On the Quest for Sustainability wherein there are contained more questions than answers Dr. Rüdiger Berlich / Dr. Torsten Antoni Karlsruhe Institute of Technology torsten.antoni@kit.edu EGI Technical Forum 20-09-11, Lyon

This presentation will … … try to put the current efforts in German Grid Computing into the context of the related European efforts … focus strongly on sustainability aspects … discuss the relationship between Grids and Clouds, and where both can learn from each other T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

What we want “Grid computing is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (I.Foster) T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Why we are still here Coordinated resource sharing will remain a current trend, no matter what it is called Aspects change and new technologies alter the means of implementation Virtualization (wasn‘t a commodity technique when Grid Computing started) Grid can learn from the Cloud We must make sure that we do not loose useful Grid developments Cloud can learn from collaborative aspects of the Grid T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Where we are Today: EGEE → EGI „The EGEE project brings together experts from more than 50 countries with the common aim of building on recent advances in Grid technology and developing a service Grid infrastructure which is available to scientists 24 hours-a-day. “ EGI: „EGI.eu is a foundation established under Dutch law to create and maintain a pan-European Grid Infrastructure (EGI) in collaboration with National Grid Initiatives (NGIs) and European International Research Organisations (EIROs), to guarantee the long-term availability of a generic e-infrastructure for all European research communities and their international collaborators. “ T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

EGI → National Grid Initiatives In Germany: A very diverse set of Grid initiatives NGI-DE is the German branch of EGI Partner: KIT (coordination), BADW-LRZ, FZJ, DESY, LUH-RRZN, DFN, FhG, D-Grid GmbH Supported particularly by Gauß-Allianz e.V. Association of large academic compute centers in Germany Has the potential for sustainable deployment of HPC and Grid services Large computing centres as components make it easier to achieve sustainability T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

EGI – National Grid Initiatives EGI – European Grid Infrastructure NGI-DE – Uplink to EGI, central services, international tasks DGI-2 – Continuation of requested services and their adaption to EGI T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

DGI-2 „D-Grid Integrationsprojekt“ Maintenance and running of the services and infrastructure used by D-Grid communities in 2011/12 Adaption of services to requirements, structures and processes of the European Grid Inrastructure (EGI), with the aim of maintaining and establishing interoperability with other European Grid providers Laying the foundations for a sustainable Grid infrastructure T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Excerpt from the D-Grid project list AstroGrid Collaborative reserch environment for astronomy C3-Grid Collaborative Climate Community and Processing Grid In-Grid “In-Grid will enable engineering projects for Grid-based application and efficient use of common compute and software resources” Gap-SLC Usage of short-lived certificates in portal-based Grids Many more past and current projects Not „just“ High-Energy Physics T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Offers and Expectations In the context of DGI-2, currently three middlewares are supported Globus, UNICORE, gLite There are services built around these Either in a national or in an international context Participating partners provide huge amounts of computing power Users have a very diverse set of use-cases and expectations T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

The Quest for Sustainability DGI-2 partners already run and maintain numerous IT-services, such as networks, HPC-systems and storage for local, regional, national and international Users This is the foundation on which sustainability can be built Central Grid services will become part of the standard offers of participating institutions Need to find suitable „business-models“ around it Active discussion in the context of EGI and NGI-DE Open Source as one incubator Cannot give a recipy T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Business Plan ?! Taking a „business plan“ approach, central questions need to be asked What is the actual product we are „selling“ and on which we want to base sustainable operation? Probably the answer is different for each NGI and even each sub-community What is our core product? Provision of computing resources? Is the available user base large enough to actually support sustainability? If not, what do we do in terms of business development? Where do we want to be in 1, 2, 5 years from now? What do we want to be sustainable? T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

The Quest for Sustainability Sustainability means Ensuring the long-term availability of Infrastructure Hardware Software Services Know-How Means of ensuring sustainability include Binding man-power and knowledge Identifying funding schemes Identifying new user communities and usage scenarios But in the end: We can pay our own bills and maintain our market position T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

The Quest for Sustainability Groundwork Need to understand the term Grid Computing in a wider scope It is the deployment scenarios and applications that define what Grid Computing means No matter what Grid Computing will be called in 20 years, its deployment scenarios will remain Need to build business model on deployment scenarios Need to make sure viable Grid techniques remain Collaboration as one of Grid‘s biggest strengths T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Conclusion (1) Grid Computing (in Germany and internationally) uses established technologies and there are very important and „future-proof“ application scenarios There will always be Grid activities (under whatever name …) Far fewer application domains will remain than were originally envisaged for „the Grid“ There will in the future be very few users from industry Some user communities will go into a different direction Application scenarios will likely be limited to those requiring the USP of „the Grid“ – The direct support for collaborative efforts The Grid infrastructure will in the future involve virtualization and dynamic extension of Grid resources Grid and Cloud (+ successors) will become indistinguishable Need to focus on the user- and application side T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Conclusion (2) There already exists a strong foundation for sustainable operation, through the participating organizations Sustainability efforts may focus on services, making the overall task easier to handle, as fewer funds are needed Focus on interoperability rather than „building the infrastructure“ Sustainability needs to be organized in the form of a business plan, or else … 10 employees cost ~750000 € / Year . Does the „Grid market“ support this ? Sustainability in the form of a business plan will likely have to focus on academic users. This is a path that is rarely being followed Keep it simple! We need many users to make it cheaper for everyone Need „business development“, adapt to the users T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon

Conclusion (3) Grid Computing is international by design EGI, and with it the NGIs will carry on duties and maintenance But need to adapt to the new opportunities posed through virtualization, allowing an almost seamless exchange of compute power (adaption already happens) At one point, Grids and Clouds will become indistinguishable From the user- and application-provider perspective, the long-term perspective is bright coordinated resource sharing will remain Current Grid providers need to adapt their mode of operation Whether all collaborative aspects of the Grids will remain depends on the success of the sustainability strategy. This is the main duty of EGI and NGIs T. Antoni | Sustainability | EGI TF | 20-09-11 | Lyon