EGEE-III and EGI Design Study Future European Grid Landscape CSC Finnish IT Center for Science

Contents CSC as a part of the international e- Infrastructures Overview of EGEE II/III EGI design study

CSC and the European e-Infrastructures Operational grids: EGEE EGEE, EGEE-II, EGEE-III DEISA DEISA, eDEISA, DEISA2 NDGF Finnish M-grid Preparotory phase PRACE EGI_DS e-IRGSP2

Enabling Grids for E-sciencE EGEE-II INFSO-RI EGEE'07; 2nd October 2007 The EGEE Infrastructure EGEE'07; 2nd October Production Service Pre-production service Certification test-beds (SA3) Test-beds & Services Operations Coordination Centre Regional Operations Centres Global Grid User Support EGEE Network Operations Centre (SA2) Operational Security Coordination Team Operations Advisory Group (+NA4) Joint Security Policy GroupEuGridPMA (& IGTF) Grid Security Vulnerability Group Security & Policy Groups Support Structures & Processes Training infrastructure (NA4) Training activities (NA3)

Enabling Grids for E-sciencE EGEE-II INFSO-RI EGEE'07; 2nd October 2007 Resources RegionCountriesSitesCPU CERN UK/I Fr De/CH It NE SEE CE SWE A-P Ru Totals EGEE'07; 2nd October 2007

Enabling Grids for E-sciencE EGEE-II INFSO-RI EGEE'07; 2nd October 2007 Increasing workloads 32% Still expect factor 5 increase for LHC experiments over next year 6 EGEE'07; 2nd October 2007

Enabling Grids for E-sciencE EGEE-II INFSO-RI EGEE'07; 2nd October 2007 Use of the infrastructure EGEE: ~250 sites, >45000 CPU 24% of the resources are contributed by groups external to the project ~>20k simultaneous jobs 7 EGEE'07; 2nd October 2007

Enabling Grids for E-sciencE EGEE-II INFSO-RI Registered Collaborating Projects Applications improved services for academia, industry and the public Support Actions key complementary functions Infrastructures geographical or thematic coverage 25 projects have registered as of Sept 2007: web pageweb page

Enabling Grids for E-sciencE EGEE-II INFSO-RI EGEE-II to EGEE-III EGEE-III –To be funded under European Commission call INFRA –32M compared to ~37M for EGEE-II Key objectives –Expand/optimise existing EGEE infrastructure, include more resources and user communities –Prepare migration from a project-based model to a sustainable federated infrastructure based on National Grid Initiatives 2 year period – May 2008 to April 2010 –No gap between EGEE-II and EGEE-III Consortium –Now structured on a national basis (National Grid Initiatives/Joint Research Units) –43 partners (compared to 90+ in EGEE-II) Networking activitiesSpecific Service Activities NA1: ManagementSA1: Operations NA2: DisseminationSA2: Networking Support NA3: TrainingSA3: Integration, testing & Cert. NA4: ApplicationsJoint Research Activities NA5: International Coop. & PolicyJRA1: Middleware engineering

Northern Europe Effort in EGEE-III

Whats special about the Grid Grids have proven to be an excellent way of federating resources across computer centres of varying sizes into much larger quasi-homogeneous infrastructures. This matches well with the needs of many disciplines, allowing resources at participating institutes to meet the needs of the entire collaboration(s). This in turn adds value to the individual sites, leading to a positive feedback situation. 11

But theres (much) more… Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: –Provisioned Large scale, long term Grand Challenge e.g. LHC (space microscopes), space telescopes, …. 12

But theres (much) more… Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: –Provisioned –Scheduled Require large resources for short periods Far too expensive to provision for a single application Not (always) time critical – disaster response? 13

But theres (much) more… Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: –Provisioned –Scheduled –Opportunistic Which includes the above but also other areas which are less real time 14

But theres (much) more… Grid Computing (potentially) offers value to a wide range of applications, broadly classified as follows: –Provisioned –Scheduled –Opportunistic These can co-exist, sharing the same Grid infrastructure(s), to each others mutual benefit! 15

Evolution Testbeds Utility Service Routine Usage National Global

Why Sustainability TODAY? Dependency: some application domains depend on grids already today Protection of Investment: Investment in grids, both from funding organizations and from users, need to be protected Perspective: Todays grid users are grid enthusiasts, tomorrows grid users ask for a longer term perspective

Evolution Testbeds Utility Service Routine Usage National Global European e-Infrastructure

Terminology European Grid Initiative (EGI) = the future EGI Organisation + the NGIs EGI Organisation = the future team with central responsibility, including the management structure EGI Design Study = the present EU FP7 project producing a blueprint for a future EGI

EGI Objectives Ensure the long-term sustainability of the European e- infrastructure Coordinate the integration and interaction between National Grid Infrastructures Operate the European level of the production Grid infrastructure for a wide range of scientific disciplines to link National Grid Infrastructures

38 European NGIs + Asia, US, Latin America + PRACE + OGF-Europe + …

EGI Design Study Project proposal: submitted to FP7-INFRASTRUCTURES , Design Studies Goal: Conceptual setup and operation of a new organizational model of a sustainable pan-European grid infrastructure Consortium: 9 Partners EGI Preparation Team

EGI Preparation Team Members: Johannes Kepler Universität Linz (GUP) Greek Research and Technology Network S.A. (GRNET) Istituto Nazionale di Fisica Nucleare (INFN) CSC - Scientific Computing Ltd. (CSC) CESNET, z.s.p.o. (CESNET) European Organization for Nuclear Research (CERN) Verein zur Förderung eines Deutschen Forschungsnetzes - DFN-Verein (DFN) Science & Technology Facilities Council (STFC) Centre National de la Recherche Scientifique(CNRS)

EGI Policy Board

EGI Design Study Project proposal: submitted to FP7-INFRASTRUCTURES , Design Studies Goal: Conceptual setup and operation of a new organizational model of a sustainable pan-European grid infrastructure Consortium: 9 Partners EGI Preparation Team NGI Representatives EGI Policy Board Person months: ~300 Duration: 1 Sept 2007 – 30 Nov 2009 (27 Months)

EGI_DS Schedule Duration 27 months: Develop EGI ProposalNGIs signing Proposal Start of EGEE-III Final Draft of EGI Blueprint Proposal EGI Blueprint Proposal EGEE-III transition to EGI-like structure EGI Entity in place EU Call Deadline for EGI Proposal Submission of EGEE-III Start of EGI Design Study EGEE-II (2YEARS) EGEE-III (2YEARS) EGI operational

Work Distribution WP2: EGI Requirements Consolidation (Fotis Karayannis, GRNET) WP3: EGI functionality definition (Laura Perini, INFN) WP4: Study of EGI legal and organisational options (Beatrice Merlin, CNRS) WP5: Establishment of EGI (Jürgen Knobloch, CERN) WP6: EGI Promotion and Links with Other Initiatives (Per Öster, CSC)

EGI Webpage

Characteristics of NGIs Each NGI … should be a recognized national body with a single point-of-contact … should mobilise national funding and resources … should operate the national e-Infrastructure … should supports user communities (application independent, and open to new user communities and resource providers) … should contribute and adhere to international standards and policies Responsibilities between NGIs and EGI are split to be federated and complementary

NGI Responsibilities Regional operations functions Resource provisioning Accounting and monitoring Application support Help desk facilities Contribution to middleware development

EGI Operations Principles 1.Reliability of Grid services and SLAs; 2.Multi-level operation model; 3.EGI, NGI and ROC; 4.Multiple middleware stacks; 5.Planning, coordination and gathering of new requirements; 6.Cooperation; 7.Federation, interoperability and data aggregation.

EGI Transition Scenario Many applications rely now on production quality grid infrastructures. Sciences using todays grid operations (supported by EGEE, DEISA, …) and other projects should be able to transit without disruption to the envisaged sustainable EGI/NGI-based model. Example: EGEE-III using JRUs as NGI equivalents

EGI Transition Scenario Important: Human Expertise Many developments and operational tasks are performed by highly skilled staff, which has built up their expertise through the lifetime of the current grid projects. Care must be taken that this expertise can be retained during and after the transition period.

Convention (Statutes) of the EGI Organisation Goal: Defining a set of agreed formal criteria and rules for the EGI organisation to be ratified by the EGI Advisory Board Contents: –Purpose of the EGI Organisation –Membership –Organisational structure (Council, Directors, …) –Budget contributions, audits, …

EGI – European Grid Initiative Future EGI Organisation = Glue between various grid communities in Europe and beyond EGI_DS defines required mechanisms and functionalities of the EGI Organisation Towards a sustainable environment for the application communities utilizing grid infrastructures for their everyday work

Acknowledgements Slides from different projects and contributors EGEE II-III Bob Jones Ian Bird EGI-DS Dieter Kranzlmüller