1. EGI: European Grid Infrastructure Steven Newhouse Interim EGI.eu Director EGI-InSPIRE Project Director Technical Director EGEE-III 1

2. European e-Infrastructure European Data Grid (EDG) –Explore concepts in a testbed Enabling Grid for E-sciencE (EGEE) –Moving from prototype to production European Grid Infrastructure (EGI) –Routine usage of a sustainable e-infrastructure 2

3. What is e-Infrastructure? Resources linked by high speed networks –Compute, Storage, Instruments,... Controlled access to shared resources –Authentication, Authorisation, Accounting,... Dependable services for others to use –Driven by availability and reliability metrics Services that are there for the long-term –Supporting experiments lasting decades Swiss Grid Day3

4. The EGEE-III Project 17,000 users 139,000 LCPUs (cores) 25Pb disk 39Pb tape 12 million jobs/month +45% in a year 268 sites +5% in a year 48 countries +10% in a year 162 Virtual Organisations +29% in a year Over 20 active communities in 112 VOs

5. EGI Collaboration NGINGI NGINGI NGINGI NGINGI Research Community Research Community Research Community Research Community EIROEIRO EIROEIRO EGI.eu Research Community 5

6. EGI.eu Coordination for European Grid resources –Roadmap to integrate HTC, HPC, Data, Instruments,... –Policy & services needed to run a grid Governance & ownership by its stakeholders –EGI Council votes proportional to GDP –EGI Council fees proportional to votes –Sustainable small coordinating organisation (EGI.eu) –Builds on resources from within its stakeholders Location selected to be Amsterdam –Distributed staff with a core (~50%) in Amsterdam –Approximately 40 staff, €3.5M/year Swiss Grid Day6

7. EC Funding National Funding Relationship between NGIs & EGI Swiss Grid Day7 NGI National Tasks (NGI) NGI International Tasks (EC & NGI) NGI National Tasks (NGI) NGI International Tasks (EC & NGI) NGI National Tasks (NGI) NGI International Tasks (EC & NGI) EGI.eu EGI Global Tasks (EC, NGI, EGI.eu)

8. EGI means Innovation Deploy Technology Innovation –Distributed Computing continues to evolve Grids  Desktops  Virtualisation  Clouds  ? Enable Software Innovation –Provide reliable persistent technology platform Today: Tools built on gLite/UNICORE/ARC Support Research Innovation –Infrastructure for data driven research Support for international research (e.g. ESFRI) Swiss Grid Day8

9. Technology Innovation Will come from outside EGI –EGI runs DCI technologies in production Partnership with technology projects –EMI (European Middleware Infrastructure) Continued evolution of gLite, ARC, UNICORE –RESERVOIR Public cloud functionality to ERA –StratusLab (Proposed) Use RESERVOIR to support gLite environments Swiss Grid Day9

10. 10 PrototypeRequirementsImplementationUsageReleaseEvaluateDeployment Feedback Collaboration Technology innovations taking place outside the production infrastructure Innovating technology being deployed within the production infrastructure NGIs & EIROs EC & National Research Projects Release

11. Software Innovation Will come from outside EGI –EGI is a neutral platform for applications EGI cannot support all services in its core –Every community needs something different Foster innovation within ‘sectors’ –High Throughput Computing –Digital Libraries –... Swiss Grid Day11

12. EGEE Maintained Components External Components Example of Deployed Services 12 Physical Resources General Services LHC File Catalogue LHC File Catalogue Hydra Workload Management Service Workload Management Service File Transfer Service File Transfer Service Logging & Book keeping Service Logging & Book keeping Service AMGA Storage Element Disk Pool Manager dCache Information Services BDII MON User Interface User Access Security Services Security Services Virtual Organisation Membership Service Virtual Organisation Membership Service Authz. Service SCAS Proxy Server LCAS & LCMAPS Compute Element CREAM LCG-CE gLExec BLAH Worker Node User Interface BDII CORE UMD SITE SERVICES (TBC) COMMUNITY SERVICES RESPECT USER Deployed by EGI Maintained & Supported through EGI Deployed by EGI Maintained & Supported by the community

13. Digital Library Community Community specific software stack –gCube from D4Science project VREs: Manage access to libraries –Big focus on usability Swiss Grid Day13 Physical Resources Core UMD Services Information Service Virtual Research Environment (VRE) Manager Service Container VRE Services VRE Services

14. High Throughput Computing Managing large collections of jobs & files –HEP, LS,... Problem specific Applications and Tools –Maintained & supported by that community Swiss Grid Day14 Physical Resources Core UMD Services GANGADIANE HTC Community Services (e.g. WMS, FTS, LFC, AMGA,...) User ApplicationsWISDOM

15. Research Innovation An infrastructure to support the ERA –Within the EU27 –Geographical Europe –Interoperability worldwide for collaboration Work with Virtual Research Communities –Groupings of aligned Virtual Organisations –Provide community specific: Support, training, consultancy, requirements etc. Swiss Grid Day15

16. The ESFRI Projects First roadmap 2006 and updated in 2008 with 44 projects Preparatory phase funding for most with second round soon Big push in FP8? Swiss Grid Day16 ESFRI: European Strategy Forum on Research Infrastructures

17. ESFRI Environmental Sciences LIFEWATCH EMSO IAGOS-ERI AURORA BOREALIS EUFAR-COPAL ICOS EURO-ARGO EPOS EISCAT-3D SIOS Status 2009 17Swiss Grid Day

18. EGI User Support USERS VOs Virtual Research Community User Forum Steering Committee USERS VOs Virtual Research Community USERS NGINGI NGI Helpdesk E G I. e u Training Events Trainers Apps. DB EGI Helpdesk VRC Helpdesk Other Helpdesk ESFRI Project VOs Virtual Research Community 18

19. Moving from EGEE to EGI What is different? –EGEE did ‘everything’ –EGI focuses just on infrastructure operations What is the same? –Running a 24/7 production quality infrastructure –Providing a support framework for the users –Collaboration to drive European DCI forward Swiss Grid Day19

20. The EGI-InSPIRE Project Integrated Sustainable Pan-European Infrastructure for Researchers in Europe Submitted to INFRA-2010 1.2.1 Targeting call objectives: –1.2.1.1: European Grid Initiative –1.2.1.2: Service deployment for heavy users A 4 year project Seeking a total €25M EC contribution –Project cost €70M

21. How much does it cost? Annual Costs EGEE EC Contribution: €16M (EGI Equivalent: ~€11M) Infrastructure Cost: ~ €80M

22. EGI Project Ecosystem EGI Software Providers Software Providers Software Providers Community VRC SSC EC Projects VRC Specialised Support Centres Supporting Projects Virtual Research Communities 22

23. Project Structure 23 Project Management External Relations: Policy, Dissemination and Standards SA3 Software Support SA2 Middleware Unit SA1 Operations JRA1 Operational Tools SA4 Services for HUCs NA3 User Support Software Provisioning and Support Operations and Tools User Support and Services

24. Project Structure EGI-InSPIRE Managerial Collaborations (e.g. EC, DEISA, OSG,...) Technical Collaborations (e.g. CHAIN, GISELR, SIENA,...) VRC & SSC Projects (e.g. ROSCOE, SAFE, CUE, TAPAS,...) Software Projects (e.g. EMI, IGE, SGI...)

25. User Support & Services Support User Communities –Researchers in International Collaborations –Scale up from the single VO to a community Provide a federated Helpdesk linking: –Discipline specific support (e.g. Bio Apps) –National infrastructure support (e.g. NGS) –Generic services (e.g. Training) Provide core services to support users –Manage VOs, Application DB, Training DB Swiss Grid Day25

26. Possible Virtual Research Communities (FP7 funding) High Energy Physics (ROSCOE) Life Sciences (ROSCOE) Computational Chemistry & Material Science (ROSCOE) Grid Observatory [Computer Science] (ROSCOE) Complexity Science (ROSCOE) Photon Science (ROSCOE) Humanities (ROSCOE) Astronomy & Astrophysics (SAFE) Fusion (SAFE) Earth Science (SAFE) Molecular Biodiversity (SIMBIOME) NMR in Structural Biology (WeNMR) Hydro-Meteorology (DRIHME) Swiss Grid Day26

27. Training Events

28. Registry of Trainers

29. Applications Database 29 http://appdb.eu-egee.org Alt. link: http://grid.ct.infn.it/egee_applications/

30. EGEE RESPECT  EGI Identify third-party software that works well with Grid –gLite: http://technical.eu-egee.org/index.php?id=290http://technical.eu-egee.org/index.php?id=290 –Expand to include UNICORE, ARC, Globus, … Simplified Access –P-GRADE, Ganga, Migrating Desktop, g-Eclipse, i2glogin, Virtual Control Room Workload Management –GridWay Metascheduler, DIANE New Resources –GRelC, Instrument Element Infrastructure Services –StoRM Swiss Grid Day30

31. Other Activities Dissemination –With NGIs, VRCs, SSCs and other projects Support for Heavy User Communities –General & community specific services Events –Two Annual meetings: Users & Technology Technology –Liaison with software providers –Definition and verification of requirements Swiss Grid Day31

32. Resources EGI.eu does not own or control resources Resources owned by individual organisations –They manage access for their user communities Middleware enables secure authorised access for external collaborators –EGI provided Unified Middleware Distribution (UMD) –EGI defined software & interfaces –UMD compatible software must be deployed –Interoperation within your country and internationally Swiss Grid Day32

33. Transition to EGI Model Has already started within EGEE-III Will continue in EGI-InSPIRE –Integrated Sustainable Pan-European Infrastructure for Researchers in Europe –4 year project, seeking €25M EC contribution User facing changes will be gradual –Focused communication from EGI & EGEE Main public points of contact have few changes –Helpdesk & software Swiss Grid Day33

34. Summary Move away from a project based model –Sustainable e-Infrastructure User services will evolve to this new model –Coordination of services provided through NGIs –Expect little change in the delivered services Start coordinating European e-Infrastructure –1 st May 2010 Contact: director@egi.eu Swiss Grid Day34