1. EGEE is a project funded by the European Union under contract INFSO-RI-508833 Middleware for the next Generation Grid Infrastructure Erwin Laure EGEE Deputy Middleware Manager A. Aimar, M. Barroso, S. Beco, P. Buncic, A. Di Meglio, A. Edlund, S. Fisher, D. Groep, L. Guy, F. Hemmer, P. Kunszt, M. Livny, O. Mulmo, F. Pacini, F. Prelz, M. Sgaravatto CHEP 2004 www.eu-egee.org

2. Chep 2004 - 2 The EGEE Project EGEE offers the largest production grid facility in the world open to many applications (HEP, BioMedical, generic) Existing production service based on LCG Next generation open source web-services middleware being re-engineered taking into account production/ deployment/ management needs Well-defined, distributed support structure to provide eInfrastructure that is available to many application domains Middleware Activity  Re-engineer and harden Grid middleware  Provide production quality middleware www.eu-egee.org Network infrastructure ( GÉANT ) Operations, Support and training Collaborations Global Grid

3. Chep 2004 - 3 Future EGEE Middleware - gLite Application requirements http://egee-na4.ct.infn.it/requirements/http://egee-na4.ct.infn.it/requirements/ Intended to replace LCG-2 Starts with existing components Aims to address LCG-2 shortcoming and advanced needs from applications (in particular feedback from DCs) Prototyping short development cycles for fast user feedback Initial web-services based prototypes being tested with representatives from the application groups Globus 2 basedWeb services based gLite-2gLite-1LCG-2LCG-1

4. Chep 2004 - 4 Guiding Principles Lightweight (existing) services  Easily and quickly deployable  Use existing services where possible as basis for re-engineering Interoperability  Allow for multiple implementations Perf/Scale. & Resilience/Fault Tolerance  Large-scale deployment and continuous usage Portable  Being built on Scientific Linux and Windows Co-existence with deployed infrastructure  Reduce requirements on participating sites  Flexible service deployment Multiple services running on the same physical machine (if possible)  Co-existence with LCG-2 and OSG (US) are essential for the EGEE Grid service Service oriented approach  Follow WSRF standardization  No mature WSRF implementations exist to-date so start with plain WS WSRF compliance is not an immediate goal, but we follow the WSRF evolution WS-I compliance is important

5. Chep 2004 - 5 gLite Services Service decomposition based on ARDA blueprint

6. Chep 2004 - 6 WMS

7. Chep 2004 - 7 CE Works in push and pull mode Site policy enforcement Exploit new globus GK and CondorC (close interaction with globus and condor team) CEA … Computing Element Acceptance JC … Job Controller MON … Monitoring LRMS … Local Resource Management System

8. Chep 2004 - 8 Data Management Scheduled data transfers (like jobs) Reliable file transfer Site self-consistency SRM based storage

9. Chep 2004 - 9 Storage Element Interfaces SRM interface  Management and control  SRM (with possible evolution) Posix-like File I/O  File Access  Open, read, write  Not real posix (like rfio) SRM interface rfiodcapchirpaio Castor dCacheNeST Disk POSIX API File I/O Control User

10. Chep 2004 - 10 Catalogs File Catalog Metadata Catalog LFN Metadata File Catalog  Filesystem-like view on logical file names  Keeps track of sites where data is stored  Conflict resolution Replica Catalog  Keeps information at a site (Meta Data Catalog)  Attributes of files on the logical level  Boundary between generic middleware and application layer Replica Catalog Site A GUIDSURL LFN Replica Catalog Site B GUIDSURL LFN GUID Site ID

11. Chep 2004 - 11 Information and Monitoring R-GMA for  Information system and system monitoring  Application Monitoring No major changes in architecture  But re-engineer and harden the system Co-existence and interoperability with other systems is a goal  E.g. MonaLisa MPP – Memory Primary Producer DbSP – Database Secondary Producer Job wrapper MPP DbSP Job wrapper MPP Job wrapper MPP e.g: D0 application monitoring:

12. Chep 2004 - 12 “The Grid” Joe Pseudonymity Service (optional) Credential Storage 1. 2. 3. 4. Obtain Grid (X.509) credentials for Joe “Joe → Zyx” “Issue Joe’s privileges to Zyx” “User=Zyx Issuer=Pseudo CA” Attribute Authority Security myProxy tbd VOMS GSI LCAS/LCMAP S

13. Chep 2004 - 13 GAS & Package Manager Grid Access Service (GAS)  Discovers and manages services on behalf of the user  File and metadata catalogs already integrated Package Manager  Provides application software at execution site  Based upon existing solutions  Details being worked out together with experiments and operations

14. Chep 2004 - 14 Approach Exploit experience and components from existing projects  AliEn, VDT, EDG, LCG, and others Design team works out architecture and design  Architecture: https://edms.cern.ch/document/476451https://edms.cern.ch/document/476451  Design: https://edms.cern.ch/document/487871/https://edms.cern.ch/document/487871/  Feedback and guidance from EGEE PTF, EGEE NA4, LCG GAG, LCG Operations, LCG ARDA Components are initially deployed on a prototype infrastructure  Small scale (CERN & Univ. Wisconsin)  Get user feedback on service semantics and interfaces After internal integration and testing components are delivered to SA1 and deployed on the pre-production service EDGVDT... LCG...AliEn

15. Chep 2004 - 15 From Prototype to Release Prototype setup at 2 sites (CERN & Wisconsin)  ~45 users registered  2 nd release mid August Many bugfixes New functionalities  More nodes (more powerful) being added at CERN  EDG WMS added at CNAF  Being ported to SLC3  Second VO being set up (core services at Wisconsin) Will use globus RLS Cf. following talk by Birger Koblitz

16. Chep 2004 - 16 From Prototype to Release Continuous integration system set up  First results being put forward to testing team  SLC3 version of prototype middleware provided SCM convergence being reached  SCM plan: https://edms.cern.ch/document/446241https://edms.cern.ch/document/446241 Common service configuration being worked out Deployment scenarios being worked out Testing sites (RAL, NIKHEF, CERN) up and running  Focus on prototype installation and testing  Plans being made for focused testing of gLite components for pre- production service Test plan defined  Based upon architecture document, release plan, and NA4 requirements  https://edms.cern.ch/document/473264/ https://edms.cern.ch/document/473264/

17. Chep 2004 - 17 gLite and LCG-2 LCG-2 focus on production, large-scale data handling The service for the 2004 data challenges Provides experience on operating and managing a global grid service Development programme driven by data challenge experience  Data handling  Strengthening the infrastructure  Operation, VO management Evolves to LCG-3 as components progressively replaced with new middleware -- target is to minimise the discontinuities of migration to the new generation Aim for migration plan by end of year gLite focus on analysis Developed by EGEE project in collaboration with VDT (US) LHC applications and users closely involved in prototyping & development (ARDA project) Short development cycles Co-existence with LCG-2 Profit as far as possible from LCG- 2 infrastructure, experience  Ease deployment – avoid separate hardware As far as possible - completed components integrated in LCG-2  improved testing, easier displacement of LCG-2 LCG-2 (=EGEE-0) prototyping product 2004 2005 LCG-3 EGEE-1 product les robertson - cern-it-17 LCG

18. Chep 2004 - 18 Summary Next generation middleware being designed and assembled  Prototype first tangible outcome BUT this is a PROTOTYPE not a release!  Architectural and design work well advanced Architecture document sent to EU Design document (draft) exists Feedback from applications and operations essential! Detailed release plan worked out  https://edms.cern.ch/document/468699 https://edms.cern.ch/document/468699 First components for pre-production service during autumn Continuous integration and testing scheme defined and adopted Technology Risk  Will WS allow for all upcoming requirements?  Divergence from standards

19. Chep 2004 - 19 Links JRA1 homepage  http://egee-jra1.web.cern.ch/egee-jra1/ http://egee-jra1.web.cern.ch/egee-jra1/ Architecture document  https://edms.cern.ch/document/476451/ https://edms.cern.ch/document/476451/ Release plan  https://edms.cern.ch/document/468699 https://edms.cern.ch/document/468699 Prototype installation  http://egee-jra1.web.cern.ch/egee-jra1/Prototype/testbed.htm http://egee-jra1.web.cern.ch/egee-jra1/Prototype/testbed.htm Test plan  https://edms.cern.ch/document/473264/ https://edms.cern.ch/document/473264/ Design document  https://edms.cern.ch/document/487871/ https://edms.cern.ch/document/487871/ gLite homepage:  http://www.glite.org/ http://www.glite.org/