1. JRA1 Activity Feedback Frédéric Hemmer EGEE Middleware Manager and the JRA1 team EGEE is a project funded by the European Union under contract IST-2003-508833 www.eu-egee.org EGEE 1 st Conference, 22-Apr-2004

2. EGEE 1 st Conference, Cork, April 22, 2004 - 2 Outline Status What has changed in Cork Important steps until next conference in November 2004  Or rather end of 2004 – MJRA1.4: Release Candidate 1 Summary

3. EGEE 1 st Conference, Cork, April 22, 2004 - 3 STATUS

4. EGEE 1 st Conference, Cork, April 22, 2004 - 4 Software Clusters Tools, Testing & Integration (CERN) clusters Development clusters:  UK  CERN  IT/CZ  Nordic Clusters have a reasonable sized (distributed) development testbed  Taken over from EDG  Nordic cluster to be finalized  Milestone MJRA1.2 (PM3) almost reached now Link with Integration & Tools clusters established Clusters up and running! Nordic (security) cluster  JRA3

5. EGEE 1 st Conference, Cork, April 22, 2004 - 5 Design Team Formed in December 2003 Current members:  UK: Steve Fisher  IT/CZ: Francesco Prelz  Nordic: David Groep  VDT: Miron Livny  CERN: Predrag Buncic, Peter Kunszt, Frederic Hemmer, Erwin Laure Started service design based on component breakdown defined by the LCG ARDA RTAG Leverage experiences and existing components from AliEn, VDT, and EDG. A working document Overall design & API’s https://edms.cern.ch/document/458972 Basis for architecture (DJRA1.1) and design (DJRA1.2) document

6. EGEE 1 st Conference, Cork, April 22, 2004 - 6 Towards a prototype Focus on key services; exploit existing components Initially an ad-hoc installation at Cern and Wisconsin Aim to have first instance ready by end of April  Open only to a small user community  Expect frequent changes (also API changes) based on user feedback and integration of further services Enter a rapid feedback cycle  Continue with the design of remaining services  Enrich/harden existing services based on early user/operations feedback Access service:  AliEn shell, APIs Information & Monitoring:  R-GMA CE:  AliEn CE, Globus gatekeeper, CondorG Security:  VOMS, myProxy Workload mgmt:  AliEn task queue; SE:  SRM (Castor), GridFTP, GFAL, aoid File Transfer Service:  AliEn FTD File and Replica Catalog:  AliEn File Catalog, RLS Initial prototype components for April’04 To be extended/ changed (e.g. WMS) This is not a release! It’s purely an ad-hoc installation

7. EGEE 1 st Conference, Cork, April 22, 2004 - 7 Integration A master Software Configuration Plan is being finalized now It contains basic principles and rules about the various areas of SCM and Integration (version control, release management, build systems, bug tracking, etc) Compliant with internationally agreed standards (ISO 10007-2003 E, IEEE SCM Guidelines series) Most EGEE stakeholders have already been involved in the process to make sure everybody is aware of, contributes to and uses the plan An EGEE JRA1 Developer's Guide will follow shortly in collaboration with JRA2 (Quality Assurance) based on the SCM Plan It is of paramount importance to deliver the plan and guide as early as possible in the project lifetime

8. EGEE 1 st Conference, Cork, April 22, 2004 - 8 SCM Contents SCM and Integration Implementation Configuration and Version Control Build Systems Release Process Other Configuration and Change Control Procedures Solid steps towards MJRA1.3 (PM5)

9. EGEE 1 st Conference, Cork, April 22, 2004 - 9 Testing The 3 initial testing sites are CERN, NIKHEF and RAL  More sites can join the testing activity at a later stage !  Must fulfil site requirements Testing activities will be driven by the test plan document Test plan being developed based on user requirements documents:  Application requirements from NA4: HEPCAL I&II, AWG documents, Bio- informatics requirements documents from EDG  Deployment requirements being discussed with SA1  ARDA working document for core Grid services  Security: work with JRA3 to design and plan security testing The test plan is a living document: it will evolve to remain consistent with the evolution of the software Coordination with NA4 testing and external groups (e.g. Globus) established Solid steps towards MJRA1.3 (PM5)

10. EGEE 1 st Conference, Cork, April 22, 2004 - 10 Resources indicators

11. EGEE 1 st Conference, Cork, April 22, 2004 - 11 Execution Plan Work Breakdown Structure defined Document needs to be completed

12. EGEE 1 st Conference, Cork, April 22, 2004 - 12 Achievements in Cork

13. EGEE 1 st Conference, Cork, April 22, 2004 - 13 Clusters Responsibilities Nordic CERN UK IT/CZ

14. EGEE 1 st Conference, Cork, April 22, 2004 - 14 Planning Evolution of the prototype  Envisaged status at end of 2004: Key services need to fulfill all requirements (application, operation, quality, security, …) and form a deployable release Remaining services available as prototype  Need to develop a roadmap Incremental changes to prototype (where possible) Early user feedback through ARDA and early deployment on SA1 pre- production service Detailed release plan being planned  Converge prototype work with integration & testing activities Need to get rolling now! First components will start using SCM in May

15. EGEE 1 st Conference, Cork, April 22, 2004 - 15 Guiding Principles Lightweight (existing) services  Easily and quickly deployable Interoperability  Allow for multiple implementations Resilience and Fault Tolerance Co-existence with deployed infrastructure  Run as an application Service oriented approach  Follow WSRF standardization  No mature WSRF implementations exist to date, hence: start with plain WS – WSRF compliance is not an immediate goal  Review situation end 2004

16. EGEE 1 st Conference, Cork, April 22, 2004 - 16 JRA1/SA1 - Process description No official delivery of requirements from SA1 to JRA1 stated in the TA The definition, discussion and agreement of the requirements has already started, done through dedicated meetings This is an ongoing process:  Not all the requirements defined yet  Set of requirements agreed, need basic agreement to start working! But can be reviewed at any time there is a valid reason for it

17. EGEE 1 st Conference, Cork, April 22, 2004 - 17 JRA1/SA1 - Requirements 1. Middleware delivery to SA1 2. Release management 3. Deployment scenarios 4. Middleware configuration  JRA1 will provide a standard set of configuration files and documentation with examples that SA1 can use to design tools. Format to be agreed between SA1-JRA1  It is the responsibility of SA1 to provide configuration tools to the sites 5. Enforcement of the procedures 6. Platforms to support Primary platform: Red Hat Enterprise 3.0, gcc 3.2.3 and icc8 compilers (both 32 and 64- bits). Secondary platform: Windows (XP/2003), vc++ 7.1 compiler (both 32 and 64-bits) 7. Versions for compilers, libraries, third party software 8. Programming languages 9. Packaging and software distribution 10. Others Sites must be allowed to organize the network as they wish, internal or external connectivity, NAT, firewall, etc, all must be possible, no special constraints. WNs must not require Outgoing IP connectivity; Not inbound connectivity either.

18. EGEE 1 st Conference, Cork, April 22, 2004 - 18 Related documents Requirement table stored at: https://edms.cern.ch/document/456865 Meeting minutes stored at: https://edms.cern.ch/document/451069

19. EGEE 1 st Conference, Cork, April 22, 2004 - 19 JRA1/JRA3 A lot of progress has been achieved here  Security Group formed, JRA1 members identified  First meeting scheduled on May 5-6, 2004 GAP analysis planned by then  VOMS Administration support clarified Handled by JRA3  Issue: VOMS effort reporting

20. EGEE 1 st Conference, Cork, April 22, 2004 - 20 JRA1/JRA4 SCM plan presented and discussed More discussions on which components of JRA4 will be required in the overall architecture/design need to take place

21. EGEE 1 st Conference, Cork, April 22, 2004 - 21 American Involvement in JRA1 UWisc  Miron Livny part of the design Team  Condor Team actively involved in reengineering resource access In collaboration with Italian Cluster ISI  Identification of potential contributions started (e.g. RLS) Focused discussions being planned Argonne  Collaboration on Testing started  Support for key Globus Components enhancements being discussed

22. EGEE 1 st Conference, Cork, April 22, 2004 - 22 March Issues related to other activities Security JRA3  Development must be part of JRA1 structure  Security is not an orthogonal activity Same for JRA4 Interaction with SA1  Support  Requirements and acceptance criteria gathering  Definition of responsibilities Testing Support  Packaging/Configuration/Distribution Mechanisms American involvement still being clarified Issue: CNRS Involvement JRA2 documents (standards, templates) need to be available early  And relevant to the complexity of the activity

23. EGEE 1 st Conference, Cork, April 22, 2004 - 23 Important steps until next conference (rather: end of 2004 – MJRA1.4: 1 st release candidate)

24. EGEE 1 st Conference, Cork, April 22, 2004 - 24 Milestones and Deliverables for 2004 Month Deliverables & Milestones Item Lead Partner Status M03MJRA1.1Tools for middleware engineering and integration deployed CERN Most tools identified (SCM) Issue: CNRS person tbi M03DJRA1.1(Document) Architecture and Planning (Release 1) CERN Design document as starting point; writing organized for May M03MJRA1.2Software cluster development and testing infrastructure available CERN Done for CERN/UK/IT/CZ Nordic to be finalized M05MJRA1.3Integration and testing infrastructure in place including test plans (Release 1) CERN Sites identified; teams established; M05DJRA1.2(Document) Design of grid services (Release 1) CERN Design document as starting point M09MJRA1.4Software for the Release Candidate 1CERN Work on prototype started; will evolve into 1 st release candidate M08 – Amsterdam conference: Tech preview of release candidate 1 available

25. EGEE 1 st Conference, Cork, April 22, 2004 - 25 JRA1 and other activities NA4  HEP: ARDA project started; ensures close relations between HEP and middleware  Bio: activities with similar spirit needed – focused meeting tentatively being planned for May SA1  Revision of requirements (platforms) JRA2  QAG started  Monthly meeting established JRA3  Necessary structures established  Focused Meeting in May JRA4  Architectural components required need to be clarified Other projects  Potential drain of resources for dissemination activities

26. EGEE 1 st Conference, Cork, April 22, 2004 - 26 Summary Activity progressing and on track (despite tight schedules) Technology Risk  Will WS allow for all upcoming requirements?  Divergence to standards Very productive conference  Lively discussions in most sessions  Unfortunately no time for a JRA1/NA4 session No amendments to TA foreseen Thanks to organizers

27. EGEE 1 st Conference, Cork, April 22, 2004 - 27 Information Systems … http://www.mylostbag.com/