1. 1 DataGRID Application Status and plans Federico.Carminati@cern.ch

2. 2DataGRID ConferenceMarch 6, 2002 Application objectives for year 1 Define use cases for the applications Define application requirements Deploy realistic applications on the TestBed 1 Evaluate TestBed 1

3. 3DataGRID ConferenceMarch 6, 2002 Achievements Dialogue within the applications on the opportunities of GRID Dialogue and coordination to understand how to do things differently and not more of the same Testbed 0 has improved our understanding of the issues A very large amount of unfunded application effort has made this possible Realistic applications using GLOBUS have been deployed Requirements expressed in June 2001 Developed detailed “long term use cases” A very large exercise leveraging unfunded effort (30-40 person- month) Funded WP8 effort (LCs) has Participated to the deployment and configuration of the TestBed Validated the TestBed with “generic” HEP applications  Resource Broker  Replica catalogue functionality

4. 4DataGRID ConferenceMarch 6, 2002 Achievements Detailed TestBed evaluation plans have been elaborated and applied Few hours after the official opening of the TestBed on December 9 a standard physics simulation job has been run Although TB1 had not reached stability much middleware functionality was demonstrated by the applications in a short period of time

5. 5DataGRID ConferenceMarch 6, 2002 Achievements Mechanisms to provide feedback to the developers have been put in place Priority list constantly updated by WP8 and discussed at weekly WP manager meetings Priority list Feedback on the release plan Detailed user requirements for crucial components (e.g. Storage Element) in preparation Large unfunded participation from experiments Probably as much as 200 person-month in the first year

6. 6DataGRID ConferenceMarch 6, 2002 Issues and actions Stability (or even existence) of the testbed Testbed existed for 10 days in December and then for ~5 days before the review!! Configuration and management of the VO’s is very difficult Our needs were not expressed too clearly and were not given enough attention Procedures should be worked out to automate it Assignment and escalation of problems is complicated It has worked well thanks to the heroic efforts of all people involved, but it will have to be better formalised Need tight coordination between site-managers, TestBed and applications

7. 7DataGRID ConferenceMarch 6, 2002 Issues and actions Coexistence of pseudo-production and development environment still a problem We have done it for years on a local environments On a distributed ones is more difficult and we are learning Deployment plans to all TestBed sites are still unclear This needs to be clarified!

8. 8DataGRID ConferenceMarch 6, 2002 Plan for the next year Continue exploitation of TB1 Deploy larger and more realistic applications on TB1, including “data challenges” and “production challenges” Use more sites as TB1 expands Refine the user requirements and long term views A common set of requirements / use cases to define common solutions This will facilitate the relation with other GRID MW projects Collaboration with DataTag has MW interoperability as specific goal An RTAG has been launched in the LCG Project with this mandate This has been possible because of the work of the DataGRID project and is heavily based on WP8's findings to date

9. 9DataGRID ConferenceMarch 6, 2002 What we want from a GRID This is the result of our experience on TB0 & TB1 OS & Net services Basic Services High level GRID middleware LHC VO common application layer Other apps ALICEATLASCMSLHCb Specific application layer Other apps GLOBU S team GRID architecture

10. 10DataGRID ConferenceMarch 6, 2002 Common use cases OS & Net services Bag of Services (GLOBUS) Specific application layer GLOBU S team MiddleWare MW1MW2MW3MW4MW5 MiddleWare MW1MW2MW3MW4MW5 LHC VO use cases & requirements Other apps If we manage to define ALICEATLASCMSLHCbOther apps MiddleWare MW1MW2MW3MW4MW5 VO use cases & requirements Common core use case Or even better LHCOther apps MiddleWare MW1MW2MW3MW4MW5 VO use cases & requirements It will be easier to arrive at Common use cases LHCOther apps

11. 11DataGRID ConferenceMarch 6, 2002 Detailed plan of work LCs have already interviewed the experiments A lot of material has been collected They will now post-process the information Hopefully one month The result will be the starting point for the RTAG RTAG will contain representative of experiments and (subject to the agreement of the experiments) of LCs The RTAG produces results A of detailed use cases for HEP in two-three months During this time LCs will devote their attention to the testbed (see later) The results will be brought back into WP8/9/10 to Discuss possible implementation Involve WP9-10

12. 12DataGRID ConferenceMarch 6, 2002 Production Scenario (PRO) VO MetaDataCatalog LFN1 LFN2 LFN3 LFN4... JDL WDL file Broker Production db UI Exp. Frame VO-ReplicaCatalog CE SE WN Configuration db GIIS GRIS MSS

13. 13DataGRID ConferenceMarch 6, 2002 VO MetaDataCatalog LFN1 LFN2 LFN3 LFN4... JDL file Broker Production db VO-ReplicaCatalog CE SE WN Configuration db Rcapi file open UI Exp. Frame SE Spawn a job EvtTag Collection Physics Analysis (ANA) scenario Retrieve result

14. 14DataGRID ConferenceMarch 6, 2002 Execution (additional data/info needed) Job on worker node Catalog (LFN) Calibration… Replica Catalogue Information Services Scheduling Grid

15. 15DataGRID ConferenceMarch 6, 2002 Postprocessing Job on worker node Catalog (LFN) Calibration… Replica Catalogue Information Services Storage Element SEn

16. 16DataGRID ConferenceMarch 6, 2002 Disclaimer A lot of this work is essentially architectural We want to provide a purely user point of view Inevitably we will trespass the border and say not only what we want but how we want it However we understand this and we will be glad to be suggested different ways to do that As long as this still does what we want

17. 17DataGRID ConferenceMarch 6, 2002 Application Priorities API’s This is a long standing requirement, now this is needed for prototyping MPI At least on local clusters, wide area MPI later Interactive access We have to agree on what it means and how it can be provided We certainly want to start trying on TB1 Linux 7.2 (summer) This is necessary for a proper deployment Solaris (release 2) Not a priority per se, but if not this year, we will never do it The DataGRID code should demonstrate some portability to be seriously considered as a baseline Tools for VO management

18. 18DataGRID ConferenceMarch 6, 2002 Application testbed requirements Three main priorities

19. 19DataGRID ConferenceMarch 6, 2002 Application testbed requirements Development and production testbeds We need a valid model to have both Consolidation & Stability of production testbed It has to stay up from tomorrow to release 2 Consolidation and complete deployment of uniform information system The question of FTREE/R-GMA/MDS has to be solved Support for the testbed This has to work much better than in TB1 till now Interface to MSS CASTOR, HPSS, DataSTORE Solve the question of installation on pre-existing clusters Light LCFG? Expansion of the testbed (up to) the famous 40 WP6 sites

20. 20DataGRID ConferenceMarch 6, 2002 Immediate consolidation May be jump started by a jam session of LCs and WPs Get together for few days till some test applications run on the testbed without problems This should be done rather quickly 3-4 days week 22th March or after Easter These applications may be bundled in a standard test package to run periodically A kind of testbed heartbeat to verify its stability

21. 21DataGRID ConferenceMarch 6, 2002 Summary Applications have been able to deploy and demonstrate large applications on the TestBed The LHC experiments are participating with enthusiasm to the project The side effect is that the pressure is high from the users for Support Stable environment Functionality Documentation Wide deployment These are healthy signs Continued positive reaction to the feedback from the users will maintain their high level of enthusiasm and participation The evolution of TB1 leading to TB2 will be a crucial test of the project’s ability to reach its final goals