2. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 1 Accomplishments of the project from the end user point of view General Perspective Federico.Carminati@cern.ch

3. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 2 Application objectives for year 1 u Define use cases for the applications u Define application requirements u Deploy realistic applications on the TestBed 1 u Evaluate TestBed 1

4. online system multi-level trigger filter out background reduce data volume level 1 - special hardware 40 MHz (40 TB/sec) level 2 - embedded processors level 3 - PCs 75 KHz (75 GB/sec) 5 KHz (5 GB/sec) 100 Hz (100 MB/sec) data recording & offline analysis les.robertson@cern.ch

5. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 4 The LHC Detectors CMS ATLAS LHCb ~6-8 PetaBytes / year ~10 8 events/year ~10 3 batch and interactive users

6. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 5 Europe: 267 institutes, 4603 users Elsewhere: 208 institutes, 1632 users CERN’s Network in the World

7. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 6 HEP & Distributed computing u The investment for LHC computing is massive n 1.25 GB/s in HI mode n ~5 PB/y of tape n ~1.5 PB of disk n ~1800 kSI95/exp (~70,000 PC2000) n ~ order of 60MSFr of hardware s Without media + personpower + infrastructure and networking u Politically, technically and sociologically it cannot be concentrated in a single location n It is unlikely that countries will make deploy massive computing at CERN n Competence is naturally distributed n Cannot ask to people to travel to CERN so often

8. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 7 The Monarc Model Tier 0+1 centre 800 kSI95 500 TB disk Robot CERN Tier3 Universit y 1 Tier3 Universit y N Tier3 Universit y 2 622 MB/s Tier 2 centre 20 kSI95 20 TB disk Robot Tier 2 centre 20 kSI95 20 TB disk Robot 622 MB/s Tier 1 centre 200 kSI95 300 TB disk Robot Lyon RAL BNL 1500 MB/s

9. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 8 Why we need the GRID? u Every physicist should have equal access to data and resources u The system will be extremely complex n Number of sites and components in each site n Different tasks performed in parallel: simulation, reconstruction, scheduled and unscheduled analysis u We need transparent access to dynamic resources u Bad news is that the basic tools are missing n Distributed resource management, file and object namespace and authentication n Local resource management of large clusters n Data replication and caching u Good news is that we are not alone n All the above issues are central to the new developments going on in the US and Europe under the collective name of GRID

10. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 9 The Grid Vision The GRID: networked data processing centres and ”middleware” software as the “glue” of resources. Researchers perform their activities regardless geographical location, interact with colleagues, share and access data Scientific instruments and experiments provide huge amount of data

11. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 10 Biomedical Applications Explore strategies that facilitate the sharing of genomic databases and test grid-aware algorithms for comparative genomics Genomics, post-genomics, and proteomics Medical images analysis Process the huge amount of data produced by digital imagers in hospitals.

12. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 11 Grid added value for biomedical applications u Data mining on genomics databases (exponential growth). u Indexing of medical databases (Tb/hospital/year). u Collaborative framework for large scale experiments (e.g. epidemiological studies). u Parallel processing for n Databases analysis n Complex 3D modelling

13. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 12 Earth Observations ESA missions: about 100 Gbytes of data per day (ERS 1/2) 500 Gbytes, for the next ENVISAT mission (2002). DataGrid contribute to EO: enhance the ability to access high level products allow reprocessing of large historical archives improve Earth science complex applications (data fusion, data mining, modelling …) Source: L. Fusco, June 2001

14. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 13 Achievements u A wide-ranging dialogue has been launched within the applications on the opportunities opened by the GRID technology n Each HEP experiment is a world-wide distributed community of some 1000-2000 researchers n Dialogue and coordination at several levels has been necessary to reach a common understanding n Understand how to do things differently and not more of the same u To improve their understanding of the issue, applications have asked to have a GLOBUS only TestBed deployed immediately n Applications produced a set of requirements for TestBed 0 in 1Q 2001 n Testbed 0 has been in operation since 2Q 2001 n A very large amount of unfunded effort, particularly from WP8, has made this possible

15. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 14 Achievements u Realistic applications using GLOBUS authentication have been deployed on TestBed 0 u Applications have expressed their requirements in June 2001 n For this it has been necessary to develop detailed “long term use cases” n A very large exercise leveraging essentially unfunded effort (some 30-40 person-month) n This has left few months to the developers to produce the middleware u Funded WP8 effort has participated to the deployment and configuration of the TestBed u Funded WP8 effort has performed a thorough validation of the TestBed with “generic” HEP applications developed by them for this purpose n Resource Broker n Replica catalogue functionality

16. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 15 Achievements u Detailed TestBed evaluation plans have been elaborated and applied n Few hours after the official opening of the TestBed on December 9 a standard physics simulation job has been run n Although TB1 had not reached stability much middleware functionality was demonstrated by the applications in a short period of time

17. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 16 Achievements u Mechanisms to provide feedback to the developers have been put in place n Priority list constantly updated by WP8 and discussed at weekly WP manager meetings Priority list n Feedback on the release plan n Detailed user requirements for crucial components (e.g. Storage Element) in preparation u Large unfunded participation from experiments n Probably as much as 200 person-month in the first year

18. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 17 Issues and actions u Configuration and management of the VO’s on the TestBed has been unexpectedly difficult n The responsibility is shared, some of our needs were not expressed too clearly and were not given enough attention n Procedures should be worked out to automate it u Assignment and escalation of problems is complicated n It has worked well thanks to the heroic efforts of all people involved, but it will have to be better formalised n Need tight coordination between site-managers, TestBed and applications u Coexistence of pseudo-production and development environment n We have done it for years on a local environments n On a distributed ones is more difficult and we are learning u Deployment plans to all TestBed sites are still unclear n This will be clarified in the next minor releases

19. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 18 What we want from a GRID u This is the result of our experience on TB0 & TB1 OS & Net services Basic Services High level GRID middleware LHC VO common application layer Other apps ALICEATLASCMSLHCb Specific application layer Other apps GLOBU S team GRID architecture

20. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 19 Common use cases OS & Net services Bag of Services (GLOBUS) Specific application layer GLOBU S team MiddleWare MW1MW2MW3MW4MW5 MiddleWare MW1MW2MW3MW4MW5 LHC VO use cases & requirements Other apps If we manage to define ALICEATLASCMSLHCbOther apps MiddleWare MW1MW2MW3MW4MW5 VO use cases & requirements Common core use case Or even better LHCOther apps MiddleWare MW1MW2MW3MW4MW5 VO use cases & requirements It will be easier to arrive at Common use cases LHCOther apps

21. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 20 Plan for the next year u Continue exploitation of TB1 n Provide continuous feedback to developers n Deploy larger and more realistic applications on TB1 n Use the TB1 for “data challenges” and “production challenges” n Use more sites as TB1 expands u Refine the user requirements and long term views n We should be able to express them in a more uniform way n A common set of requirements / use cases to define common solutions n This will also facilitate the relation with other GRID MW projects s Collaboration with DataTag WP4 has MW interoperability as specific goal n A Requirement Technical Assessment Group (RTAG) has been launched in the context of the LHC GRID Computing Project with this mandate n This has been possible because of the work of the DataGRID project and is heavily based on WP8's findings to date

22. EC Review – 01/03/2002 – F.Carminati – Accomplishments of the project from the end user point of view– n° 21 Summary u Applications have been able to deploy and demonstrate large applications on the TestBed n The LHC experiments are participating with enthusiasm to the project u The side effect is that the pressure is high from the users for n Support n Stable environment n Functionality n Documentation n Wide deployment u These are healthy signs n Continued positive reaction to the feedback from the users will maintain their high level of enthusiasm and participation u The evolution of TB1 leading to TB2 will be a crucial test of the project’s ability to reach its final goals