1. EGEE is a project funded by the European Union under contract IST-2003-508833 HEP Use Cases for Grid Computing J. A. Templon Undecided (NIKHEF) Grid Tutorial, NIKHEF Amsterdam, 3-4 June 2004 www.eu-egee.org

2. NIKHEF Grid Tutorial, 3-4 June 2004 - 2 Contents The HEP Computing Problem How it matches the Grid Computing Idea Some HEP “Use Cases” & Approaches

3. NIKHEF Grid Tutorial, 3-4 June 2004 - 3 Our Problem  Place event info on 3D map  Trace trajectories through hits  Assign type to each track  Find particles you want  Needle in a haystack!  This is “relatively easy” case

4. NIKHEF Grid Tutorial, 3-4 June 2004 - 4 More complex example

5. NIKHEF Grid Tutorial, 3-4 June 2004 - 5 interactive physics analysis batch physics analysis batch physics analysis detector event summary data raw data event reprocessing event reprocessing event simulation event simulation analysis objects (extracted by physics topic) event filter (selection & reconstruction) event filter (selection & reconstruction) processed data Data Handling and Computation for Physics Analysis

6. NIKHEF Grid Tutorial, 3-4 June 2004 - 6 Scales To reconstruct and analyze 1 event takes about 90 seconds Maybe only a few out of a million are interesting. But we have to check them all! Analysis program needs lots of calibration; determined from inspecting results of first pass.  Each event will be analyzed several times!

7. NIKHEF Grid Tutorial, 3-4 June 2004 - 7 online system multi-level trigger filter out background reduce data volume level 1 - special hardware 40 MHz (40 TB/sec) level 2 - embedded processors level 3 - PCs 75 KHz (75 GB/sec) 5 KHz (5 GB/sec) 100 Hz (100 MB/sec) data recording & offline analysis One of the four LHC detectors

8. NIKHEF Grid Tutorial, 3-4 June 2004 - 8 Scales (2) 90 seconds per event to reconstruct and analyze 100 incoming events per second To keep up, need either:  A computer that is nine thousand times faster, or  nine thousand computers working together Moore’s Law: wait 20 years and computers will be 9000 times faster (we need them in 2007!)

9. NIKHEF Grid Tutorial, 3-4 June 2004 - 9 Computational {Impli,Compli}cations Four LHC experiments – roughly 36k CPUs needed BUT: accelerator not always “on” – need fewer BUT: multiple passes per event – need more! BUT: haven’t accounted for Monte Carlo production – more!! AND: haven’t addressed the needs of “physics users” at all!

10. NIKHEF Grid Tutorial, 3-4 June 2004 - 10 LHC User Distribution

11. NIKHEF Grid Tutorial, 3-4 June 2004 - 11 Classic Motivation for Grids Trivially parallel problem Large Scales: 100k CPUs, petabytes of data  (if we’re only talking ten machines, who cares?) Large Dynamic Range: bursty usage patterns  Why buy 25k CPUs if 60% of the time you only need 900 CPUs? Multiple user groups (& purposes) on single system  Can’t “hard-wire” the system for your purposes Wide-area access requirements  Users not in same lab or even continent

12. NIKHEF Grid Tutorial, 3-4 June 2004 - 12 Solution using Grids Trivially parallel: break up problem “appropriate”-sized pieces Large Scales: 100k CPUs, petabytes of data  Assemble 100k+ CPUs and petabytes of mass storage  Don’t need to be in the same place! Large Dynamic Range: bursty usage patterns  When you need less than you have, others use excess capacity  When you need more, use others’ excess capacities Multiple user groups on single system  “Generic” grid software services (think web server here) Wide-area access requirements  Public Key Infrastructure for authentication & authorization

13. NIKHEF Grid Tutorial, 3-4 June 2004 - 13 HEP Use Cases Simulation Data (Re)Processing Physics Analysis General ideas presented here … contact us for detailed info

14. NIKHEF Grid Tutorial, 3-4 June 2004 - 14 Simulation The easiest use case  No input data  Output can be to a central location  Bookkeeping not really a problem (lost jobs OK) Define program version and parameters Tune # of events produced per run to “reasonable” value Submit (needed ev)/(ev per job) jobs Wait

15. NIKHEF Grid Tutorial, 3-4 June 2004 - 15 Data (Re)Processing Quite a bit more challenging: there are input files, and you can’t lose jobs One job per input file (so far) Data distribution strategy Monitoring and bookkeeping Software distribution Traceability of output (“provenance”)

16. NIKHEF Grid Tutorial, 3-4 June 2004 - 16 km3net Reconstruction Model Mediterranea n 10 Gb/s L1 Trigger Raw Data Cache > 1 TB > 1000 CPUs  Distributed Event Database?  Auto Distributed Files?  Single Mass Store + “Thermal Grid”? StreamService 1 Mb/s This needs work!! 2 Gbit/s is not a problem but you want many x 80 Gbit/s! Dual 1TB Circular Buffers? Distribute from shore station? Or dedicated line to better- connected location, distribute from there?? Grid useful here – get a lot but only when you need it! Grid data model applicable, but maybe not computational model …

17. NIKHEF Grid Tutorial, 3-4 June 2004 - 17 HEP Analysis Model Idea

18. NIKHEF Grid Tutorial, 3-4 June 2004 - 18 Conclusions HEP Computing well-suited to Grids HEP is using Grids now There is a lot of (fun) work to do!