1. ATLAS Data Challenges CHEP03 La Jolla, California 24 th March 2003 Gilbert Poulard ATLAS Data Challenges Co-ordinator CERN EP-ATC For the ATLAS DC & Grid team

2. March 24, 2003G. Poulard - CHEP032 Outline nIntroduction nATLAS Data Challenges activities nGrid activities in ATLAS DC1 nBeyond DC1 nSummary nConclusion

3. March 24, 2003G. Poulard - CHEP033 Introduction nSeveral talks on the LHC project and on GRID projects in this conference u I will not ‘repeat’ what is said in several other presentations on LHC, LCG and GRID nThis talk concentrates on the on-going “Data Challenges” activities of ATLAS, one of the four LHC experiments

4. March 24, 2003G. Poulard - CHEP034 ATLAS experiment Diameter25 m Barrel toroid length26 m Endcap end-wall chamber span46 m Overall weight 7000 Tons ATLAS: ~ 2000 Collaborators ~150 Institutes 34 Countries

5. March 24, 2003G. Poulard - CHEP035 ATLAS “and building 40”

6. March 24, 2003G. Poulard - CHEP036 Aims of the Experiment 5   discovery ~1 year ~3 years ~ 4 years present limit Search for the Higgs boson Measure the Standard Model Higgs Boson Detect Supersymmetric states Study Standard Model QCD, EW, HQ Physics New Physics? ATLAS H  ZZ (*)  4 ~1 year

7. March 24, 2003G. Poulard - CHEP037 Detectors Front-end Pipelines Readout Buffers Event Builder Buffers & Processing Farms Data Storage Readout Drivers LVL2 RoI –Region-of- Interest (RoI) –Specialized algorithms –Fast selection with early rejection EF u Full event available u Offline derived algorithms u Seeding by LVL2 u Best calibration / alignment u Latency less demanding LVL1 u Hardware based (FPGA and ASIC) u Coarse calorimeter granularity u Trigger muon detectors: RPCs and TGCs The ATLAS Trigger/DAQ System 1 GHz interaction rate / <75 (100) kHz O (1) kHz output rate O (100) Hz output event rate ~100 GB/s output data flow O (100) MB/s output data flow O (1) GB/s output data flow 2  s latency O (10) ms latency ~ seconds latency 40 MHz bunch-crossing rate RoI Pointers HLT

8. March 24, 2003G. Poulard - CHEP038 Data nEvery event will consist of 1-1.5 MB (all detectors together) nAfter on-line selection, events will be written to permanent storage at a rate of 100-200 Hz nTotal amount of “raw” data:  1 PB/year nTo reconstruct and analyze this data: Complex “Worldwide Computing Model” and “Event Data Model” u Raw Data @ CERN u Reconstructed data “distributed” u All members of the collaboration must have access to “ALL” public copies of the data (at a reasonable access speed)

9. March 24, 2003G. Poulard - CHEP039 ATLAS Computing Challenge nThe emerging World Wide computing model “is an answer” to the LHC computing challenge nIn this model the Grid must take care of: u data replicas and catalogues u condition data base replicas, updates and synchronization u access authorizations for individual users, working groups, production managers u access priorities and job queues nValidation of the new Grid computing paradigm in the period before the LHC requires Data Challenges of increasing scope and complexity

10. March 24, 2003G. Poulard - CHEP0310 nData Challenges are the way to test the prototype infrastructure before the start of the real experiment (2007) nATLAS plans to run one Data Challenge per year, with increasing complexity and amount of data nEach Data Challenge consists of the following steps:  Physics event generation (Pythia, Herwig,...)  Event simulation (Geant3, Geant4)  Background simulation, pile-up and detector response simulation (all these depend on luminosity)  Event reconstruction  Event analysis nData can be (re-)distributed to different production sites between any of these steps } this is the real challenge! Systems Tests: Data Challenges

11. March 24, 2003G. Poulard - CHEP0311 DC0: readiness & continuity tests (December 2001 – June 2002) n“3 lines” for “full” simulation u 1) Full chain with new geometry (as of January 2002) Generator->(Objy)->Geant3->(Zebra->Objy)->Athena recon.->(Objy)->Analysis u 2) Reconstruction of ‘Physics TDR’ data within Athena (Zebra->Objy)->Athena rec.-> (Objy) -> Simple analysis u 3) Geant4 robustness test Generator-> (Objy)->Geant4->(Objy) n“1 line” for “fast” simulation Generator-> (Objy) -> Atlfast -> (Objy) Continuity test: Everything from the same release for the full chain (3.0.2) we learnt a lot (we underestimated the implications of that statement) completed in June 2002

12. March 24, 2003G. Poulard - CHEP0312 ATLAS DC1 Phase I (July-August 2002) nPrimary concern was delivery of events to High Level Trigger (HLT) community u Goal ~10 7 events (several samples!) nPut in place the MC event generation & detector simulation chain u Switch to AthenaRoot I/O (for Event generation) u Updated geometry u Filtering u Validate the chain: Athena/Event Generator -> (Root I/O)->Atlsim/Dice/Geant3->(Zebra) nPut in place the distributed MonteCarlo production u “ATLAS kit” (rpm) u Scripts and tools (monitoring, bookkeeping)  AMI database; Magda replica catalogue; VDC u Quality Control and Validation of the full chain

13. March 24, 2003G. Poulard - CHEP0313 Tools used in DC1 AMIMagda VDC AtCom GRAT replica catalog physics metadata recipe catalog Perm. production log Trans. production log physics metadata perm production log trans production log replica catalog recipe catalog interactive production framework automatic production framework AMI physics metadata

14. March 24, 2003G. Poulard - CHEP0314 ATLAS Geometry Scale of the problem: n25,5 millions distinct volume copies n23 thousands different volume objects n4,673 different volume types nmanaging up to few hundred pile-up events none million hits per event on average

15. March 24, 2003G. Poulard - CHEP0315 Atlsim/Geant3 + Filter 10 5 events Atlsim/Geant3 + Filter Hits/ Digits MCTruth Atlsim/Geant3 + Filter As an example, for 1 sample of di-jet events:  Event generation: 1.5 x 10 7 events in 150 partitions  Detector simulation: 3000 jobs Pythia 6 Di-jet Athena-Root I/O Zebra HepMC Event generation DC1/Phase I Task Flow Detector Simulation (5000 evts) (~450 evts) Hits/ Digits MCTruth Hits/ Digits MCtruth

16. March 24, 2003G. Poulard - CHEP0316 DC1: validation & quality control nWe defined two types of validation u Validation of the sites:  We processed the same data in the various centres and made the comparison To insure that the same software was running in all production centres We also checked the random number sequences u Validation of the simulation:  We used both “old” generated data & “new” data Validation datasets: di-jets, single ,e, ,H  4e/2  /2e2  /4  About 10 7 evts reconstructed in June, July and August Comparison made also with previous simulations n“QC” is a “key issue” for the success

17. March 24, 2003G. Poulard - CHEP0317 Comparison Procedure Test sample Reference sample Superimposed Samples Contributions to  2

18. March 24, 2003G. Poulard - CHEP0318 Comparison procedure ends with a  2 -bar chart summary Give a pretty nice overview of how samples compare: Summary of Comparison

19. March 24, 2003G. Poulard - CHEP0319 Validation samples (740k events)  single particles (e, , ,  ), jet scans, Higgs events Single-particle production (30 million events)  single  (low p T ; p T =1000 GeV with 2.8<  <3.2)  single  (p T =3, …, 100 GeV)  single e and  different energies (E=5, 10, …, 200, 1000 GeV) fixed  points;  scans (|  |<2.5);  crack scans (1.3<  <1.8) standard beam spread (  z =5.6 cm); fixed vertex z-components (z=0, 4, 10 cm) Minimum-bias production (1.5 million events)  different  regions (|  |<3, 5, 5.5, 7) Data Samples I

20. March 24, 2003G. Poulard - CHEP0320 QCD di-jet production (5.2 million events)  different cuts on E T (hard scattering) during generation large production of E T >11, 17, 25, 55 GeV samples, applying particle-level filters large production of E T >17, 35 GeV samples, without filtering, full simulation within |  |<5 smaller production of E T >70, 140, 280, 560 GeV samples Physics events requested by various HLT groups (e/ , Level-1, jet/E T miss, B-physics, b-jet,  ; 4.4 million events)  large samples for the b-jet trigger simulated with default (3 pixel layers) and staged (2 pixel layers) layouts  B-physics (PL) events taken from old TDR tapes Data Samples II

21. March 24, 2003G. Poulard - CHEP0321 ATLAS DC1 Phase 1 : July-August 2002 3200 CPU‘s 110 kSI95 71000 CPU days 5*10* 7 events generated 1*10* 7 events simulated 3*10* 7 single particles 30 Tbytes 35 000 files 39 Institutes in 18 Countries 1.Australia 2.Austria 3.Canada 4.CERN 5.Czech Republic 6.France 7.Germany 8.Israel 9.Italy 10.Japan 11.Nordic 12.Russia 13.Spain 14.Taiwan 15.UK 16.USA grid tools used at 11 sites

22. March 24, 2003G. Poulard - CHEP0322 ATLAS DC1 Phase II (November 02/March 03) nProvide data with and without ‘pile-up’ for HLT studies u Pile-up production u new data samples (huge amount of requests) u “Byte stream” format to be produced nIntroduction & testing of new Event Data Model (EDM) u This includes new Detector Description nProduction of data for Physics and Computing Model studies u Both ESD and AOD produced from Athena Reconstruction nTesting of computing model & of distributed analysis using AOD nUse more widely GRID middleware

23. March 24, 2003G. Poulard - CHEP0323 Luminosity Effect Simulation Aim Study Interesting Processing at different Luminosity L (cm -2 s -1 ) Separate Simulation of Physics Events & Minimum Bias Events and cavern background for muon studies Merging of Primary Stream (Physics) Background Stream(s) (Pileup (& cavern background)) Primary Stream (KINE,HITS) Background Stream (KINE,HITS DIGITIZATION Bunch Crossing (DIGI) 1 N( L )

24. March 24, 2003G. Poulard - CHEP0324 Pile-up features nDifferent detectors have different memory time requiring very different number of minimum bias events to be read in u Silicons, Tile calorimeter: t<25 ns u Straw tracker: t<~40-50 ns u Lar Calorimeters: 100-400 ns u Muon Drift Tubes: 600 ns nStill we want the pile-up events to be the same in different detectors ! nFor Muon studies: in addition “Cavern Background”

25. March 24, 2003G. Poulard - CHEP0325 Pile-up task flow ATLSIM Minimum bias 0.5 MB (460 sec) Cavern Background 20 KB (0.4 sec) Background 0.5 MB Physics 2 MB (340 sec) Pile-up 7.5 MB (@HL) 400 sec (Mixing:80 Digitization: 220) 0.03 sec High Luminosity: 10 34  23 events/bunch crossing  61 bunch crossings Low luminosity: 2 x 10 33

26. March 24, 2003G. Poulard - CHEP0326 Higgs into two photons no pile-up

27. March 24, 2003G. Poulard - CHEP0327 Higgs into two photons L=10^34 pile-up

28. March 24, 2003G. Poulard - CHEP0328 ATLAS DC1/Phase II: November 2002-March 2003 Goals : Produce the data needed for the HLT TDR Get as many ATLAS institutes involved as possible Worldwide collaborative activity Participation : 56 Institutes nAustralia nAustria nCanada nCERN nChina nCzech Republic nDenmark * nFrance nGermany nGreece nIsrael nItaly nJapan nNorway * nPoland nRussia nSpain nSweden * nTaiwan nUK nUSA * u New countries or institutes u * using Grid

29. March 24, 2003G. Poulard - CHEP0329 Preparation for Reconstruction nOn-going activities (in several areas) u Put in place the infrastructure for the production u Get the “reconstruction” software ready and validated  Both Physics & HLT communities involved u Include the dedicated code for HLT studies  Lvl1, Lvl2 & Event Filter u Today we are in the validation phase u End of March we expect to reconstruct and analyse  a full high statistics sample without pile-up  ~ 10% of a high statistics sample with pile-up u Data being concentrated in 8 “sites” u Production both on “standard batch” or “GRID” systems

30. March 24, 2003G. Poulard - CHEP0330 Primary data (in 8 sites) Data (TB) Simulation: 23.7 (40%) Pile-up: 35.4 (60%) Lumi02: (14.5) Lumi10: (20.9) Pile-up: Low luminosity ~ 4 x 10 6 events (~ 4 x 10 3 NCU days) High luminosity ~ 3 x 10 6 events ( ~ 12 x 10 3 NCU days) Data replication using Grid tools (Magda)

31. March 24, 2003G. Poulard - CHEP0331 Grid in ATLAS DC1 * US-ATLAS EDG Testbed Prod NorduGrid part of Phase 1 reproduce part of full phase 1 & 2 production phase 1 data production Full Phase 2 several tests production [ * See other ATLAS talks for more details]

32. March 24, 2003G. Poulard - CHEP0332 DC1 production on the Grid nGrid test-beds in Phase 1 u 11 out of 39 sites (~5% of the total production)  NorduGrid (Bergen, Grendel, Ingvar, OSV, NBI,Oslo,Lund,LSCF) all production done on the Grid  US-Grid (LBL, UTA, OU) ~10% of US DC1 production (~900 CPU.days) nPhase 2 u NorduGrid (full pile-up production) u US Grid  Pile-up in progress  ~ 8TB of pile-up data, 5000 CPU.days, 6000 Jobs u Will be used for reconstruction

33. March 24, 2003G. Poulard - CHEP0333 Summary on DC1 nPhase 1 (summer 2002) was a real success nThe pile-up production ran quite smoothly u Expects to have it completed by end of March nThe concentration of the data is on its way u Replication mostly performed with “Magda” nProgress are being made in the organization u Integration of tools (production, bookkeeping, replication…) nValidation of the “offline” reconstruction software is progressing well u HLT dedicated software will then have to be added n“Massive” production for reconstruction expected by beginning of April

34. March 24, 2003G. Poulard - CHEP0334 DC2-3-4-… nDC2: u Probably Q4/2003 – Q2/2004 u Goals  Full deployment of EDM & Detector Description  Geant4 replacing Geant3  Test the calibration and alignment procedures  Use LCG common software (POOL, …)  Use widely GRID middleware  Perform large scale physics analysis  Further tests of the computing model (Analysis)  Run on LCG-1 u Scale  As for DC1: ~ 10**7 fully simulated events nDC3: Q3/2004 – Q2/2005 u Goals to be defined; Scale: 5 x DC2 nDC4: Q3/2005 – Q2/2006 u Goals to be defined; Scale: 2 X DC3

35. March 24, 2003G. Poulard - CHEP0335 Summary (1) nATLAS computing is in the middle of first period of Data Challenges of increasing scope and complexity and is steadily progressing towards a highly functional software suite, plus a World Wide computing model, which gives all ATLAS equal and equal quality of access to ATLAS data

36. March 24, 2003G. Poulard - CHEP0336 Summary (2) nThese Data Challenges are executed at the prototype tier centers and use as much as possible the Grid middleware being developed in Grid projects around the world

37. March 24, 2003G. Poulard - CHEP0337 Conclusion Quite promising start for ATLAS Data Challenges!

38. March 24, 2003G. Poulard - CHEP0338 Thanks to all DC-team members (working in 14 work packages) A-WP1: Event generation A-WP2: Geant3 simulation A-WP3: Geant4 Simulatio n A-WP4: Pile-up A-WP4: Pile-up A-WP5: Detector response A-WP6: Data Conversio n A-WP6: Data Conversio n A-WP7: Event filtering A-WP9: Analysis A-WP10: Data Management A-WP10: Data Management A-WP8: Reconstructio n A-WP11: Tools A-WP12: Teams Production Validation …. A-WP12: Teams Production Validation …. A-WP13: Tier Centres A-WP14: Fast Simulatio n