1. WLCG Workshop @CERN November 13 2008 Plan for shutdown and 2009 data-taking Kors Bos

2. WLCG Workshop @CERN November 13 2008 Plans for 2008/2009 o September + October 2008: cosmic ray data taking (see next slide) o November + December 2008: preparations re- processing and analysis o Early 2009: reprocessing 2008 cosmic ray data (reduction to ~20%) and data analysis o Official statement from CERN management next week in Council o But we already know the “optimistic” scenario:  Jan-April: Functional Tests, Rre-Processing, MC Prod., Analysis Challenges, …  May-August: cosmic ray detector performance studies  Spetember-December: collisions data taking November 28 2008CNAF, Bologna2

3. WLCG Workshop @CERN November 13 2008 Data Type June 24 th - October 28 th, 2008January 1 st - June 24 th, 2008 FilesEventsTot. Size [MB]FilesEventsTot. Size [MB] RAW 693,509464,362,1391,104,305,94387,79640,059,042136,324,739 CALRAW 17,028110,463,48817,558,049130n/a1,886 ESD 651,109434,868,635240,578,525104,64556,379,21814,017,253 ESD_FILTERED 474,716n/a7,795,53470,284n/a242,337 CBNT 651,109434,868,635404,906,622104,64556,379,21821,286,696 AOD 2,57279,046,9221,894,468n/a TAG 2,57279,046,9224,407n/a TAG_COMM 4,623325,676,27827,5581,91417,289,650388 HIST 5,378n/a183,5942278n/a77,183 NTUP_PIXELCALIB 354n/a21,974n/a NTUP_MUONCALIB 5,022n/a2,285,878n/a Glossary: RAW: physics and debug streams from DAQ CALRAW: calibration streams from DAQ ESD_FILTERED: ESDs of events with at least one ID track TAG_COMM:commissioning TAGs (produced in reconstruction step) TAG:physics TAGs (produced in AOD merging step) HIST:merged DQM histograms NTUP_*CALIB:merged n-tuples for Pixel and Muon detector calibration purposes RAW event size = 2.4 MB because of many samplings nominal = 1.6 MB

4. WLCG Workshop @CERN November 13 2008 Data Type June 24 th - October 28 th, 2008January 1 st - June 24 th, 2008 FilesEventsTot. Size [MB]FilesEventsTot. Size [MB] RAW 693,509464,362,1391,104,305,94387,79640,059,042136,324,739 CALRAW 17,028110,463,48817,558,049130n/a1,886 ESD 651,109434,868,635240,578,525104,64556,379,21814,017,253 ESD_FILTERED 474,716n/a7,795,53470,284n/a242,337 CBNT 651,109434,868,635404,906,622104,64556,379,21821,286,696 AOD 2,57279,046,9221,894,468n/a TAG 2,57279,046,9224,407n/a TAG_COMM 4,623325,676,27827,5581,91417,289,650388 HIST 5,378n/a183,5942278n/a77,183 NTUP_PIXELCALIB 354n/a21,974n/a NTUP_MUONCALIB 5,022n/a2,285,878n/a Glossary: RAW: physics and debug streams from DAQ CALRAW: calibration streams from DAQ ESD_FILTERED: ESDs of events with at least one ID track TAG_COMM:commissioning TAGs (produced in reconstruction step) TAG:physics TAGs (produced in AOD merging step) HIST:merged DQM histograms NTUP_*CALIB:merged n-tuples for Pixel and Muon detector calibration purposes ESD event size = 0.6 MB nominal 1 MB

5. WLCG Workshop @CERN November 13 2008 Data Type June 24 th - October 28 th, 2008January 1 st - June 24 th, 2008 FilesEventsTot. Size [MB]FilesEventsTot. Size [MB] RAW 693,509464,362,1391,104,305,94387,79640,059,042136,324,739 CALRAW 17,028110,463,48817,558,049130n/a1,886 ESD 651,109434,868,635240,578,525104,64556,379,21814,017,253 ESD_FILTERED 474,716n/a7,795,53470,284n/a242,337 CBNT 651,109434,868,635404,906,622104,64556,379,21821,286,696 AOD 2,57279,046,9221,894,468n/a TAG 2,57279,046,9224,407n/a TAG_COMM 4,623325,676,27827,5581,91417,289,650388 HIST 5,378n/a183,5942278n/a77,183 NTUP_PIXELCALIB 354n/a21,974n/a NTUP_MUONCALIB 5,022n/a2,285,878n/a Glossary: RAW: physics and debug streams from DAQ CALRAW: calibration streams from DAQ ESD_FILTERED: ESDs of events with at least one ID track TAG_COMM:commissioning TAGs (produced in reconstruction step) TAG:physics TAGs (produced in AOD merging step) HIST:merged DQM histograms NTUP_*CALIB:merged n-tuples for Pixel and Muon detector calibration purposes CALRAW and CBNT only at the Tier-0

6. WLCG Workshop @CERN November 13 2008 Data Type June 24 th - October 28 th, 2008January 1 st - June 24 th, 2008 FilesEventsTot. Size [MB]FilesEventsTot. Size [MB] RAW 693,509464,362,1391,104,305,94387,79640,059,042136,324,739 CALRAW 17,028110,463,48817,558,049130n/a1,886 ESD 651,109434,868,635240,578,525104,64556,379,21814,017,253 ESD_FILTERED 474,716n/a7,795,53470,284n/a242,337 CBNT 651,109434,868,635404,906,622104,64556,379,21821,286,696 AOD 2,57279,046,9221,894,468n/a TAG 2,57279,046,9224,407n/a TAG_COMM 4,623325,676,27827,5581,91417,289,650388 HIST 5,378n/a183,5942278n/a77,183 NTUP_PIXELCALIB 354n/a21,974n/a NTUP_MUONCALIB 5,022n/a2,285,878n/a Glossary: RAW: physics and debug streams from DAQ CALRAW: calibration streams from DAQ ESD_FILTERED: ESDs of events with at least one ID track TAG_COMM:commissioning TAGs (produced in reconstruction step) TAG:physics TAGs (produced in AOD merging step) HIST:merged DQM histograms NTUP_*CALIB:merged n-tuples for Pixel and Muon detector calibration purposes AOD very small for cosmics TAGs always small, but many

7. WLCG Workshop @CERN November 13 2008 2008 Cosmic Ray Data Sample o 460M events RAW and ESD at CERN o and distributed to T1’s to tape and disk  first: RAW to tape and ESD to disk  after half September: RAW to tape and disk and ESD to disk o A Total of 1.35 PB to disk and 1.1 PB to tape in Tier-1’s o in 3.5 months  ~50 HZ o nominal 200 HZ, but sometimes lower because of big event size o 110M events CALRAW to CAF for calibration o This may be a good guidance of what we will do in 2009 before collisions November 28 2008CNAF, Bologna7

8. WLCG Workshop @CERN November 13 2008 2008 Cosmic Ray Data Reduction o reduction to 20% of original volume o also produce Reduced RAW which must go to tape (and disk?) o but certainly Reduced ESD, as a matter of fact these are DPD’s o will still be ~200 TB of Redu-RAW and ~50 TB of Redu-ESD o we would like to keep this on disk in 2009 for further analysis o most original RAW on disk, limited pre-staging needed o new ESD will be distributed according to MoU o old ESD may then be deleted (keep copy on tape at CERN) November 28 2008CNAF, Bologna8

9. WLCG Workshop @CERN November 13 2008 Cosmic Ray Data Re-Processing o suitable runs and datasets have been defined for re-processing  305,000 files~500 TB o data is truly distributed, so all Tier-1’s are needed o we can at best replace 1 Tier-1 by CERN o Processing times in Tier-1’s may vary quite a bit  not similar CPU power at the Tier-1’s  not optimized data distribution o Conditions DB access was a problem, solved now (?) o first tests ongoing now with pre-released software November 28 2008CNAF, Bologna9

10. WLCG Workshop @CERN November 13 2008 several stages o not final software, just 1 dataset, only done at CERN … now  get first performance numbers o not final software, reference datasets, done everywhere  for validation: compare at event level  get first performance numbers: penalty for distribution o repeat with final software  compare again o full re-processing on 300,000 files November 28 2008CNAF, Bologna10

11. WLCG Workshop @CERN November 13 2008 Re-Processing Bandwidth o if we assume a typical 10% T1 has 1000 cores for re-processing o the reconstruction of an event takes typically 10 sec on an average CPU (maybe a little less for these cosmics) o an event is 2.4 MB (maybe less for collisions data) o so we need a bandwidth of 240 MB/sec per 1000 CPU’s o If we were to pre-stage those data from tape o we would need to bring ~1 TB/hour on-line November 28 2008CNAF, Bologna11

12. WLCG Workshop @CERN November 13 2008 November 28 2008CNAF, Bologna12

13. WLCG Workshop @CERN November 13 2008 DDM Functional Tests o we get a VERY BIG one coming up o next 2 weeks we accumulate a lot of data in the T1’s (on DATADISK) o in week 3 we then launch T1-T1 transfers all at once this will test (a new vs of) DDM at an order of magnitude higher rates o otherwise we shall always run the DDM FT o to keep testing all sites o new: now also MCDISK included in the tests o and this is badly needed, unfortunately  November 28 2008CNAF, Bologna13

14. WLCG Workshop @CERN November 13 2008 MC Production Functional Tests o running at low rate now: 200 jobs/cloud*week o need to increase the rate to become more significant o need to be brokered indiscriminately so all T2 sites get jobs o need more atomization o new dashboard available now and and shifters should look at it o we should always run them, at lower priority than real MC o also used for testing new features (like merging) o need to see if priorities work November 28 2008CNAF, Bologna14

15. WLCG Workshop @CERN November 13 2008 Merging o first in Monte Carlo o generating 1 G4 event takes 1000 seconds! o jobs with 50 events  HITS files of 100 MB o merge 40 into 1  HITS file of 4 GB and 2000 events o digi+ reco on 2000 events  RDO file of 2 GB and AOD of 200 MB o merge 10 into 1  AOD file of 2 GB November 28 2008CNAF, Bologna15

16. WLCG Workshop @CERN November 13 2008 Distributed Analysis Functional Tests o real user analyses using GANGA ran in Italian, German, UK and Dutch cloud  AOD analysis in Milano, Bologna, Roma, Frascati and Napoli  5 people submitting 50*5 jobs from UK, Germany, CERN and Italy  now automized in a script o should be expanded to all clouds  but needs people watching in (almost) all Tier-2’s  should also include the US o should also be done with Pathena  but then also in the EU clouds o need to identify real user analysis use-cases o need to be tested with Panda and WMS backend in EU clouds o need to see if priorities work November 28 2008CNAF, Bologna16

17. WLCG Workshop @CERN November 13 2008 Analysis Bandwidth o if we assume a typical T2 has 400 cores for analysis o an AOD analysis using GANGA reaches typically 15 Hz  on an average CPU, data on NFS mounted disk o an AOD event is ~0.2 MB o so we need a bandwidth of 1.2 GB/sec (10 Gb) per 400 CPU’s o don’t need to bring those AOD’s on-line from tape o don’t know if DPD or NTUP analysis is faster o have encountered LAN bandwidth problems in Italian T2’s o not using DB access yet November 28 2008CNAF, Bologna17

18. WLCG Workshop @CERN November 13 2008 Organizational o Post Mortem WS @CERN: January 19-23  T01/2/3 Jamboree on Thursday January 22  Shifters and other tutorials on Friday January 23 o T2/3 Jamborees organized per cloud  open to all people  central operations will attend o Analysis workshops  combined with the physics coordination workshops (?)  need integration between US and EU clouds  and NDGF and Asia for that purpose November 28 2008CNAF, Bologna18

19. WLCG Workshop @CERN November 13 2008 other issues for Bologna o 2009 resources: 1700 kSI2K, 736 TB disk, 832 TB tape o analysis model: ATLAS vs Italian users, Tier-3’s November 28 2008CNAF, Bologna19