Initial Planning towards The Full Dress Rehearsal Michael Ernst

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 2 Outline MotivationMotivation What to be exercisedWhat to be exercised Progressive Steps to ramp upProgressive Steps to ramp up MonitoringMonitoring Schedule (to be discussed)Schedule (to be discussed)

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 3 Introduction So far more questions than answers …So far more questions than answers …

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 4 Scope of Final Dress Rehearsal Stated by Fabiola in her IntroductionStated by Fabiola in her Introduction Generate O(10 7 ) events: few days of data taking, ~1 pb -1 at L=10 31 Mix and filter events to get correct physics mixture as expected at HLT output Pass events through G4 simulation (as-installed misaligned distorted geometry) Run Lvl1 simulation Produce byte streams  emulate raw data format Send raw data to Point1, pass through HLT nodes and SFO, write out events into streams, closing files at boundary of luminosity blocks Send events from Point 1 to Tier0; manipulate/merge files according to final model Perform calibration & alignment at Tier0 (and possibly also outside) Run reconstruction at Tier0 (and maybe Tier1s?)  produce ESD, AOD, TAG, DPD Distribute ESD, AOD, TAG, DPD to Tier1s and Tier2s; replicate databases Perform distributed analysis, use TAGs, producing additional group-specific DPD, etc. Run Data Quality at all levels of data production

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 5 Timescale & Duration Timescale?Timescale? Between June and October 2007 Duration?Duration? One problem is that the FDR competes for resources with the ongoing ATLAS detector commissioning which by then will be reaching its final stagesOne problem is that the FDR competes for resources with the ongoing ATLAS detector commissioning which by then will be reaching its final stages Primarily in the TDAQ, Tier-0 and Data Quality Monitoring Will require careful scheduling Series of 1 week “runs” separated by 2-3 weeks of analysis and preparation for the next one?Series of 1 week “runs” separated by 2-3 weeks of analysis and preparation for the next one? Would allow for 3-4 runs prior to low energy running These runs will provide good testbeds to exercise the ATLAS global shift operations infrastructureThese runs will provide good testbeds to exercise the ATLAS global shift operations infrastructure In fact these might be the schedule driver to have this in place Procedures in place to ensure all shift slots are covered, on-call rotas, etc.

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 6 Scale While the scope of the FDR should be as complete as possible, it’s important to scale it optimallyWhile the scope of the FDR should be as complete as possible, it’s important to scale it optimally Ability to inject events into the TDAQ system is likely to be the limiting factorAbility to inject events into the TDAQ system is likely to be the limiting factor It’s designed to get events out of Point1 at high rate, not to get them in Need to carefully evaluate injection alternativesNeed to carefully evaluate injection alternatives ROD, Lvl2, SFI, EF, SFO Understand what the rate restrictions are based on the existing hardwareUnderstand what the rate restrictions are based on the existing hardware It makes little sense to design and install additional hardware to support FDR Understand whether event cloning/replication can be used to increased throughputUnderstand whether event cloning/replication can be used to increased throughput We assume Lvl2/EF will be run in pass-through modeWe assume Lvl2/EF will be run in pass-through mode

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 7 Data Flow (adapted from Rob, Richard & Claude) Tier 0 Fast reco, calibrate Tier-1 transfer Prompt reco (bulk) Verify DB from online - config, calib -DCS,monitor +prompt calib digested status +digested status Tier-2 transfer +TAG DB (DQ status) Tier-1 Oracle replica Tier-2 replica RAW:200Hz320MB/s express calib ESD 100MB/s AOD 20MB/s ROD(B)s Front-end LVL1 LVL2 SFI (s) EF SFOs DCSOnlineDQA OfflineDQA Status summary Event, Lumi-Block OfflineDQA Analysis Model components missing

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 8 Proposed Strategy (D. Quarrie) Currently the emphasis has been on focused, orthogonal component testsCurrently the emphasis has been on focused, orthogonal component tests E.g. Tier-0 tests, SC4, TDAQ Large Scale Test, 3D project tests, etc. The Data Streaming test spans many of theseThe Data Streaming test spans many of these Provides valuable feedback on missing capabilities as well as it’s primary goalProvides valuable feedback on missing capabilities as well as it’s primary goal Proposal is to use this as the vehicle towards the FDRProposal is to use this as the vehicle towards the FDR In addition we now have some manpower to use it (at a smaller scale) as a regression testbed using RTTIn addition we now have some manpower to use it (at a smaller scale) as a regression testbed using RTT D. Quarrie’s much delayed Full Chain Test Identify missing or kludged functionality and adiabatically replace itIdentify missing or kludged functionality and adiabatically replace it Couple into ongoing component testsCouple into ongoing component tests

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 9 Coupling to Tier-0 Tests? Running since ~2 weeks (mixed experience, many problems in particular at CERNRunning since ~2 weeks (mixed experience, many problems in particular at CERN Another one in MayAnother one in May One constraint is number of nodes available at CERN in order to achieve bandwidth goalsOne constraint is number of nodes available at CERN in order to achieve bandwidth goals Real reconstruction code cannot be used without significantly reducing CPU time per event David proposes to use the real reconstruction code for the May testsDavid proposes to use the real reconstruction code for the May tests Perhaps with some Algorithms and/or output EDM disabled to fit bandwidth goals within cpu constraints An interim release 14.0.X should be available for that Subsequent tests should as far as possible use full reconstructionSubsequent tests should as far as possible use full reconstruction

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 10 Coupling to Calibration Data Challenge? Final phase of CDC is to test ability to determine and correct misalignment in timescale of 24 hoursFinal phase of CDC is to test ability to determine and correct misalignment in timescale of 24 hours As is required in computing model prior to Tier-0 processing Early CDC will use release 13.0.XEarly CDC will use release 13.0.X Final phase could also use release 14.0.XFinal phase could also use release 14.0.X Timescale matchesTimescale matches How best to couple to May Tier-0 tests?How best to couple to May Tier-0 tests?

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 11 Other Couplings Data Quality MonitoringData Quality Monitoring DDM & 3DDDM & 3D Metadata Catalog (AMI)Metadata Catalog (AMI) TAG DatabaseTAG Database COOL DatabaseCOOL Database Analysis ModelAnalysis Model Physicist involvementPhysicist involvement Absolutely crucial Need to understand proposed component test schedules and create plan coupling them togetherNeed to understand proposed component test schedules and create plan coupling them together Master planning document is at:Master planning document is at: ssioning?rev=6;filename=ATLAS-Offline-Computing.pdf

M. Ernst Tier 2 Meeting 8 March, 2007 UCSD 12 Some Technical Issues Implications of >2GB file sizeImplications of >2GB file size File sizes and how/where to merge?File sizes and how/where to merge? File sizes must be matched not only to e.g. DDM optimization, but also to job processing times (e.g. <<24 hour at Tier-0 per node) and operational constraints (e.g. run duration) Would it be technically feasible to perform merging via file concatenation instead of via read-in/write-out?Would it be technically feasible to perform merging via file concatenation instead of via read-in/write-out?