The ATLAS Computing Model BTeV Talk 5/3/2019 The ATLAS Computing Model Roger Jones Lancaster University CHEP06 Mumbai 13 Feb. 2006 For January Collab Meeting

Brief summary ATLAS Facilities and their roles BTeV Talk 5/3/2019 Overview Brief summary ATLAS Facilities and their roles Growth of resources CPU, Disk, Mass Storage Network requirements CERN↔ Tier 1 ↔ Tier 2 Operational Issues and Hot Topics RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

BTeV Talk 5/3/2019 Computing Resources Computing Model fairly well evolved, documented in C-TDR Externally reviewed http://doc.cern.ch//archive/electronic/cern/preprints/lhcc/public/lhcc-2005-022.pdf There are (and will remain for some time) many unknowns Calibration and alignment strategy is still evolving Physics data access patterns MAY be exercised from June Unlikely to know the real patterns until 2007/2008! Still uncertainties on the event sizes , reconstruction time Lesson from the previous round of experiments at CERN (LEP, 1989-2000) Reviews in 1988 underestimated the computing requirements by an order of magnitude! RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

ATLAS Facilities Event Filter Farm at CERN Tier 0 Center at CERN BTeV Talk 5/3/2019 ATLAS Facilities Event Filter Farm at CERN Located near the Experiment, assembles data into a stream to the Tier 0 Center Tier 0 Center at CERN Raw data  Mass storage at CERN and to Tier 1 centers Swift production of Event Summary Data (ESD) and Analysis Object Data (AOD) Ship ESD, AOD to Tier 1 centers  Mass storage at CERN Tier 1 Centers distributed worldwide (10 centers) Re-reconstruction of raw data, producing new ESD, AOD Scheduled, group access to full ESD and AOD Tier 2 Centers distributed worldwide (approximately 30 centers) Monte Carlo Simulation, producing ESD, AOD, ESD, AOD  Tier 1 centers On demand user physics analysis CERN Analysis Facility Analysis Heightened access to ESD and RAW/calibration data on demand Tier 3 Centers distributed worldwide Physics analysis RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Processing Tier-0: Tier-1: BTeV Talk 5/3/2019 Processing Tier-0: Prompt first pass processing on express/calibration physics stream 24-48 hours later, process full physics data stream with reasonable calibrations Implies large data movement from T0 →T1s Tier-1: Reprocess 1-2 months after arrival with better calibrations Reprocess all resident RAW at year end with improved calibration and software Implies large data movement from T1↔T1 and T1 → T2 RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Analysis model Analysis model broken into two components BTeV Talk 5/3/2019 Analysis model Analysis model broken into two components Scheduled central production of augmented AOD, tuples & TAG collections from ESD Derived files moved to other T1s and to T2s Chaotic user analysis of augmented AOD streams, tuples, new selections etc and individual user simulation and CPU-bound tasks matching the official MC production Modest job traffic between T2s RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Inputs to the ATLAS Computing Model (1) BTeV Talk 5/3/2019 Inputs to the ATLAS Computing Model (1) RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Inputs to the ATLAS Computing Model (2) BTeV Talk 5/3/2019 Inputs to the ATLAS Computing Model (2) RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Data Flow Tier 0 view Tier 2 view EF farm  T0 BTeV Talk 5/3/2019 Data Flow EF farm  T0 320 MB/s continuous T0 Raw data  Mass Storage at CERN T0 Raw data  Tier 1 centers T0 ESD, AOD, TAG  Tier 1 centers 2 copies of ESD distributed worldwide T1  T2 Some RAW/ESD, All AOD, All TAG Some group derived datasets T2  T1 Simulated RAW, ESD, AOD, TAG T0  T2 Calibration processing? Tier 0 view Tier 2 view RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

ATLAS partial &“average” T1 Data Flow (2008) RAW ESD (2x) AODm (10x) 1 Hz 85K f/day 720 MB/s Tape RAW ESD2 AODm2 0.044 Hz 3.74K f/day 44 MB/s 3.66 TB/day RAW 1.6 GB/file 0.02 Hz 1.7K f/day 32 MB/s 2.7 TB/day Tier-0 disk buffer AODm1 500 MB/file 0.04 Hz 3.4K f/day 20 MB/s 1.6 TB/day AODm2 500 MB/file 0.04 Hz 3.4K f/day 20 MB/s 1.6 TB/day ESD1 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AODm1 500 MB/file 0.04 Hz 3.4K f/day 20 MB/s 1.6 TB/day RAW 1.6 GB/file 0.02 Hz 1.7K f/day 32 MB/s 2.7 TB/day AOD2 10 MB/file 0.2 Hz 17K f/day 2 MB/s 0.16 TB/day ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AOD2 10 MB/file 0.2 Hz 17K f/day 2 MB/s 0.16 TB/day AODm2 500 MB/file 0.004 Hz 0.34K f/day 2 MB/s 0.16 TB/day T1 Each Tier-2 Plus simulation and analysis data flow CPU farm ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AODm2 500 MB/file 0.036 Hz 3.1K f/day 18 MB/s 1.44 TB/day AODm2 500 MB/file 0.004 Hz 0.34K f/day 2 MB/s 0.16 TB/day ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AODm2 500 MB/file 0.036 Hz 3.1K f/day 18 MB/s 1.44 TB/day T1 Other Tier-1s disk storage T1 Other Tier-1s RWL Jones 13 Feb. 2006 Mumbai

Total ATLAS Requirements in for 2008 BTeV Talk 5/3/2019 Total ATLAS Requirements in for 2008 RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Important points: Discussion on disk vs tape storage at Tier-1’s BTeV Talk 5/3/2019 Important points: Discussion on disk vs tape storage at Tier-1’s Tape in this discussion means low-access secure storage No ‘disk buffers’ included except input to Tier 0 Storage of Simulation data from Tier 2’s Assumed to be at T1s Need partnerships to plan networking Must have fail-over to other sites Commissioning Requirement of flexibility in the early stages Simulation is a tunable parameter in T2 numbers! Heavy Ion running still under discussion. RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

ATLAS T0 Resources RWL Jones 13 Feb. 2006 Mumbai BTeV Talk 5/3/2019 For January Collab Meeting

ATLAS T1 Resources RWL Jones 13 Feb. 2006 Mumbai BTeV Talk 5/3/2019 For January Collab Meeting

ATLAS T2 Resources RWL Jones 13 Feb. 2006 Mumbai BTeV Talk 5/3/2019 For January Collab Meeting

Required Network Bandwidth BTeV Talk 5/3/2019 Required Network Bandwidth Caveats No safety factors No headroom Just sustained average numbers Assumes no years/datasets are ‘junked’ Physics analysis pattern still under study… RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Mainly outward data movement BTeV Talk 5/3/2019 T1 ↔ CERN Bandwidth I+O The projected time profile of the nominal bandwidth required between CERN and the Tier-1 cloud. Mainly outward data movement RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

About half is scheduled analysis BTeV Talk 5/3/2019 T1 ↔ T1 Bandwidth I+O About half is scheduled analysis The projected time profile of the nominal bandwidth required T1 and T1 cloud. RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

T1↔ T2 Bandwidth I+O Dominated by AOD BTeV Talk 5/3/2019 T1↔ T2 Bandwidth I+O Dominated by AOD The projected time profile of the nominal aggregate bandwidth expected for an average ATLAS Tier- 1 and its three associated Tier-2s. RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Issues 1: T1 Reprocessing Reprocessing at Tier 1s is understood in principle In practice, requires efficient recall of data from archive and processing Pinning, pre-staging, DAGs all required? Requires the different storage roles to be well understood RWL Jones 13 Feb. 2006 Mumbai

Issues 2: Streaming This is *not* a theological issue All discussions are about optimisation of data access TDR has 4 streams from event filter primary physics, calibration, express, problem events Calibration stream has split at least once since! At AOD, envisage ~10 streams ESD streaming? Straw man streaming schemes (trigger based) being agreed Will explore the access improvements in large-scale exercises Will also look at overlaps, bookkeeping etc RWL Jones 13 Feb. 2006 Mumbai

TAG Access TAG is a keyed list of variables/event Two roles Direct access to event in file via pointer Data collection definition function Two formats, file and database Now believe large queries require full database Restricts it to Tier1s and large Tier2s/CAF Ordinary Tier2s hold file-based TAG corresponding to locally-held datasets RWL Jones 13 Feb. 2006 Mumbai

BTeV Talk 5/3/2019 Conclusions Computing Model Data Flow understood for placing Raw, ESD and AOD at Tiered centers Still need to understand data flow implications of Physics Analysis SC4/Computing System Commissioning in 2006 is vital. Some issues will only be resolved with real data in 2007-8 RWL Jones 13 Feb. 2006 Mumbai For January Collab Meeting

Backup Slides RWL Jones 13 Feb. 2006 Mumbai

Heavy Ion Running RWL Jones 13 Feb. 2006 Mumbai BTeV Talk 5/3/2019 For January Collab Meeting