Data Challenge with the Grid in ATLAS First Chinese-French Workshop on LHC Physics and associated Grid Computing IHEP, Beijing 14-12-2006 Ghita Rahal CC-IN2P3

Chinese-French Workshop on LHC Physics an Associated Grid Computing Guidelines Presentation of Lyon Tier-1 Test and data Challenges Atlas Data management and data Model Jobs processing and priorities Some statistics on job production at CC 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing LYON Tier 1 Situation Tier-2: GRIF CEA/DAPNIA LAL LLR LPNHE IPNO Strasbourg Tier-3: IPHC Paris-IdF Nantes Tier-2: Subatech Tier-3: LAPP Annecy Clermont-Ferrand Tier-2: LPC Lyon Tier-3: CPPM Analysis Facility Marseille Tier-1: CC-IN2P3 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing LYON Tier-1 LCG France Setup, develop and maintain an LCG Tier-1 and an Analysis Facility at CC-IN2P3 (Lyon) Promote and coordinate the integration of Tier-2/Tier-3 French (and non French) sites into the LCG collaboration Schedule Started in July 2004 Two phases 2004-2008: development and ramp-up 2009-…: cruise phase Equipment budget for Tier-1 and Analysis Facility 2005-2008: 16,4 M€ PROS: previous experience with large collaborations (BaBar, D0,…) 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

LYON TIER-1 contribution Lyon participate to the 4 LHC experiment 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Guidelines Presentation of Lyon Tier-1 Test and data Challenges Atlas Data management and data Model Jobs processing and priorities Some statistics on job production at CC 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Tests TIER-1 Lyon in 2006 Lyon part of all test challenges to gain experience Performance tests: June-July 2006 September-October 2006 Goal: transfers Tier-0 to 10 Tier-1s: 800 MB/s in total of which 120MB/s to Lyon Goal: transfer Tier-1 (Lyon) to Tier2: 75 MB/s ATLAS Functional test October 22 to 30 Goal: Check the system functionality during data transfer from CERN to Tier-1s and within Tier-1 clouds. 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Performance tests T0=>T1 July 2006 Almost reached the goal for few hours Problems from various sides (availability of the sites, of the services…. See summary) 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Performance tests T0=>T1 October 2006 Overall weaker throughput: Simultaneous tests decide to run multi VO tests in December and January. 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Atlas Performance tests T1=>T2 ATLAS: continuous transfer from T1 to T2 sites initiated by the Tier 1 July 2006: 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Atlas Performance tests T1=>T2 July 2006 Transfers to 7 Sites, T2 and non-T2 simultaneously Some problem of limitations in the bandwidth for simultaneous transfers 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Multi-VO tests 2 days tests involving multi VO Generate data at Tier-0 according to the rate transfer of each experiment Transfer to all sites 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Multi-VO tests Transfer Alice-Atlas-CMS to LYON Tier-1 Reached nominal transfer rates after few improvements… 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Conclusions from Atlas tests Achievements: Global bandwidth from T0 reached 600 MB/s (700 MB/s peak) for few hours. Transfers included most of the Tier-1 sites Lyon one of the very few sites performing the whole chain, down to the distribution to the Tier-2 and -3 sites. Problems (concerning all sites and Lyon also) Overall system still unstable: instabilities coming from the different components (DQ2, FTS, dCache..) Lack of monitoring => develop monitoring of FTS channels and dCache disks Next : Focus on Tier-1 to Tiers-2 /Tiers-3 transfers 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Guidelines Presentation of Lyon Tier-1 Test and data Challenges Atlas Data management and data Model Jobs processing and priorities Some statistics on job production at CC 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing ATLAS Data Management Atlas uses 3 grids: LCG, OSG and NorduGrid with their own services Requires an ATLAS layer over the Grid middleware Atlas Model of computing and data distribution: Storage capacity spread in T1 sites Data stored in different storage systems with different access technologies. Computing power distributed over all Tiers, 1, 2, 3 to produce MC and (re)process data Tool to Distribute the data must: Allow high performance and reliable data movement Include information about data location and replication Support multiple grid flavours. 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

ATLAS Tier-1 Data Flow (2008) Real data storage, reprocessing and distribution Tape RAW ESD2 AODm2 RAW 1.6 GB/file 0.02 Hz 1.7K f/day 32 MB/s 2.7 TB/day 0.044 Hz 3.74K f/day 44 MB/s 3.66 TB/day Plus simulation & analysis data flow Tier-0 disk buffer AODm1 500 MB/file 0.04 Hz 3.4K f/day 20 MB/s 1.6 TB/day AODm2 500 MB/file 0.04 Hz 3.4K f/day 20 MB/s 1.6 TB/day ESD1 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AODm1 500 MB/file 0.04 Hz 3.4K f/day 20 MB/s 1.6 TB/day ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AOD2 10 MB/file 0.2 Hz 17K f/day 2 MB/s 0.16 TB/day AODm2 500 MB/file 0.004 Hz 0.34K f/day 2 MB/s 0.16 TB/day RAW 1.6 GB/file 0.02 Hz 1.7K f/day 32 MB/s 2.7 TB/day T1 Tier-2s CPU farm ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AODm2 500 MB/file 0.036 Hz 3.1K f/day 18 MB/s 1.44 TB/day AODm2 500 MB/file 0.004 Hz 0.34K f/day 2 MB/s 0.16 TB/day ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day Data access for analysis: ESDm, AODm ESD2 0.5 GB/file 0.02 Hz 1.7K f/day 10 MB/s 0.8 TB/day AODm2 500 MB/file 0.036 Hz 3.1K f/day 18 MB/s 1.44 TB/day T1 Other Tier-1s disk storage T1 Other Tier-1s 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Bandwidth Requirements INPUT Source Data Type BANDWIDTH (MB/s) Tier-0 RAW1 43,20 ESD1 51,00 AODm1 20,00 Total from T0   114,20 Total BNL 26,40 Total from other T1s 14,90 Associate Tier-2s SIM MC 17,82 Total Input 173,32 OUTPUT 29,70 Other Tier-1s AODm2 21,60 Total Tier-2s 74,93 Total Output 126,23 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing ATLAS tool : DDM Tool to distribute the data within Atlas sites: based on DQ2 package. Functionalities: Transfer data between sites Catalog data at each site Remove old data at each site All data movement is automated using subscription system. How is it implemented? Based on external services: Set of DQ2 services running at each site (T0, T1) gLite FTS LFC: Local Replica Catalog Tests of the tool: Performance and functional tests shown earlier 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Guidelines Presentation of Lyon Tier-1 Test and data Challenges Atlas Data management and data Model Jobs processing and priorities Some statistics on job production at CC 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Priorities at the Sites Sites deal with different types of jobs, user’s analysis jobs, production,… Priorities must be established to favour processing/reprocessing of data or Monte-Carlo production Roles defined at the level of the certificates 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Atlas Production jobs versus all Atlas jobs Goal: 80% of production jobs running at the site All Atlas Jobs % production jobs ~93% Production Jobs 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Atlas Production Jobs: submission But works only if enough production jobs submitted by Atlas Example: Peak of numbers of prod jobs running at ~60%; Not enough jobs in queue No steady stream of jobs in queue Needs better tuning at the level of the Grid Production Jobs in queue Production Jobs running 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Guidelines Presentation of Lyon Tier-1 Test and data Challenges Atlas Data management and data Model Establishment of job priorities Some statistics on job production at CC 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Production Jobs on the French Sites 22% of LCG 15% for July-September 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Global Contribution 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing

Chinese-French Workshop on LHC Physics an Associated Grid Computing Summary Tools to transfer, process and analyse the data on the grid are identified and extensively tested. Still need to improve: reliability, stability, monitoring. Communication between computing experts community and physicist community better established but needs more sustained efforts from both sides. 14/12/2006 Chinese-French Workshop on LHC Physics an Associated Grid Computing