1. Data Challenge with the Grid in ATLAS
First Chinese-French Workshop on LHC Physics and associated Grid Computing
IHEP, Beijing
Ghita Rahal
CC-IN2P3

2. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Guidelines
Presentation of Lyon Tier-1
Test and data Challenges
Atlas Data management and data Model
Jobs processing and priorities
Some statistics on job production at CC
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

3. Chinese-French Workshop on LHC Physics an Associated Grid Computing
LYON Tier 1 Situation
Tier-2: GRIF
CEA/DAPNIA
LAL
LLR
LPNHE
IPNO
Strasbourg
Tier-3: IPHC
Paris-IdF
Nantes
Tier-2: Subatech
Tier-3: LAPP
Annecy
Clermont-Ferrand
Tier-2: LPC
Lyon
Tier-3: CPPM
Analysis Facility
Marseille
Tier-1: CC-IN2P3
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

4. Chinese-French Workshop on LHC Physics an Associated Grid Computing
LYON Tier-1 LCG France
Setup, develop and maintain an LCG Tier-1 and an Analysis Facility at CC-IN2P3 (Lyon)
Promote and coordinate the integration of Tier-2/Tier-3 French (and non French) sites into the LCG collaboration
Schedule
Started in July 2004
Two phases
: development and ramp-up
2009-…: cruise phase
Equipment budget for Tier-1 and Analysis Facility
: 16,4 M€
PROS: previous experience with large collaborations (BaBar, D0,…)
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

5. LYON TIER-1 contribution
Lyon participate to the 4 LHC experiment
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

6. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Guidelines
Presentation of Lyon Tier-1
Test and data Challenges
Atlas Data management and data Model
Jobs processing and priorities
Some statistics on job production at CC
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

7. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Tests TIER-1 Lyon in 2006
Lyon part of all test challenges to gain experience
Performance tests:
June-July 2006
September-October 2006
Goal: transfers Tier-0 to 10 Tier-1s: 800 MB/s in total of which 120MB/s to Lyon
Goal: transfer Tier-1 (Lyon) to Tier2: 75 MB/s
ATLAS Functional test
October 22 to 30
Goal: Check the system functionality during data transfer from CERN to Tier-1s and within Tier-1 clouds.
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

8. Performance tests T0=>T1
July 2006
Almost reached the goal for few hours
Problems from various sides (availability of the sites, of the services…. See summary)
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

9. Performance tests T0=>T1
October 2006
Overall weaker throughput: Simultaneous tests
decide to run multi VO tests in December and January.
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

10. Atlas Performance tests T1=>T2
ATLAS: continuous transfer from T1 to T2 sites initiated by the Tier 1
July 2006:
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

11. Atlas Performance tests T1=>T2
July 2006
Transfers to 7 Sites, T2 and non-T2 simultaneously
Some problem of limitations in the bandwidth for simultaneous transfers
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

12. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Multi-VO tests
2 days tests involving multi VO
Generate data at Tier-0 according to the rate transfer of each experiment
Transfer to all sites
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

13. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Multi-VO tests
Transfer Alice-Atlas-CMS to LYON Tier-1
Reached nominal transfer rates after few improvements…
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

14. Conclusions from Atlas tests
Achievements:
Global bandwidth from T0 reached 600 MB/s (700 MB/s peak) for few hours.
Transfers included most of the Tier-1 sites
Lyon one of the very few sites performing the whole chain, down to the distribution to the Tier-2 and -3 sites.
Problems (concerning all sites and Lyon also)
Overall system still unstable: instabilities coming from the different components (DQ2, FTS, dCache..)
Lack of monitoring => develop monitoring of FTS channels and dCache disks
Next : Focus on Tier-1 to Tiers-2 /Tiers-3 transfers
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

15. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Guidelines
Presentation of Lyon Tier-1
Test and data Challenges
Atlas Data management and data Model
Jobs processing and priorities
Some statistics on job production at CC
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

16. Chinese-French Workshop on LHC Physics an Associated Grid Computing
ATLAS Data Management
Atlas uses 3 grids: LCG, OSG and NorduGrid with their own services Requires an ATLAS layer over the Grid middleware
Atlas Model of computing and data distribution:
Storage capacity spread in T1 sites
Data stored in different storage systems with different access technologies.
Computing power distributed over all Tiers, 1, 2, 3 to produce MC and (re)process data
Tool to Distribute the data must:
Allow high performance and reliable data movement
Include information about data location and replication
Support multiple grid flavours.
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

17. ATLAS Tier-1 Data Flow (2008)
Real data storage, reprocessing and distribution
Tape
RAW
ESD2
AODm2
RAW
1.6 GB/file
0.02 Hz
1.7K f/day
32 MB/s
2.7 TB/day
0.044 Hz
3.74K f/day
44 MB/s
3.66 TB/day
Plus simulation & analysis data flow
Tier-0
disk buffer
AODm1
500 MB/file
0.04 Hz
3.4K f/day
20 MB/s
1.6 TB/day
AODm2
500 MB/file
0.04 Hz
3.4K f/day
20 MB/s
1.6 TB/day
ESD1
0.5 GB/file
0.02 Hz
1.7K f/day
10 MB/s
0.8 TB/day
AODm1
500 MB/file
0.04 Hz
3.4K f/day
20 MB/s
1.6 TB/day
ESD2
0.5 GB/file
0.02 Hz
1.7K f/day
10 MB/s
0.8 TB/day
AOD2
10 MB/file
0.2 Hz
17K f/day
2 MB/s
0.16 TB/day
AODm2
500 MB/file
0.004 Hz
0.34K f/day
2 MB/s
0.16 TB/day
RAW
1.6 GB/file
0.02 Hz
1.7K f/day
32 MB/s
2.7 TB/day T1
Tier-2s
CPU farm
ESD2
0.5 GB/file
0.02 Hz
1.7K f/day
10 MB/s
0.8 TB/day
AODm2
500 MB/file
0.036 Hz
3.1K f/day
18 MB/s
1.44 TB/day
AODm2
500 MB/file
0.004 Hz
0.34K f/day
2 MB/s
0.16 TB/day
ESD2
0.5 GB/file
0.02 Hz
1.7K f/day
10 MB/s
0.8 TB/day
Data access for analysis:
ESDm, AODm
ESD2
0.5 GB/file
0.02 Hz
1.7K f/day
10 MB/s
0.8 TB/day
AODm2
500 MB/file
0.036 Hz
3.1K f/day
18 MB/s
1.44 TB/day T1
Other Tier-1s
disk storage T1
Other Tier-1s
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

18. Bandwidth Requirements
INPUT
Source
Data Type
BANDWIDTH (MB/s)
Tier-0
RAW1
43,20
ESD1
51,00
AODm1
20,00
Total from T0
114,20
Total BNL
26,40
Total from other T1s
14,90
Associate Tier-2s
SIM MC
17,82
Total Input
173,32
OUTPUT
29,70
Other Tier-1s
AODm2
21,60
Total Tier-2s
74,93
Total Output
126,23
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

19. Chinese-French Workshop on LHC Physics an Associated Grid Computing
ATLAS tool : DDM
Tool to distribute the data within Atlas sites: based on DQ2 package.
Functionalities:
Transfer data between sites
Catalog data at each site
Remove old data at each site
All data movement is automated using subscription system.
How is it implemented?
Based on external services:
Set of DQ2 services running at each site (T0, T1)
gLite FTS
LFC: Local Replica Catalog
Tests of the tool: Performance and functional tests shown earlier
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

20. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Guidelines
Presentation of Lyon Tier-1
Test and data Challenges
Atlas Data management and data Model
Jobs processing and priorities
Some statistics on job production at CC
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

21. Priorities at the Sites
Sites deal with different types of jobs, user’s analysis jobs, production,…
Priorities must be established to favour processing/reprocessing of data or Monte-Carlo production
Roles defined at the level of the certificates
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

22. Atlas Production jobs versus all Atlas jobs
Goal: 80% of production jobs running at the site
All Atlas Jobs
% production jobs ~93%
Production Jobs
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

23. Atlas Production Jobs: submission
But works only if enough production jobs submitted by Atlas
Example: Peak of numbers of prod jobs running at ~60%;
Not enough jobs in queue
No steady stream of jobs in queue
Needs better tuning at the level of the Grid
Production Jobs in queue
Production Jobs running
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

24. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Guidelines
Presentation of Lyon Tier-1
Test and data Challenges
Atlas Data management and data Model
Establishment of job priorities
Some statistics on job production at CC
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

25. Production Jobs on the French Sites
22% of LCG
15% for July-September
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

26. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Global Contribution
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing

27. Chinese-French Workshop on LHC Physics an Associated Grid Computing
Summary
Tools to transfer, process and analyse the data on the grid are identified and extensively tested.
Still need to improve: reliability, stability, monitoring.
Communication between computing experts community and physicist community better established but needs more sustained efforts from both sides.
14/12/2006
Chinese-French Workshop on LHC Physics an Associated Grid Computing