1. The LHC Computing Grid Visit of Dr. John Marburger
Director, Office of Science and Technology Policy
Frédéric Hemmer
Deputy Head, IT Department
July 10, 2006
The LHC Computing Grid – July 2006

2. Outline The Challenge The (W)LCG Project The LCG Infrastructure
The LCG Service
Beyond LCG
The LHC Computing Grid – July 2006

3. New frontiers in data handling
ATLAS experiment:
~150 million 40MHz
~ 10 million Gigabytes per second
Massive data reduction on-line
Still ~1 Gigabyte per second to handle
The LHC Computing Grid – July 2006

4. The Data Challenge
LHC experiments will produce million Gigabytes of data each year (about 20 million CDs!)
LHC data analysis requires a computing power equivalent to ~ 100,000 of today's fastest PC processors.
Requires many cooperating computer centres, CERN providing only ~20% of the computing resources
The LHC Computing Grid – July 2006

5. (extracted by physics topic)
Data Handling and Computation for Physics Analysis
CERN
reconstruction
detector
event filter
(selection &
reconstruction)
analysis
processed
data
event
summary
data
raw
data
batch
physics
analysis
event
reprocessing
simulation
analysis objects
(extracted by physics topic)
event
simulation
interactive
physics
analysis
The LHC Computing Grid – July 2006

6. LCG Service Hierarchy Tier-0 – the accelerator centre
Data acquisition & initial processing
Long-term data curation
Data Distribution to Tier-1 centres
Canada – Triumf (Vancouver)
France – IN2P3 (Lyon)
Germany –Karlsruhe
Italy – CNAF (Bologna)
Netherlands – NIKHEF/SARA (Amsterdam)
Nordic countries – distributed Tier-1
Spain – PIC (Barcelona)
Taiwan – Academia SInica (Taipei)
UK – CLRC (Oxford)
US – FermiLab (Illinois)
– Brookhaven (NY)
Tier-1 – “online” to the data acquisition process  high availability
Managed Mass Storage –  grid-enabled data service
All re-processing passes
Data-heavy analysis
National, regional support
Tier-2 – ~100 centres in ~40 countries
Simulation
End-user analysis – batch and interactive
Services, including Data Archive and Delivery, from Tier-1s
The LHC Computing Grid – July 2006

7. LHC Computing Grid project (LCG)
More than 100 computing centres
12 large centres for primary data management: CERN (Tier-0) and eleven Tier-1s
38 federations of smaller Tier-2 centres
40 countries involved
The LHC Computing Grid – July 2006

8. The LHC Computing Grid – July 2006

9. The new European Network Backbone
LCG working group with Tier-1s and national/ regional research network organisations
New GÉANT 2 – research network backbone  Strong correlation with major European LHC centres
Swiss PoP at CERN
The LHC Computing Grid – July 2006

10. October 2005 (Lambda Triangle)
The LHC Computing Grid – July 2006

11. Summary of Computing Resource Requirements
~ 100K of today’s fastest processors
Summary of Computing Resource Requirements
All experiments
From LCG TDR - June 2005
CERN
All Tier-1s
All Tier-2s
Total
CPU (MSPECint2000s) 25 56 61
142
Disk (PetaBytes) 7 31 19 57
Tape (PetaBytes) 18 35 53
CPU
Disk
Tape
The LHC Computing Grid – July 2006

12. LHC Computing Grid Project - a Collaboration
Building and operating the LHC Grid – a global collaboration between
The physicists and computing specialists from the LHC experiments
The national and regional projects in Europe and the US that have been developing Grid middleware
The regional and national computing centres that provide resources for LHC
The research networks
Researchers
Computer Scientists & Software Engineers
Service Providers
The LHC Computing Grid – July 2006

13. LCG depends on 2 major science grid infrastructures …
The LCG service runs & relies on grid infrastructure provided by:
EGEE - Enabling Grids for E-Science
OSG - US Open Science Grid
The LHC Computing Grid – July 2006

14. EGEE Grid Sites : Q1 2006
sites
EGEE:
Steady growth over the lifetime of the project
CPU
EGEE:
> 180 sites, 40 countries
> 24,000 processors, ~ 5 PB storage
The LHC Computing Grid – July 2006

15. OSG & WLCG OSG Infrastructure is a core piece of the WLCG.
OSG delivers accountable resources and cycles for LHC experiment production and analysis.
OSG Federates with other infrastructures.
Experiments see a seamless global computing facility
The LHC Computing Grid – July 2006

16. Ramp up of OSG use last 6 months
deployment
OSG deployment
The LHC Computing Grid – July 2006

17. Data Transfer by VOs e.g. CMS
The LHC Computing Grid – July 2006

18. LCG Service planning Pilot Services – stable service from 1 June 06
2006
cosmics
LHC Service in operation – 1 Oct 06 over following six months ramp up to full operational capacity & performance
2007
LHC service commissioned – 1 Apr 07
first physics
2008
full physics
run
The LHC Computing Grid – July 2006

19. Service Challenges Jun-Sep 2006 – SC4 – pilot service
Purpose
Understand what it takes to operate a real grid service – run for weeks/months at a time (not just limited to experiment Data Challenges)
Trigger and verify Tier1 & large Tier-2 planning and deployment – - tested with realistic usage patterns
Get the essential grid services ramped up to target levels of reliability, availability, scalability, end-to-end performance
Four progressive steps from October 2004 thru September 2006
End SC1 – data transfer to subset of Tier-1s
Spring 2005 – SC2 – include mass storage, all Tier-1s, some Tier-2s
2nd half 2005 – SC3 – Tier-1s, >20 Tier-2s –first set of baseline services
Jun-Sep 2006 – SC4 – pilot service
 Autumn 2006 – LHC service in continuous operation – ready for data taking in 2007
The LHC Computing Grid – July 2006

20. SC4 – the Pilot LHC Service from June 2006
A stable service on which experiments can make a full demonstration of experiment offline chain
DAQ  Tier-0  Tier-1 data recording, calibration, reconstruction
Offline analysis - Tier-1  Tier-2 data exchange simulation, batch and end-user analysis
And sites can test their operational readiness
Service metrics  MoU service levels
Grid services
Mass storage services, including magnetic tape
Extension to most Tier-2 sites
Evolution of SC3 rather than lots of new functionality
In parallel –
Development and deployment of distributed database services (3D project)
Testing and deployment of new mass storage services (SRM 2.2)
The LHC Computing Grid – July 2006

21. Sustained Data Distribution Rates: CERN  Tier-1s
Centre
ALICE
ATLAS
CMS
LHCb
Rate into T1 MB/sec (pp run)
ASGC, Taipei X
100
CNAF, Italy
200
PIC, Spain
IN2P3, Lyon
GridKA, Germany
RAL, UK
150
BNL, USA
FNAL, USA
TRIUMF, Canada 50
NIKHEF/SARA, NL
Nordic Data Grid Facility
Totals
1,600
Design target is twice these rates to enable catch-up after problems
The LHC Computing Grid – July 2006

22. SC4 T0-T1: Results Easter w/e Target 10 day period
Target: sustained disk – disk transfers at 1.6GB/s out of CERN at full nominal rates for ~10 days
Result: just managed this rate on Easter Sunday (1/10)
Easter w/e
Target 10 day period
The LHC Computing Grid – July 2006

23. ATLAS SC4 tests From last week: initial ATLAS SC4 work ATLAS transfers
Rates to ATLAS T1 sites close to target rates
ATLAS transfers
Background transfers
The LHC Computing Grid – July 2006

24. Impact of the LHC Computing Grid in Europe
LCG has been the driving force for the European multi-science Grid EGEE (Enabling Grids for E-sciencE)
EGEE is now a global effort, and the largest Grid infrastructure worldwide
Co-funded by the European Commission (~130 M€ over 4 years)
EGEE already used for >20 applications, including…
Bio-informatics
Education, Training
Medical Imaging
The LHC Computing Grid – July 2006

25. The EGEE Project Infrastructure operation Middleware
Currently includes >200 sites across 39 countries
Continuous monitoring of grid services & automated site configuration/management
Middleware
Production quality middleware distributed under business friendly open source licence
User Support - Managed process from first contact
through to production usage
Training
Documentation
Expertise in grid-enabling applications
Online helpdesk
Networking events (User Forum, Conferences etc.)
Interoperability
Expanding interoperability with related infrastructures
The LHC Computing Grid – July 2006

26. EGEE-II Expertise & Resources
32 countries
13 federations
 Major and national Grid projects in Europe, USA, Asia
+ 27 countries through related projects:
BalticGrid
EELA
EUChinaGrid
EUIndiaGrid
EUMedGrid
SEE-GRID
The LHC Computing Grid – July 2006

27. Example: EGEE Attacks Avian Flu
EGEE used to analyse 300,000 possible potential drug compounds against bird flu virus, H5N1.
2000 computers at 60 computer centres in Europe, Russia, Taiwan, Israel ran during four weeks in April - the equivalent of 100 years on a single computer.
Potential drug compounds now being identified and ranked
Neuraminidase, one of the two major surface proteins of influenza viruses, facilitating the release of virions from infected cells. Image Courtesy Ying-Ta Wu, AcademiaSinica.
The LHC Computing Grid – July 2006

28. Example: Geocluster industrial application
The first industrial application successfully running on EGEE
Developed by the Compagnie Générale de Géophysique (CGG) in France, doing geophysical simulations for oil, gas, mining and environmental industries.
EGEE technology helps CGG to federate its computing resources around the globe.
The LHC Computing Grid – July 2006

29. Grids in Europe Great investment in developing Grid technology
Sample of National Grid projects:
Austrian Grid Initiative
Belgium: BEGrid
DutchGrid
France: e-Toile; ACI Grid
Germany: D-Grid; Unicore
Greece: HellasGrid
Grid Ireland
Italy: INFNGrid; GRID.IT
NorduGrid
UK e-Science: National Grid Service; OMII; GridPP
EGEE provides framework for national, regional and thematic Grids
Average of 180 M€ per year since 2002 (national + EC)
The LHC Computing Grid – July 2006

30. European e-Infrastructure Coordination
Evolution
European e-Infrastructure Coordination
EGEE
EGEE-II
EDG
EGEE-III
Testbeds
Utility Service
Routine Usage
The LHC Computing Grid – July 2006

31. The LHC Computing Grid – July 2006