1. A high-performance computing facility for scientific research
26/08/2018
July 5th, 2007
CC-IN2P3
A high-performance computing facility for scientific research
Visit of a Japanese delegation at CC-IN2P3

2. + some opening to other sciences
26/08/2018
CC-IN2P3 presentation
Main scientific organization in France
CC-IN2P3 federates the main computing resources
For :
High energy physics
Nuclear physics
Astroparticle physics
National Institute for Nuclear Physics and Particle Physics
Dedicated Computing Center
+ some opening to other sciences
dapnia
Atomic Energy Commission
2007 Budget: 10 M€ M€ salary
D. Boutigny

3. Mission Physics Experiments Masses of data Basic research
26/08/2018
Mission
Physics Experiments
Nuclear Physics
Particle Physics
Astroparticules
Masses of data
Basic research
Data processing
Publications
Physics Analysis
D. Boutigny

4. Why such huge needs of computing for high energy physics ?
26/08/2018
Why such huge needs of computing for high energy physics ?
Scientific experiments are more and more sophisticated
Big number of electronic channels
huge amount of data
International collaborations
Data is globally distributed
Data analysis is a geographically distributed process
Development of high-performance wide-area network links
Simulation is an important part of the scientific process
Modeling of the detectors behavior is complex
 Computing and mass storage needs
D. Boutigny

5. Computing power evolution
26/08/2018
Computing power evolution
The compute farm purchased in 2006 entered the « Top500 Supercomputer Sites » list
3700 jobs in simultaneous execution
jobs in queue
throughput: jobs/day
D. Boutigny

6. Evolution of the storage capacity
26/08/2018
Evolution of the storage capacity
5.4 PB
Mass storage (tape-based)
2.4 PB
Ramp-up for the LHC era TB
Disk-based storage
D. Boutigny

7. International Dimension
26/08/2018
International Dimension
Contributing to scientific research in an international context
Network of big centers for high energy physics data processing around the world
Active collaboration on several domains
Data acquisition using instruments built by international collaborations
D. Boutigny

8. The Large Hadron Collider (LHC): a new challenge
26/08/2018
The Large Hadron Collider (LHC): a new challenge
ATLAS
4 experiments with a level of complexity never reached before
CMS
10 billions collisions recorded each year
15 PB/year of data
D. Boutigny

9. 15 PetaBytes ? 6 cm 50 CD-ROM = 35 GB 1 hard disk: 500 GB
26/08/2018
Concorde
(15 Km)
Balloon
(30 Km)
CD stack with
1 year LHC data!
(~ 20 Km)
Mt. Blanc
(4.8 Km)
15 PetaBytes ?
50 CD-ROM
= 35 GB
6 cm
1 hard disk: 500 GB
1 year of LHC data  hard disks
Central processing of all those data is not viable
 It is necessary to integrate the data processing capabilities of several computing centers around the world
D. Boutigny

10. A global grid infrastructure
26/08/2018
A global grid infrastructure
Annecy
Marseille
TRIUMF
FZK
CC-IN2P3
Nantes
T3 (many)
T1 (11)
RAL
ASCC
CNAF
Île de France T0
Brookhaven
PIC
T2 (~100)
NIKHEF
Lyon
Fermilab
NDGF
Strasbourg
Clermont
D. Boutigny

11. 26/08/2018
EGEE Project
Enabling Grids for E-sciencE is a European project aiming at deploying a grid infrastructure for several scientific domains, coordinated by CERN
Phase I ended – Phase II started since April 2006
90 partners, 32 countries
EU funding: 32 M€, 2 years
Pilot
New
The European part of the LHC grid is operated by EGEE
CC-IN2P3 plays an important role in the EGEE grid-wide operation
11-12 people funded by EGEE
Crucial for the success of the LHC grid in France
D. Boutigny

12. 26/08/2018
D. Boutigny

13. Planned evolution of the site’s computing capacity
26/08/2018
Planned evolution of the site’s computing capacity
CPU
Disk
The planned CPU and disk storage capacities grow faster than Moore’s law
 Huge impact on the cooling and power infrastructure
1.5 M€ have been invested in the computer room upgrade  1 MW of electrical power for computing equipment
A new building is foreseen for 2010  MW
D. Boutigny

14. Grid usage 29% HEP Grid / Total Slope: +1% / month
26/08/2018
Grid usage
29%
HEP Grid / Total
Slope: +1% / month
Main grid consumers:
LHC
Virgo
ILC
Non HEP Grid / Total ~ 0.4 %
D. Boutigny

15. LHC experiments CPU consumption
26/08/2018
LHC experiments CPU consumption
1 UI = 50 SI2k.h
D. Boutigny

16. Core competencies of CC-IN2P3 (1)
26/08/2018
Core competencies of CC-IN2P3 (1)
Operation of large computing farms
Mass storage
Usage of international connectivity through high-performance links
LHC: dedicated optical circuit CC-IN2P - CERN at 10 Gbps
Data distribution
Computing and data-intensive grid
D. Boutigny

17. Core competencies of CC-IN2P3 (2)
26/08/2018
Core competencies of CC-IN2P3 (2)
High-performance storage
Storage and retrieval of masses of data
D. Boutigny

18. Collaboration projects
26/08/2018
Collaboration projects
ANR project IGTMD
Grid Interoperability and Massive Data Transfer
RENATER – LIP/RESO – CC-IN2P3 + FNAL
Dedicated 2x1 Gb/s optical circuit between CC-IN2P3 and FNAL
France-Japan Particle Physics Laboratory
NAREGI / EGEE interoperability
Software development on grid interoperability
New approach based on OGF standards: SAGA (Simple API for Grid Applications) / JSDL (Job Submission Description Language)
SRB / iRODS developments and applications
1 visitor from KEK will be working at CC-IN2P3 for 8 months
Explore possibility to use iRODS in LHC context
France-China Particle Physics Laboratory
Future France-Korea International Laboratory
D. Boutigny

19. Outreach High energy physics At CC-IN2P3
26/08/2018
Outreach
High energy physics
Traditionally high consumer of computing power
At CC-IN2P3
Development of a high quality infrastructure
Development of world-class core competencies
Usage of a fraction of the CC-IN2P3 resources for other scientific or industrial domains
 Huge impact
D. Boutigny

20. Bio-medical applications at CC-IN2P3
26/08/2018
Bio-medical applications at CC-IN2P3
Several groups use the CC-IN2P3’s resources
Genomics
Functional Medical Imaging
Research on rheumatoid arthritis …
Our contribution is focused on data management and usage of the compute farm
Helps on structuring and federating the user community
Opening possibilities for the research groups
One year worth of computing on a single (researcher’s) desktop performed at CC-IN2P3 in some days or a few weeks
D. Boutigny

21. Collaboration, Communication, Dissemination
26/08/2018
Collaboration, Communication, Dissemination
Hosting of more than 200 web sites
 Palais de la Découverte (Paris science museum) for instance
Operation of the visio-conference system shared by several nationwide scientific organizations (4500h reservation in 2007)
CERN – INSERM – CNRS - INRA
Webcast: video recording and delivery of scientific events via the web
French Academy of Sciences
CNRS national and international conferences ....
~ hits for Video On Demand in 2007
D. Boutigny