1. CERN Data Centre ‘Building 513 on the Meyrin Site’
IT Department Computing Facilities Group

2. Overview of CERN Central Computing Facilities
Overview of CERN Data Centre
Computer Rooms
Powering
Cooling
Networking
CERN Data Centre in Numbers
CERN Data Centre - 2

3. Overview of CERN Central Computing Facilities
CERN Main Data Centre – Building 513 on the Meyrin Site
All services required for the running of the laboratory
Physics data processing and notably the WLCG Tier0
Data storage
CERN Data Backup – Building 613 on the Meyrin Site
Tape Robots
Remote hosting site (Wigner Data Centre in Budapest) with 2.7MW of available power for computing
Extension of WLCG Tier0 capacity
Extension of BC capacity
CERN Data Centre - 3

4. CERN’s Main Data Centre
Data centre built in early 1970s
Has been refurbished/upgraded several times:
To cope with changing computing technologies
To increase the capacity as far as possible
To improve energy efficiency
To insolate critical services from other services and provide better security for them
24/7 operator
CERN Data Centre - 4

5. Four main computer rooms:
1440m2 main room with air cooling
1240m2 basement room mixture of air/water cooling
225m2 room for critical servers only with water cooling
207m2 room for data B/U in a separate building
A number of additional rooms for telecoms equipment (external providers)
CERN Data Centre - 5

6. Powering
The CERN site is fed from both the French and Swiss electrical networks with automatic switchover.
The CERN Data Centre is connected to an on-site HV electrical distribution ring as well as a connection to a separate on-site HV distribution line with automatic switchover.
The CERN Data Centre is also connected to the CERN Safety Network (diesels) with automatic switchover.
All computing infrastructure, as well as important cooling equipment, is on UPS.
CERN Data Centre - 6

7. Powering Six 3MVA and two 2MVA transformers (18kV/400V).
Two independent power distribution systems within the Data Centre; Critical and Physics.
Physics - four UPS Systems, one with 5*400kVA and three with 4*400kVA modules (N+1).
Critical - one UPS System, with 4*400kVA modules (N+1).
Critical systems are fed by both Physics and Critical.
CERN Data Centre - 7

8. Powering Overview IT Physics Equipment IT Critical Equipment
Physics UPS
(3/4+1)
Diesel Generator (N+1)
Critical UPS
(3+1)
600kW
3.2MW
3.8MW
Normal Network
Cooling for
18kV
CERN Data Centre - 8

9. There are multiple cooling technologies and systems employed.
Cooling Overview
There are multiple cooling technologies and systems employed.
Large AHUs allowing extensive use of free cooling.
Traditional CRAC units.
Active water-cooled racks.
Passive water-cooled racks.
CERN Data Centre - 9

10. Main Room Primary Cooling
There are four 1500 kW chillers on the roof (N+1) with two 30 m3 storage tanks.
10 AHU units (N+1) (each with a capacity of ~105 m3/h) just below the roof which can use outside air, recycled air or a mixture.
Cooled air pumped down large ducts at the side of the room and then under the false floor into the cold corridors.
Cooled air controlled within the temperature range 14-24ºC.
Allows to cool ~2.5 kW per rack on average – higher loads possible for individual racks.
CERN Data Centre - 10

11. Critical Cooling
There are three 400kW chillers outside the building with a 40 m3 storage tank.
Main Room
Ventilation units inside the building produce the cooled air which is passed into the three zones via the red ducts.
The cooled air is supplied above the racks.
Barn
Water pumped under the false floor and fed to the 90 water-cooled racks.
They have passive rear-door heat exchangers, i.e. the doors do not have fans.
Allows to cool at least 10kW per rack.
CERN Data Centre - 11

12. Vault Cooling Air Cooling Water Cooling
Water produced by the roof chillers fed to 6 CRAC units.
CRAC units blow the cooled air under the false floor.
Water Cooling
There are two 800kW chillers (N+1) at the side of the building with 30 m3 storage tank.
Water pumped under the false floor and fed to the 80 water-cooled racks.
The have active rear-door heat exchangers, i.e. the doors have fans.
Allows to cool 10kW per rack.
CERN Data Centre - 12

13. Data Centre in a few Numbers
Total computer room floor space: ~3100 m2
Number of racks: 1127
Number of servers: ~12000
Raw disk space: ~120 PB
Tape storage: ~140 PB
Number of routers: 23
Number of switches: ~500
Number of 1Gbps/10Gbps/100Gbps ports: 21,657/6008/48
Available/Used power: 3.8MW/2.3MW
DC Overview
CERN Data Centre - 18