Competences & Plans at GSI P.Malzacher@gsi.de K.Schwarz@gsi.de LSDMA Kickoff Workshop 22 March 2012, KIT

open source and community software budget commodity hardware GSI: a German National Lab for Heavy Ion Research FAIR: Facility for Ion and Antiproton Research ~2018 GSI computing today ALICE T2/T3 HADES ~ 14000 cores, ~ 5.5 PB lustre ~ 9 PB archive capacity PANDA CBM/HADES NuSTAR APPA FAIR computing 2018 CBM PANDA NuSTAR APPA LQCD 300000 cores 40 PB disk 40 PB archive open source and community software budget commodity hardware support different communities scarce manpower & FAIR MAN & FAIR Grid/Cloud

1Tb/s 1Tb/s TB/s 1Tb/s 1Tb/s

Distribution of Competences energy efficient high throughput cluster cluster file system lustre tape archive, gStore MAN Grid, Cloud Exp. Frameworks Parallel Programming Two HEP like experiments: CBM & PANDA, two research fields with a lot of smaller experiments ~3000 scientists from ~50 countries

„General Purpose“ HPC System, optimized for high I/O bandwidth Compute farms: 4000 cores, ethernet based 10000 cores IB based Debian, SGE shared usage, based on priorities by GSI, FAIR, ALICE Tier2/3 Lustre with 3 MDS HA cluster, 48 core, 300 GB RAM 180 Fileservers with 3.5 TB RAM as OSS 7.000 disks > 5.5 PB raw capacity I/O capacity about 1 Tb/s more to come IB based l

We provide a mixture of: a Tier 2 integrated in the AliEn Grid, The Alice Tier-2 at GSI We provide a mixture of: a Tier 2 integrated in the AliEn Grid, mainly simulation a Tier 3, analysis trains fast interactive responds via PROOF based on PoD calibration & alignment of TPC integrated in the standard GSI batch farm (GSI, FAIR). We are able to readjust the relative size of the different parts on request. Main contributions from Germany: Uni Heidelberg Uni Frankfurt Uni Münster Uni Darmstadt GSI TPC TRD HLT GridKa Tier-1 GSI Tier-2/3 LoeweCSC

New (dark) Datacenter: ready 2014 Minimal floor footprint Space for 800 19“ racks Planned cooling power 6 MW (building supports more) Internal temperature 30° Minimal construction cost Fast to build Autonomous rack power mgnt. Back door rack cooling Smoke detection in every rack Use of heat for building heating Shortest cable lenghts Water pressure below 1bar avoiding risk of spills Use of any commercial 19“ architecture Unmatched packing- and power density Construction cost about 11 M€

Proof of Concept: Testcontainer with PUE < 1.1 in operation since 1.5 years Mini Cube ~ 100 racks, 1.2 MW cooling power: 1Q2012

First Step to the FAIR HPC Link 12 * 10 Gb/s GSI – LOEWE CSC 2 * 10 Gb/s GSI – Uni Ffm 4 * 10 Gb/s GSI – Uni Mainz

gStore: a lightweight tape access system: 2 3584-L23 ATL, 8 TS1140 tape drives, 17 data mover resource max storage capacities: Experimentdata: Copy of rawdata overall data mover disk cache   8.8 PB 1.3 PB 0.26 PB overall gStore I/O bandwith: DM disk <-> tape (SAN) DM disk <-> clients (LAN) DAQ online clients ->DM incl. copy to lustre 2.0 GB/s 5.0 GB/s 1.0 GB/s 0.5 GB/s

Exploration of Virtualization & Cloud Computing GSI as Cloud provider: SClab: Cloud Prototype at GSI ~ 200 cores demo production for ALICE GSI as Cloud user: FLUKA jobs for Radiation Safety Studies on Amazon and Ffm cloud IaaS- cloud VM CMS server DRMS master client worker application

The FairRoot framework is used by CBM, PANDA and parts of NUSTAR Virtual MC Geant3 Geant4 FLUKA G4VMC FlukaVMC G3VMC Geometry Close contact common developments Run Manager Event Generator Magnetic Field Detector base IO Manager Tasks RTDataBase SQL Conf, Par, Geo Root files Hits, Digits, Tracks Application Cuts, processes Display Track propagation Always in close contact STS TRD TOF RICH ECAL MVD ZDC MUCH ASCII Urqmd Pluto Track finding digitizers Hit Producers Dipole Map Active Map const. field CBM Code STT MUO TOF DCH EMC MVD TPC DIRC ASCII EVT DPM Track finding digitizers Hit Producers Dipole Map Solenoid Map const. field Panda Code

Challenges for reconstruction UrQMD, central Au+Au @ 25 AGeV particles per collision of two gold nuclei results up to now D0, cτ=127 μm K- π+ Open charm: Typical signal multiplicities: O(10-6) No „easy“ trigger signatures Fast online event reconstruction, no hardware trigger: time to reconstruct 1-10ms & 107 collisions/s -> 104 – 105 cores

Exploring Different Architectures and Parallelization Strategies: eg Exploring Different Architectures and Parallelization Strategies: eg. PANDA Tracking on CPU and GPU, PROOF, PoD CPU 1 Event 1 Track Candidates GPU Track Fitting Tracks CPU 2 Event 2 CPU 3 Event 3 CPU 4 Event 4 No. of Process 50 Track/Event 2000 Track/Event 1 CPU 1.7 E4 Track/s 9.1 E2 Track/s 1 CPU + GPU (Tesla) 5.0 E4 Track/s 6.3 E5 Track/s 4 CPU + GPU (Tesla) 1.2 E5 Track/s 2.2 E6 Track/s

Remote access to data and computing via the Grid: example Alice and PANDA Grid with AliEn Panda Grid Sites

www.eofs.org

http://www.crisp-fp7.eu/ WP16: Common User Identity System WP19: Distributed Data Infrastructure

Long term goal: Distributed data management for FAIR Short/Medium term projects: Storage Federation Remote lustre SE on top of lustre global xroot redirector for ALICE and FAIR Cloud Storage interface to existing environments (ROOT, AliEn,...) Optimized job/data placement hot vs unused data check for corrupted data sets reading from several SEs 3rd party copy

Summary: FAIR Computing FAIR will serve about 20 scientific collaborations from four major fields of research and application. Basis of the software systems of the larger FAIR experiments is FairRoot - a simulation, reconstruction and analysis framework. It is designed to optimize the accessibility for new as well as experienced users, to be able to cope with future developments, and to enhance the synergy between the different experiments. Parallelization of the software on all levels is essential to harvest the performance gains of future architectures.

Data recording for CBM and PANDA require a detector read-out without hardware triggers, relying exclusively on online event filtering. The first layer of the system consists of a combination of specialized processing elements such as GPUs, CELLs or FPGAs in combination with COTS computers, connected by an efficient high-speed network. Major building blocks of the infrastructure are a new datacenter for online computing, storage systems and offline cluster, linked to a metropolitan area network connecting the compute facilities of the surrounding universities and research centers, embedded in an international Grid/Cloud infrastructure.