1. 1P. Vande Vyvre - CERN/PH ALICE DAQ Technical Design Report DAQ TDR Task Force Tome ANTICICFranco CARENA Wisla CARENA Ozgur COBANOGLU Ervin DENESRoberto DIVIA Jean-Claude MARINKlaus SCHOSSMAIER Csaba SOOSPierre VANDE VYVRE Sandro VASCOTTOLinda VIDAK P. VANDE VYVRE for the ALICE DAQ project

2. 2P. Vande Vyvre - CERN/PH Trigger Level 0,1 Trigger Level 2 High-Level Trigger Transient Data Storage (TDS) Event-Building Network Storage network Detector Digitizers Front-end Pipeline/Buffer Decision Readout Buffer Decision Sub-event Buffer Local Data Concentrators (LDC) Event Buffer Global Data Collectors (GDC) Permanent Data Storage (PDS) Decision Detector Data Link (DDL) Data Logical model DAQ System

3. 3P. Vande Vyvre - CERN/PH 25 GB/s 2.50 GB/s 1.25 GB/s Pb-Pb beam Rate Max. ev. size - Central 20 Hz86.0 MB - MB 20 Hz20.0 MB - Dimuon1600 Hz 0.5 MB - Dielectron 200 Hz 9.0 MB pp beam MB 100 Hz 2.5 MB Running modes A: DAQ B: DAQ+HLT Analysis C: DAQ+HLT Trigger Physics requirements

4. 4P. Vande Vyvre - CERN/PH GDC DAQ architecture CTP LTU TTC FERO LTU TTC FERO LDC BUSY Rare/All Event Fragment Sub-event Event File Storage Network TDS PDS L0, L1a, L2 262 DDLs EDM LDC Load Bal. LDC HLT Farm FEP DDL H-RORC 10 DDLs 10 D-RORC 10 HLT LDC 123 DDLs TDS DSS Event Building Network 329 D-RORC 175 Detector LDC 50 GDC 25 TDS 5 DSS

5. 5P. Vande Vyvre - CERN/PH GDC DAQ Software CTP LTU TTC FERO LTU TTC FERO LDC BSY Rare/All Event Fragment Sub-event Event File Storage Network TDS PDS L0, L1a, L2 262 DDLs D-RORC Detector LDC EDM LDC Load Bal. LDC HLT Farm FEP DDL H-RORC Event Building Network DDL D-RORC HLT LDC 123 DDLs TDS DDL and D-RORC software Dataflow (DATE) ROOT I/O CASTOR Control (DATE) Monitoring (AFFAIR, MOOD)

6. 6P. Vande Vyvre - CERN/PH LDC DDL architecture PCI Bus Front-End Read-Out DAQ Read-out Receiver Card (D-RORC) Source Interface Unit Forward Channel (Raw data) Backward Channel (Pedestals, control) Destination Interface Unit Detector Data Link (DDL) : - Source Interface Unit - Transmission media - Destination Interface Unit Standard detector/DAQ interface

7. 7P. Vande Vyvre - CERN/PH DDL Uniform hardware for source and destination in ½ CMC form factor Based on 2.1 Gbit/s components 200 MB/s bandwidth Detectors readout integrated with the DDL: - HMPID - SDD - TOF

8. 8P. Vande Vyvre - CERN/PH D-RORC D-RORC: Same hardware base - 1 DDL link card plugged in - 2 integrated DDL channels (2 inputs, or 1 input/1 output) TRG busy generation 200 MB/s throughput with DATE

9. 9P. Vande Vyvre - CERN/PH DAQ software framework DATE Development process –Target the final system from the start –Release and document DATE (Data Acquisition and Test Environment) –Complete ALICE DAQ software framework: Data-flow: detector readout, event building System configuration and control Performance monitoring, Data quality monitoring –Evolving with requirements and technology DATE current status –Used in the ALICE test beams and several institutes –Performance tests during ALICE Computing Data Challenges (ADC) –NA57, Compass, Harp, NA60

10. 10P. Vande Vyvre - CERN/PH LDCs and readout software DATE software Autonomous DDL readout Sub-event building Processing of HLT decision Send data to event-building LDC PCs running Linux Dual-CPU DDR memory Network Interface Card (NIC) on the motherboard Up to 5 PCI slots LDC DDL DIUDDL SIU DATE banks readout recorder D-RORC Raw data Events fragments Event Building Network HLT Detector NIC decision agent HLT LDC Sub-events

11. 11P. Vande Vyvre - CERN/PH Event building, network, GDCs DATE software –Event building –Data formatting Event Building Network –Switched Ethernet GDC – PCs running Linux – Dual-CPU – DDR memory – NIC on the motherboard GDC NIC DATE data banks event builder ROOT recorder Raw data HLT data Sub-events (raw data, HLT data), HLT decisions Storage Network Event Building Network Complete accepted events

12. 12P. Vande Vyvre - CERN/PH PCs Performances Network PC memory

13. 13P. Vande Vyvre - CERN/PH Event building scalability

14. 14P. Vande Vyvre - CERN/PH Storage GDC Storage Network PDS TDS Storage Network: Fibre Channel TDS: storage array GDC PDS GDC

15. 15P. Vande Vyvre - CERN/PH Control software DATE software –DAQ run control HI –Generation of state machines –Interface to ECS SMI++ and DIM DAQ Services Servers (DSS) –Servers or PC running Linux LDC D-RORC DDL DIUDDL SIU GDC Event Building Network Detector Electronics DDL SIU Storage Network Run Control State Machines ECS EDM

16. 16P. Vande Vyvre - CERN/PH Monitoring software: AFFAIR LDC D-RORC DDL DIUDDL SIU GDC Event Building Network Detector Electronics DDL SIU Storage Network ROOT I/O performances Round Robin DB DATE performances Fabric monitoring ROOT DB ROOT Plots ROOT Plots for Web CASTOR performances TDS Round robin DB Apache with php DIM DSS server Web browser

17. 17P. Vande Vyvre - CERN/PH Data quality monitoring: MOOD DATE + ROOT environments MOOD framework –Interfaces to detector code –Example for TPC Applications: –Raw data integrity –Detector performance DSS server

18. 18P. Vande Vyvre - CERN/PH Data Challenges: event building Event building No recording 5 days non-stop 5 days non-stop 1750 MBytes/s sustained 1750 MBytes/s sustained Event building and data recording Recording to CASTOR 7 days non-stop: 7 days non-stop: 1 ~ 140 TBytes 1 280 MBytes/s sustained

19. 19P. Vande Vyvre - CERN/PH Simulation Conditions: Realistic event sizes, distributions, detectors readout 8000 Hz tot: 800 Hz Electron (EL), 700 Muon (MU) Electron and Muon triggers considered rare Original count After Past/Future protection Final count Decrease of EL and MU triggers due to detector busy

20. 20P. Vande Vyvre - CERN/PH DAQ feedback to CTP High/low level at LDC to inform the CTP to block high bandwidth “non important” triggers (CE, MB) to prevent multi-event buffer getting full Result: almost no losses after Past/Future protection Original count After Past/Future protection Final count

21. 21P. Vande Vyvre - CERN/PH DAQ for Test & Commissioning SXL HallSX Hall UX25 (Experimental Area) PX24/CR1 (DAQ) PX24/CR2 (HLT) SR Hall Access Shaft DDL LAN DDL Patch Panel (ALICE sub-detector assembly) (Networking) PX24/CR3 (DCS) PX24/CR4 (Misc.) LAN WR 1 WR 2 ACR 1 Q 2005 : DAQ System at Point 2 12 DDLs for TPC 8 DDLs for others 3 partitions 2005: similar system in Si. lab.

22. 22P. Vande Vyvre - CERN/PH Final DAQ System SXL HallSX Hall UX25 (Experimental Area) PX24/CR1 (DAQ) PX24/CR2 (HLT) SR Hall Access Shaft DDL LAN DDL Patch Panel (ALICE sub-detector assembly) (Networking) PX24/CR3 (DCS) PX24/CR4 (Misc.) WR 1 WR 2 ACR Installation staging: (% of final DAQ performance) 2006: 20% 2007: 30% 2008: 100%

23. 23P. Vande Vyvre - CERN/PH Organization and resources Institutes: –Birmingham, Budapest, CERN, Split, Zagreb DAQ CORE: –Data transfer1’426 kCHF –Central DAQ2’734 kCHF –Monitoring 100 kCHF –Infrastructure1’735 kCHF –Total5’995 kCHF

24. 24P. Vande Vyvre - CERN/PH Key milestones DDL and D-RORC –1 Q 2004: pre-production for detector test and commissioning –2005: production DAQ reference system in DAQ lab: 2004 DAQ systems for surface tests and commissioning: –SXL2: mounting hall on the surface of point 2 (1 Q 2005) –Si. lab: ITS surface test (2005) Milestones Final DAQ –Jan 2006Final DAQ system ready with all functionalities 20 % of final performance –Nov 2006 30 % for pp and first HI run –Oct 2008100 % for second HI run (needs and budget)