András László KFKI Research Institute for Particle and Nuclear Physics New Read-out System of the NA61 Experiment at CERN SPS Zimányi Winter School ‑ 25 November 2008

25 November 2008 András László2 The NA61 experiment NA61 is a fixed target experiment at the CERN SPS. It is based on a previous experiment, called NA49. Many detector upgrades have been implemented.  Main physics goals:  p+C → π,K + X meson production xsections for T2K.  p+p,C → π,K + X meson production xsections for CR.  p+p,Pb → π,K,p + X hadron production for high p_T.  Energy – A scan for search for QCD critical point.

25 November 2008 András László3 Hardware setup

25 November 2008 András László4 Main hardware properties –Four large volume TPC chambers (40m^3, very good dE/dx). –TOFs, counters etc. –Major data contribution from TPCs: – approx 100MB/event for raw event – about 4MB/event for zero compressed event – 80Hz event rate is to be achieved with the new system

25 November 2008 András László5 Serial connection Concentrator Boxes Boxes MBMB 9-bit readout channel MotherBoards 24 1 Front-Endcards Fiber connection DDL receivers RORCRORC PCI-X PC Readout scheme CB TOF CAMAC FICFIC FICFIC CAENCAEN CAENCAEN VME 64X crates crates CAEN VME – USB link bridges Push data mode with flow control: MB  CB  PC w/o processor intervention No data loss even in case of long XOFF Poll data mode with flow control: VME  PC with processor intervention

25 November 2008 András László6 LVDS Serial connection Concentrator Box 9-bit readout channel Mother Boards 24 1 Front-Endcards DDL fiber connection to DAQ FE-MB-CB communication 1 MB performs pedestal subtraction, zero suppression, and zero/Huffman compr. CB polls for data from MBs on the LVDS lines. Serializes data to DDL line. On data arrival, the BUSY signal toward Trigger Unit is set. BUSY signal to Trigger Unit

25 November 2008 András László7 DDL architecture DAQ PC PCI Bus Front-End Read-Out DAQ Read-out Receiver Card (D-RORC) Source Interface Unit Forward Channel (Raw data) Backward Channel (Pedestals, control) Destination Interface Unit Detector/DAQ interface 100 MHz 64-bit PCI card Detector Data Link (DDL): Source Interface Unit Transmission Medium Destination Interface Unit

25 November 2008 András László8RORC DDL memory management principle Firmware data length transfer status Ready FIFO Transfer status possible values: ffffffff ffffffff empty (set by sw) xxxx0000 xxxx0000 loaded, data continues (set by RORC) xxxxelse xxxxelse loaded, data ended (set by RORC) Free FIFO page size Index of Ready FIFO start address Page aligned continuous user memory outside Linux’s memory space, reachable for DMA and user as well. (PHYSMEM module) Free pages in PC PHYSMEM for event fragments Free FIFO / Ready FIFO depth: 128 Maximum page size: 2Mbyte First, data are pushed into the SIU by hardware.

25 November 2008 András László9 Data collection scheme with VME PC USB port CAEN: V1718 VME-USB Bridge VME Bus USB Cable VSB Bus Sub Unit MM6390 VME/VSB Dual Port Mem.

25 November 2008 András László10 Memory management principle with VME First, event data are pushed into pages through VSB port into a VME/VSB dual port memory module. Cyclic buffer, data flow is independent of Central DAQ. Free pages in VME dual port memory for events Maximum page size: 24kbyte The event pages are transferred into PC PHYSMEM memory via a VME-USB bridge. (PC polls for new data.) Cyclic buffer, trying to catch up with VME cyclic buffer. VME-USB bridge Free pages in PC PHYSMEM memory for events If PC buffer cannot catch up with VME buffer, BUSY signal is set via a register module. (Flow control.)

25 November 2008 András László11 Main read-out components Mother Boards, 8 Concentrator Boxes on 8 DDL channels, PC with Ghz - 6 PCI-X slots - 8 GB CAMAC and TOF communication via VME memory module + CORBO register module + VME-USB bridge

25 November 2008 András László12 DAQ software Actual version supposes all input going into 1 PC 2 or more PCs will be possible later on, if necessary Hardware and communication setup is configurable in files HWSetup.dat (general), CBSetup.dat (DDL lines), VMESetup.dat (VME lines), MBSetup1-8.dat (MotherBoards) Basic components are graphically switchable on/off Disk, CAMAC, TOF, TPC (can switch components, enabled in config file) Illustration: # CB RORC RORC RORC MB Setup File # no rev S/N C/N setup/MBSetup1.dat # setup/MBSetup2.dat CBSetup.dat: # MB Incl?Th MBSetup1.dat:

25 November 2008 András László13 DAQ software working principle A GUI is hiding the lowlevel system (Tcl/Tk). The run control program is a command line interpreter (C). Upon program start, an Event Server is launched to serve the online monitoring (forks upon each client connection). Upon run start, 5 processes are launched (communication is performed via shared memory): 1. Logger 2. Slow Control client 3. Recorder (polls for data from receivers, writes to local disk) 4. DDL Receiver (polls for data from DDLs, sends to recorder) 5. VME Receiver (polls for data from VMEs, sends to recorder) Upon run end/crash: notification sent to E-log logging syst.

25 November 2008 András László14 Run Control software GUI

25 November 2008 András László15 Event Client (data monitoring)

25 November 2008 András László16 Performance Average event size: approximately 3MBytes with p+C. Average event size: approximately 3MBytes with p+C. Event building without disk: about 75Hz continuosly. Event building without disk: about 75Hz continuosly. Event building with disk: about 55Hz continuosly, i.e. Event building with disk: about 55Hz continuosly, i.e. without relaxation time (possible only with the RAID0 trick!) On time-average of the 10sec/40sec SPS spill structure: On time-average of the 10sec/40sec SPS spill structure: 75Hz event recording rate is achieved within spill.

25 November 2008 András László17 Thank you!