Sept TPC readoutupgade meeting, Budapest1 DAQ for new TPC readout Ervin Dénes, Zoltán Fodor KFKI, Research Institute for Particle and Nuclear Physics

Sept TPC readoutupgade meeting, Budapest2 DDL architecture PCI Bus Concentrator DAQ Read-out Receiver Card (D-RORC) Source Interface Unit Forward Channel (Raw data) Backward Channel (Pedestals, control) Destination Interface Unit Detector Data Link (DDL) : - Source Interface Unit - Transmission media - Destination Interface Unit Standard detector/DAQ interface

Sept TPC readoutupgade meeting, Budapest3 ALICE Detector Data Link

Sept TPC readoutupgade meeting, Budapest4 Read-Out Receiver Cards (D-RORC) PCI-X adapter for 2 DDL max 100 MHz PCI master: autonomous DMA

Sept TPC readoutupgade meeting, Budapest5 DDL Features Interface: Full duplex 32-bit data path on the destination interface (DIU card) Half duplex 32-bit data path on the source interface (SIU card) Full duplex flow control (XON/XOFF) Interface clock up to 66 MHz (easy integration with PCI 66) 264 MB/s peak data rate, 240 MB/s sustained bandwidth (max.) Implementation: Duplex LC optical link up to 300 m 2x FC or 2x GbE physical layer components Small Form Factor Pluggable (SFP) optical transceivers Bit error rate < Robust error detection: very low undetected bit error rate < Automatic link synchronization and management Radiation tolerant Source Interface Extras: Stand-by support (low power consumption) In-system reconfiguration / Remote system upgrade Monitoring of the aging of laser diode of optical transceivers

Sept TPC readoutupgade meeting, Budapest6 Readout System Performance Motherboard with dual Xeon 2.4 GHz Six PCI-X slots, 4 bus segments ( ), 2 controllers Linux OS ALICE Data-Acquisition software (DATE)

Sept TPC readoutupgade meeting, Budapest7 Performance: 6 D-RORCs Testing the fully populated PC using data source internal to PCI interface –Interoperability test –Measure the maximal input bandwidth PCI #6 PCI #5 PCI #4 PCI #3 PCI #2 PCI #1 Segment #1 #2 #3 #4 Controller #1 #2 1 Ch Aggregate Bandwidth [MB/s] Normalized Bandwidth [MB/s/Ch]

Sept TPC readoutupgade meeting, Budapest8 DDL Software All functions accessible as interactive commands or API Script-based interpreter for sequence of operations: –Sending command to the FEE –Reading FEE status printing the status comparing the status polling the status –Downloading data into the FEE from a file –Reading data from the FEE writing data into a file comparing data with data in a file TPC configuration: < 0.3 s FERO DDL definepedestal_addr0x1FFF defineenable_pedestal0x2C resetSIU write_commandenable_pedestal write_blockpedestal_addr pedestal.hex%x read_and_check_blockpedestal_addrpedestal.hex%x definepedestal_addr0x1FFF defineenable_pedestal0x2C resetSIU write_commandenable_pedestal write_blockpedestal_addr pedestal.hex%x read_and_check_blockpedestal_addrpedestal.hex%x D-RORC LDC

Sept TPC readoutupgade meeting, Budapest9 DDL Transactions write_command FEE command No reply from the FEE RORCConc.FEE BUS FEE command acq Writing a command to the FEE (e.g. CLEAR)

Sept TPC readoutupgade meeting, Budapest10 DDL Transactions read_and_print “ ” [ ] FEE address RORCConc.FEE BUS FEE status read_and_check read_until At least 6 FEE clocks FEE address Reading a status from the FEE acq

Sept TPC readoutupgade meeting, Budapest11 DDL Transactions write_block [ ] FEE address FEERORCConc. BUS Data FEE address block length acq Writing data to the FEE (e.g. pedestal) end transaction acq

Sept TPC readoutupgade meeting, Budapest12 DDL Transactions FEE address BUS block length Data end transaction end block FEE address acq Reading data from the FEE (e.g. read back pedestal) RORCConc.FEE BUS SW read_and_check_block [ ]

Sept TPC readoutupgade meeting, Budapest13 DDL Transactions start run BUS event length Data end transaction end of event Rdy to receive acq Data collection RORCConc.FEE BUS SW Data event length end of event end run

Sept TPC readoutupgade meeting, Budapest14 The Free FIFO D-RORC PC memory bank Firmware Event building page address Free FIFO PC CPU Allocation of free pages

Sept TPC readoutupgade meeting, Budapest15 Direct Memory Access D-RORC Firmware PC memory bank DDL No involvement PC CPU

Sept TPC readoutupgade meeting, Budapest16 The Ready FIFO D-RORC PC memory bank Event building DDL Ready FIFO Firmware lengthpage status lengthpage status lengthpage status Delivery of filled pages PC CPU

Sept TPC readoutupgade meeting, Budapest17 NA61 DAQ Configuration VTPC1VTPC2 MTPCR MTPCL Motherboards and concentrator CAMAC TOFs Trigger system PC RORC DDL links Gate & Clock to Motherboards Busy VME to PCI ?

Sept TPC readoutupgade meeting, Budapest18 Timing Diagram Gate for event Transfer to PC Readout of FEE Busy Evt 1. Evt 2. Evt 3. Read FEE to mem 1Read FEE to mem 2 Transfer evt 1. to PC Transfer evt 2.. Busy generally = sampling time + FEE readout time, if the data flow fluid Busy = 1, during the readout of the FEE or no free memory to read it

Sept TPC readoutupgade meeting, Budapest19 Bandwidth Estimations DDL/PCI bus: 100 MHz, 64 bits 800 MB/s Memory bus: 2.4 GHz, 64 bits 19.2 GB/s Requirement: 100 Hz, size: 6 MB 600 MB/s SATA speed: 6 Gb/s, 750 MB/s ?