WP5: Common DAQ David Cussans, on behalf of WP5 team: University of Bristol, DESY Hamburg, Institute of Physics AS CR Prague, University of Sussex, University College London

Outline Task 5.2 Interface, synchronisation and control of multiple-detector systems Task 5.3 Central DAQ software and run control (together with WP15 beam test infrastructure) Task 5.4 Development of data quality and slow control monitoring Task 5.5 Event model for combined DAQ Interface Specification D5.1

Tasks (Task 5.1 is management)

Task 5.2: Hardware Timing and Synchronization New Physicist/Engineer started at Bristol 1/6 Paolo Baesso – DAQ expert. 50% on AIDA-2020 for lifetime of project. Schematic for next version of mini-TLU ready for review Porting firmware from Xilinx Spartan-6 to Artix-7 underway Samer al-Kilani , UCL

T5.2 Hardware: Current TLU Open Hardware: ohwr.org “AIDA mini-TLU”

T5.2 Hardware: Next version TLU Increased number of inputs ( 46) Increased number of “Device Under Test” interfaces ( 34) Added low jitter clock generator (si5345) Allows TLUs to be used as a slave as well as a master. Can chain TLUs Low jitter clock for precision timing

T5.2 Hardware: Next version TLU Increase resistance to noise on DUT links. Target new FPGA family as carrier board Xilinx Spartan-6  Artix-7 Physically smaller carriers. Higher performance Circuit Schematics ready for review … volunteers welcome.

Task 5.3: Central DAQ software and run control Lead – University College London (M. Wing). 1 staff member for 2 years Close links with WP15 Yi Liu – IHEP/DESY lead developer Based on EUDAQ Developed in EUDET for pixel telescopes but light-weight and generic

Write separate file for each producer Changes from EUDAQ 1.x  2.x Write separate file for each producer Single data collector in 1.x doesn’t scale Works around limitations in LCIO event model LCIO assumes you know what an event is when you write the file. Can easily form chains of “producers” Easier to understand code

EUDAQ v1.x Data Flow EUDAQ v1.x EUDET Hardware Software Run Control TLU Control TLU Producer Log Collector Data Collector Online Monitor Run Control DUT1 DAQ DUT1 Producer DUT2 DAQ DUT2 Producer Telescope DAQ Telescope Prod.

EUDAQ v2 Data Flow EUDAQ v2.x EUDET Hardware Software Run Control TLU Control TLU Producer Data Collector 0 Log Collector Online Monitor Run Control Event Builder DUT1 DAQ DUT1 Producer Data Collector 1 DUT2 DAQ DUT2 Producer Data Collector 2 Telescope DAQ Telescope Prod. Data Collector 3

Event Builder …… …… EUDAQv1 case: FIFO 0 FIFO 1 EUDAQv1 case: Same event rate from sub-detector is mandated. One system trigger ensures the same event rate. DataCollector just merges them by one by one. …… FIFO tlu EUDAQv2 case: Event/Trigger N+1 Event/Trigger N FIFO 0 FIFO 1 FIFO tlu …… Time No unified system trigger Different data rate from sub-detectors Self triggered sub-detector may exist (FIFO 0) No data in FIFO 0 Event/Trigger N+1 Event/Trigger N

Processor framework (the solution for “Event Builder”) Data processing code is encapsulated as a Processor ( R. Peschke ) Processors are linked to a chain to processing data stream. Data stream can be forked and rejoined Run Independently Multiple threads capability and safety. A B C Z A B C Z D E A B C Z D A “I do everything myself.” Automatic thread instances: A: root processor F: join processor D A B C Z E F 1 2 User is completely unware of the threads when the derived processor class is implemented. Threading or NOT, is depending on the location where the processor is.

Better separation of Framework / User Code. The line between EUDAQ core framework and user/hardware specified code: EUDAQv1 compiled to a single shared library (so, dll). Device specified code (DataConverterPlugin) is inside the single library. Framework and user code is unfriendly mixed. Modification of user code will result in to re-compile all the EUDAQ lib. EUDAQv2 A core shared library of framework and a collection of user libraries (plugins). Separate user code to user folders. (done) Make plugin binaries with compile-time linking. (done) Make plugin binaries discovered and loaded at run-time. (TODO) User derived Processor and Event complied as plugin libraries binary. (TODO)

Task 5.4 Data quality and slow control monitoring Lead by University of Sussex (Fabrizio Salvatore) Decided to use DQM4HEP framework Developed for SDHCAL by Rémi Eté (IPNL) and Antoine Pingault (UGent) https://github.com/DQM4HEP Currently used in combined ECAL-SDHCAL test beam in CERN Generic data structure: compatibility with any input data

EUDAQ with DQM4HEP 16 EUDAQ DQM4HEP Disk Producer 0 Data Collector (LCIO) Producer 0 Data Collector LCIO File Streamer Producer 1 Run Control Monitor (EUDAQ) Event Collector Producer n Logger Analysis Module Monitor (DQM4HEP)

EUDAQ with DQM4HEP 17 EUDAQ DQM4HEP Disk Producer 0 Data Collector (LCIO) Producer 0 Data Collector LCIO File Streamer Producer 1 Run Control Monitor (EUDAQ) Event Collector Producer n Logger Analysis Module Monitor (DQM4HEP)

Direct interface with EUDAQ DQM Plans Direct interface with EUDAQ Currently “quasi-online” – reads from file Build a subset of events and send to monitoring Steering from EUDAQ Documentation/guide/examples for writing own analysis module

Task 5.5 - Event model for combined DAQ Lead DESY – Adrian Irles EUDAQ 1.x Event model One trigger = one particle = one event Good fit with LCIO event model Simple Inadequate when Mixing detectors of different integration times ( e.g. rolling shutter and LHC detectors. Self triggered detectors ( e.g. LC calorimeter )

AHCAL (+ BIF) as Test Case AHCAL: TCP stream. One readout = all packets with same ROC. Packets come out-of-order (Depends on data occupancy) ASICS are readout in a given BxID if the ASIC had a autotriggered signal in one of its own channels Readout cycle: L1 A1 Layer 2, ASIC 1 Layer 1, ASIC 1 Layer 1, ASIC 2 L1 A2 Start Acq Bytes Bytes Bytes L2 A1 TS hit 1 L1 A3 TS hit 2 Mcell: BxID: L3 A1 Mcell: BxID: Mcell: BxID: TS hit 2 ROC 2 L3 A2 2 2 TS hit 4 1 3 1 6 1 5 L2 A2 . 2 6 2 7 2 9 L3 A3 3 8 3 9 3 ... L1 A4 5 9 4 ... (missing BxID 6) 4 ... L2 A3 TS hit N L2 A4 Stop Acq noise L3 A4 BIF AHCAL

Developing methods of “event building” for Event Model Developing methods of “event building” for Self triggered detectors Detectors with different integration times

Use of TLU Hardware by Calorimeter Use as “Beam InFormation” device to record particle arrival times Jiri Kvasnicka - AHCAL BIF Shows correlation of different systems Shows monitoring works for two systems One triggered, one self triggered

Beam InterFace (BIF) Mini-TLU without any hardware modification FMC mezzanine card, 4 lemo inputs, 2 HDMI + 1 RJ45 DUT ports, 1 clock port Modified firmware (svn rev. 245) for operation as a slave device: Clock distribution modified, whole BIF internally runs on the external clock Learned the AHCAL fast commands from HDMI External shutter for gating of the trigger data storage (only data within AHCAL readout cycle saved) Special cable for connection to CCC Software: EUDAQ Trigger Spill CCC HDMI cable … LDAs LDA PC PC 3 pairs in RJ45 gnd Clock (diff) beam trigger BIF DIF

Results and experience Used by AHCAL group in: Testbeam Nov 2015 (Spiroc2D single HBU) Testbeam May 2016 (Spiroc2B, more layers) Useful for SPIROC TDC calibration histogram of time between two triggers Time between 2 events: 1.8 ns jitter => Single timestamp jitter: 1.3 ns Includes jitter from scintillator, PMT, discriminator and coincidence units Mini-TLU itself should be better Observed: UDP packet loss exception, but can recover 1 readout cycle is typically lost SW issue, HW counts correctly sometimes lost Test on table: 100 kHz trigger in AHCAL acquisition cycle timing sustainable (100x more than actual particle rate)

Interface Specification for Common DAQ Deliverable D5.1 Interface Specification for Common DAQ

D5.1 Common DAQ Interface Specification Deliverable needs to be submitted by end M15 ( July 2016 ) Needed for practical as well as contractual reasons. Define what is needed to participate in AIDA-2020 common beam-test Aims to be short – refers to external documents where possible. Draft for comments on AIDA-2020 WP5 pages: https://espace.cern.ch/AIDA2020-intranet/WP5/WP5%20Documents/AIDA-2020_D5-1_v0-1.pdf

Timing/Synchronization/Triggering ( with TLU) D5.1 Hardware Interface Timing/Synchronization/Triggering ( with TLU) EUDET interface ( asynchronous clocks ) Synchronize data using trigger numbers AIDA interface ( synchronous clocks ) Synchronize using timestamps and/or trigger number clock , trigger , busy , sync Points to AIDA-TLU manual and EUDET TLU memo

D5.1 Software Interface Define interfaces for: Data storage Store/retrieve data with enough meta-data to correlate with other detectors Run Control Define interface Easy for EUDAQ based systems. Define finite state machine for data producers Have extra state INITIALIZED? ( w.r.t. current EUDAQ ) Useful for Calorimeter and TPC On line monitoring ( DQM )

Test Beam Needs No dedicated DAQ development beam tests at CERN Combined beam-test driven by detector community Some DAQ development beam-tests envisaged at DESY

Summary Enhancing EUDAQ to cope with high rate beam-tests with heterogeneous detectors Generic framework for DQM being developed Based on framework written by LC Calorimeter community Proof of principle from AHCAL beam-tests Generic ways of event building for heterogeneous detectors being developed. Next version of TLU (for “mass production”) due Summer Specification for Interfacing to Common DAQ ( D5.1 ) at draft stage.