1. WP5: Common DAQ
David Cussans, on behalf of WP5 team: University of Bristol, DESY Hamburg, Institute of Physics AS CR Prague, University of Sussex, University College London

2. Outline
Task 5.2 Interface, synchronisation and control of multiple-detector systems
Task 5.3 Central DAQ software and run control (together with WP15 beam test infrastructure)
Task 5.4 Development of data quality and slow control monitoring
Task 5.5 Event model for combined DAQ
Interface Specification D5.1

3. Tasks
(Task 5.1 is management)

4. Task 5.2: Hardware Timing and Synchronization
New Physicist/Engineer started at Bristol 1/6
Paolo Baesso – DAQ expert.
50% on AIDA-2020 for lifetime of project.
Schematic for next version of mini-TLU ready for review
Porting firmware from Xilinx Spartan-6 to Artix-7 underway
Samer al-Kilani , UCL

5. T5.2 Hardware: Current TLU
Open Hardware:
ohwr.org
“AIDA mini-TLU”

6. T5.2 Hardware: Next version TLU
Increased number of inputs ( 46)
Increased number of “Device Under Test” interfaces ( 34)
Added low jitter clock generator (si5345)
Allows TLUs to be used as a slave as well as a master.
Can chain TLUs
Low jitter clock for precision timing

7. T5.2 Hardware: Next version TLU
Increase resistance to noise on DUT links.
Target new FPGA family as carrier board
Xilinx Spartan-6  Artix-7
Physically smaller carriers. Higher performance
Circuit Schematics ready for review
… volunteers welcome.

8. Task 5.3: Central DAQ software and run control
Lead – University College London (M. Wing).
1 staff member for 2 years
Close links with WP15
Yi Liu – IHEP/DESY lead developer
Based on EUDAQ
Developed in EUDET for pixel telescopes but light-weight and generic

9. Write separate file for each producer
Changes from EUDAQ 1.x  2.x
Write separate file for each producer
Single data collector in 1.x doesn’t scale
Works around limitations in LCIO event model
LCIO assumes you know what an event is when you write the file.
Can easily form chains of “producers”
Easier to understand code

10. EUDAQ v1.x Data Flow EUDAQ v1.x EUDET Hardware Software Run Control
TLU Control
TLU Producer
Log Collector
Data Collector
Online Monitor
Run Control
DUT1 DAQ
DUT1 Producer
DUT2 DAQ
DUT2 Producer
Telescope DAQ
Telescope Prod.

11. EUDAQ v2 Data Flow EUDAQ v2.x EUDET Hardware Software Run Control
TLU Control
TLU Producer
Data Collector 0
Log Collector
Online Monitor
Run Control
Event Builder
DUT1 DAQ
DUT1 Producer
Data Collector 1
DUT2 DAQ
DUT2 Producer
Data Collector 2
Telescope DAQ
Telescope Prod.
Data Collector 3

12. Event Builder …… …… EUDAQv1 case:
FIFO 0
FIFO 1
EUDAQv1 case:
Same event rate from sub-detector is mandated.
One system trigger ensures the same event rate.
DataCollector just merges them by one by one. ……
FIFO tlu
EUDAQv2 case:
Event/Trigger N+1
Event/Trigger N
FIFO 0
FIFO 1
FIFO tlu ……
Time
No unified system trigger
Different data rate from sub-detectors
Self triggered sub-detector may exist (FIFO 0)
No data in FIFO 0 Event/Trigger N+1
Event/Trigger N

13. Processor framework (the solution for “Event Builder”)
Data processing code is encapsulated as a Processor ( R. Peschke )
Processors are linked to a chain to processing data stream. Data stream can be forked and rejoined Run Independently Multiple threads capability and safety. A B C Z A B C Z D E A B C Z D A
“I do everything myself.”
Automatic thread instances:
A: root processor
F: join processor D A B C Z E F 1 2
User is completely unware of the threads when the derived processor class is implemented. Threading or NOT, is depending on the location where the processor is.

14. Better separation of Framework / User Code.
The line between EUDAQ core framework and user/hardware specified code:
EUDAQv1
compiled to a single shared library (so, dll).
Device specified code (DataConverterPlugin) is inside the single library.
Framework and user code is unfriendly mixed.
Modification of user code will result in to re-compile all the EUDAQ lib.
EUDAQv2
A core shared library of framework and a collection of user libraries (plugins).
Separate user code to user folders. (done)
Make plugin binaries with compile-time linking. (done)
Make plugin binaries discovered and loaded at run-time. (TODO)
User derived Processor and Event complied as plugin libraries binary. (TODO)

15. Task 5.4 Data quality and slow control monitoring
Lead by University of Sussex (Fabrizio Salvatore)
Decided to use DQM4HEP framework
Developed for SDHCAL by Rémi Eté (IPNL) and Antoine Pingault (UGent)
Currently used in combined ECAL-SDHCAL test beam in CERN
Generic data structure: compatibility with any input data

16. EUDAQ with DQM4HEP 16 EUDAQ DQM4HEP Disk Producer 0 Data Collector
(LCIO)
Producer 0
Data Collector
LCIO File
Streamer
Producer 1
Run Control
Monitor
(EUDAQ)
Event
Collector
Producer n
Logger
Analysis
Module
Monitor
(DQM4HEP)

17. EUDAQ with DQM4HEP 17 EUDAQ DQM4HEP Disk Producer 0 Data Collector
(LCIO)
Producer 0
Data Collector
LCIO File
Streamer
Producer 1
Run Control
Monitor
(EUDAQ)
Event
Collector
Producer n
Logger
Analysis
Module
Monitor
(DQM4HEP)

18. Direct interface with EUDAQ
DQM Plans
Direct interface with EUDAQ
Currently “quasi-online” – reads from file
Build a subset of events and send to monitoring
Steering from EUDAQ
Documentation/guide/examples for writing own analysis module

19. Task 5.5 - Event model for combined DAQ
Lead DESY – Adrian Irles
EUDAQ 1.x Event model
One trigger = one particle = one event
Good fit with LCIO event model
Simple
Inadequate when
Mixing detectors of different integration times ( e.g. rolling shutter and LHC detectors.
Self triggered detectors ( e.g. LC calorimeter )

20. AHCAL (+ BIF) as Test Case AHCAL:
TCP stream. One readout = all packets with same ROC.
Packets come out-of-order (Depends on data occupancy)
ASICS are readout in a given BxID if the ASIC had a autotriggered signal in one of its own channels
Readout cycle:
L1 A1
Layer 2, ASIC 1
Layer 1, ASIC 1
Layer 1, ASIC 2
L1 A2
Start Acq
Bytes
Bytes
Bytes
L2 A1
TS hit 1
L1 A3
TS hit 2
Mcell:
BxID:
L3 A1
Mcell:
BxID:
Mcell:
BxID:
TS hit 2
ROC 2
L3 A2 2 2
TS hit 4 1 3 1 6 1 5
L2 A2 . 2 6 2 7 2 9
L3 A3 3 8 3 9 3
...
L1 A4 5 9 4
...
(missing BxID 6) 4
...
L2 A3
TS hit N
L2 A4
Stop Acq
noise
L3 A4
BIF
AHCAL

21. Developing methods of “event building” for
Event Model
Developing methods of “event building” for
Self triggered detectors
Detectors with different integration times

22. Use of TLU Hardware by Calorimeter
Use as “Beam InFormation” device to record particle arrival times
Jiri Kvasnicka - AHCAL BIF
Shows correlation of different systems
Shows monitoring works for two systems
One triggered, one self triggered

23. Beam InterFace (BIF) Mini-TLU without any hardware modification
FMC mezzanine card, 4 lemo inputs, 2 HDMI + 1 RJ45 DUT ports, 1 clock port
Modified firmware (svn rev. 245) for operation as a slave device:
Clock distribution modified, whole BIF internally runs on the external clock
Learned the AHCAL fast commands from HDMI
External shutter for gating of the trigger data storage (only data within AHCAL readout cycle saved)
Special cable for connection to CCC
Software: EUDAQ
Trigger
Spill
CCC
HDMI cable …
LDAs
LDA PC PC
3 pairs
in RJ45
gnd
Clock (diff)
beam
trigger
BIF
DIF

24. Results and experience
Used by AHCAL group in:
Testbeam Nov 2015 (Spiroc2D single HBU)
Testbeam May 2016 (Spiroc2B, more layers)
Useful for SPIROC TDC calibration
histogram of time between two triggers
Time between 2 events: 1.8 ns jitter
=> Single timestamp jitter: 1.3 ns
Includes jitter from scintillator, PMT, discriminator and coincidence units
Mini-TLU itself should be better
Observed: UDP packet loss
exception, but can recover
1 readout cycle is typically lost
SW issue, HW counts correctly
sometimes lost
Test on table: 100 kHz trigger in AHCAL acquisition cycle timing sustainable (100x more than actual particle rate)

25. Interface Specification for Common DAQ
Deliverable D5.1
Interface Specification for Common DAQ

26. D5.1 Common DAQ Interface Specification
Deliverable needs to be submitted by end M15 ( July 2016 )
Needed for practical as well as contractual reasons.
Define what is needed to participate in AIDA-2020 common beam-test
Aims to be short – refers to external documents where possible.
Draft for comments on AIDA-2020 WP5 pages:

27. Timing/Synchronization/Triggering ( with TLU)
D5.1 Hardware Interface
Timing/Synchronization/Triggering ( with TLU)
EUDET interface ( asynchronous clocks )
Synchronize data using trigger numbers
AIDA interface ( synchronous clocks )
Synchronize using timestamps and/or trigger number
clock , trigger , busy , sync
Points to AIDA-TLU manual and EUDET TLU memo

28. D5.1 Software Interface Define interfaces for: Data storage
Store/retrieve data with enough meta-data to correlate with other detectors
Run Control
Define interface
Easy for EUDAQ based systems.
Define finite state machine for data producers
Have extra state INITIALIZED? ( w.r.t. current EUDAQ )
Useful for Calorimeter and TPC
On line monitoring ( DQM )

29. Test Beam Needs
No dedicated DAQ development beam tests at CERN
Combined beam-test driven by detector community
Some DAQ development beam-tests envisaged at DESY

30. Summary
Enhancing EUDAQ to cope with high rate beam-tests with heterogeneous detectors
Generic framework for DQM being developed
Based on framework written by LC Calorimeter community
Proof of principle from AHCAL beam-tests
Generic ways of event building for heterogeneous detectors being developed.
Next version of TLU (for “mass production”) due Summer
Specification for Interfacing to Common DAQ ( D5.1 ) at draft stage.