Data Acquisition Backbone Core J. Adamczewski-Musch, N. Kurz, S. Linev GSI, Experiment Electronics, Data processing group
DABCDABC S. Linev DAQ architectures Detector FEE buffer Readout buffer Switch Processor farm Storage Self-triggered Front-end all hits shipped to DAQ. Data push architecture High-throughput Event building First event selection done in processor farm. Readout buffer outside radiation area. Many Gbyte storage easily possible. Allows L1 decision times of ms Fast links Conventional DAQ L1 trigger HLT L1 CBM DAQ
DABCDABC S. Linev InfiniBand performance studies 2006 * 2007 ** 2011 *** Measurements: * GSI, 4-nodes SDR IB cluster ** Uni Mainz, 120-nodes DDR IB cluster *** LOEWE-CSC, Höchst, 836-nodes QDR IB cluster (preliminary)
DABCDABC S. Linev DAQ requirements Flexible – adopt different kinds of soft- and hardware Compact – use only necessary code Scalable – from small detector tests to many- nodes clusters Performing – low framework overhead Multiprocessing & multithreading
DABCDABC S. Linev Data Acquisition Backbone Core Controls core-test net-test bnet-test mbs bnet-mbs Slim (batch) DIM Java GUI ApplicationsPlugins socket IB verbs bnet mbs bnet-mbs Core Plugins: Implementation of applications (programmers) Applications: Mainly setup or testing programs (users) DABCDABC Download via
DABCDABC S. Linev DABC dataflow concept UserModule “Pool” “Input” “Output0” “Output1” File UserModule “Pool” “Input” “Output0” “Output1” UserModule “Pool” “Input” “Output0” “Output1” File DataServer NetworkDevice PCIeDevice node1 node2 Transport Transport: manages buffers queue runs in own thread decouples user code from actual transport functionality Device: represents hardware items manages several transports
DABCDABC S. Linev User code - modules dabc::Module dabc::PoolHandle dabc::Timer dabc::Port dabc::Parameter dabc::Module provides: I/O ports for communications Pools handles to request memory Timers for timeouts processing Configuration & monitoring parameters
DABCDABC S. Linev Many other features Configuration with XML files Controlling interface and GUI Factories (plugins) architecture Application and state machine Event building network (BNET) Runs on 32/64 bit Linux
DABCDABC S. Linev CBM hardware prototypes nXYTER, 128-channel self-triggered readout chip with few ns time resolution, DEFNI collaboration CBM Readout controller (ROC), Kirchhoff Institut für Physik, Heidelberg
DABCDABC S. Linev DABC as access layer to ROC file I/O online monitor ROC-udp plugin ROC-optic plugin User access layer DABCDABC Users scripts, GUIs optic PCIe Ethernet
DABCDABC S. Linev ROC software components DABC-based components roc-udp plugin roc-optic plugin combiner and sorter modules readout application rocGui and rocutil for configuration go4-based online analysis
DABCDABC S. Linev CBM beam test in CERN, Nov.10 ROC Eth SYNC A B ROC Eth SYNC A B ROC Eth SYNC A B STS station RICH MBS Eth SYNC sender Susibo USB SYNC SPADIC Susibo USB SYNC SPADIC Susibo USB SYNC SPADIC Susibo USB SYNC SPADIC Susibo USB SYNC SPADIC Susibo USB SYNC SPADIC TRD1 TRD2 TRD3 TRD4 Beam monitor PC DABC Eth USB Control room storage go4 online monitor Three different readouts: ROC UDP-based readout SPADIC/Susibo USB readout MBS readout Combination of free-running and triggered electronics Cave
DABCDABC S. Linev node3 node2 node1 DABC application SPADIC combiner Input1 Output Input0 Input2 Input3 Input4 Input5 Super combiner MBS Output0 ROC SPADIC Output1 ROC combiner Input1 Output Input0 Input2 go4 online monitor lmd file storage MBS ROCs SPADICs/Susibo
DABCDABC S. Linev CBM beam test in COSY, Dec.10 ROC SFP SYNC A B Data combiner board SFP ROC SFP SYNC A B ROC SFP SYNC A B ROC SFP SYNC A B ROC SFP SYNC A B ROC SFP SYNC A B ROC SFP SYNC A B ROC SFP SYNC A B Data combiner board SFP STS station 1 STS station 2 GEM station 1 GEM station 2 MBS Eth SYNC sender Beam monitor PC DABC AVNet PCIe SFP Eth PC DABC/ EPICS AVNet PCIe SFP Eth data readout and control via optic clock synchronization via optic optional MBS readout optional EPICS readout CaveControl room 20 m
DABCDABC S. Linev DABC application Super combiner MBS Output0 ROC EPICS Output1 go4 online monitor Lmd file storage MBS ROCs ROC combiner Input1 Output Input0 Input2 Input3 Input4 Input5 Input6 Input7 DABC/ EPICS ROC data splitter Output1 InputOutput0 Output2 Output3 Output4 Output5 Output6 Output7 optic readout
DABCDABC S. Linev Plans for version 2 More scalability and flexibility Failure detection and recovery Redesign of control interface
DABCDABC S. Linev Conclusion DABC is general-purpose DAQ framework Can be used for different purposes – from small detectors tests to multi-nodes application Because of its plugin architecture can be easy extended to specific needs Open for improvements and new ideas