1. The Application of DAQ-Middleware to the J-PARC E16 Experiment E Hamada 1, M Ikeno 1, D Kawama 2, Y Morino 1, W Nakai 3, 2, Y Obara 3, K Ozawa 1, H Sendai 1, T N Takahashi 4, M M Tanaka 1, S Yokkaichi 2 1 High Energy Accelerator Research Organization (KEK) 2 RIKEN Nishina Center 3 The University of Tokyo 4 Research Center for Nuclear Physics, Osaka University 2015/4/14CHEP20151 Track 1 Session: #3 (Data acquisition and electronics) April 14, 2015 18:00 – Village Center

2. Outline 1.DAQ-Middleware overview and features 2.DAQ System at E16 Experiment requirements and architecture 3.DAQ System Performance method and result 4.Summary and Future Plan 2015/4/14CHEP20152

3. 1. DAQ-Middleware 2015/4/14CHEP20153

4. What DAQ-Middleware Is A framework for network based DAQ software –Easy to use, configure and develop Target –Medium-scale experiments –Test benches (sensors, electronics) 2015/4/14CHEP20154

5. DAQ-Middleware Architecture Develop DAQ system by configuring DAQ components 2015/4/14CHEP20155 Dispatcher Gatherer ・・・・・・ Read-out -modules PC XML System Configuration Daq Operator Logger Monitor Command / Status software unit that achieve DAQ functions

6. DAQ Component 2015/4/14CHEP20156 InPort OutPort Service Port (command/status) Logics (for data handling) Data InPort OutPort Service Port (command/status) (for data handling) Data + = DAQ-Middleware provides users writes Logics

7. DAQ Component Features 2015/4/14CHEP20157 READOUT Flexibility Reusability Scalability logic Ring Buffer ・・ ・ Users can flexibly change DAQ component combination. Users can improve performance by adding new PCs and deploying DAQ components. Users can use a DAQ component in various DAQ system. DAQ component has a ring buffer. Users do not need to implement a buffer.

8. List of DAQ-Middleware Users 2015/4/14CHEP20158 DAQ-Middleware Working J-PARC MLF Neutron beam lines Experiments [19 experiments] - Material and Life Science - J-PARC MLF Neutron/Muon - DAQ system of Depth-resolved XMCD (KEK PF) - Elementary Particle/Nuclear Physics - J-PARCE16 Experiment - CANDLES - SuperNEMO (planning) Test benches for sensors and electronics [9 test benches] –Liquid Argon TPC –SOI Pixel Detector –ILC CCD Vertex –ADC-SiTCP ・ 2D gaseous detector with readout ASIC using printing technologies and so on….

9. 2. DAQ System at E16 Experiment 2015/4/14CHEP20159

10. Requirements 2015/4/14CHEP201510 660MB data receive 660MB data receive not receive 1spill not receive Trigger rate fluctuates due to beam rate variation. Event size per one event is almost constant. Estimation of data transfer to DAQ PCs Data rate per spill660MB/spill Trigger rate(average) 1kHz (max) 2kHz Instantaneous data rate (average) 330MB/s (max) 660MB/s

11. System Architecture 2015/4/14CHEP201511 Function ・ Event build ・ Analyze data & show the result Read-out-moduleNetwork switch Storage PC Network switchMonitor PC HDD Function ・ Read data ・ Save data on each HDD ・ Send data of a part of event to Monitor PC

12. DAQ Component Configuration 2015/4/14CHEP201512 Storage PC Monitor PC Gatherer Merger Dispatcher Logger Filter Merger Eventbuilder Monitor ComponentFunction GathererRead data MergerReceive data from multiple Gatherers DispatcherSend data to Logger and Filter LoggerSave data on each HDD FilterSend data of a part of event to Merger of Monitor PC ComponentFunction MergerReceive data from multiple Filters EventbuilderEvent build MonitorAnalyze data & show the result

13. DAQ Component Features 2015/4/14CHEP201513 READOUT Flexibility Reusability Scalability logic Ring Buffer ・・ ・ Users can flexibly change DAQ component combination. Users can improve performance by adding new PCs and deploying DAQ components. Users can use a DAQ component in various DAQ system. DAQ component has a ring buffer. Users do not need to implement a buffer.

14. If data volume increase by detector upgrade, we add Storage PC. Advantage of DAQ Component Features 2015/4/14CHEP201514 Flexibility & Scalability Reusability Ring Buffer Using DAQ component which is prepared, we saved time and effort for development. We can use no event time effectively. SampleDispatcher, Logger Gatherer, Monitor DAQ system receive process receive process

15. 3. DAQ System Performance 2015/4/14CHEP201515

16. Purpose & Method 2015/4/14CHEP201516 Emulator Storage PC Network switchMonitor PC HDD Emulator test data send data once every 10 events Purpose Evaluation of a total throughput using a DAQ-Middleware for the J-PARC E16 experiment Emulators run with 1 cycle per 6 seconds. During one cycle, emulators send test data for 2 seconds and do not send test data for 4 seconds. Monitor PC shows data value of a part of data regularly. We changed the number of emulators to change transfer data volume and measured processing data speed. Method

17. Result The result met the requirements of the E16 experiment! 2015/4/14CHEP201517 requirement throughput (Trigger rate is always max [2kHz]. ) HDD write limit requirement throughput (Trigger rate is always average [1kHz]. ) During sending test data, one event data size per one emulator is 14kB. 2kHz 1kHz

18. 4. Summary and Future Plan 2015/4/14CHEP201518

19. Summary and Future Plan DAQ-Middleware is a framework for network based DAQ software. DAQ component has following features. –Flexibility –Scalability –Reusability –Ring Buffer We have developed DAQ system for E16 experiment by using DAQ-Middleware. The requirements from E16 experiment have been met. In the future, we are going to connect DAQ system to read out module, and evaluate the system. 2015/4/14CHEP201519

20. Backup 2015/4/14CHEP201520

21. J-PARC E16 Experiment DAQ system overview 2015/4/14CHEP201521

22. Evaluation Environment 2015/4/14CHEP201522 HDD PC1 PC2 PC3 CPUIntel(R) Xeon(R) X5650 @ 2.67GHz 6Cores Memory24GB Network1Gbps x 5 OSScientific Linux 6.4 HDD Hitachi HDS724040ALE6 4TB PC1 CPUIntel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz 6Cores Memory32GB Network1Gbps x 5 OSScientific Linux 6.6 HDD Hitachi HDS724040ALE6 4TB PC2 CPUIntel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz Memory8GB Network2Gbps OSScientific Linux 6.6 PC3 Cisco Catalyst 2960G

23. HDD write speed check 2015/4/14CHEP201523

24. SRS (Scalable Readout System) 2015/4/14CHEP201524 SRS is general purpose multi-channel readout system. User can choose variety of frontend chips. FECADC FEC Front-End Card FEC Front-End Card FEC Front-End Card Eurocrate Front-End ASIC SRU

25. Evaluation by using SRS 2015/4/14CHEP201525 SRS ADC/EFC ADC/FEC send test data. Maximum transfer speed of test data is 1Gbps + 1Gbps. We configured that DAQ PC could process all of data. Storage PC Network switchMonitor PC HDD test data send data once every 10 events

26. Storage PC Performance 2015/4/14CHEP201526 Emulator Storage PC Test PC emulator CPU Intel(R) Xeon(R) X5650 @ 2.67GHz 6Cores Memory 24GB Network 1Gbps x 10 OS Scientific Linux 6.4 SSD Intel SSD520Series 240GB only receive data Throughput of 1 Storage PC is 1000MB/spill. write data on SSD test data trigger rate = 2kHz 1 event size = 14kB

27. DAQ Component & Configuration Example 2015/4/14CHEP201527 InPort OutPort Service Port (command/status) Logics (for data handling) Data InPort OutPort Service Port (command/status) Logics (for data handling) Data += DAQ-Middleware provides user writes Monitor DaqOperator Gatherer Online Monitor only Examples of DAQ component combination Logger DaqOperator Gatherer Logging only Dispatcher Logger Monitor DaqOperator Gatherer Online Monitor & Logging