Download presentation
Presentation is loading. Please wait.
1
Example of DAQ Trigger issues for the SoLID experiment
Alexandre Camsonne Workshop on Future Trends in Nuclear Physics Computing March 17th 2016
2
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
JLab DAQ specifics Almost continuous machine : 499 MHz repetition rate usually prevents to trigger on beam crossing like at collider usually trigger on detector High luminosity up to 1039 cm-2s-1 Typical scale of experiments if of order of 100 M$ ( different from HEP and CERN LHC ) Several different setup run for a few years 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
3
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Trigger goals Reduce rate so it can be handled by the electronics Front end dead time Data transfer Reduce data size Only record event of interest Improve signal to background 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
4
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Data readout VME 100 MB/s Calo Ethernet 100 MB/s CPU Cerenkov DLO6 Tape 250 MB/s SAS 250 MB/s FADC Scintillator Event Builder L3 Drive Silo APV transfer 80 MB/s CPU MPD CPU GEM APV DAQ bottle necks VME 100 MB/s 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
5
Example SoLID PVDIS experiment
Inclusive electron scattering Calorimeter and Gas Cerenkov 200 to 500 KHz of electrons Maximum 1.8 MHz total rate 30 individual sectors to reduce rate per sector to 60 KHz maximum 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
6
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
FADC readout FADC readout full waveform 10 samples Only want to readout FADC channel in the cluster to reduce number of channels readout because of background in case zero suppression does not work CTP generates a 64 bit pattern Send pattern to TI or FADC directly to trigger FADC Only channels from pattern are put in buffer 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
7
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
ECAL trigger 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
8
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Calorimeter Geometry Detector segmented in 30 sectors One crate per sector CTP CTP 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
9
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
CTP connections To neighbor CTP To neighbor CTP 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
10
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Neighboring sectors New CTP : has two additional optical links Can send Cerenkov and calorimeter edges to other sectors. CTP CTP CTP 8 Gbp/s optical link 8 Gbp/s optical link 8 Gbp/s optical link 36 calorimeters 9 Cherenkov = 150ns ns per m+ 300 ns ( data ) = 500 ns overhead Trigger decision = 500 ns (Transfer ) + 1us ( clustering ) < 4 us (APV) 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
11
PVDIS electron trigger
Coincidence ECAL and Gas Cerenkov Singles ECAL 286 KHz Singles rates Cerenkov 2 MHz Accidental 30 ns 16.6 KHz DIS electron 10.4 KHz Total rate 27 KHz 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
12
Event size FADC PVDIS with waveform
Detector Total number of channels Number of channels firing Number of samples Max size detector bytes Minimal size detector Typical size Shower 58 7 10 2784 336 772 Preshower Gas Cerenkov 9 3 432 144 Max total size 46KB 0.816KB 1.544KB Max rate Assuming 100 MB/s per crate One crate 2.1 KHz 121 KHz 60 KHz FADC data rate for 30 KHz = 60 MB/s 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
13
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
GEM readout APV25 Front GEM ASICs Up to channels APV 25 : 128 channels Readout VME based readout : 16 APV25 = 2048 channels ( ~ 10 $ / channels ) SRS readout : ethernet /PC based = 2048 channels ( ~ 3 $ / channels ) 1 crate per sectors for FADC and GEM 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
14
APV25 readout Switch Capacitor Array ASICS with buffer length 192 samples at 40 MHz : 4.8 us Look back 160 samples : 4 us APV readout time : t_APV = 141 x number_of_sample / 40 MHz t_APV(1 sample) = 3.7 us. Max rate APV front end : 270 KHz in 1 sample mode 90 KHz in 3 samples mode Will be triggered at max 60 KHz in 3 samples 100KHz Max in 1 sample Optional on chip deconvolution Front-end FPGA deconvolution being implemented Deadtimeless pipelined electronics / parallel read and write 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
15
GEM event occupancy and size
Sector Rate XY Bytes 3 samples ( bytes) 199 398 1592 4776 1 147 294 1176 3528 2 107 214 856 2568 3 102 204 816 2448 4 Total hits / sector 657 1314 5256 15768 Data rate / sector 30000 Data rate ( sector Mb/s) 157,68 473.1 Total detector ( x30) 19710 4730.4 Occupancy detector 0.14 Data rate to front end reading 3 samples Use 4 Gigabit link = 512 MB/s not for safety 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
16
GEM event occupancy and size after deconvolution
Sector Rate XY Bytes 3 samples ( bytes) 23 46 184 552 1 12 24 96 288 2 10 20 80 240 3 9 18 72 216 4 Total hits / sector 63 126 504 1296 Data rate / sector 30000 Data rate ( sector Mb/s) 15.12 45.36 Total detector (x30) 1620 453.6 1360.8 Occupancy detector 0.013 Rates with deconvolution 3 samples readout Implementation in readout electronics No issue for transfer up to 60 KHz ( 90 MB / s ) Data after processing going to tape 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
17
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Future chip VMM family Zero suppression ADC and TDC on chip Logic function with input from other chips Reduce data on the ASIC stage and then on front end stage using FPGA before sending to readout controller 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
18
Level-3 Trigger and monitoring
L3 Farm EB1 Event Builder stage 1 EB2 stage 2 Staged Event Building blocked event fragments partially recombined event fragments N x M array of nodes (exact number to be determined by available hardware at time of purchase) Level-3 Trigger and monitoring full events L3 Farm node Raid Disk ER Event Recorder Event Recording 300MB/s in 300MB/s out All nodes connected with 1GB/s links Switches connected with 10GB/s fiber optics ROC Front-End Crates Read Out Controllers ~60 crates ~50MB/s out per crate 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
19
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
PVDIS data rates Simulation and trigger were checked and optimized Trigger rate estimated to be 27 KHz GEM data rate assuming 30 KHz after deconvolution around 46 MB/s FADC data 60 MB/s Total about 108 MB/s per sector to L3 x 30 for a total 3.24 GB/s Use L3 to reduce to 250 MB/s ( similar to Hall D ) 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
20
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
L3 data reduction Can use slow detectors that cannot be used at L1 Pile up detection Only record sample for event with pile up Calorimeter clustering GEM readout and tracking Timing cut Clustering Crude tracking Tracking Improved timing to reduce accidentals 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
21
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Tape size Days Data rate Seconds Total data TB Double DLO5 in $ DLO6 in $ E Pol proton 120 250 2592 5184 259200 155520 E J/Psi 60 1296 129600 77760 E Transv. Pol. 3He 90 1944 3888 194400 116640 E Long. Pol. 3 He 35 756 1512 75600 45360 E PVDIS 169 3650.4 7300.8 365040 219024 Total 474 614304 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
22
Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Conclusion SoLID requires high rates low dead time, flexible trigger capability Use of JLab pipeline and CODA3 electronics allows almost deadtimeless electronics and makes this experiment possible Serial readout and on board processing are additional available tools L3 farm capability similar to HEP in smaller scale gives more flexibility on the trigger Limiting factor : readout speed of the data from the modules and tape price 11/21/2018 Workshop on future trends in Nuclear Physics Computing DAQ Trigger
Similar presentations
