Bachir Bouhadef, Mauro Morganti, G. Terreni (KM3-Italy Collaboration) Parallel Neutrino Triggers using GPUs for an underwater telescope KM3-NEMO Bachir Bouhadef, Mauro Morganti, G. Terreni (KM3-Italy Collaboration) INFN Pisa & Physics Department of Pisa

INFN Pisa & Physics Department of Pisa Outlines NEMO Phase II Tower DAQ system for NEMO Tower Showing a possibility of using a CPU-GPU DAQ for an online muon-track selection. Proposing a method for Parallelizing the online trigger software NEMO-II Tower. Test muon triggers in GPUs. Proposition for KM3Net-It Tower Trigger data handling. 2 INFN Pisa & Physics Department of Pisa 2

INFN Pisa & Physics Department of Pisa NEMO Tower Phase II. KM3 - NEMO Tower . 8 floors 4 PMT/Floor 8m Floor Arm length Floor 14 Floor 13 Floor 1 14 floors 6 PMT/Floor 6m Floor Arm 84 PMTs 32 PMTs X 55 kHz 1.7 Mhit/s 3 INFN Pisa & Physics Department of Pisa 3

Trigger and Data Acquisition System (NEMO Phase II) T. Chiarusi & F. Simeone VLVNT 2013 GPUs in HEP INFN Pisa & Physics Department of Pisa 4 4

Trigger and Data Acquisition System (NEMO Phase II) SC FC Onshore TS 1 TS 0 time AFC TS 0 TS 0 HM 0 TCPU 0 Gbit swich TCPU 1 HM 1 TS 1 TS 1 TS 1 TS 0 time TS is Time Slice of 200ms. The trigger in TCPU: sorting time’s PMTs hits and applying a charge threshold and time coincidences (simple coincidence, Floor coincidence). INFN Pisa & Physics Department of Pisa 5 5

Scalable Programming Model Why GPUs ? Scalable Programming Model A GPU uses blocks and threads for parallel programming SC AFC Tower D.U. INFN Pisa & Physics Department of Pisa 6 6

INFN Pisa & Physics Department of Pisa A new parallel trigger for muon detection based on time difference of muon’s hits Most of the muon track hits are Within a Time Windows (TD). We need at least 5 hits in different PMTs to reconstruct the muon track. We must look for a number of hits N in a fixed time windows. INFN Pisa & Physics Department of Pisa 7 7

INFN Pisa & Physics Department of Pisa 8 8

INFN Pisa & Physics Department of Pisa We propose a DAQ system using CPU-GPU Architecture Gbit swich Onshore Storage Unit INFN Pisa & Physics Department of Pisa 9 9

INFN Pisa & Physics Department of Pisa TGPU-CPU TCPU is replaced by TGPU-CPU, and every second the TGPU-CPU will receive 5 Time slices of 200ms each. INFN Pisa & Physics Department of Pisa GPUs in HEP 10 10

INFN Pisa & Physics Department of Pisa 5 TTS from network thread TTS4 TTS3 TTS2 TTS1 TTS0 Network THRD Step 1 5 CPU threads to put PMT hits in correct time interval, and the hits are time-order by thread Structure is ready to be treated in GPU TTS4 TTS3 TTS2 TTS1 TTS0 Step 2 CPU work The size of hits are not fixed, we must prepare a new structure for the GPU Number of threads must be multiple of 5 and 32 We can not predict how many hits per thread there are, so we fixed a max hits number by thread using the nominal rate X 3 or 6 We have considered also the edge effect between threads and TTSs. We should avoid threads with a few hits by choosing the optimal thread time interval. INFN Pisa & Physics Department of Pisa 11 11

INFN Pisa & Physics Department of Pisa Sort all PMT hits using classical Algorithms (shell) Step 1 Structure is ready to Trigger tagging Step 2 TTS0 TTS1 TTS2 TTS3 Trigger L0 In L1 Trigger all possible trigger can be implemented, and according to L0+L1 efficiencies the best is chosen to tag the event to be saved. GPU work 1 (L0) - N7TW1000 (L0) SC or AFC 2 (L1) - N7TW1000 & SC 3 (L1) - N7TW1000 & AFC 4 (L2) - N7TW1000 & SC & AFC 5 (L2) - N7TW1000 & ( (SC & AFC ) || (SC>1) || (AFC>1)) 6 (L2) - N7TW1000 & (SC || AFC)) 7 (L0) Charge > Charge_THRHD 1 2 3 4 INFN Pisa & Physics Department of Pisa 12 12

INFN Pisa & Physics Department of Pisa Muon triggers tests in GPUs GPU cards INFN Pisa & Physics Department of Pisa 13 13

INFN Pisa & Physics Department of Pisa Muon triggers tests in GPUs Trigger Time execution for one tower of 32PMT/ ~55kHz. Using 2 different GPU cards and one 1 second of raw data for 32 PMTs at background rate ~ 55 kHz and applying all triggers. The back times are measured while the CPU in Idle. The read times are measured times while the CPU executes other processes INFN Pisa & Physics Department of Pisa 14 14

INFN Pisa & Physics Department of Pisa Muon triggers tests in GPUs Using the same two GPU cards, we use one second data of 84 PMTs (6PMT X 14 Floors) at background rate 55 kHz and applying all triggers. The back numbers are the measured times while the CPU in Idle. The red numbers are the measured times while the CPU executes other processes INFN Pisa & Physics Department of Pisa 15 15

INFN Pisa & Physics Department of Pisa We have also tested a new trigger that combines between 7 different hits within 1000ns and time-space correlation with respect to first hit. In GPU the trigger was applied to data tower of 84 PMTs, and measured trigger time is 350ms in case of Tesla20c50. We are also working muon track reconstruction in GPUs. Sorting in Tesla 20c50 100 X 182 threads Alessio Bacciarelli INFN Pisa & Physics Department of Pisa 16 16

INFN Pisa & Physics Department of Pisa Proposed DAQ CPU-GPU for 8 KM3-Ita Tower INFN Pisa & Physics Department of Pisa 17 17

INFN Pisa & Physics Department of Pisa Conclusion and future work - GPUs can also take place in optical neutrino telescopes. - Still other online more selective algorithms can be applied. - Muon track reconstruction is underway. - Both Tesla 20c50 and GTX TITAN can be a good choice. INFN Pisa & Physics Department of Pisa 18 18

Thank you for attention I would like to thank all Members of the KM3Net-It Collaboration INFN Pisa & Physics Department of Pisa 19 19