White Rabbit and KM3NeT Peter Jansweijer, on behalf of KM3NeT &

Outline Intro: KM3NeT daq from a technical perspective A multi-km3 neutrino telescope in the deep-sea KM3NeT daq from a technical perspective

Potential neutrino sources Supernova Remnants Pulsar Wind Nebula ? Dark Matter Cosmogenic neutrinos Active Galactic Nuclei Micro Quasars Gamma-Ray Burst

Active Galactic Nuclei Detection method Strings with optical sensor modules Cherenkov light muon Active Galactic Nuclei neutrino Neutrino-induced muons in the deep sea

Active Galactic Nuclei Deployment Active Galactic Nuclei 640 string with optical sensors in the deep sea at 3-5 km depth

KM3NeT Artist Impression 640 strings 20 DOM/string 12800 DOMs Volume: ~5 km3 12,800 DOMs in the deep sea at 3-5 km depth with point-to-point connection to shore

Digital Optical Module (DOM) Lower Hemisphere 19 PMTs Upper Hemisphere 12 PMTs PMT Base: High Voltage Supply Analog Front-End Central Logic Board (CLB)

Central Logic Board (CLB) Readout 31 TDC’s with 1 ns resolution “Knowledge” of absolute time (1 ns resolution) Data pushed from PMTs to Shore Station I2C: PMT-HV, Threshold, Compass, Tilt Other IO: Temp, Nano beacon, Acoustics Firmware must be reconfigurable Low Power Low Cost Part of a scalable system (with respect to the complete detector) Highly reliable

White Rabbit is going deep-sea!

Central Logic Board (CLB) IP/UDP Packet Buffer Stream Selector (IPMUX) 31 TDCs Fifo TDC0 RxPacket Buffer 64KB RxPort 1 State Machine 31 PMTs RxPort 2 Rx_mac2buf Rx_buf2data Rx Stream Select Fifo TDC30 Flags RxPort_m Management & Control Pause Frame Management & Config. Fifo ADC Hydrophone TxPacket Buffer 32KB TxPort 1 TxPort 2 Tx_pkt2mac Tx_data2buf Tx Stream Select Flags Management & Control TxPort_m Nano Beacon Debug LEDs Xilinx Kintex-7 2nd CPU LM32 UART I2C I2C GPIO MEM Data UTC time & Clock (PPS, 125 MHz) Control Point to Point interconnection Debug RS232 Temp Compass Tilt Wishbone shared bus (32 bits)

Shore Station Shore Sub Sea Servers may be used for storage and 192.16.x.1 Server2 192.16.x.2 Server n 192.16.y.z Servers may be used for storage and Pre-processing Switch Server Server Buffers Server Server 10MHz GPS PPS WR Switch WR Switch Shore Sub Sea DOM 1 DOM 2 DOM 16 DOM 12798 DOM 12799 DOM 12800 Each DOM synchronizes to the absolute time (using White Rabbit) Each DOM receives a look up table with IP addresses while configuring the detector. All DOMs start at an absolute point in time which was communicated via a command over the White Rabbit network. All DOMs start their first time-slice at exactly the same time. All data is IP/UDP formatted and passed to the IP number corresponding to the time slice After ‘n’ time slices, first PC is again selected to process the data Time Slice Time Slice IP Address IP 1 Time Slice IP 192.16.x.1 2 192.16.x.2 : : n 192.16.y.z

Work to be done (with respect to White Rabbit) There is no PCIe in the deep sea… Debug via UART Firmware reconfigurable; (new) software loaded together with (new) FPGA configuration (Block Memory Mapping file) Explore the Kintex-7 deterministic latency GTX Future Artix-7 may be even more cost effective in terms of money and power Implement Oscillators, DACs (used by Soft-PLL) and SFP on FMC card which is to be plugged onto Xilinx KC705 Evaluation board (Mesfin Gebyehu) 1st goal: Replace the (slave) SPEC in the SPEC<-> SPEC test setup with the KC705 implementation for validation. Study Shore Station (e.g. White Rabbit Switch v3) broadcast

To be studied: Network Broadcast ReferenceClock Start PTP Broadcast Buffer Buffer Tx SFP Buffer Time Stamp t1 DOM Buffer SFP Optical Network Time Stamp t4 Rx t4 Stop1 DOM j: DDMTD SFP Time Stamp t4 Rx t4 Stop2 DOM j: DDMTD SFP Time Stamp t4 Rx t4 Stop3 j: DDMTD DOM SFP Time Stamp t4 Rx t4 Stop4 Main Electrical Optical Cable (MEOC) May be 100 Km long… Shore Station interface j: DDMTD Can this be done with the WR Switch-V3? Need Firmware/Software change?

(White Rabbit community) Thank you! (White Rabbit community)

Backup Slides

LM32, three boot types Generate ROM image before synthesis (used for functional simulation debug) Describe a generic RAM using “init” file Useful for functional simulation Incorporates “boot.elf” in the block-rams Download “elf” via an external interface in a running system (used for software debug) Useful for debugging purposes (the SPEC uses PCIe or JTAG) No BMM=Block Memory Mapping file needed After Place&Route (will be used in final CLB) Merge “FPGA.bit” file and “boot.elf” file (data2mem) This needs BMM=Block Memory Mapping file The “bit” file which is outputted by the merge can be used as updated configuration file This avoids synthesis each time software is updated

BMM (Block Memory Mapping) File Fpga.bit Fpga_elf.bit ROM ROM RAMB36 0x00 RAMB36 0x12 RAMB36 0x00 RAMB36 0x34 RAM RAM RAMB36 0x00 RAMB36 0x56 RAMB36 0x00 RAMB36 0x78 Data2Mem hello.elf ADDRESS_SPACE lm32_memory RAMB36 [0x00000000:0x00011FFF] BUS_BLOCK u1_u0/U0/xst_blk_mem_generator/gnativebmg.native_blk_mem_gen/valid.cstr/ramloop[15].ram.r/v6_init.ram/NO_BMM_INFO.SP.SIMPLE_PRIM36.ram [31:30]; : END_BUS_BLOCK; END_ADDRESS_SPACE; ADDRESS_SPACE lm32_data_memory RAMB36 [0x00000000:0x00011FFF] BUS_BLOCK u6_u0/U0/xst_blk_mem_generator/gnativebmg.native_blk_mem_gen/valid.cstr/ramloop[15].ram.r/v6_init.ram/NO_BMM_INFO.SP.SIMPLE_PRIM36.ram [31:30]; fpga.bmm

Optical Network (Simplified) Broadcast λ1 λ1 PIN Mod λ1 λ2 REAM λ2- λn A1 2xCu DOM1 λ2 λ3 λ1 PIN λ2 C2 Junction Box λ2 REAM λ2 λn λ3 λ3 2xCu DOM2 A2 λn DU-container λn λ1 PIN Shore station λn REAM Slow Control l1 2xCu DOMn Multiple l Single l