1. White Rabbit and KM3NeT Peter Jansweijer, on behalf of KM3NeT
&

2. Outline Intro: KM3NeT daq from a technical perspective
A multi-km3 neutrino telescope in the deep-sea
KM3NeT daq from a technical perspective

3. Potential neutrino sources
Supernova Remnants
Pulsar Wind Nebula ?
Dark Matter
Cosmogenic
neutrinos
Active Galactic Nuclei
Micro Quasars
Gamma-Ray Burst

4. Active Galactic Nuclei
Detection method
Strings with optical sensor modules
Cherenkov light
muon
Active Galactic Nuclei
neutrino
Neutrino-induced muons in the deep sea

5. Active Galactic Nuclei
Deployment
Active Galactic Nuclei
640 string with optical sensors
in the deep sea at 3-5 km depth

6. KM3NeT Artist Impression
640 strings
20 DOM/string
12800 DOMs
Volume: ~5 km3
12,800 DOMs in the deep sea at 3-5 km depth
with point-to-point connection to shore

7. Digital Optical Module (DOM)
Lower Hemisphere
19 PMTs
Upper Hemisphere
12 PMTs
PMT Base:
High Voltage Supply
Analog Front-End
Central Logic Board
(CLB)

8. Central Logic Board (CLB)
Readout 31 TDC’s with 1 ns resolution
“Knowledge” of absolute time (1 ns resolution)
Data pushed from PMTs to Shore Station
I2C: PMT-HV, Threshold, Compass, Tilt
Other IO: Temp, Nano beacon, Acoustics
Firmware must be reconfigurable
Low Power
Low Cost
Part of a scalable system (with respect to the complete detector)
Highly reliable

9. White Rabbit is going deep-sea!

10. Central Logic Board (CLB)
IP/UDP Packet Buffer
Stream Selector (IPMUX)
31 TDCs
Fifo
TDC0
RxPacket
Buffer
64KB
RxPort 1
State Machine
31 PMTs
RxPort 2
Rx_mac2buf
Rx_buf2data
Rx Stream Select
Fifo
TDC30
Flags
RxPort_m
Management
& Control
Pause Frame
Management
& Config.
Fifo
ADC
Hydrophone
TxPacket
Buffer
32KB
TxPort 1
TxPort 2
Tx_pkt2mac
Tx_data2buf
Tx Stream Select
Flags
Management
& Control
TxPort_m
Nano
Beacon
Debug LEDs
Xilinx
Kintex-7
2nd CPU
LM32
UART
I2C
I2C
GPIO
MEM
Data
UTC time & Clock (PPS, 125 MHz)
Control
Point to Point interconnection
Debug RS232
Temp
Compass
Tilt
Wishbone shared bus (32 bits)

11. Shore Station Shore Sub Sea Servers may be used for storage and
x.1
Server2
x.2
Server n
y.z
Servers may be
used for storage and
Pre-processing
Switch
Server
Server
Buffers
Server
Server
10MHz
GPS
PPS
WR Switch
WR Switch
Shore
Sub Sea
DOM 1
DOM 2
DOM 16
DOM
12798
DOM
12799
DOM
12800
Each DOM synchronizes to the absolute time (using White Rabbit)
Each DOM receives a look up table with IP addresses while configuring the detector.
All DOMs start at an absolute point in time which was communicated via a command over the White Rabbit network.
All DOMs start their first time-slice at exactly the same time.
All data is IP/UDP formatted and passed to the IP number corresponding to the time slice
After ‘n’ time slices, first PC is again selected to process the data
Time Slice
Time Slice
IP Address IP 1
Time Slice IP
x.1 2
x.2 : : n
y.z

12. Work to be done (with respect to White Rabbit)
There is no PCIe in the deep sea…
Debug via UART
Firmware reconfigurable; (new) software loaded together with (new) FPGA configuration (Block Memory Mapping file)
Explore the Kintex-7 deterministic latency GTX
Future Artix-7 may be even more cost effective in terms of money and power
Implement Oscillators, DACs (used by Soft-PLL) and SFP on FMC card which is to be plugged onto Xilinx KC705 Evaluation board (Mesfin Gebyehu)
1st goal: Replace the (slave) SPEC in the SPEC<-> SPEC test setup with the KC705 implementation for validation.
Study Shore Station (e.g. White Rabbit Switch v3) broadcast

13. To be studied: Network Broadcast
ReferenceClock
Start
PTP
Broadcast
Buffer
Buffer Tx
SFP
Buffer
Time Stamp t1
DOM
Buffer
SFP
Optical
Network
Time Stamp t4 Rx
t4 Stop1
DOM
j: DDMTD
SFP
Time Stamp t4 Rx
t4 Stop2
DOM
j: DDMTD
SFP
Time Stamp t4 Rx
t4 Stop3
j: DDMTD
DOM
SFP
Time Stamp t4 Rx
t4 Stop4
Main Electrical Optical Cable
(MEOC)
May be 100 Km long…
Shore Station interface
j: DDMTD
Can this be done with the WR Switch-V3?
Need Firmware/Software change?

14. (White Rabbit community)
Thank you!
(White Rabbit community)

15. Backup Slides

16. LM32, three boot types
Generate ROM image before synthesis (used for functional simulation debug)
Describe a generic RAM using “init” file
Useful for functional simulation
Incorporates “boot.elf” in the block-rams
Download “elf” via an external interface in a running system (used for software debug)
Useful for debugging purposes (the SPEC uses PCIe or JTAG)
No BMM=Block Memory Mapping file needed
After Place&Route (will be used in final CLB)
Merge “FPGA.bit” file and “boot.elf” file (data2mem)
This needs BMM=Block Memory Mapping file
The “bit” file which is outputted by the merge can be used as updated configuration file
This avoids synthesis each time software is updated

17. BMM (Block Memory Mapping) File
Fpga.bit
Fpga_elf.bit
ROM
ROM
RAMB36
0x00
RAMB36
0x12
RAMB36
0x00
RAMB36
0x34
RAM
RAM
RAMB36
0x00
RAMB36
0x56
RAMB36
0x00
RAMB36
0x78
Data2Mem
hello.elf
ADDRESS_SPACE lm32_memory RAMB36 [0x :0x00011FFF]
BUS_BLOCK u1_u0/U0/xst_blk_mem_generator/gnativebmg.native_blk_mem_gen/valid.cstr/ramloop[15].ram.r/v6_init.ram/NO_BMM_INFO.SP.SIMPLE_PRIM36.ram [31:30]; :
END_BUS_BLOCK;
END_ADDRESS_SPACE;
ADDRESS_SPACE lm32_data_memory RAMB36 [0x :0x00011FFF]
BUS_BLOCK
u6_u0/U0/xst_blk_mem_generator/gnativebmg.native_blk_mem_gen/valid.cstr/ramloop[15].ram.r/v6_init.ram/NO_BMM_INFO.SP.SIMPLE_PRIM36.ram [31:30];
fpga.bmm

18. Optical Network (Simplified)
Broadcast λ1 λ1
PIN
Mod λ1 λ2
REAM
λ2- λn A1
2xCu
DOM1 λ2 λ3 λ1
PIN λ2 C2
Junction Box λ2
REAM λ2 λn λ3 λ3
2xCu
DOM2 A2 λn
DU-container λn λ1
PIN
Shore station λn
REAM
Slow Control l1
2xCu
DOMn
Multiple l
Single l