July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology KM3NeT CLBv2 1

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Established connection between SPEC (WR-Master) and KC705 (WR-Slave) WR on KC705 2 Data Unit LayerStandard Host Layers Data7. Application 6. Presentation 5. Session Segment4. TransportRFC-768UDP Media Layers Packet3. NetworkRFC-791IPv4 Frame2. Data LinkIEEE 802.3MAC Bit1. PhysicalPHY OKAY! Link up! => Physical Layer okay (already reported June 19) FMC VCXO (SoftPLL) Locked! New! What does Calibrate mean? Communication! => PTP packets are exchanged between SPEC (Master) and KC705 (Slave) New! Timing servo lock still needs attention (phase tracking is ”on” but “Phase set point” stays 0) ????

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology KC705 WR Ethernet port Block diagram 3 Ethernet traffic: PTP frames Ethernet MAC

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Visual Status 4 Rx_mac2buf I2C Fifo 31 TDCs TDC 0 Management & Control Data Control Wishbone bus RxPacket Buffer 64KB IP/UDP Packet Buffer Stream Selector (IPMUX) Rx_buf2data RxPort 1 RxPort 2 RxPort_m Management & Config. Tx_pkt2mac Tx_data2buf TxPort 1 TxPort 2 TxPort_m Flags Rx Stream Select TxPacket Buffer 32KB Flags Tx Stream Select 31 PMTs Pause Frame ADC Management & Control Hydrophone Fifo TDC 30 Fifo Nano Beacon GPIO Debug LEDs I2C Debug RS232 Temp Compass Tilt Point to Point interconnection Xilinx Kintex-7 Start Time Slice UTC & Offset counter since Time Slice Start MEM S 2 nd CPU LM32 M M WB Crossbar (1x7) WB Crossbar (3x2) S M S M M S S M M M SS S UART S M M S S M M State Machine SPI S M Flash Focus on user interface WRPC IPMUX UTC time & Clock (PPS+RefClk)

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology TX/RX packets active! ◦ Problem was due to improper master/slave configuration. ◦ Ethernet packets (PTP) are now happily exchanged! Note: User interface still needs to be connected… Timing Servo does not lock => PPS not enabled PHY RXCDRLOCK_OUT lost in simulation ◦ Understood!  This was a non-issue.  Due to a de-asserted RXBYTEISALIGNED_OUT caused by misaligned IDLE ( ) character generated by the frame generator in the simulation. Summary WR status 5

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Done: ◦ LM32 + WB-Crossbar + DPRAM + UART ◦ Soft-PLL FMC layout ◦ WR without PCI-express ◦ Deterministic PHY ◦ 1-wire Currently: ◦ Soft PLL (hardware + software). Still under study… ◦ Endpoint (= MAC) <= under investigation, focus on user interface ◦ Fabric redirector <= focus on user interface To do (in order of priority): ◦ Mini-nic <= Complex, but seems to work (PTP flows) ◦ PPS generator <= relatively straightforward ◦ SysCon <= easy? Status Listing 6

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Ordered Minimum Order Quantity of 162 Available if anyone needs one (Valencia for testing TDCs for example) Expected delivery: mid-September GBX (Octopus) connectors 7

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology 1. Switch Routing Table (software?) needs to be adjusted. 2. PTP timestamps t 1 for all DOMs are equal and reside in outgoing port (needs firmware- or software-change or both) 3. MAC Control-Level multicast MAC addresses; such as “pause frames” for flow control need to be handled correctly. Example: ◦ “DOM-B” request “Pause” = Okay (request over point to point link), but… ◦ “Shore station” request “Pause” may be problematic (request over broadcast link, all ports are stalled). Address single DOM? 4. Other surprises? Shore Station Broadcast brainstorm DOM A DOM B DOM C DOM D Buffer Port-2 SFP Broadcast Optical Network Start Tx t 4 Stop1 Reference Clock PTP Time Stamp t 1 Time Stamp t 4 t 4 Stop2 Time Stamp t 4 t 4 Stop3 Time Stamp t 4 t 4 Stop4 Time Stamp t 4 Shore Station interface 8 Rx  : DDMTD Rx  : DDMTD Rx  : DDMTD Rx  : DDMTD Port-1 SFP Main Electrical Optical Cable Port-3 SFP Port-4 SFP Port-5 SFP

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology Other surprises: PTP t 1,t 4 timestamps over multiple switches 9 PTP t 1 Broadcast PTP t 4 DOM inputs PTP t 4 DOM inputs PTP t 4 DOM inputs 1. Redistribution of PTP t 1 not only within one switch… 2. Routing PTP (layer-2!; no IPv4) frames is an issue… (note that t 4 needs to be routed to the appropriate DOM via the broadcast channel) …over multiple switches (depending on detector broadcast network segmentation) This needs brainpower! One t 1 Many t 4

July 31, 2013KM3NeT, CLBv2 Meeting Peter Jansweijer Mesfin Gebyehu Nikhef Amsterdam Electronics- Technology LM32_2 nd Power Estimator 10 LM32_2nd system consumes 156 mW plus 127 mW quiscent current (total 283 mW) If we forsee that the complete CLBv2 system adds up to 6-10 times the complexity of a single lm32_2nd system then we calcutale 10 * 156mW + 127mW = 1.56 W for the FPGA This estimation doesn’t contain a GTX Expected FPGA power should be < 3 W This is still a very rough estimation!