1. Multi-Gigabit transmission BLT  GOLD Andreas Ebling, Isabel Koltermann, Jonas Kunze Andi Ebling 1

2. GOLD real-time data path and control GOLD will be operated in conjunction with the BLT, acting as a data source and control module GOLD residing in ATCA crate Currently no crate CPU (nor crate...) available Optical link to BLT (SFP) allows for module control Optical receivers (12-channel) on RTDP BLT residing in CMM slot of JEP crate Controlled by VME-- Optical control link extends VME functionality to GOLD Optical transmitter (SNAP12, up to 6.5 Gb/s) to be used as a data source GOLD is Virtex-6 based / BLT is Virtex-5 based Andi Ebling 2

3. BLT  GOLD Andi Ebling 3 BLT SFP VME GOLD SFP Control SNAP 12 AVAGO ATCA RTDP

4. Module status 1 BLT available 1 SFP link, tested up to 6.4Gb/s, ok 1 SNAP12 link , requires minor h/w mods 2 SNAP12 opto module pairs ordered in February, not yet arrived Internal clock suitable for multiples of 160 MHz GOLD not expected in very near future  use Xilinx development board as a GOLD emulator 1 ML605 available 1 SFP link Daughter module might carry additional link hardware Internal clock suitable for multiples of 100 or 125 MHz Andi Ebling 4

5. Transceiver modes Control path will probably be asynchronous eventually Requires buffering the received data where crossing clock domains Control stream will contain fill characters (comma) which can be taken out or added by clock correction circuitry so as to cope with slightly differing rates at source and sink RTDP will probably benefit from synchronous transmission Requires cleaned global clock (LHC bunch clock) Received data are coming in at correct rate, but unknown phase offset Requires phase alignment Xilinx recipe to align phase to recovered clock (within receiver) Leads to separate clock domains for each data channel Requires yet another phase alignment to global clock in the fabric Try to align to one global clock per FPGA right away Andi Ebling 5

6. Phase Alignment Andi Ebling 6

7. Delay in control mode BLT in loopback with 10 m (~ 50 ns) fibre BLT to GOLD and back to BLT (with 2 * 10 m fibre) Gray: extrapolated values Andi Ebling 7 RateDelay (Ticks)Delay (ns)Delay (without fibre) 2 Gb/s34 @ 100 MHz340290 4 Gb/s40 @ 200 MHz200150 6.4 Gb/s46 @ 320 MHz147.297.2 RateDelay (Ticks)Delay (ns)Delay (without fibre) 2 Gb/s74 @ 100 MHz740640 4 Gb/s85 @ 200 MHz425325 6.4 Gb/s97 @ 320 MHz303203

8. Delay in RTDP mode BLT in loopback with 10 m (~ 50 ns) fibre  BLT to GOLD and back to BLT (with 2 * 10 m fibre)  Using 4-Byte-interface Andi Ebling 8 RateDelay (Ticks)Delay (ns)Delay (without Cable)LHC Bunch Ticks 2 Gb/s24 @ 100 MHz2401908 4 Gb/s29 @ 200 MHz145954 6.4 Gb/s35 @ 320 MHz109593 6.4 Gb/s19 @ 160 MHz119693 RateDelay (Ticks)Delay (ns)Delay (without Cable)LHC Bunch Ticks 2 Gb/s49 @100 MHz49039016 4 Gb/s59 @ 200 MHz2951958 6.4 Gb/s71 @ 320 MHz2221225

9. Summary and Future 2 Byte mode at 6.4 Gb/s ->fabric frequency: 320 MHz 4 Byte mode at 6.4 Gb/s ->fabric frequency: 160 MHz Easier to handle, but latency + 10 ns Both modes need 3 LHC Bunch Ticks BLT-loopback-delay should be equal to delay from BLT to GOLD on RTDP Expected delay : 3 LHC Bunch Ticks in GTX + 2 LHC Bunch ticks on fibre (10 m) Future: testing SNAP 12 on BLT loopback (when arrived) testing AVAGO on RTDP (when GOLD finished) Andi Ebling 9