1 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results

2 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Summary DU communication strategy Hybrid approach –Copper Link requirements –Link implementation –Complete Copper Node block diagram –Reduced Copper Node block diagram –Mezzanine design First results of the prototype board Conclusions

3 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results DU communication strategy all-optical solution ‣ long haul: fiber ‣ DU backbone: fiber hybrid solution ‣ long haul: fiber ‣ DU backbone: copper ‣ The DU connection to on-shore must be optical: ‣ distance is about 100 km ‣ aggregate data rate from floors is high (~Gb/s) ‣ The DU backbone can be either optical or electrical ‣ link are tens or hundreds of meters long ‣ data rate can be as small as 100 Mb/s

4 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Copper Link requirements Constraints: applicable to both Twisted Pairs or Coax cables length of a single hop (maximum 50m) data rate of copper chain (~1.25 Gb/s) Copper Node Features: the chain is synchronous auto-identification of nodes payload can be dynamically allocated the node is designed as a plug-in module each node is reprogrammable node power consumption is very low (~2W)

5 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results The link is bidirectional with asymmetric data rates: ๏ Up-going Mb/s for timing and slow control ๏ Down-going Gb/s for physics data and control Link Implementation: Double Daisy Chain Scheme Pros Timing from Low Speed Nodes are identical Cons Higher Power Failure stops higher floors

6 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Complete Copper Node Block Diagram pluggable mezzanine board stand-alone for debug SerDes-like interface user transparent for PHY management reprogrammable on-the-fly dynamic allocation of payload

7 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results cNodeV1: Copper Node prototype board Only the slow “Master” chain is implemented: –PLL clock recovering and cleaning (expected p-p jitter: 105ps); –clock multiplication for high speed chain; –slow Rx and Tx data channel (stand-alone or FPGA driven); –SMA connectors for line interface: external Driver and Equalizer set allows different cables; impedance matching boards can be inserted ;

8 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Mezzanine board form factor The chosen form-factor is the Nemo mezzanine Possible testing boards: –directly pluggable into Nemo FCM (Spartan3 device) –in-directly pluggable into Xilinx ML50x series (V5)

9 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results The cNodeV1 board plugged into an FCM

10 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results The cNodeV1 plugged into the ML50x board with the adapter

11 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results First results: 2 CPPM The test cable is a 10 m long CAT-5 TP (hop 1-> 2); The system has been tested with a “BER analyzer” –no errors detected during the test (too short...) An embedded processor controls the system and the BER module Test of clock and data integrity on slow chain; –single or multi-hop test on daisy-chained boards; The clock Total Jitter after 1st and 2nd hop is ~150ps

12 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Measured jitter after CDR Receiver

13 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Measured jitter after PLL

14 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Eye diagram before equalizer

15 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Eye diagram after equalizer

16 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Preliminary results: 2 Naples Different test cables –two coax cables 20 cm long; –2 m long CAT5; –30 m long CAT5e; –300 m long CAT5e. System tested with a “BER analyzer” on board –no errors detected during the test (too short...) Embedded processor to control the whole system Test of clock and data integrity on slow chain; –single or multi-hop test on daisy-chained boards;

17 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Preliminary results: single hop 2 coax cables: –CDR clock TIE sdev: 450 ps –PLL clock TIE sdev: 14 ps 2 m CAT5 cable: –CDR clock TIE sdev: 500 ps –PLL clock TIE sdev: 15 ps 30 m CAT5e cable: –CDR clock TIE sdev: 550 ps –PLL clock TIE sdev: 18 ps

18 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Preliminary results: double hop 300 m CAT5e cable: –PLL2 clock TIE sdev: 35 ps –PLL1 clock TIE sdev: 18 ps –Period: ns (sdev ~ 6ps)

19 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Preliminary results: line analysis by SDA 300 m CAT5e cable: –PLL2 clock TIE sdev: 35 ps –PLL1 clock TIE sdev: 19 ps

20 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Conclusions Preliminary results are really encouraging –200 Mb/s link seams feasible on long distances. –This link shows a low BER –Clock can be extracted from the data and after PLL the jitter is reduced to few ten of picoseconds Other tests are required; –Much more hops (10) –Cable impedance at different pressure condition must be studied A complete version of mezzanine (low speed and high speed daisy chain) must be designed and tested

21 S. Russo, F. Ameli KM3Net WP3-WP5 Joint Meeting :: Paris :: February 2009 Copper backbone data transmission system: first results Concluding Remarks Multiple Backbones provide redundancy but also an increase in power, cost, and complexity: –Reducing the number of PMs per backbone reduces the overall rate increasing hop length: single node failure could be sectioned out! –One fiber per DU is required; groups of DU can be mux-demux by means of DWDM. –Even though the number of backbones can be easily increased, the total “cost” could result higher than with a single optical backbone... DU JB contains electronics to bridge copper and fiber; –two 1.2Gb/s channel can produce a single 2.5Gb/s stream which drives a color into a fiber: 1 color per backbone! An independent communication line over TP cable is dedicated to power management, backbone sectioning, and slow control function.