Defining serial links for SuperB A. Aloisio, R. Giordano INFN and University of Naples ‘Federico II’ aloisio@na.infn.it rgiordano@na.infn.it

SuperB Workshop - Orsay, Feb.09 Overview Serial links in SuperB: not only data SuperB clock architecture and link configurations A test bench for clock jitter analysis Jitter transfer and breakdown test results Comments & Conclusions SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 DAQ/Trigger Frame Serial links in SuperB should do much more than sending data: Low-jitter clock distribution Fixed-latency timing signal transmission (both trigger and DAQ) ad-hoc real-time protocols for FEC-to-ROM-to-FCTS communications from Breton’s talk al SuperB Meeting, Elba, Giugno 08 SuperB Workshop - Orsay, Feb.09

Embedded Transceivers GTPs Xilinx Virtex5-LXT50 FPGAs offers high speed embedded Gb/s transceivers: up to 3.5 Gbits, 10 Gbit/s available by the Q4 2009 (Altera, Xilinx) Programmable parallel and line rate, coding, symbol width, latency High-level protocols, error correction schemes to be implemented in the fabric SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 Link configuration Assuming a 60 MHz master clock we can consider different parallel and line rates: Tx clock Line Rate (10x) Recovered clock prescaler Master 4x (240 MHz) → 2.4Gbit/s → 240 MHz → ÷4 → 60MHz 5x (300 MHz) → 3.0 Gbit/s → 300 MHz → ÷5 → 60MHz 6x (360 MHz) → 3.6 Gbit/s → 360 MHz → ÷6 → 60MHz 5x: odd number, duty cycle correction needed after clock prescaling 6x: FPGA close to the physical limit, serial rate would require 10Gbit/s technology, 4x: ‘comfortable’ parallel rate, standard optical technology, 4=22 -> no duty cycle correction 10b symbols are easier to handle than 20b symbols: simpler clock tree, no needs for f and 2f clocks More robust, less jitter and skew 8b10b coding is implemented in HW (but fixed latency architecture may require custom coding !) SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 Clock analysis In this talk, we focus on clock distribution, NOT data (quite an unorthodox approach !) Two different questions to answer: I have available the master clock (possibly the machine clock). So, how good is the recovered clock compared with it ? I DO NOT have the master clock. Is the recovered clock stable enough to feed high-speed digital circuits (possibly other serdes, FPGAs, PLLs, ASICs ) ? Analysis of Timing Interval Errors (TIE) accumulated between edges of Master and derived clock is a well known, robust technique (a.k.a. TIE edge-to-edge) to answer the first question When a Master Clock is not available, many equipment vendors (Agilent, LeCroy, Tek) provide ‘ideal’ references (software PLLs, HW clock recovery, tracking algorithms, …) What ‘ideal’ means is poorly specified and even worse documented (patent pending, secrets of the trade, …). SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 TX and RX clocks LVDS RX high-speed diff. MUX RX recovered clock (240 MHz) TX TX clock (60 MHz) SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 Test bench Agilent 33250A 80 MHz function generator Sin, rand. & arb. Jitter source Tektronix DTG5334 - 3.75 Gbit/s data generator - delay resolution: 0.2 ps - built-in and external jitter inputs Xilinx ML505 boards - Virtex5 50LXT - 2x 3.5 Gbit/s GTP (Rx+Tx) - Built-in and ext. clocks via SMA Physical Layer: 2x 1m SMA cables (lab grade) LeCroy SDA6020A: - 6 GHz bandwidth - 1ps RMS jitter floor - ASDA-J jitter analysis SuperB Workshop - Orsay, Feb.09

Case 1: Master Clock avail, very low jitter Best case: Ideal Master Clock, no external jitter very low Rj 60 MHz TX clk TX clock: 60 MHz Random jitter (Rj) s = 5.8 ps Pk-Pk jitter: 54 ps TX RX RX clock (recovered): 240 MHz Random jitter (Rj) s = 22 ps Pk-Pk jitter: 220 ps 240 MHz RX clk (recovered) 240 MHz RX clk (recovered) Jitter extrapolation @ BER 10-12 Tj = 260 ps, s = 18 ps Rj = 18 ps Dj = 2 ps SuperB Workshop - Orsay, Feb.09

Case 2: Master Clock avail, Deterministic Jitter Jitter on the Master clock is transferred across the link to the recovered clock Transfer is function of the jitter spectrum The TX clock is modulated by sinusoidal jitter (200ps) at different frequencies The transfer function can be measured at a given frequency by taking the ratio of the jitter amplitude on the Master and recovered clocks SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 Case 2: Jitter transfer 2 MHz 10 MHz 20 MHz 1.5 MHz 1 MHz 100 KHz With a Master clock, the jitter transfer can be fully characterized The PLL has a ~ 2.5 MHz low pass bandwidth The higher the bandwidth, the more tolerant the link is (good for data) … … but the more the recovered clock tracks the jitter (bad for the clock) SuperB Workshop - Orsay, Feb.09

Case 2: Jitter Breakdown Random jitter is not filtered effectively by the PLL: it may survive to jitter cleaners Special care has to be taken with very low frequency jitter sources (< 10 MHz) The Tj floor is: 270 ps (pk-pk) s = 18ps Total Jitter (pk-pk) Deterministic Jitter SuperB Workshop - Orsay, Feb.09

Case 3: Master clock not avail 200ps sinusoidal Jitter @ 1 MHz, as before, but this time we do not use the master clock Instead, an ‘ideal’ clock is generated by the scope. Its edges are used as reference The ‘ideal’ clock tracks the slow jitter and hides the phase noise Tj = 180ps (s=13ps), Dj = 10ps (quite understimated …) Sinusoidal Jitter can be revealed by taking the TIE far from where the ideal clock is estimated Tj = 700ps Dj = 350ps (quite overstimated …) SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 A few comments… Our work has addressed independently many of the guidelines proposed by Dominique and Umberto: Our setup assumes a Master clock at 60MHz. The FPGA multiplies it by 4, then the link by 10, arriving at a line rate of 2.4 Gbit/s Cleaners typically fail to remove random jitter. Good for spectrum limited deterministic jitters, below 10 MHz. SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 … and a bit more Our measures show that the Total Jitter floor could be compatible with 15ps RMS @ 10 MHz and above. At lower jitter frequencies, it could be much higher. Note that data rate and jitter are not easily related. Xilinx claims better jitter figures can be achieved at higher data rate (to be tested). Well, … it looks like we started already… A closer collaboration is very welcomed in future SuperB Workshop - Orsay, Feb.09

SuperB Workshop - Orsay, Feb.09 Conclusions We configured a serial link platform compatible with a basic SuperB clock architecture, aiming at clock distribution (sending data is easier) Master clock @ 60MHz Parallel clock @ 240 MHz Line rate @ 2.4 Gbit/s 8-bit payload, 8b10b coding Tj of the recovered clock is mainly random above 10 MHz, 270ps (pk-pk) amplitude, 18ps standard deviation Low frequency deterministic jitter sources (< 10 MHz) show up in the recovered clock and should be avoided by design, or cleaned. Changing FPGA/serdes changes the frequency threshold, does not get rid of the problem. Results DO change with different algorithms: we need to agree on the technique used to measure jitter Many things still to do for the TDR: try higher rates, newer FPGAs, take into account the fibers, measure the BER, disentangle the RX contribution, test jitter cleaners, … SuperB Workshop - Orsay, Feb.09