1. Defining serial links for SuperB
A. Aloisio, R. Giordano
INFN and University of Naples ‘Federico II’

2. SuperB Workshop - Orsay, Feb.09
Overview
Serial links in SuperB: not only data
SuperB clock architecture and link configurations
A test bench for clock jitter analysis
Jitter transfer and breakdown test results
Comments & Conclusions
SuperB Workshop - Orsay, Feb.09

3. SuperB Workshop - Orsay, Feb.09
DAQ/Trigger Frame
Serial links in SuperB should do much more than sending data:
Low-jitter clock distribution
Fixed-latency timing signal transmission (both trigger and DAQ)
ad-hoc real-time protocols for FEC-to-ROM-to-FCTS communications
from Breton’s talk al SuperB Meeting, Elba, Giugno 08
SuperB Workshop - Orsay, Feb.09

4. Embedded Transceivers
GTPs
Xilinx Virtex5-LXT50
FPGAs offers high speed embedded Gb/s transceivers: up to 3.5 Gbits, 10 Gbit/s available by the Q (Altera, Xilinx)
Programmable parallel and line rate, coding, symbol width, latency
High-level protocols, error correction schemes to be implemented in the fabric
SuperB Workshop - Orsay, Feb.09

5. SuperB Workshop - Orsay, Feb.09
Link configuration
Assuming a 60 MHz master clock we can consider different parallel and line rates:
Tx clock Line Rate (10x) Recovered clock prescaler Master
4x (240 MHz) → 2.4Gbit/s → 240 MHz → ÷4 → 60MHz
5x (300 MHz) → 3.0 Gbit/s → 300 MHz → ÷ → 60MHz
6x (360 MHz) → 3.6 Gbit/s → 360 MHz → ÷ → 60MHz
5x: odd number, duty cycle correction needed after clock prescaling
6x: FPGA close to the physical limit, serial rate would require 10Gbit/s technology,
4x: ‘comfortable’ parallel rate, standard optical technology, 4=22 -> no duty cycle correction
10b symbols are easier to handle than 20b symbols:
simpler clock tree, no needs for f and 2f clocks
More robust, less jitter and skew
8b10b coding is implemented in HW (but fixed latency architecture may require custom coding !)
SuperB Workshop - Orsay, Feb.09

6. SuperB Workshop - Orsay, Feb.09
Clock analysis
In this talk, we focus on clock distribution, NOT data (quite an unorthodox approach !)
Two different questions to answer:
I have available the master clock (possibly the machine clock). So, how good is the recovered clock compared with it ?
I DO NOT have the master clock. Is the recovered clock stable enough to feed high-speed digital circuits (possibly other serdes, FPGAs, PLLs, ASICs ) ?
Analysis of Timing Interval Errors (TIE) accumulated between edges of Master and derived clock is a well known, robust technique (a.k.a. TIE edge-to-edge) to answer the first question
When a Master Clock is not available, many equipment vendors (Agilent, LeCroy, Tek) provide ‘ideal’ references (software PLLs, HW clock recovery, tracking algorithms, …)
What ‘ideal’ means is poorly specified and even worse documented (patent pending, secrets of the trade, …).
SuperB Workshop - Orsay, Feb.09

7. SuperB Workshop - Orsay, Feb.09
TX and RX clocks
LVDS RX
high-speed diff. MUX
RX recovered clock (240 MHz) TX
TX clock (60 MHz)
SuperB Workshop - Orsay, Feb.09

8. SuperB Workshop - Orsay, Feb.09
Test bench
Agilent 33250A
80 MHz function generator
Sin, rand. & arb. Jitter
source
Tektronix DTG5334
Gbit/s data generator
- delay resolution: 0.2 ps
- built-in and external jitter inputs
Xilinx ML505 boards
- Virtex5 50LXT
- 2x 3.5 Gbit/s GTP (Rx+Tx)
- Built-in and ext. clocks via SMA
Physical Layer:
2x 1m SMA cables
(lab grade)
LeCroy SDA6020A:
- 6 GHz bandwidth
- 1ps RMS jitter floor
- ASDA-J jitter analysis
SuperB Workshop - Orsay, Feb.09

9. Case 1: Master Clock avail, very low jitter
Best case:
Ideal Master Clock, no external jitter
very low Rj
60 MHz TX clk
TX clock:
60 MHz
Random jitter (Rj)
s = 5.8 ps
Pk-Pk jitter: 54 ps TX RX
RX clock (recovered):
240 MHz
Random jitter (Rj)
s = 22 ps
Pk-Pk jitter: 220 ps
240 MHz RX clk
(recovered)
240 MHz RX clk
(recovered)
Jitter BER 10-12
Tj = 260 ps, s = 18 ps
Rj = 18 ps
Dj = 2 ps
SuperB Workshop - Orsay, Feb.09

10. Case 2: Master Clock avail, Deterministic Jitter
Jitter on the Master clock is transferred across the link to the recovered clock
Transfer is function of the jitter spectrum
The TX clock is modulated by sinusoidal jitter (200ps) at different frequencies
The transfer function can be measured at a given frequency by taking the ratio of the jitter amplitude on the Master and recovered clocks
SuperB Workshop - Orsay, Feb.09

11. SuperB Workshop - Orsay, Feb.09
Case 2: Jitter transfer
2 MHz
10 MHz
20 MHz
1.5 MHz
1 MHz
100 KHz
With a Master clock, the jitter transfer can be fully characterized
The PLL has a ~ 2.5 MHz low pass bandwidth
The higher the bandwidth, the more tolerant the link is (good for data) …
… but the more the recovered clock tracks the jitter (bad for the clock)
SuperB Workshop - Orsay, Feb.09

12. Case 2: Jitter Breakdown
Random jitter is not filtered effectively by the PLL: it may survive to jitter cleaners
Special care has to be taken with very low frequency jitter sources (< 10 MHz)
The Tj floor is:
270 ps (pk-pk)
s = 18ps
Total Jitter (pk-pk)
Deterministic Jitter
SuperB Workshop - Orsay, Feb.09

13. Case 3: Master clock not avail
200ps sinusoidal 1 MHz, as before, but this time we do not use the master clock
Instead, an ‘ideal’ clock is generated by the scope. Its edges are used as reference
The ‘ideal’ clock tracks the slow jitter and hides the phase noise
Tj = 180ps (s=13ps), Dj = 10ps (quite understimated …)
Sinusoidal Jitter can be revealed by taking the TIE far from where the ideal clock is estimated
Tj = 700ps Dj = 350ps (quite overstimated …)
SuperB Workshop - Orsay, Feb.09

14. SuperB Workshop - Orsay, Feb.09
A few comments…
Our work has addressed independently many of the guidelines proposed by Dominique and Umberto:
Our setup assumes a Master clock at 60MHz. The FPGA multiplies it by 4, then the link by 10, arriving at a line rate of 2.4 Gbit/s
Cleaners typically fail to remove random jitter. Good for spectrum limited deterministic jitters, below 10 MHz.
SuperB Workshop - Orsay, Feb.09

15. SuperB Workshop - Orsay, Feb.09
… and a bit more
Our measures show that the Total Jitter floor could be compatible with 15ps 10 MHz and above. At lower jitter frequencies, it could be much higher. Note that data rate and jitter are not easily related. Xilinx claims better jitter figures can be achieved at higher data rate (to be tested).
Well, … it looks like we started already…
A closer collaboration is very welcomed in future
SuperB Workshop - Orsay, Feb.09

16. SuperB Workshop - Orsay, Feb.09
Conclusions
We configured a serial link platform compatible with a basic SuperB clock architecture, aiming at clock distribution (sending data is easier)
Master 60MHz
Parallel 240 MHz
Line 2.4 Gbit/s
8-bit payload, 8b10b coding
Tj of the recovered clock is mainly random above 10 MHz, 270ps (pk-pk) amplitude, 18ps standard deviation
Low frequency deterministic jitter sources (< 10 MHz) show up in the recovered clock and should be avoided by design, or cleaned. Changing FPGA/serdes changes the frequency threshold, does not get rid of the problem.
Results DO change with different algorithms: we need to agree on the technique used to measure jitter
Many things still to do for the TDR: try higher rates, newer FPGAs, take into account the fibers, measure the BER, disentangle the RX contribution, test jitter cleaners, …
SuperB Workshop - Orsay, Feb.09