1. JESD204B Training Achieving Deterministic Latency
May 2016

2. Abstract
JESD204B links are the latest trend in data-converter digital interfaces. These links take advantage of high speed serdes technology to offer many compelling benefits including improved channel densities and simplified board layouts. JESD204B expands upon existing JESD204 standards by including methods for achieving deterministic latency. This presentation addresses one of the challenges of adopting the new interface - understanding and designing the link latency.
The concepts and examples presented will enable a system designer to:
Understand the tradeoff between link latency and tolerance to link delay variation
Guarantee deterministic latency across the JESD204B link
Use a formulaic and procedure-based approach to design the link latency

3. Table of Contents Latency in a JESD204B link
Define Deterministic Latency
Guaranteeing Deterministic Latency
Steps to Achieve Deterministic Latency
Verifying Deterministic Latency
Bonus!

4. Latency in a JESD204B Link

5. What is Latency?
Latency is defined as the total time (in seconds or clock cycles) it takes a signal to travel from Point A to Point B
For Example:
Point A might be the input to the ADC
Point B might be the output of the DAC
In a JESD204B link:
Point A is the input to the JESD204B transmitter
Point B is the output of the JESD204B receiver’s elastic buffer

6. Latency Definitions
Link Latency — the latency from when the sampled parallel data is input to the serializer at the transmitter (ADC/FPGA) until the same data is available in parallel form at the output of the elastic buffer in the receiver (FPGA/DAC)
Here we’ve defined the “Link Latency” which is the input to the JESD204B transmitter to the output of the JESD204B receiver. You can see that part of the link latency is the link delay.

7. Latency Definitions
Link Delay — delay from when the sampled parallel data is input to the serializer at the transmitter (ADC/FGPA) until the same data is presented at the input to the elastic buffer in the receiver (FPGA/DAC)
We can define the link delay as the latency from the input to the JESD204B transmitter (shown here as the serializer) to the output of the JESD204B link layer (shown here as the deserializer).

8. Defining Deterministic Latency

9. Deterministic Latency (DL) Definition
Wikipedia: ”A deterministic system is a system in which no randomness is involved in the development of future states of the system. A deterministic model will thus always produce the same output from a given starting condition or initial state.”
 The change in the state is known and constant (repeatable across trials)
JESD204B Standard (abbr.): Latency from the frame-based data input at the TX to the frame-based data output at the RX. Latency should be programmable and repeatable over power cycles and re-sync events provided timing requirements are met.
Knowing the latency is usually not as important as it being constant
A deterministic system is defined as being a system in which no randomness is involved in the future states of the system. A deterministic model will thus always produce the same output from a given starting condition or initial state.
So, what is the starting condition in a JESD204B deterministic system? The LMFC phase alignment in each device
What is the “same output” in a JESD204B system? The latency
So we create a starting condition using SYSREF to align the LMFC clocks in each device, such that every time the system starts the LMFC’s start in the same state. We’re then able to create the same output, or latency, across the link.
So the most important aspect of deterministic latency is that the latency should stay constant from system startup to startup.
Note that having deterministic latency does not mean that you necessarily know the latency. In many cases, the total latency isn’t a concern so much as having matched latency among multiple devices.

10. Which one is Deterministic Latency referring to?
So, for a JESD204B link, which latency is deterministic latency referring to? From a purely JESD204B standpoint it would be the link latency (note the difference between link latency and link delay). A system designer may also refer to the total system latency as being deterministic.

11. Systems that Require DL
Applications sensitive to latency
Digital pre-distortion (DPD) control loops
Automatic gain control loops (AGC)
Defensive counter measures
Any system that requires multi-device synchronization
Multi-antenna communications systems
Phased array radar
Magnetic Resonance Imaging (MRI)
Now that we know what deterministic latency, which systems may actually need it? Of course any application that is sensitive to latency variations such as control loops for digital predistortion or automatic gain control. Or total latency such as defensive counter measures. And any system that requires multi-device synchronization will also need deterministic latency to guarantee that all devices send or receive the same data at the same time.

12. Example: Multi-Device Synchronization
Data is sent over serdes lanes
The device clock captures SYSREF so setup and hold must be met
Each device receives a “device clock”
Each device receives a “SYSREF” signal

13. JESD204B Features that Enable DL
(Time Reference) Low freq. time reference local to each device, synchronized deterministically to other devices in system
Local Multi-Frame Clock (LMFC)
(Alignment Detection) TXRX initialization pattern with embedded TX time reference is used by RX to detect reference alignment
Initial Lane Alignment (ILA) Sequence
(Delay Adjustment) Adjustable delay compensation at RX
RX Elastic Buffer
Which block really enables us to achieve deterministic latency from startup to startup and over operating conditions? The elastic buffer does because it is able to absorb any variations in delay between lanes and then output the data at a known time.

14. Link Initialization Sequence to Achieve DL
Time synchronization signals are sent to all devices to align LMFCs
All TX’s send ILA sequence aligned to TX LMFC
RX analyzes data alignment and adjusts elastic buffer for each lane independently to align data of all lanes
So how does JESD204B achieve deterministic latency? There’s a set sequence of events that must occur. The first step is that SYSREF is sent to all devices at the same point in time to align their internal LMFCs. Then, the transmitters will issue a /A/ character at the same point in time based on the LMFC edge. The lanes may become skewed as the data travels from the transmitter to the receiver. As each receiver receives it’s /A/ character it will begin to buffer data, essentially absorbing the latency variation. At a predetermined point in time, all receivers will release the data from the elastic buffers at the same time, thereby aligning the data.

15. Time Synchronization Signals
JESD204B defines 3 “subclasses”, 2 of which support DL
2 mechanisms to synchronize the local time references (LMFC)
Subclass
Mechanism for Deterministic Latency
Subclass 0
No support for deterministic latency (backward compatible with JESD204A).
Subclass 1
“SYSREF” signal is used to align LMFCs within all TX and RX devices.
Subclass 2
“SYNC” signal is used to align LMFCs, eliminating the need for “SYSREF”. LMFC is aligned based on when the SYNC signal is received from each data converter device.
Complex timing means that this will only work below 500 MSPS.
Here are the three subclasses for JESD204B. The subclasses simply define different mechanisms for achieving deterministic latency. Subclass 0 is the simplest method, in that it doesn’t allow you to achieve deterministic latency. Subclass 1 defines the use of a SYSREF signal used to align a reference clock within each device. Subclass 2 defines strict timing on a handshake signal that is used to align reference clocks in each device, but the strict timing limits the maximum sampling rate. Most devices are focused on subclass 1 because it is the easiest to implement and will work at higher sampling rates. We’ll focus on Subclass 1 throughout the rest of the presentation.

16. Initial Lane Alignment (ILA) Sequence
Transmitters send ILA on all lanes starting on an LMFC “boundary”
The ILA contains /A/ and /R/ characters that indicate the multi-frame boundaries
/A/ and /R/ are the alignment markers used to determine the data and time alignment and adjust the elastic buffer depth
I also want to briefly discuss the initial lane alignment sequence. This sequence is sent from all transmitters at the same time (based on the internal LMFCs) and contains alignment characters. The receivers will use these alignment characters to trigger the start of valid data.

17. Delay Compensation with Elastic Buffer
So, for a JESD204B link, which latency is deterministic latency referring to? From a purely JESD204B standpoint it would be the link latency (note the difference between link latency and link delay). A system designer may also refer to the total system latency as being deterministic.

18. Deterministic Latency Visual
1. SYSREF is used to align LMFCs
2. After SYNC is received, all lanes send ILA sequence on next LMFC boundary
3. Each lane starts buffering it’s data when it receives the R character
4. All lanes are released on the next LMFC edge
(or earlier if
RBD < K)
and are now aligned
Here’s a visual of the process. Let’s see if you can walk me through it.

19. Guaranteeing Deterministic Latency

20. Requirements to Guarantee DL?
The LMFCs in each device must have deterministic alignment every time the system starts or is re-synchronized
Data on all lanes must arrive before the RX data release point occurs
Max total link delay < LMFC period (TLMFC)
RBD setting must set release point to a time after all lanes have arrived
If the “R” character is not received by all lanes when RBD occurs, the RX data release will not occur until the next LMFC edge

21. 1. Align the LMFCs in Each Device
The LMFCs must be aligned in each device
In reality, the LMFCs are usually misaligned between TX and RX devices but the phase difference must be deterministic
SYSREF must meet setup and hold times relative to device clocks (Subclass 1)
SYNC~ must meet setup and hold times relative to device clocks (Subclass 2)

22. 2. Set Release Point to Occur After the Latest Arriving Lane
The standard requires: max(link delay) < TLMFC
Example: Serial Rate (SR) = 10 Gbps, F = 2, K = 32. What is LMFC period?
TLMFC = 10*F*K/SR = 10*2*32/10e9 = 64ns (32 frame clock cycles)
(10 is for 8b/10b encoding)
The default release point is on the next LMFC boundary (RBD = K)
RBD must be between 1 and K
1 < K < 32
17 < K*F < 1024

23. 2. Set Release Point to Occur After the Latest Arriving Lane (cont.)
The elastic buffer release point can be shifted from the LMFC rising edge by using the RBD parameter
RBD is defined as a shift in the elastic buffer release point from the LMFC boundary by “RBD” frame periods
RBD = K corresponds to the release point at the next RX LMFC boundary
RBD = K-4 shifts the release point to 4 frame cycles before the next RX LMFC boundary
Modifying RBD can be used to:
Release the buffer earlier to achieve minimum latency
Shift the release point away from a time window of link delay uncertainty

24. Example: Release Point Too Soon
The LMFC period is 50 ns
The link delay is 100 ns +/- 10 ns (PVT)
Min link delay = 90 ns
Max link delay = 110 ns
Assume the buffer release point is set to the LMFC boundary
(RBD = K)

25. Example: Release Point Too Soon
Assume MIN link delay
Buffer releases on LMFC rising edge
50 ns
Data delay is 90 ns
Total latency is 100 ns

26. Example: Release Point Too Soon
Assume MAX link delay
Buffer releases on next LMFC rising edge
50 ns
Data delay is 110 ns
Total latency is 150 ns

27. Example: Release Point Too Soon
In this example, the latency IS NOT deterministic because it is not consistent from startup to startup
How do we avoid this scenario?
Calculate the expected maximum lane delay
Choose a release point that occurs after the maximum lane delay

28. Using RBD to shift the release point
Release Point occurs “RBD” frame cycles after the LMFC boundary
Set RBD  K
50 ns
RBD
Data delay is 110 ns
Total latency is 125 ns for all cases

29. Achieving Deterministic Latency

30. 3-Step Process for Deterministic Latency
Deterministic latency can be achieved using this three step process:
Determine Alignment of LMFCs
Calculate expected link delay (with variation)
Choose release point that provides margin against error

31. Step 1: Determine Alignment of LMFCs
Things to consider:
Propagation delay of SYSREF and device clocks across board
Purposefully added delays to SYSREF, such as analog delays or dynamic digital delays (like in LMK04828)
Delays from SYSREF input pins to resetting of LMFC (internal to devices)

32. LMFC Alignment Between Devices

33. DAC38J84 SYSREF to LMFC Latency

34. Step 2: Calculate Link Delay
Variations occur due to device variances (specified in datasheet) or due to PVT effects
These delays will come from the datasheet or need to be measured or calculated
Total Link Delay = tTX_SER + tLANE + tRX_DESER +/- tVARIANCE
LMFC alignment between TX and RX has already been determined

35. Step 3: Choose Buffer Release Point
Choose release point that guarantees enough margin around the expected lane arrival

36. Summary of Calculation
We have done the following:
Determined LMFC alignment between TX and RX
Calculated total link delay
Determined the appropriate buffer release point 2 3 1

37. Total Link Latency
The total link latency can be calculated using the following formula
“n” is the number of whole RX multi-frames traversed for the case where the total link delay exceeds one LMFC cycle
Link Latency = (n * K + RBD) * TFRAME + (tRX_LMFC – tTX_LMFC)
n = 2 1 2

38. Add ADC or DAC latency for Total Latency

39. Verifying Deterministic Latency

40. Example System Diagram

41. Deterministic Latency Test Setup
Generate a pulse with FPGA
Capture FPGA pulse with ADC
Output ADC’s MSB from FPGA
Observe relative timing on scope
Power up the system many times to confirm the relative timing stays constant

42. Device Setup for Test JESD204B Parameters FPGA
ADC = ADC16DX370 (16-bit, dual channel, 370 MSPS)
Device clock = MHz (2.713 ns)
JESD204B Parameters
L = 2, M = 2, F = 2, S = 1, K = 32
Frame cycle = 10 * F / Linerate = 10 * 1 / Mbps = ns
LMFC cycle = Frame cycle * K = ns * 32 = ns
FPGA
Device clock = MHz (~10.85 ns)

43. Additional Delays in System
Account for non-device related delays in calculations

44. Latency Calculation
Per the previous slides, the latency is calculated as
For this experiment the following parameters apply
All units in Frame clock cycles
tRX_LMFC = 28, tTX_LMFC = 3.5, n = 2, K = 32, RBD=28
Link Latency = frame clock cycles
The following also extend the latency
ADC16DX370 core latency = +12.5
DEVCLK routing skew and MSB output routing delay = +1.4
SYSREF/DEVCLK sampling skew = +1.5
Additional receiver processing delays
+3 rx_tdata release delay (for earliest sample in the 4-sample set output each period of 92.5MHz clock. This delay may instead be +4)
+4 latching MSBs before output (at Fs/4 rate)
Total Calculated Latency = = cycles
Link Latency = (n * K + RBD) * TFRAME + (tRX_LMFC – tTX_LMFC)

45. Result Calculated latency for this setup is 138.9 frame cycles
Measured latency is ns / 2.7 ns = cycles
Note: A longer pulse width would remove ambiguity of input-to-output edge
Latency=379.6ns

46. Bonus!

47. Determine Release Point by Experiment
Setup a test similar to the one shown to observe relative delays
Vary RBD until a 1 LMFC period latency jump is observed
Choose release point by taking the last RBD value before the latency jump was observed and add the expected latency variation to that value (plus extra margin)
Release Point
Release Point
Release Point
Release Point
Release Point
Release Point
Total Latency
Optimal Release Point

48. End!