1. GBT-SCA Slow Control Adapter ASIC
LHCb Upgrade Electronics meeting
12 June 2014

2. Design Team Core Digital Logic:
Christian Paillard, Alessandro Caratelli
e-port interface:
Sandro Bonacini
ADC:
icSparkling IP block
DAC:
Xavier Llopart (Medipix IP block)
Chip Assembly:
Christian Palliard, Rui F. Oliveira
AOB: Kostas Kloukinas, Paolo Moreira
12 June 14

3. SCA in the GBT system
12 June 14

4. The GBT System A Single Link for: Readout (DAQ)
Embedded Electronics
Control Room
Optical Link
A Single Link for:
Readout (DAQ)
High speed unidirectional (up-link)
Trigger data (up-link)
Timing Trigger and Control (TTC)
Clock reference and synchronous control (down-link)
Trigger decisions and control (down-link)
Low and fixed latency
Experiment control (SC/DCS/ECS)
Modest bandwidth (bidirectional link)
Custom ASICs in the detectors:
Radiation Tolerant: Total dose & Single Event Upsets
Commercial components in the control room
FPGAs used to implement multi-way transceivers
The communication between the front-end electronics embedded in the detectors and the control room is manly composed of three functional subsystems
• A fast timing distribution system responsible to deliver to the experiment the system clock and several fast trigger signals and some fast signals from the detector to the control room,
• A data acquisition link carrying the collected data from the detector to the control room,
• A slow control system carrying bidirectional traffic from and to the control room and the embedded electronics in the detector.
The GBT system provides a single bidirectional link for carrng the thre types of traffic. An appropriate bandwidth is allocated to each of the three subsystems.
12 June 14

5. GBT-SCA front-end interconnects
e-port
GBTX
GBT on FPGA
Slow Control
DAQ
Timing and Trigger
GBT-SCA
FE ASICs
Embedded electronics
Control room
User busses and I/O for Control and monitoring
Data
Timing
ASIC dedicated to slow control functions.
System Upgrades for SLHC detectors.
Replacement for the CCU & DCU ASICs
CCU: Communication Control Unit
DCU: Detector Control Unit in CMS).
Flexible to match the needs of different Front-End systems.
Technology: CMOS 130nm using radiation tolerant techniques.
The slow control system is the one uncharged to distribute control sequences and collect status information from the embedded peripheral electronics in the detector. Such as front-end ASICs or monitoring chips for control of environmental variables or any other specific module which needs to be remotely controlled
the SCA chip is the embedded node of the system which is responsible to translate the unified packets sent by the control room and redirect to the selected peripheral through one of the physical ports.
It connects to a dedicated e-link port on the GBTX ASIC, with a fixed bandwidth of 80MHz.
The GBT communication is totally transparent to the slow control protocol.
12 June 14

6. The Slow Control Adapter ASIC
Dual redundant e-Ports for GBTX e-links.
Network Controller
Arbiter
16 I2C masters (7bit/10bit addressing)
1 JTAG master controller
1 SPI master (with 8 slave select)
32 General purpose digital I/O lines individually programmable as input / output / interrupt input
31 Analog inputs multiplexed in a 1 bit single slope integrative ADC
4 independent analog output (8 bits resolution)
Auxiliary I2C port for testability
The remote devices controlled by the SCAs are seen from the control computer as remote independent channels, each one with a particular set of control registers and/or allocated memory locations. \
The channels operate independently from each other in order to allow concurrent transactions and perform concurrent transfers to their end-devices.
The NC is encharged to collect the requests from the control room, interpetate it and redirect to a specific channel interface, as well collect the replyes from the interfaces when they request attention through an internal interrupt system and resend to the control room.
Figure 1. The SCA top level block diagram
12 June 14

7. Communication architecture
12 June 14

8. Communication Architecture
The Three Communication Layers of the SCA:
GBT-link Layer
Connects the GBT ASIC to the Control Room electronics via a point-to-point, bi-directional, 4.8Gbps, optical link using a special, SEU robust, transport protocol.
The transport protocol is totally transparent to the user data.
e-link Layer
Connects to the SCA ASIC via a point-to-point, bi-directional, 80Mbps link, using a packet oriented transport protocol (HDLC protocol).
Data packets are encapsulated by the GBT-link protocol.
Channel Layer
Connects to the Front-End ASICs via bi-directional links.
SCA on-chip peripherals are also Channel Blocks.
Channel Blocks are controlled via a command driven protocol.
The Channel Protocol Commands are encapsulated by the e-link protocol.
12 June 14

9. GBT-link Packet Format
Fixed packet length: 120bits
Packet transmission rate: 1/25ns
Data transmission rate: 4.8 Gbps
Fixed bandwidth allocation:
Trigger path: 640 Mbps
Control path: 160 Mbps
1 internal e-link (for GBT management)
1 external e-link (for GBT-SCA chip)
40 MHz DDR (80 Mbps)
Data path: 2.56 Gbps
Mbps
Mbps
40 80 Mbps
Data flow:
Symmetrical, Bi-directional data transmission.
Transmission of GBT-packets is continuous.
Data from e-link ports are muxed/demuxed in the GBT-link stream.
GBT data path is unaware of the e-link transfer protocol.
2 bits H IC EC
FEC
80 bits
32 bits
4 bits
Neverless it is possible to connect up to 40 SCA ASICS to 1 GBTx module, on the data path, in case the user wants to have an optical link dedicated for slow control purposes.
12 June 14

10. HDLC Packet Format HDLC Packet:
8 bits
SOF
Address
Control
DATA
FCS
EOF
16 bits
16∙n bits
6 bits
HDLC packet structure:
HDLC Packet:
SOF/EOF: Frame delimiter character.
Address: Packet destination address.
Control: Indicates the type of the data packet.
FCS: Frame Check Sequence for transmission error detection. (𝐺(𝑥) = 𝑥 𝑥 𝑥 5 + 1)
Uses bit-stuffing techniques in order to achieve symbol synchronization.
The frame delimiter contains 6 consecutive ones;
Any sequence of 5 ones in the stream is stuffed with one following zero at the transmitter side
Any sequence of more than 6 ones is considered to be frame abort or channel idle signalling
Bi-directional operation.
Each received command packet get acknowledged or rejected by the SCA
12 June 14

11. SCA Packet Formats SCA Packet: HDLC packet :
8 bits
SOF
Address
Control
DATA
FCS
EOF
16 bits
16∙n bits
6 bits
HDLC packet :
8 bits
16∙n bits
CH#
TR#
CMD
Data
LEN
SCA up-link packet structure:
ERR
SCA down-link packet structure:
SCA Packet:
CH#: Specifies the SCA Channel interface to be addressed
TR#: Incremental identification number for each transmitted command.
CMD: Is a command code that specifies the request. The operation can refer to a specific internal register of the channel or a front-end ASIC destination address. In this case an address field follows the command.
ERR: In case of an error has encountered, return an error otherwise returns 0x00.
LEN: Is a field that specifies the DATA field length.
DATA: Is an optional variable length field upon LEN value.
12 June 14

12. Channel Commands
At the reception of any request packet, the SCA replies with an acknowledge packet
The acknowledge packet include information of the status of the operation and eventually the data value required.
Each command can be directed to a specific channel interface.
Upon reception of a command, the channel performs the required operation on its interface.
The command field and the data content of a given packet is interpreted differently according to the channel to which it is addressed.
Each channel type has a specific set of valid commands.
Channels receiving an invalid command do not execute any action and the error condition is reported to the user.
Enable/Disable command for each channel
When not enabled, the interface core is kept in Power Down Mode to reduce power consumption.
At the enable, the channel interface get reset.
12 June 14

13. Implementation
12 June 14

14. e-link & e-port Block Diagram
SCA Network Controller
RX FIFO RX
control
PHY
RXCLK
RXDATA
ATLANTIC
Bus TX
control
TXDATA
TX FIFO
Clk 40 MHz
RESET
e-port implementation for the GBT-SCA
e-link interface: electrical, bi-directional chip-to-chip interconnect at 80Mbps.
HDLC (High Level Data Link Control) packet oriented communication protocol.
Special commands: CONNECT, RESET, TEST (loopback)
Acknowledge transactions using the HDLC protocol mechanism (packet numbering).
12 June 14

15. Dual Redundant e-ports
The GBT-SCA is equipped with a Primary and a Secondary e-port to be connected with two adjacent GBTX chips for redundancy
Only one e-port is needed for operation and can be active at a time
Data from the inactive e-port is discarded
Switching between ports is command driven. (HDLC “CONNECT” command)
Clock activity or noise on the differential input of the inactive link is discarded
Upon start up, the user application software should send a “CONNECT” command to activate the Primary e-port from the Primary e-link.
To switch to the Secondary e-port a “CONNECT” command should be send through the Secondary e-link.
Figure 2. GBT-SCA to GBTX connectivity
Figure 3. Logical view of HDLC e-ports
14/05/2014
GBT-SCA Design review

16. Physical Layer (electrical)
SLVS (Scalable Low Voltage Standard)
JEDEC standard: JESD8-13
Differential voltage based signaling protocol.
Voltage levels compatible with deep submicron processes.
Low Power, Low EMI
Line impedance: 100 Ohm
Differential mode: 400 mV
Common mode: 0.2V
0.2V V
200mV
12 June 14

17. Auxiliary I2C port Auxiliary I2C port for testability
Allows direct access to Network Controller by-passing the e-ports.
Receive SCA commands and returns replies
Hardwired “Test Enable” pin
Possibility to use as an expansion port
14/05/2014
GBT-SCA Design review

18. The SPI channel Synchronous bit oriented serial link.
Full duplex synchronous serial data transfer
Configurable single transaction length up to 128 bits RW
Configurable communication speed from 156KHz to 20MHz settable with 128 possible values
Asynchronous reset signal with configurable pulse length
Acknowledge packet to report the end of the communication
Supported working modes:
(0,0) (0,1) (1,0) (1,1).
Possibility to invert the logical level of the output signals during the inactivity time.
Figure 5. The SPI channel scheme
14/05/2014
GBT-SCA Design review

19. The JTAG channel
128 bit shift registers to hold the TDO/TDI and TMS bit-streams
Protocol implemented in software
Configurable single transaction length up to 128 bits RW
Configurable communication speed from 156KHz to 20MHz settable with 128 possible values
Asynchronous reset signal with configurable pulse length
Acknowledge packet to report the end of the communication
Possibility to invert the TCK edge synchronously to which the transmitted bits on the TDO and TMS lines are latched
Possibility to invert the TCK edge synchronously to which the received bits on the TDI line are evaluated.
Possibility to invert the logical level of the output signals during the inactivity time.
Figure 6. The JTAG channel scheme
14/05/2014
GBT-SCA Design review

20. The I2C channels 16 independent I2C master channels
Concurrent operation of all 16 channels
Support is limited to single bus master topologies.
7-bit and 10-bit addressing modes.
Single-byte and Multi-byte transfer modes.
Read-Modify-Write mode that allows for AND, OR, XOR logic operations with a Mask register.
Programmable transfer rate from 100Kbps to 1Mbps.
Power Down mode per channel
Figure 4. I2C channel block diagram
14/05/2014
GBT-SCA Design review

21. The GPIO channel 32 general-purpose I/O signals.
All general-purpose I/O signals are bi-directional having individually programmable direction.
All general-purpose I/O signals can be three-stated.
General-purpose I/O signals programmed as inputs can cause interrupt requests.
General-purpose I/O signals programmed as inputs can be registered at raising edge of system clock or at user programmed edge of external clock.
Alternative input reference clock signal from external interface.
Power Down mode
Figure 7 - The GPIO channel
14/05/2014
GBT-SCA Design review

22. The DAC channel 4 independent Digital to analog converters.
The single DAC is an IP Block from MEDIPIX team
Voltage output: 0.0V – 1.0V
8 bit resolution
Figure 8 – The DAC channel
14/05/2014
GBT-SCA Design review

23. ADC channel DIGITAL ANALOG ADC Control logic Calibration logic
12-bit ADC (LSB = 244 µV)
Single -slope architecture
Input range: GND < Vin < 1 V
Single supply voltage: VDD = 1.5 V
System clock frequency: 40 MHz
Max. conversion rate ≈ 3.5 KHz Max conversion time ≈ 600 μsec
Operating temperature: -30°C to +80°C
32 Multiplexed Input channels
1 channel occupied by an on chip Temp sensor
All channels feature a switchable 10 μΑ current source to support Pt sensors.
Power consumption 1.2V (420uW analog part )
Power Down mode
Automatic Offset cancelation & Gain correction
Pre-calibration at production testing phase
DIGITAL
ANALOG
ADC
Control logic
Calibration logic
E-fuses read logic
E-fuses bank
Supervising logic +
Wishbone interface
Figure 9. The ADC channel scheme
14/05/2014
GBT-SCA Design review

24. Layout E-Fuses DAC Prompt ADC analog ADC calib. logic JTAG channel
SPI channel
I2C channels
Network controller
GPIO channel
E-Link
3.6 x 3.6 mm
12 June 14

25. Packaging (draft) Package Type: LFBGA
Low Profile Fine Pitch BGA (Chip Scale Package)
Ball Pitch: 0.8 mm
Size: 12 x 12 mm
Height: mm
Pin count: 196
Preliminary pinout
Manufacturers:
ASE via IMEC
NOVAPACK
12 June 14

26. Final Verifications Functional simulation at the top level
Post Layout simulations
Simulation with SEU injection
Ultrasim Mixed-Signal simulations
SCA emulation on FPGA
Protocol tests with commercial devices
12 June 14

27. FPGA test bench 1/2 Software FPGA test firmware SCA logic 14/05/2014
I2C Slave x4
USB control logic
SCA packet gener.
E-Port Master
Parallel I/O ctrl
SPI Slave Multimode
Comm. control
JTAG slave
Transf. control flags
SCA send packet
SCA rec packet
GPIO data
SPI data
JTAG data
Slaves Configure
Test System config.
I2C0 data
I2Cn data
Drivers +
USB bluster
+ interface etc.
Low level func. to handle GBT-SCA communication
Low level func. to handle slaves
Testing software:
Start random transaction on random channels
Whait with time-out replies from the SCA logic
Compare data received on slave interfaces with the expected behaviour respect sent requests
Write operation log and error log
Test passed when no errors are encountered
Software
FPGA test firmware
14/05/2014
GBT-SCA Design review

28. JTAG test with commercial hardware
SCA logic
I2C Slave x4
USB control logic
SCA packet gener.
E-Port Master
Parallel I/O ctrl
SPI Slave Multimode
Comm. control
Transf. control flags
SCA send packet
SCA rec packet
GPIO data
SPI data
JTAG data
Slaves Configure
Test System config.
I2C0 data
I2Cn data
Drivers +
USB bluster
+ interface etc.
Low level func. to handle GBT-SCA communication
Low level func. to handle slaves
Program en external commercial FPGA through the GBT-SCA emulated on an other FPGA:
Read FPGA configuration file (ISE output file containing series of directives for JTAG interface to program the target Xilinx FPGA)
Find the target FPGA in the JTAG chain
Generate corresponding commands in SCA format
Send the commands to the GBT-SCA emulated in the main FPGA
Write error log of unsuccessful operations
Software
FPGA test firmware
I2C interface
SPI interface
JTAG interface
14/05/2014
GBT-SCA Design review

29. Command line interface
FPGA test bench 2/2
FPGA test firmware
Nios II Processor
Handle shell interface
E-port drivers
Slaves drivers
Handle communication with SCA
Real-time verification of concurrent operations performed by the SCA
Report Operation information and identified errors
Comm. control
E-Port Master
SCA logic
Comm. control
E-Port Master
I2C Slave x4
I2C Slave x4
I2C Slave x4
I2C Slave x4
Drivers and
USB bluster
Command line interface
JTAG port
Parallel I/O ctrl
Timer banks
SPI Slave Multimode
SDRAM
controller
JTAG slave
PLL
12 June 14

30. Specifications Documentation: “Users Manual” document (draft)
“Table of commands”
12 June 14

31. Status & Planning Design Readiness review on 14/5/2014 Design Status
Suggestions for design verifications
Minor design modifications (suppression of external biasing resistor)
Design Status
Ready for submission
Participate on the GBT-chipset engineering run scheduled for this month (June 2014)
Estimated TAT: 4-5 months
Yield a few hundred SCA chips
Test Bench Preparation
12 June 14

32. Thank you
12 June 14