1. Erno DAVID, Tivadar KISS Wigner Research Center for Physics (HU)
CRU Slow Control
Erno DAVID, Tivadar KISS Wigner Research Center for Physics (HU)
16 December, 2015 1 1

2. Control Functionalities via CRU
I. Hardware management of the CRU and the GBT links themselves:
1) Configuring the GBT link itself: R/W of GBTx own control registers (LHC clock phase, laser driver)
Possible: via the embedded control channel „Internal” of the GBT.
Memory mapped register R/W to GBT control registers in CRU.
CRU translates this to communicate with the GBTx (on FECs) in the GBT embedded control channel „Internal”.
2) Configuring the CRU HW (PCIe40 card) itself (e.g. MiniPODs)
Possible: with memory mapped register R/W of the CRU own control registers.
 CRU FW translates them to on-board I2C control chains of PCIe40 card (MiniPODs, external PLL, etc.).
3) Configuring the CRU Functionality (i.e. CRU FW)
Possible: memory mapped register R/W of the CRU own control registers.
II. Control of the Detector and Trigger components
4) „Slow” control (HW configuration) of FEC boards
Possible: via the embedded control channel „External” of the GBT and with the help of the GBT-SCA chip.
Memory mapped register R/W to FE control registers in CRU. (The content shall follow the HDLC protocol.)
CRU translates this to communicate with the GBT-SCA (on FECs) in the GBT embedded control channel „External”.
 GBT-SCA chip interprets and translates them to FEC-on-board I2C, SPI, MDIO, JTAG, GPIO) control chains.
5) „Fast” control of FEC boards w/ FPGA (e.g. download big data, pedestals, FE-FPGA firmware image, etc)
Possible: via GBT data channel. (Possible only for detectors where GBT data channel is not used for trigger and timing or/and FE- ASIC read-out control).
 General packet based, or single word transactions.
6) „Fast” control of trigger units w/FPGA (CTP, LTU)
Possible: via GBT data channel. (Like “Fast contorol of FEC boards w/FPGA.) 2 2 2

3. CRU Firmware Slow Control SoC Architecture
FLP Server
PCIe40 Board
Linux
CRU API
Arria 10 FPGA
O2 Software
GBT-SCA
User Logic
TTS, GBT, … (core modules)
[0..47] x 2 40 MHz
Custom Interface
[0..47] x 2 40 MHz
QSYS PCIe Endpoint 0 bar_x_master Avalon-MM Master 250 MHz x 32/64 bit
DCS Software
QSYS Interconnect (250 MHz x 32/64 bit)
??? Software
GBTx ASIC
I2C ?
[0..47] x 2 40 MHz
[0..47] x 2 40 MHz
SFP+
Si 5338
MiniPOD 0
MiniPOD 7
Arria 10 Firmware Flash
. . . 3 3 3

4. GBT-SCA IP GBT-SCA IP for the CRU prototype (few (1-4) GBT links):
Outgoing interface: 2 40 MHz, separated 40 MHz TX and RX clock domains
Avalon-MM interface v1 (single pending command): TX payload registers + go bit, RX payload registers + rcv bit, single 250 MHz PCIe clock domain
Avalon-MM interface v2 (multiple commands): TX payload and command FIFO, RX payload and packet info FIFO, single 250 MHz PCIe clock domain
Comment: Separated GBT-SCA IPs per GBT channels can be a bottleneck in the full design (it requires more PCIe bus access, no broadcast support, etc). We should investigate IP with multiple GBT links support like the LHCb solution (
Next steps:
We have to understand our GBT-SCA usage patterns. E.g. how to access a I2C slave connected to SCA (read/write), how to control the JTAG master port? What is the maximum SCA HDLC packet payload size? (the Verilog IP uses 4 byte for TX and 5 byte for RX). Is it sufficient for our needs?
We have to gain full understanding of HDLC CRC-16 calculation. We can use the ISO specification and the Caratelli's source code (which we know that has been tested with GBT-SCA).
We have to populate our GitLab repository (
Simulation difficulties: the golden reference is a Verilog file but we develop VHDL and mixed mode simulation is not supported by the Altera Modelsim Edition.
Hardware testing difficulties: needs a real GBT-SCA (no FPGA based SCA emulator exists) and an Altera based GBT link (with CRU FW+SW). 4 4 4

5. GBT-SCA Interface Variations
Avalon-MM Interface (e.g. like the Verilog IP?):
(this is just an example)
+0 : CTRL(8)
+4 : STATUS(8)
+8 : DATA_TX/RX_HI(32)
+12 : DATA_TX/RX_LO(32)
+16 : -
+20 : -
+24 : ENABLE(1)
GBT-SCA
gbt_tx_clk_i
gbt_tx_data_o[1:0]
gbt_rx_clk_i
gbt_rx_data_i[1:0]
mms_reset
mms_clk
mms_address[15:0]
mms_write
mms_writedata[31:0]
mms_read
mms_readdata[31:0]
mms_rvalid
mms_irq
GBT Interface
Avalon-MM Slave Interface
GBT-SCA IP with single GBT link support
GBT-SCA
Avalon-MM Interface:
FIFO based TX and RX interface for multiple cmd.
Burst support
gbt_tx_clk_i[N-1:0]
gbt_tx_data_o[2*N-1:0]
gbt_rx_clk_i[N-1:0]
gbt_rx_data_i[2*N-1:0]
mms_reset
mms_clk
mms_address[15:0]
mms_write
mms_writedata[31:0]
mms_read
mms_readdata[31:0]
mms_rvalid
mms_irq
mms_waitrequest
mms_burstcount[5:0]
GBT Interface
Avalon-MM Slave Interface
N = Number of the GBT Links
GBT-SCA IP with multiple GBT links support 5 5 5

6. GBT-SCA IP Simulation Setup
Testbench
2 40 MHz
or 1 80 MHz
GBT-SCA (Emulator)
GBT-SCA (Controller)
2 40 MHz
or 1 80 MHz
Avalon-MM Master
Emulator: - Receives SCA commands - Generates valid responses
Avalon-MM Master
Controller: - Sends SCA commands - Checks the responses 6 6 6