1. Totem Readout using the SRS
Adrian Fiergolski1,2, Michele Quinto1,3
1INFN-Bari, Italy
2 Warsaw University of Technology, Poland
3University of Bari, Italy
Totem Readout using the SRS
RD51 E-School, 3rd of February 2014

2. Adrian Fiergolski, Michele Quinto
Outline
Introduction of the TOTEM experiment
Integration of SRS with the TOTEM DAQ
Firmware development
Firmware verification
Tests and results of SRS-based DAQ
3-Feb-2014
Adrian Fiergolski, Michele Quinto

3. Adrian Fiergolski, Michele Quinto
TOTEM experiment
TOTEM uses 3 tracking detectors (T1,T2,RP) located symmetrically with respect to the IP5
For the luminosity independent measurement of the p-p cross section at low momentum transfer, TOTEM requires magnetic configuration of the accelerator optics with high β* (90m, 1000m, 1535m)
TOTEM demands special LHC runs, which allow RP to approach the beam
Roman Pot: 10 planes of silicon edgeless detector
T1: Cathode Strip Chamber
T2: triple GEM
3-Feb-2014
Adrian Fiergolski, Michele Quinto

4. TOTEM DAQ before Long Shutdown (LS1)
In the TOTEM standalone configuration, the VME bus bandwidth limits
the trigger rate to 1 kHz.
3-Feb-2014
Adrian Fiergolski, Michele Quinto

5. Adrian Fiergolski, Michele Quinto
VFAT2 chip
Trigger and tracking capabilities
128 channels
0.25 µm CMOS process node
supports LHC clock frequency of 40 MHz
Radiation hard
Single Event Upset (SEU) protection
3-Feb-2014
Adrian Fiergolski, Michele Quinto

6. Adrian Fiergolski, Michele Quinto
Optical transmission
Data 16
TX_en
TX_er
800Mbps
1310 nm
9/125 µm single-mode fiber
READY
K28.5
D5.6
D0.6
D27.5
D13.6
D16.6
D11.7
Comma
Data
Ethernet codding (IEEE-802.3):
3-Feb-2014
Adrian Fiergolski, Michele Quinto

7. Adrian Fiergolski, Michele Quinto
8b10b Encoding 8b
10b
Assumptions:
line code
maps 8-bit symbols to 10-bit symbols
Properties:
DC-balance
bounded disparity
reasonable clock recovery
Code structure:
Difference between the count of 1s and 0s in a string of at least 20bits is no more than 2
No more than five 1s or 0s in a row
All codes that represent the 256 data values: data (D) codes.
The codes representing 12 special non-data characters: control (K) codes
3-Feb-2014
Adrian Fiergolski, Michele Quinto

8. Adrian Fiergolski, Michele Quinto
TOTEM’s DAQ evolution
3-Feb-2014
Adrian Fiergolski, Michele Quinto

9. Scalable Readout System
Advantages:
Cost effective replacement for the currently used VME-based solution offering higher bandwidth
TOTEM’s implementation will be compatible with the CMS DAQ
Allow standalone runs of TOTEM
Enable hardware data filtration
3-Feb-2014
Adrian Fiergolski, Michele Quinto

10. TOTEM’s C-Card: Opto-FEC
The development board linking the OptoRx and the FEC.
32-bit parallel bus following S-Link protocol clocked at 40 MHz
2.5 Gbps SERDES
8 LVDS lines providing 5.36 Gbps
Clock generator/jitter cleaner
TTC interface
I2C configuration lines
TTS support
JTAG support
Independent power supply mode
3-Feb-2014
Adrian Fiergolski, Michele Quinto

12. Firmware development guidelines
Hardware description and verification in System Verilog language
compactness, syntax structures → more re-usable, less error prone code
the language consequently gains attention of industry → increasing maturity of the EDA tools
Possibility to use legacy VHDL, VERILOG modules (eg. open cores)
The communication between entities via standard interfaces
AMBA AXI4-Stream, AHB
Automatic register generation from register map specification
IDesignSpec
3-Feb-2014
Adrian Fiergolski, Michele Quinto

13. Adrian Fiergolski, Michele Quinto
Firmware scheme
System Unit Provides set of common interfaces and services. Development of mutual modules can by a shared effort of the SRS community.
Application Unit Application specific data processing part.
3-Feb-2014
Adrian Fiergolski, Michele Quinto

14. Firmware verification
The firmware is simulated using System Verilog combined with the Universal Verification Methodology (UVM):
High level of abstraction (reusable)
Random test vector generation (guided by constraints)
Coverage indicating verification progress
EDA tools provide UVM libraries to test popular interfaces (eg. Ethernet, I2C)
The verification of the FEC defines two kind of simulations:
Partial simulation → to achieve faster simulation coverage of complex modules
Full design
3-Feb-2014
Adrian Fiergolski, Michele Quinto

15. SRS based DAQ in the LHC environment
Conditions:
data from 3 full RP detectors containing about 120 VFATs
FEC was read directly by a standard PC running DATE software
commercial SATA storage medium
Results:
Without transmission error, the system acquired 10M events reaching maximum trigger rate of 10 kHz
3-Feb-2014
Adrian Fiergolski, Michele Quinto

16. Adrian Fiergolski, Michele Quinto
Even faster SRS
Modifications:
new firmware following the presented guidelines
distributed data storage on up to 3 DAQ nodes
custom, combined hardware-software solution to achieve lossless transmission via unreliable UDP protocol
Trigger rate at flat top ~25kHz
Readout bandwidth close to the link limit 118MB/s
System stability over more than 140M events
None of the event has been lost
Conclusion:
In term of trigger rate, the new DAQ is more than one order of magnitude faster, reaching 24.7kHz against 1 kHz of standard, VME based , system.
3-Feb-2014
Adrian Fiergolski, Michele Quinto