Totem Readout using the SRS Adrian Fiergolski1,2, Michele Quinto1,3 1INFN-Bari, Italy 2 Warsaw University of Technology, Poland 3University of Bari, Italy Totem Readout using the SRS RD51 E-School, 3rd of February 2014

Adrian Fiergolski, Michele Quinto Outline Introduction of the TOTEM experiment Integration of SRS with the TOTEM DAQ Firmware development Firmware verification Tests and results of SRS-based DAQ 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Adrian Fiergolski, Michele Quinto TOTEM experiment TOTEM uses 3 tracking detectors (T1,T2,RP) located symmetrically with respect to the IP5 For the luminosity independent measurement of the p-p cross section at low momentum transfer, TOTEM requires magnetic configuration of the accelerator optics with high β* (90m, 1000m, 1535m) TOTEM demands special LHC runs, which allow RP to approach the beam Roman Pot: 10 planes of silicon edgeless detector T1: Cathode Strip Chamber T2: triple GEM 3-Feb-2014 Adrian Fiergolski, Michele Quinto

TOTEM DAQ before Long Shutdown (LS1) In the TOTEM standalone configuration, the VME bus bandwidth limits the trigger rate to 1 kHz. 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Adrian Fiergolski, Michele Quinto VFAT2 chip Trigger and tracking capabilities 128 channels 0.25 µm CMOS process node supports LHC clock frequency of 40 MHz Radiation hard Single Event Upset (SEU) protection 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Adrian Fiergolski, Michele Quinto Optical transmission Data 16 TX_en TX_er 800Mbps 1310 nm 9/125 µm single-mode fiber READY K28.5 D5.6 D0.6 D27.5 D13.6 D16.6 D11.7 Comma Data Ethernet codding (IEEE-802.3): 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Adrian Fiergolski, Michele Quinto 8b10b Encoding 8b 10b Assumptions: line code maps 8-bit symbols to 10-bit symbols Properties: DC-balance bounded disparity reasonable clock recovery Code structure: Difference between the count of 1s and 0s in a string of at least 20bits is no more than 2 No more than five 1s or 0s in a row All codes that represent the 256 data values: data (D) codes. The codes representing 12 special non-data characters: control (K) codes 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Adrian Fiergolski, Michele Quinto TOTEM’s DAQ evolution 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Scalable Readout System Advantages: Cost effective replacement for the currently used VME-based solution offering higher bandwidth TOTEM’s implementation will be compatible with the CMS DAQ Allow standalone runs of TOTEM Enable hardware data filtration 3-Feb-2014 Adrian Fiergolski, Michele Quinto

TOTEM’s C-Card: Opto-FEC The development board linking the OptoRx and the FEC. 32-bit parallel bus following S-Link protocol clocked at 40 MHz 2.5 Gbps SERDES 8 LVDS lines providing 5.36 Gbps Clock generator/jitter cleaner TTC interface I2C configuration lines TTS support JTAG support Independent power supply mode 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Firmware development guidelines Hardware description and verification in System Verilog language compactness, syntax structures → more re-usable, less error prone code the language consequently gains attention of industry → increasing maturity of the EDA tools Possibility to use legacy VHDL, VERILOG modules (eg. open cores) The communication between entities via standard interfaces AMBA AXI4-Stream, AHB Automatic register generation from register map specification IDesignSpec 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Adrian Fiergolski, Michele Quinto Firmware scheme System Unit Provides set of common interfaces and services. Development of mutual modules can by a shared effort of the SRS community. Application Unit Application specific data processing part. 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Firmware verification The firmware is simulated using System Verilog combined with the Universal Verification Methodology (UVM): High level of abstraction (reusable) Random test vector generation (guided by constraints) Coverage indicating verification progress EDA tools provide UVM libraries to test popular interfaces (eg. Ethernet, I2C) The verification of the FEC defines two kind of simulations: Partial simulation → to achieve faster simulation coverage of complex modules Full design 3-Feb-2014 Adrian Fiergolski, Michele Quinto

SRS based DAQ in the LHC environment Conditions: data from 3 full RP detectors containing about 120 VFATs FEC was read directly by a standard PC running DATE software commercial SATA storage medium Results: Without transmission error, the system acquired 10M events reaching maximum trigger rate of 10 kHz 3-Feb-2014 Adrian Fiergolski, Michele Quinto

Adrian Fiergolski, Michele Quinto Even faster SRS Modifications: new firmware following the presented guidelines distributed data storage on up to 3 DAQ nodes custom, combined hardware-software solution to achieve lossless transmission via unreliable UDP protocol Trigger rate at flat top ~25kHz Readout bandwidth close to the link limit 118MB/s System stability over more than 140M events None of the event has been lost Conclusion: In term of trigger rate, the new DAQ is more than one order of magnitude faster, reaching 24.7kHz against 1 kHz of standard, VME based , system. 3-Feb-2014 Adrian Fiergolski, Michele Quinto