A microTCA Based DAQ System for the CMS GEM Upgrade Thomas Lenzi on behalf of the CMS Muon Group TWEPP 2016

GE1/1 Upgrade Project

Motivations Problem During Phase II, foreseen to start in 2025, the luminosity of the LHC will double, resulting in higher fluxes of particles which will impact the detectors. Solutions Add redundancy in the most forward region of the muon spectrometer where only CSCs are instrumented. Integrate GEM and CSC data to improve the performance of the trigger algorithms. Results Maintain the excellent trigger capabilities of the muon system at high fluxes of particles.

GE1/1 DAQ System Before the full installation during LS2 (2018-2019), a small slice will be installed in CMS in January 2017 with a near final version of the front-end electronics.

VFAT3 Front-End Zero suppression performed on the OptoHybrid GEM readout strips Connected to the GBT chipset The VFAT ASICs are mounted on small PCBs called the Hybrids which makes the link between the detector and the GEB. Shown here are the Hybrids of the VFAT2 ASICs. By CERN

GEM Electronics Board Not enough space to run flat cables 1-m-long PCB connecting the VFATs to the OptoHybrid Provides power to the VFATs and shielding to the detector Good signal integrity at 40 MHz for slice test 320 MHz for LS2 system By Lappeenranta University of Technology

OptoHybrid Components Xilinx Virtex-6 FPGA (XC6VLX130T) GBT chipset SCA chipset 2x VTRx & 2x VTTx Functions Slow control of the 24 VFAT2s Forward tracking data to the off-detector electronics (over VTRx) Pack the trigger data and send it to the CSCs and GEMs electronics (over VTTx) GBT Tested and working with two E-Links to and four E-Links from the FPGA at 320 MHz each. By Université Libre de Bruxelles and Texas A&M

Calorimeter Trigger Processor Zynq Processor (XC7Z045) Embedded Linux which includes Xilinx’s debugging tools Custom web application for monitoring uHal and IPBus over TCP/IP AXI bus to FPGA to transfer the IPBus requests to the on-detector electronics Green = GBT RX Red = GBT TX Virtex-7 FPGA (XC7VX690T) 36x GBT cores (no errors over 69h test) 24x 8b/10b links DAQ link to the AMC13 One CTP7 can control 12 OptoHybrids meaning we need 6 boards per endcap. By University of Wisconsin

Test Beam Test beam setup November 2015 test beam using SPS beam Two GEM detectors fully equipped with their DAQ system GLIB used instead of CTP7 Gas mixture used is Ar-CO2-CF4 (45%-15%-40%) Triggers provided by scintillators placed on both sides of the chambers Results Various measurements on the chamber efficiency Robustness of the DAQ system against high rate of triggers Current through the high voltage divider

GEM-CSC Integration Tests B904 setup GEM detector placed above CSC chamber Full DAQ chain Link between the OH and the CSC’s OTMB Goals Create path for trigger data between GEM and CSC Software development and testing Data taking with cosmic muons Integration in the CMS DAQ Results The data packer is running on the OptoHybrid and the trigger information is received by the CSCs.

Irradiation Setup Goals Estimate SEU rates in CMS and test aging Beam parameters Cyclotron at Louvain-La-Neuve Mono-energetic proton beam Energy between 14 and 62 MeV 8 cm of diameter beam spot centered on FPGA 10% homogeneity Flux up to 2 x 108 particles cm-2 s-1 24 hours of testing Electronics Irradiated 2 Xilinx Virtex-6 FPGAs (XC6VLX130T) on OptoHybrid boards Counting done by external OptoHybrid limiting the control logic on the irradiated FPGAs Communication done through HDMI cable Monitoring with ChipScope (JTAG)

Error Detection Sampling rate of 40 MHz Use Xilinx’s Soft Error Mitigation (SEM) tool to detect errors in the configuration memory. Use the ECC embedded in the BRAM controllers. Use triplication to detect and mitigate errors in the logic (CLB and DSP). SEM and ECC can correct single bit flips (recoverable) and detect double or more bit flips (critical).

SEU Cross Section SEM and ECC can correct single bit flips (recoverable) and detect double or more bit flips (critical). Energies within the range of background particles in CMS.

Total Ionizing Dose In CMS, the cumulated dose at the end of Phase II will be on the order of 10 kRad.

Consequences on GE1/1 Upper limits on cross sections for a luminosity of 5 x 1034 cm-2 s-1: SEU Cross section SEUs in CMS per FPGA SEM recoverable 3.08 x 10-7 cm2 27 / day* SEM critical 3.19 x 10-8 cm2 3 / day* BRAM recoverable 1.02 x 10-7 cm2 9 / day* BRAM critical 1.71 x 10-8 cm2 1 / day* Assuming 24h at nominal luminosity OptoHybrid firmware resource usage 36% of the LUTs 3% of the BRAMs Mitigation SEUs will have limited impact on the firmware due to the low resource consumption. Triplication can protect sensitive modules (results in backups). Periodic reset (every 10 minutes) will recover all errors.

Summary DAQ System The developments on the DAQ system for the slice test are nearly finished. The GBT chipset and GBT-FPGA core have been tested successfully and can communicate. 36 GBT-FPGA cores have been implemented in the CTP7. The link to the AMC13 and CMS DAQ is operational. Tests The GEM-CSC integration tests are on going in B904 at CERN. The DAQ chain has been tested during a test beam campaign in November 2015 and showed excellent integration of the components. The OptoHybrid’s FPGA has been irradiated to quantify the number of expected SEUs in CMS. Mitigation techniques (ECC, triplication, …) can correct for ~99% of the errors.

Outlooks Slice Test Installation starts in January 2017, commissioning in March 2017, and LHC data taking in May 2017. Instrumentation of 10 GEM detectors (5 superchambers) in one endcap of CMS. Real size test before full installation during LS2 LS2 Installation during the 2018-2019 long shutdown. Full ring of detectors in both endcaps. Upgrade to VFAT3 ASIC. New version of the OptoHybrid with 3 GBT chipsets to control the VFAT3s.

Backups

LHC Schedule

GEM-CSC Integration

Triple-GEM Technology

Test Beam Setup

Position Dependency

Triplication When an SEU affects the triplicated logic, is the mitigation working ? SEM takes a few milliseconds to correct errors. At high fluxes, the errors in the configuration memory accumulate and triplication fails. At low fluxes, time between errors is large and allows triplication to mitigate the consequences of the upsets.

Radiation in CMS

Slice Test