Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 1 Trigger Upgrade Wesley H. Smith University of Wisconsin Darin Acosta University of Florida Sergo Jindariani Fermilab US CMS PMG, January 16, 2013

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 2 Performance and Schedule of the LHC Need to handle PU~50, and L  TeV Effectively a factor of ~6 (or larger) increase in current L1 rates M.Lamont, CMS Week, also shown to Council, Dec.2012

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 3 Motivation: Trigger Rate Projections Trigger rate studies for the special 8 TeV high-PU runs ! Linear with PU and lumi, but rates growing large

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 4 Motivation: Projected L1 2E34 Single e/γ trigger Black - 14 TeV MC (50 PU) Red - 8 TeV data (66 PU) Lepton triggers scale by ~2 for increased center of mass energy. Muons have poor control of rates at high thresholds H T (E T sum of jets) Black - 8 TeV data (66 PU) Red - 8 TeV data (45 PU) Rates shown for L inst =2×10 34 cm -2 s -1 Single muon trigger Black - 14 TeV MC (50 PU) Red - 8 TeV data (66 PU) Jet trigger rates are strongly dependent on PU Rate in kHz ! 100

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 5 US Upg. Existing CMS Trigger & DAQ Overall Trigger & DAQ Architecture: 2 Levels: Level-1 Trigger: 25 ns input 3.2  s latency Design: Interaction rate: 1 GHz Bunch Crossing rate: 40 MHz 40 MHz x 25 PU = 1 GHz Level 1 Output: 100 kHz Output to Storage: Hz Average Event Size: 0.5 MB Data production 1 TB/day UXC   USC Calorimeters : Muon Systems:

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 6 Goals for CSC Trig. Upgrade - U. Florida, Rice, Texas A&M Remove limit of 3 segments per Muon Port Card Each Muon Port Card covers one sector Particularly problematic for collimated multi-muons w/rising occupancy Increase bandwidth in trigger links Occupancy of segments from chambers will exceed optical link bandwidth to CSC Track-Finder Sector Processors Dropped segments will degrade performance: lose momentum precision (higher rate) and/or tracks (inefficiency). Improve momentum resolution Make full use of all track information to approach best precision achievable for standalone muon reconstruction Steeper rate vs. p T threshold curve  increases safety margin to high luminosity and high pile-up Deliver higher precision on output track quantities to Global Trigger upgrade, & more μ candidates η ✕ϕ = 0.05 ✕ 2.5° → ✕ 0.015° Accommodate new algorithms like those in Higher Level Trigger invariant mass cuts, jet-lepton matching, … Possible seeding of future inner tracking trigger Requires new high bandwidth (optical) links to Global Trigger

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 7 EMU Trigger Upgrade CavernCounting Room

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 8 Muon Port Card Upgrade Use the existing MPC main board Backplane interface to TMB remains unchanged 3 original optical links are still available New mezzanine card with new fpga and new links x spares

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 9 EMU Track-Finder Upgrade “Backplane Connector” “Sector Processor” x 12 Connection to VT892 standard backplane

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 10 EMU CSCTF Chassis Not US* *Processing the resulting tracks with RPC or CSC hits is a US responsibility

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 11 Muon Trigger M&S TOTAL: $701K Muon Port Cards: 80 x $1480 = $126,400 Optical Fibers & installation: $51,600 New estimate 4x as much under investigation CSC Track-Finder: $523,246 Module Preproduction (3): $63,630 Module Production (18): $357,480 uTCA chassis, optical parts, COTS: $102,136

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 12 Upgrade Cal. Trig. Algos. - U. Wisconsin Upgrade Cal. Trig. Algos. - U. Wisconsin Particle Cluster Finder Applies tower thresholds to Calorimeter Creates overlapped 2x2 clusters Cluster Overlap Filter Removes overlap between clusters Identifies local maxima Prunes low energy clusters Cluster Isolation and Particle ID Applied to local maxima Calculates isolation deposits around 2x2,2x3 clusters Identifies particles Jet reconstruction Applied on filtered clusters Groups clusters to jets Particle Sorter Sorts particles & outputs the most energetic ones MET,HT,MHT Calculation Calculates Et Sums, Missing Et from clusters All coded in Firmware & Tested (latency/resources)

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 13 Upgrade Cal. Trig. Performance Simulation work on stage-1 calorimeter trigger by FNAL, UI-Chicago, UC Davis, UC San Diego, U Wisconsin, MIT, Ohio State U. (subset of stage 2) Also exploring muon isolation

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 14 Calo Trigger Upgrade in Parallel: Split inputs from ECAL & HCAL HCAL energy ECAL energy Regional Calo Trigger Global Calo Trigger EM candidates Region energies HF energy HCAL energy Layer 1 Calo Trigger Layer 2 Calo Trigger Current L1 Trigger System Upgrade L1 Trigger System oSLB oRM Install optical SLB and optical RM mezzanines during LS1 Install HCAL passive optical splitters during LS1 or YETS Install HCAL backend μHTR cards for input to new trigger Install HCAL frontend electronics after LS2 (finer segmentation)

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 15 Calorimeter Trigger Evolution 3 new calorimeter trigger μTCA crates  spares Optical Receiver Modules 

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 16 Calorimeter Trigger Processor Virtex-6 Prototype Board (CTP-6) Back End FPGA XC6VHX250 T/ XC6VHX380 T Front End FPGA XC6VHX250 T/ XC6VHX380 T Avago AFBR- 810B Tx Module 4X Avago AFBR-820B Rx Module MMC Circuitry JTAG/USB Console Interface Mezzanine Power Modules Dual SDRAM for dedicated DAQ and TCP/IP buffering 12x Multi Gig Backplane Connections

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 17 Calorimeter Trigger Processor Virtex-7 (CTP-7) Replace 2 Virtex- 6s with a Virtex 7 for processing+ZYNQ for embedded TCP/IP endpoint 30A, 1V power module for FPGA logic core 3x CXP Pluggable modules for 36 Tx + 36 Rx 10G optical links 2x AFBR-820 modules for 24 Rx 10G optical links Simpler design to execute than the CTP-6 36 Total + spares Virtex-7 VX690T FPGA ZYNQ XC7Z030 EPP (optional) 1.5V Supply 2.5V Supply 3.3V Supply 1V 30A Supply CXP Module 12Tx + 12 Rx CXP Module 12Tx + 12Rx CXP Module 12Tx + 12 Rx 12X Rx 12X Rx (CTP-6 CAD View)

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 18 CIO-X: crate interconnections (2/crate x 3 crates = 6 + spares) Controller (MMC and link mgmt) 4X Avago AFBR-79EQDZ QSFP+ Module Positions 4x4 Lane Bidirectiona l Multi Gig Backplane Connections Backplane Rx/Tx Redriver ICs (top and bottom sides)

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 19 VT894 Crate Test Setup (Final system: 3 crates w/ 12 CTP7 ea.) BU AMC13 Vadatech MCH UW CTP-6 TTC Downlink UW Aux

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 20 Final Calo Trigger Upgrade (“Stage 2”) Two modes of connectivity required Keep new trigger flexible in order to adapt to the needs of the evolving CMS physics program Both architectures have two processing layers Layer 1 optimized for backplane connectivity, Layer 2 for optical TMT architecture chosen as baseline Fully Pipelined Calorimeter Trigger Time Multiplexed Calorimeter Trigger Layer 1 Layer 2 Demux

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 21 Stage 2 Cal. Trig. Upgrade (TMT Architecture) US: UK:

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 22 Calorimeter Trigger M&S TOTAL: 1,149 K$ K$ each (36 needed plus 6 spares): 735 K$ FPGA: 12 K$ Optical: 3 K$ Board, Fabrication, Assembly, other 2.5 K$ 6 CTP K$ each: 105 K$ 9 2K$ each (6 needed plus 3 spares): 18 K$ Optical: 1.2 K$ Board, Fab Assembly, other 0.8 K$ 3 CIOx 2K$ each: 6 K$ 4 Vadatech μTCA crates : 40 K$ incl. MCH & PS (3 needed plus 10 K$ 4 AMC13 Modules (3 needed plus 5 K$: 15 K$ Optical Cables & Patch Panel between Layers 1 & 2: 20 K$ oRM’s: 700 (601 plus 300 $: 210 K$.

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 23 Trigger Labor & Travel (non-physicist) Total Labor: $2.07M Muon Trigger Labor over 4 years: $ 840K MPC Electronic & Firmware Engineering: $160K CSCTF Electronic & Firmware Engineering: $480K Software Engineering: $200K Calorimeter Trigger Labor over 4 years: $ 1,230K Electronic Engineering: $ 380K Firmware Engineering: $ 530K Technician: $160K Software Engineering: $160K Travel over 4 years: $130K Muon: $70K Calorimeter: $60K NB: Resource Loading is not complete

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 24 Trigger WBS High Level Notes: Starts Nov. 1, 2013 Prototyping complete Not included Dictionary structure only Next: Schedule integration & Resources Needs alignment with CMS Structure Underway All WBS US only

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 25 Trigger WBS Detail

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 26 Cal. Trig. Layer 1 Schedule (official CMS version)

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 27 EMU Trigger Schedule (official CMS version)

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 28 Cal. Trig. oSLB-oRM Schedule

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 29 Schedule & Completion Trigger threshold KPP: Demonstration of 99.9% agreement between upgrade trigger electronics and software emulation of this electronics through test pattern injection based on data taken after LS1 followed by demonstration of reduction of calorimeter and endcap muon trigger rates for electrons, photons, muons and taus with respect to the present system by a factor of two for a reduction of less than 15% in efficiency using the trigger emulator run on data taken after LS1. Incorporation of unganged ME1/1a data into the endcap muon trigger logic. Trigger objective KPP: Demonstration of 99.99% agreement between upgrade trigger electronics and software emulation of this electronics through test pattern injection based on data taken after LS1 followed by demonstration of reduction of calorimeter and endcap muon trigger rates for electrons, photons, muons and taus with respect to the present system by a factor of two for a reduction of less than 10% in efficiency using the trigger emulator run on data taken after LS1. Calorimeter Trigger electron and photon position resolution improved from DeltaEta=DeltaPhi=0.35 to DeltaEta=DeltaPhi=0.05. Incorporation of unganged ME1/1a data into the endcap muon trigger logic.

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 30 RisksRisks MPC installation incomplete by end LS1 Consequence: unable to provide full inputs to upgrade CTCTF Mitigation: installation of remaining MPC mezzanines during YETS. oSLB-oRM installation not complete by end LS1 Consequence: full parallel operation of final calorimeter trigger not possible until 2018 Mitigation: use parallel operation involving stage-1 calorimeter trigger hardware (oRSC) or use slice for validation Full HCAL μHTR system not commissioned by end 2016 Consequence: operation of final calorimeter trigger not possible during 2016 Mitigation: Continue to use stage-1 upgrade calorimeter trigger hardware (oRSC).

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 31 BACKUPBACKUP

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 32 Cal. Trig. Stage 1 UK

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 33 Muon Trigger WBS Trigger Milestones & Interfaces Endcap Muon Track-Finder Trigger Management EMUTF Production Manage Muon Trigger EMUTF Software Muon Trigger Milestones & Interfaces EMUTF Firrmware Muon Trigger Management EMUTF Components Muon Port Card Mezzanine EMUTF Optics MPCM Production Manage EMUTF FPGAs MPCM Software EMUTF Misc. Components MPCM Firmware EMUTF PCB Fabrication MPCM Components EMUTF Processing Card Fab MPCM Optics EMUTF Optics Card Fab MPCM FPGAs EMUTF Memory Card Fab MPCM Misc. Components EMUTF Backplane Connector Fab MPCM PCB Fabrication EMUTF Assembly MPCM Assembly EMUTF Processing Card Assy MPCM Testing EMUTF Optics Card Assy MPCM Ship EMUTF Memory Card Assy MPCM on-site Testing EMUTF Backplane Connector Assy MPCM Installation EMUTF Testing MPCM Comissioning EMUTF Ship MPC-EMUTF Optical Fibers EMUTF on-site Testing MPC-EMUTF OF Production Manage EMUTF Installation MPC-EMUTF OF Components EMUTF Comissioning MPC-EMUTF OF Fabrication EMU TF Infrastructure MPC-EMUTF OF Testing EMUTF Infrastructure Manage MPC-EMUTF OF Ship EMUTF Vadatech Crates MPC-EMUTF OF Installation EMUTF AMC MPC-EMUTF OF Comissioning EMUTF Optical Splitters & Patch Panel EMUTF Infrastucture Testing EMUTF Infrastructure Ship EMUTF Infrastructure on-site Testing EMUTF Infrastructure Installation EMUTF Infrastructure Comissioning

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 34 Calorimeter Trigger WBS Calorimeter Trigger CTP Infrastructure Calorimeter Trigger Milestones & Interfaces CTP Infrastructure Manage Calorimeter Trigger Management CTP Vadatech Crates Calorimeter Trigger Processor CTP AMC CTP7 Production Manage CTP Optical Cables & Patch Panel CTP7 Software CTP Infrastucture Testing CTP7 Firmware CTP Infrastructure Ship CTP7 Components CTP Infrastructure on-site Testing CTP7 Optics CTP Infrastructure Installation CTP7 FPGAs CTP Infrastucture Comissioning CTP7 Misc. Components Optical Receiver Mezzanines CTP7 PCB Fabrication oRM Production Manage CTP7 Assembly oRM Software CTP7 Testing oRM Firmware CTP7 Ship oRM Components CTP7 on-site Testing oRM Optics CTP7 Installation oRM FPGAs CTP7 Comissioning oRM Misc. Components CIOx Card oRM PCB Fabrication CIOx Production Manage oRM Assembly CIOx Software oRM Testing CIOx Firmware oRM Ship CIOx Components oRM on-site Testing CIOx Optics oRM Installation CIOx FPGAs oRM Comissioning CIOx Misc. Components oSLB-oRM Optical Fibers CIOx PCB Fabrication oSLB-oRM Production Manage CIOx Assembly oSLB-oRM OF Components CIOx Testing oSLB-oRM OF Fabrication CIOx Ship oSLB-oRM OF Testing CIOx on-site Testing oSLB-oRM OF Ship CIOx Installation oSLB-oRM OF Installation CIOx Comissioning oSLB-oRM OF Comissioning

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 35 Calorimeter Trigger Primitives Present Connections Regional Calorimeter Trigger To DAQ Via GCT HCAL HTR Cards To DAQ Via HCAL DCC2 Existing Copper Cables ECAL TCCs To DAQ Via ECAL DCC Existing Copper Cables

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 36 Calorimeter Trigger Primitives Connection Evolution Regional Calorimeter Trigger HCAL uHTR Cards ECAL TCCs SLHC Cal Trigger Processor Cards Optical Ribbons OSLBs ORMs Trigger Primitive Optical Patch Panel ECAL Opti. Ribbons ECAL Indiv. Fibers (LC) Optical Ribbons To DAQ Via BU “AMC13” To DAQ Via GCT To DAQ Via BU “AMC13” To DAQ Via ECAL DCC HCAL HTR Cards To DAQ Via HCAL DCC2 Existing Copper Cables HCAL Opti. Ribbons Optical Splitter Upstream of HCAL HTR/uHRT

Wesley Smith, U. Wisconsin, PMG, January 16, 2013 Trigger Upgrade – 37 Calorimeter Trigger Primitives Final Situation HCAL uHTR Cards ECAL TCCs SLHC Cal Trigger Processor Cards Optical Ribbons OSLBs ECAL Opti. Ribbons Optical Ribbons To DAQ Via BU “AMC13” To DAQ Via BU “AMC13” To DAQ Via ECAL DCC HCAL Opti. Ribbons Trigger Primitive Optical Patch Panel