SLHC Calorimeter Trigger S.Dasu University of Wisconsin 08 May 2009 test

Introduction SLHC Calorimeter Trigger (SLHC-CT) Goals Enable continuation of EWSB physics studies at SLHC Retain similar thresholds as LHC: x2 reduction in rate for Phase I and x10 for Phase II Single & double electrons and photons: 30-60 GeV Single & double taus: 50-80 GeV (not Ztt but for Htt ) Jets & MET: Provide better energy and position resolution SLHC Calorimeter Trigger (SLHC-CT) Upgrade Strategy Improved algorithms, more multi-object triggers & improved topological conditions for Phase I Track matching for Phase II High spatial resolution of calorimeter trigger objects so that tracks can be matched closely Track matching and track Isolation will resolve electrons from photons , taus from jets

SLHC-CT Upgrade Constraints Input to calorimeter trigger No changes to ECAL TPGs EE may change, but only in Phase II HCAL TPGs can change in Phase I if needed To be defined Output of calorimeter trigger Work closely with global trigger systems to evolve together Upgrade Commissioning Seamless transition to new trigger hardware such that LHC operations are not affected Commission alongside current trigger Ability to operate with old and new systems at the same time is needed, e.g., old EB and new HB/HE/HF Transition to upgraded trigger when validated

Calorimeter Trigger Evolution Step 1 (2009) Step 2: ↓ OR ↓ Step 3 Step 4 ETCC: TPGs HTR: TPGs ETCC: TPGs HTR: TPGs uTCA- HTR: TPG ETCC: TPGs uTCA-HTR: TPGs RCT/ uTCA GCT/ uTCA GT/GMT ETCC: TPGs uTCA-HTR: TPGs oSLB oSLB oSLB SLB SLB SLB oSLB oSLB oSLB RCT RMC RMC RCT RMC oSLB oSLB RCT oSLB oSLB RCT/ uTCA Matrix & Aux Cards  Cu GCT: Sources GCT/ uTCA GCT: Sources GCT/ uTCA GCT: Sources GCT/ uTCA FO GCT: Main GT/GMT GT/GMT GT/GMT

SHLC-CT Upgrade Studies Simulation Program Developed SLHC trigger algorithm simulation program to work within CMSSW Compare LHC and SLHC algorithm performance Interface with track trigger simulation program Studies for Phase II of upgrade Firmware & Hardware Program Firmware development in collaboration with experts from Wisconsin and Maryland EE departments Feasibility of implementing new algorithms Systematic improvements in methodologies Prototyping of mTCA cards with new FPGAs began with last portions of LHC calorimeter trigger

Optical Serial Link Board (oSLB) oSLB (TX and RX) Responsibility of Jose Carlos da Silva (LIP, Portugal) Maintain same connectivity and form factor as current SLB Replace cables by optical links Latency budget increases in most cases when using FPGAs High speed link, 850/1310 nm, commercial package PCB for both ECAL/HCAL and RCT FPGA Transceivers, deeper FIFO and Full Orbit histogram per channel Stratix II or IV (GX) or Virtex 4 or 5 Future options / reasons for upgrade

Calorimeter signatures HCAL η Δη x Δφ=0.087x0.087 Electrons/Photons Spatially confined in a cluster of 2x2 trigger towers Significantly higher ECAL contribution Isolated e/γ should have low energy deposits in the surrounding area ECAL φ e/γ HCAL η Taus Confined in 2x3 Clusters 3 prongs/1 prong + π0s have wider φ profile Small energy leak in surrounding towers ECAL φ τ η HCAL Jets Most of the energy confined in a central core For jets over 20 Gev the energy is included in a 8x8 region ECAL φ jet

Algorithm Overview Particle Cluster Finder Cluster Overlap Filter Takes Calorimeter TPG input and applies tower thresholds Creates overlapped 2x2 clusters Cluster Overlap Filter Removes overlap between clusters Identifies local maxima Prunes low energy clusters Cluster Isolation and Particle ID Applied to local maxima Calculates isolation deposits around 2x2,2x3 clusters Identifies particles Jet reconstruction Applied on filtered clusters Groups clusters to jets Particle Sorter Sorts particles and outputs the most energetic ones MET,HT,MHT Calculation Calculates Et Sums, Missing Et from clusters

SLHC-CT Simulation Results: Factor of 2 for Phase I Michalis Bachtis & Kevin Flood Isolated electrons Taus QCD Rate (kHz) QCD Rate (kHz) Factor of 2 rate reduction Isolated electrons Taus Efficiency Efficiency Higher Efficiency

Improved Position Resolution Achieved better than the target half-tower resolution (0.044)

Jet Trigger Comparison SLHC LHC LHC SLHC

Algorithm implementation (Firmware) M.Schulte, K.Compton, B.Buchli, A. F-Farmahani, T.Gregerson, S.Naumov (ECE Department/UW)‏ Now Implementing and optimizing the algorithms Implementation of thresholds and pattern comparator Comparison of LUT, Division implementation for electron ID Cluster filtering, jet finding … Shruva Bhattacharya and Will Plishker, Maryland FPGA Firmware design methodology Development management tools Algorithm implementation (Firmware)

Initial Design Platform Xilinx Virtex-5 devices contain Virtex-5 Slices (4 LUTs and 4 flip-flops) DSP48E Slices (multiplier, adder, and accumulator) Block RAMs (36 Kbits) RocketIO Transceivers GTP transfers up to 3.75 Gbps/link GTX transfers up to 6.50 Gbps/link Initial designs synthesized for Xilinx Virtex-5 LX110T and TX240T FPGAs For the initial design platform, we have been using Xilinx Virtex-5 FPGAs and Xilinx Design tools. It is important to note that the RocketIO Transceivers are full-duplex bi-direction, which means that the TX240T device has 48 RocketIO serial inputs and 48 RocketIO serial outputs. Compared to the LX110T device, the TX240T device has roughly twice as many Virtex-5 slices and Block RAM bits, 50% more DSP48E Slices, and up to six times more IO bandwidth. FPGA Virtex-5 Slices DSP48E Slices Block RAM (Kbits) RocketIO Transceivers LX110T 17,280 64 5,328 16 GTP TX240T 37,440 96 11,664 48 GTX

FPGA Firmware Results Virtex-5 Resource Utilization for Input and Output RocketIO and Buffering on the TX240T FPGA Resource 8 x 8 Grid 8 x 16 Grid 16 x 16 Grid RocketIO Input Links 25% 46% 83% Virtex-5 Slices 5% 10% 19% Resource utilization for Particle Cluster Finder theTX240T FPGA Resource 8 x 8 Grid 8 x 16 Grid 16 x 16 Grid Virtex-5 Slices 10% 20% 39% BRAMs 14% 27% 53% Virtex-5 Slice Utilization for Cluster Overlap Filter FPGA 8 x 8 Grid 8 x 16 Grid 16 x 16 Grid LX110T 14% 29% 58% TX240T 7% 27%

Overall Resource Estimates Estimated resources are given in the table below Includes RocketIO, buffers, particle cluster finder, overlap filter, and cluster weighting Additional grid sizes and FPGA devices should be considered Overall Resource Utilization on TX240T FPGA This slide shows the overall estimated resource utilization for input and output RocketIO and buffers, particle cluster finder, overlap filter, and cluster weighting. If we implement an 8 x 8 grid of clusters, we have plenty of resources left and should be able to implement additional functionality on the same FPGA. If we implement an 8 x 16 grid of clusters, then we use just over half of the Virtex-5 slices and just under half of the RocketIO links. This should still allow fairly significant upgrades to be made to the algorithms. It does not seem like a good idea to implement a 16 x 16 grid of clusters on a single Virtex-5 TX240T FPGA, since this could impact upgradability. However, since high-end Virtex-6 FPGAs will have a larger number of Virtex Slices and RocketIO links, it may be feasible to implement a 16 x 16 grid on Virtex-6 FPGAs. If each FPGA implements a 8 x 16 cluster grid, then this portion of the Calorimeter trigger would require roughly 36 FPGAs to implement to process the 2-dimensional array of 4096 towers. Resource 8 x 8 Grid 8 x 16 Grid 16x 16 Grid RocketIO Links 25% 46% 83% Virtex-5 Slices 27% 54% 105% Block RAMs 14% 53%

Estimated Latencies on TX240T FPGAs Total Estimated Latency Latency Estimates Estimated latencies are given in the table below Clock rate of 200MHz (cycle time of 5 ns) Cluster Overlap Filter operates in parallel with part of Particle Cluster Finder Estimated Latencies on TX240T FPGAs Component Latency (cycles) Latency (ns) Input RocketIO and Buffers 15 75 Particle Finder, Overlap Filter, Cluster Weighting 12 60 Output Rocket IO and Buffers 10 50 Total Estimated Latency 37 185 This slide shows the estimated latency of the RocketIO, Buffers, Particle Finder, Overlap Filter, and Cluster Weighting. With additional hardware, the Cluster Weighting can be performed in parallel with the Overlap filter to reduce the overall latency to 35 cycles (175 ns). It should also be feasible to reduce the latency of the RocketIO and Buffering compared what is shown in the table, since the latencies used are fairly pessimistic. However, even with optimizing RocketIO and Buffering, it is anticipated the RocketIO and buffering will still contribute significantly to the overall latency. Thus, an important areas for future investigation are to determine how we can best reduce the latency of the RocketIO and buffering and the number of times RocketIO and buffering are needed.

Calorimeter Trigger Prototype Combine the initial designs in a single FPGA RocketIO, buffering, particle cluster finder, cluster overlap filter, and cluster weighting Implement the rest of the Calorimeter Trigger Cluster Isolation and Particle ID Jet Reconstruction MET,HT,MHT Calculation – calculate Et sums Particle Sorter Perform more in-depth testing and analysis of all the designs Prototype the Calorimeter Trigger designs on FPGA hardware For next steps in this project, we plan to: (1) Combine the initial designs into a single FPGA for a complete M x N grid, where M and N can be varied. (2) Implement the rest of the Calorimeter Trigger firmware. We have already begun preliminary investigations of some of these modules. (a) The cluster isolation and particle ID will require adders with lots of inputs, lookup tables, and random logic. (b) The jet reconstruction will require adders with lots of inputs, weighted energy calculations, absolute differences, comparisons, and random logic. (c) The transverse energy sum calculations will also require adders with lots of inputs and the particle sorter will need to be able to sort a large number of particles and output the most energetic particles. (3) After the rest of the modules for the Calorimeter trigger have been developed, these modules will be combined, more in-depth testing will take place, and the entire design will be analyzed to determine potential optimizations. (4) Along the way, the Calorimeter Trigger designs will be prototyped on FPGA hardware and enhancements will be made to the designs, if neccessary.

SLHC-CT: Prototype Boards Data acquisition AUX card for GCT-uTCA crate for starter UW Engineer: Tom Gorski Two Grad Students for Test Firmware (Prof. Michael Schulte) Development Board & Sample Device Support from Xilinx

Aux Card in Test Fixture Single 12V Supply RS232 Interface to Terminal to control 4x4 switches TTS Output Tester (LED- based)

Test Firmware Hybrid of Microblaze soft core and custom IP blocks (HDL) for the interfaces User perspective: “C” program, communicate via RS232 terminal Core tests for hardware interfaces implemented in HDL blocks HDL Blocks for: GTPx, GTPr, TTC, TTS, S-Link, Clock Config Allows simultaneous testing at speed of all interfaces, using multiple pattern types

Possible SLHC RCT Mapping One Optical Fiber (GTX, 6+ Gb/s) replaces One Vitesse 7216 Link (4 copper links, 4.8 Gb/s combined) Double-size AMC modules (148.8mm x 181.5mm x 28.95 mm) Per RCT Input Card: 24 Links (12 HCAL, 12 ECAL) 8 Towers Per Link  96 H/E Tower-Pairs/Card 7 Input Cards/Crate Each Card covers 12φ x 8η Each Crate covers 12φ x 56η Need 6 “6U” crates to receive all H/E info. Room in existing RCT racks to support concurrently

Commitments to SLHC-CT University of Wisconsin Overall design, hardware, firmware, simulations University of Iowa Interface to Trigger Primitives Generators, Simulations Texas A&M University Simulations (especially tau for phase II) University of Minnesota, University of Maryland HCAL Trigger Primitives (in HCAL project) Princeton University Prototype hardware evaluation, Firmware development, TPG linearization Non-US participants on Trigger Projects Imperial College, London, UK, (Full calorimeter trigger) Bristol University, UK (Simulations primarily) LIP, Lisbon, Portugal (Trigger Primitives and DAQ, oSLB) Ecole Polytechnique, Palaiseau, France (ECAL Trigger Primitives) Commitments to SLHC-CT

SLHC-CT R&D Projects University of Wisconsin Prepare firmware for AUX card Procure and evaluate Matrix card and mTCA crate Develop overall design following the evolution constraints and FPGA firmware evaluation from EE team Design, build and evaluate prototype V5 or V6 based processor card Develop, synthesize, simulate and evaluate firmware in hardware Continue simulation code development and studies University of Iowa Develop software for interface to Trigger Primitives Generators Evaluate benefits of additional TPG info from HCAL Texas A&M University Continue tau simulations – rates for cross triggers with muons/electrons Track matching and isolation requirements … SLHC-CT R&D Projects

SLHC-CT Unassigned Projects Calorimeter information for Muon trigger In the current trigger calorimeter trigger provides MIP and Isolation bits to the muon trigger In the current HLT substantial reduction in rate at level-2 trigger is due to calorimeter isolation Most analyses (EWK) use calorimeter isolation Likely that isolation will degrade with increased pile-up However, it will still be used Improved position information from SLHC-CT could benefit isolation Someone should study this in simulation Specify what is needed Interface to Track trigger Simulation and specification of how calorimeter objects are combined with track information Track matching, Isolation, Primary vertex / luminous region reduction SLHC-CT Unassigned Projects

Summary SLHC Calorimeter Trigger simulation is in place Early results indicate meet Phase I requirements of x2 reduction More studies and improvements on the way Algorithm threshold/cut optimization Jet reconstruction alternatives Lining up with Tracking Trigger Simulation Is improved position in SLHC-CT sufficient to get x10 reduction in rate for Phase II? Hardware & Firmware development Continuously verifying that algorithms are implementable within FPGA bandwidth and processing constraints Improved methodologies for tracking firmware changes Prototypes with new FPGAs in mTCA formfactor being built We are preparing for CD0 review; We welcome new groups