1. SLHC Calorimeter Trigger
S.Dasu University of Wisconsin
08 May 2009
test

2. Introduction SLHC Calorimeter Trigger (SLHC-CT) Goals
Enable continuation of EWSB physics studies at SLHC
Retain similar thresholds as LHC: x2 reduction in rate for Phase I and x10 for Phase II
Single & double electrons and photons: GeV
Single & double taus: GeV (not Ztt but for Htt )
Jets & MET: Provide better energy and position resolution
SLHC Calorimeter Trigger (SLHC-CT) Upgrade Strategy
Improved algorithms, more multi-object triggers & improved topological conditions for Phase I
Track matching for Phase II
High spatial resolution of calorimeter trigger objects so that tracks can be matched closely
Track matching and track Isolation will resolve electrons from photons , taus from jets

3. SLHC-CT Upgrade Constraints
Input to calorimeter trigger
No changes to ECAL TPGs
EE may change, but only in Phase II
HCAL TPGs can change in Phase I if needed
To be defined
Output of calorimeter trigger
Work closely with global trigger systems to evolve together
Upgrade Commissioning
Seamless transition to new trigger hardware such that LHC operations are not affected
Commission alongside current trigger
Ability to operate with old and new systems at the same time is needed, e.g., old EB and new HB/HE/HF
Transition to upgraded trigger when validated

4. Calorimeter Trigger Evolution
Step 1 (2009)
Step 2: ↓ OR ↓
Step 3
Step 4
ETCC: TPGs
HTR: TPGs
ETCC: TPGs
HTR: TPGs
uTCA- HTR: TPG
ETCC: TPGs
uTCA-HTR: TPGs
RCT/
uTCA
GCT/ uTCA
GT/GMT
ETCC: TPGs
uTCA-HTR: TPGs
oSLB
oSLB
oSLB
SLB
SLB
SLB
oSLB
oSLB
oSLB
RCT
RMC
RMC
RCT
RMC
oSLB
oSLB
RCT
oSLB
oSLB
RCT/
uTCA
Matrix & Aux Cards  Cu
GCT: Sources
GCT/ uTCA
GCT: Sources
GCT/ uTCA
GCT: Sources
GCT/ uTCA FO
GCT: Main
GT/GMT
GT/GMT
GT/GMT

5. SHLC-CT Upgrade Studies
Simulation Program
Developed SLHC trigger algorithm simulation program to work within CMSSW
Compare LHC and SLHC algorithm performance
Interface with track trigger simulation program
Studies for Phase II of upgrade
Firmware & Hardware Program
Firmware development in collaboration with experts from Wisconsin and Maryland EE departments
Feasibility of implementing new algorithms
Systematic improvements in methodologies
Prototyping of mTCA cards with new FPGAs began with last portions of LHC calorimeter trigger

6. Optical Serial Link Board (oSLB)
oSLB (TX and RX)
Responsibility of Jose Carlos da Silva (LIP, Portugal)
Maintain same connectivity and form factor as current SLB
Replace cables by optical links
Latency budget increases in most cases when using FPGAs
High speed link, 850/1310 nm, commercial package
PCB for both ECAL/HCAL and RCT
FPGA Transceivers, deeper FIFO and Full Orbit histogram per channel
Stratix II or IV (GX) or Virtex 4 or 5
Future options / reasons for upgrade

7. Calorimeter signatures
HCAL η
Δη x Δφ=0.087x0.087
Electrons/Photons
Spatially confined in a cluster of 2x2 trigger towers
Significantly higher ECAL contribution
Isolated e/γ should have low energy deposits in the surrounding area
ECAL φ
e/γ
HCAL η
Taus
Confined in 2x3 Clusters
3 prongs/1 prong + π0s have wider φ profile
Small energy leak in surrounding towers
ECAL φ τ η
HCAL
Jets
Most of the energy confined in a central core
For jets over 20 Gev the energy is included in a 8x8 region
ECAL φ
jet

8. Algorithm Overview Particle Cluster Finder Cluster Overlap Filter
Takes Calorimeter TPG input and applies tower thresholds
Creates overlapped 2x2 clusters
Cluster Overlap Filter
Removes overlap between clusters
Identifies local maxima
Prunes low energy clusters
Cluster Isolation and Particle ID
Applied to local maxima
Calculates isolation deposits around 2x2,2x3 clusters
Identifies particles
Jet reconstruction
Applied on filtered clusters
Groups clusters to jets
Particle Sorter
Sorts particles and outputs the most energetic ones
MET,HT,MHT Calculation
Calculates Et Sums, Missing Et from clusters

9. SLHC-CT Simulation Results: Factor of 2 for Phase I
Michalis Bachtis & Kevin Flood
Isolated electrons
Taus
QCD Rate (kHz)
QCD Rate (kHz)
Factor of 2 rate reduction
Isolated electrons
Taus
Efficiency
Efficiency
Higher Efficiency

10. Improved Position Resolution
Achieved better than the target half-tower resolution (0.044)

11. Jet Trigger Comparison
SLHC
LHC
LHC
SLHC

12. Algorithm implementation (Firmware)
M.Schulte, K.Compton, B.Buchli, A. F-Farmahani, T.Gregerson, S.Naumov (ECE Department/UW)‏
Now Implementing and optimizing the algorithms
Implementation of thresholds and pattern comparator
Comparison of LUT, Division implementation for electron ID
Cluster filtering, jet finding …
Shruva Bhattacharya and Will Plishker, Maryland
FPGA Firmware design methodology
Development management tools
Algorithm implementation (Firmware)

13. Initial Design Platform
Xilinx Virtex-5 devices contain
Virtex-5 Slices (4 LUTs and 4 flip-flops)
DSP48E Slices (multiplier, adder, and accumulator)
Block RAMs (36 Kbits)
RocketIO Transceivers
GTP transfers up to 3.75 Gbps/link
GTX transfers up to 6.50 Gbps/link
Initial designs synthesized for
Xilinx Virtex-5 LX110T and TX240T FPGAs
For the initial design platform, we have been using Xilinx Virtex-5 FPGAs and Xilinx Design tools. It is important to note that the RocketIO Transceivers are full-duplex bi-direction, which means that the TX240T device has 48 RocketIO serial inputs and 48 RocketIO serial outputs.
Compared to the LX110T device, the TX240T device has roughly twice as many Virtex-5 slices and Block RAM bits, 50% more DSP48E Slices, and up to six times more IO bandwidth.
FPGA
Virtex-5 Slices
DSP48E
Slices
Block RAM
(Kbits)
RocketIO
Transceivers
LX110T
17,280 64
5,328
16 GTP
TX240T
37,440 96
11,664
48 GTX

14. FPGA Firmware Results
Virtex-5 Resource Utilization for Input and Output RocketIO and Buffering on the TX240T FPGA
Resource
8 x 8 Grid
8 x 16 Grid
16 x 16 Grid
RocketIO Input Links
25%
46%
83%
Virtex-5 Slices 5%
10%
19%
Resource utilization for Particle Cluster Finder theTX240T FPGA
Resource
8 x 8 Grid
8 x 16 Grid
16 x 16 Grid
Virtex-5 Slices
10%
20%
39%
BRAMs
14%
27%
53%
Virtex-5 Slice Utilization for Cluster Overlap Filter
FPGA
8 x 8 Grid
8 x 16 Grid
16 x 16 Grid
LX110T
14%
29%
58%
TX240T 7%
27%

15. Overall Resource Estimates
Estimated resources are given in the table below
Includes RocketIO, buffers, particle cluster finder, overlap filter, and cluster weighting
Additional grid sizes and FPGA devices should be considered
Overall Resource Utilization on TX240T FPGA
This slide shows the overall estimated resource utilization for input and output RocketIO and buffers, particle cluster finder, overlap filter, and cluster weighting.
If we implement an 8 x 8 grid of clusters, we have plenty of resources left and should be able to implement additional functionality on the same FPGA.
If we implement an 8 x 16 grid of clusters, then we use just over half of the Virtex-5 slices and just under half of the RocketIO links. This should still allow fairly significant upgrades to be made to the algorithms.
It does not seem like a good idea to implement a 16 x 16 grid of clusters on a single Virtex-5 TX240T FPGA, since this could impact upgradability. However, since high-end Virtex-6 FPGAs will have a larger number of Virtex Slices and RocketIO links, it may be feasible to implement a 16 x 16 grid on Virtex-6 FPGAs.
If each FPGA implements a 8 x 16 cluster grid, then this portion of the Calorimeter trigger would require roughly 36 FPGAs to implement to process the 2-dimensional array of 4096 towers.
Resource
8 x 8 Grid
8 x 16 Grid
16x 16 Grid
RocketIO Links
25%
46%
83%
Virtex-5 Slices
27%
54%
105%
Block RAMs
14%
53%

16. Estimated Latencies on TX240T FPGAs Total Estimated Latency
Latency Estimates
Estimated latencies are given in the table below
Clock rate of 200MHz (cycle time of 5 ns)
Cluster Overlap Filter operates in parallel with part of Particle Cluster Finder
Estimated Latencies on TX240T FPGAs
Component
Latency (cycles)
Latency (ns)
Input RocketIO and Buffers 15 75
Particle Finder, Overlap Filter, Cluster Weighting 12 60
Output Rocket IO and Buffers 10 50
Total Estimated Latency 37
185
This slide shows the estimated latency of the RocketIO, Buffers, Particle Finder, Overlap Filter, and Cluster Weighting.
With additional hardware, the Cluster Weighting can be performed in parallel with the Overlap filter to reduce the overall latency to 35 cycles (175 ns).
It should also be feasible to reduce the latency of the RocketIO and Buffering compared what is shown in the table, since the latencies used are fairly pessimistic. However, even with optimizing RocketIO and Buffering, it is anticipated the RocketIO and buffering will still contribute significantly to the overall latency. Thus, an important areas for future investigation are to determine how we can best reduce the latency of the RocketIO and buffering and the number of times RocketIO and buffering are needed.

17. Calorimeter Trigger Prototype
Combine the initial designs in a single FPGA
RocketIO, buffering, particle cluster finder, cluster overlap filter, and cluster weighting
Implement the rest of the Calorimeter Trigger
Cluster Isolation and Particle ID
Jet Reconstruction
MET,HT,MHT Calculation – calculate Et sums
Particle Sorter
Perform more in-depth testing and analysis of all the designs
Prototype the Calorimeter Trigger designs on FPGA hardware
For next steps in this project, we plan to:
(1) Combine the initial designs into a single FPGA for a complete M x N grid, where M and N can be varied.
(2) Implement the rest of the Calorimeter Trigger firmware. We have already begun preliminary investigations of some of these modules.
(a) The cluster isolation and particle ID will require adders with lots of inputs, lookup tables, and random logic.
(b) The jet reconstruction will require adders with lots of inputs, weighted energy calculations, absolute differences, comparisons, and random logic.
(c) The transverse energy sum calculations will also require adders with lots of inputs and the particle sorter will need to be able to sort a large number of particles and output the most energetic particles.
(3) After the rest of the modules for the Calorimeter trigger have been developed, these modules will be combined, more in-depth testing will take place, and the entire design will be analyzed to determine potential optimizations.
(4) Along the way, the Calorimeter Trigger designs will be prototyped on FPGA hardware and enhancements will be made to the designs, if neccessary.

18. SLHC-CT: Prototype Boards
Data acquisition AUX card for GCT-uTCA crate for starter
UW Engineer: Tom Gorski
Two Grad Students for Test Firmware (Prof. Michael Schulte)
Development Board & Sample Device Support from Xilinx

19. Aux Card in Test Fixture
Single 12V Supply
RS232 Interface to Terminal to control 4x4 switches
TTS Output Tester (LED- based)

20. Test Firmware
Hybrid of Microblaze soft core and custom IP blocks (HDL) for the interfaces
User perspective: “C” program, communicate via RS232 terminal
Core tests for hardware interfaces implemented in HDL blocks
HDL Blocks for: GTPx, GTPr, TTC, TTS, S-Link, Clock Config
Allows simultaneous testing at speed of all interfaces, using multiple pattern types

21. Possible SLHC RCT Mapping
One Optical Fiber (GTX, 6+ Gb/s) replaces One Vitesse Link (4 copper links, 4.8 Gb/s combined)
Double-size AMC modules (148.8mm x 181.5mm x mm)
Per RCT Input Card:
24 Links (12 HCAL, 12 ECAL)
8 Towers Per Link  96 H/E Tower-Pairs/Card
7 Input Cards/Crate
Each Card covers 12φ x 8η
Each Crate covers 12φ x 56η
Need 6 “6U” crates to receive all H/E info.
Room in existing RCT racks to support concurrently

22. Commitments to SLHC-CT
University of Wisconsin
Overall design, hardware, firmware, simulations
University of Iowa
Interface to Trigger Primitives Generators, Simulations
Texas A&M University
Simulations (especially tau for phase II)
University of Minnesota, University of Maryland
HCAL Trigger Primitives (in HCAL project)
Princeton University
Prototype hardware evaluation, Firmware development, TPG linearization
Non-US participants on Trigger Projects
Imperial College, London, UK, (Full calorimeter trigger)
Bristol University, UK (Simulations primarily)
LIP, Lisbon, Portugal (Trigger Primitives and DAQ, oSLB)
Ecole Polytechnique, Palaiseau, France (ECAL Trigger Primitives)
Commitments to SLHC-CT

23. SLHC-CT R&D Projects University of Wisconsin
Prepare firmware for AUX card
Procure and evaluate Matrix card and mTCA crate
Develop overall design following the evolution constraints and FPGA firmware evaluation from EE team
Design, build and evaluate prototype V5 or V6 based processor card
Develop, synthesize, simulate and evaluate firmware in hardware
Continue simulation code development and studies
University of Iowa
Develop software for interface to Trigger Primitives Generators
Evaluate benefits of additional TPG info from HCAL
Texas A&M University
Continue tau simulations – rates for cross triggers with muons/electrons
Track matching and isolation requirements …
SLHC-CT R&D Projects

24. SLHC-CT Unassigned Projects
Calorimeter information for Muon trigger
In the current trigger calorimeter trigger provides MIP and Isolation bits to the muon trigger
In the current HLT substantial reduction in rate at level-2 trigger is due to calorimeter isolation
Most analyses (EWK) use calorimeter isolation
Likely that isolation will degrade with increased pile-up
However, it will still be used
Improved position information from SLHC-CT could benefit isolation
Someone should study this in simulation
Specify what is needed
Interface to Track trigger
Simulation and specification of how calorimeter objects are combined with track information
Track matching, Isolation, Primary vertex / luminous region reduction
SLHC-CT Unassigned Projects

25. Summary SLHC Calorimeter Trigger simulation is in place
Early results indicate meet Phase I requirements of x2 reduction
More studies and improvements on the way
Algorithm threshold/cut optimization
Jet reconstruction alternatives
Lining up with Tracking Trigger Simulation
Is improved position in SLHC-CT sufficient to get x10 reduction in rate for Phase II?
Hardware & Firmware development
Continuously verifying that algorithms are implementable within FPGA bandwidth and processing constraints
Improved methodologies for tracking firmware changes
Prototypes with new FPGAs in mTCA formfactor being built
We are preparing for CD0 review; We welcome new groups