1. A Time-Multiplexed Track-Trigger for the CMS HL-LHC upgrade
28 May 2015
Geoff Hall
Imperial College London
NB here describing R&D on (mainly) one option under study for how to process tracker data for the L1 trigger
for that option, work carried out by UK groups in CMS

2. Silicon tracker with trigger-stub capability
see comprehensive talk by Fabio Ravera
J Mans
ECFA 2014
Silicon tracker with trigger-stub capability
Strip/Strip Modules
90 µm pitch/5 cm length
Strip/Pixel Modules
100 µm pitch/2.5 cm length 100 µm x 1.5 mm “macropixels”
Inner Pixel
Covers up to η=4.0
Geoff Hall
Elba 2015

3. Motives and design of future CMS tracker
Tracker replacement essential for Run 4 (post-2025)
because of radiation damage and high pileup
LS months
Trigger must be substantially upgraded to handle high pileup
-> Linst ~ 5 x 1034 cm-2s-1 (levelled) => <Nev> ~ 140 – 200
Calorimeter issues
isolation of e/g/t degraded by pile-up from p0 gs and hadrons
many more jets, which overlap
Muon system issues
increased combinatorial fakes, enhanced by multiple scattering
To control much higher rate of L1 triggers only significant new data comes from tracker
Geoff Hall
Elba 2015

4. Stacked-tracker principle
Compare pattern of hits in contiguous sensor elements in closely spaced layers
pT cut set by angle of track in layer
primarily depends on layer separation
but increasing separation worsens fake combinations
details depend on
pitch
thickness
charge sharing
track impact point …
EXP. pT cut: GeV/c
FIT: pT cut: GeV/c
σ: GeV/c
data from 2013 beam test
assumes r=75cm layer
Geoff Hall
Elba 2015

5. CMS Tracker ASIC evolution
RAL TD-Imperial
1999: APV µm 2011: CBC 0.13µm 2013: CBC2 0.13µm
7 mm x 8mm (128 chan) 7mm x 4mm (128 chan) 11mm x 5mm (254 chan)
pipeline
128x192
128 x preamp/shaper
APSP + 128:1 MUX
pipe logic
bias gen.
CAL
FIFO
control
logic
pipeline 128 x 256
pipeline 254 x 256
analogue data
~4 µs latency
binary data,
6.4 µs latency
wire-bondable
2015: CBC3 (final – in layout)
up to 12.8 µs latency
(512 bx)
bump-bondable,
cluster & correlation logic
Geoff Hall
Elba 2015

6. Time Multiplexed Trigger
Multiple sources send to single destination for complete event processing
as used, e.g., in CMS High Level Trigger
Requires two layers with passive switching network between them
“simple” optical fibre network
data organisation and formatting at Layer 1, event processing at Layer 2
illustration on next slide
Now possible (in CMS calo trigger from 2016) because of
large & powerful FPGAs, interfaced to
high speed and miniature packaged optical links
Geoff Hall
Elba 2015

7. New Trigger Architecture
Conventional Trigger
Time Multiplexed Trigger
Geoff Hall
Elba 2015

8. Current generation hardware: MP7
Next generation boards at prototype design phase
Current generation hardware: MP7 IN
72 x 12.5 Gbps = 0.9 Tbps
OUT
~80W
Virtex-7
Geoff Hall
Elba 2015

9. Advantages of TMT
“All” the data arrive at a single place for processing
in ideal case avoids boundaries and sharing between processors
however, does not preclude sub-division of detector into regions
Architecture is naturally matched to FPGA processing
parallel streams with pipelined steps at data link speed
Single type of processor, possibly for both layers
L1= PP: Pre-Processor L2 = MP: Main Processor
One or two nodes can validate an entire trigger
spare nodes can be used for redundancy, or algorithm development
Many conventional algorithms explode in a large FPGA
timing constraints or routing congestion for 2D algorithms
Synchronisation is required only in a single node
not across entire trigger
Geoff Hall
Elba 2015

10. The track-trigger challenge
Impossible to transfer all data off-detector for decision logic so on-detector data reduction (or selective readout) essential
but 99% tracks < 2 GeV/c
What track information is needed for the
L1 trigger?
original assumption: a few points might help
but simulations did not show big rate reductions
hence quasi-full track reconstruction for pT >~3 GeV/
How to find the tracks in ~5µs with high efficiency and acceptable fake rates?
large fraction of hits are not track-related (conversions, secondaries,…)
Geoff Hall
Elba 2015

11. Potential solutions Associative Memory + FPGA FPGA only
AM stores track patterns as templates
FPGA fits for final parameters
Used in CDF at Level-2. New architecture required for CMS [International]
FPGA only
Programmable device ( –> flexibility & technology evolution) – but can it be done with so many hits?
(i) tracklet – “conventional” track finding [US]
(ii) Hough transform [UK]
How to tell which will work best (or at all)? The architectural design may be very important
not simple to transform software algorithms to firmware
demonstrators needed to prove each concept
Geoff Hall
Elba 2015

12. Pattern matching with Associative Memories
...
The Pattern
Bank
The pattern bank is flexible
set of pre-calculated patterns:
can account for misalignment
changing detector conditions
beam movement …
The Event

13. few hundred stubs/tower
Send data to Pattern Recognition Mezzanine in each ATCA blade
Data distributed to Pulsar boards in time multiplexed mode in round robin
Perform pattern recognition using several AM chips (120k Patterns/AM chip)
Track fit with FPGA inside the Mezzanine (~1 ns/fit)

14. See poster by G. Fedi et al (INFN)

15. Layout of fully Time-Multiplexed Track-Trigger
Focusing on demonstration of the concept
entire tracker could be read out by MP7-like processing cards
requires ~150 MP cards, segmented into 5 h regions
module sharing
≤ 2 regions
simpler architecture
no deghosting
sharing defined by large luminous region in z
feed time-multiplexed data to regional processors
TM period of BX possible using MP7s
Geoff Hall
Elba 2015

16. see poster by Davide Cieri
Track-finding in FPGA
Hardware requirements already feasible, but processing in FPGA very challenging
exploring Hough transform approach:
line in real space -> point in inverse space
pipelined dataflow
natural with TM
matches FPGA needs
find stubs in 2D
2D histogram
selection to reduce number of candidates
valid track where lines intersect
i.e. stubs which share the same (m,c)
Geoff Hall
Elba 2015

17. Status of demonstrator
Hardware exists in working form (from calo trigger)
adapt for track-trigger time slice
software validation under way
Firmware implementation of Hough array ready
self-filling systolic array
integrating with infrastructure firmware
Geoff Hall
Elba 2015

18. Summary The TMT is now a proven architecture in CMS
will by used in the CMS calorimeter trigger from 2016
Existing hardware is very flexible and can be deployed for a TMTT
only a fraction of the system is required to validate the concept
installing and building should only require replicating identical nodes
a track-trigger could be built using present technology
safe to assume further technological progress in next decade
The TMT challenge is to find feasible algorithms
which is more effective and affordable? FPGAs or AM+FPGAs?
Geoff Hall
Elba 2015

19. Backup
Geoff Hall
Elba 2015

20. even simpler demonstrator
2 MP7s emulate event data from 1 out of 5 regions, one out of every 24BX
# emulated PP/FEDs 46 46
# PP->MP links 18 46
This demonstrator already exists
just need to program source data to be ready to try algorithms
# PP links/ MP 64
# PP->MP links total 64
# TM nodes 1 φ1 φ2 φ3 φ4 φ5
Geoff Hall
Elba 2015

21. Time-multiplexing 5 1 5 1 5 1 1 5
All data for 1bx from all regions in a single card!
Everything you need! 2 6 2 6 2 6 2 6 1 3 7 3 7 3 7 3 7 4 4 4 4 5 1 1 5 1 5 1 5
All data for 1bx from all regions in a single card!
Everything you need! 2 2 6 2 6 2 6 2 6 3 7 3 7 3 7 3 7 4 4 4 4 1 5 5 1 1 5 5 1
All data for 1bx from all regions in a single card!
Everything you need! 2 6 2 6 6 2 2 6 3 7 3 7 7 3 3 7 4 4 4 4 1 5 1 5 5 1 1 5 2 6 2 6 2 6 2 6 3 7 3 7 3 7 3 7 4 4 4 4
BX:4
BX:5
BX:6
BX:7
BX:3
BX:2
BX:1
Geoff Hall
Elba 2015 21

22. Why tracker input to L1 trigger?
Single µ and e L1 trigger rates will greatly exceed 100kHz
similar behaviour for jets
Single electron trigger rate
<pT> ≈ few GeV/bx/trigger tower
Isolation criteria alone are insufficient to reduce rate at L= 1035 cm-2.s-1
L = 2x1033
L = 1034 muon L1 trigger rate
1035
Geoff Hall
Elba 2015

23. CMS Outer Tracker trigger
~15000 modules transmitting
pT-stubs to L1 40 MHz
full hit data to MHz
EXP. pT cut: GeV/c
FIT: pT cut: GeV/c
σ: GeV/c
~8400 2S-modules
reconstructed pT cut of r=75cm layer
Geoff Hall
Elba 2015

24. time multiplex period
the time multiplex period is not a completely free parameter
small TM period
large TM period
full event must be quickly assembled into one MP
could allow more efficient processing of pipelined data into MP
reduces data volume per event from PP to MP
(or requires increased number of links)
increases data volume per event from PP to MP
(or reduces number of links)
reduces latency
increases latency
reduces number of MPs
increases number of MPs
min ~15bx
(PP output bandwidth without more Trigger Regions)
max ~34bx
(68 links/2 Trigger Regions)
preferred direction
TM period of 24BX chosen for case study (could be optimised in future)
Geoff Hall
Elba 2015