Overview of the ATLAS Fast Tracker (FTK) (daughter of the very successful CDF SVT) July 24, 2008 M. Shochet

What is it for? At the LHC design accelerator intensity: New phenomena:  0.05 Hz Total interaction rate:  1 GHz (40 MHz beam crossings) Many possible new phenomena produce heavy b quarks which can only be distinguished from the bulk of the background by reconstructing the individual tracks. We are proposing to significantly enhance the ability of ATLAS to rapidly identify b quarks in the trigger. Currently done in commodity PC’s. This is slow and becomes slower as the accelerator intensity and thus track density increase. The problem! beam pipe few mm July 24, 2008 M. Shochet

ATLAS July 24, 2008 M. Shochet

Inner Tracking Detectors Pixels barrel SCT barrel Pixels disks 3 pixel layers (space point) 8 strip layers (1 coordinate)  11 layers, 14 coordinates July 24, 2008 M. Shochet

Getting data into FTK on L1 accept RODs SCT  Pixels  FTK dual-output HOLA designed by Tang ROBs ROBs silicon hits silicon tracks ask for ROI’s Level 2 July 24, 2008 M. Shochet

Number of input fibers Number of crates Pixels: 120 Strips: 92 Number of crates  12 July 24, 2008 M. Shochet

How does it work? First do pattern recognition, then fit possible candidates. Prestore patterns (roads) in large content-addressable memory. coarse resolution hits full resolution hits (superbins) The Pattern Bank (4-layer) July 24, 2008 M. Shochet

Massively parallel pattern recognition track fitter 1 superbin per silicon layer Majority logic allows up to 1 missed layer. July 24, 2008 M. Shochet

Functional layout RODs ~Offline quality Track parameters Pixels & SCT PIPELINED AM overlap regions RODs EVENT # 1 EVENT # N 50~100 KHz event rate HITS (LVDS links) AM-board Data Formatter (DF) DO-board S-links SUPER BINS DATA ORGANIZER ROADS cluster finding split by layer ROADS + HITS Track Fitter ~Offline quality Track parameters Raw data ROBINs Track data ROBIN July 24, 2008 M. Shochet

Possible layout for a core crate (after DFs) AM-B7 AM-B8 AM-B1 AM-B0 DO5 DO4 DO3 DO2 DO1 DO0 Track Fitter AM-B2 AM-B3 AM-B4 AM-B5 AM-B6 AM-B9 AM-B10 AM-B11 AM-B12 July 24, 2008 M. Shochet

Data Formatter Receives raw hits from the detector (RODs) Finds hit clusters pixels silicon strips Store cluster centroids Separates clusters by silicon layer & sends to Data Organizers on 6 LVDS data busses (22 bits each) July 24, 2008 M. Shochet

Data Organizer Receives hits from Data Formatters. Stores hits at full resolution in a way that is rapidly accessible by pattern number. Sends hits at coarser resolution (superbin) to pattern recognition unit (Associative Memory). Receives patterns from AM, retrieves full resolution hits, and sends road number and hits to the Track Fitter. July 24, 2008 M. Shochet

Track Fitter Receives road # and associated hits from Data Organizers. Computes all hit combinations Calculate the track parameters curvature, azimuthal angle, polar angle, z0, impact parameter and the goodness of fit (2) using a linear correction to the mean for that sector (excellent precision over a sector). sector: a physical silicon module in each layer pixels: 1" x 2.5 " strips: 2.5" x 5 " July 24, 2008 M. Shochet

Output good tracks to ROBIN. 14 measurements, 5 parameters  9 constraints (2) Pi: 5 track parameters & 9 constraints (2 is sum of squares) xj: the hit coordinate in layer j aij, bi: the stored constants for each sector, calculated in advance from a large sample of training tracks (simulation or data) Cut on goodness of fit; among the combinations in a road, select the track with the best 2. Output good tracks to ROBIN. July 24, 2008 M. Shochet

Readout Buffer (ROBIN) Stores tracks for access by the level-2 trigger PCs. July 24, 2008 M. Shochet

Track Fitter details GigaFitter – a simplified version built for the CDF SVT 2D reconstruction (transverse to the beamline) 6 detector layers 3 track parameters (curvature, azimuthal angle, impact parameter) 3 constraints  2 July 24, 2008 M. Shochet

GigaFitter scheme Comb - FiFo Lay0-Ram Lay1- Ram Lay2- Ram Lay3-Ram ... Lay10-Ram Comb - FiFo DSP: Fit Tracks Choose best χ2 track INPUT FiFo Constants RAM DO roads & hits  input FIFO  RAMs according to detector layer Combinations (one hit/layer) calculated sequentially & stored in Comb-FIFO Each combination & the constants are sent to DSP for fitting & selection July 24, 2008 M. Shochet

DSP algorithm for SVT C1 18 39 ACC 39 FIFO Hit 18 C2 18 39 ACC 39 156 18 39 ACC C3 39 39 ACC 18 39 C4 constants RST DSP48E CTRL EV READY RST Comb-FIFO data serialized: 1 hit and its constants sent to parallel DSP slices each computing a track parameter or constraint. An additional DSP computes total 2 from individual constraints Total of 7 DSP slices in parallel each working at 200 MHz July 24, 2008 M. Shochet

DSP Slice 18 39 One DSP slice computes a track parameter in 14 clock cycles, plus 4 for the first one. July 24, 2008 M. Shochet

Xilinx XC5VSX95T 14720 V5 slices (4 LUT + 2 FF) 1% used for each fitter 1520 kbit of distributed RAM 640 DSPs 2.5% used for each fitter in the FTK version 5 parameters, 9 constraints, 1 to calculate overall 2   40 fitters/chip (remember: many combinations per road) 1 Mbyte of block RAM plenty for the SVT prototype not even close for the FTK!! July 24, 2008 M. Shochet

Missing hit problem To obtain high reconstruction efficiency, we must allow one physical detector layer to miss a hit. The constants in the parameter and constraint equations are different when there is a missing hit. Store 12 sets of constants (all hits, a miss in one of 11 layers). A lot of memory that has to be accessed very quickly. Can one look ahead for the constant set that will be needed next? Alternative is to estimate the hit location in the missing layer. How long does it take? July 24, 2008 M. Shochet

Size of the constant memory 210 words/constant set (1414 + 14) If we need 2-byte precision  420 bytes/constant set One constant set/sector. Currently estimate 100k sectors in an FTK crate.  42 Mbytes of fast memory  0.5 Gbyte if solve missing hit problem with more memory We have heard that with the latest FPGAs, there is very fast access to external computer-like memory. Is that true? July 24, 2008 M. Shochet

How many fitters are needed? The number of cycles from the time the data from a road is in the input FIFO until the track parameters are in the output FIFO is approximately: NfitCycl = Ncomb  Nhits (all 14 calculations done in parallel) A road packet takes Nhits + 1 Data Organizer clock cycles to be sent to the Track Fitter. Thus if we are to have 0 deadtime from track fitting, we need the number of parallel fitters (there are 40 in a chip), Nfitters, satisfying: This translates into a maximum average Ncomb of Nfitters  ClockRatio  (Nhits + 1)/Nhits where ClockRatio is the ratio of the fitter to DO clock speeds. We will have to satisfy this: road width, # of FPGAs. July 24, 2008 M. Shochet

Hit Warrior function One can easily get many tracks (ghosts) from a single real particle due to presence of extra random silicon hits. Compare a new track with those already found. If new one has 8 or more hits in common with a stored track, keep only the best 2 track. If space permits, add this function to the Track Fitter. July 24, 2008 M. Shochet

Spy Buffer Data flows very quickly through this system. By the time any PC using its output detects a problem, the data would already be long gone from the FTK. That makes diagnosing a problem that is occurring internally in the FTK extremely difficult. We found it very useful in the SVT to have deep buffers at the input and output of every board in the system. Then, when a problem is detected, these spy buffers can be frozen in the entire system and read out. The buffer is deep enough so the event with the error is still inside it. This allows diagnosing and fixing subtle problems. July 24, 2008 M. Shochet