A Compact Trigger Architecture for SLHC Many thanks to all that I have questioned over the last few months, particularly Jad, Magnus, John and Costas 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Trigger Requirements Task Build physics objects cluster electrons, taus, jets calculate quantities such as total/missing-et, miss-ht Sort in order of rank Identify physics topologies Must not miss interesting events Can be crude: just a trigger! Challenge How best to map task onto physical geometry? Part 1: What type of baseline hardware topology/architecture Part 2: What type of processing card Part 3: What services / infrastructure required? 22 July 2009 A Trigger Platform for SLHC

Geometry of RCT ECAL: 8bits energy + FGV (FineGrainVeto) HCAL: 8bits energy + MIP (MinIonisingParticle) Tower Region Phi Neg Eta Pos Eta 1 2 3 4 5 6 7 8 9 10 Electron or Tau Jet Region Number / Eta Index x18 Non Tau Jet Constant phi strip Phi 22 July 2009 A Trigger Platform for SLHC

Geometry & link capacity B B B B B E E E HF F F F F HF HF HF Region Number of Regions: 22 (eta) x 18 (phi) 1 Region = 4x4 Towers 1 Link at 4.8Gb/s (8B/10B) 12bytes per bx Hence 1 region = 2 links (12bits) 1 Link at 6.4Gb/s (8B/10B) 16bytes per bx Hence 1 region = 2 links (16bits) Tower 22 July 2009 A Trigger Platform for SLHC

Original idea (i.e. duplicate the data to provide boundary conditions) Apply jet + electron processing on 1x1 region. Requires 1 boundary region to contain jets. Data is duplicated x9 times Expensive... Consequence: Very large, expensive system of between 300 to 500 cards 22 July 2009 A Trigger Platform for SLHC

An alternative approach... Concentrate on electron and tau jet processing initially Small area – typically < 2x2 towers Might be able to use for subsequent jet clustering Allow capability to explore different algorithms See “Calorimeter Trigger for Phase I”, M.Bachtis et al., Wisconsin Pre cluster method used in GCT Constant phi strip 1 2 3 4 5 6 7 8 9 10 Regions 0 to 2 Regions 3 to 6 Region Number / Eta Index 22 July 2009 A Trigger Platform for SLHC

Encap (mostly) : Regions 3 to 6 Elec, Gamma & Tau Barrel: Regions 0 to 2 Encap (mostly) : Regions 3 to 6 20 e/h input links + 2 track trigger 22 e/h input links + 2 track trigger Eta0 Eta0 Eta0 Eta0 4xSERDES or 32x LVDS (e.g. 16 x 24bit objects) 4xSERDES or 32x LVDS (e.g. 16 x 24bit objects) Built in capacity for Track Trigger 2+2 2+2 Sum 2x2 towers for and transmit on 2+2 fibres (i.e. 2 copies) 22 July 2009 A Trigger Platform for SLHC

Electron / Tau Processing Regions: -6 to -3 Regions: -2 to -0 Regions: +0 to +2 Regions: +3 to +6 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Jet Processing Regions: 0 to 2 Opposite eta region 0 Regions: 3 to 6 Regions: HF 8 jet input links 8 jet input links 8 jet input links 22 July 2009 A Trigger Platform for SLHC

System view (uTCA crates) Electron Cluster Reg -6 to -3, Phi 0-8 Electron Cluster Reg -6 to -3, Phi 0-8 Electron Cluster Reg +0 to +2, Phi 0-8 Electron Cluster Reg +3 to +6, Phi 0-8 Electron Cluster Reg -6 to -3, Phi 9-15 Electron Cluster Reg -6 to -3, Phi 9-15 Electron Cluster Reg +0 to +2, Phi 9-15 Electron Cluster Reg +3 to +6, Phi 9-15 HF HF Jet Cluster Neg Eta 9 cards Jet Cluster Pos Eta 9 cards Elec and Jet Sort Elec: 4 cards, Jet: 2 cards 16 IsoElec 16 NonIsoElec, 16 TauJets, 8 ForwardJets, 8 CentralJets, Total/Missing Et/Ht, Etc Possible to use the same card for entire system Global Trigger 22 July 2009 A Trigger Platform for SLHC

Latency According to TDR RCT latency = 20bx Does this include Vitesse link sync? RCT-GCT cable delay = 4bx GCT to GT including GT link = 15bx Hence total = 39bx ~5bx 4x18=72 HF Elec PreCluster Jet 2x2 TowerSum Elec Cluster ~10bx 2x9=18 Jet Cluster ~10bx 4 2 Elec Sort Jet Sort ~10bx Optional 16->4 Sort in OGTI or New GT 22 July 2009 A Trigger Platform for SLHC

Overview ot main features Part 2: Processing Card Overview ot main features 22 July 2009 A Trigger Platform for SLHC

Virtex 5: Mini-T5 TX150T or TX240T Prices for single parts. XC5VTX150T-2FFG1759C = $5.1k QSFP XC5VTX240T-2FFG1759C = $9.7k QSFP 4 4 TX150T or TX240T PPOD-OUT 12 12 PPOD-IN 8 8 4 backplane PPOD-IN Cost per card ~ $10k Full system requires ~ 1-2 m$ -1 Max speed 4.25Gb/s -2 Max speed 5.0Gb/s 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC I/O Capabilities Primay input (2 x PPODs) AvagoTech AFBR-786 (12x6.25 Gb/s) AvagoTech AFBR-820 (12x10.3 Gb/s) Primay output (1 x PPOD) AFBR-776 (12x6.25 Gb/s) AFBR-810 (12x10.3 Gb/s) Primary sharing (2 x QSFP) AFBR-79Q4Z (BiDir, 4x10 Gb/s) AFBR-79Q5Z (BiDir, 4x5 Gb/s) Avoids crate to crate interconnect issues Auxillary I/O Samtec 2x40 LVDS interface (~Gb/s) (QTH/QSH series) Low latency inteconnect Connector on each side Flex cable available from Samtec Could also be used for daughter card Backplane 8 (Virtex5) or 4 (Virtex6) At least 1 used for Ethernet, Perhaps 1 for DAQ 10G Ready ! 22 July 2009 A Trigger Platform for SLHC

Clock Distribution V5 SMA uTCA Clk1 uTCA Clk3 CrossPoint Osc1 Osc2 Use Dynamic Reconfiguration Port within V5 to reroute clks. Already done this OGTI card. Max 3 jumps between GTPs / GTXs 2x CrossPoint 8 22 July 2009 A Trigger Platform for SLHC

Mini-T5: Current status 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Roadmap MINI-T6S XC5VTX150T XC5VTX240T XC6VLX550T Slices (k) 23 37 86 Logic Cells (k) 148 240 550 CLB FlipFlops (k) 92 150 687 Distributed RAM (kbits) 1,500 2,400 6,200 BRAM (36kbits) 228 324 632 XC6 @ 11.2G Upgrade links from 6.5Gb/s to 11.2Gb/s MINI-T6 XC6VLX550T 1FF1759C A single Virtex-6 FPGA CLB comprises two slices, with each containing four 6-input LUTs and eight Flip-Flops (twice the number found in a Virtex-4 FPGA slice), for a total of eight 6-LUTs and 16 Flip-Flops per CLB. Upgrade links from 5Gb/s to 6.5Gb/s A single Virtex-5 CLB comprises two slices, with each containing four 6-input LUTs and four Flip-Flops (twice the number found in a Virtex-4 slice), for a total of eight 6-LUTs and eight Flip-Flops per CLB MINI-T5 XC5VTX150T 2FFG1759C 22 July 2009 A Trigger Platform for SLHC

Still thinking about this Just some ideas... Part 3: Services Still thinking about this Just some ideas... 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Services Required Communication: GbE (SerDes) is the standard on uTCA DAQ data concentration Mainly on trigger systems CLK40, TTC (Chans A & B) & TTS Downstream systems may not need this. Switching GbE, PCIe or SRIO Protocol Agnostic 22 July 2009 A Trigger Platform for SLHC

How is MCH assembled Tongue 2: Complicated !! NonRedundant: - Tx AMC Clk 1 and Clk3 - Rx AMC Clk2 - Fabric B to AMC ports 1 to 6 Redundant: - Tx MCH Clk1 to AMC Clk1 or Clk3 - Rx MCH Clk2 from AMC Clk2 ports 1 to 12 NonRedundant Alt: No Clk1, Clk2 or Clk3, but Fabrics B and C Tongues 3 and 4: Fabrics D, E, F and G Switch 10GbE, PCIe or SRIO Connects to 4 ports on each AMC Tongue 1: Fabric A (i.e. GbE) IPMI (I2C management) 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Backplane allocation 22 July 2009 A Trigger Platform for SLHC

standard functionality MCH1 providing GbE and standard functionality MCH2 providing CMS functionality 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Backplane design GbE Tongue 1: AMC port 1 Tongue 2: Fabric B AMC port 3 Tongue 3 and 4: AMC ports 8-11 Tongue 2: AMC Clk3 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Option 1 MCH1 T1 GbE MCH1 T3/T4 Protocol agnostic switch 4x (12x12) or 72x72 Connects to FatPipe MCH2 T1 Must have this tongue Hence place basic services here. TTC/TTS on Fabric A (AMC Port1) MCH2 T2 LHC Clk on MCH Clk1 (AMC Clk3) MCH2 T3/T4: DAQ on port 8 Ports 9-11 unused But... Seems to waste MCH2 T3 & T4 Could be used for X-point switch (or other app) If Fabric B available then.. Protocol agnostic switch custom to particular vendors MCH. No power/communication to T3/T4. 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Option 2 MCH2 T1 DAQ on Fabric A (AMC Port 1) Perhaps here rather than because this is the main PCB May need large FPGA + optics if eventual aim is 10GbE for DAQ MCH2 T2 LHC CLK on MCH Clk1 (AMC Clk 3) TTC/TTS on Fabric B (AMC Port3) MCH2 T3/T4 Spare Protocol agnostic switch? 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC MCH for trigger system SFP+ (10GbE) Local DAQ V6 or S6 XC6VLX130T 20 links SFP+ (10GbE) Global DAQ 12 MCH2-T1 Header – 80 way DAQ from AMC Port 1 Tx, AMC Port 1 Rx unused Back pressure ? MagJack TTC CLK40 to AMC CLK3 12 Pros/Cons: Nice design. All services on just two cards T3 and T4 spare for X-point CLK2 (Unused) 12 MCH2-T2 Header – 80 way TTC to AMC Port 3 Rx 12 TTS from AMC Port 3 Tx 12 22 July 2009 A Trigger Platform for SLHC

A Trigger Platform for SLHC Conclusions First attempt to match physical geometry onto hardware platform Some spare capacity for contingency Leaves basic design flexible Processing card Flexible Should support multiple algorithms Easily upgraded to Virtex 6 without destroying mapping onto hardware Manufacture in August Card should be back in September or October Services / Crates / MCHs Still considering options. 22 July 2009 A Trigger Platform for SLHC