Implementation and Test of the First- Level Global Muon Trigger of the CMS Experiment Hannes Sakulin 1), 2), Anton Taurok 2) 1) CERN 2) Institute for High Energy Physics, Vienna, Austria 11 th Workshop on Electronics for LHC Experiments Heidelberg 13 th September, 2005 URL of this presentation:

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 2 CMS Detector and the Muon System Drift-Tube chambers (DT) - 4 stations in muon barrel - 12 (8) DT layers per station - 4 stations in muon barrel - 12 (8) DT layers per station Cathode Strip Chambers (CSC) - 4 stations in muon endcaps - 6 CSC layers per station Resistive Plate Chambers (RPC) - 6 layers in muon barrel - 6 layers in muon barrel - 4 layers in muon endcaps - 4 layers in muon endcaps Iron Yoke Superconducting Coil Tracker Electromagnetic Calorimeter Hadronic Calorimeter

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 3 CMS Level-1 Trigger  Optimal Combination  Cancel out duplicates  Best four muons  Calo confirmation + isolation  Trigger Algorithms (max. 128 in parallel) 1 , 2 , 1  +1e, 1e, 2e,  +j …topological 1 , 2 , 1  +1e, 1e, 2e,  +j …topological  Only place where thresholds are applied max. 100 kHz L1 Accept Pipelined 40 MHz, Latency < 3.2  s HFHCALECAL RPCCSCDT DT local trigger CSC local trigger DT Track Finder CSC Track Finder Pattern comparator trigger Global Calorimeter Trigger Global Muon Trigger L1 Global Trigger Regional Calorimeter Trigger 4  4+4  4  (with MIP/ISO bits) MIP+ Quiet bits e/ , j, E T, E t miss, … Calorimeter Trigger Muon Trigger

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 4 Global Muon Trigger Overview Output: 8 bit , 6 bit , 5 bit p T, 2 bits charge/synch, 3 bit quality, MIP bit, Isolation bit Inputs: 8 bit , 6 bit , 5 bit p T, 2 bits charge, 3 bit quality, 1 bit halo/  fine-coarse Best 4  4  RPC brl 4  DT 4  CSC 4  RPC fwd 252 MIP bits 252 Quiet bits

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 5 Global Muon Trigger Tasks  Synchronizing  Matching (based on ,  ) & Pairing of DT & brlRPC, CSC & fwdRPC  Merging parameters  Converting scales (  )  Detecting ghosts, fake triggers  Canceling out duplicated candidates in the overlap region  Propagating to calorimeter for MIP bit assignment / to vertex for ISO bit assignment  Ranking & Sorting

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 6 Global Muon Trigger Simulation Cancel ghosts barrel / endcap overlap region Optimal combination Project to Calorimeter assign MIP, ISO Determine best four muons sort by rank 5% <0.1%  muon, calo pTpT  GMT selec tion    Rate kHz at 14 GeV/c OR SMART AND rates for L=2x10 33 cm -2 s - 1

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 7 VME Boards & Data flow PSB-9U Pipeline Synchronizing Buffer GMT-9U 16 STP cables, SCSI connectors MHz Muon candidates (16) MIP and Quiet bits RPC-Trigger CSC Trigger DT Trigger RPC Trigger backplane GTL MHz Best 4 muons Global Trigger Backplane connection GTL+, MHz point-to-point Global Calorimeter Trigger 12 cables x 2 pairs MHz sent on 1.4 Gbit/s serial links backplane serial link max MHz sent at 280 MHz Event records (3072 bits / trigger) to DAQ via front- end card

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 8 GMT 9U VME Board

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 9 GMT 9U VME Board VME 64x Interface Altera ACEX SORT VirtexII-2000 ROP& Control VirtexII-2000 Logic fwd VirtexII-3000 Logic brl VirtexII-3000 MIP/ISO brl VirtexII-3000 MIP/ISO fwd VirtexII-3000 Input fwdRPC VirtexII-1500 Input CSC VirtexII-1500 Input DT VirtexII-1500 Input brlRPC VirtexII-1500 Input Board 12 SCSI connectors LVDS receivers 10x Xilinx Virtex-II FPGA on mezzanines XC18V04 PROMs mounted under the mezzanines up to 4 per FPGA JTAG Xilinx JTAG Altera Channel Link for readout Power Supply +Cooling Clock distribution Readout Processor VME Interface JTAG via VME Simu & Spy Control Board Control

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 10 Mezzanine board for BF957  For Xilinx Virtex-II-2000, -3000,  Connectors  2x Samtec MIT-076-L-D (up, down)  2x Samtec MIT-095-L-D (left, right)  50  impedance  Used for all FPGAs on the GMT Logic Board except Input FPGAs

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 11 Front Panel

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 12 Input Board + Front Panel Edge connectors B25 Z-pack 2 mm (Tyco AMP)

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 13 GMT in the Global Trigger Crate In three slots behind the GMT front panel GMT Logic Board 3 Pipeline Sync. Boards Global Trigger Crate Special wide input board parallel to front panel Global Trigger Boards and Trigger Control System housed in same crate Space for cables

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 14 FPGA Development GMT LUT X Lookup Method Cadence NCSIM Xilinx ISE 6.3 Synplify 7.3 VHDL Behavioral simulation Gate level simulation Chip configuration VHDL generated by LUT Framework Synthesis Implementation Concurrent Versions System to manage VHDL code CVS Server “Build System” most of design flow scripted with Makefiles Automated tests verify functionality after every change cross-check with C++ simulation look-up tables C++ GMT LUT X Lookup Method

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 15 FPGA Development  Example: Forward Logic FPGA  Xilinx XC2V3000  464 chip inputs/outputs used  92/96 18 kbit memory blocks used  75 % of chip resources used Developed VHDL Cross-checked with C++ simulation Synthesized + Implemented Verified on-board Design of all 10 FPGAs now finished GMT Forward Logic FPGA

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 16 GMT functional tests (I)  Test features in hardware  Simulation RAMs in Input FPGAs (and in MIP/ISO Assignment FPGAs)  Spy RAMs in Sort FPGA  Synchronous start through ROP & Control chip  Can test sequences at 40 MHz  Bitwise compatible C++ simulation  Part of CMS simulation software (ORCA)  Can save simulated GMT input and GMT output in hardware representation  Common configuration of simulation model and hardware  Configuration of simulation model can be transformed into hardware configuration (look-up-table and register contents)  Hardware test with simulated physics samples

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 17 GMT Online Software Overview GMTDriver Hardware Access Library JTAG Access Library GMTBoard GMTChip Board/Chip structure GMTChip GMTConfigurator GMTTester GMTTest GMTMonitor Main GMT Functions level GMTFirmwareLoader GMTLUTHandlerGMTBoardController GMTSimuSpyHandlerGMTConfigRegHandler GMTInputMonitor GMTErrorMonitor Basic services level Facade GMTEvent GMTEventReader L1MuGMTLUT Used by command line interface or by Trigger Supervisor

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 18 GMT functional tests (II) SORT ROP& Control Logic fwd Logic brl MIP/ISO brl MIP/ISO fwd Input fwdRPC Input CSC Input DT Input brlRPC Simulated data sequence loaded into simulation RAMs 4k events (by VME) Result “spied” in Spy RAM (by VME) Muon Data connections tested Cancel-out links tested VME access to all chips tested Simulated MIP/ISO bits (assignment result) Synchronous test start controlled by ROP&Control

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 19 GMT functional test results  VME access to all chips tested  Hardware timing verified  Test works at 40 MHz  Main muon data path tested  Input FPGAs  Logic FPGAs  Sort FPGAs  Several bad connections (single bits) identified  Due to soldering problems at mezzanine connectors Most problems solved (re-soldering) Remaining/future problems can be fixed by re-routing through spare signal lines between FPGAs  To be tested …  Data path Input FPGAs  MIP/ISO Assignment FPGAs  MIP/Quiet bits from PSBs to GMT (over backplane)  GMT result to GT (over backplane)  Readout records to Frontend Card (over backplane)

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 20 CSC-GMT Interconnection Test Vienna, January 2005  Objective  Test connection between CSC Trigger Muon Sorter and GMT  Test GMT input stage  Setup  Common clock source: TTCex module with optical outputs (in 6U crate)  CSC Trigger crate  Clock and Control Board (CCB) with TTCrm optical clock receiver mezzanine  Muon Sorter (MS)  SBS620 VME controller  Global Trigger / Global Muon Trigger Crate  Timing (TIM6U) board with TTCrm optical clock receiver mezzanine  Global Muon Trigger (GMT)  TTCvi board to create periodic bunch counter reset every 3564 th bx  SBS620 VME controller  4 cables: shielded twisted pair, 34 pairs, halogen free length: 11 m (max length in final system: 10 m)

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 21 CSC-GMT Interconnection Test

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 22 CSC-GMT Interconnection Test  Test Patterns  “Running one” pattern  Random Pattern  Test setups  Muon Sorter outputs connected to CSC0.. CSC3 inputs of GMT  Muon Sorter outputs connected to DT0..3, brl-RPC0..3, fwd-RPC0..3 inputs  Muon Sorter outputs connected to CSC0, DT0, brl-RPC0, fwd-RPC0 inputs  GMT Input sampling  At 160 MHz (4x per bx)  Detect switching time of input signal  Result: Switching always occurred at same phase. Stable clock relationship  Compare logic in GMT firmware  Store reference sequence in RAM (500 words)  Continuously compare incoming patterns with reference and count errors  Can perform long-term tests at maximum speed

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 23 CSC-GMT Interconnection Test  Result  Test with pseudo-random data running for 16 hrs  3.8 x bits transferred  No errors observed  GMT inputs for all regional muon triggers tested

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 24 Summary  CMS has three muon systems. All three are used in the L1 Trigger  Regional Muon Triggers find muon candidates  Global Muon Trigger combines these candidates  finds the best 4 overall candidates  correlates with calorimeter information  many advantages (proven by extensive simulation studies)  Higher efficiency  Better rate handling  Lower ghosting  Hardware development is now compete  Single 9U GMT Logic Board + 3 PSB Input boards  10 Xilinx Virtex-II FPGAs on mezzanines  Firmware design complete (based on VHDL model)  Online-Software development well underway  Performed functional tests (more to come)  Successful interconnection test with CSC Trigger Many thanks to the HEPHY Electronics team: Michael Padrta Kurt Kastner Herbert Bergauer

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 25 Backup Slides

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 26 GMT Logic Board

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 27 FPGA Development: Logic FPGA  Many Functions implemented as Look-Up Tables  Flexible: GMT can be adapted to characteristics of regional trigger systems  Xilinx Block RAM and distributed RAM  bits input (addr) bits output (data)  In Logic FPGA:  14 types of LUTs  52 sets of default values  > 200 LUT instances  Need system to handle LUTs GMT Forward Logic FPGA

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 28 Types of Look-Up-Tables in GMT  MIP & ISO assignment FPGA: 4 types of LUTs, (32 sets of default values)   -conversion 6 to 4 bits   projection stage 1   projection stage 2   projection  Logic FPGA: 14 types of LUTs, (52 sets of default values)  Matching Unit  Delta-  calculation  Match quality  Cancel-Out Unit  Overlap  conversion (6 to 4 bits for match unit inside cancel-out unit)  Cancel-Out Unit Delta-  calculation  Sort Rank Unit  Sort-Rank pt-q  Sort-Rank  -q  Sort-Rank  -   Sort Rank combine  Merge Rank Unit  Merge-Rank pt-q  Merge-Rank  -q  Merge-Rank  -   Merge-Rank combine  Muon Merger Unit  Pt mixing  Unit conversion   -conversion (input to output)  Different default values for each LUT type for DT, CSC, brlRPC, fwdRPC or combinations thereof  Different default values for each LUT type for DT, CSC, brlRPC, fwdRPC or combinations thereof  Need system to handle LUTs

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 29 Firmware Test: Data Flow  Cross-check of ORCA simulation with FPGA firmware & GMT board  Single type of hardware test file for all chips and for GMT board  All GMT inputs & outputs  Intermediate results at various points in GMT  So far tested Logic FPGAs and MIP and ISO Assignment FPGAs  Higgs  4  events  Full agreement with ORCA ORCA DT CSC RPC Calo GMT GT HW Testfile digi GMT Inputs GMT Outputs intermediate results VHDL test bench (for single chip or entire GMT) Chip or entire GMT VHDL Behavioral / Gate Level stimuliOutput signals Intermediate signals hits

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 30 Automated testing of VHDL code  Example (continued). /GMT > cvs co -r MIAU_timesim_works_03Jun2003 GMT (check out code) /GMT > cd src/MipIsoAU/test /GMT > gmake test_miau (analyze + elaborate VHDL, simulate & check using default dataset) checking event nr Simulated PHI SELECTBITS : form ORCA PHI SELECTBITS : Simulated ETA SELECTBITS : from ORCA ETA SELECTBITS : Simulated MIP/ISO bits : DTMIP: , RPCMIP: , DTISO: , RPCISO: From ORCA MIP/ISO bits : DTMIP: , RPCMIP: , DTISO: , RPCISO: **** all tests succeeded. ncsim: *W,RNQUIE: Simulation is complete. ncsim> exit /GMT >

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 31 Selection of low-quality unconfirmed muons Efficiency and Trigger Rates 50 % GMT smart: 2.9 kHz GMT OR: 5.4 kHz GMT AND: 2.0 kHz GMT Efficiency ORCA simulation  100 % trigger rate from min. bias threshold 14 GeV/c Efficiency DT CSC RPC 0 % 100 % GMT smart

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 32 Efficiency versus Pseudorapidity GMT smart DT CSC RPC  Single muon sample (both charges): 2.5 < p T < 100 GeV/cNo p T threshold applied ORCA simulation

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 33 Selection of low-quality unconfirmed muons Efficiency and Trigger Rates 50 % GMT smart: 97.3 % GMT OR: 98.1 % GMT AND: 87.4 % Single Muon Trigger Rate L=2x10 33 cm -2 s -1 from min. bias GMT smart: 2.9 kHz GMT OR: 5.4 kHz GMT AND: 2.0 kHz All values for |  | < 14 GeV/c Efficiency 14 ORCA simulation

LECC 2005 Heidelberg, 13 th September 2005 Hannes Sakulin CERN / HEPHY Vienna Implementation and Test of the First-Level Global Muon Trigger of CMS 34 Mezzanine connectors