LHCb Trigger and Data Acquisition System Beat Jost Cern / EP Presentation given at the 11th IEEE NPSS Real Time Conference June 14-18, 1999 Santa Fe, NM

RT 99, Santa Fe, 15 June 1999 Slide 2 Beat Jost, Cern Outline qIntroduction qGeneral Trigger/DAQ Architecture qTrigger/DAQ functional components ãSelected Topics åLevel-1 Trigger åEvent-Building Network Simulation qSummary

RT 99, Santa Fe, 15 June 1999 Slide 3 Beat Jost, Cern Introduction to LHCb qSpecial purpose experiment to measure precisely CP violation parameters in the BB system qDetector is a single-arm spectrometer with one dipole qTotal b-quark production rate is ~75 kHz qExpected rate from inelastic p-p collisions is ~15 MHz qBranching ratios of interesting channels range between giving interesting physics rate of ~5 Hz LHCb in Numbers

RT 99, Santa Fe, 15 June 1999 Slide 4 Beat Jost, Cern LHCb Detector

RT 99, Santa Fe, 15 June 1999 Slide 5 Beat Jost, Cern Typical Interesting Event

RT 99, Santa Fe, 15 June 1999 Slide 6 Beat Jost, Cern Trigger/DAQ Architecture Read-out Network (RN) RU Control & Monitoring RU 2-4 GB/s 4 GB/s 20 MB/s Variable latency L2 ~10 ms L3 ~200 ms LAN Read-out units (RU) Timing & Fast Control Front-End Electronics VDET TRACK ECAL HCAL MUON RICH LHC-B Detector L0 L1 Level 0 Trigger Level 1 Trigger 40 MHz 1 MHz 40 kHz Fixed latency 4.0  s Variable latency <1 ms Data rates 40 TB/s 1 TB/s Front-End Multiplexers (FEM) 1 MHz Front End Links Trigger Level 2 & 3 Event Filter SFC CPU Sub-Farm Controllers (SFC) Storage Throttle

RT 99, Santa Fe, 15 June 1999 Slide 7 Beat Jost, Cern Front-End Electronics qData Buffering for Level-0 latency qData Buffering for Level-1 latency qDigitization and Zero Suppression qFront-end Multiplexing onto Front- end links

RT 99, Santa Fe, 15 June 1999 Slide 8 Beat Jost, Cern Timing and Fast Control qProvide common and synchronous clock to all components needing it qProvide Level-0 and Level-1 trigger decisions qProvide commands synchronous in all components (Resets) qProvide Trigger hold-off capabilities in case buffers are getting full

RT 99, Santa Fe, 15 June 1999 Slide 9 Beat Jost, Cern Level-0 Trigger qLarge transverse Energy (Calorimeter) Trigger qLarge transverse momentum Muon Trigger qPile-up Veto qImplemented in FPGAs/DSPs basically hard-wired Input rate: 40 MHz Output rate: 1 MHz Latency: 4.0  s (fixed)

RT 99, Santa Fe, 15 June 1999 Slide 10 Beat Jost, Cern DAQ Functional Components qReadout Units (RUs) ãMultiplex Front-end links onto Readout Network links ãMerge input fragments to one output fragment qSubfarm Controllers (SFCs) ãassemble event fragments arriving from RUs to complete events and send them to one of the CPUs connected ãLoad balancing among the CPUs connected qReadout Network ãprovide connectivity between RUs and SFCs for event-building ãprovide necessary bandwidth (4 GB/sec sustained) qCPU farm ãexecute the high level trigger algorithms åLevel-2 (Input rate: 40 kHz, Output rate: 5 kHz) åLevel-3 (Input rate: 5 kHz, Output rate: ~100 Hz) ã~2000 processors (à 1000 MIPS) Note: There is no central event manager

RT 99, Santa Fe, 15 June 1999 Slide 11 Beat Jost, Cern Control System qCommon integrated controls system ãDetector controls (classical ‘slow control’) åHigh voltage åLow voltage åCrates åTemperatures åAlarm generation and handling åetc. ãDAQ controls åClassical RUN control åSetup and configuration of all components (FE, Trigger,DAQ) åMonitoring ãSame system for both functions

RT 99, Santa Fe, 15 June 1999 Slide 12 Beat Jost, Cern Level-1 Trigger qPurpose ãSelect events with detached secondary vertices qAlgorithm ãBased on special geometry of vertex detector (r-stations,  -stations) ãSeveral steps åtrack reconstruction in 2 dimensions (r-z) ådetermination of primary vertex åsearch for tracks with large impact parameter relative to primary vertex åfull 3 dimensional reconstruction of those tracks ãExpect rate reduction by factor 25 qTechnical Problem: 1 MHz input rate, 3 GB/s data rate, small event fragments, Latency

RT 99, Santa Fe, 15 June 1999 Slide 13 Beat Jost, Cern Level-1 Trigger (2) qImplementation ã~32 sources to switching network ãAlgorithm running in processors (~200 CPUs) ãBasic idea is to have a switching network between data sources and processors ãIn principle very similar to DAQ, however the input rate of 1 MHz poses special problems.

RT 99, Santa Fe, 15 June 1999 Slide 14 Beat Jost, Cern Event-Building Network qRequirements ã4 GB/s sustained bandwidth ãscalable ãexpandable ã~100 inputs (RUs) ã~100 outputs (SFCs) ãaffordable and if possible commercial (COTS, Commodity?) qReadout Protocol ãPure push-through protocol of complete events to one CPU of the farm ÚSimple hardware and software ÚNo central control  perfect scalability ÚFull flexibility for high-level trigger algorithms ØLarge bandwidth needed ØAvoiding buffer overflows via ‘throttle’ to trigger

RT 99, Santa Fe, 15 June 1999 Slide 15 Beat Jost, Cern Event-Building Network Simulation qSimulated technology: Myrinet ãNominal 1.28 Gb/s ãXon/Xoff flow control ãSwitches: åideal cross-bar å8x8 maximum size (currently) åwormhole routing åsource routing åNo buffering inside switches qSoftware used: Ptolemy discrete event framework qRealistic traffic patterns ãvariable event sizes ãevent building traffic

RT 99, Santa Fe, 15 June 1999 Slide 16 Beat Jost, Cern Network Simulation Results Results don’t depend strongly on specific technology (Myrinet), but rather on characteristics (flow control, etc) qFIFO buffers between switching levels allow to recover scalability q50 % efficiency “Law of nature”

RT 99, Santa Fe, 15 June 1999 Slide 17 Beat Jost, Cern Summary qLHCb is a special purpose experiment to study CP violation qTriggering poses special challenges ãSimilarity between inelastic p-p interactions and events with B-Mesons qDAQ is designed with simplicity and maintainability in mind ãPush readout protocol  Simple, e.g. No central event manager ãHarder bandwidth requirements on readout network ãSimulations suggest that readout network can be realized by adding FIFO buffers between levels of switching elements qUnified approach to Controls ãSame basic infrastructure for detector controls and DAQ controls ãBoth aspects completely integrated but operationally independent