CERN Real Time conference, Montreal May 18 – 23, 2003 Richard Jacobsson 1 Driving the LHCb Front-End Readout TFC Team: Arek Chlopik, IPJ, Poland Zbigniew Guzik, IPJ, Poland Richard Jacobsson, CERN Beat Jost, CERN

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, LHCb Readout D etector Front-End electronics L0 FE VELO L0 FE STOT RICH ECALHCALMUON SWITCH L1 FE SWITCH SFC SWITCH CPUCPU READOUT NETWORK CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU CPUCPU Event building CPU farm TFC SYSTEM L0 TRIGGER LHC CLK 40 MHz 1 MHz 40 KHz SORTER Two levels of high-rate triggers For more details see Niko Neufeld’s talk in session T3 (RT-088)

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, TFC Architecture (1) Physics trigger... Throttle SwitchTFC Switch Clock L0 / L1 Readout Supervisor 2 VELO FE ST FEOT FERICH FEECAL FE Single TFC master : Readout Supervisor Fan-out / Fan-in : Support partitioning Front-End Clock L0 / L1 Local trigger (Optional) Clock L0 / L1 Clock Readout Supervisor 1 Readout Supervisor 3 Readout Supervisor 4 VELO FE Event building network Clock Readout Supervisor 1

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, TFC Architecture (2) TFC components -CERN Trigger, Timing and Control (TTC) distribution system. Common to all LHC experiments: TTCtx (electrical-optical converters) TTCoc (optical fan-outs) TTCrx (Receiver chips) - Components endemic to LHCb: Readout Supervisors (“ODIN”) TFC Switch (“THOR”) Throttle Switches (“MUNIN”) Throttle ORs

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, CERN TTC system 1 l Developed in the CERN RD12 project: Timing, Trigger and Control distribution system based on fiber optics l Transmitting two channels multiplexed: è A: Low latency 40 MHz signal è B: Two types of broadcasts with Hamming code protection: Short broadcasts (8 bit data in 16 bit frame) Long broadcasts (16 bit data in 42 bit frame)

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, CERN TTC system 2 l LHC: è Distribute LHC clock (40.08 MHz) and LHC orbit signal ( kHz) to experiments over fiber with minimal jitter (~8ps RMS) l Experiments: è Distribute clock, trigger and control commands to the detector readout over fiber with minimal jitter Prevessin LHC Control Room Experimental hall several km

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, TTC in LHCb l Use of the TTC system in LHCb: è Channel A used to distribute (accept/reject signal) L0 trigger (40 MHz --> 1.1 MHz accept rate) è Channel B used to distribute short broadcasts with encoded: Bunch Counter Reset and L0 Event ID counter reset L1 trigger (1.1 MHz --> 40 kHz accept rate) Control commands (FE resets, calibration pulses) è Channel B used to distribute long broadcasts with: IP/Ethernet destination address for the data transmission over the network to the CPU farm l Broadcast order on channel B is handled according to a priority scheme

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, Readout Supervisor “Odin” - Clock distribution LHC clock - L0 handling & distribution L0 - Auto-trigger generator Trigger generator - Trigger controller Trigger controller Throttles - Command broadcast generator Cmd broadcast generator - ODIN Front-End Front-End DAQ - Exp. Control System interface ECS interface ECS - L1 handling & distribution L1 L1 broadcast generator Experiment orchestra director - all mastership in a single module Channel A/B - TTC encoding TTC Encoder Designed with emphasis on: Versatile - to support many different types of running modes Functions easily added and modified. Credit Card PC

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, ODIN simulation l A lot of effort put on simulation. è Specs have been simulated in behavioral model with a behavioral model of the LHC machine, trigger system, and FE, using Visual HDL è Full Readout Supervisor with actual FPGA code and models of discrete logic has been simulated in the same simulation VisualHDL test bench è FPGA code also simulated at gate level in the same model Q_IOI Q_PIPE Q_L0 Q_CMD Q_L1 AFIFO TFIFO Q_GCS Q_T1B Q_RND TTC

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, TFC Switch “THOR” l Crucial: Equal internal propagation delays. If skew too large, FE will suffer from timing alignment problems when using different Readout Supervisors. l Small jitter Pool of Readout Supervisors VELOVELO STST OTOT RICHRICH... MULTIPLEXERS DELAYS ECS interface ECS TTC information as TTC encoded electrical Clock, trigger and command distribution and support partitioning

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, Throttle Switch “Munin” Pool of Readout Supervisors VELOVELO STST OTOT RICHRICH... ECS interface ECS Throttle feed-back from detector Front-End and trigger system to the appropriate Readout Supervisor Throttle signals Throttle OR module is only a variation of the same board as 32:1 OR logic and history buffer

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23, Conclusions l TFC system architecture and use of TTC well establish l Different from the other LHC experiments with two levels of high-rate trigger l Emphasis on partitioning è Readout Supervisor All mastership in one module Provides a lot of flexibility and versatility è Switches Partitioning to support testing, calibrating, and debugging well integrated l The first prototype of the Readout Supervisor “ODIN” built and tested l Final prototype of TFC Switch built and tested l Final complete prototype of Readout Supervisor ready for production l General purpose test board “FREJA” is being designed è Self-checking scan of the functionality by producing stimuli and receiving the output of the TFC system like a FE

CERN Richard Jacobsson Real Time conference, Montreal May 18 – 23,