1. Introduction to DAQ Architecture Niko Neufeld CERN / IPHE Lausanne

2. Features Two types of data flows on the same network Level 1: 5.5 kB @ 1 MHz HLT: 40 kB @ 40 kHz One shared farm subdivided in ~ 90 sub-farms contains ~ 2000 CPU nodes (single CPU) Everything connected by Gigabit Ethernet (1000 BaseT) – use IP as transport protocol Pure push protocol, with back-pressure local: Ethernet flow-control global: throttle signal via TFC

3. LHCb DAQ Architecture Multiplexing Layer FE Switch Level-1 Traffic HLT Traffic 126-224 Links 44 kHz 5.5-11.0 GB/s 323 Links 40 kHz 1.6 GB/s 29 Switches 32 Links 94-175 SFCs Front-end Electronics Gb Ethernet Level-1 Traffic Mixed Traffic HLT Traffic 94-175 Links 5.5-10 GB/s TRM Sorter TFC System L1-Decision Storage System Readout Network Switch SFC Switch CPU SFC Switch CPU SFC Switch CPU SFC Switch CPU SFC Switch CPU CPU Farm 62-87 Switches 64-137 Links 88 kHz ~2000 CPUs

4. Front-end board Most FE boards are of the same type: TELL1 connect to the detector receive L0 and L1 trigger from TTC format data for L1 and HLT and sends to DAQ send throttle signal to TFC if buffers threaten to overflow Some (e.g. RICH) do not participate in L1

5. Properties relevant for simulation Latencies: zero suppression, … Adjustable water marks (for throttle signals) Separate data-paths for L1 and HLT Buffer sizes (L1-buffer, possibly HLT formatting buffer) should be adaptable to final specifications / functionality of TELL1

6. TFC (simplified) L0 throttle OR L0 Throttle OR VELO L1 FE TTCrx VELO L0 FE TTCrx VELO L0 FE TTCrx L1 Decision Sorter TFC switch L1 throttle switchL0 throttle switch TTCtx L1 Throttle OR VELO L0 FE TTCrx L0 FE TTCrx L1 FE TTCrx TTCoc VELO L1 FE TTCrx VELO L0 FE TTCrx VELO L0 FE TTCrx VELO L0 FE TTCrx L0 FE TTCrx L1 FE TTCrx TTCoc L1 throttle OR L0 Decision Uni/t Generator Readout Supervisor Only blue parts need to be simulated Decision sorting and RS only complex devices with several delays, buffers, etc…

7. L1 Decision Sorter L1 decisions must be sorted when they arrive from the farm L1 Decision Sorter delivers sorted decisions to Readout Supervisor L1DS checks for time-outs of decisions

8. Switching Network Parameterised switch model Nice to have parameters, which can be measured (in our test-bed) Mostly buffers and latency Possibly back-pressure (flow-control) on Ethernet links Support for various frame sizes (MTUs) Routing / switching on the basis of Ethernet or IP Much more detail in Jean-Pierre’s presentation Aggregation switches should be included in simulation  they add a “hop”, possibly “smear” arrival times at the main switch

9. Subfarm Controller Performs event-building, checks for corruption, time-outs, etc. Level-1 events: if free worker CPU  forward, set timer for maximal processing time if decision received in time forward to Level 1 decision sorter, else send default decision (programmable) if no free worker CPU, buffer until either time-out (discard) CPU free – send onwards with time-penalty  events age in the SFC buffer HLT events: send to free CPU  nodes buffer several events Can issue throttle signal (via ECS  slow!)

10. Farm node Runs algorithms for L1 and HLT Generates time-out (when algorithm does not terminate) Buffers several HLT events Has at maximum one L1 event at any given time For HLT forwards events to SFC For L1 forwards decision to SFC Can fail, loop, time-out... Communicates with SFC with a token-scheme

11. Summary The DAQ system is designed to consist of simple components of well-defined functionality Most components will have a one-to-one correspondence in the simulation Details of TFC system irrelevant Details of latencies and buffers in FE-boards and farm very important The system is scalable. Upgrades will add data to the L1 stream (can double the total required band-width)