1. CBM DAQ and Event Selection Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration International Workshop on Transition Radiation Detectors – Present and Future Cheile Gradistei, Romania, September 24-28, 2005

2. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 2 Outline CBM (very briefly)  observables  setup FEE/DAQ/Event Selection  requirements  challenges  strategies

3. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 3 CBM at FAIR... evolving SIS 100 Tm SIS 300 Tm U: 35 AGeV p: 90 GeV Compressed Baryonic Matter Experiment

4. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 4 CBM Physics Topics and Observables In-medium modifications of hadrons  onset of chiral symmetry restoration at high ρ B  measure: , ,   e + e - (μ + μ - ) open charm: D 0, D ± Strangeness in matter  enhanced strangeness production  measure: K, , , ,  Indications for deconfinement at high ρ B  anomalous charmonium suppression ?  measure: D 0, D ± J/   e + e - (μ + μ - ) Critical point  event-by-event fluctuations  measure: π, K Good e/π separation Low cross sections → High interaction rates → Selective Triggers Hadron identification Vertex detector

5. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 5 CBM Setup  Radiation hard Silicon pixel/strip detectors in a magnetic dipole field  Electron detectors: RICH & TRD & ECAL: pion suppression up to 10 5  Hadron identification: RPC, RICH  Measurement of photons, π 0, η, and muons: ECAL

6. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 6  Add ZCAL to determine centrality  Consider HPD for early lepton tagging (optional)  Consider μID detector (optional)  Consider various Silicon tracker implementations CBM Setup... evolving ZCAL HBD μID

7. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 7 CBM Trigger Requirements In-medium modifications of hadrons  onset of chiral symmetry restoration at high ρ B  measure: , ,   e + e - open charm (D 0, D ± ) Strangeness in matter  enhanced strangeness production  measure: K, , , ,  Indications for deconfinement at high ρ B  anomalous charmonium suppression ?  measure: D 0, D ± - J/   e + e Critical point  event-by-event fluctuations  measure: π, K offline assume archive rate: few GB/sec 20 kevents/sec offline trigger trigger on high p t e + - e - pair trigger on displaced vertex drives FEE/DAQ architecture

8. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 8 Open Charm Detection Example: D 0  K -  + (3.9%; c  = 124.4  m) reconstruct tracks find primary vertex find displaced tracks find secondary vertex target first two planes of vertex detector few 100 μm 5 cm standard procedure.... for CBM, this is in 1 st level 'trigger' like for LHcB or BTeV (rip)

9. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 9 CBM DAQ Requirements Profile D and J/Ψ signal drives the rate capability requirements D signal drives FEE and DAQ/Trigger requirements  Problem similar to B detection, like in LHCb or BTeV (rip)  Adopted approach: displaced vertex 'trigger' in first level, like in BTeV (rip)  Additional Problem: DC beam → interactions at random times → time stamps with ns precision needed → explicit event association needed Current design for FEE and DAQ/Trigger:  Self-triggered FEE  Data-push architecture

10. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 10 Conventional FEE-DAQ-Trigger Layout Detector Cave Shack FEE Buffer L2 Trigger L1 Trigger DAQ L1 Accept L0 Trigger f bunch Archive Trigger Primitives Especially instrumented detectors Dedicated connections Specialized trigger hardware Limited capacity Limited L1 trigger latency Modest bandwidth Standard hardware

11. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 11 Limits of Conventional Architecture Decision time for first level trigger limited. typ. max. latency 4 μs for LHC Only especially instrumented detectors can contribute to first level trigger Large variety of very specific trigger hardware Not suitable for complex global triggers like secondary vertex search Limits future trigger development High development cost

12. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 12 L1 Trigger Special hardware High bandwidth The way out.. use Data Push Architecture Detector Cave Shack FEE Buffer L2 Trigger L1 Trigger DAQ L1 Accept L0 Trigger f bunch Archive Trigger Primitives Especially instrumented detectors Dedicated connections Specialized trigger hardware Limited capacity Limited L1 trigger latency Modest bandwidth Standard hardware f clock L2 Trigger Time distribution

13. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 13 L1 Trigger Special hardware High bandwidth The way out... use Data Push Architecture Detector Cave Shack FEE DAQ Archive f clock L2 Trigger

14. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 14 L1 Select Special hardware High bandwidth The way out... use Data Push Architecture Detector Cave Shack FEE DAQ Archive f clock L2 Select Self-triggered front-end Autonomous hit detection No dedicated trigger connectivity All detectors can contribute to L1 Large buffer depth available System is throughput-limited and not latency-limited Modular design: Few multi-purpose rather many special-purpose modules Use term: Event Selection

15. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 15 Front-End for Data Push Architecture Each channel detects autonomously all hits An absolute time stamp, precise to a fraction of the sampling period, is associated with each hit All hits are shipped to the next layer (usually concentrators) Association of hits with events done later using time correlation Typical Parameters:  with few 1% occupancy and 10 7 interaction rate: some 100 kHz channel hit rate few MByte/sec per channel whole CBM detector: 1 Tbyte/sec → power efficient front-end → low jitter clock/time distribution → high bandwidth

16. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 16 Typical Front-End Electronics Chain PreAmp Shaper ADC Hit Finder Hit Finder digital Filter digital Filter Backend & Driver Backend & Driver Si Strip Pad GEM's PMT APD's Shaping time: 5-200 ns Sample rate: 10-100 MHz Dyn. range: 8...>12 bit 1/t Tail cancellation Baseline restorer Hit parameter estimators: Amplitude Time FIFO-Buffering Line protocol All in one mixed-signal chip

17. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 17 Towards a Multi-Purpose FEE Chain PreAmp ADC Hit Finder Hit Finder digital Filter digital Filter Shaper Several selectable input stages; → pos/neg signals → capacitance match switched capacitor filter → programmable shaping time programmable sampling rate programmable hit finder and estimator

18. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 18 Towards an Over All Architecture Digital Pipeline Ain Event Buffers send DAC Analog Filter sample Digital Filters WERE we stop N Pre- Amplifier Blocks TIMER CLOCK different combinations sampling rate resoluion power DATA-out ANALOG FIFO to/from next neighbor hit detection ADC start Hit TDC Hit Detector Leading Edge peak … 1..N Fast Readout...... to/from next neighbor hit detection Programmable Finite State Machine leveraging TRAP knowledge and expertise

19. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 19 A CBM M-MPW Run CBM organized a Multi-Multi Project Wafer run in UMC 0.18 μm CMOS  6 different parts combined on a 5 x 5 mm 2 wafer  submitted June to Europractice (IMEC)  dies (already cut) just delivered, now come the moments of truth AtkinFischerBrüning DeppeMuthers Tontisirin Content addressable memory 12bit 50MSPS ADC Test structures PreAmp for Si Strip DLL based TDC Clock-Data recovery Coordination: Marcus Dorn @ KIP

20. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 20 FEE Summary Concrete R&D on FEE ASIC buildings blocks  MPWs done in 2005 covering essential blocks  analyze results, plan now next steps Other ongoing developments:  Fast PreAmp-Shapers by H.K. Soltveit in 0.35 μm AMS CMOS as spin-off of DETNI in 0.13 μm IBM CMOS  next generation TAC by H. Flemming FOPI and HADES work on RPC electronic  both learn to build mid-sized systems  self-triggered multi-hit TDC based on CERN HPTDC as HADES spin-off planned

21. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 21 CBM DAQ and Online Event Selection More than 50% of total data volume relevant for first level event selection Aim for simplicity Simple two layer approach: 1. event building 2. event processing needed for D needed for J/μ usefull for J/μ STS, TRD, and ECAL data used in first level event selection

22. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 22 Logical Data Flow Concentrators: multiplex channels to high-speed links Time distribution Buffers Build Network Processing resources for first level event selection structured in small farms Connection to 'high level' selection processing

23. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 23 Bandwidth Requirements Data flow: ~ 1 TB/sec 1 st level selection: 10 ~14 operation/sec Data flow: few 10 GB/sec archive target: 1 GB/sec Moore helps Gilder helps

24. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 24 CBM DAQ / L1 Event Selection Requirements Profile Revisited So far assumed:  D and J/Ψ measured simultaneously @ 10 7 int/sec  requirements driven by D (displaced vertex @ L1) Current scenarios:  open charm runs with ~10 6 int/sec  look into D 0, D ±, D s, and Λ c  look into J/Ψ → e + e -  look into J/Ψ → μ + μ -  look into p-A and p-p less computational B/W open charm L1 select lower σ → 10 7 int/sec all new game, still open even lower σ → 10 8+ int/sec Λ c has cτ = 60μm only

25. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 25 CBM DAQ / L1 Event Selection Requirements Profile Revisited SystemObservableInteraction Rate Au+Auopen charm10 6 C+Copen charm10 7 p+p,p+Aopen charmfew 10 7 Au+Au J/Ψ 10 7 C+C J/Ψ 10 8 p+p,p+A J/Ψ few 10 8 p+A Y few 10 8 MAPS/DEPFET pile-up again primary vertex determination ? events no longer well separated in time....

26. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 26 Feasibility & Options: BNet Data flow in Event Builder ~ 1 TB/sec

27. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 27 Example: InfiniBand Switches TODAY:  Voltaire ISR 9288 InfiniBand switch  288 4x ports; non-blocking  cost today ~120 kEUR (or ~400 EUR/port)  288 GByte/sec switching bandwidth likely in a few years:  288 4x port QDR  likely same or lower cost  1152 GByte/sec switching speeds  adequate for CBM... Conclusion:  BNet switch is not a major issue  There will be a COTS solution

28. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 28 Feasibility & Options: PNet Initial proposal: Network of FPGA and CPU resources

29. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 29 Slide from CHEP'04 Dave McQueeney IBM CTO US Federal Shown in CBM Collaboration Meeting October 2004

30. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 30 The Cell Processor one 'normal' PowerPC CPU (PPE) 8 Synergistic Processing Element (SPE) each with  256 kB memory; 128 x 128 bit register  4 SP floating point units (SIMD) designed for best GFlop/W runs at ~ 3 GHz → 200 GFlop peak Software support state:  Linux integration planned for 2.6.13  SPE interface via spufs  gcc/gdb support for C/C++  SIMD instructions via instrinsics... 221 mm 2 die size in 90 nm Developed by Sony, Toshiba and IBM Market: VIDEOGAMES Budget: 500 M$

31. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 31 The Cell Processor cont. Processors like the Cell offer  massive single precision floating point capabilities  large memory bandwidth  very good power efficiency (GFlop / W) This is a very promising platform for L1 tracking  great for algorithms with a lot of arithmetic  probably much easier to handle than FPGAs Next steps  analyze CA/KF tracker algorithm  determine the SIMD potential  re-implement it for speed Note: might even improve time on a normal PC when SSE is used....  and of course, get a Cell system... Note: 1000 Cell processors have 200 SP TFlop peak.... the Even Intel has realized now that this is the essential figure of merit

32. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 32 CBM FEE/DAQ Demonstrator Mission: Provide a platform to  demonstrate essential architecture elements of the CBM FEE-DAQ concept FEE: self-triggered, data push, conditional RoI based readout CNet: combined data, time, control, and RoI traffic TNet: low jitter clock and synchronization over serial links BNet:high bandwidth, RDMA based architecture E/DCS:integrated approach for DCS/ECS  provide test bed for all future FEE/DAQ prototyping in hardware firmware controlware software  perform beam tests with detector prototypes  form basis for medium-scale applications in intermediate-term experiments Be operational by end 2006  avoid cathedrals, go for the bazaar, try and learn  build a first generation (G1) demonstrator quickly

33. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 33 Mini Setup: Ethernet based DCB MGT GE OASE FEE DCB-TC MGT GE OASE FEE DCB MGT GE OASE FEE GE Switch Next to minimal test bench setup few DCB clock/trigger from DCB-TC modest data date via GE PC GE PC GE

34. 25 September 2005 TRD Workshop, Cheile Gradistei, September 24-28, 2005 34 The End Thanks for your attention We acknowledge the support of the European Community- Research Infrastructure Activity under the FP6 "Structuring the European Research Area" programme (HadronPhysics, contract number RII3-CT-2004-506078).