1. CBM FEE/DAQ/Trigger Walter F.J. Müller, GSI CBM Collaboration Meeting, October 6-8, 2004 Status and Overview

2. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 2 CBM Requirements Profile D and J/Ψ signal drives the rate capability requirements D signal drives FEE and DAQ/Trigger requirements  Problem similar to B detection, see BTeV, LHCb  Adopted approach: displaced vertex trigger in first level, like in BTeV  Additional Problem: DC beam→ interactions at random times → time stamps with ns precision needed → explicit event association needed Current design for FEE and DAQ/Trigger:  Self-triggered FEE: All hits shipped with time stamp  Data-push architecture: L1 trigger throughput limited but not latency limited Quite different from the usual LHC style electronics Substantial R&D needed

3. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 3 Conventional FEE-DAQ-Trigger Layout Detector Cave Shack FEE Buffer L2 Trigger L1 Trigger DAQ L1 Accept L0 Trigger f bunch Archive Trigger Primitives Especially instrumented detectors Dedicated connections Specialized trigger hardware Limited capacity Limited L1 trigger latency Modest bandwidth Standard hardware

4. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 4 L1 Trigger FPGA and CPU mix High bandwidth The way out... use Data Push Architecture Detector Cave Shack FEE DAQ f clock L2 Trigger Self-triggered front-end Autonomous hit detection No dedicated trigger connectivity All detectors can contribute to L1 Large buffer depth available System is throughput-limited and not latency-limited Modular design: Few multi-purpose rather many special-purpose modules

5. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 5 Typical Self-Triggered Front-End Average 10 MHz interaction rate Not periodic like in collider On average 100 ns event spacing 051015202530 time amplitude 50 100 a: 126 t: 5.6 a: 114 t: 22.2 Use sampling ADC on each detector channel running with appropriate clock Time is determined to a fraction of the sampling period threshold

6. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 6 Front-End for Data Push Architecture Each channel detects autonomously all hits An absolute time stamp, precise to a fraction of the sampling period, is associated with each hit All hits are shipped to the next layer (usually concentrators) Association of hits with events done later using time correlation

7. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 7 Typical FEE Parameters for CBM Based on  interaction rate: 10 7 /sec  1% occupancy for central collisions  minimum bias events have ¼ of central multiplicity  typically 3 detector channels fire per particle hit  a hit can be expressed in 5 bytes one gets as 'typical' values  channel hit rate: ~ 100 kHz  channel data rate: ~ 500 kB/sec or 5 Mbps BUT:  in inner (outer) regions rates tend to be generally higher (lower)  ECAL has ~7% cell occupancy → 700 kHz channel hit rate  START (diamond micro strip): some 10 MHz channel hit rate

8. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 8 Typical FEE Parameters for CBM II For a multi-channel FEE one gets: # channels Parameter 1163264128Unit total hit rate (typ.)0.11.63.26.412.8MHz total data rate (typ.)580160320640Mbps total hit rate (ECAL)0.711.022.044.088.0MHz total data rate (ECAL)35560112022404480Mbps plausible channel/chip for most sub-systems plausible channel/chip for Si Strip detector

9. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 9 Sub-System Specifics Si-Strip  few ns time resolution  high channel density (50um pitch, strip length few to >100 mm)  material budget essential  radiation hard electronics needed (2 MRad/year at z=40cm) RICH  current assumption: use PMT's as photon detector  nominal signal amplitude: few 10 5 e; dark rate: ~ 3 KHz/PMT  dynamic range modest, time resolution few ns

10. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 10 Sub-System Specifics TRD  thin Xe absorber (typ. 6 mm), no drift time  pads size and shape changes drastically over R (square -> strip)  type of chamber (wire or a 'fast' type) tbd RPC  most likely differential detector and differential input stage  time resolution of electronics should be 25 ps or better ECAL  current assumption: use PMT's as photon detector  nominal signal amplitude to be determined  dynamic range large (12 bit), ns time resolution helpful

11. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 11 Typical Front-End Electronics Chain PreAmp pre Filter ADC Hit Finder Hit Finder digital Filter digital Filter Backend & Driver Backend & Driver Si Strip Pad GEM's PMT APD's Anti- Aliasing Filter Sample rate: 10-100 MHz Dyn. range: 8...>12 bit 'Shaping' 1/t Tail cancellation Baseline restorer Hit parameter estimators: Amplitude Time Clustering Buffering Link protocol All potentially in one mixed-signal chip

12. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 12 Towards a Multi-Purpose FEE Chain PreAmp ADC Hit Finder Hit Finder digital Filter digital Filter pre Filter Several selectable input stages; → pos/neg signals → capacitance match as option: dual range conversion ? programmable sampling rate ? programmable hit finder and estimator

13. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 13 FEE Summary About half of the experiment cost is in FEE Significant amount of development needed for Data Push FEE Task Force Mission:  Collect requirement profiles for all sub-system variants  Identify common building blocks (chip or block level)  Formulate overall concept for FEE  Search for collaboration partners outside CBM  Prepare EU D&C proposal by end 2004  Start R&D on critical building blocks Slide from February 2004 Collaboration Meeting in progress retarget... in 2005

14. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 14 First CBM FEE ASIC Meeting 9/26/2004 Follow link

15. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 15 Conclusion of 1 st FEE ASIC Meeting A multi-purpose FEE chain could handle  RICH  TRD  ECAL the RPC is dedicated development, but can share buildings blocks (ADC, communication) and technology base the Si-Strip is dedicated development (channel density, radiation hardness), might share building blocks, and can share technology base

16. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 16 TP and Workgroup Structure I Sections of FEE chapter of TP  Overall Architecture  Communication and Time/Clock Architecture Clock/Time distribution (FETA) (R. Tielert / V. Lindenstruth) Link protocol (FELA) (U. Bruening / V. Lindenstruth) OASE (R.Tielert) CNet (U. Bruening / U. Kebschull) DCS interface (V. Lindenstruth / U. Bruening)  RICH/TRD/ECAL solution PreAmp (W. Mueller / R. Tielert) ADC (R. Tielert / L. Musa) filtering (W. Mueller / V. Lindenstruth) feature extraction (V. Lindenstruth / L. Musa / R. Tielert) backend (FELA)

17. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 17 TP and Workgroup Structure II Sections of FEE chapter of TP (cont.)  RPC/START solution PreAmp/Discriminator (M. Ciobanu / E. Badura) TAC variant (H. Flemming / E. Badura) DLL variant (H. Flemming / E. Badura)  Si Strip solution (E. Atkin / A. Voronin)  MAPS interfacing (/w IReS)  Infrastructure power distribution (V. Lindenstruth / R. Tielert) DCS (U. Kebschull / V. Lindenstruth) Configuration Mgmt, Fault Tolerance (U. Kebschull)

18. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 18 FEE → DAQ/Trigger Much of my is time over by now...... and dinner is waiting outside lets face DAQ/Trigger issues anyway

19. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 19 The FutureDAQ EU Project (I3HP-JRA1) 1 st Workshop March 25-26, 2004 in Munich 2 nd Workshop September 9, 2004 in Darmstadt Follow link

20. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 20 Regular local (Da/Hd/Ma) DAQ/Trigger Meetings Follow link

21. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 21 Now lets look at the whole System The following does not explore the full design space It looks into one scenario, driven by the Ansatz do (almost) all processing done after the build stage Other scenarios are possible and should be investigated, putting more emphasis on: do all processing as early as possible transfer data only then necessary All specific 'implementations' are just examples for illustration All stated numbers are still rough estimates

22. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 22 A Network oriented System View FEE CNet BNet PNet HNet Front-end Electronics Concentrator Networks Build Network Processor Networks Compute Resources High Level Network to high level computing

23. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 23... with some numbers FEE CNet BNet PNet HNet to high level computing ~ 50000 FEE chips ~ 1 TB/Sec BW ~ 1000 links ~ 100 sub farms ~ 100 nodes per sub farm Output BW ~ 10 GB/sec

24. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 24... with some questions FEE CNet BNet PNet HNet to high level computing What is the FEE Link Protocol ? Where is the time distribution ? Interfacing of CNet to BNet ? Interfacing of BNet to PNet ? Interfacing of HNet to PNet ?

25. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 25... and now decoupling the Networks FEE CNet BNet HNet to high level computing PNet Buffer Node at CNet – BNet Interface Buffer Node at BNet – PNet Interface Processor Node at PNet – HNet Interface

26. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 26 Add Time Distribution → 5 Networks FEE CNet BNet HNet to high level computing PNet TNet Interface TNet Connect to BNet or a PNet

27. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 27 Network Characteristics FEE CNet BNet HNet to high level computing PNet TNet Data Push Datagram's errors marked but not recovered Request/Response and Data Push Transactions errors recovered

28. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 28 Where is the Cave Wall ? FEE CNet BNet HNet to high level computing PNet TNet Different options to be evaluate d

29. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 29 OASE FEE Interface Proposal (V.Lindenstruth):  handle all three interfaces with one physical serial link  use an optical interface already at the FEE chip level + no noise from data lines + low chip pin count this requires  SerDes cores on FEE chips  low-cost fiber couplings this implies  clock/time is distributed on the uplinks of the CNet FEE Hit Data (out only) Clock and Time (in only) Control (bidirectional) 3 logical interfaces might be bidirectional (cluster RoI transfers) Also: Test (JTAG) New territor y

30. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 30 Focus on PNet FEE CNet BNet HNet to high level computing PNet TNet

31. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 31 A Sub-farm Scenario to HNet from BNet Node Board Fabric Board 10 Gbps links 8 port switches 10 Gbps links 8 port switches 10 node boards (different types) double star fabric 10 Gbps links 40 GB/sec BW almost feasible today !!

32. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 32 A Sub-farm Scenario II Core pieces are already here today  can only improve... Missing: FPGA and SoC with PCIe  wait and see how market moves Essential: Power consumption  2.5 Gbps SerDes now 100-200 mW  160 Gbps switch now 6.5 W (130 nm)  PLX PEX8532 or BCM5675  in 2010: see what 45nm will do 10+2 slot VXS backplane ELMA 101VXSD712 MultiGig RT connectors 6.4 Gpbs/pin This is VME++ VITA 41.x MultiGig RT connectors 6.4 Gpbs/pin 8 port, 32 lane PCIe switch 160 Gbps bandwidth PLX PEX8532

33. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 33 Slide from CHEP'04 Dave McQueeney IBM CTO US Federal

34. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 34 Network Summary 5 different networks with very different characteristics  CNet medium distance, short messages, special requirements connects custom components (FEE ASICs)  TNet broadcast time (and tags), special requirements  BNet naturally large messages, Rack-2-Rack  PNet short distance, most efficient if already 'build-in' connects standard components (FPGA, SoCs)  HNet general purpose, to rest of world Ethernet PCIe + ASI Custom open several COTS options Whatever the implementation is, it will be called Ethernet... FEE Interfaces and CNet will be co-developed. Depends on how clock/time distribution is done Potentially build with CNet components Look at emerging technologies Stay open for changes and surprises Cost efficiency is key here !! Probably uncritical

35. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 35 From Hardware to Software Key question is whether we can achieve at least 1:1000 trigger rejection for the D and J/Ψ triggers at an affordable hardware setup. Needed:  Fast algorithms  Realistic Trigger Performance Simulations  Detector – Trigger Co-development See J. Gläß and I. Kisel talks Starting in new Framework Still to do: full tracking MAPS pileup efficiencies/noise Still on 0 th iteration....

36. October 6-8, 2004 CBM FEE/DAQ/Trigger - Status and Overview, Walter F.J. Müller, GSI 36 Timelines CBM Technical proposal due 15.1.2005  this is 101 days from now  this is 73 working days from now if you still believe in weekends and a xmas break... One part is: FEE/DAQ/Trigger architecture proposal  with implementation plan (R&D steps, milestones,....)  with risk assessment  with cost estimate Lets have dinner....