FEE/DAQ Demonstrator Walter F.J. Müller, GSI, Darmstadt for the CBM Collaboration 3 rd FutureDAQ Workshop GSI, Darmstadt, October 11, 2005

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 2 Mission Statement Provide a platform to  demonstrate essential architecture elements of the CBM FEE-DAQ concept FEE: self-triggered, data push, conditional RoI based readout CNet: combined data, time, control, and RoI traffic TNet: low jitter clock and synchronization over serial links BNet:high bandwidth, RDMA based architecture E/DCS:integrated approach for DCS/ECS  provide test bed for all future FEE/DAQ prototyping in hardware firmware controlware software  perform beam tests with detector prototypes  form basis for medium-scale applications in intermediate-term experiments Be operational by end 2006  avoid cathedrals, go for the bazaar, try and learn  build a first generation (G1) demonstrator quickly

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 3 System Layout FEE CNet BNet HNet to archive TNet FEE boards CNet links DCB: data combiner boards CDL: CBM detector links ABB: active buffer boards EB Switch Backend processors General Network Clock/Trigger distribution

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 4 G1 Demonstrator: BNet FEE CNet BNet HNet to archive TNet RDMA Key architectural element in modern high speed data transport is RDMA  e.g. Infiniband  e.g. iWARP, iSCSI Separate  data transport  data transport via zero-copy RDMA  status management  status management via reliable messages Proposal:  Architecture choice: CBM BNet based on RDMA (e.g. uDAPL or other API)  Implementation choice: use InfiniBand

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 5 G1 Demonstrator: TNet FEE CNet BNet HNet to archive TNet TNet proposal:  reuse CNet components TNet link proposal:  just a CNet link  run unidirectional  possibly use optical splitters TNet controler proposal:  just a DCB with other firmware

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 6 TNet Controller and Setup DCB-TC Use DCB as TNet controller - different firmware - little extra hardware - trigger inputs - clock generator DCB DCB DATA passive optical splitter or as alternative another layer of DCBs Trigger optionally connect to ABB hardware trigger inputs and likely also 1-2 outputs

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 7 Clock and Jitter Propagation DCB-TC Use DCB as TNet controller - different firmware - little extra hardware - trigger inputs - master clock generator DCB DATA Is a passive splitter feasible or even desirable ? Is a passive splitter feasible or even desirable ? If not, an extra layer of DCBs need for clock fan-out for more than 64 FEE boards Trigger optionally connect to ABB hardware trigger inputs and likely also some outputs..... Bottom Line: Clock distributed over 2 or 3 hops

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 8 G1 Demonstrator: FEE FEE CNet BNet HNet to archive TNet Get started with a  general purpose  chamber-mountable  modest density  ADC + FPGA based board Plausible parameters:  16 channels  100 MHz sampling  10 bit If needed, a pipeline TDC based FEE module to support Time-over-Threshold based systems might be added too

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 9 DCB Connectivity Link FPGA Link for processing: but use Virtex-4 - FX have PPC, Ethernet MAC, MGT - but: V-4 FX are hard to get these days bi-directional optical link data, trigger, RoI, control, clock Options: OASE or V4 MGT uni-directional optical link; trigger, aux. control, clock primary primary data interface: MGT+SFP auxiliary Ethernet as control interface and auxiliary data interface CNTL/DATA DATA CLK/TRG CPLD Ether SFP NVRAM MEM enough SDRAM memory to allow - operation of both PPCs - data buffering support remote reconfiguration

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 10 Micro Setup: 1 DCB + 1 PC The minimal test bench setup maximal 8 FEE local clock from DCB modest data date via GE DCB MGT GE Link PC GE FEE

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 11 Mini Setup: Ethernet based DCB MGT GE Link FEE DCB-TC MGT GE Link FEE DCB MGT GE Link FEE GE Switch Next to minimal test bench setup few DCB clock/trigger from DCB-TC modest data date via GE PC GE PC GE

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 12 Target Setup: Low Performance DCB MGT GE Link FEE DCB-TC MGT GE Link FEE DCB MGT GE Link FEE GE Switch Use GE for event building PC GE DCB MGT GE Link FEE GEABB PC GE PC GE ABB PC GE ABB GE Switch ControlsDAQ – Event Building & Selection

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 13 Target Setup: High Performance DCB MGT GE Link FEE DCB-TC MGT GE Link FEE DCB MGT GE Link FEE GE Switch Use IB for event building PC GE DCB MGT GE Link FEE HCAABB PC GE PC GEHCAABB PC GEHCAABB IB Switch ControlsDAQ – Event Building & Selection

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 14 Status ABB/DCB Prototype  wait for J. Gläß talk in a few minutes OASE Status  see presentations of R. Tielert and S. Tontisirin on Wednesday DAQ Controls  just heard H. Essel InfiniBand Test-Cluster  4 node cluster ordered, will be probably be delivered next week  Contacts established to larger IB based clusters to allow scaling tests with realistic cluster sizes in the future.

11 October rd FutureDAQ Workshop --- Walter F.J. Müller, GSI 15 The End Thanks for your attention We acknowledge the support of the European Community- Research Infrastructure Activity under the FP6 "Structuring the European Research Area" programme (HadronPhysics, contract number RII3-CT ).