HEP UCL Cambridge University Imperial College London University of Manchester Royal Holloway, University of London University College London Matthew Warren, UCL Matthew Warren, UCL 22 March 2007 EUDET/CALICE DAQ Overview

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 2 PC/s Ideal DAQ Structural Overview Detector ASICs on e.g. ECAL slabDetector ASICs on e.g. ECAL slab Front-End (FE)Front-End (FE) -FE-Interface (DIF): Detector specific -FE Link/Data Aggregator (LDA): Generic Data-link (FE to Off-Detector Receiver)Data-link (FE to Off-Detector Receiver) CCC-link (Clock+Control+Config to FE)CCC-link (Clock+Control+Config to FE) DAQ PCDAQ PC -Off-Detector Receiver/s (ODR) -Drives CCC-link -Data Store LDA CCC-link ODR Store Data-link DIF ASICs FE DIF ASICs DIF ASICs …

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 3 FE Structure Detail We have 2+ types of detector to readout. Divide the FE into a 2 part, tiered system 1) Detector Interface module (DIF) 1) Detector Interface module (DIF) -Detector specific interface -Includes power connectors -‘Local’ systems (e.g. stand-alone clock) -Debug connectors 2) Link/Data Aggregator module 2) Link/Data Aggregator module (LDA) (LDA) -Collects data from many ‘DIF’s -Drives data Off detector link -Receives and distributes C+C -FPGA Development board -BUT: We would might like to read-out slabs individually first… Link/ Data Aggregator PC ECALSlab ECAL DIF Link/ Data Aggregator PC HCALLayer HCAL DIF HCALLayer ECALSlab ECAL DIF C+C Fanout

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 4 Data-link (+CCC) Use most common networking fibre-optics:Use most common networking fibre-optics: -Multimode with LC connectors -SFP (mini-GBIC) interfaces -1Gbit rate (maybe tuned to multiple of machine freq.) -Ethernet Control up-link NOT via fibre, initially.Control up-link NOT via fibre, initially.

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 5 Off-Detector Receiver (ODR) PCI Express CardPCI Express Card Virtex 4, FX100 FPGA (big!)Virtex 4, FX100 FPGA (big!) Hosts opto-linksHosts opto-links -2xSFP, 2xHSSDC2 on board Source of C+C (Control link)Source of C+C (Control link) -Initially copper (LVDS) -Later fibre Will use external clock and sync signals for multi-board synchronous operationWill use external clock and sync signals for multi-board synchronous operation

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 6 ODR(2) - Status Firmware AND software well underway: PCIe interface DONEPCIe interface DONE Register read/write DONERegister read/write DONE DMA access DONEDMA access DONE Ethernet InterfaceEthernet Interface IN-PROGRESS IN-PROGRESS DDR2 Interface IN-PROGRESSDDR2 Interface IN-PROGRESS Linux driver DONELinux driver DONE Optimised Disk StoreOptimised Disk Store IN-PROGRESS IN-PROGRESS Manager Software IN-PROGRESS IN-PROGRESS Performance profilingPerformance profiling IN-PROGRESS IN-PROGRESS Clock and Control UplinkClock and Control Uplink NOT-STARTED NOT-STARTED Ethernet Interface DDR2 Interface PCIe Interface Control/ Status Reg. Block Internal RAM Test Data Gen Arbiter Driver Manager Software Firmware

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 7 Detector Interface (Cam, IC) -Spec + hardware DIF to Link/Data Aggregator (Cam/Man) -Spec + hardware Data aggregate, format (Man) -Hardware + firmware LDA to ODR opto-link (Man, UCL) -Hardware + firmware ODR (RHUL, UCL, Cam) -firmware ODR to disk (RHUL) -Driver software Local Software DAQ (RHUL) Full blown Software DAQ (RHUL, UCL, [IC]) LDA PC ECALSlab DIF ODR Driver Opto Opto UK Read-out work (ECAL FE)

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 8 Clock + Control, Integration Keep it simple! -System synchronising signals distributed -All data tagged with common ‘timestamp’ Clock Control (Train- start, Sync) PC ODR PC ODR Fanout (TLU?) LDA Config Slab Config LDA Slab Slave outMaster In

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 9 ‘TLU’ Requirements Lets assume 32 ‘Slabs’. -Each Slab needs: Clk; Train, (Trigger?) But could use LDA here. At 8 slabs/LDA = 4 LDAs With minimum 1 LDA/ODR = 4 ODRs -Each ODR needs: same + Fast Control(data)  4 Signals Clock, Train, Trigger, Control  Fanout of 32 slabs + 4 LDA + 4 ODE =40, OR more likely 4 LDA + 4 ODR = 8. OR more likely 4 LDA + 4 ODR = 8. -TLU acts as Master or Slave for signalling -TLU generates signals stand-alone -We presume to use LVDS everywhere. ?? Do we try to use the TLU for fanout, or just as a ‘machine’ interface?

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 10 Extra: Optical Switch

HEP UCL 22-Mar-2007EUDET/CALICE DAQ Overview 11 Optical (Layer-1) Switching Part of the UK CALICE is to evaluate the use of a “layer-1” switch. 1) DAQ PC failover - Redirect data to spare unused DAQ PC on the fly 2) “Router” - Can change data destination per bunch-train. Regulate load by sending data directly to free resources 3) Programmable optical patch panel (large installation) Manufacturers offering similar products, in same price range e.g. Glimmerglass, Polatis - difficult to differentiate between them Decided on PolatisDecided on Polatis -can switch dark fibre (i.e. not MEMS based) -Multimode fibre capable -Fastest switching time (20ms) 16x16 array with 50 μ m multimode LC connectors16x16 array with 50 μ m multimode LC connectors