1. Takeo Higuchi Institute of Particle and Nuclear Studies, KEK on behalf of the COPPER working group Apr.22,2005Super B Factory Workshop Detector (DAQ/Computing) Parallel Pipelined Electronics

2. Add-on modules Common pipelined-readout platform (motherboard) – Generalized DAQ software. Common pipelined-readout platform (motherboard) – Generalized DAQ software. Design Concept: Low Cost Sub-detector I/F User-defined module for analog part only. Minimizing the development cost. Sub-detector I/F User-defined module for analog part only. Minimizing the development cost. Readout CPU Commercially available PMC module. No hardware development cost. Readout CPU Commercially available PMC module. No hardware development cost.

3. Common Readout Platform: COPPER PMC Processor Trigger Generic PMC slot Sub-detector I/F On-board Ether Form factor = VME 9U Sub-detector I/F slot  4 Pipeline FIFO (1MB)  4 PMC slot  3 32-bit local bus 32-bit 33MHz PCI bus Local-PCI bridge FastEthernet I/F VME I/F Sub-detector I/F slot  4 Pipeline FIFO (1MB)  4 PMC slot  3 32-bit local bus 32-bit 33MHz PCI bus Local-PCI bridge FastEthernet I/F VME I/F Data transfer on COPPER measured >65 MB/s. >30 kHz L1 rate @ 1.6 kB/ev. Data size corresponds to typical SuberKEKB DC/COPPER. Data transfer on COPPER measured >65 MB/s. >30 kHz L1 rate @ 1.6 kB/ev. Data size corresponds to typical SuberKEKB DC/COPPER.

4. PMC / PMC Processor PCI Mezzanine Card – 100% compliant with the PCI. – Suitable for high density applications. – Many commercial products are available: Ethernet cards, GbE cards, memory modules, CPUs, etc. RadiSys EPC-6315 – PentiumIII 800 MHz w/ 512 MB main memory. – 32-bit 33/66 MHz PCI bus interface. Contains Linux kernel image PCI NICPMC NIC

5. Sub-Detector I/F: FINESSE FINESSE: 186  76 mm 2 daughter card on the COPPER – Receives and digitizes sub-detector analog signals. – Holds the digitized signals for the level-1 trigger latency. – Outputs the data to the COPPER. Some FINESSE variations – TDC – ADC – TDC “tandem”

6. SuperBelle CDC Readout: QTC + TDC QTC: charge to time conversion Q event 0t DRIFT L1-trigger drift charge to sense wire t T(Q)T(Q) t STOP Q-to-T conversion t t1t1 t2t2 TDC & ADC at the same time, simplifying the readout scheme. t 1 = drift time t 1 – t 2 = drift charge 9U-VME sized QTC module with LeCroy MQT300 QTC chip 9U-VME sized QTC module with LeCroy MQT300 QTC chip

7. For tracking performance Position resolution < 130  m For dE/dx performance Dynamic range10 bit Linearity< 0.5 – 1.0% Other boundary conditions Expected single rate per wire100-200 kHz Readout ch#~ 15 k in total For tracking performance Position resolution < 130  m27  m/bit (0.78 ns) For dE/dx performance Dynamic range10 bit17 bit Linearity< 0.5 – 1.0%0.49% Other boundary conditions Expected single rate per wire100-200 kHz Readout ch#~ 15 k in total24 ch/chip SuperBelle CDC Readout: QTC + TDC TDC: time to digital conversion Requirement FINESSE with AMT-3 TDC chip (originally designed for ATLAS) AMT-3 TDC chip

8. AMT - 3 TDC Chip Up to 17 bit timing measurement: coarse + fine counters – Pulse edge timings:coarse + fine – Trigger timings:coarse Trigger matching – Only the edges associated to Level-1 trigger are output. Level-1 pipeline provided (256 edges) Coarse counter:13-bit bunch crossing counter (40MHz @ LHC) Fine counter:additional 4-bit time memory cell Coarse counter:13-bit bunch crossing counter (40MHz @ LHC) Fine counter:additional 4-bit time memory cell SuperBelle system clock Bunch crossing rate: LHC = 40 MHz, KEKB = 508 MHz. We choose SuperKEKB system clock = 42.33 MHz = 508/12. SuperBelle system clock Bunch crossing rate: LHC = 40 MHz, KEKB = 508 MHz. We choose SuperKEKB system clock = 42.33 MHz = 508/12.

9. Level-1 pipeline FIFO (256 edges) Readout FIFO (64 edges) Channel buffer (4 edges) Channel buffer (4 edges) Channel buffer (4 edges) Trigger matching Trigger timing FIFO (8 events)  24 AMT-3 24-channel inputs AMT-3 output Level-1 trigger inputs AMT - 3 TDC Chip – Cont’d collision time L1 trigger trigger latency (~5  s?) edge search OKNG Trigger matching

10.  of AMT-3  No hardware output indicating FIFO full – Unexpected data loss is only notified by an “error flag” in the next event output. Readout “readout FIFO” of AMT-3 quickly to COPPER DREADY DATA AMT-3 FPGA w/ FIFO FPGA w/ FIFO SCLK: 42.3 MHz BUSY No further trigger

11. AMT-3 L1-Pipeline Depth  Less depth: 256 edges shared by 24-input channels – LeCroy TDC (we’re currently using): 16 edges / each channel. – Is it sufficiently large?  MC simulation is needed. MC parameters – Assuming SuperBelle drift chamber – # of input channels: 24 – Random wire hit: 100 kHz (Poisson distribution) – Level-1 trigger rate: 50 kHz (Poisson distribution) – # of gen. words per wire hit: 2 for signal edges + additional for noise – Level-1 trigger latency: 5  s Extreme case

12. “Toy” MC Simulation: Buffer Usage Level-1 pipeline usage sampled at every clock [edges] Entries / 1 edge 13.7 s 1.37 s 137 ms 13.7 ms FIFO full point Extrapolate … =  ~10 4 Extrapolation assuming the right side slope unchanged. FIFO/AMT-3 won’t get full before O(10 5 ) sec. Very preliminary

13. TDC FINESSE with AMT-3 Block diagram of TDC FINESSE (not finalized yet) AMT-3 Data formatter Trigger reply logic FIFO-full monitor Busy generator FIFO 42.3MHz clock Level-1 trigger Trigger busy Local-bus I/F Sub-detector COPPER event FIFO Local bus (A7D8) COPPER FIFO full Xilinx Spartan-3

14. [Readout data] COPPER The First AMT-3 Signal on COPPER Pulse generator NIM  ECL converter FINESSE AMT-3 250 ns 4.2  s w#1: 00111010010111011110110110000110 w#2: 00111010010110011110111011011010 @ ch 0x0b content tag (0x3 = edge data) ch id. (ch = 0x0b) 1: leading edge 0: trailing edge 17-bit edge timing pulse width Measured pulse width dist. peak = 251 ns rms = 0.93 ns CPU w#1: 00111010010111011110110110000110 w#2: 00111010010110011110111011011010

15. Future Plan Short term (-Jul.): System test – TDC resolution study with test input pulses. – Investigation of FIFO full behavior of AMT-3. – Stability tests: stability against random trigger, long-run stability, etc. – Software development: device driver, readout software, etc. Middle term (Aug.-Nov.): First step integration to Belle DAQ (EFC) – EFC is the smallest sub-detector (total = 320 ch). – EFC has two alternative readout paths (CDAQ + EFC own): failsafe and easier crosscheck. Long term (Dec.-): CDC readout with COPPER based system

16. First Step Integration into Belle DAQ Replace Belle EFC DAQ with COPPER EFC = Extreme Forward Calorimeter Forward EFC Backward EFC 160 ch COPPER TDC TDC  3 FASTBUS VxWorks EB VME crate COPPER  3 Ethernet switch PC TDC “Tandem” FINESSE Local STOR Local STOR

17. Summary We have developed module-structured system for SuperBelle readout consisting of COPPER boards and FINESSE cards. The QTC and TDC will be employed to readout SuperBelle CDC data. We use AMT-3 TDC chip. The TDC FINESSE card with the AMT-3 has been developed. Level-1 pipeline depth in the AMT-3 is examined by MC simulation. We have observed the AMT-3 is correctly digitizing the input pulses. The first step integration of the COPPER system into the Belle DAQ is planned in this August using EFC.

18. 16 ch 8 ch TDC “Tandem” FINESSE LeCroy TDC:6 connectors COPPER: 4 FINESSE slots To use present analog cables… – We’ve made a ‘tandem-FINESSE’ module equipped with 3 connectors. Channel density – 48 ch / tandem-FINESSE. – Same as present LeCroy TDC. Tandem-FINESSE AMT-3  2

19. Readout PC and Event Builder copper0 copper1 copper5 … Readout PC Present event builder Switching hub 100 Base-T GbE Readout PC: – CPU : Intel Dual Xeon 2.46 GHz – OS : RedHat9(2.4.18-20smp) Readout PC: – CPU : Intel Dual Xeon 2.46 GHz – OS : RedHat9(2.4.18-20smp) Data transfer performance Faster readout PC and/or software tuning are requested for SuperBelle use.

20. Calibration Run Calibration run – Daily detector calibration with test pulse and/or test trigger. Readout PCs should take responsibility to collect data – O(100) COPPERs / sub-detector  6 readout PCs  Event building mechanism is needed for calibration run.

21. CPU Boot - up Mechanism Compact Flash – Boot up procedure is well established. – Distribution of updated software to all CF cards is non- trivial. – Write access to CF card shortens the card lifetime. – CF card is getting out of date. Network boot – Distribution of updated software to all CF cards is easy. – We have no established method yet to boot-up EPC- 6315 from network. – Boot-PC farm serving multiple boot requests must be developed. It is never trivial.

22. Run Control copper0 copper1 copper5 … NSM Gateway NSM NSM or other mechanism To avoid increase of NSM nodes, we put NSM gateway (like SVD2). fbdaqprivate network Readout PC (l0efc) NSM Master