1. Belle Trigger/DAQ Workshop 2005 - Workshop Summary - R.Itoh, KEK Shinshu University, 2/17-18/2005 28 participants Shinshu Univ. Matsumoto Catsle and Mountains

2. Workshop Goals 1. Confirm the feasibility and compatibility of the near-term upgrade plans between Trigger/DAQ and detector groups. * We need prompt upgrade of DAQ to cope with the luminosity increase in coming years. * The upgrade should be as much as compatible with the upgrade for Super B. 2. Discuss readiness of our design for Super KEKB with L=5x10 35 cm -2 sec -1. * Previous design was for L=1x10 35 cm -2 sec -1 with rather pessimistic assumption on trigger rate. * Update trigger rate estimation using the latest data and verify our design.

4. CDS Agenda was used to manage program and talk slides! You can reach the page from http://belle.kek.jp/workshops/TDAQ2005

5. Current Status DAQ dead time during data taking (2/19-20/2005) w/o injection w/ injection Average dead time during data taking = 7.2 % @ 500Hz L1 rate = ~4% (readout) + ~2% (Inj. VETO) + ~1% (CDC current limit.) Dead Time Fraction ~3.5% deadtime added due to injection VETO  Estimation of dead time with increased luminosity

6. Possible DAQ condition until SuperB upgrade - L is ~1.5 x 10 34 in 2005  Ave. trigger rate ~ 500 Hz => intrinsic DAQ deadtime ~ 3-4%  can live with current DAQ - L will increase up to 2~5 x 10 34 in 2006-20xx (with crab cavity installed)  Ave. trigger rate ~ 1-2kHz => intrinsic DAQ deadtime >20%  needs effort to reduce deadtime and to increase processing power for event builder/L3 farm - L > 10 35 in 20xx (xx can be 08~11?) by SuperB upgrade  Ave. trigger rate >10kHz (max. 30kHz) => pipelined DAQ is really necessary >10 x CPU power for EB/L3 farm “ Clear and Present Danger” by Tom Clancy

7. Trigger rate (@L=5x10 35 ) ● According to present knowledge, – Luminosity term is quite large. 200Hz x Luminosity ( unit:10 34 cm -2 sec -1 ) 10kHz at L = 5x10 35 cm -2 sec -1 – To reduce it, better trigger system is necessary. – To keep tau events and to suppress two-photon events – HER beam dominantly affects the background trigger rate. ● Trigger rate  P x I HER  I 2 HER ● 200Hz at I HER =1.27A  0.2x(4.1/1.27) 2 = 2kHz ● One worry : Lower  y * 6mm  3mm ● Good point : More bumps at Tsukuba straight section to reduce the vacuum pressure. Uno

8. Iwasaki

9. Trigger Rate Readout deadtime Total efficiency A) 500Hz 4 % ~ 91 % B) 1KHz ~20 % ~ 75 % C) 2KHz >50 % <45 % D) 10 KHz ~100 % ~ 0 % SuperKEKB Current condition * We need to manage B) and C) even before SuperKEKB upgrade without interrupting scheduled accelerator running. 1. Upgrade of readout system to pipeline-based system Needs some upgrade! Expected performance without upgrade of frontend readout * Another possibility is further sub-division of slow detector readout (like ECL), but LeCroy does not support FASTBUS TDCs any longer..... 2. Upgrade of Event Builder/RFARM also. * To remove 650Hz barrier

10. * Replace existing FASTBUS TDC system with COPPER based pipelined TDC system(no dead time) detector by detector during scheduled accelerator shutdown time (Summer, Winter). - Start from the detector with the longest readout deadtime (CDC) next FY. - Repeat replacement year-by-year and move to fully pipelined readout system in 3 years. - Further upgrade (for SuperB) are managed by replacing FINESSE + increasing no. of modules. * Modularize backend DAQ (event builder+reconstruction farm) and add new units whenever more processing power is required. * Keep intrinsic dead time ~ 5% even with increased luminosity * Upgrade is consistent with further SuperB detector upgrade * Upgrade scenario is as much as independent of SuperB budget profile * Do not interrupt accelerator running by the upgrade “Smooth” adiabatic upgrade scenario to SuperB DAQ No distinction between “near-term” and “SuperB” upgrades

11. Common Readout Platform: COPPER ● O(100) of the COPPER boards are used by each sub- detector system. - Developed in cooperation with KEK electroncs group PMC Processor Trigger Generic PMC slot FINESSE On-board Ether FINESSE modules Form factor = VME 9U COPPER module by KEK. - Common Readout Module to handle pipelined readout - Digitizers are implemented as daughter cards (FINESSE) - Works at >30KHz! Higuchi Ready for Mass Production

12. Near-term Upgrade of Digitizers Replace FASTBUS TDC system with COPPER based TDC system COPPER : Common readout module to handle pipeline readout Controller TDM Gate Generator FPI LeCroy 1877S Event Builder FastEther Trigger (SEQ) VME 6U FASTBUS Network SW Readout PC COPPER LeCroy 1877S COPPER TT RX TT RX TT RX TT-SW VME 9U X n Event Builder GbE FastEther Trigger

13. Near-term Upgrade of Event Builder/RFARM Current DAQ : Readout subsystems are connected to single Event Builder through point-to-point network connection - Modularize (Event Builder + RFARM) as a unit - Have multiple units CDC TOF ECL KLM VTX Event Builder RFARM CDC TOF ECL KLM VTX Event Builder RFARM Event Builder RFARM Event Builder RFARM Transfer Network Matrix L=1.5x10 34 cm -2 sec -1 /unit

14. ..... ~1000 Pipeline ROM(COPPER) (pipelined readout) ~50 Readout PCs..... >10 Event Building Farms >10 L3 Farms Transfer Network mass storage Input: ~ 100K channels all components are Linux-based PC's Design for SuperKEKB Current DAQ

15. L1 trigger rate 500 Hz 20 kHz Maximum trigger rate 650 Hz 30 kHz Event size at L1 40 kB/ev 300 kB/ev Data flow rate at L1 25 MB/s 9 GB/sec Data flow at storage 13 MB/s 250 MB/sec Subdetector readout 26 >1000 HLT reduction 2 12 Belle(L=1.5x10 34 ) SuperKEKB(L=5x10 35 ) Estimated DAQ condition at SuperKEKB Updated!!

16. Detector Electronics @ SuperKEKB Quick Summary - SVD : CMS APV25 chip - Pixel : MAPS(CAP3) or “striplet”(APV25) - CDC : 2 stage upgrade 1) Pipelined TDC with Q-to-T conversion (shorter shaping time) 2) ADC with waveform sampling (10bit@>100MHz) - ECL : Wave form sampling needed to manage pileup effect (14bit FADC@2MHz for barrel, >20MHz for pure CsI) - TOP/RICH : Need to manage pixel photo-detector * Time stretcher, HPTDC, Analog pipeline - KLM : Readout scheme is not so much different from Belle's regardless of choice of detection device (RPC/Sci. Tile) * "hit" info multiplexing + on-board data compression

17. TDC FINESSE 24 ch LVDS input (96 ch / COPPER) – ~150 COPPER boards in total for the SuperBelle CDC. ● Time resolution 0.78 ns/bit (w/ 40MHz clock) ~ 27  m position resolution ● Dynamic range = 17 bit equivalent to 100  s pipeline depth ● Linearity = 0.49% TDC chip: AMT-3 Originally designed for ATLAS http://atlas.kek.jp/tdc Higuchi

18. * Current prototypes : channel density is too low * Needs to develop high-density 10bit/100MHz ADC (~ 100 ch / module (VME 9U) ) -> can be used for CDC and PID 8 ch differential input ● Sampling clock = 65 MHz ● Dynamic range = 12 bit ● Linearity = 1.2% ● Equips 512 word/ch FIFO (500 MHz FADC FINESSE of 2 ch / 8 bit has also been developed) The FINESSE with higher channel density is under design. Flash ADC FINESSE Higuchi

19.  128x Wilk ADCs 8 chan. * 256 samples 8x HS Analog out, 1x MUX out (COPPER2) 2004/5 Hawaii FINESSE Efforts CuEval HPTDC TOF/TOP/F-DIRC electronics LAB2 Varner

20.  HPTDC FINESSE HPTDC chip Spartan-3 FPGA COPPER Interface 32-channels (8 channel HP mode) Varner

21. Overview of APV25 ● Based on IBM 0.25µm technology ● Input stage: – shaping time: 50nsec – ENC(e)=270+38/C (pF) or 2000 electrons at 45 pF input capacitance – Analog deconvolution is possible but will not be used in Belle SVD. ● 160 out of 192 stage analog pipeline operated at 40 MHz clock – ~4 µsec trigger latency – 30 events can be queued for readout ● 40 MHz analog multiplexing (3+  ) µsec for 128 channel readout) ● 2 mW/channel SVD Tsuboyama

22. Readout electronics front end: APV25, Vienna repeater (not the final version, a new version with AC coupling and base line restore will be ready in April) 15m CAT5 cable (like SVD2) back end: Vienna readout system with programmable APV sequencer and fast ADCs (same as on CMS Pixel FED) Schwanda

23. 4 ADC 4 times 10 bit data 4 clocks with adjustable phase Altera Daughter 9 inputs 9 data proc + FIFO’s P1P1 P2P2 64 (or 32 bit) bit data bus 40 MHz+4 control lines 9 lines with information for trigger proc. Fast data transfer to PCI P3P3 VME protocol Altera Altera daughter with final FIFO Data for trigger, serial Schematics of the ADC-Pixel with the possibility of single channel processing and output with data for trigger processor (largely exists) 2 clocks phase shifted TTC TTC input optical connection 12 in p ut o pt Delayed clock and control sig.distributer, VME control CMS P0P0 Transmit crate clock, control signals and event number Control bus * Signal processing on FPGA for occupancy reduction Vienna group has two ideas on the algorithm Schwanda

24. Nishida PID

26. Schwartz * Wave form sampling is required

27. Schwartz

29. Role of FINESSE - Two implementations are cosidered 1. On-board digitizers - Original idea - ~ 100 channels / 4 FINESSE cards = 1 COPPER module - AMT based TDC, high-speed ADC cards are being developed 2. Interface to outside digitizer system - For complicated readout electronics unified with digitizers i.e. SVD (APV25), Pixel (CAP), ECL, etc. - Standard interface FINESSE is being considered ex. Optical S-link

30. Upgrade Timeline FY2004: - Complete R&D on COPPER  Almost done TDC FINESSE design and production  in progress FY2005: Summer - Replacement of EFC TDCs with COPPER TDCs - 2 nd unit of Event Builder + RFARM Winter (during crab cavity installation) - Replacement of CDC readout FY2006: - New timing distribution system - Replacement of KLM and TRG readout FY2007: - Replacement of TOF and ACC readout - 3 rd unit of Event Builder+RFARM (if necessary) - SVD readout upgrade + inner SVD upgrade - Full upgrade of readout for ECL (wave form smpl.) FY20xx: - Full upgrade with new electronics / new FINESSE - Operation with >10 units of Event Builder+RFARM Super B

31. Conclusions of TRIG/DAQ WS 2005 1. Adiabatic upgrade scenario to pipelined readout + modularized backend meets both requirements by near-term and SuperB upgrades. -> The scenario was approved. - Upgrade project has already started. 2. The design goal for DAQ at SuperKEKB has been updated and it was confirmed that current approach can satisfy the requirements even with the luminosity of L= 5 x 10 35 cm -1 sec -1. 3. COPPER based readout system is ready for mass production. 4. Development of FINESSE cards for each subdetector is now going on.

32. Backup Slides

33. 6x6 network matrix Dual Xeon(3.06GHz), RedHat9 PC 1000Base-T SenderReceiver Test Bench at KEK Performance test of Transfer Network Matrix K.Ito(Tokyo) Evolution of data transfer rate was studied by adding reciever nodes up to 6.

34. Measured Performance  # receiver PC = 1  # receiver PC = 6 At typical data size typical data size increase K.Ito - Performance of Transfer Network Matrix is proven to be scalable up to 6 output nodes. No performance degradation by adding 2 nd,3 rd... Event Builder+RFARM units

35. COPPER based TDC LeCroy 1877S Multi-Hit TDC * 96ch/module, 16hits/channel, 16bit(500ps/LSB) * equipped with LeCroy's MTD133B chip COPPER + TDC FINESSE x 4 * Based on AMT3 chip (16bit(780ps/LSB)) - Some difference from MTD133B * depth of multi-hit buffer (256 words for 24ch) * method of multi-hit recording (FIFO vs. Buffer full handling) * TDC FINESSE - 48 ch / 2 FINESSEs - The same input connectors as that of 1877S's (16 inputs * 6 ) - differential ECL input -> needs LVDS conversion Idea: Build a TDC module compatible with 1877S as possible using COPPER + FINESSE TDC => Pipelining of digitizers becomes possible without modifying frontend electronics Higuchi's talk

36. Evolution of intrinsic dead time for higher trigger rate w/o Event Builder Upgrade * Dead time evolution is not linear to trigger rate. * Prediction is obtained by a fit to 0 ~ 550 Hz region with a 2 nd order polynomial. * Prediction has a large ambiguity.