1. COPPER Status T.Higuchi / KEK Jul.4 th, 2009. B2GM Meeting.

2. Belle II Data Stream Belle II detector Signal Digitizer Digital Signal TX Signal RX Readout Electronics Readout PC Analog Signal TX Signal Digitizer Readout Electronics Readout PC On/in/at the detectorIn the electronics hut Readout PC Event Builder Online PC Farm Storage System STOR ~30m cable

3. R&D of the Readout Electronics Present FASTBUS-based DAQ will fail around L1 rate ~1 kHz with DAQ deadtime being 20%. And in Belle II, the L1 rate will be ~10kHz. Call for a new readout electronics that works up to L1 rate ~30 kHz. Keywords of R&D Equipped with pipeline (FIFO) Equipped with online CPU for data processing Usage of commercially available products Modularized structure for easier maintenance Seamless switch from the present system

4. COPPER-II Board Signal RX x 4 or Signal digitizer x 4 Add-on modules to the COPPER called FINESSE RadiSys EPC-6315 – Intel P3 800 MHz – 256 MB memory – Network boot – RedHat Linux 9 Online CPU VME9U size 100 BaseT port x 2 Data transfer and Control Trigger module Onboard FIFO 1MB x 4 on the tail side

5. Signal Digitizer – AMT3 FINESSE AMT-3 TDC chip 2 http://atlas.kek.jp/tdc – 24 LVDS inputs / chip. – 256-edge deep L1 FIFO (pipeline), 8-event deep trigger FIFO. – Only matching edges to the trigger are output. – 0.7ns timing resolution. – Linearity = 251.0±0.9 ns (RMS) for 250.0 ns width pulse. Spartan3 FPGA 2 – AMT3 register control. – Data format. – COPPER local bus interface, COPPER FIFO interface. tandem-structured FINESSE

6. PCI Mezzanine Card Standard 100% Compliant w/ PCI Good for a high density applications Many commercial products – Ethernet cards, CPUs, GbE cards, memory modules, etc. RadiSys EPC-6315 – PMC sized CPU module. – Equipped with Intel PentiumIII 800 MHz w/ 512 MB memory. – RedHat Linux 7.3 or 9, or FedoraCore 1 Linux run on it. – Price = ~\120,000 / module. Bootable from CF card or from network. PCI NICPMC NIC

7. Performance Review RX TDC FINESSE Emulator Clock/Trigger Generator PMC CPU Ethernet COPPER-II The COPPER works > 30 kHz input rate Factor-10 data reduction Required trigger rate Typical trigger rate @ 416 bytes/ev/FINESSE 010203040 Input trigger rate [kHz] 10 20 30 40 Accepted trigger rate [kHz]

8. Performance Review 0 2 4 -4 -2 AMT3 –FASTBUS (unt = AMT3 LSB) Resolution 0.61 LSB (RMS) Residual distribution Crosscheck of the COPPER output and the FASTBUS output. – Signals from the Q-T converter of Belle CDC are branched to the FASTBUS and the COPPER. COPPER FASTBUS From CDC Daisy-chain

9. Performance Review # of hits/TDC Deadtime duration (μs) Normal data size 29.5 μs 2.8 μs COPPER/AMT3 FASTBUS We achieved 90% deadtime reduction with pipelined DAQ. S.Y.Suzuki

10. COPPER History in Belle DAQ 2002-2003 – Start of R&D from a scratch. – COPPER prototype in hand. 2004 – Revision to COPPER-II. – Study of maximum acceptable L1 rate: OK up to 30kHz. 2005 – Debug of AMT3 FINESSE. – Replacement of the Belle EFC DAQ with 6 COPPERs. – Crosscheck with FASTBUS DAQ. – a 2006 – Replacement of part of the Belle CDC DAQ with COPPERs. – Crosscheck with FASTBUS DAQ. – Deadtime performance study. 2007 – Replacement of the Belle CDC DAQ totally with 89 COPPERs. – Replacement of the Belle ACC DAQ with 24 COPPERs. 2008 – Replacement of the Belle TRG DAQ with 26 COPPERs. 2009 – Replacement of the Belle KLM DAQ with 34 COPPERs. – ECL and TOF DAQ are kept untouched. Ready to go

11. COPPER-II COPPER-3 Major motivation for the upgrade – Replacement of disappeared/disappearing parts from the market to the available ones. – Bug fix (replacement bug fixing patches to onboard patterns). Minor modifications – Replacement of the onboard Ethernet to the Gbit-Ethernet. – Length normalization of CLK/TRG lines from TTRX to FINESSEs. – Easier access to a reset switch. – Easier access to VME base address configuration switches. – Change of PMC CPU interfacing connector. – Movement of VME J0 receiver location closer to the J0. – RoHS compliant parts.

12. COPPER-3 Board Looks quite compatible to the COPPER-II. (The board color is changed to turquoise blue)

13. COPPER-3 Compatibility Study Before the study: Tips to operate the COPPER-3 – Use specially created Linux image for COPPER-3 A Linux image for the COPPER-II cannot be used for the COPPER-3. The special Linux for COPPER-3 is ok for the COPPER-II.

14. COPPER-3 Compatibility Study Compatibility study setup – Same setup as the system test for the COPPER-II and AMT3-FINESSE Minor troubles in compatibility study Swap of eth0 eth1 configuration A bug found in the AMT3 device driver AMT3 FINESSE Clock/Trigger Generator PMC CPU Ethernet COPPER-3 1 to 96 signal fan-out RX Compatible data as the COPPER-II system test are readout in this study. – Trivially, the COPPER-3 can be said OK. – The CPU is booted up properly. – The CPU, AMT3-FINESSEs and trigger timing system works fine on the COPPER-3.

15. COPPER-II/COPPER-3 in Belle II DAQ The COPPER-3 is ready to use. – We use the COPPER-3 as the data RX in the electronics hut. We also use existing COPPER-IIs (~250 boards) together with the COPPER-3s. We will purchase ~250 COPPER-3s including spares in coming 2-3 years.

16. R&D of Processor PMC (PrPMC) Problems of EPC-6315 (existing PrPMC) – Future prospect of the product is unclear. Market trend is switching to ComExpress from PrPMC. Vendor (US RadiSys) may stop the product supply immediately. – Both vendor and Japanese HQ are quite unprofessional. The first release of the product was with untested BIOS. The product was revised with an incompatible network chip. Japan US communication is poor. Call for a new PrPMC with concrete future prospect and kind support. Now in bidding procedure. The company will be fixed on Jul.14 th.

17. Specification of New PrPMC CPU – Intel Atom 1.6GHz or more. Main memory – 256MB or more. Mass storage – Onboard 4GB SSD or external media like CF. Network I/F – GbE w/ Intel 82574. – 900Mbps or more. OS – Redhat Enterprise Linux 3 or CentOS 3 or later. OS bootup method – From mass storage – From USB device – From network User I/F – Serial console or USB/PS2 keyboard+mouse + VGA. Power consumption – < 15W (EPC-6315 … 18W)

18. Schedule of PrPMC R&D Jun.19 th Announcement of bid Jul.1 st Meeting w/ companies Jul.7 th Deadline of technical report from companies – Block diagram – Layout proposal of major parts Jul.14 th Box open – Company will be fixed on the day Feb.19 th End of R&D. We have two PrPMCs in hand then.

19. Summary We are confident that the COPPER-based DAQ is in a state of the art. A new COPPER board, COPPER-3, is compatible with the COPPER-II. We are to employ the COPPER-3s for Belle II readout electronics. A new PrPMC, which replaces the EPC-6315 with several problems, is under bidding process. We have them by Feb.19 th.