1. Private Notes on the 3 rd PXD DAQ/Trigger Workshop T.Higuchi (KEK) Jul.8,2011B2GM

2. Before the WS: Liked to Listen to What we mainly liked to listen to – Progress in the ATCA based readout option We understand Giessen and IHEP are working very hard on this issue. We liked to hear about the status of hardware production of revised version and performance study.

3. Before the WS: Liked to Listen to – Cluster finder algorithm Giessen students are working very hard on the cluster finding, track finding, and around. In the last workshop, there were excellent presentations which implement/convert the finding software to the FPGA firmware. We liked to listen to the progress in it. – DHH status and DHH controller status The DHH is a key components that interfaces with the Belle II DAQ and trigger system. We liked hear some progress around DHH from Igor-san.

4. Before the WS: Liked to Present What we liked to present – Progress in the PC readout option As for the backup option, we made some kind of progress on this part. We have rough estimation of the performance of RocketIO→PCIe card. We liked to present that. – PXD DQM status Itoh-san wished to present the PXD DQM status.

5. Before the WS: Liked to Work Out What we liked to work out – Interface between DHH and Belle2link All Belle II detectors other than the PXD should comply a “DAQ and trigger interfacing protocol” being developed by Prof. Liu-san. We liked to discuss interfacing signals between the DHH (controller) ↔ trigger system in the workshop. – Number of links between DHH and ATCA/PC How many links between DHH ↔ ATCA/PC is best? We meant up to which throughput, can the DHH output the data?

6. Before the WS: Liked to Work Out – System clock What system clock does the PXD want to be provided, and which component likes to receive it? It is related to the throughput per link. – PXD integration to CDAQ: ATCA/PC → event builder Especially for the ATCA option case, which option is planned to connect the ATCA to the CDAQ's event builder. SiTCP?

7. Before the WS: Liked to Work Out – PXD DQM Itoh-san wished to raise a question how to implement the DQM for PXD. Preprocessed data with some information lost by DHH/ATCA/PC may be inappropriate for DQM, whose major purpose is to detect a faulty condition. – Injection veto How to implement the injection veto in PXD? How to reduce veto duration?

8. Talk-by-talk Memos Hereafter Not all talks and topics are covered …

9. ATCA System Status... Lange DDR2 bandwidth – Measured 180MB/s... corresponds to 1% occupancy capability – If switched from ISE 10 to ISE 13, then 360MB/s I/O (2% occupancy) capability is verified. – 540MB/s write (3% occupancy) and 140MB/s read and again 14MB/s write (1/10 reduction) is requirement. – We should keep 3% occupancy agreement, while some have studied much lower occupancy may be possible.

10. ATCA System Status... cont’d New ATCA daughter board – 2 x 2GB DDR2 memory equipped. DHH ↔ ATCA daughter bard – 2 optical cables per link. Next steps – Several options on GBE IP-core (SiTCP?). – Board to board data transfer over RocketIO.

11. Compute Node Status... Thomas Temperature monitor – If fan fails and temperature raise is detected, we should stop the run. – Slow control system may detect the faulty condition, another path than the PXD data stream. Linux on embedded powerPC Event ordering (sequence) – Event ordering may be disordered by HLT. In the computing node case, re-ordering is possible.

12. RoI Algorithm... David Memory size – Capability of 4096x4096 PIXs, while only 786x250 is needed for every half ladder. Processing speed – Processing speed can be doubled by using 2 IP-cores in parallel. Algorithm realized for up to 31 ROIs Throughput – Full data throughput works up to 100MHz.

13. DHH... Igor With or without DHH controller – Start jobs which are compatible with both options. 2x SFP+ serial links – Each link has 6.5Gbps capability DHH clock – 3/20 of 508.89MHz (RF) = 76.33MHz.

14. DHH... cont’d JTAG interface (DHH ↔ DHP) – Load of configuration information. – Change of DHP readout mode from zero-suppression to full R/O mode. – Pedestal updates. PXD plans every-few-second updates! DHH-system ↔ CDAQ communication – Sergei started to study the communication part.

15. DHH... cont’d DHH status – Done Synchronous clock distribution with Xilinx FPGA Clock synthesis DHP I/F mechanism – On-going Schematic drawing – TODO Work on slow-control system Schematic-drawing finalization and start of production (around the end of June)

16. xTCA... Zhen’An Liu Production status – 10 boards are produced. New boards with XC5VFX70T will be ready in 2-3 weeks. Board specification – 5x FPGAs – 2x 6.25Gbps optical links per FPGA – 4GB DDR2 memory per FPGA

17. xTCA... Zhen’An Liu Achieved performance – 550MB/s per link! Requirement is 605MB/s per link, and it may be the next challenge. Status of upgrade – Upgraded carrier board is under progress. – PCB layout is almost being finished. DDR2 → DDR3? – To improve the memory speed, DDR2 should be replaced with DDR3, which is, in principle, possible.

18. ATCA / PC Option Decision The ATCA system is 95% sure toward the goal. Of course, there may be a showstopper in the remaining 5%, but at this moment, we conclude and state the ATCA is the default option. It is nothing wrong to continue the PC option study as a backup for an accident in the remaining 5%, and for the future DAQ upgrade other than the PXD DAQ.

19. Deadtime Generator... Igor Accidental data truncation – DHP may truncate data when memory gets full, which causes data loss Protection against the data truncation – Blocking the trigger may work to avoid the data loss. Options – Read of DHP memory fill level, but because of the smallness of its memory, this option is unrealistic. – Prediction of DHP memory fill estimated by toy Monte Carlo simulation.

20. Data Concentration... Dingfelder Track finding in the ATCA – Hit information from SVD are used. Hough transformation is used to find out tracks. Data concentrator – A data concentrator will be responsible to collect the SVD hit signals and output them to the ATCA. – 4x SFP input connectors from the SVD(FTB) and 1x SFP output connector to the ATCA.

21. Sequential Clustering.. Wassatsch Data clustering engine (DCE) for DHH – Free count of pixel per cluster, free input ordering. DCE implementation – Can the DCE be mapped to a DHH FPGA? → 

22. HLT ↔ PXD Interface... Itoh PXD ↔ DAQ interface – Event disordering problem will be (automatically) solved by ATCA option. Location of the ATCA crates and Bonn box – Server room opposite to the shift room... is it fine? Con: Link cables not only from DHH but also from SVD should be needed. – Depends on whether the EH will be accessible or not in the luminosity run → unclear yet → ask Uehara-san?

23. HLT ↔ PXD Interface... cont’d PXD and DAQ effort boundary x10 HLT Network SW PC Network SW ATCA ` ` DAQ group’s coverage PXD group’s coverage

24. PXD-DAQ-Related Tracking... Martin Situation in PANDA – In PANDA, reconstructed track momentum is biased toward positive, which is a “feature” or “known bug” of Runge-Kutta. – Better to check if this happens or not in Belle. Algorithm optimization on GPU The major remaining task – Alignment.

25. Belle II Background Study... Moll Purpose of the study – PXD group’s major concern is occupancy rather than radiation hardness of the ASICs; the PXD's readout ASICs are designed well tolerable against the radiation.

26. PXD Monitor... Sergei Time-to-time pedestal shift (monotonic change) Read of pedestals from the DHP – DHP is controlled via JTAG. – Read action will take place every 5 minutes; the action lasts for 440μs during one of the injection veto timings. → Block the next L1 trigger during the pedestal data are being readout → Blocking mechanism is possible in principle. Upload pedestals to the DHP as well Who calculates pedestals? → under discussion

27. Event Pile Up... Lange Probability of event pile up – For 1 frame, the average probability to have 1 trigger is 30kHz/50kHz = 0.6. – For 1 frame, the probability to have 2 triggers is (0.6^2/2!) x exp(-0.6) = 0.327. This probability corresponds to 2 triggers for every 3 frames. Action on this issue is needed – The calculation shows the pile up probability is more than we expected in the TDR.

28. The 4 th PXD DAQ/Trigger Workshop Dear Kiesling-san and Lange-san, As for the 4th PXD DAQ/TRG WS in Japan, we find a nice hotel in Yugawara city, where Yugawara station is located 73-min-train-ride from Tokyo station. You can find some hint about the hotel from Japanese "hotel reservation portal site", http://travel.rakuten.co.jp/HOTEL/104628/104628_std.html Japanese text only, but I hope photos are universal. The discussion will be on room charge. - Western-style single room... 12,000 yen / night (up to 12 rooms) - Western-style twin room... 14,000 yen / night (6 rooms) - Japanese-style middle-size room... 14,000 yen / night (6 rooms) - Japanese-style large-size room... 14,000 yen / night (1 room) Any kinds of rooms can be used by a single person at the rate above; i.e. 25 rooms for 25 attendants in total. The room charge includes breakfast and Japanese-style dinner (lunch is not), and conference room fee. Yes, the room charge may look expensive especially for students. (We may negotiate with the hotel to share the Japanese-style room by multiple persons, but I guess students do not like room-sharing). How do you like the hotel? Let me have your opinion over e-mail or a face-to-face discussion in the coming B2GM. Cheers, <<= Higuchi My e-mail sent out on Jun.30,2011

29. Do You Like This Hotel? Hotel name = New WelCity Yugawara Entrance Hot spa Semi-Japanese-style guest room Food