1. XFEL In-kind Review Committee Meeting, 11 May 2009 Parliament British Museum XFEL clock and control system In kind contribution proposal Development and delivery of a clock and control system for 2D detectors at XFEL and other light sources Martin Postranecky, Matthew Warren, Matthew Wing (UCL)

2. 2 Outline Introduction and motivation Conceptual design of clock and control system Project administrative details Our previous experience Summary

3. 3 Introduction Within WP 76: DAQ & Control (C. Youngman) we have started collaborating to provide the (common) clock and control system for the three planned 2D megapixel detectors. Have had discussions with the XFEL timing group and regular DAQ/Train Builder meetings to allow us to start design work. The clock and control system will potentially be operated at other light sources (e.g. LCLS, Spring8 and FLASH): — needs to interface to different bunch structures — needs to be flexible and intelligent Exploit advances in connectivity and data handling provided by the Telecommunication Crate Architecture (TCA) standards, AdvancedTCA and MicroTCA: — compatible with XFEL timing system and Train Builder — flexible, upgradeable, and standardised platform

4. 4 Motivation The megapixel detectors need to: precisely know when a bunch is coming: timed-in, ability to work at different accelerator complexes know what the bunch pattern will be know what kind of data is being taken: “physics”, “calibration”, etc. have the same calibration constants loaded into the front-end electronics A clock and control system needs to provide all of this processing power not such an issue (as with e.g. Train Builder), but needs to be highly efficient it is the link between the accelerator and detector and is an essential part of the whole readout scheme

5. 5 Link to accelerator 200ns 99.4ms 600 μ s Clock and control Master receives, formats and distributes: Clock - 5MHz, multiplied to 100MHz Train Start - Fixed time before train Bunch Pattern -Identifies filled bunches in train Bunch Veto - Fast signal post-bunch 5MHz bunches 10Hz train 5MHz bunches Pre-bunch Lower frequency Arbitrary Patterns Beamline specific structures

6. 6 Overall readout scheme 2D Pixel Detector Detector Unit Backend Processing Readout Crate C+C Crate Timing Receiver Crate Proc- essor C+C Master C+C Slave Train Builder Crate Proc- essor Train Builder PC Switch Machine Storage Controls TCP/IP Network Train Builder Detector Unit Detector Unit 64k Pixel Unit C+C Slave Detector Unit Detector Unit Detector Unit 64k Pixel Unit PC

7. 7 Timing t1 T1 T2 T1+T2 5 MHz clock in TRIGGER in 100 MHz clock out TRIGGER out VETO out D2 D1 VETO in

8. 8 Clock and control system design MicroTCA Crate Crate Processor 100 MHz Clock Start/Info/Stop Bunch Veto FEE Status C+C Master C+C Fanout FEE 5MHz Clock Trig/Telegram Bunch Veto FEE C+C Fanout Slave Timing Interface 1-20MHz Clock Trig/Data Other FEE Timing Receiver XFEL

9. 9 Project schedule Project started (even though not approved), but may need to revise schedule with WP 76 group. Design review Prototypes at UCL Technical review Final boards at UCL

10. 10 Deliverables Phase 1: Report on design and functionality of clock and control system (Aug 2010) Phase 1: Prototype clock and control system used by prototype detectors (Dec 2010) — The prototypes will be in use for prototype detectors so will have much of the functionality of the final system Phase 2: Full clock and control system delivered to 2D Megapixel detector (Jun 2013)

11. 11 Summary of resource request 7.6 FTE of mainly engineering effort with: — 6.2 FTE at UCL — 1.4 FTE at STFC Technology Division for PCB layout and testing — Will hire some new engineering effort, building on existing expertise, so need funding quickly. 82 k€ equipment with main costs: — MicroTCA system (10 k€) — Prototype boards (30 k€) — Production boards (35 k€) Travel to US and Japan to understand LCLS and Spring-8 timing. Usability at different light sources is an integral part of clock and control system. Travel to DESY (and other collaborators) not included in document but are real costs. Have assumed resources to provide system for three 1 Megapixel detectors. Total real cost of ~ 750k€ (estimate and using £1  €1.2).

12. 12 Previous experience The UCL High Energy Physics group and the specific applicants have vast experience in data acquisition, and specifically clock and control, systems for numerous detectors: UCL provided clock and control system for the ZEUS silicon vertex detector and timing for the central tracking detector at HERA Co-developed the DAQ system for current ILC calorimetry R&D programme (CALICE) Co-developer of the DAQ system for the planned CALICE R&D programme: several calorimeters, O(10 4 ) channels, different test beams. UCL provided the clock and control cards. Co-developer on DAQ for ATLAS at the LHC. UCL provided the clock and control interface for silicon strip, pixel and muon sub-detectors. All supported by the technical effort at STFC who have built electronics for a multitude of large-scale projects.

13. 13 Summary We have presented a work programme to provide a common clock and control system for the Megapixel detectors at XFEL and other light sources. The clock and control system is an essential part of the megapixel readout system and the interface to the accelerator. A conceptual design and choice of architecture has been made. The work is integrated into XFEL WP 76: DAQ and controls. We have a detailed breakdown of requested resources which we think is reasonable and will allow us to achieve a timely delivery of the project Our team has a long history of providing such systems for detectors in High Energy Physics. We now seek your approval for the work to start in earnest.

14. 14 Thank you

15. 15 Detailed functionality To receive, process and store timing signals from the timing receiver (TR) housed in the same crate. The timing signals are: 5 MHz bunch CLOCK (for 200 ns bunch spacing); Bunch train TRIGGER, a single pulse sent by the TR to the C&C Master ~15 ms before the start of each bunch train; Event Number = TRAIN ID, i.e. a unique number/identifier for the train; BUNCH PATTERN or Bunch PATTERN ID, i.e. a unique identifier for the various bunch patterns which may be chosen by the user. To receive BUNCH VETO, a hardware-generated pulse to indicate that the current bunch is to be ignored by the detector. To receive STATUS / ERROR (and possibly BUSY) from each front-end electronics (FEE) board. To distribute three fast lines to each FEE: 100 MHz un-interrupted clock, phase-locked to 5 MHz bunch CLOCK; START pulse, followed by TRAIN ID, followed by PATTERN ID, followed by STOP signal; BUNCH VETO signal. To process any BUSY information from the crate controller to stop the following START pulse. To generate all timing signals in stand-alone mode without the TR. To synchronise to other light sources’ timing systems, i.e. to accept external CLOCK and possibly TRIGGER at different frequencies. To provide diagnostic and visual indication of CLOCK, TRIGGER, etc. performance and presence or absence of any FEE.

16. 16 Clock and control Master - Clocking : 2 XTAL 5MHz Clock From TR Ext Clock 0 – 20MHz Prog. Mult/Div 100MHz MPX FEE Clock PLL Mult Local Clock 200MHz Prog. Delay Standalone 100MHz Prog. Delay MPX PLL

17. 17 Clock and control Master - Trigger and Info Trigger + Telegram From TR FEE Trigger Prog. Delay External Interface Ext. Trigger + Info Trigger Pulse Train ID Bunch Pattern BPID Clock 100MHz ETrigger Pulse SA Trigger Pulse SA Train ID SA Bunch Pattern XFEL Interface Select + Process + Store Standalone Trigger + Info LUT

18. 18 Clock and control - Slave 100 MHz Clock Start/Info/Stop Bunch Veto FEE Status C+C Master Link Fan-out FEE C+C Link Fan-in xN