EPICS Collaboration Meeting Timing Workshop April 24, 2012

General Goals of our Discussion Starting with the requirements for timing systems Overview of linac FEL timing requirements Overview of storage ring light source timing requirements Any other accelerator systems? Are there new requirements for next generation machines? Improved performance in terms of stability, drift over long distances? Higher repetition rates, broadcasting more complex timing patterns? Awareness of multiple beam destinations and AC power timeslot issues. Integration and synchronization issues with other subsystems (LLRF, feedbacks, MPS, …) Overview of currently available hardware Outlook for new hardware Form-factors for discrete modules Embedded FPGA timing receivers

Cont. - General Goals of our Discussion 2. Overview of currently supported software Available drivers Support for different platforms, operating systems Timing dependant software such as Beam Synchronous Data Acquisition (BSA) and Beam Line Data (BLD). Overview of current architecture of EPICS timing software Limitations of record processing Proposed architecture of C code for faster performance List of requested new features or improvements Hardware Firmware Software Operational issues and experience Improved timing system diagnostics User interfaces and operator GUIs

Not Covered Will not be giving a general timing tutorial! This is a workshop intended for people working in the field with an interest toward defining new requirements and new areas of development

AGENDA Opening discussion on the scope of topics and additions to the agenda The bulk of the meeting will be roundtable discussion with the following “seed” presentations: Overview of LCLS timing requirements – Patrick Krejcik LCLS Timing System and upgrades - John Dusatko Requirements for storage rings, plus issues for the SwissFEL – Timo Korhonen Requirements for the MaRIE Project including multiple beam destination and timeslot issues – Eric Bjorklund MRF products and future plans – Jukka Pietarinen BNL timing driver for MRF – Michael Davidsaver LCLS timing software and plans – Kukhee Kim.

Logistics Bldg 48 Redwood A Meeting Room 01:30pm Timing Workshop   01:30pm Timing Workshop 02:30pm Coffee Break 03:00pm Workshop (cont.) 05:00pm Workshop ends 06.30pm dinner - TBA

LCLS Timing Requirements Overview Timing Workshop April 24, 2012

What’s so special about 360 Hz?

Timing in the Context of the Overall Controls Architecture

Scope of Work Distributed timing system hardware and software for LCLS-II General Requirements (similar to LCLS-I): Independently operated timing system and network that operates synchronously with LCLS-I and other linac timeslots Provide triggers, timestamps, event codes for the linac to undulator e-beam systems and to the photon systems Timing patterns to be enhanced to become destination-aware to support beams to multiple undulators at different beam rates Timing patterns to be enhanced to become MPS-aware to support data acquisition multiple undulators at different beam rates No longer require a link to the legacy controls (SLC) timing

Timing System Design

BSA (Beam Synchronous Acquisition)

Design Design based on mature LCLS-I system design Using COTS components MicroResearch Finland Event Receivers, VME, PMC … EVRs BPMs: 1 per IOC, 5 BPMs per IOC both SL and RF. BPMS and BCS Hard and Soft lines processed separately. RTMs For BPMs only. Strip lines require two triggers per BPM (10/crate) RF BPMs one. CV01 VME control for CAMAC. Only first ten sectors costed. No one assigned as system lead. FODUs and Fiber Optic Connector Panels For fiber distribution. Sectors 20-30 FODUs have spare capacity for now. FO Jumpers Trunks estimated through DeSalvo. Jumpers necessary to connect to EVRs and complete infrastructure for timing distribution TRD - Timing Referene Distribution. 119MHz with fiducial for MPS, BCS and diagnostic Toroids

Lessons Learned from LCLS-I All fiber terminations should be done only by qualified Technicians Multimode fiber should be installed for long haul fiber runs in the linac Long haul fiber installation design should incorporate FODU termination boxes Timing system event codes should be made aware of MPS trips so that data acquisition by BPMs etc. is correctly handled Plan for greater flexibility in new event codes for photon users

LCLS II Fiber Optic Plant

Timing Distribution

Timing 119 MHz Reference Distribution

LCLS-II EVRs

Timing Software slides from Kukhee Kim

What does EVG send to EVR? Timing Fiber 360 Hz Pipeline n+1 n+2 n EVG Ioc-in20-ev01 EVR Event Codes [0..255] Time Stamp w/Pulse Id 192 Modifier Bits Meeting Name/Presentation Title Location, Date

Setting up the Beam Rate

Setting up the Beam Rate with evGUI

Creating Rate Groups

Creating Rate Definitions

Defining the Patterns

Master Beam Control Meeting Name/Presentation Title Location, Date

EVR Diag. Screen (D3) Trigger Selection Key for Front Panel (D1) Board information (D5) Regular Trigger Control (D2) Board Control and Monitoring (D4) Extended Delay Front Panel Trigger (D6) VME IRQ delay configuration Don’t Use It!

Trigger Panel (T3) matrix switches for mapping the events to the trigger channel (T1) event code for trigger generator (T2) enable/disable the event code (only for the trigger generator)

How to control the trigger Triggers Panel PVs Event Code PV Enable/Disable PV (T1) (T2) Triggers Panel PVs EVR Diag. Panel PVs Prescaler PV (D4 and D5) Matrix switches on the trigger panel (T3) Delay PV Width PV Status PV Trigger Event Polarity PV Enable/Disable An Internal Clock Prescaler Delay Counter set Polarity control clear Width Counter Hardware Channel Trigger Driver Trigger Generator

Form Factor/OS dependency Event Module for RTEMS/vxWorks Event Module for Linux EVR Processing Logic EVR Processing Logic BSA BSA erRecord erRecord devErMrf devErMrf drvErMrf drvLinuxEvr mrfCommon/mrfVme64 erapi VME EVR Hardware PMC EVR Hardware PMC EVR Hardware Works with old register map Works with modular register map (new)

High Level Screen

Issue 1: FWD/BWD Propagation Low Level PVs on Diagnostics Screen High level PVs on Events Screen Event Number on Trigger Screen Event Number(3*) Forward/Backward Propagations (*) Hard-coded Event number and Trigger Configuration(2*) Save/Restore for High Level PVs (*)

Issue 2: Event Code Invariant Delay Each Event Code has its own offset Each event code has to have different offset The delay has been hard-coded in the EVG EVG assumes there is no duplicated offset These offsets are involved in the hardware trigger calculations for trigger delay on EVR side But, the offset PV is hard-coded for each trigger channel Thus, the changing event code (or, changing trigger selection) makes different delay Event information in EVG Trigger Delay Calculation in the EVR

Event Code and Delay Delay Calculation To make “event code invariant delay”, need to fix the hard- coded part Require to detect changing event code (or, changing trigger selection) Re-calculate the forward propagation Actually, the offset of event code is a function of event code and trigger configuration Clock Rate Fiducial to Beam: Constant Desired Delay Event Code Offset by EVG

What is the Timeslot Zero Crossings at AC 3 phases lead out the 6 time slots Same Timeslot in different peroid shows exactly same AC phase configuration. Active Timeslot LCLS: TS1 and TS4 FACET: TS2 and TS5 XTA: TS3 and TS6 Primary Timeslot

Bean Synchronous Acquisition (BSA) Acquire beam dependent scalar values across multiple IOCs to analyze the correlations among the values which are acquired at the same pulse Maintain the buffer up to 2800 points The buffered values can be averaged up to 1000 samples Up to 20 different BSA requests are available Each BSA requests can specify: Beam Code Inclusion/Exclusion Masks for the Event Pattern Measurement Count (number of data points) Average per Measurement Severity Level

Bring up the EDEF screeen Event Global Screen

Make EDEF Reservation

Pipeline, Pattern & Event code EVG EVR Step 5 Pipeline Advancing in the EVR Generate New pattern at !3 pulses prior! Pipeline Advancing in the EVG Fiber connection to EVR Trigger/Event Generation by the Event Code Dealing with the next1 pattern Pipeline index =1 is hard-coded in the database Decide event code list with the !Next1! pattern Re-construct EDEF data (for BSA) from the MOD5 & EDEF Masks Construct EDEF data (for BSA) from the MOD5 & EDEF Masks

BSA processing DEDEF 0 DEDEF 1 Update EDEF table After the pipeline advancing DEDEF n Timestamp (active) Timestamp (Init) avgDone flag Severity DEDEF 19 Internal BSA Data Table BSA device name1 BSA device name2 BSA0 AO record: Data receptor BSA1 DATA PV BSA device name M for BSA device name M Update data value, timestamp, status and severity which come from the DATA PV BSA n BSA device name L BSA 19 AO record does the BLUE box and make record processing for correct BSA record(s). And the BSA record update the BSA buffer.

Glossary Beam Code – Part of the PNET broadcast. A set of 5 modifier bits. LCLS runs on beam code 1. Beam Rate Control – The act of selecting a Rate Definition within a Rate Group. BGRP (Beam Group) – MPG analogous to an EVG Rate Group BSA (Beam Synchronous Acquisition) – The act of filling buffers on an IOC at a specific specified time. Burst Mode – An operating procedure where an operator requests n pulses followed by zero rate. Event Codes – Part of the packet the EVG sends to the EVR. There are 256 event codes: [0..255]. Event Definition – A way to reserve a set of buffers used by BSA. Event System – Describes the way timing of beam pulses is set up. EVG (Event Generator) – IOC that passes event codes, time stamps, and modifier bits to the EVR. EVR (Event Receiver) – A card in an IOC that receives timing data from the EVG. evGUI (Event Graphical User Interface) – Java GUI used to program Rate Groups, Rate Definitions, and Modifier Bits for use by the EVG. Master Beam Control – A box sitting next to most control room OPIs used to select a Rate Definition. MPG (Master Pattern Generator) – The non-EPICS SLC “micro” to be replaced by the EVG. Modifier Bits – Part of the packet the EVG send to the EVR. 192 bits of specific timing information. PABIG (Pattern Bit Generator) – 3D PVs in the EVG used to hold Rate Definnitions x Modifier Bits x time for each Rate Group.

Glossary Pattern – Refers to the Beam Code + Modifier Bits Pipeline – Refers to the continuous stream of data sent from an EVG to the EVR. PNBN (PNET Bit Numbers aka Timing Pattern Bits) – Defines the names and bit position and bit width of the Modifier Bits. PNET (Pulse Id Network) – The network the MPG uses to send out the Timing Pattern. Pulse Id – A 360Hz counter that gets encoded into the nanoseconds part of the time stamp sent form the EVG to the EVR. It is used to correlate data so 1 beam pulse can be tracked as it passes by each device. Rate Definition – A specific rate to run a specific accelerator, such as “FULL RATE”. Rate Group – A group of Rate Definitions used by a specific accelerator, such as “LCLS”. Rate Limiting – The act of selecting a Rate Definition within a Rate Group. Single Shot Mode – An operating procedure where an operator requests 1 beam pulse followed by zero rate. Timing – Short hand term used to describe the Event System. VMTG (VME Master Trigger Generator) – Part of the EVG that receives the 360Hz fiducial and generates and interrupt to trigger PABIG PVs.