1. EPICS Collaboration Meeting
Timing Workshop
April 24, 2012

2. General Goals of our Discussion
Starting with the requirements for timing systems
Overview of linac FEL timing requirements
Overview of storage ring light source timing requirements
Any other accelerator systems?
Are there new requirements for next generation machines?
Improved performance in terms of stability, drift over long distances?
Higher repetition rates, broadcasting more complex timing patterns?
Awareness of multiple beam destinations and AC power timeslot issues.
Integration and synchronization issues with other subsystems (LLRF, feedbacks, MPS, …)
Overview of currently available hardware
Outlook for new hardware
Form-factors for discrete modules
Embedded FPGA timing receivers

3. Cont. - General Goals of our Discussion 2.
Overview of currently supported software
Available drivers
Support for different platforms, operating systems
Timing dependant software such as Beam Synchronous Data Acquisition (BSA) and Beam Line Data (BLD).
Overview of current architecture of EPICS timing software
Limitations of record processing
Proposed architecture of C code for faster performance
List of requested new features or improvements
Hardware
Firmware
Software
Operational issues and experience
Improved timing system diagnostics
User interfaces and operator GUIs

4. Not Covered Will not be giving a general timing tutorial!
This is a workshop intended for people working in the field with an interest toward defining new requirements and new areas of development

5. AGENDA
Opening discussion on the scope of topics and additions to the agenda
The bulk of the meeting will be roundtable discussion with the following “seed” presentations:
Overview of LCLS timing requirements – Patrick Krejcik
LCLS Timing System and upgrades - John Dusatko
Requirements for storage rings, plus issues for the SwissFEL – Timo Korhonen
Requirements for the MaRIE Project including multiple beam destination and timeslot issues – Eric Bjorklund
MRF products and future plans – Jukka Pietarinen
BNL timing driver for MRF – Michael Davidsaver
LCLS timing software and plans – Kukhee Kim.

6. Logistics Bldg 48 Redwood A Meeting Room 01:30pm Timing Workshop
01:30pm Timing Workshop
02:30pm Coffee Break
03:00pm Workshop (cont.)
05:00pm Workshop ends
06.30pm dinner - TBA

7. LCLS Timing Requirements Overview
Timing Workshop
April 24, 2012

9. What’s so special about 360 Hz?

10. Timing in the Context of the Overall Controls Architecture

11. Scope of Work
Distributed timing system hardware and software for LCLS-II
General Requirements (similar to LCLS-I):
Independently operated timing system and network that operates synchronously with LCLS-I and other linac timeslots
Provide triggers, timestamps, event codes for the linac to undulator e-beam systems and to the photon systems
Timing patterns to be enhanced to become destination-aware to support beams to multiple undulators at different beam rates
Timing patterns to be enhanced to become MPS-aware to support data acquisition multiple undulators at different beam rates
No longer require a link to the legacy controls (SLC) timing

12. Timing System Design

13. BSA (Beam Synchronous Acquisition)

14. Design Design based on mature LCLS-I system design
Using COTS components
MicroResearch Finland Event Receivers, VME, PMC …
EVRs
BPMs: 1 per IOC, 5 BPMs per IOC both SL and RF.
BPMS and BCS Hard and Soft lines processed separately.
RTMs
For BPMs only. Strip lines require two triggers per BPM (10/crate) RF BPMs one.
CV01
VME control for CAMAC. Only first ten sectors costed. No one assigned as system lead.
FODUs and Fiber Optic Connector Panels
For fiber distribution. Sectors FODUs have spare capacity for now.
FO Jumpers
Trunks estimated through DeSalvo. Jumpers necessary to connect to EVRs and complete infrastructure for timing distribution
TRD - Timing Referene Distribution.
119MHz with fiducial for MPS, BCS and diagnostic Toroids

15. Lessons Learned from LCLS-I
All fiber terminations should be done only by qualified Technicians
Multimode fiber should be installed for long haul fiber runs in the linac
Long haul fiber installation design should incorporate FODU termination boxes
Timing system event codes should be made aware of MPS trips so that data acquisition by BPMs etc. is correctly handled
Plan for greater flexibility in new event codes for photon users

16. LCLS II Fiber Optic Plant

17. Timing Distribution

18. Timing 119 MHz Reference Distribution

19. LCLS-II EVRs

20. Timing Software slides from Kukhee Kim

22. What does EVG send to EVR?
Timing Fiber 360 Hz Pipeline
n+1
n+2 n
EVG
Ioc-in20-ev01
EVR
Event Codes [0..255]
Time Stamp w/Pulse Id
192 Modifier Bits
Meeting Name/Presentation Title
Location, Date

23. Setting up the Beam Rate

24. Setting up the Beam Rate with evGUI

25. Creating Rate Groups

26. Creating Rate Definitions

27. Defining the Patterns

28. Master Beam Control
Meeting Name/Presentation Title
Location, Date

29. EVR Diag. Screen (D3) Trigger Selection Key for Front Panel
(D1) Board information
(D5) Regular Trigger Control
(D2) Board Control and Monitoring
(D4) Extended Delay Front Panel Trigger
(D6) VME IRQ delay configuration
Don’t Use It!

30. Trigger Panel
(T3) matrix switches for mapping the events to the trigger channel
(T1) event code
for trigger generator
(T2) enable/disable the event code
(only for the trigger generator)

31. How to control the trigger
Triggers Panel PVs
Event Code PV
Enable/Disable PV
(T1)
(T2)
Triggers Panel PVs
EVR Diag. Panel PVs
Prescaler PV
(D4 and D5)
Matrix switches on the trigger panel
(T3)
Delay PV
Width PV
Status PV
Trigger Event
Polarity PV
Enable/Disable
An Internal Clock
Prescaler
Delay Counter
set
Polarity control
clear
Width Counter
Hardware Channel
Trigger Driver
Trigger Generator

32. Form Factor/OS dependency
Event Module for RTEMS/vxWorks
Event Module for Linux
EVR Processing Logic
EVR Processing Logic
BSA
BSA
erRecord
erRecord
devErMrf
devErMrf
drvErMrf
drvLinuxEvr
mrfCommon/mrfVme64
erapi
VME EVR Hardware
PMC EVR Hardware
PMC EVR Hardware
Works with old register map
Works with modular register map (new)

33. High Level Screen

34. Issue 1: FWD/BWD Propagation
Low Level PVs on Diagnostics Screen
High level PVs on Events Screen
Event Number on Trigger Screen
Event Number(3*)
Forward/Backward Propagations (*)
Hard-coded Event number and Trigger Configuration(2*)
Save/Restore for High Level PVs (*)

35. Issue 2: Event Code Invariant Delay
Each Event Code has its own offset
Each event code has to have different offset
The delay has been hard-coded in the EVG
EVG assumes there is no duplicated offset
These offsets are involved in the hardware trigger calculations for
trigger delay on EVR side
But, the offset PV is hard-coded for each trigger channel
Thus, the changing event code (or, changing trigger selection) makes different delay
Event information in EVG
Trigger Delay Calculation in the EVR

36. Event Code and Delay Delay Calculation
To make “event code invariant delay”, need to fix the hard- coded part
Require to detect changing event code (or, changing trigger selection)
Re-calculate the forward propagation
Actually, the offset of event code is a function of event code and trigger configuration
Clock Rate
Fiducial to Beam: Constant
Desired Delay
Event Code Offset by EVG

37. What is the Timeslot
Zero Crossings at AC 3 phases lead out the 6 time slots
Same Timeslot in different peroid shows exactly same AC phase configuration.
Active Timeslot
LCLS: TS1 and TS4
FACET: TS2 and TS5
XTA: TS3 and TS6
Primary Timeslot

38. Bean Synchronous Acquisition (BSA)
Acquire beam dependent scalar values across multiple IOCs to analyze the correlations among the values which are acquired at the same pulse
Maintain the buffer up to 2800 points
The buffered values can be averaged up to 1000 samples
Up to 20 different BSA requests are available
Each BSA requests can specify:
Beam Code
Inclusion/Exclusion Masks for the Event Pattern
Measurement Count (number of data points)
Average per Measurement
Severity Level

39. Bring up the EDEF screeen
Event Global Screen

40. Make EDEF Reservation

41. Pipeline, Pattern & Event code
EVG
EVR
Step 5
Pipeline Advancing in the EVR
Generate New pattern
at !3 pulses prior!
Pipeline Advancing in the EVG
Fiber connection to EVR
Trigger/Event Generation
by the Event Code
Dealing with the next1 pattern
Pipeline index =1 is hard-coded in the database
Decide event code list with the !Next1! pattern
Re-construct EDEF data (for BSA) from the MOD5 & EDEF Masks
Construct EDEF data (for BSA) from the MOD5 & EDEF Masks

42. BSA processing DEDEF 0 DEDEF 1 Update EDEF table
After the pipeline advancing
DEDEF n
Timestamp (active)
Timestamp (Init)
avgDone flag
Severity
DEDEF 19
Internal BSA Data Table
BSA device name1
BSA device name2
BSA0
AO record: Data receptor
BSA1
DATA PV
BSA device name M
for BSA device name M
Update data value, timestamp, status and severity which come from the DATA PV
BSA n
BSA device name L
BSA 19
AO record does the BLUE box and make record processing for correct BSA record(s).
And the BSA record update the BSA buffer.

43. Glossary
Beam Code – Part of the PNET broadcast. A set of 5 modifier bits. LCLS runs on beam code 1.
Beam Rate Control – The act of selecting a Rate Definition within a Rate Group.
BGRP (Beam Group) – MPG analogous to an EVG Rate Group
BSA (Beam Synchronous Acquisition) – The act of filling buffers on an IOC at a specific specified time.
Burst Mode – An operating procedure where an operator requests n pulses followed by zero rate.
Event Codes – Part of the packet the EVG sends to the EVR. There are 256 event codes: [0..255].
Event Definition – A way to reserve a set of buffers used by BSA.
Event System – Describes the way timing of beam pulses is set up.
EVG (Event Generator) – IOC that passes event codes, time stamps, and modifier bits to the EVR.
EVR (Event Receiver) – A card in an IOC that receives timing data from the EVG.
evGUI (Event Graphical User Interface) – Java GUI used to program Rate Groups, Rate Definitions, and Modifier Bits for use by the EVG.
Master Beam Control – A box sitting next to most control room OPIs used to select a Rate Definition.
MPG (Master Pattern Generator) – The non-EPICS SLC “micro” to be replaced by the EVG.
Modifier Bits – Part of the packet the EVG send to the EVR bits of specific timing information.
PABIG (Pattern Bit Generator) – 3D PVs in the EVG used to hold Rate Definnitions x Modifier Bits x time for each Rate Group.

44. Glossary Pattern – Refers to the Beam Code + Modifier Bits
Pipeline – Refers to the continuous stream of data sent from an EVG to the EVR.
PNBN (PNET Bit Numbers aka Timing Pattern Bits) – Defines the names and bit position and bit width of the Modifier Bits.
PNET (Pulse Id Network) – The network the MPG uses to send out the Timing Pattern.
Pulse Id – A 360Hz counter that gets encoded into the nanoseconds part of the time stamp sent form the EVG to the EVR. It is used to correlate data so 1 beam pulse can be tracked as it passes by each device.
Rate Definition – A specific rate to run a specific accelerator, such as “FULL RATE”.
Rate Group – A group of Rate Definitions used by a specific accelerator, such as “LCLS”.
Rate Limiting – The act of selecting a Rate Definition within a Rate Group.
Single Shot Mode – An operating procedure where an operator requests 1 beam pulse followed by zero rate.
Timing – Short hand term used to describe the Event System.
VMTG (VME Master Trigger Generator) – Part of the EVG that receives the 360Hz fiducial and generates and interrupt to trigger PABIG PVs.