1. Wir schaffen Wissen – heute für morgen Automation in LLRF System Zheqiao Geng Paul Scherrer Institut (PSI), Switzerland For LLRF15 Workshop, Shanghai, China Nov. 3, 2015

2. Outline  Introduction and Motivation  Automation in LLRF System  A Systematic Concept of Automation  Implementation of Automation  Summary 2

3. Introduction and Motivation 3

4. Definition 4 Automation or automatic control, is the use of various control systems for operating equipment such as machinery, processes in factories, boilers and heat treating ovens, switching on telephone networks, steering and stabilization of ships, aircraft and other applications with minimal or reduced human intervention. - Wikipedia Applying to Accelerator RF Systems: LLRF Automation is the use of LLRF control system for operating the RF system with minimal or reduced human intervention.

5. Recall A Typical RF Station 5 Basic Functions of LLRF System: Measure the RF field seen by beam accurately and precisely. Provide correct and clean drive RF to pre-amplifier. It may be a result of a feedback controller. Implement interfaces with other subsystems to control or monitor them. Automation will be built upon the basic functions!

6. Goals for Automation of an RF Station 6 Realistic Goals:  Facilitate the operators to operate the RF/LLRF systems by  Automating the low level parameters optimization and settings  Automating the procedures to setup or adjust the RF station  Improve the RF stability by implementing slow feedbacks or feed forward (e.g. adaptive feed forward).  Improve the visibility of the RF system with complex system status detection and derived measurements.  Improve the robustness of the RF system by providing complicated exception detection and handling. Ultimate Goals: This is a dream!

7. Automation in LLRF System 7

8. Automate the Group Parameter Settings 8 A lot of registers in firmware and basic parameters in software need to be set under different working conditions. Setting the values of them as a group will be helpful. The parameter setting here is straight forward mainly related with working conditions. Examples:  Set up the ADC/DAC/Clock chips during startup.  Set a group of parameters or registers when changing the RF station from beam operation mode to RF conditioning mode, for which, the repetition rate, pulse shape or operation limits may be different.

9. Automate the Atomic Processes 9 The atomic processes are operations which are more or less not interruptible and will be finished in short time. The processes are normally based on the initial measurements and the output can be derived characteristic values of the RF system or optimal values for some parameters. Examples:  Derived measurement: measure quality factor and detuning of cavities  Optimization: optimize quality factor, feedback gain  Calibration: calibrate vector sum, drive power, cavity voltage, beam phase  Feedback: iterative pulse shape control; cavity tuning control

10. Automate the Complex System Processes 10 System processes are operations to implement complex procedures which may take long time to finish or can be interrupted and resumed by the user during execution. Normally one or more atomic processes will be executed in a system process. Examples:  RF station starting up or stopping  High power RF components conditioning  Systems status checking and automatic fault recovery

11. Automate the Control of Multiple RF Stations 11 To fit into the global control context of the accelerator, multiple RF stations, especially the ones within the same section, need to be operated in a coordinated way. Examples (for Linac):  Set the overall energy gain and beam phase for a section of accelerator.  Optimize the energy gain distribution for different RF stations to maximize the energy headroom and improve the overall reliability (e.g. energy gain limited by cavity quench limit).  Compensate the failed cavity or failed RF station.

12. Hierarchy of Automation – Functions 12 Multiple RF Station Control Automation Multiple RF Station Control Automation Individual RF Station Control … Automation of Atomic Processes Automation of System Processes Execute one or more Automation of Group Parameter Settings Setting a group of parameters can also be viewed as an atomic process

13. A Systematic Concept of Automation 13

14. 14 Automation Concepts  Job  System Process  Operation Mode  Virtual RF Station  Complete View of Automation Concepts

15. Job 15 Atomic processes including group parameter setting can be modeled as “Job”. A Job will be controlled by an event (can be user command, a timer or a command from system processes), take some inputs, executed with some parameters and generate some outputs. Most of the algorithms related with RF domain knowledge (e.g. cavity model, feedback model and RF signal processing) will be implemented in the Jobs. Job - Execute Domain Algorithms Job - Execute Domain Algorithms Inputs Control: Start, Pause, Stop, Reset… Parameters Outputs

16. Job Example 1: Correct DAC Offset 16 Goal: Remove the carrier leakage from vector modulator Control: Started by user clicking a button Inputs: Current DAC waveform; current measurement of vector modulator output Parameters: Calculation region in the pulse Outputs: Offset correction of DAC

17. Job Example 2: Identify I/Q Imbalances 17 Goal: Qualify the amplitude and phase imbalances of vector modulator Control: Started by user clicking a button Inputs: I/Q averages of DAC output and vector modulator output for each scan step Parameters: Phase scan start and step values Outputs: I/Q imbalances, amplitude and phase actuation errors

18. A List of Typical Jobs 18 Generate tables for pulsed operation (set point, feed forward …) Correct DAC offset Calibrate the loop phase and loop gain Calibrate the vector sum (for vector sum control) Calibrate the cavity drive/reflected power Calibrate the cavity voltage and beam phase Identify the cavity quality factor and detuning Optimize the relative phase between cavities (for vector sum control) Optimize the quality factor of cavities (for vector sum control) Optimize the feedback gains Adapt the feed forward Control superconducting cavity Lorenz force detuning Control normal conducting cavity detuning – with cooling system or others …… A LLRF algorithm library can be very helpful to implement the jobs!

19. 19 Automation Concepts  Job  System Process  Operation Mode  Virtual RF Station  Complete View of Automation Concepts

20. System Process 20 A system process will execute multiple jobs. The execution can be in two typical patterns:  Sequential: The system process can be modeled as a procedure.  State Dependent: The system process can be modeled as a Finite State Machine (FSM). Inputs 1 Job 1 Start Job 2 Inputs 2 Inputs n Outputs 1 Outputs 2 Outputs n … Param 1 Param 2 Param n End Job n State 1 Entry/Do/Exit State 1 Entry/Do/Exit Job x Inputs x Outputs x Param x State 2 Entry/Do/Exit State 2 Entry/Do/Exit Job y Inputs y Outputs y Param y Job z Inputs z Param z Outputs z

21. System Process Example 1: RF System Startup 21 Execute Jobs: -Check system status -Correct DAC offset Execute Jobs: -Determine and set klystron high voltage and drive Execute Jobs: -Calibrate loop gain and loop phase -Adaptive feed forward Automatic parameter setting: -Set modulator state -Set interlock system mode -Set RF switch on/off -Set feedback on/off -Set trigger delays Execute Jobs: -Recover from RF fault trips

22. System Process Example 1 (cont.) 22 Set target state Display the current state and transfer status.

23. System Process Example 2: High Power RF Conditioning 23 Courtesy: Alex Jürgen, PSI

24. System Process Example 2 (cont.) 24 Courtesy: Alex Jürgen, PSI

25. 25 Automation Concepts  Job  System Process  Operation Mode  Virtual RF Station  Complete View of Automation Concepts

26. Operation Mode 26 An operation mode identifies a recognized working situation of the RF system. Defining the operation modes helps to set the RF system to a specified working scenario with simple commands. When entering an operation mode: -A group of parameters will be set to specified values. -Some Jobs will be enabled or disabled. -For system processes, different states can be reached.

27. 27 Operation Mode Example 1: SwissFEL RF Station Mode

28. 28 Operation Mode Example 2: iPhone Flight Mode With one button: -Wi-Fi, Bluetooth and cellular connections are disabled. -No longer be able to make or receive calls, texts, or e-mails, or browse the Internet …

29. 29 Automation Concepts  Job  System Process  Operation Mode  Virtual RF Station  Complete View of Automation Concepts

30. Motivation 30  In a Linac based FEL machine, beam physicists usually adjust the phase and amplitude of a group of RF stations simultaneously in main Linac. SwissFEL LCLS

31. Virtual RF Station Concept 31 Individual RF Station Controller K1 K2 K3 K4 VRF Station Controller Individual RF Station Controller K2 Individual RF Station Controller K3 Individual RF Station Controller K4 Appear as a single RF station to the user! - User reading: amplitude and phase of VRF, the vector sum. - User writing: amplitude and phase set point of VRF. ACC Beam

32. 32 Virtual RF Station Planned for SwissFEL  Determine individual RF station amplitude and phase set points based on the overall energy gain and beam phase settings.  Calculate the total energy gain and beam phase of the VRF station with the vector sum of individual RF stations.  Simplify the interface between LLRF system and global beam based feedback system.  Compensate the failed RF stations if there are enough energy gain headroom in other RF stations.

33. 33 Automation Concepts  Job  System Process  Operation Mode  Virtual RF Station  Complete View of Automation Concepts

34. Hierarchy of Automation – Models 34 Individual RF Station Control … Job System Process Execute one or more Group Parameter Setting Operation Mode Configure Execute Multiple RF Station Control Automation Multiple RF Station Control Automation Virtual RF Station Setting a group of parameters can also be viewed as a Job

35. Implementation of Automation 35

36. 36  Requirements and General Architecture  Automation Tools in Popular Control Systems  Automation Implementation for SwissFEL LLRF Implementation of Automation

37. 37 Identify Jobs from Description of Use Cases The use cases describe how the RF system should be operated with the LLRF system. By describing use cases, the jobs can be easily identified by marking up the key words potential for automation. Example: Close feedback loop of an SC RF station 1.Tune the cavities with motor tuners to have optimal pre-detuning. 2.Switch on the piezo tuner to compensate the Lorenz force detuning. 3.Calibrate the vector sum. 4.Correct the loop phase and loop gain. 5.Close the feedback loop. Complex use cases are candidates of System Process!

38. 38 Architecture of Individual RF Station Automation  Choice 1: A single software process to implement all functions – operation modes, system processes and jobs.  Choice 2: Use multiple software processes for jobs, system processes and operation modes. They can also be distributed in different CPUs. Individual RF Station Automation Process Job 1 Job 2 Job n … System Process 1 System Process 2 System Process m … Operation Mode Control System processes can be implemented as threads Job Process Job 1 Job Process Job 2 Job Process Job n … System Proc. Process System Process 1 System Proc. Process System Process 2 System Proc. Process System Process n … Manager Process Operation Mode Control The manager process is used to manage other processes.

39. 39  Requirements and General Architecture  Automation Tools in Popular Control Systems  Automation Implementation for SwissFEL LLRF Implementation of Automation

40. 40  A tool to run programs written in State Notation Language (SNL). SNL is a “C” like language to program FSMs. The SNL maps well to the UML state diagrams.  With built-in Channel Access client to communicate with device controllers.  No GUI based state diagram editor. EPICS Sequencer

41. 41 DOOCS FSM  FSM editor provides GUI for definitions of states and transitions.  C++ code framework will be generated from the state diagrams and the user can input custom routines, compile and run the FSM.

42. 42 LabVIEW Statechart Toolkit  A state diagram editor can be used to create state charts and generate LabVIEW code framework.

43. 43 Top class for a module of high level application Implements functions to create and set up HLA modules Coordinates execution of Jobs: -Job registered with a code -Job executed by a event with the same code Coordinates execution of Jobs: -Job registered with a code -Job executed by a event with the same code Also implements the mechanism to execute a FSM. Each Local PV is a EPICS record which can be defined in other objects. Collects all functions to monitor and control a remote device. Read and write a remote PV with CA. PV name get from a map file. Customized Automation Framework at PSI (OOEPICS)

44. 44  Requirements and General Architecture  Automation Tools in Popular Control Systems  Automation Implementation for SwissFEL LLRF Implementation of Automation

45. 45 General Remarks  EPICS is used as the control platform in SwissFEL LLRF system.  Automation of an individual RF station is implemented in a centralized process as a soft IOC. The software runs in the CPU of the LLRF FPGA board close to the data source to reduce the network traffic.  An library in C language is used to implement all RF and control domain algorithms.

46. 46 Software Architecture of SwissFEL LLRF HLA Classes derived from the OOEPICS framework.

47. 47 LLRF HLA at SwissFEL C-band Test Stand

48. 48 Virtual RF Controller Functions

49. 49 VRF Control Hardware Architecture IFC1210 IFC_TC2 FM-S14

50. Summary 50

51. 51  Automation is very important for the operations of a complex RF system. It is one of the key functions that the LLRF system should provide.  There are big freedom to implement the automations. A systematic consideration is helpful to keep the implementation clean and clear.  To which level the automation should be implemented is a compromise considering the following issues:  Available man-power  Development difficulties  Maintenance cost Summary

52. The End! Thank You! 52