1. EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 TPS Timing System & Plan of Machine Protection System TPS Timing System & Plan of Machine Protection System Chun-Yi Wu TPS Control Team NSRRC, Hsinchu, Taiwan June 16, 2011

2. 2 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 TPS Timing System –Hardware –Timing network –Timestamp for TPS timing system –Operation status of TPS linac timing Plan of Machine Protection System –Hardware –MPS network structure –Response time of fast protection system –Health check Summary Outline

3. 3 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011

4. 4 Gun Linac LTB BTS Booster Ring Storage Ring 333.333 msec T RF = 2.00139 nsec T BR = T RF x 828= 1.657 μs T SR = T RF x 864= 1.729 μs T coinc = T BR x 24= 39.768 μs BR-CLK SR-CLK COINC-CLK Sync every 39 μs (24 turns of BR and 23 turns of SR) 0 0 24 23 TPS Machine Clocks Transfer to SR 3 GeV – BR Ext Energy Ramping 150 MeV – BR Inj t

5. 5 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Timing System Selection Event system is the most advanced timing system. It adopt by many synchrotron light sources. Performance and functionality are pretty well for synchrotron light source applications. Select event system are very nature! Form factor selection TPS accelerator controls will adopt cPCI as standard EPICS platform. Only 6U form factor will support for Phase I. Negotiate with MRF to redesign the EVG/EVR. Delivery of the first lot of event system module in late 2010. First operation in Spring 2011 – delivery service for the TPS linac operation. Distribution system –Long distance (~300 m): single mode fiber is most cost effective (> 100 m). –Short distance (< 50 m): OM3 multi mode fiber will be used.

6. 6 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 cPCI-EVRTG-300 (e-Gun Trigger) Fan-Out Concentrator Timing System Hardware Universal I/O TTL Interlock Input Module Universal I/O TTL Input UNIV-TTLIN Universal I/O TTL Output UNIV-TTL Universal I/O TTL Output Module w/ Delay Tuning Universal I/O NIM Output UNIV-NIM Universal I/O LVPECL Output Module Universal I/O LVPECL Output Module with Delay Tuning Universal I/O HFBR-1414 Output UNIV-HFBR-1414 Universal I/O HFBR-1528 Output UNIV-HFBR-1528 Universal I/O modules cPCI-EVG-300 cPCI-EVR-300 GUN-RC-203/300 Fibre + Transceiver

7. 7 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 cPCI-EVG-300 cPCI-EVR-300 cPCI-EVRTG-300 GUN-RC-203 Gun Trigger Receiver Event Generator Event Receiver Event Receiver with Gun Trigger and Fine Delay Fanout Concentrator cPCI-FOUT-CT-8 TPS Phase I Timing modules

8. 8 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Structure of TPS Timing System Serializer & Transmitter Registers Dividers Sync Sequence RAM 0x2A …….. 0x28 …….. 0x26 …….. 0x24 …… 0x22 …… 0x20 Optical Fan out concentrator Event Receiver (EVR) 8 IRQ Clocks Triggers Event Receiver (EVR) 8 IRQ Clocks Triggers Event Receiver (EVR) 8 IRQ Clocks Triggers Event Generator (EVG) 8 bit Ext Bus 8 Ext Triggers (1PPS, 1 MHz, Beam Loss, Beam Abort …) RF 499.654 MHz Mains 60 Hz RF/4

9. 9 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Multiplexed counter 1 SR revolution clock generation (  216) Multiplexed counter 0 Booster revolution clock generation (  207) Multiplexed counter 7 Coincidence clock generation (  4968) Event Priority Encoder Sequence RAM 1 Sequence RAM 2 0 1 2 3 4 5 6 7 Event Code Event Frame DistributionBitsDistributionBits Fiber Optic Link Divide by 4 Divide by 1 to 256 Synchronize to Coincidence clock Phase shift (delay) 0 to 25.5 msec Trigger Event 0~7 RF Master Oscillator AC mains voltage Transformer 60 Hz) cPCI-EVG-300 Sequence RAMs alternate. One event RAM may be modified while the other one is active sending events to control injection.  20 (3 Hz)  24 (2.5 Hz)  30 (2 Hz)  40 (1.5 Hz)  60 (1 Hz) 124.9135 MHz Rep. rate 499.654 MHz 578.303 kHz 603.445 kHz 25.14 kHz (39.772  s) Orbit Feedback Time Tick 125 MEvents/sec Beam Trip Command Univ Input 1~12 External Interrupt Optical Transceiver SR: 864 = 2 5 x 3 3 = 2 2 x 3 2 x 24 = 4 x 216 578.303 KHz (1729.197 nsec) BR: 828 = 2 2 x 3 2 x 23 = 4 x 207 603.445 KHz (1657.147 nsec) Coincident Freq 4 x 4968 25.14 KHz (39.772  s) SR: 518.4 m BR: 496.8 m Trigger Universal Input x Mapping Registers Distributed Bus 0~7 EVR (Uplink Event) Trigger IN

10. 10 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 De- Serializer 2 K Data Buffer cPCI Interface Delay Width F/F S C S C Pol Set Event CLK Trigger Reset Clock Recovery Recovered Event Clock Event Mapping (RAM) Univ Output 0, 1 Univ Output 2, 3 Univ Output 8, 9 Univ Output 4, 5 Univ Output 6, 7 Univ Output 10, 11 Delay Width F/F S C S C Pol Set Event CLK Trigger Reset cPCI-EVR-300 Distribution Bus Data 8 bit Events 8 bit UnivOutMapx Registers Optical Fiber from EVG Uplink Logic Trigger IN Trigger OUT Front Panel Input Mapping Registers

11. 11 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 De- Serializer 2 K Data Buffer cPCI Interface Delay Width F/F S C S C Pol Set Event CLK Trigger Reset Event Mapping (RAM) Univ Output 0, 1 Univ Output 2, 3 GTX4 Pulse Mode Frequency Mode Pattern Mode Delay Width F/F S C S C Pol Set Event CLK Trigger Reset cPCI-EVRTG-300 Distribution Bus Data 8 bit Events 8 bit UnivOutMapx Registers Optical Fiber from EVG Trigger OUT GTX5 Pulse Mode Frequency Mode Pattern Mode GTX6 Pulse Mode Frequency Mode Pattern Mode GUN-TX-203 Mode SFP GTX7 Pulse Mode Frequency Mode Pattern Mode GUN-TX-203 Mode SFP Diff OUT Fiber OUT Fiber OUT Clock Recovery Recovered Event Clock

12. 12 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 EVG EVRTG-300 Gun Trigger EVR-Linac EVR-Booster PS EVR- Injection/Extraction #1 EVR-CIA01 CIA 01 Fanout Concentrator EVR-CIA02 CIA 02 Fanout Concentrator CIA 24 Fanout Concentrator EVR-RF 1 EVR-CSCR EVR-RF 3 EVR-RF 2 EVR-Booster RF Drift Compensator 350 Meter Signal Mode Fiber 60 Meter OM3 Fiber EVR-300 Linac Trigger EVR-BBF CIA 23 Fanout Concentrator EVR-LTB/BTS Diag 400 Meter environmental temperature sensing OM3 Fiber EVR-CIA EVR- Injection/Extraction #2 EVR-CIA24 Master RF Frequency Standard Timing Master Fanout Concentrator Fanout Concentrator Fanout Concentrator Fanout Concentrator Fanout Concentrator EVR-300 Uplink Timing System Distribution Fanout Concentrator 1st levelx 1 2nd level x 4 3 rd level x 24 Total 29 units The propagation delay time due to fiber is ~5 ns/m. In order to achieve almost simultaneous receipt of event codes at EVRs. All fiber lengths are equal.

13. 13 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 GPS Antenna EVG 1 MHz 1 PPS TPS Control Network Trigger Events TPS Timing Network Mains 60 Hz cPCI EPICS IOC CPU EVR Triggers/Clocks Output Triggers, Interlock Input Triggers/Clocks Output 499.654 MHz Rubidium Frequency Standard NTP server “Second” Event Generator 0x700x71 RF RF Distribution DBUS5 0x44 0x45 Beam loss MPS 10 MHz 1 PPS EPICS IOC UTC “Second” Fanout CPU EVR Time reference for TPS control system

14. 14 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Tx Rx External 1PPS “Second” Register “Second” Counter Load “Second” Shift Register Count Up Clock cPCI or VME64x Bus Load τ Clock Generator τ Set by the host CPU on-demand (system boot, cold start) Bit 0 Existed EVG core Possibility to use cPCI-EVG-300 generate “SECOND” events Host CPU can acquire UTC second from NTP server at system boot time or anytime when need. 0x70 0x71

15. 15 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Planned Beamline Timing Interface EPICS IOC cPCI-EVRTG-300 Control Network Fiber Patch Panel to nearby CIA (4 Pairs OM3 Fiber) UNIV I/O 0 (TTL or NIM) Short Gate UNIV I/O 1 (TTL or NIM) Long Gate UNIV I/O 3 (TTL or NIM) SR Clock UNIV I/O 4 (TTL or NIM) Spare UNIV I/O 5 (PECL) RF Clock UNIV I/O 5 (PECL) RF Clock 5 ~ 20 ps Trigger or Clock (dependent on modules used) ~ 30 psec drift (worst case) for a few degree of ambient temperature variation (without drift compensation) Jitter Cleaner (Option) ~ 300 m Fiber Link (with/without drift compensation) Dedicated Fiber Link (Option) Control Network + Timing Network Fiber Link f RF or f RF /N, N = 4,5,6 ? Low Jitter Clock ( < 100 fsec ?) Synchrolock-AP, FEMTOLOCK, etc. (< 500 fs) Standard Supports Commercial fs Locker

16. 16 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Transceiver Demodulator Pulse Out GUN-RC-203 (one channel) Delay Width Trigger Encoded Clock RF CLK Encoded Clock Pulse Out cPCI-EVRTG-300 ( SFP port, GTX6 or GTX7 ) Encoded Clock Tx Rx Fiber Link Recovered Event Clock Micrel SY100EP196 Delay Line 1024 step ~ 9 psec/step Pulse Generator Output Transceiver Modulator Tx Rx To GUN-RC-203 Event Clock Delay ≡ 8 nsec step (Pulse Generator) + 2 ns (Phase Shifter) + 10 ps step (Delay Line) Width ≡ multiple of event clock period 8 ns (Pulse Generator) UNIV-TTLIN-IL@UNIV0/1 for inhibit GUN-TX-203 Mode - Operation Scheme External inhibit Iuput Phase Shifter 0, 2, 4, 6 nsec

17. 17 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 TPS E-Gun Trigger Options Single Bunch Current < 1 nsec Multi Bunch with 500 MHz Modulation Current 2 nsec 50 ~ 1000 nsec Current 2 nsec 50 ~ 1000 nsec Arbitrary bunch pattern ? MRF GUN-RC-203 MRF GUN-RC-300 We still consider e-gun trigger need to support arbitrary bunch pattern or not?

18. 18 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Prototype Operation GUI TPS Main GUI E-Gun Trigger module Timing Master Prototype TPS Timing Summary page

19. 19 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 EVG/EVR/EVRTG Configuration pages

20. 20 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 FCT #1 vs. RF < 10 ps cPCI-EVR-300 TTL output Module vs. RF < 20 ps Preliminary Jitter Measurement of Timing System LTB FCT #1 (Linac test site ) Single Bunch Beam

21. 21 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011

22. 22 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Machine Protection System – Purpose Latch input event Transmit the input event to somewhere Apply interlock rules and activate actuator Reset the latch after the input event removed

23. 23 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Global Machine Protection System (MPS) PLC Network EPICS CA access (read status, interlock reset, ~ 100 msec response time) Trip beam command, post-mortem trigger, …etc. (to event system) Strategic to achieve high reliability - High reliable PLC - Heart beat - Redundancy - Failsafe TPS SR TPS Booster Cell #.. Interlock PLC Sub Unit Cell #1,2 Interlock PLC Main Unit Beam position interlock Beam trip interlock RF status Interface to the safety/interlock which are responsible various groups … Intra-PLC Communication Link Fiber-optic FA Bus Type 2 ~ 2 msec response time ( ~ 2 msec response time) PLC with Embedded EPICS IOC Cell #.. Interlock PLC Sub Unit Cell #.. Interlock PLC Sub Unit Cell #.. Interlock PLC Sub Unit Cell #.. Interlock PLC Sub Unit Cell #.. Interlock PLC Sub Unit Cell # 21, 22 Interlock PLC Sub Unit Cell # 23, 24 Interlock PLC Sub Unit  Redundancy dual loop can be installed later or necessary  Economic design by heavily used remote I/O  Heartbeat ensure system is alive Cost Saving Design

24. 24 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 CIA 1 CIA 2 CIA 3 CIA 4 CIA 5 CIA 6 CIA 7 CIA 8 CIA 11 CIA 10 CIA 12CIA 13 CIA 14 CIA 15 CIA 18 CIA 17 CIA 16CIA 21 CIA 24 CIA 23 CIA 22 CIA 19 CIA 20 50 m Fiber 300 m Fiber Configuration of MPS PLC Communication link Main Unit PLC 10 m Fiber Sub Unit (Remote I/O) F3LR02-ON Link module

25. 25 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 TPS Fast Protection System Timing system EVR MPS system PLC Fast event (Trip RF signal) Actuator (RF station) Timing system EVR MPS system PLC Fast Protection system Slow Protection system TPS timing system can be used to transmit fast event to actuator. –EVR modules have uplink event generation capability. –Fan out concentrator modules can concentrate signals form eight EVRs/downstream and forward the signals upstream. –EVG module has external input to enable 8 event triggers.

26. 26 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 It’s necessary to measure the time delay of fast protection system. Time delay come from Fiber optic propagation time and timing system processing time –Optical fiber propagation time is around 5 ns/m. –Processing time depends on EVG/EVR/Fan-out concentrator modules. Response Time of TPS Fast Protection System

27. 27 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Measure Response Time of TPS Fast Protection System MPS-Trip In MPS-Trip Out 5 m(fiber) Delay(EVR1) ： 2.45 μs (Uplink) Delay(EVR2) ： 4.42 μs (Downlink) 10 m(fiber) 310 m (fiber) MPS-Trip (decode Uplink Event) Uplink < 5 μs response time EVG FOUT-CT EVR1 FOUT-CT EVR2 10 m(fiber) MPS-Trip Out MPS-Trip In MPS-Trip Out (EVR2) 1 2 3 1 3 2

28. 28 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Machine Protection System – Event Input Board Input from –Magnet Thermostat –Beam Line dump beam –Front End dump beam –Vacuum dump beam –Orbit interlock –DCCT failed –Heartbeat –…… Output to –MPS PLC –Fast protection system(Timing system) –cPCI EPICS IOC Latch/Reset event function

29. 29 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Machine Protection System – Actuator Output Board Input from –MPS PLC –Fast protection system (Timing System, TTL) –cPCI EPICS IOC (Contact, TTL or 24 V) Contact output to (~ a few ms response time) –e-gun inhibit –RF inhibit ? –…. Open collector output to(~ μs response time) –RF inhibit ? –Heartbeat –Power supply of magnets inhibit –…..

30. 30 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 Heartbeat Machine Protection System Actuator Output Board Actuator output will be normal when heartbeat is normal! Actuator output will be failed when heartbeat stop! Role of the Heartbeat check on Actuator Output Board MPS PLC DO EPICS IOC DO Timing EVR Output To Actuator

31. 31 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 TPS timing system –The first lot of EVG/EVR/Fan-out concentrator modules was received in December 2010 –Setup test system has been started from February 2011 –Linac timing is ready for commissioning of the TPS linac in April 2011 –cPCI-EVR-300 jitter w.r.t. RF clock < 20ps –cPCI-EVRTG-300 jitter w.r.t. RF clock < 10ps –Other issues are being planned Beam Injection/Top up mode Injection Timestamp …. Machine protection system –Provide MPS PLC system(~few ms) and fast protection system(<5us) –Need to –Define and implement MPS interlock logic –Low level Interface definition and specification –Prototype manufacture Summary

32. 32 EPICS 2011 Spring Collaboration Meeting, Hsinchu, June 13-17, 2011 32