1. ORNL is managed by UT-Battelle for the US Department of Energy Differential Beam Current and Errant Beam at SNS W. Blokland, C. Peters, N. Luttrell ESS, Lund, Sweden, November 2015

2. 2 Errant Beam at SNS Introduction CCL SCL,  = 0.61 SCL,  =0.81 DTL RFQ Source MEBT SNS Accelerator Power on Target1.4 MW at 1.0 GeV (1.2 MW now) Pulse on Target1.5 E14 protons (24µC) Macro-pulse in Linac~1000 mini pulses of ~24mA avg over 1ms at 60Hz Errant Beam at SNS: any beam outside the normal operation envelope: Abrupt beam losses Beam current drops or missing mini pulses Too high peak density beam on target

3. 3 Errant Beam at SNS Existing Loss Monitors abort in 16- 30 µs but, unexpectedly, also short losses in SCL lead to accumulating damage –Suspected mechanism: beam hitting cavity surface releases gas or particulates followed by ionization or redistribution, creating an environment for arcing. The arcing can damage the cavity surface  Errant Beam Committee to investigate Data collected from loss monitors, RF waveforms and the modified (60Hz and archiving of data) LabVIEW-based Beam Current Monitors Differential Current Monitor: Why is even short errant beam (<15 µs) important? base material Ion, molecule (radical), electron contaminants Credit:S. Kim Damage mechanism

4. 4 Errant Beam at SNS Analyzing the errant beam data Abnormal RF pulse with beam Normal RF pulse with beam Normal RF pulse with no beam LLRF output 558 µs in CCL 546 µs in HEBT Field Time (usec) 12 µs lost in the SCL

5. 5 Errant Beam at SNS Errant Beam Analysis BLM waveforms selected by clicking on scalars BLM scalars from CCL and SCL Current waveforms Directories of data with auto select of nearest data in other directories BLM scalars from CCL and SCL < 10% of BLM trips were due to the Ion Source/LEBT - BLM trips occur in the first week of a new source installation > 90% of BLM trips were due to Warm Linac RF faults - RF faults occur at different times during the pulse

6. 6 Errant Beam at SNS The majority of trips originate in the Warm Linac RF Adjustments: gradient changes, resonant frequency changes, and preventative maintenance on vacuum systems  fault frequency reduced by more than a factor of two.  SCL downtime was reduced by a factor of six FY12-1FY12-2FY13-1 BUT still need to reduce impact of remaining errant beam (losses or no losses)  either upgrade all BLMs and MPS system or implement single differential current monitor to abort as fast as possible, down to about 6 µs with special hookup to MPS. Errant beam loss from 30 to 15 events a day

7. 7 Errant Beam at SNS CCL SRF,  = 0.61 SRF,  =0.81 RFQ Protecting the Super Conducting Linac The choice was to implement a differential BCM with the last toroid before, and the first toroid after the SCL We use the PXI Platform with LabVIEW Real-time and FPGA Source MEBT Amplifier DTL Amplifier Attenuators Digitizer SNS Linac Beginning of CCL102 current waveform (long pulse) Abort HEBT BCM01 current waveform (short pulse) Electronics Buildings Layout of Differential Current Monitor

8. 8 Errant Beam at SNS Processing: FlexRIO FPGA Sampling at up to 100Mhz Pipeline delay 15 clocks cycles: 150ns Offset Delay Scale Offset Delay Scale Droop Down- stream Up- stream PreprocessDifferenceSumCompare Upstream’ previous Downstream’ previous Σ s i >? Y Y Y Y Alarm.. s 4 s 5 s 6 s 7 Σ Σ Σ Dif + - + - + - Σ s i >? Y.. s 4 s 5 s 6 s 7 Σ Processing must account for difference in cable lengths, calibration, baseline drifts, and droop.

9. 9 Errant Beam at SNS FlexRIO FPGA processing Preprocessing code Timing Decoding code Features: Send three waveforms -Any of the preprocessing stages -Differential signals -Sliding windows outputs Alarm status on each sample –Which threshold exceeded SNS Timing Library –Receive timing link and data link data [4]. We don’t need a separate (and custom) timing decoder!

10. 10 Errant Beam at SNS Real-Time Processing Real-time code @60Hz: –Archiving of errant beam waveforms on FTP server –Calculate statistics –Interface with control system using native LabVIEW EPICS [5] automatic creation of PVs and console screens All time-critical µs-level signal processing, including abort, is done in FPGA Console screen for DCM

11. 11 Errant Beam at SNS Experiment: Abort Test Temporary change to the Machine Protect System (A. Justice) DCM aborts in middle of the beam pulse:  ~8.5µs abort time = 2 to 3x improvement Through normal MPS: ~14 µs Reaction Time DCM pulls abort Beam gone Current and previous beam pulse Calculated best: 1 µs for DCM processing 2.5 µs signal cable delays 1 µs abort cable delay 1 µs chopper abort 1.5 µs beam propagation  7 µs. Shorter cables and better sensor locations can get us as low as 6 µs

12. 12 Errant Beam at SNS MPS delay test DCM downstream signal disconnected:  13.8 µs total for beam shutoff -3.2 µs for DCM abort (small pulses) -10.6 µs for MPS trip after receiving abort + time of flight (Alan Justice) ~3 µs in rack

13. 13 Errant Beam at SNS Results: Difference between locations Example of the errant beam we want to abort Analyzed to be due to a Warm Linac RF drop Alert on small window difference Alert on pulse-to-pulse downstream Alert on pulse-to-pulse upstream Alert to MPS Waveforms: 1.Upstream 2.Downstream 3.Difference after short sliding window Alert was given in <1.5 µs (could be faster with lower thresholds and still avoid false alerts) DCM waveforms

14. 14 Errant Beam at SNS Alarm Data Passed along with each sample BitAlarm 1Any Alarm 2Total Sum of Difference 3Large Sliding Window Ch0 –Ch1 4Small Sliding Window Ch0-Ch1 5Sliding Window Ch0-Ch0’ 6Sliding Window Ch1-Ch1’ 7MPS Latched 8MPS Auto Reset

15. 15 Errant Beam at SNS Results: Difference in time Example of slow drop in current from pulse-to- pulse Analyzed to be due to RF Fill problem Pre-mortem Analysis! Alert on pulse-to- pulse downstream Alert to MPS Waveforms for errant AND previous beam pulse AND next beam pulse: 1.Upstream 2.Downstream 3.Difference DBCM waveforms

16. 16 Errant Beam at SNS Results: Temporary loss then followed by abort. (Alarm: 56)

17. 17 Errant Beam at SNS Results: Temporary loss & bunch shift then followed by abort (alarm 56)

18. 18 Errant Beam at SNS Results: Loss on next pulse Lots of issues: different beam than previous and next pulse has losses Prev & now Now & next Next has beamloss Should we abort on no-loss difference to prevent future loss? Prevent RF learning?

19. 19 Errant Beam at SNS Alarm histogram Mostly alarm:48 (different pulse lengths) 25 cases of beam loss between CCL and HEBT in 3 days but including AP time Alarm Value Alarm LinesTypical cause 60all 4 alarmslarge loss and abort 56all but large difsmall loss and abort 48self Ch0&Ch1missing pulse before CCL 32self Ch1missing pulse before CCL 28all but self Ch1loss and abort 16self Ch0missing pulse before CCL

20. 20 Errant Beam at SNS Classifying Errant Beam Tree structure with 5 levels. Location Along Beam  Loss Upstream | Loss in SCL Portion of Macropulse  1 st | 2 nd | 3 rd | All Nature of Loss (Mini-pulse)  Partial Loss | Total Loss Beam Recovery  Recovery Present | No Recovery* Nature of Recovery  Large* | Small* Loss Downstream FrontPartialRecoveryLargeSmallNo recoveryFullMiddleEnd

21. 21 Errant Beam at SNS Ion Source Errant Beam Associated with ICS_Opr:IonSource:Edmp:FltCtr (Very representative of this fault) Associated with ICS_Opr:IonSource:Foc2:FltCtr Nominal beam

22. 22 Errant Beam at SNS Close up of loss point SCL Errant Beam Associated with SCL BLMs

23. 23 Errant Beam at SNS DTL Power Supply Errant Beam Associated with DTL_PS:MIOC1A:faultcount0

24. 24 Errant Beam at SNS Histogram (work in progress)

25. 25 Errant Beam at SNS Upstream current monitor is lost Current transformer needs to see magnetic fields to measure current  ceramic break in beam pipe Hole burnt into ceramic causing vacuum leak! (RF electrons?)  BCM replaced with spool piece. Replacement with internal shield is long term (aka years). No nearby current monitors that are suitable End of DCM? ceramic hole pipe

26. 26 Errant Beam at SNS Current measurements using BPMs Beam Position Monitor: Use sum from all four plates Use demo log-amp board with band-pass filter in front Add correction in FPGA with exponential function Use existing BCM as reference for calibration  We got our DCM back and extra locations! Sum signal RF Bandpass Logamp Log signal FPGA code to correct log signal Final signals. Calibration done using downstream BCM +

27. 27 Errant Beam at SNS Plans Add new MPS board to PXI crate with optical fiber output Dedicated link to MPS with mask-able input  8-9 µs total Simplified DCM (no fancy archiving) to provide high uptime –Add decoding of intended pulse length Linear RF envelope detectors (to avoid calibration procedure). Log-amps are very sensitive even the replacing of a cable requires recalibration Install new toroid and/or reroute BCM cables Second DCM for archiving –Keep the aborting DCM as simple as possible to increase robustness Upgrade MEBT DCM to same hardware The differential over time is also very powerful in showing anomalies –Possibly abort before losses or tell Low-Level RF to not learn on bad pulse Soft-IOC to combine DCM data with other data to determine cause of aborts

28. 28 Errant Beam at SNS References REFERENCES [1] S. Kim et al., “The Status of the Superconducting Linac and SRF Activities at the SNS,” 16 th International Conference on RF Superconductivity, Sep 23-27, 2013, Paris. [2] W. Blokland, “Errant Beam: Tools and Data,” SNS Errant Beam Committee, May 2012, Oak Ridge, TN. [3] C. S. Peters, “Errant Beam Update,” Accelerator Advisory Committee, May 7, 2013, Oak Ridge, TN. [4] R. Dickson, “LabVIEW FPGA SNS Timing User Guide,” SNS, May 7 2013, Oak Ridge, TN. [5] S. Zhukov, “Pure LabVIEW Implementation of EPICS Communication Protocol,” Big Physics and Science Summit at NIWeek, August 7-8, 2012. Austin, TX. [6] W. Blokland et al., "A new Differential And Errant Beam Current Monitor for the SNS Accelerator", IBIC2013. [7] A. Justice, “DBCM Measurements 9/15/2015 Results”, SNS Internal Memo, September 2015