C100 Operational Performance Curt Hovater Discussing input from many

Outline C100 Cryomodule Performance Operational Experience Summary 11/9/2018

C100 Performance Limitations Faults Definition Outcome/Issues/Comments Weekly time lost Cryogenic Cavity Detuning He pressure and or cavity boiling causes cavity to detune and fault Trip and recovery time, availability TBD Microphonic Detuning A sudden increase in microphonics (thunder, Cat D7) causes cavity to detune and fault Anomalous Quench Real quench that causes the cavity to trip. Not well known. Is there a cavity damage issue  TBD Unknown Unknown trip that doesn’t show up on the once and future fault logger Trip and recovery time, availability. Needs fault logger!! Vacuum Faults Vacuum faults caused by field emission Cryomodule recovery   Gradient Performance  Outcome/Issues/Comments Energy Loss Field Emission/Radiation Gradient reduced to decrease the radiation loading Reduced gradient, vacuum faults, radiation damage to cables and nearby components, 23.1 MeV RF Bath Heating/cryo issues Gradient reduced because of boiling and or cryo issues Reduced gradient, He pressure and level not stable TBD MV Gradient reduced to keep cavity with in the klystron power when detuned. Most noticed on pre-tuner plate modification and specifically on cavities 4 and 5. Quench/Field Emitter/Boiling/Arc/Window Original SRF commissioning limitation. Inflated gradient performance. Cavities commissioned at max gradient in SEL have to be recalibrated in GDR. Most cavities were beyond 19 MV/m so small effect.

C100 Performance Limitations Cont. Recovery Definition Outcome/Issues/Comments Recovery Time* Cavity Turn On Speed to recover a single cavity Availability 27.6 seconds Cryomodule Turn On Speed to recover a cryomodule 55 sec (Both gradient dependent due to ramping)   *Best time …actual recovery times maybe slower …..

C100 Operational Performance with Beam 98 MeV/C100 NL SL this plot that shows the C100 performance since 2012, The purple line represents 98 MeV per C100.   The small dots are the energy gain for an 8h period with CW beam delivery.   The large points are the operational point for the three periods chosen.   The plot shows several things: We tried to push the C100s hard between 2014 and 2016, especially Fall2015 and Spring2016. The push to high gradient is followed by a retreat, unfortunately. The South linac shows a modest improvement, probably due to all the C100s being Helium processed.  North Linac only NL22 was HeProc.

C100 Cryomodule Microphonic Issues Tuner mechanism (*stepper and piezo) acts as an acoustic antenna for microphonic ground motion Waveguides and other mechanical couplings channel acoustic noise (thunderstorms, cooling tower fans etc.) onto the cavities Cavities detune beyond a controllable point (RF power) and the RF system faults

C100 Susceptibility to Construction, Truck Traffic, LCW Cooling Fans, Lawnmowers, Thunder ... C100 Cavity gradients The drops show the cavity faulting during the day due to construction and truck traffic really any low frequency acoustic noise. ESH&Q Building construction halted Reason: RF Power could not compensate for the rapid detuning C100 - 0 Cavity Gradients

Coupled Cavities C100-4 Cavities 4, 6, 7, 8 responding to an applied PZT step voltage change from 4 to 3 volts in cavity 5 Adjacent Cavity coupling is ~ 10% between 1-4 and 5-8 cavities Cavities 4 and 5 have a “quasi” mechanical support between them. Ringing is the 21 Hz mechanical Mode Cavity 5 PZT moved 460 Hz Cavity Gradients 10 MV/m Locked in GDR Mode Klystron had the overhead to keep cavities locked Stepper Motor kicks in to tune the cavities

Cryomodule Heaters Increased microphonics were observed when heaters were on. The cryostat riser became a choke point as additional heat was applied. Out of the eight installed, only the even heaters (4) were initially used. When an odd cavity would turn off, additional heat would be supplied to the even cavities to compensate ..then Presently we are testing an eight heater system on one of the C100s. He Liquid Level in a cryomodule as heat was applied.

Operational Experience - Field Emission H&V nested Air Core correctors over BPM Field emission heats beamline Vacuum pump faults Vacuum interlock drops cryomodule out of RF Quad Viewer & pump drop cross End of Cryomodule Cavity Gradients impacting Beamline Vacuum activity BEAMLINE VACUUM C100-10 Cav 6 C100-10 Cav 7 GMES MV/m Radiation Damage

NL C100 FE Vacuum Trips Feb/Mar 2016

C100 & He Processing It was kind of a mixed bag ….

C100 & He Processing Next Steps Proceed with caution in 1L23 Added thermal cycle upfront Review procedures

RF Firmware/Software Focus Firmware focus has been, since the first commissioned C100s, on mitigating C100 performance issues Faults/Detuning Identifying troubled cavities quickly and softening the blow to the rest of the cryomodule Microphonic Detuning Improved algorithm for absorbing periodic microphonic excursions Cavities switch to SEL when detuning angle is > 60o Recovery Single Cavity Whole Cryomodule

C100 Cavity Fault & Recovery Solution C100s will never be fully immune from microphonic detunings Algorithm Identify cavity and switch to Self Excited Loop (SEL) at gradient Pull FSD Once microphonic has passed (driven by, stopped digging what ever…), automatically switch back to locked condition Alert operator and resume beam operations. Benefits C100s automatically recover, operations does not have to assist the C100 RF system. Potential for automated beam recovery too…. .

C100 Recovery Time vs Time Courtesy Bob Legg

Piezo Tuner JLAB cavity operating at ~ 15 MV/m with the PZT off and then on. Substantial improvement for slow detuning ( helium pressure drift or slow microphonics) Control bandwidths above 2 Hz cause oscillations Operated 2L25 for two months with Piezos. Piezo study has been a slow process Engineer left who was leading the investigation Competing projects have slowed this down PZT Off PZT On Slow tuner turns On to re-center The cavity PZT bandwidth was limited to 2 Hz Suggest to implement piezo tuners on “non stiffened” tuner C100s ….with possibility of full implementation

Summary C100 average energy at the end of the Winter/Spring Run NL ~ 89 MV SL ~ 87 MV Above 90 MV a combination of issues make operating problematic He bath and pressure issues contribute to microphonic detunings Cryomodules susceptible to ground motion and acoustic pickup FE vacuum trips bring down whole cryomodule There is Hope LLRF and more recently the Gradient Team have focused on many C100 Issues. Recovery times have steadily improved from 2012 to now RF Issues (HPA, Algorithmic etc.) have been fixed and are improving SRF is taking a closer look at C100 microphonic Issues New Heater implementation See Bob Legg’s Gradient Team talk for more hope…. Questions Can we get back to the 108 MV, 465 mA “high water” mark of 2012? Do we need to put more resources onto the C100s to get to 108 MV? The recent influx of resources (Gradient team) has improved C100 operations.

Clyde Mounts, Rick Nelson who contributed to this presentation Special thanks to Trent Allison, Rama Bachimanchi, Ed Daly, Mike Drury, George Lahti, Bob Legg, Clyde Mounts, Rick Nelson and Tomasz Plawski who contributed to this presentation

Mechanical Tuner Modification Design allows for 25 Hz Peak Detuning Microphonic Detuning* C100-1 C100-4 RMS (Hz) 2.985 1.524 6s(Hz) 17.91 9.14 Actual peak detuning (21 Hz) was higher than expected in first cryomodules A detailed vibration study was initiating which led to the following design change. Cavity C100-1-5 Cavity C100-5-5 A minor change to the tuner pivot plate substantially improved the microphonics for the CEBAF C100 Cryomodules. While both designs meet the overall system requirements the improved design results in a larger RF power margin

Cavity Fratricide Cavity Fratricide occurs when one cavity faults (typically microphonic detuning from He instability or ground motion) and the Lorentz force detuning of the faulted cavity detunes the adjacent cavities resulting in the cavity faulting too. Adjacent cavity has a 10% coupling to its brother/sister! Adjacent cavity was operating at 5 MV/m so the klystron had the overhead to absorb the detuning

Cavity/Cryomodule Turn On A digital self-excited loop (SEL) tracks the cavity frequency and quickly restores the cavity to its operational gradient. A firmware application then tunes the cavity and switches to Generator Driven Resonator (GDR) mode, locking the cavity to the reference. The procedure has evolved to the point that it is a “single button” automated turn on for the high gradient cryomodules. Each cavity is turned on in the SEL mode at some nominal gradient (18 MV/m for example) and then tuned to the reference frequency. Once on resonance the automated algorithm locks the cavity to the reference and then adjusts the gradient for the accelerator needs. Transition from SEL to GDR