CEBAF Accelerator Status Arne Freyberger Operations Department Accelerator Division CLAS Fall 2017 Meeting

Availability Challenges Range of CEBAF Availability during 6GeV Operations

CEBAF System Availability Weekly availability reviews at the weekly scheduling meeting Early identification of issues Monthly availability report Presented and discussed on the first Friday of a new month at the daily 8am meeting Look for trends Increased statistics for better information Annual availability review is used to set priority for support groups annual work plan (AWP). Feb. 2017 shown A good month

DTM Tracks all downtime events Downtime: Period longer than 5minutes Requires manual entry by operator Data quality may not be the top priority when things are not going well Operability reviews and presents every Wednesday Useful interpretation of this data requires that system owners help Operability in QA the data. Dealing with system downtime during non-beam operations not trivial Dealing with stops/starts of program not trivial Downtimes that result in program and/or schedule change may not be correctly tracked

Availability Data: DTM Failure Rate: 𝜆= 1 𝑀𝑇𝐵𝐹 MTBF: Mean Time Between Failure, desire this to be infinite MTTR: Mean Time to Repair, desire this to be zero 𝐴𝑣𝑎𝑖𝑙𝑎𝑏𝑖𝑙𝑖𝑡𝑦= 𝑀𝑇𝐵𝐹+𝑀𝑇𝑇𝑅 𝑀𝑇𝐵𝐹 lim 𝑀𝑇𝑇𝑅→0 𝐴𝑣𝑎𝑖𝑙𝑎𝑏𝑖𝑙𝑖𝑡𝑦 = 100% Count: Number of failures in a given period System Repair Time: 𝐶𝑜𝑢𝑛𝑡 ∗𝑀𝑇𝑇𝑅

It’s not personal, it’s data

System Availability

System MTBF (Large # is good) End of life WTF!!!!!!

System MTTR (Small # is good)

Cryogenics Data highly skewed by significant events that resulted in programmatic change MTBF mostly due to CHL1->SC1 faults, classic end of life.

RF MTBF: about two events per day MTTR: less than 1h, good but that’s two derailments per day. Would better training of operators improve MTTR even more? Improved controls?

CEBAF Performance Plan Define the performance goals Availability/Reliability Beam Parameters (emittance, energy spread, bunch charge, beam energy) Peak Conditions (Hall multiplicity, operating weeks,…) Perform gap analysis to identify where present CEBAF performance does not achieve the performance goals Develop a plan to close the gaps

CEBAF Performance Goals

CEBAF 20 year view

Critical Spares Top Priority Spring 2014: ZA magnet coil and vacuum failure; 3 week interruption to replace damaged coil and repair the vacuum chamber. This failure consumed the existing spare coil; the next failure will take much longer for repair and recovery. Spring 2015: Cold compressor 4 failure in 2 K cold-box, SC1; No spare at JLab, consumed the SNS cold compressor spare. Program change required: half design energy after 5 week down. Fall 2015: YR coil on 3-pass extraction generated a spontaneous leak. No 3-pass program for FY16, repaired Summer 2016, consumed a YR coil spare. Required Hall-A DVCS experiment to re-arrange its run plan. Fall 2016: Arc7 box supply failure, no spare, program change required: to single hall operation until supply repaired. Fall 2016: 5th pass separator vacuum leak, program change required, could not support 5th pass beam to Hall-A simultaneously with 5.5 pass beam to Hall-D. Spring 2017: Cold compressor 5 failure in 2 K cold-box, SC1; on-going root cause investigation, might be repairable. Scheduled program terminated.

Energy Gap and Plan Energy Gap: Presently CEBAF cannot support design energy (FY17/FY18 operations at 4% below design) Annual gradient loss exceeds the gain from one C50/year One C50/year insufficient to maintain energy reach; gap is growing! Energy Plan: C75 Upgrade: replace the C20 cavities with high current 5-cell cavities Re-use as much of the cryomodule as possible Total cost in terms of MV/$ lower than C50 or C100 options Upgrade LLRF to digital controls (similar to C100) Target specifications: Gradient: 18.5 MV/m Q0 > 8e9 Upgrade warm region vacuum and clean (SRF quality processing) all warm vacuum regions two warm girders/year (continues to end of 12GeV/era) Upgrade analog LLRF with digital LLRF in all zones two zones per year (continues to end of 12GeV era)

Energy Gap and Plan (C75 Upgrades) Proposed path to design energy (1090 MeV/linac) Comparable to C50 program post hurricane Isabel If annual degradation is mitigated: might achieve 1090 MeV/linac by FY21 and with fewer C75 modules

Preliminary Performance Plan M&S Cost Estimate C75s C100-Refurbs Preliminary Near term goal is to close the gaps by FY23 Plan emphasizes improving CEBAF availability/reliability first: Energy Reach second Operating at optimal weeks third All dollar values in FY17 direct costs

July 27 2017: CHL1 Event Modules that warmed above 50 K require a thermal cycle to room temperature and a controlled cool-down to 4K All the modules in these plots had temperature excursions above 50 K (or were already at room temperature for maintenance. Significant He contamination and loss of inventory. As of today all modules, except one, are full of LHe and @ 4K. This remaining module will be cooled and filled during the transition to 2 cryo plants

12GeV CEBAF Beam Parameters Obsolete At the request from Physics Division the beam parameter table created in 2001 for the4-6 GeV program is being updated to reflect 12GeV CEBAF capabilities. Please work with your APEL (Michael Tiefenback) in developing a table that will define the beam parameters appropriately for Hall-B experiments.

4-Hall Constraints In order to deliver beam to all 4-halls: At least one original end-station (A,B or C) must be on 5th pass The last 4 days in Dec. 2017 as published violate this rule. Hall-B will be at 5th pass for all of Dec. 2017 (unless Hall-D is offline) Lower passes will be arrange in the Jan/Feb/Mar beam delivery End-station cryogenic load can exceed the ESR capacity End-station magnet/target cool-downs must be carefully scheduled. High power target operations in one or more halls must be carefully scheduled.

The Good News CHL1 has been cleaned and is back in service: Presently maintaining CEBAF @ 4K All 5 cold compressors in SC1 have been operated up to 20% of operating value. Full SC1 spin up and 2K pump-down attempts (the first since March 10th) will take place today. Once CEBAF is @ 2K; assessment of the SRF gradient capabilities will take place. Transition from CHL1->SC1->CEBAF to 2 cryo-plants maintaining 2 linacs @2K schedule for Nov. 12. RF assessment + Magnet soak continues through Thanksgiving Re-visiting Cryo configuration during down-times: CEBAF/Cryo will remain @2K over Winter break Pro: Fewer transitions on aging 2K cold-boxes Pro: Beam Operations can be extended to Dec. 21 (NPES schedule has Dec. 18th) Con: Increased cost (power bill) SRF OPS has been producing field emission free cavities for LCLS-II; 7 out of 8 cavities FE free. 4-Hall system (lasers, separators, operations) ready for integration and first 4-hall operations. The CEBAF Performance Plan has been developed and reviewed. Presently working for Lab leadership to find funding opportunities. Funding for a new 2K cold-box (replace SC1) has been secured in FY17. 3-4 year effort, $8M.

Backup

Downtime Count

System Repair Time

Optimizing 12GeV Operations Reliability/Availability Focus 4-Hall Operations Improved Linac/Cryogenic procedures: Reduce risk of cryoplant contamination due to He connections/disconnections Linac LHe Pressure optimization Address end-of-life issues: SRF C20 Warm poly RF windows SRF C20/C50 end-can O-rings Improve C100 warm girder tolerance to radiation environment Identify and correct errant magnets West Arc to East Arc extension cable, allows for West Arc power supply to power East Arc magnets. Provides CEBAF the ability to support a full ABC program in the event of an East Arc box power supply failure. Energy Reach/Gradient Maintenance Linac energy setting: margin -> reliable operations Identify source of SRF Particulate Cleaning/processing warm girder regions Improved tunnel vacuum procedures

CEBAF Weeks of Operation Delivered Optimal # Weeks Accelerator Shutdown

From 2012 S&T Presentation

FY17 Joule Metric FY17 first year of 12 GeV operations reporting DOE metrics Beam operations for FY17 complete, with the exception of a few days of restoration at the end of FY17 (Sept 2017). Measured Reliability 76.3% Exceeds our expectations of 65% Below nominal DOE goal of 80% for all facilities

Availability/Reliability Gap & Plan Reliability and Availability inadequate for robust effective operations. Critical system failures have forced program/schedule changes to almost every run period to date. Availability/Reliability Plan: Purchase CEBAF and Cryogenic critical spares Build and commission a new 2K cold-box Purchase System immediate needs Shore up the depleted spares Purchase 6.5 kW and 13 kW klystrons Address immediate obsolescence issues

Operations Gap & Plan Operations Gap (Support 4-Hall/optimal weeks): Operator/CrewChief Staffing inadequate for 4-hall operation Group is presently 4 heads below historical 6GeV levels Technical and MCC staffing is inadequate to support optimal weeks (37 weeks/year) ESR capacity insufficient to support robust 4-hall operations Operations Plan: Increase staffing Execute projects that will reduce time required for maintenance Spare warm girders Redundant CHL carbon beds Transition from 2 PSS certifications/year to 1 certification/year Build a new ESR for robust 4-hall operations