Availability Performance of LCLS X-Ray FEL at SLAC William Colocho for the LCLS team.

2 Outline Quick LCLS Overview Availability and downtime vs. time Downtime per system and planned upgrades Tuning and Availability Operational Availability Goals Questions ARW 2011/LCLS Availability Cape Town, April 2011

3 Operated by Stanford University for the US Department of Energy Study the molecular world in ways never before possible 3 of 6 LCLS scientific instruments fully commissioned By February ,162 unique scientists from 28 countries have submitted 427 proposals to use LCLS Typically two experiments are scheduled for a five day user run. LCLS FEL at SLAC ARW 2011/LCLS Availability Cape Town, April 2011

4 A Tool for the Global Science Community ARW 2011/LCLS Availability Cape Town, April 2011 “By 2013 six instruments will give researchers unprecedented tools for a broad range of research in material science, medicine, chemistry, energy science, physics, biology and environmental science” (LCLS fact sheet: )

5 LCLS Free Electron LASER ParameterHard x-raysSoft x-raysShort pulse soft Short pulse hard unit Final e- energy6.7 – – GeV Bunch Charge150,250 20,40,80 pC Pulse Duration <10 fs (fwhm) Repetition Rate1,10,60,120 Hz Photon Bandwidth ~ ??% ARW 2011/LCLS Availability Cape Town, April 2011 For full list see: Each experiment selects a set of parameters to optimize their science goal. Each configuration has different hardware and software requirements.

6 LCLS Major Components 197 Quadrupoles 158 X, 159 Y Steering Correctors 175 Beam Position Monitors Klystrons/Modulators –4 Injector Stations –31 “Linac 2” Stations –48 “Linac 3” Stations 33 Undulators 33x5 CAM Movers ~1.8 Km of Vacuum ~124 Vacuum Pumps ~ 45 CAMAC crates 2 Injector LASER systems 33 Beam Shut Off Ion Chambers ARW 2011/LCLS Availability Cape Town, April 2011

7 ARW 2011/LCLS Availability Cape Town, April 2011 LCLS Availability: 94.8% ProgramDeliveredUser Off Config. Change TuningDown Scheduled Off PhotonsHardware USER COMM ALL Percent of Scheduled Hours (%) Plots and tables with the latest LCLS Availability information are automatically updated every shift during user runs. COMM: Hutch Commissioning Programs. Run I: 92.8% Photon, 97.0% Hardware Run II: 92.5% Photon, 97.2% Hardware Runs I and II Photon Availability combined “Tuning” and “Configuration Change” as one category.

8 LCLS Availability Terms Photon Availability: (Delivered + User Off + Configuration Change) / Scheduled On Hardware Availability: (Delivered + User Off + Configuration Change + Tuning)/ Scheduled On Scheduled On: Delivered + User Off + Configuration Change + Tuning + Down ARW 2011/LCLS Availability Cape Town, April 2011

9 Photon or Operational Availability Short term performance as important as long term. Users scheduled for machine time for five day intervals. ARW 2011/LCLS Availability Cape Town, April 2011

10 Trends Tuning, Downtime and Configuration Change time improved during the user run. Tuning and Configuration Change time reduction likely due to increased operational efficiencies. Downtime reduction not fully explained. Further analysis (Weibull analysis) needed. Notice correlation between tuning time and down time. (Down time leads to tuning time). ARW 2011/LCLS Availability Cape Town, April 2011

11 ARW 2011/LCLS Availability Cape Town, April 2011 Downtime Statistics and future upgrades Beam Shot Off Ion ChamberLinac fiducial distributionLinac Software Controls ConversionCoherent backup Laser Main Drive Line and Sub-Drive Line preventive maintenance RF head-tails phase monitor upgrade (Future) Linac Modulator Upgrade

12 MTBM and MDT Mean Time Between Maintenance Includes Corrective Maintenance and it does not include Preventive Maintenance. We accept that there will be preventive maintenance scheduled. This MTBM number should better reflect “Maintainability” when it most matters: During beam delivery to users. Only Selected Systems shown. See conference paper for full list. Data is 6 month snapshot (69.7 operating days). ARW 2011/LCLS Availability Cape Town, April 2011 System MTBM (Days) MDT (Hours) 1.3 Power supplies Linac Modulators Subbooster supplies S-band Klystrons Hi-power Subboosters sol. Stat subboosters, Low- Level RF Electrical (on site) Cooling water (mechanical) LCLS gun laser system Micros & Camac crates Timing Vacuum I&C MPS MCC:: (VMS System) PPS Controls Software Procedural Error Physicists Software PCDS Hardware

13 Tuning Reduction Task Force Members from Physics, Operations and Software groups. Working on reducing tuning time during Machine Development and Repair Days recovery. Expect this effort to also reduce tuning time during User runs. Tuning reduced operational availability by ~2% ARW 2011/LCLS Availability Cape Town, April 2011

14 Tuning Task Force Identify how tuning time is spent; Mine data from electronic logbook. Talk to Operators. Measure execution time of software procedures. Understand Machine configuration reproducibility. Ongoing development of software to simplify and automate tuning procedures. ARW 2011/LCLS Availability Cape Town, April 2011

15 Tuning: Need Metrics Pie! Injector Emittance, LI21 Emittance X, LI28 Emittance X, LTU Emittance X, Injector Matching, LI28 Matching, LTU Matching, Injector Phase Scans, Charge Change, Energy Change, Mirror reconfiguration, DL2 Dispersion with Quads, DL2 Dispersion with CORs, Undulator Re-pointing, Cathode Alignment, BSY Steering, LTU Steering, Laser Heater Alignment, Laser Heater Timing, C-Iris steering, MPS Trips at some settings, MPS Trips with Slotted Foil, Taper reconfiguration, Gas Detector Calibration, E-loss scan, Klystron Phasing, RF Amplitude Calibration, Zeroing e+ correctors, BBA, LI21 Emittance Y, LI28 Emittance Y, LTU Emittance Y, XCAV Set-points for FEL, L3 Steering… ARW 2011/LCLS Availability Cape Town, April 2011

16 LCLS Availability Goal is 95% That is 95% Operational Availability to Users. Before run III, goals did not take Tuning into account. A tuning budget (~2%) will be included in the availability plan. This sets the hardware availability goal at 97%. ARW 2011/LCLS Availability Cape Town, April 2011

17 Realizable Goals ARW 2011/LCLS Availability Cape Town, April 2011 Sub-System RUN III Previous GoalNew Goal Power Supplies (See Session 9, Wednesday) Magnets (See Session 14, Thursday) RF Vacuum Utilities Guns & lasers Controls Non-Radiation Safety Alignment Other Unassigned Photon Controls Total Goals based on past performance Remove worst two weeks or events from historical data (Life of the project ~2.5 years). System owners in charge and accountable for given system’s availability performance.

18 Conclusion LCLS science program is ongoing. Two year hardware availability ~ 97% Photon availability of 95% realizable. Multiple upgrade projects underway. Further analysis needed to better understand long term time dependence of individual subsystem and component failure rate. ARW 2011/LCLS Availability Cape Town, April 2011

19 Thank You! Questions. ARW 2011/LCLS Availability Cape Town, April 2011 If we use failure rates as a metric for determining which systems (and therefore groups) will receive attention and/or resources; are we setting up a system where failure is rewarded? In a climate of decreasing resources and budget constrains: Is there a model that correlates Availability and Reliability with Budget Allowance?

20 ARW 2011/LCLS Availability Cape Town, April 2011