WAO, Trieste, 24th - 28th September 2007 An Overview of the Reliability and the Fault Trends of the SRS Cheryl Hodgkinson Daresbury Laboratory Synchrotron Radiation Department

WAO, Trieste, 24th - 28th September 2007 Contents Introduction SRS Operational Statistics Analysis of the Operational Data –Efficiency –Fault Statistics –Ageing and Obsolete Equipment Maintaining Reliability –Capital Investment and Redundancy –Risk Analysis –Post Incident Investigation and Modifications –Preventative Maintenance and Scheduling Conclusions and Acknowledgments

WAO, Trieste, 24th - 28th September 2007 Introduction The Synchrotron Radiation Source –2GeV, 2 nd Generation Light Source –Constructed between 1975 and 1980 –Began scheduled user operations in 1981 –Some SRS systems, mainly injector components were salvaged from a previous accelerator –Operational user facility for 26 years and scheduled for closure December 2008

WAO, Trieste, 24th - 28th September – 1992 A number of upgrades HBL Undulator SC Wavelength Shifters Erratic efficiency profile due to new systems Dominated by water – vacuum incidents 01/02 (65%) SCW 9 Absorber 04/05 (85%) RF Cavity Window & Quadrupole Vessel Absorber 06/07 (78%) TVM 9 Efficiency in 06/07 was also significantly effected by Linac RF and HT systems and Gun Cathode issues. Periods between incidents in excess of 90% Stable period of operation Gradual increase in efficiency to figures in excess of 90% SRS Reliability Data

WAO, Trieste, 24th - 28th September 2007 SRS Fault Statistics Vacuum Incidents

WAO, Trieste, 24th - 28th September 2007 Vacuum Incidents Usually an isolated single failure Difficult to predict and prevent –failed RGA head –Quadrupole absorber – failed flowmeter Common Theme? –SCW Absorber & TVM 9 –Both very different failure mechanisms –Both historical – not needed in the current SRS configuration

WAO, Trieste, 24th - 28th September 2007 SRS Fault Statistics Beam Loss Cause UnknownRF Klystron PSURF SystemControls

WAO, Trieste, 24th - 28th September 2007 Ageing and Obsolete Equipment Early Equipment –Built in-house –Repairs in-situ (time) –Can be repaired Later Equipment –Plug-in –Replaced with a spare –Return to manufacturer for repair –Support issues

WAO, Trieste, 24th - 28th September 2007 Ageing and Obsolete Equipment Concern regarding condition of in vacuum water circuits –17 year old absorber removed from storage ring –Area’s milled away to examine any wall thinning RESULT No evidence of wall thinning or erosion

WAO, Trieste, 24th - 28th September 2007 Ageing and Obsolete Equipment External water circuits –Elbows –Near feedthroughs –Potential for vacuum incident during repair –Potential to cause more serious failure

WAO, Trieste, 24th - 28th September 2007 Ageing Machine – It’s not just equipment! It’s about people too! –Loss of design and build knowledge, as staff retire Less documentation for older equipment, drawings, maintenance schedules, etc. Can’t teach 20 years design, build and operational knowledge. –Succession planning important –You only know what’s been missed when the fault occurs Lapses in maintenance schedules Longer fault rectification

WAO, Trieste, 24th - 28th September 2007 Capital Investment and Redundancy Capital investment –Must be bid for against other priority items –Only essential systems and components which would significantly improve efficiency Spares –Hot Power Converters Storage Ring Klystron –Vacuum vessels Old RF PSU New Commercial Unit

WAO, Trieste, 24th - 28th September 2007 Risk Analysis Benefit difficult to assess –The biggest risk is the one you don’t know about At top level –Difficult to compare risk and focus resource –How do you compare extremely reliable long lead time items against components that fail frequently, but are a fast repair? –Does highlight major issues Linac Klystron

WAO, Trieste, 24th - 28th September 2007 Risk Analysis Most beneficial at individual technical group level Direct comparison can be made between equipment and fault hours contributed Led to successful modernisation in key area’s based on safety and operational risk

WAO, Trieste, 24th - 28th September 2007 Post Incident Investigation Learning from incidents is important Committee is always formed to recover and also determine actions to prevent reoccurrence. Actions from recent faults –Modification to TVM cooling pipes –Revised interlock testing schedule –Test rig to test thermionic valves on-site

WAO, Trieste, 24th - 28th September 2007 Preventative Maintenance and Scheduling

WAO, Trieste, 24th - 28th September 2007 Conclusions The SRS has provided reliable user operations for over 26 years Vacuum incidents are isolated incidents, which do not indicate any form of systematic failure which would limit the life of the SRS Only evidence of any systematic failure due to ageing is the failure of copper elbows on water circuits The high operating efficiency demanded by the light source community can be maintained on an ageing machine using a variety of techniques.

WAO, Trieste, 24th - 28th September 2007 Acknowledgments The author would like to thank –Previous SRS Operations Managers for compiling these statistics –Current and past Operations Teams for providing the data –Steve Griffiths of EE&PS for providing his Groups Risk Analysis Data And Finally Daresbury Staff past and present for their contributions to SRS Operations