B. Todd, A. Apollonio, M. Kwiatkowski, R. Schmidt, S. Wagner, J. Walter a Risk-Based Approach 1v2 to Machine Protection Systems

CERN Risk Based Approach to Machine Protection 1. Machine Protection in Context safety – protection – plant 2. Protection System Lifecycle assessing and specifying 3. Conclusions future work, and outlook Machine Protection fits between System Safety and Plant Systems assessment of powering system outlined, with risks & functions analysis of high risk failure cases life-cycle concept can be adapted from system safety to machine protection assessment of current implementations & specification of future

CERN Safety – Protection – Plant 3 [11] Vacuum Pressure Vacuum Pump Speed Control Fulfill operational requirements Plant Systems: Vacuum Example: maintain correct pressure

CERN Safety – Protection – Plant 4 [11] Vacuum Pressure Vacuum Pump Speed Control Vacuum Pressure Vacuum Valve Actuator Ensure plant stays within limits Plant Protection: Fulfill operational requirements Plant Systems: Vacuum Example: maintain correct pressure bad pressure = close valves

CERN Safety – Protection – Plant 5 [11] Vacuum Pump Speed Control Vacuum Pressure Vacuum Valve Actuator Sensors, Actuators and Process may be combined No rules regarding combination Must meet functional requirement Ensure plant stays within limits Fulfill operational requirements Plant Systems:

CERN Safety – Protection – Plant 6 [11] Access doors Beam absorbers personnel safe but machine at risk People in perimeter – stop machine Personnel Safety System: cannot be merged with plants Must meet legal requirement E.G. “function must meet IEC SIL 3”

CERN Safety – Protection – Plant 7 [11] Prevent damage to machine Prevent undue stress to components Machine Protection System: No rules regarding implementation Must meet functional requirement

CERN Safety – Protection – Plant 8 [11] powering protection closely coupled to powering plant Prevent damage to machine Prevent undue stress to components Machine Protection System: No rules regarding implementation Must meet functional requirement

CERN Safety – Protection – Plant 9 [11] Personnel Safety System: Plant Systems: Machine Protection System: danger will exist – prevent – extract energy danger exists – protect – extract energy

CERN Protection System Lifecycle

CERN Inspired by IEC Protection System Lifecycle

CERN Protection System Lifecycle Assess Existing Design System systems involved in protection are unique certain technologies used have never been tried on this scale before high cost of failure development and analysis of machine protection as if it were a safety system worked example Dipole Magnet Protection – 9GJ Protection System Lifecycle

CERN CERN Protection System Lifecycle

CERN 154 in series

CERN QuenchDamage protectprevent 154 in series

CERN Resistive zone appears in a magnet I 2 R losses begin Zone heats up (heat propagates to neighbouring magnets) Damage to magnets Hazard Chain: from Quench to Damage… Failure  Hazard Chain  Failure Catalogue 154 in series

CERN Resistive zone appears in a magnet I 2 R losses begin Zone heats up (heat propagates to neighbouring magnets) Damage to magnets Hazard Chain: from Quench to Damage… What Protection Functions and Protection Systems are in place? Failure  Hazard Chain  Failure Catalogue prior experience deep thinking simulations prototyping 154 in series

CERN when quench occurs… Failure  Hazard Chain  Failure Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 prior experience deep thinking simulations prototyping

CERN Detection Power Abort when quench occurs… Failure  Hazard Chain  Failure Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 prior experience deep thinking simulations prototyping

CERN Quench Heater when quench occurs… Failure  Hazard Chain  Failure Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 prior experience deep thinking simulations prototyping

CERN Resistor Extraction Switch when quench occurs… Failure  Hazard Chain  Failure Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 prior experience deep thinking simulations prototyping Energy Extraction Loop

CERN Powering Loop when quench occurs… Failure  Hazard Chain  Failure Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 prior experience deep thinking simulations prototyping

CERN when quench occurs… Failure  Hazard Chain  Failure Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 prior experience deep thinking simulations prototyping Escape Diode

CERN when quench occurs… Failure  Hazard Chain  Failure Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 prior experience deep thinking simulations prototyping

CERN classify probability and consequence using risk matrix risk, if function didn’t exist, according to system experts… Colour boundaries, probabilities, consequences machine dependent e.g. Annika’s Talk Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1

CERN classify probability and consequence using risk matrix risk, if function didn’t exist, according to system experts… Colour boundaries, probabilities, consequences machine dependent e.g. Annika’s Talk Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1

CERN Risk Matrix  Risk Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1

CERN determine risk reduction level using matrix Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1

CERN = reliability requirements determine risk reduction level using matrix Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 RRL Matrix  RRL Catalogue

CERN = reliability requirements RRL Matrix  RRL Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 determine risk reduction level using matrix

CERN RRL Matrix  RRL Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1

CERN RRL Matrix  RRL Catalogue Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1

CERN expected  assess  actual? Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 Assess existing system implementation: coverage, quality … How do we qualify a system meets a level? How about programmable logic? See paper…

CERN Turn off Power Converter = purple = 3 Propagate Quench = orange = 2 Extract Energy = purple = 3 Link Related Circuits = green = 1 Assess existing system implementation: coverage, quality … expected  assess  actual? How do we qualify a system meets a level? How about programmable logic? See paper…

CERN Failure Case 1: September 2008

CERN Failure Case 1: September commissioning circuit to 5 TeV = 9kA

CERN Failure Case 1: September commissioning circuit to 5 TeV = 9kA

CERN Failure Case 1: September commissioning circuit to 5 TeV = 9kA

CERN Failure Case 1: September commissioning circuit to 5 TeV = 9kA Interconnect

CERN Magnet Protection 40 Magnet Interconnect

CERN Superconducting Cable Tin – Silver Foils Longditudinal View – filled with Solder Cross Section View Superconducting Cable Copper Stabiliser [1]

CERN [1]

CERN Magnet Protection 43 electrical arc punctures helium line [2]

CERN Failure Case 1: September Pressure Wave propagates inside insulation vacuum enclosure 2. Rapid Pressure Rise Self actuating relief valves could not handle pressure Design: 2Kg He/s Incident: ~20 kg He/s 3. Forces on the vacuum barriers Design: 1.5 bar Incident: ~8 bar Quadrupoles Displaced by ~50 cm Cryogenic line connections damaged Vacuum to atmospheric pressure [1]

CERN Incident location Dipole Bus bar [1]

CERN Failure Case 1: September 2008 Quadrupole-dipole interconnection Quadrupole support Main Damage Area: 700m 39 dipoles 14 quadrupoles [1]

CERN Hazard Chain had been identified in initial stages… Probability classified as negligible Risk Reduction Level was therefore minimum Installation did not conform …

CERN nQPS 48

CERN nQPS 49 Interconnect impedance is measured Energy Extracted if impedance unacceptable

CERN overall repair and consolidation 14 quadrupole magnets replaced 204 interconnections repaired 4km beam-tube cleaned longitudinal restraining system quadrupoles 900 ports for helium pressure release 6500 new detectors and 250km cables for new Interconnect Protection System collateral damage mitigation 39 dipole magnets replaced [1]

CERN : Interconnect Reworking [3, 4] interconnects to be re-worked…

CERN : Interconnect Reworking [1, 2] interconnects to be re-worked…

CERN : Interconnect Reworking [1, 2] interconnects to be re-worked…

CERN Failure Case 2: January 2013

CERN quench tests forced a quadrupole magnet quench, all four protection functions failed to activate Six months earlier a thunderstorm tripped several QPS detectors Piquet team needed to manually intervene to rearm Post-Analysis: mitigation of this need by new firmware, piquet did not intervene Firmware update was not applied to this particular circuit Post-Analysis: time and revalidation pressure Missing rearm does not prevent the circuit from being powered Circuit powered and unprotected for six months Event was repeated as failure of protection functions was not identified immediately Failure of this nature on dipole circuit represents most critical risk level for CERN.

CERN QPS protection functions do not meet required RRL 1.Qualification of QPS Functions 2. Addition of Independent Energy Extraction Loop

CERN In Conclusion…

CERN Today: done using a deep-thinking argumentative approach Information is there, not organised

CERN Today: done using a deep-thinking argumentative approach Information is there, not organised If we work outside to inside = protection assessment

CERN Protection Functions  System Specifications Today: done using a deep-thinking argumentative approach Information is there, not organised If we work outside to inside = protection assessment If we work left to right = protection as a safety system build protection cases Stakeholders may want proof that their investment is secure e.g. Annika’s Talk

CERN Fin! Thank You!

CERN References P. LeBrun - LHC Performance Workshop [1] 62 Welding Interconnections on Sector 3-4, CERN Photography Service, [2] J.-P. Tock - LHC Performance Workshop [3] CERN, EDMS Document # [4]