1. CSE 466 – Fall 2000 - Introduction - 1 Safety  Examples  Terms and Concepts  Safety Architectures  Safe Design Process  Software Specific Stuff  Sources  Hard Time by Bruce Powell Douglass, which references Safeware by Nancy Leveson

2. CSE 466 – Fall 2000 - Introduction - 2 What is a Safe System? Brake Pedal Sensor ProcessorBus Brake w/ local controller Engine w/ local controller Is it safe? What does “safe” mean? How can we make it safe?

3. CSE 466 – Fall 2000 - Introduction - 3  Reliability of component i can be expressed as the probability that component i is still functioning at some time t.  Is system reliability P s (t) =  P i (t) ?  Assuming that all components have the same component reliability, Is a system w/ fewer components always more reliable ?  Does component failure  system failure ? burn in period Terms and Concepts time Pi(t) = Probability of being operational at time t Low failure rate means nearly constant probability 1/(failure rate) = MTBF

4. CSE 466 – Fall 2000 - Introduction - 4 A Safe System  A system is safe if it’s deployment involves assuming an acceptable amount of risk…acceptable to whom?  Risk factors  Probability of something bad happing  Consequences of something bad happening (Severity)  Example  Airplane Travel – high severity, low probability  Electric shock from battery powered devices – hi probability, low severity severity probability danger zone (we don’t all have the same risk tolerance!) safe zone Nuclear power plant airplane mp3 player Desktop PC?

5. CSE 466 – Fall 2000 - Introduction - 5 More Precise Terminology  Accident or Mishap: (unintended) Damage to property or harm to persons..  Release of Energy  Release of Toxins  Interference with life support functions  Damage to a credit rating (Does this fit the definition?)  Hazard: A state of the the system that will inevitably lead to an accident or mishap.  Hydrogen leak in fuel cell system  Backup battery dead on ventilator  Supplying misleading information to safety personnel or control systems. This is the desktop PC nightmare scenario. Bad information  Alarm bell broken

6. CSE 466 – Fall 2000 - Introduction - 6 Faults  A fault is an “unsatisfactory system condition or state”. A fault is not necessarily a hazard. In fact, assessments of safety are based on the notion of fault tolerance.  Main H 2 Valve stuck, back up H 2 valve working  Systemic faults  Design Errors (includes process errors such as failure to test or failure to apply a safety design process)  Faults due to software bugs are systemic  Security breech  Random Faults  Random events that can cause permanent or temporary damage to the system. Includes EMI and radiation, component failure, power supply problems, wear and tear.

7. CSE 466 – Fall 2000 - Introduction - 7 Component v. System  Reliability is a component issue  Safety and Availability are system issues  A system can be safe even if it is unreliable!  If a system has lots of redundancy the likelihood of a component failure (a fault) increases, but so may increase the safety and availability of that system.  Safety and Availability are different and sometimes at odds. Safety may require the shutdown of a system that may still be able to perform its function.  A backup system that can fully operate a nuclear power plant might always shut it down in the event of failure of the primary system.  The plant could remain available, but it is unsafe to continue operation

8. CSE 466 – Fall 2000 - Introduction - 8 Single Fault Tolerance (for safety)  The existence of any single fault does not result in a hazard  Single fault tolerant systems are generally considered to be safe, but more stringent requirements may apply to high risk cases…airplanes, power plants, etc. Backup H2 Valve Control Main H2 Valve Control watchdog protocol If the handshake fails, then either one or both can shut off the gas supply. Is this a single fault tolerant system? Assume perfectly reliable valves

9. CSE 466 – Fall 2000 - Introduction - 9 Is This? Backup H2 Valve Control Main H2 Valve Control watchdog handshake common mode failures

10. CSE 466 – Fall 2000 - Introduction - 10 Now Safe? Backup H2 Valve Control Main H2 Valve Control watchdog handshake Separate Clock Source Power Fail-Safe (non-latching) Valves Does it ever end?

11. CSE 466 – Fall 2000 - Introduction - 11 Time is a Factor  The TUV Fault Assessment Flow Chart  T1: Fault tolerance time of the first failure  T2: Time after which a second fault is likely  Captures time, and the notion of “latent faults”  T1 – tolerance time for first fault  T2 – Time after which a second fault is likely  Based on MTBF data  Safety requires that  T test <T 1 <T 2 System Unsafe yes no System Safe Fault Detected After T 2 ? yes no yes hazard? Hazard after T 1 ? First Fault 2 nd Fault

12. CSE 466 – Fall 2000 - Introduction - 12 Latent Faults  Any fault this is not detectable by the system during operation has a probability of 1 – doesn’t count in single fault tolerance assessment  Detection might not mean diagnosis. If system can detect secondary affect of device failure before a hazard arises, then this could be considered safe Backup H2 Valve Control Main H2 Valve Control watchdog handshake stuck valves could be latent if the controllers cannot check their state. May as well assume that they are stuck!

13. CSE 466 – Fall 2000 - Introduction - 13 Design for Safety 1.Hazard Identification and Fault Tree Analysis, FMEA 2.Risk Assessment 3.Define Safety Measures 4.Create Safe Requirements 5.Implement Safety 6.Assure Safety Process 7.Test,Test,Test,Test,Test

14. CSE 466 – Fall 2000 - Introduction - 14 1. Hazard Identification – Ventilator Example MishapSeverityTolerance Time Fault Example LikelihoodDetection Time MechanismExposure Time Hypo- ventilation Severe5 min.Vent Fails – No pressure in reservoir. Rare30secIndep. pressure sensor w/ alarm 40sec Esophageal intubation Medium30secC02 sensor alarm 40sec User mis- attaches breathing hoses neverN/ADifferent mechanic al fittings for intake and exhaust N/A Over- pressuriza tion Severe0.05secRelease valve failure Rare0.01secSecondary valve opens 0.01sec Human in Loop