1. Computer in Safety-Critical Systems
Chapter 6
Computer in Safety-Critical Systems
Introduction
“Safety-critical” and other definitions
How and why failures occur?
Risk analysis
Evaluating software
A case study of safety-critical failures
Model verification and validation
When human welfare is at stake, the price for haphazard practices is severe, and computing professionals must exercise extreme care to ensure that a system is safe.
Two requirements must be satisfied:
1) Have some idea of the techniques needed to develop computer systems that are as safe as is practically possible.
2) To be able to arrive at a reasonably objective assessment of exactly what that level of safety is.

2. Introduction How will an accidents happen?
Most accident are caused by a combination of:
organizational
managerial
technical, and
sociological or
political factors.
Preventing an accident requires paying attention to all the root causes.

3. “Safety-critical” and other definitions
Sometimes seen as systems with a component of real-time control that can have a direct life-threatening impact.
“Managing Murphy’s Law”
Examples:
Aircraft industry
Medical treatment system
Nuclear poser plants
Missile systems
What do we need?
A through risk assessment of a system
Risk, hazard, and reliability.

4. How and Why failures occur?
Difficulty in assessing and predicting failures
Three independent disk drives resulted in the failure of the Toronto Stock Exchange.
Is a computer model an abstraction of the real world ?
What is the danger of such an engineering view?
Risk Analysis
The responsibility of managers is to take the results of risk analysis seriously and act on them by ensuring that the system is managed conscientiously.
By Selecting:
Proper people
Training them,
Not overworking them

5. Evaluating Software Computers now have safety-critical functions in:
both military and civilian aircraft
nuclear plants
medical devices
It is incumbent upon those responsible for programming, purchasing, installing, and licensing these systems to determine whether or not the software is ready to be used.
What standard must a software product satisfy if it is to be used in safety-critical applications?
What document should be required?
How much testing is required?
How should the software be structured?

6. A case study of safety-critical failures
The Therac-25 accident, failure in a radiation-therapy system.
Errors in special timing-dependent sequences of events in the user interface.
Software error?
An appropriate software/hardware interlock could have prevented the events.
What was the role of Medical Doctors in this accident?
Should we use software in a safety-critical system?
How the standard was used in the system?
Was the redundancy check of the safety sufficient?
Model verification and validation
The numerical model of a real system must go through an extensive verification and validation process.
Summary
Risk assessment is very difficult to do
Software models of real world systems can never fully present all cases.