EE551 Real-Time Operating Systems Safety Critical Systems Analysis Course originally developed by Maj Ron Smith

Safety Critical Software Systems – ilities of Systems Software safety is one of the “ilities” of that is part of non-functional requirements specifies criteria that can be used to judge the operation of a system, rather than specific behaviors What separates safety critical software from other kinds of software is the potential that it has to contribute to hazardous system states. Hazards in a system are contributing factors in the risks of accidents that can lead to the following:

Safety Critical Software Systems – ilities of Systems Execution Qualities Usability and Operability Security Reliability Safety Fault Tolerance Evolution Qualities Maintainbility, Understandability and Modifiability Supportability (Integrated Logistics Support) Testability Portability Scalability and Extensibility Integrity – often used to encompass other ilities What separates safety critical software from other kinds of software is the potential that it has to contribute to hazardous system states. Hazards in a system are contributing factors in the risks of accidents that can lead to the following:

Safety Critical Software Systems – ilities of Systems Safety and reliability are often misinterpreted There is a school of thought that states that safety is a subset of reliability What separates safety critical software from other kinds of software is the potential that it has to contribute to hazardous system states. Hazards in a system are contributing factors in the risks of accidents that can lead to the following:

Major RW Smith Software Reliability (part1) - 5 reliability, R(t) - the probability that, when operating under stated environmental conditions, a system will perform its intended function adequately for a specified interval of time. a measure of the success with which a system conforms to some authoritative specification of its behavior most frequent hardware metric - MTBF failure rate is more universal in software probability that a program will operate correctly in a specified environment for a specified length of time. 21-Apr-17 Major RW Smith Software Reliability (part1) - 5

Safeware: System Safety and Computers Nancy G. Leveson Safety Critical Software Systems – Authoritative text Safeware: System Safety and Computers Nancy G. Leveson ISBN-10: 0201119722 | ISBN-13: 978-0201119725

Safety Critical Software Systems Potential of the software to lead to hazardous system states Hazards can lead to accidents and: Death Serious Injuries Damage to environment Significant loss of material Loss of strategic advantage What separates safety critical software from other kinds of software is the potential that it has to contribute to hazardous system states. Hazards in a system are contributing factors in the risks of accidents that can lead to the following:

Safety Critical Systems

Safety Critical Systems

Safety Critical Systems

Safety Critical Systems

Examples of failures: Medical Therac-25 (1985-87)(extreme case) Bloodbank software released over 1M “failed” plasma units on the market. Pacemakers reset to unsafe parameters due to external radiation sources (antitheft devices, microwaves,…) Infusion pumps delivering the wrong rate of medicine. The literature is full of examples of accidents involving safety critical systems. Here these examples list a few examples studied by the FDA.

Safety Critical Software Systems Safety Critical Software cannot be verified and validated using “traditional” methods to derive test cases Must use risk management and hazard analysis techniques Root Cause Analysis It is important to know that Safety Critical Software cannot be verified and validated using traditional V&V methods We must use risk management and hazard analysis techniques to prove the system safe.

Safety Critical Software Systems Hazard Analysis techniques Hazard list from similar devices Hazard and Operability (HAZOP) Analysis Fault Tree Analysis (FTA) Event Tree Analysis (ETA) Failure Modes and Effects Analysis (FMEA) Failure Modes, Effects and Criticality Analysis (FMECA) Here is a list of hazard analysis techniques that have been successfully used on safety critical software in the past. In the next few of slides, we will take a brief look at each of the methods.

Safety Critical Systems - Hazard Analysis – Hazard List Known hazards lists or reports from previous similar devices Lessons Learned DB (internal to companies) Recall notices (general public – industry wide) Food and Drug Administration Web Site (MAUDE) http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmaude/search.cfm Federal Aviation Agency http://www.faa.gov/data_research/accident_incident/ Transport Canada (CADORS) http://wwwapps.tc.gc.ca/Saf-Sec-Sur/2/CADORS-SCREAQ/m.aspx?lang=eng The simplest hazard analysis method consists of looking at known hazards list from previous similar devices. These lists can come from lessons learned, recall notices or obtained from the FDA If there are no similar systems, then a brainstorming session can be conducted. Generic lists such as the one found in Annex D of ISO 14 971 can be also be used.

Safety Critical Systems - Hazard Analysis – Hazard List Brainstorming session Generic lists (ISO 14971 Annex D) The simplest hazard analysis method consists of looking at known hazards list from previous similar devices. These lists can come from lessons learned, recall notices or obtained from the FDA If there are no similar systems, then a brainstorming session can be conducted. Generic lists such as the one found in Annex D of ISO 14 971 can be also be used.

Safety Critical Systems - Hazard Analysis – HAZOP Hazard and Operability Study Process oriented is a structured and systematic examination of a planned or existing process or operation to identify and evaluate problems that may represent risks to personnel, equipment or environment Originates from Chemical Industry The next method is HAZOP. HAZOP is ideal for process based systems. It has been widely applied for safety analysis of chemical plants. Hazop is used to identify hazards that may occur while operating a system outside of its original intent.

Safety Critical Systems - Hazard Analysis – HAZOP Analyze the behavior of a system based on operating deviations from original design or intent Decomposition of system into sub-processes or items (systems, subsystems, components) Parameters (flow, temperature, pressure,…) Systematic qualitative analysis with Guide words (less, more, inverse, too high, too low, before…) Hazop decomposes a system into items or sub-processes. Each process is described by a set of parameters. The expected behavior of the system is analyzed systematically with the use of defined guide words.

Safety Critical Systems - Hazard Analysis - FTA Fault Tree Analysis is the most used of all Hazard analysis methods for safety critical software. Each hazard is analyzed using a deductive or backward approach. You start with the hazard then you trace back to the possible events that could cause it. The resulting analysis is in the form of a Boolean tree. The interesting part of this technique is that the leaves of the tree form a test case generation oracle. The weakness of FTA is that there is no chronological ordering to the events displayed.

Safety Critical Systems - Hazard Analysis - ETA Control measures Event Tree Analysis use a forward or inductive approach to hazard analysis. The ETA tree is similar to a binary decision tree. The protection mechanisms are listed at the top of the tree. The tree starts with an event that can lead to a hazard. At each level, the tree will branch for successful and failure of the protection measure.S

Safety Critical Systems - Hazard Analysis – FME(C)A Item Failure Mode Causes Effects Criticality Prob Control measures Registration RMS error too large a. Bad configuration b. Markers too close c. Handling errors d. Tracking error e. Transformation error Cannot use IIGS Critical N/A Operator training Documentation Failure Modes, Effects and Criticality Analysis can take the form of a tree or table. Tables representations are more common due to the compact representation. Each component or subcomponent including software is analyzed. The question we are trying to ascertain is what is going to happen if this component fails. The down side of such an analysis is that it only looks at single points of failure.

Safety Critical Software Systems State Based Analysis methods Markov Chain Models Petri Nets Software Cost Reduction Methods David Parnas and Constance L. Heitmeyer Formal mathematical approach to specifications Apart from using hazard analysis methods, We can also use techniques that use state or modes and transitions to perform a safety analysis. I will not go into detail here due to time restrictions.