1. Software Testing and Quality Assurance Software Quality Assurance 1

2. Reading Assignment http://www.rvs.uni- bielefeld.de/publications/Incidents/DOCS/ ComAndRep/Warsaw/warsaw- report.html http://www.rvs.uni- bielefeld.de/publications/Incidents/DOCS/ ComAndRep/Warsaw/warsaw- report.html 2

3. 3 Objectives Software Safety ◦ Software safety is not just software Reliability Safety Engineering Approach

4. 4 Software Safety Safety in systems involving software is becoming important. For example, ◦ Computer Aided Dispatch Systems (CAD); ◦ Electronic Flight Control Systems (EFCS). ◦ Train Protection Systems; ◦ Chemical Plant control systems.

5. 5 Software Safety We wish to avoid in engineering and operating our platforms is ◦ Accidents. The system that we build must avoid the hazards that lead to accidents.

6. 6 Software Safety Accident – an event of sequence of events leading to harm; that is, death, injury, environmental damage or financial loss. Hazard – a physical situation or state of the platform that can lead to an accident.

7. 7 Software Safety To understand the safety of a system ◦ Understand how they can fail. Investigate accidents and accident sequences ◦ To understand the sequence of events leading to the accident and to try and determine which subsystem failed. Accidents are usually caused by combination of failures and circumstances.

8. 8 Software Safety Hazards are really accidents “waiting to happen”. ◦ They are the pre-conditions for an accident. In hazard identification ◦ We are concerned with thinking about the safety of the personnel or the platform. ◦ Start from known accidents or consider possible accidents and work back to hazards. ◦ Brainstorming exercise.

9. 9 Software safety is not just software Reliability Failure is key to understanding software reliability. ◦ Failure is deviation from the specified behavior of the system. For safety, ◦ it is always a deviation from the intended behavior; and where the deviation can lead to harm or damage.

10. 10 Software safety is not just software Reliability In normal usage of word failure ◦ A system may be unreliable but still safe; ◦ It may be completely reliable but totally unsafe. What failure of the system lead to an unsafe system?

11. 11 Safety Engineering Approach Hazard analysis technique to determine the safety aspects of the system ◦ Early in the development process, then ◦ Monitoring safety throughout the product development process; and ◦ Ensuring that there is enough evidence to build a safety case at the end of the product development process.

12. 12 Safety Engineering Approach Requirements Analysis and Product Definition ◦ Exploratory analyzes in the form a preliminary hazard analysis (PHA). ◦ The hazards, potential situations that can cause harm or environmental damage ◦ The potential cause list. Good understanding of the safely aspects of the software prior to going into the design and implementation phases.

13. 13 Safety Engineering Approach Design and Implementation Phase ◦ A deductive Approach – starts with potential situations of harm and works back to design or implementation elements. ◦ An inductive Approach – Starts from components or subsystems failures and  Works back through sub-systems to see if hazard result is used to verify the safety elements in the design.

14. 14 Safety Engineering Approach Defending in Depth - System that must defend against situations of harm ◦ Must be ultra-reliable or ◦ There must be sufficient redundancy Thus, safety related subsystem failure does not lead to safety related system failure.

15. 15 Language of Hazard Analysis Hazards needs to be identified and assessed as early in the development life cycle as possible. First step – hazard identification ◦ Process of identifying those situations which could lead to an accident under credible situations. ◦ Done as part of brainstorming methods.

16. 16 Language of Hazard Analysis Hazards in turn are casued by failures or failure modes. Failures – are unintended or states of the system that can lead to a hazard. Failure mode – identify specific classes of failures

17. 17 Language of Hazard Analysis Error – unintended states of the system that can lead to failure. Flaw – design/program defects that give rise to errors when certain conditions are activated. Fault – events that result when a flaw is activated.

18. 18 A Case Study – The Okecie Accident Occurred because of an accident sequence. ◦ Plane landed 750m down the runway and traveling at 170 knots instead of the more usual 150 knots in torrential rain and heavy winds. ◦ Aircraft failed to brake in time and hit an earthen wall at the end of the runway and burst into flames.

19. 19 A Case Study – The Okecie Accident

20. 20 A Case Study – The Okecie Accident In normal conditions – A320 does not need a 2.8 KM runway to stop; ◦ Even in torrential rain it can land safely and stop in under 1500m. Consequences of the accident were assessed to be major, but not catastrophic.

21. 21 A Case Study – The Okecie Accident Weather factors are believed to have contributed to the accident, ◦ Strong winds veered from a cross-wind to a tail-wind on final approach. ◦ It was raining heavily. Air Traffic Control (ATC) did not inform the crew of the change in wind direction.

22. 22 A Case Study – The Okecie Accident

23. 23 A Case Study – The Okecie Accident AG is true if (WoW > 12 tonnes) and wheels spinning > 72 Km/hr) or (wheels spinning > 72 km/hr and radio alt < 10 ft) AG  ‘at ground’ WoW  ‘weight on wheels’ & Function of both wheels Standing water on the runway caused the aircraft to Aquaplane while braking and wheels were not spinning at The required rate to activiate the braking logic.

24. 24 A Case Study – The Okecie Accident AG = False (weight on both wheels) WS = False (wheels spinning at > 72 KM/hr) Radio Alt = True (radio altimeter) Logic was implemented correctly but the system failed. This is quite probability a failure in requirements.

25. 25 A Case Study – The Okecie Accident Isolate the error, flaw and faults ◦ Error – was the assignment AG = False ◦ Flaw – we required AG = WoW > 12 tones for both wheels ◦ Failure – arose because the flaw was triggered in condition of wind and water on the runway.

26. 26 Requirements in Safety Lifecycles

27. 27 Requirements in Safety Lifecycles Initial Hazard List – the list of known hazards in the domain or a list of hazards obtained from similar systems or prior versions etc. Preliminary Hazard Analysis (PHA) ◦ Determining potential hazards for the new system as well as ◦ The causes of failure leading to these hazards

28. 28 Requirements in Safety Lifecycles Consequence Analysis – we work from design elements to hazards. ◦ Is predictive in nature and is carried out before the design is completed. ◦ Used to provide information into the design processes and is aimed to choosing between design alternatives. ◦ Event trees, failure modes and effects analysis (FMEA) etc.

29. 29 Requirements in Safety Lifecycles Causal Analysis – typcially performed top- down. ◦ Working from hazards to design or model element. ◦ Fault tree are the classic technique.

30. 30 Key points Accidents and Hazards ◦ Accidents are usually caused by combination of failures and circumstances. ◦ Hazards are really accidents “waiting to happen”. ◦ Hazards needs to be identified and assessed as early in the development life cycle as possible. Safety Engineering Approach ◦ The safety lifecycle consists of a number of activities which are aimed at determining and verifying the safely functions of the system.