1 Software Testing and Quality Assurance Lecture 34 – Software Quality Assurance

2 Lecture Objectives What is Software Reliability? Classify Failures Software Reliability Engineering

3 What is Software Reliability? Consider two simple systems consisting of two writers and a reader communicating via a shared memory on a bus. Suppose probability of failure of each channel of the bus is per operational hour.

4 What is Software Reliability?

5 For a single channel bus The probability of a bus failure, such as message becoming corrupt or the bus failing to transmit a message is ‘one failure every 11.4 years’. If we add the second channel then the failure probability decreases to One failure every 1,114,552 years, because both channels must fail at the same time for this to occur.

6 What is Software Reliability?

7 We have to be careful in not over interpreting these numbers. While improvement is significant, after all, It is only a probability. The definition of reliability depends heavily on determining, and in practice detecting, failures.

8 Classifying Failures Examples of failures include Functional failures – where the actual behavior of the system deviates from the specified functional behavior. Timing failures – where the program may deliver correct results but the program fails to meet its timing requirements. Safely failures – where an accident resulting in harm or injury is deemed to be a failure of the system.

9 Classifying Failures In practice, we need A way of observing failures; and Logging each failure as it arises. We also need A way of determining the time at which a failure occurs – called failure time; Or The rate at which failure occur – called failure rate. Assurance that the estimates are accurate

10 Classifying Failures Transient Failure Program gives incorrect result, but the program continues to execute. Hard Failure Program crashes (stack overrun….)

11 Classifying Failures Cascaded Failure Program crashes and takes down other programs Catastrophic Failure Program crashes and takes down the operating system or the entire system; A total system failure.

12 Software Reliability Engineering (SRE) PhaseSRE Activities Requirements Determine the functional profile of the system Determine and classify important failures Identify client reliability needs Conduct trade-off studies. Set reliability objectives.

13 Software Reliability Engineering (SRE) PhaseSRE Activities Design Design and evaluate systems to meet reliability goals. Allocate reliability targets to components

14 Software Reliability Engineering (SRE) PhaseSRE Activities Implementation Measure and monitor the reliability during implementation. Manage fault introduction and propagation.

15 Software Reliability Engineering (SRE) PhaseSRE Activities System and Field Testing Measure reliability growth (using reliability growth models). Track testing progress against reliability growth.

16 Software Reliability Engineering (SRE) PhaseSRE Activities Post-delivery & Maintenance Monitor field reliability against reliability objectives.

17 Software Reliability Engineering (SRE) – Some Observations Activities of SRE span the entire software development process But the activities themselves do not constitute a development process. Activities themselves need to be supported by Analytical tools – reliability models Evaluation methods – Markov Chains.

18 Software Reliability Engineering (SRE) – Some Observations Design and implementation reliable systems is supported by design principles and algorithms For example, from the discipline of “Fault tolerant systems”.

19 Key points Reliability depends heavily on determining, and in practice, detecting failures. Generally, four types of failures. Software Reliability Engineering Span the entire software development process