1. Models for Software Reliability N. El Kadri SEG3202

2. 2 Definitions 1991 IEEE standard: the probability of failure-free software operation for a specified period of time in a specified environment The quality of the product improves over time, and we talk about reliability growth Software-reliability growth problem: estimating and predicting the reliability of a program as faults are identified and attempts made to fix them. Fault density - The number of faults per thousand lines of executable source code.

3. 3 Using Reliability Models

4. 4 Taxonomy of Software Reliability Models

5. 5 Time-dependent Reliability Growth Models

6. 6

7. 7 Discussion One problem with models is an overwhelming number of models has been proposed to address the issue of software reliability assessment. We must be aware that no single model can be recommended universally to users under any circumstances. The best models may vary from time to time and differ form application to application.

8. 8 Time-Independent Reliability Models: Fault Injection N Estimates the number of faults N in the system in the case where we know the outcomes of the already detected faults. A 1.Insert into a software module a certain number A of faults 2.Proceed with its testing f i 3.Count the number of failures due to injected faults (f) Count the number of failures due to inherent faults (i ) 4.Estimate the number of remaining faults =>

9. 9 Time-Independent Reliability Models: Fault Injection Example: A=25 1.Insert into a software module A=25 faults 2.Proceed with its testing (results: 32 failures) f=17 3.Count the number of failures due to injected faults (f=17) i=15 4.Count the number of failures due to inherent faults (i=15) 5.Estimate the number of remaining faults

10. 10 Fault Injection: Confidence Levels about the Number of Faults 1. If not all artificial faults have been found (f<A), then: If all artificial faults have been found (f=A), then:

11. 11 Fault Injection: Example (cont.) 1.Confidence levels about the number of faults Let us set E to 10. f

12. 12 Fault Injection: Example 2 1.Assume all artificial faults have been found (f=A) 2.Given E=10, certain confidence level λ and i<E, how to determine the number of faults to be injected?

13. 13 Time-Independent Reliability Models: Input Domain 1. Software reliability depends on how software operates on a certain input domain. 2.This point of view relates to selecting software test cases over the input domain according to how software is used Software usage information includes the environment information where software is used, as well as the information on the actual frequency of usage of different operations, functions, or features that the system offers. The usage information is quantified through operational profiles

14. 14 Input Domain: Equivalence Classes Software Module

15. 15 Operational Profiles Such a strategy is called Statistical Testing and it has at least two benefits 1.Testing concentrates on the parts of the system that are most likely to be used and hence should result in a system that the user finds more reliable. 2.Using the techniques which we have presented earlier, we have confidence that reliability predictions based on the test results will give us an accurate prediction of reliability as seen by the user.

16. 16 Operational Profiles Methodology to develop operational profile: 1. Determine customer profile or usage context profile. 2. Determine user profile. 3. Determine system modes and their profile. 4. Determine the functional (requirements) profile. 5. Determine the operational (implementation) profile.

17. 17 Operational profile A set of relative frequencies (or probabilities) of occurrence of disjoint software operations during its operational use A software-based system may have one or more operational profiles. Operational profiles are used to select test cases and direct development, testing and maintenance efforts towards the most frequently used or most risky components.

18. 18 Operational profile Construction of an operational profile is preceded by definition of a customer/user profile, a system mode profile, and a functional profile. Profiles are constructed by creating detailed hierarchical lists of customers, users, modes, functions and operations that the software needs to provide under each set of conditions. For each item estimate the probability of its occurrence and thus provide a quantitative description of the profile. If usage is available as a rate (e.g., transactions per hour) it needs to be converted into probability.

19. 19 User Profile - Example

20. 20 System Mode

21. 21 System Modes Example We can identify five system modes: 1.Normal Traffic load 2.High Traffic load 3.Start/Restart 4.Administration 5.Troubleshooting The first three are only relevant to the subscriber user type, the fourth to the operator user type and the fifth to the customer care system user type.

22. 22 Functional Profile

23. 23 Operational Profile

24. 24 Operational Profile

25. 25 Input Domain: Equivalence Classes Software Module

26. 26 Input Domain: Equivalence Classes & Operational Profiles 1.Each Path through the Operational Profile hierarchy defines an equivalent class: Path E1: subscriber, normal traffic, MO SMS, originating MO SMS Each path Ej has its associated probability Pj stating that the inputs will come from it under normal operation of the system Assuming the inputs are independent from each other, the probability of a path is a product of the corresponding operational probabilities, i.e.: for E1 the associate probability is P1 = 0.9*0.6*0.05*0.95 = 0.35

27. 27 Input Domain Model: Reliability Estimation 1.Assumption: c equivalent classes E i 2.With each class comes its operational profile 3.Let P i be the probability stating that the inputs will come from E i under normal operation of the system  n j is the number of test cases sampled from the j th input domain E i, where f j out of them resulted in software failures 5.The estimated reliability is computed as:

28. 28 System reliability Knowing the reliability of individual components R i, one can easily compute the reliability of some architectures as follows: (a) Series configuration:(b) Parallel configuration:

29. 29 Classes of Reliability Models and their main Features

30. 30 Reliability Measures

31. 31 Availability

32. 32 ANSI/IEEE 982.1-1988 Includes guidance for the following: Applying product and project measures throughout the software life cycle Optimizing the development of reliable software with respect to constraints Maximizing the reliability in its actual use environment Developing the means to manage reliability in the same manner that cost and schedule are managed

33. 33 More Information