TESTING FOR THE RELIABILITY OF A SOFTWARE SARAT CHANDRA YADAVALLI CSC 532 TERM PAPER

Software Reliability Engineering - Introduction Reliability for software is the probability of failure free operation of a computer programme in a specified environment for some specified interval of natural units or time. Reliability for software is the probability of failure free operation of a computer programme in a specified environment for some specified interval of natural units or time. Software reliability engineering is the standard that empowers the testers and developers to simultaneously - Software reliability engineering is the standard that empowers the testers and developers to simultaneously -  Ensure that product reliability meets user needs  Speed the product to market faster  Reduce product cost

Software Reliability Engineering Process Diagram 1.List Associated Systems 2. Develop Operational Profiles 3.Engineer “Just Right” Reliability 4. Prepare for Test 5.Execute Test 6.Guide Test Requirements and Architecture Design and Implementation Test

Developing Operation Profiles Developing operational profiles will give the information about how users will employ the product being built Developing operational profiles will give the information about how users will employ the product being built With the information given, both development and test can be made more realistic With the information given, both development and test can be made more realistic

Terminology Operation is a major system logical task, of short duration which returns control to the system when complete Operation is a major system logical task, of short duration which returns control to the system when complete Operational Profile (OP) is simply the set of operations and their probabilities of occurrence. Operational Profile (OP) is simply the set of operations and their probabilities of occurrence. OP can be represented in 2 ways: OP can be represented in 2 ways:  Tabular representation  Graphical representation

 Tabular Representation

 Graphical Representation

Methodology Identify the initiators of operations Identify the initiators of operations Choose between tabular or graphical representation Choose between tabular or graphical representation Create an operations list for each initiator and consolidate the results Create an operations list for each initiator and consolidate the results Determine the occurrence rates of the individual operations Determine the occurrence rates of the individual operations Determine the occurrence probabilities Determine the occurrence probabilities

Preparing for Test The operational profile information is applied to planning for efficient test. The operational profile information is applied to planning for efficient test. Testing is the process of executing a program with the intention of finding design errors in a given environment. Testing is the process of executing a program with the intention of finding design errors in a given environment. Testing can only prove the incorrectness of the software but not its correctness. Testing can only prove the incorrectness of the software but not its correctness.

Concepts Run is a specified instance of an operation. Run is a specified instance of an operation. A run is characterized by the operation and its input state A run is characterized by the operation and its input state Input variable can be either direct or indirect Input variable can be either direct or indirect A test case is the partial specification of a run through the naming of its direct input variables and their values. A test case is the partial specification of a run through the naming of its direct input variables and their values. A run involves the execution of a test case. A run involves the execution of a test case. Test procedure is a controller that sets up environmental conditions and that invokes randomly selected test cases at random times. Test procedure is a controller that sets up environmental conditions and that invokes randomly selected test cases at random times.

Direct and Indirect Input Variables for Process Fax Call Operation of Fone Follower

Procedure Preparing test cases – It can be done by simply recording in the field all the input variables needed to initiate the runs that make up the field execution of the software Preparing test cases – It can be done by simply recording in the field all the input variables needed to initiate the runs that make up the field execution of the software Preparing test procedures Preparing test procedures

Preparing test cases Estimating the number of new test cases needed for the current release Estimating the number of new test cases needed for the current release Allocating the number of new test cases among the systems to be tested Allocating the number of new test cases among the systems to be tested Allocating the number of new test cases for each system among its new operations Allocating the number of new test cases for each system among its new operations Specify the new test cases Specify the new test cases Adding the new test cases to the test cases from previous releases. Adding the new test cases to the test cases from previous releases.

Estimating the number of new test cases needed for the current release Two factors must be taken into account: Two factors must be taken into account:  Time  Cost Compute the number of test cases you have time to prepare Compute the number of test cases you have time to prepare Compute the number of test cases you can afford to prepare Compute the number of test cases you can afford to prepare Take the minimum of these as the number of test cases you will plan to prepare Take the minimum of these as the number of test cases you will plan to prepare

Allocating the number of new test cases for each system among its new operations Convert the graphical representation into tabular Convert the graphical representation into tabular Identify the rarely occurring critical new operations and determine how many test cases to preassign to each Identify the rarely occurring critical new operations and determine how many test cases to preassign to each Determine the allocation probabilities for the other new operations Determine the allocation probabilities for the other new operations Preassign one test case to each infrequent other new operations Preassign one test case to each infrequent other new operations Assign the remaining test cases to the remaining other new operations in accordance with the allocation probabilities Assign the remaining test cases to the remaining other new operations in accordance with the allocation probabilities

Allocating the number of new test cases among the systems to be tested Bulk of the test cases should be allocated to the product itself Bulk of the test cases should be allocated to the product itself Majority of test cases are allocated to the number of operations of the variation that occur and do not occur for the product Majority of test cases are allocated to the number of operations of the variation that occur and do not occur for the product Give particular weight to the differing operations that have high occurrence probabilities Give particular weight to the differing operations that have high occurrence probabilities

Allocation of Test cases to Operations of Fone Follower

Test cases Specification Select test cases within operations with equal probability Select test cases within operations with equal probability Prepare test scripts for the selected test cases Prepare test scripts for the selected test cases When invoked, the test cases must convey the operation they belong to and their direct input variables to the system under test When invoked, the test cases must convey the operation they belong to and their direct input variables to the system under test

Conclusion “A laudable standard for software reliability is Five 9’s - software which works % of the time” We have discussed - The procedure to identify the operational profiles used in defining the test cases. The procedure to identify the operational profiles used in defining the test cases. Procedure to prepare for test Procedure to prepare for test

References Text Book : Software Reliability Engineering – John D. Musa Text Book : Software Reliability Engineering – John D. Musa Lyu, M. (Editor) Handbook of Software Reliability Engineering, ISBN , McGraw-Hill, New York. Lyu, M. (Editor) Handbook of Software Reliability Engineering, ISBN , McGraw-Hill, New York. Tierney, J SRE at Microsoft. Keynote speech at 8th International Symposium On Software Reliability Engineering, November 1977, Albuquerque, NM. Tierney, J SRE at Microsoft. Keynote speech at 8th International Symposium On Software Reliability Engineering, November 1977, Albuquerque, NM. Challenges in Software Reliability and Testing - Philip J. Boland, Department of Statistics, National University of Ireland – Dublin. Challenges in Software Reliability and Testing - Philip J. Boland, Department of Statistics, National University of Ireland – Dublin. Y. K. Malaiya and J. Denton “ Module Size Distribution and Defect Density,” Proc. IEEE International Symposium on Software Reliability Engineering, Oct. 2000, pp Y. K. Malaiya and J. Denton “ Module Size Distribution and Defect Density,” Proc. IEEE International Symposium on Software Reliability Engineering, Oct. 2000, pp Module Size Distribution and Defect Density Module Size Distribution and Defect Density Operational Profile Specification, Test Case Generation, and Reliability Estimation for Modules - D.M. Woit, Queen’s University, February Operational Profile Specification, Test Case Generation, and Reliability Estimation for Modules - D.M. Woit, Queen’s University, February N. Li and Y.K. Malaiya “ROBUST: A Next Generation Software Reliability Engineering Tool” Proc. IEEE Int. Symp. on Software Reliability Engineering, pp , Oct N. Li and Y.K. Malaiya “ROBUST: A Next Generation Software Reliability Engineering Tool” Proc. IEEE Int. Symp. on Software Reliability Engineering, pp , Oct Predicting Software Reliability from Testing Taking into Account Other Knowledge about a Program - Antonia Bertolino (IStituto di Elaborazione della Informazione del CNR, Pisa,Italy), Lorenzo Strigini (Centre for Software Reliability, City University, London). Predicting Software Reliability from Testing Taking into Account Other Knowledge about a Program - Antonia Bertolino (IStituto di Elaborazione della Informazione del CNR, Pisa,Italy), Lorenzo Strigini (Centre for Software Reliability, City University, London).