1 Software Testing and Quality Assurance Lecture 30 – Testing Systems

2 Lecture Objectives Learn how to write test cases from use cases Learn how to quantify the quality of testing

3 Complementary strategies for selecting test cases: Orthogonal Defect Classification (ODC) ODC captures information about the type of faults that are present in a software system under development

4 Use cases as sources of test cases Constructing use profiles Using scenarios to construct test cases The expected result section of a test case

5 Use cases as sources of test cases: constructing use profiles The construction of use profile begins with actors from the use case diagram. After deployment the use case frequencies can be updated based on actual data and used for regression testing.

6 Use cases as sources of test cases: Using scenarios to construct test cases The process for identifying specific values from the variables mentioned in a use case has four steps: Identify all of the values that will be supplied by the actors contained in the use case. Identify equivalence classes of values for each input data type. Construct tables that list combinations of values from the various equivalence classes. Construct test cases that combine a single permutation of values with the necessary environmental constraints.

7 Use cases as sources of test cases: Using scenarios to construct test cases— example

8 Use cases as sources of test cases: Using scenarios to construct test cases— example (cont...) Each row specifies a test.

9 Use cases as sources of test cases: the expected result section of a test case For complex systems it is very difficult to determine the expected results from a test run. E.g. spacecraft control software.

10 Use cases as sources of test cases: the expected result section of a test case Techniques to reduce that amount of effort required to develop expected results: Construct the results incrementally The test cases are written to cover some subset of a use of the system. E.g. database (from 50 to 1000 records) Grand tour test cases: results of one test case are the inputs for the next test case Disadvantage: a failure to test case 1 we can’t run test 2.

11 Testing multiple representations Systems are written using a composite of complied servers (c++), interpreted clients (java), and multiple data sources. Important features of these systems from a testing perspective: Interactions between two data models. Interactions between pieces written in different languages. Interaction between static and dynamic portions of the program.

12 Testing multiple representations (cont...) To address these features perform a thorough unit test process: All pieces of the system should be exercised Test the interactions across language and representation boundaries. Test the interfaces between two languages to determine that a complete object is transferred.

13 Testing multiple representations (cont...) Test the following: Be certain that each connection in either direction is exercised Validate the results by examining the transformed objects in detail to ensure that nothing is missing or misinterpreted Measure coverage by covering all of the primitive types of each language

14 What needs to be tested: testing for qualitative system attributes Testing for qualitative system attributes Translate each qualitative claim into measurable attribute Design test cases that can detect the presence or absence of these measurable attributes Execute the test cases and analyze the results Aggregate these results to determine if a specific claim is justified.

15 What needs to be tested: testing for qualitative system attributes Validate performance claims: Quantify the term acceptable performance (a number of transactions per second) Create new data or use historical data to use in the tests. Run the tests, collect results and timing data. Make determination (pass or fail) for the claim.

16 What needs to be tested: testing the system deployment Testing the system deployment: required for configurable systems and for applications that require dynamic interaction with the environment. Deployment testing is intended to ensure that the packaging used for the system provides adequate setup and delivers a product in working condition.

17 What needs to be tested: testing the system deployment (cont...) The initial test case if a full, complete installation. Interactions between options Certain options may not be installed Libraries or drivers may be needed for other options

18 What needs to be tested: testing the system deployment (cont...) Levels of installation: Typical Custom full

19 What needs to be tested: deployment testing technique Identify categories of platforms on which the system will be deployed. Locate at least one system of each type that has a typical environment but that has not had the system installed on it. Install the system using the deployment mechanism. Run a regression set of system tests and evaluate the results.

20 What needs to be tested: test system security Three categories of issues that could be classified as security: The ability of the application to allow authorized access and to prevent access by unauthorized persons. The ability of the code to access all of the needed resources that it needs to execute. The ability of the application to prevent unauthorized access to other system resources not related to the application.

21 What needs to be tested: test system security Modularity of executables and the dynamic aspects of the code does raise security issues: Permissions for files deployed in a directories: test one resource in each directory and one user from each security class.

22 Key points Testing the system deployment: required for configurable systems and for applications that require dynamic interaction with the environment.

23 Announcement SWE Revised Program Discussion Monday December 29, 2008 at 12:10-1:00 in 24/141