1 Software Testing and Quality Assurance Lecture 25 – Testing Interactions (Chapter 6)

2 Lecture Outline Sampling Test Cases Orthogonal array testing

3 Sampling test cases Population: all possible test cases that can be executed A sample is a subset of a population that has been selected based on some probability distribution.

4 Sampling test cases Possibilities for determining which test cases to select Based on probability distribution A probability distribution defines, for each data value in a population, a set of allowable values and the probability that value will be selected. Uniform probability distribution Each value in the population is assigned the same selection probability For example, Probability distribution based on the use profile

5 Sampling test cases (cont...) A stratified sample is a set of samples in which each sample represents a specific subpopulation Example: select a sample of test cases from the use cases of each actor Stratified sample tests are selected from a series of categories.

6 Sampling test cases (cont...) Use of random number generator in sampling Advantage: all values have equal probability Disadvantage: test cases cannot be reproduced (you may let the test driver record the generated values) Class family: a set of classes related by inheritance.

7 Sampling test cases: Orthogonal array testing (OATS) Orthogonal arrays provide a specific sampling technique that seeks to limit the explosion by defining pair-wise combinations of a set of interacting objects. An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment).

8 Sampling test cases: Orthogonal array testing (OATS) It represents a specific class family (a class and its children) E.g. 3 factors with 3 levels each (27 possibilities), with pair-wise only 9 possibilities

9 Sampling test cases: OATS Example Decide how many independent variables will be tested for interaction. This will map to the Factors of the array. Decide the maximum number of values that each independent variable will take on. This will map to the Levels of the array.

10 Sampling test cases: OATS Example Find a suitable orthogonal array. A suitable array is one that has at least as many Factors as needed from Step 1 and has at least as many levels for each of those factors as decided in Step 2. Map the Factors and values onto the array. Transcribe the Runs into test cases.

11 Sampling test cases: OATS example Number of states: Class A: 2 Class B: 3 Class P: 1 Class C: 2 Class D: 3 Class E: 3 * State transition diagrams are not shown

12 Sampling test cases: OATS example (cont...) Step 1: Identify all factors: sending hierarchy, receiving hierarchy, parameter position in the message (six factors in the example: class A hierarchy, class P hierarchy, class C hierarchy and factors associated with each class hierarchy)

13 Sampling test cases: OATS example (cont...) Step 2: Determine levels for each factor by considering the set of possible values: One factor has one level: the parameter class family only has one member: P Two factors have a maximum of two levels; B and C. Three factors have a maximum of three levels: A, D and E.

14 Sampling test cases: OATS example (cont...) Step 3: Locate a standard orthogonal array that fits the problem. 6 factors 3 levels

15 Sampling test cases: OATS example (cont...)

16 Sampling test cases: OATS example (cont...) Step 4: Establish a mapping from each factor onto the integers in the array so that standard array can be interpreted The first column in L 18 can be used to represent the sender class family which has two classes: A and B (1 corresponds to A class and 2 corresponds to class B) The second column in L 18 when there is a difference in the number of levels (class A has 2 states and class B has 3 states).

17 Sampling test cases: OATS example (cont...) The third column in L 18 represents the parameter hierarchy that only has one class, P. (any value in the third column represents P). The fourth column represents the state of P (which there are two). The fifth column represents the class C hierarchy, which has three members. The sixth column represents the states of the C, D, and E classes.

18 Sampling test cases: OATS example (cont...) Step 5: Construct test cases based on the mapping and the rows in the table. Each row in the orthogonal array specifies one specific test case.

19 Sampling test cases: OATS example— result interpretation The 10 th column represents: An instance of B in state 1 Is to send a message by passing an instance of class P in state 3 To an instance of class E in state 2.

20 Sampling test cases: OATS another example Consider a web page with three distinct sections (Top, Middle, and Bottom) that can be individually shown or hidden by the user. You wish to test the interactions of the different sections.

21 Sampling test cases: OATS another example (cont...) There are three independent variables (the sections of the page). Each variable can take on two values (hidden or visible). An L 4 orthogonal array will do the job — two levels for the values and three factors for the variables. Mapping the values onto the array where Hidden=0 and Visible=1

22 Sampling test cases: OATS another example (cont...) OA before mapping factors Factor 1Factor 2Factor 3 Run 1000 Run 2011 Run 3101 Run 4110 OA after mapping factors TopMiddleBottom Test case 1Hidden Test case 2HiddenVisible Test case 3VisibleHiddenVisible Test case 4Visible Hidden

23 Sampling test cases: adequacy criteria for OATS Exhaustive: all possible combinations of all factors are considered. Minimal: only the interactions between the base classes from each hierarchy are tested. Random: the tester haphazardly selects cases from several of the classes.

24 Sampling test cases: adequacy criteria for OATS Representative: a uniform sample that ensures that every class is tested to some level. Weighted representative: Add cases to the representative approach based on relative importance or risk associated with the class.

25 Key points Possibilities for determining which test cases to select: based on probability distribution, uniform probability distribution and probability distribution based on the use profile. A stratified sample is a set of samples in which each sample represents a specific subpopulation An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment). Adequacy criteria for OATS: exhaustive, minimal, random, representative, and weighted representative.