1. Software Testing and Quality Assurance Testing Interactions

2. 2 Reading Assignment  John McGregor and David A. Sykes, A Practical Guide to Testing Object-Oriented Software, Addison-Wesley, 2001, ISBN: 0-201-325640. Chapter 6: Testing Interactions

3. 3 Objectives To understand different types of interactions. Learn techniques on how to select test cases to run.

4. 4 Topics covered Introduction Object interactions Testing object interactions Sampling test cases Testing off-the-shelf components Protocol testing Test patterns Testing exceptions

5. 5 Object Interactions An instance of a trusted primitive class, for example, may contain no faults, but if the services of that instance are not used correctly by other program components, then the program contains faults. Thus the correct collaboration—or interaction—of objects in a program is critical to the correctness of the program.

6. 6 Introduction Object-oriented program comprises a collection of objects that collaborate to solve some problem. Most classes have collaborators:  The methods in the class interact with instances of other classes. The interactions being tested are between objects at runtime. The focus of interaction testing is ensuring that messaging occurs correctly with objects whose classes have already been tested separately.

7. 7 Object interactions An object interaction is simply a request by one object (the sender) to another (the receiver) to perform one of the receiver’s operations and all of the processing performed by the receiver to complete the request. Object interactions include:  Messages between an object and its components  Messages between an object and other objects with which it is associated. We want to consider the impact of these interactions both on the internal state of the receiving object and on those objects with which it has an association

8. 8 Partial Class Testing Partial class testing is a result of the iterative, incremental development approach. In general,  It is not possible to sequence the development of a class so that all the classes it needs to interact with are totally developed and tested.  A project's schedule is usually based on delivery of end-user functionality (use cases).

9. 9 Partial Class Testing (cont.) Lower level— Primitive classes are more likely to be completely developed at one time and tested as a complete unit. Other classes are therefore developed and tested incrementally.

10. 10 Object Interactions: Identifying Interactions A primitive class is one that can be used without any collaboration with other objects. A nonprimitive class is one that requires collaboration with other objects in order to accomplish its objectives.

11. 11 Object Interactions: Identifying Interactions (cont...) Possible collaborations:  Parameter: public operation collaborates with an object passed in as an argument (formal parameter).  Return: public operation returns an object with which it collaborates.  Internal: a method of a class creates an instance of another as part of its implementation.  Global: reference to a global instance of some class (undesired).

12. 12 Object Interactions: Identifying Interactions (cont...) Categories of nonprimitive classes based on degree of interaction:  Collection classes: maintain references to instances of other classes, but do not interact with those instances.  Collaborating classes: interacts directly with its associated classes  classes that are not collection classes.

13. 13 Object Interactions: Identifying Interactions — Collection Classes Collection classes: Use objects in their specifications but never actually collaborate with any of them. Instead they do one of the following:  Store references  Create instances  Delete instances Examples: lists, stacks, and queues.

14. 14 Object Interactions: Identifying Interactions — Collaborating Classes Collaborating Classes  Non-primitive classes that are not collection classes.  They are classes that use other objects in one or more operations as part of their implementation.  When a post condition of an operation in a class's interface refers to the state of an instance of an object and/or specifies that some attribute of that object is used or modified, then that class is a collaborating class.

15. 15 Object Interactions: Identifying Interactions — Collaborating Classes (cont...) The number of potential collaborations can become impossibly large quickly. Often the bugs that are most serious do not arise from the interaction of two objects, but from the interactions between a complete set of objects.

16. 16 Object Interactions: Identifying Interactions — Collaborating Classes (cont...) Factors that determine the size for testing:  Composition and association: The interaction of the composing object with its composed attributes is tested. The interaction between an object and its associated objects are tested as successive layers of aggregation are integrated.  The number of layers of aggregations created between interaction tests is closely related to the visibility of defects.

17. 17 Object Interactions: Identifying Interactions — Collaborating Classes (cont...)  Object complexity: the more complex the objects, the fewer that should be integrated prior to a round of testing. number of parameters for each method, number of methods, number of attributes in each object.  Trying to test a chunk that is too complex often results in defects that successfully hide from the tests.

18. 18 Testing Object Interactions: Testing Collection Classes Collection classes are tested using techniques for primitive classes:  A test driver will create instances that are passed as parameters in messages to a collection being tested.  Test cases center around ensuring that those instances are correctly incorporated into and removed from the collection class  If defensive design has been used then include negative test cases in the test suite.

19. 19 Specifying Interactions Assume that operations defined by a class are specified by preconditions, post conditions, and class invariants. It is important to know whether defensive design or design by contract has been used in creating the specification of the particular interface to be tested.

20. 20 Specifying Interactions (Cont.) Defensive design assumes that little or no checking of parameter boundaries occurs prior to a message being sent.  reduces the number of clauses in preconditions,  requires checks internally for violations of attribute constraints,  increases the number of clauses in post conditions.

21. 21 Specifying Interactions (Cont.) Design by contract assumes that appropriate preconditions are checked prior to a message being sent and that the message is not sent if any of the parameters are outside acceptable limits.  increases the number of clauses in preconditions,  requires no checking internally for violations of attribute constraints, and  reduces the number of clauses in the post condition clause.

22. 22 Testing Object Interactions: Testing Collaborator Classes The complexity of testing a collaborating class is greater than that of testing a collection class or a primitive class. For a collaborator class under test:  Identify the collaborating classes  Determine the sequence of testing Collaborating classes, Next level of collaboration Then the next level of collaboration and so on

23. 23 The Interaction between Testing and Design Approach Contract design places more responsibility on the human designer than on error-checking code. This reduces the amount of class-level testing since there are fewer paths due to a smaller amount of error-checking code. At the interaction level, there is more testing required in order to be certain that the human designer has complied with the client side of the contract using precondition constraints. A focus of interaction testing is whether the preconditions of methods in a receiving object are being met by the sending object.

24. 24 The Interaction between Testing and Design Approach (Cont.) It is not legitimate to build test cases that violate these preconditions. The intention is to determine that the sending object checks the preconditions of the receiving object before sending the message inappropriately. The test should also check whether the sending object aborts correctly, probably by throwing an appropriate exception.

25. 25 Sampling Test Cases When the number of test cases is too large to handle reasonably A systematic technique is needed for determining which ones to use. We are interested with a testing approach in which the level of coverage can be increased systematically.

26. 26 Sampling Test Cases Population: all possible test cases that can be executed. It includes all preconditions and all possible combinations of input values. A sample is a subset of a population that has been selected based on some probability distribution. Possibilities for determining which test cases to select  Based on probability distribution A probability distribution defines, for each data value in a population, a set of allowable values and the probability that value will be selected.  Uniform probability distribution Each value in the population is assigned the same selection probability

27. 27 Sampling Test Cases (cont...) A stratified sample is a set of samples in which each sample represents a specific subpopulation Stratified sample tests are selected from a series of categories. A population of tests is divided into subsets so that a subset contains all of the tests that exercise a specific component. Sampling occurs on each subset independent of the others Example: select a sample of test cases from the use cases of each actor

28. 28 Sampling Test Cases (cont...) Stratifying the test case samples by each actor provides an effective means of increasing the reliability of the system. Running the selected tests uses the system the way that it will be used in typical situations and finds those defects that are most likely to be found in typical use. Removing these defects produces the largest possible increases in reliability with the smallest effort.

29. 29 Sampling Test Cases (cont...) Use of random number generator in sampling  Advantage: all values have equal probability  Disadvantage: test cases cannot be reproduced.  Example Velocity class, set direction: values between 0 and 359, inclusive. int (random() * 360).

30. 30 Sampling Test Cases (cont...) The first dimension in sampling is considering the class family  Class family: a set of classes related by inheritance. Example, Sprite is the base class in a very large class family, which is a set of classes related by inheritance. An object from any one of the classes in the family can be substituted for the sprite parameter.

31. 31 Sampling Test Cases (cont...) The second dimension for sampling  Consider that each member of the family may have different states that can cause two objects from the same class to behave differently.  There will be the same number of states or more states in the derived class as there are in the base class.  Cover the states defined for each class with special emphasis on the new states added at that level in the inheritance hierarchy.

32. 32 Sampling Test Cases : Orthogonal Array Testing (OATS) Orthogonal arrays provide a specific sampling technique that seeks to limit the explosion by defining pair-wise combinations of a set of interacting objects. An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment). It represents a specific class family (a class and its children) E.g. 3 factors with 3 levels each (27 possibilities), with pair-wise only 9 possibilities

33. 33 Sampling Test Cases : Orthogonal Array Testing (OATS) This is a systematic way of reducing the number of test cases. Most of the errors that are encountered are between pairs of objects rather than among several objects.

34. 34 Sampling Test Cases : Orthogonal Array Testing (OATS) Decide how many independent variables will be tested for interaction - Factors of the array. Decide the maximum number of values that each independent variable will take on - the Levels of the array. Find a suitable orthogonal array - has at least as many Factors as needed - has at least as many levels for each of those factors Map the Factors and values onto the array. Transcribe the Runs into test cases.

35. 35 Sampling Test Cases : OATS Example 1 Number of states:  Class A: 2  Class B: 3  Class P: 2  Class C: 2  Class D: 3  Class E: 3 * State transition diagrams are not shown

36. 36 Sampling Test Cases : OATS Example 1 Step 1: Identify all factors:  sending hierarchy,  receiving hierarchy,  parameter position in the message six factors in the example: class A hierarchy, class P hierarchy, class C hierarchy and factors for the states associated with each class hierarchy Step 2: Determine levels for each factor by considering the set of possible values:  One factor has one level: the parameter class family only has one member: P  Two factors have a maximum of two levels; A and C; the maximum number of states for a class in the P family is two.  Three factors have a maximum of three levels: B, D and E.

37. 37 Sampling Test Cases : OATS Example 1 Step 3: Locate a standard orthogonal array that fits the problem. 6 factors 3 levels

38. 38 Sampling Test Cases : OATS Example 1 Step 4: Establish a mapping from each factor onto the integers in the array so that standard array can be interpreted  The first column in L 18 can be used to represent the sender class family which has two classes: A and B (1 corresponds to A class and 2 corresponds to class B)  The second column in L 18 when there is a difference in the number of levels (class A has 2 states and class B has 3 states).  The third column in L 18 represents the parameter hierarchy that only has one class, P. (any value in the third column represents P).  The fourth column represents the state of P (which there are two).  The fifth column represents the class C hierarchy, which has three members.  The sixth column represents the states of the C, D, and E classes.

39. 39 Sampling Test Cases : OATS Example 1 2

40. 40 Sampling Test Cases : OATS Example 1 Step 5: Construct test cases based on the mapping and the rows in the table.  Each row in the orthogonal array specifies one specific test case.

41. 41 Sampling Test Cases : OATS Example 1 — Result Interpretation The 10 th column represents:  An instance of B in state 1  Is to send a message by passing an instance of class P in state 2  To an instance of class E in state 2.

42. 42 Sampling Test Cases : OATS Example 2 Consider a web page with three distinct sections (Top, Middle, and Bottom) that can be individually shown or hidden by the user. You wish to test the interactions of the different sections.

43. 43 Sampling Test Cases : OATS Example 2 There are three independent variables (the sections of the page). Each variable can take on two values (hidden or visible). An L 4 (2 3 ) orthogonal array will do the job — two levels for the values and three factors for the variables. Mapping the values onto the array where Hidden=0 and Visible=1

44. 44 Sampling Test Cases : OATS Example 2 OA before mapping factors Factor 1Factor 2Factor 3 Run 1000 Run 2011 Run 3101 Run 4110 OA after mapping factors TopMiddleBottom Test case 1Hidden Test case 2HiddenVisible Test case 3VisibleHiddenVisible Test case 4Visible Hidden

45. 45 Sampling test cases: Adequacy Criteria for OATS Exhaustive: all possible combinations of all factors are considered. Minimal: only the interactions between the base classes from each hierarchy are tested. Random: the tester haphazardly selects cases from several of the classes. Representative: a uniform sample that ensures that every class is tested to some level. Weighted Representative: Add cases to the representative approach based on relative importance or risk associated with the class.

46. 46 Testing off-the-shelf components Functionality can be added to an application by purchasing “Components”. Quality of components varies from vendor to another. Acceptance testing for the component in the context in which it will be used is necessary. Stress tests: extreme, may be correct or incorrect values e.g., moving mouse a lot to generate a large number of mouse move events, making multiple menu selections, etc. Tests are normally based on the main class.

47. 47 Testing off-the- shelf components: example ―a component test plan for Grid

48. 48 Protocol testing Protocol testing investigates whether the implementation of a class satisfies its specification Protocol: sequence of method invocations by combining a method whose postconditions enable the precondition of another method Each protocol corresponds to a sequence of states beginning with initial states of the two objects, a sequence of states for each object, and ending with terminal states (denoted in the state diagram) A test case takes the two objects through one complete sequence of methods

49. 49 Protocol testing: Example Timer maintains list of observers to notify when a timer event occurs. When it is enabled, it notifies the observers

50. 50 Protocol testing: Example A protocol can be found by tracing through the state diagram.  One protocol would be to create the object then sending one or more attach(...) messages followed by the enable() message, the disable message, and finally, the delete() message.  This provides a life-cycle test case.  This provides an effective test of the object in the ways that it will interact with its client objects.

51. 51 Test patterns Test patterns are design patterns for test software. A design pattern: captures and reuses design knowledge that has gained widespread application in the OO software development community, e.g., Grid component is based on the Listener design pattern A pattern is a specific configuration of interactions among a set of objects that form some cluster in the overall design. For a design pattern there can be a test pattern

52. 52 Testing exceptions An exception: provides an alternative route for returning from a method that does not necessary move to the next statement after the method is invoked. Exceptions are powerful:  The exceptional return value is an object and can be arbitrary complex.  The points at which an exception is thrown varies based on the depth of the aggregation hierarchy.

53. 53 Testing exceptions (cont...) Testing exceptions provides two perspectives:  At the class level, whether each of the methods in that class does throw the exceptions that it claims to in the appropriate circumstances. The coverage criteria requires that a class throws every exception contained in the specifications of the methods. There would be at least one test case per exception.  During integration, interaction testing will determine whether the exceptions that are being thrown at correct time are being caught at correct place. The coverage criteria for this level of testing is to be certain that every exception thrown is caught in the correct location. Both of these points of view can be tested very early in the development.

54. 54 Testing exceptions: testing interactions at the system level When the components become complex it is easier to test them in the context of the application itself instead of in the environment provided by a test driver. The interactions that can be tested at the system level are only those that can be verified at the system level (e.g. user interface):

55. 55 Key points Primitive and nonprimitive classes. Possible collaborations: parameter, return, internal, global. Categories of nonprimitive classes: collection and collaborating classes Factors that determine the size for testing  Composition and association:  aggregations  Object complexity

56. 56 Key points (cont...) Possibilities for determining which test cases to select: based on probability distribution, uniform probability distribution and probability distribution based on the use profile. A stratified sample is a set of samples in which each sample represents a specific subpopulation An orthogonal array is an array of values in which each column represents a factor (a variable in an experiment). Adequacy criteria for OATS: exhaustive, minimal, random, representative, and weighted representative.

57. 57 Key points (cont...) Stress tests: extreme, may be correct or incorrect values Protocol: sequence of method invocations by combining a method whose postconditions enable the precondition of another method A design pattern: captures and reuses design knowledge that has gained widespread application in the OO software development community An exception: provides an alternative route for returning from a method that does not necessary move to the next statement after the method is invoked.