1. Testing Object Oriented Programs CSE 111 4/28/20151

2. 2 Traditional Stages in Testing Unit testing Test individual classes or small groups of classes during development Integration testing Test larger groups of classes and subsystems System testing Test the whole system. Use test scripts that describe a sequence of user- program interactions and expected resulting behavior 4/28/2015

3. Traditional Testing (and Analysis) Methods Functional or black box testing: test with normal and oddball cases. – normal: expected functionality – oddball: invalid inputs, empty arrays, etc. Coverage: makes sure all the program (e.g. branches, variables and data structures) has been used on a test Code reading: read the code, mentally simulating what it is supposed to be doing 4/28/20153

4. Specific Object-Oriented Techniques Self-testing classes Test case classes Stubs, drivers and mock objects Class design features that facilitate testing – i.e. design for testability Subsystem testing strategy for 3-tier system Inheritance and abstract test cases 4/28/20154

5. 5 Self-Testing Classes Each class has a main method that is used for testing Except for the class with the “real” main method that is used to start the program Testing a class A with its main() method: Create an instance of the class A Call its methods, checking the results Report results of tests 4/28/2015

6. 6 Test Case Class Special class T for testing another class or classes (the CUT = class under test) T contains a collection of special test and verify methods main() method in test case class T – Creates an instance of the CUT, performs initialization, runs its test methods 4/28/2015

7. 7 Tests and Test Suites Test = TestCase or TestSuite TestSuite = Collection of Tests 4/28/2015

8. 8 TestRunners Run the tests for a set of classes Needs to be able to know which methods in a class are the test methods it needs to run If test methods use special assert statements to check results, then results are automatically reported 4/28/2015

9. J-Unit Originally was far more complex and flexible, but was difficult to use JUnit 4 – Special annotation is used to indicate which method is a test method may be in the CUT or in separate test case classes – Supports specialized kinds of assert statements and other advanced, simplifying features 4/28/20159

10. Sample JUnit Test Case import static org.junit.Assert.assertEquals; public class AdditionTest { private int x = 1; private int y = 1; @Test public void addition() {int z = x + y; assertEquals(2, z); } 4/28/201510

11. Stubs and Drivers Suppose we want to test a class C which supports some responsibility, but is called by other classes, and calls classes itself How to isolate C during testing? – Create a “driver” test case class whose test methods call methods in C. The test methods contain assert statements that check the results of the tests – Create dummy “stub” objects that emulate the classes called by C. Do not need to be a full implementations, just have enough logic to handle the calls resulting from the tests 4/28/201511

12. 12 Class Design - Testability Designing classes to assist in testing – Mock objects and parameterized class constructors 4/28/2015

13. 13 Mock Objects and Parameterized Constructors 1 O/O equivalent of a stub is called a “mock object” – Suppose the Class Under Test (CUT) uses instances of another, “auxiliary” class – Suppose that the auxiliary class is not available, or want to test CUT in isolation – Need to a create an instance of a mock object class M which stands in for or simulates instances of the actual class 4/28/2015

14. 14 Mock Objects and Parameterized Constructors 2 Suppose a constructor for class A takes an instance of class B – Define the constructor parameter using an interface C, which B implements (“interface” in the Java sense) – This means that instances of alternative classes can be passed, as long as the classes implement interface C. e.g. a matching mock object class M 4/28/2015

15. 15 Mock Objects and Parameterized Constructors 3 Suppose that objects of class A send messages to an object of class B. In the design, could use attribute visibility, in which, for example, the constructor for A creates an instance of B and assigns it to a class variable in A Facilitate testing with a design that uses parameter visibility instead where the constructor for A has a parameter for passing an instance of B which is created “outside” of A Now follow the approach of the previous slide, using an interface to declare the parameter type so that either instances of B or mock objects can be passed 4/28/2015

16. DS Example The Domain Logic subsystem has to send messages to the DataBase subsystem (i.e. to the DB Java interface class) Instead of creating the DataBase subsystem instance “inside of” DomainLogic, “pre-create” the DataBase (interface) instance and pass it as an interface-typed parameter in the DomainLogic constructor Permits easy testing of DomainLogic with a stub/mock object for the DataBase, and seamless transference to use of a real DataBase instance 4/28/201516

17. 17 JUnit and Mock Objects Suppose class A uses an object of class B, and this is passed in the constructor for A, with a parameter of type C In a test case class which tests A – inside its test method(s) create a mock object of type C create an instance of the CUT (in this case B) under test, and pass the mock object in the constructor for B – requires that the C be an interface type 4/28/2015

18. Test Strategies for Our Three-Tier System Integration testing – Separate testing of GUI, DomainLogic and DataBase subsystems – Subsystems small, so may simply do integration testing for each subsystem, with no unit testing DataBase – test DB directly, test DB through the DL DomainLogic – test DL directly, using mock DB, use test cases simulate the GUI GUI – manual testing with DL mock object – possible separate testing of GUI classes using “information objects” 4/28/201518

19. 19 DataBase Testing Why not simply test the DB in the context of the whole system? Typical test case would – Set up or check DB status – Perform application methods – Check effects by accessing DB 4/28/2015

20. 20 Problems with Whole System - Strategy for Testing the DataBase Want to test DB directly to observe effects of DB method calls Want directly examine the effects of “complex” sequences of DB method calls that would occur as the result of DL activity Need to directly test the correctness of the interfaces between DB methods and the DL methods that call them 4/28/2015

21. 21 Testing the DB Directly - Individual Method Applications Test the DB by writing test cases that execute methods in the DB interface Use black box test selection, backed up by coverage analysis, for each DB interface method DB interface may have only a small number of methods, which we can focus on individually 4/28/2015

22. Testing the DB Directly - Method Sequences Problems may occur due to sequences of DB method applications BET (bounded exhaustive testing) – test exhaustively over all inputs with a “small version” of the application – e.g. DS. Only has 2-3 members, sequences of user interactions limited to length, say, 3 4/28/201522

23. Sample DB Bug in DS Example Sequence of interactions used to reveal bug 1. initialize() /*reads in files from DB, examples with 0,1,2 entries 2. delete(x) where x is not a member of the DB Note: this defect is not discoverable if a deletion of a non-existent member is performed after a deletion of an existing member has been performed. So simple testing of a non-existing member may not work. BET picks up the strange short sequence that causes the failure 4/28/201523

24. 24 Testing the DomainLogic/DB Interfaces Purpose is to test the links from one to the other – ignore functionality for now – write test cases that call methods in the DL (Java) interface, which result in calls from DomainLogic class methods to the DB (Java interface) – focus on testing each of these DL/DB interfaces for consistency, as simply and directly as possible Note: you may find interface inconsistencies during focused test construction 4/28/2015

25. 25 Test Case Classes for Testing the DB Directly TestCase contents use a class variable for DB class instance setup() method would create a DB instance and assign to the DB instance variable (this is an empty DB) tearDown() method deletes everything from DB testx() methods would create and retrieve entries from data base, using DataBase (Java interface) methods and method sequences. Assertions in the test methods will reference data returned from DB accesses 4/28/2015

26. Test Case Classes for Testing DomainLogic/DB Interfaces Test Case contents – use a class variable for DB class instance – setup() method would create an instance dB of the DB (Java interface) and possibly use dB methods to populate the data base – testx() methods would create an instance of DomainLogic, passing dB as a constructor method parameter execute DomainLogic methods use DB methods directly to check results – teardown() removes effects of tests 4/28/201526

27. Domain Logic Testing Test the methods in the DL interface using black box and coverage testing measures – include coverage testing of called methods in the DL – focus on DL (Java interface) methods Simulate the GUI – test methods would have (sequences of) calls on DL that simulate a GUI action 4/28/201527

28. 28 Test Case Classes for Domain Logic Testing 1)setUp() method in the test case could: 1)create an instance of the mock DB, 2)create an instance of the DL, passing the mock DB as a parameter. 2)testx() method could 1)execute DL methods, checking results with an assert() 4/28/2015

29. 29 GUI Testing How do we test the GUI? – We will use manual GUI testing only – Test cases will create an instance of a mock DL and pass it as a parameter in the constructor for the GUI – Need to construct scripts that document tester input and expected results for each test case Checking results: tester inspection via GUI 4/28/2015

30. Separate Testing of GUI Classes Suppose that we want to test a GUI class more directly, by itself, but it calls on/creates other GUI classes? – If other GUI classes simply present results, keep them because they are the test output – Suppose that the other GUI classes accept input and return information How to delink these subordinate GUI objects? 4/28/201530

31. 31 Delinking GUI Classes Suppose that a window A creates a dialog B for gathering some information Instead create a Java interface class Info with a method getInfo() – getInfo() returns an information object for test use, define an implementation of Info where getInfo() sets and returns an information object directly for production use, define an implementation of Info where getInfo() creates an instance of B, reads the information, and then returns it in an information object 4/28/2015

32. Setting the Info object Two possibilities – The GUI object A has a constructor with a parameter for passing in the Info object – requires an external class, to construct the subclass implementation instances of Info for test or production runs – The GUI class A can be abstract with an abstract factory method create() – Subclass A with a definition of create() that uses the correct implementation constructor, depending on whether A is being used for testing or production use 4/28/201532

33. Test Strategies for Our Three-Tier System System Testing – Combine DomainLogic and with a real DB instances, simulate actions of GUI can also be classified as a higher level of integration testing – Test whole system through the GUI – apply black box, BET coverage and code reading are not usually done at the system level 4/28/201533

34. DS System Testing Example When doing BET sequence testing included the operations of terminating and starting a new DS session In the DS if you logon as a member, and then ask to edit your data, and do not enter any new properties, and then terminate a session, you will get a crash when you try to restart the system with a new session 4/28/201534

35. 35 New Topic: Abstract Test Cases Suppose that a class C’ subclasses a class C – Can we develop a test case T for C that we can re- use for C’? – Consider the case where C is an interface class – In general, only some tests be reused, and there is obvious way to determine which 4/28/2015

36. 36 Creating Abstract Test Cases To create an abstract test, construct an abstract test class T so that it uses an abstract factory method createC() to create instances of C If C’ is a subclass of C, which implements the interface, then create a subclass T’ of T in which the implementation of the abstract factory method createC() uses the constructor for C’ to create an concrete instance of C (i.e. instance of C’) 4/28/2015

37. 37 Polymorphism and TestCases Polymorphic design pattern: – A set of classes corresponds to related alternative forms of behavior – Can capture alternatives in a set of subclasses of a class – Subclasses have a common set of methods, but exhibit different behavior depending on the subclass in which they are defined 4/28/2015

38. 38 Creating Abstract Polymorphic Test Cases Use approach similar to above, for inheritance Usefulness? Depends on how much logic in the abstract test case methods is reusable for the subclasses 4/28/2015

39. 39 Additional Test Frameworks: HttpUnit HttpUnit emulates the relevant portions of browser behavior, including form submission, JavaScript, basic http authentication, cookies and automatic page redirection, and allows Java test code to examine returned pages either as text, an XML DOM, or containers of forms, tables, and links. When combined with a framework such as JUnit, it is fairly easy to write tests that very quickly verify the functioning of a web site.JavaScript http authenticationJUnit 4/28/2015

40. Assignment 8 Use the principles in the lecture to design tests for your project – Use J-Unit to construct and document 2 test cases for 2 classes. Use normal and oddball tests. – Construct system level tests as per earlier system testing slide (33) for user inputs use normal and oddball cases use BET to test short sequences of operations 4/28/201540