Chapter 11, Testing: Unit Testing with JUnit 4 Note to Instructor: The material in this slide set is not contained in the 3rd edition of the text book. It is planned for the 4th edition. However, because the major differences between to old JUnit 3 and the new JUnit 4, it might be advisable to use at least the JUnit 4 material.

Outline of the next 2 Lectures Testing Activities Unit testing with jUnit Java Annotations and Assertions Object-Oriented Test Modeling Using object-oriented concepts also for the test model Object-oriented Testing Patterns Mock object pattern Dependency injection pattern (not only a testing pattern) Inner class test pattern Reflection test pattern Test exception pattern 4 stage testing pattern Today Next week Test model as a new abstraction on the same level as the system model Dependency injection pattern (not only a testing pattern, good for your system health )

Requirements Analysis Testing Activities Object Design Document System Design Document Requirements Analysis Document Client Expectation Unit Testing Integration Testing System Testing Acceptance Testing Developer Client

Types of Testing Unit Testing Integration Testing System Testing Individual component (class or subsystem) Carried out by developers Goal: Confirm that the component or subsystem is correctly coded and carries out the intended functionality Integration Testing Groups of subsystems (collection of subsystems) and eventually the entire system Goal: Test the interfaces among the subsystems. System Testing The entire system Carried out by developers Goal: Determine if the system meets the requirements (functional and nonfunctional) Acceptance Testing Evaluates the system delivered by developers Carried out by the client. May involve executing typical transactions on site on a trial basis Goal: Demonstrate that the system meets the requirements and is ready to use.

JUnit: Overview A Java framework for writing and running unit tests Test cases and fixtures Test suites Test runner Written by Kent Beck and Erich Gamma Written with “test first” and pattern-based development in mind Tests are written before coding the SUT Observe the ones that fail Write new SUT code or fix existing SUT code to make the test pass Allows for regression testing Facilitates refactoring JUnit is Open Source www.junit.org JUnit Version 4, released Mar 2006 The JUnit framework is very easy to understand. A basic interface exists (Test), and the test classes must implement this interface. There are two types of test classes: TestCase and TestSuite: TestCase is the main class that we extend for writing all our tests. It contains the methods that implement the individual tests with the signature public void testMethodName() and the optional private setup() and teardown() methods. TestSuite assembles a collection of TestCase(s), other TestSuite(s), or a combination of the two. Assert is the superclass of TestCase. It provides all the methods for our tests to check the values or method calls. A test can pass, fail, or have errors. TestResult accumulates the results of individual tests in a run. It keeps information about the start and the end of the test and also about the failures and errors. TestFailure provides access to the failure or error that occurs during the test. This includes detailed information on the exception. Failures are the anticipated problems with the code. We use assertions to check for the possibility of failures. Errors are unanticipated problems. If a test method does not catch an exception, that exception is raised to the JUnit framework and it reports that as an error. It is important to understand that JUnit reports the first failure in a single test. It executes each test within a separate instance of the test class. It reports failure on each test. It is better to split the assertions into individual tests rather than use JUnit for complete functional testing using one test method. The framework defines an error class called AssertionFailedError, which is thrown whenever an assertion fails. The JUnit framework catches the error and reports that the test is failed. TestRunners display the information contained within the object to the user. TestListener is an interface that can be used by any class to monitor the progress of a test run. It contains methods for recording the starting state, the end state, and any failures or errors in a test. We discussed the three types of TestRunners in a previous section. Each of them extends a BaseTestRunner and provides the interfaces to the JUnit framework.

JUnit 4 and xUnit Frameworks Version 4: Annotation-based Simplified test setup Extension hooks for new runners AssertEquals() for comparing arrays „xUnit“ frameworks nUnit (.NET) pyUnit (Python) cppUnit (C++) dUnit (Delphi)

A Java Example class Money { private int fAmount; private String fCurrency; public Money(int amount, String currency) { fAmount= amount; fCurrency= currency; } public int amount() { return fAmount; public String currency() { return fCurrency; public Money add(Money m) { return new Money(amount()+m.amount(), currency()); }

Unit Testing add() with JUnit 4.0 The unit test MoneyTest tests that the sum of two Moneys with the same currency contains a value that is the sum of the values of the two Moneys Static import of Assertion package (Older JUnit versions used inheritance) import org.junit.Test; import static org.junit.Assert.*; public class MoneyTest { @Test public void simpleAdd() { Money m12CHF= new Money(12, "CHF");  Money m14CHF= new Money(14, "CHF");  Money expected= new Money(26, "CHF");  Money observed= m12CHF.add(m14CHF);  assertTrue(expected.equals(observed)); }   Calling the SUT Method Assertion: Returns True if parameter of type Boolean evaluates to True

Assertions in JUnit 4.0 assertTrue(Predicate); fail(String)  Returns True if Predicate evaluates to True fail(String)  Let the method fail, useful to check that a certain part of the code is not reached. assertsEquals([String message], expected, actual) Returns message if the values are the same assertsEquals([String message], expected, actual, tolerance) Used for float and double; tolerance specifies the number of decimals which must be the same assertNull([message], object) Checks if the object is null and prints message if it is assertNotNull([message], object) Check if the object is not null assertSame([String], expected, actual) Check if both variables refer to the same object assertNotSame([String], expected, actual) Check that both variables refer not to the same object assertTrue([message], boolean condition) Check if the boolean condition is True try {a.shouldThroughException(); fail("Failed")} catch (RuntimeException e) {assertTrue(true);} Alternative way for checking for exceptions

Unit Testing add() with JUnit 4.0 The unit test MoneyTest tests that the sum of two Moneys with the same currency contains a value that is the sum of the values of the two Moneys Static import of Assertion package (Older JUnit versions used inheritance) import org.junit.Test; import static org.junit.Assert.*; public class MoneyTest { @Test public void simpleAdd() { Money m12CHF= new Money(12, "CHF");  Money m14CHF= new Money(14, "CHF");  Money expected= new Money(26, "CHF");  Money observed= m12CHF.add(m14CHF);  assertTrue(expected.equals(observed)); }   Annotation: Declaration of a Test Method simpleAdd() Calling the SUT Method Assertion: Returns True if parameter of type Boolean evaluates to True

Annotations in JUnit 4.0 @Test public void foo() Annotation @Test identifies that foo() is a test method @Before public void bar() Perform bar() before executing a test method @After public void foobar() A test method must finish with call to foobar() @BeforeClass public void foofoo() Perform foofoo() before the start of all tests. Used to perform time intensive activities, e.g. to connect to a database @AfterClass public void blabla() Perform blabla() after all tests have finished. Used to perform clean-up activities, e.g. to disconnect to a database @Ignore(string S) Ignore the test method prefixed by @Ignore, print out the string S instead. Useful if the code has been changed but the test has not yet been adapted @Test(expected=IllegalArgumentException.class) Tests if the test method throws the named exception @Test(timeout=100) Fails if the test method takes longer then 100 milliseconds

@Ignore: Omitting Tests There are situations where certain tests should not be executed by the test harness Example: the current release of a third-party library used in the SUT has a bug public class CalculatorTest { @Ignore(„Don‘t run now this test until bug in Foo is fixed") @Test public void testSpecialFunctionality() { ... }

@Test(timeout): Making Sure Tests are short Unit tests should be short But some tests take their time, particularly if network connectivity is involved In these cases it is recommended to set an upper bound for the test @Test(timeout=5000) public void testLengthyOperation() { ... }

Another JUnit Example import org.junit.Test; import static org.junit.Assert.*; public class CalculatorTest { private int x = 1; private int y = 1; @Test public void add() { int z = x + y; assertEquals(2, z); } In JUnit 4, the unit tests don’t have to be in a special test class anymore. They can be anywhere in the code Identifies that add() is a test method Assertion

Annotations in JUnit 4.0 @Test public void foo() Annotation @Test identifies that foo() is a test method @Before public void bar() Perform bar() before executing a test method @After public void foobar() A test method must finish with call to foobar() @BeforeClass public void foofoo() Perform foofoo() before the start of all tests. Used to perform time intensive activities, e.g. to connect to a database @AfterClass public void blabla() Perform blabla() after all tests have finished. Used to perform clean-up activities, e.g. to disconnect to a database @Ignore(string S) Ignore the test method prefixed by @Ignore, print out the string S instead. Useful if the code has been changed but the test has not yet been adapted @Test(expected=IllegalArgumentException.class) Tests if the test method throws the named exception @Test(timeout=100) Fails if the test method takes longer then 100 milliseconds

@Before and @After: Ensuring Pre- and Post Conditions Any Method can be decorated with @Before and @After: public class CalculatorTest { @Test public void add() @Test public void sub() @Before public void setupTestData(){ } //executed before every add/sub @After public void teardownTestData() {} //executed after every add/sub } A Class containing a set of tests can be decorated with @BeforeClass and @AfterClass This is useful for expensive setups that do not need to be run for every test, such as setting up a database connection. public class CalculatorTest { @BeforeClass // executed at instantiation of class public static void setupDatabase Connection() { ... } @AfterClass // executed after remove instance of class public static void teardownDatabase Connection() { ... } }

Outline of the Lecture Test Model Model-Based Testing Generation of a test model from the system model Model-Driven Testing Distinction: platform independent and platform dependent tests Testing Activities Unit testing with JUnit Java Annotations and Assertions Object-Oriented Test Modeling Using object-oriented concepts also for the test model Object-oriented Testing Patterns Mock object pattern Dependency injection pattern (not only a testing pattern) Inner class test pattern Reflection test pattern Test exception pattern 4 stage testing pattern Test model as a new abstraction on the same level as the system model Dependency injection pattern (not only a testing pattern, good for your system health )

Object-Oriented Test Modeling We start with the system model The system contains the SUT (the unit we want to test) The SUT does not exist in isolation, it collaborates with other objects in the system model The test model is derived from the SUT To be able to interact with collaborators, we add objects to the test model These objects are called test doubles Mock objects -> not synonyms System Model Test Model System under Test (SUT) Double 1 Double 2 Double 3 Collaborators (Objects interacting with the SUT)

Object-Oriented Test Modeling We start with the system model The system contains the SUT (the unit we want to test) The SUT does not exist in isolation, it collaborates with other objects in the system model The test model is derived from the SUT To be able to interact with collaborators, we add objects to the test model These objects are called test doubles These doubles are substitutes for the Collaborators during testing Mock objects -> not synonyms System Model Test Model System under Test (SUT) Double 1 Double 2 Double 3 Collaborators (Objects interacting with the SUT) Collaborators (Objects interacting with the SUT)

Subclasses of Test Doubles A test double is like a double in the movies („stunt double“) replacing the movie actor, whenever it becomes dangerous A test double is used if the collaborator in the system model is awkward to work with There are 4 types of test doubles. All doubles try to make the SUT believe it is talking with its real collaborators: Dummy object: Passed around but never actually used. Dummy objects are usually used to fill parameter lists Fake object: A fake object is a working implementation, but usually contains some type of “shortcut” which makes it not suitable for production code (Example: A database stored in memory instead of a real database) Stub: Provides canned answers to calls made during the test, but it is not able to respond to anything outside what it is programmed for Mock object: Mocks are able to mimic the behavior of the real object. They know how to deal with sequence of calls they are expected to receive. For more details see Martin Fowler‘s online paper: http://martinfowler.com/articles/mocksArentStubs.html XXX title difference between superclass and subclasses? Mock objects -> behaviour, interaction rest -> state

Motivation for the Mock Object Pattern Let us assume we a system model for an auction system with 2 types of policies. We want to unit test Auction, which is our SUT Instanzen Dynamic polymorphism. PersonDummy -> later in animation -> Mock Real objects in addition

Motivation for the Mock Object Pattern Let us assume we a system model for an auction system with 2 types of policies. We want to unit test Auction, which is our SUT The mock object test pattern is based on the idea to replace the interaction with the collaborators in the system model, that is Person, the Bidding Policy and the TimingPolicy by mock objects These mock objects can be created at startup-time with a factory pattern. Instanzen Dynamic polymorphism. PersonDummy -> later in animation -> Mock Real objects in addition

Mock-Object Pattern In the mock object pattern a mock object replaces the behavior of a real object called the collaborator and returns hard-coded values These mock objects can be created at startup-time with a factory pattern Mock objects can be used for testing state of individual objects as well as the interaction between objects, that is, to validate that the interactions of the SUT with collaborators behave is as expected. http://robkuijt.nl/testdesignpatterns/show.php?zk=unit-test-simulation-patterns.txt#Mock-Object-Pattern

Example: System Model of a Simple Inventory System

Java Code for the Simple Inventory System public class InventorySystem { private static String TALISKER = "Talisker"; private int totalStock; private Warehouse warehouse = new WarehouseImpl(); public void addToWarehouse(String item, int amount) { warehouse.add(item, amount); totalStock += amount; } public int getTotalStock() { return totalStock; public boolean processOrder(Order order) { order.fill(warehouse); return order.isFilled(); public static void main(String[] args) { InventorySystem inventorySystem = new InventorySystem(); inventorySystem.addToWarehouse(TALISKER, 50); boolean order1success = inventorySystem.processOrder(new OrderImpl(TALISKER, 50)); boolean order2success = inventorySystem.processOrder(new OrderImpl(TALISKER, 51)); System.out.println("Order1 succeeded? " + order1success + " - Order2 succeeded? " + order2success); ?

UML Model of the InventorySystem Note that in our Java example, the Client is realized as the main() method in the class InventorySystem

Java Code for the Warehouse public interface Warehouse { public boolean hasInventory(String item, int amount); public int getInventory(String item); public void add(String item, int amount); public void remove(String item, int amount); } public class WarehouseImpl implements Warehouse { private Map<String, Integer> inventory = new HashMap<String, Integer>(); public void add(String item, int amount) { inventory.put(item, amount); public int getInventory(String item) { return inventory.get(item); public boolean hasInventory(String item, int amount) { return inventory.get(item) >= amount; public void remove(String item, int amount) { inventory.put(item, inventory.get(item) - amount);

Java Code for the Order public interface Order { public boolean isFilled(); public void fill(Warehouse warehouse); } public class OrderImpl implements Order { private String item; private int amount; private boolean filled = false; public OrderImpl(String item, int amount) { this.item = item; this.amount = amount; public void fill(Warehouse warehouse) { if(warehouse.hasInventory(item, amount)) { warehouse.remove(item, amount); filled = true; public boolean isFilled() { return filled;

Test Model of the InventorySystem Assume we now want to unit testthe class Order, our SUT

A test case for testing the State public class OrderStateTester { private static String TALISKER = "Talisker"; private Warehouse warehouse = new WarehouseImpl(); @Before public void setUp() throws Exception { warehouse.add(TALISKER, 50); } @Test public void orderIsFilledIfEnoughInWarehouse() { Order order = new Order(TALISKER, 50); order.fill(warehouse); assertTrue(order.isFilled()); assertEquals(0, warehouse.getInventory(TALISKER)); @Test public void orderDoesNotRemoveIfNotEnough() { Order order = new Order(TALISKER, 51); assertFalse(order.isFilled()); assertEquals(50, warehouse.getInventory(TALISKER)); XXX Collaborator -> external Actor? XXX Mocks as Proxy? +

What if we not only want to test the state, but also the interaction of the SUT (Order) with its collaborator (Warehouse)? Here the Mock Object Pattern comes into play. It will be covered in more detail in the next lecture.

Readings Kent Beck, Erich Gamma, Junit Cookbook http://junit.sourceforge.net/doc/cookbook/cookbook.htm JUnit Source Forge: http://sourceforge.net/projects/junit/files/junit/ JUnit 4: http://www.ibm.com/developerworks/java/library/j-junit4.html JUnit Fixtureshttp://www.informit.com/articles/article.aspx?p=101374&seq Num=5 Martin Fowler, Mocks are not Stubs http://martinfowler.com/articles/mocksArentStubs.html

Additional Slides

Difference between Stub and Mock Assume the real systems send an email message if a user failed to fill an order using this interface: public interface MailService { public void send (Message msg); } During testing we don't want to send actual email messages out to customers. So we want to create a test double If we were writing a test for the mailing behavior, we can write a simple stub and driver: Stub: public class MailServiceStub implements MailService { private List<Message> messages = new ArrayList<Message>(); public void send (Message msg) { messages.add(msg); } public int numberSent() { return messages.size();} } Driver: class OrderStateTester... public void testOrderSendsMailIfUnfilled() { Order order = new Order(TALISKER, 51); MailServiceStub mailer = new MailServiceStub(); order.setMailer(mailer); order.fill(warehouse); assertEquals(1, mailer.numberSent()); }

Applicability of the Mock Object Pattern The unit test needs to test an object with nondeterministic behavior Violation of good test design: Repeatability Example: Current weather temperature An object is difficult to set up Violation of good test design: Performance Example: The set up takes too long for running many unit tests A specific behavior is hard to trigger Example: network error The methods of an object are very slow Example: Climate modeling The object has a user interface or is the user interface itself The unit test needs to confirm that a callback function was actually called The real object cannot be tested, because it does not (yet) exist This is a common problem when interfacing with subsystems built by other teams or when interfacing to new hardware systems. Problems in unit testing: