1. S Ramakrishnan1 Systems V & V, Quality and Standards Dr Sita Ramakrishnan School CSSE Monash University

2. S Ramakrishnan2 JUnit  Eric Gamma and Kent Beck created a simple & effective unit testing framework for Java called Junit in 1997. Junit in 1997. z part of XP (Extreme Programming method) but can be used independently. but can be used independently. zJunit (Framework) provides a reusable structure which can be shared between apps. which can be shared between apps. z Developers can incorporate this framework into their apps. and extend it to meet their specific their apps. and extend it to meet their specific requirements. requirements.

3. S Ramakrishnan3 JUnit  Junit (junit.org) is an open source software and is hosted on SourceForge. z defacto standard framework for developing unit tests in Java. z many unit testing frameworks available: Eiffel, PhP, Perl, ASP, C++, C# and …

4. S Ramakrishnan4 JUnit  A typical unit test: Check that a method accepts input in an expected range & returns the expected result value for each test input. z test the behaviour of a method through its interface. Test for expected values, also for exception z check if the method obeys the design by contract principles in terms of API contract.

5. S Ramakrishnan5 Unit testing frameworks z A simple Calculator class: z The core method, add: takes 2 doubles & returns the sum as a double public class Calculator { public double add(double num1, double num2) { return num1 + num2; } z Put in some work to test this simple “unit of work” will do the adding 2 doubles & returning the sum as a double correctly. And, also preserve this effort so that you can replicate this test and run the tests again later! z Objective test cases turned into reusable test cases -> Test Program

6. S Ramakrishnan6 Unit testing frameworks z A simple TestCalculator program – manual test public class TestCalculator { public static void main(String[ ] args) { Calculator calculator = new Calculator ( ); double result = calculator.add(10, 20); if (result != 30) { System.out.println(“Incorrect result: “ + result); } zThe test will pass when you compile & run this test program. If you change this code deliberately to make it fail, you are not testing the code! You should handle error conditions in Java by throwing exceptions.

7. S Ramakrishnan7 Unit testing frameworks z A simple TestCalculator program - Modified public class TestCalculator { private int errCount = 0; public void testAdd ( ); // moving the test into its own method { Calculator calculator = new Calculator ( ); double result = calculator.add(10, 20); if (result != 30) { throw new RuntimeException(“Incorrect result: “ + result); } public static void main (String [ ] args) { TestCalculator test = new TestCalculator ( ); try { test.testAdd ( ); // can add more methods like testAdd with more unit tests in the test calculator & invoke them here } catch (Throwable e) { test.errCount++; e.printStackTrace ( ); } if (test.errCount > 0) { throw new RuntimeException(“Number of Errors : “ + test.errCount “ ); }

8. S Ramakrishnan8 Unit testing frameworks z Lessons from the last simple example re: unit testing frameworks: Each unit test must run independently of all other unit tests must be easy to define which unit tests will run errors must be detected and reported for each test separately. The modified vers.(last slide) can be improved. Each unit test must be independent & run in a different classloader instance.

9. S Ramakrishnan9 Unit testing frameworks z A better option is to consider a unit test suite. problem with this approach: large try/catch block can be a maintenance headache Another approach is to use Java’s reflection & introspection features. A program looks at itself & decide to run methods which are named in a certain way, eg. those begin with letters, test. The Junit framework supports introspecting methods, supports the use of a different classloader instance for each test and reports errors for each test separately.

10. S Ramakrishnan10 Testing with JUnit Create a test script with a number of small java methods the idea is to create java objects, do something with these and check if the objects exhibit correct properties Assertions methods to check properties such as identity of objects, equality of variables can check if objects are null or non-null; equality of objects (via == or equals () depending on the type) use assertions to determine the verdict of the test case. In XP methodology, a JUnit test should be written first before any code, and executed. the implementation code should be written to get the test to pass re-execute the test with this code and it should pass.

11. S Ramakrishnan11 Junit TestCase (Usually, a test case is considered to be a single test that can pass or fail, and a testsuite is a bunch of related testcases). In Junit, run multiple test cases withTestSuite & Testcase. They are implemented as classes. No difference between running a testsuite & a testcase. A TestCase is a class and a single test is a method. TestCalculator program written with Junit import junit.framework.TestCase; // Junit needs it to automatically run the tests public class TestCalculator extends TestCase; { public void testAdd ( ); // method name follows the pattern testXXX { Calculator calculator = new Calculator ( ); // start the test by creating an instance (object under test) double result = calculator.add(10, 20); // execute the test by calling the method to test, passing it known values assetEquals ( 30, result, 0); // check the result by calling assertEquals method, inherited from base class, TestCase. } Javadoc for assertEquals method /** *Asserts that two doubles are equal concerning a delta. … */ Static public void assertEquals (double expected, double actual, double delta) In the above code, assertEquals is passed: expected = 30, actual = result, delta = 0 Often, delta parameter can be zero and can be ignored. It is used when floating point calc. are done. Delta provides a + / - factor. If the actual data is within the range of expected +/- delta, the test passes.

12. S Ramakrishnan12 Junit TestCase public void run(TestResult result) // run method in TestCase Class { result.startTest (this); setUp ( ); // set up for all tests in class try { runTest ( ); // run the test } catch (AssertionFailedError e) { result.addFailure (this, e); } catch (Throwable e) { result.addError (this, e); } finally { tearDown ( ); // cleanup for all tests in class }

13. S Ramakrishnan13 The Junit Framework TestResult Test TestCaseTestSuite yourTestClass-1 yourTestClass-n run(TestResult) runTest ( ) setup ( ) tearDown ( ) run(TestResult) addTest(Test) fName fTests *suite ( ): TestSuite

14. S Ramakrishnan14 Junit workings Junit uses reflection to build the testsuite dynamically use getClass ( ) on an object to know which class it belongs to use getMethods ( ) to find out the methods in a class use invoke ( ) on each method to run it This ensures that test suite need not be updated if a method is added or deleted from the test suite. Test class is re loaded each time the tests are run this means that JUnit test execution window need not be restarted if test cases are recompiled To run a Test object - either a TestCase or a TestSuite, simply invoke the run ( ) method.

15. S Ramakrishnan15 More info on JUnit available at Junit web site: http://junit.orghttp://junit.org JUnit Cookbook: Kent Beck and Erich Gamma http://junit.sourceforge.net/doc/cookbook/cookbook.htm JUnit Cook tour: http://junit.sourceforge.net/doc/cookstour/cookstour.htm Mocks are’nt stubs - Martin Fowler (jul 2004), http://www.martinfowler.com/articles/mocksArentStubs.html Inversion of control containers and the dependency injection pattern, M Fowler(jan 2004), http://www.martinfowler.com/articles/injection.htmlhttp://www.martinfowler.com/articles/injection.html Vincent Massol with Ted Husted (2004), JUnit in Action, Mannings Publ. J B Rainsberger (2005), JUnit Recipes, Mannings Publ. E Hatcher and S Loughran (2002), Java Development with Ant, Mannings Publ. (Refer to Ch.4 Testing with Ant and Appendix E, Ant Task Reference available Online)

16. S Ramakrishnan16 Core JUnit Classes TestCase + TestSuite + BaseTestRunner TestCase + TestSuite + BaseTestRunner = TestResult JUnit trio - JUnit trio - backbone of JUnit Framework – to create test results To create a single test case, we extended the TestCase class (slide11) To run several TestCase objects at once, create another object called a TestSuite. T estRunner is an user interface for launching test suites. BaseTestRunner is the superclass for all testrunners.

17. S Ramakrishnan17 Core JUnit Classes JUnit distribution provides 3 TestRunner classes that can be used to execute your tests: one for text console, one for AWT and one for Swing test runner. With the Swing TestRunner, the progress indicator across the screen is the JUnit green to indicate pass and the red bar to indicate a failing test. compile the simple Testcalculator program & use the swing TestRunner for running a testcase. When JUnit is started by typing java junit.swinggui.TestRunner TestCalculator the JUnit framework does 3 things: creates a TestSuite, creates a TestResult and executes the test methods (testAdd in slide 11). Ok, how to run multiple test cases? -> TestSuite! TestRunner launches (runs) the TestSuite. Which test cases to run is up to the TestSuite.

18. S Ramakrishnan18 JUnit TestSuites For the simple Testcalculator class with a testcase (slide11), a default TestSuite is: public static Test suite ( ) { return new TestSuite(TestClaculator.class); } which is equivalent to: public static Test suite ( ) { TestSuite suite = new TestSuite ( ); suite.addTest (new TestCalculator (“testAdd”)); return suite; }

19. S Ramakrishnan19 JUnit TestSuites If one adds another test case, testSubtract, the default TestSuite would automatically include it like so: public static Test suite ( ) { TestSuite suite = new TestSuite ( ); suite.addTest (new TestCalculator (“testAdd”)); suite.addTest (new TestCalculator (“testSubtract”)); return suite; } Framework can generate a default run-time testsuite for you. You must provide the test case class Provision of such automatic testsuite ensures that you don’t forget to add additional tests to the test suite. Default TestSuite is enough for simple code & simple testing. What if you need something more than what the default can do? may want to combine suites from different packages into one may want to run only a subset of the tests when adding new features & testing for those

20. S Ramakrishnan20 JUnit TestSuites TestCase and TestSuite implement the Test Interface. package junit.framework; public interface Test { public abstract int countTestCases ( ); public abstract void run (TestResult result); } the ability to add test suites & test cases to a suite makes it easier to create specific suites as well as a master TestAll class for your apps. TestAll class is a static suite method that registers which Test Objects (TestCase or TestSuite objects) your apps. should be running on a regular basis.

21. S Ramakrishnan21 A TestAll class import junit.framework.Test; import junit.framework.TestSuite; Import junitbook.sampling.TestDefaultController; public class TestAll { public static Test suite ( ) // create a suite method to call your other tests or suites { TestSuite suite = new TestSuite(“All tests from Part 1”); // give a name to testsuite to identify // it later suite.addTestSuite(TestCalculator.class); // addTestSuite to TestCase or TestSuite objects // that you want to run together suite.addTestSuite(TestDefaultController.class); // addTestSuite works with this type as well // as this method accepts a Test object as a parameter, & both TestCase // and TestSuite implement Test interface Refer to Chapter 5 of JUnit in Action text book at techniques for automating tasks so that you Don’t have to create & maintain a TestAll class. But, you still need to create specific suites to cater for discrete subsets of your tests.

22. S Ramakrishnan22 TestResult A TestResult collects the results of a TestCase execution TestResult stores the details of tests: pass or fail assertEquals(30, result, 0) ; -- If result is not equal to 30, JUnit will create a TestFailure object & store it in TestResult TestRunner uses the TestResult to report the outcome of your tests if no TestFailures in the TestResults collection, green bar! if failures, TestRunner reports the failure count and the stack trace for failing tests. JUnit differentiates between failures & errors. Failure – for eg. - assertion failing when code is modified; Error is an unexpected condition such as an exception.

23. S Ramakrishnan23 Design Patterns and JUnit Composite Pattern Compose objects into tree structures to represent part-whole hierachies. Composite lets clients treat indiv. objects & Composition of objects uniformly (E Gamma et al. GOF1995) JUnit use of the Test interface to run a single test or a suite of tests is an e.g. of the composite pattern. When you add an object to a TestSuite, you are adding a Test, and not just a TestCase. Since both TestCase & TestSuite implement Test, you can add either to a suite. If the Test is a TestCase, the single test is run, when the Test is a TestSuite, group of tests is run, which in turn can have other testsuites.

24. S Ramakrishnan24 Design Patterns and JUnit Command pattern “Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations” (E Gamma et al. 1995) E.g. of the command pattern : the use of the Test interface to provide a common run method Collecting parameter “ When you need to collect results over several methods, you should add a parameter to the method and pass an object that will collect the results for you” (K Beck, Smalltalk Best Practices Pattern, Prentice-Hall, 1996) E.g. TestResult class is an eg. of the collecting parameter pattern

25. S Ramakrishnan25 Design Patterns and JUnit Observer Pattern “Design a one-to-many dependency between objects so that when one object changes state, all its dependencies are notified and updated automatically”. TestRunner registering as a TestListener with the TestResult is and eg. of an Observer pattern.

26. S Ramakrishnan26 The Core JUnit Classes Source: JUnit in Action > Test Assert TestCase TestSuite O.. * TestResult BasicTestRunner > TestListener O.. 1 0.. *

27. S Ramakrishnan27 JUnit’s TestListener interface TestResult collects info about the test and TestRunner reports it. Can more than one object report on a test at the same time? Does an object have to be a TestRunner to report on a test? JUnit framework provides the TestListener interface to help access the TestResult and produce meaningful reports. The TestRunners implement TestListener A number of TestListeners can register with the framework and do anything required with the info provided by the TestResult. TestListener is not an interface you need to implement to write your tests with the JUnit framework, unless you are extending the framework.

28. S Ramakrishnan28 TestCase components – Fixtures and Unit Tests Fixture is a set of resources or data that you may need to run a series of tests, eg. DB connection & access test tables avoids putting the common set up code in your tests Fixture is automatically created & destroyed by a TestCase setUptearDown through its setUp & tearDown methods. setUp TestCase calls setUp before running each of its tests and calls tearDown the tearDown when each test is completed. TestCase life cycle recreates the scaffolding (fixture) for each test method. Putting a number of test methods in the same TestCase enables sharing the fixture code. setUp ()testXXX ( )tearDown ( )

29. S Ramakrishnan29 Fixtures – to manage resources Mock objects or stubs can be used to simulate db connections – refer to ch. 6 & 7 of JUnit in Action text book & urls in the reference Managing resources such as db connections Testcase may include several db tests, each needing a fresh connection to the db fixture makes it easy for you to open a connection for each test without replicating code can use a fixture to generate input files, so you don’t carry test files with your tests & also able to have a known state before a test is executed.

30. S Ramakrishnan30 Unit Test methods Assert Interface and assert methods The JUnit framework encapsulates common testing tasks with ssert assert methods. The assert methods are defined in a Assert utility class called Assert. The class provides 8 core public methods (assertXXX) for building tests: assertTrue, assertFalse, assertEquals, assertNotNull, assertNull, assertSame, assertNotSame, fail - with a number of forms for these methods to make it easy to pass various types that you may need in your test (Refer to Javadoc for details on Assert interface) commonly used form of assertEquals assertEquals (String message, Objects expected, Object actual)

31. S Ramakrishnan31 Unit Test methods TestCase implements 10 TestCase methods, which are not part of the Assert interface: countTestCase, createResult, getName, run, runBare, runTest, setName, setUp, tearDown, toString TestCases mainly use setUp & tearDown methods. Developers involved with JUnit extensions may be interested in the other 8 methods given above. 8 assert methods from Assert and 10 testcase methods from TestCase are available for building unitTests with JUnit.

32. S Ramakrishnan32 Unit Tests As the name suggests, unit tests should run independently of other unit tests. unit tests must run in any order and must not depend on effects of previous tests. Dependency between unit tests are a problem in JUnit Portability issues - JUnit finds test methods by reflection. Reflection API does not guarantee the order in which test method names are returned. If your tests depend on ordering, your testsuite may work on one JVM but fail in another JVM. Maintenance issues – if tests are dependent & you change one test, you may find that other tests are affected. Need to understand how each works. Tests become hard to read & maintain.

33. S Ramakrishnan33 JUnit Results- Reporting If you run JUnit in a text mode, it gives a very limited written report on the console. JUnit reports with Swing testrunners are visual with the green & red bar, and with exception traces for reporting failures & errors but non permanent. If reports are needed for later review and for documentation purposes, you can do it using Ant, Eclipse and some other tools..

34. S Ramakrishnan34 JUnit Results- Reporting Can improve the default JUnit reporting by using the Ant’s “JUnit’s report” feature to produce XML output & html pages the Java build tool “Ant” formatting features can turn an ordinary JUnit report into a nice set of web pages. use “Ant” to run JUnit to produce pretty printed JUnitReports.