Software Development Tools COMP220 Seb Coope Ant, Testing and JUnit (2) These slides are mainly based on “Java Development with Ant” - E. Hatcher & S.Loughran. Manning Publications, 2003 and “JUnit in Action”, 2nd edition – P. Tahchiev, F. Leme, V. Massol, G. Gregory, Manning Publications, 2011

JUnit Primer (with ANT) continued

More on writing a test case Recall that a test case is a class which imports some JUnit packages import org.junit.Test; import static org.junit.Assert.*; ... and implements one or more no-argument void @Test methods testXXX() (See 11. Ant, Testing and JUnit, JUnit Primer, Slides 5, 6.) where each testXXX() consists of several assertion methods typically comparing expected and actual values.

More on writing a test case: setUp() and tearDown() Additionally, to prevent one @Test method from affecting the behaviour of another we can annotate by @Before and implement a method public void setUp() which initializes objects under test by storing a state (fixture) as member variables to the test case Optionally, annotate by @After and implement a method public void tearDown() which releases objects/resources under test. For this to work, we also need to import from Junit: import org.junit.Before; import org.junit.After; NOTE: The setUp() and tearDown() methods are called before and after every test method is invoked, preventing one test from affecting the behaviour of another. The names setUp and tearDown are traditional after JUnit3. JUnit4 annotations @Before and @After make the behaviour of setUp and tearDown as required. (See also next slides.)

Running a test case with setUp() and tearDown() Test Runner calls the same setUp() and tearDown() methods before and after each @Test method testXXX() is invoked. Thus, each @Test method testXXX() runs under one and the same fixture created by setUp()(despite this fixture could probably be changed by other test methods testYYY()), hence, with no assumptions about the order in which test methods are called. This prevents one test from affecting the behaviour of another. The following example illustrates this behaviour by using System.out.println("some message"). Thus, each testXXX() (typically a small and simple test method) JUnit’s TestRunner, when applied to a test case, preceeds running each testXXX() by calling setUp() and proceeds by calling tearDown() to close any open connections, or in some way reset state, and to ensure there can be no side effects among test runs.

Test Case setUpTearDownTest.java : package org.example.antbook.junit; //import JUnit4 classes: import static org.junit.Assert.*; import org.junit.Test; // import org.junit.Before; //import org.junit.* also would work import org.junit.After; // public class setUpTearDownTest{ @Before //Runs before each @Test method public void setUp(){ System.out.println("setUp sets up a fixture"); } @After //Runs after each @Test method public void tearDown(){ System.out.println("tearDown releases fixture"); } @Test public void testA(){ System.out.println("testA runs"); assertTrue("MULTIPLICATION FAILED!!!", 4 == (2 * 2)); } @Test //Each method annotated by @Test runs public void testB(){ System.out.println("testB runs"); assertSame("ADDITION FAILED!!!", 4, 2 + 2); public void SomeTestC(){ System.out.println("SomeTestC runs"); assertSame("ADDITION FAILED!!!", 5, 2 + 2); Put this file under ch04\test according to the package declaration. Use your personalised package name! Here setUp() and tearDown() methods do nothing besides informing that what they intend to do. @Test methods JUnit’s TestRunner, when applied to a test case, preceeds running each testXXX() by calling setUp() and proceeds by calling tearDown() to close any open connections, or in some way reset state, and to ensure there can be no side effects among test runs. Should it fail/run ???

Do not forget to recompile each time!!! Compile and Run: C:\Antbook\ch04>javac -d build\test test\org\example\antbook\junit\setUpTearDownTest.java C:\Antbook\ch04>java -cp build\test;C:\JAVA\junit4.8.2\junit-4.8.2.jar org.junit.runner.JUnitCore org.example.antbook.junit.setUpTearDownTest JUnit version 4.8.2 .setUp sets up a fixture testA runs tearDown releases fixture testB runs Time: 0.016 OK (2 tests) Do not forget to recompile each time!!! setUp and tearDown surround each @Test annotated method. javac -d build\test test\org\example\antbook\junit\setUpTearDownTest.java java -cp build\test;C:\JAVA\junit4.8.2\junit-4.8.2.jar org.junit.runner.JUnitCore org.example.antbook.junit.setUpTearDownTest JUnit3: java -cp build\test;C:\JAVA\junit4.8.2\junit-4.8.2.jar org.junit.runner.JUnitCore org.example.antbook.junit.setUpTearDownTest JUnit’s TestRunner, when applied to a test case, preceeds running each testXXX() by calling setUp() and proceeds by calling tearDown() to close any open connections, or in some way reset state, and to ensure there can be no side effects among test runs. Note that someTestC was not run, unlike testA and testB! WHY?? Because only @Test annotated methods should be run by Junit4 test runners. testA() and testB(), succeeded, so, setUpTearDownTest succeeded too (OK). Replace 4 by 5 in testA() and recompile and run it again. What will be changed? Will setUpTearDownTest succeed?

Compile and Run: E means that the previous testA failed Your message C:\Antbook\ch04>java -cp build\test;C:\JAVA\junit4.8.2\junit-4.8.2.jar org.junit.runner.JUnitCore org.example.antbook.junit.setUpTearDownTest JUnit version 4.8.2 .setUp sets up a fixture testA runs tearDown releases fixture E.setUp sets up a fixture testB runs Time: 0.015 There was 1 failure: 1) testA(org.example.antbook.junit.setUpTearDownTest) java.lang.AssertionError: MULTIPLICATION FAILED!!! <many lines skipped> FAILURES!!! Tests run: 2, Failures: 1 Do not forget to recompile each time!!! E means that the previous testA failed javac -d build\test test\org\example\antbook\junit\setUpTearDownTest.java java -cp build\test;C:\JAVA\junit4.8.2\junit-4.8.2.jar org.junit.runner.JUnitCore org.example.antbook.junit.setUpTearDownTest JUnit 4: JUnit’s TestRunner, when applied to a test case, preceeds running each testXXX() by calling setUp() and proceeds by calling tearDown() to close any open connections, or in some way reset state, and to ensure there can be no side effects among test runs. Your message to yourself Add @Test annotation before someTestC, and compile and run it again. What will be the result? Finally, compile and run this by Ant from mybuild.xml like in Part 11, Slide 14.

@RunWith(value=Suite.class). Writing a Test Suite What if we need to run many JUnit test cases? JUnit, test classes can be grouped into a suite Just use appropriate JUnit imports and annotations @SuiteClasses and @RunWith(value=Suite.class). Grouping tests lets you build several individual test cases for a particular subsystem , and write an all-inclusive Test Suite that runs them all . See AllTests.java below. Note that Suite of Suites can also be created in the same way. 4.3.5 A Test Suite also allows specific ordering of tests, if it is important – although ideally the order of tests should not be relevant as each should be able to stand alone. Here is an example of a test suite AllTests.java:

An example of a Test Suite C:\Antbook\ch04\test\org\example\antbook\AllTests.java package org.example.antbook; import org.junit.runner.RunWith; import org.junit.runners.Suite; import org.junit.runners.Suite.SuiteClasses; @RunWith(value=Suite.class) @SuiteClasses(value= {org.example.antbook.junit.SimpleTest.class, org.example.antbook.junit.setUpTearDownTest.class, org.eclipseguide.persistence.FilePersistenceServicesTest.class } ) public class AllTests{} WAS: junit-TestRunner-AllTests End of file Test Suite to run test methods of three test cases: SimpleTest, setUpTearDownTest and FilePersistenceServicesTest. We can also compose suites of suites in the same way. EXTEND target junit-TestRunner in mybuild.xml to RUN test suite AllTests.java.

Test Suites and Ant?? Don’t need to bother with test suites when running JUnit tests using Ant: Ant itself can list a group of TestCase classes in a much more powerful way – all test cases from a directory(!), and run it as a batch from the build file itself. But <java> task is not appropriate for this purpose. OMIT???4.3.6 Obtaining and installing JUnit

The JUnit task <junit>

The JUnit task <junit> The Ant's task <junit> is much better for running JUnit test cases than <java> which we used till this point: <junit> runs one or more (even all from a directory) JUnit test cases Collects and displays test results in one or more formats Provides a way either to fail or continue a build when a test fails Recall the unnatural situation when BUILD SCCESSFUL while some tests failed. No need to mention explicitly any Test Runner (as it was necessary with <java> task) 4.5 This will be discussed and demonstrated later

The JUnit task <junit> (cont.) In order to execute some existing test cases by Ant, let us declare a special <junit> task with the names of the test cases and appropriate classpath : <target name="test-brief" depends="test-compile"> <junit> <classpath refid="test.classpath"/> <test name = "org.eclipseguide.persistence.FilePersistenceServicesTest"/> <test name="org.example.antbook.junit.SimpleTest"/> </junit> </target> <classpath> with id test.classpath formerly used for running test cases by <java> and some <javac> tasks (say, FilePersistenceServicesTest and SimpleTest) At least one benefit: no need to use (explicitly) TestRunner One or more test cases to run Extend mybuild.xml by this target. You can also add: <test name="org.example.antbook.junit.setUpTearDownTest"/>

The JUnit task <junit> (cont.) TRY it: H:\Antbook\ch04>ant -f mybuild.xml test-brief Buildfile: C:\Antbook\ch04\mybuild.xml [SOME MESSAGES OMITTED] test-brief: [junit] Test org.eclipseguide.persistence. FilePersistenceServicesTest FAILED BUILD SUCCESSFUL H:\Antbook\ch04> ant -f mybuild.xml test-brief <junit> TEST FAILED, but BUILD SUCCESSFUL (?!) It would be more natural to see here also BUILD FAILED... But let us wait for a more advanced build with <junit>. Nothing above about org.example.antbook.junit.SimpleTest because it does not fail. TRY to confirm this by appropriate experiment.

The JUnit task <junit> (cont The JUnit task <junit> (cont.) Test failure enforcing build failure There are two issues to note about these results: no details were provided about which testXXX() method in FilePersistenceServicesTest failed and why, not so bad – no cluttering messages, But, on the other hand, insufficiently informative the build completed successfully (?!) despite the test failure – looks not natural. However, ADD to the task <junit> two attributes: <junit haltonfailure="false" printsummary="true"> We now get more informative and coherent output: We still are in the dark about what caused the failure. test-brief: [junit] Running org.example.antbook.junit.SimpleTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0 sec [junit] Running org.eclipseguide.persistence.FilePersistenceServicesTest [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.01 sec BUILD FAILED C:\Antbook\ch04\mybuild.xml:113: Test org.eclipseguide.persistence.HtmlDocu mentTest failed OLD OUTPUT: [junit] Running org.eclipseguide.persistence. FilePersistenceServicesTest [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.01 sec (a default value)

The JUnit task <junit> (cont The JUnit task <junit> (cont.) Test failures enforcing build failures test-brief: [junit] Running org.eclipseguide.persistence.FilePersistenceServicesTest [junit] Tests run: 5, Failures: 2, Errors: 0, Time elapsed: 0.031 sec [junit] Test org.eclipseguide.persistence.FilePersistenceServicesTest FAILED [junit] Running org.example.antbook.junit.SimpleTest [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.016 sec BUILD SUCCESSFUL Total time: 1 second Now it is more informative: printsummary="true" prints one-line statistics for each test case. Taking haltonfailure="true" gives results only up to the first failure: test-brief: [junit] Running org.eclipseguide.persistence.FilePersistenceServicesTest [junit] Tests run: 5, Failures: 2, Errors: 0, Time elapsed: 0.031 sec BUILD FAILED C:\Antbook\ch04\mybuild.xml:140: Test org.eclipseguide.persistence.FilePersistenceServicesTest failed We still are in the dark about what caused the failure. test-brief: [junit] Running org.example.antbook.junit.SimpleTest [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0 sec [junit] Running org.eclipseguide.persistence.FilePersistenceServicesTest [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.01 sec BUILD FAILED C:\Antbook\ch04\mybuild.xml:113: Test org.eclipseguide.persistence.HtmlDocu mentTest failed Now BUILD FAILED because of first failing test case(!!)

Recall and extend the directory structure in ch04: Structure directories to accommodate testing See also Part 11, slide 13 Recall and extend the directory structure in ch04: C:\Antbook\ch04 - base directory (basedir= ".") C:\Antbook\ch04\src - source directory (${src.dir}) C:\Antbook\ch04\test - test directory (${src.test.dir}) containing (deeper) JUnit test classes C:\Antbook\ch04\build - build directory (${build.dir}) C:\Antbook\ch04\build\classes - for compiled source files (${build.classes.dir}) C:\Antbook\ch04\build\test - for compiled JUnit test cases (${build.test.dir} ) C:\Antbook\ch04\build\data - for test repots in XML format C:\Antbook\ch04\build\report - for test repots in HTML format (new directories data and report to be considered later) Inspect regularly all the content of these directories in the Lab and relate all changes to the contents of mybuild.xml. First 3 red coloured directories and their contents are created by yourself. Other directories should be created automatically by your Ant build file.

Four main steps to do in Ant build file related to JUnit testing Adding testing into our build process is straightforward: Add a few targets or tasks: target test-init to initialize (<mkdir>) the testing directory structure (except directories created by yourself manually; these are last three directories on the previous slide), target test-compile to compile the test code with <javac>, target test (or the like) to execute the tests with <junit> (or <java>), <junitreport> and <report> tasks to generate the test reports (to be considered later). Note that besides testing we need as usually also steps and targets dealing with source code in src not mentioned above. Draw it yourself: the target dependency graph of the build file mybuild.xml which is described in these lectures step-by-step. 4.5.2

Ant build process related to JUnit testing Some more details Before above four steps, we should start with assigning properties to various directories used by our (compile and) test targets: <property name="src.dir" value="src"/> <property name="src.test.dir" value="test"/> <property name="build.dir" location="build"/> More properties... C:\Antbook\ch04\src C:\Antbook\ch04\test C:\Antbook\ch04\build Benefits of presentation of directories by properties: If you want to change your system of directories, this can be done only in one place of build file.

Ant build process related to JUnit testing Some more details Benefits of presentation of directories by properties: Also, we can easily control e.g. where Ant places test reports by overriding test.reports.dir in command line as -Dtest.reports.dir=C:\REPORTS to temporary change the standard (default) behaviour described in our Ant build file. Thus, in mybuild.xml you should present all directories by properties (except those directories related to package declarations which you create manually for your source files and test cases and need not mention in mybuild.xml)! so that we could place them, say, in a directory served by a web server <property name="junit.jar" location="${junit.dir}/${junit.subdir}/junit.jar"/> <property name="junit.dir" location="${ant.home}/lib"/> <property name="ant.home" value="${env.ANT_HOME}" /> <property name="junit.subdir" value=""/> junit.jar = D:\ant-161\lib\junit.jar

Ant build process related to JUnit testing Some more details For compiling and running source files and tests cases we need classpath which has been already introduced into mybuild.xml in 11. Ant, Testing and JUnit , Slides 14 and 29 (and repeated now): <path id="test.classpath"> <pathelement location="${build.test.dir}"/> <!-- build/test --> <pathelement location="C:\JAVA\junit4.8.2\junit-4.8.2.jar"/> <pathelement location="${build.classes.dir}"/> <!-- build/classes --> </path> CHECK all targets, tasks, properties and paths in mybuild.xml required for running test-brief target (and for other future test targets).

Ant build process related to JUnit testing Some more details In mybuild.xml, testing can possibly be preceded by a <copy> task This brings over all non-.java resources: For example, assume that an html file is required as an argument of a Java program which should do something with this file (read it, parse it, search some information in it, etc). Then we need an example of such file test.html somewhere under C:\Antbook\ch04\test , and so test.html, should be copied into the ${build.test.dir}, to allow our compiled JUnit test cases to access test data file – test.html – easily . <copy todir="${build.test.dir}"> <fileset dir="${src.test.dir}" excludes="**/*.java,**/*.class"/> </copy> C:\Antbook\ch04\build\test C:\Antbook\ch04\test Because of dependency checking, the <copy> task does not impact incremental build times until those files change