Unit Testing CSSE 514 Programming Methods 4/19/01

Overview Code that's Easy to Test Unit Testing Testing against Contract Writing Unit Tests Using Test Harnesses JUnit Primer Reference: Andrew Hunt, David Thomas, The Pragmatic Programmer, Addison Wesley, 2000 Reference: Mike Clark, JUnit Primer,

Code that's Easy to Test The "Software IC" metaphor Software should be tested more like hardware, with: –Built-in self testing –Internal diagnostics –Test harness Need to build in testability from the very beginning Need to test each piece thoroughly before trying to wire them together

Unit Testing Roughly equivalent to chip-level testing for hardware Its testing done to each module, in isolation, to verify its behavior Typically the unit test will establish some sort of artificial environment and then invoke routines in the module being tested It then checks the results returned against either some known value or against the results from previous runs of the same test (regression testing) When the modules are assembled we can use the same tests to test the system as a whole

Testing against Contract When we write unit tests we want to write test cases that ensure a given unit honors its contract –This will tell us whether the code meets the contract and whether the contract means what we think it means iContract for square root routine /* argument >= 0 abs((result*result)-argument)<epsilon */

Testing against Contract The above contract tells us what to test: –Pass in a negative argument and ensure that it is rejects –Pass in an argument of zero to ensure that it is accepted (this is a boundary value) –Pass in values between zero and the maximum expressible argument and verify that the difference between the square of the result and the original argument is less than some value epsilon

Testing against Contract When you design a module or even a single routine, you should design both its contract and the code to test that contract By designing code to pass a test and fulfill its contract, you might consider boundary conditions and other issues that you wouldn't consider otherwise The best way to fix errors is to avoid them in the first place By building the tests before you implement the code you get to try out the interface before you commit to it

Writing Unit Tests Unit test should be conveniently located –For small projects you can imbed the unit test for a module in the module itself –For larger projects you should keep the tests in the package directory or a /test subdirectory of the package By making the code accessible to developers you provide them with: –Examples of how to use all the functionality of your module –A means to build regression tests to validate any future changes to the code In Java, you can use the main routine to run your unit tests

Using Test Harnesses A test harness can handle common operations such as –Logging status –Analyzing output for expected results –Selecting and running the tests Harnesses can be: –GUI driven –Written in the same language as the rest of the project –May be implemented as a combination of make files and scripts

Using Test Harnesses A test harness should include the following capabilities: –A standard way to specify setup and cleanup –A method for selecting individual tests or all available tests –A means of analyzing output for expected (or unexpected) results –A standardized form of failure reporting Tests should be composable: that is, a test can be composed of subtests of subcomponents to any depth

JUnit Primer This short primer demonstrates how to write and run simple test cases and test suites using the JUnit testing framework

Why Use JUnit JUnit allows you to write code faster while increasing quality JUnit is elegantly simple JUnit tests check their own results and provide immediate feedback JUnit tests can be composed into a hierarchy of test suites Writing JUnit tests is inexpensive JUnit tests increase the stability of software JUnit tests are developer tests JUnit tests are written in Java JUnit is free

Design of JUnit JUnit is designed around two key design patterns: the Command pattern and the Composite pattern A TestCase is a command object –Any class that contains test methods should subclass the TestCase class A TestSuite is a composite of other tests, either TestCase instances or other TestSuite instances

Step 1: Write a Test Case To write a test case, follow these steps: 1.Define a subclass of TestCase. 2.Override the setUp() method to initialize object(s) under test. 3.Override the tearDown() method to release object(s) under test. 4.Define one or more testXXX() methods that exercise the object(s) under test. 5.Define a suite() factory method that creates a TestSuite containing all the testXXX() methods of the TestCase. 6.Define a main() method that runs the TestCase.

import junit.framework.Test; import junit.framework.TestCase; import junit.framework.TestSuite; public class ShoppingCartTest extends TestCase { private ShoppingCart _bookCart; /** * Constructs a ShoppingCartTest with the specified name. * name Test case name. */ public ShoppingCartTest(String name) { super(name); } /** * Sets up the text fixture. * * Called before every test case method. */ protected void setUp() { _bookCart = new ShoppingCart(); Product book = new Product("Extreme Programming", 23.95); _bookCart.addItem(book); } /** * Tears down the text fixture. * * Called after every test case method. */ protected void tearDown() { _bookCart = null; }

/** * Tests the emptying of the cart. */ public void testEmpty() { _bookCart.empty(); assert(_bookCart.isEmpty()); } /** * Tests adding a product to the cart. */ public void testProductAdd() { Product book = new Product("Refactoring", 53.95); _bookCart.addItem(book); double expectedBalance = book.getPrice(); double currentBalance = _bookCart.getBalance(); double tolerance = 0.0; assertEquals(expectedBalance, currentBalance, tolerance); int expectedItemCount = 2; int currentItemCount = _bookCart.getItemCount(); assertEquals(expectedItemCount, currentItemCount); }

/** * Tests removing a product from the cart. * ProductNotFoundException If the * product was not in the cart. */ public void testProductRemove() throws ProductNotFoundException { Product book = new Product("Extreme Programming", 23.95); _bookCart.removeItem(book); double expectedBalance = book.getPrice(); double currentBalance = _bookCart.getBalance(); double tolerance = 0.0; assertEquals(expectedBalance, currentBalance, tolerance); int expectedItemCount = 0; int currentItemCount = _bookCart.getItemCount(); assertEquals(expectedItemCount, currentItemCount); }

/** * Tests removing an unknown product from the cart. * * This test is successful if the * ProductNotFoundException is raised. */ public void testProductNotFound() { try { Product book = new Product("Ender's Game", 4.95); _bookCart.removeItem(book); fail("Should raise a ProductNotFoundException"); } catch(ProductNotFoundException pnfe) { // successful test }

/** * Assembles and returns a test suite for * all the test methods of this test case. * A non-null test suite. */ public static Test suite() { // // Reflection is used here to add all // the testXXX() methods to the suite. // TestSuite suite = new TestSuite(ShoppingCartTest.class); // // Alternatively, but prone to error when adding more // test case methods... // // TestSuite suite = new TestSuite(); // suite.addTest(new ShoppingCartTest("testEmpty")); // suite.addTest(new // ShoppingCartTest("testProductAdd")); // suite.addTest(new // ShoppingCartTest("testProductRemove")); // suite.addTest(new // ShoppingCartTest("testProductNotFound")); // return suite; }

/** * Runs the test case. * * Uncomment either the textual UI, Swing UI, or AWT UI. */ public static void main(String args[]) { String[] testCaseName = {ShoppingCartTest.class.getName()}; //junit.textui.TestRunner.main(testCaseName); //junit.swingui.TestRunner.main(testCaseName); junit.ui.TestRunner.main(testCaseName); }

Step 2: Write a Test Suite To write a test suite, follow these steps: 1.Define a subclass of TestCase. 2.Define a suite() factory method that creates a TestSuite containing all the TestCase instances and TestSuite instances contained in the TestSuite. 3.Define a main() method that runs the TestSuite.

public class EcommerceTestSuite extends TestCase { /** * Constructs a EcommerceTestSuite with the specified name. * name Test suite name. */ public EcommerceTestSuite(String name) { super(name); } /** * Assembles and returns a test suite * containing all known tests. * * New tests should be added here! * A non-null test suite. */ public static Test suite() { TestSuite suite = new TestSuite(); // // The ShoppingCartTest we created above. // suite.addTest(ShoppingCartTest.suite()); // // Another example test suite of tests. // suite.addTest(CreditCartTestSuite().suite()); return suite; } /** * Runs the test suite. * * Uncomment either the textual UI, Swing UI, or AWT UI. */ public static void main(String args[]) { String[] testCaseName = {EcommerceTestSuite.class.getName()}; //junit.textui.TestRunner.main(testCaseName); //junit.swingui.TestRunner.main(testCaseName); junit.ui.TestRunner.main(testCaseName); }

Step 3: Run the Tests Now that we've written a test suite containing a collection of test cases and other test suites, we can run either the test suite or any of its test cases individually Running a TestSuite will automatically run all of its subordinate TestCase instances and TestSuite instances. Running a TestCase will automatically invoke all of its defined testXXX() methods

Step 4: Organize the Tests 1.Create test cases in the same package as the code under test. For example, the com.mydotcom.ecommerce package would contain all the application-level classes as well as the test cases for those components. If you want to avoid combining application and testing code in your source directories, it's recommended to create a parallel, mirrored directory structure that contains the test code. 2.For each Java package in your application, define a TestSuite class that contains all the tests for verifying the code in the package. 3.Define similar TestSuite classes that create higher-level and lower-level test suites in the other packages (and sub-packages) of the application. 4.Make sure your build process includes the compilation of all test suites and test cases. This helps to ensure that your tests are always up- to-date with the latest code and keeps the tests fresh.

Testing Idioms Keep these things in mind when testing: Code a little, test a little, code a little, test a little... Run your tests as often as possible, at least as often as you run the compiler. Run all the tests in the system at least once per day (or night). Begin by writing tests for the areas of code that you're most worried about breaking. Write tests that have the highest possible return on your testing investment. When you need to add new functionality to the system, write the tests first. If you find yourself debugging using System.out.println(), write a test case instead. When a bug is reported, write a test case to expose the bug. The next time someone asks you for help debugging, help them write a test. Don't deliver software that doesn't pass all of its tests.