1. Unit, Regression, and Behavioral Testing Based On: Unit Testing with JUnit and CUnit by Beth Kirby beth@apl.washington.edu206-685-0310 Dec 13, 2002 Jules White jules@dre.vanderbilt.edu

2. One minute summary ‘Code that isn’t tested doesn’t work’ Why test? Why test? Code that isn’t regression tested suffers from code rot (breaks eventually) Why regression test? Why regression test? A unit testing framework is unit & regression testing on steroids

3. What is unit testing? Testing is often divided into categories such as: Unit testing Unit testing Testing an isolatable ‘unit’ of code, usually a class e.g. AStack Integration testing Integration testing Testing a module of code (e.g. a package) Testing AStack + Analyzer, LStack + Analyzer Application testing Application testing Testing the code as the user would see it (black box) Testing Delimiters

4. Writing a unit test - example public void testAnalyzer() { Analyzer analyzer = new Analyzer (….); analyzer.process (“{“); analyzer.process (“}”); //checks to see if the delimiters matched assertTrue (analyzer.match ()); }

5. What is a testing framework? A test framework provides reusable test functionality which: Is easier to use (e.g. don’t have to write the same code for each class) Is easier to use (e.g. don’t have to write the same code for each class) Is standardized and reusable Is standardized and reusable Provides a base for regression tests Provides a base for regression tests

6. Why formalize unit testing? Unit testing has often been done, but in an ad hoc manner E.g. adding a main method to a class, which runs tests on the class E.g. adding a main method to a class, which runs tests on the classAxiom: Code that isn’t tested doesn’t work Code that isn’t tested doesn’t work ‘If code has no automated test case written for it to prove that it works, it must be assumed not to work.’ (Hightower and Lesiecki) ‘If code has no automated test case written for it to prove that it works, it must be assumed not to work.’ (Hightower and Lesiecki)

7. Why use a testing framework? Each class must be tested when it is developed Each class needs a regression test Regression tests need to have standard interfaces Thus, we can build the regression test when building the class and have a better, more stable product for less work

8. Regression testing New code and changes to old code can affect the rest of the code base ‘Affect’ sometimes means ‘break’ ‘Affect’ sometimes means ‘break’ I need to run tests on the old code, to verify it works – these are regression tests Regression testing is required for a stable, maintainable code base

9. Regression testing and refactoring ‘Refactoring is a technique to restructure code in a disciplined way.’ (Martin Fowler) Refactoring is an excellent way to break code. Regression testing allows developers to refactor safely – if the refactored code passes the test suite, it works

10. Running automated tests The real power of regression tests happens when they are automated This requires they report pass/fail results in a standardized way This requires they report pass/fail results in a standardized way Can set up jobs to Clean & check out latest build tree Clean & check out latest build tree Run tests Run tests Put results on a web page & send mail (if tests fail) Put results on a web page & send mail (if tests fail) JUnit & ant have code to do all of this

11. Some background eXtreme programming (XP) is a ‘programming methodology that stresses (among other things) testing and refactoring The Apache community provides open- source software, including the Jakarta project (server side java tools, e.g. tomcat (servlet engine)) Ant is a build tool (like make) Part of the Jakarta project Part of the Jakarta project Is becoming the de facto standard for java projects Is becoming the de facto standard for java projects Is written in java, uses xml to specify build targets Is written in java, uses xml to specify build targets

12. Stateful Testing What is Stateful Testing? Example: public void testAnalyzer() { Analyzer analyzer = new Analyzer(…..); analyzer.process (“{“); assertEqual (“{”, analyzer.getStack().topOfStack()); analyzer.process (“}”); assertEqual( 0, analyzer.getStack().size() ); assertEqual( 0, analyzer.getStack().size() );}

13. What is Wrong with Stateful Testing? What happens if Analyzer is refactored to use a data structure other than a stack? Example: public void testAnalyzer() { Analyzer analyzer = new Analyzer(…..); analyzer.process (“{“); assertEqual (“{”, analyzer.getStack().topOfStack()); analyzer.process (“}”); assertEqual( 0, analyzer.getStack().size() ); assertEqual( 0, analyzer.getStack().size() );}

14. What is Wrong with Stateful Testing? Stateful testing does not allow the interface and implementation to vary. Stateful testing tightly couples the testing code to the implementation so that refactoring breaks the testing code Stateful testing also prevents testing code to be reused as new classes are added that implement the same interface e.g. a test targeted specifically for AStack cannot be used by LStack e.g. a test targeted specifically for AStack cannot be used by LStack}

15. Behavioral Testing Tests should check the expected behavior (input / output) Tests should be coded to the specification not the implementation Example: public void testAnalyzer() { Analyzer analyzer = new Analyzer(……); analyzer.process (“{“); analyzer.process (“}”); Boolean expected = new Boolean(true); expected.equals( analyzer.match () ); expected.equals( analyzer.match () );}

16. Mock Objects How do you isolate an object for unit testing when it relies on other objects? Example: public void testAnalyzer() { Analyzer analyzer = new Analyzer(……); analyzer.process (“{“); analyzer.process(“}”); assertEqual( true, analyzer.match ()); // A stack is used here assertEqual( true, analyzer.match ()); // A stack is used here}

17. Mock Objects Mock Objects are used to stand in for the real objects that the class relies on e.g Stack is replaced by a MockStack e.g Stack is replaced by a MockStack Mock Objects check to ensure that the dependent object calls the correct methods in the correct order They can guarantee the input and output to the dependent object and isolate the source of errors

18. Mock Objects public void testAnalyzer() { Analyzer analyzer = new Analyzer(new MockObjectFactory()); …..} MockObjectFactory { …. Stack make_stack () { return new MockStackObject ();} }

19. Mock Objects MockStackObject { …. boolean popCalled = false; boolean pushCalled = false; boolean pushCalledFirst = false; public void push (String str) { pushCalled = true; pushCalled = true; pushCalledFirst = !popCalled; pushCalledFirst = !popCalled;} public String pop() { popCalled = true; popCalled = true; return “{“; return “{“;}

20. Mock Objects Example: public void testAnalyzer() { MockObjectFactory fact = new MockObjectFactory() MockObjectFactory fact = new MockObjectFactory() Analyzer analyzer = new Analyzer(fact); analyzer.process (“{“); analyzer.process (“}”); assertTrue( fact.mockStack.pushCalledFirst()); assertTrue( fact.mockStack.popCalled ()); assertTrue( true, analyzer.match ()); // A MockStack is used here assertTrue( true, analyzer.match ()); // A MockStack is used here}

21. What is Junit? JUnit is a regression testing framework written by Erich Gamma and Kent Beck It is found at www.junit.org www.junit.org It consists of classes that the developer can extend to write a test – notably junit.framework.TestCase - and related structure to run and report the tests

22. Tool integration Java IDEs Eclipse – http://www.eclipse.org Eclipse – http://www.eclipse.orghttp://www.eclipse.org Extensive support for JUnit Netbeans - http://www.netbeans.org/index.html Netbeans - http://www.netbeans.org/index.html http://www.netbeans.org/index.html Idea - http://www.intellij.com/idea/ Idea - http://www.intellij.com/idea/http://www.intellij.com/idea/

23. Writing a test Directions can be found in the Junit cookbook - http://junit.sourceforge.net/doc/cookbook/cookbook.ht m http://junit.sourceforge.net/doc/cookbook/cookbook.ht m http://junit.sourceforge.net/doc/cookbook/cookbook.ht m Soundbite summary 1. Create an instance of TestCase: TestCase 2. Write methods which run your tests, calling them test 2. Write methods which run your tests, calling them test 3. Call a test runner with your test 4. When you want to check a value, call an assert method (e.g. assertTrue()) and pass a condition that is true if the test succeeds

24. Writing a test - example public void testAnalyzer() { Analyzer analyzer = new Analyzer(……); analyzer.process (“{“); analyzer.process (“}”); Boolean expected = new Boolean(true); expected.equals( analyzer.match () ); expected.equals( analyzer.match () );}

25. Specifying tests JUnit The framework will look in every class given to it which implements TestCase The framework will look in every class given to it which implements TestCase It uses reflection to find all methods that start with ‘test’ (e.g. testMoneyMoneyBag) It uses reflection to find all methods that start with ‘test’ (e.g. testMoneyMoneyBag) It runs each such method as a test case It runs each such method as a test case

26. Test suites In practice, you will want to run a group of related tests (e.g. all the tests for a class) To do so, group your test methods in a class which extends TestCase Override the constructor and (if desired) the setUp and tearDown method

27. Test Suite example Package test.mypackage; import junit.framework.TestCase; Import mypackage; Public class MyClassTest extends TestCase { public MyClassTest(String name) { public MyClassTest(String name) { super(name); super(name); public void setUp throws Exception { public void setUp throws Exception { // set up for each test // set up for each test } public void tearDown throws Exception { public void tearDown throws Exception { // releases resources // releases resources } public void testMyMethod throws Exception { public void testMyMethod throws Exception { // Run tests // Run tests }}

28. Test Suite methods setUp – does initialization common to all tests It is run each time a test is run (can’t store state in variables that are set in setUp) It is run each time a test is run (can’t store state in variables that are set in setUp) tearDown – releases resources, e.g. database connections. Can also have other helper methods, just don’t name them test Can also have other helper methods, just don’t name them test

29. Running tests Need to call a runner run method, e.g junit.textui.TestRunner.run(suite()); junit.textui.TestRunner.run(suite()); Example main class for running tests // imports public class ATestSuite { public static TestSuite suite () { TestSuite suite = new TestSuite(“MyPackageTests”); suite.addTestSuite(“MyClassTests.class”); return suite; } public static void main(String args[]) { } public static void main(String args[]) { junit.textui.TestRunner.run(suite()); junit.textui.TestRunner.run(suite()); }}

30. Assert methods There are a variety of assert methods available, e.g. AssertTrue, assertFalse, AssertNull, assertSame, etc AssertTrue, assertFalse, AssertNull, assertSame, etc In general, the syntax is assert ( expected, testVal) assert ( expected, testVal) Assert (String message, expected, testVal) Assert (String message, expected, testVal)

31. Project structure One way to organize tests is to put them in a parallel directory tree E.g. MyPackage. …. Has a parallel tree of tests.MyPackage. …. E.g. MyPackage. …. Has a parallel tree of tests.MyPackage. …. Test suites can be collected, so each class has a test class each class has a test class each directory has test suite with a runner, which runs the tests in that directory and the suites in all subdirectories each directory has test suite with a runner, which runs the tests in that directory and the suites in all subdirectories

32. Junit addons Junit has a bunch of add on stuff that has been written for it (your mileage may vary) Examples Code generators – make guesses at what tests are needed Code generators – make guesses at what tests are needed Load testers Load testers Code coverage Code coverage Testers for thread safety Testers for thread safety ‘Helpful classes’ (e.g. RecursiveTestSuite is a TestSuite that will recursively walk through the directory structure looking for subclasses of TestCase and will add them.) ‘Helpful classes’ (e.g. RecursiveTestSuite is a TestSuite that will recursively walk through the directory structure looking for subclasses of TestCase and will add them.)RecursiveTestSuite

33. Cactus, HttpUnit, Mock Objects Cactus (from jakarta) is a simple test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters,...). HttpUnit emulates the relevant portions of browser behavior to allow automated testing (on sourceforge) Mock objects emulate objects from other parts of the application, but have controlled behaviour

34. Resources Videos http://video.google.com/videosearch?q=techtalks&pa ge=1&lv=0&so=1 http://video.google.com/videosearch?q=techtalks&pa ge=1&lv=0&so=1 Web sites Junit - http://www.junit.org Junit - http://www.junit.orghttp://www.junit.org CppUnit - http://cppunit.sourceforge.net/ CppUnit - http://cppunit.sourceforge.net/http://cppunit.sourceforge.net/ Ant - http://jakarta.apache.org/ant/index.html Ant - http://jakarta.apache.org/ant/index.htmlhttp://jakarta.apache.org/ant/index.html Book Java Tools for Extreme Programming: Mastering Open Source Tools Including Ant, JUnit, and Cactus Java Tools for Extreme Programming: Mastering Open Source Tools Including Ant, JUnit, and Cactus