Chapter 8 Testing and Debugging Goals To learn how to carry out unit tests To understand the principles of test case selection and evaluation To learn how to use logging and assertions To become familiar with the debugger To learn strategies for effective debugging

Unit Test  Test classes in isolation, outside the program in which they are used  Test one class at a time  Supply test inputs through test harness  Test harness = program that feeds test inputs to a class

Root Approximator Example program to illustrate testing: square root approximator Algorithm known to the ancient Greeks (Heron) Task: to compute the square root of a Given: a guess x (ok to start with 1) Actual square root lies between x and a/x

Take midpoint (x + a/x) / 2 as a better guess Method converges rapidly. Square root of 100: Guess #1: 50.5 Guess #2: Guess #3: Guess #4: Guess #5: Guess #6: Guess #7: Guess #8: 10.0

File RootApproximator.java 1 /** 2 Computes approximations to the square root of 3 a number, using Heron's algorithm 4 */ 5 public class RootApproximator 6 { 7 /** 8 Constructs a root approximator for a given number aNumber the number from which to extract the square root 10 (Precondition: aNumber >= 0) 11 */ 12 public RootApproximator(double aNumber) 13 { 14 a = aNumber; 15 xold = 1; 16 xnew = a; 17 } 19 /** 20 Compute a better guess from the current guess. the next guess 22 */ 23 public double nextGuess() 24 { 25 xold = xnew; 26 if (xold != 0) 27 xnew = (xold + a / xold) / 2; 28 return xnew; 29 }

31 /** 32 Compute the root by repeatedly improving the current guess until two successive guesses are approximately equal. the computed value for the square root 35 */ 36 public double getRoot() 37 { 38 while (!Numeric.approxEqual(xnew, xold)) 39 nextGuess(); 40 return xnew; 41 } 42 private double a; // the number whose square root is computed 44 private double xnew; // the current guess 45 private double xold; // the old guess 46}

File RootApproximatorTest.java 1 import javax.swing.JOptionPane; 2 3 /** 4 This program prints ten approximations for a square root. 5 */ 6 public class RootApproximatorTest 7 { 8 public static void main(String[] args) 9 { 10 String input = 11 JOptionPane.showInputDialog("Enter a number"); 12double x = Double.parseDouble(input); 13 RootApproximator r = new RootApproximator(x); 14 final int MAX_TRIES = 10; 15 for (int tries = 1; tries <= MAX_TRIES; tries++) 16 { 17 double y = r.nextGuess(); 18 System.out.println("Guess #" + tries + ": " + y); 19 } 20 System.exit(0); 21 } 22}

Unit Test with BlueJ

File RootApproximatorTest2.java 1 import javax.swing.JOptionPane; 2 3 /** 4 This program computes square roots of user-supplied inputs. 5 */ 6 public class RootApproximatorTest2 7 { 8 public static void main(String[] args) 9 { 10 boolean done = false; 11 while (!done) 12 { 13String input = JOptionPane.showInputDialog( 14 "Enter a number, Cancel to quit"); if (input == null) 17 done = true; 18 else 19 { 20double x = Double.parseDouble(input); 21 RootApproximator r = new RootApproximator(x); 22 double y = r.getRoot(); System.out.println("square root of " + x + " = " + y); 26 } 27 } 28 System.exit(0); 29 } 30}

Read Inputs from File  Prepare a file with test inputs, such as –  Use input redirection –java RootApproximatorTest3 < test.in  Output –square root of = 10.0 square root of 20.0 =

File RootApproximatorTest3.java 1 import java.io.BufferedReader; 2 import java.io.InputStreamReader; 3 import java.io.IOException; 4 5 /** 6 This program computes square roots of inputs supplied through System.in 8 */ 9 public class RootApproximatorTest3 10{ 11 public static void main(String[] args) 12 throws IOException 13 { 14 BufferedReader console 15= new BufferedReader(new InputStreamReader(System.in)); 16 boolean done = false; 17 while (!done) 18 { 19 String input = console.readLine(); 20 if (input == null) done = true; 21 else 22 { 23 double x = Double.parseDouble(input); RootApproximator r = new RootApproximator(x); 25 double y = r.getRoot(); 26 System.out.println("square root 27 of " + x+ " = " + y); 29 } 30 } 31 } 32}

Sources of Test Data Provided by humans RootApproximatorTest3 Computer-generated sequence RootApproximatorTest4 Random sequence RootApproximatorTest5

RootApproximatorTest4 1 /** Computes square roots of input 3 values supplied by a loop. 4 */ 5 public class RootApproximatorTest4 6 { 7 public static void main(String[] args) 8 { 9 final double MIN = 1; 10 final double MAX = 10; 11 final double INCREMENT = 0.5; 12 for (double x = MIN; x <= MAX; x = x + INCREMENT) 13 { 14RootApproximator r = new RootApproximator(x); 15 double y = r.getRoot(); 16 System.out.println("square root of " 17 + x + " = " + y); 18 } }} 1 import java.util.Random; 3 /** 4 Computes square roots of random inputs. 5 */ 6 public class RootApproximatorTest5 7 { 8 public static void main(String[] args) 9 { 10 final double SAMPLES = 100; 11 Random generator = new Random(); 12 for (int i = 1; i <= SAMPLES; i++) 13 { // generate random test value double x = 1.0E6 * generator.nextDouble(); 16 RootApproximator r = new RootApproximator(x); 17 double y = r.getRoot(); 18 System.out.println("square root of " 19 + x + " = " + y); 20 } } } RootApproximatorTest5

Test Cases  Positive test case: expect positive outcome E.g square root of 100  Negative test case: expect negative outcome E.g square root of 100  Boundary test case: at boundary of domain Frequent cause for errors E.g square root of 0  Manual RootApproximatorTest3  Check property of result E.g. square root squared = original value RootApproximatorTest6  Oracle = slower way of computing answer E.g. square root of x = x 1/2 RootApproximatorTest7 Evaluation

File RootApproximatorTest6.java 1 import java.util.Random; 3 /** 4 This program verifies the computation 5 of square root values by checking a mathematical property of square roots. 6 */ 7 public class RootApproximatorTest6 8 { 9 public static void main(String[] args) 10 { 11 final double SAMPLES = 100; 12 int passcount = 0; 13 int failcount = 0; 14 Random generator = new Random(); 15 for (int i = 1; i <= SAMPLES; i++) 16 { double x = 1.0E6 * generator.nextDouble(); 20 RootApproximator r = new RootApproximator(x); 21 double y = r.getRoot(); 22 System.out.println("square root of " 23 + x + " = " + y); 25 if (Numeric.approxEqual(y * y, x)) 28 { 29 System.out.println("Test passed."); 30 passcount++; 31 } 32 else { 34 System.out.println("Test failed."); 35 failcount++; 36 } 37 } 38 System.out.println("Pass: " + passcount); 39 System.out.println("Fail: " + failcount); 40 } 41}

File RootApproximatorTest7.java 1 import java.util.Random; 3 /** 4 Verifies the computation of square 5 root values by using an oracle. 6 */ 7 public class RootApproximatorTest7 8 { 9 public static void main(String[] args) 10 { 11 final double SAMPLES = 100; 12 int passcount = 0; 13 int failcount = 0; 14 Random generator = new Random(); 15 for (int i = 1; i <= SAMPLES; i++) 16 { 19 double x = 1.0E6 * generator.nextDouble(); 20 RootApproximator r = new RootApproximator(x); 21 double y = r.getRoot(); 22 System.out.println("square root of " 23 + x + " = " + y); 25 double oracleValue=Math.pow(x,0.5); 26 if (Numeric.approxEqual(y, oracleValue)) 30 { 31 System.out.println("Test passed."); 32 passcount++; 33 } 34 else { 36 System.out.println("Test failed."); 37 failcount++; 38 } 39 } 40 System.out.println("Pass: " + passcount); 41 System.out.println("Fail: " + failcount); 42 } 43}

Regression Testing Save test cases Automate testing java Program test1.out java Program test2.out java Program test3.out Repeat test whenever progam changes Test suite = collection of test cases Cycling = bug that is fixed but reappears in later versions Regression testing = testing against past failures

Test Coverage Black-box testing: test functionality without understanding internal structure White-box testing: take internal structure into account when designing tests Test coverage: the code that is actually executed during test cases Easy to overlook error branches during testing Make sure to execute each branch in at least one test case

Program Trace  Output statements in your program for diagnostic purposes if (status == SINGLE) { System.out.println("status is SINGLE");... }...  Stack trace tells you the contents of the call stack Throwable t = new Throwable(); t.printStackTrace(System.out); Looks like exception report: –java.lang.Throwable at TaxReturn.getTax(TaxReturn.java:26) at TaxReturnTest.main(TaxReturnTest.java:30)  Drawback of trace messages: Need to remove them when testing is complete, stick them back in when another error is found

Logging  Get global logger object: Logger logger = Logger.getLogger("global");  Log a message logger.info("status is SINGLE");  By default, logged messages are printed. Turn them off with logger.setLevel(Level.OFF); Assertions  Assertion = assumption that you believe to be true –assert y >= 0; root = Math.sqrt(y);  If assertion fails, program is terminated  Can be used to assert pre/postconditions  Must compile/run with special flags –javac -source 1.4 MyProg.java java -enableassertions MyProg

The Debugger  Debugger = program to run your program, interrupt it, and inspect variables  Three key commands:  Set Breakpoint  Single Step  Inspect Variable  Two variations of Single Step  Step Over = don't enter method call  Step Inside = enter method call

The Debugger Stopping at a Breakpoint

Inspecting Variables

Sample Debugging Session  Word class counts syllables in a word  Each group of adjacent vowels (aeiouy) is a syllable  However, an e at the end of a word doesn't count  If algorithm gives count of 0, increment to 1  Constructor removes non-letters at beginning and end  Buggy output: Syllables in hello: 1 Syllables in regal: 1 Syllables in real: 1

File Word.java 1 public class Word 2 { 3 /** 4 Constructs a word by removing leading and trailing non- letter characters, such as punctuation marks. s the input string 7 */ 8 public Word(String s) 9 { 10 int i = 0; 11 while (i < s.length() && 12 !Character.isLetter(s.charAt(i))) 13 i++; 14 int j = s.length() - 1; 15 while (j > i && !Character.isLetter(s.charAt(j))) 16 j--; 17 text = s.substring(i, j + 1); 18 } 19 /** 20 Returns the text of the word, after removal of the 21 leading and trailing non- letter characters. the text of the word 23 */ 24 public String getText() 25 { 26 return text; 27 } 29 /** 30 Counts the syllables in the word. the syllable count 32 */

33 public int countSyllables() 34 { 35 int count = 0; 36 int end = text.length() - 1; 37 if (end < 0) return 0; 38 char ch = Character.toLowerCase (text.charAt(end)); 41 if (ch == 'e') end--; 43 boolean insideVowelGroup 44 = false; 44 for (int i = 0; i <= end; i++) 45 { 46 ch= Character.toLowerCase (text.charAt(i)); 47 if ("aeiouy".indexOf(ch) >= 0) 48 { 49 // ch is a vowel 50 if (!insideVowelGroup) 51 { 52 // start of new vowel group 53 count++; 54 insideVowelGroup = true; 55 } 56 } 57 else 58 insideVowelGroup = false; 59 } 61// every word has at least one syllable 62 if (count == 0) 63 count = 1; 65 return count; 66 } 68 private String text; 69}

File WordTest.java 1 import java.util.StringTokenizer; 2 import javax.swing.JOptionPane; 4 public class WordTest 5 { 6public static void main(String[] args) 7 { 8 String input = 9 JOptionPane.showInputDialog ("Enter a sentence"); 10 StringTokenizer tokenizer = new StringTokenizer(input); 11 while (tokenizer.hasMoreTokens()) 12 { 13 String token = tokenizer.nextToken(); 14Word w = new Word(token); 15int syllables = w.countSyllables(); 16System.out.println( "Syllables in " + w.getText() + ": " + syllables); 18 } 19 System.exit(0); 20 } 21}

Final Letter Test is Not Correct  Set breakpoint in line 35 (first line of countSyllables)  Start program, supply input hello  Method checks if final letter is 'e'  Run to line 41  Inspect variable ch  Should contain final letter but contains 'l'

Debugging the countSyllables Method

The Current Values of the Local and Instance Variables

More Problems Found  end is set to 3, not 4  text contains "hell", not "hello"  No wonder countSyllables returns 1  Culprit is elsewhere  Can't go back in time  Restart and set breakpoint in Word constructor

Debugging the Word Constructor  Supply "hello" input again  Break past the end of second loop in constructor  Inspect i and j  They are 0 and 4--makes sense since the input consists of letters  Why is text set to "hell"?  Off-by-one error: Second parameter of substring is the first position not to include  text = substring(i, j); should be text = substring(i, j + 1);

Debugging the Word Constructor

Another Error  Fix the error  Recompile  Test again: –Syllables in hello: 2 Syllables in regal: 1 Syllables in real: 1  Oh no, it's still not right  Start debugger  Erase all old breakpoints  Supply input "regal"

Debugging countSyllables (again)  Break in the beginning of countSyllables  Single-step through loop  First iteration ('r'): skips test for vowel  Second iteration ('e'): passes test, increments count  Third iteration ('g'): skips test  Fourth iteration ('a'): passes test, but doesn't increment count  insideVowelGroup was never reset to false  Fix and retest: All test cases pass  Is the program now bug-free? The debugger can't answer that.

The First Bug

Therac-25 Facility