Comp1004: Building Better Classes I Testing and Debugging Partly based on BlueJ Book – Chapter 6

Software Errors Are A Problem Mars Climate Orbiter – $125 million dollar probe – Communications relay for future NASA Mars missions Launched successfully on Dec 11, 1998 and travelled 669 million km in 286 days And finally fired its engines, causing it to plunge into the atmosphere and never be seen again :-(

Software Errors Are A Problem The Problem? NASA built the engines to take navigational instructions in metric units (m/sec) Lockhead Martin provided them in imperial units (ft/sec) So the probe dropped too far, and its engines overheated in the atmosphere

Coming up we aim to save $125 million dollars Debugging – Syntax and Logical Errors Testing Strategies – Equivalence Classes and Boundary Values – Test Automation Debuggers

We have to deal with errors Early errors are usually syntax errors. – The compiler will spot these. Later errors are usually logic errors. – The compiler cannot help with these. – Also known as bugs. Some logical errors have no immediately obvious manifestation. – Commercial software is rarely error free.

Syntax Errors? ArrayList kennel = new ArrayList ; kennel.add(new Dog(“Rover”)); kennel.add(new Dog(“Fido”)); kennel.add(new Dog(“Patch”)); kennel.add(new Dog(“Eric”)); kennel.add(new Dog(“Binky”)); for(int i = 0; i <= kennel.size(); i == i + 1) { kennel.get(i).bark() }

Syntax Errors? ArrayList kennel = new ArrayList (); kennel.add(new Dog(“Rover”)); kennel.add(new Dog(“Fido”)); kennel.add(new Dog(“Patch”)); kennel.add(new Dog(“Eric”)); kennel.add(new Dog(“Binky”)); for(int i = 0; i <= kennel.size(); i == i + 1) { kennel.get(i).bark(); } Forgot to call the Constructor Forgot to terminate the statement with a semi- colon

Logical Errors? ArrayList kennel = new ArrayList (); kennel.add(new Dog(“Rover”)); kennel.add(new Dog(“Fido”)); kennel.add(new Dog(“Patch”)); kennel.add(new Dog(“Eric”)); kennel.add(new Dog(“Binky”)); for(int i = 0; i <= kennel.size(); i == i + 1) { kennel.get(i).bark(); } Forgot to call the Constructor Forgot to terminate the statement with a semi- colon

Logical Errors? ArrayList kennel = new ArrayList (); kennel.add(new Dog(“Rover”)); kennel.add(new Dog(“Fido”)); kennel.add(new Dog(“Patch”)); kennel.add(new Dog(“Eric”)); kennel.add(new Dog(“Binky”)); for(int i = 0; i < kennel.size(); i = i + 1) { kennel.get(i).bark(); } Forgot to call the Constructor Forgot to terminate the statement with a semi- colon Need to use < or the loop will overrun the arraylist Need to use assignment operator = rather than the equality operator ==

Prevention vs Detection (Developer vs Maintainer) We can lessen the likelihood of errors. – Use software engineering techniques, like encapsulation. We can improve the chances of detection. – Use software engineering practices, like modularization and documentation. We can develop detection skills.

Testing and debugging These are crucial skills. Testing searches for the presence of errors. Debugging searches for the source of errors. – The manifestation of an error may well occur some ‘distance’ from its source.

Method 1: Unit testing Each unit of an application may be tested. – Method, class, module (package in Java). Can (should) be done during development. – Finding and fixing early lowers development costs (e.g. programmer time). – A test suite is built up.

Testing fundamentals Understand what the unit should do – its contract. – You will be looking for violations. – Use positive tests and negative tests. Test boundaries. – Zero, One, Full. Search an empty collection. Add to a full collection.

Equivalence Classes and Boundary Values “A theme park is testing a new ticketing system. Adults aged 18 or over pay £20, children age 5 years or younger get in free, and other children pay £12. In all cases if the person is a member of the theme park fan club the cost is reduced by 50%.” What are the equivalence classes? (the sets of values that have the same result) Member Age 18+ Member Age 18+ £10

Equivalence Classes and Boundary Values “A theme park is testing a new ticketing system. Adults aged 18 or over pay £20, children age 5 years or younger get in free, and other children pay £12. In all cases if the person is a member of the theme park fan club the cost is reduced by 50%.” What are the equivalence classes? (the sets of values that have the same result) Member Age 18+ Member Age 18+ Non-Member Age 18+ Non-Member Age 18+ Member Age Member Age Non-Member Age Non-Member Age Member or Non-Member Age Member or Non-Member Age £10 £20 £6 £12 FREE

Equivalence Classes and Boundary Values “A theme park is testing a new ticketing system. Adults aged 18 or over pay £20, children age 5 years or younger get in free, and other children pay £12. In all cases if the person is a member of the theme park fan club the cost is reduced by 50%.” Member Age 18+ Member Age 18+ Non-Member Age 18+ Non-Member Age 18+ Member Age Member Age Non-Member Age Non-Member Age Member or Non-Member Age Member or Non-Member Age £10 £20 £6 £12 FREE What are the boundary values? (the points at which the classes change)

Age? 5/6 Equivalence Classes and Boundary Values “A theme park is testing a new ticketing system. Adults aged 18 or over pay £20, children age 5 years or younger get in free, and other children pay £12. In all cases if the person is a member of the theme park fan club the cost is reduced by 50%.” What are the boundary values? (the points at which the classes change) Member Age 18+ Member Age 18+ Non-Member Age 18+ Non-Member Age 18+ Member Age Member Age Non-Member Age Non-Member Age Member or Non-Member Age Member or Non-Member Age £10 £20 £6 £12 FREE Member? true/false Age? 17/18

Test automation Good testing is a creative process, but thorough testing is time consuming and repetitive. Regression testing involves re-running tests to see if new changes have caused old code to break Use of a test rig or test harness can relieve some of the burden. – Classes are written to perform the testing.

Age? 5/6 Test Harness What might our test harness look like? Member Age 18+ Member Age 18+ Non-Member Age 18+ Non-Member Age 18+ Member Age Member Age Non-Member Age Non-Member Age Member or Non-Member Age Member or Non-Member Age £10 £20 £6 £12 FREE Member? true/false Age? 17/18

Age? 5/6 Test Harness What might our test harness look like? Member Age 18+ Member Age 18+ Non-Member Age 18+ Non-Member Age 18+ Member Age Member Age Non-Member Age Non-Member Age Member or Non-Member Age Member or Non-Member Age £10 £20 £6 £12 FREE Member? true/false Age? 17/18 public testCosts() { //Test Harness System.out.println(“Starting Cost Tests”); int cost; cost = calcCost(18, true); if(cost == 10) System.out.println(“Test 1 Pass”); else System.out.println(“Test 1 Fail”); cost = calcCost(18, false); if(cost == 20) System.out.println(“Test 2 Pass”); else System.out.println(“Test 2 Fail”); cost = calcCost(17, true); if(cost == 6) System.out.println(“Test 1 Pass”); else System.out.println(“Test 1 Fail”); cost = calcCost(17, false); if(cost == 12) System.out.println(“Test 1 Pass”); else System.out.println(“Test 1 Fail”); //etc – until all boundaries are tested System.out.println(“Ending Cost Tests”); }

Method 2: Manual walkthroughs Relatively underused. – A low-tech approach. – More powerful than appreciated. Get away from the computer! ‘Run’ a program by hand. High-level (Step) or low-level (Step into) views.

Tabulating object state An object’s behavior is usually determined by its state. Incorrect behavior is often the result of incorrect state. Tabulate the values of all fields. Document state changes after each method call.

Method 3: Verbal walkthroughs Explain to someone else what the code is doing. – They might spot the error. – The process of explaining might help you to spot it for yourself. Group-based processes exist for conducting formal walkthroughs or inspections.

Method 4: Print statements The most popular technique. No special tools required. All programming languages support them. Only effective if the right methods are documented. Output may be voluminous! Turning off and on requires forethought. public int sumAllCosts() { int costs[] = getAllCosts(); int totalcost = 0; for(int n : costs){ totalcost += n; } return totalcost; }

Method 4: Print statements The most popular technique. No special tools required. All programming languages support them. Only effective if the right methods are documented. Output may be voluminous! Turning off and on requires forethought. public int sumAllCosts() { int costs[] = getAllCosts(); if(costs == null) System.out.println(“Costs array is null!”); else { System.out.println(“Costs array returned”); System.out.println(“Size:” + costs.size()); } int totalcost = 0; for(int n : costs){ System.out.println(totalcost); totalcost += n; System.out.println(totalcost); } System.out.println(“Finished”); System.out.println(totalcost); return totalcost; }

Choosing a test strategy Be aware of the available strategies. Choose strategies appropriate to the point of development. Automate whenever possible. – Reduces tedium. – Reduces human error. – Makes (re)testing more likely.

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state.

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent Call Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent markWork Call Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent markWork Call Return Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent Call Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent markWork Call Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent markWork Call Return Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent Call Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent markWork Call Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent markWork Call Return Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent Call Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent Call Return Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents Call stack

Debuggers Debuggers are both language- and environment-specific. – BlueJ and Eclipse have integrated debugger. Support breakpoints. Step and Step-into controlled execution. Call sequence (stack). Object state. public void markWork(Work w) { int mark = allocateMark(w); if(w.late()) mark = mark * w.getPenalty(); w.setMark(mark); } public void markStudent(Student s) { Work work[] = s.getWork(); if(work != null) { for(Work w : work){ markWork(w); } public void markAllStudents() { Students students[] = getStudents(); if(students != null) { for(Student s : students){ markStudent(s); } markAllStudents markStudent Call Call stack Etc…

Summary we aimed to save $125 million dollars Debugging – Syntax and Logical Errors Testing Strategies – Equivalence Classes and Boundary Values – Test Automation Debuggers