1. EECE 310: Software Engineering Exception handling and Testing

2. Learning Objectives Define specifications with exceptions Choose between checked/unchecked exceptions and implement them Decide when to handle exceptions or propagate them Define the goals and characteristics of testing Write black-box test cases for programs Write glass-box test cases for programs

3. What’s wrong with this ? Caller must remember to not call factorial() with n < 0  behavior is undefined otherwise. These are called partial procedures as they have a REQUIRES clause, as opposed to full procedures. public static int factorial (int n) // REQUIRES: n >= 0 // EFFECTS: returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0.

4. Why not do this ? public static int factorial (int n) // EFFECTS: returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0. // and -1 for n < 0 Why is this not an elegant solution ? 1.Not really the purpose of a return value 2.Caller may forget to check for this case 3.What if every return value is a possible result ?

5. Exceptions Mechanism to signal abnormal/unexpected inputs or conditions in the program Typically used for signaling errors at runtime May be used during development to indicate situations that haven’t yet been handled

6. Exceptions in Java Exceptions need to be specified in procedure signature (declaration) with throws clause – Strictly true only for checked exceptions (later) – Separate multiple exceptions with commas public static int factorial( int n ) throws NonPositiveException

7. Exceptions: Specifications EFFECTS clause must reflect both the exception and the condition under which it is thrown REQUIRES clause should not list the condition No changes to the MODIFIES clause (usually) public static int factorial(int n) throws NonPositiveException { // EFFECTS: If (n < 0) throws NonPositiveException, // otherwise returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0

8. Learning Objectives Define specifications with exceptions Choose between checked/unchecked exceptions and implement them Decide when to handle exceptions or propagate them Define the goals and characteristics of testing Write black-box test cases for programs Write glass-box test cases for programs

9. Exceptions: Implementation public static int factorial(int n) throws NonPositiveException { // EFFECTS: If (n < 0) throws NonPositiveException, // otherwise returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0 if (n < 0) throw new NonPositiveException(“Factorial”); …. Exception class must be defined by the user

10. Checked vs. Unchecked checkedunchecked listed in the procedure’s signature mandatoryoptional user code must handle can handle

11. Exception types in Java Throwable Error Exception RuntimeException unchecked exceptions checked exceptions NullPointerException IOException

12. What happens when an exception is thrown in Java ? If it is a checked exception, then the calling procedure must handle the exception or explicitly propagate it If it is an unchecked exception, it propagates in stack till it finds an appropriate catch block foo fooCaller main stack growth propagation direction Stack

13. Some Rules of Thumb … Always use Checked Exceptions unless – When you know the caller is unlikely to exercise the exceptional condition e.g., internal routine – When the situation being handled is truly out of the ordinary (e.g., Out of memory error) – There is a convenient way to check for the condition apriori e.g., NullPointerException – The exception is meaningless outside its context

14. Exceptions: Group Activity Consider the search procedure below. Can you rewrite it to use exceptions ? Which of the exceptions will you make checked and why ? public static int search( int[] a, int x) { // REQUIRES: a is not NULL // MODIFIES: None // EFFECTS: Examines each of a[0], a[1]… in any // order and returns the index of an array // element that equals to x. If no such // element is found, return -1.

15. Learning Objectives Define specifications with exceptions Choose between checked/unchecked exceptions and implement them Decide when to handle exceptions or propagate them Define the goals and characteristics of testing Write black-box test cases for programs Write glass-box test cases for programs

16. How to handle exceptions ? Using the try-catch-finally construct in Java try { doSearch();// Can throw three types of exceptions } catch (NullPointerException np) { // Take appropriate action 1 } catch (ElementNotFoundException ep) { // Take appropriate action 2 } finally { // Common cleanup code for all catch blocks // Called even if the exception handling code throws exceptions }

17. What can we do in the catch block ? Handling the exception Correct the circumstances that led to the exception or print a descriptive error message. catch(ElementNotFoundExceptio n e) { // Can use the variable e system.out.println(e); } Throw a new exception Sometimes the exception may indicate a larger problem in the program, which makes sense only to caller. The caller then throws a new exception. catch (NullPointerException np) { // Possible out of memory throw new OutOfMemException(“foo”); }

18. Exception Propagation When is it better to not handle the exception – Because you have insufficient info at the caller – Because you know the exception is handled later, say by your caller (s), so avoid duplication of effort How to propagate an exception ? static void insertIfNotFound(int[] a, int x) throws NullPointerException { // EFFECTS: If the element x is not present in a, // insert it into a. Otherwise, do nothing.

19. When should you propagate an exception ? Propagate an Exception if it is handled elsewhere OR if you have insufficient information to handle the error at this level Make sure you write the exception in the proc. declaration if it is a checked exception Handle an exception if you have sufficient information to bypass the error, print a error message or if you need to re-interpret it (i.e., throw a new exception) Make sure that you don’t throw the same exception again

20. Learning Objectives Define specifications with exceptions Choose between checked/unchecked exceptions and implement them Decide when to handle exceptions or propagate them Define the goals and characteristics of testing Write black-box test cases for programs Write glass-box test cases for programs

21. Validation Does the software do what was wanted? – Increase confidence that a program will function as desired. – This is difficult to determine and involves subjective judgements Does the software meet its specification? – This can be objective if the specifications are sufficiently precise Experiment with the program (Testing) Experiment with the program (Testing) Reason about the program (Formal verification) Reason about the program (Formal verification) Inspect the program (Reviews) Inspect the program (Reviews) Validation

22. Code Reviews Inspect other’s code for bugs (manually) Why don’t code reviews alone work ? – Ad-hoc techniques, no standard way of doing it – Not reproducible – too much reliance on humans – Errors can occur even when all standard coding practices are followed (corner cases) – Often, things that are difficult for one person are also difficult for another (Knight & Leveson study)

23. Formal Verification Tests the correctness of a piece of code without executing it – Requires detailed formal specifications of behavior – Can reveal the presence of hard-to-find bugs Theorem-proving -Can prove correctness in all cases (inductive) -Requires significant expert intervention Model-Checking -Proves correctness up to a finite depth bound -Can be completely automated, but not always scalable Formal Verification

24. EECE 310 - 24 Testing Testing shows that a program meets its specification Testing can never be exhaustive for non-trivial programs So pick a partition of the input space for testing Systematic testing depends on partitioning 1. Partition the set of possible behaviors of the system 2. Choose representative samples from each partition 3. Make sure we covered all partition How do you identify suitable partitions? That’s what testing is all about!!! Methods: black box, glass box,...

25. Learning Objectives Define specifications with exceptions Choose between checked/unchecked exceptions and implement them Decide when to handle exceptions or propagate them Define the goals and characteristics of testing Write black-box test cases for programs Write glass-box test cases for programs

26. Black-box Testing Generate test cases from the specification only (don’t look at code) Advantages: – Avoids making the same assumptions as the programmer – Test data is independent of the procedure’s implementation – Results can be interpreted without knowing implementation details – Forces you to write a good specification

27. Test cases for black box testing Paths through the spec – E.g., choose test cases that cover each part of the ‘requires’, ‘modifies’ and ‘effects’ clauses Boundary conditions – Choose test cases that are at or close to boundaries for ranges of inputs – Test for aliasing errors (e.g., two parameters referring to the same object) Invalid input – The program should degrade gracefully, without experiencing data loss

28. Black Box Testing: Example Paths through the spec: – “x  0” means “x>0 or x = 0”, so test both “paths” – x < 0 is another path Boundary conditions: – x >=0 choose: 1, 0, as values for x – x < 0 choose: -1 as value for x – Also, try very large & very small values for x Other cases Invalid output: Not allowed by the type-system Aliasing: Int is a primitive type, so not possible Values for epsilon > 0.001, values for epsilon < 0.00001 int abs(int x) { // returns: x returns -x // otherwise => returns x int abs(int x) { // returns: x returns -x // otherwise => returns x

29. Aliasing: Example Static void appendVector(Vector v1, Vector v2) throws NullPointerException { // REQUIRES: v1 and v2 contain non-NULL elements // EFFECTS: if v1 or v2 is null, throws NullPointerException // else removes all elements of v2 and appends them // in reverse order to the end of v1 if (v1 == null) throw new NullPointerException(“Vector.appendVector”); while (v2.size ( ) > 0 ) { v1.addElement( v2.lastElement( ) ); v2.removeElementAt(v2.size() – 1 ); } } What happens if v1==v2 ?

30. Black Box Testing: Abs Program Can you spot the error in the code below ? – Will this error be detected by the test suite ? int abs(int x) { // returns: x returns -x // otherwise => returns x if (x < -5) return –x; else return x; } int abs(int x) { // returns: x returns -x // otherwise => returns x if (x < -5) return –x; else return x; }

31. Group Activity Write black box test cases for the search procedure with exceptions. Also specify for each test case, what aspect it tests (e.g., invalid input). public static int search( int[] a, int x) throw NullPointerException, ElementNotFoundException{ // MODIFIES: None // EFFECTS: Examines each of a[0], a[1]… in any // order and returns the index of an array // element that equals to x. Otherwise, If // no such element is found, throw // an ElementNotFoundException. // if a is NULL, throw NullPointerException

32. Learning Objectives Define specifications with exceptions Choose between checked/unchecked exceptions and implement them Decide when to handle exceptions or propagate them Define the goals and characteristics of testing Write black-box test cases for programs Write glass-box test cases for programs

33. Glass Box testing Examine the code and test all paths – Because black box testing can never guarantee we exercised all the code Path completeness: – A path is a sequence of statements in the code – A test set is path complete if each path through the code is exercised by at least one case in the test set Not the same as saying each statement in the code is reached!! (Why ?)

34. Glass Box Testing: Example There are four paths in the following code So we need at least 4 test-cases. Example: – x=3, y=2, z=1 – x=3, y=2, z=4 – x=2, y=3, z=2 – x=2, y=3, z=4 Are the above tests sufficient ? int maxval(int x,int y,int z) { // EFFECTS: returns maximum // value of the three inputs if (x > y) { if (x > z) return x else return z } else { if (y > z) return y else return z } } int maxval(int x,int y,int z) { // EFFECTS: returns maximum // value of the three inputs if (x > y) { if (x > z) return x else return z } else { if (y > z) return y else return z } } x > y x > z y > z

35. Weaknesses of path completeness Path completeness is insufficient Consider the test case x=4, y=1, z=2. This is path complete. The program performs correctly on this test case but the program is still wrong !! Path completeness is usually infeasible – How many paths are there through this program segment (for any a[i])? – Loops are problematic. Try: test 0, 1, 2, n-1, and n iterations, (n is the max number of iterations possible) for (j=0, i=0; i<100; i++) if a[i]=true then j=j+1 for (j=0, i=0; i<100; i++) if a[i]=true then j=j+1 int maxval (int x, y, z) { /* EFFECTS: returns maximum value of the three inputs */ return z; } int maxval (int x, y, z) { /* EFFECTS: returns maximum value of the three inputs */ return z; }

36. How to approximate Path Completeness ? 1. Loops: – With variable amounts of iteration 0, 1, 2 iterations All possible ways to exit loop for each case 2. For each statement where an exception could be raised – Test case that attempts to cause the exception 3. Straight-line code All paths through the code THESE ARE ALL HEURISTICS. THERE IS NO SILVER BULLET.

37. Path Completeness Approximation: Example Test NPException: – factorial(-1) Paths through the loop: – 0 iterations (n = 1) returns 1 – 1 iteration (n = 2) returns 1 (Bug found !) 2 iterations (n = 3) returns 2 (Bug found !) The program has a bug  the loop condition should be (i <= n) not (i < n) int factorial( int n ) { int result = 1;int i; if (n < 0) throw new NPException(“fact”); for (i = 1; i < n; i++) { result = result * i; } return result; }

38. Group Activity Write glass-box test cases for an implementation of search that was written by some other group in the class. Do not attempt to correct any bugs unless they’re egregious. The goal is to use glass-box tests to find them.

39. Learning Objectives Define specifications with exceptions Choose between checked/unchecked exceptions and implement them Decide when to handle exceptions or propagate them Define the goals and characteristics of testing Write black-box test cases for programs Write glass-box test cases for programs

40. To do before next class 1. Read in the textbook: – sections 10.1, 10.2, Chapter 4 – Exercises 4.1 to 4.4 and 10.1 2. Show up in the lab to submit your first assignment (due next week !)