2. Program Correctness and Efficiency Chapter 2

3. Chapter 2: Program Correctness and Efficiency2 Chapter Objectives To understand the differences between the three categories of program errors To understand the effect of an uncaught exception and why you should catch exceptions To become familiar with the Exception hierarchy and the difference between checked and unchecked exceptions To learn how to use the try-catch-finally sequence to catch and process exceptions To understand what it means to throw an exception and how to throw an exception in a method

4. Chapter 2: Program Correctness and Efficiency3 Chapter Objectives (continued) To understand the different testing strategies and when and how they are performed To learn how to write special methods to test other methods and classes To become familiar with debugging techniques and debugger programs To be introduced to the process of program verification and the use of assertions and loop invariants To understand the meaning of big-O notation and how it is used to analyze an algorithm’s efficiency

5. Chapter 2: Program Correctness and Efficiency4 Program Defects and “Bugs” A program may be efficient, but is worthless if it produces a wrong answer Defects often appear in software after it is delivered Testing can never demonstrate the complete absence of defects In some situations it is very difficult to test a software product completely in the environment in which it is used Debugging: removing defects

6. Chapter 2: Program Correctness and Efficiency5 Examples of Bad Software Bad software is everywhere! NASA Mars Lander (cost $165 million) Crashed into Mars Error in converting English and metric units of measure Denver airport Buggy baggage handling system Delayed airport opening by 11 months Cost of delay exceeded $1 million/day MV-22 Osprey Advanced military aircraft Lives have been lost due to faulty software

7. Chapter 2: Program Correctness and Efficiency6 Software and Complexity A new car contains more lines of code (LOC) than was required to land the Apollo astronauts on the moon Netscape17,000,000 Space shuttle10,000,000 Linux1,500,000 Windows XP40,000,000 Boeing 7777,000,000 system LOC

8. Chapter 2: Program Correctness and Efficiency7 Syntax Errors Syntax errors are mistakes in the grammar of a language The Java compiler detects syntax errors during compilation and requires you to correct them before successfully compiling the program Some common syntax errors include: Omitting or misplacing braces Performing an incorrect type of operation on a primitive type value Invoking an instance method not defined Not declaring a variable before using it Providing multiple declarations of a variable

9. Chapter 2: Program Correctness and Efficiency8 Run-time Errors or Exceptions Run-time errors Occur during program execution Occur when the JVM detects an operation that it knows to be incorrect Cause the JVM to throw an exception Examples of run-time errors include Division by zero Array index out of bounds Number format and Input mismatch error Null pointer exceptions

10. Chapter 2: Program Correctness and Efficiency9 Run-time Errors or Exceptions (continued)

11. Chapter 2: Program Correctness and Efficiency10 Logic Errors A logic error occurs when the programmer or analyst Made a mistake in the design of a class or method Implemented an algorithm incorrectly Most logic errors do not cause syntax or run-time errors and are thus difficult to find Sometimes found through testing Sometimes found during real-world operation of the program

12. Chapter 2: Program Correctness and Efficiency11 The Exception Class Hierarchy When an exception is thrown, one of the Java exception classes is instantiated Exceptions are defined within a class hierarchy that has the class Throwable as its superclass Classes Error and Exception are subclasses of Throwable RuntimeException is a subclass of Exception

13. Chapter 2: Program Correctness and Efficiency12 The Class Throwable Throwable is the superclass of all exceptions All exception classes inherit the methods of throwable

14. Chapter 2: Program Correctness and Efficiency13 The Class Throwable (continued)

15. Chapter 2: Program Correctness and Efficiency14 Checked and Unchecked Exceptions Two categories of exceptions: checked and unchecked Checked exception normally not due to programmer error and is beyond the control of the programmer Unchecked exception may result from Programmer error Serious external conditions that are unrecoverable

16. Chapter 2: Program Correctness and Efficiency15 Checked and Unchecked Exceptions

17. Chapter 2: Program Correctness and Efficiency16 Catching and Handling Exceptions When an exception is thrown, the normal sequence of execution is interrupted Default behavior Program stops JVM displays an error message The programmer may override the default behavior by Enclosing statements in a try block Processing the exception in a catch block

18. Chapter 2: Program Correctness and Efficiency17 Uncaught Exceptions When an exception occurs that is not caught, the program stops and the JVM displays an error message and a stack trace The stack trace shows the sequence of method calls, starting at the method that threw the exception and ending at main

19. Chapter 2: Program Correctness and Efficiency18 The try-catch-finally Sequence Avoid uncaught exceptions Write a try-catch sequence to catch an exception Handle it rather than relying on the JVM Catch block is skipped if all statements within the try block execute without error

20. Chapter 2: Program Correctness and Efficiency19 Handling Exceptions to Recover from Errors Exceptions provide the opportunity to Recover from errors Report errors User error is a common source of error and should be recoverable Catch block within the first catch clause having an appropriate exception class executes, others are skipped Compiler displays an error message if it encounters an unreachable catch clause

21. Chapter 2: Program Correctness and Efficiency20 The finally Block When an exception is thrown, the flow of execution is suspended and there is no return to the try block There are situations in which allowing a program to continue after an exception could cause problems The code in the finally block is executed either after the try block is exited or after a catch clause is exited The finally block is optional

22. Chapter 2: Program Correctness and Efficiency21 Throwing Exceptions Instead of catching an exception in a lower-level method, it can be caught and handled by a higher-level method Declare that the lower-level method may throw a checked exception by adding a throws clause to the method header Can throw the exception in the lower-level method, using a throw statement The throws clause is useful if a higher-level module already contains a catch clause for this exception type

23. Chapter 2: Program Correctness and Efficiency22 Throwing Exceptions (continued) Can use a throw statement in a lower-level method to indicate that an error condition has been detected Once the throw statement executes, the lower-level method stops executing immediately

24. Chapter 2: Program Correctness and Efficiency23 Catching Exceptions Example

25. Chapter 2: Program Correctness and Efficiency24 Programming Style You can always avoid handling exceptions by declaring that they are thrown, or throwing them and letting them be handled farther back in the call chain It is usually best to handle the exception instead of passing it along The following are recommended guidelines: If an exception is recoverable in the current method, handle the exception in the current method If a checked exception is likely to be caught in a higher-level method, declare that it can occur using a throws clause It is not necessary to use a throws clause with unchecked exceptions

26. Chapter 2: Program Correctness and Efficiency25 Testing (Textbook View) There is no guarantee that a program that is syntax and run-time error free will also be void of logic errors The “best” situation is a logic error that occurs in a part of the program that always executes; otherwise, it may be difficult to find the error The worst kind of logic error is one that occurs in an obscure part of the code (infrequently executed)

27. Chapter 2: Program Correctness and Efficiency26 Structured Walkthroughs Most logic errors arise during the design phase and are the result of an incorrect algorithm Logic errors may also result from typographical errors that do not cause syntax or run-time errors One form of testing is hand-tracing the algorithm before implementing Structured walkthrough: designer must explain the algorithm to other team members and simulate its execution with other team members looking on

28. Chapter 2: Program Correctness and Efficiency27 Levels and Types of Testing Testing: exercising a program under controlled conditions and verifying the results Purpose is to detect program defects after all syntax errors have been removed and the program compiles No amount of testing can guarantee the absence of defects in sufficiently complex programs Unit testing: checking the smallest testable piece of the software (a method or class) Integration testing: testing the interactions among units

29. Chapter 2: Program Correctness and Efficiency28 Levels and Types of Testing (continued) System testing: testing the program in context Acceptance testing: system testing designed to show that the program meets its functional requirements Black-box testing: tests the item based on its interfaces and functional requirements White-box testing: tests the software with the knowledge of its internal structure

30. Chapter 2: Program Correctness and Efficiency29 Preparations for Testing A test plan should be developed early in the design phase Aspects of a test plan include deciding how the software will be tested, when the tests will occur, who will do the testing, and what test data will be used If the test plan is developed early, testing can take place concurrently with the design and coding A good programmer practices defensive programming and includes code to detect unexpected or invalid data

31. Chapter 2: Program Correctness and Efficiency30 Testing Tips for Program Systems Most of the time, you will test program systems that contain collections of classes, each with several methods If a method implements an interface, its specification should document input parameters and expected results Carefully document each method parameter and class attribute using comments as you write the code Leave a trace of execution by displaying the method name as you enter it Display values of all input parameters upon entry to a method

32. Chapter 2: Program Correctness and Efficiency31 Testing Tips for Program Systems (continued) Display the values of any class attributes that are accessed by this method Display the values of all method outputs after returning from a method Plan for testing as you write each module rather than after the fact

33. Chapter 2: Program Correctness and Efficiency32 Developing the Test Data Test data should be specified during the analysis and design phases for the different levels of testing: unit, integration, and system In black-box testing, we are concerned with the relationship between the unit inputs and outputs There should be test data to check for all expected inputs as well as unanticipated data In white-box testing, we are concerned with exercising alternative paths through the code Test data should ensure that all if statement conditions will evaluate to both true and false

34. Chapter 2: Program Correctness and Efficiency33 Testing Boundary Conditions When hand-tracing through an algorithm or performing white-box testing, you must exercise all paths Check special cases called boundary conditions

35. Chapter 2: Program Correctness and Efficiency34 Testing Normally testing is done by The programmer Other members of the software team who did not code the module being tested Final users of the software product Do not rely on programmers for testing as they are often blind to their own oversights Companies also have quality assurance (QA) organizations that verify that the testing process is performed correctly In extreme programming, programmers work in pairs where one writes the code and the other writes the tests

36. Chapter 2: Program Correctness and Efficiency35 Stubs It may be difficult to test a method or class that interacts with other methods or classes The replacement of a method that has not yet been implemented or tested is called a stub A stub has the same header as the method it replaces, but its body only displays a message indicating that the stub was called

37. Chapter 2: Program Correctness and Efficiency36 Drivers A driver program declares any necessary object instances and variables, assigns values to any of the method’s inputs, calls the method, and displays the values of any outputs returned by the method You can put a main method in a class to serve as the test driver for that class’s methods

38. Chapter 2: Program Correctness and Efficiency37 Using a Test Framework A test framework is a software product that facilitates writing test cases, organizing the test cases into test suites, running the test suites, and reporting the results A test framework often used for Java products is JUnit, an open-source product that can be used in a stand- alone mode and is available from junit.org

39. Chapter 2: Program Correctness and Efficiency38 Debugging a Program Debugging is the major activity performed by programmers during the testing phase Testing determines if there is an error, debugging determines the cause of it Debugging is like detective work Inspect carefully the information displayed by your program Insert additional diagnostic output statements in the method to determine more information

40. Chapter 2: Program Correctness and Efficiency39 Debugging Example What does programmer intend, and what is the problem? public static String getSentence() { String sentence = “”; int count = 0; String word = JOptionPane.showInputDialog(“Enter word, *** to quit”); while (word != “***” && word != null && count < 10) { sentence = sentence + word + “ “; count++; word = JOptionPane.showInputDialog(“Enter word, *** to quit”); } return sentence; }

41. Chapter 2: Program Correctness and Efficiency40 Using a Debugger Debuggers often are included with IDEs A debugger can execute your program incrementally rather than all at once Single-step execution executes in increments as small as one program statement Breakpoints are used to traverse large portions of code before stopping The actual mechanics of using a debugger depend on the IDE that you are using

42. Chapter 2: Program Correctness and Efficiency41 Using a Debugger (continued)

43. Chapter 2: Program Correctness and Efficiency42 Reasoning about Programs: Assertions and Loop Invariants Assertions: logical statements about a program that are claimed to be true; generally written as a comment Preconditions and postconditions are assertions A loop invariant is an assertion Helps prove that a loop meets it specification True before loop begins, at the beginning of each repetition of the loop body, and just after loop exit

44. Chapter 2: Program Correctness and Efficiency43 Assertions and Loop Invariants Example

45. Chapter 2: Program Correctness and Efficiency44 Testing (My View) Can be shown that probability of an error appearing after t units of testing is about E = K/twhere K is a constant This approximation holds over large range of t Then the “mean time between failures” is MTBF = t/K The good news: quality improves with testing The bad news: quality only improves linearly with testing!

46. Chapter 2: Program Correctness and Efficiency45 MTBF and Testing The “mean time between failures” is approximately MTBF = t/K To have 1,000,000 hours without failure, must test (on the order of) 1,000,000 hours! Ordinary users are, in effect, “testing” the code What does this imply about large complex pieces of software? Consider Windows XP, for example…

47. Chapter 2: Program Correctness and Efficiency46 Testing The fundamental problem Developers (or good guys) must find (almost) all flaws/bugs Users (or bad guy) only “need” 1 flaw/bug Aside: Software reliability far more difficult in security than elsewhere Why? See the next slide…

48. Chapter 2: Program Correctness and Efficiency47 Security Testing: Do the Math Recall that MTBF = t/K Suppose 10 6 security flaws in some software Say, Windows XP Suppose each bug has MTBF of 10 9 hours Expect to find 1 bug for every 10 3 hours testing Good guys spend 10 7 hours testing: find 10 4 bugs Good guys have found 1% of all the bugs Bad guy spends 10 3 hours of testing: finds 1 bug Chance good guys found bad guy’s bug is only 1% !!!

49. Chapter 2: Program Correctness and Efficiency48 Types of Testing Function testing  verify that system functions as it is supposed to Performance testing  other requirements such as speed, resource use, etc. Acceptance testing  customer involved Installation testing  test at install time Regression testing  test after any change

50. Chapter 2: Program Correctness and Efficiency49 Other Testing Issues Active fault detection Don’t wait for system to fail Actively try to make it fail  users/attackers will! Fault injection Insert faults into the process Even if no obvious way for such a fault to occur Bug injection Insert bugs into code See how many of injected bugs are found Can use this to estimate number of bugs Assumes injected bugs similar to unknown bugs

51. Chapter 2: Program Correctness and Efficiency50 A Testing Case History In one system with 184,000 lines of code Flaws found 17.3% inspecting system design 19.1% inspecting component design 15.1% code inspection 29.4% integration testing 16.6% system and regression testing Conclusion: must do many kinds of testing Overlapping testing is necessary Provides a form of “defense in depth”

52. Chapter 2: Program Correctness and Efficiency51 Testing: The Bottom Line Testing is inherently difficult Security testing far more demanding than non- security testing Non-security testing  does system do what it is supposed to? Security testing  does system do what it is supposed to… …and nothing more? Usually impossible to do exhaustive testing How much testing is enough?

53. Chapter 2: Program Correctness and Efficiency52 Testing: The Bottom Line How much testing is enough? Recall MTBF = t/K Seems to imply testing is nearly hopeless! But there is some hope… If we can eliminate an entire class of flaws then statistical model breaks down For example, if we have a single test (or a few tests) to eliminate all buffer overflows

54. Chapter 2: Program Correctness and Efficiency53 Efficiency of Algorithms Difficult to get a precise measure of the performance of an algorithm or program Can characterize a program by how the execution time or memory requirements increase as a function of increasing input size Big-O notation A simple way to determine the big-O of an algorithm or program is to look at the loops and to see whether the loops are nested

55. Chapter 2: Program Correctness and Efficiency54 Efficiency of Algorithms Consider for (int i = 0; i< x.length; i++) { if(x[i] == target) return i; } Suppose n is x.length Then “work” is T(n) = n Since 1 statement each time thru the loop

56. Chapter 2: Program Correctness and Efficiency55 Efficiency of Algorithms Consider for (int i = 0; i< x.length; i++) { for (int j = 0; j< x.length; j++) { if(i != j && x[i] == x[j]) return false; } Suppose n is x.length Then “work” is T(n) = n 2

57. Chapter 2: Program Correctness and Efficiency56 Efficiency of Algorithms (continued) Consider: First time through outer loop, inner loop is executed n-1 times; next time n-2, and the last time once. So we have T(n) = 3(n – 1) + 3(n – 2) + … + 3 or T(n) = 3(n – 1 + n – 2 + … + 1)

58. Chapter 2: Program Correctness and Efficiency57 Efficiency of Algorithms (continued) We can reduce the expression in parentheses to: n x (n – 1) 2 So, T(n) = 1.5n 2 – 1.5n This polynomial is zero when n is 1. For values greater than 1, 1.5n 2 is always greater than 1.5n 2 – 1.5n Therefore, we say that T(n) is O(n 2 ) This is “big O” notation Upper bound on performance Assumes n is “big enough”, and up to constant factor For example, T(n) = n 2 + 5n + 25 is O(n 2 )

59. Chapter 2: Program Correctness and Efficiency58 Efficiency of Algorithms (continued)

60. Chapter 2: Program Correctness and Efficiency59 Chapter Review Three kinds of defects can occur in programs Syntax errors Run-time errors Logic errors All exceptions in the Exception class hierarchy are derived from a common superclass called Throwable The default behavior for exceptions is for the JVM to catch them by printing an error message and a call stack trace and then terminating the program

61. Chapter 2: Program Correctness and Efficiency60 Chapter Review (continued) Two categories of exceptions: checked and unchecked A method that can throw a checked exception must either catch it or declare that it is thrown “throw” statement throws an unchecked exception Program testing is done at several levels starting with the smallest testable piece of a program called a unit Integration testing: once units are individually tested, they can then be tested together System testing: once the whole program is put together, it is tested as a whole

62. Chapter 2: Program Correctness and Efficiency61 Chapter Review (continued) Acceptance programming involves testing in an operational manner demonstrating its functionality Black-box testing tests the item based on its functional requirements without knowledge of its internal structure White-box testing tests the item using knowledge of its internal structure Test drivers and stubs are tools used in testing Test drivers exercise a method or class Stubs stand in for called methods Big-O notation determines the efficiency of a program