1. Correctness Correctness

2. Quality Perceptions  The perception of quality associated with your code is typically bound to:  Correctness  Efficiency (speed of execution usually)  Cost (if it costs more, it must be better right?)  Robustness  Flexibility  Functionality  Maintainability  Security  Usability  Whatever the user likes.  The perception of quality associated with your code is typically bound to:  Correctness  Efficiency (speed of execution usually)  Cost (if it costs more, it must be better right?)  Robustness  Flexibility  Functionality  Maintainability  Security  Usability  Whatever the user likes.

3. Correctness  We will focus on correctness in this class.  Often has an indirect impact on:  Cost  Robustness  Flexibility  Functionality  Maintainability  Security  What is correctness?  We will focus on correctness in this class.  Often has an indirect impact on:  Cost  Robustness  Flexibility  Functionality  Maintainability  Security  What is correctness?

4. Definition of Software Correctness  Correct software must accomplish the following:  Compute accurate results  Operate safely  Implement the requirements and meet the specifications  Achieve the above for all possible inputs  Recognize input from outside its domain  What can we do to help us achieve this?  Correct software must accomplish the following:  Compute accurate results  Operate safely  Implement the requirements and meet the specifications  Achieve the above for all possible inputs  Recognize input from outside its domain  What can we do to help us achieve this?

5. Standard Techniques  Use design patterns – don’t reinvent the wheel  Use standard libraries  Code re-use  If it we know something is correct from past experience, then why start from scratch to re-invent it?  Need to be mindful of legal issues – are you allowed to use the software, are there licensing issues (use of freeware in software you want to sell)?  Use design patterns – don’t reinvent the wheel  Use standard libraries  Code re-use  If it we know something is correct from past experience, then why start from scratch to re-invent it?  Need to be mindful of legal issues – are you allowed to use the software, are there licensing issues (use of freeware in software you want to sell)?

6. Choice of Programming Language  Choose a language with strict type checking.  A type checker is like a theorem prover which confirms your assertions about the functions/methods and variables you use in the program. Integer f (float x) { return … }  When the type checker checks the code, it confirms:  f is called with one argument and it is a value of type float  If f produces a result, then that result belongs to the class Integer.  Choose a language with strict type checking.  A type checker is like a theorem prover which confirms your assertions about the functions/methods and variables you use in the program. Integer f (float x) { return … }  When the type checker checks the code, it confirms:  f is called with one argument and it is a value of type float  If f produces a result, then that result belongs to the class Integer.

7. Choice of Programming Language  While type checking (and other useful support from programming languages, environments, etc) does not directly check that we match the requirements and specification of the application, it does help reduce coding errors.  You no longer have to check that the function f maps a float onto an Integer.  Does not alleviate the need for testing.  Only true for languages with sound type systems (Ada, Java, ML, Haskell, …). Not true for languages such as C/C++ because the function f could consume a bit pattern which is another type (i.e., string ) and interpret it as a float.  While type checking (and other useful support from programming languages, environments, etc) does not directly check that we match the requirements and specification of the application, it does help reduce coding errors.  You no longer have to check that the function f maps a float onto an Integer.  Does not alleviate the need for testing.  Only true for languages with sound type systems (Ada, Java, ML, Haskell, …). Not true for languages such as C/C++ because the function f could consume a bit pattern which is another type (i.e., string ) and interpret it as a float.

8. Assertions  Type checking is limited in the support it provides. We want something more.  An assertion is a claim about the state of our program (values of variables and their relationship to each other) at various points in the execution of our program.  We are probably used to writing comments that states assertions: int p; // p is a prime number int x;... x = x * x; // x is now positive  Problem – these are just statements and are not checked for us in the same way that the type checker checks things automatically.  Type checking is limited in the support it provides. We want something more.  An assertion is a claim about the state of our program (values of variables and their relationship to each other) at various points in the execution of our program.  We are probably used to writing comments that states assertions: int p; // p is a prime number int x;... x = x * x; // x is now positive  Problem – these are just statements and are not checked for us in the same way that the type checker checks things automatically.

9. Assertions  Assertions are often the result of the execution of statements.  Used to argue that code fragments execute as expected.  Often referred to as pre-conditions and post-conditions (of statements).  Use logic to express or assert facts that we believe or expect to be true: Sort (A); // for all i, j such that 1  i  j  p // A[i]  A[j]  PRECONDITION { Code fragment } POSTCONDITION  Assertions are often the result of the execution of statements.  Used to argue that code fragments execute as expected.  Often referred to as pre-conditions and post-conditions (of statements).  Use logic to express or assert facts that we believe or expect to be true: Sort (A); // for all i, j such that 1  i  j  p // A[i]  A[j]  PRECONDITION { Code fragment } POSTCONDITION

10. Example  Consider: interface Queue { // is the queue empty? boolean empty(); // add Item x to the end of the Queue Queue push (Item x); // return and remove the Item at the head of the Queue Item pop (); // how many elements in the Queue? int size(); }  Consider: interface Queue { // is the queue empty? boolean empty(); // add Item x to the end of the Queue Queue push (Item x); // return and remove the Item at the head of the Queue Item pop (); // how many elements in the Queue? int size(); }

11. Adding Assertions  Questions:  Can the method be called in all situations. If not, when can it be called?  Is there anything special about the result value or state?  It is always possible to call empty. It always returns a true|false value.  It is always possible to call push. The argument must be of type Item (checked by type checker). The result is a non-empty queue whose size is one more than it was before the call. The call only makes sense if the queue is not already full.  Questions:  Can the method be called in all situations. If not, when can it be called?  Is there anything special about the result value or state?  It is always possible to call empty. It always returns a true|false value.  It is always possible to call push. The argument must be of type Item (checked by type checker). The result is a non-empty queue whose size is one more than it was before the call. The call only makes sense if the queue is not already full.

12. Adding Assertions  It is only possible to call pop when the queue contains at least one element. At the end of the call, the queue has one less element than at the start of the call.  It is always possible to call size. The result is an integer in the range 0 to the maximum possible size of the queue.  How do we describe this?  It is only possible to call pop when the queue contains at least one element. At the end of the call, the queue has one less element than at the start of the call.  It is always possible to call size. The result is an integer in the range 0 to the maximum possible size of the queue.  How do we describe this?

13. Example  size´() is the value of the size function BEFORE the current method was called – the old value if you will. interface Queue { // is the queue empty? // PRE: none boolean empty(); //POST: none // add Item x to the end of the Queue //PRE: size() < MAX_LENGTH Queue push (Item x); // POST: !(empty()) && size() = size´() + 1  size´() is the value of the size function BEFORE the current method was called – the old value if you will. interface Queue { // is the queue empty? // PRE: none boolean empty(); //POST: none // add Item x to the end of the Queue //PRE: size() < MAX_LENGTH Queue push (Item x); // POST: !(empty()) && size() = size´() + 1

14. Example // return and remove the Item at the head of the Queue // PRE: !(empty()) Item pop (); // POST: size() = size´() - 1 // how many elements in the Queue? // PRE: none int size(); // POST: 0  RESULT  MAX_LENGTH } // return and remove the Item at the head of the Queue // PRE: !(empty()) Item pop (); // POST: size() = size´() - 1 // how many elements in the Queue? // PRE: none int size(); // POST: 0  RESULT  MAX_LENGTH }

15. Assertions  Q: How do we automatically enforce these assertions?  A: Write a class to help you.  See sample Correctness.java in ~michael/CS351 on esus. class Correctness { public static void pre(boolean expr, Error ex) {…} public static void post (boolean expr) {…} public static void beginRequire () {…} public static void endRequire () {…} public static void beginEnsure () {…} public static void endEnsure () {…} }  Q: How do we automatically enforce these assertions?  A: Write a class to help you.  See sample Correctness.java in ~michael/CS351 on esus. class Correctness { public static void pre(boolean expr, Error ex) {…} public static void post (boolean expr) {…} public static void beginRequire () {…} public static void endRequire () {…} public static void beginEnsure () {…} public static void endEnsure () {…} }

16. Assertions  Pre-conditions must be within a beginRequire() … endRequire() bracket.  Post-conditions must be within a beginEnsure() … endEnsure() bracket.  This helps identify a post-condition that fails within a pre-condition evaluation compared to post-condition that fails at the end of a method. (Read the code).  May have multiple pre- and post-conditions within appropriate bracket.  May have a pre-conditions and post-conditions associated with any group of statements – it is not restricted to the entire method.  Pre-conditions must be within a beginRequire() … endRequire() bracket.  Post-conditions must be within a beginEnsure() … endEnsure() bracket.  This helps identify a post-condition that fails within a pre-condition evaluation compared to post-condition that fails at the end of a method. (Read the code).  May have multiple pre- and post-conditions within appropriate bracket.  May have a pre-conditions and post-conditions associated with any group of statements – it is not restricted to the entire method.

17. Assertions - Example public final Edge getEdge(int id) { Edge Result; // variable declarations // pre-condition checking Correctness.beginRequire(); Correctness.pre( id >= 0 && id < numEdges(), new IdOutOfRange() ); Correctness.endRequire(); Result = impl.getEdge(id); // evaluate result to be returned // post-condition checking Correctness.beginEnsure(); Correctness.post( hasEdge(Result() == true ); Correctness.endEnsure(); return Result; // finally return the result. } public final Edge getEdge(int id) { Edge Result; // variable declarations // pre-condition checking Correctness.beginRequire(); Correctness.pre( id >= 0 && id < numEdges(), new IdOutOfRange() ); Correctness.endRequire(); Result = impl.getEdge(id); // evaluate result to be returned // post-condition checking Correctness.beginEnsure(); Correctness.post( hasEdge(Result() == true ); Correctness.endEnsure(); return Result; // finally return the result. }

18. Expectation  Use assertions when developing your code.  When testing your code, if a pre- or post-condition fails you will get an error message. It helps you identify the failure and correct it far more quickly.  Use them in your project.  Use assertions when developing your code.  When testing your code, if a pre- or post-condition fails you will get an error message. It helps you identify the failure and correct it far more quickly.  Use them in your project.

19. Correctness  Two basic techniques for attempting to produce programs without bugs:  Testing: run the program on various sets of data and see if it behaves correctly in these cases.  Proving correctness: show mathematically that the program always does what it is supposed to do.  Both techniques have their particular problems:  Testing is only as good as the test cases selected.  A proof of correctness may contain errors.  Two basic techniques for attempting to produce programs without bugs:  Testing: run the program on various sets of data and see if it behaves correctly in these cases.  Proving correctness: show mathematically that the program always does what it is supposed to do.  Both techniques have their particular problems:  Testing is only as good as the test cases selected.  A proof of correctness may contain errors.

20. Correctness  A detailed formal proof is typically a lot of work. However, even an informal proof is helpful in clarifying your understanding of how a program works and in convincing yourself that it is probably correct.  Informal proofs are little more than a way of describing your understanding of how the program works – such proofs can easily be produced while writing the program in the first place  Excellent program documentation!  A detailed formal proof is typically a lot of work. However, even an informal proof is helpful in clarifying your understanding of how a program works and in convincing yourself that it is probably correct.  Informal proofs are little more than a way of describing your understanding of how the program works – such proofs can easily be produced while writing the program in the first place  Excellent program documentation!

21. Program Correctness  Before looking at program proving in detail, there is something else that must be pointed out:  A program can only be judged correct in relation to a set of specifications for what it is supposed to do.  All programs do something correctly; the question is: does it do what it is supposed to do?  A really formal proof amounts to showing that a (mathematical) description of what the program does is the same as a (mathematical) description of what it should do.  Before looking at program proving in detail, there is something else that must be pointed out:  A program can only be judged correct in relation to a set of specifications for what it is supposed to do.  All programs do something correctly; the question is: does it do what it is supposed to do?  A really formal proof amounts to showing that a (mathematical) description of what the program does is the same as a (mathematical) description of what it should do.

22. Program Correctness  Aspects of a program's correctness include: (1)Partial correctness: whenever the program terminates, it performs correctly. (2)Termination: the program always terminates. (1) + (2) ⇒ Program is totally correct.  Aspects of a program's correctness include: (1)Partial correctness: whenever the program terminates, it performs correctly. (2)Termination: the program always terminates. (1) + (2) ⇒ Program is totally correct.

23. Program Correctness Proofs  Consider the handout "Proof of Program Correctness" and the function "exponentiate" on the first page. function exponentiate (x: in integer) return integer is –– Evaluates 2**x, for x  0–– {1} i, sum: integer; begin sum := 1; –– sum = 2**0`–– {2} for i in 1.. x loop sum := sum + sum –– sum = 2**i, i>0–– {3} end loop; –– sum = 2**x, x  0–– {4} return sum; end exponentiate;  Consider the handout "Proof of Program Correctness" and the function "exponentiate" on the first page. function exponentiate (x: in integer) return integer is –– Evaluates 2**x, for x  0–– {1} i, sum: integer; begin sum := 1; –– sum = 2**0`–– {2} for i in 1.. x loop sum := sum + sum –– sum = 2**i, i>0–– {3} end loop; –– sum = 2**x, x  0–– {4} return sum; end exponentiate;

24. Program Correctness Proofs {1} lists the goals of the function {2} asserts the initial value of "sum" We can prove {3} by induction. The first time {3} is reached we have i=1 sum=1 + 1 = 2 = 2 0 = 2 i Assume that the nth time {3} is reached sum=2 n then the (n+1)th time sets sum'=sum + sum =2 n + 2 n =2 n+1 therefore {3} always holds. {1} lists the goals of the function {2} asserts the initial value of "sum" We can prove {3} by induction. The first time {3} is reached we have i=1 sum=1 + 1 = 2 = 2 0 = 2 i Assume that the nth time {3} is reached sum=2 n then the (n+1)th time sets sum'=sum + sum =2 n + 2 n =2 n+1 therefore {3} always holds.

25. Program Correctness Proofs If {4} is ever reached, there are two possibilities: a)The loop was never executed, in which case x=0, and sum remains unchanged from {2}, i.e., sum = 1 = 20. b)The loop was executed, in which case {3} was reached x times. Hence at {4}, sum = 2 x. See handout for further examples involving induction. If {4} is ever reached, there are two possibilities: a)The loop was never executed, in which case x=0, and sum remains unchanged from {2}, i.e., sum = 1 = 20. b)The loop was executed, in which case {3} was reached x times. Hence at {4}, sum = 2 x. See handout for further examples involving induction.

26. Program Correctness Proofs  For large programs, a major obstacle of program correctness proofs is an inability of the human to visualize the entire operation.  The remedy is modularization. As we can not write a large program without the aid of modularization and top- down design, we can not understand an algorithm and prove correctness unless it is modularized.  As a module is designed, an informal proof of correctness can be produced to show that the module matches the specification which describes its inputs and outputs.  For large programs, a major obstacle of program correctness proofs is an inability of the human to visualize the entire operation.  The remedy is modularization. As we can not write a large program without the aid of modularization and top- down design, we can not understand an algorithm and prove correctness unless it is modularized.  As a module is designed, an informal proof of correctness can be produced to show that the module matches the specification which describes its inputs and outputs.

27. Program Correctness Proofs  A proof of correctness for a module relying on "lower level" modules is only interested in what they do and not how they do it. The lower level modules are assumed to meet the specifications which state what they do.  The specification of a module consists of two parts:  specification of the range of inputs of the module.  desired effect of the module.  In addition to pre- and post-conditions, a complex algorithm should contain assertions at key points. The more complex the algorithm, the more assertions that are necessary to bridge the gap between pre- and post-conditions.  The assertions should be placed so that it is fairly easy to understand the flow of control from one assertion to the next. In practice, this usually means placing at least one assertion in each loop.  Consider...  A proof of correctness for a module relying on "lower level" modules is only interested in what they do and not how they do it. The lower level modules are assumed to meet the specifications which state what they do.  The specification of a module consists of two parts:  specification of the range of inputs of the module.  desired effect of the module.  In addition to pre- and post-conditions, a complex algorithm should contain assertions at key points. The more complex the algorithm, the more assertions that are necessary to bridge the gap between pre- and post-conditions.  The assertions should be placed so that it is fairly easy to understand the flow of control from one assertion to the next. In practice, this usually means placing at least one assertion in each loop.  Consider...

28. Program Correctness Proofs procedure binary is –– binary search algorithm N: constant...;–– some number  1} x: array (1..N) of float; key: float; L, R, K: integer; found: boolean; begin key :=...; –– (x[I]  x[J] iff 1  I  J  N) and (X[1]  key  x[N}){0} L := 1; R := N; found := false; -- 1  L  R  N and x(L)  key  x(R){1} while (L  R) and (not found) loop K := (L+R) div 2; –– 1  L  K  R  N and (  x(L)  key  x(R)){2} found := (x(K) = key); if not found then–– x(K)  key{3} if key<x(K) then R := K–1;––   key  x(R){4} else L := K+1;––  x(L)  key end if;––{5} ––  x(L)  key  x(R){6} end if; end loop; –– found=  and (  x(K)=key){7} end binary; [  is “key is present in the array x”] procedure binary is –– binary search algorithm N: constant...;–– some number  1} x: array (1..N) of float; key: float; L, R, K: integer; found: boolean; begin key :=...; –– (x[I]  x[J] iff 1  I  J  N) and (X[1]  key  x[N}){0} L := 1; R := N; found := false; -- 1  L  R  N and x(L)  key  x(R){1} while (L  R) and (not found) loop K := (L+R) div 2; –– 1  L  K  R  N and (  x(L)  key  x(R)){2} found := (x(K) = key); if not found then–– x(K)  key{3} if key<x(K) then R := K–1;––   key  x(R){4} else L := K+1;––  x(L)  key end if;––{5} ––  x(L)  key  x(R){6} end if; end loop; –– found=  and (  x(K)=key){7} end binary; [  is “key is present in the array x”]

29. Program Correctness Proofs  {0} is a pre-condition describing what this module expects of its input.  {1} is a pre-condition describing the initial conditions before entering the loop.  {2} is an assertion true at that point for each iteration of the loop.  {3} is an assertion true whenever the if condition evaluates to true.  {4} holds if the then clause is executed.  {5} holds if the else clause is executed.  {6} holds after the if statement. It is true irrespective of whether the then or else clause was executed.  {7} is the post-condition of the module.  {0} is a pre-condition describing what this module expects of its input.  {1} is a pre-condition describing the initial conditions before entering the loop.  {2} is an assertion true at that point for each iteration of the loop.  {3} is an assertion true whenever the if condition evaluates to true.  {4} holds if the then clause is executed.  {5} holds if the else clause is executed.  {6} holds after the if statement. It is true irrespective of whether the then or else clause was executed.  {7} is the post-condition of the module.

30. Termination  A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be reasonable confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.  In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.  In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.  A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be reasonable confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.  In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.  In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.

31. Termination  A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be relatively confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.  In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.  A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be relatively confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.  In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.

32. Termination  In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.  Consider the following function:  In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.  Consider the following function: function Ackermann(x, y: in integer) return integer is –– x and y must be nonnegative integers begin –– Ackermann if x = 0 then return (y+1); elsif y = 0 then return Ackermann((x-1), 1); else return Ackermann((x-1), Ackermann(x, (y-1))); end if; end Ackermann;

33. Reading  Rex Page, “Engineering Software Correctness”, ACM, FDPE’05, September 25, 2005, Tallinn, Estonia, pp 39-46.  Bertrand Meyer, ‘Applying “Design by Contract”’, IEEE Computer, October 1992, pp 40-51.  Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, Raymie Stata, “Extended Static Checking for Java”, ACM, PLDI’02, June 17-19, 2002, Berlin, Germany.  Rex Page, “Engineering Software Correctness”, ACM, FDPE’05, September 25, 2005, Tallinn, Estonia, pp 39-46.  Bertrand Meyer, ‘Applying “Design by Contract”’, IEEE Computer, October 1992, pp 40-51.  Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, Raymie Stata, “Extended Static Checking for Java”, ACM, PLDI’02, June 17-19, 2002, Berlin, Germany.

34. Summary  Correctness is an important aspect of software quality.  Use appropriate tools (including programming language choice) to help.  Use assertions.  Reason about the code to prove correctness when necessary.  It is easier and quicker to use assertions etc than it is to test the code to demonstrate compliance with the requirements and specifications.  Still need to test the code – could have flaws in your logic!  Correctness is an important aspect of software quality.  Use appropriate tools (including programming language choice) to help.  Use assertions.  Reason about the code to prove correctness when necessary.  It is easier and quicker to use assertions etc than it is to test the code to demonstrate compliance with the requirements and specifications.  Still need to test the code – could have flaws in your logic!