1. 50.530: Software Engineering
Sun Jun
SUTD

2. Week 10: Invariant Generation

3. Problem
{pre}while B do program{post} if there exists an invariant inv such that the following are satisfied:
(1) pre => inv
(2) {inv && B} program {inv}
(3) inv && !B => post
and the loop terminates.
How do we find inv so as to complete the proof?

4. Big View
inv
pre
pre => inv

5. Big View B !B
inv
post
pre
inv && !B => post

6. Big View B !B
inv
post
pre
one iteration
{inv && B}program{inv}

7. Static/Dynamic Analysis
Static analysis: infer (loop) invariants based on source code without executing the program (treating programs a mathematical formula)
Dynamic analysis: infer (loop) invariants based on testing results.
It’s about learning something about the invariants and making guesses!

8. Exercise 1 x = 0.1; y = 0; while (x < 2) { k = 4 – x*x;
y = sqrt(4-k);
x += 0.001; }
if (y < 0) {
error();
Show that the error is not occurring.

9. Ernst et al. IEEE Transactions on Software Engineering 2001
Dynamically Discovering Likely Program Invariants to Support Program Evolution

10. The Approach
Seem familiar?

11. Instrumentation
Instrument at the beginning/end of each method and the start of loops.
Daikon only supports two forms of data: scalar numbers (including characters and Booleans) and sequence of scalars;
Convert other values into one of these forms.

12. Example: Instrumentation
public int sumUp (int[] B, int N) {
int i = 0;
int s = 0;
while (i != N) {
i = i+1;
s = s +B[i] }
return s;
public int sumUp (int[] B, int N) {
//add code to output values
int i = 0;
int s = 0;
while ( i != N) {
i = i+1;
s = s +B[i]; }
return s;

13. Example: Testing
100 randomly-generated arrays of length 7 to 13, in which each element was a random number in the range of -100 to 100. The following s is the learned pre-condition.

14. Example: Testing
100 randomly-generated arrays of length 7 to 13, in which each element was a random number in the range of -100 to 100. The following s is the learned post-condition.

15. Example: Testing
100 randomly-generated arrays of length 7 to 13, in which each element was a random number in the range of -100 to 100. The following loop invariants are learned .

16. Discussion
What invariants should we infer?

17. What Invariants to Infer?
Invariants over any variables
Constant value, e.g., x = a;
Uninitialized, e.g., x = uninit;
Invariants over a single numeric variable
Range limit, e.g., x >= a, x <= b, a <= x <= b
Nonzero, e.g., x != 0
Modulus, e.g., x mod b = a
Non-modulus, e.g., x mod b != a

18. What Invariants to Infer?
Invariants over two numeric variables
Linear relationship, e.g., y = ax+b
Ordering comparison: x < y, x <= y, x > y, x >= y, x = y, x != y
Functions, e.g., y = fn(x) or x = fn(y) where fn is one of Python’s built-in unary functions like absolute values, negation, etc.
Invariants over x+y: any invariant from the list of invariants over a single numeric variable, such as (x+y) mod b = a
Invariants over x-y: as for x+y;

19. What Invariants to Infer?
Invariants over three numeric variables
Linear relationship, e.g., z = ax+by+c
Functions, e.g., z = fn(x, y) or x = fn(y) where fn is one of Python’s built-in binary functions like min, max, GCD, and, or, etc.
How about four variables and more?

20. What Invariants to Infer?
Invariants over a single sequence variable
Range: minimum and maximum sequence values, ordered lexicographically; for instance, this can indicate the range of string or array values
Element ordering: whether the elements of each sequence are non-decreasing, non-increasing, or equal
Invariants over all the sequence elements (treated as a single large collection)

21. What Invariants to Infer?
Invariants over two sequence variables
Linear relationship: y = ax + b, element-wise
Comparison: x < y, x <= y, x > y, x >= y, x = y, x != y, perform lexicographically
Subsequence relationship: x is a subsequence of y or vice versa
Reversal: x is the reverse of y
Invariants over a sequence and a numeric variable
Membership: i in s

22. What Invariants to Infer?
Derived variables
Derived from any sequence s
Length: size(s)
Extremal elements: s[0], s[1], s[size(s)-1], s[size(s)-2]
Derived from any numeric sequence s
sum: sum(s)
Minimum elements: min(s)
Maximum elements: max(s)
Derived from any sequence s and any numeric variable i
Element at the index: s[i], s[i-1]
Subsequences: s[0..i], s[0..i-1]
Derived from function invocations: number of calls so far

23. Algorithm
Collect samples at a program point (through instrumentation and testing)
For all variables, test every potential invariant (defined above)
Remove an invariant if it is violated by a sample.

24. Exercise 2 int inc(int *x, int y) { *x += y;
return *x; }
Given the program and the collected data, what are the invariants?

25. Filtering Invariants
Too many potentially invariants could discourage programmers from looking through them.
A better test suite could help.
Daikon filters invariants by computing an invariant confidence: assume a random input, what is the chance of the invariant would appear?

26. Invariant Confidence: Example
A range for numeric ranges like x in [ ] are reported only if the limits appear to be non-coincidental: if several values near the extremes all appear about as often as would be expected (assuming uniform distribution).

27. Invariant Confidence: Example
Suppose the reported value for variable x fall in a range of size r that includes 0
Suppose that x != 0 holds for all test cases
The probability of x != 0 is: (1-1/r)^n where n is the number of samples
If the probability is less than a user-defined confidence threshold, then x != 0 is reported.

28. Scalability Daikon’s invariant detection time is
Potentially cubic in the number of variables in scope at a program point (not the total number of variables in the program)
Linear in the number of samples (the number of times a program point is executed)
Linear in the number of instrumented program points.

29. Case Study: Invariant Stability
Warming:
One program!

30. Case Study: Invariant Stability
Conclusion:
Stable?

31. More Invariants, Better Programs?
Experiment setup
424 student programs from a single assignment for CSE 142 at University of Washington
The quality of the programs is measured by their scores.
Invariant detection was performed over 200 executions of each program, resulting in 3 to 28 invariants per program.
Conclusion: No co-relation

32. Discussion
For invariant generation, shall we use random test case generation or systematic test case generation?
How do we measure the usefulness of the generated invariants?
How do we test whether a generated invariant is really a loop invariant?
How do we identify the useful templates for invariants?
Can we discover disjunctive invariants?

33. Unbounded Symbolic execution for program Verification
Jaffar et al. RV’11
Unbounded Symbolic execution for program Verification

34. Motivation
Symbolic execution doesn’t handle loops well: path explosion
Loop invariants are essential to handle loops.
Idea: learn loop invariant through symbolic execution

35. Iterative Deepening
Step 1: execute path L0,1,4,5 symbolically x = 0 && //from L0 x >= n && //from L1 x < 0 //from L4 Interpolant at L4: x >= 0
L0 x = 0;
L1 while (x < n) {
L x++;
L3 }
L4 if (x < 0) {
L error();
L6 }

36. Iterative Deepening
Step 2: check if x >= 0 is a loop invariant by checking whether the following is satisfiable. x >= 0 && x < n && x1 = x+1 && x1 < 0 No! Thus x >= 0 is a loop invariant. Complete the proof with Hoare logic rules.
L0 x = 0;
L1 while (x < n) {
L x++;
L3 }
L4 if (x < 0) {
L error();
L6 }

37. Another Look Initially, L0 L0 x = 0; L1 while (x < n) { L2 x++;
L4 if (x < 0) {
L error();
L6 } L1
x<n
x>=n L2 L4
x<0 L3
error

38. Another Look With the loop invariant, L0 L0 x = 0;
L1 while (x < n) {
L x++;
L3 }
L4 if (x < 0) {
L error();
L6 } L1
x>=0
x<n
x>=n L2 L4
x<0 L3
error
This serves as a proof that error is not reachable.
Finding a loop invariant is to find this label at this a loop head!

39. Iterative Deepening L0 L0 lock=0;new=old+1 L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L1
new=old
new!=old L2 L6
lock=0 L3
error L4 L5
Is error happening?
What label shall we generate at L1?

40. Iterative Deepening
Step 1: execute path L0,1,6,7 symbolically lock=0&&new=old+1&& //from L0 new==old && //from L1 lock==0 //from L6 Interpolant at L6: lock!=0 Is lock!=0 an invariant during the loop?
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error();

41. Iterative Deepening Step 1: execute path L0,1,6,7 symbolically
lock=0&&new=old+1&& //from L0
new=old && //from L1
lock==0 //from L6
What is the interpolant at L1?
That is,
A is lock=0&&new=old+1
B is new=old&&lock=0
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error();

42. Ideal Case
The interpolant at L1 is new!=old || lock != 0 Exercise 3: Is this a loop invariant strong enough to prove that error is not possible?
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error();
Recall existing techniques only return conjunctive interpolants. The interpolant at L1 thus may be either new!=old or lock!=0, neither of which is a loop invariant.

43. Iterative Deepening
Step 2: execute path L0,1,2,3,5,1,6,7 symbolically lock=0&&new=old+1&& //from L0 new!=old && //from L1 lock1=1&old1=new && //from L2 new=old1&& //from L1 lock1==0 //from L6 Interpolant at L1?
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error();

44. Iterative Deepening
Step 2: execute path L0,1,2,3,4,5,1,6,7 symbolically lock=0&&new=old+1&& //from L0 new!=old && //from L1 lock1=1&old1=new && //from L2 lock2=0&new1=new+1 && //from L2 new=old1&& //from L1 lock1==0 //from L6 Interpolant at L1?
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error();
It doesn’t help to execute more iterations

45. Alternative Approach L0 lock=0;new=old+1 L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L0 L1
lock=0&&new=old+1 L2 L3
Assume there is a label Inv at L1 which is a loop invariant; The following is true.
lock=0&&new=old+1 => Inv
lock=1&&old=new => Inv L5
L1’
lock=1&&old=new

46. Alternative Approach L0 lock=0;new=old+1 L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L0 L1
lock=0&&new=old+1 L2
lock=0&&new=old+1
Ideally, we let Inv be
(lock=0&&new=old+1) ||
(lock=1&&old=new) ||
(lock=0&&new=old+1)
Exercise: check if Inv is indeed a loop invariant. L3 L4 L5
L1’ L5
L1’
lock=1&&old=new

47. Invariant Validation L0 L1 (lock=0&&new=old+1) || (lock=1&&old=new)
error L4
Since it is a loop invariant, we can label L1 now. Is it strong enough? L5

48. An Ideal Algorithm
Identify paths which end at the loop head for the first time.
Test if the disjunction of the path conditions is a loop invariant strong enough for the proof
If positive, terminate
Otherwise, identify paths which end at the loop head for the second time. …

49. Discussion First time: i = 0; Second time: i = 1; int i = 0;
Third time: i = 2; …
How do we make the jump to i <= 1000?
int i = 0;
while (i < 1000) {
i++; }

50. Another Look at Daikon L0
{(lock=0,old=*, new=*+1), (lock=1,old=*+1, new=*+1), …} L1
new=old
Pre-defined
abstraction
new!=old L2 L6
lock=0
lock=0
new=old
new=old+1 L3
error L4 L5
Can Daikon find the right invariant in this case?

51. New Approach: USE Step 1: execute symbolically L0 L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L1 L6 L7

52. New Approach: USE Step 2: Compute interpolant L0 L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L1 L6
lock!=0 L7

53. New Approach: USE Step 3: Label loop head L0 L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L1
{lock=0, new=old+1} L6
lock!=0 L7

54. New Approach: USE
Step 4: abstract loop head labels based on the new condition.
The loop head L1 is visited with a different path with a new condition.
Abstract the labels on L1 so that it is implied by the new condition. L0 L1
lock=0&&new=old+1 L2 L3 L5
L1’
lock=1&&old=new

55. New Approach: USE
Step 4: abstract loop head labels based on the new condition.
Remove labels at L1 until the conjunction of the remaining labels is implied by the new condition L0 L1
lock=0&&new=old+1
true L2 L3 L5
Do we need to continue from L1’ given now it is stronger than an ancestor L1?
L1’
lock=1&&old=new

56. New Approach: USE
Step 5: execute symbolically Since lock=0&&new=old+1 (at L1’) implies true (at L1). We stop. L0 L1
true L2 L3
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L4 L5
L1’
lock=0&&new=old+1

57. USE: First AbstractionL0 L1
true
Is this abstraction safe or not? It is safe iff error is not reachable if it is not reachable based on this abstraction.
new=old
new!=old L2 L6
lock=0 L3
error
Is error reachable or not based on this abstraction? L4 L5

58. USE: Checking
Run DFS/BFS algorithm on this graph shows that error is reachable. L0 -> L1 -> L6 -> error A counterexample based on the abstraction might not be a real counterexample! L0 L1
true
new=old
new!=old L2 L6
lock=0 L3
error L4 L5

59. USE: Spuriousness Checking
Run DFS/BFS algorithm on this graph shows that error is reachable. L0 -> L1 -> L6 -> error Symbolically execute the above path and conclude that it is spurious. L0 L1
true
new=old
new!=old L2 L6
lock=0 L3
error L4 L5
Why it is spurious?

60. USE: Refinement The path L0,L1,L6,error is spurious.
One (or more) loop head in this path must be too abstract.
Find an interpolant at the loop head (L1) L0 L1
true
new=old
new!=old L2 L6
lock=0 L3
error L4 L5
lock=0&&new=old+1&& new=old && lock=0
Assume the interpolant found at L1 is: new!=old

61. USE: Refinement The path L0,L1,L6,error is spurious.
One (or more) loop head in this path must be too abstract.
Find an interpolant at the loop head (L1) L0
new!=old
new=old L6
lock=0
error

62. USE: Re-explore
Since the label at L1 has changed, we need to re-explore. This time, we can’t remove the label at L1. We continue instead. L0 L1
new!=old L2 L3 L5
L1’
lock=1&&old=new

63. USE: Re-explore
Continue with L6, symbolic execution proves that it is not possible. L0 L1
new!=old L2
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L3 L5
L1’
lock=1&&old=new L6

64. USE: Re-explore
Continue with L6, symbolic execution proves that it is not possible. L0 L1
new!=old L2
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L3 L5
L1’
lock=1&&old=new
We can’t go further since lock==1. L6

65. USE: Re-explore
Backtrack to L1’ and continue with L2, symbolic execution shows it is not feasible. L0 L1
new!=old L2
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L3 L5
L1’
lock=1&&old=new
L2’
We can’t go further since old==new.

66. USE: Re-Explore
Backtrack to L3, continue with L4,L5,L1. We can stop at L1’ because lock=0&&new=old+1 implies new!=old. L0 L1
new!=old L2
L0 lock=0;new=old+1
L1 while (new!=old) {
L lock=1;old=new;
L if (*) {
L4 lock=0;new++;}
L5 };
L6 if (lock==0)
L error(); L3 L4 L5
L1’
lock=0&&new=old+1

67. Recap: the USE ApproachL0 L1
new!=old
new!=old L2 L3 L5 L4 L1 L5 L6 L2 L1
subsumed by new!=old

68. Recap: the USE Approach
This approach acknowledges the difficulty in finding (disjunctive) loop invariants and compensates it with a combination of state space exploring and abstraction-refinement.

69. Case Study
Iterative Deepening
New Approach

70. Exercise 4 The path L0,L1,L6,error is spurious.
One (or more) loop head in this path must be too abstract.
Find an interpolant at the loop head (L1) L0 L1
new=old
new!=old L2 L6
lock=0 L3
error L4 L5
lock=0&&new=old+1&& new=old && lock=0
What if the interpolant at L1 is: new=old+1?