1. Proofs and Counterexamples Rupak Majumdar

2. In the good old days… Decision Procedure  yes no

3. Now… Sooh, short answer: yes with an if, long answer, no with a but. –Reverend Lovejoy Decision Procedure  yes no Interpolant  +,  - Model Predicate Abstraction Boolean SAT Pre- Process Proof Invariant Generation

4. It’s the Application! Validating large scale software w.r.t. partial specifications Rules in documentation –Order of operations & data access –Resource management –Incomplete, unenforced, wordy Violated rules ) bad behavior –System crash or deadlock –Unexpected exceptions –Failed runtime checks

5. Property: Double Locking “An attempt to re-acquire an acquired lock or release a released lock will cause a deadlock.” Calls to lock and unlock must alternate. lock unlock

6. Example Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } lock unlock Does program Example satisfy the locking spec?

7. Ok, but what does this have to do with Decision Procedure  yes no Decision Procedure  yes no Interpolant  +,  - Model Predicate Abstraction Boolean SAT Pre- Process Proof Invariant Generation vs

8. Automatic Verification of Programs Search Refine Program Safe Trace Yes No Property [POPL 02] [POPL 04] Automatic Tool Model and analyze program behavior in the language of logic, reduce reasoning about programs to reasoning about logic

9. What a program really is… State Transition 3: unlock(); new++; 4:} … 3: unlock(); new++; 4:} … pc lock old new q  3   5  0x133a pc lock old new q  4   5  6  0x133a Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return;} Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return;}

10. The Safety Verification Problem Initial Error Is there a path from an initial to an error state ? Problem: Infinite state graph Solution : Set of states ' logical formula Safe

11. Representing States as Formulas [F] states satisfying F {s | s ² F } F FO fmla over prog. vars [F 1 ] Å [F 2 ]F1 Æ F2F1 Æ F2 [F 1 ] [ [F 2 ]F 1 Ç F 2 [F][F] : F [F 1 ] µ [F 2 ]F 1 implies F 2 i.e. F 1 Æ: F 2 unsatisfiable

12. Idea 1: Symbolic Analysis Initial Error Program State Space 1.Given F, and a program operation, describe a formula for the set of successor states “Strongest postcondition” 2. Iterate the SP, checking at each stage if Error is hit, or if new states are added

13. Idea 1: Symbolic Analysis Iterate the SP, checking at each stage if Error is hit, or if new states are added CurrentReach Æ Error is unsat OldReach Ç SP(OldReach)  CurrentReach Problem: Too many details tracked Symbolic Iteration may not terminate

14. Idea 1’: Symbolic Analysis + Annotations Ask programmer for Invariant Check Invariant does not hit Error, and Invariant is closed under SP Invariant Æ Error is unsat Invariant Ç SP(Invariant)  Invariant No more iterations: Programmer provides an inductive assertion, just make two queries Problem: Increased work for the programmer

15. Idea 2: Predicate Abstraction Initial Error Program State Space Abstraction: Predicates on program state –Signs: x > 0 –Aliasing:&x  &y States satisfying the same preds are equivalent –Merged into single abstract state Abstraction [Graf-Saidi 97]

16. Example: Predicate Abstraction For the red region above, the predicate abstraction is the set of “boxes” that cover the red region For each small box, ask the theorem prover if Æ is satisfiable

17. Search finite state space symbolically Conservative – Abstraction safe ) System safe – Too coarse ) spurious counterexample Abstract Idea 2: Predicate Abstraction

18. BUG 1. Feasible ) BUG Abstract Counterexamples: Analyze counterexamples to check feasibility Idea 3: Counterexample Analysis Get satisfying assignment from decision procedure: test demonstrating bug

19. BUG 1. Feasible ) BUG 2. Infeasible ) What predicates distinguish states across cut? 3. Build refined abstraction Abstract Counterexamples: Analyze counterexamples to check feasibility Idea 3: Counterexample Analysis

20. Idea 3: Counterex.-Guided Refinement Abstract Refine Add predicates to rule out spurious trace Repeat reachability –Till safe or real trace is found –Or go on refining forever [Kurshan et al. 93] [Clarke et al. 00] [Ball-Rajamani 01]

21. safe Abstract Refine Idea 3: Counterex.-Guided Refinement Add predicates to rule out spurious trace Repeat reachability –Till safe or real trace is found –Or go on refining forever

22. Problem: Abstraction is Expensive Reachable Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Observe Fraction of state space reachable #Preds ~ 100’s, #States ~ 2 100, #Reach ~ 1000’s

23. Solution1 : Only Abstract Reachable States Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries

24. Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Solution1 : Only Abstract Reachable States

25. Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Solution1 : Only Abstract Reachable States

26. Safe Solution Build abstraction on-the-fly, during search Problem #abstract states = 2 #predicates Exponential Thm. Prover queries Solution1 : Only Abstract Reachable States

27. Key Idea: Reachability Tree 5 1 2 3 4 3 Unroll Abstraction 1. Pick tree-node (=abs. state) 2. Add children (=abs. successors) 3. On re-visiting abs. state, cut-off Counterexample Analysis -Learn new predicates locally -Rebuild subtree with new predicates “Lazy Abstraction” Initial [HenzingerJhalaM.Sutre02]

28. Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 Reachability Tree

29. Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 lock() old = new q=q->next LOCK 2 2 Reachability Tree

30. Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK 2 2 [q!=NULL] 3 3 Reachability Tree

31. Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK 2 2 3 3 q ->data = new unlock() new++ 4 4 : LOCK Reachability Tree

32. Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK 2 2 3 3 4 4 : LOCK [new==old] 5 5 Reachability Tree

33. Build-and-Search Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK 2 2 3 3 4 4 : LOCK 5 5 unlock() : LOCK Reachability Tree

34. Analyze Counterexample Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK 2 2 3 3 4 4 : LOCK 5 5 Reachability Tree lock() old = new q=q->next [q!=NULL] q ->data = new unlock() new++ [new==old] unlock()

35. Analyze Counterexample Predicates: LOCK : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK 2 2 3 3 4 4 : LOCK 5 5 [new==old] new++ old = new Inconsistent new == old Reachability Tree

36. Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 Reachability Tree

37. Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old 2 2 lock() old = new q=q->next Reachability Tree

38. Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old 2 2 3 3 4 4 q ->data = new unlock() new++ : LOCK, : new = old Reachability Tree

39. Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old 2 2 3 3 4 4 : LOCK, : new = old [new==old] Reachability Tree

40. Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 LOCK, new==old 2 2 3 3 4 4 : LOCK, : new = old : LOCK, : new == old 1 [new!=old] 4 Reachability Tree

41. Repeat Build-and-Search Predicates: LOCK, new==old : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 2 2 3 3 4 4 1 4 LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE Reachability Tree LOCK, new==old : LOCK, : new = old : LOCK, : new == old

42. BLAST Abstract Refine C Program Safe Trace Yes No Property [HenzingerJhalaM.Sutre02]

43. Scaling BLAST Abstract Refine C Program Safe Trace Yes No Property To find: 1. Which predicates to track? 2.Where to track them? Key Problem: Find good predicates

44. Tracking lock not enough #Predicates grows with program size Problem: p 1,…,p n needed for verification Exponential reachable abstract states while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } TFTTFT

45. #Predicates grows with program size Problem: p 1,…,p n needed for verification Exponential reachable abstract states while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } : LOCK : LOCK, p 1 p1p2p1p2 p 1 : p 2 : p 1 p 2 : p 1 : p 2 LOCK, p 1 : LOCK, : p 1 2 n Abstract States : LOCK

46. Predicates useful locally while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } while(1){ 1: if (p 1 ) lock() ; if (p 1 ) unlock() ; … 2: if (p 2 ) lock() ; if (p 2 ) unlock() ; … n: if (p n ) lock() ; if (p n ) unlock() ; } : LOCK : LOCK, p 1 LOCK, p 1 : LOCK, : p 1 2n Abstract States p1p1 p2p2 pnpn : LOCK LOCK, p 2 : LOCK, : p 2 : LOCK Solution: Use predicates only where needed

47. Predicates: LOCK, new==old Refinement Main Questions : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 2 2 3 3 4 4 1 4 LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE LOCK, new==old : LOCK, : new = old : LOCK, : new == old Q. How to find good predicates ? Where to track each predicate?

48. Counterexample Traces lock() old = new q=q->next [q!=NULL] q ->data = new unlock() new++ [new==old] unlock() TraceSSA Trace lock 1 = 1 old 1 = new 0 q 1 = q 0 - >next assume(q 1 != NULL) (q 1 -> data ) 1 = new 0 lock 2 = 0 new 1 = new 0 +1 assume(new 1 =old 1 ) assert(lock 2 =1) Trace Feasibility Formula Thm: Trace is feasible, TF is satisfiable lock 1 = 1 Æ old 1 = new 0 Æ q 1 = q 0 - >next Æ q 1 != NULL Æ (q 1 -> data ) 1 = new 0 Æ lock 2 = 0 Æ new 1 = new 0 +1 Æ new 1 =old 1

49. Predicates: old=new, new=new+1, new=old Proof of Unsatisfiability old 1 = new 0 new 1 = new 0 + 1 new 1 = old 1 +1 new 1 = old 1 ; Proof of Unsatisfiability lock 1 = 1 Æ old 1 = new 0 Æ q 1 = q 0 - >next Æ q 1 != NULL Æ (q 1 -> data ) 1 = new 0 Æ lock 2 = 0 Æ new 1 = new 0 +1 Æ new 1 =old 1 Trace Feasibility Formula Add: old=new [HenzingerJhalaM.Sutre02]

50. Decision Procedure Must Produce Proofs Decision Procedure  yes no Boolean SAT Sat asgn Proof Proofs important in other applications as well: e.g., Proof carrying code [Necula97,HenzingerJhalaM.NeculaSutreWeimer02]

51. Counterexample Traces: Take 2 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) y = x +1 1: x = ctr; 2: ctr = ctr + 1; 3: y = ctr; 4: if (x = i-1){ 5: if (y != i){ ERROR: } } 1: x = ctr; 2: ctr = ctr + 1; 3: y = ctr; 4: if (x = i-1){ 5: if (y != i){ ERROR: } }

52. Trace Formulas 1: x = ctr 2: ctr = ctr+1 3: y = ctr 4: assume(x=i-1) 5: assume(y  i) Trace SSA Trace x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 1: x 1 = ctr 0 2: ctr 1 = ctr 0 +1 3: y 1 = ctr 1 4: assume(x 1 =i 0 -1) 5: assume(y 1  i 0 ) Trace Feasibility Formula

53. Proof of Unsatisfiability Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 x 1 = ctr 0 x 1 = i 0 -1 ctr 0 = i 0 -1 ctr 1 = ctr 0 +1 ctr 1 = i 0 y 1 = ctr 1 y 1 = i 0 y 1  i 0 ; Proof of Unsatisfiability

54. The Present State… 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace … is all the information the executing program has here 1. … after executing trace past (prefix) 2. … knows present values of variables 3. … makes trace future (suffix) infeasible State… At pc 4, which predicate on present state shows infeasibility of future ?

55. What Predicate is needed ? Trace 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0

56. What Predicate is needed ? Trace 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 1. … after executing trace prefix Relevant Information … implied by TF prefix Predicate …

57. 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) 1. … after executing trace prefix 2. … has present values of variables What Predicate is needed ? Trace Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 … implied by TF prefix … on common variables Predicate … x1x1 x1x1 Relevant Information y1y1 y1y1

58. 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) What Predicate is needed ? Trace Trace Formula (TF) x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 Predicate … 1. … after executing trace prefix 2. … has present values of variables 3. … makes trace suffix infeasible … implied by TF prefix … on common variables … & TF suffix is unsatisfiable Relevant Information

59. Interpolant = Predicate ! -- ++ Interpolate  Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 y 1 = x 1 + 1 Predicate … … implied by TF prefix … on common variables … & TF suffix is unsatisfiable Craig Interpolant [Craig 57] Computable from Proof of Unsat 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Trace Predicate at 4: y= x+1

60. Another interpretation … -- ++ Interpolate  Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 y 1 = x 1 + 1 Unsat = Empty Intersection = Trace Infeasible Predicate at 4: y= x+1 -- ++ After exec prefix Can exec suffix  Interpolant  = Overapproximation of states after prefix that cannot execute suffix

61. Predicates: LOCK, new==old Refinement Main Questions : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 2 2 3 3 4 4 1 4 LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE LOCK, new==old : LOCK, : new = old : LOCK, : new == old Q. How to find good predicates ? Where to track each predicate? Q: How to compute interpolants? (And do they always exist?)

62. Another Proof of Unsatisfiability x 1 – ctr 0 =0 x 1 -i 0 +1=0 ctr 0 -i 0 +1=0 ctr 1 - ctr 0 -1=0 ctr 1 -i 0 =0 y 1 -ctr 1 =0 y 1 -i 0 =0 y 1 -i 0  0 0  0 Rewritten Proof £ 1 £ (-1) £ 1 x 1 = ctr 0 x 1 = i 0 -1 ctr 0 = i 0 -1 ctr 1 = ctr 0 +1 ctr 1 = i 0 y 1 = ctr 1 y 1 = i 0 y 1  i 0 ; Proof of Unsatisfiability

63. Interpolant from Rewritten Proof ? Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 x 1 – ctr 0 =0 x 1 -i 0 +1=0 ctr 0 -i 0 +1=0 ctr 1 - ctr 0 -1=0 ctr 1 -i 0 =0 y 1 -ctr 1 =0 y 1 -i 0 =0 y 1 -i 0  0 0  0 Rewritten Proof £ 1 £ (-1) £ 1 Interpolate

64. Interpolant from Rewritten Proof ? Trace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 x 1 – ctr 0 =0 ctr 1 - ctr 0 -1=0 y 1 -ctr 1 =0 Interpolant ! £ (-1) £ 1 Interpolate y 1 -x 1 -1=0 y 1 =x 1 +1

65. Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i -- ++ x 1 = ctr 0 Predicate Map 2: x= ctr 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Interpolate

66. Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 -- ++ 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Predicate Map 2: x = ctr 3: x= ctr-1 x 1 = ctr 1 -1 Interpolate Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i

67. Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Predicate Map 2: x = ctr 3: x= ctr - 1 4: y= x + 1 y 1 = x 1 +1 -- ++ Interpolate Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i

68. Building Predicate Maps TraceTrace Formula x 1 = ctr 0 Æ ctr 1 = ctr 0 + 1 Æ y 1 = ctr 1 Æ x 1 = i 0 - 1 Æ y 1  i 0 1: x = ctr 2: ctr = ctr + 1 3: y = ctr 4: assume(x = i-1) 5: assume(y  i) Predicate Map 2: x = ctr 3: x= ctr - 1 4: y= x + 1 5: y = i y 1 = i 0 -- ++ Interpolate Cut + Interpolate at each point Pred. Map: pc i  Interpolant from cut i

69. Local Predicate Use Predicate Map 2: x = ctr 3: x= ctr - 1 4: y= x + 1 5: y = i Use predicates needed at location #Preds. grows with program size #Preds per location small Local Predicate use Ex: 2n states Global Predicate use Ex: 2 n states Verification scales …

70. pc 1 : x 1 = 3 pc 2 : assume (x 1 >0) pc 3 : x 3 = f 1 (x 1 ) pc 4 : y 2 = y 1 pc 5 : y 3 = f 2 (y 2 ) pc 6 : z 2 = z 1 +1 pc 7 : z 3 = 2*z 2 pc 8 : return z 3 pc 9 : return y 3 pc 10 : x 4 = x 3 +1 pc 11 : x 5 = f 3 (x 4 ) pc 12 : assume(w 1 <5) pc 13 : return w 1 pc 14 : assume x 4 >5 pc 15 : assume (x 1 =x 3 +2) Traces with Procedure Calls Trace Formula i Trace i pc 1 : x 1 = 3 pc 2 : assume (x 1 >0) pc 3 : x 3 = f 1 (x 1 ) pc 4 : y 2 = y 1 pc 5 : y 3 = f 2 (y 2 ) pc 6 : z 2 = z 1 +1 pc 7 : z 3 = 2*z 2 pc 8 : return z 3 pc 9 : return y 3 pc 10 : x 4 = x 3 +1 pc 11 : x 5 = f 3 (x 4 ) pc 12 : assume(w 1 <5) pc 13 : return w 1 pc 14 : assume x 4 >5 pc 15 : assume(x 1 =x 3 +2)

71. Interprocedural Analysis Trace Formula i Trace i Require at each point i : Scoped predicates YES: Variables visible at i NO: Caller’s local variables Find predicate needed at point i YES NO

72. Problems with Cutting Trace Formula i Trace i -- ++ Caller variables common to  - and  + Unsuitable interpolant: not well-scoped

73. Scoped Cuts Trace Formula i Call begins Trace i

74. Scoped Cuts -- ++ Trace Formula i Call begins Trace i Predicate at pc i = Interpolant from cut i

75. Common Variables Formals Current locals Trace Formula Predicate at pc i = Interpolant from i-cut i Trace i -- ++ Common Variables Formals Well-scoped

76. Question: When Do Interpolants Exist? Craig’s Theorem guarantees existence for first order logic But we are interpreting formulas over theories (arithmetic, theories of data structures)

77. The Good News Interpolants always exist for recursively enumerable theories –The proof is a simple application of compactness So: interpolants exist for Presburger arithmetic, sets with cardinality constraints, theory of lists, (quantifier-free) theory of arrays, multisets, …

78. The Bad News “The proof is a simple application of compactness” –May be algorithmically inefficient –Daunting engineering task to construct interpolating decision procedure for each individual theory

79. An Alternate Path: Reduction Want to compile formulas in a new theory to formulas in an old theory such that interpolation in the old theory imply interpolation in the new theory T reduces to R: can compile formulas in theory T to formulas in theory R –And use decision procedures for R to answer decision questions for T Technically: Given theories T and R, with R µ T, a reduction is a computable map  from T formulas to R formulas such that for any T-formula  : –  and  (  ) are T-equivalent –  is T-satisfiable iff  (  ) is R-satisfiable

80. Example: Theory of Sets Theory of sets reduces to theory of equality with uninterpreted functions x = y 8 e. e 2 x, e 2 y x= ;8 e.e  x x=U 8 e.e 2 x x={e}e 2 x Æ 8 e’.e’ 2 x ) e = e’ x=y [ z 8 e.e 2 x, e 2 y Ç e 2 z x=y Å z 8 e.e 2 x, e 2 y Æ e 2 z

81. Example: Theory of Multisets Theory of multisets reduces to the combination theory of equality with uninterpreted functions and linear arithmetic x = y 8 e. count(x,e) = count(y,e) x= ;8 e.count(x,e) = 0 x=[(e,n)]count(x,e)=max(0,n) Æ 8 e’.e’  e ) count(x,e’)=0 x=y ] z 8 e.count(x,e)= count(y,e)+count(z,e) x=y [ z 8 e. count(x,e) = max(count(y,e), count(z,e)) x=y Å z 8 e. count(x,e) = min(count(y,e), count(z,e))

82. Reduction and Interpolation  - and  + in Theory T  - and  + in Theory R Interpolant  in Theory R as well as T Quantifier-free interpolant Reduction from T to R Interpolate in R Eliminate quantifiers in T or R

83. Reduction Theorem Interpolants for the theory of arrays, sets, and multisets can be computed by reduction to the combination theory of linear arithmetic and equality with uninterpreted functions –We already have interpolating decision procedures for this latter theory [KapurM.Zarba06]

84. Lazy Abstraction Abstract Refine C Program Safe Trace Yes No Property Refine Trace Feas Formula Thm Pvr Proof of Unsat Pred. Map PC  Preds. Ctrex. Trace Interpolate Solution: Localize pred. use, find where preds. needed Problem: #Preds grows w/ Program Size

85. So far … Lazy Abstraction Predicates: –Abstract infinite program states Counterexample-guided Refinement: –Find predicates tailored to prog, property 1.Abstraction : Expensive Reachability Tree 2.Refinement : Find predicates, use locations Proof of unsat of TF + Interpolation

86. What the “decision procedure” must provide Decision Procedure  yes no Interpolant  +,  - Boolean SAT Sat asgn Proof

87. Predicates: LOCK, new==old Refinement : LOCK Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4:}while(new != old); 5: unlock (); } 1 1 2 2 3 3 4 4 1 4 LOCK, new=old 4 4 : LOCK, new==old 5 5 SAFE LOCK, new==old : LOCK, : new = old : LOCK, : new == old So how well does all this work? Quite well, if the program and property are control-dominated Not so well when data is involved…

88. Refinement Failure: Unrolling Loops x = 0; y = 50; while ( x<100 ) { if ( x>=50 ) y = y+1; x = x+1; } assert( y==100 ); counterexample: x=0; y=50; x>=100; y==100 refinement: x==0 counterexample: x=0; y=50; x =100; y==100 refinement: x==1 counterexample: x=0; y=50; x<100; x=x+1; x<100; x=x+1; x>=100; y==100 refinement: x==2...

89. Refinement Failure: Unfolding Arrays for (i=0; i<n; i++) { a[i]=i; } for (j=0; j<n; j++) { assert( a[j]==j ); } counterexample: i=0; i =n; j=0; j<n; a[j]!=j refinement: a[0]==0 counterexample: i=0; i =n; j=0; j<n; a[j]==j; j++; j<n; a[j]!=j refinement: a[1]==1...

90. What went Wrong? Consider all unrolled counterexamples at once –Convergence of abstraction discovery Inspect families of counterexamples of unbounded length –Justification for unbounded universal quantification Looking at one counterexample path at a time is too weak [JhalaMcMillan05,JhalaMcMillan06]

91. Path Programs Treat counterexamples as programs –“Close” the loops A B C B D stmt1 stmt2 stmt3 stmt4 A B C D stmt1 stmt2 stmt3 stmt4 counterexample path path program, contains loops, etc [BeyerHenzingerM.Rybalchenko07]

92. Meaning of Path Programs Path program ´ (Possibly unbounded) sets of counterexamples: Unbounded counterexamples Property-determined fragment of original program –Can be analyzed independently to find good abstractions

93. Path Invariants Invariant for path programs ´ path invariant Abstraction refinement using path invariants –Elimination of all counterexamples within path program –Justification for unbounded quantification

94. Invariant Generation Given a path program, with a designated error location, find an invariant that demonstrates error is not reachable –Can scale: Reduced obligation to program fragment –Outer model checking loop integrates path invariants into program invariant Can use any technique We use constraint-based invariant generation [SankaranarayananSipmaManna04,BeyerHenzingerM.Rybalchenko07]

95. What we need Decision Procedure  yes no Interpolant  +,  - Model Boolean SAT Pre- Process Proof Invariant Generation

96. Summary Traditional distinctions between theorem proving, model checking, and dataflow analysis are getting blurred Is Blast a state space exploration engine with a theorem proving subroutine? A theorem prover with a model checking tactic? An abstract interpreter with a rich and variable domain? It is all of the above, and none of the above –The distinctions are mostly of historical interest –New algorithms are re-shaping the way we think about, and build, tools

97. More generally… What does it take to get verified software? –Blast is a small part in the whole process –How can we make the programmer’s job easier, through a combination of Language support Process support Tool support Debugging and visualization support? –How can we write software so that it is verifiable?

98. Acknowledgments Dirk Beyer Tom Henzinger Ranjit Jhala Deepak Kapur Ken McMillan Andrey Rybalchenko Gregoire Sutre Calogero Zarba

99. Trace Formula TraceTrace Formula m1: assume(flag!=0) m2: y=inc(x,flag) i1: assume(sign!=0) i2: rv = a+1 i4: return rv m3: assume(y<=x) Æ flag 0  0 Æ a 0 =x 0 Æ sign 0 = flag 0 Æ sign 0  0 Æ rv 0 =a 0 +1 Æ y 0 =rv 0 Æ y 0 <= x 0 i -- ++ i rv 0 a0a0 a0a0 rv 0 > a 0 Interpolate

100. partition(int *a, int n) { int i, gelen, ltlen; int ge[n], lt[n]; gelen = 0; ltlen = 0; L1: for(i = 0; i < n; i++) { if ( a[i] >= 0 ) { L2: ge[gelen] = a[i]; gelen++; } else { L3: lt[ltlen] = a[i]; ltlen++; } } L4: for(i = 0; i < gelen; i++) { assert( ge[i] >= 0 ); } L5: for(i = 0; i < ltlen; i++) { assert( lt[i] < 0 ); } } Example: quantified path invariant Counterexample L1,L2,L1,L4,Err Path invariant: L1 : 8 0 · k < gelen ! ge[k] ¸ 0 L4 : 8 i · k < gelen ! ge[k] ¸ 0 Counterexample L1,L3,L1,L4,L5,Err Path invariant: L1 : 8 0 · k < ltlen ! lt[k] < 0 L4 : 8 0 · k < ltlen ! lt[k] < 0 L5 : 8 i · k < ltlen ! lt[k] < 0

101. Verification by Theorem Proving 1. Loop Invariants 2. Logical formula 3. Check Validity Invariant: lock Æ new = old Ç : lock Æ new  old Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; }

102. Verification by Theorem Proving 1. Loop Invariants 2. Logical formula 3. Check Validity - Loop Invariants - Multithreaded Programs + Behaviors encoded in logic + Decision Procedures - Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Precise [ESC]

103. Verification by Program Analysis 1. Dataflow Facts 2. Constraint System 3. Solve constraints Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } - Imprecision due to fixed facts + Abstraction + Type/Flow Analyses Scalable [CQUAL, ESP, MC]

104. Verification by Model Checking 1. (Finite State) Program 2. State Transition Graph 3. Reachability Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } Example ( ) { 1: do{ lock(); old = new; q = q->next; 2: if (q != NULL){ 3: q->data = new; unlock(); new ++; } 4: } while(new != old); 5: unlock (); return; } - Pgm ! Finite state model -State explosion + State Exploration + Counterexamples Precise [SPIN, SMV, Bandera,JPF ]

105. Combining Strengths Theorem Proving - loop invariants + Behaviors encoded in logic Refine + Theorem provers Computing Successors,Refine Program Analysis - Imprecise + Abstraction Shrink state space Model Checking - Finite-state model, state explosion + State Space Exploration Path Sensitive Analysis + Counterexamples Finding Relevant Facts Lazy Abstraction