Program Verification as Probabilistic Inference Sumit Gulwani Nebojsa Jojic Microsoft Research, Redmond

1 Problem of Program Verification Given a program with a pre/post-condition pair, discover proof of validity or invalidity. Proof is in the form of an invariant at each program point that can be locally verified.

2 Example 1 y := 50; y = 100 False x := x +1; y := y +1; x < 50 x <100 True False 11 22 33 44 55 66 77 88 x = 0 Prog. Point Invariant  entry x=0x=0 11 x = 0 Æ y = 50 22 x · 50 ) y = 50 Æ 50 · x ) x = y Æ x · 100 33 x · 50 ) y = 50 Æ 50 · x ) x = y Æ x <100 44 x <50 Æ y = 50 55 x · 50 Æ y = 50 66 50 · x <100 Æ x = y 77 50< x · 100 Æ x = y 88 x · 50 ) y = 50 Æ 50 · x ) x = y Æ x · 100  exit y = 100 Proof of Validity  entry  exit

3 Machine Learning Algorithm for Program Verification Initialize invariants at all program points to any element (from an abstract domain over which the proof exists) Pick a program point (randomly) whose invariant is locally inconsistent & update it to make it less inconsistent.

4 Outline  Inconsistency Measure Algorithm Experiments

5 Consistency of an invariant I at program point  I is consistent at  iff Post(  ) ) I Æ I ) Pre(  ) Post(  ) is the strongest postcondition of “the invariants at the predecessors of  ” at  Pre(  ) is the weakest precondition of “the invariants at the successors of  ” at  Example: I Q  P R c 11   Post(  2 ) = StrongestPost(P,s) Pre(  2 ) = (c ) Q) Æ ( : c ) R) s

6 Measuring Inconsistency of an invariant I at  Local inconsistency of invariant I at program point  = IM(Post(  ), I) + IM(I, Pre(  )) Where the inconsistency measure IM(  1,  2 ) is some approximation of the number of program states that violate  1 )  2

7 Example of an inconsistency measure IM Consider the abstract domain of Boolean formulas (with the usual implication as the partial order). Let  1 ´ a 1 Ç … Ç a n in DNF and  2 ´ b 1 Æ … Æ b m in CNF IM(  1,  2 ) =  (a i,b j ) where  (a i,b j ) = 0, if a i ) b j = 1, otherwise

8 Outline Inconsistency Measure & Penalty Function  Algorithm Experiments

9 Algorithm Search for proof of validity and invalidity in parallel. Same algorithm with different boundary conditions. Proof of Validity –I exit = Postcondition –I entry = Precondition Proof of Invalidity –I exit = : Postcondition –I entry ) Precondition, and I entry is satisfiable –This assumes that program terminates on all inputs.

10 Algorithm (Continued) Initialize invariant I j at program point  j to any element (from an abstract domain over which the proof exists) While invariant at some point is locally inconsistent: –Choose j randomly s.t. I j is inconsistent at  j –Update I j s.t. inconsistency of I j at  j is minimized [Sandwich Step] More precisely, I j is chosen randomly with probability inversely proportional to its inconsistency at  j (to avoid getting stuck in a local minima). But now, termination is only probabilistic.

11 Comparison with Interpolants Interpolant Given  1,  2 such that  1 )  2, find  such that:  1 )  )  2 Vars(  ) µ Vars(  1 ) Å Vars(  2 ) Sandwich Step Given  1,  2, find  such that: IM(  1,  ) + IM( ,  2 ) is minimum (i.e., # of states violating  1 )  )  2 is minimum)  is from a given abstract domain

12 Intersection of Forward & Backward Analysis y := 50; y = 100 False x := x +1; y := y +1; x < 50 x <100 True False 11 22 33 44 55 66 77 88 x = 0 Prog. Point Invariant 22 x <100 Ç y = 100 55 x ¸ 0 Æ x · 50 Æ y = 50 77 x ¸ 51 Æ x · 100 Æ x = y 88 - Assume abstract elements can have at most 3 conjuncts. Post(  8 ): x ¸ 0 Æ x · 100 Æ ( x · 50 Ç x = y ) Æ ( y = 50 Ç x ¸ 51). Dropping any conjunct is a valid choice at  8 in a forward analysis. -But backward guidance from  2 calls for keeping x · 100 and ( x · 50 Ç x = y )

13 Outline Inconsistency Measure & Penalty Function Algorithm  Experiments

14 Example 1 y := 50; y = 100 False x := x +1; y := y +1; x < 50 x <100 True False 11 22 33 44 55 66 77 88 x = 0 Prog. Point Invariant  entry x=0x=0 11 x = 0 Æ y = 50 22 x · 50 ) y = 50 Æ 50 · x ) x = y Æ x · 100 33 x · 50 ) y = 50 Æ 50 · x ) x = y Æ x <100 44 x <50 Æ y = 50 55 x · 50 Æ y = 50 66 50 · x <100 Æ x = y 77 50< x · 100 Æ x = y 88 x · 50 ) y = 50 Æ 50 · x ) x = y Æ x · 100  exit y = 100 Proof of Validity  entry  exit

15 Stats: Proof vs Incremental Proof of Validity Black: Proof of Validity Grey: Incremental Proof of Validity Incremental proof requires fewer updates

16 Stats: Different Sizes of Boolean Formulas Grey: 5*3, Black: 4*3, White: 3*2 n*m denotes n conjuncts & m disjuncts Larger size requires fewer updates

17 Example 2 x := 0; m := 0; n · 0 Ç 0 · m < n False m := x ; x := x +1; * x < n True 11 22 33 44 66 55 77 88 true Prog. Point Invariant  entry true 11 x=0 Æ m=0x=0 Æ m=0 22 n · 0 Ç (0 · x Æ 0 · m < n ) 33 n · 0 Ç (0 · x < n Æ 0 · m < n ) 44 55 66 77 88 n · 0 Ç (0 · x · n Æ 0 · m < n )  exit n · 0 Ç (0 · m < n ) Proof of Validity  entry  exit

18 Stats: Proof of Validity Example 2 is “easier” than Example 1. Easier example requires fewer updates.

19 Related Work: Probabilistic Techniques Used successfully in several areas of computer science. Yields more efficient, precise, even simpler algorithms. An earlier technique: Random Interpretation [POPL ’03-’05] –Discovers program invariants –Monte Carlo Algorithm: May generate invalid invariants with a small probability. Running time is bounded. –“Random Testing” + “Abstract Interpretation” This talk: Machine Learning –Discovers proof of validity/invalidity of a Hoare triple. –Las Vegas Algorithm: Generates a correct proof. Running time is probabilistic. –“Forward Analysis” + “Backward Analysis”

20 Conclusion Combining Randomized & Symbolic techniques is powerful –Interprocedural Random Interpretation [POPL ’05] –DART [PLDI ’05], Yogi [FSE ’06] –This work Machine Learning Algorithm –Inconsistency Measure for an abstract domain: How far are two abstract elements from satisfying the partial order? –Algorithm: Pick a program point (randomly) whose invariant is locally inconsistent & update it to make it less inconsistent. –Intersection of forward and backward analysis.