1. CS357 Lecture 13: Symbolic model checking without BDDs Alex Aiken David Dill 1

2. Bounded model checking Simple idea: “Unroll” description, use SAT to find a property violation. State description: –I(s) – initial state –N(s,t) – t is a successor of s –P(s) – property of all states k-step violation of the property would satisfy: I(s 0 )  N(s 0,s 1 )  N(s 1,s 2 ) ...  N(s k-1,s k )  P(s k ) Use SAT to check this for k = 0, 1,... until it gets too big. 2

3. Verification with BMC Bounded model checking is a good way to search for counterexamples (up to some depth). For proving “always properties” (previous slide) –Define “depth” of a state as the length of the shortest path from a start state to the state. –Searching all k up to the maximum-depth state is sufficient to prove property P. –If there are |V| Boolean state variables, there is a bound of 2 |V| on this path length. –The maximum depth is usually much less than this. –This is impractically large in most cases. –It’s not helpful for infinite-state systems (e.g., unbounded integers). 3

4. Liveness properties Reminder: Properties like “eventually P” are liveness properties. They don’t have finite-length counterexamples. But, for finite-state systems, there is a counterexample that is a single infinite path with a loop. The following says that there exists a loop where P is always false (a violation of “AF P”) I(s 0 )  N(s 0,s 1 ) ...  N(s i,s i+1 )  P(s i+1 )...  P(s k-1 )  N(s k-1,s i )  P(s i ) 4

5. Better model checking with SAT Interpolants give some of the same advantages of constrain in BDDs Def: Given a pair of formulas (A, B) s.t. A&B is not satisfiable, and interpolant P is a formula: –A  P –P  B is unsatisfiable –P contains only variables that occur in both A and B. Equivalently: If A  C, there is some P s.t. –A  P and P  C –P contains only variables that occur in both A and C –(C =  B) 5

6. Interpolants in unbounded model checking Check I(s 0 )  N(s 0,s 1 )  N(s 1,s 2 ) ...  N(s k-1,s k )  [  P(s 1 )   P(s 2 ) ...  P(s k )] If satisfiable, we have a property violation Otherwise, find interpolant Q of I(s 0 )  N(s 0,s 1 ) and N(s 1,s 2 ) ...  N(s k-1,s k )  P(s k ) –Q only has variables from s 1 –I(s 0 )  N(s 0,s 1 )  Q(s 1 ) -- Q(s 1 ) is an overapproximation of states reachable in 1 step. – Q(s 1 )  [N(s 1,s 2 ) ...  N(s k-1,s k )  P(s k )] unsatisfiable, so Q(s 1 ) is an underapproximation of states that are backwards reachable from  P(s k ) in k-1 steps. 6

7. Interpolants in unbounded model checking R(s) := I(s) while true { if R(s)  N(s,t)  P(t) is satisfiable return “error found” R’(t) := ITP[R(s)  N(s, t),  P(t)]  R(t) if R’(s)  R(s), return “P holds” (inductive invariant) R(s) := R’(s)  R(s) } (Note: R(s) is R(t) with variable s substituted for t.) 7

8. Fix the bug This line could produce false positives: if R(s)  N(s,t)  P(t) is satisfiable return “error found” Since R is an overapproximation of the state space, there might be some unreachable state s that satisfies the formula. Correct algorithm uses N(s 1,s 2 ) ...  N(s k-1,s k )  [  P(s 1 )  P(s 2 ) ...  P(s k )] instead of  P(t) 8

9. Termination Theorem: There exists a k where this terminates with the correct answer. k is the reverse depth of the state graph. Reverse depth of a state s is the length of the shortest path to a state satisfying  P(t). Reverse depth of state graph is the maximum of this value over all states. Algorithm often terminates much faster by reaching a fixed point. 9

10. Finding a propositional interpolant General idea: Derive from proof of unsatisfiability –Works more generally (some quantifier-free first-order formulas). Simple linear-time algorithm on resolution tree (see McMillan paper). Based on resolution proofs related to GRASP implication graph. 10

11. Predicate Refinement First-order interpolation can be used to find new predicates in CEGAR loop of predicate abstraction. Used in BLAST and other program verifiers. 11 x=y Good Predicates Bad Predicates x=0, y=0 x=1, y=1 x=2, y=2... x=i; y=i; while(x!=0) {x--; y--;} assert y==0; Program

12. Divergence example Most heuristics derive predicates in some way from the refutation of the counterexample. Example: refinement using weakest procondition (WP) x=i,y=i [x!=0] x--, y-- [x=0] [y!=0] Error! x=i; y=i; while(x!=0) {x--; y--;} assert y==0; False x=0  y=0 x=1  y=1 i=1  i=1 x=0,y=0 x  0,y  0 x=0,y  0 Add these predicate From McMillan & Jhala, TACAS 2006 slides

13. Interpolants and concrete counterexamples. This prevents introduction of unnecessary symbols in predicates. Starting point for refinement: Unsatisfiable concretized counterexample  1 (s 1 )  N(s 1,s 2 )  2 (s 2 ) ...  n (s n )  FALSE Create a “generalized interpolant” R 1 (s 1 ),R 2 (s 2 ),...,R n (s n ) –R i-1 (s i-1 )  N(s i-1,s i )  R i (s i ) –R i (s i )  N(s i,s i+i )  i+i (s i+i ) ...  n (s n )  FALSE –R i only has symbols from  i 13

14. Interpolants as Floyd-Hoare proofs False x 1 =y 0 True y 1 >x 1    1. Each formula implies the next 2. Each is over common symbols of prefix and suffix 3. Begins with true, ends with false x=y; y++; [x=y] x 1 = y 0 y 1 =y 0 +1 x1y1x1y1 {False} {x=y} {True} {y>x} x = y y++ [x == y] From Ken McMillan, 2010

15. Limited language Limited expressiveness of predicates forces generalization (to a good predicate). E.g., avoid predicates x=1, y=1, x=2, y=2, etc. Force it to use x=y, instead. Idea: restrict language, gradually relax restrictions only if proof is impossible with current language. –Example: Only allow constants up to size k in formulas. Increase k only when proof is impossible with current k. 15

16. First-order interpolation Interpolation quantifier-free first-order logic uses same kind of trick as propositional Constructs: Equality, uninterpreted terms, arrays, simple arithmetic. Constrain prover to find interpolants during proofs. –“split proof” – deduction rules restricted to vocabulary common to two successive formulas. 16