Exploiting SAT solvers in unbounded model checking K. L. McMillan Cadence Berkeley Labs

Overview We will examine the performance of two SAT-based abstraction methods Counterexample-based abstraction Proof-based abstraction This tells us something about The nature of decision heuristics in SAT solvers The strengths are weaknesses of SAT solvers for bounded model checking and related applications. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Outline Background: SAT and BMC Localization abstraction Cex-based Proof-based Performance study what it tells us about SAT Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

DPLL-style SAT solvers SATO,GRASP,CHAFF,BERKMIN Objective: Check satisfiability of a CNF formula literal: v or Øv clause: disjunction of literals CNF: conjunction of clauses Approach: Branch: make arbitrary decisions Propagate implication graph Use conflicts to guide inference steps Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

The Implication Graph (BCP) (Øa Ú b) Ù (Øb Ú c Ú d) d b a Øc Decisions Assignment: a Ù b Ù Øc Ù d Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Resolution a Ú b Ú Øc Øa Ú Øc Ú d b Ú Øc Ú d When a conflict occurs, the implication graph is used to guide the resolution of clauses, so that the same conflict will not occur again. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Conflict Clauses (Øa Ú b) Ù (Øb Ú c Ú d) Ù (Øb Ú Ø d) (Øa Ú c) resolve Decisions Conflict! Conflict! d Assignment: a Ù b Ù Øc Ù d Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Conflict Clauses (cont.) Are generated by resolution Are implied by existing clauses Are in conflict in the current assignment Are safely added to the clause set Many heuristics are available for determining when to terminate the resolution process. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Basic SAT algorithm A = Æ empty clause? y UNSAT n conflict? Deduce conflict clause and backtrack y is A total? Branch: add some literal to A y SAT Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Generating refutations Refutation = a proof of the null clause Record a DAG containing all resolution steps performed during conflict clause generation. When null clause is generated, we can extract a proof of the null clause as a resolution DAG. Original clauses Derived clauses Null clause Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Circuit SAT Can the circuit output be 1? CNF(p) (a Ú Øg) Ù (b Ú Øg) Ù(Øa Ú Øb Ú g) CNF(p) input variables a g (Øg Ú p) Ù (Øc Ú p) Ù(g Ú c Ú Øp) b output variable p c p is satisfiable when the formula CNF(p) Ù p is satisfiable Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Bounded Model Checking BCCZ99 Given A finite transition system M A property p Determine Does M allow a counterexample to p of k transitions of fewer? This problem can be translated to a SAT problem Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Models Transition system described by a set of constraints Model: g = a Ù b, p = g Ú c, c' = p } Each circuit element is a constraint note: a = at and a' = at+1 g = a Ù b p = g Ú c c' = p a b c p g Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Properties We restrict our attention to safety properties. Characterized by: Initial condition I Final condition F (representing "bad" states) A counterexample is a path from a state satisfying I to state satisfying F, where every transition satisfies C. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Unfolding Unfold the model k times: Uk = C0 Ù C1 Ù ... Ù Ck-1 a b c p g a b c p g a b c p g ... I0 Fk Use SAT solver to check satisfiability of I0 Ù Uk Ù Fk A satisfying assignment is a counterexample of k steps Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

BMC applications Debugging: Proving properties: Can find counterexamples using a SAT solver Proving properties: Only possible if a bound on the length of the shortest counterexample is known. I.e., we need a diameter bound. The diameter is the maximum lenth of the shortest path between any two states. Worst case is exponential. Obtaining better bounds is sometimes possible, but generally intractable. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Localization abstraction Kurshan Property: G (c Þ X c) free variable Model: C = { g = a Ù b, p = g Ú c, c' = p } ' a b c p g C'Þ property, C Þ C' C Þ property Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Constraint granularity Most authors use constraints at "latch" granularity... a b c p g Model: C = { c' = (a Ù b) Ú c } ...however, techniques we will consider can be applied at both "gate" and "latch" granularity. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Localization, cont C' may refer to fewer state variables than C reduction in the state explosion problem Key issue: how to choose constraints in C' counterexample-based proof-based Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Algorithm Kurshan Choose initial C' Model check true, done abstraction C' true, done Cex Can extend Cex from C'to C? yes, Cex SAT uses no Add constraints to C' Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Abstract counterexamples Assume simple safety property: initial condition I and final condition F w.l.o.g., assume I and F are atomic formulas to make this true, add constraints in C: vI Û I vF Û F Abstract variables V' = support(C',I,F) Abstract counterexample A' is a truth assignment to: { vt | v in V', t in 0..k } where k is the number of steps. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Counterexample extension CGJLV 2000 Abstract counterexample A' satisfies: I0 Ù U'k Ù Fk where U'k = C'0 Ù C'1 Ù ... Ù C'k-1 Find A consistent with A', satisfying: I0 Ù Uk Ù Fk where Uk = C0 Ù C1 Ù ... Ù Ck-1 That is, A is any satisfying assignment to: A' Ù I0 Ù Uk Ù Fk I.e., to extend an abstract counterexample, we just apply it as a constraint in BMC. If unsat, abstract counterexample is "false". Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Abstraction refinement Refinement = adding constraints to C' to eliminate false counterexamples. Many heuristsics used for this. Too many to cover here. Recall that a SAT solver can produce a resolution-based refutation in the UNSAT case.... Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Proof-based refinement Recall, to extend abstract Cex A', we check: A' Ù I0 Ù Uk Ù Fk If UNSAT, we obtain refutation proof P proof that A' cannot be extended to concrete Cex Let E be set of constraints used in proof P: E = { c Î C | some ci occurs in P } A' cannot be extended to a Cex for E P is the proof of this. Thus, add E to C' and continue... Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

In other words... The refutation of the formula: A' Ù I0 Ù Uk Ù Fk gives us a sufficient set of constraints to rule out the abstract counterexample. We continue ruling out counterexamples until either the abstraction C' proves the property or we can extend an abstract counterexample to a concrete one. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

CCKSVW approach (FMCAD02) Find the shortest prefix of Cex A' that cannot be extended. That is, A' Ù I0 Ù Uk Ù Fk is feasible for all k < i, but not for k=i. OK OK OK OK NO! s0 s1 s2 ... si-1 si Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

CCKSVW approach cont. Let P be a refutation of A' Ù I0 Ù Ui Ù Fi Let E be set of constraints used in proof P only on state si-1: E = { c Î C | ci-2 occurs in P } OK OK OK OK NO! s0 s1 s2 ... si-1 si add constraints used here Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Weakness of Cex-based approach Arbitrarily chosen abstract Cex may be refutable for many reasons not related to property. Thus, may add irrelevant constraints. To remedy, may try to characterize a set of Cex's rather than just one (e.g., GKM-HFV,TACAS03). Alternative: don't use counterexamples Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Proof-based abstraction MA,TACAS03 Cex? BMC at depth k done No Cex? Use refutation to choose abstraction Increase k MC abstraction True? done False? Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

BMC phase Unfold the model k times: U = C0 Ù C1 Ù ... Ù Ck-1 Use SAT solver to check satisfiability of I0 Ù U Ù Fk If unsatisfiable: property has no Cex of length k produce a refutation proof P Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Abstraction phase Let C' be set of constraints used in proof P: C' = { c Î C | some ci occurs in P } C' admits no counterexample of length k let U' = C'0 Ù C'1 Ù ... Ù C'k-1 P is a refutation of I0 Ù U' Ù Fk Model check property on C' property true for C' implies true for C else Cex of length k' > k (why?) restart for k = k' Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Algorithm Cex? BMC done C at depth k No Cex? Refutation P induces abstraction C' let k = k' Model check C' True? done Cex of depth k'? Notice: MC counterexample is thrown away! Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Termination Depth k increases at each iteration Eventually k > d, diameter of C' If k > d, no counterexample is possible In practice, termination uses occurs when k » d/2 Usually, diameter C' << diameter of C Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Weakness of proof-based abs BMC must refute all counterexamples of length k, while in Cex-based, BMC must refute only one (partial) counterexample. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

PicoJavaII benchmarks Hardware Java virtual machine implementation Properties derived from verification of ICU handles cache, instruction prefetch and decode Original abstraction was manual Added neigboring IFU to make problem harder properties ICU IFU Mem, Cache Integer unit No properties can be verified by standard model checking! Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Abstraction results solid = original, gray = manual, open = proof-based abstraction Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Inference SAT solver seems to be very effective at narrowing down the proof to relevant facts. In most cases, it did better than manual abstraction. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Comparing CBA and PBA Apples-apples comparison same SAT solver same model checker only differences are: For CBA previous A' is kept as a constriaint for BMC, C' is cumulative. For PBA previous A' and C' are thrown away each iteration. Note these are my implementations. This says nothing about performance of specific tools! Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Run time comparison Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Abstraction comparison Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Possible explanation Internally, SAT solver is really doing CBA a=0 decision stack = abstract Cex A' refutation of A' decision heuristic moves proof variables up, into A' Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

PBA run-time breakdown solid = BMC time, open = MC time Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

CBA run-time breakdown solid = BMC time, open = MC time Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

A (fuzzy) hypothesis SAT-based BMC "succeeds" when number of relevant variables is small, and fails otherwise. "success" is BMC for k = diameter of relevant logic Parameterized models allowing no abstraction Model Max state vars German protocol 42 "swap" 21 Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Industrial benchmarks Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Implications for model checking Most of the time if bounded model checking succeeds, unbounded model checking also succeeds using abstraction. No need to settle for time bounded result Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Conclusions SAT solvers are very effective at ignoring irrelevant facts Can think of decision heuristic as a form of CBA implications for improving heuristics? Hence very similar performance of CBA and PBA for localization abstraction SAT solver performance is tied to number of relevant variables Performs well if there is a small UNSAT "core" Performs badly when all variables relevant. Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.