1. Exploiting SAT solvers in unbounded model checking
K. L. McMillan
Cadence Berkeley Labs

2. Overview
We will examine the performance of two SAT-based abstraction methods
Counterexample-based abstraction
Proof-based abstraction
This tells us something about
The nature of decision heuristics in SAT solvers
The strengths are weaknesses of SAT solvers for bounded model checking and related applications.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

3. Outline Background: SAT and BMC Localization abstraction
Cex-based
Proof-based
Performance study
what it tells us about SAT
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

4. DPLL-style SAT solvers
SATO,GRASP,CHAFF,BERKMIN
Objective:
Check satisfiability of a CNF formula
literal: v or Øv
clause: disjunction of literals
CNF: conjunction of clauses
Approach:
Branch: make arbitrary decisions
Propagate implication graph
Use conflicts to guide inference steps
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

5. The Implication Graph (BCP)
(Øa Ú b) Ù (Øb Ú c Ú d) d b a Øc
Decisions
Assignment: a Ù b Ù Øc Ù d
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

6. Resolution a Ú b Ú Øc Øa Ú Øc Ú d b Ú Øc Ú d
When a conflict occurs, the implication graph is
used to guide the resolution of clauses, so that the
same conflict will not occur again.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

7. Conflict Clauses (Øa Ú b) Ù (Øb Ú c Ú d) Ù (Øb Ú Ø d) (Øa Ú c) resolve
Decisions
Conflict!
Conflict! d
Assignment: a Ù b Ù Øc Ù d
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

8. Conflict Clauses (cont.)
Are generated by resolution
Are implied by existing clauses
Are in conflict in the current assignment
Are safely added to the clause set
Many heuristics are available for determining
when to terminate the resolution process.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

9. Basic SAT algorithm A = Æ empty clause? y UNSAT n conflict?
Deduce conflict
clause and
backtrack y
is A
total?
Branch:
add some literal
to A y
SAT
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

10. Generating refutations
Refutation = a proof of the null clause
Record a DAG containing all resolution steps performed during conflict clause generation.
When null clause is generated, we can extract a proof of the null clause as a resolution DAG.
Original clauses
Derived clauses
Null clause
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

11. Circuit SAT Can the circuit output be 1? CNF(p) (a Ú Øg) Ù (b Ú Øg)
Ù(Øa Ú Øb Ú g)
CNF(p)
input
variables a g
(Øg Ú p) Ù (Øc Ú p)
Ù(g Ú c Ú Øp) b
output
variable p c
p is satisfiable when the
formula CNF(p) Ù p
is satisfiable
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

12. Bounded Model Checking
BCCZ99
Given
A finite transition system M
A property p
Determine
Does M allow a counterexample to p of k transitions of fewer?
This problem can be translated to a SAT problem
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

13. Models Transition system described by a set of constraints Model:
g = a Ù b,
p = g Ú c,
c' = p }
Each circuit element is a constraint
note: a = at and a' = at+1
g = a Ù b
p = g Ú c
c' = p a b c p g
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

14. Properties We restrict our attention to safety properties.
Characterized by:
Initial condition I
Final condition F (representing "bad" states)
A counterexample is a path from a state satisfying I to state satisfying F, where every transition satisfies C.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

15. Unfolding Unfold the model k times:
Uk = C0 Ù C1 Ù ... Ù Ck-1 a b c p g a b c p g a b c p g
... I0 Fk
Use SAT solver to check satisfiability of
I0 Ù Uk Ù Fk
A satisfying assignment is a counterexample of k steps
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

16. BMC applications Debugging: Proving properties:
Can find counterexamples using a SAT solver
Proving properties:
Only possible if a bound on the length of the shortest counterexample is known.
I.e., we need a diameter bound. The diameter is the maximum lenth of the shortest path between any two states.
Worst case is exponential. Obtaining better bounds is sometimes possible, but generally intractable.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

17. Localization abstraction
Kurshan
Property: G (c Þ X c)
free variable
Model:
C = {
g = a Ù b,
p = g Ú c,
c' = p } ' a b c p g
C'Þ property, C Þ C'
C Þ property
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

18. Constraint granularity
Most authors use constraints at "latch" granularity... a b c p g
Model:
C = {
c' = (a Ù b) Ú c }
...however, techniques we will consider can be applied at
both "gate" and "latch" granularity.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

19. Localization, cont C' may refer to fewer state variables than C
reduction in the state explosion problem
Key issue: how to choose constraints in C'
counterexample-based
proof-based
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

20. Algorithm Kurshan Choose initial C' Model check true, done
abstraction C'
true, done
Cex
Can extend Cex
from C'to C?
yes, Cex
SAT uses no
Add constraints
to C'
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

21. Abstract counterexamples
Assume simple safety property:
initial condition I and final condition F
w.l.o.g., assume I and F are atomic formulas
to make this true, add constraints in C:
vI Û I vF Û F
Abstract variables V' = support(C',I,F)
Abstract counterexample A' is a truth assignment to:
{ vt | v in V', t in 0..k }
where k is the number of steps.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

22. Counterexample extension
CGJLV 2000
Abstract counterexample A' satisfies:
I0 Ù U'k Ù Fk where U'k = C'0 Ù C'1 Ù ... Ù C'k-1
Find A consistent with A', satisfying:
I0 Ù Uk Ù Fk where Uk = C0 Ù C1 Ù ... Ù Ck-1
That is, A is any satisfying assignment to:
A' Ù I0 Ù Uk Ù Fk
I.e., to extend an abstract counterexample, we just
apply it as a constraint in BMC. If unsat, abstract
counterexample is "false".
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

23. Abstraction refinement
Refinement = adding constraints to C' to eliminate false counterexamples.
Many heuristsics used for this.
Too many to cover here.
Recall that a SAT solver can produce a resolution-based refutation in the UNSAT case....
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

24. Proof-based refinement
Recall, to extend abstract Cex A', we check:
A' Ù I0 Ù Uk Ù Fk
If UNSAT, we obtain refutation proof P
proof that A' cannot be extended to concrete Cex
Let E be set of constraints used in proof P:
E = { c Î C | some ci occurs in P }
A' cannot be extended to a Cex for E
P is the proof of this.
Thus, add E to C' and continue...
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

25. In other words... The refutation of the formula:
A' Ù I0 Ù Uk Ù Fk
gives us a sufficient set of constraints to rule out the abstract counterexample.
We continue ruling out counterexamples until either the
abstraction C' proves the property or we can extend an
abstract counterexample to a concrete one.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

26. CCKSVW approach (FMCAD02)
Find the shortest prefix of Cex A' that cannot be extended.
That is,
A' Ù I0 Ù Uk Ù Fk
is feasible for all k < i, but not for k=i. OK OK OK OK
NO! s0 s1 s2
...
si-1 si
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

27. CCKSVW approach cont. Let P be a refutation of
A' Ù I0 Ù Ui Ù Fi
Let E be set of constraints used in proof P only on state si-1:
E = { c Î C | ci-2 occurs in P } OK OK OK OK
NO! s0 s1 s2
...
si-1 si
add constraints used here
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

28. Weakness of Cex-based approach
Arbitrarily chosen abstract Cex may be refutable for many reasons not related to property.
Thus, may add irrelevant constraints.
To remedy, may try to characterize a set of Cex's rather than just one (e.g., GKM-HFV,TACAS03).
Alternative: don't use counterexamples
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

29. Proof-based abstraction
MA,TACAS03
Cex?
BMC
at depth k
done
No Cex?
Use refutation to
choose abstraction
Increase k
MC abstraction
True?
done
False?
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

30. BMC phase Unfold the model k times:
U = C0 Ù C1 Ù ... Ù Ck-1
Use SAT solver to check satisfiability of
I0 Ù U Ù Fk
If unsatisfiable:
property has no Cex of length k
produce a refutation proof P
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

31. Abstraction phase Let C' be set of constraints used in proof P:
C' = { c Î C | some ci occurs in P }
C' admits no counterexample of length k
let U' = C'0 Ù C'1 Ù ... Ù C'k-1
P is a refutation of I0 Ù U' Ù Fk
Model check property on C'
property true for C' implies true for C
else Cex of length k' > k (why?)
restart for k = k'
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

32. Algorithm Cex? BMC done C at depth k No Cex? Refutation P induces
abstraction C'
let k = k'
Model check C'
True?
done
Cex of depth k'?
Notice: MC counterexample is thrown away!
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

33. Termination Depth k increases at each iteration
Eventually k > d, diameter of C'
If k > d, no counterexample is possible
In practice, termination uses occurs when k » d/2
Usually, diameter C' << diameter of C
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

34. Weakness of proof-based abs
BMC must refute all counterexamples of length k, while in Cex-based, BMC must refute only one (partial) counterexample.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

35. PicoJavaII benchmarks
Hardware Java virtual machine implementation
Properties derived from verification of ICU
handles cache, instruction prefetch and decode
Original abstraction was manual
Added neigboring IFU to make problem harder
properties
ICU
IFU
Mem,
Cache
Integer
unit
No properties can be verified by standard model checking!
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

36. Abstraction results
solid = original, gray = manual, open = proof-based abstraction
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

37. Inference
SAT solver seems to be very effective at narrowing down the proof to relevant facts.
In most cases, it did better than manual abstraction.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

38. Comparing CBA and PBA Apples-apples comparison same SAT solver
same model checker
only differences are:
For CBA previous A' is kept as a constriaint for BMC, C' is cumulative.
For PBA previous A' and C' are thrown away each iteration.
Note these are my implementations. This says nothing
about performance of specific tools!
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

39. Run time comparison
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

40. Abstraction comparison
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

41. Possible explanation Internally, SAT solver is really doing CBA a=0
decision stack
= abstract Cex A'
refutation of A'
decision heuristic
moves proof variables
up, into A'
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

42. PBA run-time breakdown
solid = BMC time, open = MC time
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

43. CBA run-time breakdown
solid = BMC time, open = MC time
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

44. A (fuzzy) hypothesis
SAT-based BMC "succeeds" when number of relevant variables is small, and fails otherwise.
"success" is BMC for k = diameter of relevant logic
Parameterized models allowing no abstraction
Model
Max state vars
German protocol 42
"swap" 21
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

45. Industrial benchmarks
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

46. Implications for model checking
Most of the time if bounded model checking succeeds, unbounded model checking also succeeds using abstraction.
No need to settle for time bounded result
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

47. Conclusions
SAT solvers are very effective at ignoring irrelevant facts
Can think of decision heuristic as a form of CBA
implications for improving heuristics?
Hence very similar performance of CBA and PBA for localization abstraction
SAT solver performance is tied to number of relevant variables
Performs well if there is a small UNSAT "core"
Performs badly when all variables relevant.
Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.