SAT-based Model Checking Yakir Vizel Computer Science Department, Technion, Israel Based on slides from K.L. McMillan, A.R. Bradley and Yakir Vizel.

Slides:



Advertisements
Similar presentations
Model Checking Base on Interoplation
Advertisements

Automated abstraction refinement II Heuristic aspects Ken McMillan Cadence Berkeley Labs.
The behavior of SAT solvers in model checking applications K. L. McMillan Cadence Berkeley Labs.
Exploiting SAT solvers in unbounded model checking
Quantified Invariant Generation using an Interpolating Saturation Prover Ken McMillan Cadence Research Labs TexPoint fonts used in EMF: A A A A A.
Exploiting SAT solvers in unbounded model checking K. L. McMillan Cadence Berkeley Labs.
Applications of Craig Interpolation to Model Checking K. L. McMillan Cadence Berkeley Labs.
Hybrid BDD and All-SAT Method for Model Checking Orna Grumberg Joint work with Assaf Schuster and Avi Yadgar Technion – Israel Institute of Technology.
CS 267: Automated Verification Lecture 8: Automata Theoretic Model Checking Instructor: Tevfik Bultan.
Automatic Verification Book: Chapter 6. What is verification? Traditionally, verification means proof of correctness automatic: model checking deductive:
Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License:
ECE Synthesis & Verification - L271 ECE 697B (667) Spring 2006 Synthesis and Verification of Digital Systems Model Checking basics.
Propositional and First Order Reasoning. Terminology Propositional variable: boolean variable (p) Literal: propositional variable or its negation p 
Methods of Proof Chapter 7, second half.. Proof methods Proof methods divide into (roughly) two kinds: Application of inference rules: Legitimate (sound)
Aaron Bradley University of Colorado, Boulder
The Theory of NP-Completeness
Rigorous Software Development CSCI-GA Instructor: Thomas Wies Spring 2012 Lecture 13.
Towards More Efficient SAT-Based Model Checking Joao Marques-Silva Electronics & Computer Science University of Southampton LAA C&V Workshop, Isaac Newton.
© Anvesh Komuravelli IC3/PDR Overview of IC3/PDR Anvesh Komuravelli Carnegie Mellon University.
Properties of SLUR Formulae Ondřej Čepek, Petr Kučera, Václav Vlček Charles University in Prague SOFSEM 2012 January 23, 2012.
SAT and Model Checking. Bounded Model Checking (BMC) A.I. Planning problems: can we reach a desired state in k steps? Verification of safety properties:
Tuning SAT-checkers for Bounded Model-Checking A bounded guided tour Ofer Strichman Carnegie Mellon University.
Weizmann Institute Tuning SAT-checkers for Bounded Model-Checking A bounded guided tour Ofer Shtrichman Weizmann Institute & IBM (HRL)
Proof-based Abstraction Presented by Roman Gershman Ken McMillan, Nina Amla.
Constraint Logic Programming Ryan Kinworthy. Overview Introduction Logic Programming LP as a constraint programming language Constraint Logic Programming.
Bounded Model Checking EECS 290A Sequential Logic Synthesis and Verification.
Technion 1 Generating minimum transitivity constraints in P-time for deciding Equality Logic Ofer Strichman and Mirron Rozanov Technion, Haifa, Israel.
Efficient Reachability Checking using Sequential SAT G. Parthasarathy, M. K. Iyer, K.-T.Cheng, Li. C. Wang Department of ECE University of California –
Formal Verification Group © Copyright IBM Corporation 2008 IBM Haifa Labs SAT-based unbounded model checking using interpolation Based on a paper “Interpolation.
Weizmann Institute Tuning SAT-checkers for Bounded Model-Checking A bounded guided tour Ofer Shtrichman Weizmann Institute & IBM-HRL.
Computing Over­Approximations with Bounded Model Checking Daniel Kroening ETH Zürich.
1 Completeness and Complexity of Bounded Model Checking.
1 Abstraction Refinement for Bounded Model Checking Anubhav Gupta, CMU Ofer Strichman, Technion Highly Jet Lagged.
CS 267: Automated Verification Lecture 13: Bounded Model Checking Instructor: Tevfik Bultan.
272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 4: SMT-based Bounded Model Checking of Concurrent Software.
7/13/2003BMC A SAT-Based Approach to Abstraction Refinement in Model Checking Bing Li, Chao Wang and Fabio Somenzi University of Colorado at Boulder.
Binary Decision Diagrams (BDDs)
1 The Theory of NP-Completeness 2012/11/6 P: the class of problems which can be solved by a deterministic polynomial algorithm. NP : the class of decision.
1 Completeness and Complexity of Bounded Model Checking.
SAT and SMT solvers Ayrat Khalimov (based on Georg Hofferek‘s slides) AKDV 2014.
Introduction to Satisfiability Modulo Theories
Advanced Topics in Propositional Logic Chapter 17 Language, Proof and Logic.
Incremental formal verification of hardware Hana Chockler Alexander Ivrii Arie Matsliah Shiri Moran Ziv Nevo IBM Research - Haifa.
Daniel Kroening and Ofer Strichman 1 Decision Procedures An Algorithmic Point of View BDDs.
Lazy Annotation for Program Testing and Verification Speaker: Chen-Hsuan Adonis Lin Advisor: Jie-Hong Roland Jiang November 26,
1 The Theory of NP-Completeness 2 Cook ’ s Theorem (1971) Prof. Cook Toronto U. Receiving Turing Award (1982) Discussing difficult problems: worst case.
LDK R Logics for Data and Knowledge Representation Propositional Logic: Reasoning First version by Alessandro Agostini and Fausto Giunchiglia Second version.
Daniel Kroening and Ofer Strichman 1 Decision Procedures An Algorithmic Point of View BDDs.
© Copyright 2008 STI INNSBRUCK Intelligent Systems Propositional Logic.
SAT-Based Model Checking Without Unrolling Aaron R. Bradley.
CS357 Lecture 13: Symbolic model checking without BDDs Alex Aiken David Dill 1.
Bounded Model Checking A. Biere, A. Cimatti, E. Clarke, Y. Zhu, Symbolic Model Checking without BDDs, TACAS’99 Presented by Daniel Choi Provable Software.
CIS 540 Principles of Embedded Computation Spring Instructor: Rajeev Alur
1 Alan Mishchenko Research Update June-September 2008.
Trading-off SAT search and Variable Quantifications for effective Unbounded Model Checking G. Cabodi P. Camurati L. Garcia M. Murciano S. Nocco S. Quer.
Knowledge Repn. & Reasoning Lecture #9: Propositional Logic UIUC CS 498: Section EA Professor: Eyal Amir Fall Semester 2005.
Dana Nau: Lecture slides for Automated Planning Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License:
Efficient Generation of Small Interpolants in CNF (for Model Checking)
Symbolic model checking with SAT/SMT
אימות אוטומטי Intertwined Forward-Backward Reachability Analysis Using Interpolants Work by: Yakir Vizel, Orna Grumberg and Sharon Shoham (TACAS 2013)
Hybrid BDD and All-SAT Method for Model Checking
Introduction to Software Verification
Inference and search for the propositional satisfiability problem
Solving Linear Arithmetic with SAT-based MC
Introduction to Software Verification
K. L. McMillan Cadence Berkeley Labs
Propositional Calculus: Boolean Algebra and Simplification
Scalability in Model Checking
Predicate Abstraction
Verifying Clausal Proofs, DRUPing and Interpolants SAT/SMT Seminar
Presentation transcript:

SAT-based Model Checking Yakir Vizel Computer Science Department, Technion, Israel Based on slides from K.L. McMillan, A.R. Bradley and Yakir Vizel

Outline Background –Symbolic Model Checking –DPLL-style SAT solvers –Bounded Model Checking SAT-based model checking methods –K-induction –Interpolation –Interpolation Sequence –IC3/PDR –There are more…

Model checking Problem definition: –Does every run of a (finite-state) transition system satisfy a given temporal property? Result: –Yes –No + counterexample Examples: –Is every request to this bus arbiter eventually acknowledged? –Does this program every dereference a null pointer?

Transition systems Tuple (S,I,T), where: –S is the (finite) set of states –I  S is the set of initial states –T  S  S is the set of transitions A run of (S,I,T) is  S , where: –  0  I –for all i  0, (  i,  i+1 )  T That is, a run is an infinite path in the state graph strating with an initial state

Reachability Problem def: –Does a transition system have a finite run ending in a state contained in the failure set F? –More precisely, does there exist  0...  k  S k s.t.:  0  I and  k  F for all 0  i < k, (  i,  i+1 )  T Using automata-theoretic methods, model checking safety properties reduces to reachability analysis. –Given a t.s. M and a property P, we can construct M P and F P such that M satisfies P exactly when F P is not reachable in M P.

State explosion problem Reachability analysis can be done by BFS or DFS on the state graph. However, |S| is exponential in system size –for example 2 n, where n is number of registers Impractical to construct the state graph explicitly. Our topic is essentially how to use a SAT solver to tackle this problem.

Symbolic transition systems Tuple (V,I,T), where: –V is a signature (set of variables), –I is a formula over V (the initial condition) –T is a formula over V  V' (the transition condition) States:  = V  {0,1} (a valuation to V) A run of (V,I,T) is  , where: –I[  0 ] –for all i  0, T[  i,  i+1 ] Note: T[  i,  i+1 ] means T[  i  ' i+1 ]

Example T is a conjunction of constraits, one per component. a b cp g g = a  b p = g  c c' = p T =  { g = a  b, p = g  c, c' = p }

Symbolic Reachability IF R1R1 R2R2... R = I  Img(I,T) = R 1  Img(R 1,T) Idea: represent reachable states by a formula Essentially a BFS with symbolic representation.

Symbolic reachability, cont. Reachability fixed point: R 0 = I R i+1 = R i  Img(R i,T) R =  R i F is reachable iff R  F  false Image operator: Img(Q,T) =  V. (Q(V)  T(V,V ’ )) We need a way to eliminate the quantifier, to get us back to an ordinary Boolean formula.

DPLL-style SAT solvers Objective: –Check satisfiability of a CNF formula literal: v or  v clause: disjunction of literals CNF: conjunction of clauses Approach: –Branch: make arbitrary decisions –Propagate implication graph –Use conflicts to guide inference steps SATO,GRASP,CHAFF,BERKMIN SAT solvers can also generate refutation proofs!

The Implication Graph (BCP) (  a  b)  (  b  c  d) a cc Decisions b Assignment: a  b   c  d d

Propositional Resolution a  b   c  a   c  d b   c  d When a conflict occurs, the implication graph is used to guide the resolution of clauses, so that the same conflict will not occur again.

Conflict Clauses (  a  b)  (  b  c  d)  (  b   d) a cc Decisions b Assignment: a  b   c  d d Conflict! (  b  c ) resolve Conflict! (  a  c) resolve Conflict!

Generating refutations Refutation = a proof of the null clause –Record a DAG containing all resolution steps performed during conflict clause generation. –When null clause is generated, we can extract a proof of the null clause as a resolution DAG. Original clauses Derived clauses Null clause

Circuit SAT a b c p g Can the circuit output be 1? input variables output variable (a   g)  (b   g)  (  a   b  g) (  g  p)  (  c  p)  (g  c   p) CNF(p) p is satisfiable when the formula CNF(p)  p is satisfiable

Bounded Model Checking Given –A finite transition system M –A property P Determine –Does M allow a counterexample to P of k transitions of fewer? This problem can be translated to a SAT problem BCCZ99

Symbolic Models (recall) Transition system described by constraints... a b cp g g = a  b p = g  c c' = p New notation: Q means "add n primes to the symbols in Q" T =  { g = a  b, p = g  c, c' = p }

Bounded model checking Unfold the model k times: U = T  T ...  T a b cp g a b cp g a b cp g... I F Use SAT solver to check satisfiability of I  U  F If unsatisfiable: property has no Cex of length k can produce a refutation proof P Biere,et al. TACAS99

Bounded Model Checking I R1R1 R2R2 = I  Img(I,T) = R 1  Img(R 1,T) F I  T  T   F ……

BMC applications Debugging: –Can find counterexamples using a SAT solver Proving properties: –Only possible if a bound on the length of the shortest counterexample is known. I.e., we need a diameter bound. The diameter is the maximum lenth of the shortest path between any two states. –Worst case is exponential. Obtaining better bounds is sometimes possible, but generally intractable.

Unbounded Model Checking We consider a variety of methods to explioit SAT and BMC for unbounded model checking: –K-step induction –Abstraction Counterexample-based Non-counterexample-based –Exact image computations SAT solver tests for fixed point SAT solver computes image –Over-approximate image computations

Induction The simple case: P is an inductive invariant –I => P –P  T => P’ Usually, P is not an inductive invariant BUT – a stronger inductive invariant R may exist (strengthening) –I => R –R  T => R’ –R => P

Induction P R I

K-induction Induction: SSS2000 P(s 0 )  i : P(s i )  P(s i+1 )  i : P(s i ) k-step induction: P(s 0..k-1 )  i : P(s i..i+k-1 )  P(s i+k )  i : P(s i )

K-induction with a SAT solver Recall: U k = T  T ...  T Two formulas to check: –Base case: I  U k-1  P...P –Induction step: U k  P...P  P If both are valid, then P always holds. If not, increase k and try again.

Simple path assumption Unfortunately, k-induction is not complete. –Some properties not k-inductive for any k. Simple path restriction: –There is a path to  P iff there is a simple path to  P (path with no repeated states). PP PP

Induction over simple paths Let simple(s 0..k ) be defined as: –  i,j in 0..k : (i  j)  s i  s j k-induction over simple paths: P(s 0..k-1 )  i : simple(s 0..k )  P(s i..i+k-1 )  P(s i+k )  i : P(s i ) Must hold for k large enough, since a simple path cannot be unboundedly long. Length of longest simple path is called recurrence diameter.

...with a SAT solver For simple path restriction, let: S k =  t=0..k, u=t+1..k:   v in V : v t = v u (where V is the set of state variables). Two formulas to check: –Base case: I  U k-1  P...P –Induction step: S k  U k  P...P  P If both are valid, then P always holds. If not, increase k and try again.

Termination Termination condition: k is the length of the longest simple path of the form P*  P This can be exponentially longer than the diameter. –example: loadable mod 2 N counter where P is (count  2 N -1) diameter = 1 longest simple path = 2 N Nice special cases: –P is a tautology (k=0) –P is inductive invariant (k=1)

Image computation methods Symbolic model checking without BDD's –Use SAT solver just for fixed-point detection Abdulla, Bjesse and Een 2000 Williams, Biere, Clarke and Gupta 2000 –Adapt SAT solver to compute image directly McMillan, 2002

Image over-approximation BMC and Craig interpolation allow us to compute image over-approximation relative to property. –Avoid computing exact image. –Maintain SAT solver's advantage of filtering out irrelevant facts.

Interpolation If A  B = false, there exists an interpolant A' for (A,B) such that: A  A' A'  B = false A' refers only to common variables of A,B Example: –A = p  q, B =  q  r, A' = q New result –given a resolution refutation of A  B, A' can be derived in linear time. (Craig,57) (Pudlak,Krajicek,97)

Interpolation-based MC Interpolation gives us –SAT-based algorithm for over-approximate image computation, using interpolation –SAT-only symbolic model checking (McMillan,2003)

Reachability Is there a path from I to F satisfying transitions T? Reachability fixed point: R 0 = I R i+1 = R i  Img(R i,T) R =  R i Image operator: Img(Q,T) =  V. (Q  T) F is reachable iff R  F  false

Overapproximation An overapproximate image op. is Img' s.t. for all Q, Img(Q,T) implies Img'(Q,T) Overapprimate reachability: R' 0 = I R' i+1 = R' i  Img'(R' i,T) R' =  R' i Img' is adequate (w.r.t.) F, when –if Q cannot reach F, Img ’ (Q,T) cannot reach F If Img' is adequate, then –F is reachable iff R'  F  false

Adequate image Q F Img(Q,T) Reached from Q Can reach F Img ’ (Q,T) But how do you get an adequate Img'?

k-adequate image operator Img' is k-adequate (w.r.t.) F, when –if Q cannot reach F, Img ’ (Q,T) cannot reach F within k steps Note, if k > diameter, then k-adequate is equivalent to adequate.

Interpolation-based image Idea -- use unfolding to enforce k-adequacy A = Q  T B = T  T  T  F k F k = ¬P ∨ ¬P ∨ … ∨ ¬P Q F TTTTTTT AB t=1 t=k Let Img'(Q) 0 = A', where A' is an interpolant for (A,B)... Img' is k-adequate!

Given the following BMC formula. A B A’A’

Huh? A  A' –Img(Q,T)  Img'(Q,T) A'  B = false – Img’(Q,T) cannot reach F in k steps Hence Img' is k-adequate overapprox. Q F TTTTTTT AB t=1 t=k A' But note, Img' is partial -- not defined if A  B is sat.

k-adequate I F1F1 F2F2 = I  Img’(I,T) = F 1  Img’(F 1,T) P ¬P …… k k

A1A1 A2A2 Using Interpolants

A’1A’ Using Interpolants (2)

I R1R1 R1R1 I R2R2 R2R2 R3R3 R3R3 ¬P A’1A’1 A’1A’1 A’2A’2 A’2A’2 A’3A’3 A’3A’3 Analogy To Reachability Analysis

Reachability algorithm let k = 0 repeat if I can reach F within k steps, answer reachable R = I while Img'(R,T)  F = false R' = Img'(R,T)  R if R' = R answer unreachable R = R' end while increase k end repeat

Termination Since k increases at every iteration, eventually k > d, the diameter, in which case Img' is adequate, and hence we terminate. Notes: –don't need to know when k > d in order to terminate –often termination occurs with k << d

Interpolation-based MC Fully SAT-based. Inherits SAT solvers ability to concentrate on facts relevant to a property. Most effective when –Very large set of facts is available –Only a small subset are relevant to property For true properties, appears to converge for smaller k values.

Interpolation-Sequence If A 1  A 2  A k = false, there exists an interpolation-sequence A’ 0, A’ 1,…, A’ k+1 for (A 1,  A k ) such that: A’ 0 =T and A’ k+1 =F A’ j  A j+1  A’ j+1 A’ j - over common variables of A 1,  Aj and A j+1,  A k A’ j equals the interpolant of A=A 1  A j and B=A j+1  A k –Given the same resolution graph

50 A1A1 A2A2 A3A3 AkAk A k+1 A’1A’1 A’2A’2 A’3A’3 A’ k-1 A’kA’k BMC formula partitioned in a different manner: Interpolation-Sequence based MC (Vizel and Grumberg,2009)

51 I 1,1 I 1,2 I 2,2 I1I1 I1I1 Using Interpolation-Sequence

52 INIT R1R1 R1R1 R2R2 R2R2 R3R3 R3R3 A1A1 A1A1 A A A3A3 A3A3 ¬P A 1,1 A 2,2 A 1,2 A1A1 A1A1 A2A2 A2A2 A 3,3 A 2,3 A 1,3 Analogy to Forward Reachability

Conclusion SAT solvers are very effective at ignoring irrelevant facts SAT solvers can produce refutations We can exploit in a number of ways: –BMC –Abstraction for UMC (either CBA or PBA) –Abstract image computations using interpolation This makes it possible to model check localizable properties large systems.

IC3 – The Breakthrough IC3 = Incremental Construction of Inductive Clauses for Indubitable Correctness The Goal: Find an Inductive Invariant stronger than P by learning relatively inductive facts (incrementally) –Recall: F is inductive invariant if I => F F  T => F’ –F is stronger than P, therefore F => P F  P  T => F’ => P’ (Bradley,2010)

What Makes IC3 Special? No unrolling/unfolding of the transition function T is required All previous approaches require unrolling –Searching for an inductive invariant –Unrolling = A form of strengthening IC3 strengthen in a different way

Example M 1 : –I: x=1, y=1 –T: x’= x+1, y’= y+x M 2 : –I: x=1, y=1 –T: x’= x+y, y’= y+x P: y ≥ 1

Induction on M 1 I => P: –x=1  y=1 => y ≥ 1 BUT: P  T ≠> P’ –y ≥ 1  x’=x+1  y’=x+y => y’ ≥ 1 Incremental Proof: –F = x ≥ 0 –x ≥ 0  x’=x+1 =>  x’ ≥ 0 –Clearly, F  P  T => P’

Induction on M 2 I => P: –x=1  y=1 => y ≥ 1 BUT: P  T ≠> P’ –y ≥ 1  x’=x+y  y’=x+y => y’ ≥ 1 Monolithic Proof: –Choose a new P*, stronger than P –Note that F = x ≥ 0 is also not inductive Set P* = x ≥ 0  y ≥ 1 –I => P* –P*  T => P*’

IC3 Basics Compute Over-approximated Reachability Sequence (OARS) s.t. –F 0 = I –F i => P –F i => F i+1 –F i  T => F’ i+1 Notations: –Cube s: conjunction of literals v 1  v 2  ¬v 3 - Represents a state –s is a cube => ¬s is a clause (DeMorgan) –P is inductive relative to F if F is inductive and F  P  T => P’

OARS I R1R1 R2R2 = I  Img(I,T) = R 1  Img(R 1,T) P F1F1 F2F2 ¬P

A Backward Search Search for a predecessor s to some error state: P  T  ¬P’ –If none exists, property holds: P  T  ¬P’ = ¬(P  T  => P’) Try to block s –P = P  ¬s

IC3 - Initialization Check the following two formulas: –I  ¬P’ –I  T  ¬P’ If both are unsatisfiable then: –I => P –I  T => P’ Therefore –F 0 = I, F 1 = P

IC3 - Initialization I F0F0 P F1F1

Iteration I F1F1 F2F2 P F k+1 …… FkFk F k-1

IC3 - Iteration Given an OARS, define F k+1 =P Apply a backward search –Find predecessor s in F k that can reach a bad state F k  T => P’ (F k  T  ¬P’) –If none exists, move to next iteration –If exists, try to find a predecessor t to s in F k-1 F k-1  T => ¬s’ (F k-1  T  s’) –If non exists, s can be removed from F k F k = F k  ¬s –Otherwise: Recur on (t,k-1) We call (t,k-1) a proof obligation If we can reach I, a CEX exists

That Simple? Looks simple But this “simple” does NOT work Simple = States Enumeration –Too many states… Are we enumerating states? –Yes (when it doesn’t work) –But, not really

Observation 1 A state s in F k can reach a bad state in one transition Important Fact: s is not in F k-1 (!!) –F k-1  T => F k –F k => P –If s was in F k-1 we would have found it in an earlier iteration Therefore: F k-2  T => ¬s’

Inductive Generalization A state s in F k can reach a bad state in one transition Assuming s in not reachable: –F k-1  T => ¬s’ BUT, this is stronger: F k-1  ¬s  T => ¬s’ This looks familiar! –I => ¬s Otherwise, CEX! (I ≠> ¬s  s is in I) –¬s is inductive relative to F k-1

Inductive Generalization So we know F k-1  ¬s  T => ¬s’ And, ¬s is a clause Generalize: Find a sub-clause c ¬s s.t. F k-1  c  T => c’ (and I => c) –Sub clause means less literals –Less literals implies less satisfying assignments (a ∨ b ∨ c) vs. (a ∨ b) –Stronger inductive fact More states are removed from F k, making it stronger (closer to R k )

Observation 2 A state s in F i can reach a bad state in a number of transitions s is also in F j for j > i, a longer CEX may exist –F i => F j –s may not be reachable in i steps, but it may be reachable in j steps

Push Forward I F1F1 F2F2 P F k+1 …… FkFk F k-1

Push Forward s is removed from F i –by conjoining a sub-clause c –F i = F i  c Pushing forward for j > i –F j  c  T => c’ holds –c is inductive relative to F j –F j = F j  c s cannot be blocked at level j > i –Add a proof obligation (s,j) –If s is reachable from I, CEX!

IC3 – Key Ingredients Backward Search –Find a state s that can reach a bad state in a number of steps –s may not be reachable (over-approximations) Block a State –Do it efficient, block more than s Generalization Push Forward –An inductive fact at frame i, may also be inductive at higher frames –If not, a longer CEX is found

The End