Tuning SAT-checkers for Bounded Model-Checking A bounded guided tour Ofer Strichman Carnegie Mellon University.

Slides:



Advertisements
Similar presentations
Model Checking Base on Interoplation
Advertisements

Exploiting SAT solvers in unbounded model checking
Exploiting SAT solvers in unbounded model checking K. L. McMillan Cadence Berkeley Labs.
Model Checking Lecture 3. Specification Automata Syntax, given a set A of atomic observations: Sfinite set of states S 0 Sset of initial states S S transition.
SAT-based Bounded and Unbounded Model Checking Edmund M. Clarke Carnegie Mellon University Joint research with C. Bartzis, A. Biere, P. Chauhan, A. Cimatti,
CS 267: Automated Verification Lecture 8: Automata Theoretic Model Checking Instructor: Tevfik Bultan.
Introduction to Formal Methods for SW and HW Development 09: SAT Based Abstraction/Refinement in Model-Checking Roberto Sebastiani Based on work and slides.
SAT Based Abstraction/Refinement in Model-Checking Based on work by E. Clarke, A. Gupta, J. Kukula, O. Strichman (CAV’02)
Automatic Verification Book: Chapter 6. What is verification? Traditionally, verification means proof of correctness automatic: model checking deductive:
Daniel Kroening and Ofer Strichman 1 Decision Procedures An Algorithmic Point of View SAT.
The Theory of NP-Completeness
1 Temporal Claims A temporal claim is defined in Promela by the syntax: never { … body … } never is a keyword, like proctype. The body is the same as for.
SAT and Model Checking. Bounded Model Checking (BMC) A.I. Planning problems: can we reach a desired state in k steps? Verification of safety properties:
Weizmann Institute Tuning SAT-checkers for Bounded Model-Checking A bounded guided tour Ofer Shtrichman Weizmann Institute & IBM (HRL)
SAT-based Bounded Model Checking
Proof-based Abstraction Presented by Roman Gershman Ken McMillan, Nina Amla.
Heuristics for Efficient SAT Solving As implemented in GRASP, Chaff and GSAT.
1 Carnegie Mellon UniversitySPINFlavio Lerda SPIN An explicit state model checker.
Ofer Strichman, Technion 1 Decision Procedures in First Order Logic Part III – Decision Procedures for Equality Logic and Uninterpreted Functions.
1 Deciding separation formulas with SAT Ofer Strichman Sanjit A. Seshia Randal E. Bryant School of Computer Science, Carnegie Mellon University.
1 Completeness and Complexity of Bounded Model Checking Ed Clarke Daniel Kroening Joel Ouaknine Carnegie Mellon University, Pittsburgh, USA Ofer Strichman.
Witness and Counterexample Li Tan Oct. 15, 2002.
Analysis of Algorithms CS 477/677
Formal Verification Group © Copyright IBM Corporation 2008 IBM Haifa Labs SAT-based unbounded model checking using interpolation Based on a paper “Interpolation.
Review of the automata-theoretic approach to model-checking.
Pruning techniques for the SAT-based Bounded Model-Checking problem Ofer Shtrichman Weizmann Institute of Science & IBM - HRL.
Weizmann Institute Tuning SAT-checkers for Bounded Model-Checking A bounded guided tour Ofer Shtrichman Weizmann Institute & IBM-HRL.
Computing Over­Approximations with Bounded Model Checking Daniel Kroening ETH Zürich.
1 Completeness and Complexity of Bounded Model Checking.
1 Abstraction Refinement for Bounded Model Checking Anubhav Gupta, CMU Ofer Strichman, Technion Highly Jet Lagged.
1 A propositional world Ofer Strichman School of Computer Science, Carnegie Mellon University.
Witness and Counterexample Li Tan Oct. 15, 2002.
1 Formal Engineering of Reliable Software LASER 2004 school Tutorial, Lecture1 Natasha Sharygina Carnegie Mellon University.
CS 267: Automated Verification Lecture 13: Bounded Model Checking Instructor: Tevfik Bultan.
Formal Verification of SpecC Programs using Predicate Abstraction Himanshu Jain Daniel Kroening Edmund Clarke Carnegie Mellon University.
Flavio Lerda 1 LTL Model Checking Flavio Lerda. 2 LTL Model Checking LTL –Subset of CTL* of the form: A f where f is a path formula LTL model checking.
272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 4: SMT-based Bounded Model Checking of Concurrent Software.
1 Carnegie Mellon UniversitySPINFlavio Lerda Bug Catching SPIN An explicit state model checker.
Regular Model Checking Ahmed Bouajjani,Benget Jonsson, Marcus Nillson and Tayssir Touili Moran Ben Tulila
1 The Theory of NP-Completeness 2012/11/6 P: the class of problems which can be solved by a deterministic polynomial algorithm. NP : the class of decision.
Model Checking Lecture 4 Tom Henzinger. Model-Checking Problem I |= S System modelSystem property.
1 Completeness and Complexity of Bounded Model Checking.
Daniel Kroening and Ofer Strichman 1 Decision Procedures in First Order Logic Decision Procedures for Equality Logic Range Allocation.
1 Agenda Modeling problems in Propositional Logic SAT basics Decision heuristics Non-chronological Backtracking Learning with Conflict Clauses SAT and.
CIS 842: Specification and Verification of Reactive Systems Lecture Specifications: LTL Model Checking Copyright , Matt Dwyer, John Hatcliff,
Daniel Kroening and Ofer Strichman 1 Decision Procedures An Algorithmic Point of View BDDs.
1 The Theory of NP-Completeness 2 Cook ’ s Theorem (1971) Prof. Cook Toronto U. Receiving Turing Award (1982) Discussing difficult problems: worst case.
Learning Symbolic Interfaces of Software Components Zvonimir Rakamarić.
Verification & Validation By: Amir Masoud Gharehbaghi
SAT-Based Model Checking Without Unrolling Aaron R. Bradley.
Heuristics for Efficient SAT Solving As implemented in GRASP, Chaff and GSAT.
1 Temporal logic. 2 Prop. logic: model and reason about static situations. Example: Are there truth values that can be assigned to x,y simultaneously.
© 2006 Carnegie Mellon University Introduction to CBMC: Part 1 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Arie Gurfinkel,
CS357 Lecture 13: Symbolic model checking without BDDs Alex Aiken David Dill 1.
Variants of LTL Query Checking Hana ChocklerArie Gurfinkel Ofer Strichman IBM Research SEI Technion Technion - Israel Institute of Technology.
Bounded Model Checking A. Biere, A. Cimatti, E. Clarke, Y. Zhu, Symbolic Model Checking without BDDs, TACAS’99 Presented by Daniel Choi Provable Software.
SAT Solving As implemented in - DPLL solvers: GRASP, Chaff and
Today’s Agenda  Quiz 4  Temporal Logic Formal Methods in Software Engineering1.
© 2006 Carnegie Mellon University Introduction to CBMC: Part 1 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Arie Gurfinkel,
Bernd Fischer RW714: SAT/SMT-Based Bounded Model Checking of Software.
Model Checking Lecture 2. Model-Checking Problem I |= S System modelSystem property.
Presentation Title 2/4/2018 Software Verification using Predicate Abstraction and Iterative Refinement: Part Bug Catching: Automated Program Verification.
Symbolic model checking with SAT/SMT
Bounded Model Checking
Introduction to Software Verification
Heuristics for Efficient SAT Solving
Decision Procedures An Algorithmic Point of View
Translating Linear Temporal Logic into Büchi Automata
Scalability in Model Checking
SAT Based Abstraction/Refinement in Model-Checking
Presentation transcript:

Tuning SAT-checkers for Bounded Model-Checking A bounded guided tour Ofer Strichman Carnegie Mellon University

Model Checking  Given a: Finite transition system M(S, I,  A temporal property   The model checking problem: Does M satisfy 

Model Checking  Temporal properties: “Always x=y ” (G( x=y )) “Every Send is followed by Ack ” (G( Send  F A ck )) “ Reset can always be reached” (GF Reset ) “From some point on, always switch_on” (FG switch_on) “Safety” properties “Liveness” properties

Advances in Model Checking  Explicit model checking (1980 – )  Symbolic Model Checking with Binary Decision Diagrams (1991 – )  Symbolic Bounded Model Checking with SAT solvers (1999 –)

Bounded Model Checking  A.I. Planning problems: can we reach a desired state in k steps?  Verification of safety properties: can we find a bad state in k steps?  Verification: can we find a counterexample in k steps ? (Biere, Cimatti, Clarke, Zhu, 1999)

Bounded Model Checking  Most safety properties can be reduced to “Always p” where p is propositional.  Is there a state reachable within k cycles that satisfies  p ?... s0s0 s1s1 s2s2 s k-1 sksk pp p pp p

Reducing the BMC problem to SAT p is preserved up to cycle k iff  (k) is unsatisfiable:... s0s0 s1s1 s2s2 s k-1 sksk pp p pp p

Example: a two bit counter Property: Always (  l   r)  (2) is unsatisfiable.  (3) is satisfiable. Initial state: Transition:

Bounded Model Checking  All Linear-time Temporal Logic (LTL) can be checked with BMC  BMC can be applied to software, e.g. C programs (Kroening, Clarke, 2002): Unwind each loop k times Represent in Single Assignment Form (SAF) Solve the resulting bit-vector verification condition

Bounded Model-Checking of software  while() loops are unwinded void f(...) {... cond while(cond) { Body; Body; } Rest; } void f(...) {... cond while(cond) { Body; Body; } Rest; }

Bounded Model-Checking of software  while() loops are unwinded void f(...) {... cond if(cond) { Body; cond while(cond) { Body; Body; } Rest; } void f(...) {... cond if(cond) { Body; cond while(cond) { Body; Body; } Rest; }

Bounded Model-Checking of software  while() loops are unwinded void f(...) {... cond if(cond) { Body; cond if(cond) { Body; cond while(cond) { Body; Body; } Rest; } void f(...) {... cond if(cond) { Body; cond if(cond) { Body; cond while(cond) { Body; Body; } Rest; }

Bounded Model-Checking of software  while() loops are unwinded iteratively  Assertion may be inserted after last iteration: violated if program runs longer than bound permits void f(...) {... cond if(cond) { Body; cond if(cond) { Body; cond if(cond) { Body; Body; cond while(cond) { Body; Body; } Rest; } void f(...) {... cond if(cond) { Body; cond if(cond) { Body; cond if(cond) { Body; Body; cond while(cond) { Body; Body; } Rest; }

Bounded Model-Checking of software  while() loops are unwinded iteratively  Assertion my be inserted after last iteration: violated if program runs longer than bound permits void f(...) {... cond if(cond) { Body; cond if(cond) { Body; cond if(cond) { Body; Body; cond if(cond) { assert(FALSE); } Rest; } void f(...) {... cond if(cond) { Body; cond if(cond) { Body; cond if(cond) { Body; Body; cond if(cond) { assert(FALSE); } Rest; }

Bounded Model Checking - First impression…  First experiments with BMC in the industry showed that it is rarely faster than model checkers, unless k is very small.  But: Model checkers enjoyed more than 10 years of R&D….

The Davis-Putnam procedure Given  k  in CNF: (x,y,z),(x,y),( : y,z),( : x, : y, : z) Decide() Deduce() ( ~1000) Diagnose()  

Tuning SAT for BMC 1.Restrict Decide() to a small set of variables 2.Use the variable dependency graph for smarter orderings 3.Exploit  (k)’s structure to restrict the state-space: Learn more by exploiting the symmetry of  (k) Reuse information between the SAT instances

1. Restricting Decide()  Restricting Decide() to a smaller set of variables that uniquely determines the satisfiability of  (k): Model variables (~ 15 % of  (k)’s variables) Input variables (~ 5 % of  (k)’s variables)  Less variables to Decide() implies more variables to Deduce()

 For a general CNF formula, Dynamic strategies are typically better: Most Frequent in unsatisfied clauses (DLCS) Satisfies the most clauses (DLIS) Satisfies the most shortest clauses (MOM, JW) Conflict Driven (VSIDS) : 2. Variable ordering Q: How well do they work with BMC formulas ?

A (CNF) dependency graph D (V,E): A partitioning C 1..C n : An abstract dependency graph D’(V’, E’): 2. Variable ordering (Abstract dependency graphs)

For  (k) there exists a partition C 1..C n s.t. the abstract dependency graph is linear C0C0 C1C1 C2C2 CkCk C3C3 C k-1 V0V0 V1V1 V2V2 VkVk V3V3 V k Variable ordering (The natural order of  (k))

I0I0 ~Pk~Pk With general-purpose Decide() strategies, local sets of variables are satisfied a-synchronically 2. Variable ordering

General-purpose Vs. tailor-made Decide() strategies...  :...  (x 5 = ( y 4  z 5  u 4 )) ... x 5 = T y 4 = F z 5 = F u 4 = T General purpose Back- track x 5 = T y 4 = F z 5 = F u 4 = T Use  ‘s structure to resolve conflicts on a more local level... Tailor made Back- track

I0I0 PkPk Riding on unreachable states...  k  should satisfy I 0 I0I0 Riding on legal executions...  (k) should satisfy  P k PkPk 2. Variable ordering (simple static ordering)

Given an order, guess a value  Dynamic decision  Constant value  Previous value  ‘Flat’ computation ... Previous value x 2 = 1 y 7 = 0 z 2 = 0 y 3 = 1 x 2 = 0 y 7 = 0 z 2 = 0 y 3 = 1

Can this regularity be used to speed up the search ? 3. Exploiting  (k)’s structure

 Conflict clauses is the main mechanism for learning  If (x 3 =1, y 7 = 0, z 5 = 1 ) leads to a conflict, add the conflict clause C: (  x 3  y 7   z 5 )

3. Exploiting  (k)’s structure (Replicated clauses)  If x 3 =1, y 7 = 0, z 5 = 1 leads to a conflict, then so will x 2 =1, y 6 = 0, z 4 = 1  Therefore, we can also add: (  x 2  y 6   z 4 )  …  (  x 0  y 4   z 2 ) and... (  x 4  y 8   z 6 )  …  (  x k-4  y k   z k-2 )  Yet,  (k) is not fully symmetric because of I 0. Check whether the clauses that caused the conflict include I 0 variables.

4. Exploiting  (k)’s structure (Reusing clauses)  When can a conflict clause C that was learned while solving  (k) be reused for solving  (k+1)?  Answer: all clauses that together implied C are in  (k) Å  (k+1)  All clauses except the property are in  (k) Å  (k+1)

Results (Sec.) * * * = exceeds 10,000 sec. )Today, Chaff solves all in 7 minutes…)

Results (sec.)

The Conclusion  The original conclusion (2000): Many models that cannot be solved by BDD symbolic model checkers, can be solved with the optimized SAT Bounded Model Checker. The other direction is true as well  Today: BMC with SAT is dominant in finding shallow errors. BDD-based procedures are mainly used for proving their absence.

How big should k be?  For every model M and LTL property  there exists k s.t.  The minimal such k is the Completeness Threshold (CT)

How big should k be?  Diameter d = longest shortest path from an initial state to any other reachable state.  Recurrence Diameter rd = longest loop-free path.  rd ¸ d d = 2 rd = 3

How big should k be?  Theorem: for Gp properties CT = d s0s0 pp Arbitrary path

How big should k be?  Theorem: for Fp properties CT= rd s0s0 pp pp pp pp pp  Open Problem: The value of CT for general Linear Temporal Logic properties is unknown

The General case  Buchi automata B: h S,S 0, ,F,L i S - States S 0 µ S - Initial states  µ S £ S - Transition relation F µ S - Accepting set L: S ! 2 AT - Labeling function  Let inf(W) be the set of states visited infinite no. of times by a run W  B accepts W iff there exists f 2 F s.t. inf(W) Å f  ;

The General case  Every LTL formula  can be represented by a Buchi automaton B 

LTL model checking  Given M, , construct B    LTL model checking: is  : M £ B   empty ?  Emptiness checking: is there a path to a loop with an accepting state ?  ! witness to G true with fairness constraint  M ²  iff  is empty

LTL Bounded Model Checking  “Unroll”  k times  Find a witness to Gtrue with the fairness constraint s0s0 f