SAT Based Abstraction/Refinement in Model-Checking Based on work by E. Clarke, A. Gupta, J. Kukula, O. Strichman (CAV’02)

2 Model Checking Given a: Finite transition system M(S, I, R, L) A temporal property p The model checking problem: Does M satisfy p ?

3 Model Checking Temporal properties: “Always x=y” (G(x=y)) “Every Send is followed immediately by Ack” (G(Send  X Ack)) “Reset can always be reached” (GF Reset) “From some point on, always switch_on” (FG switch_on) “Safety” properties “Liveness” properties

4 Model Checking (safety) I Add reachable states until reaching a fixed-point = bad state

5 Model Checking (safety) I Too many states to handle ! = bad state

6 Abstraction hhhhh Abstraction Function h : S ! S’ S S’

7 Abstraction Function Partition variables into visible( V ) and invisible( I ) variables. The abstract model consists of V variables. I variables are made inputs. The abstraction function maps each state to its projection over V.

8 Abstraction Function h x1 x2 x3 x4 x1 x2 Group concrete states with identical visible part to a single abstract state.

9 Building Abstract Model M’ can be computed efficiently if M is in functional form, e.g. sequential circuits. Abstract x1 x2 x3 x4 x1 x2 i1 i2 i1 i2 x3 x4

10 Existential Abstraction I I

11 Model Checking Abstract Model Preservation Theorem The counterexample may be spurious Converse does not hold

12 Checking the Counterexample Counterexample : (c 1, …,c m ) Each c i is an assignment to V. Simulate the counterexample on the concrete model.

13 Checking the Counterexample Concrete traces corresponding to the counterexample: (Initial State) (Unrolled Transition Relation) (Restriction of V to Counterexample)

14 Abstraction-Refinement Loop Check Counterexample Refine Model CheckAbstract M’, pM, p, h No Bug Pass Fail Bug Real Spurious h’

15 Refinement methods… P Frontier Inputs Invisible Visible (R. Kurshan, 80’s) Localization

16 Generate all counterexamples. Prioritize variables according to their consistency in the counterexamples. X1 x2 x3 x4 (Glusman et al., 2002) Intel’s refinement heuristic Refinement methods…

17 Simulate counterexample on concrete model with SAT If the instance is unsatisfiable, analyze conflict Make visible one of the variables in the clauses that lead to the conflict (Chauhan, Clarke, Kukula, Sapra, Veith, Wang, FMCAD 2002) Abstraction/refinement with conflict analysis Refinement methods…

18 Why spurious counterexample? I I Deadend states Bad States Failure State f

19 Refinement Problem: Deadend and Bad States are in the same abstract state. Solution: Refine abstraction function. The sets of Deadend and Bad states should be separated into different abstract states.

20 Refinement h’ Refinement : h’ h’

21 Refinement Deadend States

22 Refinement Deadend States Bad States

23 Refinement as Separation d1d1 b1b1 b2b2 I V Refinement : Find subset U of I that separates between all pairs of deadend and bad states. Make them visible. Keep U small !

24 Refinement as Separation d1d1 b1b1 b2b I V Refinement : Find subset U of I that separates between all pairs of deadend and bad states. Make them visible. Keep U small !

25 Refinement as Separation The state separation problem Input: Sets D, B Output: Minimal U  I s.t.:  d  D,  b  B,  u  U. d(u)  b(u) The refinement h’ is obtained by adding U to V.

26 Two separation methods ILP-based separation Minimal separating set. Computationally expensive. Decision Tree Learning based separation. Not optimal. Polynomial.

27 Separation with Decision Tree learning (Example) Separating Set : {v 1,v 2,v 4 } DB BDBD 1001 b1b1 d2d2 d1d1 b2b2 v1v1 v4v4 v2v2 01 {d1,b2}{d1,b2}{d2,b1}{d2,b1} DBDB Classification:

28 Separation with 0-1 ILP (Example)

29 Separation with 0-1 ILP One constraint per pair of states. v i = 1 iff v i is in the separating set.

30 Refinement as Learning For systems of realistic size Not possible to generate D and B. Expensive to separate D and B. Solution: Sample D and B Infer separating variables from the samples. The method is still complete: counterexample will eventually be eliminated.

31 Efficient Sampling DB db Let  (D,B) be the smallest separating set of D and B. Q: Can we find it without deriving D and B ? A: Search for smallest d,b such that  (d,b) =  (D,B)

32 Efficient Sampling Direct search towards samples that contain more information. How? Find samples not separated by the current separating set (Sep).

33 Efficient Sampling Recall:  D characterizes the deadend states  B characterizes the bad states  D  B is unsatisfiable Samples that agree on the sep variables: Rename all v i  B to v i ’

34 Efficient Sampling Sep = {} d,b = {} Run SAT solver on  (Sep) STOP unsat Compute Sep:=  (d,b) Add samples to d and b sat Sep is the minimal separating set of D and B

35 The Tool NuSMV Cadence SMV MC Chaff SAT LpSolve Dec Tree Sep

36 Results Property 1

37 Results Property 2 Efficient Sampling together with Decision Tree Learning performs best. Machine Learning techniques are useful in computing good refinements.

38 Current trends (3/3) (Chauhan, Clarke, Kukula, Sapra, Veith, Wang, FMCAD 2002) Abstraction/refinement with conflict analysis

39 The End