Seminar on Scalability in Model Checking Symmetry, parameterized systems, abstraction, and more Orna Grumberg 13.3.2012

Model Checking An efficient procedure that receives: A finite-state model M, describing a system A temporal logic formula , describing a property It returns yes, M |=  no + Counterexample, otherwise

Model of systems S - Set of states. I  S - Initial states. M=<S, I, R, L> S - Set of states. I  S - Initial states. R  S x S - Total transition relation. L: S 2AP - Labeling function. AP – Set of atomic propositions If an atomic proposition is not in L(s) then it is false in s

CTL* -calculus: More expressive than CTL* Allows any combination of temporal operators and path quantifiers More expressive than LTL or CTL ACTL / ACTL* The universal fragments of CTL/CTL* with only universal path quantifiers -calculus: More expressive than CTL*

Main limitations The state explosion problem: Model checking is efficient in time but suffers from high space requirements: The number of states in the system model grows exponentially with the number of variables the number of components in the system Works on finite representations of systems

solutions to the state explosion problem: Symbolic model checking with BDDs SAT-based Bounded and Unbounded model checking

More solutions to the state explosion problem: Abstraction Modular verification Partial order reduction Symmetry Handling infinite-state systems: Verifying parameterized systems Static analysis

Possible solution Replacing the system model by a smaller one (less states and transitions) that still preserves properties of interest Used for instance in: Abstraction Symmetry Parameterized systems Modular verification

Simulation and Bisimulation and their logical characterization

equivalence between models that strongly preserves CTL* (-calculus) We define: equivalence between models that strongly preserves CTL* (-calculus) If M1  M2 then for every CTL* formula , M1 |=   M2 |=  preorder on models that weakly preserves ACTL* If M2  M1 then for every ACTL* formula , M2 |=   M1 |= 

The simulation preorder [Milner] Given two models M1 = (S1,I1,R1,L1), M2 = (S2,I2,R2,L2) H  S1 x S2 is a simulation iff for every (s1, s2 )  H : s1 and s2 satisfy the same propositions For every successor t1 of s1 there is a successor t2 of s2 such that (t1,t2) H Notation: s1  s2

The simulation preorder [Milner] Given two models M1 = (S1,I1,R1,L1), M2 = (S2,I2,R2,L2) H  S1 x S2 is a simulation iff for every (s1, s2 )  H : L(s1) = L(s2) t1 [ (s1,t1)  R1  t2 [ (s2,t2)  R2  (t1,t2) H ] ] Notation: s1  s2

Simulation preorder (cont.) H  S1 x S2 is a simulation from M1 to M2 iff H is a simulation and for every s1  I1 there is s2  I2 s.t. (s1, s2)  H Notation: M1  M2

Bisimulation relation [Park] For models M1 and M2, H  S1 x S2 is a bisimulation iff for every (s1, s2 )  H : L(s2) = L(s1) t1 [ (s1,t1)  R1  t2 [ (s2,t2)  R2  (t1,t2) H ] ] t2 [(s2,t2)  R2  t1 [(s1,t1)  R1  (t1,t2) H ]] Notation: s1  s2 Addition to simulation: for every successor t2 of s2 there is a successor t1 of s1 so that (t1,t2) are in H

Bisimulation relation (cont.) H  S1 x S2 is a Bisimulation between M1 and M2 iff H is a bisimulation and for every s1  I1 there is s2  I2 s.t. (s1, s2 )  H and for every s2  I2 there is s1  I1 s.t. (s1, s2 )  H Notation: M1  M2 If H is a simulation and is symmetric then it is a bisimulation bisimulation over SxS is an equivalence relation Bisimulation over models is an equivalence relation

Bisimulation equivalence M1  M2 a b d c 1 4 3 6 2 5 M1 a b c d 1’ 2’ 3’ 4’ 5’ 6’ M2 bisimulations are not sensitive to: duplications unwinding

H={ (1,1’), (2,4’), (4,2’), (3,5’), (3,6’), (5,3’), (6,3’) } Bisimulation equivalence M1  M2 a b d c 1 4 3 6 2 5 M1 a b c d 1’ 2’ 3’ 4’ 5’ 6’ M2 H={ (1,1’), (2,4’), (4,2’), (3,5’), (3,6’), (5,3’), (6,3’) } bisimulations are not sensitive to: duplications unwinding

H={ (1,1’), (2,4’), (4,2’), (3,5’), (3,6’), (5,3’), (6,3’) } Bisimulation equivalence M1  M2 a b d c 1 4 3 6 2 5 M1 a b c d 1’ 2’ 3’ 4’ 5’ 6’ M2 H={ (1,1’), (2,4’), (4,2’), (3,5’), (3,6’), (5,3’), (6,3’) } a b bisimulations are not sensitive to: duplications unwinding

Simulation preorder M1 M2 wait coin pepsi coke

M1 M2 a b c d M1  M2

M1 M2 a b c d M1  M2

M1 M2 d c a b a b c d M1  M2 and M1  M2 but not M1  M2

(bi)simulation and logic preservation Theorem: If M1  M2 then for every CTL* formula , M1 |=   M2 |=  If M2  M1 then for every ACTL* formula , M2 |=   M1 |=  Holds also for -calculus and its universal fragment True also for mu-calculus and universal mu-calculus respectively

What is the logical characterization M1 sim M2 ? b c d M1  M2 and M2  M1: M1 sim M2 What is the logical characterization M1 sim M2 ?

Abstraction

Abstraction One of the most useful ways to fight the state explosion problem Enables handling infinite-state systems as well They should preserve properties of interest: properties that hold for the abstract model should hold for the concrete model Abstractions should be constructed directly from the program

Abstraction Removes or simplifies details Removes entire components that are irrelevant to the property under consideration, thus reducing # of states Manual abstraction requires great creativity.

Outline for abstraction Define an abstract model that preserves the checked property Consider different types of abstractions Automatically construct an abstract model Different constructions for different types Automatically refine it, if the abstraction is not detailed enough

We first define an abstract model Mh based on a concrete (full) model M of the system Goal: constructing Mh directly from the program text P  Mh M Abstract model Concrete model

Abstraction preserving ACTL/ACTL* We use Existential Abstraction in which the abstract model is an over-approximation of the concrete model: The abstract model has more behaviors But no concrete behavior is lost Every ACTL/ACTL* property true in the abstract model is also true in the concrete model

Existential Abstraction Given an abstraction function h : S  Sh, the concrete states are grouped and mapped into abstract states : Mh M < Mh h M

How to define an abstract model: Given M and , choose Sh - a set of abstract states AP – a set of atomic propositions that label concrete and abstract states h : S  Sh - a mapping from S on Sh that satisfies: h(s) = h(t) only if L(s)=L(t)

The abstract model Mh = ( Sh, Ih, Rh, Lh ) sh  Ih  sI : h(s) = sh (sh,th)  Rh  s,t [ h(s) = sh  h(t) = th  (s,t)R ] Lh(sh) = L(s) for some s where h(s) = sh This is an exact abstraction In other words: s: Lh(h(s)) = L(s)

An approximated abstraction (an approximation ) sh  Ih  sI : h(s) = sh (sh,th)  Rh  s,t [ h(s) = sh  h(t) = th  (s,t)R ] Lh is as before Notation: Mr – reduced (exact) Mh - approximated

Logic preservation Mh (exact or approximated) has less states but more behaviors: Mh  M therefore: Theorem If  is an ACTL/ACTL* specification over AP, then Mh |=   M |= 

Example Program with one variable x over the integers Initially x may be either 0 or 1 At any step, x may non-deterministically either decrease or increase by 1

Example (cont.) Abstraction 1: Sh1 = { a–, a0, a+ } a+ if s(x)>0 h1(s) = a0 if s(x)=0 a– if s(x)<0 Abstraction 2: Sh2 = { aeven, aodd } h2(s) = if even( |s(x)| ) then aeven else aodd

The concrete model Abstraction 2 Abstraction 1 x=0 x=1 x=-1 x=-2 x=3 aeven aodd Abstraction 2 a0 a+ a– Abstraction 1

Types of Abstraction (Sh, h, AP) Localization reduction: each variable either keeps its concrete behavior or is fully abstracted (has free behavior) [Kurshan94] Predicate abstraction: concrete states are grouped together according to the set of predicates they satisfy [GS97,SS99] Data abstraction: the domain of each variable is abstracted into a small abstract domain [CGL94,LONG94] BL94: Balarin& Sangiovanni-vincentelli – localization reduction BLO98 – predicate abstractions

Depending on h and the size of M, Mh (I. e Depending on h and the size of M, Mh (I.e. Ih, Rh ) can be built using: BDDs or SAT solver or Theorem prover / Decision procedures / SMT (sat Modulo Theory) We later demonstrate such constructions for specific types of abstractions

Predicate abstraction Given a program over variables V Predicate Pi is a first-order atomic formula over V Examples: x+y < z2 , x=5 Choose: AP = { P1,…,Pk } that includes the atomic formulas in the property  and conditions in if, while statements of the program

Labeling of concrete states: L(s) = { Pi | s |= Pi }

Abstract model Abstract states are defined over Boolean variables { B1,...,Bk }: Sh  { 0,1 }k h(s) = sh  for all 1jk : [ s |= Pj  sh |= Bj ] Lh(sh) = { Pj | sh |= Bj }

Example Program over natural variables x, y AP = { P1, P2, P3 } where P1 = x≤1 , P2 = x>y , P3 = y=2 AP = { x≤1 , x>y , y=2 } L((0,0)) = L((1,1)) = L(0,1)) = { P1 } L((0,2)) = L((1,2)) = { P1, P3 } L((2,0)) = { P2 }

s,t [ h(s) = sh  h(t) = th  (s,t)R ] h((0,0)) = L((1,1)) = L(0,1)) = (1,0,0) h((0,2)) = L((1,2)) = (1,0,1) h((2,0)) = (0,1,0) No concrete state is mapped to (1,1,1) Lh((1,0,0)) = { P1 } Lh((1,0,1)) = { P1, P3 } The concrete state and its abstract state are labeled identically (sh,th)  Rh  s,t [ h(s) = sh  h(t) = th  (s,t)R ]

Computing Rh (same example) Program with one statement: x := x+1

( (b1,b2,b3) , (b’1,b’2,b’3) )  Rh  xyx’y’ [ P1(x,y)  b1  P2(x,y)  b2  P3(x,y)  b3  x’=x+1  y’=y  P1( x’,y’ )  b’1  P2( x’,y’ )  b’2  P3( x’,y’ )  b’3 ]

Computing Rh using BDDs If the program is over finite, relatively small domains, use BDDs We need BDDs for: the concrete transition relation R (possibly partitioned) the abstraction mapping h Rh is computed according to the formula above, using BDD operations

Computing Rh using SAT or Theorem Prover If the program is over finite, relatively large domains, use SAT If the program is over infinite domains, use theorem prover Theorem prover and SAT solve the problem for each pair of states separately: |Sh|x|Sh| applications

Computing Rh with SAT or Theorem Prover (example cont.) Given Sh = (1,0,0) th = (0,0,0) To determine if (sh, th)  Rh , check: xyx’y’ [ h((x,y)) = (1,0,0)  h((x’,y’)) = (0,0,0)  ((x,y),(x’y’))R ]  xyx’y’ [ P1(x,y) P2(x,y) P3(x,y)  x’=x+1  y’=y  P1(x’,y’) P2(x’,y’)  P3(x’,y’) ]

Using BDDs, Rh is computed based on the concrete model M Using SAT or Theorem prover, Rh can be viewed as computed directly from the program text We later show a more direct construction from the program text

Approximation For BDDs: Approximate the computation of Rh For SAT and theorem prover: In case of check failure for a pair (sh,th), include the pair in Rh

Data Abstraction Given a program over variables v1,…vn where vi is over a domain Di Program states (concrete) S = D1 x…x Dn Choose for each vi: an abstract domain Ai hi : Di  Ai Sh = A1 x…x An

AP = { viA =a | a  Ai , i=1,…n } where viA refers to the abstract value of vi h : S  Sh is defined: h ((d1,…,dn)) = (a1,…,an) where ai = hi(di) L ((d1,…,dn)) ={ viA =ai | hi(di) = ai , i=1,…,n } Lh ((a1,…,an)) ={ viA =ai | i=1,…,n } As before: L(s) = Lh (h(s))

Constructing Mh from program text we assume that the program is given by first order formulas I(V) – describing the initial states R(V, V’ ) – describing the transition relation where V=( v1,...,vn ) and V'=( v1’,...,vn’ ) Instead of a specific programming language we assume that the program is given by first order formulas I [xidi] means that for every i, xi is assigned the value di=s(xi)

Representing a program by formulas: example k: x:=e k’ Formula R(x, pc, x’, pc’) : pc=k  x’=e  pc’=k’

Representing a program by formulas: example (cont) k: if x=0 then k1: x:=1 else k2: x:=x+1 k’ Formula R (x, pc, x’, pc’): ( pc=k  x=0  x’=x  pc’ = k1) ( pc=k  x 0  x’=x  pc’ = k2) ( pc=k1  x’=1  pc’=k’)  ( pc=k2  x’=x+1  pc’=k’)

What does it mean for a formula to represent a model M? I (V) and R (V, V’) represent (describe) the model M=(S, I, R, L) as follows: Let s=(d1,...dn), s’=(d1’,...,dn’) s  I  I [vidi] = true (s, s’)  R  R [vi di, vi’ di’ ] = true

For a formula  over variables v1, ...,vk Notation: For a formula  over variables v1, ...,vk [] (v1A, ..., vkA) = v1,...,vk ( h(v1)= v1A  ...  h(vk)= vkA  (v1, ...,vk) ) Note: [I (V) ] and [R (V, V’)] are formulas over abstract variables [I (V)] and [R (V, V’)] represent Mr For simplicity we assume here that all hi = h, all Di = D and all Ai = A

I(V) and R(V, V’), representing the “program text”, we can construct Given first order formulas I(V) and R(V, V’), representing the “program text”, we can construct [I (V)] and [R (V, V’)], representing Mr For simplicity we assume here that all hi = h, all Di = D and all Ai = A

Problem: Given [I (V) ] and [R (V, V’ )], in order to determine if sr  Ir, we need to find a state s  I (a satisfying assignment for I(V)) so that h(s) = sr Similarly, for (sr, tr)  Rr we look for a satisfying assignment for R (V, V’) This is a difficult task due to the size and complexity of the two formulas

Simplifying the formulas For  in negation normal form over basic predicates pi and pi, T() simplifies [] by “pushing” the existential quantifiers inward: T(pi(v1,...,vn)) = [pi](v1A,...,vnA) T(pi(v1,...,vn)) = [pi](v1A,...,vnA) T(1  2) = T(1)  T(2) T(1  2) = T(1)  T(2) (*) (*) does not preserve equivalence

Approximated model Theorem: []  T() In particular, [I ]  T(I ) and [R ]  T(R ) Corollary: The approximation model Mh, defined by T(I ) and T(R ) satisfies: Mh  Mr  M by the simulation preorder Ma has more initial states and more transitions then Mr and it is therefore greater by the simulation preorder

Approximated model (cont.) Defined over the same set of abstract states as Mr Easier to compute since existential quantifiers are applied to simpler formulas Less precise: has more initial states an more transitions than Mr

Computing approximation model from the text No need to construct formulas. The approximation model can be constructed directly from the program text The user should provide abstract predicates [pi] and [pi] for every basic action (assignment or condition) in the program

Abstract predicates provided by the user: Example statement: x := y+z predicate p(x’,y,z): x’ = y+z A = {aeven, aodd } [p](x’A,yA,zA) = { (aeven, aodd, aodd), (aeven, aeven, aeven), (aodd, aodd, aeven), (aodd, aeven, aodd) } [p](aeven, aodd, aodd) iff x’,y,z ( h(x’) = aeven  h(y) = aodd  h(z)= aodd  x’=y+z )

This is data abstract

Refinement

Logic preservation Theorem Theorem If  is an ACTL/ACTL* specification over AP, then Mh |=   M |=  However, the reverse may not be valid.

Traffic Light Example Property:  =AG AF ¬ (state=red) Abstraction function h maps green, yellow to go. red red go Mh green M |=   Mh |=  yellow M

Traffic Light Example (Cont) If the abstract model invalidates a specification, the actual model may still satisfy the specification. Property:  =AG AF (state=red) red red green go M |=  but Mh |=  yellow Spurious Counterexample: red,go,go, ... M Mh

CounterExample-Guided Abstraction-Refinement (CEGAR)

The CEGAR Methodology M and  Mh Mh |=  Mh |=  generate initial abstraction M and  model check Mh refinement Th is spurious stop Mh |=  generate counterexample Th Mh |=  check spurious counterexample Th Th is not spurious

Refinement versus approximation Remark: Refinement builds new Sh and h, then recomputes Rh and Ih (exact or approximated) Approximation uses the same Sh and h and just adds more transitions and initial states

Assumptions for BDD-based implementation The concrete model M is finite but too big to directly apply model checking on R and I can be held as BDDs in memory, R possibly partitioned: R(V,V’) = i Ri (V, vi’ ) h is held as a BDD over concrete and abstract states CEGAR can also be implemented with SAT or Theorem Prover

Generating the Initial Abstraction If we use predicate abstraction then predicates are extracted from the program’s control flow and the checked property If we use localization reduction then the unabstracted variables are those appearing in the predicates above Predicate abstraction-like is used for the initial abstraction. Differences: we do not define Bi for each predicate; The refinement does not necessarily add new predicates. It just change h.

AP = { reset=TRUE, x<y, x=y, y=2 } Example init(x) := 0 next(x) := case : 0; : x + 1; else : x; esac; init(y) := 1; next(y) := case : 0;  ¬ : y + 1; else : y; esac; reset=TRUE reset=TRUE x < y x=y x = y x < y x=y x = y y=2 y=2  = AF x=y AP = { reset=TRUE, x<y, x=y, y=2 }

Model Check The Abstract Model Given the abstract model Mh If Mh | , then the model checker generates a counterexample trace (Th) Most current model checkers generate paths or loops Question : is Th spurious?

Counterexamples For AGp it is a path to a state satisfying p For AFp it is a infinite path represented by a path+loop, where all states satisfy p On the other hand, For EFp we need to return the whole computation tree (the whole model)

Path Counterexample Assume that we have four abstract states {1,2,3}   {4,5,6}   {7,8,9}   {10,11,12}   Abstract counterexample Th= , , ,      therefore, M |=  Th is not spurious,

Spurious Path Counterexample failure state The concrete states mapped to the failure state are partitioned into 3 sets Th is spurious

Refining The Abstraction Goal : refine h so that the dead-end states and bad states do not belong to the same abstract state. For this example, two possible solutions.

Computing a Concrete Counterexample from an Abstract one Let T = (a1,…an) be a path abstract counterexample h-1(a) = { s | h(s) = a } The set of concrete counterexamples corresponding to T: h-1(T) = { (s1,…sn) | i h(si)=ai  I(s1)  iR(si,si+1) }

BDD-based computation of h-1(a1),…, h-1(an) S1 = h-1(a1)  I For i = 2,…,n do Si = Image(Si-1)  h-1(ai) if Si= then dead-end := Si-1 return(i-1, dead-end) print (“counterexample exists”) Return (S1,…,Sn) If h is an approximation of existential abstraction, then S1 may also be empty The concrete counterexample can be constructed from Sn backwards.

Computing a concrete counterexample from S1,…,Sn tn = choose (Sn) For i = n-1 to 1 ti = choose ( predecessors(ti+1)  Si ) Return ( (t1,…,tn) )

Remark: If h(si) = ai then for every atomic formula f, si |= f  a |= f In particular, if an |= p then sn |= p Therefore, if (a1,…an) is an abstract counterexample then (s1,…sn) is a concrete counterexample

Refining the abstraction Refinement separates dead-end states from bad states, thus, eliminating the spurious transition from Si-1 to Si

Refinement versus approximation Remark: Refinement builds new Sh and h, then recomputes Rh and Ih (exact or approximated) Approximation uses the same Sh and h and just adds more transitions and initial states

Implementing CEGAR With BDDs: The concrete model M is finite but too big to directly apply model checking on R and I can be held as BDDs in memory, R possibly partitioned: R(V,V’) = i Ri (V, vi’) h is held as BDD over concrete and abstract states Can also be implemented with SAT or Theorem Prover

Conclusion We defined several types of abstractions We showed how to extract them from the concrete model or from the program text We presented the CEGAR framework for abstraction-refinement We considered several possible implementations

More on abstraction 3-valued and even multi-valued abstraction Static analysis for computing abstract models directly from program text Abstract interpretation: Instead of h:  : 2S -> Sh : Sh -> 2S

THE END

BDDs: R. E. Bryant, Graph-based Algorithms for Boolean Function Manipulation, IEEE transactions on Computers, 1986 BDD-based model checking: J.R. Burch, E.M. Clarke, K.L. McMillan, D.L. Dill, L.J. Hwang, Symbolic Model Checking: 10^20 States and Beyond, LICS’90 SAT-based Bounded model checking: Symbolic model checking using SAT procedures instead of BDDs, A. Biere, A. Cimatti, E. M. Clarke, M. Fujita, Y. Zhu, DAC'99

Existential abstraction + data abstraction: E. M. Clarke, O Existential abstraction + data abstraction: E. M. Clarke, O. Grumberg, D. E. Long, Model Checking and Abstraction, TOPLAS, 1994. Localization reduction: R. P. Kurshan, Computer-Aided Verification of coordinating processes – the automata theoretic approach, 1994

Predicate abstraction: S. Graf and H Predicate abstraction: S. Graf and H. Saidi, Construction of abstract state graphs with PVS, CAV’97 H. Saidi and N. Shankar, Abstract and Model Check while you Prove, CAV’99 BDD-based CEGAR: Clarke, Grumberg, Jha, Lu, Veith, Counterexample-guided Abstraction Refinement, CAV2000, JACM2003

Which states satisfy f1 = E(a U c) ? a,b a a b,c b a,c a,b c

Which states satisfy f1 = E(a U c) ? a,b a a b,c b a,c a,b c

Which states satisfy f2 = EG b ? a,b a a b,c b a,c a,b c

Which states satisfy f2 = EG b ? a,b a a b,c b a,c a,b c

If the model is given explicitly, e. g If the model is given explicitly, e.g. by adjacent matrix, then only systems of very restricted size can be handled.

Efficient Explicit-state model checking Uses hash table in order to (partially) hold reachable states Collisions may result in repeated work Heuristics such as partial order and symmetry are used to reduce the explored state space Successful for software – can handle 10s of millions of states