Chair of Software Engineering Software Verification Stephan van Staden Lecture 10: Model Checking

2 Overview  The big picture  Example to be verified  Ingredients of a model checking system  The state explosion problem  Applications of model checking  SLAM

3 The big picture  Application domain: mostly concurrent, reactive systems  Verification of particular properties  Verify that a system satisfies a property: 1. Model the system in the model checker’s description language. Call this model M 2. Express the property to be verified in the specification language of the model checker. Call this formula ϕ 3. Run the model checker to show that M satisfies ϕ, i.e. M ⊨ ϕ  Automatic for finite-state models

4 Example 2 processes execute in parallel Each undergoes transitions n → r → c → n → … where  n denotes “not in critical section”  r denotes “requesting to enter critical section”  c denotes “in critical section” Requirements:  Safety: Only one process may execute critical section code at any point  Liveness: Whenever a process requests to enter its critical section, it will eventually be allowed to do so  Non-blocking: A process can always request to enter its critical section

5 Example (cont) We write a program P to fulfill these requirements. But is it really doing its job? We construct a model M for P where M captures the relevant behavior of P: n1n2n1n2 r1n2r1n2 r1r2r1r2 r1c2r1c2 c1r2c1r2 n1r2n1r2 n1c2n1c2 c1n2c1n2 s2s2 s5s5 s7s7 s4s4 s1s1 s0s0 s3s3 s6s6

6 Example (cont) Based on a definition of when a model satisfies a property, we can determine whether M satisfies P’s required properties Example: M satisfies the safety requirement if no state reachable from s 0 (including s 0 ) is labeled c 1 c 2. Thus our M satisfies P’s safety requirement We can formalize the idea Note: the conclusion that P satisfies these requirements depends on the (unverified) assumption that M is a faithful representation of all the relevant aspects of P

7 Ingredients of a model checking system Models, specifications, a satisfaction relation, a satisfaction checking algorithm. Tasks: 1.Define what models are 2.Define what specifications are 3.Define when a model satisfies a specification 4.Devise an efficient algorithm to decide satisfaction

8 1. Define what models are Many formalisms, e.g. Kripke models, LTSs, … A transition system M = (S, →, L) is a set of states S with a transition relation → (a binary relation on S), such that every s S has some s’ S with s → s’, and a labeling function L: S → Ρ(Atoms), where Atoms is a set of atomic descriptions. Our example: a transition system with  S = {s 0, s 1, s 2, s 3, s 4, s 5, s 6, s 7 }  → = {(s 0,s 1 ), (s 0,s 5 ), (s 1,s 2 ), (s 1,s 3 ), (s 4, s 5 ), …}  L(s 0 ) = {n 1,n 2 }, L(s 1 ) = {r 1,n 2 }, L(s 3 ) = {r 1,r 2 }, … where Atoms = {n 1, r 1, c 1, n 2, r 2, c 2 } Define a concrete syntax in which models can be described

9 2. Define what specifications are They describe properties of models Temporal logics (e.g. LTL, CTL) are often used to specify properties of concurrent, reactive systems. Syntax of LTL formulas: ϕ ::= true | false | p | ( ¬ ϕ ) | ( ϕ ˄ ϕ ) | ( ϕ v ϕ ) | ( ϕ → ϕ ) | (X ϕ ) | (G ϕ ) | (F ϕ ) | ( ϕ U ϕ ) | ( ϕ W ϕ ) | ( ϕ R ϕ ) Example specifications: G ¬ (c 1 ˄ c 2 ), G (r 1 → F c 1 ) and G (r 2 → F c 2 ) Define a concrete syntax for specifications

10 3. Define the satisfaction relation For a path π = s 1 → s 2 →... in a model M & LTL formula ϕ : π ⊨ true π ⊨ false π ⊨ p iff p L(s 1 ) π ⊨ ¬ ϕ iff π ⊨ ϕ π ⊨ ϕ 1 ˄ ϕ 2 iff π ⊨ ϕ 1 and π ⊨ ϕ 2 π ⊨ ϕ 1 v ϕ 2 iff π ⊨ ϕ 1 or π ⊨ ϕ 2 π ⊨ ϕ 1 → ϕ 2 iff π ⊨ ϕ 2 whenever π ⊨ ϕ 1 π ⊨ X ϕ iff π 2 ⊨ ϕ (π i = s i → s i+1 →...) π ⊨ G ϕ iff for all i ≥ 1, π i ⊨ ϕ π ⊨ F ϕ iff there is some i ≥ 1 such that π i ⊨ ϕ π ⊨ ϕ 1 U ϕ 2 iff there is some i ≥ 1 such that π i ⊨ ϕ 2 and for all 1 ≤ j < i, π j ⊨ ϕ 1 / / neXt Globally Future Until Weak-until Release

11 3. Define the satisfaction relation (cont.) π ⊨ ϕ 1 W ϕ 2 iff either π ⊨ ϕ 1 U ϕ 2 or for all i ≥ 1, π i ⊨ ϕ 1 π ⊨ ϕ 1 R ϕ 2 iff either there is some i ≥ 1 such that π i ⊨ ϕ 1 and for all 1 ≤ j ≤ i we have π j ⊨ ϕ 2, or for all k ≥ 1, π k ⊨ ϕ 2 Suppose M = (S, →, L) is a model, s S and ϕ is an LTL formula. M,s ⊨ ϕ iff for every path π of M starting at s we have π ⊨ ϕ Thus we are able to prove for our example that:  M,s 0 ⊨ G ¬ (c 1 ˄ c 2 ) (the safety property holds).  M,s 0 ⊨ G (r 1 → F c 1 ) (the liveness property does not hold – a counterexample path is s 0 → s 1 → s 3 → s 7 → s 1 → s 3 → s 7 →...).  The non-blocking property is not expressible in LTL. /

12 4. Provide a checking algorithm Devise an efficient algorithm to decide M ⊨ ϕ Typically without user interaction Approaches: semantic, automata, tableau, … Provide a counterexample when M ⊨ ϕ. The counterexample provides a clue to what is wrong:  The system might be incorrect  The model might be too abstract (in need of refinement)  The specification might not be the intended one /

13 The state explosion problem The number of states grow exponentially with the number of system components For our example: 3 processes execute in parallel, 4 processes, etc. Techniques to make state explosion less severe:  Abstraction  Induction  Partial order reduction  …

14 Applications Verification of specific properties of:  Hardware circuits  Communication protocols  Control software  Embedded systems  Device drivers (e.g. SLAM project of Microsoft)  … Real-world problems: “10 20 states and beyond” Useful: 2007 Turing Award (Clarke, Emerson and Sifakis) We now turn to SLAM

15 SLAM introduction Faulty drivers often responsible for OS crashes Verifying proper driver API usage E.g. a lock is never released without being first acquired Prevent

16 SLAM introduction (cont) Input: a C program P (the device driver code) and temporal safety properties expressed as a SLIC specification ϕ Output (if terminates): P ⊨ ϕ or P ⊨ ϕ /

17 SLIC property specification A SLIC spec ϕ is a state machine Example: state { enum {Unlocked = 0, Locked = 1} state = Unlocked; } KeAcquireSpinLock.return { if (state==Locked) abort; else state = Locked; } KeReleaseSpinLock.return { if (state==Unlocked) abort; else state = Unlocked; } UnlockedLocked Error Rel Acq Rel

18 Translating SLIC spec to C state { enum {Unlocked = 0, Locked = 1} state = Unlocked; } KeAcquireSpinLock.return { if (state==Locked) abort; else state = Locked; } KeReleaseSpinLock.return { if (state==Unlocked) abort; else state = Unlocked; } enum {Unlocked = 0, Locked = 1} state = Unlocked; void slic_abort() { SLIC_ERROR: ; } void KeAcquireSpinLock_return { if (state==Locked) slic_abort(); else state = Locked; } void KeReleaseSpinLock_return { if (state==Unlocked) slic_abort(); else state = Unlocked; }

19 The device driver code P do { KeAcquireSpinLock(); nPacketsOld = nPackets; if (request) { request = request->Next; KeReleaseSpinLock(); nPackets++; } } while (nPackets != nPacketsOld); KeReleaseSpinLock();

20 The instrumented program P’ Compose P with C version of ϕ to obtain an instrumented P’. If SLIC_ERROR is reachable in P’, then P ⊨ ϕ else P ⊨ ϕ. Example P’: / enum {Unlocked = 0, Locked = 1} state = Unlocked; void slic_abort() { SLIC_ERROR: ; } void KeAcquireSpinLock_return { if (state==Locked) slic_abort(); else state = Locked; } void KeReleaseSpinLock_return { if (state==Unlocked) slic_abort(); else state = Unlocked; } do { KeAcquireSpinLock(); A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; if (request) { request = request->Next; KeReleaseSpinLock(); B: KeReleaseSpinLock_return(); nPackets++; } } while (nPacketsOld != nPackets); KeReleaseSpinLock(); C: KeReleaseSpinLock_return();

21 The SLAM verification process Iterative process to determine whether SLIC_ERROR is reachable in P’: prog. P’ prog. P SLIC rule boolean program path predicates slic c2bp bebop newton

22 SLAM iteration Let E 0 be the predicates present in the conditionals of ϕ. For iteration i: 1.Apply c2bp to construct the boolean program BP(P’,E i ) 2.Apply bebop to check if there is a path p i in BP(P’,E i ) that reaches SLIC_ERROR. If bebop determines that SLIC_ERROR is not reachable, then ϕ is valid in P and the algorithm terminates 3.If there such a path p i, then newton checks whether p i is feasible in P’. Two possible outcomes: 1. “yes”: ϕ has been invalidated in P and the algorithm terminates with counterexample p i 2. “no”: newton finds a set of predicates F i that explain the infeasibility of p i in P’ 4.Let E i+1 := E i ∪ F i and i := i + 1 and proceed to the next iteration

23 Some observations The original satisfaction problem has been translated into a boolean program reachability problem. For every statement s in P’ and every predicate e E i, c2bp determines the effect of s on e to build the boolean program.

24 Iteration 0 E 0 = {state = Locked, state = Unlocked} BP(P’,E 0 ): decl {state=Unlocked}, {state=Locked}; {state=Unlocked} := T; {state=Locked} := F; void slic_abort() { SLIC_ERROR: skip; } void KeAcquireSpinLock_return { if ({state=Locked}) slic_abort(); else { {state=Locked} := T; {state=Unlocked} := F; } } void KeReleaseSpinLock_return { if ({state=Unlocked}) slic_abort(); else { {state = Unlocked} := T; {state = Locked} := F; } } do { skip; A: KeAcquireSpinLock_return(); skip; if (*) { skip; B: KeReleaseSpinLock_return(); skip; } } while (*); skip; C: KeReleaseSpinLock_return();

25 Iteration 0 (cont) bebop provides path p 0 = A → A → SLIC_ERROR newton determines that p 0 is infeasible in P’ and returns F 0 = {nPacketsOld = nPackets} as explanation – it must be both true and false on p 0 : do { KeAcquireSpinLock(); A: KeAcquireSpinLock_return(); nPacketsOld = nPackets; if (request) { request = request->Next; KeReleaseSpinLock(); B: KeReleaseSpinLock_return(); nPackets++; } } while (nPacketsOld != nPackets); KeReleaseSpinLock(); C: KeReleaseSpinLock_return();

26 Iteration 1 E 1 = {state = Locked, state = Unlocked, nPacketsOld = nPackets} BP(P’,E 1 ): decl {state=Unlocked}, {state=Locked}; {state=Unlocked} := T; {state=Locked} := F; void slic_abort() { SLIC_ERROR: skip; } void KeAcquireSpinLock_return { if ({state=Locked}) slic_abort(); else { {state=Locked} := T; {state=Unlocked} := F; } } void KeReleaseSpinLock_return { if ({state=Unlocked}) slic_abort(); else { {state = Unlocked} := T; {state = Locked} := F; } } do { skip; A: KeAcquireSpinLock_return(); b := T; if (*) { skip; B: KeReleaseSpinLock_return(); b := F; } } while (!b); skip; C: KeReleaseSpinLock_return();

27 Iteration 1 (cont) bebop establishes that SLIC_ERROR is unreachable in BP(P’,E 1 ) SLAM concludes that P ⊨ ϕ

28 Sources Logic in Computer Science – Modelling and Reasoning about Systems, 2 nd edition, Michael Huth & Mark Ryan, Cambridge University Press, Automatically Validating Temporal Safety Properties of Interfaces, T. Ball & S. K. Rajmani, SPIN 2001 Tutorial on "Software Model Checking with SLAM" at PLDI 2003