1. Bernd Fischer bfischer@cs.sun.ac.za RW714: SAT/SMT-Based Bounded Model Checking of Software

2. Organization Lecture plan: Lecture 1: Modeling and Foundations of BMC Lecture 2: Design of typical software BMC tools (CBMC, ESBMC, LLBMC, bit-blasting, SMT) Lecture 3: Concurrency verification (schedule exploration, sequentialization) Assignment: not yet clear set with last lecture, hand-in Jun 8 (?)

3. “Model” checking software Traditional view of model checking: informal system description model M spec S finite state machine temporal logic: CTL*,CTL,LTL… model checker program P Does M satisfy S?If so, does P satisfy S? software ⋏

4. “Model” checking software Problems in model checking software: building models of programs is difficult and tedious –write Promela-model of Linux kernel ??? ⇒ extract models automatically / check programs directly properties might hold on model but not on program –integer overflow, struct padding,... ⇒ extract (bit-) precise models writing specifications is difficult and tedious –AG(mutex==1 ⇒ EF mutex==0) –AG(“no null-pointer dereference”) ??? ⇒ use language safety conditions and assert

5. Modeling software as transition system Symbolic representation of systems: finite set of variables V = {x 1,…,x n }, over a finite domain D, to encode system states (nodes) describe the set of initial states with a formula over the set of variables:init(x 1,…,x n ) represent transitions (edges) with a formula over variables:trans(x 1,…,x n, x 1 ’,…,x n ’) –use valuation of x 1,…,x n to represent state before transition –use valuation of x 1 ’,…,x n ’ (copy of the variables) to represent state after transition

6. Modeling software as transition system Example: tic-tac-toe V = {x 1, x 2, …, x 9, t}, D = {−, X, O, A, B} variable x i encodes the content of cell i –“−” stands for empty cell –“X” stands for marked by player A –“O” stands for marked by player B variable t encodes the player that has to move next

7. Modeling software as transition system Example: tic-tac-toe Initial configurations –all cells are empty and either player A or B can start init: (x 1 = − ∧ x 2 = − ∧ … ∧ x 9 = − ) ∧ (t=A ∨ t=B)

8. Modeling software as transition system Example: tic-tac-toe Transition relation –player A or B can move, if it is her turn, and mark one of the empty cells trans: ∨ i=1..9 ( move (A,i) ∨ move (B,i) ) move (A,i) : (t = A ∧ t’ = B) ∧ (x i = − ∧ x i ’ = X) ∧ ∧ j=1,..,9, j≠i (x j ’ = x j ) it is A’s turn to move, and B will move next Cell-i was empty, and now marked with X All the other cells maintain their content 8

9. Modeling software as transition system Example: tic-tac-toe Winning condition for player A –three marks in a row winA: (x 1 = X ∧ x 2 = X ∧ x 3 = X) ∨ (x 1 = X ∧ x 5 = X ∧ x 9 = X) ∨ … 9

10. Modeling software as transition system Boolean programs: simple programming language with only Boolean variables only assignments, if -, and while -statements non-deterministic assignments: x := *; no procedure calls (here, but possible in general) Boolean programs are useful as intermediate language: 10 Device driver Program Device driver Program Boolean Program Boolean Program Abstraction Engine Model Checker Counter- example Counter- example

11. Modeling software as transition system Modeling Boolean programs: Add a distinct label to each assignment statement if else endif while endwhile end begin x 1 = false; while (x 1 ∧ x 2 ) do x 1 = true; endwhile if (x 1 ∨ x 2 ) then x 1 = x 1 ↔ x 2 ; else x 2 = x 1 ↔ x 2 ; endif end L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10

12. Modeling software as transition system Modeling Boolean programs: begin x 1 = false; while (x 1 ∧ x 2 ) do x 1 = true; endwhile if (x 1 ∨ x 2 ) then x 1 = x 1 ↔ x 2 ; else x 2 = x 1 ↔ x 2 ; endif end L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10 V = {x 1, …, x n, pc} init: (pc = L 1 ) original program variables additional variable to encode program counter D = {F, T, L 1,…, L 10 } trans: ??? ⇒ need to encode effect of each statement type

13. Modeling software as transition system Modeling Boolean programs: assignments trans i : pc = L i ∧ pc’ = L i+1 (update prgm counter) ∧ x i ’ ↔ b(x 1,…,x n ) (update var x i ) ∧ ∧ j=1..n (x j ’ = x j ) (copy all other vars) j≠i x i = b(x 1,…,x n );... L i L i+1 b is a Boolean formula over the program vars

14. Modeling software as transition system Modeling Boolean programs: conditionals trans i : ( pc = L i ∧ pc’ = L i+1 ∧ b(x 1,…,x n ) (update prgm counter) ∨ pc = L i ∧ pc’ = L j+1 ∧ ¬ b(x 1,…,x n ) ∨ pc = L j ∧ pc’ = L k+1 ∨ pc = L k ∧ pc’ = L k+1 ) ∧ ∧ j=1..n (x j ’ = x j ) (copy all vars) if(b(x 1,…,x n )) then 〈 then-statement 〉 else 〈 else-statement 〉 endif... L i L i+1 L j L j+1 L k L k+1

15. Modeling software as transition system Modeling Boolean programs: loops trans i : ( pc = L i ∧ pc’ = L i+1 ∧ b(x 1,…,x n ) (enter body) ∨ pc = L i ∧ pc’ = L j+1 ∧ ¬ b(x 1,…,x n ) (exit loop) ∨ pc = L j ∧ pc’ = L i (back to loop head) ) ∧ ∧ j=1..n (x j ’ = x j ) (copy all vars) while(b(x 1,…,x n )) do 〈 body-statement 〉 endwhile... L i L i+1 L j L j+1

16. Example: Modeling software as transition system trans(x 1, x 2, PC, x 1 ’, x 2 ’, PC’) = (PC=L 1 ∧ PC’=L 2 ∧ x 1 ’=F ∧ x 2 ’=x 2 ) ∨ (( (PC=L 2 ∧ PC’=L 3 ∧ (x 1 ∧ x 2 ) ) ∨ (PC=L 2 ∧ PC’=L 5 ∧ ¬(x 1 ∧ x 2 ) ) ∨ (PC=L 4 ∧ PC’=L 2 ) ) ∧ (x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 )) ∨ (PC=L 3 ∧ PC’=L 4 ∧ (x 1 ’= T) ∧ (x 2 ’= x 2 )) ∨ (( (PC=L 5 ∧ PC’=L 6 ∧ (x 1 ∨ x 2 ) ) ∨ (PC=L 5 ∧ PC’=L 8 ∧ ¬ (x 1 ∨ x 2 ) ) ∨ (PC=L 7 ∧ PC’=L 10 ) ∨ (PC=L 9 ∧ PC’=L 10 ) ) ∧ (x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 )) ∨ (PC=L 6 ∧ PC’=L 7 ∧ (x 1 ’= (x 1 ↔x 2 )) ∧ (x 2 ’= x 2 )) ∨ (PC=L 8 ∧ PC’=L 9 ∧ (x 2 ’= (x 1 ↔x 2 )) ∧ (x 1 ’= x 1 )) Begin x 1 = false; while (x 1 ∧ x 2 ) do x 1 = true; endwhile if (x 1 ∨ x 2 ) then x 1 = x 1 ↔ x 2 ; else x 2 = x 1 ↔ x 2 ; endif end L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10 variables: V={x 1, x 2, PC}

17. Modeling software as transition system begin while (x 1 ∧ x 2 ) do x 1 = true; while (x 1 ↔x 2 ) do x=(x 1 ∨ x 2 ); endwhile if (x 1 ∨ x 2 ) then x 3 = x 1 ↔ x 2 ; x 2 = x 1 ∨ x 2 ; else x 2 = x 1 ∧ x 2 ; endif end begin x 1 = false; while (x 1 ∧ x 2 ) do x 1 = true; while (x 1 ↔x 2 ) do if (x 1 ∨ x 2 ) then x 3 =(x 1 ∨ ¬x 2 ); else x 3 =(¬x 1 ∨ x 2 ); endif endwhile end Exercise: encode the following two programs

18. Modeling software as transition system begin while (x 1 ∧ x 2 ) do x 1 = true; while (x 1 ↔x 2 ) do x 1 =(x 1 ∨ x 2 ); endwhile if (x 1 ∨ x 2 ) then x 3 = x 1 ↔ x 2 ; x 2 = x 1 ∨ x 2 ; else x 2 = x 1 ∧ x 2 ; endif end L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10 L 11 L 12 L 13 trans(x 1, x 2, x 3, PC, x 1 ’, x 2 ’, x 3 ’, PC’) = (( (PC=L 1 ∧ PC’=L 2 ∧ (x 1 ∧ x 2 ) ) ∨ (PC=L 1 ∧ PC’=L 7 ∧ ¬(x 1 ∧ x 2 ) ) ∨ (PC=L 6 ∧ PC’=L 1 ) ) ∧ ((x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 ) ∧ (x 3 ’ = x 3 ))) ∨ (PC=L 2 ∧ PC’=L 3 ∧ (x 1 ’= T) ∧ (x 2 ’= x 2 ) ∧ (x 3 ’= x 3 )) ∨ (( (PC=L 3 ∧ PC’=L 4 ∧ (x 1 ↔x 2 ) ) ∨ (PC=L 3 ∧ PC’=L 6 ∧ ¬(x 1 ↔x 2 ) ) ∨ (PC=L 5 ∧ PC’=L 3 ) ) ∧ ((x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 ) ∧ (x 3 ’ = x 3 ))) ∨ (PC=L 4 ∧ PC’=L 5 ∧ (x 1 ’= x 1 ∨ x 2 ) ∧ (x 2 ’= x 2 ) ∧ (x 3 ’= x 3 )) ∨ (( (PC=L 7 ∧ PC’=L 8 ∧ (x 1 ∨ x 2 ) ) ∨ (PC=L 7 ∧ PC’=L 11 ∧ ¬ (x 1 ∨ x 2 ) ) ∨ (PC=L 10 ∧ PC’=L 13 ) ∨ (PC=L 12 ∧ PC’=L 13 ) ) ∧ ((x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 ) ∧ (x 3 ’ = x 3 ))) ∨ (PC=L 8 ∧ PC’=L 9 ∧ (x 3 ’= (x 1 ↔ x 2 )) ∧ (x 1 ’= x 1 ) ∧ (x 2 ’= x 2 )) ∨ (PC=L 9 ∧ PC’=L 10 ∧ (x 2 ’= (x 1 ∨ x 2 )) ∧ (x 1 ’= x 1 ) ∧ (x 3 ’= x 3 )) ∨ (PC=L 11 ∧ PC’=L 12 ∧ (x 2 ’= (x 1 ∧ x 2 )) ∧ (x 1 ’= x 1 ) ∧ (x 3 ’= x 3 ))

19. Modeling software as transition system Begin x 1 = false; while (x 1 ∧ x 2 ) do x 1 = true; while (x 1 ↔x 2 ) do if (x 1 ∨ x 2 ) then x 3 =(x 1 ∨ ¬x 2 ); else x 3 =(¬x 1 ∨ x 2 ); endif endwhile end L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10 L 11 L 12 trans(x 1, x 2, x 3, PC, x 1 ’, x 2 ’, x 3 ’, PC’) = (PC=L 1 ∧ PC’=L 2 ∧ (x 1 ’= F) ∧ (x 2 ’= x 2 ) ∧ (x 3 ’= x 3 )) ∨ ( ( (PC=L 2 ∧ PC’=L 3 ∧ (x 1 ∧ x 2 ) ) ∨ (PC=L 2 ∧ PC’=L 12 ∧ ¬(x 1 ∧ x 2 ) ) ∨ (PC=L 11 ∧ PC’=L 2 ) ) ∧ ((x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 ) ∧ (x 3 ’ = x 3 ))) ∨ (PC=L 3 ∧ PC’=L 4 ∧ (x 1 ’= T) ∧ (x 2 ’= x 2 ) ∧ (x 3 ’= x 3 )) ∨ (( (PC=L 4 ∧ PC’=L 5 ∧ (x 1 ↔x 2 ) ) ∨ (PC=L 4 ∧ PC’=L 11 ∧ ¬(x 1 ↔x 2 ) ) ∨ (PC=L 10 ∧ PC’=L 4 ) ) ∧ ((x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 ) ∧ (x 3 ’ = x 3 ))) ∨ (( (PC=L 5 ∧ PC’=L 6 ∧ (x 1 ∨ x 2 ) ) ∨ (PC=L 5 ∧ PC’=L 8 ∧ ¬ (x 1 ∨ x 2 ) ) ∨ (PC=L 7 ∧ PC’=L 10 ) ∨ (PC=L 9 ∧ PC’=L 10 ) ) ∧ ((x 1 ’ = x 1 ) ∧ (x 2 ’ = x 2 ) ∧ (x 3 ’ = x 3 ))) ∨ (PC=L 6 ∧ PC’=L 7 ∧ (x 3 ’= (x 1 ∨ ¬ x 2 )) ∧ (x 1 ’= x 1 ) ∧ (x 2 ’= x 2 )) ∨ (PC=L 8 ∧ PC’=L 9 ∧ (x 3 ’= (¬x 1 ∨ x 2 )) ∧ (x 1 ’= x 1 ) ∧ (x 2 ’= x 2 ))

20. Reminder: “Model” checking software Problems in model checking software: building models of programs is difficult and tedious –write Promela-model of Linux kernel ??? ⇒ extract models automatically / check programs directly properties might hold on model but not on program –integer overflow, struct padding,... ⇒ extract (bit-) precise models writing specifications is difficult and tedious –AG(mutex==1 ⇒ EF mutex==0) –AG(“no null-pointer dereference”) ??? ⇒ use language safety conditions and assert

21. Checking program transition systems Standard xTL model checking: explore transition system check property AG( ∧ i pc ≠ L assert i ) Check satisfiability: build BDD for transition system add conjunct ∧ i pc ≠ L assert i... but two (tiny) problems: state space explosion ⇒ DPLL-based SAT methods loops ⇒ bounded model checking

22. State space explosion Boolean programs only require one bit per variable... but C programs require 32 ... not to forget heap-allocated memory   ⇒ BDDs do not scale to (large/real) programs Alternative: clausal satisfiability checking (aka SAT solving) still NP-complete, but huge practical gains very efficient implementation techniques

23. SAT solving as enabling technology

24. DPLL satisfiability solving Given a propositional Boolean formula φ in clausal form {{a, b}, {¬ a, b}, { a, ¬b}, {¬ a, ¬b}} determine whether a satisfying assignment of variables to truth values exists. Solvers based on Davis-Putnam-Logemann-Loveland algorithm: 1.If φ = ∅ then SAT 2.If ⃞ ∈ φ then UNSAT 3.If φ = φ’ ∪ {x}then DPLL( φ’[ x ↦ true]) If φ = φ’ ∪ {¬x}then DPLL( φ’[ x ↦ false]) 4.Pick arbitrary x and return DPLL( φ[ x ↦ false]) ∨ DPLL( φ[ x ↦ true]) +NP-complete but many heuristics and optimizations ⇒ can handle problems with 100,000’s of variables {{a, b}, {¬ a, b}, { a, ¬b}} {{ b}, {¬b}} { {b}} {⃞}{⃞} {⃞}{⃞} ∅ a ↦ false a ↦ true b ↦ false b ↦ true

25. Bounded model checking (BMC) Problem: transition relation describes only one step need to glue together multiple copies (one per step) –otherwise get inconsistent formulas (x==x+1) for loops we don’t know how many copies  ⇒ Solution: bound program execution –mainly: number of loop iterations ⇒ This is not a bug, it is a feature: many errors are “shallow”

26. Bounded model checking (BMC) MC: check if a property holds for all states BMC: check if a property holds for a subset of states Init error... k

27. Bounded model checking (BMC) IS THERE ANY ERROR? IS THERE ANY ERROR IN k STEPS? no yes completeness threshold reached k+1 still tractable k+1 intractable no yes M, S ok fail bound MC: BMC: “never” happens in practice

28. Bounded model checking (BMC) Basic Idea: check negation of given property  up to given depth transition system M unrolled k times –for programs: unroll loops, unfold arrays, … translated into verification condition  such that  satisfiable iff  has counterexample of max. depth k has been applied successfully to verify (sequential) software... M0M0 M1M1 M2M2 M k-1 MkMk ¬0¬0 ¬1¬1 ¬2¬2 ¬  k-1 ¬k¬k counterexample trace     transition system property bound k+1 copies

29. BMC: system unwinding – variables Each S 0, S 1, S 2, …, S k uses its own copies of X S 0 = {x 1 0, x 2 0, …, x n 0 } S 1 = {x 1 1, x 2 1, …, x n 1 } … S k = {x 1 k, x 2 k, …, x n k } S0S0 S0S0 S1S1 S1S1 S2S2 S2S2 SkSk SkSk init(S 0 ) trans(S 0,S 1 ) trans(S 1,S 2 ) target(S i ) =TRUE ??? for some i=0,1,2,…,k

30. BMC: system unwinding – formula Ψ k = init(S 0 ) ∧ ( ∧ i=0,…,k-1 ( trans(S i, S i+1 ) ) ) ∧ ( ∨ i=0,…,k ( target(S i ) ) ) S0S0 S0S0 S1S1 S1S1 S2S2 S2S2 SkSk SkSk init(S 0 ) trans(S 0,S 1 ) trans(S 1,S 2 ) target(S i ) =TRUE ??? for some i=0,1,2,…,k Ψ k is satisfiable iff target is reachable within k steps Size: |init| k |trans| k |target|

31. BMC: completeness threshold Computing CT is as hard as model checking. Idea: Compute an over-approximation to actual CT –Consider system P as a graph. –Compute CT from structure of P. IS THERE ANY ERROR IN k STEPS? completeness threshold reached k+1 still tractable k+1 intractable no yes M, S ok fail bound BMC: “never” happens in practice

32. BMC: completeness threshold Theorem: for AGp properties CT = D I (M) Diameter D(M) = longest shortest path between any two reachable states. Recurrence Diameter RD(M) = longest loop-free path between any two reachable states. The initialized versions: D I (M) and RD I (M) start from an initial state. D(M) = 2 RD(M) = 3 D I (M) = RD I (M) =

33. BMC: proving correctness Correctness can be shown via BMC: in k steps: no error is reachable in k steps: all states are reachable incomp k = init(S 0 ) ∧ ( ∧ i=0,…,k-1 trans(S i, S i+1 ) ) // there is a path ∧ ( ∧ i=0,…,k-1 ∧ j=i+1,…,k S i ≠ S j ) // all states are distinct If incomp k is not satisfiable then in k steps we reach all states. Hence, the system is correct if Ψ k is not satisfiable incomp k is not satisfiable