1. Carnegie Mellon University Formal Verification of Infinite-State Systems Using Boolean Methods Formal Verification of Infinite-State Systems Using Boolean Methods http://www.cs.cmu.edu/~bryant Randal E. Bryant Contributions by graduate students: Sanjit Seshia, Shuvendu Lahiri

2. – 2 – Outline Task Formally verify hardware and software systems Build on success in verifying finite models Infinite-State Models Need logic that is suitably expressive, yet remains reasonably tractable Verification Techniques Solve problems by mapping into propositional logic Proof engines can use powerful Boolean methods Different levels of automation and capacity

3. – 3 – Truly Infinite-State Systems Systems where want to model real-world values (temperature, speed,...) Hybrid systems Very difficult to verify Systems with real-valued time constraints E.g., timed automata Somewhat easier to verify, since all clocks move at same rate Air Bag Controller Speedometer Reading Accelerometer Reading Deploy!

4. – 4 – Theoretically Infinite-State Systems Systems with unbounded buffers Even though can’t really build one tailhead In Use

5. – 5 – Arbitrarily Large Finite-State Systems Synchronization protocol that should work for arbitrary number of processes Verify for arbitrary N Circular buffer with fixed, but arbitrary capacity Verify for arbitrary value of Max tail head In Use 0 Max-1 P2P2 PNPN P1P1

6. – 6 – Very Large Finite-State Systems Abstract 32-bit words as arbitrary integers View memories as having unbounded capacity Reg. File IF/ID Instr Mem +4 PC ID/EX ALUALU EX/WB = = Rd Ra Rb Imm Op Adat Control

7. – 7 – Microprocessor Report, Oct. 28, 1996 Example: HP/Compaq Alpha 21264 Pipeline State Multiple caches Instruction queues Dynamically- allocated registers Memory queue Many buffers between stages Verification Tasks Does it implement the Alpha ISA?

8. – 8 – Abstracting Data from Bits to Integers View Data as Symbolic “Terms” Arbitrary integers Verification proves correctness of design for all possible word sizes Can store in memories & registers Can select with multiplexors ITE: If-Then-Else operation x0x0 x1x1 x2x2 x n-1 x  1010 x y p ITE(p, x, y) 1010 x y T x 1010 x y F y

9. – 9 – Abstraction Via Uninterpreted Functions For any Block that Transforms or Evaluates Data: Replace with generic, unspecified function Only assumed property is functional consistency: a = x  b = y  f (a, b) = f (x, y) ALUALU f

10. – 10 – Abstraction Via Uninterpreted Functions For any Block that Transforms or Evaluates Data: Replace with generic, unspecified function Also view instruction memory as function Reg. File IF/ID Instr Mem +4 PC ID/EX ALUALU EX/WB = = Rd Ra Rb Imm Op Adat Control F1F1 F2F2 F3F3

11. – 11 – EUF: Equality with Uninterp. Functs Decidable fragment of first order logic Formulas ( F ) Boolean Expressions  F, F 1  F 2, F 1  F 2 Boolean connectives T 1 = T 2 Equation P (T 1, …, T k ) Predicate application Terms ( T ) Integer Expressions ITE(F, T 1, T 2 ) If-then-else Fun (T 1, …, T k ) Function application Functions ( Fun ) Integer  Integer f Uninterpreted function symbol Read, Write Memory operations Predicates ( P ) Integer  Boolean p Uninterpreted predicate symbol

12. – 12 – Decision Problem Logic of Equality with Uninterpreted Functions (EUF) Truth Values Dashed Lines Model Control Logical connectives Equations Integer Values Solid lines Model Data Uninterpreted functions If-Then-Else operationTask Determine whether formula is universally valid True for all interpretations of variables and function symbols

13. – 13 – Finite Model Property for EUF Observation Any formula has limited number of distinct expressions Only property that matters is whether or not different terms are equal x0x0 d0d0 f (x0)f (x0) f (d0)f (d0)

14. – 14 – Boolean Encoding of Integer Values For Each Expression Either equal to or distinct from each preceding expression Boolean Encoding Use Boolean values to encode integers over small range EUF formula can be translated into propositional logic Tautology iff original formula valid Expression Possible Values Bit Encoding x0x0{0}00 d0d0{0,1}0 b 10 f (x0)f (x0){0,1,2} b 21 b 20 f (d0)f (d0){0,1,2,3} b 31 b 30

15. – 15 – An Out-of-order Processor (OOO) Data Dependencies Resolved by Register Renaming Map register ID to instruction in reorder buffer that will generate register value Inorder Retirement Managed by Retirement Buffer FIFO buffer keeping pending instructions in program order Reorder Buffer Fields PC Program memory Reorder Buffer valid value src1valid src1val src1tag src2valid src2val src2tag dest op result bus DECODEDECODE Register Rename Unit valid tag val ALU headtail incr dispatch retire execute result 1 st Operand 2 nd Operand

16. – 16 – ALU execute Directly Addressable Select particular entry for execution Retrieve result value from executed instruction Access Modes for Reorder Buffer FIFO Insert when dispatch Remove when retire Content Addressable Broadcast result to all entries with matching source tag headtail RetireDispatch result busGlobal Flush all queue entries when instruction at head causes exception

17. – 17 – Required Logic Increased Expressive Power Model queue pointers Increment & decrement operations Relative ordering Ability to construct complex memory structures Not just set of fixed memory types Don’t Go Too Far Want practical decision procedures Efficient reduction to propositional logic

18. – 18 – EUF  CLU Terms ( T ) ITE(F, T 1, T 2 ) If-then-else Fun (T 1, …, T k ) Function application Formulas ( F )  F, F 1  F 2, F 1  F 2 Boolean connectives T 1 = T 2 Equation P(T 1, …, T k ) Predicate application succ (T) Increment pred (T) Decrement T 1 < T 2 Inequality

19. – 19 – EUF  CLU (Cont.) Functions ( Fun ) f Uninterpreted function symbol Read, Write Memory operations Predicates ( P ) p Uninterpreted predicate symbol x 1, …, x k. T Function lambda expression x 1, …, x k. F Predicate lambda expression Arguments can only be terms Lambdas are just mutable arrays

20. – 20 – Modeling Memories with ’s Memory M Modeled as Function M(a): Value at location aInitially Arbitrary state Modeled by uninterpreted function m 0 Writing Transforms Memory M = Write(M, wa, wd) a. ITE(a = wa, wd, M(a)) Future reads of address wa will get wd M a M a m0m0 M M a 1010 wd = wa

21. – 21 – Modeling Unbounded FIFO Buffer Queue is Subrange of Infinite Sequence Q.head = h Index of oldest element Q.tail = t Index of insertion location Q.val = q Function mapping indices to values q(i) valid only when h  i < t Initial State: Arbitrary Queue Q.head = h 0, Q.tail = t 0 Impose constraint that h 0  t 0 Q.val = q 0 Uninterpreted function q(h–2) q(h–1) q(h) q(h+1) q(t–2) q(t–1) q(t) q(t+1) tail head Already Popped Not Yet Inserted increasing indices

22. – 22 – Modeling FIFO Buffer (cont.) t q(h–2) q(h–1) q(h) q(h+1) q(t–2) q(t–1) q(t) q(t+1) h next[h] := ITE(operation = POP, succ(h), h) next[q] := (i). ITE((operation = PUSH & i=t), x, q(i)) next[t] := ITE(operation = PUSH, succ(t), t) q(h–2) q(h–1) q(h) q(h+1) q(t–2) q(t–1) x q(t+1) next[t] next[h] op = PUSH Input = x

23. – 23 – Systems of Identical Processes Each Process has k State Variables Each state variable represented as array Indexed by process Id sv 1 sv 2 sv k State of Process i

24. – 24 – Modeling System of Identical Processes On Each Step: Select arbitrary process index p As if chosen by nondeterministic scheduler Update state for selected process next[state] := lambda(i) case i = p & state(i) = IDLE: TRYING i = p & state(i) = TRYING & inuse : TRYING i = p & state(i) = TRYING & !inuse: CRITICAL default: state(i) esac IDLE TRYING CRITICAL state 0/1 inusep

25. – 25 – Decision Procedure Operation Series of transformations leading to propositional formula Propositional formula checked with BDD or SAT tools Bryant, Lahiri, Seshia [CAV02] Lambda Expansion Function & Predicate Elimination Convert to Boolean Formula Boolean Satisfiability CLU Formula -free Formula Function-free Formula Boolean Formula

26. – 26 – Finite Model Property for CLU Observation Need to encode all possible relative orderings of expressions Each symbolic value has maximum range of increments & decrements Can use Boolean encodings of small integer ranges x  y  succ(x) > pred(y) xx+1 y –1y y xx+1 y –1y xx+1 y –1y xx+1x y –1y x = 0, y = 3x = 2, y = 1

27. – 27 – Verifying OOO Lahiri, Seshia, & Bryant, FMCAD 2002Goal Show that OOO implements Instruction Set Architecture (ISA) model For all possible execution sequencesChallenges No bound on program length OOO holds partially executed instructions in reorder buffer States of two systems match only when reorder buffer flushed ISA Reg. File PC OOO Reg. File PC Reorder Buffer

28. – 28 – Adding Shadow State McMillan, ‘98 Arons & Pnueli, ‘99 Provides Link Between ISA & OOO Models Additional entries in ROB Do not affect OOO behavior Generated when instruction dispatched Predict values of operands and result From ISA model ISA Reg. File PC OOO Reg. File PC Reorder Buffer

29. – 29 – Adding Shadow Structures shdw.src1val[rob.tail]  Rf isa (src1) shdw.src2val[rob.tail]  Rf isa (src2) shdw.value[rob.tail]  ALU(Rf isa (src1), Rf isa (src2), op) shdw.value shdw.src1val shdw.src2val Shadow Fields Updated directly from the ISA model during dispatch result bus Reorder Buffer Fields PC Program memory Reorder Buffer valid value src1valid src1val src1tag src2valid src2val src2tag dest op DECODEDECODE Register Rename Unit valid tag val ALU headtail incr dispatch retire execute

30. – 30 – Invariant Checking Formulas I 1, …, I n I j (s 0 ) holds for any initial state s 0, for 1  j  n I 1 (s)  I 2 (s)  …  I n (s)  I j (s ) for any current state s and successor state s for 1  j  n Invariants for OOO (13) Refinement maps (2) Show relation between ISA and OOO models Shadow state (3) Shadow values correctly predict OOO values State consistency (8) Properties of OOO state that ensure proper operation Overall Correctness Follows by induction on time

31. – 31 – Refinement Maps Correspondence with a sequential ISA model OOO and ISA synchronized at dispatch For Register File Contents  r. reg.valid(r)  reg.val(r) = Rf isa (r) For Program Counter PC ooo = PC isa shdw.value shdw.src1val shdw.src2val Shadow Fields valid value src1valid src1val src1tag src2valid src2val src2tag dest op Reorder Buffer Fields PC Program memory Reorder Buffer result bus DECODEDECODE Register Rename Unit valid tag val ALU headtail incr dispatch retire execute

32. – 32 – Shadow Invariants 1. 1.  rob t. rob.valid(t)  rob.value(t) = shdw.value(t) 2. 2.  rob t. rob.src1valid(t)  rob.src1val(t) = shdw.src1val(t) 3. 3.  rob t. rob.src2valid(t)  rob.src2val(t) = shdw.src2val(t) shdw.value shdw.src1val shdw.src2val Shadow Fields result bus Reorder Buffer Fields PC Program memory Reorder Buffer valid value src1valid src1val src1tag src2valid src2val src2tag dest op DECODEDECODE Register Rename Unit valid tag val ALU headtail incr dispatch retire execute

33. – 33 – State Consistency Invariants Tag Consistency invariants (2) Instructions only depend on instruction preceding in program order Register Renaming invariants (2) Tag in a rename-unit should be in the ROB, and the destination register should match  r.  reg.valid(r)  ( rob.head  reg.tag(r) < rob.tail  rob.dest(reg.tag(r)) = r ) For any entry, the destination should have reg.valid as false and tag should contain this or later instruction  rob t.(  reg.valid(rob.dest(t))  t  reg.tag(rob.dest(t)) < rob.tail)

34. – 34 – State Consistency Invariants (cont.) Executed instructions have operands ready  rob t. rob.valid(t)  rob.src1valid(t)  rob.src2valid(t) Shadow-Value-Operands Relationship  rob t. shdw.value(t) = Alu(shdw.src1val(t),shdw.src2val(t),rob.op(t)) Producer-Consumer Values (2)  rob t.  rob.src1valid(t)  shdw.src1val(t) = shdw.value(rob.src1tag(t))

35. – 35 – Quantified Invariants and Proofs Allowed Form  x 1  x 2 …  x k  (x 1 …x k )  (x 1 …x k ) is a CLU formula without quantifiers x 1 …x k are integer variables free in  (x 1 …x k ) Proving these invariants requires quantifiers |= (  x 1  x 2 …  x k  (x 1 …x k ))   y 1  y 2 …  y m  (y 1 …y m ) Prove  x 1  x 2 …  x k [  (x 1 …x k )   (y 1 …y m )] is not satisfiable Undecidable Automatic instantiation of with concrete terms Automatic instantiation of x 1 …x k with concrete terms Sound but incomplete method Reduce the quantified formula to a CLU formula Can use the decision procedure for CLU

36. – 36 – Proving Invariants Proved Automatically Quantifier instantiation was sufficient in these cases Time spent = 54s on 1.4GHz machine Total effort = 2 person daysComparison Previous efforts using theorem provers took weeks of effort

37. – 37 – Extending the OOO Processor base Executes ALU instructions only exc Handles arithmetic exceptions Must flush reorder buffer exc/br Handles branches Predicts branch & speculatively executes along path exc/br/mem-simp Adds load & store instructions Store commits as instruction retires exc/br/mem Stores held in buffer Can commit later Loads must scan buffer for matching addresses

38. – 38 – Comparative Verification Effort baseexcexc / brexc / br / mem-simp exc / br / mem Total Invariants 1334396771 Manually instantiate 00048 UCLID time 54 s236 s403 s1594 s2200 s Person time 2 days5 days2 days15 days10 days

39. – 39 – “I Just Want a Loaf of Bread” Ingredients Recipe Result

40. – 40 – Cooking with Invariants Ingredients: Predicates Recipe: Invariants Result: Correctness reg.valid(r)  r,t.  reg.valid(r)  reg.tag(r) = t  ( rob.head  reg.tag(r) < rob.tail  rob.dest(t) = r ) rob.head  reg.tag(r) reg.tag(r) = t rob.dest(t) = r

41. – 41 – Automatic Recipe Generation Want Something More Given any set of ingredients Generate best recipe possible Ingredients Recipe Creator Result

42. – 42 – Automatic Predicate Abstraction Graf & Saïdi, CAV ‘97Idea Given set of predicates P 1 (s), …, P k (s) Boolean formulas describing properties of system state View as abstraction mapping: States  {0,1} k Defines abstract FSM over state set {0,1} k Form of abstract interpretation Do reachability analysis similar to symbolic model checking Prior Implementations Very weak inference capabilities Call theorem prover or decision procedure to test each potential transition Little support for quantified predicates

43. – 43 – Abstract State Space Concrete States Abstract States  P 1 (s), …, P k (s)  sAbstraction Function  t Abstraction Concrete States Abstract States st Concretization Function  Concretization

44. – 44 – Abstract State Machine Transitions in abstract system mirror those in concrete Abstract  Concrete System Abstract System s Concretize  t t s Concrete Transition Abstract Transition

45. – 45 – Overapproximation by Abstract Model Path in abstract state space may not correspond to one in concrete OK when verifying safety properties Possible false negatives, but no false positives Concrete System Abstract System

46. – 46 – Predicate Abstraction Example State Space State variables: { x, y } Initial State { (2, 1) } Next State Behavior x   x y   y Verification Task Prove all bad states unreachable Initial State Bad States

47. – 47 – Precise Analysis Reachable States { (2, 1), (  2,  1) } Reachable States Bad States

48. – 48 – Predicates Use 3-valued predicates in this example c x:3 L E G cx:ycx:y L E G c y:0 G E L

49. – 49 – Abstract Initial State Reached Set #0 { LGG } c x:3 L cx:ycx:y G c y:0 G

50. – 50 – Step 1: Concretize Reached Set #0 Reached Set #0 { LGG } L G G c x:3 cx:ycx:y c y:0 (Note loss of precision) s Concretize  s  s 

51. – 51 – Compute Possible Successor States x  xy  yx  xy  y s Concrete Transition s Concretize  s  s  s Concrete Transition s

52. – 52 – Abstract Newly Reached States 000 Reached Set #1 { LLL, LGG } L L L c x:3 cx:ycx:y c y:0 s Concrete Transition Abstract    s Concretize  s  s  s Concrete Transition s

53. – 53 – Step 2: Concretize Reached Set #1 Reached Set #1 { LLL, LGG } L c x:3 cx:ycx:y c y:0 (Note loss of precision) L L s Concretize  s  s 

54. – 54 – Compute Possible Successor States x  xy  yx  xy  y s Concrete Transition s Concretize  s  s  s Concrete Transition s

55. – 55 – Abstract Newly Reached States Reached Set #2 { LLL, LGG, EGG, GGG } c x:3 G E cx:ycx:y G c y:0 G s Concrete Transition Abstract    s Concretize  s  s  s Concrete Transition s

56. – 56 – EGG Final Reached State Set LLL LGGGGG Bad States

57. – 57 – Symbolic Formulation of Step 2 Concretized State Set Encode each 3-valued {L, E, G} predicate with 2 Boolean variables ( l, g ) Represent state set as formula ( l 1   g 1  l 2   g 2  l 3   g 3 )  ( l 1   g 1   l 2  g 2   l 3  g 3 ) Reached Set #1 { LLL, LGG } LLL LGG l 1 : x < 3 g 1 : x > 3 l 2 : x < y g 2 : x > y g 3 : y > 0 l 3 : y < 0

58. – 58 – Next-State Predicates Next State (x, y ) Get predicates l 1, l 2, l 3, g 1, g 2, g 3 Determine conditions under which predicates will hold in next state Express in terms of current state (x, y) Next State Predicate Condition x =  x y =  yCurrentStateMatches l1l1 x < 3  x < 3x >  3 — l2l2 x < yx < y x < yx < y x > yx > y g2g2 l3l3 y < 0  y < 0 y > 0 g3g3 g1g1 x > 3  x > 3x < 3x < 3 — g2g2 x > yx > y x > yx > y x < yx < y l2l2 g3g3 y > 0  y > 0 y < 0 l3l3

59. – 59 – Consistency Constraints Eliminate impossible predicate combinations In general, may need to introduce additional variables To express more complex transitivity constraints l1l1 g1g1 l2g2l2g2 g2l2g2l2 g3l3g3l3 l3g3l3g3 l1l1 g1g1 (g1  g1)(g1  g1) (g1  l1)(g1  l1) (g2  g3  l1)(g2  g3  l1)

60. – 60 – Symbolic Form Formulation Express compatible combinations of current-state & next- state variables Quantify out current-state variables Gives formula over next-state variables lglglg [( l 1   g 1  l 2   g 2  l 3   g 3 ) lglglg  ( l 1   g 1   l 2  g 2   l 3  g 3 ) ] Current State l 1, l 2, l 3, g 1, g 2, g 3  l 1, l 2, l 3, g 1, g 2, g 3 Consistency Constraints ggglggl   ( g 1  g 1 )   ( g 1  l 1 )  ( g 2  g 3  l 1 ) lggl  l 2  g 2  g 2  l 2 lggl  l 3  g 3  g 3  l 3

61. – 61 – l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 Next State 101010000101EGG 101010010101GGG 101010100101LGG 100101101010LLL l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 Next State 101010000101EGG Extracting Next-State Set Run SAT checker over formula Generate blocking clause for each newly generated state lglglg [( l 1   g 1  l 2   g 2  l 3   g 3 ) lglglg  ( l 1   g 1   l 2  g 2   l 3  g 3 ) ] ggglggl   ( g 1  g 1 )   ( g 1  l 1 )  ( g 2  g 3  l 1 ) lggl  l 2  g 2  g 2  l 2 lggl  l 3  g 3  g 3  l 3   (  l 1   g 1   l 2  g 2   l 3  g 3 )

62. – 62 – General Principle Lahiri, Bryant, and Cook CAV 2003Basis Abstract state set described as formula over Boolean variables P = p 1, …, p k Current state given by formula  ( p 1, …, p k ) Generate Quantified CLU Formula Describing Next Abstract State Set Concretize set of current states as formula  (P 1 (s), …, P k (s)) Combine predicates with next-state functions to get next- state predicates P 1 (s, x), …, P k (s, x) Variables x encode nondeterministic choices Get formula  s, x [  (P 1 (s), …, P k (s))  p 1  P 1 (s, x)  …  p k  P k (s, x) ]

63. – 63 – General Principle (cont.) Transform into Quantified Boolean Formula Formula of form Next( P ) =  Z  (Z, P ) Z: Integer and function variables P : Abstract state variables Translate into Boolean formula of form  B  ( B, P ) B : Boolean variables arising from small-domain encoding Key Property { P |  (Z, P ) satisfiable } = { P |  ( B, P ) satisfiable } Form Encountered in Symbolic Model Checking Next( S ) =  S [ Curr( S )  Trans( S, S ) ] S : Boolean variables encoding current state S : Boolean variables encoding next state Solve using either SAT or BDDs

64. – 64 – Quantified Invariant Generation User supplies predicates containing free variables Generate globally quantified invariantExample Predicates p 1 : reg.valid(r) p 2 : rob.dest(t) = r p 3 : reg.tag(r) = t Abstract state satisfying ( p 1  p 2   p 3 ) corresponds to concrete state satisfying  r,t[  reg.valid(r)  reg.tag(r) = t  rob.dest(t) = r] rather than  r[  reg.valid(r)]   r,t[reg.tag(r) = t]   r,t[rob.dest(t) = r]

65. – 65 – Generating Quantified Invariants Use Quantifier Instantiation to Approximate  During Concretization Causes even greater overapproximation Similar technique used by Flanagan & Qadeer, POPL ‘02

66. – 66 – Systems Verified with Predicate Abstraction Very general models Unbounded processes, buffers, cache lines, … Safety properties only Model PredicatesIterationsCPU Time Out-Of-Order Execution Unit2592,613s German’s Cache Protocol219122s German’s Protocol, unbounded channels 301915,000s Bounded Retransmission Buffer22911s Lamport’s Bakery Algorithm24 5,211s

67. – 67 – Other Uses of UCLID Verifier Invariant Checking More complex version of OOO including speculative execution, exceptions, & buffered loads & stores Lahiri & Bryant, CAV 2003 Predicate Abstraction Core algorithm used to generate weakest Boolean precondition for software model checking SLAM project at Microsoft Pipelined Processor Verification Verify checker processor from U. Michigan Model extracted directly from Verilog Bounded check of load-store unit from industrial microprocessor

68. – 68 – Conclusions CLU is Useful Logic Expressive enough to model wide range of systems Systems with unbounded resources Abstract away most data operations Simple enough to be tractable Small domain property allows exploiting Boolean methods Predicate Abstraction is Powerful Tool Removes requirement to hand-generate invariants Benefits similar to model checking

69. – 69 – Further Work Support for Proofs of Liveness Must make argument that progress being made Greater Automation Automatic generation of predicates More efficient implementation of predicate abstraction More Powerful Logic Linear arithmetic would be useful Potential blow-up when translate to Boolean formula Apply to Other Systems Software Network protocols