Carnegie Mellon University Symbolic Approaches to Invariant Checking and Automatic Predicate Abstraction Randal E. Bryant Contributions by graduate students: Sanjit Seshia, Shuvendu Lahiri

– 2 – Outline Task Prove safety properties of term-level systemsMethod Generate & prove inductive invariants by predicate abstractionFeatures Symbolic approach using SAT enumeration Handles important class of quantified predicates Augment with automated predicate discovery

– 3 – Reachable States Verifying Safety Properties State Machine Model State encoded as Booleans, integers, and functions Next state function expresses how updated on each step Prove: System will never reach bad state Reset States Bad States  Present State Next State Inputs (Arbitrary) Reset

– 4 – RnRn R2R2 True Model Checking Reach Fixed-Point R n = R n+1 = Reachable Impractical for Term-Level Models Many systems never reach fixed point Can keep adding elements to buffer Convergence test undecidable Bad States R1R1 Reset States 

– 5 – I Inductive Invariant Checking Key Properties of System that Make it Operate Correctly Formulate as formula I Prove Inductive Holds initially I (s 0 ) Preserved by all state changes I (s)  I (  (i, s)) Reachable States Reset States Bad States 

– 6 – Inductive Invariants Formulas I 1, …, I n I j (s 0 ) holds for any initial state s 0, for 1  j  n I 1 (s)  I 2 (s)  …  I n (s)  I j (s ) for any current state s and successor state s for 1  j  n Overall Correctness Follows by induction on time Restricted form of invariants  x 1  x 2 …  x k  (x 1 …x k )  (x 1 …x k ) is a CLU formula without quantifiers x 1 …x k are integer variables free in  (x 1 …x k )

– 7 – Restricted Invariants and Proofs Proving invariants inductive requires quantifiers |= (  x 1  x 2 …  x k  (x 1 …x k ))   y 1  y 2 …  y m  (y 1 …y m ) |=  x 1  x 2 …  x k   (x 1 …x k )   y 1  y 2 …  y m  (y 1 …y m ) Universally Quantified Variables “Free” Validity proves holds for all interpretations Existentially Quantified Variables Make Problem Undecidable Automatic instantiation of x 1 …x k with concrete terms Sound but incomplete method Reduce the quantified formula to a CLU formula Can use the decision procedure for CLU

– 8 – Constructing Invariants from Predicates Recipe: Invariants Result: Correctness reg.valid(r)  r,t.  reg.valid(r)  reg.tag(r) = t  ( rob.head  reg.tag(r) < rob.tail  rob.dest(t) = r ) rob.head  reg.tag(r) reg.tag(r) = t rob.dest(t) = r

– 9 – Automatic Predicate Abstraction Graf & Saïdi, CAV ‘97Idea Given set of predicates P 1 (s), …, P k (s) Boolean formulas describing properties of system state View as abstraction mapping: States  {0,1} k Defines abstract FSM over state set {0,1} k Form of abstract interpretation Do reachability analysis similar to symbolic model checkingImplementation Early ones had weak inference capabilities Call theorem prover or decision procedure to test each potential transition Recent ones make better use of symbolic encodings

– 10 – Abstract State Space Concrete States Abstract States  P 1 (s), …, P k (s)  sAbstraction Function  t Abstraction Concrete States Abstract States st Concretization Function  Concretization

– 11 – Abstract State Machine Transitions in abstract system mirror those in concrete Abstract  Concrete System Abstract System s Concretize  t t s Concrete Transition Abstract Transition

– 12 – Generating Concrete Invariant Reach Fixed-Point on Abstract System Termination guaranteed, since finite state Equivalent to Computing Invariant for Concrete System Strongest possible invariant that can be expressed by formula over these predicates RnRn R2R2 R1R1 Reset States AA Abstract System Concretize  Concrete System I Reset States CC

– 13 – Predicate Abstraction Example State Space State variables: { x, y } Initial State { (2, 1) } Next State Behavior x   x y   y Verification Task Prove all bad states unreachable Initial State Bad States

– 14 – Precise Analysis Reachable States { (2, 1), (  2,  1) } Reachable States Bad States

– 15 – Predicates Use 3-valued predicates in this example c x:3 L E G cx:ycx:y L E G c y:0 G E L

– 16 – Abstract Initial State Reached Set #0 { LGG } c x:3 L cx:ycx:y G c y:0 G

– 17 – Step 1: Concretize Reached Set #0 Reached Set #0 { LGG } L G G c x:3 cx:ycx:y c y:0 (Note loss of precision) s Concretize  s  s 

– 18 – Compute Possible Successor States x  xy  yx  xy  y s Concrete Transition s Concretize  s  s  s Concrete Transition s

– 19 – Abstract Newly Reached States 000 Reached Set #1 { LLL, LGG } L L L c x:3 cx:ycx:y c y:0 s Concrete Transition Abstract    s Concretize  s  s  s Concrete Transition s

– 20 – Step 2: Concretize Reached Set #1 Reached Set #1 { LLL, LGG } L c x:3 cx:ycx:y c y:0 (Note loss of precision) L L s Concretize  s  s 

– 21 – Compute Possible Successor States x  xy  yx  xy  y s Concrete Transition s Concretize  s  s  s Concrete Transition s

– 22 – Abstract Newly Reached States Reached Set #2 { LLL, LGG, EGG, GGG } c x:3 G E cx:ycx:y G c y:0 G s Concrete Transition Abstract    s Concretize  s  s  s Concrete Transition s

– 23 – EGG Final Reached State Set LLL LGGGGG Bad States

– 24 – Conventional Implementation of P.A. Basis Abstract state sets described as formulas over Boolean variables B = b 1, …, b k Current state given by formula  ( b 1, …, b k ) Check whether candidate state  ( b 1, …, b k ) is successor Concrete System Abstract System Concretize  [P/B][P/B]   [P/B][P/B] Concretize  Intersect? Abstract Transition?  [P/ B ][  / S ] Predecessor

– 25 – Drawbacks of Conventional Implementation Very Slow Guess at possible next state Construct term-level formula and test for satisfiability Possibly 2 k calls to decision procedure Can Only Handle Proposition Predicates Cannot construct quantified invariants [P/B][P/B] Intersect?  [P/ B ][  / S ]   [P/ B ]   [P/ B ][  / S ] Satisfiable?

– 26 – Symbolic Approach to P.A. Lahiri, Bryant, Cook, CAV 2003 Generate Quantified Formula Describing Next Abstract State Set Current state given by formula  ( B ) Generate formula  ( B ) describing all successors Abstract System  All Abstract Transitions   S, X  ( B, S, X ) How to reach abstract state B via concrete states S and X

– 27 – Symbolic Approach (cont.) Transform into Quantified Boolean Formula Formula of form Next( B ) =  S, X  ( S, X, B ) S, X : Integer and function variables B : Abstract state variables Translate into Boolean formula of form  A  ( A, B ) A : Boolean variables encoding integer & function values Key Property { B |  ( S, X, B ) satisfiable } = { B |  ( A, B ) satisfiable } Solve using either SAT enumeration or BDD quantification

– 28 – Symbolic Formulation of Step 2 Concretized State Set Encode each 3-valued {L, E, G} predicate with 2 Boolean variables ( l, g ) Represent state set as formula ( l 1   g 1  l 2   g 2  l 3   g 3 )  ( l 1   g 1   l 2  g 2   l 3  g 3 ) Reached Set #1 { LLL, LGG } LLL LGG l 1 : x < 3 g 1 : x > 3 l 2 : x < y g 2 : x > y g 3 : y > 0 l 3 : y < 0

– 29 – Next-State Predicates Next State (x, y ) Get predicates l 1, l 2, l 3, g 1, g 2, g 3 Determine conditions under which predicates will hold in next state Express in terms of current state (x, y) Next State Predicate Condition x =  x y =  yCurrentStateMatches l1l1 x < 3  x < 3x >  3 — l2l2 x < yx < y x < yx < y x > yx > y g2g2 l3l3 y < 0  y < 0 y > 0 g3g3 g1g1 x > 3  x > 3x < 3x < 3 — g2g2 x > yx > y x > yx > y x < yx < y l2l2 g3g3 y > 0  y > 0 y < 0 l3l3

– 30 – Consistency Constraints Eliminate impossible predicate combinations In general, may need to introduce additional variables To express more complex transitivity constraints l1l1 g1g1 l2g2l2g2 g2l2g2l2 g3l3g3l3 l3g3l3g3 l1l1 g1g1 (g1  g1)(g1  g1) (g1  l1)(g1  l1) (g2  g3  l1)(g2  g3  l1)

– 31 – Symbolic Form Formulation Express compatible combinations of current-state & next- state variables Quantify out current-state variables Gives formula over next-state variables lglglg [( l 1   g 1  l 2   g 2  l 3   g 3 ) lglglg  ( l 1   g 1   l 2  g 2   l 3  g 3 ) ] Current State l 1, l 2, l 3, g 1, g 2, g 3  l 1, l 2, l 3, g 1, g 2, g 3 Consistency Constraints ggglggl   ( g 1  g 1 )   ( g 1  l 1 )  ( g 2  g 3  l 1 ) lggl  l 2  g 2  g 2  l 2 lggl  l 3  g 3  g 3  l 3

– 32 – l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 Next State EGG GGG LGG LLL l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 l1l1 g1g1 l2l2 g2g2 l3l3 g3g3 Next State EGG Extracting Next-State Set Run SAT checker over formula Generate blocking clause for each newly generated state lglglg [( l 1   g 1  l 2   g 2  l 3   g 3 ) lglglg  ( l 1   g 1   l 2  g 2   l 3  g 3 ) ] ggglggl   ( g 1  g 1 )   ( g 1  l 1 )  ( g 2  g 3  l 1 ) lggl  l 2  g 2  g 2  l 2 lggl  l 3  g 3  g 3  l 3   (  l 1   g 1   l 2  g 2   l 3  g 3 )

– 33 – Quantified Invariant Generation (Lahiri & Bryant, VMCAI 2004) User supplies predicates containing free variables Generate globally quantified invariantExample Predicates p 1 : reg.valid(r) p 2 : rob.dest(t) = r p 3 : reg.tag(r) = t Abstract state satisfying ( p 1  p 2   p 3 ) corresponds to concrete state satisfying  r,t[  reg.valid(r)  reg.tag(r) = t  rob.dest(t) = r] rather than  r[  reg.valid(r)]   r,t[reg.tag(r) = t]   r,t[rob.dest(t) = r]

– 34 – Systems Verified with Predicate Abstraction Very general models Unbounded processes, buffers, cache lines, … Safety properties only Model PredicatesIterationsCPU Time Out-Of-Order Execution Unit2591,207s German’s Cache Protocol13914s German’s Protocol, unbounded channels s Bounded Retransmission Buffer22911s Lamport’s Bakery Algorithm s

– 35 – Automatic Predicate Discovery Strength of Predicate Abstraction If give it right set of predicates, PA will put them together into invariantWeakness Gets nowhere without right set of predicates Typical failure mode: Generate “true” as invariantChallenges Too many predicates will overwhelm PA engine Our use of quantified invariants precludes counterexample- generated refinement techniques

– 36 – Iterative Generation of Predicates Lahiri & Bryant, CAV ’04 Generate new set of predicates if current predicates not sufficient Generate inductive invariant using predicate abstraction over P WP-based new predicate generation from P Property proved? P No Initial set of atomic predicates from the invariant  Yes, Done

– 37 – Case Study 1: N-Bakery N-Process mutual exclusion protocol N-Process mutual exclusion protocol [Lamport ’76] Each process contains An unbounded ticket A counter with range [1…N] Other Boolean state variables Safety: Mutual exclusion property Constructs inductive invariant in 3 iterations of WP- based predicate discovery  Iteration 1: # Predicates = 1, Time to construct inv =.81s Does not imply mutual exclusion  Iteration 2: # Predicates = 18, Time = 55.8s Does not imply mutual exclusion  Iteration 3: # Predicates = 33, Time = 471s Implies mutual exclusion

– 38 – Case Study 2: German’s Cache Protocol N-Client Directory based Cache Coherence Protocol N-Client Directory based Cache Coherence Protocol [German, IBM] Each client contains Boolean state variables 3 single-entry channels to communicate with central process Central “home” process contains Directory : [1..N]  {0,1} of clients sharing a line Current client id [1…N] Boolean variables Safety: Mutual exclusion property Constructs inductive invariant in 4 iterations of WP-based predicate discovery  Iteration 1: # Predicates = 4, Time (to construct inv) = 1.46 s  Iteration 2: # Predicates = 12, Time = s  Iteration 3: # Predicates = 24, Time = 207 s  Iteration 4: # Predicates = 28, Time = 1266s Implies mutual exclusion

– 39 – Extension of German’s cache protocol Each client communicates with home with unbounded FIFO channels Unbounded number of unbounded channels Verification complexity goes up considerably 2 manually provided predicates for FIFOs required Predicates involved constant offsets Time to construct inductive invariant = 3 hours

– 40 – Predicate Abstraction Convergences Powerful method for generating & evaluating abstract model of system Applicable to variety of systems with different modeling levels HardwareSoftware Word-LevelUCLID Seshia, Lahiri, Bryant, CAV ‘02 SLAM Ball, Rajamani, SPIN ‘01 Bit-LevelClarke, Talupar, Wang, SAT ‘03 CBMC Kroening, Clarke, ICCAD ‘04

– 41 – Observations Predicate Abstraction Combines features of theorem proving & model checking Very general and powerful technique Lots of ways to generalize Making More Efficient Symbolic formulation very general SAT enumeration limits capacity to ~25 predicates Making Easier to Use Automatic predicate discovery Limitation: Hard to find counterexamples