Interpolating Property Directed Reachability Author Software Engineering Institute 9/11/2018 Interpolating Property Directed Reachability Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Arie Gurfinkel and Yakir Vizel July 18, 2015 Title Slide Title and Subtitle text blocks should not be moved from their position if at all possible.

http://arieg.bitbucket.org/avy/

Verification by Successive Under-Approximation bound 2 bound 3 bound 1 bounded proof Lemma2 Lemma1 Lemma3 Inductive? bounded proof Lemma2 Lemma1 Lemma3 Inductive? bounded proof Lemma2 Lemma1 Lemma3 Inductive? BMC BMC BMC No No No

Reachability Analysis Is Bad reachable? …Rn R2 R1 Bad INIT

Outline Interpolating Model Checking IC3 / Property Directed Reachabilty Avy: Interpolating Property Directed Reachability DRUP Interpolants Fast Interpolating BMC Future Directions

Interpolating Model Checking Introduced by McMillan in 2003 Kenneth L. McMillan: Interpolation and SAT-Based Model Checking. CAV2003: 1-13 based on pairwise Craig interpolation Extended to sequences and DAGs Yakir Vizel, Orna Grumberg: Interpolation-sequence based model checking. FMCAD 2009: 1-8 uses interpolation sequence Kenneth L. McMillan: Lazy Abstraction with Interpolants. CAV 2006: 123-136 IMPACT: interpolation sequence on each program path Aws Albarghouthi, Arie Gurfinkel, Marsha Chechik: From Under-Approximations to Over-Approximations and Back. TACAS 2012: 157-172 UFO: interpolation sequence on the DAG of program paths Key Idea turn SAT/SMT proofs of bounded safety to inductive traces repeat forever until a counterexample or inductive invariant are found

IMC: Interpolating Model Checking BMCN SAT CEX UNSAT N:=N+1 SeqItp trace F = [F0, …, FN] Is F closed No Yes SAFE

Programs, Safety, Cexs, Invariants A transition system P = (V, Init, Tr, Bad) P is UNSAFE if and only if there exists a number N s.t. P is SAFE if and only if there exists a safe inductive invariant Inv s.t. Inductive Safe

Bounded Model Checking …… R2 R1 INIT Rk INIT(V0) ∧Tr(V0,V1) ∧…∧Tr(Vk-1,Vk)∧Bad(Vk)

Inductive Trace An inductive trace of a transition system P = (V, Init, Tr, Bad) is a sequence of formulas [F0, …, FN] such that Init ⇒F0 ∀0 · i < N , Fi(v) ∧ Tr (v, u) ⇒ Fi+1 (u) A trace is safe iff ∀ 0 ≤ i ≤ N , Fi ⇒¬Bad A trace is monotone iff ∀ 0 · i < N , Fi ⇒ Fi+1 A trace is closed iff ∃ 1 ≤ i ≤ N, Fi ⇒(F0 ∨ … ∨ Fi-1) A transition system P is SAFE iff it admits a safe closed trace

Inductive Trace in Pictures …FN F2 F1 Bad INIT

Craig Interpolation Theorem Theorem (Craig 1957) Let A and B be two First Order (FO) formulae such that A ) :B, then there exists a FO formula I, denoted ITP(A, B), such that A ⇒ I I ⇒¬B atoms(I) ⊆ atoms(A) ∩ atoms(B) A Craig interpolant ITP(A, B) can be effectively constructed from a resolution proof of unsatisfiability of A ∧ B In Model Checking, Craig Interpolation Theorem is used to safely over-approximate the set of (finitely) reachable states

Craig Interpolant I B A

Craig Interpolant as a Circuit Let F = A(x, z) ∧ B(z, y) be UNSAT, where x and y are distinct Note that for any assignment v to z either A(x, v) is UNSAT, or B(v, y) is UNSAT An interpolant is a circuit I(z) such that for every assignment v to z I(v) = A only if A(x, v) is UNSAT I(v) = B only if B(v, y) is UNSAT A proof system S has a feasible interpolation if for every refutation Π of F in S, F has an interpolant polynomial in the size of Π propositional resolution has feasible interpolation extended resolution does not have feasible interpolation

Interpolation Sequence Given a sequence of formulas A = {Ai}i=0n, an interpolation sequence ItpSeq(A) = {I1, …, In-1} is a sequence of formulas such that Ik is an ITP (A0 ∧ … ∧ Ak-1, Ak ∧ … ∧ An), and ∀ k<n . Ik ∧ Ak+1⇒ Ik+1 ⇒ ) ⇒ ⇒ ) ⇒ A0 A1 A2 A3 A4 A5 A6 I0 I1 I2 I3 I4 I5 Can compute by pairwise interpolation applied to different cuts of a fixed resolution proof (very robust property of interpolation)

From Interpolants to Traces A Sequence Interpolant of a BMC instance is an inductive trace ( Init(v0) )0 ∧ ( Tr (v0,v1) )1 ∧ … ∧ ( Tr (vN-1, vN) )N ∧ Bad(vN) F0(v0) F1(v1) FN(vN) A trace computed by a sequence interpolant is safe NOT necessarily monotone NOT necessarily closed BMCN trace

Inductive Trace in Pictures …FN F2 F1 Bad INIT

IMC: Interpolating Model Checking ImcMkSafe BMCN SAT CEX UNSAT N:=N+1 SeqItp trace F = [F0, …, FN] Is F closed No Yes SAFE

IMC: Strength and Weaknesses elegant global bounded safety proof many different interpolation algorithms available easy to extend to SMT theories Weaknesses the naïve version does not converge easily interpolants are weaker towards the end of the sequence not incremental no information is reused between BMC queries size of interpolants hard to guide

IC3: Property Directed Reachability IC3: A SAT-based Hardware Model Checker Incremental Construction of Inductive Clauses for Indubitable Correctness A. Bradley: SAT-Based Model Checking without Unrolling. VMCAI 2011 PDR: Explained and extended the implementation Property Directed Reachability N. Eén, A. Mishchenko, R. K. Brayton: Efficient implementation of property directed reachability. FMCAD 2011 Very active area of research Key Idea: carefully manage SAT solving while building an inductive proof one inductive lemma at a time

IC3/PDR F = [Init] MkSafe CEX G = [G0, …, GN] PDR trace Push F = [F0, …, FN] F = [F0, …, FN] 9 i, Fi = Fi+1 No Yes SAFE

PDR Trace Recall that an inductive trace of a transition system P = (V, Init, Tr, Bad) is a sequence of formulas [F0, …, FN] such that Init ⇒ F0 ∀ 0 ≤ i < N , Fi(v) ∧ Tr (v, u) ⇒ Fi+1 (u) A trace is clausal if every Fi is in CNF A delta-compressed trace (or δ-trace) is a sequence of clauses s.t. each clause c belongs to a unique frame Fi ∀ 0 ≤ i ≤ n , ∀ j < i , (c ∈ Fi) ⇒ (c ∉ Fj) A PDR trace is a monotone, clausal, safe (up to N-1) PDR trace is often represented compactly by a δ-trace

PdrMkSafe IC3/PDR in Pictures

IC3/PDR in Pictures PdrMkSafe cex Cex Queue Trace lemma Frame F0

PdrPush IC3/PDR in Pictures Inductive

IC3/PDR in Pictures PdrPush PDR Invariants Fi ⇒ ¬Bad Init ⇒ Fi Inductive PDR Invariants Fi ⇒ ¬Bad Init ⇒ Fi Fi ⇒ Fi+1 Fi ∧ Tr ⇒Fi+1

PDR Strength and Weaknesses Strengths elegant incremental many opportunities for guidance fine-grained proof management fine-grained generalization of lemmas Weaknesses local backward search for a counterexample CNF explosion

AVY: Interpolating PDR This talk Yakir Vizel, Arie Gurfinkel: Interpolating Property Directed Reachability. CAV 2014: 260-276 Key Idea combine global BMC reasoning of IMC with local strengthening of IC3/PDR use interpolation for PDR use PDR for interpolation

Avy: Interpolating PDR Bounded verification with BMC Global trace using sequence interpolation Locally convert (and strengthen) to PDR trace Re-use old trace G in new BMC step Compute strengthening of old trace G by interpolation G = [G0, …, GN] BMCN SAT CEX UNSAT N:=N+1 SeqItp trace F = [F0, …, FN] MkPdrTrace PDR trace G = [G0, …, GN] 9 i, Gi = Gi+1 No Yes SAFE

Extending a Trace Incrementally Input: A transition system P=(Init,Tr,Bad); a clausal trace F= [F0, …, FN] Problem: Find (if possible) a stronger safe trace G=[G0, …, GN] Init(v0)∧Tr (v0,v1) ∧ … ∧ Tr (vN-1, vN) ∧ Bad(vN) F0 F1 FN-1 FN G0 G1 GN-1 GN

Extending a Trace Incrementally Input: A transition system P=(Init,Tr,Bad); a clausal trace F= [F0, …, FN] Problem: Find (if possible) a stronger safe trace G=[G0, …, GN] Let φ = (F0 ∧ Tr0)0 ∧ (F1 ∧ Tr1)1 … ∧ (FN ∧ BadN)N if φ is SAT then return [ ] I1, …, In = SequenceItp (φ) G0 = Init, ∀ 1 ≤ i ≤ N . Gi = Fi ∧ Ii return [G0, …, GN]

Monotone Traces by Interpolation Input: A transition system P=(Init,Tr,Bad); a safe trace F= [F0, …, FN] Problem: Find (if possible) a monotone safe trace G=[G0, …, GN] Solution: Take a sequence G0 = Init G1 = Itp (Init’ ∨ (Init ∧ Tr) , : (Init’ ∨ F’1) ) … Gi = Itp (G’i-1 ∨ (Gi-1 ∧ Tr) , : (G’i-1 ∨ F’i) ) Claim: G = [G0, …, GN] is a monotone and safe trace Gi ⇒ Gi+1 Gi ⇒ ¬Bad Gi ∧ Tr ⇒G’i+1 Gi ⇒∨{Fj | 0 ≤ j ≤ i }

The Tricky Part of the Proof Given a sequence G0 = Init G1 = Itp (G’0 ∨ (G0 ∧ Tr) , ¬(G’0 ∨ F’1) G2 = Itp (G’1 ∨ (G1 ∧ Tr), ¬(G’1 ∨ F’2) … Need to show that G1 ∧ Tr ⇒ (G’1 ∨ F’2) by property of interpolation G1 ⇒ (G0 ∨ F1) because F is a trace, F1 ∧ Tr ⇒ F’2 by property of interpolation G0 ∧ Tr ⇒ G’1 BUT the trace G=[G0, …, GN] is not monotone and likely to be large

Using PDR for Interpolation Given mutually unsatisfiable pair of formulas A and B Construct a SAFE transition system P = (A, ID, B) with initial state A transition relation ID over common variables of A and B ID = ∧ { x=x’ | x ∈ Vars (A) ∩ Vars (B) } bad states B Run PDR/IC3 on P Claim: The frame F1 is a CNF interpolant between A and B A ∧ ID ⇒ F’1 == A ⇒ F1 F1 ⇒ ¬B

Extending Monotone Clausal Traces by PDR Given a PDR trace F = [Init, F1] of transition system P = (Init, Tr, Bad) G2 -- an over over-approximation of the forward image of F1 i.e., F1 ∧ Tr ⇒ G’2 Construct SAFE transition system T = (Init, Tr, Bad) where Bad = ¬(G2 ∨ F1) Run PDR on T starting with a trace [Init, F1, True] Claim: The sequence [Init, F1, F2] is a SAFE PDR trace

Extending a Trace by PDR PdrMkSafe Extending a Trace by PDR ¬ (G2 ∨ F1) Observations: [Init, F1, F2] is a PDR trace F2 is stronger than G2 Ç F1 F1 after is stronger than F1 before!!! Frame F0 Frame F1 Frame F2

Avy global trace reuse prev. frame strengthen curr. trace strengthen future trace syntactic termination

What is a “good” bounded proof? Proof size is not a good indicator the smallest resolution proof is usually not good depends too much on the initial state depends too much on the bound A “good” proof is abstract works for many ‘similar’ transition systems A proof is “good” if it extends a previously good proof re-uses existing facts

Searching for a “good” proof assumption for a frame assumption for wires min-suffix strategy incrementally “cut” the wires to find the proof with the shortest suffix min-core strategy let SAT solver find the smallest number of wires needed for UNSAT Need better support for expressing priorities over cores!!!

Experiments Started with an implementation based on ABC slightly modified PDR engine with external API added Sequence Interpolation SAT solving with MiniSAT and Glucose search for a good proof with one solver re-solve to compute interpolants Performs differently from PDR virtual best is much better than either one in isolation Status AVY PDR ITP Virtual Best SAFE 76 72 62 112 UNSAFE 24 15 26 29

Results from HWMCC’14

DRUPing for Interpolants trivial resolution learned clause A CDCL proof is build out of trivial resolutions terminated by a learned clause A sub-proof for each learned clause can be re-constructed in polynomial time negation of clause + BCP leads to a conflict A clausal proof is a sequence of learned clauses in the order they are learned Interpolate while replaying the proof Arie Gurfinkel, Yakir Vizel: DRUPing for interpolats. FMCAD 2014: 99-106

MiniDRUP SAT Trim Replay CNF SAT with DRUP proofs Interpolation-oriented BCP in Trim Learn near CNF interpolants in Replay SAT Clausal Proof BCP Trim core proof BCP +Learning Replay Interpolant

Fast BMC and Interpolation State-of-the-art in Bounded Model Checking (Fast BMC) each successive bound is exponentially harder to solve many advancement in SAT since first BMC many BMC-specific advancements circuit-aware simplifications (sweeping, constant propagation, etc.) use of incremental SAT for increasing verification depth lazy addition of constraints (incremental cone-of-influence) BMC used in IMC/Avy is different than BMC used for BMC  interpolation algorithms assume naïve BMC circuit-aware simplifications change the structure of the formula no correspondence between constraints and circuit steps! incremental SAT makes interpolation more difficult many SAT queries, but one proof what to log? Yakir Vizel, Arie Gurfinkel, Shard Malik: Fast Interpolating BMC. CAV 2015.

Future Directions Extending to theories Extending to programs easy for theories with existing interpolation procedures BUT, still need PDR-like interpolation procedure Extending to programs DAG extension for handling CFG is straight forward handling procedures (non-linear Horn clauses) is tricky no efficient BMC. inlining == exponential explosion Many implementation decisions remain unexplored other metrics for ‘goodness’ of bounded proofs (i.e., sequence interpolants) and corresponding proof optimization procedures switching between PDR and IMC tactics searching for a CNF interpolant vs adapting a given one

http://arieg.bitbucket.org/avy/

Inductive Generalization A clause φ is inductive relative to F iff Init ⇒ φ (Initialization) and φ∧ F ∧Tr ⇒φ’ (Inductiveness) Implemented by first letting φ = ¬m and generalizing φ by iteratively dropping literals while checking the inductiveness condition Theorem: Let F0, F1, …, FN be a valid IC3 trace. If φ is inductive relative to Fi, 0 ≤ i < N, then, for all j ≤ i, φ is inductive relative to Fj. Follows from the monotonicity of the trace if j < i then Fj ⇒ Fi if Fj ⇒Fi then (φ∧Fi ∧ Tr ⇒φ’) ⇒ (φ ∧ Fj ∧ Tr ⇒φ’)

Author Software Engineering Institute 9/11/2018 Contact Information Arie Gurfinkel, Ph. D. Sr. Researcher CSC/SSD Telephone: +1 412-268-5800 Email: info@sei.cmu.edu U.S. Mail Software Engineering Institute Customer Relations 4500 Fifth Avenue Pittsburgh, PA 15213-2612 USA Web www.sei.cmu.edu www.sei.cmu.edu/contact.cfm Email: info@sei.cmu.edu Telephone: +1 412-268-5800 SEI Phone: +1 412-268-5800 SEI Fax: +1 412-268-6257