1. Interpolating Property Directed Reachability
Author
Software Engineering Institute
9/11/2018
Interpolating Property Directed Reachability
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA
Arie Gurfinkel and Yakir Vizel
July 18, 2015
Title Slide
Title and Subtitle text blocks should not be moved from their position if at all possible.

3. Verification by Successive Under-Approximation
bound 2
bound 3
bound 1
bounded proof
Lemma2
Lemma1
Lemma3
Inductive?
bounded proof
Lemma2
Lemma1
Lemma3
Inductive?
bounded proof
Lemma2
Lemma1
Lemma3
Inductive?
BMC
BMC
BMC No No No

4. Reachability Analysis
Is Bad reachable?
…Rn R2 R1
Bad
INIT

5. Outline
Interpolating Model Checking IC3 / Property Directed Reachabilty Avy: Interpolating Property Directed Reachability DRUP Interpolants Fast Interpolating BMC Future Directions

6. Interpolating Model Checking
Introduced by McMillan in 2003
Kenneth L. McMillan: Interpolation and SAT-Based Model Checking. CAV2003: 1-13
based on pairwise Craig interpolation
Extended to sequences and DAGs
Yakir Vizel, Orna Grumberg: Interpolation-sequence based model checking. FMCAD 2009: 1-8
uses interpolation sequence
Kenneth L. McMillan: Lazy Abstraction with Interpolants. CAV 2006:
IMPACT: interpolation sequence on each program path
Aws Albarghouthi, Arie Gurfinkel, Marsha Chechik: From Under-Approximations to Over-Approximations and Back. TACAS 2012:
UFO: interpolation sequence on the DAG of program paths
Key Idea
turn SAT/SMT proofs of bounded safety to inductive traces
repeat forever until a counterexample or inductive invariant are found

7. IMC: Interpolating Model Checking
BMCN
SAT
CEX
UNSAT
N:=N+1
SeqItp
trace F = [F0, …, FN]
Is F closed No
Yes
SAFE

8. Programs, Safety, Cexs, Invariants
A transition system P = (V, Init, Tr, Bad) P is UNSAFE if and only if there exists a number N s.t. P is SAFE if and only if there exists a safe inductive invariant Inv s.t.
Inductive
Safe

9. Bounded Model Checking…… R2 R1
INIT Rk
INIT(V0)
∧Tr(V0,V1)
∧…∧Tr(Vk-1,Vk)∧Bad(Vk)

10. Inductive Trace
An inductive trace of a transition system P = (V, Init, Tr, Bad) is a sequence of formulas [F0, …, FN] such that
Init ⇒F0
∀0 · i < N , Fi(v) ∧ Tr (v, u) ⇒ Fi+1 (u)
A trace is safe iff ∀ 0 ≤ i ≤ N , Fi ⇒¬Bad
A trace is monotone iff ∀ 0 · i < N , Fi ⇒ Fi+1
A trace is closed iff ∃ 1 ≤ i ≤ N, Fi ⇒(F0 ∨ … ∨ Fi-1)
A transition system P is SAFE iff it admits a safe closed trace

11. Inductive Trace in Pictures
…FN F2 F1
Bad
INIT

12. Craig Interpolation Theorem
Theorem (Craig 1957)
Let A and B be two First Order (FO) formulae such that A ) :B, then there exists a FO formula I, denoted ITP(A, B), such that
A ⇒ I I ⇒¬B
atoms(I) ⊆ atoms(A) ∩ atoms(B)
A Craig interpolant ITP(A, B) can be effectively constructed from a resolution proof of unsatisfiability of A ∧ B
In Model Checking, Craig Interpolation Theorem is used to safely over-approximate the set of (finitely) reachable states

13. Craig Interpolant I B A

14. Craig Interpolant as a Circuit
Let F = A(x, z) ∧ B(z, y) be UNSAT, where x and y are distinct
Note that for any assignment v to z either
A(x, v) is UNSAT, or
B(v, y) is UNSAT
An interpolant is a circuit I(z) such that for every assignment v to z
I(v) = A only if A(x, v) is UNSAT
I(v) = B only if B(v, y) is UNSAT
A proof system S has a feasible interpolation if for every refutation Π of F in S, F has an interpolant polynomial in the size of Π
propositional resolution has feasible interpolation
extended resolution does not have feasible interpolation

15. Interpolation Sequence
Given a sequence of formulas A = {Ai}i=0n, an interpolation sequence ItpSeq(A) = {I1, …, In-1} is a sequence of formulas such that
Ik is an ITP (A0 ∧ … ∧ Ak-1, Ak ∧ … ∧ An), and
∀ k<n . Ik ∧ Ak+1⇒ Ik+1 ⇒ ) ⇒ ⇒ ) ⇒
A0 A1 A2 A3 A4 A5 A6
I0 I1 I2 I I4 I5
Can compute by pairwise interpolation applied to different cuts of a fixed resolution proof (very robust property of interpolation)

16. From Interpolants to Traces
A Sequence Interpolant of a BMC instance is an inductive trace
( Init(v0) )0 ∧ ( Tr (v0,v1) )1 ∧ … ∧ ( Tr (vN-1, vN) )N ∧ Bad(vN)
F0(v0) F1(v1) FN(vN)
A trace computed by a sequence interpolant is
safe
NOT necessarily monotone
NOT necessarily closed
BMCN
trace

17. Inductive Trace in Pictures
…FN F2 F1
Bad
INIT

18. IMC: Interpolating Model Checking
ImcMkSafe
BMCN
SAT
CEX
UNSAT
N:=N+1
SeqItp
trace F = [F0, …, FN]
Is F closed No
Yes
SAFE

19. IMC: Strength and Weaknesses
elegant
global bounded safety proof
many different interpolation algorithms available
easy to extend to SMT theories
Weaknesses
the naïve version does not converge easily
interpolants are weaker towards the end of the sequence
not incremental
no information is reused between BMC queries
size of interpolants
hard to guide

20. IC3: Property Directed Reachability
IC3: A SAT-based Hardware Model Checker
Incremental Construction of Inductive Clauses for Indubitable Correctness
A. Bradley: SAT-Based Model Checking without Unrolling. VMCAI 2011
PDR: Explained and extended the implementation
Property Directed Reachability
N. Eén, A. Mishchenko, R. K. Brayton: Efficient implementation of property directed reachability. FMCAD 2011
Very active area of research
Key Idea:
carefully manage SAT solving while building an inductive proof one inductive lemma at a time

21. IC3/PDR F = [Init] MkSafe CEX G = [G0, …, GN] PDR trace Push
F = [F0, …, FN]
F = [F0, …, FN]
9 i, Fi = Fi+1 No
Yes
SAFE

22. PDR Trace
Recall that an inductive trace of a transition system P = (V, Init, Tr, Bad) is a sequence of formulas [F0, …, FN] such that
Init ⇒ F0
∀ 0 ≤ i < N , Fi(v) ∧ Tr (v, u) ⇒ Fi+1 (u)
A trace is clausal if every Fi is in CNF
A delta-compressed trace (or δ-trace) is a sequence of clauses s.t.
each clause c belongs to a unique frame Fi
∀ 0 ≤ i ≤ n , ∀ j < i , (c ∈ Fi) ⇒ (c ∉ Fj)
A PDR trace is a monotone, clausal, safe (up to N-1)
PDR trace is often represented compactly by a δ-trace

23. PdrMkSafe
IC3/PDR in Pictures

24. IC3/PDR in Pictures PdrMkSafe cex Cex Queue Trace lemma Frame F0

25. PdrPush
IC3/PDR in Pictures
Inductive

26. IC3/PDR in Pictures PdrPush PDR Invariants Fi ⇒ ¬Bad Init ⇒ Fi
Inductive
PDR Invariants
Fi ⇒ ¬Bad Init ⇒ Fi
Fi ⇒ Fi Fi ∧ Tr ⇒Fi+1

27. PDR Strength and Weaknesses
Strengths
elegant
incremental
many opportunities for guidance
fine-grained proof management
fine-grained generalization of lemmas
Weaknesses
local backward search for a counterexample
CNF explosion

28. AVY: Interpolating PDR
This talk
Yakir Vizel, Arie Gurfinkel: Interpolating Property Directed Reachability. CAV 2014:
Key Idea
combine global BMC reasoning of IMC with local strengthening of IC3/PDR
use interpolation for PDR
use PDR for interpolation

29. Avy: Interpolating PDR
Bounded verification with BMC
Global trace using sequence interpolation
Locally convert (and strengthen) to PDR trace
Re-use old trace G in new BMC step
Compute strengthening of old trace G by interpolation
G = [G0, …, GN]
BMCN
SAT
CEX
UNSAT
N:=N+1
SeqItp
trace F = [F0, …, FN]
MkPdrTrace
PDR trace G = [G0, …, GN]
9 i, Gi = Gi+1 No
Yes
SAFE

30. Extending a Trace Incrementally
Input: A transition system P=(Init,Tr,Bad); a clausal trace F= [F0, …, FN]
Problem: Find (if possible) a stronger safe trace G=[G0, …, GN]
Init(v0)∧Tr (v0,v1) ∧ … ∧ Tr (vN-1, vN) ∧ Bad(vN) F0 F1
FN-1 FN G0 G1
GN-1 GN

31. Extending a Trace Incrementally
Input: A transition system P=(Init,Tr,Bad); a clausal trace F= [F0, …, FN]
Problem: Find (if possible) a stronger safe trace G=[G0, …, GN]
Let φ = (F0 ∧ Tr0)0 ∧ (F1 ∧ Tr1)1 … ∧ (FN ∧ BadN)N
if φ is SAT then return [ ]
I1, …, In = SequenceItp (φ)
G0 = Init, ∀ 1 ≤ i ≤ N . Gi = Fi ∧ Ii
return [G0, …, GN]

32. Monotone Traces by Interpolation
Input: A transition system P=(Init,Tr,Bad); a safe trace F= [F0, …, FN]
Problem: Find (if possible) a monotone safe trace G=[G0, …, GN]
Solution: Take a sequence
G0 = Init
G1 = Itp (Init’ ∨ (Init ∧ Tr) , : (Init’ ∨ F’1) ) …
Gi = Itp (G’i-1 ∨ (Gi-1 ∧ Tr) , : (G’i-1 ∨ F’i) )
Claim: G = [G0, …, GN] is a monotone and safe trace
Gi ⇒ Gi+1
Gi ⇒ ¬Bad
Gi ∧ Tr ⇒G’i+1
Gi ⇒∨{Fj | 0 ≤ j ≤ i }

33. The Tricky Part of the Proof
Given a sequence
G0 = Init
G1 = Itp (G’0 ∨ (G0 ∧ Tr) , ¬(G’0 ∨ F’1)
G2 = Itp (G’1 ∨ (G1 ∧ Tr), ¬(G’1 ∨ F’2) …
Need to show that G1 ∧ Tr ⇒ (G’1 ∨ F’2)
by property of interpolation G1 ⇒ (G0 ∨ F1)
because F is a trace, F1 ∧ Tr ⇒ F’2
by property of interpolation G0 ∧ Tr ⇒ G’1
BUT the trace G=[G0, …, GN] is not monotone
and likely to be large

34. Using PDR for Interpolation
Given mutually unsatisfiable pair of formulas A and B
Construct a SAFE transition system P = (A, ID, B) with
initial state A
transition relation ID over common variables of A and B
ID = ∧ { x=x’ | x ∈ Vars (A) ∩ Vars (B) }
bad states B
Run PDR/IC3 on P
Claim: The frame F1 is a CNF interpolant between A and B
A ∧ ID ⇒ F’1 == A ⇒ F1
F1 ⇒ ¬B

35. Extending Monotone Clausal Traces by PDR
Given a PDR trace F = [Init, F1] of transition system P = (Init, Tr, Bad)
G2 -- an over over-approximation of the forward image of F1
i.e., F1 ∧ Tr ⇒ G’2
Construct SAFE transition system T = (Init, Tr, Bad)
where Bad = ¬(G2 ∨ F1)
Run PDR on T starting with a trace [Init, F1, True]
Claim: The sequence [Init, F1, F2] is a SAFE PDR trace

36. Extending a Trace by PDR
PdrMkSafe
Extending a Trace by PDR
¬ (G2 ∨ F1)
Observations: [Init, F1, F2] is a PDR trace F2 is stronger than G2 Ç F1 F1 after is stronger than F1 before!!!
Frame F0
Frame F1
Frame F2

37. Avy global trace reuse prev. frame strengthen curr. trace
strengthen future trace
syntactic termination

38. What is a “good” bounded proof?
Proof size is not a good indicator
the smallest resolution proof is usually not good
depends too much on the initial state
depends too much on the bound
A “good” proof is abstract
works for many ‘similar’ transition systems
A proof is “good” if it extends a previously good proof
re-uses existing facts

39. Searching for a “good” proof
assumption for a frame
assumption for wires
min-suffix strategy
incrementally “cut” the wires to find the proof with the shortest suffix
min-core strategy
let SAT solver find the smallest number of wires needed for UNSAT
Need better support for expressing priorities over cores!!!

40. Experiments Started with an implementation based on ABC
slightly modified PDR engine with external API
added Sequence Interpolation
SAT solving with MiniSAT and Glucose
search for a good proof with one solver
re-solve to compute interpolants
Performs differently from PDR
virtual best is much better than either one in isolation
Status
AVY
PDR
ITP
Virtual Best
SAFE 76 72 62
112
UNSAFE 24 15 26 29

41. Results from HWMCC’14

42. DRUPing for Interpolants
trivial resolution
learned clause
A CDCL proof is build out of trivial resolutions
terminated by a learned clause
A sub-proof for each learned clause can be re-constructed in polynomial time
negation of clause + BCP leads to a conflict
A clausal proof is a sequence of learned clauses in the order they are learned
Interpolate while replaying the proof
Arie Gurfinkel, Yakir Vizel: DRUPing for interpolats. FMCAD 2014:

43. MiniDRUP SAT Trim Replay CNF
SAT with DRUP proofs Interpolation-oriented BCP in Trim Learn near CNF interpolants in Replay
SAT
Clausal Proof
BCP
Trim
core proof
BCP +Learning
Replay
Interpolant

44. Fast BMC and Interpolation
State-of-the-art in Bounded Model Checking (Fast BMC)
each successive bound is exponentially harder to solve
many advancement in SAT since first BMC
many BMC-specific advancements
circuit-aware simplifications (sweeping, constant propagation, etc.)
use of incremental SAT for increasing verification depth
lazy addition of constraints (incremental cone-of-influence)
BMC used in IMC/Avy is different than BMC used for BMC 
interpolation algorithms assume naïve BMC
circuit-aware simplifications change the structure of the formula
no correspondence between constraints and circuit steps!
incremental SAT makes interpolation more difficult
many SAT queries, but one proof
what to log?
Yakir Vizel, Arie Gurfinkel, Shard Malik: Fast Interpolating BMC. CAV 2015.

45. Future Directions Extending to theories Extending to programs
easy for theories with existing interpolation procedures
BUT, still need PDR-like interpolation procedure
Extending to programs
DAG extension for handling CFG is straight forward
handling procedures (non-linear Horn clauses) is tricky
no efficient BMC. inlining == exponential explosion
Many implementation decisions remain unexplored
other metrics for ‘goodness’ of bounded proofs (i.e., sequence interpolants)
and corresponding proof optimization procedures
switching between PDR and IMC tactics
searching for a CNF interpolant vs adapting a given one

47. Inductive Generalization
A clause φ is inductive relative to F iff
Init ⇒ φ (Initialization) and φ∧ F ∧Tr ⇒φ’ (Inductiveness)
Implemented by first letting φ = ¬m and generalizing φ by iteratively dropping literals while checking the inductiveness condition
Theorem: Let F0, F1, …, FN be a valid IC3 trace. If φ is inductive relative to Fi, 0 ≤ i < N, then, for all j ≤ i, φ is inductive relative to Fj.
Follows from the monotonicity of the trace
if j < i then Fj ⇒ Fi
if Fj ⇒Fi then (φ∧Fi ∧ Tr ⇒φ’) ⇒ (φ ∧ Fj ∧ Tr ⇒φ’)

48. Author Software Engineering Institute
9/11/2018
Contact Information
Arie Gurfinkel, Ph. D.
Sr. Researcher
CSC/SSD
Telephone:
U.S. Mail
Software Engineering Institute
Customer Relations
4500 Fifth Avenue
Pittsburgh, PA
USA
Web
Telephone:
SEI Phone:
SEI Fax: