Prof. Necula CS Lecture 111 Theorem Proving for FOL Satisfiability Procedures CS Lecture 11

Prof. Necula CS Lecture 112 Review Recall the we use the following logic: Goals: G ::= L | true | G 1 Æ G 2 | H ) G | 8 x. G Hypotheses: H ::= L | true | H 1 Æ H 2 Literals: L ::= p(E 1, …, E k ) Expressions: E ::= n | f(E 1, …, E m ) This is sufficient for VCGen iff: –The invariants, preconditions and postcond. are all from H

Prof. Necula CS Lecture 113 A Simple and Complete Prover Define the following symbolic “prove” algorithm –Prove(H, G) - prove the goal “H ) G” Prove(H, true) = true Prove(H, G 1 Æ G 2 ) = prove(H, G 1 ) && prove(H, G 2 ) Prove(H, H 1 ) G 2 ) = prove(H Æ H 1, G 2 ) Prove(H, 8 x. G) = prove(H, G[a/x]) (a is “fresh”) Prove(H, L) = unsat(H Æ : L) We have a simple, sound and complete prover –If we have a way to check unsatisfiability of sets of literals

Prof. Necula CS Lecture 114 How Powerful is Our Prover? With VCGen in mind we must restrict invariants to H ::= L | true | H 1 Æ H 2 No disjunction, implication or quantification ! –Is that bad ? Consider the function: void insert(LIST *a, LIST * b) { LIST *t = a->next; a->next = b; b->next = t; } And the problem is to verify that –It preserves linearity: all list cells are pointed to by at most one other list cell –Provided that b is non-NULL and not pointed to by any cell

Prof. Necula CS Lecture 115 Lists and Linearity A bit of formal notation (remember the sel/upd): –We write sel(n, a) to denote the value of “a->next” given the state of the “next” field is “n” –We write upd(n, a, b) to denote the new state of the “next” field after “a->next = b” Code is void insert(LIST *a, LIST * b) { LIST *t = a->next; a->next = b; b->next = t; } Pre is ( 8 q. q  0 ) 8 p 1. 8 p 2. sel(n, p 1 ) = sel(n, p 2 ) = q ) p 1 = p 2 ) Æ b  0 Æ 8 p. sel(n, p)  b) Æ a  0 Post is ( 8 q. q  0 ) 8 p 1. 8 p 2. sel(n, p 1 ) = sel(n, p 2 ) = q ) p 1 = p 2 ) VC is Pre ) Post[upd(upd(n, a, b), b, sel(n, a)) / n] Not a G !

Prof. Necula CS Lecture 116 Two Solutions So it is quite easy to want to step outside H So what can we do? 1.Extend the language of H –And then extend the prover 2.Push the complexity of invariants into literals –And then extend the unsatisfiability procedure

Prof. Necula CS Lecture 117 Goal Directed Theorem Proving (1) Finally we extend the use of quantifiers: G ::= L | true | G 1 Æ G 2 | H ) G | 8 x. G | 9 x. G H ::= L | true | H 1 Æ H 2 | 8 x. H We have now introduced an existential choice –Both in “H ) 9 x. G” and “ 8 x.H ) G” Existential choices are postponed –Introduce unification variables + unification prove(H, 9 x.G) = prove(H, G[u/x] ) (u is a unif var) prove(H, u = t) = instantiate u with t if u  FV(t) Still sound and complete goal directed proof search ! –Provided that unsat can handle unification variables !

Prof. Necula CS Lecture 118 Goal Directed Theorem Proving (2) We can add disjunction (but only to goals): G ::= true | L | G 1 Æ G 2 | H ) G | 8 x. G | G 1 Ç G 2 Extend prover as follows: prove(H, G 1 Ç G 2 ) = prove(H, G 1 ) || prove(H, G 2 ) This introduces a choice point in proof search –Called a “disjunctive choice” –Backtracking is complete for this choice selection But only in intuitionistic logic !

Prof. Necula CS Lecture 119 Goal Directed Theorem Proving (3) Now we extend a bit the language of hypotheses –Important since this adds flexibility for invariants and specs. H ::= L | true | H 1 Æ H 2 | G ) H We extend the prover as follows: prove(H, (G 1 ) H 1 ) ) G) = prove(H, G) || (prove(H Æ H 1, G) && prove(H, G 1 )) –This adds another choice (clause choice in Prolog) expressed here also as a disjunctive choice –Still complete with backtracking

Prof. Necula CS Lecture 1110 Goal Directed Theorem Proving (4) The VC for linear lists can be proved in this logic ! –This logic is called Hereditary Harrop Formulas But the prover is not complete in a classical sense –And thus complications might arise with certain theories Still no way to have disjunctive hypotheses –The prover becomes incomplete even in intuitionistic logic –E.g., cannot prove even that P Ç Q ) Q Ç P Let’s try the other method instead …

Prof. Necula CS Lecture 1111 A Theory of Linear Lists Push the complexity into literals Define new literals: linear(n) = def 8 q. q  0 ) 8 p 1. 8 p 2. sel(n, p 1 ) = sel(n, p 2 ) = q ) p 1 = p 2 rc0(n, b) = def b  0 ) 8 p. sel(n, p)  b Now the predicates become: Pre is linear(n) Æ rc0(n, b) Æ a  0 Æ b  0 Post is linear(n) VC is linear(n) Æ rc0(n, b) Æ a  0 Æ b  0 ) linear(upd(upd(n, a, b), b, sel(n, a)))) This is a G ! The hard work is now in the satisfiability procedure

Prof. Necula CS Lecture 1112 A Theory of Linear Lists In order to allow the prover to work with “linear” and “rc0” we must define their meaning: –Semantically (by giving the definitions from before) –Axiomatically (by giving a set of axioms that define them): linear(n) a  0 rc0(n, b) linear(upd(n, a, b)) linear(n) a  0 rc0(n, b) rc0(upd(n, a, b), sel(n, a)) Now we can prove the VC with just three uses of these axioms Is this set of axioms complete?

Prof. Necula CS Lecture 1113 Discussion It makes sense to push hard work in literals: –Can be handled in a customized way within the Sat procedures –The hand-crafted inference rules guide the prover –The inference rules are useful lemmas –Important technique #3 Just like in type inference, or data flow analysis : Theorem ProvingType InferenceData Flow Analysis LiteralsType systemLattice Inference rulesTyping rulesTransfer functions Sat. procedureInference algorithmIterative D.F.A.

Prof. Necula CS Lecture 1114 Theories Now we turn to unsat(L 1, …, L k ) A theory consists of: –A set of function and predicate symbols (syntax) –Definitions for the meaning of these symbols (semantics) Semantic or axiomatic definitions Example: –Symbols: 0, 1, -1, 2, -2, …, +, -, =, < (with the usual meaning) Theory of integers with arithmetic (Presburger arithmetic)

Prof. Necula CS Lecture 1115 Decision Procedures for Theories The Decision Problem: –Decide whether a formula in a theory + FOL is true Example: –Decide whether 8 x. x > 0 ) ( 9 y. x = y + 1) in { N, +, =, >} A theory is decidable when there is an algorithm that solves the decision problem for the theory –This algorithm is the decision procedure for the theory

Prof. Necula CS Lecture 1116 Satisfiability Procedures for Theories The Satisfiability Problem –Decide whether a conjunction of literals in the theory is satisfiable –Factor out the FOL part of the decision problem This is what we need to solve in our simple prover We will explore a few useful theories and satisfiability procedures for them …

Prof. Necula CS Lecture 1117 Examples of Theories. Equality. The theory of equality with uninterpreted functions Symbols: =, , f, g, … Axiomatically defined: E = E E 2 = E 1 E 1 = E 2 E 1 = E 2 E 2 = E 3 E 1 = E 3 E 1 = E 2 f(E 1 ) = f(E 2 ) Example of a satisfiability problem: g(g(g(x)) = x Æ g(g(g(g(g(x))))) = x Æ g(x)  x

Prof. Necula CS Lecture 1118 A Satisfiability Procedure for Equality Definitions: –Let R be a relation on terms –The equivalence closure of R is the smallest relation that is closed under reflexivity, symmetry and transitivity An equivalence relation Equivalence classes –Given a term t we say that t * is its representative –Two terms t 1 and t 2 are equivalent iff t 1 * = t 2 * –Computable in near-linear time (union-find) The congruence closure of a relation is the smallest relation that is closed under equivalence and congruence

Prof. Necula CS Lecture 1119 A Representation for Symbolic Terms We represent terms as DAGs –Share common subexpressions –E.g. f(f(a, b), b): f f ab Equalities are represented as dotted edges –E.g. f(f(a, b), b) = a –We consider the transitive closure of dotted edges

Prof. Necula CS Lecture 1120 Computing Congruence Closure We pick arbitrary representativs for all equivalence classes (nodes connected by dotted edges) For all nodes t = f(t 1, …, t n ) and s = f(s 1, …, s n ) –If t i * = s i * for all i = 1..n (find) –We add an edge between t* and s* and pick one of them as the representative for the entire class (union) f f ab f f ab

Prof. Necula CS Lecture 1121 Computing Congruence Closure (Cont.) Congruence closure is an inference procedure for the theory of equality –Always terminates because it does not add nodes The hard part is to detect the congruent pairs or terms –There are tricks to do this in O(n log n) We say that f(t 1, …, t n ) is represented in the DAG if there is a node f(s 1, …, s n ) such that s i * = t i *

Prof. Necula CS Lecture 1122 Satisfiability Procedure for Equality 1.Given F = Æ i t i = t i ’ Æ Æ j u j  u j ’ 2.Represent all terms in the same DAG 3.Add dotted edges for t I = t I ’ 4.Construct the congruence closure of those edges 5.Check that 8 j. u j *  u j ’ * Theorem: F is satisfiable iff 8 j. u j *  u j ’ *

Prof. Necula CS Lecture 1123 Example with Congruence Closure Consider: g(g(g(x)) = x Æ g(g(g(g(g(x))))) = x Æ g(x)  x g g x g g g g g x g g g x g g g g gg g g x g g Contra- diction

Prof. Necula CS Lecture 1124 Congruence Closure. Discussion. The example from before has little to do with program verification But equality is still very useful The congruence closure algorithm is the basis for many unification-based satisfiability procedures –We add the additional axiom: –Or equivalently: f(E 1 ) = f(E 2 ) E 1 = E 2 f -1 (E 1 ) = f -1 (E 2 )

Prof. Necula CS Lecture 1125 Presburger Arithmetic The theory of integers with +, -, =, > The most useful in program verification after equality –And quite useful for program analysis also Example of a satisfiability problem: y > 2x + 1 Æ y + x > 1 Æ y < 0 Satisfiability of a system of linear inequalities –Known to be in P (with rational solutions) –Some of the algorithms are quite simple –If we add the requirement that solutions are in Z then the problem is NP-complete

Prof. Necula CS Lecture 1126 Difference Constraints A special case of linear arithmetic All constraints of the form: x i - x j · c or x i - 0 · c or 0 - x j · c The most common form of constraint Construct a directed graph with: –A node for 0 –A node for each variable x i –A edge from x i to x j of weight c for each x i - x j · c xixi xjxj c

Prof. Necula CS Lecture 1127 Difference Constraints Theorem: A set of difference constraints is satisfiable iff there is no negative weight cycle in the graph Can be solved with Bellman-Ford in O(n 2 ) –In practice n is typically quite small –In practice we use incremental algorithms (to account for assumptions being pushed and popped) Algorithm is complete ! Was used successfully in array-bounds checking elimination and induction variable discovery

Prof. Necula CS Lecture 1128 Extensions of Difference Constraints Shostak extended the algorithm to ax + by · c Construct a graph as before –One node for each variable –One undirected edge for each constraint An admissible loop in this graph is a loop in which any two adjacent edges “ax + by · c” and “dy + ez · f” have sgn(b)  sgn(d) –The residue of such adjacent edges is a constraint on x and z a|d| x + e|b| z · c|d| + f|b| –The residue for a loop is an inequality without variables Theorem: The inequalities are satisfiable iff all residues for simple loops are satisfiable

Prof. Necula CS Lecture 1129 How Complete are These Procedures? Consider: 3x ¸ 2y Æ 3y ¸ 4 Æ 3 ¸ 2x xy 0 ¢ 3 ¢ 2 ¢ 4.5 The unsat procedure is sound: unsat Q ) unsat Z But it is incomplete ! Not a problem in practice Or the problem goes away with tricks like this: Transform “ax ¸ b” into “x ¸ d b/a e ” Residue is: 13.5 ¸ 8 ) satisfiable But only in Q, not in Z

Prof. Necula CS Lecture 1130 Arithmetic. Discussion There are many satisfiability algorithms –Even for the general case (e.g. Simplex) –Except for difference constraints, all are incomplete in Z –But Z can be handled well with heuristics There are no practical satisfiability procedures for ( Q, £ ) and the satisfiability of ( Z, £ ) is only semi- decidable

Prof. Necula CS Lecture 1131 Combining Satisfiability Procedures We have developed sat. procedures for several theories –We considered each theory in part –Can we combine several sat. procedures? Consider equality and arithmetic f(f(x) - f(y))  f(z)x  yy + z  x0  z false f(x) = f(y) y  x x = y 0 = z f(x) - f(y) = z

Prof. Necula CS Lecture 1132 Combining Satisfiability Procedures Combining satisfiability procedures is non trivial And that was to be expected: –( Z, +) and ( Q, £ ) are decidable, but ( Z, £ ) is not –Equality was solved by Ackerman in 1924, arithmetic by Fourier even before, but E + A only in 1979 ! Yet in any single verification problem we will have literals from several theories: –Equality, arithmetic, lists, … When and how can we combine separate satisfiability procedures?

Prof. Necula CS Lecture 1133 Nelson-Oppen Method (1) 1.Represent all conjuncts in the same DAG f(f(x) - f(y))  f(z) Æ y ¸ x Æ x ¸ y + z Æ z ¸ 0 f - ff y x ¸ z0 ¸ + f ¸

Prof. Necula CS Lecture 1134 Nelson-Oppen Method (2) 2.Run each sat. procedure Require it to report all contradictions (as usual) Also require it to report all equalities between nodes f - ff y x ¸ z0 ¸ + f ¸

Prof. Necula CS Lecture 1135 Nelson-Oppen Method (3) 3.Broadcast all discovered equalities and re-run sat. procedures Until no more equalities are discovered or a contradiction arises f - ff y x ¸ z0 ¸ + f ¸ x Contradiction

Prof. Necula CS Lecture 1136 Puzzle: Constructive vs. Classical Proofs Prove the following fact: Hint: Try