1. 1 CS 2710, ISSP 2160 Chapter 9 Inference in First-Order Logic

2. 2 Pages to skim Storage and Retrieval (p. starts bottom 328) Efficient forward chaining (starts p. 333) through Irrelevant facts (ends top 337) Efficient implementation of logic programs (starts p. 340) through Constraint logic programming (ends p. 345) Completeness of resolution (starts p. 350) (though see notes in slides)

3. 3 Inference with Quantifiers Universal Instantiation: –Given  X (person(X)  likes(X, sun)) –Infer person(john)  likes(john,sun) Existential Instantiation: –Given  x likes(x, chocolate) –Infer: likes(S1, chocolate) –S1 is a “Skolem Constant” that is not found anywhere else in the KB and refers to (one of) the individuals that likes sun.

4. 4 Reduction to Propositional Inference Simple form (pp. 324-325) not efficient. Useful conceptually. Replace each universally quantified sentence by all possible instantiations –All X (man(X)  mortal(X)) replaced by –man(tom)  mortal(tom) –man(chocolate)  mortal(chocolate) –… Now, we have propositional logic. Use propositional reasoning algorithms from Ch 7

5. 5 Reduction to Propositional Inference Problem: when the KB includes a function symbol, the set of term substitutions is infinite. father(father(father(tom))) … Herbrand 1930: if a sentence is entailed by the original FO KB, then there is a proof using a finite subset of the propositionalized KB Since any subset has a maximum depth of nesting in terms, we can find the subset by generating all instantiations with constant symbols, then all with depth 1, and so on

6. 6 First-Order Inference We have an approach to FO inference via propositionalization that is complete: any entailed sentence can be proved Entailment for FOPC is semi-decidable: algorithms exist that say yes to every entailed sentence, but no algorithm exists that also says no to every nonentailed sentence. Our proof procedure could go on and on, generating more and more deeply nested terms, but we will not know whether it is stuck in a loop, or whether the proof is just about to pop out

7. 7 Generalized Modus Ponens This is a general inference rule for FOPC that does not require universal instantiation first Given: – p1’, p2’ … pn’, (p1  … pn)  q –Subst(theta, pi’) = subst(theta, pi) for all i Conclude: –Subst(theta, q)

8. 8 GMP is a lifted version of MP GMP “lifts” MP from propositional to first-order logic Key advantage of lifted inference rules over propositionalization is that they make only substitutions which are required to allow particular inferences to proceed

9. 9 GMP Example  x,y,z ((parent(x,y)  parent(y,z))  grandparent(x,z)) parent(james, john), parent(james, richard), parent(harry, james) We can derive: –Grandparent(harry, john), bindings: {x/harry,y/james,z/john} –Grandparent(harry, richard), bindings: {x/harry,y/james,z/richard}

10. 10 Unification Process of finding all legal substitutions Key component of all FO inference algorithms Unify(p,q) = theta, where Subst(theta,p) == Subst(theta,q) Assuming all variables universally quantified

11. 11 Standardizing apart All X knows(john,X). All X knows(X,elizabeth). These ought to unify, since john knows everyone, and everyone knows elizabeth. Rename variables to avoid such name clashes Note: all X p(X) == all Y p(Y) All X (p(X) ^ q(X)) == All X p(X) ^ All Y q(Y)

12. 12 def Unify (p, q, bdgs): d = disagreement(p, q) # If there is no disagreement, then success. if not d: return bdgs elif not isVar(d[0]) and not isVar(d[1]): return 'fail' else: if isVar(d[0]): var = d[0] ; other = d[1] else: var = d[1] ; other = d[0] if occursp (var,other): return ‘fail’ # Make appropriate substitutions and recurse on the result. else: pp = replaceAll(var,other,p) qq = replaceAll(var,other,q) return Unify (pp,qq, bdgs + [[var,other]]) For code, see “resources” on the webpage

13. 13 ================================ unify: ['loves', ['dog', 'var_x'], ['dog', 'fred']] ['loves', 'var_z', 'var_z'] subs: [['var_z', ['dog', 'var_x']], ['var_x', 'fred']] result: ['loves', ['dog', 'fred'], ['dog', 'fred']] ================================ unify: ['loves', ['dog', 'fred'], 'fred'] ['loves', 'var_x', 'var_y'] subs: [['var_x', ['dog', 'fred']], ['var_y', 'fred']] result: ['loves', ['dog', 'fred'], 'fred'] ================================ unify: ['loves', ['dog', 'fred'], 'mary'] ['loves', ['dog', 'var_x'], 'var_y'] subs: [['var_x', 'fred'], ['var_y', 'mary']] result: ['loves', ['dog', 'fred'], 'mary'] ================================

14. 14 unify: ['loves', ['dog', 'fred'], 'mary'] ['loves', ['dog', 'var_x'], 'var_y'] subs: [['var_x', 'fred'], ['var_y', 'mary']] result: ['loves', ['dog', 'fred'], 'mary'] ================================ unify: ['loves', ['dog', 'fred'], 'fred'] ['loves', 'var_x', 'var_x'] failure ================================ unify: ['loves', ['dog', 'fred'], 'mary'] ['loves', ['dog', 'var_x'], 'var_x'] failure ================================ unify: ['loves', 'var_x', 'fred'] ['loves', ['dog', 'var_x'], 'fred'] var_x occurs in ['dog', 'var_x'] failure

15. 15 unify: ['loves', 'var_x', ['dog', 'var_x']] ['loves', 'var_y', 'var_y'] var_y occurs in ['dog', 'var_y'] failure ================================ unify: ['loves', 'var_y', 'var_y'] ['loves', 'var_x', ['dog', 'var_x']] var_x occurs in ['dog', 'var_x'] failure ================================ unify: (fails because vars not standardized apart) ['hates', 'agatha', 'var_x'] ['hates', 'var_x', ['f1', 'var_x']] failure ================================ unify: ['hates', 'agatha', 'var_x'] ['hates', 'var_y', ['f1', 'var_y']] subs: [['var_y', 'agatha'], ['var_x', ['f1', 'agatha']]] result: ['hates', 'agatha', ['f1', 'agatha']]

16. 16 Most General Unifier The Unify algorithm returns a MGU L1 = p(X,f(Y),b) L2 = p(X,f(b),b) Subst1 = {X\a, Y\b} Result1 = p(a,f(b),b) Subst2 = {Y\b} Result2 = p(X,f(b),b) Subst1 is more restrictive than Subst2. In fact, Subst2 is a MGU of L1 and L2.

17. 17 Storage and retrieval Hash statements by predicate for quick retrieval (predicate indexing), e.g., of all sentences that unify with tall(X) Why attempt to unify –tall(X) and silly(dog(Y)) Instead –Predicates[tall] = {all tall facts} –Unify(tall(X),s) for s in Predicates[tall] Subsumption lattice for efficiency (see p. 329 for your interest)

18. 18 Inference Methods Unification (prerequisite) Forward Chaining –Production Systems –RETE Method (OPS) Backward Chaining –Logic Programming (Prolog) Resolution –Transform to CNF –Generalization of Prop. Logic resolution

19. 19 Resolution Theorem Proving (FOL) Convert everything to CNF Resolve, with unification –Save bindings as you go! If resolution is successful, proof succeeds If there was a variable in the item to prove, return variable’s value from unification bindings

20. 20 Converting to CNF

21. 21 Converting sentences to CNF 1. Eliminate all ↔ connectives (P ↔ Q)  ((P  Q) ^ (Q  P)) 2. Eliminate all  connectives (P  Q)  (  P  Q) 3. Reduce the scope of each negation symbol to a single predicate  P  P  (P  Q)   P   Q  (P  Q)   P   Q  (  x)P  (  x)  P  (  x)P  (  x)  P 4. Standardize variables: rename all variables so that each quantifier has its own unique variable name

22. 22 Converting sentences to clausal form: Skolem constants and functions 5. Eliminate existential quantification by introducing Skolem constants/functions (  x)P(x)  P(c) c is a Skolem constant (a brand-new constant symbol that is not used in any other sentence) (  x)(  y)P(x,y) becomes (  x)P(x, F(x)) since  is within the scope of a universally quantified variable, use a Skolem function F to construct a new value that depends on the universally quantified variable f must be a brand-new function name not occurring in any other sentence in the KB. E.g., (  x)(  y)loves(x,y) becomes (  x)loves(x,F(x)) In this case, F(x) specifies the person that x loves E.g.,  x1  x2  x3  y P(… y …) becomes  x1  x2  x3 P(… FF(x1,x2,x3) …) (FF is a new name)

23. 23 Converting sentences to clausal form 6. Remove universal quantifiers by (1) moving them all to the left end; (2) making the scope of each the entire sentence; and (3) dropping the “prefix” part Ex: (  x)P(x)  P(x) 7. Put into conjunctive normal form (conjunction of disjunctions) using distributive and associative laws (P  Q)  R  (P  R)  (Q  R) (P  Q)  R  (P  Q  R) 8. Split conjuncts into separate clauses 9. Standardize variables so each clause contains only variable names that do not occur in any other clause

24. 24 An example (  x)(P(x)  ((  y)(P(y)  P(F(x,y)))   (  y)(Q(x,y)  P(y)))) 2. Eliminate  (  x)(  P(x)  ((  y)(  P(y)  P(F(x,y)))   (  y)(  Q(x,y)  P(y)))) 3. Reduce scope of negation (  x)(  P(x)  ((  y)(  P(y)  P(F(x,y)))  (  y)(Q(x,y)   P(y)))) 4. Standardize variables (  x)(  P(x)  ((  y)(  P(y)  P(F(x,y)))  (  z)(Q(x,z)   P(z)))) 5. Eliminate existential quantification (  x)(  P(x)  ((  y)(  P(y)  P(F(x,y)))  (Q(x,G(x))   P(G(x))))) 6. Drop universal quantification symbols (  P(x)  ((  P(y)  P(F(x,y)))  (Q(x,G(x))   P(G(x)))))

25. 25 An Example 7. Convert to conjunction of disjunctions (  P(x)   P(y)  P(F(x,y)))  (  P(x)  Q(x,G(x)))  (  P(x)   P(G(x))) 8. Create separate clauses  P(x)   P(y)  P(F(x,y))  P(x)  Q(x,G(x))  P(x)   P(G(x)) 9. Standardize variables  P(x)   P(y)  P(F(x,y))  P(z)  Q(z,G(z))  P(w)   P(G(w)) Note: Now that quantifiers are gone, we do need the upper/lower-case distinction

26. 26 1. all X (read (X) --> literate (X)) 2. all X (dolphin (X) --> ~literate (X)) 3. exists X (dolphin (X) ^ intelligent (X)) (a translation of ``Some dolphins are intelligent'') ``Are there some who are intelligent but cannot read?'' 4. exists X (intelligent(X) ^ ~read (X)) Set of clauses (1-3): 1. ~read(X) v literate(X) 2. ~dolphin(Y) v ~literate(Y) 3a. dolphin (a) 3b. intelligent (a) Negation of 4: ~(exists Z (intelligent(Z) ^ ~read (Z))) In Clausal form: ~intelligent(Z) v read(Z) Resolution proof: in lecture.

27. 27 More complicated example Did Curiosity kill the cat Jack owns a dog. Every dog owner is an animal lover. No animal lover kills an animal. Either Jack or Curiosity killed the cat, who is named Tuna. Did Curiosity kill the cat? These can be represented as follows: A. (  x) (Dog(x)  Owns(Jack,x)) B. (  x) (((  y) (Dog(y)  Owns(x, y)))  AnimalLover(x)) C. (  x) (AnimalLover(x)  ((  y) Animal(y)   Kills(x,y))) D. Kills(Jack,Tuna)  Kills(Curiosity,Tuna) E. Cat(Tuna) F. (  x) (Cat(x)  Animal(x) ) G. Kills(Curiosity, Tuna) GOAL

28. 28 Convert to clause form A1. (Dog(D)) A2. (Owns(Jack,D)) B. (  Dog(y),  Owns(x, y), AnimalLover(x)) C. (  AnimalLover(a),  Animal(b),  Kills(a,b)) D. (Kills(Jack,Tuna), Kills(Curiosity,Tuna)) E. Cat(Tuna) F. (  Cat(z), Animal(z)) Add the negation of query:  G: (  Kills(Curiosity, Tuna)) D is a skolem constant

29. 29 The resolution refutation proof R1:  G, D, {}(Kills(Jack, Tuna)) R2: R1, C, {a/Jack, b/Tuna}(~AnimalLover(Jack), ~Animal(Tuna)) R3: R2, B, {x/Jack} (~Dog(y), ~Owns(Jack, y), ~Animal(Tuna)) R4: R3, A1, {y/D}(~Owns(Jack, D), ~Animal(Tuna)) R5: R4, A2, {}(~Animal(Tuna)) R6: R5, F, {z/Tuna}(~Cat(Tuna)) R7: R6, E, {} FALSE

30. 30 The proof tree GG D C B A1 A2 F A R1: K(J,T) R2:  AL(J)   A(T) R3:  D(y)   O(J,y)   A(T) R4:  O(J,D),  A(T) R5:  A(T) R6:  C(T) R7: FALSE {} {a/J,b/T} {x/J} {y/D} {} {z/T} {}

31. 31 Decidability and Completeness Resolution is a refutation complete inference procedure for First-Order Logic –If a set of sentences contains a contradiction, then a finite sequence of resolutions will prove this. –If not, resolution may loop forever (“semi-decidable”) Here are notes by Charles Elkan that go into this more deeplynotes by Charles Elkan

32. 32 Decidability and Completeness Refutation Completeness: If KB |= A then KB |- A –If it’s entailed, then there’s a proof Semi-decidable: –If there’s a proof, we’ll halt with it. –If not, maybe halt, maybe not Logical entailment in FOL is semi-decidable: if the desired conclusion follows from the premises, then eventually resolution refutation will find a contradiction

33. 33 Decidability and Completeness Propositional logic –logical entailment is decidable –There exists a complete inference procedure First-Order logic –logical entailment is semi-decidable –Resolution procedure is refutation complete

34. 34 Strategies (heuristics) for efficient resolution include –Unit preference. If a clause has only one literal, use it first. –Set of support. Identify “useful” rules and ignore the rest. (p. 305) –Input resolution. Intermediately generated sentences can only be combined with original inputs or original rules. –Subsumption. Prune unnecessary facts from the database.

35. 35 Horn Clauses A Horn Clause is a CNF clause with at most one positive literal Horn Clauses form the basis of forward and backward chaining The Prolog language is based on Horn Clauses Deciding entailment with Horn Clauses is linear in the size of the knowledge base

36. 36 Reasoning with Horn Clauses Forward Chaining –For each new piece of data, generate all new facts, until the desired fact is generated –Data-directed reasoning Backward Chaining –To prove the goal, find a clause that contains the goal as its head, and prove the body recursively –Goal-directed reasoning The state space is an AND-OR graph; see 7.5.4

37. 37 Forward Chaining over FO Definite (Horn) Clauses Clauses (disjunctions) with at most one positive literal First-order literals can include variables, which are assumed to be universally quantified Use GMP to perform forward chaining (Semi-decidable as for full FOPC)

38. 38 Def FOL-FC-Ask(KB,A) returns subst or false KB: set of FO definite clauses with variables standardized apart A: the query, an atomic sentence Repeat until new is empty new  {} for each implication (p1 ^ … ^ pn  q) in KB: for each T such that SUBST(T,p1^…^pn) = SUBST(T,p1’^…^pn’) for some p1’,…,pn’ in KB q’  SUBST(T,q) if q’ is not a renaming of a sentence already in KB or new: add q’ to new S  Unify(q’,A) if S is not fail then return S add new to KB Return false Process can be made more efficient; read on your own, for interest

39. 39 Backward Chaining over Definite (Horn) Clauses Logic programming Prolog is most popular form Depth-first search, so space requirements are lower, but suffers from problems from repeated states

40. 40 american(X) ^ weapon(Y) ^ sells(X,Y,Z) ^ hostile(Z)  criminal(X). owns(nono,m1). missile(m1). missile(X1) ^ owns(nono,X1)  sells(west,X1,nono). missile(X2)  weapon(X2). enemy(X3,america)  hostile(X3). american(west). enemy(nono,america). Goal: criminal(west). Backward chaining proof: in lecture In Prolog: criminal(X) :- american(X), weapon(Y), sells(X,Y,Z), hostile(Z).

41. 41 Horn clauses are all of the form: L1 ^ L2 ^... ^ Ln -> Ln+1 Or, equivalently, in clausal form: ~L1 v ~L2 v... v ~Ln v Ln+1 Prolog (like databases) makes the "closed world assumption": if P cannot be proved, infer not P Think of the system as an arrogant know-it-all: "If it were true, I would know it. Since I can't prove it, it must not be true" Thus, it uses "negation as failure".

42. 42 neighbor(canada,us) neighbor(mexico,us) neighbor(pakistan,india) ?- neighbor(canada,india). no In full first-order logic, you would have to be able to infer “~neighbor(canada,india)" for "neighbor(canada,india)" to be false. Be careful! “~neighbor(canada,india) is not entailed by the Sentences above!

43. 43 bachelor(X) :- male(X), \+ married(X). male(bill). male(jim). married(bill). married(mary). An individual is a bachelor if it is male and it is not married. \+ is the negation-as-failure operator in Prolog. | ?- bachelor(bill). no | ?- bachelor(jim). yes | ?- bachelor(mary). no | ?- bachelor(X). X = jim; no | ?-

44. 44 Comparing backward chaining in prolog with resolution [In lecture]

45. 45 WrapUp You are responsible for everything in Chapter 9 except the following (though you are encouraged to read them): –Storage and Retrieval (p. starts bottom 328) –Efficient forward chaining (starts p. 333) through Irrelevant facts (ends top 337) –Efficient implementation of logic programs (starts p. 340) through Constraint logic programming (ends p. 345) –Completeness of resolution (starts p. 350) (though see notes in slides) Also, see files posted on the schedule (clausal form conversion, resolution, etc.)