1. ITCS 3153 Artificial Intelligence Lecture 17 First-Order Logic Chapter 9 Lecture 17 First-Order Logic Chapter 9

2. Inference in first-order logic Our goal is to prove that KB entails a fact,  We use logical inferenceWe use logical inference  Forward chaining  Backward chaining  Resolution All three logical inference systems rely on search to find a sequence of actions that derive the empty clause Our goal is to prove that KB entails a fact,  We use logical inferenceWe use logical inference  Forward chaining  Backward chaining  Resolution All three logical inference systems rely on search to find a sequence of actions that derive the empty clause

3. Search and forward chaining Start with KB full of first-order definite clauses Disjunction of literals with exactly one positiveDisjunction of literals with exactly one positive –Equivalent to implication with conjunction of positive literals on left (antecedent / body / premise) and one positive literal on right (consequent / head / conclusion) –Propositional logic used Horn clauses, which permit zero or one to be positive Look for rules with premises that are satisfied (use substitution to make matches) and add conclusions to KBLook for rules with premises that are satisfied (use substitution to make matches) and add conclusions to KB Start with KB full of first-order definite clauses Disjunction of literals with exactly one positiveDisjunction of literals with exactly one positive –Equivalent to implication with conjunction of positive literals on left (antecedent / body / premise) and one positive literal on right (consequent / head / conclusion) –Propositional logic used Horn clauses, which permit zero or one to be positive Look for rules with premises that are satisfied (use substitution to make matches) and add conclusions to KBLook for rules with premises that are satisfied (use substitution to make matches) and add conclusions to KB

4. Search and forward chaining Which rules have premises that are satisfied (modus ponens)?Which rules have premises that are satisfied (modus ponens)? –A ^ E => C… nope –B ^ D => E… yes –E ^ C ^ G ^ H => I… nope  A ^ E = C… yes  E ^ C ^ G ^ H ^ I… nope one more try… yes Which rules have premises that are satisfied (modus ponens)?Which rules have premises that are satisfied (modus ponens)? –A ^ E => C… nope –B ^ D => E… yes –E ^ C ^ G ^ H => I… nope  A ^ E = C… yes  E ^ C ^ G ^ H ^ I… nope one more try… yes Breadth First A, B, D, G, HA, B, D, G, H A ^ E => CA ^ E => C B ^ D => EB ^ D => E E ^ C ^ G ^ H => IE ^ C ^ G ^ H => I Breadth First A, B, D, G, HA, B, D, G, H A ^ E => CA ^ E => C B ^ D => EB ^ D => E E ^ C ^ G ^ H => IE ^ C ^ G ^ H => I

5. Search and forward chaining Would other search methods work? Yes, this technique falls in standard domain of all searchesYes, this technique falls in standard domain of all searches Would other search methods work? Yes, this technique falls in standard domain of all searchesYes, this technique falls in standard domain of all searches

6. Search and backward chaining Start with KB full of implications Find all implications with conclusion matching the queryFind all implications with conclusion matching the query Add to fringe list the unknown premisesAdd to fringe list the unknown premises –Adding could be to front or rear of fringe (depth or breadth) Start with KB full of implications Find all implications with conclusion matching the queryFind all implications with conclusion matching the query Add to fringe list the unknown premisesAdd to fringe list the unknown premises –Adding could be to front or rear of fringe (depth or breadth)

7. Search and backward chaining Are all the premises of I satisfied? NoAre all the premises of I satisfied? No –For each (C E G H) are each of their premises satisfied?  C? no, put its premises on fringe –For each (A and E) are their premises satisfied? A… yes E… no, add premises for each B and D B… yes D… yes –E, G, H… yes Are all the premises of I satisfied? NoAre all the premises of I satisfied? No –For each (C E G H) are each of their premises satisfied?  C? no, put its premises on fringe –For each (A and E) are their premises satisfied? A… yes E… no, add premises for each B and D B… yes D… yes –E, G, H… yes Depth First A, B, D, G, HA, B, D, G, H A ^ E => CA ^ E => C B ^ D => EB ^ D => E C ^ E ^ G ^ H => IC ^ E ^ G ^ H => I Depth First A, B, D, G, HA, B, D, G, H A ^ E => CA ^ E => C B ^ D => EB ^ D => E C ^ E ^ G ^ H => IC ^ E ^ G ^ H => I

8. Search and backward chaining Are all the premises of I satisfied? NoAre all the premises of I satisfied? No –For each (C E G H) are each of their premises satisfied?  C? no, put its premises on fringe end –E? no, put its premises on fringe end –G, H… yes –Are C’s premises (A E) satisfied? A… yes E… no, add premises –Are E’s premises (B D) satisfied? Yes –Return to C and I Are all the premises of I satisfied? NoAre all the premises of I satisfied? No –For each (C E G H) are each of their premises satisfied?  C? no, put its premises on fringe end –E? no, put its premises on fringe end –G, H… yes –Are C’s premises (A E) satisfied? A… yes E… no, add premises –Are E’s premises (B D) satisfied? Yes –Return to C and I Breadth First A, B, D, G, HA, B, D, G, H A ^ E => CA ^ E => C B ^ D => EB ^ D => E C ^ E ^ G ^ H => IC ^ E ^ G ^ H => I Breadth First A, B, D, G, HA, B, D, G, H A ^ E => CA ^ E => C B ^ D => EB ^ D => E C ^ E ^ G ^ H => IC ^ E ^ G ^ H => I

9. Backward/forward chaining Don’t explicitly tie search method to chaining direction

10. Inference with resolution We put each first-order sentence into conjunctive normal formWe put each first-order sentence into conjunctive normal form –We remove quantifiers –We make each sentence a disjunction of literals (each literal is universally quantified) We show KB ^  is unsatisfiable by deriving the empty clauseWe show KB ^  is unsatisfiable by deriving the empty clause –Resolution inference rule is our method  Keep resolving until the empty clause is reached We put each first-order sentence into conjunctive normal formWe put each first-order sentence into conjunctive normal form –We remove quantifiers –We make each sentence a disjunction of literals (each literal is universally quantified) We show KB ^  is unsatisfiable by deriving the empty clauseWe show KB ^  is unsatisfiable by deriving the empty clause –Resolution inference rule is our method  Keep resolving until the empty clause is reached

11. Resolution Look for matching sentences Shared literal with opposite signShared literal with opposite sign –Substitution may be required [Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)][Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)] –F(x) = animal unloved by x –G(x) = someone who loves x Look for matching sentences Shared literal with opposite signShared literal with opposite sign –Substitution may be required [Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)][Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)] –F(x) = animal unloved by x –G(x) = someone who loves x

12. Resolution What does this mean in English? [Animal (F(x)) V Loves (G(x), x)][Animal (F(x)) V Loves (G(x), x)] –F(x) = animal unloved by x –G(x) = someone who loves x [~Loves (u,v) V ~Kills (u, v)][~Loves (u,v) V ~Kills (u, v)] What does this mean in English? [Animal (F(x)) V Loves (G(x), x)][Animal (F(x)) V Loves (G(x), x)] –F(x) = animal unloved by x –G(x) = someone who loves x [~Loves (u,v) V ~Kills (u, v)][~Loves (u,v) V ~Kills (u, v)] For all people, either a person doesn’t love an animal or someone loves the personFor all people, either a person doesn’t love an animal or someone loves the person Nobody loves anybody or nobody kills anybodyNobody loves anybody or nobody kills anybody For all people, either a person doesn’t love an animal or someone loves the personFor all people, either a person doesn’t love an animal or someone loves the person Nobody loves anybody or nobody kills anybodyNobody loves anybody or nobody kills anybody

13. Resolution [Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)][Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)] –Loves and ~Loves cancel with substitution  u/G(x) and v/x Resolvent clauseResolvent clause –[Animal (F(x)) v ~Kills (G(x), x)] [Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)][Animal (F(x)) V Loves (G(x), x)] and [~Loves (u,v) V ~Kills (u, v)] –Loves and ~Loves cancel with substitution  u/G(x) and v/x Resolvent clauseResolvent clause –[Animal (F(x)) v ~Kills (G(x), x)]

14. Example

15. Resolution example

16. Inference with resolution What resolves with what for proof? Unit preferenceUnit preference –Start with single-literal sentences and resolve them with more complicated sentences –Every resolution reduces the size of the sentence by one  Consistent with our goal to find a sentence of size 0 –Resembles forward chaining What resolves with what for proof? Unit preferenceUnit preference –Start with single-literal sentences and resolve them with more complicated sentences –Every resolution reduces the size of the sentence by one  Consistent with our goal to find a sentence of size 0 –Resembles forward chaining

17. Inference with resolution What resolves with what for proof? Set of supportSet of support –Build a special set of sentences –Every resolution includes one sentence from set  New resolvent is added to set –Resembles backward chaining if set of support initialized with negated query What resolves with what for proof? Set of supportSet of support –Build a special set of sentences –Every resolution includes one sentence from set  New resolvent is added to set –Resembles backward chaining if set of support initialized with negated query

18. Theorem provers Logical inference is a powerful way to “reason” automatically Prover should be independent of KB syntaxProver should be independent of KB syntax Prover should use control strategy that is fastProver should use control strategy that is fast Prover can support a human byProver can support a human by –Checking a proof by filling in voids –Person can kill off search even if semi-decidable Logical inference is a powerful way to “reason” automatically Prover should be independent of KB syntaxProver should be independent of KB syntax Prover should use control strategy that is fastProver should use control strategy that is fast Prover can support a human byProver can support a human by –Checking a proof by filling in voids –Person can kill off search even if semi-decidable

19. Practical theorem provers Boyer-MooreBoyer-Moore –First rigorous proof of Godel Incompleteness Theorem OTTEROTTER –Solved several open questions in combinatorial logic EQPEQP –Solved Robbins algebra, a proof of axioms required for Boolean algebra  Problem posed in 1933 and solved in 1997 after eight days of computation Boyer-MooreBoyer-Moore –First rigorous proof of Godel Incompleteness Theorem OTTEROTTER –Solved several open questions in combinatorial logic EQPEQP –Solved Robbins algebra, a proof of axioms required for Boolean algebra  Problem posed in 1933 and solved in 1997 after eight days of computation

20. Practical theorem provers Verification and synthesis of hard/soft ware SoftwareSoftware –Verify a program’s output is correct for all inputs –There exists a program, P, that satisfies a specification HardwareHardware –Verify that interactions between signals and circuits is robust  Will CPU work in all conditions? –There exists a circuit, C, that satisfies a specification Verification and synthesis of hard/soft ware SoftwareSoftware –Verify a program’s output is correct for all inputs –There exists a program, P, that satisfies a specification HardwareHardware –Verify that interactions between signals and circuits is robust  Will CPU work in all conditions? –There exists a circuit, C, that satisfies a specification

21. Statistical Learning Methods Chapter 20 Statistical learning (Bayes, maximum likelihood)Statistical learning (Bayes, maximum likelihood) Hidden variables (expectation maximization, Markov models)Hidden variables (expectation maximization, Markov models) Instance-based (Nearest neighbor)Instance-based (Nearest neighbor) Neural networksNeural networks Chapter 20 Statistical learning (Bayes, maximum likelihood)Statistical learning (Bayes, maximum likelihood) Hidden variables (expectation maximization, Markov models)Hidden variables (expectation maximization, Markov models) Instance-based (Nearest neighbor)Instance-based (Nearest neighbor) Neural networksNeural networks

22. Rational agents Up until now Many rules were available and rationality was piecing rules together to accomplish a goalMany rules were available and rationality was piecing rules together to accomplish a goal –Inference and deduction Now Lots of data available (cause/effect pairs) and rationality is improving performance with dataLots of data available (cause/effect pairs) and rationality is improving performance with data –Model building, generalization, prediction Up until now Many rules were available and rationality was piecing rules together to accomplish a goalMany rules were available and rationality was piecing rules together to accomplish a goal –Inference and deduction Now Lots of data available (cause/effect pairs) and rationality is improving performance with dataLots of data available (cause/effect pairs) and rationality is improving performance with data –Model building, generalization, prediction

23. How early will my son be born? Logic from first principles I think he will be born tomorrowI think he will be born tomorrow –20 literals corresponding to 20 dates –Well-fed (mom(x)) => late(x) –late(x) ^ impatient(father(x)) => thisWeekend (x) –late(x) ^ impatient(mother(x)) => tomorrow(x) –… Logic from first principles I think he will be born tomorrowI think he will be born tomorrow –20 literals corresponding to 20 dates –Well-fed (mom(x)) => late(x) –late(x) ^ impatient(father(x)) => thisWeekend (x) –late(x) ^ impatient(mother(x)) => tomorrow(x) –…

24. How early will my son be born? Statistical Learning Histogram of birthsHistogram of births Data from family treeData from family tree Multidimensional correlations between early and ethnicityMultidimensional correlations between early and ethnicity … Statistical Learning Histogram of birthsHistogram of births Data from family treeData from family tree Multidimensional correlations between early and ethnicityMultidimensional correlations between early and ethnicity …

25. Function Approximator Build a function that maps input to output Start with a model of functionStart with a model of function Use statistics to set values of coefficientsUse statistics to set values of coefficients –Pick m and b such that line defined by terms minimizes the sum of distances between each observed (x, y) and (x, f(x)) Build a function that maps input to output Start with a model of functionStart with a model of function Use statistics to set values of coefficientsUse statistics to set values of coefficients –Pick m and b such that line defined by terms minimizes the sum of distances between each observed (x, y) and (x, f(x)) x y f(x) = mx + b = y

26. Slightly more complicated Parabola Select a, b, cSelect a, b, c Goal is y – ax 2 –bx – c = 0Goal is y – ax 2 –bx – c = 0 –If we have three points and three unknowns we can solve –If we have more points we must use another technique Parabola Select a, b, cSelect a, b, c Goal is y – ax 2 –bx – c = 0Goal is y – ax 2 –bx – c = 0 –If we have three points and three unknowns we can solve –If we have more points we must use another technique x y f(x) = ax 2 + bx + c

27. Mappings These function approximators are mappings They map inputs to outputsThey map inputs to outputs –We hope the outputs match similar observations The mappings become better with more informationThe mappings become better with more information This is what neural networks do But the beauty of neural networks is in how they do what they doBut the beauty of neural networks is in how they do what they do These function approximators are mappings They map inputs to outputsThey map inputs to outputs –We hope the outputs match similar observations The mappings become better with more informationThe mappings become better with more information This is what neural networks do But the beauty of neural networks is in how they do what they doBut the beauty of neural networks is in how they do what they do

28. Neural Networks Biologically inspiredBiologically inspired –We have neurons in our bodies that transmit signals based on inputs  Internal dynamics dependent on chemical gradients  Connections between neurons are important –Tolerates noisy input –Tolerates partial destruction –Perform distributed computation Biologically inspiredBiologically inspired –We have neurons in our bodies that transmit signals based on inputs  Internal dynamics dependent on chemical gradients  Connections between neurons are important –Tolerates noisy input –Tolerates partial destruction –Perform distributed computation

29. Neural Networks Synthetic A neural network unit accepts a vector as input and generates a scalar output dependent on activation functionA neural network unit accepts a vector as input and generates a scalar output dependent on activation function Links within network controlled through weightsLinks within network controlled through weightsSynthetic A neural network unit accepts a vector as input and generates a scalar output dependent on activation functionA neural network unit accepts a vector as input and generates a scalar output dependent on activation function Links within network controlled through weightsLinks within network controlled through weights