Inference and Resolution for Problem Solving Comp3710 Artificial Intelligence Computing Science Thompson Rivers University

Inference and Resolution Course Outline Part I – Introduction to Artificial Intelligence Part II – Classical Artificial Intelligence, and Searching Knowledge Representation Searching Search Methodologies Advanced Search Genetic Algorithms (relatively new study area) Knowledge Represenation and Automated Reasoning Propositinoal and Predicate Logic Inference and Resolution for Problem Solving Rules and Expert Systems Part III – Machine Learning Part IV – Advanced Topics TRU-COMP3710 Inference and Resolution

Inference and Resolution Chapter Objectives How to convert BNF to CNF Use of the resolution rule for some problems How to construct Horn clauses from given rules and facts Use of forward chaining Use of backward chaining TRU-COMP3710 Inference and Resolution

Inference and Resolution Chapter Outline Introduction Resolution in propositional logic An application of resolution – graph coloring Inference methods – forward and backward chaining Horn clauses Forward chaining Backward chaining TRU-COMP3710 Inference and Resolution

Inference and Resolution Topics 1. Introduction Problem solving using propositional calculus: Based on rules, knowledge, observations, and facts, Decide if a given query is valid. Example Rules and facts (or observation results) If it is raining, Jenny does not go camping. It is raining. It is a hot day. It is afternoon. Question: Does Jenny go fishing? Can you convert them to a logical form, i.e., wff (or sentence)? (Raining  ~Camping)  Raining  HotDay  Afternoon  ~Fishing ??? We need to see the above sentence is contradictory to answer “Does Jenny go fishing?”. TRU-COMP3710 Inference and Resolution

2. Resolution in Propositional Logic CNF(Conjunctive Normal Form) How to convert propositions to CNFs Clauses instead of CNFs for convenience The Resolution rule Resolution refutation Proof by refutation TRU-COMP3710 Inference and Resolution

Conjunctive Normal Forms (CNF) 2. Conjunctive Normal Forms (CNF) BNF (Backus-Naur Form) is not convenient to use. A wff is in Conjunctive Normal Form (CNF) if it is a conjunction of disjunctions: conjunction of disjunctions of literals clauses E.g., (A  B)  (B  C  D) E.g., A  (B  C)  (¬A  ¬B  ¬C  D) Every sentence of propositional logic is logically equivalent to a conjunction of disjunctions of literals. [Q] Really? How to convert? [Q] Why do we need to use CNF? Similarly, a wff is in Disjunctive Normal Form (DNF) if it is a disjunction of conjunctions. E.g., A  (B  C)  (¬A  ¬B  ¬C  D) TRU-COMP3710 Inference and Resolution

Inference and Resolution Converting to CNF Any wff can be converted to CNF by using the following equivalences: A ↔ B  (A → B)  (B → A) biconditional elimination A → B  (¬A  B) implication elimination ¬(A  B)  (¬A  ¬B) de Morgan’s law ¬(A  B)  (¬A)  (¬B) de Morgan’s law ¬ ¬ A  A double-negation elimination A  (B  C)  (A  B)  (A  C) distributivity Importantly, this can be converted into an algorithm – this will be useful when we come to automating resolution. However, the computational complexity of conversion to CNF: NP-hard Examples – convert the following propositions to CNFs. (¬A → B) → ¬C A  (B  C) TRU-COMP3710 Inference and Resolution

Inference and Resolution 2. Examples – convert the following BNF propositions to CNFs. (~A  B)  ~C ~(~A  B)  ~C (~~A  B)  ~C (A  B  ~C) A  (B  C) (A  B)  (B  A) (~A  B)  (~B  A) A  (B  C) TRU-COMP3710 Inference and Resolution

Inference and Resolution 2. Clauses Having converted a wff to CNF, it is usual to write it as a set of clauses. E.g.: (A  B)  C In CNF is: (A  C)  (~B  C) [Q] How? In clause form, we write: {(A, C), (~B, C)} [Q] How to use CNF? Disjunction Conjunction TRU-COMP3710 Inference and Resolution

Inference and Resolution 2. The Resolution Rule The resolution rule is written as follows: A  B, ¬B  C (A  B)  (¬B  C) ╞ (A  C) A  C A  B, ¬B (A  B)  ¬B ╞ A A If the upper part is valid, then the lower part should be valid. This tells us that if we have two clauses that have a literal and its negation, we can combine them by removing that literal. [Q] Can you prove it using the truth table? E.g.: if we have {(A, C), (~A, D)}, we would apply resolution to get {(C, D)} TRU-COMP3710 Inference and Resolution

Resolution Refutation 2. Resolution Refutation Let us resolve: {(~A, B), (~B, C), A, ~C} [Q] We begin by resolving the first clause with the second clause, thus eliminating B and ~B: {(~A, B), (~B, C), A, ~C} => {(~A, C), A, ~C} B is eliminated. => {C, ~C} A is eliminated. => falsum Why falsum? Because the initial assumption, ~C, is not valid. Therefore C is valid. Now we can resolve both remaining literals, which gives falsum: ^ If we reach falsum, we have proved that our initial set of clauses were inconsistent. This can be written as: {(~A, B), (~B, C), A, ~C} ╞ ^ Note {…, A, …, ~A, …} ╞ ^ TRU-COMP3710 Inference and Resolution

Inference and Resolution Proof by Refutation If we want to prove that a logical argument is valid, we negate its conclusion, convert it to clause form, and then try to derive falsum using resolution. If we derive falsum, then our clauses were inconsistent, meaning the original argument was valid, since we negated its conclusion. Proof by refutation is also called proof by contradiction. TRU-COMP3710 Inference and Resolution

Inference and Resolution [Q] Is the following argument valid? (A ˄ ¬B) → C A ˄ ¬B C Start with the negation of the conclusion, and convert to clauses: {(~A, B, C), A, ~B, ~C} [Q] How? Let’s resolve: {(~A, B, C), A, ~B, ~C} => {(B, C), ~B, ~C} How? => {C, ~C} How? =>  We have reached falsum. [Q] Why? Because of the assumption of ~C. Therefore C. Hence, our original argument is valid. TRU-COMP3710 Inference and Resolution

Inference and Resolution Another example: A  B B  C C  D D  E  F  A  F Is this argument valid? ~(A  F)  ~(~A  F)  A  ~F Clause form? {(~A, B), (~B, C), (~C, D), (~D, E, F), A, ~F } Resolution? TRU-COMP3710 Inference and Resolution

Inference and Resolution 2. Topics It is often sensible to represent a refutation proof in tree form: In this case, the proof has failed, as we are left with E instead of falsum. [Q] What does this mean? A  F is not valid nor invalid. How to automate the resolution? TRU-COMP3710 Inference and Resolution

3. Applications of Resolution Resolution refutation can be used to determine if a solution exists for a particular combinatorial problem. For example, for graph coloring, we represent the assignment of colors to the nodes and the constraints regarding edges as propositions, and attempt to prove that the complete set of clauses is satisfiable (i.e., to be deduced to non-false sum). This does not tell us how to color the graph, simply that it is possible. TRU-COMP3710 Inference and Resolution

Inference and Resolution Topics 3-coloring problem: Available colors are r (red), g (green) and b (blue). Ar means that node A has been coloured red. Each node must have exactly one color: Ar  Ag  Ab ¬Ar  ¬Ag ( ¬(Ar  Ag)) ¬Ag  ¬Ab ¬Ab  ¬Ar If (A, B) is an edge, then: ¬Ar  ¬Br ( ¬(Ar  Br)) ¬Ab  ¬Bb ¬Ag  ¬Bg (B, C), (A, C), (C, D) and (A, D) are also edges. Now we construct the complete set of clauses for our graph, and try to see if they are satisfiable, using the resolution rule. It will take long time though. TRU-COMP3710 Inference and Resolution

4. Forward and backward chaining Horn clauses – a simpler form Forward chaining Backward chaining TRU-COMP3710 Inference and Resolution

Inference and Resolution Horn Clauses Horn Form (a more restricted version of propositional language) – conjunction of Horn clauses Horn clause = propositional symbol; or -- facts or observations (conjunction of symbols)  symbol -- rules E.g., C  (B  A)  (C  D  B) Fact or observation: C Rules: if B then A; if C and D then B Modus Ponens (for Horn Form): complete for Horn KBs α1[, … ,αn], α1 [ …  αn]  β β Meaning: Facts and the rules with the facts  then  is a fact. E.g., Rule: if red light, then stop; Fact: red light; then ??? Can be used with forward chaining or backward chaining. These algorithms are very natural and run in linear time. TRU-COMP3710 Inference and Resolution

How to model Horn clauses? Example: Rules: IF P THEN Q IF L and M THEN P IF B and L THEN M IF A and P THEN L IF A and B THEN L Facts or observations: A B Question: Q ? [Q] Horn clauses? and-or graph TRU-COMP3710 Inference and Resolution

Inference and Resolution Example: Rules: If X croaks and X eats flies, then X is a frog. If X chirps and X sings, then X is a canary. If X is a frog, then X is green. If X is a canary, then X is yellow. Observations: Fritz croaks. Fritz eats flies. Question: What is the color of Fritz? [Q] How to answer? [Q] Can you convert the problem to Horn clauses? TRU-COMP3710 Inference and Resolution

Inference and Resolution Another more complex example – ZooKeeper: If X has hair, then X is a mammal. If X gives milk, then X is a mammal. If X has feathers, then X is a bird. If X flies and lays eggs, then X is a bird If X is a mammal and X eats meat, then X is a carnivore. If X is a mammal and X has pointed teeth and X has claws and X has forward pointing eyes, then X is a carnivore. If X is a mammal and X has hoofs, then X is an ungulate. If X is a mammal and X chews cud, then X is an ungulate. If X is a carnivore and X has tawny color and X has dark spots, then X is a cheetah. If X is a carnivore and X has tawny color and X has black strips, then X is a tiger. If X is an ungulate and X has long legs and X has a long neck and X has tawny color and X has dark spots, then X is a giraffe. If X is an ungulate and X has white color and X has black strips, then X is a zebra. TRU-COMP3710 Inference and Resolution

Inference and Resolution 4. If X is a bird and X does not fly and X has long neck and X has long legs and X is black and white, then X is an ostrich. If X is a bird and X does not fly and X swims and X is black and white, then X is a penguin. If X is a bird and X is a good flier, then X is an albatross. Facts/observations: Stretch has hair. Stretch chew cud. Stretch has long legs. Stretch has a long neck. Stretch has tawny color. Stretch has dark spots. Question: What animal is Stretch? [Q] How to answer the above query? Can we use “proof by refutation”? Any problem with that? TRU-COMP3710 Inference and Resolution

Inference and Resolution Forward chaining [Q] The query Q is valid? [Q] Can we use “proof by refutation”? Any problem with that? Knowledge base Facts Query = Q TRU-COMP3710 Inference and Resolution

Inference and Resolution Idea: Start with facts, and fire any rule whose premises are satisfied in the KB; Keep adding its conclusion to the KB, until query is found. Knowledge base OR AND Facts Query = Q AND-OR graph TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Query Facts Rules Rule fired triggered A, B OR AND # of premises Not inferred TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B Reduced by 1 Set true TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 OR AND TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 OR AND TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L OR AND TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L 3 3 TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L 3 3 A, B, L, M TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L 3 3 A, B, L, M 2 2 TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L 3 3 A, B, L, M 2 2 A, B, L, M, P TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L 3 3 A, B, L, M 2 2 A, B, L, M, P 4, 1 TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L 3 3 A, B, L, M 2 2 A, B, L, M, P 4, 1 4 A, B, L, M, P 1 TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 Facts Rules Rule fired triggered A, B 5 5 A, B, L 3 3 A, B, L, M 2 2 A, B, L, M, P 4, 1 4 A, B, L, M, P 1 1 A, B, L, M, P, Q STOP TRU-COMP3710 Inference and Resolution

Inference and Resolution Example – Fritz: Rules Croak  EatFly  Frog Chirp  Sing  Canary Frog  Green Canary  Yellow Facts Croak EatFly Question The color of Fritz. Green Yellow Frog Canary Croak EatFly Chirp Sing Facts Rules triggered Rule fired Croak, EatFly TRU-COMP3710 Inference and Resolution

Inference and Resolution Example – Fritz: Rules Croak  EatFly  Frog Chirp  Sing  Canary Frog  Green Canary  Yellow Facts Croak EatFly Question The color of Fritz. Green Yellow Frog Canary Croak EatFly Chirp Sing Facts Rules triggered Rule fired Croak, EatFly TRU-COMP3710 Inference and Resolution

Inference and Resolution Example – Fritz: Rules Croak  EatFly  Frog Chirp  Sing  Canary Frog  Green Canary  Yellow Facts Croak EatFly Question The color of Fritz. Green Yellow Frog Canary Croak EatFly Chirp Sing Facts Rules triggered Rule fired Croak, EatFly TRU-COMP3710 Inference and Resolution

Inference and Resolution Example – Fritz: Rules Croak  EatFly  Frog Chirp  Sing  Canary Frog  Green Canary  Yellow Facts Croak EatFly Question The color of Fritz. Green Yellow Frog Canary Croak EatFly Chirp Sing Facts Rules triggered Rule fired Croak, EatFly 1 1 TRU-COMP3710 Inference and Resolution

Inference and Resolution 4. Example – Fritz: Rules: Croak  EatFly  Frog Chirp  Sing  Canary Frog  Green Canary  Yellow Facts: Croak EatFly Question: The color of Fritz? Green Yellow Frog Canary Croak EatFly Chirp Sing Facts Rules triggered Rule fired Croak, EatFly 1 1 …, Frog … TRU-COMP3710 Inference and Resolution

Inference and Resolution Backward chaining [Q] Problem of forward chaining (FC)? FC starts with all the asserted symbols, i.e., facts -> longer execution time (We do not have to use all the facts to conclude the given query.) Idea of backward chaining (BC): work backwards from the query q: to prove q by BC, check if q is known already, or prove by BC – only the premises of some rules concluding q Something to consider Avoid loops: check if new subgoal is already on the goal stack Avoid repeated work: check if new subgoal has already been proved true, or has already failed TRU-COMP3710 Inference and Resolution

Inference and Resolution Unknown Facts Goals Matching rules A, B Q Q:? 1 2 3 4 5 TRUE TRU-COMP3710 Inference and Resolution

Inference and Resolution Unknown (subgoal) Facts Goals Matching rules A, B Q 1 A, B P P:? 1 2 3 4 5 TRU-COMP3710 Inference and Resolution

Inference and Resolution Facts Goals Matching rules A, B Q 1 A, B P 2 A, B [L, M] L: ? 1 2 3 4 5 TRU-COMP3710 Inference and Resolution

Inference and Resolution Already in the stack of subgoals Facts Goals Matching rules A, B Q 1 A, B P 2 A, B [L, M] L: 4, 5 1 2 3 4 5 Not fired TRU-COMP3710 Inference and Resolution

Inference and Resolution Facts Goals Matching rules A, B Q 1 A, B P 2 A, B [L, M] L: 4, 5 1 2 3 4 5 TRU-COMP3710 Inference and Resolution

Inference and Resolution Facts Goals Matching rules A, B Q 1 A, B P 2 A, B [L, M] L: 4, 5 A, B, L M M:? 1 2 3 4 5 Became TRUE TRU-COMP3710 Inference and Resolution

Inference and Resolution Facts Goals Matching rules A, B Q 1 A, B P 2 A, B [L, M] L: 4, 5 A, B, L M 3 1 2 3 4 5 TRU-COMP3710 Inference and Resolution

Inference and Resolution Facts Goals Matching rules A, B Q 1 A, B P 2 A, B [L, M] L: 4, 5 A, B, L M 3 A, B, L, M  STOP 1 2 3 4 5 TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 TRU-COMP3710 Inference and Resolution

Inference and Resolution 1 2 3 4 5 TRU-COMP3710 Inference and Resolution

Inference and Resolution Example – Fritz: Rules Croak  EatFly  Frog Chirp  Sing  Canary Frog  Green Canary  Yellow Facts Croak EatFly Question The color of Fritz? Green Yellow Frog Canary Croak EatFly Chirp Sing Facts Goals Matching rules Croak, EatFly Yellow 4 Croak, EatFly Canary 2 Croak, EatFly [Chirp, Sing]  Failure Croak, EatFly Green … … … => Chirp –> false => Canary –> false => Yellow –> false TRU-COMP3710 Inference and Resolution

Inference and Resolution How to use BC to solve ZooKeeper? Animal candidates – Cheetah, Tiger, Giraffe, Zebra, Ostrich, Penguin, Albatross For each of the above animals, Run BC to see if the animal can be deduced. Can you implement BC for ZooKeeper? One good thing of Horn clauses is the conclusion part of if-then rules is a propositional symbol. The condition part of if-then rules is a conjunction of propositional symbols, and hence AND operation. There can be multiple rules having the same conclusion, i.e., propositional symbol. Anyone of the rules can be used, and hence OR operation. As you can see in the example of Yellow in the previous slide, recursion can be used. TRU-COMP3710 Inference and Resolution

Inference and Resolution Recursion with the next representation of rules and facts. If X is a mammal AND X eats meat, then X is a carnivore. If X is a mammal AND X has pointed teeth AND X has claws AND X has forward pointing eyes, then X is a carnivore. … Stretch has hair. Stretch chew cud. var rules = []; rules['Carnivore'] = [ ['Mammal', 'EatMeat'], ['Mammal', 'PointedTooth', 'Claw', 'ForwardPointingEye'] ]; // (Mammal  EatMeat)  // (Mammal  PointedTooth  Claw  ForwardPointingEye) ... var facts = []; facts[0] = 'Hair'; facts[1] = 'ChewCud'; OR DNF (Disjunctive Normal Form) TRU-COMP3710 Inference and Resolution

Inference and Resolution // DNF for rules // rules: associative array in which property values are 2D arrays. rules['Mammal'] = [['Hair'], ['Milk']]; rules['Ungulate'] = [['Mammal', 'Hoof'], ['Mammal', 'ChewCud']]; rules['Giraffe'] = [['Ungulate', 'LongLeg', 'LongNeck', 'TawnyColor', 'DarkSpot']]; rules['Zebra'] = [['Ungulate', 'WhiteColor', 'BlackStrip']];   // Linear array // This array starts with the facts obtained by observing an animal. facts = ['ChewCud', 'DarkSpot', 'Hair', 'LongLeg', LongNeck', 'TawnyColor']; TRU-COMP3710 Inference and Resolution

Inference and Resolution Facts: Stretch has hair. Stretch chew cud. Stretch has long legs. Stretch has a long neck. Stretch has tawny color. Stretch has dark spots. Let’s find if that animal is giraffe, using backward chaining – recursion.   * BC – the recursive function for backward chaining * Facts? – is the propositional symbol fact? * Rules? – is there any rule used for the propositional symbol? BC(…) Fact? Rule? Rule0 Rule1 Return; Action BC(‘Giraffe’) No Yes [‘Ungulate’, ‘LongLeg’, ‘LongNeck’, ‘TawnyColor’, ‘DarkSpot’]   BC(‘Ungulate’) && BC(‘LongLeg’) && BC(‘LongNeck’) && BC(‘TawnyColor’) && BC(‘DarkSpot’) BC(‘Ungulate’) [‘Mammal’, ‘Hoof’] [‘Mammal’, ‘ChewCud’] BC(‘Mammal’) && BC(‘Hoof’) || BC(‘Mammal’) && BC(‘ChewCud’) BC(‘Mammal’) [‘Hair’] [‘Milk’] BC(‘Hair’) || BC(‘Milk’) BC(‘Hair’) TRUE ... TRU-COMP3710 Inference and Resolution

Forward vs. backward chaining 4. Forward vs. backward chaining Topics FC is data-driven, automatic, unconscious processing, E.g., object recognition, routine decisions May do lots of work that is irrelevant to the goal BC is goal-driven, appropriate for problem-solving, E.g., Where are my keys? E.g., How do I get into a PhD program? E.g., My car won’t start. What is the problem? Complexity of BC can be much less than linear in size of KB TRU-COMP3710 Inference and Resolution

Topics Project ideas Resolution refutation for the ZooKeeper problem, with explanation Forward chaining for the ZooKeeper problem, with explanation TRU-COMP3710 Advanced Search