1. Today’s Topics 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 111 Representing Knowledge in a Logic One way to get domain knowledge into a computer, which is still a largely unsolved, major AI challenge Propositional Logic Natural deduction, Resolution theorem proving Satisfiability Efficiency Techniques: Pure Symbols and Unit Clauses Predicate Logic (aka First-Order Predicate Calculus, FOPC; also First-Order Logic, FOL) Next lecture

2. Why Logic? Historically important in AI However, less practical ‘success stories’ than several other parts of AI –Good, Old-Fashioned AI? (GOFAI) –Importance 5, 10, 25 years from now? Probabilistic logic growing area (later in term) Useful in other parts of CS –Formal analysis of programs and protocols, computer security –Relates to SQL of databases 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 2

3. Two Goals 1.Learn how to represent (simple) English sentences in a formal representation Need a way to enter ‘world facts’ into computers in a manner that algo’s can use effectively ML weak at making use of ‘domain knowledge,’ but some research on this (including mine) 2.Learn how to mechanically make sound (ie, ‘correct’) inferences/deductions 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 3

4. Physical Symbol Hypothesis (Newell & Simon, Turing Award Lecture, 1976) Intelligence (human or machine) achieved through the use of 1.Symbols that represent the significant aspects of the given problem domain 2.Operations on these basic and compound symbols that generate potential solutions 3.Search to find a solution among the possibilities 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 4 Allen Newell 1927 -1992

5. FOPC A well-defined representation of knowledge (though maybe too black-and-white) Sound: inferred knowledge correct (‘logically follows’) Complete: all possible inferences can be mechanically produced 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 5

6. Propositional Logic: Subset of FOPC Components of Prop Logic Atomic sentences P, Q, ItIsRaining, etc like random variables, but are ‘truth valued’ Compound sentences if P and Q are sentences, so are (P) ¬P P ˄ Q P ˅ Q P → QP ↔ Q 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 6 Key ideas of logical reasoning easier to learn in the simpler case of propositional logic

7. Well-Formed Formula (wff’s) Well formed? P ˄ ˅ Q ¬ ¬ ¬ P P ˄ Q ˅ R - should use parens, though: P ˄ (Q ˅ R) We’ll later use P ˄ Q ˄ ¬R → Z as ‘shorthand’ for (P ˄ Q ˄ (¬R)) → Z 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 7

8. Propositional Logic: Connectives 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 8 Symbols Used AND: ˄ & comma OR: ˅ NOT: ¬ ~ ! IMPLIES: → EQUIVALANCE: ↔

9. P → Q as a Venn Diagram - Which One is Most Accurate? 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 25, Week 11 9 P Q Q P Q P P Q Q P

10. P → Q as a Venn Diagram - Which One is Most Accurate? 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 25, Week 11 10 Q P If we know P is true we can also infer Q is true Knowing Q is false also tells us that P is false But if P is false, we can’t infer anything about Q

11. Some Propositional Logic Practice Let R = is it raining H = it is hot outside U = I’ll take my umbrella “It is raining but not hot.” R ˄ ¬ H “It is neither raining nor hot.” ¬ (R ˅ H) “It is not raining and it is not hot.” ¬ R ˄ ¬ H // Same as above via ‘DeMorgan’s Law’ “When it is raining, I take my umbrella.” R → U 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 11 Remember, in logic everything ‘evaluates’ to TRUE or FALSE

12. Interpretations and Models https://en.wikipedia.org/wiki/Interpretation_(logic) https://en.wikipedia.org/wiki/Interpretation_(logic) Technical note: an interpretation assigns a truth-value to each proposition we have (like a ‘complete world state’) models are interpretations that make all of our wff’s true Our mechanical-inference algorithm needs to be told the ‘semantics’ (ie, ‘meaning’) of our atomic symbols. It can then compute truth value of any compound wff. 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 12 Duck or rabbit?

13. Example: Models Models of { P ˅ ¬Q, P → Q } ? 1.P = TRUE and Q = TRUE 2.P = FALSE and Q = FALSE Ie, there are FOUR interpretations of { P, Q }, TWO of which are models 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 13

14. Some Definitions A logical sentence is called valid (or tautological) if TRUE for ALL models satisfiable if TRUE for SOME models unsatisfiable if TRUE for NO models Valid? Satisfiable? Unsatisfiable? P P ˅ ¬P P ˄ ¬P (P → Q) ↔ (¬P ˅ Q) next slide 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 14

15. Using Truth Tables to Judge Compound Sentences 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 15 PQP → Q ¬P ˅ Q(P → Q) ↔ (¬P ˅ Q) FF FT TF TT All possible models

16. Using Truth Tables to Judge Compound Sentences 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 16 PQP → Q ¬P ˅ Q(P → Q) ↔ (¬P ˅ Q) FFTTT FTTTT TFFFT TTTTT So TAUTOLOGICAL All possible models Given an interpretation, we can judge the truth value of any compound WFF by creating one row of a table like this

17. ‘Natural Deduction’ in Prop Logic 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 17 Inference rules allow us to deduce NEW wff’s from known wff’s - an alternative to generating all or part of a truth table Notation for inference rules (these will be our operators/actions in a search process): given these wffs are know to be true we can logical deduce these wffs

18. Deduction in Prop Logic (2) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 18 AND Elimination (AE) wff 1 ˄ wff 2 ˄ … ˄ wff N wff i If a conjunct is true, then so is each wff in it AND Introduction (AI) wff 1, wff 2, …, wff N wff 1 ˄ wff 2 ˄ … ˄ wff N If we know a bunch of wff’s are true, their conjunction is also true

19. Deduction in Prop Logic (3) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 19 OR Elimination (OE) – not an inference rule wff 1 ˅ wff 2 ˅ … ˅ wff N nothing can be deduced If a disjunct is true, we cannot deduce anything OR Introduction (OI) wff 1 wff 1 ˅ wff 2 If we know a wff’s is true, we can ‘disjoin’ it with any other wff and that new wff is also true

20. Deduction in Prop Logic (4) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 20 Double Negation Elimination (DNE) ¬ ¬ wff wff Two negations cancel out Double Negation Introduction (DNI) wff ¬ ¬ wff If we ‘need’ a negation sign, we can introduce TWO

21. Deduction in Prop Logic (5) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 21 Equivalence Elimination (EE) wff 1 ↔ wff 2 wff 1 → wff 2 ˄ wff 2 → wff 1

22. Deduction in Prop Logic (6) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 22 Modus Ponens (MP) wff 1 → wff 2, wff 1 wff 2 If the IF is true in an implication, then we can deduce the THEN Modus Ponens (alternate form) ¬ wff 1 ˅ wff 2, wff 1 wff 2

23. Deduction in Prop Logic (7) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 23 Unit Resolution (UR) wff 1 ˅ wff 2, ¬ wff 2 wff 1 Given we know wff 2 is false, it cannot satisfy the OR Unit Resolution (alternate form) ¬ wff 1 → wff 2, ¬ wff 2 wff 1 Aside: this is called ‘modus tollens’

24. Deduction in Prop Logic (8) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 24 Resolution (Rez) wff 1 ˅ wff 2, ¬ wff 2 ˅ wff 3 wff 1 ˅ wff 3 Can understand via a ‘case analysis’ (ie, wff 2 is either true or false, in one case we know wff 1 and the other we know wff 3 ) Resolution is really the ONLY inference rule we need, IF we pre-process our given knowledge and create a ‘canonical’ form called conjunctive normal form (CNF) – more later

25. Deduction in Prop Logic (9 ) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 25 Another Way to View the Resolution Rule wff 1 ˅ wff 2, ¬ wff 2 ˅ wff 3 wff 1 ˅ wff 3 Convert to implications ¬ wff 1  wff 2, wff 2  wff 3 ¬ wff 1  wff 3 Let wff 4 = ¬ wff 1 wff 4  wff 2, wff 2  wff 3 wff 4  wff 3

26. Some IMPLICIT Rules (we’ll assume some special-purpose s/w handles these outside of our search algo; in paper-and-pencil questions, no need to justify using them) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 26 wff 1 ˄ wff 2 wff 2 ˄ wff 1 (wff) wff wff 1 ˅ wff 2 wff 2 ˅ wff 1 wff (wff)

27. EXAMPLE: Searching for Proofs Given: (1) P (2) Q (3) P ˄ Q → R Show: R State: Initial State: Goal Test: Actions: Search Strategy: 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 27

28. EXAMPLE: Searching for Proofs Given: (1) P (2) Q (3) P ˄ Q → R Show: R State: the wff’s we know (deduction only ADDs to this list) Initial State: the givens Goal Test: the wff ‘to show’ is in the state Actions: our inference rules Search Strategy: some sort of best-first search? 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 28

29. Searching for a Proof 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 29 1: P 2: Q 3: P ˄ Q → R 1, 2, 3 4: P ˄ Q 1, 2, 3 4: P ˅ Q 1, 2, 3 4: ¬¬P 1, 2, 3, 4 5: R... MP on 3 and 4 DNI on 1 AI on 1 and 2 OI on 1 and 2 In logic, we only ADD wffs, so each node could simply show the NEW wffss that are added to all the wff’s of all ancestors

30. We’ll Search on ‘Scratch Paper’ and Only Show the SOLUTION PATH Given: (1) P (2) ¬ Q (3) P → (R ˄ W) (4) R ˄ ¬ Q → S Show: S # WFF Justification 1 P given 2 ¬Q given 3 P → (R ˄ W) given 4 R ˄ ¬ Q → S given 5 R ˄ W MP on 1 and 3 6 R AE on 5 7 R ˄ ¬ Q AI on 2 and 6 8 S MP on 4 and 7 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 30

31. Resolution Theorem Proving (how computers do logical inferencing) First put all our wff’s in clausal form (aka, conjunctive normal form; CNF) –Essentially, use many of our inference rules in some preprocessing code (more details later) –Results in an AND of ORs with some NOTs inside (¬A ˅ B ˅ ¬Z) ˄ (A) ˄ … ˄ (D ˅ ¬F ˅ S ˅ ¬W) –NEGATE the wff we want to prove –Use the RESOLUTION RULE to derive a contradiction, ie … ˄ (false) ˄ … –If successful (might not be), we have proven the wff 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 31

32. To show wff 1 given some other wff’s in CNF assume ¬ wff 1 demonstrate this leads to a contradiction via something of this form wff ˅ false, ¬ wff ˅ false false ˅ false or equivalently wff, ¬ wff □ Resolution Theorem Proving Recap (basis of Prolog) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 32 □ is used to represent the EMPTY clause

33. A Convenient Notation for Searching for Resolution Proofs (same problem as previous example) P 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 33 ¬ Q ¬ P ˅ W¬ R ˅ Q ˅ S ¬S ¬ P ˅ R ¬ P ˅ Q ˅ S ¬ P ˅ S S □ We could also simply leave this box empty

34. Putting WFF’s into Clausal/CNF Form (pg 253-254 of text) 1.Eliminate all the ↔’s using EE rule 2.Eliminate →’s using P → Q same as ¬ P ˅ Q 3.Push NOT signs next to atomic symbols using i.¬ ¬ P becomes P ii.¬(P ˄ Q) becomes ¬P ˅ ¬ Q // DeMorgan’s Law iii.¬(P ˅ Q) becomes ¬P ˄ ¬ Q // DeMorgan’s Law 4.Distribute ORs over ANDs using (P ˅ (Q ˄ R)) becomes ((P ˅ Q) ˄ (P ˅ R)) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 34

35. EXAMPLE: Putting WFF’s into Clausal/CNF Form (P ˄ R → ¬ Q) ˄ (Q ↔ R) 1.Eliminate all the ↔’s using EE rule (P ˄ R → ¬ Q) ˄ (Q → R) ˄ (R → Q) 2.Eliminate →’s using P → Q same as ¬ P ˅ Q (¬ (P ˄ R) ˅ ¬ Q) ˄ (¬ Q ˅ R) ˄ (¬ R ˅ Q) 3.Push NOT signs next to atomic symbols using … (¬ P ˅ ¬ R ˅ ¬ Q) ˄ (¬ Q ˅ R) ˄ (¬ R ˅ Q) 4.Distribute ORs over ANDs using No change (ie, not needed in this example) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 35

36. More Practice 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 36 WFFs p ˄ r ↔ q w ˅ ¬ z → a c ˄ d → e ˄ ¬ f m ˄ g → ¬ b ˅ h

37. More Practice 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 37 WFFs (p ˄ r) ↔ q (w ˅ ¬ z) → a (c ˄ d) → (e ˄ ¬ f) (m ˄ g) → (¬ b ˅ h) CNF (recall: combined by AND) ¬ p ˅ ¬ r ˅ q ¬ q ˅ p ¬ q ˅ r ¬ w ˅ a z ˅ a ¬ c ˅ ¬ d ˅ e ¬ c ˅ ¬ d ˅ ¬ f ¬ m ˅ ¬ g ˅ ¬ b ˅ h Good to remember that (a ˄ b ˄ c ˄ d) → e becomes ¬ a ˅ ¬ b ˅ ¬ c ˅ ¬ d ˅ e and p → (q ˄ r) same as (p → q) ˄ (p → r) Each row is called a clause

38. Satisfying a Set of WFFs Sometimes we want to find a MODEL of a set of WFFs –Eg, a solution to a set of logical constraints –Recall a model is a setting of the truth values of the atomic symbols that makes all the wff’s true –Satisfiability is fundamental to NP completeness in CS theory (https://en.wikipedia.org/wiki/Boolean_satisfiability_problem)https://en.wikipedia.org/wiki/Boolean_satisfiability_problem 11/24/15CS 540 - Fall 2015 (Shavlik©), Lecture 26, Week 12 38

39. Searching for Models If our set of WFF’s has N propositions, we might need to consider 2 N ‘world states’ Can set up as a search task Some often-useful special cases greatly help –one is that we meet satisfy all wff’s without needing to set ALL the prop’s (‘short circuiting’) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 39 Set A=true Set A=false Set B=true Set B=false

40. Useful Preprocessing Tricks for Satisfiability (recall ‘constraint propagation’ in Sudoku) We have a set of clauses (ie, WFFs in CNF) Pure Symbol: if an atomic symbol always appears with the same ‘sign’ (eg, only P appears or only  P appears), then we can set its truth value, so no need to try both TRUE and FALSE Unit Clause Propagation: if a clause has only ONE item in it, then we have a ‘forced move,’ eg … {  p  q  r  …   z } … {  r } … {  b  …   r } … forces us to set r = false and we get … {  p  q  false  …   z } … { true } … {  b  …  true } … which simplifies to … {  p  q  …   z } … 11/24/15CS 540 - Fall 2015 (Shavlik©), Lecture 26, Week 12 40

41. Example 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 41 (Q  false  ¬ R) (P  ¬ R) (P  ¬ B  ¬ Q) (¬ P  R) (true) (Q  ¬ R) (P  ¬ R) (P  true  ¬ Q) (¬ P  R) (Q  ¬ R) (P  ¬ R) (¬ P  R) (true  ¬ R) (P  ¬ R) (¬ P  R) (P  ¬ R) (¬ P  R) (Q  ¬ A  ¬ R) (P  ¬ R) (P  ¬ B  ¬ Q) (¬ P  R) (A) simplify A must be true let B = false (Q  ¬ R) (P  ¬ R) (P  ¬ B  ¬ Q) (¬ P  R) let Q = true simplify Now only 2 2 combo’s to try instead of 2 5

42. Where We Are Have a well-defined, though simple, logic Can represent ‘world/domain knowledge’ (or the rules of chess, taxes, calculus, etc) We have mechanisms for determining what logically follows (don’t want to derive EVERYTHING that follows since unbounded, eg P, ¬¬P, ¬¬¬¬P, etc) –truth tables (brute-force enumeration) –‘natural deduction’ –resolution theorem proving Can search for models of sets of WFFs (‘satisfiability’) 11/17/15CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11 42