Today’s Topics 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 111 Representing Knowledge in a Logic One way to get domain knowledge into a computer, which is still a largely unsolved, major AI challenge Propositional Logic Natural deduction, Resolution theorem proving Satisfiability Efficiency Techniques: Pure Symbols and Unit Clauses Predicate Logic (aka First-Order Predicate Calculus, FOPC; also First-Order Logic, FOL) Next lecture

Why Logic? Historically important in AI However, less practical ‘success stories’ than several other parts of AI –Good, Old-Fashioned AI? (GOFAI) –Importance 5, 10, 25 years from now? Probabilistic logic growing area (later in term) Useful in other parts of CS –Formal analysis of programs and protocols, computer security –Relates to SQL of databases 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 2

Two Goals 1.Learn how to represent (simple) English sentences in a formal representation Need a way to enter ‘world facts’ into computers in a manner that algo’s can use effectively ML weak at making use of ‘domain knowledge,’ but some research on this (including mine) 2.Learn how to mechanically make sound (ie, ‘correct’) inferences/deductions 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 3

Physical Symbol Hypothesis (Newell & Simon, Turing Award Lecture, 1976) Intelligence (human or machine) achieved through the use of 1.Symbols that represent the significant aspects of the given problem domain 2.Operations on these basic and compound symbols that generate potential solutions 3.Search to find a solution among the possibilities 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 4 Allen Newell

FOPC A well-defined representation of knowledge (though maybe too black-and-white) Sound: inferred knowledge correct (‘logically follows’) Complete: all possible inferences can be mechanically produced 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 5

Propositional Logic: Subset of FOPC Components of Prop Logic Atomic sentences P, Q, ItIsRaining, etc like random variables, but are ‘truth valued’ Compound sentences if P and Q are sentences, so are (P) ¬P P ˄ Q P ˅ Q P → QP ↔ Q 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 6 Key ideas of logical reasoning easier to learn in the simpler case of propositional logic

Well-Formed Formula (wff’s) Well formed? P ˄ ˅ Q ¬ ¬ ¬ P P ˄ Q ˅ R - should use parens, though: P ˄ (Q ˅ R) We’ll later use P ˄ Q ˄ ¬R → Z as ‘shorthand’ for (P ˄ Q ˄ (¬R)) → Z 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 7

Propositional Logic: Connectives 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 8 Symbols Used AND: ˄ & comma OR: ˅ NOT: ¬ ~ ! IMPLIES: → EQUIVALANCE: ↔

P → Q as a Venn Diagram - Which One is Most Accurate? 11/17/15CS Fall 2015 (Shavlik©), Lecture 25, Week 11 9 P Q Q P Q P P Q Q P

P → Q as a Venn Diagram - Which One is Most Accurate? 11/17/15CS Fall 2015 (Shavlik©), Lecture 25, Week Q P If we know P is true we can also infer Q is true Knowing Q is false also tells us that P is false But if P is false, we can’t infer anything about Q

Some Propositional Logic Practice Let R = is it raining H = it is hot outside U = I’ll take my umbrella “It is raining but not hot.” R ˄ ¬ H “It is neither raining nor hot.” ¬ (R ˅ H) “It is not raining and it is not hot.” ¬ R ˄ ¬ H // Same as above via ‘DeMorgan’s Law’ “When it is raining, I take my umbrella.” R → U 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Remember, in logic everything ‘evaluates’ to TRUE or FALSE

Interpretations and Models Technical note: an interpretation assigns a truth-value to each proposition we have (like a ‘complete world state’) models are interpretations that make all of our wff’s true Our mechanical-inference algorithm needs to be told the ‘semantics’ (ie, ‘meaning’) of our atomic symbols. It can then compute truth value of any compound wff. 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Duck or rabbit?

Example: Models Models of { P ˅ ¬Q, P → Q } ? 1.P = TRUE and Q = TRUE 2.P = FALSE and Q = FALSE Ie, there are FOUR interpretations of { P, Q }, TWO of which are models 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 13

Some Definitions A logical sentence is called valid (or tautological) if TRUE for ALL models satisfiable if TRUE for SOME models unsatisfiable if TRUE for NO models Valid? Satisfiable? Unsatisfiable? P P ˅ ¬P P ˄ ¬P (P → Q) ↔ (¬P ˅ Q) next slide 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 14

Using Truth Tables to Judge Compound Sentences 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week PQP → Q ¬P ˅ Q(P → Q) ↔ (¬P ˅ Q) FF FT TF TT All possible models

Using Truth Tables to Judge Compound Sentences 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week PQP → Q ¬P ˅ Q(P → Q) ↔ (¬P ˅ Q) FFTTT FTTTT TFFFT TTTTT So TAUTOLOGICAL All possible models Given an interpretation, we can judge the truth value of any compound WFF by creating one row of a table like this

‘Natural Deduction’ in Prop Logic 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Inference rules allow us to deduce NEW wff’s from known wff’s - an alternative to generating all or part of a truth table Notation for inference rules (these will be our operators/actions in a search process): given these wffs are know to be true we can logical deduce these wffs

Deduction in Prop Logic (2) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week AND Elimination (AE) wff 1 ˄ wff 2 ˄ … ˄ wff N wff i If a conjunct is true, then so is each wff in it AND Introduction (AI) wff 1, wff 2, …, wff N wff 1 ˄ wff 2 ˄ … ˄ wff N If we know a bunch of wff’s are true, their conjunction is also true

Deduction in Prop Logic (3) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week OR Elimination (OE) – not an inference rule wff 1 ˅ wff 2 ˅ … ˅ wff N nothing can be deduced If a disjunct is true, we cannot deduce anything OR Introduction (OI) wff 1 wff 1 ˅ wff 2 If we know a wff’s is true, we can ‘disjoin’ it with any other wff and that new wff is also true

Deduction in Prop Logic (4) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Double Negation Elimination (DNE) ¬ ¬ wff wff Two negations cancel out Double Negation Introduction (DNI) wff ¬ ¬ wff If we ‘need’ a negation sign, we can introduce TWO

Deduction in Prop Logic (5) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Equivalence Elimination (EE) wff 1 ↔ wff 2 wff 1 → wff 2 ˄ wff 2 → wff 1

Deduction in Prop Logic (6) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Modus Ponens (MP) wff 1 → wff 2, wff 1 wff 2 If the IF is true in an implication, then we can deduce the THEN Modus Ponens (alternate form) ¬ wff 1 ˅ wff 2, wff 1 wff 2

Deduction in Prop Logic (7) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Unit Resolution (UR) wff 1 ˅ wff 2, ¬ wff 2 wff 1 Given we know wff 2 is false, it cannot satisfy the OR Unit Resolution (alternate form) ¬ wff 1 → wff 2, ¬ wff 2 wff 1 Aside: this is called ‘modus tollens’

Deduction in Prop Logic (8) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Resolution (Rez) wff 1 ˅ wff 2, ¬ wff 2 ˅ wff 3 wff 1 ˅ wff 3 Can understand via a ‘case analysis’ (ie, wff 2 is either true or false, in one case we know wff 1 and the other we know wff 3 ) Resolution is really the ONLY inference rule we need, IF we pre-process our given knowledge and create a ‘canonical’ form called conjunctive normal form (CNF) – more later

Deduction in Prop Logic (9 ) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Another Way to View the Resolution Rule wff 1 ˅ wff 2, ¬ wff 2 ˅ wff 3 wff 1 ˅ wff 3 Convert to implications ¬ wff 1  wff 2, wff 2  wff 3 ¬ wff 1  wff 3 Let wff 4 = ¬ wff 1 wff 4  wff 2, wff 2  wff 3 wff 4  wff 3

Some IMPLICIT Rules (we’ll assume some special-purpose s/w handles these outside of our search algo; in paper-and-pencil questions, no need to justify using them) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week wff 1 ˄ wff 2 wff 2 ˄ wff 1 (wff) wff wff 1 ˅ wff 2 wff 2 ˅ wff 1 wff (wff)

EXAMPLE: Searching for Proofs Given: (1) P (2) Q (3) P ˄ Q → R Show: R State: Initial State: Goal Test: Actions: Search Strategy: 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 27

EXAMPLE: Searching for Proofs Given: (1) P (2) Q (3) P ˄ Q → R Show: R State: the wff’s we know (deduction only ADDs to this list) Initial State: the givens Goal Test: the wff ‘to show’ is in the state Actions: our inference rules Search Strategy: some sort of best-first search? 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 28

Searching for a Proof 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week : P 2: Q 3: P ˄ Q → R 1, 2, 3 4: P ˄ Q 1, 2, 3 4: P ˅ Q 1, 2, 3 4: ¬¬P 1, 2, 3, 4 5: R... MP on 3 and 4 DNI on 1 AI on 1 and 2 OI on 1 and 2 In logic, we only ADD wffs, so each node could simply show the NEW wffss that are added to all the wff’s of all ancestors

We’ll Search on ‘Scratch Paper’ and Only Show the SOLUTION PATH Given: (1) P (2) ¬ Q (3) P → (R ˄ W) (4) R ˄ ¬ Q → S Show: S # WFF Justification 1 P given 2 ¬Q given 3 P → (R ˄ W) given 4 R ˄ ¬ Q → S given 5 R ˄ W MP on 1 and 3 6 R AE on 5 7 R ˄ ¬ Q AI on 2 and 6 8 S MP on 4 and 7 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 30

Resolution Theorem Proving (how computers do logical inferencing) First put all our wff’s in clausal form (aka, conjunctive normal form; CNF) –Essentially, use many of our inference rules in some preprocessing code (more details later) –Results in an AND of ORs with some NOTs inside (¬A ˅ B ˅ ¬Z) ˄ (A) ˄ … ˄ (D ˅ ¬F ˅ S ˅ ¬W) –NEGATE the wff we want to prove –Use the RESOLUTION RULE to derive a contradiction, ie … ˄ (false) ˄ … –If successful (might not be), we have proven the wff 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 31

To show wff 1 given some other wff’s in CNF assume ¬ wff 1 demonstrate this leads to a contradiction via something of this form wff ˅ false, ¬ wff ˅ false false ˅ false or equivalently wff, ¬ wff □ Resolution Theorem Proving Recap (basis of Prolog) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week □ is used to represent the EMPTY clause

A Convenient Notation for Searching for Resolution Proofs (same problem as previous example) P 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week ¬ Q ¬ P ˅ W¬ R ˅ Q ˅ S ¬S ¬ P ˅ R ¬ P ˅ Q ˅ S ¬ P ˅ S S □ We could also simply leave this box empty

Putting WFF’s into Clausal/CNF Form (pg of text) 1.Eliminate all the ↔’s using EE rule 2.Eliminate →’s using P → Q same as ¬ P ˅ Q 3.Push NOT signs next to atomic symbols using i.¬ ¬ P becomes P ii.¬(P ˄ Q) becomes ¬P ˅ ¬ Q // DeMorgan’s Law iii.¬(P ˅ Q) becomes ¬P ˄ ¬ Q // DeMorgan’s Law 4.Distribute ORs over ANDs using (P ˅ (Q ˄ R)) becomes ((P ˅ Q) ˄ (P ˅ R)) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 34

EXAMPLE: Putting WFF’s into Clausal/CNF Form (P ˄ R → ¬ Q) ˄ (Q ↔ R) 1.Eliminate all the ↔’s using EE rule (P ˄ R → ¬ Q) ˄ (Q → R) ˄ (R → Q) 2.Eliminate →’s using P → Q same as ¬ P ˅ Q (¬ (P ˄ R) ˅ ¬ Q) ˄ (¬ Q ˅ R) ˄ (¬ R ˅ Q) 3.Push NOT signs next to atomic symbols using … (¬ P ˅ ¬ R ˅ ¬ Q) ˄ (¬ Q ˅ R) ˄ (¬ R ˅ Q) 4.Distribute ORs over ANDs using No change (ie, not needed in this example) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 35

More Practice 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week WFFs p ˄ r ↔ q w ˅ ¬ z → a c ˄ d → e ˄ ¬ f m ˄ g → ¬ b ˅ h

More Practice 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week WFFs (p ˄ r) ↔ q (w ˅ ¬ z) → a (c ˄ d) → (e ˄ ¬ f) (m ˄ g) → (¬ b ˅ h) CNF (recall: combined by AND) ¬ p ˅ ¬ r ˅ q ¬ q ˅ p ¬ q ˅ r ¬ w ˅ a z ˅ a ¬ c ˅ ¬ d ˅ e ¬ c ˅ ¬ d ˅ ¬ f ¬ m ˅ ¬ g ˅ ¬ b ˅ h Good to remember that (a ˄ b ˄ c ˄ d) → e becomes ¬ a ˅ ¬ b ˅ ¬ c ˅ ¬ d ˅ e and p → (q ˄ r) same as (p → q) ˄ (p → r) Each row is called a clause

Satisfying a Set of WFFs Sometimes we want to find a MODEL of a set of WFFs –Eg, a solution to a set of logical constraints –Recall a model is a setting of the truth values of the atomic symbols that makes all the wff’s true –Satisfiability is fundamental to NP completeness in CS theory ( 11/24/15CS Fall 2015 (Shavlik©), Lecture 26, Week 12 38

Searching for Models If our set of WFF’s has N propositions, we might need to consider 2 N ‘world states’ Can set up as a search task Some often-useful special cases greatly help –one is that we meet satisfy all wff’s without needing to set ALL the prop’s (‘short circuiting’) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week Set A=true Set A=false Set B=true Set B=false

Useful Preprocessing Tricks for Satisfiability (recall ‘constraint propagation’ in Sudoku) We have a set of clauses (ie, WFFs in CNF) Pure Symbol: if an atomic symbol always appears with the same ‘sign’ (eg, only P appears or only  P appears), then we can set its truth value, so no need to try both TRUE and FALSE Unit Clause Propagation: if a clause has only ONE item in it, then we have a ‘forced move,’ eg … {  p  q  r  …   z } … {  r } … {  b  …   r } … forces us to set r = false and we get … {  p  q  false  …   z } … { true } … {  b  …  true } … which simplifies to … {  p  q  …   z } … 11/24/15CS Fall 2015 (Shavlik©), Lecture 26, Week 12 40

Example 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week (Q  false  ¬ R) (P  ¬ R) (P  ¬ B  ¬ Q) (¬ P  R) (true) (Q  ¬ R) (P  ¬ R) (P  true  ¬ Q) (¬ P  R) (Q  ¬ R) (P  ¬ R) (¬ P  R) (true  ¬ R) (P  ¬ R) (¬ P  R) (P  ¬ R) (¬ P  R) (Q  ¬ A  ¬ R) (P  ¬ R) (P  ¬ B  ¬ Q) (¬ P  R) (A) simplify A must be true let B = false (Q  ¬ R) (P  ¬ R) (P  ¬ B  ¬ Q) (¬ P  R) let Q = true simplify Now only 2 2 combo’s to try instead of 2 5

Where We Are Have a well-defined, though simple, logic Can represent ‘world/domain knowledge’ (or the rules of chess, taxes, calculus, etc) We have mechanisms for determining what logically follows (don’t want to derive EVERYTHING that follows since unbounded, eg P, ¬¬P, ¬¬¬¬P, etc) –truth tables (brute-force enumeration) –‘natural deduction’ –resolution theorem proving Can search for models of sets of WFFs (‘satisfiability’) 11/17/15CS Fall 2015 (Shavlik©), Lecture 24, Week 11 42