1. 1 Natural Language Processing Chapter 14

2. 2 Transition First we did words (morphology) Then we looked at syntax Now we’re moving on to meaning.

3. 3 Semantics What things mean Approach: meaning representations which bridge the gap from linguistic forms to knowledge of the world Serve the practical purposes of a program doing semantic processing

4. 4 Semantic Processing Representations that allow a system to –Answer questions –Determine truth –Perform inference –…

5. 5 Types of Meaning Representations First-order predicate calculus Semantic networks Conceptual dependency Frame-based representations See lecture for examples…

6. 6 Verifiability Does Spice Island serve vegetarian food? Serves(spiceisland,vegetarianfood) Verifiability: the system’s ability to compare representations to facts in memory

7. 7 Current Focus in Class Conventional meanings of words Ignore context Literal meaning

8. 8 Ambiguity I want to eat someplace that’s close to Pitt Mary kissed her husband and Joan did too I baked the cake on the table Old men and women go to the park Every student ate a sandwich

9. 9 Canonical Form Does Spice Island have vegetarian dishes? Do they have vegetarian food at Spice Island? Are vegetarian dishes served at Spice Island? Does Spice Island serve vegetarian fare? Canonical form: inputs that mean the same thing should have the same meaning representations

10. 10 Canonical Form Simplifies reasoning Makes representations more compact (fewer different representations) BUT: makes semantic analysis harder –Need to figure out that “have” and “serve” mean the same thing in the previous examples; same for the various phrases for vegetarian food –BUT: can perform word sense disambiguation; use a single representation for all senses in a synset

11. 11 Representational Schemes We’re going to make use of First Order Predicate Calculus (FOPC) as our representational framework –Not because we think it’s perfect –All the alternatives turn out to be either too limiting or too complicated, or –They turn out to be notational variants

12. 12 Knowledge Based Agents Central component: knowledge base, or KB. A set of sentences in a knowledge representation language Generic Functions –TELL (add a fact to the knowledge base) –ASK (get next action based on info in KB) Both often involve inference, which is? Deriving new sentences from old

13. 13 Fundamental Concepts of logical representation and reasoning Information is represented in sentences, which must have correct syntax ( 1 + 2 ) * 7 = 21 vs. 2 ) + 7 = * ( 1 21 The semantics of a sentence defines its truth with respect to each possible world W is a model of S means that sentence S is true in world W What do the following mean? –X |= Y –X entails Y –Y logically follows from X

14. 14 Entailment A |= B In all worlds in which A is true, B must be true as well All models of A are models of B Whenever A is true, B must be true as well A entails B B logically follows from A

15. 15 Inference KB |-i A Inference algorithm i can derive A from KB i derives A from KB i can derive A from KB A can be inferred from KB by i

16. 16 Propositional Logic Syntax Sentence -> AtomicSent | complexSent AtomicSent -> true|false| P, Q, R … ComplexSent ->  sentence| ( sentence  sentence ) | ( sentence  sentence ) | ( sentence  sentence ) | ( sentence  sentence ) | ( sentence ) [no predicate or function symbols]

17. 17 Propositional Logic Sentences If there is a pit at [1,1], there is a breeze at [1,0] P 11  B 10 There is a breeze at [2,2], if and only if there is a pit in the neighborhood B 22  ( P 21  P 23  P 12  P 32 ) There is no breeze at [2,2]  B 22

18. 18 Semantics of Prop Logic In model-theoretic semantics, an interpretation assigns elements of the world to sentences, and defines the truth values of sentences Propositional logic: easy! Assign T or F to each proposition symbol; then assign truth values to complex sentences in the obvious way

19. 19 Propositional Logic A ^ B is true if both A and B are true A v B is true if one or both A and B are true P  Q equiv ~P v Q. Thus, P  Q is false if P is true and Q is false. Otherwise, P  Q is true. ~A is true if A is false

20. 20 Proofs A derivation A sequence of applications of (usually sound) rules of inference Reasoning by Search Example KB = A  B, B  C, D  E, E  F, D Forward chaining: Add A, infer B, infer C Backward chaining:F? E? D? Yes… Sound but not complete inference procedures

21. 21 Resolution Resolution allows a sound and complete inference mechanism (search-based) using only one rule of inference Resolution rule: –Given: P 1  P 2  P 3 …  P n, and  P 1  Q 1 …  Q m –Conclude: P 2  P 3 …  P n  Q 1 …  Q m Complementary literals P 1 and  P 1 “cancel out” For your information only; resolution won’t be on exam

22. 22 Resolution Any complete search algorithm, applying only the resolution rule, can derive any conclusion entailed by any KB in propositional logic. Refutation completeness: Given A, we cannot use resolution to generate the consequence A v B. But we can answer the question, is A v B true. I.e., resolution can be used to confirm or refute a sentence Again, for your information only; will not be on the exam

23. 23 Unsound (but useful) Inference Where there is smoke, there is fire Example KB: Fire  Smoke, Smoke Abduction: conclude Fire Unsound: –Example KB1: Fire  Smoke, DryIce  Smoke, Smoke –DryIce rather than Fire could be true

24. 24 Propositional Logic  FOPC B11  (P12 v P21) B23  (P32 v P 23 v P34 v P 43) … “Internal squares adjacent to pits are breezy”: All X Y (B(X,Y) ^ (X > 1) ^ (Y > 1) ^ (Y < 4) ^ (X < 4))  (P(X-1,Y) v P(X,Y-1) v P(X+1,Y) v (X,Y+1))

25. 25 FOPC Worlds Rather than just T,F, now worlds contain: Objects: the gold, the wumpus, people, ideas, … “the domain” Predicates: holding, breezy, red, sisters Functions: fatherOf, colorOf, plus Ontological commitment

26. 26 FOPC Syntax Add variables and quantifiers to propositional logic

27. 27 Sentence  AtomicSentence | (Sentence Connective Sentence) | Quantifier Variable, … Sentence | ~Sentence AtomicSentence  Predicate(Term,…) | Term = Term Term  Function(Term,…) | Constant | Variable Connective   | ^ | v |  Quantifier  all, exists Constant  john, 1, … Variable  A, B, C, X Predicate  breezy, sunny, red Function  fatherOf, plus Knowledge engineering involves deciding what types of things Should be constants, predicates, and functions for your problem

28. 28 Examples Everyone likes chocolate –  X (person(X)  likes(X, chocolate)) Someone likes chocolate –  X (person(X) ^ likes(X, chocolate)) Everyone likes chocolate unless they are allergic to it –  X ((person(X) ^  allergic (X, chocolate))  likes(X, chocolate))

29. 29 Quantifiers All X p(X) means that p holds for all elements in the domain Exists X p(X) means that p holds for at least one element of the domain In well-formed FOPC, all variables are bound by (in the scope of) a quantifier

30. 30 Nesting of Variables 1.Everyone likes some kind of food 2.There is a kind of food that everyone likes 3.Someone likes all kinds of food 4.Every food has someone who likes it

31. 31 Answers Everyone likes some kind of food All P (person(P)  Exists F (food(F) and likes(P,F))) There is a kind of food that everyone likes Exists F (food(F) and (All P (person(P)  likes(P,F)))) Someone likes all kinds of food Exists P (person(P) and (All F (food(F)  likes(P,F)))) Every food has someone who likes it All F (food (F)  Exists P (person(P) and likes(P,F)))

32. 32 Semantics of FOPC: Interpretation Specifies which objects, functions, and predicates are referred to by which constant symbols, function symbols, and predicate symbols.

33. 33 Example 3 people: John, Sally, Bill John is tall Sally and Bill are short John is Bill’s father Sally is Bill’s sister Interpretation 1 (others are possible): –“John”, “Sally”, and “Bill” as you think –“person”  {John, Sally,Bill} –“short”  {Sally,Bill} –“tall”  {John} –“sister”  { } A 2-ary predicate –“father”  { } A 1-ary function

34. 34 Example tall(father(bill)) ^ ~sister(sally,bill) Assign meanings to terms: –“bill”  Bill; “sally”  Sally; “father(bill)”  John Assign truth values to atomic sentences –Tall(father(bill)) is T because John is in the set assigned to “tall” –~sister(sally,bill) is F because is in the set assigned to “sister” – So, sentence is false, because T ^ F is F

35. 35 Determining Truth Values Exist X tall(X) : true, because the set assigned to “tall” isn’t {} All X short(X) : false, because there are objects that are not in the set assigned to “short”

36. 36 Representational Schemes What are the objects, predicates, and functions?

37. 37 Choices: Functions vs Predicates Rep-Scheme 1: tall(fatherOf(bob)). Rep-Scheme 2: Exists X (fatherOf(bob,X) ^ tall(X) ^ (All Y (fatherOf(bob,Y)  X = Y))) “fatherOf” in both cases is assigned a set of 2-tuples: {,,…} But {,,,…} is possible if it is a predicate

38. 38 Choices: Predicates versus Constants Rep-Scheme 1: D = {a,b,c,d,e}. red: {a,b,c}. pink: {d,e}. Some true sentences: red(a). red(b). pink(d). ~(All X red(X)). All X (red(X) v pink(X)). But what if we want to say that pink is pretty?

39. 39 Choices: Predicates versus Constants Rep-Scheme 2: D = {a,b,c,d,e,red,pink} colorof:,,,, } pretty: {pink} primary: {red } Some true sentences: colorOf(a,red). colorOf(b,red). colorOf(d,pink). ~(All X colorOf(X,red)). All X (colorOf(X,red) v colorOf(X,pink)). pretty(pink). primary(red). We have reified predicates pink and red: made them into objects

40. 40 Inference with Quantifiers Universal Instantiation: –Given  X (person(X)  likes(X, sun)) –Infer person(john)  likes(john,sun) Existential Instantiation: –Given  x likes(x, chocolate) –Infer: likes(S1, chocolate) –S1 is a “Skolem Constant” that is not found anywhere else in the KB and refers to (one of) the individuals who likes sun.

41. 41 FOPC This choice isn’t completely arbitrary or driven by the needs of practical applications FOPC reflects the semantics of natural languages because it was designed that way by human beings In particular…

42. 42 Meaning Structure of Language The semantics of human languages… –Display a basic predicate-argument structure –Make use of variables –Make use of quantifiers –Use a partially compositional semantics

43. 43 Predicate-Argument Structure Events, actions and relationships can be captured with representations that consist of predicates and arguments to those predicates. Languages display a division of labor where some words and constituents function as predicates and some as arguments.

44. 44 Example Mary gave a list to John Giving(Mary, John, List) More precisely –Gave conveys a three-argument predicate –The first arg refers to the subject (the giver) –The second is the recipient, which is conveyed by the NP in the PP –The third argument refers to the thing given, conveyed by the direct object

45. 45 Is this a good representation? John gave Mary a book for Susan –Giving (john,mary,book,susan) John gave Mary a book for Susan on Wednesday –Giving (john,mary,book,susan,wednesday) John gave Mary a book for Susan on Wednesday in class –Giving (john,mary,book,susan,wednesday,inClass) John gave Mary a book for Susan on Wednesday in class after 2pm –Giving (john,mary,book,susan,wednesday,inClass,>2pm)

46. 46 Reified Representation Exist b,e (ISA(e,giving) ^ agent(e,john) ^ beneficiary(e,sally) ^ patient(e,b) ^ ISA(b,book)) “That happened on Sunday” Add later (assuming S2 is the skolem for e): –happenedOnDay(s2,sunday)

47. 47 Representing Time in Language I ran to Oakland I am running to Oakland I will run to Oakland Now, all represented the same: Exist w (ISA(w,running) ^ agent(w,speaker) ^ dest(w,oakland))

48. 48 Representing Time Events are associated with points or intervals in time. 1.Exist w,i (ISA(w,running) ^ agent(w,speaker) ^ dest(w,oakland) ^ interval(w,i) ^ precedes(i,now)) 2.… member(i,now) 3.… precedes(now,i)

49. 49 Determining Temporal Relations is complex (largely unsolved) Ok, so for Christmas, we fly to Dallas then to El Paso (refers to the future, but the tense is present) Let’s see, flight 1390 will be at the gate an hour now (refers to an interval starting in the past using the future tense) I take the bus in the morning but the incline in the evening (habitual – not a specific morning or evening)

50. 50 Reference Point Flight 2020 arrived Flight 2020 had arrived What’s the difference? What do you expect in the second example?

51. 51 Reference Point Reichenbach (1947) introduced notion of Reference point (R), separated out from Speech time (S) and Event time (E) Example: –When Mary's flight departed, I made the call –When Mary's flight departed, I had made the call Departure event specifies reference point.

52. 52 Reichenbach Applied to Tenses S S R,S S,R,ES,R S We refer to the S,R,E notation as a Basic Tense Structure (BTS)

53. 53 Aspect: A property of Verb Phrases Combo of several qualities of events Statives: express a property of the experiencer; do not imply anything about the time bounds –“I like the Yankees in the World Series” –“I want to go first class” –“I have the flu” Tests for statives: –cannot be used as commands: *“Want to go first class!” –Odd in progressive: “I am wanting to go first class” Subinterval property: a subinterval of having the flue is also having the flue

54. 54 Aspect (continued) Activities: have a time span, but not necessarily an end point –Running, playing the piano, looking for an umbrella Tests: (progressive and commands are fine) –Odd when modified by temporal expressions with “in”: *He was running in 5 minutes Subinterval property: e.g., any subinterval of playing the piano is playing the piano

55. 55 Aspect (continued) Accomplishments: have a well-defined end point, and cause a new state of affairs to exist “I booked a reservation” Test: “in” adverbials ok: “I booked a reservation in 5 minutes” Test and property: Cannot be modified by “stopped” without changing the meaning What happens when you change “booked” to the progressive?

56. 56 Aspect (continued) Achievements: occur at a singular point in time; cause a new state of affairs to exist –“Lost an umbrella”, “noticed the picture”, “arrived at the station” Test: cannot occur over a time span –*I lost an umbrella for three months Note: “I noticed the picture for three months” has an iterative meaning (this is not a simple achievement)