1. CPSC 503 Computational Linguistics
Representing Meaning
Lecture 15
Giuseppe Carenini
12/8/2018
CPSC503 Spring 2004

2. Knowledge-Formalisms Map (including probabilistic formalisms)
State Machines (and prob. versions)
(Finite State Automata,Finite State Transducers, Markov Models)
Morphology
Syntax
Rule systems (and prob. versions)
(e.g., (Prob.) Context-Free Grammars)
Semantics
Pragmatics
Discourse and Dialogue
Logical formalisms
(First-Order Logics)
AI planners
12/8/2018
CPSC503 Spring 2004

3. Next four classes What meaning is and how to represent it
How to map sentences into their meaning
Meaning of individual words
Tasks:
Information Extraction
Information Retrieval
How the mening of a sentence depends on the emaning of its constituents
phrases and words
compositional semantics
12/8/2018
CPSC503 Spring 2004

4. Today 12/3 Semantics / Meaning Meaning Representations
First-Order Logics
Linguistically relevant Concepts
12/8/2018
CPSC503 Spring 2004

5. Semantics
Def. Semantics: The study of the meaning of words, intermediate constituents and sentences
Def1. Meaning: a representation that expresses the linguistic input in terms of objects, actions, events, time, space… beliefs, attitudes...relationships
Def2. Meaning: a representation that links the linguistic input to knowledge of the world
How the meaning of a phrase is related to the meaning of its parts
What sorts of meaning structures and meaning relations obtain in natural language
Semantic relations among words
Link linguistic inputs to a non-linguistic representation of the world
First-Order Logics
Symbol structures that corresponds to objects and relations among objects in some world being represented
…Representation for a particular state of affairs in some world
Language independent!
12/8/2018
CPSC503 Spring 2004

6. Semantic Relations involving Sentences
Same truth conditions
Paraphrase: have the same meaning
I gave the apple to John vs. I gave John the apple
I bought a car from you vs. you sold a car to me
The thief was chased by the police vs. ……
Entailment: “implication”
The park rangers killed the bear vs. The bear is dead
Nemo is a fish vs. Nemo is an animal
If one is true the other must be true
Some linguists feel that languages do not permit two or more structures to have exactly identical meanings….
Contradiction:
I am in Vancouver vs. I am in California
12/8/2018
CPSC503 Spring 2004

7. Words from Nonlexical categories
Grammaticization
Concept
Affix
-ed -s
re-
in-, un-, de-
Past
More than one
Again
Negation
Words from Nonlexical categories
Obligation
Possibility
Definite, Specific
Indefinite, Non-specific
Disjunction
Negation
Conjunction
must
may
the a or
not
and
Of the indefinitely large set of concepts expressible in human language, a relatively small set enjoys a special status
concepts that are lexicalized as affixes and nonlexical (functional / closed-class) categories in most of the world’s languages
Two kinds of category
Closed class (generally are function words)
Prepositions, articles, conjunctions, pronouns, determiners, aux, numerals
Open class
Nouns (proper/common; mass/count), verbs, adjectives, adverbs
If you run across an unknown word….??
12/8/2018
CPSC503 Spring 2004

8. Common Meaning Representations
I have a car
FOL
Semantic Nets
They all share a common foundation: Meaning Representation consists of structures composed of sets of symbols
Symbol structures are objects and relations among objects
We’re going to take the same basic approach to meaning that we took to syntax and morphology
We’re going to create representations of linguistic inputs that capture the meanings of those inputs.
But unlike parse trees and the like these representations aren’t primarily descriptions of the structure of the inputs…
Conceptual Dependency
Frames
12/8/2018
CPSC503 Spring 2004

9. Requirements for Meaning Representations
Sample NLP Task: giving advice about restaurants
Accept queries in NL
Generate appropriate responses by consulting a KB
e.g,
Does Maharani serve vegetarian food?
-> Yes
What restaurants are close to the ocean?
-> C and Monks
12/8/2018
CPSC503 Spring 2004

10. Verifiability (in the world?)
Example: Does LeDog serve vegetarian food?
Knowledge base (KB) expressing our world model
Convert question to KB language and verify its truth value against the KB content
It must be possible to use the representation to verify whether the sentence is true in the world (with respect to our representation of the world)
Sample entry in KB: Serves(LeDog,Vegetarian Food)
System can match input representation against representations in knowledge base. If it finds a match, it can return Yes; Otherwise No.
12/8/2018
CPSC503 Spring 2004

11. Gozzilla interpretation
Unambiguousness
Gozzilla interpretation
Example: I want to eat some place near campus.
Final representations should be unambiguous
Vagueness: I want to eat Spanish food.
Single linguistic input can have different meaning representations
Each representation unambiguously characterizes one meaning
Answer will depends on which interpretation is chosen
System should allow us to represent vagueness
12/8/2018
CPSC503 Spring 2004

12. Canonical form
Paraphrases should be mapped into the same representation.
Does LeDog have vegetarian dishes?
Do they have vegetarian food at LeDog?
Are vegetarian dishes served at LeDog?
Does LeDog serve vegetarian fare?
……………
Alternatives use different words widely varying syntactic analysis
they should not be mapped in substantially different meanings
Computational purpose that meaning representation serve...
in particular in this domain we want to give the same answer!
What about to have/serve a tennis ball
word sense disambiguation
Having vs. serving
Food vs. fare vs. dishes
12/8/2018
CPSC503 Spring 2004

13. How to Produce a Canonical Form
Systematic Meaning Representations can be derived from thesaurus
food ___
dish ___|____one overlapping meaning sense
fare ___|
We can systematically relate syntactic constructions
[S [NP Maharani] serves [NP vegetarian dishes]]
[S [NP vegetarian dishes] are served at [NP Maharani]]
12/8/2018
CPSC503 Spring 2004

14. Inference and Expressiveness
Consider a more complex request
Can vegetarians eat at Maharani?
Vs: Does Maharani serve vegetarian food?
Why do these result in the same answer?
Inference: System’s ability to draw valid conclusions based on the meaning representations of inputs and its KB
serve(Maharani,VegetarianFood) => CanEat(Vegetarians,At(Maharani))
Not because they mean the same thingQ
Expressiveness: system must be able to handle a wide range of subject matter
12/8/2018
CPSC503 Spring 2004

15. Non Yes/No Questions
Example: I'd like to find a restaurant where I can get vegetarian food.
Indefinite reference <-> variable
serve(x,VegetarianFood)
Matching succeeds only if variable x can be replaced by known object in KB.
12/8/2018
CPSC503 Spring 2004

16. Meaning Structure of Language
How does language convey meaning?
Grammaticization
Tense systems
Conjunctions
Quantifiers ……
Display a partially compositional semantics
Display a basic predicate-argument structure
12/8/2018
CPSC503 Spring 2004

17. Predicate-Argument Structure
Represent relationships among concepts
Some words act like arguments and some words act like predicates:
Nouns as concepts or arguments: red(ball)
Adj, Adv, Verbs as predicates: red(ball)
Subcategorization frames specify number, position, and syntactic category of arguments
Examples: give NP2 NP1, find NP, sneeze []
All human languages
a specific relation holds between the concepts expressed by the words or the phrases
Events, actions and relationships can be captured with representations that consist of predicates and arguments.
Languages display a division of labor where some words and constituents function as predicates and some as arguments.
One of the most important roles of the grammar is to help organize this pred-args structure
12/8/2018
CPSC503 Spring 2004

18. Semantic (Thematic) Roles
This can be extended to the realm of semantics
Semantic Roles: Participants in an event
Agent: George hit Bill. Bill was hit by George
Theme: George hit Bill. Bill was hit by George
Source, Goal, Instrument, Force…
Verb subcategorization: Allows linking arguments in surface structure with their semantic roles
Predicates
Primarily Verbs, VPs, PPs, Sentences
Sometimes Nouns and NPs
Arguments
Primarily Nouns, Nominals, NPs
But also everything else; as we’ll see it depends on the context
Mary gave/sent/read a book to Ming
Agent Theme Goal
Mary gave/sent/read Ming a book
Agent Goal Theme
12/8/2018
CPSC503 Spring 2004

19. Selectional Restrictions
Semantic (Selectional) Restrictions: Constrain the types of arguments verbs take
George assassinated the senator
*The spider assassinated the fly
Predicates
Primarily Verbs, VPs, PPs, Sentences
Sometimes Nouns and NPs
Arguments
Primarily Nouns, Nominals, NPs
But also everything else; as we’ll see it depends on the context
12/8/2018
CPSC503 Spring 2004

20. First Order Predicate Calculus (FOPC)
FOPC provides sound computational basis for verifiability, inference, expressiveness…
Supports determination of truth
Supports Canonical Form
Supports compositionality of meaning
Supports question-answering (via variables)
Supports inference
Allows for…
The analysis of truth conditions
Allows us to answer yes/no questions
Supports the use of variables
Allows us to answer questions through the use of variable binding
Supports inference
Allows us to answer questions that go beyond what we know explicitly
FOPC reflects the semantics of natural languages because it was designed that way by human beings
12/8/2018
CPSC503 Spring 2004

21. FOPC Syntax Give(sister-of(John), Mary, x)
AtomicFormula  Predicate (Term, …)
Term  Function (Term,…) | Constant | Variable
Constant  B | VegetarianFood | LeDog
Variable  x | y | …
Predicate  Serves | Near | …
Function  LocationOf | CuisineOf | …
Give(sister-of(John), Mary, x)
Formula  AtomicFormula | Formula Connective Formula | Quantifier Variable, … Formula | Ø Formula | (Formula)
Connective  |  | Ù | Þ
Quantifier  " | $
Context free grammar specification of FOL
examine this bottom up fashion
terms represent objects
functions often expressed in English as genitives
statements about unknown objects
Terms
Constants: Maharani
Functions: LocationOf(Maharani)
Variables: x in LocationOf(x)
Predicates: Relations that hold among objects
Serves(Maharani,VegetarianFood)
Logical Connectives: Permit compositionality of meaning
I only have $5 and I don’t have a lot of time
Have(I,$5) Ù Have(I,LotofTime)
12/8/2018
CPSC503 Spring 2004

22. FOPC Semantics
Formulas in FOPC can be assigned truth values True or False
Database semantics for atomic formulas
LeDog is near campus.
Near(LocationOf(LeDog),LocationOf(campus))
Truth tables for connectives
12/8/2018
CPSC503 Spring 2004
No common-sense

23. Variables and Quantifiers
Existential ($): There exists
A restaurant that serves Mexican food near UBC
($x) Restaurant(x) Ù Serves(x,MexicalFood) Ù Near(LocationOf(x),LocationOf(UMD))
Universal ("): For all
All vegetarian restaurants serve vegetarian food
("x) VegetarianRestaurant(x) Þ Serves(x,VegetarianFood)
12/8/2018
CPSC503 Spring 2004

24. Connectives I only have five dollars and I don’t have a lot of time.
Have(Speaker,FiveDollars) Ù Ø Have(Speaker,LotOfTime)
12/8/2018
CPSC503 Spring 2004

25. Inference Modus ponens:   Þ  
Example: VegetarianRestaurant(Joe’s)  x: VegetarianRestaurant(x) Þ Serves(x,VegetarianFood) Serves(Joe’s,VegetarianFood)
One of the most important desiderata for a meaning representation is that it should support inference
12/8/2018
CPSC503 Spring 2004

26. Uses of modus ponens
Forward chaining: as individual facts are added to the KB, all derived inferences are generated
  Þ  
Backward chaining: starts from queries. E.g., the Prolog programming language
father(X, Y) :- parent(X, Y), male(X). parent(john, bill). parent(jane, bill). female(jane). male (john). ?- father(M, bill).
12/8/2018
CPSC503 Spring 2004

27. Linguistically Relevant Concepts in FOL
Categories & Events (Reification)
Representing Time
Beliefs
Aspects
12/8/2018
CPSC503 Spring 2004

28. Categories & Events Categories: Events:
VegetarianRestaurant (Joe’s) - relation vs. object
MostPopular(Joe’s,VegetarianRestaurant)
ISA (Joe’s,VegetarianRestaurant)
AKO (VegetarianRestaurant,Restaurant)
Reification
Events:
Reservation (Hearer,Joe’s,Today,8PM,2)
Problems:
Determining the correct number of roles
Representing facts about the roles associated with an event
Ensuring that all and only the correct inferences can be drawn
Represent all concept we want to make statements about as full-fledged objects
In this way categories are relations instead of objects
12/8/2018
CPSC503 Spring 2004

29. MUC-4 Example
On October 30, 1989, one civilian was killed in a reported FMLN attack in El Salvador.
INCIDENT: DATE 30 OCT 89 INCIDENT: LOCATION EL SALVADOR INCIDENT: TYPE ATTACK INCIDENT: STAGE OF EXECUTION ACCOMPLISHED INCIDENT: INSTRUMENT ID INCIDENT: INSTRUMENT TYPE PERP: INCIDENT CATEGORY TERRORIST ACT PERP: INDIVIDUAL ID "TERRORIST" PERP: ORGANIZATION ID "THE FMLN" PERP: ORG. CONFIDENCE REPORTED: "THE FMLN" PHYS TGT: ID PHYS TGT: TYPE PHYS TGT: NUMBER PHYS TGT: FOREIGN NATION PHYS TGT: EFFECT OF INCIDENT PHYS TGT: TOTAL NUMBER HUM TGT: NAME HUM TGT: DESCRIPTION "1 CIVILIAN" HUM TGT: TYPE CIVILIAN: "1 CIVILIAN" HUM TGT: NUMBER 1: "1 CIVILIAN" HUM TGT: FOREIGN NATION HUM TGT: EFFECT OF INCIDENT DEATH: "1 CIVILIAN" HUM TGT: TOTAL NUMBER
Message Understanding Conference
12/8/2018
CPSC503 Spring 2004

30. Subcategorization frames
I ate
I ate a turkey sandwich
I ate a turkey sandwich at my desk
I ate at my desk
I ate lunch
I ate a turkey sandwich for lunch
I ate a turkey sandwich for lunch at my desk
no fixed “arity”!
12/8/2018
CPSC503 Spring 2004

31. One possible solution
Eating1 (Speaker)
Eating2 (Speaker, TurkeySandwich)
Eating3 (Speaker, TurkeySandwich, Desk)
Eating4 (Speaker, Desk)
Eating5 (Speaker, Lunch)
Eating6 (Speaker, TurkeySandwich, Lunch)
Eating7 (Speaker, TurkeySandwich, Lunch, Desk)
Meaning postulates are used to tie semantics of predicates: " w,x,y,z: Eating7(w,x,y,z) Þ Eating6(w,x,y)
12/8/2018
CPSC503 Spring 2004

32. Reification Again “I ate a turkey sandwich for lunch”
$ w: Isa(w,Eating) Ù Eater(w,Speaker) Ù Eaten(w,TurkeySandwich) Ù MealEaten(w,Lunch)
Reification Advantages:
No need to specify fixed number of arguments for a given surface predicate
No more roles are postulated than mentioned in the input
No need for meaning postulates to specify logical connections among closely related examples
12/8/2018
CPSC503 Spring 2004

33. Representing Time
Events are associated with points or intervals in time.
We can impose an ordering on distinct events using notion of precedes.
Temporal logic notation: ($w,x,t) Arrive(w,x,t)
Constraints on variable t I arrived in New York ($ t) Arrive(I,NewYork,t) Ù precedes(t,Now)
12/8/2018
CPSC503 Spring 2004

34. Interval Events Need tstart and tend
“She was driving to New York until now”
($ tstart,tend)
Drive(She,NewYork, Ù
precedes(tstart,Now) Ù
Equals(tend,Now)
12/8/2018
CPSC503 Spring 2004

35. Relation Between Tenses and Time
Relation between simple verb tenses and points in time is not straightforward
Present tense used like future:
We fly from Baltimore to Boston at 10
Complex tenses:
Flight 1902 arrived late
Flight 1902 had arrived late
12/8/2018
CPSC503 Spring 2004

36. Reference Point
Reichenbach (1947) introduced notion of Reference point (R), separated out from Speech time (S) and Event time (E)
Example:
When Mary's flight departed, I ate lunch
When Mary's flight departed, I had eaten lunch
Departure event specifies reference point.
12/8/2018
CPSC503 Spring 2004

37. Next Time
Read Chp. 15
12/8/2018
CPSC503 Spring 2004