CPSC 503 Computational Linguistics Representing Meaning Lecture 15 Giuseppe Carenini 12/8/2018 CPSC503 Spring 2004

Knowledge-Formalisms Map (including probabilistic formalisms) State Machines (and prob. versions) (Finite State Automata,Finite State Transducers, Markov Models) Morphology Syntax Rule systems (and prob. versions) (e.g., (Prob.) Context-Free Grammars) Semantics Pragmatics Discourse and Dialogue Logical formalisms (First-Order Logics) AI planners 12/8/2018 CPSC503 Spring 2004

Next four classes What meaning is and how to represent it How to map sentences into their meaning Meaning of individual words Tasks: Information Extraction Information Retrieval How the mening of a sentence depends on the emaning of its constituents phrases and words compositional semantics 12/8/2018 CPSC503 Spring 2004

Today 12/3 Semantics / Meaning Meaning Representations First-Order Logics Linguistically relevant Concepts 12/8/2018 CPSC503 Spring 2004

Semantics Def. Semantics: The study of the meaning of words, intermediate constituents and sentences Def1. Meaning: a representation that expresses the linguistic input in terms of objects, actions, events, time, space… beliefs, attitudes...relationships Def2. Meaning: a representation that links the linguistic input to knowledge of the world How the meaning of a phrase is related to the meaning of its parts What sorts of meaning structures and meaning relations obtain in natural language Semantic relations among words Link linguistic inputs to a non-linguistic representation of the world First-Order Logics Symbol structures that corresponds to objects and relations among objects in some world being represented …Representation for a particular state of affairs in some world Language independent! 12/8/2018 CPSC503 Spring 2004

Semantic Relations involving Sentences Same truth conditions Paraphrase: have the same meaning I gave the apple to John vs. I gave John the apple I bought a car from you vs. you sold a car to me The thief was chased by the police vs. …… Entailment: “implication” The park rangers killed the bear vs. The bear is dead Nemo is a fish vs. Nemo is an animal If one is true the other must be true Some linguists feel that languages do not permit two or more structures to have exactly identical meanings…. Contradiction: I am in Vancouver vs. I am in California 12/8/2018 CPSC503 Spring 2004

Words from Nonlexical categories Grammaticization Concept Affix -ed -s re- in-, un-, de- Past More than one Again Negation Words from Nonlexical categories Obligation Possibility Definite, Specific Indefinite, Non-specific Disjunction Negation Conjunction must may the a or not and Of the indefinitely large set of concepts expressible in human language, a relatively small set enjoys a special status concepts that are lexicalized as affixes and nonlexical (functional / closed-class) categories in most of the world’s languages Two kinds of category Closed class (generally are function words) Prepositions, articles, conjunctions, pronouns, determiners, aux, numerals Open class Nouns (proper/common; mass/count), verbs, adjectives, adverbs If you run across an unknown word….?? 12/8/2018 CPSC503 Spring 2004

Common Meaning Representations I have a car FOL Semantic Nets They all share a common foundation: Meaning Representation consists of structures composed of sets of symbols Symbol structures are objects and relations among objects We’re going to take the same basic approach to meaning that we took to syntax and morphology We’re going to create representations of linguistic inputs that capture the meanings of those inputs. But unlike parse trees and the like these representations aren’t primarily descriptions of the structure of the inputs… Conceptual Dependency Frames 12/8/2018 CPSC503 Spring 2004

Requirements for Meaning Representations Sample NLP Task: giving advice about restaurants Accept queries in NL Generate appropriate responses by consulting a KB e.g, Does Maharani serve vegetarian food? -> Yes What restaurants are close to the ocean? -> C and Monks 12/8/2018 CPSC503 Spring 2004

Verifiability (in the world?) Example: Does LeDog serve vegetarian food? Knowledge base (KB) expressing our world model Convert question to KB language and verify its truth value against the KB content It must be possible to use the representation to verify whether the sentence is true in the world (with respect to our representation of the world) Sample entry in KB: Serves(LeDog,Vegetarian Food) System can match input representation against representations in knowledge base. If it finds a match, it can return Yes; Otherwise No. 12/8/2018 CPSC503 Spring 2004

Gozzilla interpretation Unambiguousness Gozzilla interpretation Example: I want to eat some place near campus. Final representations should be unambiguous Vagueness: I want to eat Spanish food. Single linguistic input can have different meaning representations Each representation unambiguously characterizes one meaning Answer will depends on which interpretation is chosen System should allow us to represent vagueness 12/8/2018 CPSC503 Spring 2004

Canonical form Paraphrases should be mapped into the same representation. Does LeDog have vegetarian dishes? Do they have vegetarian food at LeDog? Are vegetarian dishes served at LeDog? Does LeDog serve vegetarian fare? …………… Alternatives use different words widely varying syntactic analysis they should not be mapped in substantially different meanings Computational purpose that meaning representation serve... in particular in this domain we want to give the same answer! What about to have/serve a tennis ball word sense disambiguation Having vs. serving Food vs. fare vs. dishes 12/8/2018 CPSC503 Spring 2004

How to Produce a Canonical Form Systematic Meaning Representations can be derived from thesaurus food ___ dish ___|____one overlapping meaning sense fare ___| We can systematically relate syntactic constructions [S [NP Maharani] serves [NP vegetarian dishes]] [S [NP vegetarian dishes] are served at [NP Maharani]] 12/8/2018 CPSC503 Spring 2004

Inference and Expressiveness Consider a more complex request Can vegetarians eat at Maharani? Vs: Does Maharani serve vegetarian food? Why do these result in the same answer? Inference: System’s ability to draw valid conclusions based on the meaning representations of inputs and its KB serve(Maharani,VegetarianFood) => CanEat(Vegetarians,At(Maharani)) Not because they mean the same thingQ Expressiveness: system must be able to handle a wide range of subject matter 12/8/2018 CPSC503 Spring 2004

Non Yes/No Questions Example: I'd like to find a restaurant where I can get vegetarian food. Indefinite reference <-> variable serve(x,VegetarianFood) Matching succeeds only if variable x can be replaced by known object in KB. 12/8/2018 CPSC503 Spring 2004

Meaning Structure of Language How does language convey meaning? Grammaticization Tense systems Conjunctions Quantifiers …… Display a partially compositional semantics Display a basic predicate-argument structure 12/8/2018 CPSC503 Spring 2004

Predicate-Argument Structure Represent relationships among concepts Some words act like arguments and some words act like predicates: Nouns as concepts or arguments: red(ball) Adj, Adv, Verbs as predicates: red(ball) Subcategorization frames specify number, position, and syntactic category of arguments Examples: give NP2 NP1, find NP, sneeze [] All human languages a specific relation holds between the concepts expressed by the words or the phrases Events, actions and relationships can be captured with representations that consist of predicates and arguments. Languages display a division of labor where some words and constituents function as predicates and some as arguments. One of the most important roles of the grammar is to help organize this pred-args structure 12/8/2018 CPSC503 Spring 2004

Semantic (Thematic) Roles This can be extended to the realm of semantics Semantic Roles: Participants in an event Agent: George hit Bill. Bill was hit by George Theme: George hit Bill. Bill was hit by George Source, Goal, Instrument, Force… Verb subcategorization: Allows linking arguments in surface structure with their semantic roles Predicates Primarily Verbs, VPs, PPs, Sentences Sometimes Nouns and NPs Arguments Primarily Nouns, Nominals, NPs But also everything else; as we’ll see it depends on the context Mary gave/sent/read a book to Ming Agent Theme Goal Mary gave/sent/read Ming a book Agent Goal Theme 12/8/2018 CPSC503 Spring 2004

Selectional Restrictions Semantic (Selectional) Restrictions: Constrain the types of arguments verbs take George assassinated the senator *The spider assassinated the fly Predicates Primarily Verbs, VPs, PPs, Sentences Sometimes Nouns and NPs Arguments Primarily Nouns, Nominals, NPs But also everything else; as we’ll see it depends on the context 12/8/2018 CPSC503 Spring 2004

First Order Predicate Calculus (FOPC) FOPC provides sound computational basis for verifiability, inference, expressiveness… Supports determination of truth Supports Canonical Form Supports compositionality of meaning Supports question-answering (via variables) Supports inference Allows for… The analysis of truth conditions Allows us to answer yes/no questions Supports the use of variables Allows us to answer questions through the use of variable binding Supports inference Allows us to answer questions that go beyond what we know explicitly FOPC reflects the semantics of natural languages because it was designed that way by human beings 12/8/2018 CPSC503 Spring 2004

FOPC Syntax Give(sister-of(John), Mary, x) AtomicFormula  Predicate (Term, …) Term  Function (Term,…) | Constant | Variable Constant  B | VegetarianFood | LeDog Variable  x | y | … Predicate  Serves | Near | … Function  LocationOf | CuisineOf | … Give(sister-of(John), Mary, x) Formula  AtomicFormula | Formula Connective Formula | Quantifier Variable, … Formula | Ø Formula | (Formula) Connective  |  | Ù | Þ Quantifier  " | $ Context free grammar specification of FOL examine this bottom up fashion terms represent objects functions often expressed in English as genitives statements about unknown objects Terms Constants: Maharani Functions: LocationOf(Maharani) Variables: x in LocationOf(x) Predicates: Relations that hold among objects Serves(Maharani,VegetarianFood) Logical Connectives: Permit compositionality of meaning I only have $5 and I don’t have a lot of time Have(I,$5) Ù Have(I,LotofTime) 12/8/2018 CPSC503 Spring 2004

FOPC Semantics Formulas in FOPC can be assigned truth values True or False Database semantics for atomic formulas LeDog is near campus. Near(LocationOf(LeDog),LocationOf(campus)) Truth tables for connectives 12/8/2018 CPSC503 Spring 2004 No common-sense

Variables and Quantifiers Existential ($): There exists A restaurant that serves Mexican food near UBC ($x) Restaurant(x) Ù Serves(x,MexicalFood) Ù Near(LocationOf(x),LocationOf(UMD)) Universal ("): For all All vegetarian restaurants serve vegetarian food ("x) VegetarianRestaurant(x) Þ Serves(x,VegetarianFood) 12/8/2018 CPSC503 Spring 2004

Connectives I only have five dollars and I don’t have a lot of time. Have(Speaker,FiveDollars) Ù Ø Have(Speaker,LotOfTime) 12/8/2018 CPSC503 Spring 2004

Inference Modus ponens:   Þ   Example: VegetarianRestaurant(Joe’s)  x: VegetarianRestaurant(x) Þ Serves(x,VegetarianFood) Serves(Joe’s,VegetarianFood) One of the most important desiderata for a meaning representation is that it should support inference 12/8/2018 CPSC503 Spring 2004

Uses of modus ponens Forward chaining: as individual facts are added to the KB, all derived inferences are generated   Þ   Backward chaining: starts from queries. E.g., the Prolog programming language father(X, Y) :- parent(X, Y), male(X). parent(john, bill). parent(jane, bill). female(jane). male (john). ?- father(M, bill). 12/8/2018 CPSC503 Spring 2004

Linguistically Relevant Concepts in FOL Categories & Events (Reification) Representing Time Beliefs Aspects 12/8/2018 CPSC503 Spring 2004

Categories & Events Categories: Events: VegetarianRestaurant (Joe’s) - relation vs. object MostPopular(Joe’s,VegetarianRestaurant) ISA (Joe’s,VegetarianRestaurant) AKO (VegetarianRestaurant,Restaurant) Reification Events: Reservation (Hearer,Joe’s,Today,8PM,2) Problems: Determining the correct number of roles Representing facts about the roles associated with an event Ensuring that all and only the correct inferences can be drawn Represent all concept we want to make statements about as full-fledged objects In this way categories are relations instead of objects 12/8/2018 CPSC503 Spring 2004

MUC-4 Example On October 30, 1989, one civilian was killed in a reported FMLN attack in El Salvador. INCIDENT: DATE 30 OCT 89 INCIDENT: LOCATION EL SALVADOR INCIDENT: TYPE ATTACK INCIDENT: STAGE OF EXECUTION ACCOMPLISHED INCIDENT: INSTRUMENT ID INCIDENT: INSTRUMENT TYPE PERP: INCIDENT CATEGORY TERRORIST ACT PERP: INDIVIDUAL ID "TERRORIST" PERP: ORGANIZATION ID "THE FMLN" PERP: ORG. CONFIDENCE REPORTED: "THE FMLN" PHYS TGT: ID PHYS TGT: TYPE PHYS TGT: NUMBER PHYS TGT: FOREIGN NATION PHYS TGT: EFFECT OF INCIDENT PHYS TGT: TOTAL NUMBER HUM TGT: NAME HUM TGT: DESCRIPTION "1 CIVILIAN" HUM TGT: TYPE CIVILIAN: "1 CIVILIAN" HUM TGT: NUMBER 1: "1 CIVILIAN" HUM TGT: FOREIGN NATION HUM TGT: EFFECT OF INCIDENT DEATH: "1 CIVILIAN" HUM TGT: TOTAL NUMBER Message Understanding Conference 12/8/2018 CPSC503 Spring 2004

Subcategorization frames I ate I ate a turkey sandwich I ate a turkey sandwich at my desk I ate at my desk I ate lunch I ate a turkey sandwich for lunch I ate a turkey sandwich for lunch at my desk no fixed “arity”! 12/8/2018 CPSC503 Spring 2004

One possible solution Eating1 (Speaker) Eating2 (Speaker, TurkeySandwich) Eating3 (Speaker, TurkeySandwich, Desk) Eating4 (Speaker, Desk) Eating5 (Speaker, Lunch) Eating6 (Speaker, TurkeySandwich, Lunch) Eating7 (Speaker, TurkeySandwich, Lunch, Desk) Meaning postulates are used to tie semantics of predicates: " w,x,y,z: Eating7(w,x,y,z) Þ Eating6(w,x,y) 12/8/2018 CPSC503 Spring 2004

Reification Again “I ate a turkey sandwich for lunch” $ w: Isa(w,Eating) Ù Eater(w,Speaker) Ù Eaten(w,TurkeySandwich) Ù MealEaten(w,Lunch) Reification Advantages: No need to specify fixed number of arguments for a given surface predicate No more roles are postulated than mentioned in the input No need for meaning postulates to specify logical connections among closely related examples 12/8/2018 CPSC503 Spring 2004

Representing Time Events are associated with points or intervals in time. We can impose an ordering on distinct events using notion of precedes. Temporal logic notation: ($w,x,t) Arrive(w,x,t) Constraints on variable t I arrived in New York ($ t) Arrive(I,NewYork,t) Ù precedes(t,Now) 12/8/2018 CPSC503 Spring 2004

Interval Events Need tstart and tend “She was driving to New York until now” ($ tstart,tend) Drive(She,NewYork, Ù precedes(tstart,Now) Ù Equals(tend,Now) 12/8/2018 CPSC503 Spring 2004

Relation Between Tenses and Time Relation between simple verb tenses and points in time is not straightforward Present tense used like future: We fly from Baltimore to Boston at 10 Complex tenses: Flight 1902 arrived late Flight 1902 had arrived late 12/8/2018 CPSC503 Spring 2004

Reference Point Reichenbach (1947) introduced notion of Reference point (R), separated out from Speech time (S) and Event time (E) Example: When Mary's flight departed, I ate lunch When Mary's flight departed, I had eaten lunch Departure event specifies reference point. 12/8/2018 CPSC503 Spring 2004

Next Time Read Chp. 15 12/8/2018 CPSC503 Spring 2004