Lecture 21 Computational Lexical Semantics CSCE 771 Natural Language Processing Lecture 21 Computational Lexical Semantics Topics Features in NLTK III Computational Lexical Semantics Semantic Web USC Readings: NLTK book Chapter 10 Text Chapters 20 April 3, 2013

Overview Readings: Text 19,20 NLTK Book: Chapter 10 Last Time (Programming) Features in NLTK NL queries  SQL NLTK support for Interpretations and Models Propositional and predicate logic support Prover9 Today Last Lectures slides 25-29 Computational Lexical Semantics Readings: Text 19,20 NLTK Book: Chapter 10 Next Time: Computational Lexical Semantics II

Model Building in NLTK - Chapter 10continued Mace model builder lp = nltk.LogicParser() # install Mace4 config_mace4('c:\Python26\Lib\site-packages\prover9') a3 = lp.parse('exists x.(man(x) & walks(x))') c1 = lp.parse('mortal(socrates)') c2 = lp.parse('-mortal(socrates)') mb = nltk.Mace(5) print mb.build_model(None, [a3, c1]) True print mb.build_model(None, [a3, c2]) print mb.build_model(None, [c1, c2]) False

>>> a4 = lp. parse('exists y. (woman(y) & all x >>> a4 = lp.parse('exists y. (woman(y) & all x. (man(x) -> love(x,y)))') >>> a5 = lp.parse('man(adam)') >>> a6 = lp.parse('woman(eve)') >>> g = lp.parse('love(adam,eve)') >>> mc = nltk.MaceCommand(g, assumptions=[a4, a5, a6]) >>> mc.build_model() True

10.4 The Semantics of English Sentences Principle of compositionality --

Representing the λ-Calculus in NLTK (33) a. (walk(x) ∧ chew_gum(x)) b. λx.(walk(x) ∧ chew_gum(x)) c. \x.(walk(x) & chew_gum(x)) -- the NLTK way!

Lambda0.py import nltk from nltk import load_parser lp = nltk.LogicParser() e = lp.parse(r'\x.(walk(x) & chew_gum(x))') print e \x.(walk(x) & chew_gum(x)) e.free() print lp.parse(r'\x.(walk(x) & chew_gum(y))') \x.(walk(x) & chew_gum(y))

Simple β-reductions >>> e = lp.parse(r'\x.(walk(x) & chew_gum(x))(gerald)') >>> print e \x.(walk(x) & chew_gum(x))(gerald) >>> print e.simplify() [1] (walk(gerald) & chew_gum(gerald))

Predicate reductions >>> e3 = lp.parse('\P.exists x.P(x)(\y.see(y, x))') >>> print e3 (\P.exists x.P(x))(\y.see(y,x)) >>> print e3.simplify() exists z1.see(z1,x)

Figure 19.7 Inheritance of Properties Exists e,x,y Eating(e) ^ Agent(e, x) ^ Theme(e, y) “hamburger edible?” from wordnet Speech and Language Processing, Second Edition Daniel Jurafsky and James H. Martin Copyright ©2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved.

Figure 20.1 Possible sense tags for bass Chapter 20 – Word Sense disambiguation (WSD) Machine translation Supervised vs unsupervised learning Semantic concordance – corpus with words tagged with sense tags

Feature Extraction for WSD Feature vectors Collocation [wi-2, POSi-2, wi-1, POSi-1, wi, POSi, wi+1, POSi+1, wi+2, POSi+2] Bag-of-words – unordered set of neighboring words Represent sets of most frequent content words with membership vector [0,0,1,0,0,0,1] – set of 3rd and 7th most freq. content word Window of nearby words/features

Naïve Bayes Classifier w – word vector s – sense tag vector f – feature vector [wi, POSi ] for i=1, …n Approximate by frequency counts But how practical?

Looking for Practical formula . Still not practical

Naïve == Assume Independence Now practical, but realistic?

Training = count frequencies . Maximum likelihood estimator (20.8)

Decision List Classifiers Naïve Bayes hard for humans to examine decisions and understand Decision list classifiers - like “case” statement sequence of (test, returned-sense-tag) pairs

Figure 20.2 Decision List Classifier Rules

WSD Evaluation, baselines, ceilings Extrinsic evaluation - evaluating embedded NLP in end-to-end applications (in vivo) Intrinsic evaluation – WSD evaluating by itself (in vitro) Sense accuracy Corpora – SemCor, SENSEVAL, SEMEVAL Baseline - Most frequent sense (wordnet sense 1) Ceiling – Gold standard – human experts with discussion and agreement

Figure 20.3 Simplified Lesk Algorithm gloss/sentence overlap

Simplified Lesk example The bank can guarantee deposits will eventually cover future tuition costs because it invests in adjustable rate mortgage securities.

SENSEVAL competitions http://www.senseval.org/ Check the Senseval-3 website.

Corpus Lesk weights applied to overlap words inverse document frequency idfi = log (Ndocs / num docs containing wi)

20.4.2 Selectional Restrictions and Preferences  

Wordnet Semantic classes of Objects

Minimally Supervised WSD: Bootstrapping Yarowsky algorithm Heuritics: one sense per collocations one sense per discourse

Figure 20.4 Two senses of plant

Figure 20.5

Figure 20.6 Path Based Similarity

Figure 20.6 Path Based Similarity . \ simpath(c1, c2)= 1/pathlen(c1, c2) (length + 1)

Information Content word similarity  

Figure 20.7 Wordnet with P(c) values

Figure 20.8

Figure 20.9

Figure 20.10

Figure 20.11

Figure 20.12

Figure 20.13

Figure 20.14

Figure 20.15

Figure 20.16