1. Lecture 21 Computational Lexical Semantics
CSCE Natural Language Processing
Lecture 21 Computational Lexical Semantics
Topics
Features in NLTK III
Computational Lexical Semantics
Semantic Web USC
Readings:
NLTK book Chapter 10
Text Chapters 20
April 3, 2013

2. Overview Readings: Text 19,20 NLTK Book: Chapter 10
Last Time (Programming)
Features in NLTK
NL queries  SQL
NLTK support for Interpretations and Models
Propositional and predicate logic support
Prover9
Today
Last Lectures slides 25-29
Computational Lexical Semantics
Readings:
Text 19,20
NLTK Book: Chapter 10
Next Time: Computational Lexical Semantics II

3. Model Building in NLTK - Chapter 10continued
Mace model builder
lp = nltk.LogicParser()
# install Mace4
config_mace4('c:\Python26\Lib\site-packages\prover9')
a3 = lp.parse('exists x.(man(x) & walks(x))')
c1 = lp.parse('mortal(socrates)')
c2 = lp.parse('-mortal(socrates)')
mb = nltk.Mace(5)
print mb.build_model(None, [a3, c1])
True
print mb.build_model(None, [a3, c2])
print mb.build_model(None, [c1, c2])
False

4. >>> a4 = lp. parse('exists y. (woman(y) & all x
>>> a4 = lp.parse('exists y. (woman(y) & all x. (man(x) -> love(x,y)))') >>> a5 = lp.parse('man(adam)') >>> a6 = lp.parse('woman(eve)') >>> g = lp.parse('love(adam,eve)') >>> mc = nltk.MaceCommand(g, assumptions=[a4, a5, a6]) >>> mc.build_model() True

5. 10.4 The Semantics of English Sentences
Principle of compositionality --

6. Representing the λ-Calculus in NLTK
(33) a. (walk(x) ∧ chew_gum(x)) b. λx.(walk(x) ∧ chew_gum(x)) c. \x.(walk(x) & chew_gum(x)) -- the NLTK way!

7. Lambda0.py
import nltk from nltk import load_parser lp = nltk.LogicParser() e = lp.parse(r'\x.(walk(x) & chew_gum(x))') print e \x.(walk(x) & chew_gum(x)) e.free() print lp.parse(r'\x.(walk(x) & chew_gum(y))') \x.(walk(x) & chew_gum(y))

8. Simple β-reductions
>>> e = lp.parse(r'\x.(walk(x) & chew_gum(x))(gerald)') >>> print e \x.(walk(x) & chew_gum(x))(gerald) >>> print e.simplify() [1] (walk(gerald) & chew_gum(gerald))

9. Predicate reductions
>>> e3 = lp.parse('\P.exists x.P(x)(\y.see(y, x))') >>> print e3 (\P.exists x.P(x))(\y.see(y,x)) >>> print e3.simplify() exists z1.see(z1,x)

10. Figure 19.7 Inheritance of Properties
Exists e,x,y Eating(e) ^ Agent(e, x) ^ Theme(e, y) “hamburger edible?” from wordnet
Speech and Language Processing, Second Edition
Daniel Jurafsky and James H. Martin
Copyright ©2009 by Pearson Education, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved.

11. Figure 20.1 Possible sense tags for bass
Chapter 20 – Word Sense disambiguation (WSD)
Machine translation
Supervised vs unsupervised learning
Semantic concordance – corpus with words tagged with sense tags

12. Feature Extraction for WSD
Feature vectors Collocation [wi-2, POSi-2, wi-1, POSi-1, wi, POSi, wi+1, POSi+1, wi+2, POSi+2] Bag-of-words – unordered set of neighboring words Represent sets of most frequent content words with membership vector [0,0,1,0,0,0,1] – set of 3rd and 7th most freq. content word Window of nearby words/features

13. Naïve Bayes Classifier
w – word vector s – sense tag vector f – feature vector [wi, POSi ] for i=1, …n Approximate by frequency counts But how practical?

14. Looking for Practical formula
. Still not practical

15. Naïve == Assume Independence
Now practical, but realistic?

16. Training = count frequencies
. Maximum likelihood estimator (20.8)

17. Decision List Classifiers
Naïve Bayes hard for humans to examine decisions and understand Decision list classifiers - like “case” statement sequence of (test, returned-sense-tag) pairs

18. Figure 20.2 Decision List Classifier Rules

19. WSD Evaluation, baselines, ceilings
Extrinsic evaluation - evaluating embedded NLP in end-to-end applications (in vivo) Intrinsic evaluation – WSD evaluating by itself (in vitro) Sense accuracy Corpora – SemCor, SENSEVAL, SEMEVAL Baseline - Most frequent sense (wordnet sense 1) Ceiling – Gold standard – human experts with discussion and agreement

20. Figure 20.3 Simplified Lesk Algorithm
gloss/sentence overlap

21. Simplified Lesk example
The bank can guarantee deposits will eventually cover future tuition costs because it invests in adjustable rate mortgage securities.

22. SENSEVAL competitions
Check the Senseval-3 website.

23. Corpus Lesk
weights applied to overlap words inverse document frequency idfi = log (Ndocs / num docs containing wi)

24. 20.4.2 Selectional Restrictions and Preferences

25. Wordnet Semantic classes of Objects

26. Minimally Supervised WSD: Bootstrapping
Yarowsky algorithm
Heuritics:
one sense per collocations
one sense per discourse

27. Figure 20.4 Two senses of plant

28. Figure 20.5

29. Figure 20.6 Path Based Similarity

30. Figure 20.6 Path Based Similarity
. \ simpath(c1, c2)= 1/pathlen(c1, c2) (length + 1)

31. Information Content word similarity

32. Figure 20.7 Wordnet with P(c) values

33. Figure 20.8

35. Figure 20.9

36. Figure 20.10

37. Figure 20.11

38. Figure 20.12

39. Figure 20.13

40. Figure 20.14

41. Figure 20.15

42. Figure 20.16