Lexical Ambiguity Resolution / Sense Disambiguation Supervised methods Non-supervised methods Class-based models Seed models Vector models EM Iteration Unsupervised clustering Sense induction Anaphosa Resolution -- CS466 Lecture XVIII --

For sense disambiguation, ** Ambiguous verbs (e.g., to fire) depend heavily on words in local context (in particular, their objects). ** Ambiguous nouns (e.g., plant) depend on wider context. For example, seeing [ greenhouse, nursery, cultivation ] within a window of +/- 10 words is very indicative of sense. -- CS466 Lecture XVI --

Deficiency of “Bag-of-words” Approach context is treated as an unordered bag of words -> like vector model (and also previous neural network models etc.) -- CS466 Lecture XVI --

Words tend to exhibit only one sense in a given collocation or Observations Words tend to exhibit only one sense in a given collocation or word association 2 word Collocations (word to left or word to the right) Prob(container) Prob(vehicle) .99 + .01 - .96 + .04 - oxygen Tank Panzer Empty P (Person) P (Place) .01 .99 .95 .05 .02 .98 .96 .04 In Madison With Dr. Madison Airport Madison mayor Mayor Formally P (sense | collocation) is a low entropy distribution

Collocation Means (originally): - “in the same location” - “co-occurring” in some defined relationship Adjacent (bigram allocations) Verb/Object collocations Co-occurrence within +/- k words collocations Fire her Fire the long rifles Made of lead, iron, silver, … Other Interpretation: An idiomatic (non-compositional high frequency association) Eg. Soap opera, Hong Kong -- CS466 Lecture XVI --

Order and Sequence Matter: plant pesticide  living plant pesticide plant  manufacturing plant a solid lead  advantage or head start a solid wall of lead  metal a hotel in Madison  place I saw Madison in a hotel bar  person -- CS466 Lecture XVI --

Adjacent words more salient than those 20 words away Observation Distance matters Adjacent words more salient than those 20 words away All positions give same weight -- CS466 Lecture XVI --

Words tend to exhibit only one sense in a given discourse or document Observations Words tend to exhibit only one sense in a given discourse or document = word form Very unlikely to have living Plants / manufacturing plants referenced in the same document (tendency to use synonym like factory to minimize ambiguity) communicative efficiency (Grice) Unlikely to have Mr. Madison and Madison City in the same document Unlikely to have Turkey (both country and bird) in the same document -- CS466 Lecture XVI --

Vector Models for Word Sense KEY Sense 1 Centroid Sense 1 Sense 2 Sense 2 Centroid -- CS466 Lecture XVI --

S1 – S2 for all terms in sum vec[sum][term] != 0 Plant 1 2 3 4 5 6 * S1 Xi Sum += V[i] For each vector Sim (1, i) For each term in vecs[docn] Sum[term] += vec[docn] S2 Sim (2,i) S1 > S2 assign sense 1 else sense 2 Sum 1 2 3 4 5 6 * S1 – S2 for all terms in sum vec[sum][term] != 0 -- CS466 Lecture XVI --

Vector Models for Person / Place KEY PERSON CENTROID PERSON PLACE PLACE CENTROID -- CS466 Lecture XVI --

Vector Models for Lexical Ambiguity Resolution / Lexical Classification Treat labeled contexts as vectors Class W-3 W-2 W-1 W0 W1 W2 W3 PLACE long way from Madison to Chicago COMPANY When Madison investors issued a Convert to a traditional vector just like a short query V328 V329 -- CS466 Lecture XVI --

Training Space (Vector Model) Per Pl Pl Pl Per Pl Per Per Pl Per Pl Per Per Pl Person Centroid Place Centroid new example Eve Co Company Centroid Co Eve Co Co Eve Co Co Event Centroid Co -- CS466 Lecture XVI --

Problem with supervised methods Tagged training data is expensive (time, resources) Solution: Class discriminators can serve as effective wordsense discriminators And are much less costly to train if we can tolerate some noise in the models -- CS466 Lecture XVIII --

Pseudo-Class Discriminators What if class lists (like Rogets) are not available? Create small classes optimized for the target ambiguity Class (Crane 1) = heron, stork, eagle, condor, … Class (Crane 2) = derrick, forklift, bulldozers, … Class (Tank 1) = Jeep, Vehicle, Humvee, Bradley, Abrams, … Class (Tank 2) = Vessel, container, flask, pool Include synonyms, hype-nyms, hyponyms, topically related Smaller and potentially more specific but less robust (parent in tree) (child in tree) -- CS466 Lecture XVIII --

Goal: Iterative Refinement Sorting State: Output of (poor) class models Context that match reliable Small sets of hand tagged data Seed Words: Reliable collocations of target words need not be synonyms. Plant 1 Plant 2 Plant life Plant species Microscopic plant Living plant Animal within +/- 10 words Employee within +/- 10 words Assembly plant Plant closure -- CS466 Lecture XIX --