Using Semantic Relatedness for Word Sense Disambiguation Siddharth Patwardhan patw0006@d.umn.edu 10/24/2002

The Lesk1 Algorithm Two Hypotheses: The intended sense of the target word in a given context is semantically related to other word senses in the context. Semantically related words have greater number of overlaps of their dictionary definitions. 1[Lesk 1986]

The rate of interest at this bank is high. An Example The rate of interest at this bank is high. interest (involvement) interest (interestingness) interest (sake) interest (charge for loan) interest (pastime) interest (stake) bank (financial institution) bank (river) bank (stock) bank (building) bank (arrangement) bank (container) rate (charge per unit) rate (change with time) rate (pace) rate: amount of a charge or payment relative to some basis; "a 10-minute phone call at that rate would cost $5“ interest: a fixed charge for borrowing money; usually a percentage of the amount borrowed; "how much interest do you pay on your mortgage?“ bank: a financial institution that accepts deposits and channels the money into lending activities; "he cashed a check at the bank"; "that bank holds the mortgage on my home"

Adapting Lesk to WordNet2 Banerjee and Pedersen [2001] adapt the Lesk algorithm to use the rich source of knowledge in WordNet. hypernym: gloss hypernym: gloss hypernym: gloss bank: gloss interest: gloss rate: gloss hyponym: gloss hyponym: gloss hyponym: gloss 2 [Fellbaum 1998]

Semantic Relatedness by counting edges. Rada, et al [1989] introduce a notion of relatedness between words by counting the number of edges between the them in a “broader-than” hierarchy (MeSH: a hierarchy of medical terms). Leacock and Chodorow [1998] use a similar approach to measure semantic relatedness between concepts by finding the length of the shortest path between the two concepts in the is-a hierarchy of WordNet. They scale this value by the maximum depth of the taxonomy and get a formula for relatedness: relatedness = -log(pathLength/(2·maxDepth))

Information Content Introduced by Resnik [1995]. Indicates the specificity or generality of a concept. More specific concepts have higher information content, while more general concepts have less information content. For example concepts like dime, clinker and hayfork are rather specific or topical, would be localized in a discourse and would greatly restrict the choice of concepts that can be used around them (in the context). Computed from large (ideally sense-tagged) corpora. IC(concept) = -log(Probability of occurrence of the concept in a large corpus)

Information Content *Root* *Root* +1 +1 Motor vehicle Motor vehicle +1 car car +1 +1 cab cab +1 minicab minicab +1

Measures of Semantic Relatedness based on a concept hierarchy LOWEST COMMON SUBSUMER CONCEPT c2 CONCEPT c1

Measures of Semantic Relatedness Resnik[1995] relatedness = IC(lcs) Jiang Conrath[1997] distance = 2 x IC(lcs) – (IC(c1) + IC(c2)) Lin[1998] relatedness = 2 x IC(lcs) IC(c1) + IC(c2)

Measures of Semantic Relatedness The Hirst-St.Onge[1998] measure c2 Extra Strong Relation – between two occurrences of the same word. (2) Strong Relation – three rules. Synonyms Horizontal Relation Compound – Word Relation (3) Medium Strong Relation – if there exists an allowable path between the two concepts. Is a kind of (UPWARD) Opposite (HORIZONTAL) c4 c1 Has part (DOWNWARD) c3

Word Sense Disambiguation using Measures of Semantic Relatedness CONTEXT Word1 Target Word2 W11 W12 T1 T2 W21 W22 SENSES LOCAL APPROACH W11 W12 W21 W22 W11 W12 W21 W22 S11 S21 T1 S12 T2 S22 S13 S23 S24 S14 Score(T1) = S11 + S12 + S13 + S14 Score(T2) = S21 + S22 + S23 + S24

Word Sense Disambiguation using Measures of Semantic Relatedness GLOBAL APPROACH a b W11 T1 W21 S1 = a + b + c S5 W11 T2 W21 c T2 W12 T1 W22 S6 W12 W22 S2 W11 T1 W22 W11 T2 W22 S3 S7 W12 T1 W21 W12 T2 S8 W21 S4 Combination with the highest score is selected.

Some Results The experiments were performed on the noun instances of the SENSEVAL-2 data (1723 instances). A context window size of 3 with the local scoring approach was considered for the experiments. Information content was calculated using the SemCor semantically tagged corpus and from the Brown corpus. SemCor Brown Resnik 0.295 0.290 Jiang-Conrath 0.330 0.331 Lin 0.328 0.363 Leacock-Chodorow 0.305 Hirst-St.Onge 0.316 Adapted Lesk 0.391

References [Lesk 1986] M. Lesk. Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone In Proceedings of International Conference on Machine Learning, Madison, Wisconsin, August 1998. [Budanitsky Hirst 2001] A. Budanitsky and G. Hirst. Semantic Distance in WordNet: An experimental application-oriented evaluation of five measures. In Workshop on WordNet and Other Lexical Resources, Second meeting of the North American Chapter of the Association for Computational Linguistics, Pittsburgh, June 2001. [Banerjee Pedersen 2002] S. Banerjee and T. Pedersen. An adapted Lesk algorithm for word sense disambiguation using WordNet. In Proceedings of the Third International Conference on Intelligent Text Processing and Computational Linguistics, Mexico City, Feb 2002. [Resnik 1995] P. Resnik. Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, August 1995.

References [Jiang Conrath 1997] J. Jiang and D. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings on International Conference on Research in Computational Linguistics, Taiwan, 1997. [Lin 1998] D. Lin. An information-theoretic definition of similarity. In Proceedings of International Conference on Machine Learning, Madison, Wisconsin, August 1998. [Leacock Chodorow 1998] C. Leacock and M. Chodorow. Combining local context and WordNet similarity for word sense identification. In Fellbaum, pp. 265 – 283, 1998. [Hirst St-Onge 1998] G. Hirst and D. St-Onge. Lexical chain as representations of context for the detection and correction of malapropisms. In Fellbaum, pp. 305 – 332, 1998. [Fellbaum 1998] C. Fellbaum, editor. WordNet: An electronic lexical database. MIT Press, 1998. [Rada et al, 1989] R. Rada, H. Mili, E. Bicknell and M. Blettner. Development and application of a metric on semantic nets. IEEE Transactions on Systems, Man and Cybernetics, 19(1):17-30, February 1989.