Presentation is loading. Please wait.

Presentation is loading. Please wait.

Coarse to Fine Grained Sense Disambiguation in Wikipedia Hui Shen [Ohio University] Razvan Bunescu [Ohio University] Rada Mihalcea [University of North.

Published byCandace Watson Modified over 9 years ago

Similar presentations

Presentation on theme: "Coarse to Fine Grained Sense Disambiguation in Wikipedia Hui Shen [Ohio University] Razvan Bunescu [Ohio University] Rada Mihalcea [University of North."— Presentation transcript:

1 Coarse to Fine Grained Sense Disambiguation in Wikipedia Hui Shen [Ohio University] Razvan Bunescu [Ohio University] Rada Mihalcea [University of North Texas] *SEM, June 13, 2013

2 Question Answering Q: What is the composition of Earth’s atmosphere? 2 As the Earth is 4.5 billion years old, it would have lost its atmosphere by now if there were no protective magnetosphere... The atmosphere is composed of 78% nitrogen and 21% oxygen.

3 KB Question Answering Q: What is the composition of Earth’s atmosphere? 3 As the Earth is 4.5 billion years old, it would have lost its atmosphere by now if there were no protective magnetosphere... The atmosphere is composed of 78% nitrogen and 21% oxygen. Atmosphere of Earth

4 Fine to Coarse Grained Word Sense Disambiguation 4 As the Earth is 4.5 billion years old, it would have lost its atmosphere by now if there were no protective magnetosphere... The atmosphere is composed of 78% nitrogen and 21% oxygen. AtmosphereAtmosphere (unit)Atmosphere (ambiance) sense A. MarsA. EarthA. VenusStellar A. referencespecific sense

5 Word Sense Disambiguation Select the correct sense of a word based on the context: –Use a repository of senses such as WordNet: Static resource, short glosses, too fine grained. –Unsupervised: Similarity between context and sense definition or gloss. –Supervised: Train on text manually tagged with word senses. –Limited amount of manually labeled data. 5 Use Wikipedia for WSD: –Large sense repository, continuously growing. –Large training dataset.

6 Wikipedia for WSD 6 ’’’Palermo’’’ is a city in [[Southern Italy]], the [[capital city | capital]] of the [[autonomous area | autonomous region]] of [[Sicily]]. wiki Palermo is a city in Southern Italy, the capital of the autonomous region of Sicily. html capital city capital (economics) financial capital human capital capital (architecture)

7 WSD Trained on Wikipedia Corpus – atmosphere – 1.Collect all WP titles that are linked from the anchor word atmosphere. => Atmosphere, Atmosphere of Earth, Mood (Pshychology), … 2.Create a sense repository from all titles that have sufficient support in WP. => { Atmosphere, Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere, Atmosphere (unit), Atmosphere (music group) } 3.For each sense, use the links as labeled examples and train a classifier to distinguish between alternative senses of the word atmosphere: –extract features from the word context. –each WP sense acts as a different label in the classification model. 7

8 Major Shortcoming of WP Corpus for WSD Classification algorithms work with disjoint categories. Sense labels collected from WP are not disjoint! –Many instances that are linked to Atmosphere could have been linked to more specific titles: Atmosphere of Earth or Atmosphere of Mars. Atmosphere category is ill defined.  the learning algorithm will underperform, since it tries to separate Atmosphere examples from Atmosphere of Earth examples. 8

9 Sense & Reference Annotations in Wikipedia 9 The Beagle 2 lander objectives were to characterize the physical properties of the atmosphere and surface layers. Atmosphere sense Atmosphere of Mars reference

10 Sense & Reference Annotations in Wikipedia 10 The Orbiter has been successfully performing scientific measurements since early 2004, namely high-resolution imaging and study of the interaction of the atmosphere with the interplanetary medium. Atmosphere sense Atmosphere of Mars reference

11 Sense & Reference Annotations in Wikipedia 11 Assuming the planet’s atmosphere is close to chemical equilibrium, it is predicted that 55 Cancri d is covered in a layer of water clouds. Atmosphere sense Atmosphere of Cancri reference no article yet in WP

12 Sense & Reference Annotations in Wikipedia 12 An aerogravity assist, or AGA, is a spacecraft maneuver designed to change velocity when arriving at a body with an atmosphere. Atmosphere sense generic use

13 Annotation Inconsistencies in Wikipedia: Potential Causes 1.Editors may be unaware that an article exists in Wikipedia for the actual reference of a word, or for a more specific sense of the word: –end up using a link to an article describing the general sense of the word. 2.More specific articles are introduced only in newer versions of Wikipedia: –and thus earlier annotations could not have been aware of these more recent articles. 13

14 Annotation Inconsistencies in Wikipedia: Potential Causes 3.Annotating words with the most specific sense or reference available in WP may require substantial cognitive effort: –editors will then choose to link to a general sense of the word. 14 Trajan was nominated as Consul and brought Apollodorus of Damascus with him to Rome around 91. Rome reference Ancient Rome more specific reference

15 15 An animal sleeps on the couch. A cat sleeps on the couch. A white siameze cat sleeps on the couch.

16 16 An cat sleeps on the piece of furniture. A cat sleeps on the couch. A cat sleeps on the white leather couch.

17 Possible Solutions (1) 1)Group overlapping senses and references into one general sense category. 17 Atmosphere of Earth Atmosphere of Mars Atmosphere of Venus Stellar Atmosphere Atmosphere => { Atmosphere, Atmosphere (unit), Atmosphere (music group) }

18 Possible Solutions (1) Straightforward to implement: –Train multiclass classifier to distinguish the disjoint categories { Atmosphere, Atmosphere (unit), Atmosphere (music group) } Disadvantage is loss of disambiguation resolution: –Resulting WSD system cannot link atmosphere to the more specific senses { Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere } 18

19 Possible Solutions (2) 2)Keep the original sense repository, but change the definition of some sense categories such that all categories in the repository become mutually disjoint. 19 Atmosphere Atm. Earth Atm. Mars Atm. Venus Stelar Atm.

20 Possible Solutions (2) 2)Keep the original sense repository, but change the definition of some sense categories such that all categories in the repository become mutually disjoint. 20 Atm. Earth Atm. Mars Atm. Venus Stelar Atm. Atmosphere (G)

21 Possible Solutions (2) => { Atmosphere (G), Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere, Atmosphere (unit), Atmosphere (music group) } 21 Atm. Earth Atm. Mars Atm. Venus Stelar Atm. Atmosphere (G) Atmosphere (G) = generic uses, or senses/references not in WP.

22 Possible Solutions (2) Advantage is WSD system can make more fine grained annotations: –annotate senses down to reference level i.e. Atmosphere of Earth, Atmosphere of Mars, Stellar Atmosphere. Not as straightforward to train: 22 Atmosphere (G) unknown set of training examplesknown set of training examples

23 Proposed WSD Architecture 23 Atmosphere (unit)Atmosphere (music group)Atmosphere “In global climate models, the properties of the atmosphere are specified at a number of …” Level 1

24 Proposed WSD Architecture 24 Atmosphere (unit)Atmosphere (music group)Atmosphere “In global climate models, the properties of the atmosphere are specified at a number of …” Level 1

25 Proposed WSD Architecture 25 Atmosphere (unit)Atmosphere (music group)Atmosphere “In global climate models, the properties of the atmosphere are specified at a number of …” Atmosphere (G)Atmosphere (S) Level 1 Level 2

26 Proposed WSD Architecture 26 Atmosphere (unit)Atmosphere (music group)Atmosphere “In global climate models, the properties of the atmosphere are specified at a number of …” Atmosphere (G)Atmosphere (S) Level 1 Level 2

27 Proposed WSD Architecture 27 Atmosphere (unit)Atmosphere (music group)Atmosphere “In global climate models, the properties of the atmosphere are specified at a number of …” Atmosphere (G)Atmosphere (S) A. MarsA. EarthA. VenusStellar A. Level 1 Level 2 Level 3

28 Proposed WSD Architecture 28 Atmosphere (unit)Atmosphere (music group)Atmosphere “In global climate models, the properties of the atmosphere are specified at a number of …” Atmosphere (G)Atmosphere (S) A. MarsA. EarthA. VenusStellar A. Level 1 Level 2 Level 3

29 Straightforward Training WSD for L1 & L3: Use multiclass classifiers 29 Atmosphere (unit)Atmosphere (music group)Atmosphere Level 1 Atmosphere, Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere training examples A. MarsA. EarthA. VenusStellar A. Level 3

30 Training Level 2 Classifier 30 Atmosphere (G)Atmosphere (S) Level 2 Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere training examples ? Atmosphere (G) unknown set of training examplesknown set of training examples

31 Naïve SVM 31 Atmosphere (G)Atmosphere (S) Level 2 Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere training examples Atmosphere 60% of the examples linked to Atmosphere should actually belong to Atmosphere (S).  underperforming classifier.

32 Training for Level 2: Semi-supervised  learning with positive and unlabeled examples. 32 Atmosphere (G)Atmosphere (S) Level 2 Atmosphere of Earth, Atmosphere of Mars, Atmosphere of Venus, Stellar Atmosphere positive training examples unlabeled training examples Atmosphere

33 Learning with Positive and Unlabeled Examples 1)Adaptation of the Biased SVM [Lee and Liu, ICML 2003] : 33 decision function f(x) s(x) = +1, if x positive (P) s(x) = −1, if x unlabeled (U)

34 Learning with Positive and Unlabeled Examples 1)Adaptation of the Biased SVM [Lee and Liu, ICML 2003] : 34 control penalties for errors on positive (C P ) vs. unlabeled (C U ) examples how do we set C P and C U ? want C P > C U

35 1)Adaptation of Biased SVM [Lee and Liu, ICML 2003] : –Use development data to tune C P and C U. [Lee and Liu, 03] [this work, 13] Learning with Positive and Unlabeled Examples 35 both estimated un positive and unlabeled examples from dev. data

36 Learning with Positive and Unlabeled Examples 2)Weighted Samples SVM [Elkan and Noto, KDD 2008] : I.Train prob. classifier g(x) to compute p(s  1|x) on P and U. II.Train final decision function f(x) on weighted examples sampled as follows:  sample each example in P with weight 1.  sample each example from U as two examples:  one positive, with weight p(y  +1 | x, s  0).  one negative, with weight p(y  −1 | x, s  0). 36 both estimated based on g(x)

37 Evaluation Datasets 37

38 Evaluation Datasets 38

39 Evaluation Datasets 39

40 Evaluation Datasets 40

41 Experimental Results: Level 2 Accuracy Trained only on WP links, tested on manual annotations. –Averaged over 4-fold cross-validation experiments. 41

42 Experimental Results: Level 2 F-measure Trained only on WP links, tested on manual annotations. –Averaged over 4-fold cross-validation experiments. 42

43 Experimental Results: Overall Accuracy 43 Atmosphere (unit)Atmosphere (music group)Atmosphere... atmosphere... Atmosphere (G)Atmosphere (S) A. MarsA. EarthA. VenusStellar A. Level 1 Level 2 Level 3

44 Experimental Results: Overall Accuracy Leaf nodes as sense repository. 44

45 Future Work Extend learning with positive and unlabeled data to multiclass: Automatically add fine grained links to entire Wikipedia.. 45 Atmosphere (unit)Atmosphere (music group)Atmosphere “In global climate models, the properties of the atmosphere are specified at a number of …” Atmosphere (O)A. MarsA. EarthA. VenusStellar A. Level 1 Level 2

46 Questions ? 46

Download ppt "Coarse to Fine Grained Sense Disambiguation in Wikipedia Hui Shen [Ohio University] Razvan Bunescu [Ohio University] Rada Mihalcea [University of North."

Similar presentations

January 23 rd, 2011 1. Document classification task We are interested to solve a task of Text Classification, i.e. to automatically assign a given document.

January 23 rd, Document classification task We are interested to solve a task of Text Classification, i.e. to automatically assign a given document.
Classification Classification Examples

Classification Classification Examples
Machine learning continued Image source: https://www.coursera.org/course/mlhttps://www.coursera.org/course/ml.

Machine learning continued Image source:
Text Categorization Karl Rees Ling 580 April 2, 2001.

Text Categorization Karl Rees Ling 580 April 2, 2001.
Automatic Identification of Cognates, False Friends, and Partial Cognates University of Ottawa, Canada University of Ottawa, Canada.

Automatic Identification of Cognates, False Friends, and Partial Cognates University of Ottawa, Canada University of Ottawa, Canada.
Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)

Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)
GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.

GENERATING AUTOMATIC SEMANTIC ANNOTATIONS FOR RESEARCH DATASETS AYUSH SINGHAL AND JAIDEEP SRIVASTAVA CS DEPT., UNIVERSITY OF MINNESOTA, MN, USA.
Evaluation.

Evaluation.
CS 47051 Word Sense Disambiguation. 2 Overview A problem for semantic attachment approaches: what happens when a given lexeme has multiple ‘meanings’?

CS Word Sense Disambiguation. 2 Overview A problem for semantic attachment approaches: what happens when a given lexeme has multiple ‘meanings’?
CS347 Review Slides (IR Part II) June 6, 2001 ©Prabhakar Raghavan.

CS347 Review Slides (IR Part II) June 6, 2001 ©Prabhakar Raghavan.
1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.

1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.
Collective Word Sense Disambiguation David Vickrey Ben Taskar Daphne Koller.

Collective Word Sense Disambiguation David Vickrey Ben Taskar Daphne Koller.
Sentiment Lexicon Creation from Lexical Resources BIS 2011 Bas Heerschop Erasmus School of Economics Erasmus University Rotterdam

Sentiment Lexicon Creation from Lexical Resources BIS 2011 Bas Heerschop Erasmus School of Economics Erasmus University Rotterdam
1 Noun Homograph Disambiguation Using Local Context in Large Text Corpora Marti A. Hearst Presented by: Heng Ji Mar. 29, 2004.

1 Noun Homograph Disambiguation Using Local Context in Large Text Corpora Marti A. Hearst Presented by: Heng Ji Mar. 29, 2004.
Taking the Kitchen Sink Seriously: An Ensemble Approach to Word Sense Disambiguation from Christopher Manning et al.

Taking the Kitchen Sink Seriously: An Ensemble Approach to Word Sense Disambiguation from Christopher Manning et al.
KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.

KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.
1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions,

1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions,
Introduction to Machine Learning Approach Lecture 5.

Introduction to Machine Learning Approach Lecture 5.
Ensemble Learning (2), Tree and Forest

Ensemble Learning (2), Tree and Forest
Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.

Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.

Similar presentations

Ads by Google