Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Systematic Exploration of the Feature Space for Relation Extraction Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois,

Published byFay Cameron Modified over 9 years ago

Similar presentations

Presentation on theme: "A Systematic Exploration of the Feature Space for Relation Extraction Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois,"— Presentation transcript:

1 A Systematic Exploration of the Feature Space for Relation Extraction Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois, Urbana-Champaign

2 Apr 23, 2007NAACL-HLT2 What Is Relation Extraction? …hundreds of Palestinians converged on the square… person (entity type) bounded-area (entity type) relation ?

3 Apr 23, 2007NAACL-HLT3 What Is Relation Extraction? …hundreds of Palestinians converged on the square… person (entity type) bounded-area (entity type) located (relation type)

4 Apr 23, 2007NAACL-HLT4 Existing Methods Rule-based [Califf & Mooney 98] Generative-model-based [Miller et al. 00] Discriminative-model-based –Feature-based [Zhou et al. 05] –Kernel-based [Bunescu & Mooney 05b] [Zhang et al. 06]

5 Apr 23, 2007NAACL-HLT5 Feature-Based Methods Entity info –arg 1 is a Person entity & arg 2 is a Bounded-Area entity POS tagging –there is a preposition between arg 1 and arg 2 Syntactic parsing –arg 2 is inside a prepositional phrase following arg 1 Dependency parsing –arg 2 is dependent on a preposition, which in turn is dependent on a verb …hundreds of Palestinians arg1 converged on the square arg2 … located Other features?

6 Apr 23, 2007NAACL-HLT6 Kernel-Based Methods Define a kernel function to measure the similarity between two relation instances Convolution kernels –Defined on sequence or tree representation of relation instances –Corresponding to a feature space, where features are sub-structures such as sub- sequences and sub-trees

7 Apr 23, 2007NAACL-HLT7 Convolution Tree Kernel (sub-tree features) NNS hundreds IN of NNP Palestinians VBD converged IN on DT the NN square NPB PP NP S VP PP NPB

8 Apr 23, 2007NAACL-HLT8 Convolution Tree Kernel (sub-tree features) NNS hundreds IN of NNP Palestinians VBD converged IN on DT the NN square NPB PP NP S VP PP NPB

9 Apr 23, 2007NAACL-HLT9 Convolution Tree Kernel (sub-tree features) NNS hundreds IN of NNP Palestinians VBD converged IN on DT the NN square NPB PP NP S VP PP NPB

10 Apr 23, 2007NAACL-HLT10 Convolution Tree Kernel (sub-tree features) NNS hundreds IN of NNP Palestinians VBD converged IN on DT the NN square NPB PP NP S VP PP NPB

11 Apr 23, 2007NAACL-HLT11 Convolution Tree Kernel (sub-tree features) NNS hundreds IN of NNP Palestinians VBD converged IN on DT the NN square NPB PP NP S VP PP NPB

12 Apr 23, 2007NAACL-HLT12 Convolution Tree Kernel (sub-tree features) NNS hundreds IN of NNP Palestinians VBD converged IN on DT the NN square NPB PP NP S VP PP NPB NOT included by original definition! Useful? Yes Choices of features are also critical in kernel methods!

13 Apr 23, 2007NAACL-HLT13 Is it possible to define the complete set of potentially useful features ?

14 Apr 23, 2007NAACL-HLT14 Outline of Our Work Defined a graphic representation of relation instances Presented a general definition of features Proposed a bottom-up search strategy to explore the feature space Evaluated different types of features

15 Apr 23, 2007NAACL-HLT15 A Graphic Representation of Relation Instances hundredsofPalestiniansconvergedonthesquare Each node can have multiple labels –Word, POS tag, entity type, etc.

16 Apr 23, 2007NAACL-HLT16 A Graphic Representation of Relation Instances NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area Each node can have multiple labels –Word, POS tag, entity type, etc. Each node has an argument tag set to 0, 1, 2, or 3

17 Apr 23, 2007NAACL-HLT17 A Graphic Representation of Relation Instances NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010002 Each node can have multiple labels –Word, POS tag, entity type, etc. Each node has an argument tag set to 0, 1, 2, or 3

18 Apr 23, 2007NAACL-HLT18 Graphic Representation Based on Syntactic Parse Trees NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010002 NPB PP NP 1 1 0 1 S VP PP NPB 3 2 2 2

19 Apr 23, 2007NAACL-HLT19 Graphic Representation Based on Dependency Parse Trees NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010002

20 Apr 23, 2007NAACL-HLT20 A General Definition of Features NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010002 Sub-graphs

21 Apr 23, 2007NAACL-HLT21 A General Definition of Features NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 001000 2 Sub-graph Subset of the original label set

22 Apr 23, 2007NAACL-HLT22 A General Definition of Features NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 001000 2 Sub-graph Subset of the original label set

23 Apr 23, 2007NAACL-HLT23 A General Definition of Features NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 001000 2 Sub-graph Subset of the original label set Unigram Feature

24 Apr 23, 2007NAACL-HLT24 NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 00 10 002 A General Definition of Features

25 Apr 23, 2007NAACL-HLT25 NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 00 10 002 A General Definition of Features Bigram Feature

26 Apr 23, 2007NAACL-HLT26 More Examples NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010002 NPB PP NP 1 1 0 1 S VP PP NPB 3 2 2 2

27 Apr 23, 2007NAACL-HLT27 More Examples NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010 0 02 NPB PP NP 1 1 0 1 S VP PP NPB 3 2 2 2 Production Feature

28 Apr 23, 2007NAACL-HLT28 More Examples NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010 002 NPB PP NP 1 1 0 1 S VP PP NPB 3 2 2 2

29 Apr 23, 2007NAACL-HLT29 More Examples NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010 0 0 2 NPB PP NP 1 1 0 1 S VP PP NPB 3 2 2 2

30 Apr 23, 2007NAACL-HLT30 More Examples NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 00 1 000 2 NPB PP NP 1 1 0 1 S VP PP NPB 3 2 2 2

31 Apr 23, 2007NAACL-HLT31 More Examples NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0010002 NPB PP NP 1 1 0 1 S VP PP NPB 3 2 2 2

32 Apr 23, 2007NAACL-HLT32 Coverage of the Feature Definition NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 001000 2 Entity attributes [Zhao & Grishman 05] [Zhou et al. 05] –Unigram features with entity attributes

33 Apr 23, 2007NAACL-HLT33 Coverage of the Feature Definition Bag-of-word features [Zhao & Grishman 05] [Zhou et al. 05] –Unigram features with words NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 001000 2

34 Apr 23, 2007NAACL-HLT34 Coverage of the Feature Definition Bigram features [Zhao & Grishman 05] –Bigram features with words NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 00 10 002

35 Apr 23, 2007NAACL-HLT35 Coverage of the Feature Definition Grammar production features [Zhang et al. 06] –Production features Dependency relation and dependency path features [Bunescu & Mooney 05a] [Zhao and Grishman 05] [Zhou et al. 05] –Bigram and n-gram features with words NNS hundreds IN of NNP Palestinians Person VBD converged IN on DT the NN square Bounded-Area 0 0 1 0002

36 Apr 23, 2007NAACL-HLT36 Exploring the Feature Space We consider three feature subspaces: –Sequence, syntactic parse tree, dependency parse tree A bottom-up strategy –Start with unigram features, and gradually increase the size/complexity of the features –First search in each subspace, then merge features from different subspaces

37 Apr 23, 2007NAACL-HLT37 Empirical Evaluation Data set: –ACE (Automatic Content Extraction) 2004 –7 types of relations Preprocessing –Assume entities are correctly identified –Brill Tagger –Collins Parser Learning algorithms –Maximum entropy models –SVM

38 Apr 23, 2007NAACL-HLT38 Evaluation A commonly used setup: –Consider all pairs of entities in each single sentence –Multi-class classification: # relation types + 1 (no relation between the two entities) –5-fold cross validation –Precision (P), Recall (R) and F1 (F)

39 Apr 23, 2007NAACL-HLT39 Increase Feature Complexity

40 Apr 23, 2007NAACL-HLT40 Combine Features from Different Subspaces SynSyn + SeqSyn + DepAll MEP0.7260.7370.6950.724 R0.6880.6940.7310.702 F0.6830.7150.7120.713 SVMP0.6790.6890.6870.691 R0.6810.6860.6820.686 F0.6800.6880.6840.688

41 Apr 23, 2007NAACL-HLT41 Heuristics to Prune Features H1: in Syn, to remove words before and after the arguments H2: in Seq, to remove features that contain articles, adjectives and adverbs H3: in Syn, to remove features that contain articles, adjectives and adverbs H4: in Seq, to remove words before and after the arguments

42 Apr 23, 2007NAACL-HLT42 Effects of Heuristics

43 Apr 23, 2007NAACL-HLT43 Conclusions A general graphic view of feature space Evaluated 3 subspaces (seq, syn, dep) Findings –Combination of unigrams, bigrams, and trigrams works the best –Combination of complementary feature subspaces (seq + syn) is beneficial –Additional heuristics can be used to further improve the performance

44 Apr 23, 2007NAACL-HLT44 Future Work Best feature configuration and relation types Principled ways to prune or to weight features –Feature selection (information gain, chi square, etc.) –Inclusion of more complex features –Feature weighting

45 Apr 23, 2007NAACL-HLT45 References [Bunescu & Mooney 05a] A shortest path dependency kernel for relation extraction. In Proceedings of HLT/EMNLP, 2005. [Bunescu & Mooney 05b] Subsequence kernels for relation extraction. In NIPS, 2005. [Califf & Mooney 98] Relational learning of pattern-match rules for information extraction. In Proceedings of AAAI Spring Symposium on Applying Machine Learning to Discourse Processing, 1998. [Miller et al. 00] A novel use of statistical parsing to extract information from text. In Proceedings of NAACL, 2000. [Zhang et al. 06] Exploring syntactic features for relation extraction using a convolution tree kernel. In Proceedings of HLT/NAACL, 2006. [Zhao & Grishman 05] Extracting relations with integrated information using kenrel methods. In Proceedings of ACL, 2005. [Zhou et al. 05] Exploring various knowledge in relation extraction. In Proceedings of ACL, 2005.

46 Thanks!

Download ppt "A Systematic Exploration of the Feature Space for Relation Extraction Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois,"

Similar presentations

School of something FACULTY OF OTHER School of Computing FACULTY OF ENGINEERING Chunking: Shallow Parsing Eric Atwell, Language Research Group.

School of something FACULTY OF OTHER School of Computing FACULTY OF ENGINEERING Chunking: Shallow Parsing Eric Atwell, Language Research Group.
Progress update Lin Ziheng. System overview 2 Components – Connective classifier Features from Pitler and Nenkova (2009): – Connective: because – Self.

Progress update Lin Ziheng. System overview 2 Components – Connective classifier Features from Pitler and Nenkova (2009): – Connective: because – Self.
Proceedings of the Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2007) Learning for Semantic Parsing Advisor: Hsin-His.

Proceedings of the Conference on Intelligent Text Processing and Computational Linguistics (CICLing-2007) Learning for Semantic Parsing Advisor: Hsin-His.
Sequence Classification: Chunking Shallow Processing Techniques for NLP Ling570 November 28, 2011.

Sequence Classification: Chunking Shallow Processing Techniques for NLP Ling570 November 28, 2011.
Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley.

Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley.
May 2006CLINT-LN Parsing1 Computational Linguistics Introduction Approaches to Parsing.

May 2006CLINT-LN Parsing1 Computational Linguistics Introduction Approaches to Parsing.
1 Relational Learning of Pattern-Match Rules for Information Extraction Presentation by Tim Chartrand of A paper bypaper Mary Elaine Califf and Raymond.

1 Relational Learning of Pattern-Match Rules for Information Extraction Presentation by Tim Chartrand of A paper bypaper Mary Elaine Califf and Raymond.
计算机科学与技术学院 Chinese Semantic Role Labeling with Dependency-driven Constituent Parse Tree Structure Hongling Wang, Bukang Wang Guodong Zhou NLP Lab, School.

计算机科学与技术学院 Chinese Semantic Role Labeling with Dependency-driven Constituent Parse Tree Structure Hongling Wang, Bukang Wang Guodong Zhou NLP Lab, School.
Statistical NLP: Lecture 3

Statistical NLP: Lecture 3
Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009.

Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009.
Part-of-speech tagging. Parts of Speech Perhaps starting with Aristotle in the West (384–322 BCE) the idea of having parts of speech lexical categories,

Part-of-speech tagging. Parts of Speech Perhaps starting with Aristotle in the West (384–322 BCE) the idea of having parts of speech lexical categories,
Semantic Role Labeling Abdul-Lateef Yussiff 20-01-2011.

Semantic Role Labeling Abdul-Lateef Yussiff
Learning with Probabilistic Features for Improved Pipeline Models Razvan C. Bunescu Electrical Engineering and Computer Science Ohio University Athens,

Learning with Probabilistic Features for Improved Pipeline Models Razvan C. Bunescu Electrical Engineering and Computer Science Ohio University Athens,
Made with OpenOffice.org 1 Sentiment Classification using Word Sub-Sequences and Dependency Sub-Trees Pacific-Asia Knowledge Discovery and Data Mining.

Made with OpenOffice.org 1 Sentiment Classification using Word Sub-Sequences and Dependency Sub-Trees Pacific-Asia Knowledge Discovery and Data Mining.
Recognizing Implicit Discourse Relations in the Penn Discourse Treebank Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng Department of Computer Science National.

Recognizing Implicit Discourse Relations in the Penn Discourse Treebank Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng Department of Computer Science National.
Part II. Statistical NLP Advanced Artificial Intelligence Part of Speech Tagging Wolfram Burgard, Luc De Raedt, Bernhard Nebel, Lars Schmidt-Thieme Most.

Part II. Statistical NLP Advanced Artificial Intelligence Part of Speech Tagging Wolfram Burgard, Luc De Raedt, Bernhard Nebel, Lars Schmidt-Thieme Most.
1 CSC 594 Topics in AI – Applied Natural Language Processing Fall 2009/2010 8. Shallow Parsing.

1 CSC 594 Topics in AI – Applied Natural Language Processing Fall 2009/ Shallow Parsing.
1 Combining Lexical and Syntactic Features for Supervised Word Sense Disambiguation Saif Mohammad Ted Pedersen University of Toronto University of Minnesota.

1 Combining Lexical and Syntactic Features for Supervised Word Sense Disambiguation Saif Mohammad Ted Pedersen University of Toronto University of Minnesota.
Reporter: Longhua Qian School of Computer Science and Technology

Reporter: Longhua Qian School of Computer Science and Technology
Page 1 Generalized Inference with Multiple Semantic Role Labeling Systems Peter Koomen, Vasin Punyakanok, Dan Roth, (Scott) Wen-tau Yih Department of Computer.

Page 1 Generalized Inference with Multiple Semantic Role Labeling Systems Peter Koomen, Vasin Punyakanok, Dan Roth, (Scott) Wen-tau Yih Department of Computer.

Similar presentations

Ads by Google