Presentation is loading. Please wait.

Presentation is loading. Please wait.

Convolution Kernels on Constituent, Dependency and Sequential Structures for Relation Extraction Advisor: Hsin-His Chen Reporter: Chi-Hsin Yu Date: 2009.08.16.

Similar presentations


Presentation on theme: "Convolution Kernels on Constituent, Dependency and Sequential Structures for Relation Extraction Advisor: Hsin-His Chen Reporter: Chi-Hsin Yu Date: 2009.08.16."— Presentation transcript:

1 Convolution Kernels on Constituent, Dependency and Sequential Structures for Relation Extraction Advisor: Hsin-His Chen Reporter: Chi-Hsin Yu Date: 2009.08.16 From EMNLP 2009 Truc-Vien T. Nguyen and Alessandro Moschitti and Giuseppe Riccardi Department of Information Engineering and Computer Science University of Trento Italy

2 Outlines Introduction SVM and Kernel Methods Kernels for Relation Extraction Experiments Discussion and Conclusion

3 Introduction Relation extraction ▫Defined in ACE (Automatic Content Extraction)  Recognition of relations  The relation detection and characterization task (RDC)

4 Introduction (Cont.) There are ▫five general types of relations, ▫some of which are further sub-divided, yielding a total of 24 types/subtypes of relations

5 SVM and Kernel methods The (integral) transform of f(t) by the kernel k(v,t) Time domain  frequency domain

6 SVM and Kernel methods(cont.) Nonlinear separating hyperplane ▫Nonlinear mapping ▫Kernel trick Ex: 256 dim.  billion dim.

7 Kernel Matrix Terminologies ▫Gram matrix K (kernel matrix)  K ij = k(x i, x j ) ▫psd: positive semi-definite,  v t Kv >= 0 for all v  Inner product space in a Hilbert space ▫PSD is closed under  Linear combination, polynomial expansion, and normalization (Schlkopf and Smola, 2001) Valid kernel function ▫Gram matrix K is psd iff function k(.,.) is a kernel. ▫Ref: Kernel Methods for Pattern Analysis (Theorem 3.11) +

8 Kernel Methods Kernel methods ▫Used in different algorithms, such as SVM, PCA, LDA, …… ▫Implicitly operate on feature space with much higher dimension ▫Involved only inner product of input samples Feature-based approach: Sample x 1 =(x 11, x 12, …, x 1d ) n samples Kernel methods approach:

9 Kernel Methods (cont.) x 1 =(x 11, x 12, …, x 1d )x 2 =(x 21, x 22, …, x 2d ) SVM (Quadratic Programming) solver Q

10 Kernel Methods in NLP Discrete structure ▫Tree ▫Word sequence ▫Haussler:  Convolution Kernels on Discrete Structures (1999) Feature-based approach: Sample x 1 =(x 11, x 12, …, x 1d ) n samples

11 Kernels for Relation Extraction – Tree Structures Constituent tree and dependency tree PET: path-enclosed tree

12 Kernels for Relation Extraction – Tree Structures (cont.) DW: dependency word tree GR: Grammatical relation tree GRW: Grammatical relation and words

13 Kernels for Relation Extraction – Sequential Structures SK1: ▫T2-LOC Washington, U.S. T1-PER officials SK2 ▫T2-LOC NN, NNP T1-PER NNS SK3 ▫T2-LOC pobj, nn T1-PER nsubj SK4 ▫Washington T2-LOC In working T1-PER officials GPE U.S. SK5 ▫pobj T2-LOC prep ROOT T1-PER nsubj GPE nn SK6 ▫NN T2-LOC IN VBP T1-PER NNS GPE NNP

14 Kernel Matrix Computation A: Tree similarity, K(T i, T j ) ▫ST kernel: Sub-Tree kernel ▫SST kernel: subset tree kernel ▫PT kernel: partial tree kernel B: Sequence similarity, K(S 1, S j ) ▫WSK kernel: word sequence kernel  Cancedda et al. JMLR 2003

15 Kernel Matrix Computation (cont.) Sample ▫R i = ( E 1, E 2 ) ▫R 1 : (Washington, U.S.) ▫R 2 : (Washington, Officials) C: Polynomial kernel Features ▫entity headword, entity type, entity subtype, mention, type, and LDC2 mention type Feature-based approach: Sample x 1 =(x 11, x 12, …, x 1d ) n samples

16 Kernel Matrix Combinations Sequential structures Tree structures +

17 Experiments Setup ▫Corpus  ACE 2004  348 documents, 4400 relation instances  Seven entity types and seven relation types  Physical, Person/Social, Employment/Membership/Subsidiary, Agent-Artifact, PER/ORG Affiliation, GPE Affiliation, and Discourse ▫Generates 38696 negative examples ▫One v.s. all ▫5-fold cross-validation

18 Experiments Results

19 Experiments Results (cont.)

20 Discussion and Conclusion Conclusion ▫Investigating many kernel methods for RE Future work ▫Designing new kernels ▫Capturing other knowledge into kernel, such as ontologies …

21 Thanks!!


Download ppt "Convolution Kernels on Constituent, Dependency and Sequential Structures for Relation Extraction Advisor: Hsin-His Chen Reporter: Chi-Hsin Yu Date: 2009.08.16."

Similar presentations


Ads by Google