Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July.

Published byEdwin Mutch Modified over 10 years ago

Similar presentations

Presentation on theme: "A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July."— Presentation transcript:

1 A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July 12, 2010 Tohoku University Nara Institute of Science and Technology

2 Page  2 Predicate-Argument Structure Analysis (Semantic Role Labeling)  Task of analyzing predicates and its arguments –A predicate represents a state or an event, and its arguments have relations to the predicate –Each of arguments has a particular semantic role (Agent, Theme, etc)  In recent years, predicate sense disambiguation has been included in predicate-argument structure analysis [Surdeanu+ 08, Hajič+ 09] –‘sell.01’ means that ‘sold’ is an instance of the first sense of ‘sell’  Important for many NLP applications –MT, QA, RTE, etc. Theme Location Temporal luxuryautomakerlastTheyearsold1,214carsintheU.S. maker.01 sell.01 Product Agent

3 Page  3 drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Two Types of Dependencies of Elements in Predicate- Argument Structures (1)Inter-dependencies between a predicate and its arguments –A1: car => we can infer that the correct sense is drive.01 (2)Non-local dependencies among arguments Two or more arguments do not have the same role Basically, obligatory roles of the predicate should appear in sentences drive.01 A0 A1 SBJ NMOD OBJ Pauldrovehiscar In order to realize robust predicate-argument structure analysis, it is necessary to deal with these types of dependencies

4 Page  4 Previous Work (1)Non-local dependencies among arguments: Re-ranking [Johansson and Nugues 2008, etc.] Generate N-best assignments of argument roles, then obtain global features for each assignment, finally select the argmax using the re-ranker Can not explicitly capture inter-dependencies between a predicate and its arguments (2)Inter-dependencies between a predicate and its arguments: Markov Logic Networks [Meza-Ruiz and Riedel 2009, etc.] Jointly learn and classify pred. senses and arg. roles simultaneously MLN can not deal with particular types of global features Currently, no existing (discriminative) approach sufficiently handles both types of dependencies

5 Page  5 Previous Work (1)Non-local dependencies among arguments: Re-ranking [Johansson and Nugues 2008, etc.] Generate N-best assignments of argument roles, then obtain global features for each assignment, finally select the argmax using the re-ranker Can not explicitly capture inter-dependencies between a predicate and its arguments (2)Inter-dependencies between a predicate and its arguments: Markov Logic Networks [Meza-Ruiz and Riedel 2009, etc.] Jointly learn and classify pred. senses and arg. roles simultaneously MLN can not deal with particular types of global features Currently, no existing (discriminative) approach sufficiently handles both types of dependencies We propose a structured model that can capture both types of dependencies simultaneously

6 Page  6 The proposed model SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion

7 Page  7 The proposed model A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Expand the possible labels of predicate senses and argument roles

8 Page  8 The proposed model A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Expand the possible labels of predicate senses and argument roles We use four types of factors which score labels of elements in predicate- argument structures

9 Page  9 The proposed model A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE SBJ NMOD OBJ Pauldrovehiscar drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion Expand the possible labels of predicate senses and argument roles These factors are defined by (linear model) These factors are defined by (linear model) We use four types of factors which score labels of elements in predicate- argument structures

10 Page  10 A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE The proposed model 1.4754 0.7268 SBJ NMOD OBJ Pauldrovehiscar FPFP drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion use a factor which scores sense labels of the predicate

11 Page  11 A0 drive.01 drive.02 … A1 A0 … Paulcar drove NONE The proposed model 1.784 0.238 -1.665-1.235 0.876 -1.482 SBJ NMOD OBJ Pauldrovehiscar FAFA FPFP drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion use a factor which scores role labels of each argument

12 Page  12 A0 … A1 A0 … Paulcar drove NONE The proposed model 0.764 0.261 SBJ NMOD OBJ Pauldrovehiscar F PA FAFA FPFP drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01 drive.02 add a factor which scores label pairs of a predicate sense and a semantic role of an argument

13 Page  13 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0,drive01,A1 … A0,drive01,A1 … 1.865 A0 drive.01 A1 SBJ NMOD OBJ Pauldrovehiscar FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion add a factor which captures plausibility of the whole predicate-argument structure (use global features) add a factor which captures plausibility of the whole predicate-argument structure (use global features)

14 Page  14 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0,drive01,A1 … A0,drive01,A1 … 1.865 A0 drive.01 A1 SBJ NMOD OBJ Pauldrovehiscar FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion add a factor which captures plausibility of the whole predicate-argument structure (use global features) add a factor which captures plausibility of the whole predicate-argument structure (use global features) The predicate ‘drive’ has all obligatory roles A0 and A1 => F G assigns the higher score to the weight corresponds to this feature The predicate ‘drive’ has all obligatory roles A0 and A1 => F G assigns the higher score to the weight corresponds to this feature

15 Page  15 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0 drive.01 A1 NONE 1.784 0.238 -1.665-1.235 0.876 -1.482 1.4754 0.7268 0.764 0.425 SBJ NMOD OBJ Pauldrovehiscar A0,drive01,A1 … A0,drive01,A1 … 1.865 FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion The proposed model combines these types of factors

16 Page  16 The proposed model drive.02 … A1 A0 … Paulcar drove NONE A0 drive.01 A1 NONE drive.01 A0 A1 1.784 0.238 -1.665-1.235 0.876 -1.482 1.4754 0.7268 0.764 0.425 SBJ NMOD OBJ Pauldrovehiscar A0,drive01,A1 … A0,drive01,A1 … 1.865 FPFP F PA FAFA FGFG drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion drive.01: drive a vehicle A0: driver A1: vehicle drive.02: cause to move A0: driver A1: things in motion The proposed model combines these types of factors The highest scoring assignment is returned by the proposed model

17 Page  17 Dealing with global (non-local) features  Introduce the fundamental idea of [Kazama and Torisawa 2007] –Features are divided into local features and global features –Inference: N-best based approach (1) Generate N-best assignments using only local features (2) Obtain global features in the N-best assignments (3) Select the argmax –Learning: train parameters with two margin constraints All: train parameters so as to ensure a sufficient margin using all features (both local features and global features) Local only: when the constraint All is satisfied, train parameters so as to ensure a sufficient margin using only local features K&T proposed a Margin-Perceptron Learning Algorithm

18 Page  18 Inference and Learning Algorithm of the Proposed Model Inference: generate N-best assignments for each predicate sense Learning: the online Passive-Aggressive Algorithm [Crammer 2006] The parameters are trained by solving the optimization problem used in PA with the two margin constraints: All (local + global) and Local only Inference: generate N-best assignments for each predicate sense Learning: the online Passive-Aggressive Algorithm [Crammer 2006] The parameters are trained by solving the optimization problem used in PA with the two margin constraints: All (local + global) and Local only (1) All (local + global) margin (2) Local only margin positive other positive other

19 Page  19 Results on the CoNLL-2009 ST Dataset (average) feature selection Overall (Sem. F1) WSD (Acc.) SRL (Lab. F1) F P +F A no79.1789.6572.20 F P +F A +F PA no79.5889.7872.74 F P +F A +F G no80.4289.8374.11 ALLno80.7590.1574.46 Björkelundyes80.80 Zhaoyes80.47 Meza-Ruizno77.46 sense FPFP F PA FGFG FAFA … role 1 role 2 role N  The best performance is obtained by using the all factors  Our model achieved the competitive results with the top system in the CoNLL-2009 Shared Task without any feature selection procedure

20 Page  20 Results on the CoNLL-2009 ST Dataset (average) feature selection Overall (Sem. F1) WSD (Acc.) SRL (Lab. F1) F P +F A no79.1789.6572.20 F P +F A +F PA no79.5889.7872.74 F P +F A +F G no80.4289.8374.11 ALLno80.7590.1574.46 Björkelundyes80.80 Zhaoyes80.47 Meza-Ruizno77.46 sense FPFP F PA FGFG FAFA … role 1 role 2 role N  By adding two types of factors F PA and F G, we obtained performance improvements in both tasks (predicate sense disambiguation and argument role labeling) => Succeeded in joint learning

21 Page  21 Summary  We proposed a structured model that can capture two types of dependencies (1)Non-local dependencies among arguments (2)Inter-dependencies between a predicate and its arguments  The proposed model achieved the competitive results with the state-of- the-art SRL systems without any feature selection procedure  By adding two types of factors, we obtained performance improvements on both predicate sense disambiguation and argument role labeling => succeeded in joint learning  Future Work –exploiting unlabeled data (unsupervised or semi-supervised predicate-argument structure analysis)

Download ppt "A Structured Model for Joint Learning of Argument Roles and Predicate Senses Yotaro Watanabe Masayuki Asahara Yuji Matsumoto ACL 2010 Uppsala, Sweden July."

Similar presentations

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Latent Variables Naman Agarwal Michael Nute May 1, 2013.

Latent Variables Naman Agarwal Michael Nute May 1, 2013.
January 23 rd, 2011 1. Document classification task We are interested to solve a task of Text Classification, i.e. to automatically assign a given document.

January 23 rd, Document classification task We are interested to solve a task of Text Classification, i.e. to automatically assign a given document.
Face Alignment by Explicit Shape Regression

Face Alignment by Explicit Shape Regression
Document Summarization using Conditional Random Fields Dou Shen, Jian-Tao Sun, Hua Li, Qiang Yang, Zheng Chen IJCAI 2007 Hao-Chin Chang Department of Computer.

Document Summarization using Conditional Random Fields Dou Shen, Jian-Tao Sun, Hua Li, Qiang Yang, Zheng Chen IJCAI 2007 Hao-Chin Chang Department of Computer.
Exploring the Effectiveness of Lexical Ontologies for Modeling Temporal Relations with Markov Logic Eun Y. Ha, Alok Baikadi, Carlyle Licata, Bradford Mott,

Exploring the Effectiveness of Lexical Ontologies for Modeling Temporal Relations with Markov Logic Eun Y. Ha, Alok Baikadi, Carlyle Licata, Bradford Mott,
Date: 2014/05/06 Author: Michael Schuhmacher, Simon Paolo Ponzetto Source: WSDM’14 Advisor: Jia-ling Koh Speaker: Chen-Yu Huang Knowledge-based Graph Document.

Date: 2014/05/06 Author: Michael Schuhmacher, Simon Paolo Ponzetto Source: WSDM’14 Advisor: Jia-ling Koh Speaker: Chen-Yu Huang Knowledge-based Graph Document.
Perceptron Learning Rule

Perceptron Learning Rule
Online Max-Margin Weight Learning for Markov Logic Networks Tuyen N. Huynh and Raymond J. Mooney Machine Learning Group Department of Computer Science.

Online Max-Margin Weight Learning for Markov Logic Networks Tuyen N. Huynh and Raymond J. Mooney Machine Learning Group Department of Computer Science.
Coreference Based Event-Argument Relation Extraction on Biomedical Text Katsumasa Yoshikawa 1), Sebastian Riedel 2), Tsutomu Hirao 3), Masayuki Asahara.

Coreference Based Event-Argument Relation Extraction on Biomedical Text Katsumasa Yoshikawa 1), Sebastian Riedel 2), Tsutomu Hirao 3), Masayuki Asahara.
Active Learning for Streaming Networked Data Zhilin Yang, Jie Tang, Yutao Zhang Computer Science Department, Tsinghua University.

Active Learning for Streaming Networked Data Zhilin Yang, Jie Tang, Yutao Zhang Computer Science Department, Tsinghua University.
CONSTRAINED CONDITIONAL MODELS TUTORIAL Jingyu Chen, Xiao Cheng.

CONSTRAINED CONDITIONAL MODELS TUTORIAL Jingyu Chen, Xiao Cheng.
Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute.

Jointly Identifying Temporal Relations with Markov Logic Katsumasa Yoshikawa †, Sebastian Riedel ‡, Masayuki Asahara †, Yuji Matsumoto † † Nara Institute.
Page 1 SRL via Generalized Inference Vasin Punyakanok, Dan Roth, Wen-tau Yih, Dav Zimak, Yuancheng Tu Department of Computer Science University of Illinois.

Page 1 SRL via Generalized Inference Vasin Punyakanok, Dan Roth, Wen-tau Yih, Dav Zimak, Yuancheng Tu Department of Computer Science University of Illinois.
Machine learning continued Image source: https://www.coursera.org/course/mlhttps://www.coursera.org/course/ml.

Machine learning continued Image source:
Max-Margin Matching for Semantic Role Labeling David Vickrey James Connor Daphne Koller Stanford University.

Max-Margin Matching for Semantic Role Labeling David Vickrey James Connor Daphne Koller Stanford University.
A Joint Model For Semantic Role Labeling Aria Haghighi, Kristina Toutanova, Christopher D. Manning Computer Science Department Stanford University.

A Joint Model For Semantic Role Labeling Aria Haghighi, Kristina Toutanova, Christopher D. Manning Computer Science Department Stanford University.
Confidence-Weighted Linear Classification Mark Dredze, Koby Crammer University of Pennsylvania Fernando Pereira Penn  Google.

Confidence-Weighted Linear Classification Mark Dredze, Koby Crammer University of Pennsylvania Fernando Pereira Penn  Google.
Supervised classification performance (prediction) assessment Dr. Huiru Zheng Dr. Franscisco Azuaje School of Computing and Mathematics Faculty of Engineering.

Supervised classification performance (prediction) assessment Dr. Huiru Zheng Dr. Franscisco Azuaje School of Computing and Mathematics Faculty of Engineering.
Page 1 Generalized Inference with Multiple Semantic Role Labeling Systems Peter Koomen, Vasin Punyakanok, Dan Roth, (Scott) Wen-tau Yih Department of Computer.

Page 1 Generalized Inference with Multiple Semantic Role Labeling Systems Peter Koomen, Vasin Punyakanok, Dan Roth, (Scott) Wen-tau Yih Department of Computer.

Similar presentations

Ads by Google