Presentation is loading. Please wait.

Presentation is loading. Please wait.

Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009.

Published byNorma Jennings Modified over 9 years ago

Similar presentations

Presentation on theme: "Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009."— Presentation transcript:

1 Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009

2 Aug 5, 2009ACL-IJCNLP 20092 Relation Extraction Task definition: to label the semantic relation between a pair of entities in a sentence (fragment) …[leader arg-1 ] of a minority [government arg-2 ]… PHYSPER-SOCEMP-ORGNIL PHYS: Physical PER-SOC: Personal / Social EMP-ORG: Employment / Membership / Subsidiary

3 Aug 5, 2009ACL-IJCNLP 20093 Supervised Learning Current solution: supervised machine learning (e.g. [Zhou et al. 2005], [Bunescu & Mooney 2005], [Zhang et al. 2006]) Training data is needed for each relation type …[leader arg-1 ] of a minority [government arg-2 ]… arg-1 word: leaderarg-2 type: ORG dependency: arg-1  of  arg-2 EMP-ORGPHYSPER-SOCNIL

4 Aug 5, 2009ACL-IJCNLP 20094 Challenge in Practice New relation type (in a new domain): no training data or a few seed instances In this work, we study weakly-supervised relation extraction –A few seed instances of the target relation type –Many instances of other auxiliary relation types –Additional human knowledge about the target relation type Main idea: Auxiliary relation types can help!

5 Aug 5, 2009ACL-IJCNLP 20095 Syntactic Similarity across Relation Types …[leader arg-1 ] of a minority [government arg-2 ]… arg-1 word: leaderarg-2 type: ORG dependency: arg-1  of  arg-2 the youngest [son arg-1 ] of ex-director [Suharto arg-2 ] the [Socialist People’s Party arg-1 ] of [Montenegro arg-2 ] EMP-ORG PER-SOC GPE-AFF

6 Aug 5, 2009ACL-IJCNLP 20096 Syntactic Similarity Syntactic Pattern Relation InstanceRelation Type (Subtype) arg-2 arg-1Arab leadersOTHER-AFF (Ethnic) his fatherPER-SOC (Family) South Jakarta Prosecution Office GPE-AFF (Based-in) arg-1 [verb] arg-2Yemen [sent] planes to Baghdad ART (User-or- Owner) His wife [had] three young children PER-SOC (Family) Jody Scheckter [paced] Farrari to both victories EMP-ORG (Employ- Staff)

7 Aug 5, 2009ACL-IJCNLP 20097 Problem Formulation based on Transfer Learning Domain adaptation and transfer learning (e.g. [Blitzer et al. 2006], [Hal Daume III 2007]) our goal: PER-SOCEMP-ORG We apply our previous framework ([Jiang & Zhai 2007b]) –Similar in spirit to [Evgeniou & Pontil 2004] and [Daume III, 2007]

8 Aug 5, 2009ACL-IJCNLP 20098 Review of Relation Extraction Basics Linear classifier …[leader arg-1 ] of a minority [government arg-2 ]… 10..1..10..1.. arg-2 type: ORG arg-2 type: PER dependency: arg-1  of  arg-2 4.5 0.3. 6.7. arg-2 type: ORG feature vectorweight vector in linear classifier dependency: arg-1  of  arg-2 EMP-ORG

9 Aug 5, 2009ACL-IJCNLP 20099 General vs. Specific Features Assumption: some features are commonly useful for different relation types, while other features are specific for individual relation types : weight vector for target type : weight vector for k’th auxiliary type common weight vector in a lower H dimensional space

10 Aug 5, 2009ACL-IJCNLP 200910 Learning Framework loss function on the target seed instances loss function on the auxiliary training instances 10 4 1

11 Aug 5, 2009ACL-IJCNLP 200911 General Features Which subset of features should be captured by ? common weight vector in a lower H dimensional space

12 Aug 5, 2009ACL-IJCNLP 200912 Feature Separation Automatic separation within the learning framework (see [Jiang & Zhai 2007b]) Human guidance –Argument word features: features that contain head word of an argument E.g. arg-1 word: sister –Entity type features: features that contain the entity type (subtype) of an argument E.g. arg-2 type: ORG Combined

13 Aug 5, 2009ACL-IJCNLP 200913 Imposing Entity Type Constraint Fix the possible entity types for the arguments for the target relation type Filter out the relation instances that do not satisfy the constraint in the end

14 Aug 5, 2009ACL-IJCNLP 200914 Experiment Setup ACE 2004, 7 relation types –6 types  auxiliary types 1 type  target type 5-fold cross validation # seed instances: 10

15 Aug 5, 2009ACL-IJCNLP 200915 Methods Compared BL: train on seed instances only BL-A: train on seed and auxiliary training instances together w/o feature separation TL-auto: transfer learning w/ automatic feature separation TL-guide: transfer learning w/ human-guided feature separation TL-comb: automatic feature separation combined with human guidance TL-NE: TL-comb + entity type constraint

16 Aug 5, 2009ACL-IJCNLP 200916 Comparison Target TypeBLBL-ATL-autoTL- guide TL- comb TL-NE PhysicalP0.0000.16920.29200.29340.33250.5056 R0.0000.08480.16960.17220.23830.2316 F0.0000.11300.21460.21700.27770.3176 Personal/SocialP1.0000.08040.10050.30690.32140.6412 R0.03860.17080.15980.72450.76860.7631 F0.07430.10930.12340.43110.45330.6969 Employment /Membership /Subsidiary P0.92310.35610.52300.54280.59730.7145 R0.00750.18500.26170.26480.36320.3601 F0.01480.24350.34880.35590.45180.4789 AverageP0.81240.14750.24120.27030.29920.4231 R0.02120.24320.38320.47640.55090.5464 F0.04060.15320.25320.29580.34230.4132

17 Aug 5, 2009ACL-IJCNLP 200917 Effect of λ λμTλμT 100100010000 P0.62650.31620.2992 R0.11700.39590.5509 F0.18470.29830.3423 Performance of TL-comb. λ μ k = 10 4, λ ν = 1.

18 Aug 5, 2009ACL-IJCNLP 200918 Number of Seed Instances

19 Aug 5, 2009ACL-IJCNLP 200919 Sensitivity of H

20 Aug 5, 2009ACL-IJCNLP 200920 Conclusions We proposed to apply a multi-task transfer learning framework to the weakly-supervised relation extraction problem. We defined two kinds of type-specific features. Our experiments show that automatic feature separation combined with human guidance and entity type constraint can significantly outperform the baselines.

21 Aug 5, 2009ACL-IJCNLP 200921 Thank You! Questions?

22 Aug 5, 2009ACL-IJCNLP 200922 Related Work [Zhou et al. 2008]: Different way of modeling commonality among relation types. [Banko & Etzioni, 2008]: Open-domain relation extraction. No target relation type. [Xu et al. 2008]: Rule-based adaptation. Same type.

23 Aug 5, 2009ACL-IJCNLP 200923 Hypothesized Type-Specific Features

Download ppt "Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009."

Similar presentations

Explanation-Based Learning (borrowed from mooney et al)

Explanation-Based Learning (borrowed from mooney et al)
Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao, Wei Fan, Jing Jiang, Jiawei Han l Motivate Solution Framework Data Sets Synthetic.

Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao, Wei Fan, Jing Jiang, Jiawei Han l Motivate Solution Framework Data Sets Synthetic.
Latent Variables Naman Agarwal Michael Nute May 1, 2013.

Latent Variables Naman Agarwal Michael Nute May 1, 2013.
A Survey on Transfer Learning Sinno Jialin Pan Department of Computer Science and Engineering The Hong Kong University of Science and Technology Joint.

A Survey on Transfer Learning Sinno Jialin Pan Department of Computer Science and Engineering The Hong Kong University of Science and Technology Joint.
Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?
Progress update Lin Ziheng. System overview 2 Components – Connective classifier Features from Pitler and Nenkova (2009): – Connective: because – Self.

Progress update Lin Ziheng. System overview 2 Components – Connective classifier Features from Pitler and Nenkova (2009): – Connective: because – Self.
Semi-supervised Relation Extraction with Large-scale Word Clustering Ang Sun Ralph Grishman Satoshi Sekine New York University June 20, 2011 NYU.

Semi-supervised Relation Extraction with Large-scale Word Clustering Ang Sun Ralph Grishman Satoshi Sekine New York University June 20, 2011 NYU.
Machine learning continued Image source: https://www.coursera.org/course/mlhttps://www.coursera.org/course/ml.

Machine learning continued Image source:
1 Latent Semantic Mapping: Dimensionality Reduction via Globally Optimal Continuous Parameter Modeling Jerome R. Bellegarda.

1 Latent Semantic Mapping: Dimensionality Reduction via Globally Optimal Continuous Parameter Modeling Jerome R. Bellegarda.
Proposed concepts illustrated well on sets of face images extracted from video: Face texture and surface are smooth, constraining them to a manifold Recognition.

Proposed concepts illustrated well on sets of face images extracted from video: Face texture and surface are smooth, constraining them to a manifold Recognition.
计算机科学与技术学院 Chinese Semantic Role Labeling with Dependency-driven Constituent Parse Tree Structure Hongling Wang, Bukang Wang Guodong Zhou NLP Lab, School.

计算机科学与技术学院 Chinese Semantic Role Labeling with Dependency-driven Constituent Parse Tree Structure Hongling Wang, Bukang Wang Guodong Zhou NLP Lab, School.
A Two-Stage Approach to Domain Adaptation for Statistical Classifiers Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois.

A Two-Stage Approach to Domain Adaptation for Statistical Classifiers Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois.
Frustratingly Easy Domain Adaptation

Frustratingly Easy Domain Adaptation
Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots Chao-Yeh Chen and Kristen Grauman University of Texas at Austin.

Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots Chao-Yeh Chen and Kristen Grauman University of Texas at Austin.
Unsupervised Transfer Classification Application to Text Categorization Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University.

Unsupervised Transfer Classification Application to Text Categorization Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University.
Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?
Supervised Distance Metric Learning Presented at CMU’s Computer Vision Misc-Read Reading Group May 9, 2007 by Tomasz Malisiewicz.

Supervised Distance Metric Learning Presented at CMU’s Computer Vision Misc-Read Reading Group May 9, 2007 by Tomasz Malisiewicz.
Queensland University of Technology An Ontology-based Mining Approach for User Search Intent Discovery Yan Shen, Yuefeng Li, Yue Xu, Renato Iannella, Abdulmohsen.

Queensland University of Technology An Ontology-based Mining Approach for User Search Intent Discovery Yan Shen, Yuefeng Li, Yue Xu, Renato Iannella, Abdulmohsen.
Extracting Interest Tags from Twitter User Biographies Ying Ding, Jing Jiang School of Information Systems Singapore Management University AIRS 2014, Kuching,

Extracting Interest Tags from Twitter User Biographies Ying Ding, Jing Jiang School of Information Systems Singapore Management University AIRS 2014, Kuching,
Relaxed Transfer of Different Classes via Spectral Partition Xiaoxiao Shi 1 Wei Fan 2 Qiang Yang 3 Jiangtao Ren 4 1 University of Illinois at Chicago 2.

Relaxed Transfer of Different Classes via Spectral Partition Xiaoxiao Shi 1 Wei Fan 2 Qiang Yang 3 Jiangtao Ren 4 1 University of Illinois at Chicago 2.

Similar presentations

Ads by Google