Presentation is loading. Please wait.

Presentation is loading. Please wait.

Domain Adaptation Slide 1 Hal Daumé III Frustratingly Easy Domain Adaptation Hal Daumé III School of Computing University of Utah

Published byWillis Boone Modified over 8 years ago

Similar presentations

Presentation on theme: "Domain Adaptation Slide 1 Hal Daumé III Frustratingly Easy Domain Adaptation Hal Daumé III School of Computing University of Utah"— Presentation transcript:

1 Domain Adaptation Slide 1 Hal Daumé III (me@hal3.name)‏ Frustratingly Easy Domain Adaptation Hal Daumé III School of Computing University of Utah me@hal3.name

2 Domain Adaptation Slide 2 Hal Daumé III (me@hal3.name)‏ Problem  My tagger expects data like: But the unknown culprits, who had access to some of the company's computers for an undetermined period... ...but then I give it data like: you know it is it's pretty much general practice now you know “was trained on” Source Domain Target Domain

3 Domain Adaptation Slide 3 Hal Daumé III (me@hal3.name)‏ Solutions...  “LDC Solution” -- Annotate more data!  Pros: will give us good models  Cons: Too expensive, wastes old effort, no fun  “NLP Solution” -- Just use our news model on non-news  Pros: Easy  Cons: Performs poorly, no fun  “ML Junkie Solution” -- Build new learning algorithms  Pros: Often works well, fun  Cons: Often hard to implement, computationally expensive  “Our Solution” – Preprocess the data  Pros: Works well, easy to implement, computationally cheap  Cons:...?

4 Domain Adaptation Slide 4 Hal Daumé III (me@hal3.name)‏ Problem Setup Training TimeTest Time Source Data Target Data Target Data We assume all data is labeled. If you only have unlabeled target data, talk to John Blitzer

5 Domain Adaptation Slide 5 Hal Daumé III (me@hal3.name)‏ Prior Work – Chelba and Acero Training TimeTest Time Source Data Target Data Target Data MaxEnt model Prior on weights MaxEnt model Straightforward to generalize to any regularized linear classifier (SVM, perceptron)‏

6 Domain Adaptation Slide 6 Hal Daumé III (me@hal3.name)‏ Prior Work – Daumé III and Marcu Training TimeTest Time Source Data Target Data Target Data Source MaxEnt General MaxEnt Target MaxEnt “Mixture model” Inference by Conditional Expectation Maximization

7 Domain Adaptation Slide 7 Hal Daumé III (me@hal3.name)‏ State of Affairs Perf.Impl.SpeedGenerality Baselines (Numerous)BadGoodGoodGood Prior (Chelba+GoodOkayGoodOkay Acero) MegaM (Daume+GreatTerribleTerribleOkay Marcu) Proposed approachVery GoodGreatGoodGreat

8 Domain Adaptation Slide 8 Hal Daumé III (me@hal3.name)‏ “MONITOR” versus “THE” News domain: “MONITOR” is a verb “THE” is a determiner Technical domain: “MONITOR” is a noun “THE” is a determiner Key Idea: Share some features (“the”)‏ Don't share others (“monitor”)‏ (and let the learner decide which are which)‏

9 Domain Adaptation Slide 9 Hal Daumé III (me@hal3.name)‏ Feature Augmentation WemonitorthetrafficThemonitorisheavy NVDNDNVR W:monitor P:we N:the C:a+ W:the P:monitor N:traffic C:a+ W:monitor P:the N:is C:a+ W:the P: N:monitor C:Aa+ SW:monitor SP:we SN:the SC:a+ SW:the SP:monitor SN:traffic SC:a+ TW:monitor TP:the TN:is TC:a+ TW:the TP: TN:monitor TC:Aa+ In feature-vector lingo: Φ (x) → ‹ Φ(x), Φ(x), 0 › (for source domain)‏ Φ(x) → ‹ Φ(x), 0, Φ(x) › (for target domain)‏ Why should this work? Original Features Augmented Features

10 Domain Adaptation Slide 10 Hal Daumé III (me@hal3.name)‏ A Kernel Perspective In feature-vector lingo: Φ (x) → ‹ Φ(x), Φ(x), 0 › (for source domain)‏ Φ(x) → ‹ Φ(x), 0, Φ(x) › (for target domain)‏ K aug (x,z) = 2K(x,z) if x,z from same domain K(x,z)otherwise

11 Domain Adaptation Slide 11 Hal Daumé III (me@hal3.name)‏ Experimental Setup  Lots of data sets:  ACE: Named entity recognition (6 domains)‏  CoNLL: Named entity recognition (2 domains)‏  PubMed: POS tagging (2 domains)‏  CNN: recapitalization (2 domains)‏  Treebank: Chunking (3 or 10 domains)‏  Always: 75% train, 12.5% dev, 12.5% test  Lots of baselines...  Evaluation metric: Hamming loss (+McNemar)  Sequence labeling using S EARN

12 Domain Adaptation Slide 12 Hal Daumé III (me@hal3.name)‏ Obvious Approach 1: SrcOnly Training TimeTest Time Source Data Target Data Target Data Source Data

13 Domain Adaptation Slide 13 Hal Daumé III (me@hal3.name)‏ Obvious Approach 2: TgtOnly Training TimeTest Time Source Data Target Data Target Data Target Data

14 Domain Adaptation Slide 14 Hal Daumé III (me@hal3.name)‏ Obvious Approach 3: All Training TimeTest Time Source Data Target Data Target Data Unioned Data Source Data Target Data

15 Domain Adaptation Slide 15 Hal Daumé III (me@hal3.name)‏ Obvious Approach 4: Weighted Training TimeTest Time Source Data Target Data Target Data Unioned Data Source Data Target Data

16 Domain Adaptation Slide 16 Hal Daumé III (me@hal3.name)‏ Obvious Approach 5: Pred Training TimeTest Time Source Data Target Data Target Data SrcOnly Target Data (w/ SrcOnly Predictions)‏

17 Domain Adaptation Slide 17 Hal Daumé III (me@hal3.name)‏ Obvious Approach 6: LinInt Training TimeTest Time Source Data Target Data Target Data SrcOnlyTgtOnly α SrcOnlyTgtOnly α

18 Domain Adaptation Slide 18 Hal Daumé III (me@hal3.name)‏ Results – Error Rates Task Dom SrcOnly TgtOnly Baseline Prior Augment bn4.982.37 2.11 (pred) 2.06 1.98 bc4.544.07 3.53 (weight) 3.47 3.47 ACE-nw4.783.71 3.56 (pred) 3.68 3.39 NERwl2.452.45 2.12 (all) 2.412.12 un3.672.46 2.10 (linint) 2.03 1.91 cts2.080.46 0.40 (all) 0.340.32 CoNLLtgt2.492.95 1.75 (wgt/li) 1.89 1.76 PubMedtgt12.024.15 3.95 (linint) 3.99 3.61 CNNtgt10.293.82 3.44 (linint) 3.35 3.37 wsj6.634.35 4.30 (weight) 4.27 4.11 swbd315.904.15 4.09 (linint) 3.60 3.51 br-cf5.166.27 4.72 (linint) 5.22 5.15 Treebr-cg4.325.36 4.15 (all) 4.254.90 bank-br-ck5.056.32 5.01 (prd/li) 5.27 5.41 Chunkbr-cl5.666.60 5.39 (wgt/prd) 5.99 5.73 br-cm3.576.59 3.11 (all) 4.084.89 br-cn4.605.56 4.19 (prd/li) 4.48 4.42 br-cp4.825.62 4.55 (wgt/prd/li) 4.87 4.78 br-cr5.789.13 5.15 (linint) 6.71 6.30 Treebank-brown6.355.75 4.72 (linint) 4.72 4.65

19 Domain Adaptation Slide 19 Hal Daumé III (me@hal3.name)‏ Hinton Diagram: /bush/ on ACE-NER General BC-news Conversations Newswire Weblogs Usenet Telephone PER GPE ORG LOC

20 Domain Adaptation Slide 20 Hal Daumé III (me@hal3.name)‏ Hinton Diagram: /P=the/ on ACE-NER General BC-news Conversations Newswire Weblogs Usenet Telephone PER GPE ORG LOC “the Iraqi people” “the Pentagon” “the Bush (advisors|cabinet|...)” “the South”

21 Domain Adaptation Slide 21 Hal Daumé III (me@hal3.name)‏ Discussion  What's good?  Works well (if T<S), applicable to any classifier  Easy to implement – 10 lines of Perl: http://hal3.name/easyadapt.pl.gz  Very fast – leverages any classifier  What could perhaps be slightly better maybe?  Theory – why should this help?  Unannotated target data? Thanks! Questions?

Download ppt "Domain Adaptation Slide 1 Hal Daumé III Frustratingly Easy Domain Adaptation Hal Daumé III School of Computing University of Utah"

Similar presentations

CSC321: Introduction to Neural Networks and Machine Learning Lecture 24: Non-linear Support Vector Machines Geoffrey Hinton.

CSC321: Introduction to Neural Networks and Machine Learning Lecture 24: Non-linear Support Vector Machines Geoffrey Hinton.
Integrated Instance- and Class- based Generative Modeling for Text Classification Antti PuurulaUniversity of Waikato Sung-Hyon MyaengKAIST 5/12/2013 Australasian.

Integrated Instance- and Class- based Generative Modeling for Text Classification Antti PuurulaUniversity of Waikato Sung-Hyon MyaengKAIST 5/12/2013 Australasian.
Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?
Search-Based Structured Prediction

Search-Based Structured Prediction
Imbalanced data David Kauchak CS 451 – Fall 2013.

Imbalanced data David Kauchak CS 451 – Fall 2013.
Machine learning continued Image source: https://www.coursera.org/course/mlhttps://www.coursera.org/course/ml.

Machine learning continued Image source:
Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009.

Multi-Task Transfer Learning for Weakly- Supervised Relation Extraction Jing Jiang Singapore Management University ACL-IJCNLP 2009.
Event Extraction Using Distant Supervision Kevin Reschke, Martin Jankowiak, Mihai Surdeanu, Christopher D. Manning, Daniel Jurafsky 30 May 2014 Language.

Event Extraction Using Distant Supervision Kevin Reschke, Martin Jankowiak, Mihai Surdeanu, Christopher D. Manning, Daniel Jurafsky 30 May 2014 Language.
Frustratingly Easy Domain Adaptation

Frustratingly Easy Domain Adaptation
Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots Chao-Yeh Chen and Kristen Grauman University of Texas at Austin.

Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots Chao-Yeh Chen and Kristen Grauman University of Texas at Austin.
Discriminative and generative methods for bags of features

Discriminative and generative methods for bags of features
Semi-supervised learning and self-training LING 572 Fei Xia 02/14/06.

Semi-supervised learning and self-training LING 572 Fei Xia 02/14/06.
1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes, one of the attributes is the class.

1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes, one of the attributes is the class.
Semantic Role Labeling using Maximum Entropy Model Joon-Ho Lim NLP Lab. Korea Univ.

Semantic Role Labeling using Maximum Entropy Model Joon-Ho Lim NLP Lab. Korea Univ.
Introduction to Machine Learning Approach Lecture 5.

Introduction to Machine Learning Approach Lecture 5.
Learning from Multiple Outlooks Maayan Harel and Shie Mannor ICML 2011 Presented by Minhua Chen.

Learning from Multiple Outlooks Maayan Harel and Shie Mannor ICML 2011 Presented by Minhua Chen.
Machine learning Image source: https://www.coursera.org/course/ml.

Machine learning Image source:
Introduction to domain adaptation

Introduction to domain adaptation
Machine learning Image source: https://www.coursera.org/course/mlhttps://www.coursera.org/course/ml.

Machine learning Image source:
TransRank: A Novel Algorithm for Transfer of Rank Learning Depin Chen, Jun Yan, Gang Wang et al. University of Science and Technology of China, USTC Machine.

TransRank: A Novel Algorithm for Transfer of Rank Learning Depin Chen, Jun Yan, Gang Wang et al. University of Science and Technology of China, USTC Machine.

Similar presentations

Ads by Google