Presentation is loading. Please wait.

Presentation is loading. Please wait.

Hal Daumé III Microsoft Research University of Maryland

Published byAnna Laine Modified over 5 years ago

Similar presentations

Presentation on theme: "Hal Daumé III Microsoft Research University of Maryland"— Presentation transcript:

1 Hal Daumé III Microsoft Research University of Maryland
@haldaume3 he/him/his image credit: Lyndon Wong

2 We’ve all probably seen figures like this…
Left-to-right monotonic structure Efficient learning: a series of text classification Separation between learning and inference Search is still an issue in principle, but works well in practice. Better search algorithms can improve the quality of generation. THIS STUFF WORKS (this one in particular is thanks to Kyunghyun Cho)

3 New Tasks New Models Given that these neural autoregressive models work, what is left to do?

4 Upcoming presentation at Widening NLP Workshop at ACL’19
New Tasks Sudha Rao Trista Cao Upcoming presentation at Widening NLP Workshop at ACL’19

5 New Tasks Rao Cao [Louis & Nenkova, IJCNLP’11, Gao, Zhong, Pretiuc-Pietro & Li, AAAI’19]

6 New Tasks Rao Cao

7 New Tasks New Models Given that these neural autoregressive models work, what is left to do?

8 New Models Sean Welleck Kianté Brantley . you i wish <stop>
study could lol a work lot Sean Welleck Kianté Brantley Also featuring Kyunghyun Cho (not pictured); to appear at ICML 2019 next week

9 Linearizing the hierarchical prediction
Welleck Brantley + Kyunghyun Cho, ICML’19 . you . i study ??? i <stop> <stop> wish ??? could lol <stop> a <stop> <stop> ??? <stop> <stop> <stop> <stop> work lot <stop> <stop> <stop> <stop>

10 Imitation learning w/ equivocating expert
Welleck Brantley + Kyunghyun Cho, ICML’19 . you i wish you could study lol . <stop> . i study ??? i <stop> <stop> wish ??? could lol <stop> a <stop> <stop> ??? <stop> <stop> <stop> <stop> work lot Target: i wish you could study lol . <stop> <stop> <stop> <stop>

11 Imitation learning w/ equivocating expert
Welleck Brantley + Kyunghyun Cho, ICML’19 . you i wish you could study lol . <stop> . i study i <stop> <stop> wish ??? could lol <stop> a <stop> <stop> ??? <stop> <stop> <stop> <stop> work lot Target: i wish you could study lol . <stop> <stop> <stop> <stop>

12 Imitation learning w/ equivocating expert
Welleck Brantley + Kyunghyun Cho, ICML’19 . you i wish you could study lol . <stop> . i could i <stop> <stop> wish ??? could lol <stop> a <stop> <stop> ??? <stop> <stop> <stop> <stop> work lot Target: i wish you could study lol . <stop> <stop> <stop> <stop>

13 Quicksort-esque expert policy
{the, on, mat, ., sat, cat, the} on Welleck Brantley + Kyunghyun Cho, ICML’19 The cat sat on the mat . {The, sat, cat} {mat, ., the} sat mat {The, cat} {<stop>} {the} {.} cat <stop> the . {The} {<stop>} {<stop>} {<stop>} {<stop>} {<stop>} <stop> The <stop> <stop> <stop> <stop> {<stop>} {<stop>} <stop> <stop>

14 Model structure on top of quicksort
Welleck Brantley {the, on, mat, ., sat, cat, the} + Kyunghyun Cho, ICML’19 on The cat sat on the mat . {The, sat, cat} {mat, ., the} sat Valid items on sat Loss . mat the

15 Formalizing the expert policy
Welleck Brantley + Kyunghyun Cho, ICML’19 where {The, sat, cat} sat Valid items on sat Loss . mat the

16 Distributing mass across equivocations
Welleck Brantley + Kyunghyun Cho, ICML’19 Uniform Oracle Coaching Oracle [He et al., 2012] Annealed Coaching Oracle Valid items . mat the

17 Training via imitation learning
Welleck Brantley This is a special case of imitation learning with an optimal oracle Extensively studied and used in NLP [Goldberg&Nivre, 2012; Vlachos&Clark, 2014 and many more] Extensively studied and used in robotics and control [Ross et al., 2011; and many more recent work from Abeel and Levine et al.] Learning-to-search* for non-monotonic sequential generation Roll-in by a oracle/learned policy Roll-out by an oracle policy Easy to swap roll-in and roll-out policies + Kyunghyun Cho, ICML’19

18 Results on unconditional generation
Welleck Brantley + Kyunghyun Cho, ICML’19 Implicit probabilistic model: sampling 👍 normalized probability 👎 Difficult to analyze quantitatively, but we tried: All the models were trained on utterances from a dialogue data [ConvAI PersonaChat]

19 Results on unconditional generation
Welleck Brantley Results on unconditional generation + Kyunghyun Cho, ICML’19 Implicit probabilistic model: sampling 👍 normalized probability 👎 We can also do a bit of more analysis:

20 Results on unconditional generation
Welleck Brantley Results on unconditional generation + Kyunghyun Cho, ICML’19 Implicit probabilistic model: sampling 👍 normalized probability 👎 We can also do a bit of more analysis:

21 Welleck Brantley Word descrambling + Kyunghyun Cho, ICML’19

22 Welleck Brantley Machine translation + Kyunghyun Cho, ICML’19 Lags behind left-to-right, monotonic generation in MT: Though, how much it lags depends on how you measure the quality

23 Welleck Brantley Machine translation + Kyunghyun Cho, ICML’19

24 Summary and discussion
Welleck Brantley Summary and discussion Rao Cao Lots of fun stuff to do moving to new tasks, models Promising results in non-monotonic generation But still haven’t “cracked” it Should we improve modeling/representations? Should we improve training algorithms? Some contemp work: [Gu et al., arxiv’19; Stern et al., arxiv’19] Code at Thanks! Questions?

Download ppt "Hal Daumé III Microsoft Research University of Maryland"

Similar presentations

A Support Vector Method for Optimizing Average Precision

A Support Vector Method for Optimizing Average Precision
Latent Variables Naman Agarwal Michael Nute May 1, 2013.

Latent Variables Naman Agarwal Michael Nute May 1, 2013.
ATMAN HB summary seminar #2 17.09.2010. Challenges 2 ATMAN project 9/17/2010.

ATMAN HB summary seminar # Challenges 2 ATMAN project 9/17/2010.
Support Vector Machines and Kernels Adapted from slides by Tim Oates Cognition, Robotics, and Learning (CORAL) Lab University of Maryland Baltimore County.

Support Vector Machines and Kernels Adapted from slides by Tim Oates Cognition, Robotics, and Learning (CORAL) Lab University of Maryland Baltimore County.
Deep Learning in NLP Word representation and how to use it for Parsing

Deep Learning in NLP Word representation and how to use it for Parsing
Data Visualization STAT 890, STAT 442, CM 462

Data Visualization STAT 890, STAT 442, CM 462
Frustratingly Easy Domain Adaptation

Frustratingly Easy Domain Adaptation
Self Taught Learning : Transfer learning from unlabeled data Presented by: Shankar B S DMML Lab 07-27-2007 Rajat Raina et al, CS, Stanford ICML 2007.

Self Taught Learning : Transfer learning from unlabeled data Presented by: Shankar B S DMML Lab Rajat Raina et al, CS, Stanford ICML 2007.
Presented by Li-Tal Mashiach Learning to Rank: A Machine Learning Approach to Static Ranking 049011 - Algorithms for Large Data Sets Student Symposium.

Presented by Li-Tal Mashiach Learning to Rank: A Machine Learning Approach to Static Ranking Algorithms for Large Data Sets Student Symposium.
Adapted by Doug Downey from Machine Learning EECS 349, Bryan Pardo Machine Learning Clustering.

Adapted by Doug Downey from Machine Learning EECS 349, Bryan Pardo Machine Learning Clustering.
1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions,

1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions,
Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.

Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.
Maria-Florina Balcan A Theoretical Model for Learning from Labeled and Unlabeled Data Maria-Florina Balcan & Avrim Blum Carnegie Mellon University, Computer.

Maria-Florina Balcan A Theoretical Model for Learning from Labeled and Unlabeled Data Maria-Florina Balcan & Avrim Blum Carnegie Mellon University, Computer.
Text Classification Using Stochastic Keyword Generation Cong Li, Ji-Rong Wen and Hang Li Microsoft Research Asia August 22nd, 2003.

Text Classification Using Stochastic Keyword Generation Cong Li, Ji-Rong Wen and Hang Li Microsoft Research Asia August 22nd, 2003.
Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)

Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)
A support Vector Method for Multivariate performance Measures

A support Vector Method for Multivariate performance Measures
Online Learning Algorithms

Online Learning Algorithms
CS 60050 Machine Learning. What is Machine Learning? Adapt to / learn from data  To optimize a performance function Can be used to:  Extract knowledge.

CS Machine Learning. What is Machine Learning? Adapt to / learn from data  To optimize a performance function Can be used to:  Extract knowledge.
Presented By Wanchen Lu 2/25/2013

Presented By Wanchen Lu 2/25/2013
Translated Learning Wenyuan Dai, Yuqiang Chen, Gui-Rong Xue, Qiang Yang, and Yong Yu. Translated Learning. In Proceedings of Twenty- Second Annual Conference.

Translated Learning Wenyuan Dai, Yuqiang Chen, Gui-Rong Xue, Qiang Yang, and Yong Yu. Translated Learning. In Proceedings of Twenty- Second Annual Conference.

Similar presentations

Ads by Google