Presentation is loading. Please wait.

Presentation is loading. Please wait.

Progress Report WANG XUN 2015/10/02.

Published byNorma Chapman Modified over 6 years ago

Similar presentations

Presentation on theme: "Progress Report WANG XUN 2015/10/02."— Presentation transcript:

1 Progress Report WANG XUN 2015/10/02

2 Outline Enhanced Word Embedding from a Hierarchical Neural Language Model Coordination Structure Detection with Long Short Memory Network

3 Enhanced Word Embedding from a Hierarchical Neural Language Model
Word2Vec CBOW SKP Paragraph Vector GloVe

4 Unified Learning Frame
Given a sequence of words g: Averaging neighboring CBOW Dot production SKP Concatenate & project to another layer Concatenation Convolving using recurrent network Convolution

5 Horizontally Vertically Markov Property
Kid embeddings influenced by their parents (Preceding sibling & parent)

6 Objective Function: Maximize the likelihood: a

7 SKP CBOW Concatenation

8

9 Conclusion The Hierarchical Model improves word embeddings
Performances drop for sent/para/doc embeddings

10 Outline Enhanced Word Embedding from a Hierarchical Neural Language Model Coordination Structure Detection with Long Short Memory Network

11 Coordination Structure Detection with Long Short Memory Network
A and B Coordination Structure Detection Conjunct: (left)(right) Coordinator: and/or/but/,/…

12 Binary Classification
Existing Work I like cats and dogs. Binary Classification Cats, dogs 1 I like cats, dogs 0 Like cats, dogs 0 Cats, dogs 0

13 I like cats and dogs. I like cats and dogs.
No1: cats : dogs All the following examples are wrong. Some are more wrong than others. No2: like cats : dogs 1/3 No3: I like cats : dogs 2/4 3 2 1 I like cats dogs

14 I I I like cats

15 score Tensor 3 2 I like dogs cats

16 score Tensor 3 2 I like dogs cats

17 score Tensor 3 2 I like dogs cats

18 Performance Comparison
WSJ: 10-fold cross folding (not fine-tuned yet…): @1 : 71+ @2 : 72+ @5 : 74 Next Step Genia NP detection Best,

19 Conclusion LSTM Tensor

20 Memory NN for Multi-Speaker Dialogue Analysis
Input Output(words) Model Hidden layers Memory Memory Memory Memory Memory Memory Memory Memory Memory Memory Memory Input

21 Entity Based Discourse Analysis System
Input Output(words) Model Hidden layers Memory Memory Memory Memory Memory Memory Memory Memory Memory Memory Memory Input

Download ppt "Progress Report WANG XUN 2015/10/02."

Similar presentations

Limin Wang, Yu Qiao, and Xiaoou Tang

Limin Wang, Yu Qiao, and Xiaoou Tang
Dual-domain Hierarchical Classification of Phonetic Time Series Hossein Hamooni, Abdullah Mueen University of New Mexico Department of Computer Science.

Dual-domain Hierarchical Classification of Phonetic Time Series Hossein Hamooni, Abdullah Mueen University of New Mexico Department of Computer Science.
Profile-profile alignment using hidden Markov models Wing Wong.

Profile-profile alignment using hidden Markov models Wing Wong.
Speaker Adaptation for Vowel Classification

Speaker Adaptation for Vowel Classification
Isolated-Word Speech Recognition Using Hidden Markov Models

Isolated-Word Speech Recognition Using Hidden Markov Models
Deep Learning Neural Network with Memory (1)

Deep Learning Neural Network with Memory (1)
Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA ImprovingWord.

Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA ImprovingWord.
Image Captioning Approaches

Image Captioning Approaches
Parsing Natural Scenes and Natural Language with Recursive Neural Networks INTERNATIONAL CONFERENCE ON MACHINE LEARNING (ICML 2011) RICHARD SOCHER CLIFF.

Parsing Natural Scenes and Natural Language with Recursive Neural Networks INTERNATIONAL CONFERENCE ON MACHINE LEARNING (ICML 2011) RICHARD SOCHER CLIFF.
Xintao Wu University of Arkansas Introduction to Deep Learning 1.

Xintao Wu University of Arkansas Introduction to Deep Learning 1.
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation EMNLP’14 paper by Kyunghyun Cho, et al.

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation EMNLP’14 paper by Kyunghyun Cho, et al.
A Hierarchical Deep Temporal Model for Group Activity Recognition

A Hierarchical Deep Temporal Model for Group Activity Recognition
Survey on state-of-the-art approaches: Neural Network Trends in Speech Recognition Survey on state-of-the-art approaches: Neural Network Trends in Speech.

Survey on state-of-the-art approaches: Neural Network Trends in Speech Recognition Survey on state-of-the-art approaches: Neural Network Trends in Speech.
A Sentence Interaction Network for Modeling Dependence between Sentences Biao Liu, Minlie Huang Tsinghua University.

A Sentence Interaction Network for Modeling Dependence between Sentences Biao Liu, Minlie Huang Tsinghua University.
Audio-Based Multimedia Event Detection Using Deep Recurrent Neural Networks Yun Wang, Leonardo Neves, Florian Metze 3/23/2016.

Audio-Based Multimedia Event Detection Using Deep Recurrent Neural Networks Yun Wang, Leonardo Neves, Florian Metze 3/23/2016.
Unsupervised Learning of Video Representations using LSTMs

Unsupervised Learning of Video Representations using LSTMs
Intrusion Detection using Deep Neural Networks

Intrusion Detection using Deep Neural Networks
CNN-RNN: A Unified Framework for Multi-label Image Classification

CNN-RNN: A Unified Framework for Multi-label Image Classification
CS 4501: Introduction to Computer Vision Computer Vision + Natural Language Connelly Barnes Some slides from Fei-Fei Li / Andrej Karpathy / Justin Johnson.

CS 4501: Introduction to Computer Vision Computer Vision + Natural Language Connelly Barnes Some slides from Fei-Fei Li / Andrej Karpathy / Justin Johnson.
Deep Learning for Bacteria Event Identification

Deep Learning for Bacteria Event Identification

Similar presentations

Ads by Google