Understanding LSTM Networks

Slides:

Advertisements

Similar presentations

Introduction to Recurrent neural networks (RNN), Long short-term memory (LSTM) Wenjie Pei In this coffee talk, I would like to present you some basic.

Advertisements

Deep Learning Neural Network with Memory (1)

Computer Science 210 Computer Organization Control Circuits Decoders and Multiplexers.

Haitham Elmarakeby.  Speech recognition

ECE 6504: Deep Learning for Perception Dhruv Batra Virginia Tech Topics: –LSTMs (intuition and variants) –[Abhishek:] Lua / Torch Tutorial.

Convolutional LSTM Networks for Subcellular Localization of Proteins

Predicting the dropouts rate of online course using LSTM method

NOTE: To change the image on this slide, select the picture and delete it. Then click the Pictures icon in the placeholder to insert your own image. SHOW.

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation EMNLP’14 paper by Kyunghyun Cho, et al.

Lecture 7: Cognitive Science A Necker Cube

Attention Model in NLP Jichuan ZENG.

Deep Learning RUSSIR 2017 – Day 3

Best viewed with Computer Modern fonts installed

Unsupervised Learning of Video Representations using LSTMs

SUNY Korea BioData Mining Lab - Journal Review

Learning linguistic structure with simple and more complex recurrent neural networks Psychology February 2, 2017.

SD Study RNN & LSTM 2016/11/10 Seitaro Shinagawa.

Best viewed with Computer Modern fonts installed

End-To-End Memory Networks

CS 388: Natural Language Processing: LSTM Recurrent Neural Networks

CS 4501: Introduction to Computer Vision Computer Vision + Natural Language Connelly Barnes Some slides from Fei-Fei Li / Andrej Karpathy / Justin Johnson.

Recurrent Neural Networks for Natural Language Processing

Van-Khanh Tran and Le-Minh Nguyen

Recurrent Neural Networks

Neural Machine Translation by Jointly Learning to Align and Translate

Show and Tell: A Neural Image Caption Generator (CVPR 2015)

Figure 1. Examples of e-cigarette discussions in social media

Deep Learning: Model Summary

ICS 491 Big Data Analytics Fall 2017 Deep Learning

Different Units Ramakrishna Vedantam.

Synthesis of X-ray Projections via Deep Learning

Attention Is All You Need

Master’s Thesis defense Ming Du Advisor: Dr. Yi Shang

Zhu Han University of Houston

Recursive Structure.

Advanced Recurrent Architectures

Grid Long Short-Term Memory

RNN and LSTM Using MXNet Cyrus M Vahid, Principal Solutions Architect

Advanced Artificial Intelligence

Paraphrase Generation Using Deep Learning

Image Captions With Deep Learning Yulia Kogan & Ron Shiff

A First Look at Music Composition using LSTM Recurrent Neural Networks

Recurrent Neural Networks

RNNs & LSTM Hadar Gorodissky Niv Haim.

ECE599/692 - Deep Learning Lecture 14 – Recurrent Neural Network (RNN)

Code Completion with Neural Attention and Pointer Networks

The Big Health Data–Intelligent Machine Paradox

Other Classification Models: Recurrent Neural Network (RNN)

Lecture 16: Recurrent Neural Networks (RNNs)

Recurrent Encoder-Decoder Networks for Time-Varying Dense Predictions

Machine Translation(MT)

Report by: 陆纪圆.

A connectionist model in action

实习生汇报 ——北邮张安迪.

LSTM: Long Short Term Memory

Meta Learning (Part 2): Gradient Descent as LSTM

Attention for translation

Recurrent Neural Networks (RNNs)

Neural Machine Translation using CNN

Question Answering System

Week 3 Presentation Ngoc Ta Aidean Sharghi.

Recurrent Neural Networks

Deep learning: Recurrent Neural Networks CV192

Cengizhan Can Phoebe de Nooijer

Bidirectional LSTM-CRF Models for Sequence Tagging

LHC beam mode classification

Beating the market -- forecasting the S&P 500 Index

Neural Machine Translation by Jointly Learning to Align and Translate

Presentation transcript:

Understanding LSTM Networks with Colah’s figures Colah’s blog: http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Recurrent Neural Network http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Recurrent Neural Network http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Long-Term Dependencies The clouds are in the sky http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Longer-Term Dependencies http://colah.github.io/posts/2015-08-Understanding-LSTMs/

LSTM comes in! Long Short Term Memory This is just a standard RNN. http://colah.github.io/posts/2015-08-Understanding-LSTMs/

LSTM comes in! Long Short Term Memory This is the LSTM! This is just a standard RNN. http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Overall Architecture Output (Cell) state Next (Cell) State Forget Gate (Cell) state Next (Cell) State Input Gate Output Gate Hidden State Next Hidden State Input Output = Hidden state http://colah.github.io/posts/2015-08-Understanding-LSTMs/

The Core Idea http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Step-by-Step Forget Gate Input Gate Decide what information we’re going to throw away from the cell state. Input Gate Decide what new information we’re going to store in the cell state. http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Step-by-Step Update (cell state) Output Gate (hidden state) Update, scaled by how much we decide to update : input_gate*curr_state + forget_gate*prev_state Output Gate (hidden state) Output based on the updated state : output_gate*updated_state http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Again Output (Cell) state Next (Cell) State Hidden State Input Gate (Cell) state Next (Cell) State Forget Gate Output Gate Hidden State Next Hidden State Input http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Gated Recurrent Unit Cho, Kyunghyun, et al. "Learning phrase representations using RNN encoder-decoder for statistical machine translation." arXiv preprint arXiv:1406.1078 (2014). http://colah.github.io/posts/2015-08-Understanding-LSTMs/