Visualizing and Understanding Neural Models in NLP

Slides:

Advertisements

Similar presentations

Introduction to Recurrent neural networks (RNN), Long short-term memory (LSTM) Wenjie Pei In this coffee talk, I would like to present you some basic.

Advertisements

Distributed Representations of Sentences and Documents

The Interpreter Pattern. Defining the Interpreter Intent Given a language, define a representation for its grammar along with an interpreter that uses.

Richard Socher Cliff Chiung-Yu Lin Andrew Y. Ng Christopher D. Manning

Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA ImprovingWord.

Kai Sheng-Tai, Richard Socher, Christopher D. Manning

Convolutional LSTM Networks for Subcellular Localization of Proteins

Predicting the dropouts rate of online course using LSTM method

NOTE: To change the image on this slide, select the picture and delete it. Then click the Pictures icon in the placeholder to insert your own image. SHOW.

Short Text Similarity with Word Embedding Date: 2016/03/28 Author: Tom Kenter, Maarten de Rijke Source: CIKM’15 Advisor: Jia-Ling Koh Speaker: Chih-Hsuan.

Learning to Answer Questions from Image Using Convolutional Neural Network Lin Ma, Zhengdong Lu, and Hang Li Huawei Noah’s Ark Lab, Hong Kong

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation EMNLP’14 paper by Kyunghyun Cho, et al.

A Sentence Interaction Network for Modeling Dependence between Sentences Biao Liu, Minlie Huang Tsinghua University.

Graph-based Dependency Parsing with Bidirectional LSTM Wenhui Wang and Baobao Chang Institute of Computational Linguistics, Peking University.

S.Bengio, O.Vinyals, N.Jaitly, N.Shazeer

R-NET: Machine Reading Comprehension With Self-Matching Networks

End-To-End Memory Networks

CS 388: Natural Language Processing: LSTM Recurrent Neural Networks

Deep Learning for Bacteria Event Identification

Sentence Modeling Representation of sentences is the heart of Natural Language Processing A sentence model is a representation and analysis of semantic.

Recursive Neural Networks

Recurrent Neural Networks for Natural Language Processing

Inference as a Feedforward Network

Recurrent Neural Networks

A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis

Deep Learning: Model Summary

Intro to NLP and Deep Learning

ICS 491 Big Data Analytics Fall 2017 Deep Learning

CSCI 5922 Neural Networks and Deep Learning: Image Captioning

Different Units Ramakrishna Vedantam.

Are End-to-end Systems the Ultimate Solutions for NLP?

Neural Networks 2 CS446 Machine Learning.

Shunyuan Zhang Nikhil Malik

Image Question Answering

Recursive Structure.

Attention-based Caption Description Mun Jonghwan.

Advanced Artificial Intelligence

Image Captions With Deep Learning Yulia Kogan & Ron Shiff

Enhanced Dependency Jiajie Yu Wentao Ding.

A First Look at Music Composition using LSTM Recurrent Neural Networks

A Fast Unified Model for Parsing and Sentence Understanding

Final Presentation: Neural Network Doc Summarization

RNNs & LSTM Hadar Gorodissky Niv Haim.

Understanding LSTM Networks

Seminar Topics and Projects

Word embeddings based mapping

Code Completion with Neural Attention and Pointer Networks

Introduction to Machine Reading Comprehension

Other Classification Models: Recurrent Neural Network (RNN)

Lecture 16: Recurrent Neural Networks (RNNs)

Classification Boundaries

Recurrent Encoder-Decoder Networks for Time-Varying Dense Predictions

Machine Translation(MT)

Natural Language to SQL(nl2sql)

Report by: 陆纪圆.

Graph Neural Networks Amog Kamsetty January 30, 2019.

LSTM: Long Short Term Memory

Word Embedding 모든 단어를 vector로 표시 Word vector Word embedding Word

Deep Learning Authors: Yann LeCun, Yoshua Bengio, Geoffrey Hinton

-- Ray Mooney, Association for Computational Linguistics (ACL) 2014

Learn to Comment Mentor: Mahdi M. Kalayeh

Recurrent Neural Networks (RNNs)

The Updated experiment based on LSTM

The experiments based on Recurrent Neural Networks

Baseline Model CSV Files Pandas DataFrame Sentence Lists

Week 3 Presentation Ngoc Ta Aidean Sharghi.

Recurrent Neural Networks

Deep learning: Recurrent Neural Networks CV192

Bidirectional LSTM-CRF Models for Sequence Tagging

Visual Grounding.

Presentation transcript:

Visualizing and Understanding Neural Models in NLP Jiwei Li, Xinlei Chen, Eduard Hovy and Dan Jurafsky Presentation by Rohit Gupta Roll No. 15111037

Motivation Vector-based models produced by applying neural networks to natural language are very difficult to interpret

Dataset used Stanford Sentiment Treebank dataset sentiment labels for every parse tree constituent, from sentences to to individual words, for 11,855 sentences

Models studied Standard recurrent sequence models with TANH activation functions LSTMs: Long Short Term Memory Bidirectional LSTMs

Visualizations Compositionality Negation Intensification Concessive Salience (contribution of unit to the final composed meaning) Gradient back-propagation The variance of a token from the average word node LSTM-style gates that measure information flow

Local compositionality

t-SNE visualization on representations for negation

Concessive clause composition

Salience Method 1: First derivatives

Salience method 2: Variance of word from sentence mean

Salience method 3: Gate models