Storylines from Streaming Text The Infinite Topic Cluster Model Amr Ahmed, Jake Eisenstein, Qirong Ho Alex Smola, Choon Hui Teo, Eric Xing Carnegie Mellon.

Slides:

Advertisements

Similar presentations

Sinead Williamson, Chong Wang, Katherine A. Heller, David M. Blei

Advertisements

Scaling Up Graphical Model Inference

A probabilistic model for retrospective news event detection

WWW2011-Unified Analysis of Streaming News Amr Ahmed, Qirong Ho, Jacob Eisenstein, Eric Xing, Carnegie Mellon University, Alexander J. Smola, Choon Hui.

Topic models Source: Topic models, David Blei, MLSS 09.

Teg Grenager NLP Group Lunch February 24, 2005

Xiaolong Wang and Daniel Khashabi

Markov Chain Sampling Methods for Dirichlet Process Mixture Models R.M. Neal Summarized by Joon Shik Kim (Thu) Computational Models of Intelligence.

Hierarchical Dirichlet Process (HDP)

Ouyang Ruofei Topic Model Latent Dirichlet Allocation Ouyang Ruofei May LDA.

Hierarchical Dirichlet Processes

Title: The Author-Topic Model for Authors and Documents

Visual Tracking CMPUT 615 Nilanjan Ray. What is Visual Tracking Following objects through image sequences or videos Sometimes we need to track a single.

Statistical Topic Modeling part 1

Decoupling Sparsity and Smoothness in the Discrete Hierarchical Dirichlet Process Chong Wang and David M. Blei NIPS 2009 Discussion led by Chunping Wang.

Unsupervised and Weakly-Supervised Probabilistic Modeling of Text Ivan Titov April TexPoint fonts used in EMF. Read the TexPoint manual before.

Part IV: Monte Carlo and nonparametric Bayes. Outline Monte Carlo methods Nonparametric Bayesian models.

Statistical Models for Networks and Text Jimmy Foulds UCI Computer Science PhD Student Advisor: Padhraic Smyth.

Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation James Foulds 1, Levi Boyles 1, Christopher DuBois 2 Padhraic Smyth.

Sparse Word Graphs: A Scalable Algorithm for Capturing Word Correlations in Topic Models Ramesh Nallapati Joint work with John Lafferty, Amr Ahmed, William.

Latent Dirichlet Allocation a generative model for text

Unsupervised discovery of visual object class hierarchies Josef Sivic (INRIA / ENS), Bryan Russell (MIT), Andrew Zisserman (Oxford), Alyosha Efros (CMU)

Multiscale Topic Tomography Ramesh Nallapati, William Cohen, Susan Ditmore, John Lafferty & Kin Ung (Johnson and Johnson Group)

Correlated Topic Models By Blei and Lafferty (NIPS 2005) Presented by Chunping Wang ECE, Duke University August 4 th, 2006.

Topic Models in Text Processing IR Group Meeting Presented by Qiaozhu Mei.

(Infinitely) Deep Learning in Vision Max Welling (UCI) collaborators: Ian Porteous (UCI) Evgeniy Bart UCI/Caltech) Pietro Perona (Caltech)

Annealing Paths for the Evaluation of Topic Models James Foulds Padhraic Smyth Department of Computer Science University of California, Irvine* *James.

Bayesian Hierarchical Clustering Paper by K. Heller and Z. Ghahramani ICML 2005 Presented by HAO-WEI, YEH.

1 Linmei HU 1, Juanzi LI 1, Zhihui LI 2, Chao SHAO 1, and Zhixing LI 1 1 Knowledge Engineering Group, Dept. of Computer Science and Technology, Tsinghua.

Hierarchical Topic Models and the Nested Chinese Restaurant Process Blei, Griffiths, Jordan, Tenenbaum presented by Rodrigo de Salvo Braz.

27. May Topic Models Nam Khanh Tran L3S Research Center.

Eric Xing © Eric CMU, Machine Learning Latent Aspect Models Eric Xing Lecture 14, August 15, 2010 Reading: see class homepage.

Integrating Topics and Syntax -Thomas L

Timeline: A Dynamic Hierarchical Dirichlet Process Model for Recovering Birth/Death and Evolution of Topics in Text Stream (UAI 2010) Amr Ahmed and Eric.

Summary We propose a framework for jointly modeling networks and text associated with them, such as networks or user review websites. The proposed.

Latent Dirichlet Allocation D. Blei, A. Ng, and M. Jordan. Journal of Machine Learning Research, 3: , January Jonathan Huang

Hierarchical Dirichlet Process and Infinite Hidden Markov Model Duke University Machine Learning Group Presented by Kai Ni February 17, 2006 Paper by Y.

Probabilistic Models for Discovering E-Communities Ding Zhou, Eren Manavoglu, Jia Li, C. Lee Giles, Hongyuan Zha The Pennsylvania State University WWW.

1 Dirichlet Process Mixtures A gentle tutorial Graphical Models – Khalid El-Arini Carnegie Mellon University November 6 th, 2006 TexPoint fonts used.

CMU at TDT 2004 — Novelty Detection Jian Zhang and Yiming Yang Carnegie Mellon University.

Topic Models Presented by Iulian Pruteanu Friday, July 28 th, 2006.

Topic Modeling using Latent Dirichlet Allocation

Latent Dirichlet Allocation

Discovering Objects and their Location in Images Josef Sivic 1, Bryan C. Russell 2, Alexei A. Efros 3, Andrew Zisserman 1 and William T. Freeman 2 Goal:

Bayesian Multi-Population Haplotype Inference via a Hierarchical Dirichlet Process Mixture Duke University Machine Learning Group Presented by Kai Ni August.

CS246 Latent Dirichlet Analysis. LSI  LSI uses SVD to find the best rank-K approximation  The result is difficult to interpret especially with negative.

Discovering Evolutionary Theme Patterns from Text - An Exploration of Temporal Text Mining Qiaozhu Mei and ChengXiang Zhai Department of Computer Science.

Nonparametric Bayesian Models. HW 4 x x Parametric Model Fixed number of parameters that is independent of the data we’re fitting.

CS Statistical Machine learning Lecture 25 Yuan (Alan) Qi Purdue CS Nov

NIPS 2013 Michael C. Hughes and Erik B. Sudderth

Hierarchical Beta Process and the Indian Buffet Process by R. Thibaux and M. I. Jordan Discussion led by Qi An.

APPLICATIONS OF DIRICHLET PROCESS MIXTURES TO SPEAKER ADAPTATION Amir Harati and Joseph PiconeMarc Sobel Institute for Signal and Information Processing,

Text-classification using Latent Dirichlet Allocation - intro graphical model Lei Li

A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation Yee W. Teh, David Newman and Max Welling Published on NIPS 2006 Discussion.

Topic Modeling for Short Texts with Auxiliary Word Embeddings

Online Multiscale Dynamic Topic Models

The topic discovery models

J. Zhu, A. Ahmed and E.P. Xing Carnegie Mellon University ICML 2009

The topic discovery models

Collapsed Variational Dirichlet Process Mixture Models

Hierarchical Topic Models and the Nested Chinese Restaurant Process

The topic discovery models

Topic Modeling Nick Jordan.

Bayesian Inference for Mixture Language Models

Stochastic Optimization Maximization for Latent Variable Models

Michal Rosen-Zvi University of California, Irvine

Junghoo “John” Cho UCLA

Topic models for corpora and for graphs

Topic Models in Text Processing

Presentation transcript:

Storylines from Streaming Text The Infinite Topic Cluster Model Amr Ahmed, Jake Eisenstein, Qirong Ho Alex Smola, Choon Hui Teo, Eric Xing Carnegie Mellon UniversityYahoo! Research

Outline Visualizing a news stream Goals Clusters & Content analysis The story-topic model Recurrent Chinese Restaurant Process Latent Dirichlet Allocation Examples

News Stream

Realtime news stream Multiple sources (Reuters, AP, CNN,...) Same story from multiple sources Stories are related Goals Aggregate articles into a storyline Analyze the storyline (topics, entities)

Precursors

Evolutionary Clustering / RCRP Assume active story distribution at time t Draw story indicator Draw words from story distribution Down-weight story counts for next day Ahmed & Xing, 2008

Clustering / RCRP Pro Nonparametric model of story generation (no need to model frequency of stories) No fixed number of stories Efficient inference via collapsed sampler Con We learn nothing! No content analysis

Latent Dirichlet Allocation Generate topic distribution per article Draw topics per word from topic distribution Draw words from topic specific word distribution Blei, Ng, Jordan, 2003

Latent Dirichlet Allocation Pro Topical analysis of stories Topical analysis of words (meaning, saliency) More documents improve estimates Con No clustering

Named entities are special, topics less (e.g. Tiger Woods and his mistresses) Some stories are strange (topical mixture is not enough - dirty models) Articles deviate from general story (Hierarchical DP) More Issues

Storylines

Storylines Model Topic model Topics per cluster RCRP for cluster Hierarchical DP for article Separate model for named entities Story specific correction

Inference We receive articles as a stream Want topics & stories now Variational inference infeasible (RCRP, sparse to dense, vocabulary size) We have a ‘tracking problem’ Sequential Monte Carlo Use sampled variables of surviving particle Use ideas from Cannini et al. 2009

Proposal distribution - draw stories s, topics z using Gibbs Sampling for each particle Reweight particle via Resample particles if l 2 norm too large (resample some assignments for diversity, too) Compare to multiplicative updates algorithm In our case predictive likelihood yields weights Estimation past state new data

Inheritance Tree n o t t h r e a d s a f e

Extended Inheritance Tree write only in the leaves (per thread)

Results

Numbers... TDT5 (Topic Detection and Tracking) macro-averaged minimum detection cost: This is the best performance on TDT5! Yahoo News data... beats all other clustering algorithms timeentitiestopicsstory words

Stories

Related Stories

Issues

To Do Hierarchical story representation (H-RCRP) Adams, Jordan & Ghahramani 2010 Promotion of stories (corporate process) Recurrent formulation (reweight statistics) Possibly via hierarchical ddCRP metaphor Summarization of results Submodular objective Time dependence Personalization (explore/exploit)