Download presentation
Presentation is loading. Please wait.
Published byCecily Bates Modified over 6 years ago
1
Document Clustering Based on Non-negative Matrix Factorization
Wei Xu et al, SIGIR’03 Presented by Weilin Xu, 11/06/2014
3
Motivation Document clustering applications
Narrow Broad Variety of information needs Document clustering applications News clustering Extractive summarization … Why traditional clustering methods not applicable? Agglomerative clustering (bottom-up hierarchical) Cons: Computationally expensive Document partitioning (flat clustering), e.g. K-means Cons: Harsh simplifying assumptions on distributions Hierarchical: computationally expensive (O(n2logn)) Flat: based on weak assumptions, compact shape clusters in K-means, independent dimensions in Naïve Bayes, Gaussian distribution assumption in Gaussian mixture model.
4
Motivation Existing document clustering methods
Latent semantic indexing (LSI) Singular Value Decomposition Negative coefficients Graph-based spectral clustering Presented by Jinghe Leads to computing singular vectors or eigenvectors Equivalent to each other under certain conditions. Both need additional clustering methods. LSI: project each document into the singular vector space, then conducts document clustering using traditional clustering Graph: lead to the computation of singular vectors or eigenvectors of certain graph affinity matrices.
5
Proposed method Non-negative Matrix Factorization-based clustering
𝑋≈𝑈 𝑉 𝑇 U: term-concept matrix; V: document-concept matrix. Non-negative coefficients Latent semantic space is not necessarily orthogonal Factorization, Approximation
6
Roadmap Representations Method NMF vs. SVD Experiment Summary
7
Representations 𝑋 𝑚×𝑛 = 𝑥 11 ⋯ 𝑥 1𝑛 ⋮ ⋱ ⋮ 𝑥 𝑚1 ⋯ 𝑥 𝑚𝑛 , term-document matrix E.g. TF-IDF vector of document i 𝑋 𝑚×𝑛 ≈ 𝑈 𝑚×𝑘 𝑉 𝑛×𝑘 𝑇 𝑈 𝑚×𝑘 : term-topic matrix, 𝑉 𝑛×𝑘 : document-topic matrix
8
Method 𝑋≈𝑈 𝑉 𝑇 Minimize the objective function:
Derivation process (omitted) Updating equations: Normalize U: Normalize U to get unique solution of U and V, Frobenius norm Convergence is guaranteed
9
NMF vs. SVD
10
Experiment - Corpora #doc #cluster
11
Experiment – Compared methods
Spectral clustering: Average Association (AA) Equivalent to [LSI + K-means], if <Xi,Xj> similarity Spectral clustering: Normalized Cut (NC) Standard form of NMF (NMF) NC weighted form (NMF-NCW)
12
Experiment - Evaluations
Accuracy Mutual information Normalized: Map(), find by Kuhn-Munkres algorithm.
13
Experiment - Results 4 2 3 1 NMF-NCW > NC > NMF > AA
14
Experiment - Results 4 2 3 1 NMF-NCW > NC > NMF > AA
15
Summary NMF-based document partitioning method
Differ from SVD-based LSI and spectral clustering Latent semantic space not need to be orthogonal Non-negative values in latent semantic directions Without additional clustering Outperform the best methods.
16
Thanks
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.