Presentation is loading. Please wait.

Presentation is loading. Please wait.

Estimating the Number of Clusters (k) Selection Approaches: Use a Criterion to select among the solutions for several values of k (kmeans or GMMs are used)

Published byMyrtle Evans Modified over 8 years ago

Similar presentations

Presentation on theme: "Estimating the Number of Clusters (k) Selection Approaches: Use a Criterion to select among the solutions for several values of k (kmeans or GMMs are used)"— Presentation transcript:

1 Estimating the Number of Clusters (k) Selection Approaches: Use a Criterion to select among the solutions for several values of k (kmeans or GMMs are used) Criterion(k): Training Objective(k) + Model Complexity(k) Model Complexity: Bayesian arguments (BIC): L(k) – M(k) lnN Information theory (MDL, MML) Marginal Likelihood (Bayesian GMMs) Gap Statistic (Tibshirani et al, 2001) Stability-based methods Incremental methods are convenient

2 Estimating the Number of Clusters (k) Optimal solutions wrt clustering error do not always reveal the true clustering structure

3 Estimating the Number of Clusters (k) Dynamic Approaches (add or remove components on-line) Bottom-up (GMM) Starting from many initial components gradually eliminate redundant ones Methods: MML-GMM (Law & Figueiredo, IEEE TPAMI 2002)MML-GMM Bayesian GMM (Corduneanou & Bishop, AISTATS 2001)Bayesian GMM Newtonian Clustering (Blekas & Lagaris, PR 2007)

4 Bayesian GMM (C-B) C-B method: Results depend on the number of initial components initialization of components specification of the scale matrix V of the Wishart prior p(T)

5 Bayesian GMM (C-B)

6 Estimating the Number of Clusters (k) Top – down (incremental) Starting from one component Iteratively add components (usually through splitting) Until no component can be further splitted

7 Estimating the Number of Clusters (k) Top – down (incremental) X-means (BIC criterion for 2 clusters) (Pelleg & Moore, ICML 2000)X-means G-means (1d test for Gaussianity, PCA-based projection) (Hamerly & Elkan, NIPS 2003)G-means PG-means (1d tests for GMM fitness, random projections) (Feng & Hamerly, NIPS 2006)PG-means VBGMM (Incremental Variational Bayes) (Constantinopoulos & Likas, IEEE TNN 2007)VBGMM Dip-means (test for unimodality) (Kalogeratos & Likas, NIPS 2012)Dip-means

Download ppt "Estimating the Number of Clusters (k) Selection Approaches: Use a Criterion to select among the solutions for several values of k (kmeans or GMMs are used)"

Similar presentations

University of Joensuu Dept. of Computer Science P.O. Box 111 FIN- 80101 Joensuu Tel. +358 13 251 7959 fax +358 13 251 7955 www.cs.joensuu.fi Gaussian Mixture.

University of Joensuu Dept. of Computer Science P.O. Box 111 FIN Joensuu Tel fax Gaussian Mixture.
Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.

Unsupervised Learning Clustering K-Means. Recall: Key Components of Intelligent Agents Representation Language: Graph, Bayes Nets, Linear functions Inference.
Unsupervised Learning

Unsupervised Learning
CS479/679 Pattern Recognition Dr. George Bebis

CS479/679 Pattern Recognition Dr. George Bebis
1 Bayesian Adaptation in HMM Training and Decoding Using a Mixture of Feature Transforms Stavros Tsakalidis and Spyros Matsoukas.

1 Bayesian Adaptation in HMM Training and Decoding Using a Mixture of Feature Transforms Stavros Tsakalidis and Spyros Matsoukas.
Shinichi Nakajima Sumio Watanabe　 Tokyo Institute of Technology

Shinichi Nakajima Sumio Watanabe　 Tokyo Institute of Technology
Supervised Learning Recap

Supervised Learning Recap
Machine Learning and Data Mining Clustering

Machine Learning and Data Mining Clustering
Lecture 17: Supervised Learning Recap Machine Learning April 6, 2010.

Lecture 17: Supervised Learning Recap Machine Learning April 6, 2010.
ETHEM ALPAYDIN © The MIT Press, 2010 Lecture Slides for.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, 2004 Lecture Slides for.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
10/6/2015Nikos Hourdakis, MSc Thesis1 Design and Evaluation of Clustering Approaches for Large Document Collections, The “BIC-Means” Method Nikolaos Hourdakis.

10/6/2015Nikos Hourdakis, MSc Thesis1 Design and Evaluation of Clustering Approaches for Large Document Collections, The “BIC-Means” Method Nikolaos Hourdakis.
An Introduction to Bayesian Inference Michael Betancourt April 8, 2009 8.882.

An Introduction to Bayesian Inference Michael Betancourt April 8,
Regulatory Network (Part II) 11/05/07. Methods Linear –PCA (Raychaudhuri et al. 2000) –NIR (Gardner et al. 2003) Nonlinear –Bayesian network (Friedman.

Regulatory Network (Part II) 11/05/07. Methods Linear –PCA (Raychaudhuri et al. 2000) –NIR (Gardner et al. 2003) Nonlinear –Bayesian network (Friedman.
Incremental Learning of Temporally-Coherent Gaussian Mixture Models Ognjen Arandjelović, Roberto Cipolla Engineering Department, University of Cambridge.

Incremental Learning of Temporally-Coherent Gaussian Mixture Models Ognjen Arandjelović, Roberto Cipolla Engineering Department, University of Cambridge.
Dan Phelleg, Andrew Moore Carnegie Mellon University

Dan Phelleg, Andrew Moore Carnegie Mellon University
Amos Storkey, School of Informatics. Density Traversal Clustering and Generative Kernels a generative framework for spectral clustering Amos Storkey, Tom.

Amos Storkey, School of Informatics. Density Traversal Clustering and Generative Kernels a generative framework for spectral clustering Amos Storkey, Tom.
ECE 5984: Introduction to Machine Learning

ECE 5984: Introduction to Machine Learning
Study of Sparse Online Gaussian Process for Regression EE645 Final Project May 2005 Eric Saint Georges.

Study of Sparse Online Gaussian Process for Regression EE645 Final Project May 2005 Eric Saint Georges.
(1) A probability model respecting those covariance observations: Gaussian Maximum entropy probability distribution for a given covariance observation.

(1) A probability model respecting those covariance observations: Gaussian Maximum entropy probability distribution for a given covariance observation.

Similar presentations

Ads by Google