K-means and Gaussian Mixture Model 王养浩 2013 年 11 月 20 日

Outline K-means Gaussian Mixture Model Expectation Maximum

K-means Gather data points to a few cohesive ‘Clusters’ Unsupervised Learning

K-means

Easy Fast Euclidean distance? K needs input ？ Convergence ？

Determination of K Rule of Thumb ： Elbow Method Cross Validation

K-means Convergence x (i) data points μ c(i) cluster centroids Coordinate descent

Coordinate Descent

K-means Convergence Non-circle Clusters

K-means Convergence Local minimum – The optimization object is non-convex

Gaussian Mixture Model Mixture of Gaussian distribution

Gaussian Mixture Model Log likelihood Maximum likelihood – Expectation Maximum

Expectation Maximum

Jenson inquality

Expectation Maximum

Construct lower bound

Expectation Maximum

Repeat until convergence

Generalized Expectation Maximum Difficulty in M-step

Summary K-means – Coordinate descent Gaussian Mixture Model – Expectation Maximum Expectation Maximum – MLE for models with latent variables – Generalized EM

Thanks!