K-means conditional mixture models Lecture 14 K-means conditional mixture models

K-means VEM is a general bound optimization algorithm where we can choose parameterized posteriors. If we use delta functions instead of general responsibilities in MoG, we get the k-means algorithm. At each E-step we are forced to pick a winner cluster, instead of soft assignments. We minimize a cost function, but not the log-likelihood of the MoG model, so we wont get ML parameters. K-means is typically fast, but prone to local minima.

Conditional Mixture Models Recall the generative and discriminative methods for classification/regression. We will now make these model more flexible by using mixture models. Generative: p(x|z,y) p(z|y) p(y) Discriminative: p(y|z,x) p(z|x). p(z|x) are soft-swithes that are input dependent. They switch between the different models. p(y|z,x) are expert-models, such as linear regression models, but different for each value of z. For regression we get soft piecewise linear curve fitting (with uncertainty). Use EM to learn the parameters.