9. Gaussian Mixture Example: Start

10. After First Iteration

11. After 2nd Iteration

12. After 3rd Iteration

13. After 4th Iteration

14. After 5th Iteration

15. After 6th Iteration

16. After 20th Iteration

17. A Gaussian Mixture Model for Clustering Assume that data are generated from a mixture of Gaussian distributions For each Gaussian distribution Center:  i Variance:  (ignore) For each data point Determine membership

18. Learning Gaussian Mixture Model with the known covariance

19. Log-likelihood of Data  Apply MLE to find optimal parameters

20. Learning a Gaussian Mixture (with known covariance)

21. E-Step Learning Gaussian Mixture Model

22. M-Step Learning Gaussian Mixture Model

23. Mixture Model for Document Clustering A set of language models

24. Mixture Model for Documents Clustering A set of language models  Probability

25. A set of language models  Probability Mixture Model for Document Clustering

26. A set of language models  Probability Introduce hidden variable z ij z ij : document d i is generated by the j-th language model  j.

27. Learning a Mixture Model E-Step K: number of language models

28. Learning a Mixture Model M-Step N: number of documents