CS224N Section 2: EM Nate Chambers April 17, 2009 (Thanks to Bill MacCartney, Jenny Finkel, and Sushant Prakash for these materials)

Outline for today EM review EM examples NLP example Spreadsheet MT PA2 EM Alignment

EM Review Observed data -- x Model of how data is generated --  Point cloud, sentences, feature vectors Model of how data is generated --  Want to perform MLE estimation of :  = arg max L(x|) = arg max ∏i p(xi|)C(xi) But this problem is typically very hard, so we introduce unobserved data -- y Class labels, clusters, speakers of sentences Easier to perform:  = arg max L(x,y|)

EM Review Steps of EM: Initialize with some model parameters  E-step: use current  to calculate completions of unobserved data y: Compute y by p(y|x,) – soft counts! Use model parameters to fit the unobserved data M-step: use completions y to maximize model parameters: Compute arg max L(x,y|) Use completed data to fit model parameters