Valentin I. Spitkovsky April 16, 2010 CS224N Section 2: EM Valentin I. Spitkovsky April 16, 2010 (Thanks to Nate Chambers, Bill MacCartney, Jenny Finkel, and Sushant Prakash for these materials!)

Outline for today Interactive Session!! EM review (just a few slides) EM examples Weights for interpolating language models (PA1) Speaker identification (related; another spreadsheet) Word alignment for machine translation (PA2)

EM Review Observed data -- x Model of how data is generated --  Point cloud, sentences, feature vectors Model of how data is generated --  Want to perform MLE estimation of :  = arg max L(x|) = arg max ∏i p(xi|)C(xi) But this problem is typically very hard, so we introduce unobserved data -- y Class labels, clusters, speakers of sentences Easier to perform:  = arg max L(x,y|)

EM Review Steps of EM: Initialize with some model parameters  E-step: use current  to complete the unobserved data y: Weight y by p(y|x,) – soft counts! Use model parameters to fit the unobserved data M-step: use completions y to maximize model parameters: Compute arg max L(x,y|) Use completed data to fit model parameters