1. Valentin I. Spitkovsky April 16, 2010
CS224N Section 2: EM
Valentin I. Spitkovsky
April 16, 2010
(Thanks to Nate Chambers, Bill MacCartney, Jenny Finkel, and Sushant Prakash for these materials!)

2. Outline for today Interactive Session!! EM review (just a few slides)
EM examples
Weights for interpolating language models (PA1)
Speaker identification (related; another spreadsheet)
Word alignment for machine translation (PA2)

3. EM Review Observed data -- x Model of how data is generated -- 
Point cloud, sentences, feature vectors
Model of how data is generated -- 
Want to perform MLE estimation of :
 = arg max L(x|) = arg max ∏i p(xi|)C(xi)
But this problem is typically very hard, so we introduce unobserved data -- y
Class labels, clusters, speakers of sentences
Easier to perform:  = arg max L(x,y|)

4. EM Review Steps of EM: Initialize with some model parameters 
E-step: use current  to complete the unobserved data y:
Weight y by p(y|x,) – soft counts!
Use model parameters to fit the unobserved data
M-step: use completions y to maximize model parameters:
Compute arg max L(x,y|)
Use completed data to fit model parameters