1. EM Algorithm in HMM and Linear Dynamical Systems by Yang Jinsan

2. 2 References An Introduction to the Kalman Filter (1999), Technical Report, by G. Welch and G. Bishop Parameter Estimation for Linear Dynamical Systems (1999), Technical Report, by Z. Gharhamani and G.E. Hinton From Hidden Markov Models to Linear Dynamical Systems (1999), Technical Report, by T.P. Minka Gaussian Process (1999), Technical Report, by D. Mackay A comparison between the EM and subspace identification algorithms for time-invariant linear dynamic systems (2000), Technical Report, by GA. Smith and AJ Robinson.

3. 3 Outline HMM  Forward Algorithm  Backward Algorithm  Baum-Welch Algorithm  EM Algorithm Linear Dynamic System  Kalman Filter  EM Algorithm HMM and Linear Dynamic System

4. 4 HMM-Forward Algorithm Question: What is the probability of having ‘Sunny’ weather today if the seaweed was dry, damp, soggy during last three days ? Answer: P (‘Sunny’ today) = Sum of all the path for P(‘Sunny’; a path to‘Sunny’) # summation = (# state)(#state)…(#state)

5. 5 Belief Propagation in Markov Chain:

6. 6 (Probability of arriving to state i k+1 with x 1 ~ x k+1 : Forward probability from i 1 ) (Where i k+1 represents the state of hidden variable in time step k+1) (Probability of starting from state i k with x k+1 ~ x N : Backward probability from i N )

7. 7 Baum-Welch re-estimation: The probability of passing through state i k with all the observations: The probability of passing through state i k and i k+1 with all the observations:

8. 8 The estimate for the number of occurrence of state i during N stages given the model S and N observations : The estimate for the number of transitions (state i  state j) during N stages given the model S and N observations : Re-estimation formulas for the unknown model parameters:

9. 9 Calculation of Likelihood: Re-estimation Algorithm (A special case of the EM algorithm) 1. Estimate indicator ( ,  ) for the transition and emission process. (E- Step) from the given HMM parameters. 2. Update S = (A, B,  ) using indicator from step 1.

10. 10 Linear Dynamic System S 1 S 2 S 3..... S N X 1 X 2 X 3..... X N

11. 11 Posterior estimation by Kalman gain K. ( V k :posterior covariance of s k | x k, : prior covariance of ) For smaller R, x is trusted more For smaller prior covariance of estimate for s, predicted measurement(estimate) using prior x is trusted more Forecast Correct Hidden State (Predict: Forward) (Update)

12. 12

13. 13 Kalman Filter

14. 14

15. 15 Step 1 (E): Given the model, estimate hidden states using Kalman filter Step 2 (M): Update parameters: A, B, R, Q,    Q 1 using the estimations of hidden state from step 1 and the log-likelihood  :

16. 16

17. 17 (From Shumway and Stoffer (1982). An approach to time series smoothing and forecasting using the EM algorithm. J. Time Series Analysis, 3(4):253-264. )

18. 18 Methods by forward and backward propagation (Minka (1999))

19. 19 Calculation of data likelihood

20. 20