1. Part 4 b Forward-Backward Algorithm & Viterbi Algorithm CSE717, SPRING 2008 CUBS, Univ at Buffalo

2. Production Probability Computation of : Summing over all possible state sequence s Time complexity

3. Forward Variable Define can be computed by Forward Algorithm

4. Initial Values of Forward Variables

5. Recursive Computation of Forward Variables O1O1 s1s1 stst OtOt s t+1 O t+1

6. Forward Algorithm Let 1. Initialization 2. Recursion for all times t, t=1,…,T–1 3. Termination Complexity

7. Backward Variable Define Posterior probability of state i at time t

8. Backward Algorithm Let 1. Initialization 2. Recursion for all times t, t=T–1,…,1 3. Termination

9. Decoding Problem (Find a state sequence s* that produces the observations with maximal posterior probability) 12 N 312 N 312 N 312 N 3

10. Partially Optimal Path Probability Define s* can be obtained from partially optimal paths given by the computation of

11. Viterbi Algorithm Define s* can be obtained from partially optimal paths given by the computation of

12. Viterbi Algorithm Let 1. Initialization 2. Recursion for all times t, t=1,…,T–1 3. Termination 4. Back-Tracking of Optimal Path for all times t, t= T–1,…,1

13. Configuration of Forward-Backward Algorithm and Viterbi Algorithm Configuration Forward algorithm computes Viterbi Algorithm computes Parallel Implementation

14. Discussion Problem in Part 3 Question: given N samples x 1, …, x N of a mixture density of two normal distributions can you get maximum likelihood estimators of parameters ?

15. Single Gaussian The log likelihood is a polynomial and can be maximized by means of taking differentials. Mixture of 2-Gaussian Exponential cannot be removed by taking log () It’s hard to find expressions of ML estimators

16. Discussion Problem in Part 3 Question: given N samples x 1, …, x N of a mixture density of two normal distributions can you get maximum likelihood estimators of parameters ? Conclusion: No analytic solution, but numerical solutions may exist