1. www.kingston.ac.uk/dirc Tutorial on Hidden Markov Models

2. www.kingston.ac.uk/dirc Overview Markov chains Mixture Models Hidden Markov Model Definition Three basic problems Issues

3. www.kingston.ac.uk/dirc Markov chain: an example Weather model: 3 states {rainy, cloudy, sunny} Problem: Forecast weather state, based on the current weather state

4. www.kingston.ac.uk/dirc Markov chain – Model Definition N States, {S 1, S 2,… S N } Sequence of states Q ={q 1, q 2,…} Initial probabilities π={π 1, π 2,… π N } π i =P(q 1 =S i ) Transition matrix A NxN a ij =P(q t+1 =S j | q t =S i )

5. www.kingston.ac.uk/dirc Mixture Models: an example Weather model: 3 “hidden” states {rainy, cloudy, sunny} Measure weather-related variables (e.g. temperature, humidity, barometric pressure) Problem: Given the values of the weather variables, what is the state?

6. www.kingston.ac.uk/dirc Gaussian Mixture Model Definition Ν states observed through an observation x Model parameter θ={p 1 …p N, μ 1...μ Ν, Σ 1...Σ Ν }

7. www.kingston.ac.uk/dirc HMM: an example Weather model: 3 “hidden” states {rainy, cloudy, sunny} Measure weather-related variables (e.g. temperature, humidity, barometric pressure) Problem: Forecast the weather state, given the current weather variables

8. www.kingston.ac.uk/dirc Hidden Markov Model Definition (1/2) N “hidden” States, {S 1, S 2,… S N } Sequence of states Q ={q 1, q 2,…} Sequence of observations O={O 1, O 2, …}

9. www.kingston.ac.uk/dirc Hidden Markov Model Definition (2/2) λ=(A, B, π): Hidden Markov Model A={a ij }: State transition probabilities a ij =P(q t+1 =S j | q t =S i ) π={π i }: initial state distribution π i =P(q 1 =S i ) Β={b i (v)}: Observation probability distribution b i (v)=P(O t =v | q t =S i ) Similar to Markov Chain Similar to Mixture Model

10. www.kingston.ac.uk/dirc HMM Graph Similar to Markov Chain Similar to Mixture Model

11. www.kingston.ac.uk/dirc The three basic problems Evaluation O, λ → P(O|λ) Uncover the hidden part O, λ → Q that P(Q|O, λ) is maximum Learning {Ο} → λ that P(O|λ) is maximum

12. www.kingston.ac.uk/dirc Evaluation O, λ → P(O|λ) Solved by using the forward- backward procedure Applications Evaluation of a sequence of observations Find most suitable HMM Used in the other two problems

13. www.kingston.ac.uk/dirc Uncover the hidden part O, λ → Q that P(Q|O, λ) is maximum Solved by Viterbi algorithm Applications Find the real states Learn about the structure of the model Estimate statistics of the states Used in the learning problem

14. www.kingston.ac.uk/dirc Learning {Ο} → λ that P(O|λ) is maximum No analytic solution Usually solved by Baum-Welch (EM variation) Applications Unsupervised Learning (single HMM) Supervised Learning (multiple HMM)

15. www.kingston.ac.uk/dirc Some issues Limitations imposed by Markov chain Mixture model Scalability Learning Initialisation Model order

16. www.kingston.ac.uk/dirc Questions?