1. 1

2. Markov Processes (Briefly)‏
We begin with data generated from a Markov process.
The process has several states (loaded and unloaded dice) which behave differently.
The goal of the HMM is to estimate which state we were in for each data point. 2

3. Markov Processes (Cont.)‏
The probabilistic switching between states is described by a transition matrix, A.
The behavior of the states is described by an emission matrix, b. 3

4. Forward-Backward Algorithm
Takes an initial estimate of the A and b matrices and constructs an estimate of the likelihood that we were in a given state at each data point.
It does this by finding the likelihood of the data up to time t, given that we are in state i at time t, denoted alpha:
It also finds the likelihood of the data after t given that we are in state i at time t. This is denoted beta: 4

5. Baum-Welch Re-Estimation
Uses the initial guesses at A and b and the calculated alpha and beta values to re- estimate A and b.
It's magic!
When used iteratively, this often converges to the right answer, even if the initial guesses are random!
It's SCIENCE! 5

6. HMM Solution to the Dice Problem6

7. 7

8. HMM Solution to the Dice Problem8

9. Errors in Estimated Parameters
Finite Data set, ~10 transitions
Simpler problem: assuming A and B are mostly correct, what are the errors due to counting statistics?
Use A and the total number of rolls, M, to determine how many rolls you would observe for each die.
“Limiting distribution” X is the normalized left eigenvector of A with an eigenvalue of 1.
Experimental physics trick: count N events, set the error to sqrt(N). 9

10. Errors in Estimated Parameters?10

11. Errors in Estimated Parameters?11

12. Errors in Estimated Parameters?12

13. 13

14. 14

15. Daily Temperature as a Markov Process?15

16. Daily Temperature as a Markov Process?16

17. Daily Temperature as a Markov Process?17

18. Daily Temperature as a Markov Process?18

19. 19