1. Class 5 Hidden Markov models

2. Markov chains Read Durbin, chapters 1 and 3 Time is divided into discrete intervals, t i At time t, system is in one of a finite set of states, x i For each pair of states, s and t, there is a probability of transition from s to t, a st

3. Example: The drunkard’s walk A drunk (D) in Lineland ‘paces’ in a room of length 4 (positions -2, -1, 0, 1, 2) In each time step, he takes one step forward or backward at random From the ‘wall’ (-1, 1), he can only take a step away

4. Transition matrix representation 2D table, with all possible states on x and y axes each cell, a st, is probability of transition from x s to x t write a transition matrix for the Lineland drunk D do you see constraints on the row, column sums?

5. Finite state machine representation Each state is a node in a graph Each (directed) edge is a transition The edge weight is the probability of the transition Draw the fsm for the Lineland drunk D

6. Begin and end states It is possible to add a begin state B to drunkard’s walk This corresponds to an entrance It is possible to add an end state E to drunkard’s walk This corresponds to an exit (trapdoor) E is usually omitted (=> walk of arbitrary length)

7. Where to find D? Assume D enters at position 0 Calculate the probability of his position at times 0.. 5 Do the probabilities stabilize? What is the meaning of the pattern which emerges?

8. What if room is odd length? Calculate for t = 0..5 (now) for room of size (-1,.. 2) HW problem: Find P i as t become large

9. Another variation If D is not at a wall, he may move left, right or not all all, with equal probability If D is at a wall, he may move away or not all all, with equal probability For room (-1, 0, 1), give the transition matrix Calculate D’s probable state at times 0.. 5

10. Keeping more history We’ve assumes that transition depends only on current state, not prior history What is assumption if next state doesn’t depend on current state? We can also make transition dependent on some amount of history

11. Training Assume some amount of history (the ‘order’ of the Markov model) We need to fill in proper parameters, e.g., the transition probabilities These are inferred from a training set of trusted data On enough data, this provides maximum likelihood (ML) parameter values

12. Hidden Markov model Assume: a Markov model of a certain ‘type’ (set of parameters) Given: a set of data Find: the set of transitions most likely (ML) to have generated the data

13. The crooked casino One good die, one crooked die 1: 1/6 2: 1/6 3: 1/6 4: 1/6 5: 1/6 6: 1/6 1: 1/10 2: 1/10 3: 1/10 4: 1/10 5: 1/10 6: 1/2 0.05 0.1 0.95 0.9

14. What we see in Atlantic City 6251445436316566265 Two states: fair die (F) and loaded die (L) What we see: a sequence of rolls of the die What is hidden: the switching of the die (the state) We want to guess which die was most likely being used for each roll

15. Terminology Symbols:6251445436316566265 States: FFFFFFFFFFFFLLLLLLL The sequence of symbols is denoted x The sequence of states is called the path  The path is a Markov chain Probability of rolling b {1..6} with die k {F,L} is the emission probability e k (b) = P(x i = b |  i = k)

16. Probability of (x,  ) P(x,  ) = a 0  1  e  i (x i )a  i  i+1 So, finding the likelihood of a given sequence and path is easy Does this tell us when the casino switched dice? L i=1

17. Viterbi algorithm Intuition: find the most likely path for the observed sequence Method: backtracking 2D array –row: symbol for all states –column: observed symbol

18. V 6 3 2 B 1 0 1f 0 0 2f 0 0 3f 0 0 4f 0 0 5f 0 0 6f 0 0.16 1l 0 0 2l 0 0 3l 0 0 4l 0 0 5l 0 0 6l 0 0.5