1. CS621: Artificial Intelligence Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 33,34– HMM, Viterbi, 14 th Oct, 18 th Oct, 2010

2. HMM Definition Set of states : S where |S|=N Output Alphabet : O where |O|=K Transition Probabilities : A = {a ij } Emission Probabilities : B = {b j (o k )} Initial State Probabilities : π

3. Markov Processes Properties Limited Horizon: Given previous t states, a state i, is independent of preceding 0 to t- k+1 states. P(X t =i|X t-1, X t-2,… X 0 ) = P(X t =i|X t-1, X t-2 … X t-k ) Order k Markov process Time invariance: (shown for k=1) P(X t =i|X t-1 =j) = P(X 1 =i|X 0 =j) …= P(X n =i|X n-1 =j)

4. Three basic problems (contd.) Problem 1: Likelihood of a sequence Forward Procedure Backward Procedure Problem 2: Best state sequence Viterbi Algorithm Problem 3: Re-estimation Baum-Welch ( Forward-Backward Algorithm )

5. Probabilistic Inference O: Observation Sequence S: State Sequence Given O find S * where called Probabilistic Inference Infer “Hidden” from “Observed” How is this inference different from logical inference based on propositional or predicate calculus?

6. Essentials of Hidden Markov Model 1. Markov + Naive Bayes 2. Uses both transition and observation probability 3. Effectively makes Hidden Markov Model a Finite State Machine (FSM) with probability

7. Probability of Observation Sequence Without any restriction, Search space size= |S| |O|

8. Continuing with the Urn example Urn 1 # of Red = 30 # of Green = 50 # of Blue = 20 Urn 3 # of Red =60 # of Green =10 # of Blue = 30 Urn 2 # of Red = 10 # of Green = 40 # of Blue = 50 Colored Ball choosing

9. Example (contd.) U1U1 U2U2 U3U3 U1U1 0.10.40.5 U2U2 0.60.2 U3U3 0.30.40.3 Given : Observation : RRGGBRGR What is the corresponding state sequence ? and RGB U1U1 0.30.50.2 U2U2 0.10.40.5 U3U3 0.60.10.3 Transition Probability Observation/output Probability

10. Diagrammatic representation (1/2) U1U1 U2U2 U3U3 0.1 0.2 0.4 0.6 0.4 0.5 0.3 0.2 0.3 R, 0.6 G, 0.1 B, 0.3 R, 0.1 B, 0.5 G, 0.4 B, 0.2 R, 0.3 G, 0.5

11. Diagrammatic representation (2/2) U1U1 U2U2 U3U3 R,0.02 G,0.08 B,0.10 R,0.24 G,0.04 B,0.12 R,0.06 G,0.24 B,0.30 R, 0.08 G, 0.20 B, 0.12 R,0.15 G,0.25 B,0.10 R,0.18 G,0.03 B,0.09 R,0.18 G,0.03 B,0.09 R,0.02 G,0.08 B,0.10 R,0.03 G,0.05 B,0.02

12. Observations and states O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 OBS: RRG G B R G R State: S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S i = U 1 /U 2 /U 3 ; A particular state S: State sequence O: Observation sequence S* = “best” possible state (urn) sequence Goal: Maximize P(S*|O) by choosing “best” S

13. Goal Maximize P(S|O) where S is the State Sequence and O is the Observation Sequence

14. Baye’s Theorem P(A) -: Prior P(B|A) -: Likelihood

15. State Transitions Probability By Markov Assumption (k=1)

16. Observation Sequence probability Assumption that ball drawn depends only on the Urn chosen

17. Grouping terms P(S).P(O|S) =[P(O 0 |S 0 ).P(S 1 |S 0 )]. [P(O 1 |S 1 ).P(S 2 |S 1 )]. [P(O 2 |S 2 ).P(S 3 |S 2 )]. [P(O 3 |S 3 ).P(S 4 |S 3 )]. [P(O 4 |S 4 ).P(S 5 |S 4 )]. [P(O 5 |S 5 ).P(S 6 |S 5 )]. [P(O 6 |S 6 ).P(S 7 |S 6 )]. [P(O 7 |S 7 ).P(S 8 |S 7 )]. [P(O 8 |S 8 ).P(S 9 |S 8 )]. We introduce the states S 0 and S 9 as initial and final states respectively. After S 8 the next state is S 9 with probability 1, i.e., P(S 9 |S 8 )=1 O 0 is ε-transition O 0 O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 Obs: ε RRG G B R G R State: S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9

18. Introducing useful notation S0S0 S1S1 S8S8 S7S7 S9S9 S2S2 S3S3 S4S4 S5S5 S6S6 O 0 O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 Obs: ε RRG G B R G R State: S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 ε R R GGBR G R P(O k |S k ).P(S k+1 |S k )=P(S k  S k+1 ) OkOk

19. Viterbi Algorithm for the Urn problem (first two symbols) S0S0 U1U1 U2U2 U3U3 0.5 0.3 0.2 U1U1 U2U2 U3U3 0.03 0.08 0.15 U1U1 U2U2 U3U3 U1U1 U2U2 U3U3 0.06 0.02 0.18 0.24 0.18 0.015 0.040.075*0.0180.006 0.048*0.036 *: winner sequences ε R

20. Markov process of order>1 (say 2) Same theory works P(S).P(O|S) =P(O 0 |S 0 ).P(S 1 |S 0 ). [P(O 1 |S 1 ).P(S 2 |S 1 S 0 )]. [P(O 2 |S 2 ).P(S 3 |S 2 S 1 )]. [P(O 3 |S 3 ).P(S 4 |S 3 S 2 )]. [P(O 4 |S 4 ).P(S 5 |S 4 S 3 )]. [P(O 5 |S 5 ).P(S 6 |S 5 S 4 )]. [P(O 6 |S 6 ).P(S 7 |S 6 S 5 )]. [P(O 7 |S 7 ).P(S 8 |S 7 S 6 )]. [P(O 8 |S 8 ).P(S 9 |S 8 S 7 )]. We introduce the states S 0 and S 9 as initial and final states respectively. After S 8 the next state is S 9 with probability 1, i.e., P(S 9 |S 8 S 7 )=1 O 0 is ε-transition O 0 O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 Obs: ε RRG G B R G R State: S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9

21. Adjustments Transition probability table will have tuples on rows and states on columns Output probability table will remain the same In the Viterbi tree, the Markov process will take effect from the 3 rd input symbol (εRR) There will be 27 leaves, out of which only 9 will remain Sequences ending in same tuples will be compared Instead of U1, U2 and U3 U 1 U 1, U 1 U 2, U 1 U 3, U 2 U 1, U 2 U 2,U 2 U 3, U 3 U 1,U 3 U 2,U 3 U 3

22. Probabilistic FSM (a 1 :0.3) (a 2 :0.4) (a 1 :0.2) (a 2 :0.3) (a 1 :0.1) (a 2 :0.2) (a 1 :0.3) (a 2 :0.2) The question here is: “what is the most likely state sequence given the output sequence seen” S1S1 S2S2

23. Developing the tree Start S1S2S1S2S1S2S1S2S1S2 1.00.0 0.1 0.30.2 0.3 1*0.1=0.1 0.30.0 0.1*0.2=0.02 0.1*0.4=0.040.3*0.3=0.090.3*0.2=0.06.... € a1a1 a2a2 Choose the winning sequence per state per iteration 0.2 0.40.3 0.2

24. Tree structure contd… S1S2S1S2S1S2 0.1 0.30.2 0.3 0.0270.012.. 0.090.06 0.09*0.1=0.009 0.018 S1 0.3 0.0081 S2 0.2 0.0054 S2 0.4 0.0048 S1 0.2 0.0024. a1a1 a2a2 The problem being addressed by this tree is a1-a2-a1-a2 is the output sequence and μ the model or the machine

25. Path found : (working backward) S1S1 S2S2 S1S1 S2S2 S1S1 a2a2 a1a1 a1a1 a2a2 Problem statement : Find the best possible sequence Start symbolState collectionAlphabet set Transitions T is defined as

26. Tabular representation of the tree €a1a1 a2a2 a1a1 a2a2 S1S1 1.0(1.0*0.1,0.0*0.2 )=(0.1,0.0) (0.02, 0.09) (0.009, 0.012)(0.0024, 0.0081) S2S2 0.0(1.0*0.3,0.0*0.3 )=(0.3,0.0) (0.04,0.0 6) (0.027,0.018)(0.0048,0.005 4) Ending state Latest symbol observed Note: Every cell records the winning probability ending in that state Final winner The bold faced values in each cell shows the sequence probability ending in that state. Going backward from final winner sequence which ends in state S2 (indicated By the 2 nd tuple), we recover the sequence.

27. Algorithm (following James Alan, Natural Language Understanding (2 nd edition), Benjamin Cummins (pub.), 1995 Given: 1. The HMM, which means: a. Start State: S 1 b. Alphabet: A = {a 1, a 2, … a p } c. Set of States: S = {S 1, S 2, … S n } d. Transition probability which is equal to 2. The output string a 1 a 2 …a T To find: The most likely sequence of states C 1 C 2 …C T which produces the given output sequence, i.e., C 1 C 2 …C T =

28. Algorithm contd… Data Structure: 1. A N*T array called SEQSCORE to maintain the winner sequence always (N=#states, T=length of o/p sequence) 2. Another N*T array called BACKPTR to recover the path. Three distinct steps in the Viterbi implementation 1. Initialization 2. Iteration 3. Sequence Identification

29. 1.Initialization SEQSCORE(1,1)=1.0 BACKPTR(1,1)=0 For(i=2 to N) do SEQSCORE(i,1)=0.0 [ expressing the fact that first state is S 1 ] 2.Iteration For(t=2 to T) do For(i=1 to N) do SEQSCORE(i,t) = Max (j=1,N) BACKPTR(I,t) = index j that gives the MAX above

30. 3.Seq. Identification C(T) = i that maximizes SEQSCORE(i,T) For i from (T-1) to 1 do C(i) = BACKPTR[C(i+1),(i+1)] Optimizations possible: 1.BACKPTR can be 1*T 2.SEQSCORE can be T*2 Homework:- Compare this with A*, Beam Search [Homework] Reason for this comparison: Both of them work for finding and recovering sequence