1. CS344: Introduction to Artificial Intelligence (associated lab: CS386) Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 6-7: Hidden Markov Model 18 th and 20 th Jan, 2011

2. Observations leading to why probability is needed Many intelligence tasks are sequence labeling tasks Tasks carried out in layers Within a layer, there are limited windows of information This naturally calls for strategies for dealing with uncertainty Probability and Markov process give a way

3. “I went with my friend to the bank to withdraw some money, but was disappointed to find it closed” POS Sense Pronoun drop SCOPE Co-referencing Bank (N/V)closed (V/ adj) Bank (financial institution)withdraw (take away) But I/friend/money/bank was disappointed With my friend It -> bank

4. HMM

5. A Motivating Example Urn 1 # of Red = 30 # of Green = 50 # of Blue = 20 Urn 3 # of Red =60 # of Green =10 # of Blue = 30 Urn 2 # of Red = 10 # of Green = 40 # of Blue = 50 Colored Ball choosing U1U1 U2U2 U3U3 U1U1 0.10.40.5 U2U2 0.60.2 U3U3 0.30.40.3 Probability of transition to another Urn after picking a ball:

6. Example (contd.) U1U1 U2U2 U3U3 U1U1 0.10.40.5 U2U2 0.60.2 U3U3 0.30.40.3 Given : Observation : RRGGBRGR State Sequence : ?? Not so Easily Computable. and RGB U1U1 0.30.50.2 U2U2 0.10.40.5 U3U3 0.60.10.3

7. Diagrammatic representation (1/2) U1U1 U2U2 U3U3 0.1 0.2 0.4 0.6 0.4 0.5 0.3 0.2 0.3 R, 0.6 G, 0.1 B, 0.3 R, 0.1 B, 0.5 G, 0.4 B, 0.2 R, 0.3 G, 0.5

8. Diagrammatic representation (2/2) U1U1 U2U2 U3U3 R,0.02 G,0.08 B,0.10 R,0.24 G,0.04 B,0.12 R,0.06 G,0.24 B,0.30 R, 0.08 G, 0.20 B, 0.12 R,0.15 G,0.25 B,0.10 R,0.18 G,0.03 B,0.09 R,0.18 G,0.03 B,0.09 R,0.02 G,0.08 B,0.10 R,0.03 G,0.05 B,0.02

9. Example (contd.) Here : S = {U1, U2, U3} V = { R,G,B} For observation: O ={o 1 … o n } And State sequence Q ={q 1 … q n } π is U1U1 U2U2 U3U3 U1U1 0.10.40.5 U2U2 0.60.2 U3U3 0.30.40.3 RGB U1U1 0.50.2 U2U2 0.10.40.5 U3U3 0.60.10.3 A = B=

10. Observations and states O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 OBS: RRG G B R G R State: S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S i = U 1 /U 2 /U 3 ; A particular state S: State sequence O: Observation sequence S* = “best” possible state (urn) sequence Goal: Maximize P(S*|O) by choosing “best” S

11. Goal Maximize P(S|O) where S is the State Sequence and O is the Observation Sequence

12. False Start By Markov Assumption (a state depends only on the previous state) O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 OBS: RRG G B R G R State: S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8

13. Baye’s Theorem P(A) -: Prior P(B|A) -: Likelihood

14. State Transitions Probability By Markov Assumption (k=1)

15. Observation Sequence probability Assumption that ball drawn depends only on the Urn chosen

16. Grouping terms P(S).P(O|S) =[P(O 0 |S 0 ).P(S 1 |S 0 )]. [P(O 1 |S 1 ).P(S 2 |S 1 )]. [P(O 2 |S 2 ).P(S 3 |S 2 )]. [P(O 3 |S 3 ).P(S 4 |S 3 )]. [P(O 4 |S 4 ).P(S 5 |S 4 )]. [P(O 5 |S 5 ).P(S 6 |S 5 )]. [P(O 6 |S 6 ).P(S 7 |S 6 )]. [P(O 7 |S 7 ).P(S 8 |S 7 )]. [P(O 8 |S 8 ).P(S 9 |S 8 )]. We introduce the states S 0 and S 9 as initial and final states respectively. After S 8 the next state is S 9 with probability 1, i.e., P(S 9 |S 8 )=1 O 0 is ε-transition O 0 O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 Obs: ε RRG G B R G R State: S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9

17. Introducing useful notation S0S0 S1S1 S8S8 S7S7 S9S9 S2S2 S3S3 S4S4 S5S5 S6S6 O 0 O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8 Obs: ε RRG G B R G R State: S 0 S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 ε R R GGBR G R P(O k |S k ).P(S k+1 |S k )=P(S k  S k+1 ) OkOk

18. Probabilistic FSM (a 1 :0.3) (a 2 :0.4) (a 1 :0.2) (a 2 :0.3) (a 1 :0.1) (a 2 :0.2) (a 1 :0.3) (a 2 :0.2) The question here is: “what is the most likely state sequence given the output sequence seen” S1S1 S2S2

19. Developing the tree Start S1S2S1S2S1S2S1S2S1S2 1.00.0 0.1 0.30.2 0.3 1*0.1=0.1 0.30.0 0.1*0.2=0.02 0.1*0.4=0.040.3*0.3=0.090.3*0.2=0.06.... € a1a1 a2a2 Choose the winning sequence per state per iteration 0.2 0.40.3 0.2

20. Tree structure contd… S1S2S1S2S1S2 0.1 0.30.2 0.3 0.0270.012.. 0.090.06 0.09*0.1=0.009 0.018 S1 0.3 0.0081 S2 0.2 0.0054 S2 0.4 0.0048 S1 0.2 0.0024. a1a1 a2a2 The problem being addressed by this tree is a1-a2-a1-a2 is the output sequence and μ the model or the machine

21. Path found : (working backward) S1S1 S2S2 S1S1 S2S2 S1S1 a2a2 a1a1 a1a1 a2a2 Problem statement : Find the best possible sequence Start symbolState collectionAlphabet set Transitions T is defined as