Download presentation
Presentation is loading. Please wait.
Published bySibyl Phelps Modified over 6 years ago
1
Combined Lecture CS621: Artificial Intelligence (lecture 19) CS626/449: Speech-NLP-Web/Topics-in-AI (lecture 20) Hidden Markov Models Pushpak Bhattacharyya Computer Science and Engineering Department IIT Bombay
2
Example : Blocks World STRIPS : A planning system – Has rules with precondition deletion list and addition list Robot hand Robot hand A C B A B C START GOAL on(B, table) on(A, table) on(C, A) hand empty clear(C) clear(B) on(C, table) on(B, C) on(A, B) hand empty clear(A)
3
Rules R1 : pickup(x) Precondition & Deletion List : handempty, on(x,table), clear(x) Add List : holding(x) R2 : putdown(x) Precondition & Deletion List : holding(x) Add List : handempty, on(x,table), clear(x)
4
Rules R3 : stack(x,y) Precondition & Deletion List :holding(x), clear(y) Add List : on(x,y), clear(x), handempty R4 : unstack(x,y) Precondition & Deletion List : on(x,y), clear(x),handempty Add List : holding(x), clear(y)
5
Plan for the block world problem
For the given problem, Start Goal can be achieved by the following sequence : Unstack(C,A) Putdown(C) Pickup(B) Stack(B,C) Pickup(A) Stack(A,B) Execution of a plan: achieved through a data structure called Triangular Table.
6
(discussion based on the book “Automated Planning” by Dana Nau)
Why Probability? (discussion based on the book “Automated Planning” by Dana Nau)
7
Motivation c a b In many situations, actions may have more than one possible outcome Action failures e.g., gripper drops its load Exogenous events e.g., road closed Would like to be able to plan in such situations One approach: Markov Decision Processes Intended outcome c a b Grasp block c a b c Unintended outcome
8
Stochastic Systems Stochastic system: a triple = (S, A, P)
S = finite set of states A = finite set of actions Pa (s | s) = probability of going to s if we execute a in s s S Pa (s | s) = 1
9
Example Robot r1 starts at location l1 Objective is to get r1 to
State s1 in the diagram Objective is to get r1 to location l4 State s4 in the diagram Start Goal
10
Example No classical plan (sequence of actions) can be a solution, because we can’t guarantee we’ll be in a state where the next action is applicable e.g., π = move(r1,l1,l2), move(r1,l2,l3), move(r1,l3,l4) Start Goal
11
Another Example A colored ball choosing example : Urn 1 # of Red = 30 # of Green = 50 # of Blue = 20 Urn 3 # of Red =60 # of Green =10 # of Blue = 30 Urn 2 # of Red = 10 # of Green = 40 # of Blue = 50 Probability of transition to another Urn after picking a ball: U1 U2 U3 0.1 0.4 0.5 0.6 0.2 0.3
12
Example (contd.) R G B U1 0.3 0.5 0.2 U2 0.1 0.4 U3 0.6 U1 U2 U3 0.1
Given : and Observation : RRGGBRGR State Sequence : ?? Not so Easily Computable.
13
Example (contd.) Here : For observation: And State sequence π is
S = {U1, U2, U3} V = { R,G,B} For observation: O ={o1… on} And State sequence Q ={q1… qn} π is U1 U2 U3 0.1 0.4 0.5 0.6 0.2 0.3 A = R G B U1 0.3 0.5 0.2 U2 0.1 0.4 U3 0.6 B=
14
Hidden Markov Models
15
Model Definition Set of states : S where |S|=N Output Alphabet : V
Transition Probabilities : A = {aij} Emission Probabilities : B = {bj(ok)} Initial State Probabilities : π
16
Markov Processes Properties
Limited Horizon :Given previous n states, a state i, is independent of preceding 0…i-n+1 states. P(Xt=i|Xt-1, Xt-2 ,… X0) = P(Xt=i|Xt-1, Xt-2… Xt-n) Time invariance : P(Xt=i|Xt-1=j) = P(X1=i|X0=j) = P(Xn=i|X0-1=j)
17
Three Basic Problems of HMM
Given Observation Sequence O ={o1… oT} Efficiently estimate P(O|λ) Get best Q ={q1… qT} i.e. Maximize P(Q|O, λ) How to adjust to best maximize Re-estimate λ
18
Three basic problems (contd.)
Problem 1: Likelihood of a sequence Forward Procedure Backward Procedure Problem 2: Best state sequence Viterbi Algorithm Problem 3: Re-estimation Baum-Welch ( Forward-Backward Algorithm )
19
Problem 2 Given Observation Sequence O ={o1… oT} Solution :
Get “best” Q ={q1… qT} i.e. Solution : Best state individually likely at a position i Best state given all the previously observed states and observations Viterbi Algorithm
20
Example Output observed – aabb
What state seq. is most probable? Since state seq. cannot be predicted with certainty, the machine is given qualification “hidden”. Note: ∑ P(outlinks) = 1 for all states
21
Probabilities for different possible seq
1 1,2 1,1 0.4 0.15 1,1,2 0.06 1,1,1 0.16 1,2,1 0.0375 1,2,2 0.0225 1,1,1,1 0.016 1,1,1,2 0.056 1,1,2,1 0.018 1,1,2,2 0.018 ...and so on
22
P(si|si-1, si-2) (order 2 HMM)
Viterbi for higher order HMM If P(si|si-1, si-2) (order 2 HMM) then the Markovian assumption will take effect only after two levels. (generalizing for n-order… after n levels)
23
Forward and Backward Probability Calculation
24
A Simple HMM r q a: 0.2 a: 0.3 b: 0.2 b: 0.1 a: 0.2 b: 0.1 b: 0.5
25
Forward or α-probabilities
Let αi(t) be the probability of producing w1,t-1, while ending up in state si αi(t)= P(w1,t-1,St=si), t>1
26
Initial condition on αi(t)
1.0 if i=1 αi(t)= 0 otherwise
27
Probability of the observation using αi(t)
P(w1,n) =Σ1 σ P(w1,n, Sn+1=si) = Σi=1 σ αi(n+1) σ is the total number of states
28
Recursive expression for α
αj(t+1) =P(w1,t, St+1=sj) =Σi=1 σ P(w1,t, St=si, St+1=sj) =Σi=1 σ P(w1,t-1, St=sj) P(wt, St+1=sj|w1,t-1, St=si) =Σi=1 σ P(w1,t-1, St=si) P(wt, St+1=sj|St=si) = Σi=1 σ αj(t) P(wt, St+1=sj|St=si)
29
The forward probabilities of “bbba”
Time Ticks 1 2 INPUT ε b bb bbb bbba 1.0 0.2 0.0 0.1 P(w,t) 0.3
30
Backward or β-probabilities
Let βi(t) be the probability of seeing wt,n, given that the state of the HMM at t is si βi(t)= P(wt,n,St=si)
31
Probability of the observation using β
P(w1,n)=β1(1)
32
Recursive expression for β
βj(t-1) =P(wt-1,n |St-1=sj) =Σj=1 σ P(wt-1,n, St=si |St-1=si) =Σi=1 σ P(wt-1, St=sj|St-1=si) P(wt,n,|wt-1,St=sj, St-1=si) =Σi=1 σ P(wt-1, St=sj|St-1=si) P(wt,n, |St=sj) (consequence of Markov Assumption) = Σj=1 σ P(wt-1, St=sj|St-1=si) βj(t)
33
Problem 1 of the three basic problems
34
Problem 1 (contd) Order 2TNT Definitely not efficient!!
Is there a method to tackle this problem? Yes. Forward or Backward Procedure
35
Forward Procedure Forward Step:
36
Forward Procedure
37
Backward Procedure
38
Backward Procedure
39
Forward Backward Procedure
Benefit Order N2T as compared to 2TNT for simple computation Only Forward or Backward procedure needed for Problem 1
40
Problem 2 Given Observation Sequence O ={o1… oT} Solution :
Get “best” Q ={q1… qT} i.e. Solution : Best state individually likely at a position i Best state given all the previously observed states and observations Viterbi Algorithm
41
Viterbi Algorithm Define such that,
i.e. the sequence which has the best joint probability so far. By induction, we have,
42
Viterbi Algorithm
43
Viterbi Algorithm
44
Problem 3 How to adjust to best maximize Solutions : Re-estimate λ
To re-estimate (iteratively update and improve) HMM parameters A,B, π Use Baum-Welch algorithm
45
Baum-Welch Algorithm Define Putting forward and backward variables
46
Baum-Welch algorithm
47
Define Then, expected number of transitions from Si And, expected number of transitions from Sj to Si
49
Baum-Welch Algorithm Baum et al have proved that the above equations lead to a model as good or better than the previous
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.