Hidden Markov Models (HMM) Rabiner’s Paper Markoviana Reading Group Computer Eng. & Science Dept. Arizona State University

Stationary and Non-stationary Stationary Process: Its statistical properties do not vary with time Non-stationary Process: The signal properties vary over time Markoviana Reading Group Fatih Gelgi – Feb, 2005

HMM Example - Casino Coin 0.9 Two CDF tables 0.2 0.1 Fair Unfair State transition Pbbties. States 0.8 Symbol emission Pbbties. 0.5 0.5 0.3 0.7 Observation Symbols H T H T Observation Sequence HTHHTTHHHTHTHTHHTHHHHHHTHTHH FFFFFFUUUFFFFFFUUUUUUUFFFFFF State Sequence Motivation: Given a sequence of H & Ts, can you tell at what times the casino cheated? Markoviana Reading Group Fatih Gelgi – Feb, 2005

Properties of an HMM First-order Markov process Time is discrete qt only depends on qt-1 Time is discrete Markoviana Reading Group Fatih Gelgi – Feb, 2005

Elements of an HMM N, the number of States M, the number of Symbols States S1, S2, … SN Observation Symbols O1, O2, … OM l, the Probability Distributions a, b, p Markoviana Reading Group Fatih Gelgi – Feb, 2005

HMM Basic Problems Given an observation sequence O=O1O2O3…OT and l, find P(O|l) Forward Algorithm / Backward Algorithm Given O=O1O2O3…OT and l, find most likely state sequence Q=q1q2…qT Viterbi Algorithm Given O=O1O2O3…OT and l, re-estimate l so that P(O|l) is higher than it is now Baum-Welch Re-estimation Markoviana Reading Group Fatih Gelgi – Feb, 2005

Forward Algorithm Illustration at(i) is the probability of observing a partial sequence O1O2O3…Ot such that the state Si. Markoviana Reading Group Fatih Gelgi – Feb, 2005

Forward Algorithm Illustration (cont’d) at(i) is the probability of observing a partial sequence O1O2O3…Ot such that the state Si. Total of this column gives solution State Sj SN pNbN(O1) S (a1(i) aiN) bN(O2) … S6 p6b6(O1) S (a1(i) ai6) b6(O2) S5 p5b5(O1) S (a1(i) ai5) b5(O2) S4 p4b4(O1) S (a1(i) ai4) b4(O2) S3 p3b3(O1) S (a1(i) ai3) b3(O2) S2 p2b2(O1) S (a1(i) ai2) b2(O2) S1 p1b1(O1) S (a1(i) ai1) b1(O2) at(j) O1 O2 O3 O4 OT Observations Ot Markoviana Reading Group Fatih Gelgi – Feb, 2005

Forward Algorithm Definition: Initialization: Induction: Problem 1 Answer: at(i) is the probability of observing a partial sequence O1O2O3…Ot such that the state Si. Complexity: O(N2T) Markoviana Reading Group Fatih Gelgi – Feb, 2005

Backward Algorithm Illustration t(i) is the probability of observing a partial sequence Ot+1Ot+2Ot+3…OT such that the state Si. Markoviana Reading Group Fatih Gelgi – Feb, 2005

Backward Algorithm Definition: Initialization: Induction: t(i) is the probability of observing a partial sequence Ot+1Ot+2Ot+3…OT such that the state Si. Markoviana Reading Group Fatih Gelgi – Feb, 2005

Q2: Optimality Criterion 1 * Maximize the expected number of correct individual states Definition: Initialization: Problem 2 Answer: t(i) is the probability of being in state Si at time t given the observation sequence O and the model . Problem: If some aij=0, the optimal state sequence may not even be a valid state sequence. Markoviana Reading Group Fatih Gelgi – Feb, 2005

Q2: Optimality Criterion 2 * Find the single best state sequence (path), i.e. maximize P(Q|O,). Definition: dt(i) is the highest probability of a state path for the partial observation sequence O1O2O3…Ot such that the state Si. Markoviana Reading Group Fatih Gelgi – Feb, 2005

Viterbi Algorithm The major difference from the forward algorithm: Maximization instead of sum Markoviana Reading Group Fatih Gelgi – Feb, 2005

Viterbi Algorithm Illustration dt(i) is the highest probability of a state path for the partial observation sequence O1O2O3…Ot such that the state Si. Max of this col indicates traceback start State Sj SN pN bN(O1) max [d1(i) aiN] bN(O2) … S6 p6 b6(O1) max [d1(i) ai6] b6(O2) S5 p5 b5(O1) max [d1(i) ai5] b5(O2) S4 p4 b4(O1) max [d1(i) ai4] b4(O2) S3 p3 b3(O1) max [d1(i) ai3] b3(O2) S2 p2 b2(O1) max [d1(i) ai2] b2(O2) S1 p1 b1(O1) max [d1(i) ai1] b1(O2) dt(j) O1 O2 O3 O4 OT Observations Ot Markoviana Reading Group Fatih Gelgi – Feb, 2005

Relations with DBN Forward Function: Backward Function: Viterbi Algorithm: t+1(j) bj(Ot+1) aij t(i) t(i) bj(Ot+1) t+1(j) aij T(i)=1 t+1(j) bj(Ot+1) aij t(i) Markoviana Reading Group Fatih Gelgi – Feb, 2005

Some more definitions gt(i) is the probability of being in state Si at time t xt(i,j) is the probability of being in state Si at time t, and Sj at time t+1 Markoviana Reading Group Fatih Gelgi – Feb, 2005

Baum-Welch Re-estimation Expectation-Maximization Algorithm Expectation: Markoviana Reading Group Fatih Gelgi – Feb, 2005

Baum-Welch Re-estimation (cont’d) Maximization: Markoviana Reading Group Fatih Gelgi – Feb, 2005

Notes on the Re-estimation If the model does not change, it means that it has reached a local maxima. Depending on the model, many local maxima can exist Re-estimated probabilities will sum to 1 Markoviana Reading Group Fatih Gelgi – Feb, 2005

Implementation issues Scaling Multiple observation sequences Initial parameter estimation Missing data Choice of model size and type Markoviana Reading Group Fatih Gelgi – Feb, 2005

Scaling calculation: Recursion to calculate: Markoviana Reading Group Fatih Gelgi – Feb, 2005

Scaling (cont’d) calculation: Desired condition: * Note that is not true! Markoviana Reading Group Fatih Gelgi – Feb, 2005

Scaling (cont’d) Markoviana Reading Group Fatih Gelgi – Feb, 2005

Maximum log-likelihood Initialization: Recursion: Termination: Markoviana Reading Group Fatih Gelgi – Feb, 2005

Multiple observations sequences Problem with re-estimation Markoviana Reading Group Fatih Gelgi – Feb, 2005

Initial estimates of parameters For  and A, Random or uniform is sufficient For B (discrete symbol prb.), Good initial estimate is needed Markoviana Reading Group Fatih Gelgi – Feb, 2005

Insufficient training data Solutions: Increase the size of training data Reduce the size of the model Interpolate parameters using another model Markoviana Reading Group Fatih Gelgi – Feb, 2005

References L Rabiner. ‘A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition.’ Proceedings of the IEEE 1989. S Russell, P Norvig. ‘Probabilistic Reasoning Over Time’. AI: A Modern Approach, Ch.15, 2002 (draft). V Borkar, K Deshmukh, S Sarawagi. ‘Automatic segmentation of text into structured records.’ ACM SIGMOD 2001. T Scheffer, C Decomain, S Wrobel. ‘Active Hidden Markov Models for Information Extraction.’ Proceedings of the International Symposium on Intelligent Data Analysis 2001. S Ray, M Craven. ‘Representing Sentence Structure in Hidden Markov Models for Information Extraction.’  Proceedings of the 17th International Joint Conference on Artificial Intelligence 2001. Markoviana Reading Group Fatih Gelgi – Feb, 2005