Dongfang Xu School of Information Hidden Markov Model Dongfang Xu School of Information

Outline Markov Model Hidden Markov Model Part of speech tag example Concept Discrimination Hidden Markov Model Notation & Example Three basic problem Part of speech tag example Goal & Idea

Markov model Markov Model Markov property stochastic model a linear sequence of events Markov Property Markov property the conditional probability distribution future states only depend on present state

Markov model 4 kinds of Markov Model System state partially observable System state fully observable System state partially observable System autonomous Markov Chain Hidden Markov Model System controlled Markov Decision Process Partially observable Markov decision process

Markov model Markov Chain (Visible Markov Model) Rain Dry 0.7 0.3 0.2 0.8 Two states : ‘Rain’ and ‘Dry’. Transition probabilities: P(‘Rain’|‘Rain’)=0.3 , P(‘Dry’|‘Rain’)=0.7 , P(‘Rain’|‘Dry’)=0.2, P(‘Dry’|‘Dry’)=0.8 Initial probabilities: say P(‘Rain’)=0.4 , P(‘Dry’)=0.6 . The entire state of Markov Model

Markov model Hidden Markov Model All states are unknown. Low High 0.7 0.3 0.2 0.8 All states are unknown. 2. Observation is probabilistically related with the state in Markov Model. 0.6 0.6 0.4 0.4 Rain Dry

Outline Markov Model Hidden Markov Model Part of speech tag example Concept Discrimination Hidden Markov Model Notation & Example Three basic problem Part of speech tag example Goal & Idea

Hidden Markov model Problem notation Set of states: State transition probabilities: A = {aij}, i, j ∈ S, aij= P(si | sj). Symbol emission probabilities: B=(bi (vm )), bi(vm ) = P(vm | si) Initial state probabilities: =(i), i = P(si) . State sequence X = (X1, . . . , XT+1) Xt : S {1, . . . , N} Output sequence O = (o1, . . . , oT) ot ∈ K Suppose we want to calculate a probability of a sequence of observations in our example, {‘Dry’, ’Rain’}.

Hidden Markov model Calculation of observation sequence probability Consider all possible hidden state sequences: P({‘Dry’,’Rain’} ) = P({‘Dry’,’Rain’} , {‘Low’,’Low’}) + P({‘Dry’,’Rain’} , {‘Low’,’High’}) + P({‘Dry’,’Rain’} , {‘High’,’Low’}) + P({‘Dry’,’Rain’} , {‘High’,’High’}) where first term is : P({‘Dry’,’Rain’} , {‘Low’,’Low’})= P({‘Dry’,’Rain’} | {‘Low’,’Low’}) P({‘Low’,’Low’}) = P(‘Low’) P(‘Dry’|’Low’) P(‘Low’|’Low)P(‘Rain’|’Low’) = 0.4*0.4*0.3*0.6

Hidden Markov model Three basic problem Evaluation problem. Given the HMM M=(A, B, ) and the observation sequence O=o1 o2 ... oK , calculate the probability that model M has generated sequence O . Decoding problem. Given the HMM M=(A, B, ) and the observation sequence O=o1 o2 ... oK , calculate the most likely sequence of hidden states si that produced this observation sequence O. Learning problem. Given some training observation sequences O=o1 o2 ... oK and general structure of HMM (numbers of hidden and visible states), determine HMM parameters M=(A, B, ) that best fit training data.

Outline Markov Model Hidden Markov Model Part of speech tag example Concept Discrimination Hidden Markov Model Notation & Example Three basic problem Part of speech tag example Goal & Idea

POS tag example Goal & Idea To find the most probable tag sequence for a sequence of words. Two assumptions for the words: ---words are independent of each other; --- a word’s identity only depends on its tag.

POS tag example Goal & Idea Simplified formulation Observation wi: the word at position i in the corpus Set of states ti: the tag of wi State transition probabilities: P(tk|tj) = C(tj, tk)/C(tj) Symbol emission probabilities: P(wl|tj ) =C(wl : tj )/C(tj )

POS tag example Goal & Idea Points bigram Model: P(tk|tj); Data Sparse: unseen & rare words in corpus; Training corpus & computation complexity. A combination of HMM and Visible Markov Model Reference: Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. MIT press. David D. (2003). Introduction to Hidden Markov Models [PowerPoint slides]. Retrieved from  www.cedar.buffalo.edu/~govind/CS661/Lec12.ppt

Q&A Thank you!