1. Markov Models Brian Jackson Rob Caldwell March 9, 2010

2. Markov Property The probability of arriving at the next state depends only on which state you're in at that moment A modelling problem can be greatly simplified if we can assume the transitional probability is the only contributor to the likelihood of the next state in the chain.

3. How Markov Chains are used Given a Markov chain, two kinds of questions can be answered: 1) What is the most likely sequence produced given a number of steps and the start state(s)? [Follow the most likely transitions] 2) What is the probability of arriving in state X from state Y in a certain number of steps? Sum[ Probability Products of all Paths starting at Y and ending at X ]

4. Playing Tag What is the probability that Stefan is 'it' in two moves if Antje is 'it' now?

5. Graphical & Matrix Representation

6. Playing Tag = P(Antje->Alexandra)*P(Alexandra->Stefan) + P(Antje->Joachim)*P(Joachim->Stefan) = (0.5)(.75)+(.33)(.33) =.484

7. Another Example: CpG Islands From 48 sequences of Human DNA containing regions of CpG islands (nearly 60,000 nucleotides), two Markov Chains were produced: CpG-Positive and CpG-Negative

8. CpG Islands Log Likelihood Ratio = log ( P(x | +) / P(x | -) )

9. CpG Islands Summing the log-likelihood of each transition in CpG-positive and CpG-negative regions, then dividing by the number of molecules and plotting the result on a histogram: CpG-Positive: Dark Grey; CpG-Negative: Light Grey

10. Motor Units: Markov Chain

11. Motor Units: Most Likely Path

12. Belief Networks Two-Headed Coin A series of states connected by transitions which can be given probabilistic weights Normal Coin HeadsTails 0.5 0 1 0.80.2

13. Hidden Markov Model The observed sequence is “emitted” by one of several hidden states. The hidden states are a Markov chain where the transition probabilities are (generally) known.

14. Questions We Can Answer with a Markov Model Given a sequence of nucleotides, are there any promoter regions characterized by unusual probability of CpG dinucleotides? Is a player at casino blackjack cheating, based on his pattern of betting? Is Eddie Murphy in an acting slump, given that his last four films have been “Norbit”, another Shrek movie, a Sci-Fi movie grossing less than its budget, and a children's movie produced by Nickelodeon, in that order?

15. http://en.wikipedia.org/wiki/Hidden_Markov_model

16. Three Classes of Problems 1) Probability that a given state sequence occurs Given: Hidden state model, state-transition and observation matrices, and sequence Tool: Forward Algorithm

17. Three Classes of Problems 2) Most likely state sequence Given: Hidden state model and state-transition and observation matrices Tool: Viterbi Algorithm

18. Three Classes of Problems 3) Determine state and observation transition matrices Given: Observation data and proper hidden state topology Tool: Baum-Welch Algorithm

19. Viterbi Algorithm Given our CpG transition matrices, what is the most likely state sequence that produced observation sequence 'CGCG'?

20. Viterbi Algorithm Initialization

21. Viterbi Algorithm Recursion (1) 0.13 is 1.0 divided by 8 possible initial states

22. Viterbi Algorithm Recursion (2) 0.034 = 0.13 * P(C to G+) 0.034 = 0.13 * 0.274 0.010 = 0.13 * P(C to G-) 0.010 = 0.13 * 0.078

23. Viterbi Algorithm Recursion (3) 0.012 = 0.034 * P(G to C+) 0.0026 = 0.010 * P(G to C-)

24. Viterbi Algorithm Recursion (4) 0.0032 = 0.012 * P(C to G+) 0.00021 = 0.010 * P(C to G-)

25. Viterbi Algorithm Traceback Find most probable end-state, then trace back through all steps taken to arrive there. The most probable hidden state sequence producing CGCG is: C+ → G+ → C+ → G+