Marjolijn Elsinga & Elze de Groot1 Markov Chains and Hidden Markov Models Marjolijn Elsinga & Elze de Groot

Marjolijn Elsinga & Elze de Groot2 Andrei A. Markov Born: 14 June 1856 in Ryazan, Russia Died: 20 July 1922 in Petrograd, Russia Graduate of Saint Petersburg University (1878) Work: number theory and analysis, continued fractions, limits of integrals, approximation theory and the convergence of series

Marjolijn Elsinga & Elze de Groot3 Todays topics Markov chains Hidden Markov models - Viterbi Algorithm - Forward Algorithm - Backward Algorithm - Posterior Probabilities

Marjolijn Elsinga & Elze de Groot4 Markov Chains (1) Emitting states

Marjolijn Elsinga & Elze de Groot5 Markov Chains (2) Transition probabilities Probability of the sequence

Marjolijn Elsinga & Elze de Groot6 Key property of Markov Chains The probability of a symbol x i depends only on the value of the preceding symbol x i-1

Marjolijn Elsinga & Elze de Groot7 Begin and End states Silent states

Marjolijn Elsinga & Elze de Groot8 Example: CpG Islands CpG = Cytosine – phosphodiester bond – Guanine 100 – 1000 bases long Cytosine is modified by methylation Methylation is suppressed in short stretches of the genome (start regions of genes) High chance of mutation into a thymine (T)

Marjolijn Elsinga & Elze de Groot9 Two questions How would we decide if a short strech of genomic sequence comes from a CpG island or not? How would we find, given a long piece of sequence, the CpG islands in it, if there are any?

Marjolijn Elsinga & Elze de Groot10 Discrimination 48 putative CpG islands are extracted Derive 2 models - regions labelled as CpG island (‘+’ model) - regions from the remainder (‘-’ model) Transition probabilities are set - Where C st + is number of times letter t follows letter s

Marjolijn Elsinga & Elze de Groot11 Maximum Likelihood Estimators Each row sums to 1 Tables are asymmetric

Marjolijn Elsinga & Elze de Groot12 Log-odds ratio

Marjolijn Elsinga & Elze de Groot13 Discrimination shown

Marjolijn Elsinga & Elze de Groot14 Simulation: ‘+’ model

Marjolijn Elsinga & Elze de Groot15 Simulation: ‘-’ model

Marjolijn Elsinga & Elze de Groot16 Todays topics Markov chains Hidden Markov models - Viterbi Algorithm - Forward Algorithm - Backward Algorithm - Posterior Probabilities

Marjolijn Elsinga & Elze de Groot17 Hidden Markov Models (HMM) (1) No one-to-one correspondence between states and symbols No longer possible to say what state the model is in when in xi Transition probability from state k to l: πi is the ith state in the path (state sequence)

Marjolijn Elsinga & Elze de Groot18 Hidden Markov Models (HMM) (2) Begin state: a 0k End state: a 0k In CpG islands example:

Marjolijn Elsinga & Elze de Groot19 Hidden Markov Models (HMM) (3) We need new set of parameters because we decoupled symbols from states Probability that symbol b is seen when in state k:

Marjolijn Elsinga & Elze de Groot20 Example: dishonest casino (1) Fair die and loaded die Loaded die: probability 0.5 of a 6 and probability 0.1 for 1-5 Switch from fair to loaded: probability 0.05 Switch back: probability 0.1

Marjolijn Elsinga & Elze de Groot21 Dishonest casino (2) Emission probabilities: HMM model that generate or emit sequences

Marjolijn Elsinga & Elze de Groot22 Dishonest casino (3) Hidden: you don’t know if die is fair or loaded Joint probability of observed sequence x and state sequence π:

Marjolijn Elsinga & Elze de Groot23 Three algorithms What is the most probable path for generating a given sequence? Viterbi Algorithm How likely is a given sequence? Forward Algorithm How can we learn the HMM parameters given a set of sequences? Forward-Backward (Baum-Welch) Algorithm

Marjolijn Elsinga & Elze de Groot24 Viterbi Algorithm CGCG can be generated on different ways, and with different probabilities Choose path with highest probability Most probable path can be found recursively

Marjolijn Elsinga & Elze de Groot25 Viterbi Algorithm (2) v k (i) = probability of most probable path ending in state k with observation i

Marjolijn Elsinga & Elze de Groot26 Viterbi Algorithm (3)

Marjolijn Elsinga & Elze de Groot27 Viterbi Algorithm Most probable path for CGCG

Marjolijn Elsinga & Elze de Groot28 Viterbi Algorithm Result with casino example

Marjolijn Elsinga & Elze de Groot29 Three algorithms What is the most probable path for generating a given sequence? Viterbi Algorithm How likely is a given sequence? Forward Algorithm How can we learn the HMM parameters given a set of sequences? Forward-Backward (Baum-Welch) Algorithm

Marjolijn Elsinga & Elze de Groot30 Forward Algorithm (1) Probability over all possible paths Number of possible paths increases exponentonial with length of sequence Forward algorithm enables us to compute this efficiently

Marjolijn Elsinga & Elze de Groot31 Forward Algorithm (2) Replacing maximisation steps for sums in viterbi algorithm Probability of observed sequence up to and including x i, requiring π i = k

Marjolijn Elsinga & Elze de Groot32 Forward Algorithm (3)

Marjolijn Elsinga & Elze de Groot33 Three algorithms What is the most probable path for generating a given sequence? Viterbi Algorithm How likely is a given sequence? Forward Algorithm How can we learn the HMM parameters given a set of sequences? Forward-Backward (Baum-Welch) Algorithm

Marjolijn Elsinga & Elze de Groot34 Backward Algorithm (1) Probability of observed sequence from xi to the end of the sequence, requiring πi = k

Marjolijn Elsinga & Elze de Groot35 Disadvantage Algorithms Multiplying many probabilities gives very small numbers which can lead to underflow errors on the computer  can be solved by doing the algorithms in log space, calculating log(v l (i))

Marjolijn Elsinga & Elze de Groot36 Backward Algorithm

Marjolijn Elsinga & Elze de Groot37 Posterior State Probability (1) Probability that observation x i came from state k, given the observed sequence Posterior probability of state k at time i when the emitted sequence is known: P(π i = k | x)

Marjolijn Elsinga & Elze de Groot38 Posterior State Probability (2) First calculate probability of producing entire observed sequence with the ith symbol being produced by state k P(x, π i = k) = f k (i) · b k (i)

Marjolijn Elsinga & Elze de Groot39 Posterior State Probability (3) Posterior probabilities will then be: P(x) is result of forward or backward calculation

Marjolijn Elsinga & Elze de Groot40 Posterior Probabilities (4) For the casino example

Marjolijn Elsinga & Elze de Groot41 Two questions How would we decide if a short strech of genomic sequence comes from a CpG island or not? How would we find, given a long piece of sequence, the CpG islands in it, if there are any?

Marjolijn Elsinga & Elze de Groot42 Prediction of CpG islands First way: Viterbi Algorithm -Find most probable path through the model -When this path goes through the ‘+’ state, a CpG island is predicted

Marjolijn Elsinga & Elze de Groot43 Prediction of CpG islands Second Way: Posterior Decoding - function: -g(k) = 1 for k Є {A +, C +, G +, T + } -g(k) = 0 for k Є {A -, C -, G -, T - } -G(i|x) is posterior probability according to the model that base i is in a CpG island

Marjolijn Elsinga & Elze de Groot44 Summary (1) Markov chain is a collection of states where a state depends only on the state before Hidden markov model is a model in which the states sequence is ‘hidden’

Marjolijn Elsinga & Elze de Groot45 Summary (2) Most probable path: viterbi algorithm How likely is a given sequence?: forward algorithm Posterior state probability: forward and backward algorithms (used for most probable state of an observation)