Markov chains Assume a gene that has three alleles A, B, and C. These can mutate into each other. Transition probabilities Transition matrix Probability.

Slides:



Advertisements
Similar presentations
Divide-and-Conquer CIS 606 Spring 2010.
Advertisements

Markov chains. Probability distributions Exercise 1.Use the Matlab function nchoosek(n,k) to implement a generic function BinomialPMF(k,n,p) for calculating.
Discrete time Markov Chain
1 Introduction to Discrete-Time Markov Chain. 2 Motivation  many dependent systems, e.g.,  inventory across periods  state of a machine  customers.
. Markov Chains. 2 Dependencies along the genome In previous classes we assumed every letter in a sequence is sampled randomly from some distribution.
Hidden Markov Models (1)  Brief review of discrete time finite Markov Chain  Hidden Markov Model  Examples of HMM in Bioinformatics  Estimations Basic.
Lecture 6  Calculating P n – how do we raise a matrix to the n th power?  Ergodicity in Markov Chains.  When does a chain have equilibrium probabilities?
CS433 Modeling and Simulation Lecture 06 – Part 03 Discrete Markov Chains Dr. Anis Koubâa 12 Apr 2009 Al-Imam Mohammad Ibn Saud University.
Lecture 4 The Gauß scheme A linear system of equations Matrix algebra deals essentially with linear linear systems. Multiplicative elements. A non-linear.
Presented By Cindy Xiaotong Lin
1 Markov Chains (covered in Sections 1.1, 1.6, 6.3, and 9.4)
. Computational Genomics Lecture 7c Hidden Markov Models (HMMs) © Ydo Wexler & Dan Geiger (Technion) and by Nir Friedman (HU) Modified by Benny Chor (TAU)
Topics Review of DTMC Classification of states Economic analysis
11 - Markov Chains Jim Vallandingham.
TCOM 501: Networking Theory & Fundamentals
1 Part III Markov Chains & Queueing Systems 10.Discrete-Time Markov Chains 11.Stationary Distributions & Limiting Probabilities 12.State Classification.
Андрей Андреевич Марков. Markov Chains Graduate Seminar in Applied Statistics Presented by Matthias Theubert Never look behind you…
Hidden Markov Models Fundamentals and applications to bioinformatics.
Tutorial 8 Markov Chains. 2  Consider a sequence of random variables X 0, X 1, …, and the set of possible values of these random variables is {0, 1,
MAT 4830 Mathematical Modeling 4.4 Matrix Models of Base Substitutions II
Overview of Markov chains David Gleich Purdue University Network & Matrix Computations Computer Science 15 Sept 2011.
Markov Chains Lecture #5
1 CE 530 Molecular Simulation Lecture 8 Markov Processes David A. Kofke Department of Chemical Engineering SUNY Buffalo
Maximum Likelihood. Likelihood The likelihood is the probability of the data given the model.
Eigenvalues and eigenvectors
What is the probability that of 10 newborn babies at least 7 are boys? p(girl) = p(boy) = 0.5 Lecture 10 Important statistical distributions Bernoulli.
Lecture 4 The Gauß scheme A linear system of equations Matrix algebra deals essentially with linear linear systems. Multiplicative elements. A non-linear.
Lecture 6 Ordination Ordination contains a number of techniques to classify data according to predefined standards. The simplest ordination technique is.
1 Markov Chains Tom Finke. 2 Overview Outline of presentation The Markov chain model –Description and solution of simplest chain –Study of steady state.
Computational statistics 2009 Random walk. Computational statistics 2009 Random walk with absorbing barrier.
Maximum Likelihood. Historically the newest method. Popularized by Joseph Felsenstein, Seattle, Washington. Its slow uptake by the scientific community.
Chapter 4: Stochastic Processes Poisson Processes and Markov Chains
Review of Matrix Algebra
1 Markov Chains Algorithms in Computational Biology Spring 2006 Slides were edited by Itai Sharon from Dan Geiger and Ydo Wexler.
Finite Mathematics & Its Applications, 10/e by Goldstein/Schneider/SiegelCopyright © 2010 Pearson Education, Inc. 1 of 60 Chapter 8 Markov Processes.
Markov Chains Chapter 16.
MOHAMMAD IMRAN DEPARTMENT OF APPLIED SCIENCES JAHANGIRABAD EDUCATIONAL GROUP OF INSTITUTES.
Stochastic Process1 Indexed collection of random variables {X t } t   for each t  T  X t is a random variable T = Index Set State Space = range.
6. Markov Chain. State Space The state space is the set of values a random variable X can take. E.g.: integer 1 to 6 in a dice experiment, or the locations.
Final Exam Review II Chapters 5-7, 9 Objectives and Examples.
Markov Chain Monte Carlo and Gibbs Sampling Vasileios Hatzivassiloglou University of Texas at Dallas.
Lecture 10 – Models of DNA Sequence Evolution Correct for multiple substitutions in calculating pairwise genetic distances. Derive transformation probabilities.
PHARMACOECONOMIC EVALUATIONS & METHODS MARKOV MODELING IN DECISION ANALYSIS FROM THE PHARMACOECONOMICS ON THE INTERNET ®SERIES ©Paul C Langley 2004 Maimon.
Markov Chains X(t) is a Markov Process if, for arbitrary times t1 < t2 < < tk < tk+1 If X(t) is discrete-valued If X(t) is continuous-valued i.e.
Chapter 61 Continuous Time Markov Chains Birth and Death Processes,Transition Probability Function, Kolmogorov Equations, Limiting Probabilities, Uniformization.
The generalization of Bayes for continuous densities is that we have some density f(y|  ) where y and  are vectors of data and parameters with  being.
Discrete Time Markov Chains
Markov Chains and Absorbing States
Markov Chains Part 4. The Story so far … Def: Markov Chain: collection of states together with a matrix of probabilities called transition matrix (p ij.
Markov Chains Part 3. Sample Problems Do problems 2, 3, 7, 8, 11 of the posted notes on Markov Chains.
Markov Chains. What is a “Markov Chain” We have a set of states, S = {s1, s2,...,sr}. A process starts in one of these states and moves successively from.
COMS Network Theory Week 5: October 6, 2010 Dragomir R. Radev Wednesdays, 6:10-8 PM 325 Pupin Terrace Fall 2010.
Lecture 6 Ordination Ordination contains a number of techniques to classify data according to predefined standards. The simplest ordination technique is.
11. Markov Chains (MCs) 2 Courtesy of J. Bard, L. Page, and J. Heyl.
Markov Processes What is a Markov Process?
15 October 2012 Eurecom, Sophia-Antipolis Thrasyvoulos Spyropoulos / Absorbing Markov Chains 1.
Goldstein/Schnieder/Lay: Finite Math & Its Applications, 9e 1 of 60 Chapter 8 Markov Processes.
Matrices, Vectors, Determinants.
Matrices. Variety of engineering problems lead to the need to solve systems of linear equations matrixcolumn vectors.
Ch 4. Language Acquisition: Memoryless Learning 4.1 ~ 4.3 The Computational Nature of Language Learning and Evolution Partha Niyogi 2004 Summarized by.
Discrete-time Markov chain (DTMC) State space distribution
Distance based phylogenetics
Discrete-time markov chain (continuation)
Much More About Markov Chains
Discrete Time Markov Chains (cont’d)
Markov Chains Mixing Times Lecture 5
Matrix Multiplication
PageRank and Markov Chains
Markov Chains Part 5.
Lecture 4: Algorithmic Methods for G/M/1 and M/G/1 type models
Presentation transcript:

Markov chains Assume a gene that has three alleles A, B, and C. These can mutate into each other. Transition probabilities Transition matrix Probability matrix Left probability matrix: The column sums add to 1. Right probability matrix: The row sums add to 1. Transition matrices are always square The trace contains the probabilities of no change. A B C A B C 68% of A stays A, 12% mutates into B and 20% into C. 7% mutates from B to A and 10% from C to A.

Calculating probabilities Probabilities to reach another state in the next step. Probabilities to reach another state in exactly two steps. The probability to reach any state in exactly n steps is given by

Assume for instance you have a virus with N strains. Assume further that at each generation a strain mutates to another strain with probabilities a i →j. The probability to stay is therefore 1-Σa i →j. What is the probability that the virus is after k generations the same as at the beginning?

Initial allele frequencies Allele frequencies in the first generation Given initial allele frequencies. What are the frequencies in the next generation?

A Markov chain is a process where step n depends only on the transition probabilities at step n-1 and the realized values at step n. A Marcov chain doesn’t have a memory. Andrey Markov ( ) Transition probabilities might change. The model assumes constant transition probabilities.

Does our mutation process above reach in stable allele frequencies or do they change forever? Do we get stable frequencies? X n is a steady-state, stationary probability, or equilibrium vector. The associated eigenvalue is 1. The equilibrium vector is independent of the initial conditions. The largest eigenvalue (principal eigenvalue) of every probability matrix equals 1 and there is an associated stationary probability vector that defines the equilibrium conditions (Perron- Frobenius theorem).

Eigenvalues and eigenvectors of probability matrices Column sums of probability matrices are 1. Row sums might be higher. The eigenvalues of probability matrices and their transposes are identical. One of the eigenvalues of a probability matrix is 1. If one of the entries of P is 1, the matrix is called absorbing. In this case the eigenvector of the largest eigenvalue contains only zeros and one 1. Absorbing chains become monodominant by one element. To get frequencies the eigenvector has to be rescaled (normalized).

Normalizing the stationary state vector Frequencies have to add to unity! Stationary frequencies

Final frequencies The sum of the eigenvector entries have to be rescaled. N=1000

Do all Markov chains converge? Closed part Recurrent part Periodic chain Recurrent and aperiodic chains are called ergodic. The probability matrix theorem tells that every irreducible ergodic transition matrix has a steady state vector T to which the process converges. You can leave every state. State D cannot be left. The chain is absorbing.

Absorbing chains A C D B It is impossible to leave state D A chain is called absorbing if it containes states without exit. The other states are called transient. Any absorbing Markov chain finally converges to the absorbing states. Closed part Absorbing part

The time to reach the absorbing state HomeBar Assume a druncard going randomly through five streets. In the first street is his home, in the last a bar. At either home or bar he stays. 0.5

The canonical form We rearrange the transition matrix to have the s absorbing states in the upper left corner and the t transient states in the lower right corner. We have four compartments After n steps we have; The unknown matrix contains information about the frequencies to reach an absorbing state from stateB, C, or D. Transient part

Multiplication of probabilities gives ever smaller values Simple geometric series The entries n ij of the matrix B contain the probabilities of ending in an absorbing state i when started in state j. The entries n ij of the fundamental matrix N of Q contain the expected numbers of time the process is in state i when started in state j.

The sum of all rows of N gives the expected number of times the chain is is state i (afterwards it falls to the absorbing state). t is a column vector that gives the expected number of steps (starting at state i) before the chain is absorbed. The druncard’s walk The expected number of steps to reach the absorbing state. The probability of reaching the absorbing state from any of the transient states.

Periodic chains do not have stable points.

Expected return (recurrence) times C A D E B If we start at state D, how long does it take on average to return to D? If u is the rescaled eigenvector of the probability matrix P, the expected return time t ii of state i back to i is given by the inverse of the i th element u i of the eigenvector u. The rescaled eigenvector u of the probability matrix P gives the steady state frequencies to be in state i In the long run it takes about 9 steps to return to D

First passage times in ergodic chains If we start at state D, how long does it take on average to reach state A? C A D E B Applied to the original probability matrix P the fundamental matrix N of P contains information on expected number of times the process is in state i when started in state j. D C A D E B D E B A C A We have to consider all possible ways from D to A. The inverse of the sum of these probabilities gives the expected number of steps to reach from point j to point k. The fundamental matrix of an ergodic chain D E D C A …… W is the matrix containing only the rescaled stationary point vector. The expected average number of steps t jk to reach from j to k comes from the entries of the fundamental matrix N divided through the respective entry of the (rescaled) stationary point vector.

Average first passage time

You have sunny, cloudy, and rainy days with respective transition probabilities. How long does it take for a sunny day to folow a rainy day? How long does it take that a sunny day comes back?

Probabilities of DNA substitution We assume equal substitution probabilities. If the total probability for a substitution is p: A T CG p p p p p The probability that A mutates to T, C, or G is P ¬A =p+p+p The probability of no mutation is p A =1-3p Independent events The probability that A mutates to T and C to G is P AC =(p)x(p) p(A →T)+ p(A →C) +p(A →G) +p(A →A) =1 The construction of evolutionary trees from DNA sequence data

The probability matrix A T C G A T C G What is the probability that after 5 generations A did not change? The Jukes - Cantor model (JC69) now assumes that all substitution probabilities are equal.

Arrhenius model The Jukes Cantor model assumes equal substitution probabilities within these 4 nucleotides. Substitution probability after time t Transition matrix Substitution matrix t A,T,G,C A The probability that nothing changes is the zero term of the Poisson distribution The probability of at least one substitution is The probability to reach a nucleotide from any other is The probability that a nucleotide doesn’t change after time t is

Probability for a single difference This is the mean time to get x different sites from a sequence of n nucleotides. It is also a measure of distance that dependents only on the number of substitutions What is the probability of n differences after time t? We use the principle of maximum likelihood and the Bernoulli distribution

Gorilla Pan paniscus Pan troglodytes Homo sapiens Homo neandertalensis Time Divergence - number of substitutions Phylogenetic trees are the basis of any systematic classificaton