A First Course in Stochastic Processes Chapter Two: Markov Chains
X2=X2= X 1 =1 X 2 =2X 3 =1X 4 =3
X1X1 X2X2 X3X3 X4X4 X5X5 etc
P =
Example Two: Nucleotide evolution A G C T
Types of point mutation AG TC Purine PyramidineTransitions Transversions αββββ α
A AGCT T C G Kimura’s 2 parameter model (K2P) P =
GGTCAC A G C T A G C T CTATGA AGTTCGC
The Markov Property GGTCAC A G C T A G C T CTATGA
Markov Chain properties accessibleaperiodic communicate recurrent irreducible transient
A AGCT T C G Accessible P = 0
A AGCT T C G Accessible P = A (and G ) are no longer accessible from C (or T ).
A AGCT T C G Accessible P = But C (and T ) are still accessible from A (or G ).
A AGCT T C G Communicate P = Reciprocal accessibility
A AGCT T C G Irreducible P = All elements communicate
Non-irreducible A AGCT T C G P = = P1P1 P2P2 0 0 P 1 = A G AG C T CT P 2 =
Reflexivity Symmetry Transitivity Repercussions of communication
Periodicity P =
Periodicity The period d(i) of an element i is defined as the greatest common divisor of the numbers of the generations in which the element is visited. Most Markov Chains that we deal with do not exhibit periodicity. A Markov Chain is aperiodic if d(i) = 1 for all i.
Recurrence recurrent transient
More on Recurrence and i is recurrent then j is recurrent In a one-dimensional symmetric random walk the origin is recurrent In a two-dimensional symmetric random walk the origin is recurrent In a three-dimensional symmetric random walk the origin is transient
Markov Chain properties accessibleaperiodic communicate recurrent irreducible transient
Markov Chains Examples
X 1 =1 X1X1 X2X2 X3X3 X4X4 X5X5 etc
P =
Diffusion across a permeable membrane (1D random walk)
Brownian motion (2D random walk)
Wright-Fisher allele frequency model X 1 =1
Haldane (1927) branching process model of fixation probability
P i,j = coefficient of s j in the above generating function
Haldane (1927) branching process model of fixation probability Probability of fixation = 2s
Markov Chain properties accessibleaperiodic communicate recurrent irreducible transient