1. A First Course in Stochastic Processes Chapter Two: Markov Chains

2. X2=X2= X 1 =1 X 2 =2X 3 =1X 4 =3

3. X1X1 X2X2 X3X3 X4X4 X5X5 etc

4. P =

5. Example Two: Nucleotide evolution A G C T

6. Types of point mutation AG TC Purine PyramidineTransitions Transversions αββββ α

7. A AGCT T C G Kimura’s 2 parameter model (K2P) P =

8. GGTCAC A G C T A G C T CTATGA AGTTCGC

9. The Markov Property GGTCAC A G C T A G C T CTATGA

11. Markov Chain properties accessibleaperiodic communicate recurrent irreducible transient

12. A AGCT T C G Accessible P = 0

13. A AGCT T C G Accessible P = 00 00 A (and G ) are no longer accessible from C (or T ).

14. A AGCT T C G Accessible P = 00 00 But C (and T ) are still accessible from A (or G ).

15. A AGCT T C G Communicate P = Reciprocal accessibility

16. A AGCT T C G Irreducible P = All elements communicate

17. Non-irreducible A AGCT T C G P = 00 00 00 00 = P1P1 P2P2 0 0 P 1 = A G AG C T CT P 2 =

18. Reflexivity Symmetry Transitivity Repercussions of communication

19. Periodicity P =

20. Periodicity The period d(i) of an element i is defined as the greatest common divisor of the numbers of the generations in which the element is visited. Most Markov Chains that we deal with do not exhibit periodicity. A Markov Chain is aperiodic if d(i) = 1 for all i.

21. Recurrence recurrent transient

22. More on Recurrence and i is recurrent then j is recurrent In a one-dimensional symmetric random walk the origin is recurrent In a two-dimensional symmetric random walk the origin is recurrent In a three-dimensional symmetric random walk the origin is transient

23. Markov Chain properties accessibleaperiodic communicate recurrent irreducible transient

24. Markov Chains Examples

25. X 1 =1 X1X1 X2X2 X3X3 X4X4 X5X5 etc

26. P =

27. Diffusion across a permeable membrane (1D random walk)

28. Brownian motion (2D random walk)

29. Wright-Fisher allele frequency model X 1 =1

30. Haldane (1927) branching process model of fixation probability 2344442

32. P i,j = coefficient of s j in the above generating function

33. Haldane (1927) branching process model of fixation probability Probability of fixation = 2s

34. Markov Chain properties accessibleaperiodic communicate recurrent irreducible transient