1. To be presented by Maral Hudaybergenova IENG 513 FALL 2015

2. Markov chains Chapter 4. Markov chains 4.4 Limiting probabilities  The Hardy-Weinberg Law and Markov Chain in Genetics Markov Chain in Genetics  Transition probabilities  Stationary probabilities

3. Introduction  A Markov chain, named after Andrey Markov, is a random process that undergoes transitions from one state to another on a state space. It must possess a property that is usually characterized as "memorylessness": the probability distribution of the next state depends only on the current state and not on the sequence of events that preceded it. Concequently, in this sequence the present state, the future and past states are independent.Andrey Markovrandom processstate space"memorylessness"  A state i is said to be transient if, given that we start in state i, there is a non-zero probability that we will never return to i. The probability of going from state i to state j in n time steps is transient evolution. Pij = P(X n =j Ɩ X 0 = i )  State i is recurrent if it is not transient. Recurrent states are guaranteed (with probability 1) to have a finite hitting time. Recurrence and transience are class properties, that is, they either hold or do not hold equally for all members of a communicating class.

4. What does Hardy-Weinberg law state?  It’s allow us predict allele (subset of genes) frequencies in living populations.  The allele frequency tends to remain the same from generation to generation unless acted by outside influences (mechanisms of evolution).  In other words, we can predict what phenotypes/traits will be in a given population.

5. Importance of p & q  No matter the allele letter assign that dominant allele is p  No matter the allele letter assign that recessive allele is q  The combinations of the p and q alleles must equal 100% of the population, so p + q =1

6. What does it mean for us?  There are AA genotypes which are homogenous  There are aa genotypes which are homogenous  There are Aa genotypes which are heterogenous so : AA=p 2 Aa=2pq aa=q 2 means, that p 2 + 2pq+ q 2 =1 (100% of population)

7. Let’s consider fish length…  Assume genes A and a are present in fishes and length in proportion to their frequency in population (i.e. 0.6 and 0.4 respectively)  Also assume that long and short fishes meet at random (one big gene pool).  Then we can calculate expected genotype frequencies.

8.  AA: To produce a long fish, each gene must contain an A allele. P ( A ) = 0.6 x 0.6 = 0.36  Similarly, we can calculate frequency of short fish P (aa) = 0.4 x 04 = 0.16.  P ( Aa) is given by probability the first gene contains A (0.6) times probability the second gene contains a (0.4). 0.6 x 04 = 0.24.  But, there’s a second way to produce an Aa individual fish (first gene contains a and other one contains A). Same probability as before: 0.4 x 0.6= 0.24. Therefore, the overall P ( Aa) = 0.24 + 0.24 = 0.48

9. p 2 + 2pq+ q 2 =1  Genotypes in next generation of fish length:  AA = 0.36  aa = 0.16  Aa= 0.48  These frequencies add up to one. AA p 2 =0.36 Aa pq=0.24 Aa qp=0.24 aa q 2 =0,16 A a a 80% (p=0.8) 20% (q=0.2) A

10. Example 4.20 (The Hardy–Weinberg Law and a Markov Chain in Genetics) Consider a large population of individuals, each of whom possesses a particular pair of genes, of which each individual gene is classified as being of type A or type a. Assume that the proportions of individuals whose gene pairs are AA, aa, or Aa are, respectively, p0, q0, and r0 (p0 + q0 + r0 = 1). When two individuals mate, each contributes one of his or her genes, chosen at random, to the resultant offspring. Assuming that the mating occurs at random, in that each individual is equally likely to mate with any other individual, we are interested in determining the proportions of individuals in the next generation whose genes are AA, aa, or Aa. Let’s call these proportions p, q, and r, and by focusing attention on an individual of the next generation obtain then determine the probabilities for the gene pair of that individual. Note: that randomly choosing a parent and then randomly choosing one of its genes is equivalent to just randomly choosing a gene from the total gene population.

11. By conditioning on the gene pair of the parent, we see that a randomly chosen gene will be type A with probability P{A} = P{A|AA}p0 + P{A|aa}q0 + P{A|Aa}r0 = p0 +r0/2 Similarly, it will be type a with probability P{a} = q0 +r0/2 With random mating a randomly chosen member of the next generation will be type AA with probability p, where p = P{A}P{A} = (p0 +r0/2)2 Similarly, the randomly chosen member will be type aa with probability q = P{a}P{a} = (q0 +r0/2)2 and will be type Aa with probability r = 2P{A}P{a} = 2(p0 + r0/2)(q0 +r0/2)

12. Since each member of the next generation will independently be of each of the three gene types with probabilities p, q, r, it follows that the percentages of the members of the next generation that are of type AA, aa, or Aa are respectively p, q, and r. If we now consider the total gene pool of this next generation, then p + r/2, the fraction of its genes that are A, will be unchanged from the previous generation. This follows that the total gene pool has not changed from generation to generation : p + r/2 = (p0 + r0/2)2 +(p0 + r0/2)(q0 +r0/2) = (p0 + r0/2)[p0 + r0/2+ q0 +r0/2] = p0 +r0/2 since p0 +r0 +q0 = 1 = P{A} Thus, the fractions of the gene pool that are A and a are the same as in the initial generation. From this it follows that, under random mating, in all successive generations after the initial one the percentages of the population having gene pairs AA, aa, and Aa will remain fixed at the values p, q, and r. This is known as the Hardy–Weinberg law.

13. If we suppose now that the gene pair population has stabilized in the percentages p, q, r, and let us follow the genetic history of a single individual and her descendants.(For simplicity, we can assume that each individual has exactly one offspring.) So, for a given individual, let Xn denote the genetic state of her descendant in the nth generation. The transition probability matrix of this Markov chain, verified by conditioning on the state of the randomly chosen mate AA aa Aa AA p + r 0 q + r 2 2 aa 0 q+ r p+ r 2 2 Aa p + r q + r p + q + r 2 4 2 4 2 2 2

14. Let’s assume that person with gene A is African, with gene a is Asian, and with genes Aa is African-Asian. By looking at this tree we can see what type of gene we will get if they mate with each other:

15. It is quite intuitive (why?) that the limiting probabilities for this Markov chain (which also equal the fractions of the individual’s descendants that are in each of the three genetic states) should just be p, q, and r. Because if it is redundant, it suffices to show that

16. Conclusions from Hardy-Weinberg Law  Allele frequencies in a population will not change from one generation to the next.  Genetic equilibrium is a theoretical state, factors can affect it.

17. Example 4.21 Production process changes states in accordance with an irreducible, positive recurrent Markov chain having transition probabilities P i j, i, j = 1,..., n, and certain of the states are considered acceptable and the remaining unacceptable. Let A denote the acceptable states and A c the unacceptable ones. If the production process is said to be “up” when in an acceptable state and “down” when in an unacceptable state, determine: 1. the rate at which the production process goes from up to down (that is, the rate of breakdowns); 2. the average length of time the process remains down when it goes down; 3. the average length of time the process remains up when it goes up.

18. Solution: Let πk, k = 1,..., n, denote the long-run proportions. Now for i ∈ A and j ∈ Ac the rate at which the process enters state j from state i is and so the rate at which the production process enters state j from an acceptable state is Hence, the rate at which it enters an unacceptable state from an acceptable one (which is the rate at which breakdowns occur) is Now let U and D denote the average time the process remains up when it goes up and down when it goes down. Because there is a single breakdown every U + D time units on the average, it follows heuristically that

19. Second equation relating.U and.D, consider the percentage of time the process is up, which, of course, is equal to Σ i ∈ A πi. However, since the process is up on the average.U out of every.U +.D time units, it follows (again somewhat heuristically) that the and so now we can obtain

20. For example, suppose the transition probability matrix is given by where the acceptable (up) states are 1, 2 and the unacceptable (down) ones are 3, 4. The limiting probabilities satisfy

21. By solving this equation we will get: Now we can calculate rate of breakdowns following these formulas: up to down= π1(P13 + P14)+ π2(P23 + P24)=9/32 down to down= π3(P33 + P34)+ π4(P43 + P44)=14/9 up to up = π1(P11 + P12)+ π2(P21 + P22)=2

22. Remarks (i) The long run proportions πj, j 0, are often called stationary probabilities. The reason being that if the initial state is chosen according to the probabilities πj, j 0, then the probability of being in state j at any time n is also equal to πj. That is, if P{X0 = j} = πj, j ≥ 0 then P{Xn = j} = πj for all n, j ≥ 0 if we suppose it true for n − 1, then P{Xn = j} = Σ P{Xn = j |Xn−1 = i }P{Xn−1 = i } = ΣP i jπ i by the induction hypothesis = πj (ii) For state j, define m j j to be the expected number of transitions until a Markov chain, starting in state j, returns to that state. Since, on the average, the chain will spend 1 unit of time in state j for every m j j units of time, it follows that πj = 1 mjj In words, the proportion of time in state j equals the inverse of the mean time between visits to j. And it calls strong law of renewal processes.

23. Thank You For Attention!