1. Problems, cont. 3. where k=0?

2. When are there stationary distributions? Theorem: An irreducible chain has a stationary distribution  iff the states are positive persistent. Then  is unique and given by Note: This is often an easier way to calculate  i.

3. Example Consider C 1 = {0,1}

4. Periodic chains Recall that the random walk starting at 0 only could return to 0 at even times. Such a state is called periodic. The period of a state i is A state is aperiodic if its period is 1. Fact: The period of a state is an equivalence class property. So we call an irreducible chain with a periodic state a periodic chain.

5. Limit probabilities Theorem: For an aperiodic, irreducible chain Note: If the chain is transient or null persistent we have the n-step transition probabilities converging to 0. Also, the limit does not depend on i.

6. What about periodic chains? Let It has but no limiting distribution. But if X n is an irreducible chain with period d, then Y n =X nd is aperiodic, and thus since P Y = P d.

7. Proof for positive persistent chains We will use a technique called coupling. Let X and Y be independent chains with the same transition matrix P. Z=(X,Y) is a Markov chain with transition matrix p ij,kl =P(Z n+1 =(k,l)  Z n =(i,j)) = P(X n+1 =k  X n =i)P(Y n+1 =l  Y n =j) =p ik p jl X positive persistent so has stationary distribution , so Z has stationary distribution with ij =  i  j

8. Proof, cont. Let T=min{n:Z n =(s,s)}. T is finite with probability one. Now note that if T≤n, X n and Y n have the same distribution. Let Z 0 =(i,j). Using the same calculation with X and Y interchanged we see that

9. Proof, cont. So if has a limit, it does not depend on i. But by bounded convergence.