1. Group exercise For 0≤t 1 <…<t n real and 0≤r 1 ≤…≤r n integers, define X(t) by X(0)=0 and (a)Find P(X(t)=0) (b) Determine P(X(t)=k)

2. (c) Show that X(t 1 ) and X(t 2 )-X(t 1 ) are independent (d) Show that Kolmogorov’s consistency condition is satisfied (try this at home, xc)

3. Markov chains Chapters 5 and 6

5. Precipitation data January data from Snoqualmie Falls, Washington, 1948-1983 325 dry and 791 wet days R t =1(rain day t) ~ Bern(p) independently E(#WW days) = N x 30 x p 2  36 x 30 x (791/1116) 2 = 543

6. A conditional model P(R t = 1 | R t-1 = 1) = p 11 P(R t = 1 | R t-1 = 0) = p 01 Special case of i k =0 or 1 Transition matrix p d = P(rain following dry day)

7. More generally Consider a stochastic process (X n, n≥0) taking values in a discrete state space S. It is a Markov chain if The quantities are called the transition probabilities. The chain is homogeneous if the transition probabilities do not depend on n. We will usually assume this.

8. The transition matrix The matrix P=(p ij ) is called the transition matrix. Theorem: P has nonnegative entries and all row sums are one. Such matrices are called stochastic. Proof:

9. n-step transitions are called n-step transition probabilities. The matrix of them is denoted P (n). Theorem (Chapman-Kolmogorov): P (n+m) = P (n) P (m) Proof: using the Markov property.

10. Consequences 1.P (n) = P n 2.Let  k (n) = P(X n = k) and  (n) = (  k (n) ). Then  (m+n) =  (n) P m 3.In particular,  (n) =  (0) P n

11. Back to precipitation Let  1 (0) =p 1. Then

12. p 00 = p 11 = 1 P(X n =1) = p 1 p 01 ≠ p 11 If it rains on Jan 1, what is the chance that it rains on Jan 6?

13. Generating functions a=(a 0,a 1,a 2,...) sequence of real numbers is its generating function Special case: probability generating function (pgf)

14. Examples Bernoulli: G X (s) = s 0 (1-p)+sp = 1+p(s-1) Geometric: G X (s) = Poisson: G X (s) =

15. Convolutions The convolution of sequences a and b is c=a*b where c i =a 0 b i +a 1 b i-1 +...+a i b 0 Theorem: G c (s) = G a (s)G b (s) Proof:

16. Sum of iid random variables X i nonnegative integervalued Markov chain? E(S n ) == nE(X) P(S n =1) = Bernoulli case: Random walk case: X i =1 w pr p, -1 w pr 1-p