1. 1 ORDER STATISTICS AND LIMITING DISTRIBUTIONS

2. 2 ORDER STATISTICS Let X 1, X 2,…,X n be a r.s. of size n from a distribution of continuous type having pdf f(x), a<x<b. Let X (1) be the smallest of X i, X (2) be the second smallest of X i,…, and X (n) be the largest of X i. X (i) is the i -th order statistic.

3. ORDER STATISTICS It is often useful to consider ordered random sample. Example: suppose a r.s. of five light bulbs is tested and the failure times are observed as (5,11,4,100,17). These will actually be observed in the order of (4,5,11,17,100). Interest might be on the k th smallest ordered observation, e.g. stop the experiment after k th failure. We might also be interested in joint distributions of two or more order statistics or functions of them (e.g. range=max – min) 3

4. 4 ORDER STATISTICS If X 1, X 2,…,X n be a r.s. of size n from a population with continuous pdf f(x), then the joint pdf of the order statistics X (1), X (2),…,X (n) is The joint pdf of ordered sample is not same as the joint pdf of unordered sample. Order statistics are not independent. Future reference: For discrete distributions, we need to take ties into account (two X’s being equal). See, Casella and Berger, 1990, pg 231.

5. Example Suppose that X 1, X 2, X 3 is a r.s. from a population with pdf f(x)=2x for 0<x<1 Find the joint pdf of order statistics and the marginal pdf of the smallest order statistic. 5

6. 6 ORDER STATISTICS The Maximum Order Statistic: X (n)

7. 7 ORDER STATISTICS The Minimum Order Statistic: X (1)

8. 8 ORDER STATISTICS k -th Order Statistic y y1y1 y2y2 y k-1 ykyk y k+1 ynyn … … P(X<y k )P(X>y k ) f X (y k ) # of possible orderings n!/{(k  1)!1!(n  k)!}

9. Example Same example but now using the previous formulas (without taking the integrals): Suppose that X 1, X 2, X 3 is a r.s. from a population with pdf f(x)=2x for 0<x<1 Find the marginal pdf of the smallest order statistic. 9

10. Example X~Uniform(0,1). A r.s. of size n is taken. Find the p.d.f. of kth order statistic. Solution: Let Y k be the kth order statistic. 10

11. 11 ORDER STATISTICS Joint p.d.f. of k -th and j-th Order Statistic (for k<j) y y1y1 y2y2 y k-1 ykyk y k+1 ynyn … … P(X<y k )P(y k <X<y j ) f X (y k ) # of possible orderings n!/{(k  1)!1!(j-k-1)!1!(n  j)!} y j-1 yjyj y j+1 k-1 itemsj-k-1 itemsn-j items 1 item P(X>y j ) f X (y j )

12. LIMITING DISTRIBUTION 12

13. Motivation The p.d.f. of a r.v. often depends on the sample size (i.e., n) If X 1, X 2,…, X n is a sequence of rvs and Y n =u(X 1, X 2,…, X n ) is a function of them, sometimes it is possible to find the exact distribution of Y n (what we have been doing lately) However, sometimes it is only possible to obtain approximate results when n is large  limiting distributions. 13

14. 14 CONVERGENCE IN DISTRIBUTION Consider that X 1, X 2,…, X n is a sequence of rvs and Y n =u(X 1, X 2,…, X n ) be a function of rvs with cdfs F n (y) so that for each n=1, 2,… where F(y) is continuous. Then, the sequence Y n is said to converge in distribution to Y.

15. 15 CONVERGENCE IN DISTRIBUTION Theorem: If for every point y at which F(y) is continuous, then Y n is said to have a limiting distribution with cdf F(y). The term “Asymptotic distribution” is sometimes used instead of “limiting distribution” Definition of convergence in distribution requires only that limiting function agrees with cdf at its points of continuity.

16. 16 Example Let X 1,…, X n be a random sample from Unif(0,1). Find the limiting distribution of the max order statistic, if it exists.

17. 17 Example Let {X n } be a sequence of rvs with pmf Find the limiting distribution of X n, if it exists.

18. Example Let Y n be the nth order statistic of a random sample X 1, …, X n from Unif(0,θ). 18 Find the limiting distribution of Z n =n(θ -Y n ), if it exists

19. 19 Example Let X 1,…, X n be a random sample from Exp(θ). Find the limiting distribution of the min order statistic, if it exists.

20. Example Suppose that X1, X2, …, Xn are iid from Exp(1). 20 Find the limiting distribution of the max order statistic, if it exists.

21. 21 LIMITING MOMENT GENERATING FUNCTIONS Let rv Y n have an mgf M n (t) that exists for all n. If then Y n has a limiting distribution which is defined by M(t).

22. 22 Example Let =np. The mgf of Poisson( ) The limiting distribution of Binomial rv is the Poisson distribution.

23. 23 CONVERGENCE IN PROBABILITY (STOCHASTIC CONVERGENCE) A rv Y n convergence in probability to a rv Y if for every  >0. Special case: Y=c where c is a constant not depending on n. The limiting distribution of Y n is degenerate at point c.

24. 24 CHEBYSHEV’S INEQUALITY Let X be an rv with E(X)=  and V(X)=  2. The Chebyshev’s Inequality can be used to prove stochastic convergence in many cases.

25. 25 CONVERGENCE IN PROBABILITY (STOCHASTIC CONVERGENCE) The Chebyshev’s Inequality proves the convergence in probability if the following three conditions are satisfied. 1. E(Y n )=  n where

26. 26 Example Let X be an rv with E(X)=  and V(X)=  2 < . For a r.s. of size n, is the sample mean. Is

27. 27 WEAK LAW OF LARGE NUMBERS (WLLN) Let X 1, X 2,…,X n be iid rvs with E(X i )=  and V(X i )=  2 0, converges in probability to . WLLN states that the sample mean is a good estimate of the population mean. For large n, the probability that the sample mean and population mean are close to each other with probability 1. But more to come on the properties of estimators.

28. Example Let X 1, X 2,…,X n be iid rvs from Ber(p). Then, E(X)=p and Var(X)=p(1-p)<∞. By WLLN, 28

29. 29 STRONG LAW OF LARGE NUMBERS Let X 1, X 2,…,X n be iid rvs with E(X i )=  and V(X i )=  2 0, that is, converges almost sure to .

30. Relation between convergences Almost sure convergence is the strongest. (reverse is not true) 30

31. 31 THE CENTRAL LIMIT THEOREM Let X 1, X 2,…,X n be a sequence of iid rvs with E(X i )=  and V(X i )=  2 <∞. Define. Then, or Proof can be found in many books, e.g. Bain and Engelhardt, 1992, page 239

32. Example Let X 1, X 2,…,X n be iid rvs from Unif(0,1) and Find approximate distribution of Yn. Find approximate values of – –The 90 th percentile of Yn. Warning: n=20 is probably not a big enough number of size. 32

33. 33 SLUTKY’S THEOREM If X n  X in distribution and Y n  a, a constant, in probability, then a) Y n X n  aX in distribution. b) X n +Y n  X+a in distribution.

34. 34 SOME THEOREMS ON LIMITING DISTRIBUTIONS If X n  c>0 in probability, then for any function g(x) continuous at c, g( X n )  g(c) in prob. e.g. If X n  c in probability and Y n  d in probability, then aX n +bY n  ac+bd in probability. X n Y n  cd in probability 1/X n  1/c in probability for all c  0.

35. 35 Example X~Gamma( , 1). Show that