1. 1 STATISTICAL INFERENCE PART II POINT ESTIMATION

2. 2 SUFFICIENT STATISTICS X, f(x;  ),  X 1, X 2,…,X n be a sample rvs Y=U(X 1, X 2,…,X n ) is a statistic. A sufficient statistic, Y is a statistic which contains all the information for the estimation of .

3. 3 SUFFICIENT STATISTICS Given the value of Y, the sample contains no further information for the estimation of . Y is a sufficient statistic (ss) for  if the conditional distribution h(x 1,x 2,…,x n |y) does not depend on  for every given Y=y. A ss for  is not unique. If Y is a ss for , then a 1-1 transformation of Y, say Y 1 =fn(Y) is also a ss for .

4. 4 SUFFICIENT STATISTICS The conditional distribution of sample rvs given the value of y of Y, is defined as If Y is a ss for , then ss for  may include y or constant. Not depend on  for every given y. Also, the conditional range of X i given y not depend on .

5. 5 SUFFICIENT STATISTICS EXAMPLE: X~Ber(p). For a r.s. of size n, show that is a ss for p.

6. 6 SUFFICIENT STATISTICS Neyman’s Factorization Theorem: Y is a ss for  iff where k 1 and k 2 are non-negative functions and k 2 does not depend on  or y. The likelihood function Does not contain any other x i Not depend on  for every given y (also in the conditional range of x i.)

7. 7 EXAMPLES 1. X~Ber(p). For a r.s. of size n, find a ss for p if exists.

8. 8 EXAMPLES 2. X~Beta(θ,2). For a r.s. of size n, find a ss for θ.

9. 9 SUFFICIENT STATISTICS A ss may not exist. Jointly ss Y 1,Y 2,…,Y k may be needed. Example: Example 10.2.5 in Bain and Engelhardt (page 342 in 2 nd edition), X (1) and X (n) are jointly ss for  If the MLE of  exists and unique and if a ss for  exists, then MLE is a function of a ss for .

10. 10 EXAMPLE X~N( ,  2 ). For a r.s. of size n, find jss for  and  2.

11. MINIMAL SUFFICIENT STATISTICS Ifis a ss for θ, then, is also a SS for θ. But, the first one does a better job in data reduction. A minimal ss achieves the greatest possible reduction. 11

12. 12 MINIMAL SUFFICIENT STATISTICS A ss T(X) is called minimal ss if, for any other ss T’(X), T(x) is a function of T’(x). THEOREM: Let f(x;  ) be the pmf or pdf of a sample X 1, X 2,…,X n. Suppose there exist a function T(x) such that, for two sample points x 1,x 2,…,x n and y 1,y 2,…,y n, the ratio is constant as a function of  iff T(x)=T(y). Then, T(X) is a minimal sufficient statistic for .

13. 13 EXAMPLE X~N( ,  2 ) where  2 is known. For a r.s. of size n, find minimal ss for . Note: A minimal ss is also not unique. Any 1-to-1 function is also a minimal ss.

14. 14 RAO-BLACKWELL THEOREM Let X 1, X 2,…,X n have joint pdf or pmf f(x 1,x 2,…,x n ;  ) and let S=(S 1,S 2,…,S k ) be a vector of jss for . If T is an UE of  (  ) and  (T)=E(T  S), then i)  (T) is an UE of  (  ). ii)  (T) is a fn of S, so it is also jss for . iii)Var(  (T) )  Var(T) for all .  (T) is a uniformly better unbiased estimator of  (  ).

15. RAO-BLACKWELL THEOREM Notes:  (T)=E(T  S) is at least as good as T. For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs. 15

16. Example Hogg & Craig, Exercise 10.10 X 1,X 2 ~Exp(θ) Find joint p.d.f. of ss Y 1 =X 1 +X 2 for θ and Y 2 =X 2. Show that Y 2 is UE of θ with variance θ². Find φ(y 1 )=E(Y 2 |Y 1 ) and variance of φ(Y 1 ). 16

17. 17 ANCILLARY STATISTIC A statistic S ( X ) whose distribution does not depend on the parameter  is called an ancillary statistic. An ancillary statistic contains no information about .

18. Example Example 6.1.8 in Casella & Berger, page 257: Let X i ~Unif(θ,θ+1) for i=1,2,…,n Then, range R=X (n) -X (1) is an ancillary statistic because its pdf does not depend on θ. 18

19. 19 COMPLETENESS Let {f(x;  ),  } be a family of pdfs (or pmfs) and U(x) be an arbitrary function of x not depending on . If requires that the function itself equal to 0 for all possible values of x; then we say that this family is a complete family of pdfs (or pmfs).

20. 20 EXAMPLES 1. Show that the family {Bin(n=2,  ); 0<  <1} is complete.

21. 21 EXAMPLES 2. X~Uniform( ,  ). Show that the family {f(x;  ),  >0} is not complete.

22. 22 BASU THEOREM If T(X) is a complete and minimal sufficient statistic, then T(X) is independent of every ancillary statistic. Example: X~N( ,  2 ). (n-1)S 2 /  2 ~ Ancillary statistic for  By Basu theorem, and S 2 are independent. S2S2

23. 23 COMPLETE AND SUFFICIENT STATISTICS (css) Y is a complete and sufficient statistic (css) for  if Y is a ss for  and the family is complete. The pdf of Y. 1) Y is a ss for . 2) u(Y) is an arbitrary function of Y. E(u(Y))=0 for all  implies that u(y)=0 for all possible Y=y.

24. THE MINIMUM VARIANCE UNBIASED ESTIMATOR Rao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then  (T)=E(T  S) is –an UE of , i.e.,E[  (T)]=E[E(T  S)]=  and –the MVUE of . 24

25. 25 LEHMANN-SCHEFFE THEOREM Let Y be a css for . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of . Y css for . T(y)=fn(y) and E[T(Y)]= .  T(Y) is the UMVUE of .  So, it is the best estimator of .

26. 26 THE MINIMUM VARIANCE UNBIASED ESTIMATOR Let Y be a css for . Since Y is complete, there could be only a unique function of Y which is an UE of . Let U 1 (Y) and U 2 (Y) be two function of Y. Since they are UE’s, E( U 1 (Y)  U 2 (Y) )=0 imply W(Y)=U 1 (Y)  U 2 (Y)=0 for all possible values of Y. Therefore, U 1 (Y)=U 2 (Y) for all Y.

27. Example Let X 1,X 2,…,X n ~Poi(μ). Find UMVUE of μ. Solution steps: –Show that is css for μ. –Find a statistics (such as S*) that is UE of μ and a function of S. –Then, S* is UMVUE of μ by Lehmann-Scheffe Thm. 27

28. Note The estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique. 28