Download presentation
Presentation is loading. Please wait.
Published byΚαλλιστώ Βικελίδης Modified over 6 years ago
1
STATISTICAL INFERENCE PART II SOME PROPERTIES OF ESTIMATORS
2
SOME PROPERTIES OF ESTIMATORS
θ: a parameter of interest; unknown Previously, we found good(?) estimator(s) for θ or its function g(θ). Goal: Check how good are these estimator(s). Or are they good at all? If more than one good estimator is available, which one is better?
3
SOME PROPERTIES OF ESTIMATORS
UNBIASED ESTIMATOR (UE): An estimator is an UE of the unknown parameter , if Otherwise, it is a Biased Estimator of . Bias of for estimating If is UE of ,
4
SOME PROPERTIES OF ESTIMATORS
ASYMPTOTICALLY UNBIASED ESTIMATOR (AUE): An estimator is an AUE of the unknown parameter , if
5
SOME PROPERTIES OF ESTIMATORS
CONSISTENT ESTIMATOR (CE): An estimator which converges in probability to an unknown parameter for all is called a CE of . For large n, a CE tends to be closer to the unknown population parameter. MLEs are generally CEs.
6
EXAMPLE For a r.s. of size n, By WLLN,
7
MEAN SQUARED ERROR (MSE)
The Mean Square Error (MSE) of an estimator for estimating is If is smaller, is a better estimator of .
8
MEAN SQUARED ERROR CONSISTENCY
Tn is called mean squared error consistent (or consistent in quadratic mean) if E{Tn}20 as n. Theorem: Tn is consistent in MSE iff i) Var(Tn)0 as n. If E{Tn}20 as n, Tn is also a CE of .
9
EXAMPLES X~Exp(), >0. For a r.s of size n, consider the following estimators of , and discuss their bias and consistency. Which estimator is better?
10
SUFFICIENT STATISTICS
X, f(x;), X1, X2,…,Xn Y=U(X1, X2,…,Xn ) is a statistic. A sufficient statistic, Y, is a statistic which contains all the information for the estimation of .
11
SUFFICIENT STATISTICS
Given the value of Y, the sample contains no further information for the estimation of . Y is a sufficient statistic (ss) for if the conditional distribution h(x1,x2,…,xn|y) does not depend on for every given Y=y. A ss for is not unique: If Y is a ss for , then any 1-1 transformation of Y, say Y1=fn(Y) is also a ss for .
12
SUFFICIENT STATISTICS
The conditional distribution of sample rvs given the value of y of Y, is defined as If Y is a ss for , then Not depend on for every given y. ss for may include y or constant. Also, the conditional range of Xi given y not depend on .
13
SUFFICIENT STATISTICS
EXAMPLE: X~Ber(p). For a r.s. of size n, show that is a ss for p.
14
SUFFICIENT STATISTICS
Neyman’s Factorization Theorem: Y is a ss for iff The likelihood function Does not depend on xi except through y Not depend on (also in the range of xi.) where k1 and k2 are non-negative functions.
15
EXAMPLES 1. X~Ber(p). For a r.s. of size n, find a ss for p if exists.
16
EXAMPLES 2. X~Beta(θ,2). For a r.s. of size n, find a ss for θ.
17
SUFFICIENT STATISTICS
A ss may not exist. Jointly ss Y1,Y2,…,Yk may be needed. Example: Example in Bain and Engelhardt (page 342 in 2nd edition), X(1) and X(n) are jointly ss for If the MLE of exists and unique and if a ss for exists, then MLE is a function of a ss for .
18
EXAMPLE X~N(,2). For a r.s. of size n, find jss for and 2.
19
MINIMAL SUFFICIENT STATISTICS
If is a ss for θ, then, is also a ss for θ. But, the first one does a better job in data reduction. A minimal ss achieves the greatest possible reduction.
20
MINIMAL SUFFICIENT STATISTICS
A ss T(X) is called minimal ss if, for any other ss T’(X), T(x) is a function of T’(x). THEOREM: Let f(x;) be the pmf or pdf of a sample X1, X2,…,Xn. Suppose there exist a function T(x) such that, for two sample points x1,x2,…,xn and y1,y2,…,yn, the ratio is constant with respect to iff T(x)=T(y). Then, T(X) is a minimal sufficient statistic for .
21
EXAMPLE X~N(,2) where 2 is known. For a r.s. of size n, find minimal ss for . Note: A minimal ss is also not unique. Any 1-to-1 function is also a minimal ss.
22
RAO-BLACKWELL THEOREM
Let X1, X2,…,Xn have joint pdf or pmf f(x1,x2,…,xn;) and let S=(S1,S2,…,Sk) be a vector of jss for . If T is an UE of () and (T)=E(TS), then (T) is an UE of () . (T) is a fn of S, so it is also jss for . Var((T) ) Var(T) for all . (T) is a uniformly better unbiased estimator of () .
23
RAO-BLACKWELL THEOREM
Notes: (T)=E(TS) is at least as good as T. For finding the best UE, it is enough to consider UEs that are functions of a ss, because all such estimators are at least as good as the rest of the UEs.
24
Example Hogg & Craig, Exercise 10.10 X1,X2~Exp(θ)
Find joint p.d.f. of ss Y1=X1+X2 for θ and Y2=X2. Show that Y2 is UE of θ with variance θ². Find φ(y1)=E(Y2|Y1) and variance of φ(Y1).
25
ANCILLARY STATISTIC A statistic S(X) whose distribution does not depend on the parameter is called an ancillary statistic. Unlike a ss, an ancillary statistic contains no information about .
26
Example Example 6.1.8 in Casella & Berger, page 257:
Let Xi~Unif(θ,θ+1) for i=1,2,…,n Then, range R=X(n)-X(1) is an ancillary statistic because its pdf does not depend on θ.
27
COMPLETENESS Let {f(x; ), } be a family of pdfs (or pmfs) and U(x) be an arbitrary function of x not depending on . If requires that the function itself equal to 0 for all possible values of x; then we say that this family is a complete family of pdfs (or pmfs). i.e., the only unbiased estimator of 0 is 0 itself.
28
EXAMPLES 1. Show that the family {Bin(n=2,); 0<<1} is complete.
29
EXAMPLES 2. X~Uniform(,). Show that the family {f(x;), >0} is not complete.
30
COMPLETE AND SUFFICIENT STATISTICS (css)
Y is a complete and sufficient statistic (css) for if Y is a ss for and the family is complete. The pdf of Y. 1) Y is a ss for . 2) u(Y) is an arbitrary function of Y. E(u(Y))=0 for all implies that u(y)=0 for all possible Y=y.
31
BASU THEOREM If T(X) is a complete and minimal sufficient statistic, then T(X) is independent of every ancillary statistic. Example: X~N(,2). (n-1)S2/ 2 ~ S2 Ancillary statistic for By Basu theorem, and S2 are independent.
32
BASU THEOREM Example: Let T=X1+ X2 and U=X1 - X2
We know that T is a complete minimal ss. U~N(0, 2) distribution free of T and U are independent by Basu’s Theorem X1, X2~N(,2), independent, 2 known.
33
THE MINIMUM VARIANCE UNBIASED ESTIMATOR
Rao-Blackwell Theorem: If T is an unbiased estimator of , and S is a ss for , then (T)=E(TS) is an UE of , i.e.,E[(T)]=E[E(TS)]= and the MVUE of .
34
LEHMANN-SCHEFFE THEOREM
Let Y be a css for . If there is a function Y which is an UE of , then the function is the unique Minimum Variance Unbiased Estimator (UMVUE) of . Y css for . T(y)=fn(y) and E[T(Y)]=. T(Y) is the UMVUE of . So, it is the best estimator of .
35
THE MINIMUM VARIANCE UNBIASED ESTIMATOR
Let Y be a css for . Since Y is complete, there could be only a unique function of Y which is an UE of . Let U1(Y) and U2(Y) be two function of Y. Since they are UE’s, E(U1(Y)U2(Y))=0 imply W(Y)=U1(Y)U2(Y)=0 for all possible values of Y. Therefore, U1(Y)=U2(Y) for all Y.
36
Example Let X1,X2,…,Xn ~Poi(μ). Find UMVUE of μ. Solution steps:
Show that is css for μ. Find a statistics (such as S*) that is UE of μ and a function of S. Then, S* is UMVUE of μ by Lehmann-Scheffe Thm.
37
Note The estimator found by Rao-Blackwell Thm may not be unique. But, the estimator found by Lehmann-Scheffe Thm is unique.
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.