1. Stat
Lab 9

2. ? The form of f is known, but θ is unknown. Distribution: f(x; θ) x12

3. Point Estimation How to estimate θ ? Methods of finding estimators
Method of Moment estimation
Maximum Likelihood estimation
Point Estimation
Given observed data, only one single value is obtained.

4. However, in some sense, this single value of the estimator cannot provide us any useful information about the actual value of the unknown parameter.
If , then we cannot conclude that the true value is close to 1.2 or not.
The estimator is a function of r.v, so we will get another different single value when we draw another sample.
Based on (x1’,…, xn’),
So, the actual value of θ is close to 1.2 or 5.8? Or inbetween or far away from these two numbers?

5. How? Problem: Come from the variability of the estimator.
Useless? Why still consider the point estimator?
Problem: Come from the variability of the estimator.
In addition to the estimated value of the estimator,
we also have to consider the variance of the estimator.
How?
Use the single value and the variance of the estimator to form an interval that has a high probability to cover the unknown parameter.
This method including the variance of the point estimator is called interval estimation, or "confidence interval".

6. How to construct the interval?
For example,
Then
is the quantile of the standard normal distribution.
NO!!
μ is inside this interval with prob 0.95?
=> If all X are observed, i.e. x1,…,xn , then 95% C.I for μ is that
Probability is a measure of a random quantity. However, μ is fixed and is known.
μ is either inside or outside the interval.

7. Interpretation of C.I.
Given an observed x, mu is either in or not in the interval.
"0.95=P(|u - X|<z)=\int P(|u-x|<z |X=x)f(x) dx = \int I_{|u-x|<z}f(x) dx
<--1/n\sum I_{|u-x_i|<z}, where x_i are from f"
If we construct this random interval by drawing different samples repeatedly, say 100 times.
Then for each sample x, we have a quantity to measure if μ is inside the fixed interval based on the observed x or not. If yes, then 1; otherwise, 0.
1st sample: Y
2nd sample: Y
3rd sample: N
...
100 sample: Y
95% C.I. for μ means that μ is contained in 95 out of these 100 intervals. However, we dont know what these 95 intervals are.

8. Procedure to find the interval estimation/C.I.
Step 1: Find a point estimator of θ
Step 2: Find its EXACT distribution.
Step 3: Based on the exact distribution found in Step 2 to construct the C.I.
One problem in Step 2 is that it is often hard to find the exact distribution of the estimator.
So, we use the approximate distribution. Strictly speaking, we are more interested in finding the limiting distribution of the estimator, i.e. the distribution of the estimator when n goes to infinity.
Step 2': Hard to find the exact distribution. Then use the limiting distribution.
Step 3': Based on the limiting distribution to construct the C.I.

9. What is the Central Limit Theorem?
For example,
X1,...,Xn ~ F,
where F is unknown with E(X)=μ and Var(X)=1.
If the point estimator of μ is and now we don’t know its exact distribution, then we have to find the limiting distribution so that we can construct the C.I. for μ.
The limiting distribution can be found by the CLT.
What is the Central Limit Theorem?
As n goes to infinity,
if Xi ~ F with mean μ and variance σ2.
Assumptions: iid, finite mean and finite variance OR iid and finite E(X2).
So the approximate 95% C.I. for μ can be found by
The approximate 95% C.I. for μ is

10. If the estimator is MLE and it is hard to find its exact distribution, then we can use the limiting distribution of MLE.
Under some regularity conditions, we known that MLE has a limiting normal distribution.
where is called the Fisher Information matrix.

11. Discussion problems: