1. Standard Errors
Beside reporting a value of a point estimate we should consider some indication of its precision.
For this we usually quote standard error.
Standard error of an estimator is the standard deviation of its sampling distribution.
STA248 week 4

2. Example Suppose X1, X2,…, Xn are iid N(μ, σ2) and we are interested in
estimating μ.
We have seen that both the method of moments estimator and the
MLE for μ is
The standard error of is:
If σ is also an unknown parameter, we substitute an estimate for σ, say
The estimated standard error of is
STA248 week 4

3. Example
Bernoulli example...
STA248 week 4

4. Some Special Cases of Sampling Distributions
Suppose X1, X2,…, Xn are iid N(μ, σ2). The estimator of μ is We
have that
Suppose X1, X2,…, Xn are iid N(μ, σ2). The estimator of σ2 is s2 - the
sample standard deviation. The sampling distribution of s2 is given by
This is the Chi-squared distribution with parameter n­1. The
parameter is called the “degrees of freedom”.
Some properties of the Chi-squared distribution are…
STA248 week 4

5. Important Note
By the Central limit theorem, estimator that is the sum of iid random
variables will have approximately a Normal distribution for large n.
Example…
STA248 week 4

6. Bootstrap Estimates of Standard Errors
Suppose we have an estimator of a parameter and we want to
express its accuracy by its s.e. but its sampling distribution is too
complicated to derive theoretically.
A possible solution for this problem is to use Bootstrap – substitute computation for theory.
STA248 week 4

7. Parametric Bootstrap
Suppose data are realization of a random variable with a probability
distribution with density f (x | θ) with θ unknown.
We begin the bootstrap process by first estimating θ from the data to get
Next we simulate B “bootstrap samples” from the density f (x | θ)
with θ being replaced by and for each bootstrap sample we
calculate a “bootstrap estimate” of θ denoted by
Note that the bootstrap samples are always the same size as the original data set.
The bootstrap estimate of the s.e. of is the sample standard
deviation of the bootstrap estimates
STA248 week 4

8. Example
Consider a data set containing breakdown times of an insolative fluid between electrodes.
The theoretical model for this data assumes that this is an i.i.d sample from an exponential distribution…
The method of moment estimator of λ is….
We want the s.e of this estimator and for this we use parametric bootstrap (see R).
STA248 week 4

9. Nonparametric Bootstrap
If we could take an infinite number of samples of size n from the
probability distribution that generated the data and for each sample
find , we would know the sampling distribution of .
In the nonparametric bootstrap procedure we get bootstrap samples
of size n by re-sampling from the data.
Recall, the empirical distribution of the data puts probability mass
1/n at each data point and is used as the sampling distribution that
generated the data.
Re-sampling is sampling with replacement from this empirical
distribution.
See R for example…
STA248 week 4

10. Parametric Versus Nonparametric Bootstrap
In the parametric bootstrap we have to make an assumption about
the form of the distribution that generated the data
Non-parametric – if n is small can behave oddly.
STA248 week 4