1. The Regression Model
Suppose we wish to estimate the parameters of the following
relationship:
A common method is to choose parameters to minimise the
residual sum of squares:

2. This can be shown to yield the following pair of equations
known as the least-squares normal equations.
Solving these equations yields the least-squares estimates:

4. Distribution of the OLS estimator
First we will establish the conditions under which the OLS
estimator is:
1. Unbiased
2. Efficient The variance is the lowest in the class
of linear unbiased estimators.
These results will depend on the Gauss-Markov assumptions.
The Gauss-Markov assumptions are a set of assumptions
about the nature of the error term in the regression equation.

5. The Gauss-Markov assumptions
Under a specific set of assumptions the OLS estimates can
be shown to be the best linear unbiased estimates (BLUE).
The error has expected value zero
The errors are serially uncorrelated
The errors have constant variance
The X variable is non-stochastic
(fixed in repeated samples)

6. Unbiasedness
The proof of the unbiasedness of the OLS estimator relies
on only two of the Gauss-Markov assumptions.
These are assumptions 1 and 4.
Efficiency
The proof of the efficiency of the OLS estimator relies on
all the Gauss-Markov assumptions

7. Distribution of the OLS estimator
To derive the distribution of the OLS estimator we need to
make some assumption about the distribution of the errors.
If we assume that the errors follow a normal distribution then
we can show that the OLS estimates are also normally
distributed.

8. Effects of increasing the sample size on the distribution of
the OLS estimator

9. Hypothesis Testing
Suppose we wish to test
Under the null hypothesis
However, in practice we don’t know the error variance.

10. Distribution of the OLS estimator when the error variance
is unknown
If the sample variance is unknown then we can replace it with
an unbiased estimator.
When this is used then we have: