1. ELG5377 Adaptive Signal Processing
Lecture 14: Method of Least Squares Continued

2. Introduction In the last lecture, we saw that
This filter minimizes the sum of error squares between the filter’s output and its desired output.
We also saw that the filter’s input and output are orthogonal to the error signal time sequence.
Therefore the energy in the desired is the sum of the energy in the output plus the energy in the error signal on the duration of interest.

3. The Energy of the Output, Eest.

4. The Energy of the Error Signal, Emin.

5. Properties of the time average correlation matrix
It is a Hermitian matrix
It is nonnegative definite
Is nonsingular only if its determinant is 0
Its eigenvalues are all real and nonnegative (from properties 1 and 2)
It is the product of two rectangular Toeplitz matrices that are Hermitian transposes of each other

6. Reformulation of the normal equations in terms of data matrices

7. Example

8. Properties of Least Squares Estimates
The LS estimate, , is unbiased provided the measurement error has zero mean.
Proof on blackboard
When the measurement error process is white with 0 mean and variance s2, the covariance matrix of the least squares estimate is s2j-1.
When the measurement error process is white with 0 mean, the best squares estimate is the best linear unbiased estimate