1. Least Squares Problems From Wikipedia, the free encyclopedia The method of least squares is a standard approach to the approximate solution of overdetermined systems, i.e., more equations than unknowns. The most important application is in data fitting The least-squares method was first described by Carl Friedrich Gauss around 1794 Legendre was the first to publish the method, however.

2. The Problem:

3. The Problem Range of A x Ax b

4. If we have 21 data points we can find a unique polynomial interpolant to these points by solving: Data-Fitting

5. Without changing the data points we can do better by reducing the degree of the polynomial In the previous example: Polynomial of degree 8: Polynomial Least Squares Fitting

6. Orthogonal Projection and the Normal Equations Theorem:

7. Pseudoinverse exists If A has full rank then Is called the Pseudoinverse, and Is the least squares solution

8. Four Algorithms 1.Find the Pseudoinverse 2.Solve the Normal Equation (A full rank): Then calculate Requires A to have full rank Is positive definite and we use the Cholesky factorization

9. Four Algorithms 3- QR Factorization: Reduced QR Orthogonal projector onto range(A)

11. Four Algorithms 4- SVD Reduced SVD Orthogonal projector onto range(A)