1. 6 6.5 © 2016 Pearson Education, Ltd. Orthogonality and Least Squares LEAST-SQUARES PROBLEMS

2. Slide 6.5- 2 © 2016 Pearson Education, Ltd. LEAST-SQUARES PROBLEMS

3. Slide 6.5- 3 LEAST-SQUARES PROBLEMS  Solution of the General Least-Squares Problem  Given A and b, apply the Best Approximation Theorem to the subspace Col A.  Let © 2016 Pearson Education, Ltd.

4. Slide 6.5- 4 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

5. Slide 6.5- 5 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

6. Slide 6.5- 6 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Since each is a row of A T, (2)  Thus  These calculations show that each least-squares solution of satisfies the equation (3)  The matrix equation (3) represents a system of equations called the normal equations for.  A solution of (3) is often denoted by.

7. Slide 6.5- 7 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Theorem 13: The set of least-squares solutions of coincides with the nonempty set of solutions of the normal equation.  Proof: The set of least-squares solutions is nonempty and each least-squares solution satisfies the normal equations.  Conversely, suppose satisfies.  Then satisfies (2), which shows that is orthogonal to the rows of A T and hence is orthogonal to the columns of A.  Since the columns of A span Col A, the vector is orthogonal to all of Col A.

8. Slide 6.5- 8 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

9. Slide 6.5- 9 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Example 1: Find a least-squares solution of the inconsistent system for  Solution: To use normal equations (3), compute:

10. Slide 6.5- 10 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Then the equation becomes

11. Slide 6.5- 11 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM  Row operations can be used to solve the system on the previous slide, but since A T A is invertible and, it is probably faster to compute and then solve as

12. Slide 6.5- 12 © 2016 Pearson Education, Ltd. SOLUTION OF THE GENREAL LEAST-SQUARES PROBLEM

13. Slide 6.5- 13 © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS

14. Slide 6.5- 14 © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS  Now that is known, we can solve.  But this is trivial, since we already know weights to place on the columns of A to produce.  It is clear from (5) that

15. Slide 6.5- 15 © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS

16. Slide 6.5- 16 © 2016 Pearson Education, Ltd. ALTERNATIVE CALCULATIONS OF LEAST- SQUARES SOLUTIONS  The columns of Q form an orthonormal basis for Col A. (by Theorem 12)  Hence, by Theorem 10, QQ T b is the orthogonal projection of b onto Col A.  Then, which shows that is a least-squares solution of.  The uniqueness of follows from Theorem 14.