1. MA/CS 375 Fall 20021 MA/CS 375 Fall 2002 Lecture 21

2. MA/CS 375 Fall 20022 This Lecture Introduce the idea of pivoting This allows us to deal with the case where the algorithm we discussed broke down i.e. what to do to avoid the diagonal term we need to invert became very small or even zero.

3. MA/CS 375 Fall 20023 Matrices and Systems

4. MA/CS 375 Fall 20024 Example: Upper Triangular Systems Review

5. MA/CS 375 Fall 20025 Upper Triangular Systems 1) Solve the N th equation 2) Use the value from step 1 to solve the (N-1) th equation 3) Use the values from steps 1 and 2 to solve the (N-2) th equation 4) keep going until you solve the whole system. Review

6. MA/CS 375 Fall 20026 Backwards Substitution Review

7. MA/CS 375 Fall 20027 LU Factorization By now you should be persuaded that it is not difficult to solve a problem of the form Lx=b or Ux=b using forward/backward elimination Suppose we are given a matrix A and we are able to find a lower triangular matrix L and an upper triangular matrix U such that: A = LU Then to solve Ax=b we can solve two simple problems: Ly = b Ux = y (sanity check LUx = L(Ux) = Ly =b) Review

8. MA/CS 375 Fall 20028 LU Factorization (in words) Construct a sequence of multiplier matrices (M 1, M 2, M 3, M 4,…., M N-1 ) such that: (M N-1 …. M 4 M 3 M 2 M 1 )A is upper triangle. A suitable candidate for M 1 is the matrix ( 4 by 4 case): Review

9. MA/CS 375 Fall 20029 LU Factorization Sequence cont. Review

10. MA/CS 375 Fall 200210 LU Factorization cont. What does M look like: Review

11. MA/CS 375 Fall 200211 LU Factorization cont. What does M -1 look like: Review Notice: M and its inverse are very closely related.

12. MA/CS 375 Fall 200212 LU Factorization cont. In summary: 1)set L=M -1 2)set U=MA 3)the L is upper triangular 4)the U is lower triangular 5)and A = LU, we are done. Review

13. MA/CS 375 Fall 200213 Matlab Routine For LU Factorization Build L and U A is modified in place Total cost O(N 3 ) ( we can get this by noticing the triply nested loop) Review

14. MA/CS 375 Fall 200214 Matlab Routine For LU Factorization And Solution of Ax=b Build rhs Solve Ly=b Solve Ux=y Review

15. MA/CS 375 Fall 200215 Class Example Try using your LU solver on the following matrix: The inverse is: What L and U do you get?

16. MA/CS 375 Fall 200216 When Basic LU Does Not Work Remember that we are required to compute 1/A(i,i) for all the successive diagonal terms What happens when we get to the i’th diagonal term and this turns out to be 0 or very small ?? At any point we can swap the i’th row with one lower down which does not have a zero (or small entry) in the i’th column. This is known as partial pivoting. New stuff:

17. MA/CS 375 Fall 200217 Partial Pivoting Remember before that we constructed a set of M matrices so that is upper triangle Now we construct a sequence of M matrices and P matrices so that: is upper triangle. (M N-1 …. M 4 M 3 M 2 M 1 )A (M N-1 P N-1 …. M 4 P 4 M 3 P 3 M 2 P 2 M 1 P 1 )A

18. MA/CS 375 Fall 200218 Example of a Permutation Matrix P What does P do to a matrix A under multiplication ?

19. MA/CS 375 Fall 200219 Example of a Permutation Matrix P What does P do to a matrix A under multiplication ?

20. MA/CS 375 Fall 200220 Example of a Permutation Matrix P In this case P has swapped the two bottom rows of A.

21. MA/CS 375 Fall 200221 LU with Partial Pivoting

22. MA/CS 375 Fall 200222 LU with Partial Pivoting 1)find pivot 2)swap rows 3)Do elimination

23. MA/CS 375 Fall 200223 LU with Partial Pivoting 1)build matrix A 2)call our LU routine 3)check that: A(piv,:) = L*U 4)Looks good

24. MA/CS 375 Fall 200224 Team Exercise Make sure you all have a copy of the.m files on your laptops (courtesy of Coutsias) Locate GEpiv For N=100,200,400,600 –build a random NxN matrix A –time how long GEpiv takes to factorize A end Plot the cputime taken per test against N using loglog On the same graph plot N^3 using loglog Observation

25. MA/CS 375 Fall 200225 Notes on LU Factorization Big fuss over nothing, right ? What happens if I want to solve 40000 problems with Ax=b for 40000 different b vectors?. Cost is O(40000*N 2 ) for the solves and O(N 3 ) for the LU factorization. i.e. the cost of the factorization is negligable compared to the cost of the solves !!.

26. MA/CS 375 Fall 200226 From Now On When you need to solve a square non- symmetric system of equations (Ax=b) with lots of different b vectors use the Matlab built in routine lu: [L,U,P] = lu(A); y = L\(Pb); x = U\y; For a one time solve use: x=A\b

27. MA/CS 375 Fall 200227 Interlude on Norms We all know that the “size” of 2 is the same as the size of -2, namely: ||2|| = sqrt(2*2) = 2 ||-2|| = sqrt((-2)*(-2)) = 2 The ||..|| notation is known as the norm function.

28. MA/CS 375 Fall 200228 Norm of A Vector We can generalize the scalar norm to a vector norm, however now there are more choices for a vector :

29. MA/CS 375 Fall 200229 Norm of A Vector Matlab command: 1)norm(x,1) 2)norm(x,2) 3)norm(x,inf)

30. MA/CS 375 Fall 200230 Norm of A Matrix We can generalize the vector norm to a matrix norm, however now there are more choices for a matrix :

31. MA/CS 375 Fall 200231 Norm of A Matrix Matlab command: 1)norm(A,1) 2)norm(A,2) 3)norm(A,inf) 4)norm(A,fro)

32. MA/CS 375 Fall 200232 Matrix Norm in Action Theorem 5: (page 185 van Loan) –If is the stored version of then where and i.e. an error of order ||A|| 1 eps occurs when a real matrix is stored in floating point.

33. MA/CS 375 Fall 200233 Next Lecture We will see matrix norms in action… Those of you who own a digital camera, please bring it in. Otherwise bring a cd with some picture files on (jpeg, gif, tiff,…) – I have the previously taken pictures too.. We will start talking about interpolation