1. MA/CS 375 Fall 20021 MA/CS 375 Fall 2002 Lecture 18 Sit in your groups

2. MA/CS 375 Fall 20022 Projects? Demo from each group

3. MA/CS 375 Fall 20023 Matrices and Systems

4. MA/CS 375 Fall 20024 Recall Given a matrix M then if it exists the inverse of a matrix M-1 is that matrix which satisfies:

5. MA/CS 375 Fall 20025 Examples If what is If A is an NxN matrix how can we calculate its inverse ?

6. MA/CS 375 Fall 20026 More Examples If what is More generally when does an inverse exist

7. MA/CS 375 Fall 20027 Example (Problems Calculating an Inverse) Previously we saw how Matlab can get wrong answers when finite precision effects come into play (monsters) The same type of behavior can be seen when Matlab calculates the inverse of a matrix. Let’s consider the simple matrix:

8. MA/CS 375 Fall 20028 Example cont The exact inverse of A can be calculated as: Now let’s see how well Matlab does:

9. MA/CS 375 Fall 20029 Example cont The inverse of A can be calculated as: Now let’s see how well this exact solution works in Matlab:

10. MA/CS 375 Fall 200210 Example cont With this formulation of the product of A and its inverse only satisfies the definition to 6 decimal places for delta=0.001

11. MA/CS 375 Fall 200211 Example cont Now let us compare the answer Matlab gives us using inv(A):

12. MA/CS 375 Fall 200212 Example cont Now let us compare the answer Matlab gives us using inv(A): Looks pretty good, doesn’t it!

13. MA/CS 375 Fall 200213 Example cont But not so quick. Now try A*inv(A) Now Matlab is only good to 6 decimal places.

14. MA/CS 375 Fall 200214 Example comments So what it is it about this matrix which is causing this odd behaviour?. For small delta it is nearly singular. For those with some background the determinant of A is: i.e. the inverse of A nearly does not exist Matlab is actually solving for a matrix near to A

15. MA/CS 375 Fall 200215 Example team exercise Test the accuracy of Matlab’s inv(A) Team 1 use delta=1e-2 Team 2 use delta = 1e-4 Team 3 use delta = 1e-6 Team 4 use delta= 1e-9 Team 5 use delta = 1e-11 Try both A*inv(A) and inv(A)*A

16. MA/CS 375 Fall 200216 Solving Triangular Systems So let’s go back to basics. There are some systems of equations which are “easy” to solve. One example is a system with a triangular matrix structure.

17. MA/CS 375 Fall 200217 Lower Triangular Systems equationsmatrix form

18. MA/CS 375 Fall 200218 Lower Triangular Systems equations Volunteer to solve this? i.e. find x,y so that these two equations are satisfied

19. MA/CS 375 Fall 200219 Lower Triangular Systems equations 1)note that we can easily figure out x 2)then knowing x we can easily figure out y. 3)i.e. 2x = 1  x = ½ 4)3x+2y = 2  y = ½(2-3x) = ½(2-3/2) = 1/4

20. MA/CS 375 Fall 200220 Larger Problem Volunteers?. 1)on the board 2)using Matlab

21. MA/CS 375 Fall 200221 General Triangular System (matrix form)

22. MA/CS 375 Fall 200222 General Triangular System We can write down an algorithm to solve this problem using the “Forward substitution algorithm”:

23. MA/CS 375 Fall 200223 Forward Substitution Algorithm 1) Solve the 1 st equation 2) Use the value from step 1 to solve the 2 nd equation 3) Use the values from steps 1 and 2 to solve the 3 rd equation 4) keep going until you solve the whole system.

24. MA/CS 375 Fall 200224 Solving Triangular Systems in Matlab Given a lower triangular matrix L Given a vector b Solve Lx = b for the unknown vector x

25. MA/CS 375 Fall 200225 Solving Triangular Systems in Matlab In action: Given a lower triangular matrix L Given a vector b Solve Lx = b for the unknown vector x

26. MA/CS 375 Fall 200226 Solving Triangular Systems in Matlab Observations Given a lower triangular matrix L Given a vector b Solve Lx = b for the unknown vector x The algorithm scales as O(N 2 ) We relied on the diagonal terms of L to be non-zero What happens if there is a zero diagonal term ?

27. MA/CS 375 Fall 200227 Class Exercise Write the lower triangular system solver using only one loop i.e. vectorize the inner loop Test your algorithm against the built in Matlab matrix division routine. You have 10 minutes from when I say go – hard limit (no hand in required) replace

28. MA/CS 375 Fall 200228 Upper Triangular Systems

29. MA/CS 375 Fall 200229 Upper Triangular Systems Volunteer: tell us how to solve this? (hint: this is not difficult)

30. MA/CS 375 Fall 200230 Example: Upper Triangular Systems

31. MA/CS 375 Fall 200231 Upper Triangular Systems 1) Solve the N th equation 2) Use the value from step 1 to solve the (N-1) th equation 3) Use the values from steps 1 and 2 to solve the (N-2) th equation 4) keep going until you solve the whole system.

32. MA/CS 375 Fall 200232 Matlab Code for Backward Substitution

33. MA/CS 375 Fall 200233 Testing Backwards Substitution

34. MA/CS 375 Fall 200234 Finite Precision Effects Notice that x(1) depends on the values of x(2),x(3),…,x(N) Remembering that round off rears its ugly head when we deal with (small+large) numbers Volunteer to design a U matrix and b vector, which do not give good answers when used in this algorithm. (goal is to give you some intuition for cases which will break an algorithm)

35. MA/CS 375 Fall 200235 Summary of Lecture 18 We have seen how finite precision can effect the calculation of an inverse matrix, by example. We have discussed algorithms to solve Lx=b and Ux=b We have seen how to vectorize part of the forward/backward substitution algorithms We tried to break the forward/backward substitution algorithms

36. MA/CS 375 Fall 200236 Next Lecture Ok – so we know how to solve L or U matrices We will see how to write fairly general matrices as a product of a lower and upper matrix. E.g. A = LU We will investigate LU factorization, including sensitivity to the structure of A Introduce the idea of pivoting