MA/CS 375 Fall MA/CS 375 Fall 2002 Lecture 18 Sit in your groups

MA/CS 375 Fall Projects? Demo from each group

MA/CS 375 Fall Matrices and Systems

MA/CS 375 Fall Recall Given a matrix M then if it exists the inverse of a matrix M-1 is that matrix which satisfies:

MA/CS 375 Fall Examples If what is If A is an NxN matrix how can we calculate its inverse ?

MA/CS 375 Fall More Examples If what is More generally when does an inverse exist

MA/CS 375 Fall Example (Problems Calculating an Inverse) Previously we saw how Matlab can get wrong answers when finite precision effects come into play (monsters) The same type of behavior can be seen when Matlab calculates the inverse of a matrix. Let’s consider the simple matrix:

MA/CS 375 Fall Example cont The exact inverse of A can be calculated as: Now let’s see how well Matlab does:

MA/CS 375 Fall Example cont The inverse of A can be calculated as: Now let’s see how well this exact solution works in Matlab:

MA/CS 375 Fall Example cont With this formulation of the product of A and its inverse only satisfies the definition to 6 decimal places for delta=0.001

MA/CS 375 Fall Example cont Now let us compare the answer Matlab gives us using inv(A):

MA/CS 375 Fall Example cont Now let us compare the answer Matlab gives us using inv(A): Looks pretty good, doesn’t it!

MA/CS 375 Fall Example cont But not so quick. Now try A*inv(A) Now Matlab is only good to 6 decimal places.

MA/CS 375 Fall Example comments So what it is it about this matrix which is causing this odd behaviour?. For small delta it is nearly singular. For those with some background the determinant of A is: i.e. the inverse of A nearly does not exist Matlab is actually solving for a matrix near to A

MA/CS 375 Fall Example team exercise Test the accuracy of Matlab’s inv(A) Team 1 use delta=1e-2 Team 2 use delta = 1e-4 Team 3 use delta = 1e-6 Team 4 use delta= 1e-9 Team 5 use delta = 1e-11 Try both A*inv(A) and inv(A)*A

MA/CS 375 Fall Solving Triangular Systems So let’s go back to basics. There are some systems of equations which are “easy” to solve. One example is a system with a triangular matrix structure.

MA/CS 375 Fall Lower Triangular Systems equationsmatrix form

MA/CS 375 Fall Lower Triangular Systems equations Volunteer to solve this? i.e. find x,y so that these two equations are satisfied

MA/CS 375 Fall Lower Triangular Systems equations 1)note that we can easily figure out x 2)then knowing x we can easily figure out y. 3)i.e. 2x = 1  x = ½ 4)3x+2y = 2  y = ½(2-3x) = ½(2-3/2) = 1/4

MA/CS 375 Fall Larger Problem Volunteers?. 1)on the board 2)using Matlab

MA/CS 375 Fall General Triangular System (matrix form)

MA/CS 375 Fall General Triangular System We can write down an algorithm to solve this problem using the “Forward substitution algorithm”:

MA/CS 375 Fall Forward Substitution Algorithm 1) Solve the 1 st equation 2) Use the value from step 1 to solve the 2 nd equation 3) Use the values from steps 1 and 2 to solve the 3 rd equation 4) keep going until you solve the whole system.

MA/CS 375 Fall Solving Triangular Systems in Matlab Given a lower triangular matrix L Given a vector b Solve Lx = b for the unknown vector x

MA/CS 375 Fall Solving Triangular Systems in Matlab In action: Given a lower triangular matrix L Given a vector b Solve Lx = b for the unknown vector x

MA/CS 375 Fall Solving Triangular Systems in Matlab Observations Given a lower triangular matrix L Given a vector b Solve Lx = b for the unknown vector x The algorithm scales as O(N 2 ) We relied on the diagonal terms of L to be non-zero What happens if there is a zero diagonal term ?

MA/CS 375 Fall Class Exercise Write the lower triangular system solver using only one loop i.e. vectorize the inner loop Test your algorithm against the built in Matlab matrix division routine. You have 10 minutes from when I say go – hard limit (no hand in required) replace

MA/CS 375 Fall Upper Triangular Systems

MA/CS 375 Fall Upper Triangular Systems Volunteer: tell us how to solve this? (hint: this is not difficult)

MA/CS 375 Fall Example: Upper Triangular Systems

MA/CS 375 Fall Upper Triangular Systems 1) Solve the N th equation 2) Use the value from step 1 to solve the (N-1) th equation 3) Use the values from steps 1 and 2 to solve the (N-2) th equation 4) keep going until you solve the whole system.

MA/CS 375 Fall Matlab Code for Backward Substitution

MA/CS 375 Fall Testing Backwards Substitution

MA/CS 375 Fall Finite Precision Effects Notice that x(1) depends on the values of x(2),x(3),…,x(N) Remembering that round off rears its ugly head when we deal with (small+large) numbers Volunteer to design a U matrix and b vector, which do not give good answers when used in this algorithm. (goal is to give you some intuition for cases which will break an algorithm)

MA/CS 375 Fall Summary of Lecture 18 We have seen how finite precision can effect the calculation of an inverse matrix, by example. We have discussed algorithms to solve Lx=b and Ux=b We have seen how to vectorize part of the forward/backward substitution algorithms We tried to break the forward/backward substitution algorithms

MA/CS 375 Fall Next Lecture Ok – so we know how to solve L or U matrices We will see how to write fairly general matrices as a product of a lower and upper matrix. E.g. A = LU We will investigate LU factorization, including sensitivity to the structure of A Introduce the idea of pivoting