Numerical Computation Lecture 9: Vector Norms and Matrix Condition Numbers United International College

Review During our Last Class we covered: – Operation count for Gaussian Elimination, LU Factorization – Accuracy of Matrix Methods – Readings: Pav, section Moler, section 2.8

Today We will cover: – Vector and Matrix Norms – Matrix Condition Numbers – Readings: Pav, section 1.3.2, 1.3.3, Moler, section 2.9

Vector Norms A vector norm is a quantity that measures how large a vector is (the magnitude of the vector). For a number x, we have |x| as a measurement of the magnitude of x. For a vector x, it is not clear what the “ best ” measurement of size should be. Note: we will use bold-face type to denote a vector. ( x )

Vector Norms Example: x = ( 4 -1 ) – is the standard Pythagorean length of x. This is one possible measurement of the size of x. x

Vector Norms Example: x = ( 4 -1 ) – |4| + |-1| is the “ Taxicab ” length of x. This is another possible measurement of the size of x. x

Vector Norms Example: x = ( 4 -1 ) – max(|4|,|-1|) is yet another possible measurement of the size of x. x

Vector Norms A vector norm is a quantity that measures how large a vector is (the magnitude of the vector). Definition: A vector norm is a function that takes a vector and returns a non-zero number. We denote the norm of a vector x by The norm must satisfy: – Triangle Inequality: – Scalar: – Positive:,and = 0 only when x is the zero vector.

Our previous examples for vectors in R n : Manhattan Euclidean Chebyshev All of these satisfy the three properties for a norm. Vector Norms

Vector Norms Example

Definition: The L p norm generalizes these three norms. For p > 0, it is defined on R n by: p=1 L 1 norm p=2 L 2 norm p= ∞ L ∞ norm Vector Norms

Distance

Class Practice: – Find the L 2 distance between the vectors x = (1, 2, 3) and y = (4, 0, 1). – Find the L ∞ distance between the vectors x = (1, 2, 3) and y = (4, 0, 1). Distance

The answer depends on the application. The 1-norm and ∞-norm are good whenever one is analyzing sensitivity of solutions. The 2-norm is good for comparing distances of vectors. There is no one best vector norm! Which norm is best?

In Matlab, the norm function computes the L p norms of vectors. Syntax: norm(x, p) >> x = [ ]; >> n = norm(x,2) n = >> n = norm(x,1) n = 8 >> n = norm(x, inf) n = 4 Matlab Vector Norms

Definition: Given a vector norm the matrix norm defined by the vector norm is given by: Example: Matrix Norms

Example: What does a matrix norm represent? It represents the maximum “ stretching ” that A does to a vector x -> (Ax). Matrix Norms

|| A || > 0 if A ≠ O || c A || = | c| * ||A || if A ≠ O || A + B || ≤ || A || + || B || || A B || ≤ || A || * ||B || || A x || ≤ || A || * ||x || Matrix Norm Properties

Multiplication of a vector x by a matrix A results in a new vector Ax that can have a very different norm from x. The range of the possible change can be expressed by two numbers, =||A|| Here the max, min are over all non-zero vectors x. Matrix

Definition: The condition number of a nonsingular matrix A is given by: κ(A) = M/m by convention if A is singular (m=0) then κ(A) = ∞. Note: If we let Ax = y, then x = A -1 y and Matrix Condition Number

Theorem: The condition number of a nonsingular matrix A can also be given as: κ(A) = || A || * || A -1 || Proof: κ(A) = M/m. Also, M = ||A|| and by the previous slide m = 1 / (||A -1 ||). QED Matrix Condition Number

Properties of the Matrix Condition Number For any matrix A, κ(A) ≥ 1. For the identity matrix, κ(I) = 1. For any permutation matrix P, κ(P) =1. For any matrix A and nonzero scalar c, κ(c A) = κ(A). For any diagonal matrix D = diag(d i ), κ(D) = (max|d i |)/( min | d i | )

What does the condition number tell us? The condition number is a good indicator of how close is a matrix to be singular. The larger the condition number the closer we are to singularity. It is also very useful in assessing the accuracy of solutions to linear systems. In practice we don ’ t really calculate the condition number, it is merely estimated, to perhaps within an order of magnitude.

Condition Number And Accuracy Consider the problem of solving Ax = b. Suppose b has some error, say b + δb. Then, when we solve the equation, we will not get x but instead some value near x, say x + δx. A(x + δx) = b + δb Then, A(x + δx) = b + δb

Condition Number And Accuracy Class Practice: Show:

Condition Number And Accuracy The quantity ||δb||/||b|| is the relative change in the right-hand side, and the quantity ||δx||/||x|| is the relative error caused by this change. This shows that the condition number is a relative error magnification factor. That is, changes in the right-hand side of Ax=b can cause changes κ(A) times as large in the solution.