1. CPSC 491 Xin Liu November 19, 2010

2. Norm Norm is a generalization of Euclidean length in vector space Definition 2

3. l p -Norms Definition Examples l 1 -norm l 2 -norm (Euclidean length) l ∞-norm 3

4. Matrix norms Definition A matrix norm is a function ||  || : R mxn  R that satisfies Frobenius norm Matrix norms induced by vector norms (see next page) 4

5. Induced Matrix Norms Definition The supremum of ||Ax|| / ||x|| The “amplification factor” of vector norms Examples The matrix norms of A induced by L1, L2, L∞ norms 5

6. Induced Matrix Norms 6

7. Induced Matrix Norms The 1-Norm of a matrix A Write A in terms of its columns On the sphere ||x|| 1 = 1, we have When x = e j, the upper bound is attained. Therefore, 7

8. Properties of Matrix Norms For (1) norms induced by vector norms and (2) Frobenius norm, we have the following conclusions Bound of ||AB|| ||AB|| ≤||A|| ||B|| Invariance under Unitary Multiplication If Q is a unitary matrix, we have ||QA|| = ||AQ|| = ||A|| 8

9. Singular Value Decomposition Definition A singular value decomposition (SVD) of A is a factorization The diagonal entries of Σ are nonnegative and in nonincreasing order Existence and Uniqueness 9

10. Geometric interpretation The image AS of the unit sphere S under any mxn matrix A is a hyperellipse. The singular values of A σ 1, σ 2, …, σ n, are the lengths of the principal semiaxes of AS The left singular vectors of A, {u 1, u 2, …, u n } oriented in the directions of the principal semiaxes of AS The right singular vectors of A, {v 1, v 2, …, v n } are the pre-images of the principal semiaxes of AS, i.e., Av j = σ j u j 10

11. Reduced SVD If m ≥ n, a full-ranked matrix A has rank n, exactly n singular values are nonzero. We can reduce the full decomposition by removing the zero rows in Σ, and related columns in U  A mxn U cap mxn Σ cap nxn V nxn 11