CPSC 491 Xin Liu November 22, 2010

A change of Bases A mxn =UΣV T, U mxm, V nxn are unitary matrixes Σ mxn is a diagonal matrix Columns of a unitary matrix form a basis Any b in R m can be expanded in {u 1, u 2, …, u m } b=Ub’ b’=U T b Any x in R n can be expanded in {v 1, v 2, …, v m } x=Vx’ x’=V T x b=Ax U T b = U T Ax = U T UΣV T x b’= Σx’ A reduces to the diagonal matrix Σ when the range is expressed in the basis of columns of U and the domain is expressed in the basis of columns of V 2

Matrix Properties via SVD Theorem 1: The rank of A is r, the # of nonzero singular values. Proof: A mxn =UΣV T Rank (Σ) = r U, V are of full rank Theorem 2: range (A) = and null (A) = Theorem 3: ||A|| 2 = σ 1 and ||A|| F = sqrt (σ 1 2 +σ … + σ r 2 ) 3

Matrix Properties via SVD Theorem 4: The nonzero singular values of A are the square roots of the nonzero eigenvalues of A T A or AA T if Ax = λx (x is non-zero vector), then λ is an eigenvalue of A Theorem 5: If A = A T, then the singular values of A are the absolute values of the eigenvalues of A. Theorem 6: For A mxm, |det(A)| = Π i=1 m σ i Compute the determinant Proof: |det (A)| = |det (UΣV T )| = |det (U)| |det(Σ)| |det(V T )| = |det(Σ)| = Π i=1 m σ i 4

Low-Rank Approximations Theorem 7: A is the sum of r rank-one matrices A = Σ j=1 r σ j u j v j T Proof: Σ = diag(σ 1, 0, …, 0) + … + diag(0,..0, σ r, 0, …, 0) matrix multiplications The partial sum captures as much of the energy of A as possible “Energy” is defined by either the 2-norm or the Frobenius norm For any 0 ≤ v ≤ r 5

Applications Determine the rank of a matrix Find an orthonormal basis of a range/nullspace of a matrix Solve linear equation systems Compute ||A|| 2 Least squares fitting 6