Outline Basic Theories on the Subspace Subspace projection

W = span{u1, u2, …, um} = {a1u1+a2u2 +…+amum} Subspace The Set of a Space, but Also a Space Closure on addition and scalar multiplication Zero elements belongs to the set … Representation of a subspace: the set of all linear combinations on a certain set W = span{u1, u2, …, um} = {a1u1+a2u2 +…+amum} Other representations?

Subspace Remove Redundant Vectors from the Set W = span{u1, u2, …, um} = {a1u1+a2u2 +…+amum} Vector Removal Criteria If some vectors are the linear combination of other vectors, then all other vectors also spans the same vector space Some linearly independent set of vectors also form the same subspace

Subspace Properties Orthogonality of Subspace: Si ┴Sj Any two vectors in the subspaces Si and Sj are orthogonal to each other Example: x-y plane and z axis Orthogonal complement Set of vectors orthogonal to S: S┴ = {x|xTy = 0, y in S} Orthogonal Complement Also a Subspace How to prove?

Subspace Properties Dimension of Subspace and Its Orthogonal? dim(S) + dim(S┴) = dim(V) How to Prove this? … Orthogonal Subspace and EVD Null(A - λI), the set of vectors u such that (A - λI)u = 0 the eigenspace corresponding to eigenvalue λ

Subspace Projection The Subspace Spanned by Column Vectors of A Subspace: S = span(A) = {Au} Subspace Projection of Vector x The vector in S = span(A) closest to vector x Least square method: minu||x – Au||2 Least Square Solution: u = (AHA)-1AHx Projection Matrix: PS = A(AHA)-1AH

Subspace Projection Projection Matrix: PS = A(AHA)-1AH Projection^2 = Projection: PSPS = PS Orthogonal Matrix Operation： Matrix: I - PS = I - A(AHA)-1AH Properties: (I - PS) (I - PS) = (I - PS) (I - PS) PS = 0

Subspace Projection Distances of Two Subspaces More Discussions: what is this stand for? Angle between the two subspaces? The angle on all dimensions? …

Column and Row Space Column space: Col(A) Row space: Row(A) the subspace spanned by all columns of A Row space: Row(A) the subspace spanned by all rows of A Zero space: Null(A) = {x|Ax = 0} Properties on Column and Row Spaces Null(A) = (Row(A))┴ Null(AH) = (Col(A))┴

Span{a1, a2,…, ak} = Span{q1, q2,…, qk} Basic Results Rank(A) + dim[Null(A)] = n Let S = span(A), We have: Rank(A) = dim(S), dim[Null(A)] = dim(S┴) dim(S) + dim(S┴) = n  Rank(A) + dim[Null(A)] = n Full rank matrix A, QR Decomposition A = QR: columns of A and Q Span{a1, a2,…, ak} = Span{q1, q2,…, qk}