1. Outline
Basic Theories on the Subspace
Subspace projection

2. W = span{u1, u2, …, um} = {a1u1+a2u2 +…+amum}
Subspace
The Set of a Space, but Also a Space
Closure on addition and scalar multiplication
Zero elements belongs to the set …
Representation of a subspace: the set of all linear combinations on a certain set
W = span{u1, u2, …, um} = {a1u1+a2u2 +…+amum}
Other representations?

3. Subspace Remove Redundant Vectors from the Set
W = span{u1, u2, …, um} = {a1u1+a2u2 +…+amum}
Vector Removal Criteria
If some vectors are the linear combination of other vectors, then all other vectors also spans the same vector space
Some linearly independent set of vectors also form the same subspace

4. Subspace Properties Orthogonality of Subspace: Si ┴Sj
Any two vectors in the subspaces Si and Sj are orthogonal to each other
Example: x-y plane and z axis
Orthogonal complement
Set of vectors orthogonal to S: S┴ = {x|xTy = 0, y in S}
Orthogonal Complement Also a Subspace
How to prove?

5. Subspace Properties Dimension of Subspace and Its Orthogonal?
dim(S) + dim(S┴) = dim(V)
How to Prove this? …
Orthogonal Subspace and EVD
Null(A - λI), the set of vectors u such that (A - λI)u = 0
the eigenspace corresponding to eigenvalue λ

6. Subspace Projection The Subspace Spanned by Column Vectors of A
Subspace: S = span(A) = {Au}
Subspace Projection of Vector x
The vector in S = span(A) closest to vector x
Least square method: minu||x – Au||2
Least Square Solution:
u = (AHA)-1AHx
Projection Matrix: PS = A(AHA)-1AH

7. Subspace Projection Projection Matrix: PS = A(AHA)-1AH
Projection^2 = Projection: PSPS = PS
Orthogonal Matrix Operation：
Matrix: I - PS = I - A(AHA)-1AH
Properties:
(I - PS) (I - PS) = (I - PS)
(I - PS) PS = 0

8. Subspace Projection Distances of Two Subspaces
More Discussions: what is this stand for?
Angle between the two subspaces?
The angle on all dimensions? …

9. Column and Row Space Column space: Col(A) Row space: Row(A)
the subspace spanned by all columns of A
Row space: Row(A)
the subspace spanned by all rows of A
Zero space: Null(A) = {x|Ax = 0}
Properties on Column and Row Spaces
Null(A) = (Row(A))┴
Null(AH) = (Col(A))┴

10. Span{a1, a2,…, ak} = Span{q1, q2,…, qk}
Basic Results
Rank(A) + dim[Null(A)] = n
Let S = span(A),
We have: Rank(A) = dim(S), dim[Null(A)] = dim(S┴)
dim(S) + dim(S┴) = n  Rank(A) + dim[Null(A)] = n
Full rank matrix A, QR Decomposition A = QR: columns of A and Q
Span{a1, a2,…, ak} = Span{q1, q2,…, qk}