1. Linear Spaces Row and Columns Spaces
From: D.A. Harville, Matrix Algebra from a Statistician’s Perspective, Springer. Chapter 4

2. Definitions - I
Column Space of mxn matrix A ≡ set of all m-dimensional column vectors that can be expressed as linear combinations of columns of A
Row Space of mxn matrix A ≡ set of all n-dimensional row vectors that can be expressed as linear combinations of rows of A

3. Definitions - II
Linear Spaces: A non-empty set V of matrices of the same dimensions is a linear space if:
For every A in V, and every B in V, A+B is in V
For every A in V, and every scalar k, kA is in V
A1,…,Ak in V and scalars x1,…,xk  x1A1+…+xkAk is in V
Examples
Column space of any mxn matrix is a linear space of mx1 vectors
Set containing all mxn matrices is a linear space
Set of all nxn symmetric matrices is a linear space (sums and scalar multiples)
All linear spaces contain the null matrix of correct dimension
{0} is a linear space with only the null matrix included

4. Notation C(A) ≡ Column space of the matrix A
R(A) ≡ Row space of the matrix A
Rmn ≡ Linear space of all mxn matrices
Rn ≡ Linear space of all nx1 column vectors (1xn row vectors)
R(In) = Rn ≡ Linear space of all 1xn row vectors
C(In) = Rn ≡ Linear space of all nx1 column vectors
x  S  x is an element of S
x S  x is not an element of S

5. 2 Lemmas Involving Row/Column Spaces and Linear Spaces
Lemma For any matrix A, y  C(A) if and only if y’  R(A’)
y  C(A)  y = Ax for some x  y’ = (Ax)’ = x’A’  y’  R(A’)
y’  R(A’)  y’ = x’A’ for some x  (y’)’ = (x’A’)’ = Ax  y = Ax  y  C(A)
Lemma B ≡ mxn V ≡ Linear space of mxn matrices Then for any A  V, A+B  V iff B  V
If A  V and B  V, then A+B  V by definition of a linear space
If A  V and A+B  V then k1A + k2(A+B)  V by definition of a linear space.
Let k1 = -1, k2 = 1  k1A + k2(A+B) = B and thus B  V

6. Subspaces
Subset U of a linear space V is termed a subspace of V if it is a linear space.
Trivial Cases: (1) The null set {0} and (2) the entire set V
Column space C(A) of mxn matrix A is a subspace of Rm (set of all m- dimensional vectors)
Row space R(A) of mxn matrix A is a subspace of Rn (set of all n- dimensional vectors)
For any 2 subsets, S and T of a given set (say 2 subspaces of Rmxn), S is contained in T if all elements of S are elements of T (S  T)
If S  T and T  S, then S = T

7. Results Involving Subspaces
Lemma A ≡ mxn Then for any subspaces U of Rm and V of Rn :
C (A)  U iff every column of A is in U, R(A)  V iff every row of A is in V where A = [a1 … an]
If C (A)  U then all ai  U (Let xi = (0,…,0,1,0,…,0)’)
If ai  U i=1,…,n then y  C (A)  y = Ax  y = x1a1 + … + xnan and since U is linear space, y  U
Lemma A ≡ mxn B ≡ mxp C ≡ qxn :
C (B)  C (A) iff there exists nxp F such that B = AF R(C)  R(A) iff there exists qxm L such that C = LA
Corollary For any A ≡ mxn F ≡ nxp L ≡ qxm : C (AF)  C (A) and R(LA)  R(A)
Corollary A ≡ mxn E ≡ nxk F ≡ nxp L ≡ qxm T ≡ sxm :
If C (E)  C (F) then C (AE)  C (AF) and if C (E) = C (F) then C (AE) = C (AF)
If R(L)  R(T) then R(LA)  R(TA) and if R(L) = R(T) then R(LA) = R(TA)
Lemma A ≡ mxn B ≡ mxp : (1) C (A)  C (B) iff R(A’)  R(B’) (2) C (A) = C (B) iff R(A’) = R(B’)
(1)  A = BF  A’ = (BF)’ = F’B’

8. Bases Span of a finite set of matrices of common dimensions:
Finite nonempty set {A1,…,Ak}: set of all matrices being linear combinations of {A1,…,Ak}
Empty set: {0} (unique basis – see below for definition of basis)
Span of a finite set S of matrices written as sp(S) which is a linear space
sp({A1,…,Ak}) ≡ sp(A1,…,Ak) ≡ span of the set {A1,…,Ak}
Finite set S of matrices in linear space V spans V if sp(S) = V
Basis for V is a finite linearly independent set of matrices in V that spans V
C(A) for A ≡ mxn is spanned by the set of its n columns. If lin. indep.  basis
R(A) for A ≡ mxn is spanned by the set of its m rows. If lin. indep.  basis
If the columns or rows of A are not linearly independent, they are not a basis

9. Natural Bases for Rmn and Linear Space of nxn Symmetric Matrices

10. Results Involving a Basis - I
Lemma A1,…,Ap and B1,…,Bq in linear space V
If {A1,…,Ap} spans V, then so does {A1,…,Ap,B1,…,Bq}
If {A1,…,Ap,B1,…,Bq} spans V and if B1,…,Bq are linear functions of A1,…,Ap then {A1,…,Ap} spans V
Theorem V ≡ linear space spanned by a set of r matrices. Let S ≡ set of k lin. indep. Matrices in V
Then k ≤ r and if k = r, then S is a basis for V
Corollary The number of matrices in a linearly independent set of mn matrices cannot exceed mn.
Number of matrices in a linearly independent set of nn symmetric matrices cannot exceed n+n(n-1)/2 = n(n+1)/2
Theorem Every linear space of mn matrices has a basis
Lemma Matrix A in linear space V can be written as a unique linear combination of matrices in any particular basis {A1,…,Ak} with unique coefficients x1,…,xk and A = x1A1+…+xkAk

11. Results Involving a Basis - II
Theorem Any two bases of the same linear space V have the same number of matrices.
The number of matrices in the basis is its dimension: dim(V)
Theorem If linear space V is spanned by set of r matrices, then dim(V) ≤ r.
If there is set of k linearly independent matrices in V, dim(V) ≥ k
Theorem If U is a subspace of linear space V, dim(U) ≤ dim(V)
Theorem Any set of r linearly independent matrices in r-dimensional linear space V is a basis for V
Theorem U, V ≡ linear spaces of mn matrices with U  V and dim(U) = dim(V), then U = V.
Aside: dim(Rmn) = mn  dim(V) ≤ mn
Theorem Any set S that spans a linear space V of mn matrices contains a subset that is basis for V. The number of matrices in the subset is dim(V)
Theorem For any set S of r lin. indep. matrices in k-dimensional V, there exists a basis for V containing the r matrices in S and k-r additional linearly independent matrices

12. Rank of a Matrix
Row Rank – Dimension of the row space of A (number of lin. indep. Rows)
Column Rank – Dimension of the column space of A
Theorem For any matrix A, row rank = column rank
Theorem Nonnull A ≡ mxn with row rank = r, column rank = c.
Then  B ≡ mc and L ≡ cn such that A = BL. Similarly,  K ≡ mr and T ≡ rn such that A = KT.
Theorem
Basis for C(A) has c vectors: {b1,…,bc} B=[b1…bc] . C(B) = sp(b1,…,bc) = C(A)   L ≡ cn such that A = BL
Basis for R(A) has r vectors: {t1’,…,tr’} T=[t1…tr]’ . R(T) = sp(t1’,…,tr’) = R(A)   K ≡ mr such that A = KT
Theorem Assume A ≡ nonnull (A = 0  r = c = 0) A = BL A = KT  R(A)  R(L) and C(A)  C(K)
R(L) spanned by the c rows of L C(K) spanned by the r columns of K
 r ≤ dim(R(L)) ≤ c and c ≤ dim(C(K)) ≤ r  r = c

13. Results on Dimensions/Ranks
Lemma For any A ≡ mn: rank(A) ≤ m, rank(A) ≤ n
Theorem A ≡ mn, B ≡ mp, C ≡ qn: [Theorem ]
If C(B)  C(A) then rank(B) ≤ rank(A) If R(C)  R(A) then rank(C) ≤ rank(A)
Corollary A ≡ mn, F ≡ np: rank(AF) ≤ rank(A) rank(AF) ≤ rank(F) [Corollary ]
Theorem A ≡ mn, B ≡ mp, C ≡ qn: [Theorem ]
If C(B)  C(A) and rank(B) = rank(A) then C(B) = C(A). If R(C)  R(A) and rank(C) = rank(A) then R(C) = R(A)
Corollary A ≡ mn, F ≡ np:
If rank(AF) = rank(A) then C(AF) = C(A). If rank(AF) = rank(F) then R(AF) = R(F)

14. Nonsingular, Full Row Rank, and Full Column Rank Matrices
Among mn matrices, maximum rank = min(m,n)
Mininimum rank is 0, only for the null matrix 0
A ≡ mn is said to be full row rank if rank(A) = m
A ≡ mn is said to be full column rank if rank(A) = n
A ≡ nn is said to be nonsingular if rank(A) = n (full row and column rank)
Theorem A ≡ mn nonnull matrix of rank r. Then  B ≡ mr and T ≡ rn such that A = BT
For any B ≡ mr and T ≡ rn s.t. A = BT, rank(B) = rank(T) = r  B ≡ full column rank and T ≡ full row rank
By Theorem :  B ≡ mr and T ≡ rn such that A = BT
By their dimensions: rank(B) ≤ r and rank(T) ≤ r (Lemma 4.4.3). By definition: rank(A) = rank(BT) = r
By Corollary : r = rank(BT) ≤ rank(B) ≤ r, r = rank(BT) ≤ rank(T) ≤ r  rank(B) = rank(T) = r

15. Decomposition of Rectangular Matrix

16. Definitions and Results Involving Ranks
Theorem A ≡ mn with rank(A) = r  A has r linearly independent rows and columns
The rr submatrix Ar made up of the r lin. Indep. rows and the r lin. indep. columns is nonsingular.
Any of the remaining rows or columns are linearly dependent of those of Ar .
There is no submatrix of A with rank > r
Corollary Any nn symmetric matrix of rank r has an rr principal submatrix that is nonsingular
Useful Equalities
rank(A’) = rank(A) (r = max # of linearly independent rows and columns of matrix)
For any nonzero scalar k, rank(kA) = rank(A) (No linear dependencies changed)
rank(-A) =rank(A) (special case of 2))

17. Partitioned Matrices and Sums of Matrices - I

18. Partitioned Matrices and Sums of Matrices - II

19. Partitioned Matrices and Sums of Matrices - III

20. Partitioned Matrices and Sums of Matrices - IV