1. Linear Dependence and Independence
From: D.A. Harville, Matrix Algebra from a Statistician’s Perspective, Springer. Chapter 3

2. Linear Dependence and Independence - I
Finite set of matrices (including row and column vectors as special cases): A1,…,Ak
Linearly Dependent  There exist scalars x1,…,xk (not all 0) such that x1A1+…+xkAk = 0
Linearly Independent  x1A1+…+xkAk = 0 iff x1 = … = xk = 0
{} Empty set is considered to be linearly independent
Set containing a single matrix is said to be linearly independent unless the matrix is the null matrix 0
Lemma Set {A1,…,Ak} of k ≥ 2 mxn matrices is linearly dependent if at least one matrix can be written as linear combination of the others
Linearly dependent  x1A1+…+xkAk = 0  Aj = -(x1/xj) A1 + … + -(xj-1/xj) Aj-1 + -(xj+1/xj) Aj+1 + … + -(xk/xj) Ak
(so long as xj ≠ 0). Aj = x1A1 + … + xkAk  -x1A1+… + -xj-1 Aj-1 + Aj + -xj+1 Aj+1 + … + -xkAk = 0

3. Linear Dependence and Independence - II
Lemma Set {A1,…,Ak} of k ≥ 2 mxn matrices with Aj ≠ 0
is linearly dependent iff at least one matrix can be written as linear combination of the previous ones:
Aj = x1A1 + … + xj-1Aj-1
Equivalently: {A1,…,Ak} is linearly independent if none of the matrices can be written in terms of previous ones.
Aj = x1A1 + … + xj-1Aj-1  -x1A1+… + -xj-1 Aj-1 + Aj + (0)Aj+1 + … + (0)Ak = 0
Let j be lowest integer such that {A1,…,Aj} ≡ lin. dep.  x1A1 + … + xj-1Aj-1 + xjAj = 0 for some (x1,…,xj) not all 0
and xj ≠ 0  Aj = (-x1/xj) A1 + … + (-xj-1 /xj) Aj-1
Corollary Set {A1,…,Ak} of k ≥ 2 mxn linearly independent matrices and A is an mxn matrix.
Then {A1, …, Ak, A} is linearly independent iff A ≠ x1A1 + … + xkAk for not all xi = 0
Suppose A = x1A1 + … + xkAk for not all xi = 0. Then x1A1 + … + xkAk + (-1)A = 0  {A1, …, Ak, A} lin. dependent
 A ≠ x1A1 + … + xkAk is only way for {A1, …, Ak, A} to be linearly independent

4. Linear Dependence and Independence - III