1. Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems
Shang-Hua Teng

2. Linear Independence Linear Combination Linear Independence
is linearly independent if only if none of them can
be expressed as a linear combination of the others

3. Examples

4. Linear Independence and Null Space
Theorem/Definition
is linearly independent if and only
a1v1+a2v2+…+anvn=0 only happens when all a ’s are zero
The columns of a matrix A are linearly independent when only solution to Ax=0 is x = 0

5. 2D and 3D v w u v

6. How do we determine a set of vectors are independent?
Make them the columns of a matrix
Elimination
Computing their null space

7. Permute Rows and Continuing Elimination (permute columns)

8. There must be free variables.
Theorem
If Ax = 0 has more have more unknown than equations (m > n: more columns than rows), then it has nonzero solutions.
There must be free variables.

9. Echelon Matrices
Free variables

10. Reduced Row Echelon Matrix R
Free variables

11. Computing the Reduced Row Echelon Matrix
Elimination to Echelon Matrix
E1PA = U
Divide the row of pivots by the pivots
Upward Elimination
E2E1PA = R

12. Example: Gauss-Jordan Method for Matrix Inverse
[A I]
E1[A I] = [U, I]
In its reduced Echelon Matrix
A-1 [A I] = [I A-1]

13. A Close Look at Reduced Echelon Matrix
The last equation of R x = 0 is redundant
0 = 0
Rank of A is the number of pivots rank(A).

14. What is the Rank of Outer Product

15. Rank and Reduced Row Echelon Matrix
Free variables
Theorem/Definition
Rank(A) = number of independent rows
Rank(A) = number of independent columns

16. Dimension of the Column Space and Null Space
The dimensions of the column space of A is equal to Rank(A).
The dimension of the null space of A is equal to the number of free variables which is n – Rank(A)
A is an m by n matrix

17. Rank and Reduced Row Echelon Matrix
The Pivot columns are not combinations of earlier columns
Pivot columns
Free variables
Free Columns

18. Reduced Echelon and Null Space Matrix
Special Solutions

19. Null Space Matrix
Ax=0 has n-Rank(A) free variables and special solutions
The Nullspace matrix has n-Rank(A) columns
The columns of the nullspace matrix are independent
The dimension of the Null space is n – rank(A)

20. Complete Solution of Ax = 0
After column permutation, we can write
r pivot columns
n-r free columns
Nullspace matrix
Pivot variables
Free variables
Moreover: RN = [0]

21. Complete Solution to Ax = b
A is an m by n matrix, and b is an n-place vector
Unique solution
Infinitely many solution
No solution
Suppose Ax = b has more then one solution, say x1, x2 then
A x1 = b
A x2 = b So
A (x1 - x2 ) = 0
(x1 - x2 ) is in nullspace(A)

22. Complete Solution to Ax = b
Suppose we found a particular solution xp to Ax = b i.e,
A xp = b
Let F be the indexes of free variables of Ax = 0
Let xF be the column vector of free variables
Let N be the nullspace matrix of A
Then
defines the complete set of solutions to Ax = b xp

23. Example: Complete Solution to Ax = b

24. Augmented matrix [A b]
Elimination to obtain [R d]

25. Set free variables to 0 to find a particular solution
Compute the nullspace matrix
Complete solution is

26. Full Rank Matrix Suppose A is an m by n matrix. Then
A is full column if rank(A) = n
columns of A are independent
A is full row rank if rank(A) = m
Rows of A are independent

27. Full Column Rank Matrix
Columns are independent
All columns of A are pivot columns
There are non free variables or special solutions
The nullspace N(A) contains only the zero vector
If Ax=b has a solution (it might not) then it has only one solution
n by n
m-n rows of zeros

28. Full Row Rank Matrix Rows are independent
All rows of A have pivots, R has no zero rows
Ax=b has a solution for every right hand side b
The column space is the whole space Rm
There are n-m special solutions in the null space of A

29. The Whole Picture Rank(A) = m = n Ax=b has unique solution
Ax=b has n-m dimensional solution
Rank(A) = n < m
Ax=b has 0 or 1 solution
Rank(A) < n, Rank(A) < m
Ax=b has 0 or n-rank(A) dimensions

30. Basis and Dimension of a Vector Space
A basis for a vector space is a sequence of vectors that
The vectors are linearly independent
The vectors span the space: every vector in the vector can be expressed as a linear combination of these vectors

31. Basis for 2D and n-D (1,0), (0,1) (1 1), (-1 –2)
The vectors v1,v2,…vn are basis for Rn if and only if they are columns of an n by n invertible matrix

32. Column and Row Subspace
C(A): the space spanned by columns of A
Subspace in m dimensions
The pivot columns of A are a basis for its column space
Row space: the space spanned by rows of A
Subspace in n dimensions
The row space of A is the same as the column space of AT, C(AT)
The pivot rows of A are a basis for its row space
The pivot rows of its Echolon matrix R are a basis for its row space

33. Important Property I: Uniqueness of Combination
The vectors v1,v2,…vn are basis for a vector space V, then for every vector v in V, there is a unique way to write v as a combination of v1,v2,…vn .
v = a1 v1+ a2 v2+…+ an vn
v = b1 v1+ b2 v2+…+ bn vn
So: 0=(a1 - b1) v1 + (a2 -b2 )v2+…+ (an -bn )vn

34. Important Property II: Dimension and Size of Basis
If a vector space V has two set of bases
v1,v2,…vm . V = [v1,v2,…vm ]
w1,w2,…wn . W= [w1,w2,…wn ].
then m = n
Proof: assume n > m, write W = VA
A is m by n, so Ax = 0 has a non-zero solution
So VAx = 0 and Wx = 0
The dimension of a vector space is the number of vectors in every basis
Dimension of a vector space is well defined

35. Dimensions of the Four Subspaces Fundamental Theorem of Linear Algebra, Part I
Row space: C(AT) – dimension = rank(A)
Column space: C(A)– dimension = rank(A)
Nullspace: N(A) – dimension = n-rank(A)
Left Nullspace: N(AT) – dimension = m –rank(A)