Lecture 6 Matrix Operations and Gaussian Elimination for Solving Linear Systems Shang-Hua Teng

Matrix (Uniform Representation for Any Dimension) An m by n matrix is a rectangular table of mn numbers

Matrix (Uniform Representation for Any Dimension) Can be viewed as m row vectors in n dimensions

Matrix (Uniform Representation for Any Dimension) Or can be viewed as n column vectors in m dimensions

Squared Matrix An n by n matrix is a squared table of n 2 numbers

Some Special Squared Matrices All zeros matrix Identity matrix

Matrix Operations Addition Scalar multiplication Multiplication

1. Matrix Addition: Matrices have to have the same dimensions What is the complexity?

2. Scalar Multiplication: What is the complexity?

3. Matrix Multiplication Two matrices have to be conformal What is the complexity?

Matrix Multiplication Two matrices have to be conformal

The Laws of Matrix Operations A + B = B + A (commutative) c(A+B) = cA + c+B (distributive) A + (B + C) = (A + B) + C (associative) C(A+B) = CA + CB (distributive from left) (A+B)C = AC+BC (distributive from right) A(BC) = (AB)C (associative) But in general:

Counter Example

Special Matrices Identity matrix I –IA = AI = A Square Matrix A

Elimination: Method for Solving Linear Systems Linear Systems == System of Linear Equations Elimination: –Multiply the LHS and RHS of an equation by a nonzero constant results the same equations –Adding the LHSs and RHSs of two equations does not change the solution

Elimination in 2D Multiply the first equation by 3 and subtracts from the second equation (to eliminate x) The two systems have the same solution The second system is easy to solve

Geometry of Elimination (3,1) 8y = 8 Reduce to a 1-dimensional problem.

Upper Triangular Systems and Back Substitution Back substitution –From the second equation y = 1 –Substitute the value of y to the first equation to obtain x-2=1 –Solve it we have: x = 3 So the solution is (3,1)

How Much to Multiply before Subtracting Pivot: first nonzero in the row that does the elimination Multiplier: (entry to eliminate) divided by (pivot) Multiply: = 3/1

How Much to Multiply before Subtracting Pivot: first nonzero in the row that does the elimination Multiplier: (entry to eliminate) divided by (pivot) Multiply: = 3/2 The pivots are on the diagonal of the triangle after the elimination

Breakdown of Elimination What is the pivot is zero == one can’t divide by zero!!!! Eliminate x: No Solution!!!!: this system has no second pivot

Geometric Intuition (Row Pictures) Two parallel lines never intersect (3,1) 8y = 8

Geometric Intuition (Column Picture) Two column vectors are co-linear!!!!

Geometric Intuition Geometric degeneracy cause failure in elimination!

Failure in Elimination May Indicate Infinitely Many Solutions y is free, can be number! Geometric Intuition (row picture): The two line are the same Geometric Intuition (column picture): all three column vectors are co-linear

Failure in Elimination (Temporary and can be Fixed) First pivot position contains zero Exchange with the second equation Can be solved by backward substitution!

Singular Systems versus Non-Singular Systems A singular system has no solution or infinitely many solution –Row Picture: two line are parallel or the same –Column Picture: Two column vectors are co- linear A non-singular system has a unique solution –Row Picture: two non-parallel lines –Column Picture: two non-colinear column vectors

Gaussian Elimination in 3D Using the first pivot to eliminate x from the next two equations

Gaussian Elimination in 3D Using the second pivot to eliminate y from the third equation

Gaussian Elimination in 3D Using the second pivot to eliminate y from the third equation

Now We Have a Triangular System From the last equation, we have

Backward Substitution And substitute z to the first two equations

Backward Substitution We can solve y

Backward Substitution Substitute to the first equation

Backward Substitution We can solve the first equation

Backward Substitution We can solve the first equation

Generalization How to generalize to higher dimensions? What is the complexity of the algorithm? Answer: Express Elimination with Matrices

Step 1 Build Augmented Matrix Ax = b [A b]

Pivot 1: The elimination of column 1

Pivot 2: The elimination of column 2 Upper triangular matrix

Backward Substitution 1: from the last column to the first Upper triangular matrix

Expressing Elimination by Matrix Multiplication

Elementary or Elimination Matrix The elementary or elimination matrix That subtracts a multiple l of row j from row i can be obtained from the identity entry by adding (-l) in the i,j position

Elementary or Elimination Matrix

Pivot 1: The elimination of column 1 Elimination matrix

The Product of Elimination Matrices