1. Linear Systems
Numerical Methods

2. Linear equations
N unknowns, M equations
coefficient matrix
where

3. Determinants and Cramer’s Rule
[A] : coefficient matrix
D : Determinant of A matrix

4. Solving methods Direct methods Iterative methods Gauss elimination
Gauss-Jordan elimination
LU decomposition
Singular value decomposition …
Iterative methods
Jacobi iteration
Gauss-Seidel iteration

5. Linear Systems
Solve Ax=b, where A is an nn matrix and b is an n1 column vector
Can also talk about non-square systems where A is mn, b is m1, and x is n1
Overdetermined if m>n: “more equations than unknowns”
Underdetermined if n>m: “more unknowns than equations” Can look for best solution using least squares

6. Gauss Elimination Solve Ax = b Consists of two phases: Forward
Forward elimination
Back substitution
Forward Elimination
reduces Ax = b to an upper triangular system Tx = b’
Back substitution can then solve Tx = b’ for x
Forward
Elimination
Back
Substitution

7. Gauss Elimination Fundamental operations:
Replace one equation with linear combination of other equations
Interchange two equations
Re-label two variables
Combine to reduce to trivial system
Simplest variant only uses #1 operations, but get better stability by adding #2 (partial pivoting) or #2 and #3 (full pivoting)

8. Gaussian Elimination Forward Elimination x1 - x2 + x3 = 6
-(3/1)
-(3/7)
-(2/1)
x1 - x2 + x3 = 6
x2 - x3 = -9
(4/7)x3=-(8/7)
Solve using BACK SUBSTITUTION: x3 = x2= x1 =3

9. Back Substitution 1x0 +1x1 –1x2 +4x3 = 8 – 2x1 –3x2 +1x3 = 5 2x2 – 3x3
2x3 = 4
x3 = 2

10. Back Substitution
1x0
+1x1
–1x2 =
– 2x1
–3x2 = 3
2x2 = 6
x2 = 3

11. Back Substitution
1x0
+1x1 = 3
– 2x1 = 12
x1 = –6

12. Back Substitution
1x0 = 9
x0 = 9

13. Back Substitution (* Pseudocode *) for i  n down to 1 do
/* calculate xi */
x [ i ]  b [ i ] / a [ i, i ]
/* substitute in the equations above */
for j  1 to i-1 do
b [ j ]  b [ j ]  x [ i ] × a [ j, i ]
endfor
Time Complexity?  O(n2)

14. Forward Elimination 4x0 +6x1 +2x2 – 2x3 = 8 2x0 +5x2 – 2x3 = 4 -(2/4)U L T I P E R S
4x0
+6x1
+2x2
– 2x3 = 8
2x0
+5x2
– 2x3 = 4
-(2/4)
–4x0
– 3x1
– 5x2
+4x3 = 1
-(-4/4)
8x0
+18x1
– 2x2
+3x3 = 40
-(8/4)

15. Forward Elimination 4x0 +6x1 +2x2 – 2x3 = 8 – 3x1 +4x2 – 1x3 = +3x1U L T I P E R S
4x0
+6x1
+2x2
– 2x3 = 8
– 3x1
+4x2
– 1x3 =
+3x1
– 3x2
+2x3 = 9
-(3/-3)
+6x1
– 6x2
+7x3 = 24
-(6/-3)

16. Forward Elimination 4x0 +6x1 +2x2 – 2x3 = 8 – 3x1 +4x2 – 1x3 = 1x2U L T I P E R
– 3x1
+4x2
– 1x3 =
1x2
+1x3 = 9
2x2
+5x3 = 24 ??

17. Forward Elimination 4x0 +6x1 +2x2 – 2x3 = 8 – 3x1 +4x2 – 1x3 = 1x2
1x2
+1x3 = 9
3x3 = 6

18. Gauss-Jordan Elimination
b11
x22
b22
x33
b33
x44
b44
x55
b55
x66
b66
x77
b77
x88
b66
x99
b66

19. Gauss-Jordan Elimination: Example
Scaling R2:
R2  R2/(-1)
R2  R2 - (-1)R1
R3  R3 - ( 3)R1
R1  R1 - (1)R2
R3  R3-(4)R2
Scaling R3:
R3  R3/(18)
R1  R1 - (7)R3
R2  R2-(-5)R3
RESULT:
x1=8.45, x2=-2.89, x3=1.23
Time Complexity?  O(n3)

20. Gauss-Jordan Elimination
Solve:
Only care about numbers – form “tableau” or “augmented matrix”:

21. Gauss-Jordan Elimination
Given:
Goal: reduce this to trivial system and read off answer from right column

22. Gauss-Jordan Elimination
Basic operation 1: replace any row by linear combination with any other row
Here, replace row1 with 1/2 * row1 + 0 * row2

23. Gauss-Jordan Elimination
Replace row2 with row2 – 4 * row1
Negate row2

24. Gauss-Jordan Elimination
Replace row1 with row1 – 3/2 * row2
Read off solution: x1 = 2, x2 = 1

25. Augmented Matrices
Matrices are rectangular arrays of numbers that can aid us by eliminating the need to write the variables at each step of the reduction.
For example, the system
may be represented by the augmented matrix
Coefficient Matrix

26. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

27. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

28. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

29. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

30. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

31. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

32. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

33. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes

34. Matrices and Gauss-Jordan
Every step in the Gauss-Jordan elimination method can be expressed with matrices, rather than systems of equations, thus simplifying the whole process:
Steps expressed as systems of equations:
Steps expressed as augmented matrices:
Toggle slides back and forth to compare before and changes
Row Reduced Form of the Matrix