1. Chapter 11 Gaussian Elimination (I)
Speaker: Lung-Sheng Chien
Reference book: David Kincaid, Numerical Analysis

2. OutLine
Basic operation of matrix - representation - three elementary matrices
Example of Gaussian Elimination (GE)
Formal description of GE
MATLAB usage

3. Matrix notation in MATLAB6 -2 2 4 12 -8 6 10 3
-13 9 3 -6 4 1
-18

4. Matrix-vector product
Inner-product based 6 -2 2 4 x1
6 x1 + (-2) x2 + 2 x3 + 4 x4 12 -8 6 10 x2
12 x1 + (-8) x2 + 6 x x4 3
-13 9 3 x3
3 x1 + (-13) x2 + 9 x3 + 3 x4 -6 4 1
-18 x4
(-6) x1 + 4 x2 + 1 x3 + (-18) x4
outer-product based 6 -2 2 4 x1 6 -2 2 4 12 -8 6 10 x2 12 -8 6 10 x1 x2 x3 x4 3
-13 9 3 x3 3
-13 9 3 -6 4 1
-18 x4 -6 4 1
-18

5. Matrix-vector product: MATLAB implementation
matvec.m
Inner-product based
outer-product based
Question: which one is better

6. Column-major nature in MATLAB
physical index : 1D 6 1 12 2 3 3
Question: how does column-major affect inner-product based matrix-vector product and outer-product based matrix-vector product? -6 4
Logical index : 2D -2 5 -8 6 6 -2 2 4
-13 7 12 -8 6 10 4 8 3
-13 9 3 2 9 -6 4 1
-18 6 10 9 11 1 12 4 13 10 14 3 15
-18 16

7. Matrix representation: outer-product
where
is outer-product representation

8. Elementary matrix [1] (1) The interchange of two rows in A:
Define permutation matrix
How to explain?
Question 1: why
Question 2: how to easily obtain

9. Concatenation of permutation matrices
such that
such that
Question:
implies
Direct calculation

10. Elementary matrix [2] (2) Multiplying one row by a nonzero constant:
Define scaling matrix
How to explain?
since

11. Elementary matrix [3] (3) Adding to one row a multiple of another:
Define GE (Gaussian Elimination) matrix
How to explain?
outer-product representation

12. Use MATLAB notation

13. Concatenation of GE matrices
such that
Suppose
such that
Question:

14. OutLine Basic operation of matrix
Example of Gaussian Elimination (GE) - forward elimination to upper triangle form - backward substitution
Formal description of GE
MATLAB usage

15. 6 -2 2 4 x1 12 12 -8 6 10 x2 34 3
-13 9 3 x3 27 -6 4 1
-18 x4
-38 12 -8 6 10 34 6 -2 2 4 x1 12 -4 2 2 x2 10 6 -2 2 4 12 3
-13 9 3 x3 27 -6 4 1
-18 x4
-38 -4 2 2 10 3
-13 9 3 27 6 -2 2 4 x1 12 -4 2 2 x2 10 6 -2 2 4 12
-12 8 1 x3 21 -6 4 1
-18 x4
-38
-12 8 1 21

16. -6 4 1
-18
-38 6 -2 2 4 x1 12 -4 2 2 x2 10 6 -2 2 4 12
-12 8 1 x3 21 2 3
-14 x4
-26 2 3
-14
-26 6 -2 2 4 x1 12
First row does not change thereafter -4 2 2 x2 10
-12 8 1 x3 21 2 3
-14 x4
-26
-12 8 1 21 6 -2 2 4 x1 12 -4 2 2 x2 10 -4 2 2 10 2 -5 x3 -9 2 3
-14 x4
-26 2 -5 -9

17. 2 3
-14
-26 6 -2 2 4 x1 12 -4 2 2 x2 10 -4 2 2 10 2 -5 x3 -9 4
-13 x4
-21 4
-13
-21 6 -2 2 4 x1 12 -4 2 2 x2 10 2 -5 x3 -9 4
-13 x4
-21 4
-13
-21 6 -2 2 4 x1 12 -4 2 2 x2 10 2 -5 -9 2 -5 x3 -9 -3 -3 -3 x4 -3

18. Backward substitution: inner-product-based6 -2 2 4 x1 12 6 -2 2 4 x1 12 -4 2 2 x2 10 -4 2 2 x2 10 2 -5 x3 -9 2 -5 x3 -9 -3 x4 -3 x4 6 -2 2 4 x1 12 6 -2 2 4 x1 12 -4 2 2 x2 10 x2 x3 x3 x4 x4

19. Backward substitution: outer-product-based6 -2 2 4 x1 12 -4 2 2 x2 10 2 -5 x3 -9 -3 x4 -3 6 -2 2 x1 12 4 8 -4 2 x2 10 x4 2 8 2 x3 -9 -5 -4 6 -2 x1 8 2 12 x3 -4 x2 8 2 12 6 x1 12 x2 -2 6

20. OutLine Basic operation of matrix Example of Gaussian Elimination (GE)
Formal description of GE - component-wise and column-wise representation - recursive structure
MATLAB usage

21. 6 -2 2 4 x1 12 12 -8 6 10 x2 34 3
-13 9 3 x3 27 -6 4 1
-18 x4
-38 6 -2 2 4 x1 12 -4 2 2 x2 10 3
-13 9 3 x3 27 -6 4 1
-18 x4
-38 6 -2 2 4 x1 12 6 -2 2 4 x1 12 12 -8 6 10 x2 34 12 -8 6 10 x2 34
-12 8 1 x3 21 3
-13 9 3 x3 27 -6 4 1
-18 x4
-38 2 3
-14 x4
-26

22. Eliminate first column: component-wise6 -2 2 4 x1 12 6 -2 2 4 x1 12 12 -8 6 10 x2 34 -4 2 2 x2 10 3
-13 9 3 x3 27
-12 8 1 x3 21 -6 4 1
-18 x4
-38 2 3
-14 x4
-26
where
Exercise: check
by MATLAB

23. Eliminate first column: column-wise
define
and
then
Exercise: check
by MATLAB
Notation:

24. Eliminate second column: component-wise6 -2 2 4 x1 12 -4 2 2 x2 10
-12 8 1 x3 21 2 3
-14 x4
-26 6 -2 2 4 x1 12 -4 2 2 x2 10 2 -5 x3 -9 2 3
-14 x4
-26 6 -2 2 4 x1 12 -4 2 2 x2 10 2 -5 x3 -9 4
-13 x4
-21

25. Eliminate second column: column-wise
define
Exercise: check
Exercise: check
, why? Give a simple explanation.
Notation:

26. Eliminate third column: component-wise6 -2 2 4 x1 12 -4 2 2 x2 10 2 -5 x3 -9 4
-13 x4
-21 6 -2 2 4 x1 12 -4 2 2 x2 10
define 2 -5 x3 -9 -3 x4 -3
then

27. LU-decomposition Notation: LU-decomposition Exercise: check
Question: Do you have any effective method to write down matrix
Question: why don’t we care about right hand side vector

28. Problems about LU-decomposition
Question 1: what condition does LU-decomposition fail?
Question 2: does any invertible matrix has LU-decomposition?
Question 3: How to measure “goodness of LU-decomposition“?
Question 4: what is order of performance of LU-decomposition (operation count)?
Question 5: How to parallelize LU-decomposition?
Question 6: How to implement LU-decomposition with help of GPU?

29. OutLine Basic operation of matrix Example of Gaussian Elimination (GE)
Formal description of GE
MATLAB usage - website resource - “help” command - M-file

30. MATLAB website

31. MATLAB: create matrix

32. MATLAB: LU-factorization

33. Start MATLAB 2008

34. Command “help” : documentation
!= in C

35. Loop in MATLAB

36. Decision-making in MATLAB

37. IO in MATLAB

38. LU-factorization in MATLAB

39. M-file in MATLAB matvec.m
Description of function matvec, write specificaiton of input parameter
Consistency check
Inner-product based

40. Move to working directory
M-file

41. Execute M-file Construct matrix
Execute M-file, like function call in C-language

42. Exercise: forward / backward substitution
Forward substitution
since
backward substitution
since
Write MATLAB code to do forward substitution and backward substitution

43. Exercise: LU-decomposition
You should find recursive structure of the decomposition