1. Chapter 12 Gaussian Elimination (II) Speaker: Lung-Sheng Chien Reference book: David Kincaid, Numerical Analysis Reference lecture note: Wen-wei Lin, chapter 2, matrix computation http://math.ntnu.edu.tw/~min/matrix_computation.html

2. OutLine Weakness of A=LU - erroneous judgment: A is invertible but A=LU does not exist - unstable, A is far from LU A=LU versus PA=LU Pivoting strategy Implementation of PA=LU MATLAB usage

3. Fail of LU: singular of leading principal minor Question: How about interchanging row 1 and row 2? and

4. unstable of LU: near singular of leading principal minor [1] ? Theoretical: Numerical: ifthen Problem: for double-precision, we only have 16 digit-accuracy, we cannot accept, why?

5. backward substitution step 1: step 2: Question: why not due to rounding error unstable of LU: near singular of leading principal minor [2]

6. Case 1:Case 2: unstable of LU: near singular of leading principal minor [3] wrong solution

7. Question: How about interchanging row 1 and row 2? LU-factorization backward substitution step 1: step 2: unstable of LU: near singular of leading principal minor [4] Key observation: without pivoting (rounding normal number) pivoting (rounding error does not occur for normal number)

8. Case 1:Case 2: unstable of LU: near singular of leading principal minor [5] Why not 1?

9. without pivotingpivoting (NOT stable) (stable) Objective: control growth rate of control multiplier unstable of LU: cause and controllability definelargest component of matrix

10. OutLine Weakness of A=LU A=LU versus PA=LU - controllability of lower triangle matrix L Pivoting strategy Implementation of PA=LU MATLAB usage

11. 6-224 12-8610 3-1393 -641-18 6-224 0-422 0-1281 023-14 1 21 0.51 1 since is not maximum among 6-224 0-422 0-1281 023-14 1 1 31 -0.51 6-224 0-422 002-5 004-13 since is not maximum among Recall LU example in chapter 11 [1]

12. Recall LU example in chapter 11 [2] 6-224 0-422 002-5 004-13 1 1 1 21 6-224 0-422 002-5 000-3 since is not maximum among 6-224 12-8610 3-1393 -641-18 1 21 0.531 -0.521 6-224 -422 2-5 -3 Question: How can we control, is uniform bound possible?

13. PA = LU in MATLAB

14. 12-8610 3-1393 -641-18 6-224 12-8610 0-117.50.5 004-13 02 1 0.251 -0.51 0.51 sinceis maximum among PA = LU: assume P is given [1] 12-8610 0-117.50.5 004-13 02 12-8610 0-117.50.5 004-13 004/11-10/11 1 1 01 -2/111 since is maximum among

15. PA = LU: assume P is given [2] 12-8610 0-117.50.5 004-13 00 0 3/11 12-8610 0-117.50.5 004-13 00 4/11-10/11 1 1 1 1/111 since is maximum among 12-8610 3-1393 -641-18 6-224 1 0.251 -0.501 0.5-2/111/111 12-8610 -117.50.5 4-13 3/11 Observation: though proper permutation, we can control Question: in fact, we cannot know permutation matrix in advance, how can we do?

16. Sequence of matrices during LU-decomposition

17. OutLine Weakness of A=LU A=LU versus PA=LU Pivoting strategy - partial pivoting (we adopt this formulation) - complete pivoting Implementation of PA=LU MATLAB usage

18. (1) find a such that (2) swap rowand row Partial pivoting and complete pivoting thenwith Partial pivoting (1) find a such that (2) swap rowand row thenwith (2) swap columnand column complete pivoting

19. Define permutation matrix Recall permutation matrix interchange row 2, 3 interchange column 2, 3 Let 1 2 3 4 5 6 7 8 9 interchange row 2, 3 1 2 3 7 8 9 4 5 6 interchange column 2, 3 13 2 79 8 46 5 Symmetric permutation:

20. x1 x2 x3 1 23 4 5 6 7 8 9 Meaning of matrix coefficient relationship between node and node constraint on node node is named x1 x3x3 x2x2 1 23 4 5 6 7 8 9 change node 2 to node 3 and node 3 to node 2 write constraint for now node index 13 2 79 8 46 5 x1 x2x2 x3x3 x2x2x3x3 1 2 3 4 5 6 7 8 9 x2x3 x1 x2 x3

21. Identity matrix is invariant under symmetric permutation x1 x2 x3 1 1 1 1 1 1 x1 x2 x3 x2x3 change node 2 to node 3 and node 3 to node 2 x1 x3x3 x2x2 1 1 1 1 1 x2x2 x3x3 x2x2x3x3 1

22. PA = LU: partial pivoting [1] 6-224 12-8610 3-1393 -641-18 12-8610 6-224 3-1393 -641-18 12-8610 6-224 3-1393 -641-18 1 0.51 0.251 -0.51 12-8610 02 0-117.50.5 004-13 12-8610 02 0-117.50.5 004-13 12-8610 0-117.50.5 02 004-13

23. PA = LU: partial pivoting [2] where 1 0.51 0.251 -0.51 Interchange columns 2, 3 1 0.51 0.251 -0.51 Interchange rows 2, 3 1 0.251 0.51 -0.51 12-8610 3-1393 6-224 -641-18 1 0.251 0.51 -0.51 12-8610 0-117.50.5 02 004-13 verify

24. PA = LU: partial pivoting [3] 12-8610 0-117.50.5 02 004-13 1 1 -2/111 01 12-8610 0-117.50.5 004/11-10/11 004-13 12-8610 0-117.50.5 004/11-10/11 004-13 12-8610 0-117.50.5 004-13 004/11-10/11 where

25. 1 0.251 0.5-2/111 -0.51 Interchange columns 3,4 1 0.251 0.5-2/111 -0.501 Interchange rows 3.4 1 0.251 -0.5 0 1 0.5-2/111 PA = LU: partial pivoting [4] 12-8610 3-1393 -641-18 6-224 1 0.251 -0.501 0.5-2/111 12-8610 0-117.50.5 004-13 004/11-10/11 verify

26. PA = LU: partial pivoting [5] 12-8610 0-117.50.5 004-13 004/11-10/11 1 1 1 1/111 12-8610 0-117.50.5 004-13 0003/11 where 1 0.251 -0.501 0.5-2/111/111 12-8610 -117.50.5 4-13 3/11 we have

27. OutLine Weakness of A=LU A=LU versus PA=LU Pivoting strategy Implementation of PA=LU - commutability of GE matrix and permutation - recursive formula MATLAB usage

28. 1 0.51 0.251 -0.51 Interchange rows and columns 2, 3 1 0.251 0.51 -0.51 1 0.251 0.5-2/111 -0.51 Interchange rows and columns 3, 4 1 0.251 -0.5 0 1 0.5-2/111 Implementation issue: commutability of GE matrix and permutation [1]

29. Implementation issue: commutability of GE matrix and permutation [2] Observation: If we define, then where interchanges rows or columns and interchange

30. Implementation issue: commutability of GE matrix and permutation [3] partial rows (k and p) interchange symmetric permutation on an Identity matrix

31. we have compute update original matrix stores in lower triangle matrix Algorithm ( PA = LU ) [1] Given full matrix, construct initial lower triangle matrix for use permutation vector to record permutation matrix let,and

32. Algorithm ( PA = LU ) [2] define permutation matrix then after swapping rows, we have compute by swappingand then for ifthen endif swap rowand row find a such that 1 2 3 4

33. Algorithm ( PA = LU ) [3] endfor by updating by updating matrix then decomposewhere 5 (recursion is done)

34. Question: recursion implementation Initialization - check algorithm holds for k=1 Recursion formula - check algorithm holds for k, if k-1 is true Termination condition - check algorithm holds for k=n Breakdown of algorithm - check which condition PA=LU fails Exception: algorithm works only for square matrix? normal abnormal Extension of algorithm

35. Exception: algorithm works only for square matrix? [1] Case 1: Case 2:

36. Case 3: Exception: algorithm works only for square matrix? [2] we can simplify it as Question: what is termination condition of case 3 ?

37. OutLine Weakness of A=LU A=LU versus PA=LU Pivoting strategy Implementation of PA=LU MATLAB usage - abs - max

38. Command “abs”: take absolute value In PA=LU algorithm, we need to take absolute value of one column vector for pivoting abs also works for matrix

39. Command “max”: take maximum value