1. CISE301_Topic3KFUPM1 SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17: KFUPM Read Chapter 9 of the textbook

2. CISE301_Topic3KFUPM2 Lecture 12 Vector, Matrices, and Linear Equations

3. CISE301_Topic3KFUPM3 VECTORS

4. CISE301_Topic3KFUPM4 MATRICES

5. CISE301_Topic3KFUPM5 MATRICES

6. CISE301_Topic3KFUPM6 Determinant of a MATRICES

7. CISE301_Topic3KFUPM7 Adding and Multiplying Matrices

8. CISE301_Topic3KFUPM8 Systems of Linear Equations

9. CISE301_Topic3KFUPM9 Solutions of Linear Equations

10. CISE301_Topic3KFUPM10 Solutions of Linear Equations  A set of equations is inconsistent if there exists no solution to the system of equations:

11. CISE301_Topic3KFUPM11 Solutions of Linear Equations  Some systems of equations may have infinite number of solutions

12. CISE301_Topic3KFUPM12 Graphical Solution of Systems of Linear Equations solution

13. CISE301_Topic3KFUPM13 Cramer’s Rule is Not Practical

14. CISE301_Topic3KFUPM14  Naive Gaussian Elimination  Examples Lecture 13 Naive Gaussian Elimination

15. CISE301_Topic3KFUPM15 Naive Gaussian Elimination  The method consists of two steps: Forward Elimination: the system is reduced to upper triangular form. A sequence of elementary operations is used. Backward Substitution: Solve the system starting from the last variable.

16. CISE301_Topic3KFUPM16 Elementary Row Operations  Adding a multiple of one row to another  Multiply any row by a non-zero constant

17. CISE301_Topic3KFUPM17 Example Forward Elimination

18. CISE301_Topic3KFUPM18 Example Forward Elimination

19. CISE301_Topic3KFUPM19 Example Forward Elimination

20. CISE301_Topic3KFUPM20 Example Backward Substitution

21. CISE301_Topic3KFUPM21 Forward Elimination

22. CISE301_Topic3KFUPM22 Forward Elimination

23. CISE301_Topic3KFUPM23 Backward Substitution

24. CISE301_Topic3KFUPM24  Summary of the Naive Gaussian Elimination  Example  How to check a solution  Problems with Naive Gaussian Elimination Failure due to zero pivot element Error Lecture 14 Naive Gaussian Elimination

25. CISE301_Topic3KFUPM25 Naive Gaussian Elimination oThe method consists of two steps oForward Elimination: the system is reduced to upper triangular form. A sequence of elementary operations is used. oBackward Substitution: Solve the system starting from the last variable. Solve for x n,x n-1,…x 1.

26. CISE301_Topic3KFUPM26 Example 1

27. CISE301_Topic3KFUPM27 Example 1

28. CISE301_Topic3KFUPM28 Example 1 Backward Substitution

29. CISE301_Topic3KFUPM29 How Do We Know If a Solution is Good or Not  Given AX=B  X is a solution if AX-B=0  Due to computation error AX-B may not be zero  Compute the residuals R=|AX-B|  One possible test is ?????

30. CISE301_Topic3KFUPM30 Determinant

31. CISE301_Topic3KFUPM31 How Many Solutions Does a System of Equations AX=B Have?

32. CISE301_Topic3KFUPM32 Examples

33. CISE301_Topic3KFUPM33 Lectures 15-16: Gaussian Elimination with Scaled Partial Pivoting  Problems with Naive Gaussian Elimination  Definitions and Initial step  Forward Elimination  Backward substitution  Example

34. CISE301_Topic3KFUPM34 Problems with Naive Gaussian Elimination oThe Naive Gaussian Elimination may fail for very simple cases. (The pivoting element is zero). oVery small pivoting element may result in serious computation errors

35. CISE301_Topic3KFUPM35 Example 2

36. CISE301_Topic3KFUPM36 Example 2 Initialization step Scale vector: disregard sign find largest in magnitude in each row

37. CISE301_Topic3KFUPM37 Why Index Vector?  Index vectors are used because it is much easier to exchange a single index element compared to exchanging the values of a complete row.  In practical problems with very large N, exchanging the contents of rows may not be practical since they could be stored at different locations.

38. CISE301_Topic3KFUPM38 Example 2 Forward Elimination-- Step 1: eliminate x1

39. CISE301_Topic3KFUPM39 Example 2 Forward Elimination-- Step 1: eliminate x1 First pivot equation

40. CISE301_Topic3KFUPM40 Example 2 Forward Elimination-- Step 2: eliminate x2

41. CISE301_Topic3KFUPM41 Example 2 Forward Elimination-- Step 2: eliminate x2

42. CISE301_Topic3KFUPM42 Example 2 Forward Elimination-- Step 3: eliminate x3 Third pivot equation

43. CISE301_Topic3KFUPM43 Example 2 Backward Substitution

44. CISE301_Topic3KFUPM44 Example 3

45. CISE301_Topic3KFUPM45 Example 3 Initialization step

46. CISE301_Topic3KFUPM46 Example 3 Forward Elimination-- Step 1: eliminate x1

47. CISE301_Topic3KFUPM47 Example 3 Forward Elimination-- Step 1: eliminate x1

48. CISE301_Topic3KFUPM48 Example 3 Forward Elimination-- Step 2: eliminate x2

49. CISE301_Topic3KFUPM49 Example 3 Forward Elimination-- Step 2: eliminate x2

50. CISE301_Topic3KFUPM50 Example 3 Forward Elimination-- Step 2: eliminate x2

51. CISE301_Topic3KFUPM51 Example 3 Forward Elimination-- Step 3: eliminate x3

52. CISE301_Topic3KFUPM52 Example 3 Forward Elimination-- Step 3: eliminate x3

53. CISE301_Topic3KFUPM53 Example 3 Backward Substitution

54. CISE301_Topic3KFUPM54 How Good is the Solution?

55. CISE301_Topic3KFUPM55 Remarks:  We use index vector to avoid the need to move the rows which may not be practical for large problems.  If we order the equation as in the last value of the index vector, we have a triangular form.  Scale vector is formed by taking maximum in magnitude in each row.  Scale vector do not change.  The original matrices A and B are used in checking the residuals.

56. CISE301_Topic3KFUPM56 Lecture 17 Tridiagonal & Banded Systems and Gauss-Jordan Method  Tridiagonal Systems  Diagonal Dominance  Tridiagonal Algorithm  Examples  Gauss-Jordan Algorithm

57. CISE301_Topic3KFUPM57 Tridiagonal Systems:  The non-zero elements are in the main diagonal, super diagonal and subdiagonal.  a ij =0 if |i-j| > 1 Tridiagonal Systems

58. CISE301_Topic3KFUPM58  Occur in many applications  Needs less storage (4n-2 compared to n 2 +n for the general cases)  Selection of pivoting rows is unnecessary (under some conditions)  Efficiently solved by Gaussian elimination Tridiagonal Systems

59. CISE301_Topic3KFUPM59  Based on Naive Gaussian elimination.  As in previous Gaussian elimination algorithms Forward elimination step Backward substitution step  Elements in the super diagonal are not affected.  Elements in the main diagonal, and B need updating Algorithm to Solve Tridiagonal Systems

60. CISE301_Topic3KFUPM60 Tridiagonal System

61. CISE301_Topic3KFUPM61 Diagonal Dominance

62. CISE301_Topic3KFUPM62 Diagonal Dominance

63. CISE301_Topic3KFUPM63 Diagonally Dominant Tridiagonal System  A tridiagonal system is diagonally dominant if  Forward Elimination preserves diagonal dominance

64. CISE301_Topic3KFUPM64 Solving Tridiagonal System

65. CISE301_Topic3KFUPM65 Example

66. CISE301_Topic3KFUPM66 Example

67. CISE301_Topic3KFUPM67 Example Backward Substitution  After the Forward Elimination:  Backward Substitution:

68. CISE301_Topic3KFUPM68 Gauss-Jordan Method  The method reduces the general system of equations AX=B to IX=B where I is an identity matrix.  Only Forward elimination is done and no substitution is needed.  It has the same problems as Naive Gaussian elimination and can be modified to do partial scaled pivoting.  It takes 50% more time than Naive Gaussian method.

69. CISE301_Topic3KFUPM69 Gauss-Jordan Method Example

70. CISE301_Topic3KFUPM70 Gauss-Jordan Method Example

71. CISE301_Topic3KFUPM71 Gauss-Jordan Method Example

72. CISE301_Topic3KFUPM72 Gauss-Jordan Method Example