1. Chapter 3 Solution of Algebraic Equations 1 ChE 401: Computational Techniques for Chemical Engineers Fall 2009/2010 DRAFT SLIDES

2. 1.Basic definitions 2.Consistency of equations and existence of solutions 3.Gaussian Elimination 4.LU Factorization 5.Gauss-Seidel Method Chapter 1 Lecture Outline 2 Chapter 3 DRAFT SLIDES

3. Chapter 1 Basic definitions 3 Chapter 3  Representation of linear systems DRAFT SLIDES

4. Chapter 1 4 Chapter 3 Matrix rank The rank r of matrix A is defined as the order of the largest nonsingular square matrix within A. OR The rank of a matrix is the maximum number of independent rows (or, the maximum number of independent columns). Singular matrix is a matrix that has zero determinant DRAFT SLIDES

5. Chapter 1 5 Chapter 3 Matrix rank Example 1: consider the following (3  4) matrix: There are four sub-matrices of order (3  3): det = 0 r = 2 DRAFT SLIDES

6. Chapter 1 6 Chapter 3 DRAFT SLIDES

7. Chapter 1 7 Chapter 3 Consistency of equations and existence of solutions The solution of a system of linear equations may or may not exist, and it may or may not be unique. Consider the following general system. N unknowns are related by M equations DRAFT SLIDES

8. Chapter 1 8 Chapter 3 Consistency of equations and existence of solutions M = N has unique solution if - no row is a linear combination of the others (row degeneracy), or - no column is a linear combination of the others (column degeneracy) Nonsingular matrix has a unique solution Mathematically these statements are exact, but, Numerically….. DRAFT SLIDES

9. Chapter 1 9 Chapter 3 Consistency of equations and existence of solutions: numerical issues If a system is too close to linear dependence …  an algorithm may fail altogether to get a solution  round off errors can produce apparent linear dependence at some point in the solution process f 1 = t+1 f 2 = 0.95t +1.01  The numerical procedure will fail totally DRAFT SLIDES

10. Chapter 1 10 Chapter 3 Consistency of equations and existence of solutions: numerical issues If a system is too close to linear dependence …  an algorithm may still work but produce nonsense  accumulated round of errors can swamp the solution - particularly in close-to-singular systems - particularly if N is too large  not algorithmic failure, but answer is (wildly) incorrect  Error can be confirmed by direct substitution in original equations How can we check for the error in answers in this case? DRAFT SLIDES

11. Chapter 1 11 Chapter 3 Consistency of equations and existence of solutions Under-determined systems…..  M<N OR M = N with degenerate equations  Fewer equations than unknowns  May be no solution, OR  May be an infinite number of solutions i.e. subspaces of solutions - arbitrary values must be assigned to (N-M) unknowns. - changing the values of (N-M) unknowns, results in new values for the rest of unknowns. - singular value decomposition is a powerful technique DRAFT SLIDES

12. Chapter 1 12 Chapter 3 Consistency of equations and existence of solutions Over-determined systems…..  M>N AND no degenerate  More equations than unknowns  Generally there is no solution  The best compromise solution is sought - closest to satisfy all equations - requires quantification of “closeness” to correct solution - most often: sum of squares of differences between left and right hand sides is minimized  linear least squares problem DRAFT SLIDES

13. Chapter 1 13 Chapter 3 Consistency of equations and existence of solutions: second approach… The solution of a system of linear equations may or may not exist, and it may or may not be unique. Existence of solutions can be determined by comparing the rank of the Matrix A with the rank of the augmented matrix A a. N unknowns are related by M equations DRAFT SLIDES

14. Chapter 1 14 Chapter 3 Consistency of equations and existence of solutions The set of equations has a solution if, and only if, the rank of the augmented matrix is equal to the rank of the coefficient matrix. If in addition: rank (A) = N  unique solution rank (A) < N & rank (A) = rank (A a )  Infinite number of solutions rank (A) < N & rank (A) < rank (A a )  No solution DRAFT SLIDES

15. Chapter 1 15 Chapter 3 Some examples Solve the following set of 2 equations in 2 unknowns for x and y. DRAFT SLIDES

16. Chapter 1 16 Chapter 3 DRAFT SLIDES

17. Chapter 1 17 Chapter 3 Some examples Solution DRAFT SLIDES

18. Direct methods: It consists of techniques that do not require iterations to find the solution. Most direct methods involve manipulations of the matrix (A)and vector b to obtain easy to solve equations. Chapter 1 Solution of Algebraic Equations 18 Chapter 3 Indirect (Iterative) methods: These methods start with initial approximation (guess) x o to the solution x and generate a sequence of vectors x k that converge to the solution vector. These techniques are efficiently used to solve large systems where computer storage and computational time are important considerations. DRAFT SLIDES

19. - It systematically eliminates unknowns from the problem till only one equation and unknown are left. - The value of this unknown is determined and the remaining unknowns are calculated in turn. Chapter 1 Gaussian Elimination (GE) 19 Chapter 3 Gaussian elimination is very efficient method in solving large sets of linear equations. For large n, it requires n 3 /2 calculations to get the solution. DRAFT SLIDES

20. Consider: Chapter 1 Gaussian Elimination 20 Chapter 3 We use row operations to simplify the system. e.g. eliminate element A 21 by subtracting A 21 /A 11 = d 21 times row 1 from row 2. Pivot row Pivot element 0 0 0 DRAFT SLIDES A ’ 22 A ’ 23 A ’ 32 A ’ 33 A ” 33

21. Chapter 1 Gaussian Elimination (A 21 ) 21 Chapter 3 DRAFT SLIDES

22. Chapter 1 Gaussian Elimination (A 31 ) 22 Chapter 3 DRAFT SLIDES

23. Chapter 1 Gaussian Elimination (A 32 ) 23 Chapter 3 DRAFT SLIDES

24. Chapter 1 Backsubstitutation 24 Chapter 3 DRAFT SLIDES

25. Chapter 1 25 Chapter 3 Example  -4/3*-4 8/3 -4 32/3 DRAFT SLIDES

26. Chapter 1 26 Chapter 3 Example -1/3* -1 2/3 -1 -8/3 DRAFT SLIDES

27. Chapter 1 27 Chapter 3 Example -5/14* 0 -5/3 30/14 130/42 DRAFT SLIDES

28. Chapter 1 28 Chapter 3 Example DRAFT SLIDES

29. Chapter 1 29 Chapter 3 DRAFT SLIDES

30. Chapter 1 30 Chapter 3 Example 2.1 in the textbook to be discussed…. DRAFT SLIDES

31. Chapter 1 31 Chapter 3 Example: 2.2  0 PIVOTINGPARTIAL There is no restriction upon which equation is placed first or which variable is eliminated first…. DRAFT SLIDES

32. Chapter 1 32 Chapter 3 Use Gaussian elimination to solve the following set of equations 2-digit arithmetic d 21 = 100 a 21 = 0 a 22 = 0.01+ 100  100 b 2 = 1-100  -100 x 2 = -1 x 1 = (1.0-1.0)/0.01 = 0 Exact solution… x 2 = -0.9899 x 1 = 1.0099 1.0 % error 100 % error DRAFT SLIDES

33. Chapter 1 33 Chapter 3 Try this…… 2-digit arithmetic d 21 = 0.01 a 22 = -1-0.0001  -1.0 b 2 = 1-0.01  -1.0 x 2 = -1 x 1 = 1+0.01  1. 0 Exact solution… x 2 = -0.9899 x 1 = 1.0099 1.0 % error USE PIVOTING What did you learn ? DRAFT SLIDES

34. Chapter 1 34 Chapter 3 What about this??? Multiply the first equation by 200, what do you find? TRY IT BY YOURSELF...... DRAFT SLIDES

35. LU factorization (without pivoting) Let: A = L.U with L unit lower triangular, U upper triangular, i.e. Chapter 1 LU Factorization 35 Chapter 3 DRAFT SLIDES

36. LU factorization (without pivoting) This factorization can be used to solve the following linear set A.x = (L.U).x = L.(U.x) = L.y = b Such that L.y = b (forward substitution) U.x = y (backward substitution) Chapter 1 LU Factorization 36 Chapter 3 How many set of linear equations do we have now? What is the advantage of breaking up one linear set into two successive ones? DRAFT SLIDES

37. Algorithm…. This method is called Doolittle’s factorization Partition A, L, U as block matrices: with L 22 unit-lower triangle and U 22 upper triangle Chapter 1 LU Factorization 37 Chapter 3 DRAFT SLIDES

38. Determine L, U from A = L.U, i.e. this gives u 11 = a 11, U 12 = A 12, L 21 = (1/a 11 )A 21 L 22 U 22 = A 22 – L 21 U 12 Chapter 1 LU Factorization 38 Chapter 3 DRAFT SLIDES

39. Example…. Factorize matrix A Chapter 1 LU Factorization 39 Chapter 3 Solution… DRAFT SLIDES

40. Chapter 1 40 Chapter 3 Solution… Step 1: First row of U and first column of L Step 2: Find l 32, u 22, u 23 and u 33 L 22 U 22 = A 22 – L 21 U 12 = - DRAFT SLIDES

41. Chapter 1 41 Chapter 3 Solution… Step 2: Find l 32, u 22, u 23 and u 33 = - = DRAFT SLIDES

42. Note….  Re-solve the previous example using GE method Chapter 1 LU Factorization 42 Chapter 3 What are the values for d ij ? d 21 = 1/2 d 31 = 6/8 d 32 = 11/16 Compare DRAFT SLIDES

43. Note….  Re-solve the previous example using GE method Chapter 1 LU Factorization 43 Chapter 3 After elimination steps we got Compare DRAFT SLIDES

44. Note….  Solving a set of equations using LU decomposition takes the same number of calculations as Gaussian elimination since both methods are essentially equivalent. Why to use this method? Chapter 1 LU Factorization 44 Chapter 3 1. The solution of a triangular set of equations is quite trivial, forward substitution is used to find y elements and vector x is obtained using back substitution scheme DRAFT SLIDES

45. Note….  Solving a set of equations using LU decomposition takes the same number of calculations as Gaussian elimination since both methods are essentially equivalent. Why to use this method? Chapter 1 LU Factorization 45 Chapter 3 2. Once A is transformed into L and U, it can be used with any right-hand-side vector, b. DRAFT SLIDES

46. MATLAB has built in routines for solving linear systems. For LU decomposition, use the command “lu”. Consider the previous example Chapter 1 LU Factorization 46 Chapter 3 A =[8 2 9;4 9 4;6 7 9]; [L,U]=lu(A) b=[1;2;3]; Y=L \ b; X=U\Y DRAFT SLIDES

47. Example 2.4, p.26 Chapter 1 47 Chapter 3 DRAFT SLIDES

48. Example 2.4, p.26 Chapter 1 48 Chapter 3 [ ] x b A DRAFT SLIDES

49. Example 2.4, p.26 Chapter 1 49 Chapter 3 For MATLAB code, see m-file called Example 24 DRAFT SLIDES

50. Chapter 1 50 Chapter 3 Example 2.3 in the textbook to be discussed…. DRAFT SLIDES

51. A three-stages absorption system, shown in the figure below, is described by the following set of equations: Chapter 1 Example…. 51 Chapter 3 Solve for the values of x i and y i, for the following data: x f = 0.01, y f = 0.06, L = 40.8 lb/min, G = 66.7 lb/min, a=0.72. Note that y i = ax i 51DRAFT SLIDES

52. Chapter 1 Solution 52 Chapter 3 Use MATLAB…. 52DRAFT SLIDES Write your code

53. Chapter 1 53 Chapter 3 DRAFT SLIDES System Condition Condition number ~ 10 4,It is ill-conditioned system Determinant = 1  10 -4

54. Chapter 1 Gauss-Seidel Method 54 Chapter 3 Why to use this method? -It requires fewer calculations than GE for very large systems of linear equations or for linear equations with a sparse coefficient matrix which is not banded. -It is an efficient method for the solution of ill-conditioned set of linear equations. -If the physics of the problem are understood, a close initial guess can be made, decreasing the number of iterations needed. DRAFT SLIDES An iterative approach

55. Chapter 1 Gauss-Seidel Method 55 Chapter 3 Basic Procedure: -Algebraically solve each linear equation for x i -Assume an initial guess solution array -Solve for each x i -Check for error after each iteration to check if error is within a pre-specified tolerance. -Repeat until error is within the tolerance. DRAFT SLIDES

56. Chapter 1 Gauss-Seidel Method 56 Chapter 3 Algorithm A set of n equations and n unknowns:.. If the diagonal elements are non- zero Rewrite each equation solving for the corresponding unknown For example: First equation, solve for x 1 n equation, solve for x n DRAFT SLIDES

57. Chapter 1 Gauss-Seidel Method 57 Chapter 3 Algorithm Rewriting each equation From Equation 1 From equation 2 From equation n-1 From equation n DRAFT SLIDES

58. Chapter 1 Gauss-Seidel Method 58 Chapter 3 Algorithm General Form for any row ‘i’ DRAFT SLIDES

59. Chapter 1 Gauss-Seidel Method 59 Chapter 3 Solve for the unknowns Assume an initial guess for [X] Use rewritten equations to solve for each value of x i. Remember to use the most recent value of x i. DRAFT SLIDES

60. Chapter 1 Gauss-Seidel Method 60 Chapter 3 Check for the tolerance using a pre-defined criterion such as The iterations are stopped when the error is less than a pre-specified tolerance for all unknowns. DRAFT SLIDES

61. Chapter 1 Gauss-Seidel Method 61 Chapter 3 Example…. Consider the following set of linear equations: DRAFT SLIDES

62. Chapter 1 Gauss-Seidel Method 62 Chapter 3 Applying the initial guess and solving for x i Initial Guess When solving for x 2, how many of the initial guess values were used? DRAFT SLIDES

63. Chapter 1 Gauss-Seidel Method 63 Chapter 3 Finding the percentage relative error At the end of the first iteration The maximum percentage relative error is 125.47% DRAFT SLIDES

64. Chapter 1 Gauss-Seidel Method 64 Chapter 3 Iteration #2 Using from iteration #1 the values of a i are found: DRAFT SLIDES

65. Chapter 1 Gauss-Seidel Method 65 Chapter 3 Finding the percentage relative error At the end of the second iteration The maximum percentage relative error is 85.695% DRAFT SLIDES

66. Chapter 1 Gauss-Seidel Method 66 Chapter 3 Iteration x1x1 x2x2 x3x3 123456123456 3.6720 12.056 47.182 193.33 800.53 3322.6 72.767 69.543 74.447 75.595 75.850 75.906 −7.8510 −54.882 −255.51 −1093.4 −4577.2 −19049 125.47 85.695 78.521 76.632 76.112 75.972 −155.36 −798.34 −3448.9 −14440 −60072 −249580 103.22 80.540 76.852 76.116 75.963 75.931 Repeating more iterations, we got Notice – The relative errors are not decreasing Also, the solution is not converging to the true solution DRAFT SLIDES

67. Chapter 1 Gauss-Seidel Method 67 Chapter 3 DRAFT SLIDES What is the problem? This example illustrates a drawback of the Gauss-Siedel method: not all systems of equations will converge. Is it fixable? One class of system of equations always converges: One with a diagonally dominant coefficient matrix. Diagonally dominant: A in A.X = b is diagonally dominant if:

68. Chapter 1 Gauss-Seidel Method 68 Chapter 3 DRAFT SLIDES Which coefficient matrix is diagonally dominant? Most physical systems do result in simultaneous linear equations that have diagonally dominant coefficient matrices.

69. Chapter 1 Gauss-Seidel Method 69 Chapter 3 DRAFT SLIDES Example 2 Given the system of equations With an initial guess of The coefficient matrix is: Will the solution converge using the Gauss-Siedel method?

70. Chapter 1 Gauss-Seidel Method 70 Chapter 3 DRAFT SLIDES Checking if the coefficient matrix is diagonally dominant The inequalities are all true; therefore the solution should converge using the Gauss-Seidel Method

71. Chapter 1 Gauss-Seidel Method 71 Chapter 3 DRAFT SLIDES Rewriting each equationWith an initial guess of

72. Chapter 1 Gauss-Seidel Method 72 Chapter 3 DRAFT SLIDES The absolute relative approximate error The maximum percentage relative error after the first iteration is 100%

73. Chapter 1 Gauss-Seidel Method 73 Chapter 3 DRAFT SLIDES Iterationx1x1 x2x2 x3x3 123456123456 0.50000 0.14679 0.74275 0.94675 0.99177 0.99919 100.00 240.61 80.236 21.546 4.5391 0.74307 4.9000 3.7153 3.1644 3.0281 3.0034 3.0001 100.00 31.889 17.408 4.4996 0.82499 0.10856 3.0923 3.8118 3.9708 3.9971 4.0001 67.662 18.876 4.0042 0.65772 0.074383 0.00101 Repeating more iterations, the following values are obtained The solution obtained Note that the exact solution is.

74. Chapter 1 Gauss-Seidel Method 74 Chapter 3 DRAFT SLIDES Example 3 Given the system of equations With an initial guess of

75. Chapter 1 Gauss-Seidel Method 75 Chapter 3 DRAFT SLIDES Iterationx1x1 x2x2 x3x3 123456123456 21.000 −196.15 −1995.0 −20149 2.0364×10 5 −2.0579×10 5 95.238 110.71 109.83 109.90 109.89 0.80000 14.421 −116.02 1204.6 −12140 1.2272×10 5 100.00 94.453 112.43 109.63 109.92 109.89 50.680 −462.30 4718.1 −47636 4.8144×10 5 −4.8653×10 6 98.027 110.96 109.80 109.90 109.89 Conducting six iterations, the following values are obtained The values are not converging? Your comments……..

76. Chapter 1 Gauss-Seidel Method 76 Chapter 3 DRAFT SLIDES Example 3 Example 2 If a system of linear equations is not diagonally dominant, check to see if rearranging the equations can form a diagonally dominant matrix.