Chapter 9 Gauss Elimination

Gauss Elimination 9.1 Solving small numbers of equations 9.2 Naive Gauss Elimination 9.3 Pivoting 9.4 Tridiagonal Systems MATLAB M-files GaussNaive, GaussPivot, Tridiag

Small Matrices Graphical Cramer's rule Elimination For small numbers of equations, can be solved by hand Graphical Cramer's rule Elimination

Graphical Method x1 + x2 = 3 2x1 – x2 = 3 One solution

Graphical Method No solution 2x1 – x2 = – 1 2x1 – x2 = 3

Graphical Method Infinite many solution 2x1 – x2 = 3 6x1 – 3x2 = 9

Graphical Method Ill conditioned 2x1 – x2 = 3 2.1x1 – x2 = 3

Cramer’s Rule Compute the determinant D 2 x 2 matrix 3 x 3 matrix

Cramer’s Rule To find xk for the following system Replace kth column of as with bs (i.e., aik  bi )

Example 3 x 3 matrix

Ill-Conditioned System What happen if the determinant D is very small or zero? Divided by zero (linearly dependent system) Divided by a small number: Round-off error Loss of significant digits

Elimination Method Eliminate x2  Subtract to get Not very practical for large number (> 4) of equations

MATLAB’s Methods Forward slash ( / ) Back-slash ( \ ) Multiplication by the inverse of the quantity under the slash

Gauss Elimination Manipulate equations to eliminate one of the unknowns Develop algorithm to do this repeatedly The goal is to set up upper triangular matrix Back substitution to find solution (root)

Basic Gauss Elimination Direct Method (no iteration required) Forward elimination Column-by-column elimination of the below-diagonal elements Reduce to upper triangular matrix Back-substitution

Naive Gauss Elimination Begin with Multiply the first equation by a21 / a11 and subtract from second equation

Gauss Elimination Reduce to Repeat the forward elimination to get

Gauss Elimination First equation is pivot equation a11 is pivot element Now multiply second equation by a'32 /a'22 and subtract from third equation

Gauss Elimination Repeat the elimination of ai2 and get Continue and get

Back Substitution Now we can perform back substitution to get {x} By simple division Substitute this into (n-1)th equation Solve for xn-1 Repeat the process to solve for xn-2 , xn-3 , …. x2, x1

Back Substitution Back substitution: starting with xn Solve for xn1 , xn2 , … , 3, 2, 1 for i = n1, n2, …, 1 Naive Gauss Elimination

Elimination of first column

Elimination of second column

Elimination of third column Upper triangular matrix

Back-Substitution Upper triangular matrix

Example

Forward Elimination

Upper Triangular Matrix

Back-Substitution

MATLAB Script File: GaussNaive

Print all factor and Aug (do not suppress output) >> format short >> x = GaussNaive(A,b) m = 4 n = Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1 factor = -1 0 2 4 0 0 6 0 2 -10 -14 -5 factor = 0.5000 Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 2 -10 -14 -5 1 0 0 -14 -14 -5 14 0 0 0 -70 -33 x = 0.4714 -0.1143 0.2286 -0.1857 x4 Aug = [A, b] x3 x2 Eliminate second column x1 Back-substitution Eliminate third column Print all factor and Aug (do not suppress output) Eliminate first column

Algorithm for Gauss elimination 1. Forward elimination for each equation j, j = 1 to n-1 for all equations k greater than j (a) multiply equation j by akj /ajj (b) subtract the result from equation k This leads to an upper triangular matrix 2. Back-Substitution (a) determine xn from (b) put xn into (n-1)th equation, solve for xn-1 (c) repeat from (b), moving back to n-2, n-3, etc. until all equations are solved

Operation Count Important as matrix gets large For Gauss elimination Elimination routine uses on the order of O(n3/3) operations Back-substitution uses O(n2/2)

Operation Count Total operation counts for elimination stage = 2n3/3 + O(n2) Total operation counts for back substitution stage = n2 + O(n)

Operation Count Number of flops (floating-point operations) for Naive Gauss elimination Computation time increase rapidly with n Most of the effort is incurred in the elimination step Improve efficiency by reducing the elimination effort

Partial Pivoting Problems with Gauss elimination division by zero round off errors ill conditioned systems Use “Pivoting” to avoid this Find the row with largest absolute coefficient below the pivot element Switch rows (“partial pivoting”) complete pivoting switch columns also (rarely used)

Round-off Errors A lot of chopping with more than n3/3 operations More important - error is propagated For large systems (more than 100 equations), round-off error can be very important (machine dependent) Ill conditioned systems - small changes in coefficients lead to large changes in solution Round-off errors are especially important for ill-conditioned systems

Ill-conditioned System 2x1 – x2 = 3 2.1x1 – x2 = 3

Ill-Conditioned System Consider Since slopes are almost equal Divided by small number

Determinant Calculate determinant using Gauss elimination

Gauss Elimination with Partial Pivoting Forward elimination for each equation j, j = 1 to n-1 first scale each equation k greater than j then pivot (switch rows) Now perform the elimination (a) multiply equation j by akj /ajj (b) subtract the result from equation

Partial (Row) Pivoting

Forward Elimination Interchange rows 1 & 4

Forward Elimination No interchange required

Back-Substitution Interchange rows 3 & 4

MATLAB M-File: GaussPivot Partial Pivoting (switch rows) largest element in {x} [big,i] = max(x) Partial Pivoting index of the largest element

Eliminate first column >> format short >> x=GaussPivot0(A,b) Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1 big = 6 i = 4 ipr = factor = -0.1667 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000 0.1667 0 -0.3333 1.6667 2.3333 0.8333 Aug = [A b] Find the first pivot element and its index Interchange rows 1 and 4 Eliminate first column No need to interchange

Save factors fij for LU Decomposition Back substitution big = 2.3333 i = 1 ipr = 2 factor = 0.4286 Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 -0.3333 1.6667 2.3333 0.8333 -0.1429 0 0 2.0000 2.0000 0.7143 4 Second pivot element and index No need to interchange x = 0.4714 -0.1143 0.2286 -0.1857 Eliminate second column Third pivot element and index Interchange rows 3 and 4 Save factors fij for LU Decomposition Eliminate third column

TRUSS 4 5 F45 F35 F14 F24 F25   1 3 H1 2 F23 F12 V1 V3   W = 100 kg

Statics: Force Balance Node 1 Node 2 Node 3 Node 4 Node 5

Example: Forces in a Simple Truss

Define Matrices A and b in script file function [A,b]=Truss(alpha,beta,gamma,delta) A=zeros(10,10); A(1,1)=1; A(1,5)=sin(alpha); A(2,2)=1; A(2,4)=1; A(2,5)=cos(alpha); A(3,7)=sin(beta); A(3,8)=sin(gamma); A(4,4)=-1; A(4,6)=1; A(4,7)=-cos(beta); A(4,8)=cos(gamma); A(5,3)=1; A(5,9)=sin(gamma); A(6,6)=-1; A(6,9)=-cos(delta); A(7,5)=-sin(alpha); A(7,7)=-sin(beta); A(8,5)=-cos(alpha); A(8,7)=cos(beta); A(8,10)=1; A(9,8)=-sin(gamma); A(9,9)=-sin(delta); A(10,8)=-cos(gamma); A(10,9)=cos(delta); A(10,10)=-1; b=zeros(10,1); b(3,1)=100;

Gauss Elimination with Partial Pivoting >> alpha=pi/6; beta=pi/3; gamma=pi/4; delta=pi/3; >> [A,b] = Truss(alpha,beta,gamma,delta) A = 1.0000 0 0 0 0.5000 0 0 0 0 0 0 1.0000 0 1.0000 0.8660 0 0 0 0 0 0 0 0 0 0 0 0.8660 0.7071 0 0 0 0 0 -1.0000 0 1.0000 -0.5000 0.7071 0 0 0 0 1.0000 0 0 0 0 0 0.7071 0 0 0 0 0 0 -1.0000 0 0 -0.5000 0 0 0 0 0 -0.5000 0 -0.8660 0 0 0 0 0 0 0 -0.8660 0 0.5000 0 0 1.0000 0 0 0 0 0 0 0 -0.7071 -0.8660 0 0 0 0 0 0 0 0 -0.7071 0.5000 -1.0000 b = 100 >> x = GaussPivot(A,b) x = 40.5827 48.5140 70.2914 -81.1655 34.3046 46.8609 84.0287 -68.6091 -93.7218 Simple truss

Banded Matrix HBW: Half Band Width

Banded Matrix ai,j= 0 if j > i + HB or j< i - HB HB: Half Bandwidth B: Bandwidth B = 2*HB + 1 In this example HB = 1 & B = 3

Tridiagonal Matrix Only three nonzero elements in each equation (3n instead of n2 elements) Subdiagonal, diagonal, superdiagonal Row Scaling (not implemented in textbook) -- scale the diagonal element to aii = 1 Solve by Gauss elimination

Tridiagonal Matrix Special case of banded matrix with bandwidth = 3 Save storage, 3  n instead of n  n

Tridiagonal Matrix Forward elimination Back substitution Use factor = ek / fk1 to eliminate subdiagonal element Apply the same matrix operations to right hand side

Hand Calculations: Tridiagonal Matrix (a) Forward elimination (b) Back substitution

MATLAB M-file: Tridiag

Example: Tridiagonal matrix function [e,f,g,r] = example e=[ 0 -2 4 -0.5 1.5 -3]; f=[ 1 6 9 3.25 1.75 13]; g=[-2 4 -0.5 1.5 -3 0]; r=[-3 22 35.5 -7.75 4 -33]; » [e,f,g,r] = example e = 0 -2.0000 4.0000 -0.5000 1.5000 -3.0000 f = 1.0000 6.0000 9.0000 3.2500 1.7500 13.0000 g = -2.0000 4.0000 -0.5000 1.5000 -3.0000 0 r = -3.0000 22.0000 35.5000 -7.7500 4.0000 -33.0000 » x = Tridiag (e, f, g, r) x = 1 2 3 -1 -2 -3 Note: e(1) = 0 and g(n) = 0

Chapter 8 Problem 8.2 (20), 8.3(10), 8.7(15) Chapter 9 CVEN 302-501 Homework No. 6 Chapter 8 Problem 8.2 (20), 8.3(10), 8.7(15) Chapter 9 Problem 9.5 (15), 9.8(20), 9.11(20). Due Monday 10/06/08 at the beginning of the period