Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 ~ Linear Algebraic Equations ~ Gauss Elimination Chapter.

Slides:



Advertisements
Similar presentations
Ordinary Differential Equations
Advertisements

Chapter 2 Solutions of Systems of Linear Equations / Matrix Inversion
Algebraic, transcendental (i.e., involving trigonometric and exponential functions), ordinary differential equations, or partial differential equations...
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Parallel Programming in C with MPI and OpenMP Michael J. Quinn.
Simultaneous Linear Equations
SOLVING SYSTEMS OF LINEAR EQUATIONS. Overview A matrix consists of a rectangular array of elements represented by a single symbol (example: [A]). An individual.
CISE301_Topic3KFUPM1 SE301: Numerical Methods Topic 3: Solution of Systems of Linear Equations Lectures 12-17: KFUPM Read Chapter 9 of the textbook.
Numerical Algorithms Matrix multiplication
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 61.
Part 3 Chapter 9 Gauss Elimination
Solution of linear system of equations
Chapter 9 Gauss Elimination The Islamic University of Gaza
Major: All Engineering Majors Author(s): Autar Kaw
Ecuaciones Algebraicas lineales. An equation of the form ax+by+c=0 or equivalently ax+by=- c is called a linear equation in x and y variables. ax+by+cz=d.
Linear Algebraic Equations
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 Ordinary Differential Equations Equations which are.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 51.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Martin Mendez UASLP Chapter 61 Unit II.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Chapter 181 Interpolation Chapter 18 Estimation of intermediate.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Parallel Programming in C with MPI and OpenMP Michael J. Quinn.
Algorithm for Gauss elimination 1) first eliminate for each eq. j, j=1 to n-1 for all eq.s k greater than j a) multiply eq. j by a kj /a jj b) subtract.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 181 Interpolation.
Mujahed AlDhaifallah (Term 342) Read Chapter 9 of the textbook
Systems of Linear Equations
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 171 Least.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Part 31 Chapter.
Linear Simultaneous Equations
Copyright © Cengage Learning. All rights reserved. 7.6 The Inverse of a Square Matrix.
Major: All Engineering Majors Author(s): Autar Kaw
1 1.1 © 2012 Pearson Education, Inc. Linear Equations in Linear Algebra SYSTEMS OF LINEAR EQUATIONS.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 31.
Copyright © 2014, 2010, 2007 Pearson Education, Inc.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 Part 2 Roots of Equations Why? But.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Part 31 Linear.
MATH 250 Linear Equations and Matrices
Engineering Analysis ENG 3420 Fall 2009 Dan C. Marinescu Office: HEC 439 B Office hours: Tu-Th 11:00-12:00.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 111.
MECN 3500 Inter - Bayamon Lecture 7 Numerical Methods for Engineering MECN 3500 Professor: Dr. Omar E. Meza Castillo
ΑΡΙΘΜΗΤΙΚΕΣ ΜΕΘΟΔΟΙ ΜΟΝΤΕΛΟΠΟΙΗΣΗΣ 4. Αριθμητική Επίλυση Συστημάτων Γραμμικών Εξισώσεων Gaussian elimination Gauss - Jordan 1.
Chapter 3 Solution of Algebraic Equations 1 ChE 401: Computational Techniques for Chemical Engineers Fall 2009/2010 DRAFT SLIDES.
Chemical Engineering Majors Author(s): Autar Kaw
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 ~ Roots of Equations ~ Open Methods Chapter 6 Credit:
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 71.
Lecture 7 - Systems of Equations CVEN 302 June 17, 2002.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 Part 3 - Chapter 9 Linear Systems of Equations: Gauss Elimination.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 Part 3- Chapter 12 Iterative Methods.
Gaussian Elimination Electrical Engineering Majors Author(s): Autar Kaw Transforming Numerical Methods Education for.
Chapter 9 Gauss Elimination The Islamic University of Gaza
Linear Systems Numerical Methods. Linear equations N unknowns, M equations N unknowns, M equations where coefficient matrix.
Slide Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley A set of equations is called a system of equations. The solution.
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. by Lale Yurttas, Texas A&M University Chapter 271 Boundary-Value.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 Chapter 27.
Gaussian Elimination Industrial Engineering Majors Author(s): Autar Kaw Transforming Numerical Methods Education for.
Gaussian Elimination Mechanical Engineering Majors Author(s): Autar Kaw Transforming Numerical Methods Education for.
Lecture 6 - Single Variable Problems & Systems of Equations CVEN 302 June 14, 2002.
Part 3 Chapter 9 Gauss Elimination PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright © The McGraw-Hill Companies,
Gaussian Elimination Civil Engineering Majors Author(s): Autar Kaw Transforming Numerical Methods Education for STEM.
1 Numerical Methods Solution of Systems of Linear Equations.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 Part 3 - Chapter 9.
Pitfalls of Gauss Elimination ● Division by zero (avoid by pivoting) ● Round-off errors – Important with >100 eqns – Substitute answer into original eqn.
Part 3 Chapter 9 Gauss Elimination
Simultaneous Linear Equations
Chapter 10 and Matrix Inversion LU Decomposition
Part 3 - Chapter 9.
Major: All Engineering Majors Author(s): Autar Kaw
Numerical Analysis Lecture10.
Linear Systems Numerical Methods.
Simultaneous Linear Equations Gaussian Elimination with Partial Pivoting
Presentation transcript:

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1 ~ Linear Algebraic Equations ~ Gauss Elimination Chapter 9 Credit: Prof. Lale Yurttas, Chemical Eng., Texas A&M University

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 2 Solving Systems of Equations A linear equation in n variables: a 1 x 1 + a 2 x 2 + … + a n x n = b For small (n ≤ 3), linear algebra provides several tools to solve such systems of linear equations: –Graphical method –Cramer’s rule –Method of elimination Nowadays, easy access to computers makes the solution of very large sets of linear algebraic equations possible

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 3 Determinants and Cramer’s Rule [A] : coefficient matrix D : Determinant of A matrix

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 4 Computing the Determinant

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 5 Gauss Elimination Solve Ax = b Consists of two phases: –Forward elimination –Back substitution Forward Elimination reduces Ax = b to an upper triangular system Tx = b’ Back substitution can then solve Tx = b’ for x Forward Elimination Back Substitution

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 6 Gaussian Elimination Forward Elimination x 1 - x 2 + x 3 = 6 3x 1 + 4x 2 + 2x 3 = 9 2x 1 + x 2 + x 3 = 7 x 1 - x 2 + x 3 = x 2 - x 3 = x 2 - x 3 = -5 x 1 - x 2 + x 3 = 6 0 7x 2 - x 3 = (4/7)x 3 =-(8/7) -(3/1) Solve using BACK SUBSTITUTION: x 3 = 2 x 2 =-1 x 1 =3 -(2/1) -(3/7)

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 7 Back Substitution 1x01x0 +1x 1 –1x2–1x2 +4x 3 8= – 2x 1 –3x2–3x2 +1x 3 5= 2x22x2 – 3x 3 0= 2x32x3 4= x 3 = 2

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 8 1x01x0 +1x 1 –1x2–1x2 0= – 2x 1 –3x2–3x2 3= 2x22x2 6= Back Substitution x 2 = 3

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 9 1x01x0 +1x 1 3= – 2x 1 12= Back Substitution x 1 = –6

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1x01x0 9= Back Substitution x 0 = 9

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 11 for i  n down to 1 do /* calculate x i */ x [ i ]  b [ i ] / a [ i, i ] /* substitute in the equations above */ for j  1 to i-1 do b [ j ]  b [ j ]  x [ i ] × a [ j, i ] endfor Back Substitution (* Pseudocode *) Time Complexity?  O(n 2 )

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 12 Gaussian Elimination Forward Elimination

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 13 Forward Elimination 4x04x0 +6x 1 +2x 2 – 2x 3 =8 2x02x0 +5x 2 – 2x 3 =4 –4x0–4x0 – 3x 1 – 5x 2 +4x 3 =1 8x08x0 +18x 1 – 2x 2 +3x 3 =40 -(2/4) MULTIPLIERSMULTIPLIERS -(-4/4) -(8/4)

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 14 4x04x0 +6x 1 +2x 2 – 2x 3 =8 +4x 2 – 1x 3 =0 +3x 1 – 3x 2 +2x 3 =9 +6x 1 – 6x 2 +7x 3 =24 – 3x 1 -(3/-3) MULTIPLIERSMULTIPLIERS Forward Elimination -(6/-3)

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 15 4x04x0 +6x 1 +2x 2 – 2x 3 =8 +4x 2 – 1x 3 =0 1x21x2 +1x 3 =9 2x22x2 +5x 3 =24 – 3x 1 ?? MULTIPLIERMULTIPLIER Forward Elimination

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 16 4x04x0 +6x 1 +2x 2 – 2x 3 =8 +4x 2 – 1x 3 =0 1x21x2 +1x 3 =9 3x33x3 =6 – 3x 1 Forward Elimination

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 17 Gaussian Elimination Operation count in Forward Elimination n TOTAL 1 st column: 2n(n-1)  2n 2 2(n-1) 2 2(n-2) 2 ……. b 11 b 22 b 33 b 44 b 55 b 66 b 77 b 66

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 18 Pitfalls of Elimination Methods Division by zero It is possible that during both elimination and back-substitution phases a division by zero can occur. For example: 2x 2 + 3x 3 = x 1 + 6x 2 + 7x 3 = -3 A =467 2x 1 + x 2 + 6x 3 = 5216 Solution: pivoting (to be discussed later)

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 19 Pitfalls (cont.) Round-off errors Because computers carry only a limited number of significant figures, round-off errors will occur and they will propagate from one iteration to the next. This problem is especially important when large numbers of equations (100 or more) are to be solved. Always use double-precision numbers/arithmetic. It is slow but needed for correctness! It is also a good idea to substitute your results back into the original equations and check whether a substantial error has occurred.

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. ill-conditioned systems - small changes in coefficients result in large changes in the solution. Alternatively, a wide range of answers can approximately satisfy the equations. (Well-conditioned systems – small changes in coefficients result in small changes in the solution) Problem: Since round off errors can induce small changes in the coefficients, these changes can lead to large solution errors in ill-conditioned systems. Example: x 1 + 2x 2 = x 1 + 2x 2 = 10.4 x 1 + 2x 2 = x 1 + 2x 2 = 10.4 Pitfalls (cont.)

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Surprisingly, substitution of the erroneous values, x 1 =8 and x 2 =1, into the original equation will not reveal their incorrect nature clearly: x 1 + 2x 2 = 108+2(1) = 10 (the same!) 1.1x 1 + 2x 2 = (8)+2(1)=10.8 (close!) IMPORTANT OBSERVATION: An ill-conditioned system is one with a determinant close to zero If determinant D=0 then there are infinitely many solutions  singular system Scaling (multiplying the coefficients with the same value) does not change the equations but changes the value of the determinant in a significant way. However, it does not change the ill-conditioned state of the equations! DANGER! It may hide the fact that the system is ill-conditioned!! How can we find out whether a system is ill-conditioned or not? Not easy! Luckily, most engineering systems yield well-conditioned results! One way to find out: change the coefficients slightly and recompute & compare ill-conditioned systems (cont.) –

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 22 COMPUTING THE DETERMINANT OF A MATRIX USING GAUSSIAN ELIMINATION The determinant of a matrix can be found using Gaussian elimination. Here are the rules that apply: Interchanging two rows changes the sign of the determinant. Multiplying a row by a scalar multiplies the determinant by that scalar. Replacing any row by the sum of that row and the multiple of any other row does NOT change the determinant. The determinant of a triangular matrix (upper or lower triangular) is the product of the diagonal elements. i.e. D=t 11 t 22 t 33

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Techniques for Improving Solutions

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 24 Gauss-Jordan Elimination a 11 x 22 0 x x x x 66 x 77 x b 11 b 22 b 33 b 44 b 55 b 66 b 77 b 88 b x 99

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 25 Gauss-Jordan Elimination: Example R2  R2 - (-1)R1 R3  R3 - ( 3)R1 Scaling R2: R2  R2/(-1) R1  R1 - (1)R2 R3  R3-(4)R2 Scaling R3: R3  R3/(18) R1  R1 - (7)R3 R2  R2-(-5)R3 RESULT: x 1 =8.45, x 2 =-2.89, x 3 =1.23 Time Complexity?  O(n 3 )

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 26 Systems of Nonlinear Equations Locate the roots of a set of simultaneous nonlinear equations: Systems of Nonlinear Equations

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. First-order Taylor series expansion of a function with more than one variable: The root of the equation occurs at the value of x 1 and x 2 where f 1(i+1) =0 and f 2(i+1) =0 Rearrange to solve for x 1(i+1) and x 2(i+1) Since x 1(i), x 2(i), f 1(i), and f 2(i) are all known at the i th iteration, this represents a set of two linear equations with two unknowns, x 1(i+1) and x 2(i+1) You may use several techniques to solve these equations

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 28 General Representation for the solution of Nonlinear Systems of Equations Solve this set of linear equations at each iteration:  Jacobian Matrix

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 29 Solution of Nonlinear Systems of Equations Solve this set of linear equations at each iteration: Rearrange:

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. n non-linear equations n unknowns to be solved: Jacobian Matrix: Need to solve this set of Linear Equations in each and every iterative solution of the original Non-Linear system of equations. For n unknowns, the size of the Jacobian matrix is n 2 and the time it takes to solve the linear system of equations (one iteration of the non-linear system) is proportional to O(n 3 ) There is no guarantee for the convergence of the non-linear system. There may also be slow convergence. In summary, finding the solution set for the Non-Linear system of equations is an extremely compute-intensive task. 30 Notes on the solution of Nonlinear Systems of Equations