ENGR-25_Linear_Equations-1.ppt 1 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Bruce Mayer, PE Licensed Electrical & Mechanical Engineer Engr/Math/Physics 25 Chp8 Linear Algebraic Eqns-2

ENGR-25_Linear_Equations-1.ppt 2 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Learning Goals  Define Linear Algebraic Equations  Solve Systems of Linear Equation by Hand using Gaussian Elimination Cramer’s Method  Distinguish between Equation System Conditions: Exactly Determined, Overdetermined, Underdetermined  Use MATLAB to Solve Systems of Eqns

ENGR-25_Linear_Equations-1.ppt 3 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method for Eqn Sys  Solves equations using determinants.  Gives insight into the existence and uniqueness of solutions –Identifies SINGULAR (a.k.a. Divide by Zero) Systems effects of numerical inaccuracy –Identifies ILL-CONDITIONED (a.k.a. Stiff) Systems

ENGR-25_Linear_Equations-1.ppt 4 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method – Illustrated-1  Cramer’s Method can Solve “Square” Systems; i.e., [No. Eqns] = [No. Unknowns]  Consider Sq Sys  Calc Cramer’s Determinant, D c Also Called the “Characteristic” or “Denominator” Determinant  D c  Determinant of the Coefficients

ENGR-25_Linear_Equations-1.ppt 5 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method – Illustrated-2  Now, to Find The Individual Solns, Sub The Constraint Vector for the Variable Coefficients and Compute the Determinant for Each unknown, D k  In this Example Find D x, D y, D z as

ENGR-25_Linear_Equations-1.ppt 6 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method – Illustrated-3  Once We’ve Calculated all these Determinants, The Rest is Easy  These Eqns Ilustrate the most UseFul Feature of Cramer’s Method D c appears in all THREE Denominators

ENGR-25_Linear_Equations-1.ppt 7 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method – Illustrated-4  Since –However, We can ANTICIPATE Problems if |D c | << than the SMALLEST Coefficient  Completing the Example  Can ID “Condition” by Calculating D c SINGULAR Systems → D c = 0 ILL-CONDITIONED Systems → D c = “Small” –Small is technically relative to the D k

ENGR-25_Linear_Equations-1.ppt 8 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method – Illustrated-6  Calc the Determinants  First Recall The SIGN pattern for Determinants  Find D c  Dc is LARGE → WELL Conditioned System

ENGR-25_Linear_Equations-1.ppt 9 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method – Illustrated-7  Solve using MATLAB’s det Function >> EqnSys = [ ; ; ]; >> Dc = det(EqnSys(:,1:3)) Dc = 1146 >> Dx = det([EqnSys(:,4),EqnSys(:,2:3)]) Dx = 1146 >> Dy = det([EqnSys(:,1),EqnSys(:,4), EqnSys(:,3)]) Dy = 2292 All Row Elements of Cols 1-3 All Row Elements of Cols 4, 2-3 All Row Elements of Cols 1, 4, 3

ENGR-25_Linear_Equations-1.ppt 10 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Method – Illustrated-8  Solve using MATLAB’s det Function >> Dz = det([EqnSys(:,1:2),EqnSys(:,4)]) Dz = 3438 >> x = Dx/Dc x = 1 >> y = Dy/Dc y = 2 >> z = Dz/Dc z = 3 All Row Elements of Cols 1-2, 4

ENGR-25_Linear_Equations-1.ppt 11 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer vs Homogenous:  In general, for a set of HOMOGENEOUS linear algebraic equations that contains the same number of equations as unknowns a nonzero solution exists only if the set is singular; that is, if Cramer’s determinant is zero furthermore, the solution is not unique. If Cramer’s determinant is not zero, the homogeneous set has a zero solution; that is, all the unknowns are zero

ENGR-25_Linear_Equations-1.ppt 12 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Cramer’s Rule Summary  Cramer’s determinant gives some insight into ill-conditioned problems, which are close to being singular.  A Cramer’s determinant close to zero indicates an ill-conditioned problem.

ENGR-25_Linear_Equations-1.ppt 13 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods UnderDetermined Systems  An UNDERdetermined system does not contain enough information to solve for ALL of the unknown variables Usually because it has fewer equations than unknowns.  In this case an INFINITE number of solutions can exist, with one or more of the unknowns dependent on the remaining unknowns. For such systems the Matrix-Inverse and Cramer’s methods will NOT work

ENGR-25_Linear_Equations-1.ppt 14 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods UnderDetermined Example-1  A simple UnderDetermined system is the equation  All we can do is solve for one of the unknowns in terms of the other; for example, x = 6 – 3y OR y = −x/3 + 2 An INFINITE number of (x,y) solutions satisfy this equation

ENGR-25_Linear_Equations-1.ppt 15 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods UnderDetermined Example-2  When there are more Unknowns than Equations, the LEFT-DIVISION method will give a solution with some of the unknowns set equal to ZERO For Example  which corresponds to x = 0 y = 2 >>A = [1, 3]; b = 6; >>solution = A\b solution = 0 2

ENGR-25_Linear_Equations-1.ppt 16 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods More UnderDetermined Systems  An infinite number of solutions might exist EVEN when the number of equations EQUALS the number of knowns Predict by Cramer as:  For such systems the Matrix Inverse method and Cramer’s method will also NOT work MATLAB’s left-division method generates an error message warning us that the matrix A is singular

ENGR-25_Linear_Equations-1.ppt 17 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Minimum Norm Solution  When det(A) = 0, We can use the PSEUDOINVERSE method to find ONE Solution, x, such that the Euclidean (or Pythagorean) Length of x is MINIMIZED  In MATLAB:  MATLAB will return the MINIMUM NORM SOLUTION → x = pinv(A)*b

ENGR-25_Linear_Equations-1.ppt 18 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Existence and Uniqueness  The set Ax = b with m equations and n unknowns has solutions if and only if  Rank[A] is the maximum number of LINEARLY INDEPENDENT rows of A Linear Independence → No Row of A is a SCALAR multiple of ANY OTHER Combinations of Rows

ENGR-25_Linear_Equations-1.ppt 19 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods An INconsistent Example  Consider Ax = b  Since The Ranks are Unequal → this system of equations is NOT solvable  Graphically  ERO: Multiply the 1 st row by −2 and add to the 2 nd row Rank[A]=1 Rank[Ab]=2

ENGR-25_Linear_Equations-1.ppt 20 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Existence and Uniqueness  Recall Rank for m-Eqns & n-Unknwns  Now Let r = rank[A] r = n uniqueIf condition (1) is satisfied and if r = n, then the solution is unique r < n infinitesolutionsIf condition (1) is satisfied but r < n, an infinite number of solutions exists and –r unknown variables can be expressed as linear combinations of the other n−r unknown variables, whose values are ARBITRARY

ENGR-25_Linear_Equations-1.ppt 21 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Homogenous Case  The homogeneous set Ax = 0 is a special case in which b = 0  For this case rank[A] = rank[Ab] always, and thus the system in all cases has the trivial solution x = 0  A nonzero solution, in which at least one unknown is nonzero, exists if and only if rank[A] < n (n  No. Unknowns)  If m < n, the homogeneous set always has a nonzero solution

ENGR-25_Linear_Equations-1.ppt 22 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods ILLconditioned Systems  An ill-conditioned set of equations is a set that is CLOSE to being SINGULAR  The ill-conditioned status depends on the ACCURACY with which the solution calculations are made When the internal numerical accuracy used by MATLAB is INsufficient to obtain a solution, then MATLAB prints a message to warn you that the matrix is close to singular and that the results might be INaccurate

ENGR-25_Linear_Equations-1.ppt 23 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods OVERdetermined Systems - 1  By Contrast an OVERdetermined system is a set of equations that has MORE independent equations than unknowns  For such a system the Matrix Inverse method and Cramer’s method will NOT work because the A matrix is not square  However, SOME overdetermined systems have exact solutions, and they can be obtained with the left division method x = A\b

ENGR-25_Linear_Equations-1.ppt 24 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods OVERdetermined Systems - 2  For OTHER OverDetermined systems, NO exact solution exists. In some of these cases, the left-division method does not yield an answer, while in other cases the left-division method gives an answer that satisfies the equation set only in a “LEAST SQUARES” sense –When MATLAB gives an answer to an overdetermined set, it does NOT tell us whether the answer is Exact or Least-Squares in Nature  We need to check the ranks of A and [Ab] to know whether the answer is the exact solution.

ENGR-25_Linear_Equations-1.ppt 25 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Example  Under/Over Determined % Bruce Mayer, PE * 27Feb08 % ENGR25 * Under/OverDetermined Linear Systems % file = Over_Under_Lin_Sys_Example_0904.m % Solve Under & Over Determined Systems for [x1; x2; x3; x4] % % An UNDERdetermined case => 3-Eqns in 4-Unknwns Au = [ ; ; ] bu = [5; -9; 7] disp('Showing UNDERdetermined solution & check - Hit Any Key to continue') xu = Au\bu bu_chk = Au*xu pause % % An OVERdetermined case => 5-Eqns in 4-Unknwns Ao = [ ; ; ; ; ] bo = [5; -9; 7; 2; -6] disp('Showing OVERdetermined solution & check') xo = Ao\bo bo_chk= Ao*xo

ENGR-25_Linear_Equations-1.ppt 26 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Uniqueness of solutions  A system has a unique solution If, and Only If (IFF): Rank[A]=Rank[Ab] = n Where n is the ORDER of the system; i.e., the Number of UNKNOWNS  Such systems are called full-rank systems

ENGR-25_Linear_Equations-1.ppt 27 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods OverDetermined Rank  To interpret MATLAB answers correctly for an OVERdetermined system, first check the ranks of [A] and [Ab] to see whether an exact solution exists  r[Ab] = r[A] = n (the no. of Unknowns)  If r[Ab] = r[A] > n then a unique soln does not exist, then you know that the left- division answer is a least squares solution Can easily Chk: Ax soln = b (exactly)? –This is at least as easy as the rank calc

ENGR-25_Linear_Equations-1.ppt 28 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 1  If the number of equations (m) in the set equals the number of unknown variables (n), the matrix A is square and MATLAB provides two ways of solving the equation set Ax = b: 1.The matrix inverse method; solve for x by typing x = inv(A)*b. 2.The matrix left-division method; solve for x by typing x = A\b.

ENGR-25_Linear_Equations-1.ppt 29 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 2  If A is square and if MATLAB does not generate an error message when you use one of these methods, then the set has a unique solution, which is given by the left-division method.  You can always CHECK the solution for x by typing A*x to see if the result is the same as b. Use this to avoid the rank calcs

ENGR-25_Linear_Equations-1.ppt 30 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 3  If when you type A*x you receive an error message, the set is UNDERdetermined, and either it does not have a solution or it has more than one solution  In such a case, if you need more information, you must use the following procedures.

ENGR-25_Linear_Equations-1.ppt 31 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 4  For UNDERdetermined and OVERdetermined sets, MATLAB provides three ways of dealing with the equation set Ax = b. (Note that the matrix inverse method will NEVER work with such sets.) 1.The matrix left-division method; solve for x by typing x = A\b 2.The pseudoinverse method; solve for x by typing x = pinv(A)*b 3.the Reduced Row Echelon Form (RREF) method. This method uses the MATLAB function rref to obtain a solution.

ENGR-25_Linear_Equations-1.ppt 32 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 5  UNDERdetermined Systems In an underdetermined system NOT enough information is given to determine the values of all the unknown variables. –An INFINITE number of solutions might exist in which one or more of the unknowns are dependent on the remaining unknowns –For such systems Cramer’s method and the matrix inverse method will not work because either A is not square or because |A| = 0.

ENGR-25_Linear_Equations-1.ppt 33 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 6 Underdetermined Systems cont –The left-division method will give a solution with some of the unknowns arbitrarily set equal to zero, but this solution is NOT the general solution; it’s just ONE solution (of Trillions) –An infinite number of solutions might exist even when the number of equations equals the number of unknowns. The left-division method fails to give a solution in such cases –In cases that have an infinite number of solutions, some of the unknowns can be expressed in terms of the remaining unknowns, whose values are arbitrary. The rref function can be used to find these relations.

ENGR-25_Linear_Equations-1.ppt 34 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 7  OVERdetermined Systems An overdetermined system is a set of equations that has more independent equations than unknowns –For such a system Cramer’s method and the matrix inverse method will not work because the A matrix is NOT SQUARE –Some overdetermined systems have exact solutions, which can be obtained with the left- division method A\b.

ENGR-25_Linear_Equations-1.ppt 35 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Solving Linear Eqns Summary - 8 OVERdetermined systems cont. –For overdetermined systems that have no exact solution, the estimated-answer given by the left-division method satisfies the equation set only in a LEAST SQUARES sense –When we use MATLAB to solve an overdetermined set, the program does not tell us whether the solution is exact. We must determine this information ourselves. The first step is to check the ranks of A and [A b] to see whether a solution exists; if no solution exists, then we know that the left-division solution is a least squares answer.

ENGR-25_Linear_Equations-1.ppt 36 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods FlowChart for LinSys Solution Can Construct Least-Squares Approximation

ENGR-25_Linear_Equations-1.ppt 37 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Descison Tree for LinSys Solns if Rank(A)==Rank(Ab) & Rank(A)==NoUnknowns disp('EXACT Solution') elseif Rank(A)==Rank(Ab) & Rank(A)<NoUnknowns disp('INFINITE Solutions') elseif Rank(Ab) > NoUnknowns disp('LEAST SQS Approximation') else disp('NO solution; can least-sqs est.') end

ENGR-25_Linear_Equations-1.ppt 38 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods All Done for Today Gabriel Cramer Born: 31 July 1704 in Geneva, Switzerland Died: 4 Jan 1752 in Bagnols-sur-Cèze, France

ENGR-25_Linear_Equations-1.ppt 39 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods Bruce Mayer, PE Licensed Electrical & Mechanical Engineer Engr/Math/Physics 25 Appendix

ENGR-25_Linear_Equations-1.ppt 40 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods MATLAB LinSys Solver Code - 1 % Script file lineq.m % Solves the set Ax = b, given A and b. % Check the ranks of A and [A b]. if rank(A) == rank([A b]) % Then The ranks are equal. Size_A = size(A); % Does the rank of A equal the number of unknowns? if rank(A) == size_A(2) % Then the Rank of A equals the number of unknowns. disp(’There is a unique solution, which is:’) x = A\b % Solve using left division.

ENGR-25_Linear_Equations-1.ppt 41 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods MATLAB LinSys Solver Code - 2 else % Rank of A does not equal the number of unknowns. disp(’There is an infinite number of solutions.’) disp(’The augmented matrix of the reduced system is:’) rref([A b]) % Compute the augmented matrix. end else % The ranks of A and [A b] are not equal. disp(’There are no solutions.’) end

ENGR-25_Linear_Equations-1.ppt 42 Bruce Mayer, PE Engineering/Math/Physics 25: Computational Methods MATLAB rank  rank Rank of matrix  Syntax k = rank(A)  Description The rank function provides an estimate of the number of linearly independent rows or columns of a full matrix.