ME 1202: Linear Algebra & Ordinary Differential Equations (ODEs) Dr. Faraz Junejo

Row-echelon form: (1, 2, 3) A matrix is said to be in row echelon form if it has the following three properties. (1) All row consisting entirely of zeros occur at the bottom of the matrix. (2) All entries in a column below a leading entry i.e. 1 are zero.. (3) For two successive nonzero rows, the leading 1 in the higher row is farther to the left than the leading 1 in the lower row. Reduced row-echelon form: (1, 2, 3, 4) If a matrix is in echelon form and satisfy following additional condition, then it is in reduced echelon from. (4) Every column that contains a leading 1 has zeros everywhere else.

Echelon & Reduced Echelon Forms: Notes  A matrix may be row reduced (i.e. transformed by elementary row operations) into more than one matrix in echelon form, using different sequence of row operations. In summary, echelon form obtained from a given matrix may not necessarily be unique.  However, the reduced echelon from one obtains from a matrix is unique.  A matrix in reduced row-echelon form is of necessity in row-echelon form, but not conversely.

Example 1 Row-Echelon & Reduced Row-Echelon form Reduced row-echelon form: Row-echelon form:

Example 2 More on Row-Echelon and Reduced Row- Echelon form All matrices of the following types are in row-echelon form ( any real numbers substituted for the * ’ s. ) : All matrices of the following types are in reduced row-echelon form ( any real numbers substituted for the * ’ s. ) :

Exercise(Row-echelon form or reduced row-echelon form)

Example 3 (a) Solutions of Linear Systems Solution (a) the corresponding system of equations is : Suppose that the augmented matrix for a system of linear equations have been reduced by row operations to the given reduced row-echelon form. Solve the system.

Example 3 (b) Solutions of Linear Systems Solution (b) 1. The corresponding system of equations is : leading variables free variables

Example 3 (b) Solutions of Linear Systems 2. We see that the free variable can be assigned an arbitrary value, say t, which then determines values of the leading variables. 3. There are infinitely many solutions, and the general solution is given by the formulas

Example 3 (c) Solutions of Linear Systems Solution (c) 1.The 4th row of zeros leads to the equation places no restrictions on the solutions (why?). Thus, we can omit this equation.

Example 3 (c) Solutions of Linear Systems Solution (c) 2.Solving for the leading variables (x 1,x 3,x 4 ) in terms of the free variables (x 2,x 5 ) : 3. The free variable can be assigned an arbitrary value, there are infinitely many solutions, and the general solution is given by the formulas.

Example 3 (d) Solutions of Linear Systems Solution (d): the last equation in the corresponding system of equation is Since this equation cannot be satisfied, there is no solution to the system.

Elimination Methods (1/6) There should be a step-by-step elimination procedure that can be used to reduce any matrix to reduced row- echelon form.

Elimination Methods (2/6) Step1. Locate the leftmost pivot column i.e. a coulmn that does not consist entirely of zeros. Step2. Interchange the top row with another row, to bring a nonzero entry (Pivot) to top of the column found in Step1. Leftmost pivot or nonzero column The 1 st and 2 nd rows in the preceding matrix were interchanged. Pivot

Elimination Methods (3/6) Step3. If the entry that is now at the top of the column found in Step1 is a, multiply the first row by 1/a in order to introduce a leading 1. Step4. Add suitable multiples of the top row to the rows below so that all entries below the leading 1 become zeros. The 1st row of the preceding matrix was multiplied by 1/ times the 1st row of the preceding matrix was added to the 3rd row.

Elimination Methods (4/6) Step5. Now cover the top row in the matrix and begin again with Step1 applied to the sub-matrix that remains. Continue in this way until the entire matrix is in row-echelon form. The 1st row in the sub- matrix was multiplied by -1/2 to introduce a leading 1. Leftmost nonzero or pivot column in the sub-matrix Pivot

Elimination Methods (5/6) Step5 (cont.) -5 times the 1st row of the sub- matrix was added to the 2nd row of the sub-matrix to introduce a zero below the leading 1. The top row in the sub-matrix was covered, and we returned again Step1. The first (and only) row in the new submetrix was multiplied by 2 to introduce a leading 1. Leftmost nonzero or pivot column in the new sub-matrix The entire matrix is now in row-echelon form. Pivot

Elimination Methods (6/6) Step6. Beginning with last nonzero row and working upward, add suitable multiples of each row to the rows above to introduce zeros above the leading 1 ’ s. 7/2 times the 3rd row of the preceding matrix was added to the 2nd row. -6 times the 3rd row was added to the 1st row. The last matrix is in reduced row-echelon form. 5 times the 2nd row was added to the 1st row.

The first nonzero column Produce leading 1 Zeros elements below leading 1 leading 1 Produce leading 1 The first nonzero column Summary: Elimination Methods Example Submatrix

Zeros elements below leading 1 Zeros elsewhere leading 1 Produce leading 1 Example (Contd.) Submatrix

Summary: Elimination Methods Step1~Step5: the above procedure produces a row-echelon form and is called Gaussian elimination. Step1~Step6: the above procedure produces a reduced row-echelon form and is called Gaussian-Jordan elimination. Two matrices are said to be row equivalent if one matrix can be obtained from the other using elementary row operations Every matrix has a unique reduced row-echelon form but a row-echelon form of a given matrix is not unique.

Gaussian elimination and back substitution Method To solve a system of linear equations, we first simplify the system through succession of elementary row operations, until we arrive at an augmented matrix in row-echelon form. This process is called the gaussian elimination. and we may solve the simplest equation. Then we successively substitute the results into more and more complicated equations to get the whole solution. This process is called the backward substitution.

Example 1 Solve the system 3x 1 + x 2 - x 3 = 2 x 1 - x 2 + x 3 = 2 2x 1 + 2x 2 + x 3 = 6 Note that the variable x 1 appears in all 3 equations. We try to eliminate x 1 so that it appears in only 1 equation. Specifically, by multiplying -3 to the 2nd equation and then adding to the 1st equation, x 1 is eliminated from the 1st equation. 4x 2 - 4x 3 = -4 x 1 - x 2 + x 3 = 2 2x 1 + 2x 2 + x 3 = 6

Next the operation (-2)[equation 2] + [equation 3] (i.e. multiply -2 to the 2nd equation and then add to the 3rd equation) further eliminates x 1 from the 3rd equation. 4x 2 - 4x 3 = -4 x 1 - x 2 + x 3 = 2 4x 2 - x 3 = 2 Example 1 (contd.) The system is as simple as we can get as far as x 1 is concerned. Next we try to simplify the system regarding x 2. The operation (-1)[equation 1] + [equation 3] eliminates x 2 from the 3rd equation. 4x 2 - 4x 3 = -4 x 1 - x 2 + x 3 = 2 3x 3 = 6

Example 1 (contd.)  By exchanging the 1st and the 2nd equations (denoted [equation 1] ↔ [equation 2]), we rearrange the equations from the most complicated down to the simplest, i.e. x 1 - x 2 + x 3 = 2 4x 2 - 4x 3 = -4 3x 3 = 6  Finally, the coefficients can be further simplified by multiplying 1/4 and 1/3 to the 2nd and the 3rd equations (denoted (1/4)[equation 2] and (1/3)[equation 3]). x 1 - x 2 + x 3 = 2 x 2 - x 3 = -1 OR x 3 = 2 This completes the simplification process. Augmented Matrix in Row-Echelon form

Example 1 (contd.)  Now we start solving the system by backward substitution. From the 3rd equation (the simplest equation), we have x 3 = 2  Substituting this into the 2nd equation (the next simplest), we get x 2 = -1 + x 3 = => x 2 = 1.  Further substituting into the 1st equation (the most complicated), we get x 1 = 2 + x 2 - x 3 = => x 1 = 1. We conclude that the system has a unique solution x 1 = 1, x 2 = 1, x 3 = 2.

Example 1 (contd.) In the example 1, we have used the elementary row operations in following order to simplify given system.  A constant multiple of one row is added to another row  Two rows are interchanged  A row is multiplied by a nonzero constant In general, any system of linear equations can be simplified, by these three operations, to a simple system in which equations are arranged from the most complicated to the simplest. This process is called the Gaussian elimination method. After completing the gaussian elimination, we may solve the simplest equation. Then we successively substitute the results into more and more complicated equations to get the whole solution. This process is called the backward substitution.

Example 2 Solve by Gaussian elimination and back-substitution. Solution – We convert the augmented matrix – to the row-echelon form – The system corresponding to this matrix is

Example 2 Solution – Solving for the leading variables – Substituting the bottom equation into those above – Substituting the 2nd equation into the top

Exercise: 1 Let us consider the set of linearly independent equations. Augmented matrix for the set is:

Exercise: 1 (Contd.) Step 1: Eliminate x from the 2nd and 3rd equation.

Exercise: 1 (Contd.) Step 2: Eliminate y from the 3rd equation. 13R’ 2 +R’ 3 R’’ 3 Step 3: 0.5R’ 1 R’ 1 -R’ 2 R’’ 2 (1/168)R’’ 3 R’’’ 3

Exercise: 1 (Contd.) From Row 3, z = 4 From Row 2, y -14.5z = -61 or, y (4) = 61 or, y = - 3 From Row 1, x – 2y + 2.5z = 18 or, x – 2 (- 3) (4) = 18 or, x = 2

Exercise 2 Solve the linear system using Gaussian elimination and back substitution method 9x 1 + 3x 2 + 4x 3 = 7 4x 1 + 3x 2 + 4x 3 = 8 x 1 + x 2 + x 3 = 3 Answer: x 1 = - 1/5 x 2 = 4 x 3 = - 4/5

Exercise 3 Solve the linear system using Gaussian elimination and back substitution method 3x 1 + x 2 - x 3 = 2 x 1 - x 2 + x 3 = 2 2x 1 - x 2 + x 3 = 6  This is the simplest shape (roe echelon form) we can get by row operations. The corresponding system is x 1 - x 2 + x 3 = 2 x 2 - x 3 = -1 0 = 3  Since the 3rd equation 0 = 3 is a contradiction implying the original system has no solution.

Exercise 4 A garden supply centre buys flower seed in bulk then mixes and packages the seeds for home garden use. The supply center provides 3 different mixes of flower seeds: “Wild Thing”, “Mommy Dearest” and “Medicine Chest”. 1) Wild Thing seed mix contains 50% of wild flower seed, 25% of Echinacea seed and 25% of Chrysanthemum seed. 2) Mommy Dearest mix consists 75% Chrysanthemum seed and 25% wild flower seed. 3) The Medicine Chest mix contains 90% Echinacea seed, 10% other seeds. In a single order, the store received 17 grams of wild flower seed, 15 grams of Echinacea seed and 21 grams of Chrysanthemum seed. Use matrices and complete Gauss-Jordan Elimination to determine how much of each mixture the store can prepare. Assume that the garden center has an ample supply of other seeds used in Medicine chest on hand.

Exercise: 4 (contd.) Solution: – Assign variables to the amount of each mix that will be produced. – Perform a balance on each of the components that are available. Let X = Amount of Wild Thing Let Y = Amount of Mommy Dearest Let Z = Amount of Medicine Chest Wild flower0.5X Y + 0Z = 17g Echinacea0.25X + 0Y + 0.9Z = 15g Chrysanthemum 0.25X Y + 0Z = 21g In matrix form, this can be written as

Exercise: 4 (contd.) Before the matrices are populated, it is (sometimes) helpful to re-arrange the equations to reduce the number of steps in the Gauss Elimination. To do this (if there seems like an easy solution), attempt to move zeros to the bottom left, and try to maintain the first row with non-zeros except for the last entry, since row 1 is used to reduce other rows. By moving the last column (Z) to the front, and switching the first and second row, the new set of equations becomes: Echinacea0.9Z X + 0Y = 15g Wild flower0Z + 0.5X Y = 17g Chrysanthemum0Z X Y = 21g

Exercise: 4 (contd.) – Apply the Gauss Elimination: Z = 10, X = 24, and Y = 20

Summary: Gauss Elimination for Solving A System of Equations 1.Write the augmented matrix of the system. 2. Use elementary row operations to construct a row equivalent matrix in row-echelon form. 3. Write the system of equations corresponding to the matrix in row-echelon form. 4. Use back-substitution to find the solutions to this system.

Gauss-Jordan Elimination In Gauss-Jordan elimination, we continue the reduction of the augmented matrix until we get a row equivalent matrix in reduced row- echelon form. (r-e form where every column with a leading 1 has rest zeros)

Example: 1 Let us consider the set of linearly independent equations. Augmented matrix for the set is:

Example: 1 (Contd.) Step 1: Eliminate x from the 2nd and 3rd equation.

Example: 1 (Contd.) Step 2: Eliminate y from the 3rd equation. 13R’ 2 +R’ 3 R’’ 3 Step 3: 0.5R’ 1 R’ 1 -R’ 2 R’’ 2 (1/168)R’’ 3 R’’’ 3

Example: 1 (Contd.) Step 4: Eliminate z from the 2 nd equation

Example: 1 (Contd.) Step 5-1: Eliminate y from the 1st equation

Example: 1 (Contd.) Step 5-2: Eliminate z from the 1st equation

Example 2: Gauss-Jordan Elimination Solve by Gauss-Jordan Elimination Solution: The augmented matrix for the system is

Example 2(Contd.) Adding -2 times the 1st row to the 2nd and 4th rows gives Multiplying the 2nd row by -1 and then adding -5 times the new 2nd row to the 3rd row and - 4 times the new 2nd row to the 4th row gives

Example 2 (contd.) Interchanging the 3rd and 4th rows and then multiplying the 3rd row of the resulting matrix by 1/6 gives the row-echelon form. Adding -3 times the 3rd row to the 2nd row and then adding 2 times the 2nd row of the resulting matrix to the 1st row yields the reduced row-echelon form.

Example 2 (contd.) The corresponding system of equations is Solution The augmented matrix for the system is We assign the free variables, and the general solution is given by the formulas:

Exercise: 1 Answer

Exercise 2: Solve a system by Gauss-Jordan elimination method (only one solution) Sol:

Exercise 3 ： Solve a system by Gauss-Jordan elimination method (infinitely many solutions) Sol:

Exercise: 3 (Contd.) Let So this system has infinitely many solutions.

Homogeneous systems of linear equations A system of linear equations is said to be homogeneous if all the constant terms are zero. A homogenous system of linear equations is always consistent, since x 1 = 0,x 2 = 0,….x n = 0 will satisfy each equation in the system.

Trivial solution Nontrivial solution: other solutions Notes: (1) Every homogeneous system of linear equations is consistent. (2) If the homogenous system has fewer equations than variables, then it must have an infinite number of solutions. (3) For a homogeneous system, exactly one of the following is true. (a) The system has only the trivial solution. OR (b) The system has infinitely many nontrivial solutions in addition to the trivial solution. Homogeneous systems of linear equations (Contd.)

Solve the following homogeneous system Let Sol:

Example 4 Solve the following homogeneous system of linear equations by using Gauss-Jordan elimination. Solution The augmented matrix Reducing this matrix to reduced row- echelon form

Example 4 (contd.) Solution (cont) The corresponding system of equation Solving for the leading variables is Thus the general solution is Note: the trivial solution is obtained when s= t = 0.

Example 4 (Contd.) None of the three row operations alters the final column of zeros, so the system of equations corresponding to the reduced row-echelon form of the augmented matrix must also be a homogeneous system.

Application of Linear Systems  A mathematical model is an equation or system of equations that represent a real life situation.  One main reason to solve systems of linear equations is to solve real world problems.

Applications of Systems To solve problems using a system 1.Determine the unknown quantities 2.Let different variables represent those quantities 3.Write a system of equations – one for each variable Example : In a recent year, the national average spent on two athletes, one female and one male, was $6050 for Division I-A schools. However, average expenditures for a male athlete exceeded those for a female athlete by $3900. Determine how much was spent per athlete for each gender.

Applications of Systems Solution Let x = average expenditures per male y = average expenditures per female Average spent on one male and one female Average Expenditure per male: $8000, and per female: from (2) y = 8000 – 3900 = $4100.

Exercise: 2 A man walks at a rate of 3 miles per hour and jogs at a rate of 5 miles per hour. He walks and jogs a total distance of 3.5 miles in 0.9 hours. How long does the man jog? Solution: Let x represent the amount of time spent walking and y represent the amount of time spent jogging. Since the total time spent walking and jogging is 0.9 hours, we have the equation x + y = 0.9. We are given the total distance traveled as 3.5 miles. Since Distance = Rate x Time, distance walking = 3x and distance jogging = 5y. Then total distance is represented by 3x + 5y = 3.5.

Exercise: 2 (contd.) We can solve the system using substitution. 1. Solve the first equation for y 2. Substitute this expression into the second equation. 3. Solve second equation for x 4. Find the y value by substituting this x value back into the first equation. 5. Answer the question: Time spent jogging is 0.4 hours. Solution:

Bob and Mary went to the movies. Bob purchased 3 medium bags of popcorns and 2 large drinks and paid $ Mary purchased 2 medium bags of popcorns and 4 large drinks and ended up paying $ What was the price of a medium bag of popcorn and what was the price of a large drink? Denote p the price of a medium bag of popcorn and d the price of a large drink. Then we have 3p + 2d = 11 2p + 4d = 12

3p + 2d = 11 2p + 4d = 12 -6p - 4d = -22 2p + 4d = 12 -4p = -10 p = 2.50 Substituting back and solving for d we get d = (multiply the first by -2) (add the first to the second) Medium bag of popcorn costs $2.50 and a large drink costs $1.75.

Exercise: 4 A restaurant serves two types of fish dinners- small for $5.99 each and large for $8.99. One day, there were 134 total orders of fish, and the total receipts for these 134 orders was $ How many small dinners and how many large dinners were ordered? Answer: 60 small orders and 74 large orders

Application of Homogenous equation in Chemistry Balance the chemical equation Solution: We would like to determine x 1,x 2,x 3 and x 4 so that  Now for a balance equation, the number of atoms of each element must be same on each side of above equation, so we have Carbon (C): 2x 1 = x 3 Hydrogen (H): 6x 1 = 2x 4 OR Oxygen (O): 2x 2 = 2x 3 + x 4

Exercise: 5 (Contd.) 1. Augmented matrix can be written as 2. Now performing row operations to obtain reduced row-echelon form

3. The corresponding system of equations is : leading variables free variable x 4 Exercise: 5 (Contd.) 2. Reduced row-echelon matrix :

Exercise: 5 (Contd.) 4. We see that the free variable (x 4 ) can be assigned an arbitrary value, say t, which then determines values of the leading variables. 5. There are infinitely many solutions, and the general solution is given by the formulas. Note that in this case t must be a positive integer chosen in such a manner that x 1,x 2 and x 3 are positive integers, say t = 6 gives x 1 = 2, x 2 = 7, x 3 = 4, x 4 = 6. The balanced chemical equation is:

Gaussian elimination: The procedure for reducing a matrix to a row- echelon form. Gauss-Jordan elimination: The procedure for reducing a matrix to a reduced row- echelon form. Notes (1) Every matrix has an unique reduced row echelon form. (2) A row-echelon form of a given matrix is not unique. (Different sequences of row operations can produce different row-echelon forms.) Summary

Summary (Contd.) A system of linear equations is said to be homogeneous if all the constant terms are zero. Every homogeneous system of linear equations is consistent A homogeneous system of linear equations with more unknowns than equations has infinitely many solutions