1. 1 資訊科學數學 13 : Solutions of Linear Systems 陳光琦助理教授 (Kuang-Chi Chen) chichen6@mail.tcu.edu.tw

2. 2 Linear Equations and Matrices Solutions of Linear Systems of Equations

3. 3 Solutions of Linear Systems of Equations 1.6 Solutions of Linear Systems of Equations

4. 4 Row Echelon Form Definition – Row echelon form (r.e.f.) Definition – Row echelon form (r.e.f.) An m  n matrix A is said to be in row echelon form if (a) All zero rows, if there are any, appear at the bottom of the matrix (b) The first nonzero entry from the left of a nonzero row is a 1; a leading one of the row (c) For each nonzero row, the leading one appears to the right and below any leading one’s in preceding rows

5. 5 Reduced Row Echelon Form Definition – Reduced row echelon form Definition – Reduced row echelon form An m  n matrix A is said to be in reduced row echelon form if An m  n matrix A is said to be in reduced row echelon form if (a) A is in row echelon form (b) If a column contains a leading one, then all other entries in that column are zero ( 列梯形式 ; 簡化之列梯形式 ) ( 列梯形式 ; 簡化之列梯形式 )

6. 6 Example 1 - in row echelon form E.g. 1 E.g. 1

7. 7 Example 2 – reduced row echelon form E.g. 2 E.g. 2

8. 8 E.g. 2 – not reduced row echelon form E.g. 2 E.g. 2,,, Nonzero element above leading 1 in row 2

9. 9 Three Basic Types of Elementary Row Operations Type 1 – Interchange Type 1 – Interchange row i and row j are interchanged row i and row j are interchanged Type 2 – Multiply Type 2 – Multiply row i = row i times c row i = row i times c Type 3 – Add Type 3 – Add Add d times row r of A to row s of A Add d times row r of A to row s of A row s = row s + d  row r row s = row s + d  row r

10. 10 Example 3 E.g. 3 E.g. 3  E2 E2 E1⇒E1⇒ E3E3

11. 11 Row Equivalence Definition – Row Equivalence Definition – Row Equivalence An m  n matrix A is said to be row equivalence to an m  n matrix B if B can be obtained by applying a finite sequence of elementary row operations to the matrix A. An m  n matrix A is said to be row equivalence to an m  n matrix B if B can be obtained by applying a finite sequence of elementary row operations to the matrix A.

12. 12 Example 4 E.g. 4 E.g. 4 E1E1 E3⇒E3⇒ E2⇒E2⇒

13. 13 Theorem 1.5 Theorem 1.5 Theorem 1.5 Every m  n matrix is row equivalent to a matrix in row echelon form. Every m  n matrix is row equivalent to a matrix in row echelon form.

14. 14 E.g. 5 - Procedure of Row Echelon Form E.g. 5 E.g. 5 Step 1 – Find the pivotal column Step 2 – Identify the pivot in the pivotal column Pivot column Pivot

15. 15 (cont’d) E.g. 5 E.g. 5 Step 3 – Interchange if necessary so that the pivot is in the 1 st row Step 4 – Multiply so that the pivot equals to 1 pivot

16. 16 (cont’d) E.g. 5 E.g. 5 Step 5 – Make all entries in the pivot column, except the entry where the pivot was located, equal to zero

17. 17 (cont’d) E.g. 5 E.g. 5 Step 6 – Ignore the first row and repeat ⇒ ⇒ … … … ⇒ ⇒ … … … ⇒

18. 18 Example 6 Example 6 Example 6

19. 19 Remark Remark Remark - There may be more than one matrix in row echelon form that is row equivalent to a given matrix. - There may be more than one matrix in row echelon form that is row equivalent to a given matrix. - A matrix in row echelon form (r.e.f.) that is row equivalent to A is called - A matrix in row echelon form (r.e.f.) that is row equivalent to A is called “a row echelon form of A”. “a row echelon form of A”.

20. 20 Theorem 1.6 Theorem 1.6 Theorem 1.6 - Every m  n matrix is row equivalent to a unique matrix in reduced row echelon form. - Every m  n matrix is row equivalent to a unique matrix in reduced row echelon form.

21. 21 Example 7 – r.e.f. to reduced r.e.f. E.g. 7 E.g. 7

22. 22 Theorem 1.7 Theorem 1.7 Theorem 1.7 Let Ax = b and Cx = d be two linear systems each of m equations in n unknowns. If the augmented matrices [A|b] and [C|d] of these systems are row equivalent, then both linear systems have the same solutions. Let Ax = b and Cx = d be two linear systems each of m equations in n unknowns. If the augmented matrices [A|b] and [C|d] of these systems are row equivalent, then both linear systems have the same solutions.

23. 23 Corollary 1.1 Corollary 1.1 Corollary 1.1 If A and C are row equivalent m  n matrices, then the linear system Ax = 0 and Cx = 0 have exactly the same solutions. If A and C are row equivalent m  n matrices, then the linear system Ax = 0 and Cx = 0 have exactly the same solutions.

24. 24 Gauss-Jordan Reduction Procedure The Gauss-Jordan reduction procedure The Gauss-Jordan reduction procedure Step 1. Form the augmented matrix [A|b] Step 2. Obtain the reduced row echelon form [C|d] of the augmented matrix [A|b] by using elementary row operations Step 3. For each nonzero row of [C|d], solve the corresponding equation. (augmented matrix 擴增矩陣 ) (augmented matrix 擴增矩陣 )

25. 25 Gauss Elimination Procedure The Gauss elimination procedure The Gauss elimination procedure Step 1. Form the augmented matrix [A|b] Step 2. Obtain a row echelon form [C|d] of the augmented matrix [A|b] by using elementary row operations Step 3. Solving the linear system corresponding to [C|d], by back substitution ( 後代入法 ).

26. 26 Example 8 E.g. 8 E.g. 8 Solve the linear system by Gauss-Jordan reduction Solve the linear system by Gauss-Jordan reduction - Step 1 - Step 1

27. 27 (cont ’ d) E.g. 8 - Solve the linear system by Gauss-Jordan reduction E.g. 8 - Solve the linear system by Gauss-Jordan reduction - Step 2 - Step 2

28. 28 (cont ’ d) E.g. 8 - Solve the linear system by Gauss-Jordan reduction E.g. 8 - Solve the linear system by Gauss-Jordan reduction - Step 3 x = 2 - Step 3 x = 2 y = -1 y = -1 z = 3 z = 3

29. 29 Example 9 Example 9 Example 9 - Solve the linear system by Gauss-Jordan reduction - Solve the linear system by Gauss-Jordan reduction x + y + 2z – 5w = 3 x + y + 2z – 5w = 3 2x + 5y – z – 9w = -3 2x + 5y – z – 9w = -3 2x + y – z + 3w = -11 2x + y – z + 3w = -11 x – 3y + 2z + 7w = -5 x – 3y + 2z + 7w = -5

30. 30 (cont ’ d) Example 9 Example 9 - Step 1 - Step 1 - Step 2 - Step 2

31. 31 (cont ’ d) E.g. 9 - Step 3 E.g. 9 - Step 3 leading variables leading variables a free variable a free variable

32. 32 Example 10 Example 10 Example 10 - Solve the linear system by Gauss-Jordan reduction - Solve the linear system by Gauss-Jordan reduction x 1 + 2x 2 – 3x 4 + x 5 = 2 x 1 + 2x 2 – 3x 4 + x 5 = 2 x 1 + 2x 2 + x 3 – 3x 4 + x 5 + 2x 6 = 3 x 1 + 2x 2 + x 3 – 3x 4 + x 5 + 2x 6 = 3 x 1 + 2x 2 – 3x 4 + 2x 5 + x 6 = 4 x 1 + 2x 2 – 3x 4 + 2x 5 + x 6 = 4 3x 1 + 6x 2 + x 3 – 9x 4 + 4x 5 + 3x 6 = 9 3x 1 + 6x 2 + x 3 – 9x 4 + 4x 5 + 3x 6 = 9

33. 33 (cont ’ d) Example 10 Example 10 - Step 1 - Step 1 - Step 2 - Step 2

34. 34 (cont ’ d) Example 10 - Step 3 Example 10 - Step 3 leading variables leading variables free variables free variables

35. 35 Example 11 Example 11 Example 11 - Solve the linear system by Gauss elimination - Solve the linear system by Gauss elimination x + 2y + 3z = 9 x + 2y + 3z = 9 2x – y + z = 8 2x – y + z = 8 3x – z = 3 3x – z = 3

36. 36 (cont ’ d) Example 11 Example 11 - Step 1 - Step 1 - Step 2 - Step 2

37. 37 (cont ’ d) Example 11 - Step 3 Example 11 - Step 3 - By back substitution - By back substitution

38. 38 Example 12 Example 12 Example 12 - Solve the linear system by Gauss elimination - Solve the linear system by Gauss elimination x + 2y + 3z + 4w = 5 x + 2y + 3z + 4w = 5 x + 3y + 5z + 7w = 11 x + 3y + 5z + 7w = 11 x – z – w = -6 x – z – w = -6

39. 39 (cont ’ d) Example 12 Example 12 - Step 1 - Step 1 - Step 2 - Step 2 - Step 3 ⇒ 0x + 0y + 0z + 0w = 1 ⇒ No solutions !! - Step 3 ⇒ 0x + 0y + 0z + 0w = 1 ⇒ No solutions !!

40. 40 Consistent and Inconsistent Consistent and inconsistent Consistent and inconsistent - Consistent: Linear systems with at least one solution - Consistent: Linear systems with at least one solution - Inconsistent: Linear systems with no solutions - Inconsistent: Linear systems with no solutions

41. 41 Homogeneous Systems A system of linear equations is said to be homogeneous if all the constant terms are zeros. A system of linear equations is said to be homogeneous if all the constant terms are zeros. a 11 x 1 + a 12 x 2 + … + a 1n x n = 0 a 11 x 1 + a 12 x 2 + … + a 1n x n = 0 a 21 x 1 + a 22 x 2 + … + a 2n x n = 0 a 21 x 1 + a 22 x 2 + … + a 2n x n = 0 … a m1 x 1 + a m2 x 2 + … + a mn x n = 0 a m1 x 1 + a m2 x 2 + … + a mn x n = 0 ⇒ Ax = 0 ⇒ Ax = 0 Thus, a homogeneous system always has the solution x 1 = x 2 = … = x n = 0 → the trivial solution Thus, a homogeneous system always has the solution x 1 = x 2 = … = x n = 0 → the trivial solution

42. 42 Example 13 Example 13 Example 13

43. 43 Example 14 Example 14 Example 14

44. 44 Theorem 1.8 Theorem 1.8 Theorem 1.8 A homogeneous system of m equations in n unknowns has a non-trivial solution if m < n, that is, if the number of unknowns exceeds the number of equations. A homogeneous system of m equations in n unknowns has a non-trivial solution if m < n, that is, if the number of unknowns exceeds the number of equations. namely, a homogeneous system has more variables than equations has many solutions. namely, a homogeneous system has more variables than equations has many solutions. (a homogeneous system 齊次系統 ; non-trivial solution 非零解 ) (a homogeneous system 齊次系統 ; non-trivial solution 非零解 )

45. 45 Example 15 - A Homogeneous System E.g. 15 E.g. 15

46. 46 (cont ’ d) If let If let

47. 47 A Homogeneous System Example x = x p + x h x = x p + x h x p is a particular solution to the given system x p is a particular solution to the given system Ax p = b, where b = [3 -3 -11 -5] T Ax p = b, where b = [3 -3 -11 -5] T x h is a solution to the associated x h is a solution to the associated homogeneous system Ax h = 0. homogeneous system Ax h = 0.

48. 48 Polynomial Interpolation Polynomial Interpolation Polynomial Interpolation - The general form - The general form y = a n – 1 x n – 1 + a n – 2 x n – 2 + … + a 1 x + a 0 y = a n – 1 x n – 1 + a n – 2 x n – 2 + … + a 1 x + a 0 E.g. n = 3, y = a 2 x 2 + a 1 x + a 0 Given three distinct points (x 1, y 1 ), (x 2, y 2 ), (x 3, y 3 ), we have y 1 = a 2 x 1 2 + a 1 x 1 + a 0 y 1 = a 2 x 1 2 + a 1 x 1 + a 0 y 2 = a 2 x 2 2 + a 1 x 2 + a 0 y 2 = a 2 x 2 2 + a 1 x 2 + a 0 y 3 = a 2 x 3 2 + a 1 x 3 + a 0 y 3 = a 2 x 3 2 + a 1 x 3 + a 0

49. 49 Example 16 Example 16 - Find the quadratic polynomial that interpolates the points (1, 3), (2, 4), (3, 7) Example 16 - Find the quadratic polynomial that interpolates the points (1, 3), (2, 4), (3, 7)

50. 50 Example 17 – Temperature Distribution T 1 = (260 – 100 + T 2 + T 3 )/4 or 4T 1 – T 2 – T 3 = 160 T 1 = (260 – 100 + T 2 + T 3 )/4 or 4T 1 – T 2 – T 3 = 160 T 2 = (T 1 + 100 + 40 + T 4 )/4 or -T 1 + 4T 2 – T 4 = 140 T 2 = (T 1 + 100 + 40 + T 4 )/4 or -T 1 + 4T 2 – T 4 = 140 T 3 = (60 + T 1 + T 4 + 0)/4 or -T 1 + 4T 3 – T 4 = 60 T 3 = (60 + T 1 + T 4 + 0)/4 or -T 1 + 4T 3 – T 4 = 60 T 4 = (T 2 + T 3 + 40 + 0)/4 or -T 2 – T 3 + 4T 4 = 40 T 4 = (T 2 + T 3 + 40 + 0)/4 or -T 2 – T 3 + 4T 4 = 40 ⇒ ⇒ T 1 = 65, T 2 = 60, T 3 = 40, T 4 = 35. ⇒ T 1 = 65, T 2 = 60, T 3 = 40, T 4 = 35.

51. 51 Linear Equations and Matrices The Inverse of A Matrix

52. 52 The Inverse of A Matrix 1.7 The inverse of a matrix 1.7 The inverse of a matrixDefinition - An n  n matrix A is called nonsingular (or invertible 可逆的 ) if there exists an n  n matrix B such that AB = BA = I n. - An n  n matrix A is called nonsingular (or invertible 可逆的 ) if there exists an n  n matrix B such that AB = BA = I n. - The matrix B is called the inverse of A - The matrix B is called the inverse of A - If there exists no such matrix B, then A is called singular (or noninvertible) - If there exists no such matrix B, then A is called singular (or noninvertible) - A is also an inverse of B - A is also an inverse of B

53. 53 Example 1 Example 1 Example 1 ⇒ AB = BA = I 2 - B is an inverse of A and A is nonsingular. - B is an inverse of A and A is nonsingular.

54. 54 Theorem 1.9 Theorem 1.9 Theorem 1.9 An inverse of a matrix, if exists, is unique. (proof) Let B and C be inverses of A. Let B and C be inverses of A. Then AB = BA = I n, and AC = CA = I n. Then AB = BA = I n, and AC = CA = I n. Thus, C(AB) = CI n Thus, C(AB) = CI n (CA)B = C (CA)B = C I n B = C, i.e., B = C. I n B = C, i.e., B = C.

55. 55 Example 2 - Find the Inverse For the matrix A, find the inverse If exists, let the inverse A -1 be such that

56. 56 (cont ’ d) and and

57. 57 Example 3 Example 3 Example 3 ⇒ No solution; singular and and

58. 58 Theorem 1.10 Thm. 1.10 - Properties of an inverse Thm. 1.10 - Properties of an inverse - If A is nonsingular, then A -1 is nonsingular - If A is nonsingular, then A -1 is nonsingular and (A -1 ) -1 = A ; and (A -1 ) -1 = A ; - If A and B are nonsingular matrices, then AB is nonsingular and (AB) -1 = B -1 A -1 ; - If A and B are nonsingular matrices, then AB is nonsingular and (AB) -1 = B -1 A -1 ; - If A is a nonsingular matrix, then (A T ) -1 = (A -1 ) T. - If A is a nonsingular matrix, then (A T ) -1 = (A -1 ) T.

59. 59 Example 4 Example 4 Example 4 ⇒ and ⇒ and

60. 60 Corollary1.2 Corollary 1.2 Corollary 1.2 - If A 1, A 2, …, A r are n  n nonsingular matrices, then (A 1 A 2 … A r ) is nonsingular and (A 1 A 2 … A r ) -1 = A r -1 … A 2 -1 A 1 -1. - If A 1, A 2, …, A r are n  n nonsingular matrices, then (A 1 A 2 … A r ) is nonsingular and (A 1 A 2 … A r ) -1 = A r -1 … A 2 -1 A 1 -1.

61. 61 Theorem 1.11 Theorem 1.11 Theorem 1.11 Suppose that A, B are n  n matrices, Suppose that A, B are n  n matrices, - If AB = I n, then BA = I n ; - If AB = I n, then BA = I n ; - If BA = I n, then AB = I n. - If BA = I n, then AB = I n.

62. 62 The Way to Find A -1 A practical method for finding A -1 A practical method for finding A -1 Step 1. Form the 2  2n matrix [A | I n ] obtained by adjoining the identity matrix I n to the given matrix A Step 2. Compute the reduced row echelon form of the matrix obtained in Step 1 by using elementary row operations Step 3. Suppose that Step 2 has produced the matrix [C | D] in reduced row echelon form: If C = I n, then D = A -1 ; If C = I n, then D = A -1 ; If C ≠ I n, then C has a row of zeros and the matrix A is singular. If C ≠ I n, then C has a row of zeros and the matrix A is singular.

63. 63 Example 5 – Find the Inverse E.g. 5 – Find the inverse E.g. 5 – Find the inverse

64. 64 Example 6 – Find the Inverse E.g. 6 - Find the inverse E.g. 6 - Find the inverse The left-half matrix cannot have a one in the (3, 3) location, the reduced echelon form cannot be I 3. Thus A -1 does not exist.

65. 65 Theorem 1.12 & 1.13 Theorem 1.12 Theorem 1.12 An n  n matrix is nonsingular iff it is row equivalence to I n. An n  n matrix is nonsingular iff it is row equivalence to I n. Theorem 1.13 Theorem 1.13 If A is an n  n matrix, the homogeneous system Ax = 0 has a nontrivial solution iff If A is an n  n matrix, the homogeneous system Ax = 0 has a nontrivial solution iff A is singular. A is singular.

66. 66 Proof of Theorem 1.13 Proof of Theorem 1.13 Proof of Theorem 1.13 Suppose that A is nonsingular, then A -1 exists and Suppose that A is nonsingular, then A -1 exists and A -1 (Ax) = A -1 0 A -1 (Ax) = A -1 0 (A -1 A)x = 0 (A -1 A)x = 0 I n x = 0 I n x = 0 x = 0 ⇒ Ax = 0 has a trivial solution x = 0 ⇒ Ax = 0 has a trivial solution (contradiction to a non-trivial solution, hence A must be singular) (contradiction to a non-trivial solution, hence A must be singular)

67. 67 Example 8 Example 8 Example 8 Consider the homogeneous system Ax = 0, where A is the matrix (A is nonsingular) Consider the homogeneous system Ax = 0, where A is the matrix (A is nonsingular) Gauss-Jordan reduction The trivial solution x = 0

68. 68 Example 9 Example 9 Example 9 - Consider the homogeneous system Ax = 0, where A is the matrix (A is singular) - Consider the homogeneous system Ax = 0, where A is the matrix (A is singular)

69. 69 Theorem 1.14 Theorem 1.14 Theorem 1.14 If A is an n  n matrix, then A is nonsingular iff the linear system Ax = b has a unique solution for every n  1 matrix b. If A is an n  n matrix, then A is nonsingular iff the linear system Ax = b has a unique solution for every n  1 matrix b.

70. 70 Summary The Symmetry, Singularity, Inverse of A Matrix

71. 71 Some Special Matrix 4. A square matrix A is said to be antisymmetric if -A T = A. (i) If A is square, prove that A + A T is symmetric and A – A T is antisymmetric ; (ii) any square matrix A can be decomposed into the sum of a symmetric matrix B and an antisymmetric matrix C: A = B + C. (ii) any square matrix A can be decomposed into the sum of a symmetric matrix B and an antisymmetric matrix C: A = B + C. 5. G iven two symmetric matrices of the same size, A and B, then a necessary and sufficient condition for the product AB to be symmetric is that AB = BA.

72. 72 Some Special Matrix 1. A square matrix A is said to be normal if AA T = A T A. All symmetric matrices are normal ; 2. A square matrix A is said to be idempotent if A 2 = A. If A is idempotent then A T is also idempotent ; 3. A square matrix A is said to nilpotent if there is a positive integer p such that A p = O. The least integer such that A p = O is called the degree of nilpotency of the matrix. If A is nilpotent, then A T is also nilpotent with the same degree of nilpotency.

73. 73 List of Nonsingular Equivalences Nonsingular equivalences Nonsingular equivalences 1. A is nonsingular ; 2. x = 0 is the only solution to Ax = 0 ; 3. A is row equivalence to I n ; 4.The linear system Ax = b has a unique solution for every n  1matrix b.

74. 74 Properties of Matrix Inverse Properties of Matrix Inverse Properties of Matrix Inverse 1. (A -1 ) -1 = A ; 2. (cA) -1 = (1/c)A -1, where c is a nonzero scalar; 3. (AB) -1 = B -1 A -1 ; 4. (A n ) -1 = (A -1 ) n ; 5. (A T ) -1 = (A -1 ) T, where T : transpose.

75. 75 Conditions of Matrix Inverse A matrix has no inverse, if A matrix has no inverse, if (i) two rows are equal; (i) two rows are equal; (ii) two columns are equal; (Use the transpose) (ii) two columns are equal; (Use the transpose) (iii) it has a column of zeros. (iii) it has a column of zeros.

76. 76 The Inverse of 2  2 Matrix If A =, show that A -1 =. If A =, show that A -1 =. Note: The cancellation law doesn’t hold. That is, AB = AC doesn’t imply that B = C. Also, AB = O doesn’t imply that A = O or B = O. However, if A is an invertible matrix, then if AB = AC, then B = C ; if AB = AC, then B = C ; if AB = 0, then B = 0. if AB = 0, then B = 0.