資訊科學數學13 : Solutions of Linear Systems 陳光琦助理教授 (Kuang-Chi Chen) chichen6@mail.tcu.edu.tw

Linear Equations and Matrices Solutions of Linear Systems of Equations

Solutions of Linear Systems of Equations

Row Echelon Form Definition – Row echelon form (r.e.f.) An mn matrix A is said to be in row echelon form if (a) All zero rows, if there are any, appear at the bottom of the matrix (b) The first nonzero entry from the left of a nonzero row is a 1; a leading one of the row (c) For each nonzero row, the leading one appears to the right and below any leading one’s in preceding rows Echelon 1.【軍事】梯隊，梯陣；梯列。4.【物理學】階層光柵。3.特勤部隊。

Reduced Row Echelon Form Definition – Reduced row echelon form An mn matrix A is said to be in reduced row echelon form if (a) A is in row echelon form (b) If a column contains a leading one, then all other entries in that column are zero (列梯形式; 簡化之列梯形式)

Example 1 - in row echelon form E.g. 1

Example 2 – reduced row echelon form E.g. 2

E.g. 2 – not reduced row echelon form , , , Nonzero element above leading 1 in row 2

Three Basic Types of Elementary Row Operations Type 1 – Interchange row i and row j are interchanged Type 2 – Multiply row i = row i times c Type 3 – Add Add d times row r of A to row s of A row s = row s + d  row r

Example 3 E.g. 3 E1 ⇒  E2 E3

Row Equivalence Definition – Row Equivalence An mn matrix A is said to be row equivalence to an mn matrix B if B can be obtained by applying a finite sequence of elementary row operations to the matrix A .

Example 4 E.g. 4 E3 ⇒ E1 E2 ⇒

Theorem 1.5 Theorem 1.5 Every mn matrix is row equivalent to a matrix in row echelon form .

E.g. 5 - Procedure of Row Echelon Form Step 1 – Find the pivotal column Step 2 – Identify the pivot in the pivotal column Pivot Pivot column

Step 3 – Interchange if necessary so that the pivot is in the 1st row (cont’d) E.g. 5 Step 3 – Interchange if necessary so that the pivot is in the 1st row Step 4 – Multiply so that the pivot equals to 1 pivot

(cont’d) E.g. 5 Step 5 – Make all entries in the pivot column, except the entry where the pivot was located, equal to zero

Step 6 – Ignore the first row and repeat (cont’d) E.g. 5 Step 6 – Ignore the first row and repeat ⇒ ⇒ … … … ⇒

Example 6 Example 6

Remark Remark - There may be more than one matrix in row echelon form that is row equivalent to a given matrix. - A matrix in row echelon form (r.e.f.) that is row equivalent to A is called “a row echelon form of A”.

Theorem 1.6 Theorem 1.6 - Every mn matrix is row equivalent to a unique matrix in reduced row echelon form.

Example 7 – r.e.f. to reduced r.e.f. E.g. 7 Row Echelon Form (r.e.f.)

Theorem 1.7 Theorem 1.7 Let Ax = b and Cx = d be two linear systems each of m equations in n unknowns. If the augmented matrices [A|b] and [C|d] of these systems are row equivalent, then both linear systems have the same solutions.

Corollary 1.1 Corollary 1.1 If A and C are row equivalent mn matrices, then the linear system Ax = 0 and Cx = 0 have exactly the same solutions.

Gauss-Jordan Reduction Procedure The Gauss-Jordan reduction procedure Step 1. Form the augmented matrix [A|b] Step 2. Obtain the reduced row echelon form [C|d] of the augmented matrix [A|b] by using elementary row operations Step 3. For each nonzero row of [C|d], solve the corresponding equation. (augmented matrix 擴增矩陣)

Gauss Elimination Procedure The Gauss elimination procedure Step 1. Form the augmented matrix [A|b] Step 2. Obtain a row echelon form [C|d] of the augmented matrix [A|b] by using elementary row operations Step 3. Solving the linear system corresponding to [C|d], by back substitution (後代入法).

Example 8 E.g. 8 Solve the linear system by Gauss-Jordan reduction - Step 1

- Step 2 E.g. 8 - Solve the linear system by Gauss-Jordan reduction (cont’d) E.g. 8 - Solve the linear system by Gauss-Jordan reduction - Step 2

(cont’d) E.g. 8 - Solve the linear system by Gauss-Jordan reduction - Step 3 x = 2 y = -1 z = 3

Example 9 Example 9 - Solve the linear system by Gauss-Jordan reduction x + y + 2z – 5w = 3 2x + 5y – z – 9w = -3 2x + y – z + 3w = -11 x – 3y + 2z + 7w = -5

(cont’d) Example 9 - Step 1 - Step 2

(cont’d) E.g. 9 - Step 3 leading variables a free variable

Example 10 Example 10 - Solve the linear system by Gauss-Jordan reduction x1 + 2x2 – 3x4 + x5 = 2 x1 + 2x2 + x3 – 3x4 + x5 + 2x6 = 3 x1 + 2x2 – 3x4 + 2x5 + x6 = 4 3x1 + 6x2 + x3 – 9x4 + 4x5 + 3x6 = 9

(cont’d) Example 10 - Step 1 - Step 2

(cont’d) Example 10 - Step 3 leading variables free variables

Example 11 Example 11 - Solve the linear system by Gauss elimination x + 2y + 3z = 9 2x – y + z = 8 3x – z = 3

(cont’d) Example 11 - Step 1 - Step 2

(cont’d) Example 11 - Step 3 - By back substitution

Example 12 Example 12 - Solve the linear system by Gauss elimination x + 2y + 3z + 4w = 5 x + 3y + 5z + 7w = 11 x – z – w = -6

- Step 3 ⇒ 0x + 0y + 0z + 0w = 1 ⇒ No solutions !! (cont’d) Example 12 - Step 1 - Step 2 - Step 3 ⇒ 0x + 0y + 0z + 0w = 1 ⇒ No solutions !!

Consistent and Inconsistent - Consistent: Linear systems with at least one solution - Inconsistent: Linear systems with no solutions

Homogeneous Systems A system of linear equations is said to be homogeneous if all the constant terms are zeros. a11x1 + a12x2 + … + a1nxn = 0 a21x1 + a22x2 + … + a2nxn = 0 … am1x1 + am2x2 + … + amnxn = 0 ⇒ Ax = 0 Thus, a homogeneous system always has the solution x1 = x2 = … = xn = 0 → the trivial solution

Example 13 Example 13

Example 14 Example 14

Theorem 1.8 Theorem 1.8 A homogeneous system of m equations in n unknowns has a non-trivial solution if m < n, that is, if the number of unknowns exceeds the number of equations. namely, a homogeneous system has more variables than equations has many solutions. (a homogeneous system齊次系統; non-trivial solution 非零解)

Example 15 - A Homogeneous System E.g. 15

(cont’d) If let

A Homogeneous System Example x = xp + xh xp is a particular solution to the given system Axp = b , where b = [3 -3 -11 -5]T xh is a solution to the associated homogeneous system Axh = 0 .

Polynomial Interpolation - The general form y = an – 1xn – 1 + an – 2xn – 2 + … + a1x + a0 E.g. n = 3, y = a2x2 + a1x + a0 Given three distinct points (x1 , y1), (x2 , y2), (x3 , y3), we have y1 = a2x12 + a1x1 + a0 y2 = a2x22 + a1x2 + a0 y3 = a2x32 + a1x3 + a0 Polynomial Interpolation 2.【數學】多項式的。3. 【數學】插值法，內插法，內推法。

Example 16 Example 16 - Find the quadratic polynomial that interpolates the points (1, 3), (2, 4), (3, 7)

Example 17 – Temperature Distribution T1 = (260 – 100 + T2 + T3 )/4 or 4T1 – T2 – T3 = 160 T2 = (T1 + 100 + 40 + T4 )/4 or -T1 + 4T2 – T4 = 140 T3 = (60 + T1 + T4 + 0)/4 or -T1 + 4T3 – T4 = 60 T4 = (T2 + T3 + 40 + 0)/4 or -T2 – T3 + 4T4 = 40 ⇒ ⇒ T1 = 65, T2 = 60, T3 = 40, T4 = 35 .

Linear Equations and Matrices The Inverse of A Matrix

The Inverse of A Matrix 1.7 The inverse of a matrix Definition - An nn matrix A is called nonsingular (or invertible 可逆的) if there exists an nn matrix B such that AB = BA = In . - The matrix B is called the inverse of A - If there exists no such matrix B, then A is called singular (or noninvertible) - A is also an inverse of B

Example 1 ⇒ AB = BA = I2 - B is an inverse of A and A is nonsingular.

Theorem 1.9 An inverse of a matrix, if exists, is unique. Theorem 1.9 (proof) Let B and C be inverses of A. Then AB = BA = In, and AC = CA = In. Thus, C(AB) = CIn (CA)B = C InB = C , i.e., B = C .

Example 2 - Find the Inverse For the matrix A, find the inverse If exists, let the inverse A-1 be such that

(cont’d) and

⇒ No solution; singular Example 3 Example 3 and ⇒ No solution; singular

Theorem 1.10 Thm. 1.10 - Properties of an inverse - If A is nonsingular, then A-1 is nonsingular and (A-1)-1 = A ; - If A and B are nonsingular matrices, then AB is nonsingular and (AB)-1 = B-1 A-1 ; - If A is a nonsingular matrix, then (AT)-1 = (A-1)T .

Example 4 Example 4 ⇒ and

Corollary1.2 Corollary 1.2 - If A1 , A2 , … , Ar are nn nonsingular matrices, then (A1 A2 … Ar) is nonsingular and (A1 A2 … Ar)-1 = Ar-1 … A2-1 A1-1 .

Theorem 1.11 Theorem 1.11 Suppose that A, B are nn matrices, - If AB = In , then BA = In ; - If BA = In , then AB = In .

The Way to Find A-1 A practical method for finding A-1 Step 1. Form the 22n matrix [A | In] obtained by adjoining the identity matrix In to the given matrix A Step 2. Compute the reduced row echelon form of the matrix obtained in Step 1 by using elementary row operations Step 3. Suppose that Step 2 has produced the matrix [C | D] in reduced row echelon form: If C = In , then D = A-1 ; If C ≠ In , then C has a row of zeros and the matrix A is singular .

Example 5 – Find the Inverse E.g. 5 – Find the inverse

Example 6 – Find the Inverse E.g. 6 - Find the inverse The left-half matrix cannot have a one in the (3, 3) location, the reduced echelon form cannot be I3. Thus A-1 does not exist.

Theorem 1.12 & 1.13 Theorem 1.12 An nn matrix is nonsingular iff it is row equivalence to In . Theorem 1.13 If A is an nn matrix, the homogeneous system Ax = 0 has a nontrivial solution iff A is singular.

Proof of Theorem 1.13 Proof of Theorem 1.13 Suppose that A is nonsingular, then A-1 exists and A-1(Ax) = A-1 0 (A-1A)x = 0 In x = 0 x = 0 ⇒ Ax = 0 has a trivial solution (contradiction to a non-trivial solution, hence A must be singular)

Example 8 Example 8 Consider the homogeneous system Ax = 0, where A is the matrix (A is nonsingular) Gauss-Jordan reduction The trivial solution x = 0

Example 9 Example 9 - Consider the homogeneous system Ax = 0, where A is the matrix (A is singular)

Theorem 1.14 Theorem 1.14 If A is an nn matrix, then A is nonsingular iff the linear system Ax = b has a unique solution for every n1 matrix b .

Summary The Symmetry, Singularity, Inverse of A Matrix

Some Special Matrix 4. A square matrix A is said to be antisymmetric if -AT = A. (i) If A is square, prove that A + AT is symmetric and A – AT is antisymmetric; (ii) any square matrix A can be decomposed into the sum of a symmetric matrix B and an antisymmetric matrix C: A = B + C . 5. Given two symmetric matrices of the same size, A and B, then a necessary and sufficient condition for the product AB to be symmetric is that AB = BA.

Some Special Matrix 1. A square matrix A is said to be normal if AAT = ATA. All symmetric matrices are normal ; 2. A square matrix A is said to be idempotent if A2 = A. If A is idempotent then AT is also idempotent ; 3. A square matrix A is said to nilpotent if there is a positive integer p such that Ap = O. The least integer such that Ap = O is called the degree of nilpotency of the matrix. If A is nilpotent, then AT is also nilpotent with the same degree of nilpotency.

List of Nonsingular Equivalences 1. A is nonsingular ; 2. x = 0 is the only solution to Ax = 0 ; 3. A is row equivalence to In ; 4. The linear system Ax = b has a unique solution for every n1matrix b .

Properties of Matrix Inverse 1. (A-1)-1 = A ; 2. (cA)-1 = (1/c)A-1 , where c is a nonzero scalar; 3. (AB)-1 = B-1A-1 ; 4. (An)-1 = (A-1)n ; 5. (AT)-1 = (A-1)T , where T : transpose.

Conditions of Matrix Inverse A matrix has no inverse, if (i) two rows are equal; (ii) two columns are equal; (Use the transpose) (iii) it has a column of zeros.

The Inverse of 22 Matrix Note: The cancellation law doesn’t hold. If A = , show that A-1 = . Note: The cancellation law doesn’t hold. That is, AB = AC doesn’t imply that B = C . Also, AB = O doesn’t imply that A = O or B = O. However, if A is an invertible matrix, then if AB = AC , then B = C ; if AB = 0, then B = 0 .