Unit #1 Linear Systems Fall Dr. Jehad Al Dallal

Linear Equations One equation 7x=21 … A system of n simultaneous linear equations can be represented in matrix notation as: Ax = b where A is an nXn matrix, and x and b are vectors of length n. Can write solution as x = A -1 b where A -1 is the inverse of A.

Examples If A is a 2X2 matrix, for example If A is a 3X3 matrix each of the three equations represents a plane, and the solution is the point lying at the intersection of the three planes. then 2x 1 – x 2 = 3 and 3x 1 + 4x 2 = -1. Each represents a straight line and the solution of the above is given by their intersection.

Solution methods Exact solution methods Example: Gaussian Elimination method Approximation methods Example: Gauss-Seidel method

Gaussian Elimination Eliminate x 1 from all the equations after the first. Then eliminate x 2 from all the equations after the second. Then eliminate x 3 from all the equations after the third. And so on, until after n-1 steps we have eliminated x j from all the equations after the jth, for j = 1, 2, …, n-1. These steps are referred to as the forward elimination stage of Gaussian elimination.

Example Subtract -3/10 times equation 1 from equation 2, and 5/10 times equation 1 from equation 3. Next we swap equations 2 and 3. This is called partial pivoting. It is done to get the largest absolute value on or below the diagonal in column 2 onto the diagonal. This makes the algorithm more stable with respect to round-off errors (see later).

Example (continued) Now subtract -0.1/2.5 times equation 2 from equation 3. This completes the forward elimination stage of the Gaussian elimination algorithm.

Back Substitution After the forward elimination phase, the matrix has been transformed into upper triangular form. Equation n just involves x n and so can now be solved immediately. Equation n-1 just involves x n-1 and x n, and since we already know x n we can find x n-1. Working our way backwards through the equations we can find x n, x n-1, …, x 1. This is called the back substitution phase of the Gaussian elimination algorithm.

The Example Again Equation 3 is 6.2x 3 = 6.2, so x 3 = 1. This value is substituted into equation 2: 2.5x 2 + (5)(1) = 2.5 so x 2 = -1. Substituting for x 2 and x 3 in equation 1: 10x 1 + (-7)(-1) = 7 so x 1 = 0.

LU Factorisation The Gaussian elimination process can be expressed in terms of three matrices. The first matrix has 1 ’ s on the main diagonal and the multipliers used in the forward elimination below the diagonal. This is a lower triangular matrix with unit diagonal, and is usually denoted by L.

LU Factorisation The second matrix, denoted by U, is the upper triangular matrix obtained at the end of the forward elimination. The third matrix, denoted by P, is a permutation matrix representing the row interchanges performed in pivoting.

L, U, and P  The original matrix can be expressed as: LU = PA

Norms and condition numbers Norm of a vector : single number that measures the general size of the elements of the vector. Most common norms use p = 1, 2, and ¥

Example X=[ ] ||x|| 1 = =20 ||x|| 2 =( ) 0.5 ||x|| ¥ =|-8|=8

Condition Number The condition number measures the amplification of the relative error.

Example

Uses of Condition Number  As a measure of the amplification of relative error due to changes in matrix A.  As a measure of how close a matrix is to being singular. If K(A) is large then A is close to singular.