Spring 2009-2010 Dr. Jehad Al Dallal Unit #1 Linear Systems Spring 2009-2010 Dr. Jehad Al Dallal

Linear Equations One equation 7x=21 … A system of n simultaneous linear equations can be represented in matrix notation as: Ax = b where A is an nXn matrix, and x and b are vectors of length n. Can write solution as x = A-1b where A-1 is the inverse of A.

Examples If A is a 2X2 matrix, for example then 2x1 – x2 = 3 and 3x1 + 4x2 = -1. Each represents a straight line and the solution of the above is given by their intersection. If A is a 3X3 matrix each of the three equations represents a plane, and the solution is the point lying at the intersection of the three planes.

Solution methods Exact solution methods Approximation methods Example: Gaussian Elimination method Approximation methods Example: Gauss-Seidel method

Gaussian Elimination Eliminate x1 from all the equations after the first. Then eliminate x2 from all the equations after the second. Then eliminate x3 from all the equations after the third. And so on, until after n-1 steps we have eliminated xj from all the equations after the jth, for j = 1, 2, …, n-1. These steps are referred to as the forward elimination stage of Gaussian elimination.

Example Subtract -3/10 times equation 1 from equation 2, and 5/10 times equation 1 from equation 3. Next we swap equations 2 and 3. This is called partial pivoting. It is done to get the largest absolute value on or below the diagonal in column 2 onto the diagonal. This makes the algorithm more stable with respect to round-off errors (see later).

Example (continued) Now subtract -0.1/2.5 times equation 2 from equation 3. This completes the forward elimination stage of the Gaussian elimination algorithm.

Back Substitution After the forward elimination phase, the matrix has been transformed into upper triangular form. Equation n just involves xn and so can now be solved immediately. Equation n-1 just involves xn-1 and xn, and since we already know xn we can find xn-1. Working our way backwards through the equations we can find xn, xn-1,…, x1. This is called the back substitution phase of the Gaussian elimination algorithm.

The Example Again Equation 3 is 6.2x3 = 6.2, so x3 = 1. This value is substituted into equation 2: 2.5x2 + (5)(1) = 2.5 so x2 = -1. Substituting for x2 and x3 in equation 1: 10x1 + (-7)(-1) = 7 so x1 = 0.

LU Factorisation The Gaussian elimination process can be expressed in terms of three matrices. The first matrix has 1’s on the main diagonal and the multipliers used in the forward elimination below the diagonal. This is a lower triangular matrix with unit diagonal, and is usually denoted by L.

LU Factorisation The second matrix, denoted by U, is the upper triangular matrix obtained at the end of the forward elimination. The third matrix, denoted by P, is a permutation matrix representing the row interchanges performed in pivoting.

L, U, and P The original matrix can be expressed as: LU = PA

Norms and condition numbers Norm of a vector : single number that measures the general size of the elements of the vector. Most common norms use p = 1, 2, and ¥

Example X=[2 4 6 -8] ||x||1=2+4+6+8=20 ||x||2 =(22+42+62+82)0.5

Condition Number The condition number measures the amplification of the relative error.

Example

Uses of Condition Number As a measure of the amplification of relative error due to changes in matrix A. As a measure of how close a matrix is to being singular. If K(A) is large then A is close to singular.