1. Spring 2009-2010 Dr. Jehad Al Dallal
Unit #1 Linear Systems
Spring
Dr. Jehad Al Dallal

2. Linear Equations One equation 7x=21…
A system of n simultaneous linear equations can be represented in matrix notation as:
Ax = b
where A is an nXn matrix, and x and b are vectors of length n.
Can write solution as x = A-1b where A-1 is the inverse of A.

3. Examples If A is a 2X2 matrix, for example
then 2x1 – x2 = 3 and 3x1 + 4x2 = -1. Each represents a straight line and the solution of the above is given by their intersection.
If A is a 3X3 matrix each of the three equations represents a plane, and the solution is the point lying at the intersection of the three planes.

4. Solution methods Exact solution methods Approximation methods
Example: Gaussian Elimination method
Approximation methods
Example: Gauss-Seidel method

5. Gaussian Elimination
Eliminate x1 from all the equations after the first.
Then eliminate x2 from all the equations after the second.
Then eliminate x3 from all the equations after the third.
And so on, until after n-1 steps we have eliminated xj from all the equations after the jth, for j = 1, 2, …, n-1.
These steps are referred to as the forward elimination stage of Gaussian elimination.

6. Example
Subtract -3/10 times equation 1 from equation 2, and 5/10 times equation 1 from equation 3.
Next we swap equations 2 and 3. This is called partial pivoting. It is done to get the largest absolute value on or below the diagonal in column 2 onto the diagonal. This makes the algorithm more stable with respect to round-off errors (see later).

7. Example (continued)
Now subtract -0.1/2.5 times equation 2 from equation 3.
This completes the forward elimination stage of the Gaussian elimination algorithm.

8. Back Substitution
After the forward elimination phase, the matrix has been transformed into upper triangular form.
Equation n just involves xn and so can now be solved immediately.
Equation n-1 just involves xn-1 and xn, and since we already know xn we can find xn-1.
Working our way backwards through the equations we can find xn, xn-1,…, x1.
This is called the back substitution phase of the Gaussian elimination algorithm.

9. The Example Again
Equation 3 is 6.2x3 = 6.2, so x3 = 1. This value is substituted into equation 2:
2.5x2 + (5)(1) = 2.5
so x2 = -1. Substituting for x2 and x3 in equation 1:
10x1 + (-7)(-1) = 7
so x1 = 0.

10. LU Factorisation
The Gaussian elimination process can be expressed in terms of three matrices.
The first matrix has 1’s on the main diagonal and the multipliers used in the forward elimination below the diagonal. This is a lower triangular matrix with unit diagonal, and is usually denoted by L.

11. LU Factorisation
The second matrix, denoted by U, is the upper triangular matrix obtained at the end of the forward elimination.
The third matrix, denoted by P, is a permutation matrix representing the row interchanges performed in pivoting.

12. L, U, and P
The original matrix can be expressed as:
LU = PA

13. Norms and condition numbers
Norm of a vector : single number that measures the general size of the elements of the vector.
Most common norms use p = 1, 2, and ¥

14. Example X=[2 4 6 -8] ||x||1=2+4+6+8=20 ||x||2 =(22+42+62+82)0.5

15. Condition Number
The condition number measures the amplification of the relative error.

16. Example

17. Uses of Condition Number
As a measure of the amplification of relative error due to changes in matrix A.
As a measure of how close a matrix is to being singular. If K(A) is large then A is close to singular.