1 Systems of Linear Equations Error Analysis and System Condition

2 Question Suppose we use a computer to solve A 1 x 1 = b 1 and A 2 x 2 = b 2 and obtain x' 1 and x' 2. Which of these two solutions is more accurate (as compare relatively to the true solutions)?

3 Error Analysis Suppose we want to find the solution of Ax = b Would there be a x' which gives b – Ax' ≈ 0 ? –i.e., is the system solvable? Would a small rounding error in b or in A results in large change in the solution x ?

4 Error Analysis Relationship between error and residual Suppose x' is an estimated solution of Ax = b. Let the residual be r = b – Ax' ---- (1) Ax = b =>0 = b – Ax---- (2) (1) – (2)=>r = A(x – x') =>x – x' = A -1 r Thus, if A -1 has elements much larger than 1 (assuming A has been scaled), a small residual, r, may result in a large error x – x'. (i.e., ill-condition cases) The inverse of A can indicate if A is ill-condition.

5 Question If a matrix A is scaled (s.t. the largest element is 1), then A -1 can indicate whether A is ill-condition or not. Suppose both A 1 and A 2 are scaled. How can you tell which of them is more ill-condition? We need a way to quantify the condition.

6 Condition number A formal way to state the condition of a system (ill-condition or not) A way to check the condition is the matrix condition number. Cond(A) = || A || · || A -1 || where || · || is the norm, which is a real-valued function that measures the size of length of vectors or matrices. Cond(A) ≥ 1

7 Various definitions of norm for Vectors p-norm Euclidean-norm 1-norm ∞-norm

8 Various definitions of norm for Matrices p-norm Euclidean-norm 1-norm (max column) ∞-norm (max row)

9 Error Analysis Relationship between error in x and (rounding error in A ) If there is a small change in the value of A, what would be the change in x ? It can also be shown that If the condition number is considerably greater than unity, it suggest that the system is ill-conditioned.

10 Examples

11 Iterative Refinement In some cases, we can reduce round-off errors by the following procedure (can repeat for several times): Suppose in solving Ax = b, we obtain an approximated solution x' s.t. x = x' + ε. Substituting x' back into the system yields Ax' = b'---- (1) Ax = b => A(x' + ε) = b---- (2) (2) – (1) =>Aε = b – b'---- (3) Solving the system of equations in (3) yields ε. By adding ε (called the correction factor) to x', we can improve the solution to Ax = b.

12 Summary The inverse of a matrix can indicate if the matrix is ill-condition or not. The matrix condition number can be calculated as Cond(A) = || A || · || A -1 || Large condition number implies matrix is ill-condition.