1. Solving Scalar Linear Systems A Little Theory For Jacobi Iteration
Lecture 15
MA/CS 471
Fall 2003

2. Review:
Using Kirchoff’s second law we build the loop current circuit matrix. 3 1W 4W 4W 6W + - 4 5 2 V4 3W 7W 2W 1 1W + - V1
Note we have boosted the center cell to ensure diagonal dominance (hack)

3. System:
Jacobi iterative approach

4. Jacobi v. Gauss-Seidel
Jacobi
Gauss-Seidel

5. Convergence Proof

6. Definition of Spectral Radius
We define the spectral radius of a matrix A as:

7. Stage 1: Unique Limit The equation we wish to solve is:
We consider the following iterator:
For some easily invertible matrix Q
Suppose this iteration does indeed converge as , i.e.
Then xtilde will satisfy:

8. cont
And yields
So if the iteration converges, then the limit vector will be a solution to the original system.

9. Second Stage of Convergence Proof
We first prove that the Jacobi (and Gauss-Seidel) methods converge if and only if the spectral radius of

10. Theorem
Suppose and that both Q and A are non-singular. If the spectral radius of is strictly bounded above by 1 then the iterates defined by:
converge to for any starting vector

11. Proof
Let denote the error in the n’th iterate. We combine the following relationships:
to obtain:
simplifying:

12. Proof cont
The iteration converges with increasing n if and only if (proof omitted).

13. Third Stage of Convergence
Now we are left with the task of proving that for the choice of Q the matrix has
i.e. we have to prove that the absolute value of all of the eigenvalues of the matrix is less than one.
So we use Gershgorin’s theorem to find the range of the eigenvalues of

14. Recall: Gershgorin’s Circle Theorem
Let A be a square NxN matrix. Around every element
aii on the diagonal of the matrix draw a circle with
radius
Such circles are known as Gershgorin’s disks.
Theorem: every eigenvalue of A lies in one of these Gershgorin’s disks.

15. Jacobi Iteration
Recall the generic iterative scheme required a Q matrix which is “easily” invertible.
Let’s take Q=diag(A) (i.e. a matrix with zeros everywhere, apart from the diagonal entries which are the same as those of A)
Let’s write: A=L+D+U D U L

16. Cont. Then the scheme becomes: i.e.
We can determine conditions under which this scheme will converge.
Recall the necessary and sufficient condition that

17. Cont. We can use Gershgorin’s theorem after we note that
has zero entries on the diagonal – so all the Gershgorin disks will be centered at zero and have maximum radius:

18. cont So if Then we are done. Note, a matrix which satisfies:
Is called diagonally dominant.

19. Summary
The first stage of the convergence proof showed that the unique possible convergent limit of the scheme is the actual solution to the linear system.
Secondly, we showed that the scheme converges if and only if
Thirdly, we showed that for the choice of Q = diagonal of A, that (2) was satisfied.
i.e. Jacobi iteration converges for any initial guess for x if A is diagonally dominant.