1. Introduction to Numerical Analysis I MATH/CMPSC 455 Conjugate Gradient Methods

2. A-O RTHOGONAL B ASIS form a basis of, where is the i-th row of the identity matrix. They are orthogonal in the following sense: They are linearly independent, and form a basis. Introduce a set of nonzero vectors, They satisfy the following condition: We say they are A-orthogonal, or conjugate w.r.t A.

3. C ONJUGATE D IRECTION M ETHOD Theorem: For any initial guess, the sequence generated by the above iterative method, converges to the solution of the linear system in at most n iterations. Question: How to find the A-orthogonal bases?

4. C ONJUGATE G RADIENT METHOD Answer: Each conjugate direction is chosen to be a linear combination of the residual and the previous direction Conjugate Gradient Method: Conjugate direction method on this particular basis.

5. CG (O RIGINAL V ERSION ) While End While

6. Theorem : Let A be a symmetric positive-definite matrix. In the Conjugate Gradient Method, we have

7. CG (P RACTICAL V ERSION ) While End While

8. Example :