1. Conjugate Gradient Method
invented by Hestenes and Stiefel around 1951
Conjugate Gradient Method
It is an iterative method to solve the linear system of equations

2. Conjugate Gradient Method
Example: Solve:
k=1
k=2
k=3
k=4 x1 x2 x3 X4

3. Quadratic function Define: Example:
We want to solve the following linear system
Define:
quadratic function
Example:

4. Quadratic Function
Example:
Remark:

5. Minimization equivalent ot linear system
Remark:
Problem (1)
Problem (2)
IDEA:
Search for the minimum

6. Conjugate Gradient Method
Example:
minimum

7. Minimum
IDEA:
Search for the minimum
Remark:

8. Conjugate Gradient Method
“search direction”
“step length”

9. Conjugate Gradient Method
vectors
constants

10. Conjugate Gradient Method

11. Conjugate Gradient Method

12. INNER
PRODUCT

13. Inner Product DEF: Example: Example: We say that
Is an inner product if
Example:
Example:
A is SPD
We define the norm

14. Inner Product
DEF:
DEF:
where A is SPD
Example:

15. Conjugate
Gradient

16. Conjugate Gradient Method

17. Conjugate Gradient Method

18. Conjugate Gradient Method
HW:

19. Conjugate Gradient Method
Lemma:[Elman,Silvester,Wathen Book]
vectors
Orthogonal
A-Orthogonal

20. Error and Residual vectors REMARK REMARK Orthogonal A-Orthogonal
Minimizes the A-norm of the error

21. Conjugate Gradient Method

22. Connection to Lanczos

23. Introduction to Krylov Subspace Methods
DEF:
Krylov sequence
Example:
Krylov sequence

24. Introduction to Krylov Subspace Methods
DEF:
Krylov subspace
Example:
Krylov subspace
DEF:
Example:
Krylov matrix

25. Introduction to Krylov Subspace Methods
DEF:
Example:
Krylov matrix
Remark:

26. Lanczos method Lanczos: The Lanczos algorithm is defined as follows
An orthogonal basis for