1. Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 A Look-Back Technique of Restart for the GMRES(m) Method Akira IMAKURA † Tomohiro SOGABE ‡ Shao-Liang ZHANG † † Nagoya University, Japan. ‡ Aichi Prefectural University, Japan. 1/29

2. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction Main subject -- Nonsymmetric linear systems of the form Krylov subspace methods -- Symmetric linear systems Based on Lanczos algorithm : CG, MINRES, … -- Nonsymmetric linear systems Based on two-sided Lanczos algorithm : Bi-CG, Bi-CGSTAB, GPBi-CG… Based on Arnoldi algorithm : GCR, GMRES, … ＋ restart → GMRES(m) method 2/29

3. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction GMRES(m) method [Y. Saad and M. H. Schultz:1986] -- Algorithm (focus on restart) Algorithm : GMRES(m) Run m iterations of GMRES Input:, Output: -- Update the initial guess l : number of restart cycle 3/29

4. Motivation 1.0 0.0 May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction GMRES(m) method [Y. Saad and M. H. Schultz:1986] -- Residual polynomials : residual polynomial l : number of restart 1.0 0.0 4/29

5. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction GMRES(m) method with readjustment -- Numerical result compared with GMRES(m) method : parameters CAVITY05XENON1 ― GMRES(m)― GMRES(m) with readjustment Readjustment leads to be better convergence Question How does we readjust the function s.t. the root is moved? 5/29

6. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction Look-Back GMRES(m) method -- Algorithm (focus on update the initial guess) Algorithm : Look-Back GMRES(m) Run m iterations of GMRES Input:, Output: We can simply achieve the readjustment 6/29

7. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction Look-Back GMRES(m) method -- Extension of the GMRES(m) method -- A Look-Back technique of restart → Analyze based on error equations → Analyze based on residual polynomials 7/29

8. Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 Extension of the GMRES(m) method -- Analysis based on error equation -- 8/29

9. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Extension of the GMRES(m) method Algorithm : GMRES(m) Run m iterations of GMRES Input:, Output: Run m iterations of GMRES Input:, Output: Algorithm : Extension of GMRES(m) ? 9/29

10. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Extension of the GMRES(m) method Analysis based on error equations -- Introduction of error eq. and iterative refinement scheme Definition : error equation Let and be the exact solution and the numerical solution respectively. Then the error vector can be computed by solving the so-called error equation, i.e., where is residual vector corresponding to. Definition : iterative refinement scheme The technique based on solving error equations recursively to achieve the higher accuracy of the numerical solution is called the iterative refinement scheme. 10/29

11. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Extension of the GMRES(m) method Algorithm : GMRES(m) Run m iterations of GMRES Input:, Output: Run m iterations of GMRES Input:, Output: Algorithm : Extension of GMRES(m) Algorithm : Iterative refinement Run m iterations of GMRES Input:, Output: Mathematically equivalent Natural extension Run m iterations of GMRES Input:, Output: Algorithm : Iterative refinement 11/29

12. Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 A Look-Back technique of restart -- Analysis based on residual polynomials -- 12/29

13. Extension of GMRES(m) method GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart Difference between GMRES(m) and its Extension l : number of restart residual polynomial If we set. Then the rational function s.t. is exist. 13/29

14. l : number of restart Extension of GMRES(m) method Look-Back GMRES(m) method GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart Difference between GMRES(m) and its Extension Set by some techniqueLook-Back technique residual polynomial 14/29

15. Extension of GMRES(m) method Look-Back GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart A Look-Back technique Set by Look-Back technique Look-Back strategy → Readjust s.t. root is moved Look-Back at the past polynomials and rational functions 15/29

16. Extension of GMRES(m) method Look-Back GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart A Look-Back technique Motivation 1.0 0.0 Root of the function is moved ( → ) It is expected that readjustment leads to be high convergence 16/29

17. Extension of GMRES(m) method Look-Back GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart A Look-Back technique e.g. 17/29

18. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart Proposal of Look-Back GMRES(m) method -- Algorithm (focus on update the initial guess) Run m iterations of GMRES Input:, Output: -- extra costs for Look-Back technique : 1 matrix-vector multiplication per 1 restart Algorithm : Look-Back GMRES(m) 18/29

19. Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 Numerical experiments -- Comparison with GMRES(m) -- 19/29

20. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Compared methods (without preconditioner) -- GMRES(m) method -- Look-Back GMRES(m) method Parameters -- right-hand-side: -- initial guess: -- stopping criterion: Experimental conditions -- AMD Phenom II X4 940 (3.0GHz); -- Standard Fortran 77 using double precision. Test problems [obtained from UF Sparse Matrix Collection] -- CAVITY05,CAVITY16,CHIPCOOL0, MEMPLUS,NS3DA,RAJAT03, RDB5000,XENON1,XENON2. (m = 30, 100) 20/29

21. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 30 CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 ― GMRES(m)― Look-Back GMRES(m) 21/29

22. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 30 CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart 1.86 E-02 1.87 E-02 7.30 E-02 7.33 E-02 5.16 E-02 5.13 E-02 2.30 E-01 2.35 E-01 1.93 E-01 1.29 E-02 2.44 E-01 2.51 E-01 8.36 E-01 8.39 E-01 2.94 E+00 1.25 E+00 † 3.15 E+01 † 7.83 E+01 1.14 E+01 3.93 E+00 1.78E +01 1.84 E+01 † 9.16 E+00 4.00 E-01 4.42 E-01 9.47 E+01 1.57 E+01 4.38 E+02 6.64 E+01 4.43 E-03 4.57 E-03 ― GMRES(m)― Look-Back GMRES(m) 22/29

23. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 100 ― GMRES(m)― Look-Back GMRES(m) CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 23/29

24. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 100 ― GMRES(m)― Look-Back GMRES(m) Time [sec.] Total1 Restart †1.23 E-01 2.38 E+011.24 E-01 Time [sec.] Total1 Restart †5.47 E-01 1.28 E+025.52 E-01 Time [sec.] Total1 Restart 1.35 E+014.49 E-01 7.78 E+004.50 E-01 Time [sec.] Total1 Restart 2.11 E+011.07 E+00 2.11 E+011.08 E+00 Time [sec.] Total1 Restart †1.71 E-01 8.56 E+001.71 E-01 Time [sec.] Total1 Restart 4.18 E-011.10 E-01 3.93 E-011.10 E-01 Time [sec.] Total1 Restart 6.41 E+011.65 E+00 3.11 E+011.63 E+00 Time [sec.] Total1 Restart 2.70 E+025.49 E+00 1.23 E+025.43 E+00 Time [sec.] Total1 Restart 2.65 E+003.09 E-02 1.15 E+003.09 E-02 CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 24/29

25. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Test problems -- From discretization of partial differential equation of the form over the unit square. 25/29

26. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Compared methods (with ILU(0) preconditioner) -- GMRES(m) method -- Look-Back GMRES(m) method Parameters -- initial guess: -- stopping criterion: Experimental conditions -- AMD Phenom II X4 940 (3.0GHz); -- Standard Fortran 77 using double precision. (m = 10) 26/29

27. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results ― GMRES(m)― Look-Back GMRES(m) α = 0α = 0.5 α = 1.0α = 2.0 27/29

28. Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 Conclusion and Future works 28/29

29. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Conclusion and Future works Conclusion -- In this talk, from analysis based on residual polynomials, we proposed the Look-Back GMRES(m) method. -- From our numerical experiments, we learned that the Look- Back GMRES(m) method shows a good convergence than the GMRES(m) method in many cases. -- Therefore the Look-Back GMRES(m) method will be an efficient variant of the GMRES(m) method. Future works -- Analyze details of the Look-Back technique. -- Compare with other techniques for the GMRES(m) method. 29/29

30. Thank you for your kind attention

31. May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments 31/29