Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 A Look-Back Technique of Restart for the GMRES(m) Method Akira IMAKURA † Tomohiro SOGABE ‡ Shao-Liang ZHANG † † Nagoya University, Japan. ‡ Aichi Prefectural University, Japan. 1/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction Main subject -- Nonsymmetric linear systems of the form Krylov subspace methods -- Symmetric linear systems Based on Lanczos algorithm : CG, MINRES, … -- Nonsymmetric linear systems Based on two-sided Lanczos algorithm : Bi-CG, Bi-CGSTAB, GPBi-CG… Based on Arnoldi algorithm : GCR, GMRES, … ＋ restart → GMRES(m) method 2/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction GMRES(m) method [Y. Saad and M. H. Schultz:1986] -- Algorithm (focus on restart) Algorithm : GMRES(m) Run m iterations of GMRES Input:, Output: -- Update the initial guess l : number of restart cycle 3/29

Motivation May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction GMRES(m) method [Y. Saad and M. H. Schultz:1986] -- Residual polynomials : residual polynomial l : number of restart /29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction GMRES(m) method with readjustment -- Numerical result compared with GMRES(m) method : parameters CAVITY05XENON1 ― GMRES(m)― GMRES(m) with readjustment Readjustment leads to be better convergence Question How does we readjust the function s.t. the root is moved? 5/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction Look-Back GMRES(m) method -- Algorithm (focus on update the initial guess) Algorithm : Look-Back GMRES(m) Run m iterations of GMRES Input:, Output: We can simply achieve the readjustment 6/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Introduction Look-Back GMRES(m) method -- Extension of the GMRES(m) method -- A Look-Back technique of restart → Analyze based on error equations → Analyze based on residual polynomials 7/29

Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 Extension of the GMRES(m) method -- Analysis based on error equation -- 8/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Extension of the GMRES(m) method Algorithm : GMRES(m) Run m iterations of GMRES Input:, Output: Run m iterations of GMRES Input:, Output: Algorithm : Extension of GMRES(m) ? 9/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Extension of the GMRES(m) method Analysis based on error equations -- Introduction of error eq. and iterative refinement scheme Definition : error equation Let and be the exact solution and the numerical solution respectively. Then the error vector can be computed by solving the so-called error equation, i.e., where is residual vector corresponding to. Definition : iterative refinement scheme The technique based on solving error equations recursively to achieve the higher accuracy of the numerical solution is called the iterative refinement scheme. 10/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Extension of the GMRES(m) method Algorithm : GMRES(m) Run m iterations of GMRES Input:, Output: Run m iterations of GMRES Input:, Output: Algorithm : Extension of GMRES(m) Algorithm : Iterative refinement Run m iterations of GMRES Input:, Output: Mathematically equivalent Natural extension Run m iterations of GMRES Input:, Output: Algorithm : Iterative refinement 11/29

Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 A Look-Back technique of restart -- Analysis based on residual polynomials -- 12/29

Extension of GMRES(m) method GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart Difference between GMRES(m) and its Extension l : number of restart residual polynomial If we set. Then the rational function s.t. is exist. 13/29

l : number of restart Extension of GMRES(m) method Look-Back GMRES(m) method GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart Difference between GMRES(m) and its Extension Set by some techniqueLook-Back technique residual polynomial 14/29

Extension of GMRES(m) method Look-Back GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart A Look-Back technique Set by Look-Back technique Look-Back strategy → Readjust s.t. root is moved Look-Back at the past polynomials and rational functions 15/29

Extension of GMRES(m) method Look-Back GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart A Look-Back technique Motivation Root of the function is moved ( → ) It is expected that readjustment leads to be high convergence 16/29

Extension of GMRES(m) method Look-Back GMRES(m) method May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart A Look-Back technique e.g. 17/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider A Look-Back technique of restart Proposal of Look-Back GMRES(m) method -- Algorithm (focus on update the initial guess) Run m iterations of GMRES Input:, Output: -- extra costs for Look-Back technique : 1 matrix-vector multiplication per 1 restart Algorithm : Look-Back GMRES(m) 18/29

Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 Numerical experiments -- Comparison with GMRES(m) -- 19/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Compared methods (without preconditioner) -- GMRES(m) method -- Look-Back GMRES(m) method Parameters -- right-hand-side: -- initial guess: -- stopping criterion: Experimental conditions -- AMD Phenom II X4 940 (3.0GHz); -- Standard Fortran 77 using double precision. Test problems [obtained from UF Sparse Matrix Collection] -- CAVITY05,CAVITY16,CHIPCOOL0, MEMPLUS,NS3DA,RAJAT03, RDB5000,XENON1,XENON2. (m = 30, 100) 20/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 30 CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 ― GMRES(m)― Look-Back GMRES(m) 21/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 30 CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart Time [sec.] Total1 Restart 1.86 E E E E E E E E E E E E E E E E+00 † 3.15 E+01 † 7.83 E E E E E+01 † 9.16 E E E E E E E E E-03 ― GMRES(m)― Look-Back GMRES(m) 22/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 100 ― GMRES(m)― Look-Back GMRES(m) CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 23/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results for m = 100 ― GMRES(m)― Look-Back GMRES(m) Time [sec.] Total1 Restart †1.23 E E E-01 Time [sec.] Total1 Restart †5.47 E E E-01 Time [sec.] Total1 Restart 1.35 E E E E-01 Time [sec.] Total1 Restart 2.11 E E E E+00 Time [sec.] Total1 Restart †1.71 E E E-01 Time [sec.] Total1 Restart 4.18 E E E E-01 Time [sec.] Total1 Restart 6.41 E E E E+00 Time [sec.] Total1 Restart 2.70 E E E E+00 Time [sec.] Total1 Restart 2.65 E E E E-02 CAVITY05CAVITY16CHIPCOOL0 MEMPLUSNS3DARAJAT03 RDB5000XENON1XENON2 24/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Test problems -- From discretization of partial differential equation of the form over the unit square. 25/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Compared methods (with ILU(0) preconditioner) -- GMRES(m) method -- Look-Back GMRES(m) method Parameters -- initial guess: -- stopping criterion: Experimental conditions -- AMD Phenom II X4 940 (3.0GHz); -- Standard Fortran 77 using double precision. (m = 10) 26/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments Numerical results ― GMRES(m)― Look-Back GMRES(m) α = 0α = 0.5 α = 1.0α = /29

Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 Conclusion and Future works 28/29

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Conclusion and Future works Conclusion -- In this talk, from analysis based on residual polynomials, we proposed the Look-Back GMRES(m) method. -- From our numerical experiments, we learned that the Look- Back GMRES(m) method shows a good convergence than the GMRES(m) method in many cases. -- Therefore the Look-Back GMRES(m) method will be an efficient variant of the GMRES(m) method. Future works -- Analyze details of the Look-Back technique. -- Compare with other techniques for the GMRES(m) method. 29/29

Thank you for your kind attention

May 25, 2010Applied Linear Algebra - in honor of Hans Schneider Numerical experiments 31/29