Krylov-Subspace Methods - II Lecture 7 Alessandra Nardi Thanks to Prof. Jacob White, Deepak Ramaswamy, Michal Rewienski, and Karen Veroy

Last lectures review Overview of Iterative Methods to solve Mx=b –Stationary –Non Stationary QR factorization –Modified Gram-Schmidt Algorithm –Minimization View of QR General Subspace Minimization Algorithm Generalized Conjugate Residual Algorithm –Krylov-subspace –Simplification in the symmetric case –Convergence properties Eigenvalue and Eigenvector Review –Norms and Spectral Radius –Spectral Mapping Theorem

Arbitrary Subspace Methods Residual Minimization

Use Gram-Schmidt on Mw i’s ! Arbitrary Subspace Methods Residual Minimization

kth order polynomial Krylov Subspace Methods Krylov Subspace

Krylov Subspace Methods Subspace Generation The set of residuals also can be used as a representation of the Krylov-Subspace Generalized Conjugate Residual Algorithm Nice because the residuals generate next search directions

Determine optimal stepsize in kth search direction Update the solution (trying to minimize residual) and the residual Compute the new orthogonalized search direction (by using the most recent residual) Krylov-Subspace Methods Generalized Conjugate Residual Method (k-th step)

Vector inner products, O(n) Matrix-vector product, O(n) if sparse Vector Adds, O(n) O(k) inner products, total cost O(nk) If M is sparse, as k (# of iters) approaches n, Better Converge Fast! Krylov-Subspace Methods Generalized Conjugate Residual Method (Computational Complexity for k-th step)

Summary What is an iterative non stationary method: x (k+1) =x (k) +a k p k How search to calculate: –Search directions (p k ) –Step along search directions (a k ) Krylov Subspace  GCR GCR is O(k 2 n) –Better converge fast!  Now look at convergence properties of GCR

Krylov Methods Convergence Analysis Basic properties

GCR Optimality Property Therefore Any polynomial which satisfies the constraints can be used to get an upper bound on Krylov Methods Convergence Analysis Optimality of GCR poly

Theorem: Any induced norm is a bound on the spectral radius Proof: Eigenvalues and eigenvectors review Induced norms

Given a polynomial Apply the polynomial to a matrix Then Useful Eigenproperties Spectral Mapping Theorem

Krylov Methods Convergence Analysis Overview where is any (k+1)-th order polynomial subject to:  may be used to get an upper bound on Matrix norm propertyGCR optimality property

Review on eigenvalues and eigenvectors –Induced norms: relate matrix eigenvalues to the matrix norms –Spectral mapping theorem: relate matrix eigenvalues to matrix polynomials Now ready to relate the convergence properties of Krylov Subspace methods to eigenvalues of M Krylov Methods Convergence Analysis Overview

Cond(V) Krylov Methods Convergence Analysis Norm of matrix polynomials

1) The GCR Algorithm converges to the exact solution in at most n steps 2) If M has only q distinct eigenvalues, the GCR Algorithm converges in at most q steps Krylov Methods Convergence Analysis Important observations

If M = M T then 2) M has real eigenvalues 1) M has orthonormal eigenvectors Krylov Methods Convergence Analysis Convergence for M T =M - Residual Polynomial

* = evals(M) - = 5th order poly - = 8th order poly 1 Krylov Methods Convergence Analysis Residual Polynomial Picture (n=10)

Strategically place zeros of the poly Krylov Methods Convergence Analysis Residual Polynomial Picture (n=10)

Krylov Methods Convergence Analysis Convergence for M T =M – Polynomial min-max problem

= The Chebyshev Polynomial Krylov Methods Convergence Analysis Convergence for M T =M – Chebyshev solves min-max

Chebychev Polynomials minimizing over [1,10]

Krylov Methods Convergence Analysis Convergence for M T =M – Chebyshev bounds

Krylov Methods Convergence Analysis Convergence for M T =M – Chebyshev result

For which problem will GCR Converge Faster? Krylov Methods Convergence Analysis Examples

Iteration Which Convergence Curve is GCR?

GCR Algorithm can eliminate outlying eigenvalues by placing polynomial zeros directly on them. Krylov Methods Convergence Analysis Chebyshev is a bound

Iterative Methods - CG Convergence is related to: –Number of distinct eigenvalues –Ratio between max and min eigenvalue Why ? How? Now we know

Reminder about GCR –Residual minimizing solution –Krylov Subspace –Polynomial Connection Review Eigenvalues –Induced Norms bound Spectral Radius –Spectral mapping theorem Estimating Convergence Rate –Chebyshev Polynomials Summary