Consider Preconditioning – Basic Principles Basic Idea: is to use Krylov subspace method (CG, GMRES, MINRES …) on a modified system such as The matrix need not be formed explicitly; only need to solve whenever needed. With: requirement is that it should be easy to solve for an arbitrary vector z.

Left, Right, and Symmetric Preconditioners Left preconditioning : Right preconditioning : Symmetric preconditioning :

Left, Right, and Symmetric Preconditioners Symmetric preconditioning : The matrix in brackets is symmetric positive definite. ( so we can use CG) The matrix in brackets is similar to It is enough to examine the eigenvalues of the nonsymmetric matrix to investigate convergence. No loss of symmetry

Example Example: CG iterations converges slowly without preconditioner. This is better than direct method

Example Example: Preconditioner: No precond: after 40 iterations, achieves about 5-digit residual reduction. PCG: after 30 iterations, achieves 15-digits

Practical Implementations Consider why No…NO?? Loss of sparsity, matrix multiplication, finding inverse CG

Practical Implementations CGPCG-ver1

Practical Implementations PCG(ver1)PCG(ver2)

Practical Implementations PCG(ver2)PCG

Good Preconditioner Good Preconditioner M: 1)Few number of iterations (fast convergence) 2) Cheap to solve the system Mx = y textbook The preconditioners used in practice are sometimes as simple as this one (diagonal), but they are often far more complicated.

Example: Five items Item 1: (minimal polynomial with small degree) DEF: p(x) is the minimal polynomial of the nxn matrix A if p(x) is the monic polynomial of least degree such that p(A)=0. A = Characteristic polynomial minimal polynomial

Example: Item 1: (minimal polynomial with small degree) DEF: p(x) is the minimal polynomial of the nxn matrix A if p(x) is the monic polynomial of least degree such that p(A)=0. A = minimal polynomial WHY? Five items

Item 2: (A with few distinct eigenvalues) Note and why: Item 3: (small condition number) Note: it is only one way ( large condition number ) Example: Five items CG GMRES

Item 4: (residual and error) Five items The inequality implies that condition number provide an indication of the connection between the residual vector and the accuracy of the approximation In general, the relative error is bounded by the product of condition number with the relative residual

Item 5: (A with a good clustering of the eigenvalues) Textbook: A preconditioner M is good if M^(-1)A is not too far from normal and its eigenvalues are clustered. Five items

Item 6: (M is close enough to A) Textbook: If the eigenvalues of are close to 1 and is small, then any of the iterations we have discussed can be expected to converge quickly. Five items

Survey of Preconditioners Textbook: The preconditioners used in practice are sometimes as simple as this one (diag(A)), but they are often far more complicated. 1) Diagonal Scaling: Choose the preconditioner to be M = diag(c) where c is a suitable vector with nonzero entries. Problem:

Survey of Preconditioners 2) Incomplete Cholesky Factorization: (IC) Incomplete Cholesky factorization Sparse, SPD Incomplete Cholesky conjugate Gradient Method 3) Incomplete LU Factorization: (ILU) Incomplete LU factorization nonsymmetric Use GMRES with M as preconditioner

Survey of Preconditioners 4) Local Approximation The matrix A represents coupling between elements both near and far from one another. It may be worth considering M analogous to A but with the longer- range interactions omitted – a short-range approximation A. In the simplest cases of this kind, M may consist simply of a few of the diagonals of A near the main diagonal, making this a generalization of the idea of a diagonal preconditioner.

Survey of Preconditioners 5) Block Preconditioners This is another kind of local approximation, in that local effects within certain components are considered while connections to other components are ignored.

Survey of Preconditioners 7) Constant-coefficient approximation PDE with variable coefficient Use the discritezed matrix as a preconditioner for the first problem 8) symmetric approximation If a differential equation is not self-adjoint but is close in some sense to a self-adjoint equation that can be solved more easily, then the latter may sometimes sevre as a preconditioner

Survey of Preconditioners 6) Domain Decomposition In which solvers for certain subdomains of a problem are composed in flexible ways to form preconditioners for the global problem. This method combine mathematical power with natural paralleizability

Survey of Preconditioners 6) Domain Decomposition In which solvers for certain subdomains of a problem are composed in flexible ways to form preconditioners for the global problem. This method combine mathematical power with natural paralleizability Note: the problem can be parallized

Survey of Preconditioners 7) Low-order discertization Often a differential or integral equation is discretized by a higher-order method. Bringing a gain in accuracy but making the discretization stencils bigger and the matrix less sparse. A lower-order approximation of the same problem, with its sparser matrix, may be an effective preconditioner. 9-point formula. 5-point formula

Survey of Preconditioners 8) saddle-point system 9) Generalized saddle-point system

Survey of Preconditioners 10) Polynomial preconditioner

Survey of Preconditioners 11) Splitting Many applications involve combinations of physical effects, such as the diffusion and convection that combine tp make up the Navier-Stokes equations of fluid mechanics. Example: Laplacian in two or three dimensions is composed of analogous operators in each of the dimensions separately. This idea may form the basis of a preconditioner.