Cojugate Gradient Method Zhengru Zhang ( 张争茹 ) Office: Math. Building 413(West) 2010 年教学实践周

Outline Aim Method of Gauss Elimination Basic Iterative Methods Conjugate Gradient Method –Derivation –Theory –Algorithm References Homework & Project

Aim Solve linear algebraic system like a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2... a n1 x 1 + a n2 x a nn x n = b n Using matrix, the above system can be written as Ax=b A is a N x N matrix, b is a N x 1 vector Consider the case: A is large and sparse

Method of Gauss Elimination

U = A, L = I for k = 1 to N-1 for j = k +1 to N l jk = u jk /u kk u j,k:m = u j,k:m – l jk u k,k:m Algorithm of Gaussian Elimination without Pivoting LU Factorization, let A=LU Solve Ly=b Solve Ux=y

Operation Count of Gauss Elimination Gauss Elimination and Back Substitution There are 3 loops There are 2 flops per entry For each k, the inner loop is repeated for rows k +1, …, N Cost: about About N 3 flops

Instability of Gaussian Elimination without Pivoting A1=A1= A2=A2= Examples Remedy Pivoting Partial Pivoting Complete Pivoting

Algorithm of Gaussian Elimination with Partial Pivoting U = A, L = I For k = 1 to N-1 for j = k +1 to N l jk = u jk /u kk u j,k:m = u j,k:m – l jk u k,k:m

Basic Iterative Methods How to construct iterative sequence? Convergence? Conditions? Convergence rate?

Jacobi iteration X [k+1] = D -1 (L+U) X [k] + D -1 b B = D -1 (L+U) Gauss Seidel iteration X [k+1] = (D-L) -1 U X [k] + (D-L) -1 b B = (D-L) -1 U Iterative method X [k+1] = BX [k] +g converges if and only if  (B) < 1 Convergence rate ||X [k] -X * ||  ||X [1] -X [0] ||, where q =||B||<1

Steepest Decent Method Consider the case: A is symmetric positive definite Quadratic functional  (x) = x T Ax - 2b T x The solution of Ax=b is equivalent to find the minimizer of the functional  (x) Method of optimization: find a direction p k and a step  k

Steepest Decent Method Determine p k and  k Suppose that p k is determined. Let’s start from x k Let f(  ) =  (x k +  p k ) = (x k +  p k ) T A(x k +  p k )-2b T (x k +  p k ) =  2 p k T Ap k - 2  r k T p k +  (x k ) where r k = b - Ax k (Residual) By calculas f ’ (  ) = 2  p k T Ap k - 2r k T p k =0 Then let x k+1 = x k +  k p k

Algorithm for Steepest Decent Method Verify  (x k+1 ) -  (x k ) =  (x k +  k p k ) -  (x k ) =  k 2 p k T Ap k - 2  k r k T p k How to determine the direction p k ? take as the negative gradient p k = r k

Algorithm Convergence Theorem Suppose the eigenvalues of A then there holds where

Conjugate Gradient Method Derivation Negative gradient direction r k is the locally steepest decent direction, but it may not be the global one Consider a new direction: combination of r k and p k-1 Initially, take p 0 = r 0, x 1 = x 0 +  0 p 0 For step k +1, choose  and  to minimize By calculas

The corresponding minimizer is  0 and  0 satisfy take Let In summary, 0

Algorithm for CG method Operations involved: Transpose, Scalar Multiply, Matrix Add, Matrix Multiply Where  and  are obtained in a simple form

Properties for CG method Orthogonal properties Theoretically, CG method is an exact method. Actually, works as an iterative method. Convergence rate: where

References 徐树方，高立，张平文， 数值线性代数，北京大学出版社，北京， 2007 袁亚湘，孙文瑜， 最优化理论与方法，科学出版社，北京， 2000 Yousef Saad, Iterative Methods for Sparse Linear Systems, 2000

Home & Project Due at the end of this week Solve the following linear systems using CG method where Set n = 100, 200, 300, 400, 500 Use Matlab to graph the solution (j, u j ) Problem: Minimize the functional E(u)=∫(|  u| 2 +u 2 -2fu )dx The corresponding Euler-Lagrange equation is  E/  u=-2  u+2u-2f=0 or -  u+u=f -u xx + u =f 0<x<1 f=(1+4  2 )sin2  x u(0)=u(1)=0

Home & Project Due at the end of this week Solve the following linear systems using CG method The unknowns can be ordered as below

The coefficient matrix is of the block tridiagonal form Where S Tridiagonal matrix with diagonal entry: other entry: Set n=20,40,80,100. Find the solution Use Matlab to graph the solution (i, j, u ij ) -  u+u=f (x,y)  (0,1)  (0,1) u(x,y)=100(x 2 -x)(y 2 -y) f=200(y-y 2 ) + 200(x-x 2 ) + 100(x 2 -x)(y 2 -y)

The End