Download presentation
Presentation is loading. Please wait.
Published byBonnie Owens Modified over 6 years ago
1
Conjugate Gradient Problem: SD too slow to converge if NxN H matrix is ill-conditioned. SD: dx = - g (slow but no inverse to store or compute) CG: dx = -p (fast but no inverse to compute+store) GN: dx = -H-1 g (fast but expensive) Solution: Conjugate Gradient converges in N iterations if NxN H is S.P.D. Quasi-Newton Condition: g’ – g = Hdx’ (g’-g)/dx’= d2g/dx2
2
Outline CG Algorithm Step Length: Polak-Ribiere vs Fletcher-Reeves
CG Soln to Even & Overdetermined Equations Regularized CG Preconditioned CG Non-Linear CG
3
Conjugate Gradient . dxT g=0 -g dx
Quasi-Newton Condition: g’ – g = Hdx’ (1) g’ dx’Tg’= dx’T f(x*)T = 0 D dxT g=0 . dx’ x* -g dx’ dx’ dx’ Kiss point dx For dx’ at the bullseye x*, g’=0 so eqn. 1 becomes, after multiplying by dx and recalling dxT g=0, dxT (g’-g)=0 zero at bullseye. Hence, Conjugacy Condition: 0 = dxTHdx’ (2) x’ = x a p (where p is conjugate to previous direction) (3)
4
(no longer going downhill)
Conjugate Gradient Quasi-Newton Condition: g’ – g = Hdx’ (1) Conjugacy Condition: 0 = dxHdx’ (2) x’ = x a p (where p is conjugate to previous direction and a linear combo of dx & g) (3) For i = 1:nit end 0 = dxT H(bdx - g) Solve for b s.t. dx conjugate to dx’ find b find a p { dxTHdx dxT Hg b = p= bdx - g x* g dx’ dx’ = dx + ap x=x+ dx’ Solve for a s.t. dx’ kisses contour (no longer going downhill) Kiss point dx dxTHdx dxT g a =
5
Conjugate Gradient For i = 1:nit find b p= bdx - g find a
dxTHdx dxT Hg b = find b p= bdx - g dxTHdx dxT g a = find a dx’ = dx + ap x=x+ dx’ end Recall, aHd (k-1) = g(k) - g(k-1)
6
Outline CG Algorithm Step Length: Polak-Ribiere vs Fletcher-Reeves
CG Soln to Even & Overdetermined Equations Regularized CG Preconditioned CG Non-Linear CG
7
Conjugate Gradient For i = 1:nit find b p= dx + bg find a
dx’ = dx + ap For i = 1:nit find b find a p= dx + bg x=x+ dx’ end dxTHdx dxT Hg b = dxT g a = Fletcher-Reeves Polak-Ribierre Not going downhill if moving perpindicular to gradient -g
8
Outline CG Algorithm Step Length: Polak-Ribiere vs Fletcher-Reeves
CG Soln to Even & Overdetermined Equations Regularized CG Preconditioned CG Non-Linear CG
9
Conjugate Gradient: Lx=d
dxT g=0 . dx’ x* -g dx’ dx’ dx’ Kiss point dx Conjugate Gradient: Lx=d
10
x* -g dk1 Kiss point dk
11
Conjugate Gradient: LTLx=LTd
Compared to square system of equations, the gradient for overdetermined system of equations has an extra LT However, LLT has squared condition number
12
Conjugate Gradient Convergence
Well conditioned In most dimensions Poorly conditioned In every dimension If NxN H is linear SPD then convergence in N iterations, but in practice much sooner. Stopping sooner is a form of regularization by excluding small eigenvalue components
13
Outline CG Algorithm Step Length: Polak-Ribiere vs Fletcher-Reeves
CG Soln to Even & Overdetermined Equations Regularized CG Preconditioned CG Non-Linear CG
14
Regularized Conjugate Gradient
Balance between solution That minimizes misfit and one that minimizes penalty
15
Outline CG Algorithm Step Length: Polak-Ribiere vs Fletcher-Reeves
CG Soln to Even & Overdetermined Equations Regularized CG Preconditioned CG Non-Linear CG
16
Preconditioned Conjugate Gradient
Find a cheap approximate inverse P~H-1 so that PH~I. Thus, Ill-conditioned system of equations: Hx=-g Well-conditioned system of equations: PHx=-Pg PHx=-Pg A cheap approximate inverse is [H-1]ii ~ 1/Hii . Warning: PH should be SPD
17
Outline CG Algorithm Step Length: Polak-Ribiere vs Fletcher-Reeves
CG Soln to Even & Overdetermined Equations Regularized CG Preconditioned CG Non-Linear CG
18
Non-linear Conjugate Gradient
Reset to gradient direction after every approximately 3-5 iterations Locally quadratic
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.