1. Powell ’ s method 박성진

2. Overview Minimizing a function of multiple variables Proceeding from starting point in some vector direction n -> minimizing f(p) along the line n by one- dimensional methods Critical part : Choosing the next direction n Not computing the function ’ s gradient

3. Simple method Taking the unit vectors as a set of directions Moving along the first direction to its minimum, then from there along the second direction to its minimum Stop criteria : stop decreasing of function value Problem : in case of long, narrow valley at some angle to the coordinate basis vectors, requiring a series of many tiny steps Solution : needing a better set of directions than the unit vectors

4. Starting point

5. Powell ’ s quadratically convergent method Save starting point as For i=1, …,N, move to the minimum along direction and call this point For i=1, …,N-1, set Set Move to the minimum along direction and call this point

6. Starting point

7. Powell ’ s quadratically convergent method Problem : at each stage, throwing in favor of -> becoming linearly dependent solution : Reinitializing the set of directions to the basis vectors after every iterations Resetting the direction set to calculated principle directions of the matrix A Giving up the property of quadratic convergence -> finding a few good directions along narrow valleys

8. Modified Powell ’ s method Keeping the old set of directions in two cases Assume : If The average direction is played out If Not primary direction due to any single direction ’ s decrease large substantial second derivative

9. Starting point

10. Code void powell(float p[], float **xi, int n, float ftol, int *iter, float *fret, float (*func)(float [])) { void linmin(float p[], float xi[], int n, float *fret, float (*func)(float [])); int i,ibig,j; float del,fp,fptt,t,*pt,*ptt,*xit; pt=vector(1,n); ptt=vector(1,n); xit=vector(1,n); *fret=(*func)(p); for (j=1;j<=n;j++) pt[j]=p[j]; Save the initial point. for (*iter=1;;++(*iter)) { fp=(*fret); ibig=0; // ibig : 가장 큰 감소를 이룬 direction 의 인덱스 del=0.0; for (i=1;i<=n;i++) { for (j=1;j<=n;j++) xit[j]=xi[j][i]; fptt=(*fret); linmin(p,xit,n,fret,func); // minimize along it, if (fptt-(*fret) > del) { del=fptt-(*fret); so far. ibig=i; } if (2.0*(fp-(*fret)) <= ftol*(fabs(fp)+fabs(*fret))+TINY) { free_vector(xit,1,n); Termination criterion. free_vector(ptt,1,n); free_vector(pt,1,n); return; } if (*iter == ITMAX) nrerror("powell exceeding maximum iterations."); for (j=1;j<=n;j++) { ptt[j]=2.0*p[j]-pt[j]; xit[j]=p[j]-pt[j]; // xit : average direction pt[j]=p[j]; } fptt=(*func)(ptt); Function value at extrapolated point. if (fptt < fp) { t=2.0*(fp-2.0*(*fret)+fptt)*SQR(fp-(*fret)-del)-del*SQR(fp-fptt); if (t < 0.0) { linmin(p,xit,n,fret,func); for(j=1;j<=n;j++) { xi[j][ibig]=xi[j][n]; // 가장 큰 감소를 이룬 direction 제거 xi[j][n]=xit[j]; }