2. Graphical optimization Some problems are cheap to simulate or test. Even if they are not, we may fit a surrogate that is cheap to evaluate. Relying on optimization software to find the optimum is foolhardy. It is better to thoroughly explore manually. In two dimensions, graphical optimization is a good way to go. For problems with more variables, it may still be good to identify two important variables and proceed graphically. In higher dimensions, two dimensional cuts can help understand the relationship between local optima.

3. Example Plot and estimate minimum of In the range Can do contour plot or mesh plot x=linspace(-2,0,40); y=linspace(0,3,40); [X,Y]=meshgrid(x,y); Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2; cs=contour(X,Y,Z); cs=surfc(X,Y,Z); xlabel('x_1'); ylabel('x_2'); zlabel('f(x_1,x_2)');

4. Then zoom Zooming by reducing the range does not change the estimate of the position (-1,1.5), but the value is now below 0.8. x=linspace(-1.5,-0.5,40); y=linspace(1,2,40); [X,Y]=meshgrid(x,y); Z=2+X-Y+2*X.^2+2*X.*Y+Y.^2

5. In higher dimensions Often different solutions of optimization problems are obtained from multiple starting points or even from same starting point. It is tempting to ascribe this to local optima. Often it is the result of algorithmic or software failure. Generate line connecting two solutions

6. Example Consider Two candidate local optima (0,0,0) and (0.94,0.94,0.94). Line connecting first and second point indicates that they may be local optima. alpha=linspace(0,1,101); x1=[0 0 0]; x2=[0.94 0.94 0.94]; x=x2(1)*alpha+x1(1)*(1-alpha); y=x2(2)*alpha+x1(2)*(1-alpha); z=x2(3)*alpha+x1(3)*(1-alpha); f=abs(sin(2*pi.*x).*sin(2*pi.*y).* sin(2*pi.*z))+x.*sqrt(y).*z.^2; plot(alpha,f);

7. Two-dimensional cut

8. Example Optimize g=@(x) (sin(2*pi*x(1))*sin(2*pi*x(2))*sin(2*pi*x(3)))+x(1)*sqrt(x(2))*x(3)^2; X=fminsearch(@(x) g(x), [0,0,0 ])

9. Unconstraiend optimization in Matlab [X,gval,exitflag]=fminsearch(@(x) g(x), [0,0,0]) X =0.2492 0.2496 -0.2485 gval =-0.9923 exitflag =1 [X,gval,exitflag]=fminsearch(@(x) g(x), [0.5,0.5,0.5]) X =0.2487 0.7498 0.2473 gval =-0.9867 exitflag =1 1 Maximum coordinate difference between current best point and other points in simplex is less than or equal to TolX, and corresponding difference in function values is less than or equal to TolFun. 0 Maximum number of function evaluations or iterations reached. -1 Algorithm terminated by the output function. Can set convergence criteria in optimset.

10. Alternative using derivatives [X,gval,exitflag]=fminunc(@(x) g(x), [0.24,0.24,-0.24]) Warning: Gradient must be provided for trust-region algorithm; using line-search algorithm instead. > In fminunc at 383 Local minimum found. Optimization completed because the size of the gradient is less than the default value of the function tolerance. X =0.2492 0.2496 -0.2484 gval = -0.9923 exitflag =1 [X,gval,exitflag]=fminunc(@(x) g(x), [0,0,0]) Warning: Gradient must be provided for trust-region algorithm; using line-search algorithm instead. > In fminunc at 383 Initial point is a local minimum. Optimization completed because the size of the gradient at the initial point is less than the default value of the function tolerance. X = 0 0 0 gval = 0 exitflag = 1

11. 10 dimensional constrained example Aerodynamic design of supersonic transport. Objective function, take-off gross weight is a cheap and simple function of design variables. Design variables define geometry of wing and fuselage. Constraints include range, take-off and landing constraints, and maneuvrability constraints. The constraints are all uni-modal functions, but in combination, they create a complex, non-convex feasible domain.

12. Simple constraints create non-convex design space. Can that happen with linear constraints? Two local optima and a third point

13. Problems