2. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 1 Mathematics vs. “Heuristics”  Heuristics/Experimental Optimisation : When a functional relationship between the variables and the objective function is unknown we experiment on a real object or a scale model. Ø Experiments : Systematic investigation or a strategy ? Ø Strategy : Systematic exploitation of available information. Information can be gained during the optimisation process and it should be implemented in the strategy.  Indirect (or analytic) Methods : Attempts to reach the optimum in a single calculation step, without tests or trials. It is based on special mathematical properties of the objective function at the position of an extremum. (A mathematical strategy !)

3. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 2 Solving Optimisation Problems  Direct Methods : Solution is approached in a step by step manner (iteratively), at each step (hopefully) improving the value of the objective function. If this cannot be guaranteed proceed by trial and error.  Indirect (or analytic) Methods : Attempts to reach the optimum in a single calculation step, without tests or trials. It is based on special mathematical properties of the objective function at the position of an extremum.

4. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 3 Analytic Procedures Mac Laurin, 1742 : Necessary and sufficient conditions for a minimum Scheeffer,, 1886 : Proof for multivariable functions. Necessary Condition : Minimize the gradient - system of equations Sufficient conditions : Keep differentiating - in 1D : the lowest order non-vanishing derivative is positive and of even order _-> minimum If the derivative is negative it represents a maximum (if the order is odd we have a saddle point.) In N dimensions : The determinant of the Hessian matrix must be positive as well as the further N-1 subdeterminants of this matrix.

5. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 4 When Analytic Procedures go bad  Discontinuity of the objective function and its derivatives  Differentiation may be impossible (e.g. use of a black-box code or experiments) or inaccurate (e.g. noisy data).  Optima can be local or saddle points  Systems of equations (especially non-linear) may be non-soluble or very expensive to solve  ………………

6. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 5 Non-Gradient Based Methods  Direct Search Methods (R.Hooke & T.A. Jeeves, 1961)   We use the phrase “direct search” to describe sequential examination of trial solutions involving comparison of each trial solution with the “best” obtained up to that time together with a strategy for determining (as a function of earlier results) what the next trial solution will be.  The phrase implies our preference, based on experience, for straightforward search strategies, which employ no techniques of classical analysis, except where there is a demonstratable advantage in doing so.”

7. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 6 Direct Search Methods (DSM)  The key feature is that they do not require numerical function values : the relative rank of objective values is sufficient.  (Think of skiing - you know the way down but you do not know the exact altitude - and that is all that matters)

8. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 7 The EVolutionary OPeration Method  Developed by G.E. Box : The key feature is that they do not require numerical function values : the relative rank of objective values is sufficient.  (Think of skiing - you know the way down but you do not know the exact altitude - and that is all that matters)  EVOP was first used in a process engineering environment, due to shortage of personnel to the dynamic maximization of chemical processes. It was applied in real experiments which sometimes took place over a few years.

9. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 8 The EVolutionary OPeration Method Ø The method originally changes two or three parameters at a time (if possible those that have the strongest influence). 1. For two parameters a square is constructed with an initial condition at its center. 2. The corners represent the points in a cycle of trials 3. The corners are tested sequentially, several times if perturbations are acting. 4. The point with the best result becomes the mid-point in the next cycle. Ø(scaling can be changed, as well as choice of points to be taken for the next cycle)

10. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 9 DSM : Parallel Methods 1. Determine the function value at various points. 2. Declare point with smallest value the minimum. ØThese methods are also called grid methods or tabulation method. ØIncredibly slow - number of trial inversely proportiona,l to accuracy Ø (but parallel which was not so good when it was invented 1960’s)

11. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 10 DSM : Sequential Methods Ø In sequential methods : Ø Trials are made sequentially Ø Intermediate results are used in order to locate the next point They can be classified as : pattern search methods, simplex methods and methods with adaptive sets of search directions. Ø Examples : Boxing the minimum, Interval Division Ø Slow - number of trial proportional to log of accuracy Ø (but sequential which was good when it was invented in 1960’s).

12. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 11 DSM : Hooke and Jeeves Two types of moves :Exploratory Move - An extrapolation along the line of the first and last move before the variables are varied again individually.

13. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 12 DSM : Pattern Search Methods As quoted by Davidon (1967) : “ Enrico Fermi and Nicholas Metropolis used one of the first digital computers, the Los Alamos Maniac, to determine which values of certain theoretical parameters best fit experimental data. They varied one theoretical parameter at a time by steps of the same magnitude, and when no such increase or decrease in any one parameter further improved the fit to the experimental data, they halved the step size and repeated the process untill the steps were deemed sufficiently small. This simple procedure was slow, but sure….” Ø Pattern search methods are characterized by a series of exploratory move that consider the behaviour of the objective function at a pattern of points all of which lie on a rationale lattice. Ø The exploratory moves consist of a systematic strategy for visiting the points in the lattice in the immediate vicinity of the current iterate. The key feature is that points remain on a lattice.

14. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 13 Pattern Search Methods Converge Polak (1971) Theorem : Ø If {Xk} is a sequence constructed by the method of pattern search then any accumulation point X* satisfies that Vf(X*) = 0. Ø Key observation : Ø The method can construct only a finite number of intermediate points before reducing the step size by half. Hence the algorithm cannot jam up at a point. Ø Using the lattice property of the method global convergence results can be obtained.

15. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 14 DSM : Simplex Search Ø Simplex methods were motivated by the fact that earlier DSM’s required from 2n to 2 n objective function evaluations to complete the search for an improvement of an iterate. The key observation of Spendley, Hext and Himmworth was that it should not take more than n+1 values for the of the objective to identify a gradient (uphill/downhill) direction. Ø (it makes sense as n+1 points would be needed in order to define a gradient of a function Ø A simplex is a set of n+1 points in R n. Ø In 2D : a Triangle, In 3D a tetrahedron, etc.

16. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 15 DSM : Simplex Search Ø An initial simplex is constructed Ø Identify the vertex with the worst fitness Ø Reflect the worst vertex across the centroid of the opposite Ø The point with the best result becomes the mid-point in the next cycle. Ø( be careful so as not to keep reflecting back and

17. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 16 DSM : Simplex Search With a simplex search : Do we have a new candidate for a minimiser ? Ø Yes, when a reflected vertex produces a strict decrease on the value of the objective of the best vertex. With a simplex search are we near a minimiser ? Ø A circling sequence of simplices could indicate that a neighborhood has been identified.

18. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 17 DSM : Simplex Search Ø The simplex can be deformed/adapted so as to take into account the shape of the objective function.

19. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 18 DSM : Rosenbrock’s rotating coords.

20. Evolutionary Computation (P. Koumoutsakos) www.icos.ethz.ch 19 I. EVOLUTION STRATEGIES B contains information for the evolution path - Correlations of successful mutations - PCA of paths The environment is identified through mutation/success The (1,1) - ES Covariance Matrix Adaptation ES - (N. Hansen)