1. ZEIT4700 – S1, 2015 Mathematical Modeling and Optimization School of Engineering and Information Technology

2. Optimization - basics Maximization or minimization of given objective function(s), possibly subject to constraints, in a given search space Minimize f1(x),..., fk(x) (objectives) Subject to gj(x) < 0, i = 1,...,m (inequality constraints) hj(x) = 0, j = 1,..., p (equality constraints) Xmin1 ≤ x1 ≤ Xmax1 (variable / search space) Xmin2 ≤ x2 ≤ Xmax2.

3. Classical optimization techniques  Bracketing (Exhaustive search) / Region elimination methods  Simplex based search  Gradient based search Nelder Mead simplex method (Image source : http://upload.wikimedia.org/wikipedia/commons/9/96/Nelder_Mead2.gif) Interval halving method (Image source : K.Deb, Optimization for engineering design)

4. Classical optimization techniques (cntd.) Gradient based (Cauchy’s steepest descent method) Image source : K. Deb, Multi-objective optimization using Evolutionary Algorithms, John Wiley and Sons, 2002.

5. Drawbacks of classical optimization methods Gradient based methods : Assumptions on continuity / derivability Region elimination methods : For single variable problem only Search for local optimum only Constraint handling is not inherently included No provisions for handling multiple objectives

6. Optimization – Heuristics/meta-heuristics A heuristic is a technique which seeks good (i.e., near optimal) solutions at a reasonable computational cost without being able to guarantee either feasibility or optimality, or even in many cases to state how close to optimality a particular feasible solution is. - Reeves, C.R.: Modern Heuristic Techniques for Combinatorial Problems. Orient Longman (1993)

7. Simple “Hill climb” Start from random X (while termination criterion not met) { Perturb X to get a new point X’ If F(X’) > F(X), move to X’, else not } Maximize f(x) XX’ F(x) XX’ “Greedy” Local

8. Simulated Annealing Start from random X (while termination criterion not met) { Perturb X to get a new point X’ If F(X’) > F(X), move to X’, else Calculate P = exp(-(F(X) – F(X’))/T) move to X’ with probability P } Maximize f(x) XX’ F(x) XX’ Attempts to escape local minima by accepting occasional ‘worse’ moves

9. Genetic / Evolutionary algorithms From point-to-point methods to population based methods.. EAs are nature inspired optimization methods which search for the optimum solution(s) by evolving a population of solutions. Require no assumptions on differentiability / continuity of functions, hence can handle much more complex functions as compared to classical optimization techniques. Can deliver the whole Pareto Optimal Front in a single run as opposed to conventional methods. Its an Intelligent hit and trial !

10. Evolutionary Algorithms (EA) Initialization (population of solutions) Parent selection Recombination / Crossover Mutation Ranking (parent+child pop) Reduction Termination criterion met ? Yes No Output best solution obtained “Evolve” childpop Evaluate childpop

11. Gen 1 Gen 25 Gen 50 Gen 100 Evolutionary Algorithms (contd.)

12. Evolutionary Algorithm (cntd) Minimize f(x) = (x-6)^2 0 ≤ x ≤ 31 Binary GAReal Parameter GA RepresentationBinaryReal Parent selectionBinary tournatment/ Roulett wheel Binary tournatment/ Roulett wheel CrossoverOne point/multi-pointSBX,PCX … MutationBinary flipPolynomial

13. Resources Course material and suggested reading can be accessed at http://seit.unsw.adfa.edu.au/research/sites/mdo/ Hemant/design-2.html http://seit.unsw.adfa.edu.au/research/sites/mdo/ Hemant/design-2.html