1. Bio-Inspired Optimization

2. Our Journey – For the remainder of the course A brief review of classical optimization methods The basics of several stochastic optimization methods evolutionary algorithms (EA) artificial neural networks (ANN) particle swarm optimization (PSO) ant colony optimization (ACO) The basic biology and physics behind the methods How to implement and apply the methods

3. http://www.youtube.com/watch?v=A042J0IDQK4

4. Optimization: Highest achievable performance by maximizing desired factors and minimizing undesired ones Maximization: achieving the highest result regardless of cost or expense $$ Quality of life Etc.

5. Optimization In general, optimization is the problem of finding the (global) minimum (or maximum) of an objective function Sometimes (but not always!) the objective function is a specific, well-defined mathematical function F = f(x1,x2, x3,...)

6. Optimization methods Optimization methods can be divided into two broad categories Classical (deterministic) optimization methods Stochastic optimization methods Classical optimization (Covered in ME517) Gradient descent (following the steepest slope) Newton’s method Penalty methods Lagrange multiplier methods

7. Optimization methods Classical optimization (Covered in ME517) Gradient descent (following the steepest slope) http://www.youtube.com/watch?v=HvLJUsEc6dw Newton’s method Penalty methods Lagrange multiplier methods

8. Limitations of classical optimization Classical methods are less useful in cases with non-differentiable objective functions objective functions whose values can only be obtained as a result of a (lengthy) simulation Varying number of variables (as in optimization of neural networks) For such problems, stochastic optimization methods are more suitable.

9. Stochastic optimization methods Stochastic optimization algorithms can be used for solving many problems where classical methods are insufficient More frequently used in industry, particularly in large and complex problems The number of application areas is steadily increasing As the name implies, stochastic optimization methods contain an element of stochasticity (randomness) Many stochastic optimization methods are inspired from biological phenomena An important subset of stochastic optimization methods are biologically inspired optimization methods.

10. Biologically inspired optimization Discussion Why would someone use inspiration from biology when defining an optimization method? What are good examples?

11. Biologically inspired optimization characteristics In comparison with other optimization algorithms, bio-Inspired optimization algorithms have the following characteristics: The individual components are distributed and autonomous, there is no central control, and the fault of an individual cannot influence solving the whole problem, these characteristics ensure this kind of algorithms is more robust Is the individual intelligent?

12. Biologically inspired optimization characteristics They don’t demand to meet the requirement of differentiability, convexity, and other conditions for mathematical description of the problem This is because of the random search Use basic mathematical operations, therefore, they are simple and easy to be implemented computationally These advantages enabled bio-mimetic optimization algorithms to be widely used in a very short period, such as power system, vehicle routing, mechanical design, robotics, etc.

13. Biologically inspired optimization Nature is all about adaptation, which can be seen as a kind of optimization Finding the ideal solution for a specific case (types of eyes) However, note that in nature the target is constantly moving – unlike the case in engineering problems Moving surface

14. Example 1: Shark skin A good example of adaptation (biological optimization) The skin of sharks has evolved to allow low friction very fast swimming.

15. Example 1: Shark skin The rib-like structures affect the interaction between the surface of the shark and the surrounding water, essentially reducing drag Prime example of biological inspiration for engineering concepts: Similar ideas are being applied in order to reduce drag (and noise) of aircraft and other vehicles Shark skin has also inspired the development of the fabric used in swimsuits worn by athletes

16. Example 2: Evolution of the eye The evolution of eyes is another interesting example Faced with the problem of generating light-gathering devices, evolution has come up with no less than 40 completely independent solutions to the problem

17. Example 3: Swarming Some species (particularly ants, bees, and termites) display very advanced forms of cooperation Specific example: Weaver ants (Oecophylla)

18. Example 4: Swarming Many organisms (e.g. some bird and fish species) display swarming behavior for protection against predators, efficient food gathering, etc...

19. Biologically inspired optimization Several different optimization methods have been developed, based on biological phenomena: Evolutionary algorithms (inspired by evolution) Artificial neural networks (inspired by the brain) Ant colony optimization (inspired by cooperative behavior) Particle swarm optimization (inspired by swarming)

20. EA

21. Brief introduction to EAs Evolutionary algorithms = EAs. Based on darwin evolution, which is a process involving gradual, hereditary changes of biological organisms, over long periods of time Basic biological concepts: Population, fitness, selection, mutation Individuals that are well adapted to their environment have a high probability of generating offspring

22. Brief introduction to EAs Random changes to the genotype (mutations), sometimes leading to large changes of the phenotype. Mutations provide new material for evolution to work with Mutations are random, however selection is not

23. Basic mode of operation of EAs In an EA, a population of candidate solutions (to the problem at hand) is formed (random initialization) Each individual in the population is evaluated and assigned a fitness score based on its performance. New individuals are formed through the processes of selection, crossover, and mutation. The new individuals form the second generation, which is evaluated in the same way as the first generation etc. The process is repeated until a satisfactory solution has been found to the problem at hand.

24. Evolutionary Algorithms Encoding: the variables of the problem are encoded in strings of digits known as chromosomes. The exact encoding procedure varies from problem to problem – in the simple case of optimization of a function f(x1,x2,x3,...xn) the variables xi, i = 1,...,n should be encoded. In more complex problems (e.g. optimization of artificial brains for robots or optimization of neural networks), the specification of the encoding procedure can be much more complicated.

25. Evolutionary Algorithms Random initialization of a population containing N chromosomes

26. Evolutionary Algorithms Crossover Mutation

27. Evolutionary Algorithms

29. Introduction to Genetic Algorithms http://www.youtube.com/watch?v=zwYV11a__HQ

30. Optimization Project – More details on wiki 1. Describe the problem. Identify the specific performance characteristics that are being min/max-imized against each other. Contrast classical and stochastic optimization, and identify which is more appropriate for your optimization problem. 2. Identify a suitable stochastic optimization algorithm, and explain why you chose it. (You may want to do some outside research.) 3. Solve the optimization problem using a programming language

31. The fun begins: Boxcar2d.com