1. Selected Topics in Evolutionary Algorithms II Pavel Petrovič Department of Applied Informatics, Faculty of Mathematics, Physics and Informatics ppetrovic@acm.org July 10 th 2008

2. 2 Evolutionary Computation Search for solutions to a problem Solutions uniformly encoded Fitness: objective quantitative measure Population: set of randomly generated solutions Principles of natural evolution:  selection, recombination, mutation Run for many generations Selected Topics in Evolutionary Algorithms II, July 10 th 2008

3. 3 Solving problems with EA Define and implement representation Define and implement objective function Design and implement initialization, mutation and recombination operators Select appropriate algorithm and selection method Setup and tune evolutionary parameters:  Mutation rate  Crossover rate  Population size  Selection parameters  Termination criterion Selected Topics in Evolutionary Algorithms II, July 10 th 2008

4. 4 EA Concepts genotype and phenotype fitness landscape diversity, genetic drift premature convergence exploration vs. exploitation selection methods: roulette wheel (fit.prop.), tournament, truncation, rank, elitist selection pressure direct vs. indirect representations fitness space Selected Topics in Evolutionary Algorithms II, July 10 th 2008

5. 5 Genotype and Phenotype Genotype – all genetic material of a particular individual (genes)‏ Phenotype – the real features of that individual Selected Topics in Evolutionary Algorithms II, July 10 th 2008

6. 6 Fitness landscape Genotype space – difficulty of the problem – shape of fitness landscape, neighborhood function Selected Topics in Evolutionary Algorithms II, July 10 th 2008

7. 7 Population diversity Must be kept high for the evolution to advance Selected Topics in Evolutionary Algorithms II, July 10 th 2008

8. 8 Premature convergence important building blocks are lost early in the evolutionary run Selected Topics in Evolutionary Algorithms II, July 10 th 2008

9. 9 Premature convergence Selected Topics in Evolutionary Algorithms II, July 10 th 2008

10. 10 Genetic drift Loosing the population distribution due to the sampling error Selected Topics in Evolutionary Algorithms II, July 10 th 2008

11. 11 Exploration vs. Exploitation Exploration phase: localize promising areas Exploitation phase: fine-tune the solution Selected Topics in Evolutionary Algorithms II, July 10 th 2008

12. 12 Selection methods roulette wheel (fitness proportionate selection), tournament selection truncation selection rank selection elitist strategies Selected Topics in Evolutionary Algorithms II, July 10 th 2008

13. 13 Selection pressure Influenced by the problem Relates to evolutionary operators Selected Topics in Evolutionary Algorithms II, July 10 th 2008

14. 14 Direct vs. Indirect Representations Selected Topics in Evolutionary Algorithms II, July 10 th 2008

15. 15 Fitness Space (Floreano) ‏ Functional vs. behavioral Explicit vs. implicit External vs. internal Selected Topics in Evolutionary Algorithms II, July 10 th 2008

16. 16 Evolutionary Robotics Solution: Robot’s controller Fitness: how well the robot performs Simulation or real robot Selected Topics in Evolutionary Algorithms II, July 10 th 2008

17. 17 Fitness Influenced by Robot’s abilities (sensors, actuators)‏ Incremental change during evolution: Incremental Evolution Task difficulty Environment difficulty Controller abilities T Robot Morphology Selected Topics in Evolutionary Algorithms II, July 10 th 2008

18. 18 Evolvable Tasks Wall following Obstacle avoidance Docking and recharging Artificial ant following Box pushing Lawn mowing Legged walking T-maze navigation Foraging strategies Trash collection Vision discrimination and classification tasks Target tracking and navigation Pursuit-evasion behaviors Soccer playing Navigation tasks Selected Topics in Evolutionary Algorithms II, July 10 th 2008

19. 19 Evolutionary algorithms Genetic algorithm Genetic programming Evolutionary Strategies Evolutionary Programming Classifier systems Ant-colony optimisation Memetic algorithms Artificial Immune Systems Selected Topics in Evolutionary Algorithms II, July 10 th 2008

20. 20 Example: Travelling Salesman Problem (TSP) ‏ Finding a closed path that visits all cities Difficult problem (NP-complete)‏ Selected Topics in Evolutionary Algorithms II, July 10 th 2008

21. 21 Example: Travelling Salesman Problem (TSP) ‏ Trivial representation:( 4, 1, 7, 2, 5, 3, 6 ) - list of cities visited Representation is a permutation, however standard crossover results in descendants that are not permutations Not suitable for standard recombination Need a different representation or recombination! Selected Topics in Evolutionary Algorithms II, July 10 th 2008

22. 22 TSP Example: Partially matched crossover (PMX) ‏ 2 sites picked, intervening section specifies “cities” to interchange between parents: A = 9 8 4 | 5 6 7 | 1 3 2 10 B = 8 7 1 | 2 3 10 | 9 5 4 6 A’ = 9 8 4 | 2 3 10 | 1 6 5 7 B’ = 8 10 1| 5 6 7 | 9 2 4 3 some ordering information from each parent is preserved, and no infeasible solutions are generat Selected Topics in Evolutionary Algorithms II, July 10 th 2008

23. 23 TSP Example: Order Crossover (OX) ‏ 2 sites picked, intervening section specifies “cities” to interchange between parents: A = 9 8 4 | 5 6 7 | 1 3 2 10 ‏ B = 8 7 1 | 2 3 10 | 9 5 4 6 B* = 8 H 1 | 2 3 10 | 9 H 4 H B** = 2 3 10 | H H H | 9 4 8 1 B’ = 2 3 10 | 5 6 7 | 9 4 8 1 A’ = 5 6 7 | 2 3 10 | 1 9 8 4 Order crossover preserves more information about RELATIVE ORDER than does PMX, but less about ABSOLUTE POSITION of each “city” (for TSP example)‏ Selected Topics in Evolutionary Algorithms II, July 10 th 2008

24. 24 TSP Example: Operator MPX 2 sites picked, intervening section specifies “cities” to interchange between parents: A = 9 8 4 | 5 6 7 | 1 3 2 10 B = 8 7 1 | 2 3 10 | 9 5 4 6 C = 5 7 1 | 2 3 10 | 9 8 6 4 D = 6 4 1 | 2 3 10 | 9 5 7 8 C' = 5 | 5 6 7 | 7 1 | 2 3 10 | 9 8 6 4 D' = 6 4 1 | 2 3 10 | 9 5 | 5 6 7 | 7 8 C'' = * | 5 6 7 | * 1 | 2 3 10 | 9 8 * 4 C''' = 5 6 7 1 2 3 10 9 8 4 Selected Topics in Evolutionary Algorithms II, July 10 th 2008

25. 25 TSP Example: Cyclic Crossover CX Cycle crossover forces the city in each position to come from that same position on one of the two parents: A = 9 8 2 1 7 4 5 10 6 3 B = 1 2 3 4 5 6 7 8 9 10 A' = 9 - - - - - - - - - 9 - - 1 - - - - - - 9 - - 1 - 4 - - 6 - 9 2 - 1 - 4 - 8 6 10 A'' = 9 2 3 1 - 4 - 8 6 10 = 9 2 3 1 7 4 5 8 6 10 A''' = 9 2 3 1 5 4 7 8 6 10 Selected Topics in Evolutionary Algorithms II, July 10 th 2008

26. 26 Multiple-objective optimisation Several objectives to optimize Usually no single optimal solution Decision maker selects a solution from finite set by making compromises First MOEAs in mid 80s, since then huge number of papers on EMOO EAs are good for MOO: Inherently parallel Less susceptible to the shape or continuity of MO search space Selected Topics in Evolutionary Algorithms II, July 10 th 2008

27. 27 Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008

28. 28 Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008

29. 29 Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008 P current (t)‏ P known (t)‏ P true (t)‏

30. 30 Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008 MOEA is an extension on an EA in which two main issues are considered: How to select individuals such that nondominated solutions are preferred over those which are dominated How to maintain diversity as to be able to maintain in the population as many elements of the Pareto optimal set as possible.

31. 31 Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008 Preference of nondominated solutions: All non-dominated individuals get the same probability to reproduce This probability is higher than the one corresponding to the individuals which are dominated = PARETO RANKING

32. 32 Multiple-objective optimisation Selected Topics in Evolutionary Algorithms II, July 10 th 2008 Maintaining diversity: Fitness sharing Niching Clustering Geographically-based schemes to distribute solutions Use of entropy

33. 33 Multiple-objective EAs Selected Topics in Evolutionary Algorithms II, July 10 th 2008 Aggregating functions combining objectives into single fitness: cannot generate non-convex portions of the Pareto front regardless of the weight combination used

34. 34 Multiple-objective EAs Selected Topics in Evolutionary Algorithms II, July 10 th 2008 Population-based approaches concept of Pareto dominance is not directly incorporated into the selection process population of an EA is used to diversify the search VEGA = Vector Evaluated Genetic Algorithm At each generation, a number of sub-populations are generated by performing proportional selection according to each objective function in turn Problem: selection scheme is opposed to the concept of Pareto dominance

35. 35 Multiple-objective EAs Selected Topics in Evolutionary Algorithms II, July 10 th 2008 Pareto-Based Approaches Goldberg's Pareto Ranking Multi-Objective Genetic Algorithm (MOGA)‏ The Nondominated Sorting Genetic Algorithm (NSGA)‏ NSGA II = NSGA + elitism & crowded comparison operator (makes the search faster)‏ Niched Pareto Genetic Algorithm (NPGA) – tournament Strength Pareto Evolutionary Algorithm (SPEA) – special clustering method to maintain diversity SPEA2 – different clustering method (nearest neighbor)‏ many other...

36. 36 Neuroevolution through augmenting topologies (NEAT) ‏ The most successful method for evolution of artificial neural networks Sharing fitness Starting with simple solutions Global counter i.e. Topological crossover – very important for preserving evolved structures Selected Topics in Evolutionary Algorithms II, July 10 th 2008

37. 37 GECCO Contest GECCO is the largest EA conference (European alternative: PPSN)‏ Humies awards Contest tasks with prizes... Selected Topics in Evolutionary Algorithms II, July 10 th 2008

38. 38 Further information... Conferences: GECCO, PPSN, CEC (now part of WCCI, EvoWorkshops, EA) Journals: Evolutionary Computation, Genetic Programming and Evolvable Machines, IEEE Transactions on Evolutionary Computation Scientific body: ACM SIGEVO, with newsletter Mailing list: ec-digest with archive: http://ec-digest.research.ucf.edu/ Recent publication about GP: Riccardo Poli, William B Langdon, Nicholas Freitag McPhee: A Field Guide to Genetic Programming http://www.lulu.com/content/2167025 Selected Topics in Evolutionary Algorithms II, July 10 th 2008