Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer Sci. & Eng. Seoul National University

Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer Sci. & Eng. Seoul National University http://bi.snu.ac.kr/

Contents Part I: Introduction to Evolutionary Computation Part II: Genetic Programming Part III: Advanced Topics Part IV: Applications Part V: Summary and Further Info (C) 2000-2009 SNU CSE Biointelligence Lab2

Part I: Introduction to Evolutionary Computation

Basic Concept Use of Darwinian-like evolutionary processes to solve difficult computational problems.  Hence the name, “Evolutionary Computation” Biological basis  Biological systems adapt themselves to a new environment by evolution.  Generations of descendants are produced that perform better than do their ancestors. Biological evolution  Production of descendants changed from their parents  Selective survival of some of these descendants to produce more descendants 5(C) 2000-2009 SNU CSE Biointelligence Lab

Basic Concept What is the Evolutionary Computation?  Stochastic search (or problem solving) techniques that mimic the metaphor of natural biological evolution. Metaphor EVOLUTION Individual Fitness Environment PROBLEM SOLVING Candidate Solution Quality Problem 6

(C) 2000-2009 SNU CSE Biointelligence Lab General Framework of EC Generate Initial Population Evaluate Fitness Select Parents Generate New Offspring Termination Condition? Yes No Fitness Function Crossover, Mutation Best Individual 7

Components of EC (2/9) An example: the 8 queens problem  Place 8 queens on an 8x8 chessboard in such a way that they cannot check each other. 9

Components of EC (3/9) Representations  How to represent the space to be searched?  Genotype  Internal representation of solutions in EC.  Minimum domain knowledge.  Phenotype  External representation of solutions for the problem.  Require domain knowledge. Be careful so that every feasible solution can be represented in genotype space. 10(C) 2000-2009 SNU CSE Biointelligence Lab

12345678 Genotype: a permutation of the numbers 1 - 8 Phenotype: a board configuration Obvious mapping 11 Representation Example: 8 Queens Problem

Components of EC (4/9) Population  Usually has a fixed size and is a multiset of genotypes  Some sophisticated EAs also assert a spatial structure on the population e.g., a grid.  Diversity of a population refers to the number of different fitnesses / phenotypes / genotypes present (note not the same thing). Population sizing  Parent population size (M), offspring population size (K).  Typically M = K.  In typical ES (explained later), M << K.  In steady state algorithms, M > K. 12(C) 2000-2009 SNU CSE Biointelligence Lab

Components of EC (5/9) Fitness function  Represents the requirements that the population should adapt to  a.k.a. quality function or objective function  Assigns a single real-valued fitness to each phenotype which forms the basis for selection  So the more discrimination (different values) the better  Typically we talk about fitness being maximised  Some problems may be best posed as minimisation problems, but conversion is trivial. 13

Fitness Function Example: 8 Queens Problem Penalty for a queen:  The number of queens she can check Penalty for a configuration:  The sum of penalties of all queens Note: penalty is to be minimized Fitness of a configuration:  inverse penalty to be maximized 12345678 1 2 1 0 1 2 2 1 14(C) 2000-2009 SNU CSE Biointelligence Lab

Components of EC (6/9) Parent selection mechanism  Assigns probabilities for an individual to be selected as parents.  Selection probabilities are relative to current population.  Different probabilities can be assigned to the same individuals.  Usually depends on the individual’s fitness and probabilistic.  High quality solutions more likely to become parents than low quality ones  Selection pressure – the degree of correlation between the individual’s fitness and its selection probability.  High selection pressure results in reducing search scope.  Even worst in current population usually has non-zero probability of becoming a parent  This stochastic nature can aid escape from local optima. 15

Selection Mechanism Example: 8 Queens Problem 12345687 68317524 63125784 82654137 1/9 = 0.11 1/10 = 0.1 1/11 = 0.091 1/6 = 0.17 IndividualProportionalRanking (1)0.230.3 (2)0.210.2 (3)0.190.1 (4)0.360.4 (1) (2) (3) (4) Different selection mechanisms assign different selection probabilities for the same individual. 16(C) 2000-2009 SNU CSE Biointelligence Lab

Components of EC (7/9) Crossover (Recombination)  Mix information from parents into offspring in stochastic way.  Most offspring may be worse, or the same as the parents.  Hope is that some are better by combining elements of genotypes that lead to good traits.  Principle has been used for millennia by breeders of plants and livestock Example 87 64253 1 1352467 8 87 6451 2 3 1356287 4 17

(C) 2000-2009 SNU CSE Biointelligence Lab Components of EC (8/9) Mutation  It is applied to one genotype and delivers a (slightly) modified mutant, the child or offspring of it.  Element of randomness is essential.  The role of mutation in EC is different in various EC subtypes. Example  swapping values of two randomly chosen positions 12345678 12345678 18

(C) 2000-2009 SNU CSE Biointelligence Lab Components of EC (9/9) Initialization usually done at random  Need to ensure even spread and mixture of possible allele values  Can include existing solutions, or use problem-specific heuristics, to “ seed ” the population Termination condition checked every generation  Reaching some (known/hoped for) fitness  Reaching some maximum allowed number of generations  Reaching some minimum level of diversity  Reaching some specified number of generations without fitness improvement 19

(C) 2000-2009 SNU CSE Biointelligence Lab Paradigms in EC Evolutionary Programming (EP)  [L. Fogel et al., 1966]  FSMs, mutation only, tournament selection Evolution Strategy (ES)  [I. Rechenberg, 1973]  Real values, mainly mutation, ranking selection Genetic Algorithm (GA)  [J. Holland, 1975]  Bitstrings, mainly crossover, proportionate selection Genetic Programming (GP)  [J. Koza, 1992]  Trees, mainly crossover, proportionate selection 20

Appendix: Evolution Strategy

Evolution Strategy (ES) Problem of real-valued optimization Find extremum (minimum) of function F(X): R n ->R Operate directly on real-valued vector X Generate new solutions through Gaussian mutation of all components Selection mechanism for determining new parents (C) 2000-2009 SNU CSE Biointelligence Lab22

ES: Representation One individual: The three parts of an individual: : Object variables : Standard deviations : Rotation angles Fitness Variances Covariances (C) 2000-2009 SNU CSE Biointelligence Lab23

ES: Evolution Strategy vs. Genetic Algorithm Create random initial population Evaluate population Select individuals for variation Vary Insert into population Create random initial population Evaluate population Vary Selection Insert into population (C) 2000-2009 SNU CSE Biointelligence Lab27

Part II: Genetic Programming

Genetic Programming (GP) GP is an domain- independent method that evolves a population of programs to solve a problem. GP uses variable-size tree- representations rather than fixed-length strings of binary values in typical GA. GP has been successful in many domains. 30

(C) 2000-2009 SNU CSE Biointelligence Lab Tree-based Representations (1/6) Function set: internal nodes  Functions, predicates, or actions which take one or more arguments Terminal set: leaf nodes  Program constants, actions, or functions which take no arguments S-expression: (+ 3 (/ (  5 4) 7)) Terminals = {3, 4, 5, 7} Functions = {+, , /} 31

(C) 2000-2009 SNU CSE Biointelligence Lab Tree-based Representations (2/6) Trees can represent:  Arithmetic formula  Logical formula  Program  And many others depending on the definition of terminals and nonterminals. (x  true)  (( x  y )  (z  (x  y))) i =1; while (i < 20) { i = i +1 } 32

(C) 2000-2009 SNU CSE Biointelligence Lab Tree-based Representations (6/6) In GA, ES, EP chromosomes are linear structures (bit strings, integer string, real-valued vectors, permutations) Tree shaped chromosomes are non-linear structures. In GA, ES, EP the size of the chromosomes is fixed. Trees in GP may vary in depth and width. 36

Setting up a GP Run Users need to specify: 1.The terminal / function set 2.The fitness measure 3.Algorithm parameters to control the run  Population size, maximum number of generations  Selection / variation operators and their probabilities  The termination criterion  Maximum depth of a GP tree, etc. 37(C) 2000-2009 SNU CSE Biointelligence Lab

Example: Wall-Following Robot (Step 1) Program Representation in GP  Functions  AND (x, y) = 0 if x = 0; else y  OR (x, y) = 1 if x = 1; else y  NOT (x) = 0 if x = 1; else 1  IF (x, y, z) = y if x = 1; else z  Terminals  Actions: move the robot one cell to each direction { north, east, south, west }  Sensory input: its value is 0 whenever the coressponding cell is free for the robot to occupy; otherwise, 1. { n, ne, e, se, s, sw, w, nw } 38

Example: Wall-Following Robot (Step 1) Be careful when specifying function / terminal set. To avoid syntactic error  All programs in the initial population should be valid, executable programs.  The genetic operators during the run should produce valid, executable programs as offspring. To avoid run-time error  All functions can take any terminal or the results produced by any other functions as input. 40(C) 2000-2009 SNU CSE Biointelligence Lab

Example: Wall-Following Robot (Step 2) Fitness measures can be  Error between program’s output and the desired output.  Accuracy in recognizing patterns or classifying objects into classes.  Payoff a game-playing program produces. Often each program is executed over a representative sample of fitness cases. In our wall-following robot problem  the number of cells next to the wall that are visited during 60 steps over 10 independent runs.  Perfect score (320) : One Run (32)  10 randomly chosen starting points 41(C) 2000-2009 SNU CSE Biointelligence Lab

Example: Wall-Following Robot (Step 3) Population size: 5,000 Termination condition: found perfect solution Creating Next Generation  500 programs (10%) are copied directly into next generation (elitism).  Tournament selection 7 programs are randomly selected from the population 5,000. The most fit of these 7 programs is chosen.  4,500 programs (90%) are generated by crossover.  Parents are each chosen by tournament selection.  In this example, mutation was not used. (C) 2000-2009 SNU CSE Biointelligence Lab44

(C) 2000-2009 SNU CSE Biointelligence Lab Result (1/5) Generation 0  The most fit program (fitness = 92)  Starting in any cell, this program moves east until it reaches a cell next to the wall; then it moves north until it can move east again or it moves west and gets trapped in the upper-left cell. 46

(C) 2000-2009 SNU CSE Biointelligence Lab Result (2/5) Generation 2  The most fit program (fitness = 117)  Smaller than the best one of generation 0, but it does get stuck in the lower-right corner. 47

(C) 2000-2009 SNU CSE Biointelligence Lab Result (4/5) Generation 10  The most fit program (fitness = 320)  Following the wall around clockwise and moves south to the wall if it doesn’t start next to it. 49

Part III: Advanced Topics and Applications

Mathematical Models of Simple GAs (1/7) Based on (Vose & Liepins, 1991). The model simple GA 1.Start with a random population of binary strings of length l. 2.Calculate the fitness f(x) of each string x in the population. 3.Choose (with replacement) two parents from the population with probability proportional to each string’s relative fitness in the population. 4.Crossover the two parents (at single random point) with probability p c to from two offspring and discard one at random. 5.Mutate each bit in the selected offspring with probability p m and replace it in the new population. 6.Go to Step 3 until new population is complete. 7.Go to Step 1. 53(C) 2000-2009 SNU CSE Biointelligence Lab

Notations  Each string is represented by binary string between 0 and 2 l -1. Mathematical Models of Simple GAs (2/7) 11 11 01 1 0 Proportion of string i in the population The selection probability of string i, assuming that fitness is equal to the number of 1’s in the string. 54(C) 2000-2009 SNU CSE Biointelligence Lab

Mathematical Models of Simple GAs (4/7) Including crossover and mutation.  Define recombination matrix M for crossover and mutation.  r i,j (k) – the probability that string k will be produced, given that string i and j are selected to mate.  Defining r i,j (k) and M is tricky. We will show defining r i,j (0) only. 56(C) 2000-2009 SNU CSE Biointelligence Lab

Mathematical Models of Simple GAs (5/7) Defining r i,j (0)  Case 1: crossover does not occur and the selected offspring is mutated to all zeros.  Case 2: crossover occurs and the selected offspring is mutated to all zeros. Case 1  |i|: the number of 1’s in a string i.  The probability that i be mutated to all zeros  57(C) 2000-2009 SNU CSE Biointelligence Lab

Mathematical Models of Simple GAs (6/7) Case 2  Temporary definition  |h|=|i|-|i 2 |+|j 2 | & |k|=|j|-|j 2 |+|i 2 |.  But, gives the number of 1’s in rightmost c bits of i.  Thus, i: 0 0 0 1 0 1 0 1 j: 1 1 1 0 1 0 1 0 h: 0 0 0 0 1 0 1 0 k: 1 1 1 1 0 1 0 1 i1i1 i2i2 j1j1 j2j2 i: 0 0 0 0 1 0 1 0 2 5 -1: 0 0 0 1 1 1 1 1 58(C) 2000-2009 SNU CSE Biointelligence Lab

Bloat Phenomena in GP (2/3) Why does it happen?  No certain explanation.  Replication accuracy theory  GP evolves towards (bloated) representations that increase replication accuracy.  Crossover bias theory  Short programs are less fit than long ones  long programs have a selective advantage  bloat 61(C) 2000-2009 SNU CSE Biointelligence Lab

Bloat Phenomena in GP (3/3) Techniques limiting bloat  Fixed size or depth limit: programs exceeding the limit are discarded and a parent is kept.  Dangerous: acts as advantage for programs that violate the limit.  Parsimony pressure: a penalty term for larger programs is added to the fitness value.  (Fitness_parsimony) = (Fitness_raw) – c * (program_size) 62(C) 2000-2009 SNU CSE Biointelligence Lab

Part IV: Applications

(C) 2000-2009 SNU CSE Biointelligence Lab Applications of EC Numerical, Combinatorial Optimization System Modeling and Identification Planning and Control Engineering Design Data Mining Machine Learning Artificial Life 64

Co-evolving Soccer Softbots (1/5) At RoboCup there are two "leagues": the "real" robot league and the "virtual" softbot league How do you do this with GP?  GP breeding strategies: homogeneous and heterogeneous  Decision of the basic set of function with which to evolve players  Creation of an evaluation environment for our GP individuals Application Example 1 Co-evolving Soccer Softbots With Genetic Programming (C) 2000-2009 SNU CSE Biointelligence Lab65

Evolutionary Arts Exploits the process of evolution to create an artwork according to an evolutionary algorithm. Selection may be done by the viewer explicitly by selecting individuals which are aesthetically pleasing, or implicitly according to the length of time a viewer spends near a piece of evolving art. EvoMUSART workshops 2009  http://evostar.na.icar.cnr.it/EvoWorkshops/ EvoMUSART/EvoMUSART.html Application Example 2 70(C) 2000-2009 SNU CSE Biointelligence Lab

Part V: Summary & Further Information

(C) 2000-2009 SNU CSE Biointelligence Lab Recapitulation of EA EAs fall into the category of “ generate and test ” algorithms. They are stochastic, population-based algorithms. Variation operators (recombination and mutation) create the necessary diversity and thereby facilitate novelty. Selection reduces diversity and acts as a force pushing quality. Fundamental trade-off between exploration and exploitation 72

Exploration vs. Exploitation There are many possibilities to set the parameters to increase the explorative capability of an EA  Increase variation probability  Increase mutation rate  Increase crossover rate  Increase population size  Decrease selection pressure  Add noise to fitness function  Add noise in selection  Increase rank ratio in ranking selection  Decrease tournament size in tournament selection Important is the combined effect of various parameter settings. (C) 2000-2009 SNU CSE Biointelligence Lab73

(C) 2000-2009 SNU CSE Biointelligence Lab Typical Behavior of an EA Phases in optimizing on a 1-dimensional fitness landscape Early phase: quasi-random population distribution Mid-phase: population arranged around/on hills Late phase: population concentrated on high hills 74

Are Long Runs Beneficial? (C) 2000-2009 SNU CSE Biointelligence Lab75 Best fitness in population Time (number of generations) Progress in 1 st half Progress in 2 nd half Answer: - It depends how much you want the last bit of progress - It may be better to do more shorter runs

(C) 2000-2009 SNU CSE Biointelligence Lab Advantages of EC No presumptions w.r.t. problem space Widely applicable Low development & application costs Easy to incorporate other methods Solutions are interpretable (unlike NN) Can be run interactively, accommodate user proposed solutions Provide many alternative solutions 77

(C) 2000-2009 SNU CSE Biointelligence Lab Disadvantages of EC No guarantee for optimal solution within finite time Weak theoretical basis May need parameter tuning Often computationally expensive, i.e. slow 78

(C) 2000-2009 SNU CSE Biointelligence Lab Further Information on EC Conferences  IEEE Congress on Evolutionary Computation (CEC)  Genetic and Evolutionary Computation Conference (GECCO)  Parallel Problem Solving from Nature (PPSN)  Foundation of Genetic Algorithms (FOGA)  EuroGP, EvoCOP, and EvoWorkshops  Int. Conf. on Simulated Evolution and Learning (SEAL) Journals  IEEE Transactions on Evolutionary Computation  Evolutionary Computation  Genetic Programming and Evolvable Machines 79

(C) 2000-2009 SNU CSE Biointelligence Lab References Evolutionary Computation by Kenneth A. De Jong, 2006. Evolutionary Computation Evolutionary Computation: Toward a New Philosophy of Machine Intelligence, Third Edition (IEEE Press Series on Computational Intelligence) by David B. Fogel, 2005, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence, Third Edition (IEEE Press Series on Computational Intelligence) Introduction to Evolutionary Computing by A. E. Eiben and J. E Smith, Springer, 2003. Genetic Programming: On the Programming of Computers by Means of Natural Selection (Complex Adaptive Systems) by John R. Koza, 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection (Complex Adaptive Systems) Genetic Programming: An Introduction by W. Banzhaf, P. Nordin, R. Keller, and F. Francone, Morgan Kaufmann, 1997. A Field Guide to Genetic Programming by R. Poli and W. B. Langdon, 2008. Online free book – http://www.gp-field-guide.org.uk/ 80

Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer Sci. & Eng. Seoul National University

Similar presentations

Presentation on theme: "Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer Sci. & Eng. Seoul National University"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer Sci. & Eng. Seoul National University

Similar presentations

Presentation on theme: "Evolutionary Computation: A Tutorial September 2009 Biointelligence Laboratory School of Computer Sci. & Eng. Seoul National University"— Presentation transcript:

Similar presentations

About project

Feedback