Genetic Algorithms 虞台文

Content Evolutional Algorithms Genetic Algorithms Main Components of Genetic Algorithms – Encoding – Fitness Function – Recombination – Mutation – Population – Selection Models

Genetic Algorithms Evolutional Algorithms

Biological Evolution

Motivated from Nature Nature evolve strikingly complex organisms in response to complex environmental adaptation problems with apparent ease Localize and extract principles from nature Apply them to design algorithms

Darwinian Evolution 1: Survival of the Fittest All environments have finite resources – can only support a limited number of individuals Lifeforms have basic instinct/lifecycles geared towards reproduction – Therefore some kind of selection is inevitable – Those individuals that compete for the resources most effectively have increased chance of reproduction Note: fitness in natural evolution is a derived, secondary measure, i.e., we (humans) assign a high fitness to individuals with many offspring

Darwinian Evolution 2: Diversity Drives Change Phenotypic traits: – Behaviour/physical differences that affect response to environment – Partly determined by inheritance, partly by factors during development – Unique to each individual, partly as a result of random changes If phenotypic traits: – Lead to higher chances of reproduction – Can be inherited then they will tend to increase in subsequent generations and, hence, lead to new combinations of traits …

Darwinian Evolution:Summary Population consists of diverse set of individuals Combinations of traits that are better adapted tend to increase representation in population Individuals are “units of selection” Variations occur through random changes yielding constant source of diversity, coupled with selection means that: Population is the “unit of evolution” Note the absence of “guiding force”  w/o God

Population General Scheme of Evolutional Algorithms (EAs) Parents Offspring Initialization Termination Parent Selection Survivor Selection Recombination Mutation

Population General Scheme of Evolutional Algorithms (EAs) Parents Offspring Initialization Termination Parent Selection Survivor Selection Recombination Mutation A set of individuals corresponding to candidate solutions to a problem. Each individual has a fitness value. How to represent the individuals? How to define the fitness function? A set of individuals corresponding to candidate solutions to a problem. Each individual has a fitness value. How to represent the individuals? How to define the fitness function?

Population General Scheme of Evolutional Algorithms (EAs) Parents Offspring Initialization Termination Parent Selection Survivor Selection Recombination Mutation Better individuals get higher probability to become the parents of the next generation. Variation operators

Population General Scheme of Evolutional Algorithms (EAs) Parents Offspring Initialization Termination Parent Selection Survivor Selection Recombination Mutation Replace some old individuals with newly born ones.

Pseudo-Code for Typical EA

Typical Behavior of an EA Early phase: quasi-random population distribution Mid-phase: population arranged around/on hills Late phase: population concentrated on high hills

Components of EAs Representation  definition of individuals Evaluation function  fitness function Population Parent selection mechanism Variation operators – Recombination – Mutation Survivor selection mechanism  replacement Termination

Different Types of EAs Historically, different flavours of EAs have been associated with different representations – Binary strings : Genetic Algorithms – Real-valued vectors : Evolution Strategies – Finite state Machines: Evolutionary Programming – LISP trees: Genetic Programming

Different Types of EAs Historically, different flavours of EAs have been associated with different representations Binary strings : Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state Machines: Evolutionary Programming LISP trees: Genetic Programming Binary strings : Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state Machines: Evolutionary Programming LISP trees: Genetic Programming Genetic Algorithms Evolution Strategies

Different Types of EAs Historically, different flavours of EAs have been associated with different representations Binary strings : Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state Machines: Evolutionary Programming LISP trees: Genetic Programming Binary strings : Genetic Algorithms Real-valued vectors : Evolution Strategies Finite state Machines: Evolutionary Programming LISP trees: Genetic Programming Evolutionary Programming Genetic Programming

Genetic Algorithms

The Genetic Algorithm Directed search algorithms based on the mechanics of biological evolution Developed by John Holland, University of Michigan (1970’s) – To understand the adaptive processes of natural systems – To design artificial systems software that retains the robustness of natural systems

Genetic Algorithms Holland’s original GA is now known as the simple genetic algorithm (SGA) Other GAs use different: – Representations – Mutations – Crossovers – Selection mechanisms

SGA Technical Summary Tableau Representation Binary strings Recombination N-point or uniform Mutation Bitwise bit-flipping with fixed probability Parent selection Fitness-Proportionate Survivor selection All children replace parents Speciality Emphasis on crossover

Representation Genotype space Phenotype space Encoding (representation) Decoding (inverse representation)

SGA Evolution Cycle 1.Select parents for the mating pool (size of mating pool = population size) 2.Shuffle the mating pool 3.For each consecutive pair apply crossover with probability p c, otherwise copy parents 4.For each offspring apply mutation (bit-flip with probability p m independently for each bit) 5.Replace the whole population with the resulting offspring 1.Select parents for the mating pool (size of mating pool = population size) 2.Shuffle the mating pool 3.For each consecutive pair apply crossover with probability p c, otherwise copy parents 4.For each offspring apply mutation (bit-flip with probability p m independently for each bit) 5.Replace the whole population with the resulting offspring

SGA Operators: 1-Point Crossover Choose a random point on the two parents Split parents at this crossover point Create children by exchanging tails p c typically in range (0.6, 0.9)

SGA Operators: Mutation Alter each gene independently with a probability p m p m is called the mutation rate – Typically between 1/pop_size and 1/chromosome_length

SGA Operators: Selection Main idea: better individuals get higher chance – Chances proportional to fitness – Implementation  roulette wheel technique Assign to each individual a part of the roulette wheel Spin the wheel n times to select n individuals fitness(A) = 3 fitness(B) = 1 fitness(C) = 2

An Example after Goldberg ‘89 Simple problem: max x 2 over {0,1,…,31} GA approach: – Representation: binary code, e.g  13 – Population size: 4 – 1-point xover, bitwise mutation – Roulette wheel selection – Random initialization

x 2 Example: Initial Population

x 2 Example: Selection

x 2 Example: Selection

x 2 Example: Crossover

x 2 Example: Mutation

Exercise: The OneMax Problem The OneMax Problem (or BitCounting) is a simple problem consisting in maximizing the number of ones of a bitstring. Formally, this problem can be described as finding a string x = (x 1, x 2, …, x n ), with x i  {0, 1}, that maximizes the following equation:

Genetic Algorithms Main Components of GA’s

Nature-to-Computer Mapping NatureComputer IndividualSolution to a problem PopulationSet of solutions FitnessQuality of a solution ChromosomeEncoding for a solution GenePart of the encoding of a solution CrossoverSearch Operator (Local) MutationSearch Operator (Global) Nature SelectionReuse of good (sub-)solutions

Main Components of GA’s Encoding principles (gene, chromosome) Initialization procedure (creation) Selection of parents (reproduction) Genetic operators (mutation, recombination) Fitness function (environment) Termination condition

Encoding The process of representing the solution in the form of a string that conveys the necessary information. Just as in a chromosome, each gene controls a particular characteristic of the individual, similarly, each bit in the string represents a characteristic of the solution. Hence, the encoding mechanism depends on the nature of the problem variables.

Commonly Used Encoding Schemes Binary Encoding – most common and simplest one Permutation Encoding – used in “ordering problems”, e.g., TSP Direct Value Encoding – used in problems with complicated values Tree Encoding – used mainly for evolving programs or expressions, e.g., genetic programming

Binary Encoding Permutation Encoding Direct Value Encoding Tree Encoding Most common method of encoding. – Chromosomes are strings of 1s and 0s and each position in the chromosome represents a particular characteristic of the problem Chromosome B Chromosome A

Permutation Encoding Binary Encoding Permutation Encoding Direct Value Encoding Tree Encoding Useful in ordering problems such as the Traveling Salesman Problem (TSP). – Every chromosome is a string of numbers, each of which represents a city to be visited Chromosome B Chromosome A

Direct Value Encoding Binary Encoding Permutation Encoding Direct Value Encoding Tree Encoding Used in problems where complicated values, such as real numbers, are used and where binary encoding would not suffice. Good for some problems, but often necessary to develop some specific crossover and mutation techniques for these chromosomes. (left), (back), (left), (right), (forward)Chromosome B Chromosome A

Tree Encoding Binary Encoding Permutation Encoding Direct Value Encoding Tree Encoding Tree encoding is used mainly for evolving programs or expressions for genetic programming. In tree encoding every chromosome is a tree of some objects, such as functions or commands in programming language.

More on Binary Encoding  Gray coding of integers IntegerBinaryGray

More on Binary Encoding  Gray coding of integers Gray coding is a mapping that “attempts” to improve causality, i.e., small changes in the genotype cause small changes in the phenotype, unlike binary coding. – “Smoother” genotype-phenotype mapping makes life easier for the GA. Nowadays it is generally accepted that it is better to encode numerical variables directly as integers and floating point variables

More on Binary Encoding  Interval Representation z  [x, y]   represented by {a 1,…,a L }  {0,1} L [x, y]  {0,1} L must be invertible  : {0,1} L  [x, y] defines the representation

Crossover It is the process in which two chromosomes (strings) combine their genetic material (bits) to produce a new offspring which possesses both their characteristics. Two strings are picked from the mating pool at random to cross over. The method chosen depends on the Encoding Method.

Crossover It is the process in which two chromosomes (strings) combine their genetic material (bits) to produce a new offspring which possesses both their characteristics. Two strings are picked from the mating pool at random to cross over. The method chosen depends on the Encoding Method.

Crossover Operators Binary Encoding – 1-Point Crossover – 2-Point Crossover – N-Point Crossover – Uniform Crossover Permutation Encoding

Binary Encoding  1-Point Crossover Chromosome | Chromosome | Offspring | Offspring |

Binary Encoding  2-Point Crossover Chromosome | | Chromosome | | Offspring | | Offspring | |

Binary Encoding  N-Point Crossover Choose N random crossover points Split along those points Glue parts, alternating between parents

Binary Encoding  Uniform Crossover Each gene (bit) is selected randomly from one of the corresponding genes of the parent chromosomes. Chromosome Chromosome Offspring Offspring

Permutation Encoding  Order Crossover (OX) Builds offspring by choosing a sub-sequence of a tour from one parent and preserving the relative order of cities from the other parent and feasibility

Permutation Encoding  Order Crossover (OX)    Builds offspring by choosing a sub-sequence of a tour from one parent and preserving the relative order of cities from the other parent and feasibility

Permutation Encoding  Order Crossover (OX)    Builds offspring by choosing a sub-sequence of a tour from one parent and preserving the relative order of cities from the other parent and feasibility 21893

Permutation Encoding  Order Crossover (OX)    Builds offspring by choosing a sub-sequence of a tour from one parent and preserving the relative order of cities from the other parent and feasibility Chromosome Chromosome Offspring Offspring

Permutation Encoding  Partially Mapped Crossover (PMX)

Step 1

Permutation Encoding  Partially Mapped Crossover (PMX) Step

Permutation Encoding  Partially Mapped Crossover (PMX) Step

Permutation Encoding  Partially Mapped Crossover (PMX) Step 1

Permutation Encoding  Partially Mapped Crossover (PMX) Step

Permutation Encoding  Partially Mapped Crossover (PMX) Step

Permutation Encoding  Partially Mapped Crossover (PMX) Chromosome Chromosome Offspring Offspring

Permutation Encoding  Cycle Crossover

Step 1: identify cycles

Permutation Encoding  Cycle Crossover Step 1: identify cycles

Permutation Encoding  Cycle Crossover Step 1: identify cycles

Permutation Encoding  Cycle Crossover Step 1: identify cycles Step 2: copy alternate cycles into offspring

Permutation Encoding  Edge Recombination Step 1: Construct edge table

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 6

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 6

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 62

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 62

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 628

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 628

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 6287

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 6287

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 62873

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done 62873

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done

Permutation Encoding  Edge Recombination 1 Step 1: Construct edge table Step 2: Randomly choose start node Step 3: Remove visited nodes Step 4: Pick common edge if existent Pick one with the shortest list Ties are split at random 5 Step 5: Loop to Step 3 until done

Crossover for Real Valued GAs Discrete: – Each allele value in offspring z comes from one of its parents (x, y) with equal probability: z i = x i or y i – Could use n-point or uniform Intermediate – exploits idea of creating children “between” parents (hence a.k.a. arithmetic recombination) – z i =  x i + (1   ) y i where  : 0    1. – The parameter  can be: constant: uniform arithmetical crossover variable (e.g. depend on the age of the population) picked at random every time

Single Arithmetic Crossover Parents:  x 1,…, x n  and  y 1,…, y n  Pick a single gene (k) at random child 1 is: reverse for other child. e.g. with  = 0.5

Simple Arithmetic Crossover Parents:  x 1,…, x n  and  y 1,…, y n  Pick random gene (k) after this point mix values child 1 is: reverse for other child. e.g. with  = 0.5

Whole Arithmetic Crossover Parents:  x 1,…, x n  and  y 1,…, y n  child 1 is: reverse for other child. e.g. with  = 0.5

More on Crossover Crossover between 2 good solutions MAY NOT ALWAYS yield a better or as good a solution. Since parents are good, probability of the child being good is high. If offspring is not good (poor solution), it will be removed in the next iteration during “Selection”.

Crossover or Mutation Decade long debate: Which one is better / necessary / main-background? Answer (at least, rather wide agreement): – it depends on the problem, but – in general, it is good to have both – both have another role – mutation-only-EA is possible, xover-only-EA would not work in general

Crossover or Mutation Exploration: – Discovering promising areas in the search space, i.e. gaining information on the problem – Crossover is “usually” explorative, it makes a big jump to an area somewhere “in between” two (parent) areas Exploitation: – Optimizing within a promising area, i.e. using information – Mutation is “usually” exploitative, it creates random small diversions, thereby staying near (in the area of ) the parent

Crossover or Mutation Crossover can combine information from two parents Mutation can introduce new information (alleles) Crossover does not change the allele frequencies of the population To hit the optimum you often need a ‘lucky’ mutation

Mutation Operators Binary Encoding Value Encoding – Integers – Floating points Permutation Encoding

Mutation Operator for Binary Encoding For chromosomes using Binary Encoding, randomly selected bits are inverted. Offspring Mutated Offspring NOTE: The number of bits to be inverted depends on the Mutation Probability.

Mutation Operator for Value Encoding Creep  – Adding small offset to randomly chosen allele(s) Random choice  – Assign random value to randomly chosen allele(s) Offspring Mutated Offspring

Mutation Operators for Permutation Encoding Insert Mutation Swap Mutation Inversion mutation

Elitism When creating a new population by crossover or mutation the best chromosome might be lost. Forces GAs to retain some number of the best individuals at each generation. Has been found that elitism significantly improves performance. Elitism is a method which copies the best chromosome to the new offspring population before crossover and mutation.

Population Models SGA uses a Generational model: – each individual survives for exactly one generation – the entire set of parents is replaced by the offspring At the other end of the scale are Steady-State models: – one offspring is generated per generation, – one member of population replaced, Generation Gap – the proportion of the population replaced – 1.0 for GGA, 1/pop_size for SSGA

Fitness Function A fitness function quantifies the optimality of a solution (chromosome) so that that particular solution may be ranked against all the other solutions. A fitness value is assigned to each solution depending on how close it actually is to solving the problem. Ideal fitness function correlates closely to goal + quickly computable. Example: In TSP, f(x) is sum of distances between the cities in solution. The lesser the value, the fitter the solution is.

Selection Models Selection can occur in two places: – Parent selection  Selection from current generation to take part in mating – Survivor selection  Selection from parents + offspring to go into next generation Distinction between selection – operators: define selection probabilities – algorithms: define how probabilities are implemented

Parent Selection Fitness-Proportionate Selection – Roulette wheel algorithm – Baker’s SUS algorithm: Rank-Based Selection – Linear Ranking – Exponential Ranking Tournament Selection

Fitness-Proportionate Selection Roulette wheel algorithm: – Given a probability distribution, spin a 1-armed wheel n times to make n selections Baker’s SUS algorithm: – Stochastic Universal Sampling – n evenly spaced arms on wheel and spin once

Fitness-Proportionate Selection Problems – One highly fit member can rapidly take over if rest of population is much less fit: Premature Convergence – At end of runs when fitnesses are similar, lose selection pressure – Highly susceptible to function transposition Scaling can fix last two problems – Windowing: f’(i) = f(i) -  t where  is worst fitness in this (last n) generations – Sigma Scaling: f’(i) = max[ f(i) – (  f  - c  f ), 0.0] where c is a constant, usually 2.0

Function Transposition for FPS

Rank-Based Selection Attempt to remove problems of FPS by basing selection probabilities on relative rather than absolute fitness Rank population according to fitness and then base selection probabilities on rank where fittest has rank  and worst rank 1

Rank-Based Selection Linear Ranking Exponential Ranking

Tournament Selection

All methods above rely on global population statistics – Could be a bottleneck esp. on parallel machines – Relies on presence of external fitness function which might not exist: e.g., evolving game players Informal Procedure: – Pick k members at random then select the best of these – Repeat to select more individuals – k = 2: binary tournament selection

Tournament Selection

Some Demonstrations html html

References A Survey of Genetic Algorithm Genetic Algorithms: A Survey A Genetic Algorithm Tutorial