0. Genetic Algorithms and Genetic Programming

1. Evolutionary Computation
Computational procedures patterned after biological evolution
Search procedure that probabilistically applies search operators to a set of points in the search space.

2. Biological Evolution Lamarck and others Darwin and Wallace
Species “transmute” over time
Darwin and Wallace
Consistent heritable variation among individuals in the population
Natural selection of the fittest
Mendel and genetics
A mechanism for inheriting traits
genotype -> phenotype mapping

3. Biological Terminology
gene
functional entity that codes for a specific feature e.g. eye color
set of possible alleles
allele
value of a gene e.g. blue, green, brown
codes for a specific variation of the gene/feature
locus
position of a gene on the chromosome
genome
set of all genes that define a species
the genome of a specific individual is called genotype
the genome of a living organism is composed of several
chromosomes
population
set of competing genomes/individuals
In GA’s genome and chromosome are used as synonyms
These terms are used freely in GA terminology although the biological counterparts
are by far more complex and sophisticated

4. Genotype versus Phenotype
blue print that contains the information to construct an
organism e.g. human DNA
genetic operators such as mutation and recombination
modify the genotype during reproduction
genotype of an individual is immutable
(no Lamarckian evolution)
phenotype
physical make-up of an organism
selection operates on phenotypes
(Darwin’s principle : “survival of the fittest”
Well, I am sure you don’t regard this issue from this perspective while you have sex
but that it is to which it pretty much boils down from an evolutionary perspective

5. The Genetic Algorithm
Directed search algorithms based on the mechanics of biological evolution
Developed by John Holland, University of Michigan (1970’s)
To understand the adaptive processes of natural systems
To design artificial systems software that retains the robustness of natural systems

6. The Genetic Algorithm (cont.)
Provide efficient, effective techniques for optimization and machine learning applications
Widely-used today in business, scientific and engineering circles

7. Components of a GA A problem to solve, and ...
Encoding technique (gene, chromosome)
Initialization procedure (creation)
Evaluation function (environment)
Selection of parents (reproduction)
Genetic operators (mutation, recombination)
Parameter settings (practice and art)

8. Genotype Operators recombination (crossover)
combines two parent genotypes into a new offspring
generates new variants by mixing existing genetic material
stochastic selection among parent genes
mutation
random alteration of genes
maintain genetic diversity
in genetic algorithms crossover is the major operator
whereas mutation only plays a minor role

9. Representation Phenotype space Genotype space = {0,1}L Encoding
Decoding
(inverse representation) 9

10. Simple Genetic Algorithm{
initialize population;
evaluate population;
while TerminationCriteriaNotSatisfied
select parents for reproduction;
perform recombination and mutation; }

11. The GA Cycle of Reproduction
children
reproduction
modification
modified
children
parents
population
evaluation
evaluated children
deleted
members
discard

12. Population population Chromosomes could be:
Bit strings ( )
Real numbers ( )
Permutations of element (E11 E3 E7 ... E1 E15)
Lists of rules (R1 R2 R3 ... R22 R23)
Program elements (genetic programming)
... any data structure ...
population

13. Reproduction
children
reproduction
parents
population
Parents are selected at random with selection chances biased in relation to chromosome evaluations.

14. Chromosome Modification
Modifications are stochastically triggered
Operator types are:
Mutation
Crossover (recombination)
children
modification
modified children

15. The simple GA Has been subject of many (early) studies
still often used as benchmark for novel GAs
Shows many shortcomings, e.g.
Representation is too restrictive
Mutation & crossovers only applicable for bit-string & integer representations
Selection mechanism sensitive for converging populations with close fitness values
Generational population model (step 5 in SGA repr. cycle) can be improved with explicit survivor selection

16. Representing Hypotheses
(Outlook = Overcast v Rain) ^ (Wind = Strong) by
Outlook Wind
IF Wind = Strong THEN PlayTennis = yes
Outlook Wind PlayTennis

17. SGA operators: 1-point crossover
Choose a random point on the two parents
Split parents at this crossover point
Create children by exchanging tails
Pc typically in range (0.6, 0.9)

18. SGA operators: mutation
Alter each gene independently with a probability pm
pm is called the mutation rate
Typically between 1/pop_size and 1/ chromosome_length

19. Alternative Crossover Operators
Performance with 1 Point Crossover depends on the order that variables occur in the representation
more likely to keep together genes that are near each other
Can never keep together genes from opposite ends of string
This is known as Positional Bias
Can be exploited if we know about the structure of our problem, but this is not usually the case

20. n-point crossover Choose n random crossover points
Split along those points
Glue parts, alternating between parents
Generalisation of 1 point (still some positional bias)

21. Uniform crossover Assign 'heads' to one parent, 'tails' to the other
Flip a coin for each gene of the first child
Make an inverse copy of the gene for the second child
Inheritance is independent of position

22. Order 1 crossover
Idea is to preserve relative order that elements occur
Informal procedure:
1. Choose an arbitrary part from the first parent
2. Copy this part to the first child
3. Copy the numbers that are not in the first part, to the first child:
starting right from cut point of the copied part,
using the order of the second parent
and wrapping around at the end
4. Analogous for the second child, with parent roles reversed

23. Order 1 crossover example
Copy randomly selected set from first parent
Copy rest from second parent in order 1,9,3,8,2

24. Crossover OR mutation?
Decade long debate: which one is better / necessary / main-background
Answer (at least, rather wide agreement):
it depends on the problem, but
in general, it is good to have both
both have another role
mutation-only-EA is possible, xover-only-EA would not work

25. Crossover OR mutation? (cont’d)
Exploration: Discovering promising areas in the search space, i.e. gaining information on the problem
Exploitation: Optimising within a promising area, i.e. using information
There is co-operation AND competition between them
Crossover is explorative, it makes a big jump to an area somewhere “in between” two (parent) areas
Mutation is exploitative, it creates random small diversions, thereby staying near (in the area of ) the parent

26. Crossover OR mutation? (cont’d)
Only crossover can combine information from two parents
Only mutation can introduce new information (alleles)
Crossover does not change the allele frequencies of the population (thought experiment: 50% 0’s on first bit in the population, ?% after performing n crossovers)
To hit the optimum you often need a ‘lucky’ mutation

27. Evaluation evaluation modified children evaluated children
The evaluator decodes a chromosome and assigns it a fitness measure
The evaluator is the only link between a classical GA and the problem it is solving
modified
children
evaluated
children
evaluation

28. Selection Schemes stochastic sampling roulette wheel selection
spin wheel N times
stochastic universal sampling
roulette wheel selection
single spin, wheel has N equally
spaced markers
tournament selection
choose k candidates at random with uniform probability
pick best one for reproduction
expected number of offspring
best :  k , average  k ½ k-1 , worst  k 1/N k-1

29. SGA operators: Selection
Main idea: better individuals get higher chance
Chances proportional to fitness
Implementation: roulette wheel technique
Assign to each individual a part of the roulette wheel
Spin the wheel n times to select n individuals A C
1/6 = 17%
3/6 = 50% B
2/6 = 33%
fitness(A) = 3
fitness(B) = 1
fitness(C) = 2

30. Replacement Schemes generational replacement
entire population is replaced each generation
non-overlapping population
10111
01000
01011
10010
01001
11001
Selection
Crossover
Mutation
steady state replacement
a single individual (worst, random) is replaced by
one offspring
overlapping population
10010
01001
11001
10010
01001
11001
Selection
Crossover
Mutation
01110

31. Deletion population discard discarded members
Generational GA: entire populations replaced with each iteration
Steady-state GA: a few members replaced each generation
population
discarded members
discard

32. An Abstract Example Distribution of Individuals in Generation 0
Distribution of Individuals in Generation N

33. A Simple Example The Traveling Salesman Problem:
Find a tour of a given set of cities so that
each city is visited only once
the total distance traveled is minimized

34. Representation Representation is an ordered list of city
numbers known as an order-based GA.
1) London 3) Dunedin ) Beijing 7) Tokyo
2) Venice ) Singapore 6) Phoenix 8) Victoria
CityList1 ( )
CityList2 ( )

35. Crossover Crossover combines inversion and recombination:
* *
Parent ( )
Parent ( )
Child ( )
This operator is called the Order1 crossover.

36. Mutation Mutation involves reordering of the list:
* *
Before: ( )
After: ( )

37. TSP Example: 30 Cities

38. Solution i (Distance = 941)

39. Solution j(Distance = 800)

40. Solution k(Distance = 652)

41. Best Solution (Distance = 420)

42. Overview of Performance

43. Population Models SGA uses a Generational model:
each individual survives for exactly one generation
the entire set of parents is replaced by the offspring
At the other end of the scale are Steady-State models:
one offspring is generated per generation,
one member of population replaced,
Generation Gap
the proportion of the population replaced
1.0 for GGA, 1/pop_size for SSGA

44. GABIL [de Jong et al, 1993]
Learn disjunctive set of propositional rules; competitive with C4.5
Fitness
Fitness(h) = (correct(h))2
Representation
IF a1=T ^ a2=F THEN c=T; IF a2=T THEN c=F
represented by:
a1 a2 c a1 a2 c
Genetic operators???
want variable length rule sets
want only well-formed bit-string hypotheses

45. Crossover with variable-length bit-strings
Start with:
a1 a2 c a1 a2 c h h
choose crossover points for h1, e.g. after bits 1, 8
now restrict points in h2 to those that produce bitstrings with well-defined semantics, e.g. 〈1,3〉, 〈1,8〉, 〈6,8〉.
if we choose 〈1,3〉, the results is:
a1 a2 c
h3:
a1 a2 c a1 a2 c a1 a2 c
h4:

46. GABIL extensions
Add new genetic operators, also applied probabilistically:
AddAlternative: generalise constraint on a1 by changing a 0 to 1
DropCondition: generalise constraint on ai by changing every 0 to 1.
And, add new field to bitstring to determine whether to allow these:
a1 a2 c a1 a2 c AA DC
so now the learning strategy also evolves.

47. GABIL Results
Performance of GABIL comparable to symbolic rule/tree learning methods C4.5, ID5R, AQ14.
Average performance on a set of 12 synthetic problems:
GABIL without AA and DC operators: 92.1% accuracy
GABIL with AA and DC operators 95.2% accuracy
symbolic learning methods ranged from 91.2% to 96.6%

48. Extensions to the Simple GA
Encoding schemes
gray encoding
messy genetic algorithms
Replacement schemes
generational replacement
steady state replacement
Fitness scaling
linear scaling
- truncation
ranking
Selection schemes
stochastic sampling
tournament selection

49. Genetic Programming automatic generation of computer programs + X1 *
by means of natural evolution see Koza 1999
programs are represented by a parse tree (LISP expression)
tree nodes correspond to functions :
- arithmetic functions {+,-,*,/}
- logarithmic functions {sin,exp}
leaf nodes correspond to terminals :
- input variables {X1, X2, X3}
- constants {0.1, 0.2, 0.5 } + X1 *
tree is parsed from left to right:
(+ X1 (* X2 X3)) X1+(X2*X3) X2 X3

50. Genetic Programming : Crossover- / X1 X2 X3 + * X3 X2 X1
parent A
parent B - * + /
offspring B
offspring A X1 X2 X2 X3 - X1 X2 X3

51. Block-Stacking ProblemV E U N
stack U
table A L S I R V E
objective: place the blocks in the correct order such
that the stack forms the word universal
functions: set of actions, logical operators, do-until loop
terminals: set of sensors that indicate top block on stack,
next suitable block on table etc.
each program tree is tested on 166 different initial
configurations
fitness: #configurations for which the stack was correct
after program execution

52. Block-Stacking ProblemCS N NN U TB A L S I R V E
Sensors:
CS: current stack, name of the top block of the stack
TB: top correct block, name of the topmost block on the stack
such that it and all blocks underneath are in correct order
NN: next block needed, name of the block needed above TB

53. Block-Stacking Problem
MS TB
MS NN N U A L S I R V E
Functions:
MS(X): move block X to the top of the stack, return value X
MT(X): moves the block on top of the stack to the table
if X is anywhere in the stack returns X
DU(exp1, exp2): execute exp1 until the predicate exp2
becomes true
NOT(exp1) : negation of expression exp1
EQ(exp1, exp2) : returns true if exp1 and exp2 are equal

54. Block-Stacking ProblemNN U CS A TB L S I R V E N
(EQ (MS NN) (EQ (MS NN) (MS NN)))
move next needed block to the stack three times in a row
(succeeds with a stack VERSAL and U N I on the table
(DU (MS NN) (NOT NN))
move next needed block to the stack until no more blocks
are needed
(EQ (DU (MT CS) (NOT CS)) (DU (MS NN) (NOT NN)))
empty the stack on the table and then build the stack in
the correct order
(EQ (DU (MT CS) (EQ (CS TB))) (DU (MS NN) (NOT NN)))

55. Learned Program Trained to fit 166 test problems
Using population of 300 programs, found this after 10 generations:
(EQ (DU (MT CS)(NOT CS))(DU (MS NN)(NOT NN)))

56. Genetic Programming
More interesting example: design electronic filter circuits
individual are programs that transform beginning cct to final cct, by adding/subtracting components and connections.
Use population of 640,000 run on 64 node parallel processor
Discover circuits competitive with best human desgn

57. GP for classifying images
Teller & Veloso, 1997
Fitness: based on coverage & accuracy
Representation:
Primitives include Add, Sub, Mult, Div, Not, Max, Min, Read, Write, If-Then-Else, Either, Pixel, Least, Most, Ave, Variance, Difference, Mini, Library
Mini refers to a local subroutine that is separately co-evolved
Library refers to a global library subroutine (evolved by selecting the most useful minis)
Genetic operators:
Crossover mutation
Create “mating pools” and use rank proportionate reproduction.

58. Biological Evolution Lamarck (19th century)
believed individual genetic makeup was altered by lifetime experience
but current evidence contradicts this view
What is the impact of individual learning on population evolution?

59. Baldwin Effect
Assume
individual learning has no direct influence on individual DNA
But ability to learn reduces need to “hard wire” traitsin DNA
Then
Ability of individuals to learn will support more diverse gene pool
because learning allow individuals with various “hard wired” traits to be successful
More diverse gene pool will support faster evolution of gene pool.
> individual learning (indirectly) increases rate of evolution.

60. Baldwin Effect Plausible example:
New predator appears in the environment
Individual who can learn (to avoid it) will be selected
Increase in learning individuals will support more diverse gene pool
resulting in faster evolution
possibly resulting in new non-learned traits such as instinctive fear of predator

61. Computer experiements on Baldwin Effect
Evolve simple neural nets
[Hinton & Nowlan, 1987]
some network weights fixed during lifetime, others trainable
genetic makeup determines which are fixed and their weight values
Results:
With no individual learning, population failed to improve over time
when individual learning allowed
early generations: population contained many individuals with many trainable weights
later generations: higher fitness, while number of trainable weights decreased.