1. Crossover Can Provably be Useful in Evolutionary Computation
Benjamin Doerr, Edda Happ, Christian Klein
Abstract: We present the first combinatorial optimization problem where crossover provably helps.

2. Evolutionary Computation
Imitates the natural evolution.
Components:
Population of individuals
Variation
Mutation
Crossover
Selection
Successfully used in many applications.
Theoretical results still far behind.
some funny animation deleted for copyright reasons"
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

3. Do we Need Crossover? Often used in practice, because
justified by “bio-inspiredness”
seems to work well
people like it:
feels more powerful
feels more clever than mutation
→ often small mutation rate
But: Little proof/understanding of its usefulness
Roadmap for this talk
Difficulties to build a provably useful crossover
A classic CO problem where crossover provably helps
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

4. Building Block Hypothesis
Building block hypothesis (Holland, 1975):
Crossover works well when short building blocks of high fitness can be recombined to form larger blocks with even higher fitness.
“Royal Road” functions (Mitchell et. al., 1992/94) designed accordingly to show good performance of GAs.
Experiments
showed that these
assumptions were
not fulfilled. s1
******** s2 s3 s4
opt x x x
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

5. some funny animation deleted for copyright reasons"
Jump Function
jump function (Jansen & Wegener, 1999, 2005):
Mutation-only EA needs an
expected time.
Using uniform crossover with rate
improves this to
Critique: Needs very small crossover rate. )
some funny animation deleted for copyright reasons" j m : f ; 1 g n ! R j m ( x ) = 8
>
< : + k 1 n ; e l s ( n m ) 1 = n l o g O ( p o l y n ) 2 m
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

6. Real Royal Road Functions
Real royal road functions (Storch & Wegener, 2003)
complicated pseudo-boolean function
exponential runtime with mutation only
quadratic runtime with mutation and crossover
works for any constant crossover rate
0^is beginning of rayal road, definition of royal road function brings individuals here. Then road is followed because royal road function awards next position on road higher value. Lo_1 is local opt with high value. 2 individuals are not allowed to be the same, so other individual keeps climbing road untill it gets to lo_2. A uniform crossover has prob \Theta(1/n)
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

7. Ising Model O ( n ) 2 Simplified Ising Model:
Minimize number of dichromatic
edges in a 2-colored graph.
Fischer & Wegener, 2004: Rings
Sudholt, 2005: Binary Trees
-EA needs mutation steps.
-GA needs steps.
Critique: Only works with suitable fitness-sharing mechanism. 2 ( n ) ( + ) O ( n 3 ) ( 2 + )
Fischer & Wegener (Gecco2004) get exp. Opt time of O(n^3) for (\lambda+1)-EA and can only analyse GIGA (gene invariant) and IGA (idealized) to get O(n^2).
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

8. Previous Work: Difficult to implement successful crossover
Building block hypothesis does not suffice.
Few successful examples are not completely satisfying:
Artificial
pseudo-boolean functions
finding a mono-chromatic coloring
Need additional/unusual features
very low crossover rate
fitness sharing
→ Need for crossover not fully explained.
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

9. Our Result
For the classical all-pairs shortest path problem, the natural EA/GA has an expected optimization time of
, if we use mutation only
this bound is sharp
, if we use mutation and crossover
any constant crossover rate does the job ( + 1 ) O ( n 4 ) O ( n 3 : 5 + ) ( n 4 ) O ( n 3 : 5 + ) O ( n 3 : 5 + ) O ( n 3 : 5 + )
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

10. All-Pairs Shortest Path
All-Pairs Shortest Path problem:
For every pair of vertices in graph G=(V,E) find a shortest path.
Natural GA:
Individuals: Paths
Population: For each pair of vertices at most one path.
Mutation: Change an edge at either end of an individual.
Crossover: Cut two paths and combine them.
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

11. Genetic Algorithm x x 2 x x x x ; x ; initialize: I=E; pick I; pick I;
probability p
probability 1-p
pick I;
= mutate( );
pick I;
= crossover( ); x 1 2 x 1 ; 2 x x x 1 x 1 ; 2
I = selection(I, ); x
repeat
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

12. Three Crossover Operatorsx 1 x 2
: append to .
: cut , append rear part to .
: cut and , append rear part of to front
part of . 1 x 2 x 1 2 x 2 x 1 3 x 2 x 1 x 2 x 1
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

13. Successful Crossover All 3 crossover operators yield good runtimes.
For proofs we partially need constraints:
Fitness function prefers paths with fewer edges.
Assume unique shortest paths.
Experiments show: Constraints are not needed. 2 : 3 :
mutation
only
Optimization times
using only mutation
or mutation and a
crossover operator.
-> prob \Omega(\frac{k}{n^4}) to create x_new in one step.
-> exp. # steps to create all paths having up to \ell edges is O(\frac{n^4 log n}{k}).
Assume first only mutation upto length \sqrt{n log n}, then only crossover gives O(n^{3.5}\sqrt{log n}). 3 2 1
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

14. Proof Idea and Why it Works
For assume unique shortest paths.
Chance to pick particular pair and positions for crossover is
But: possible crossovers create .
Choosing and appropriately and summing over
gives runtime. 3 k z } | { x 1 x 2 | { z } k x | { z } ` 1 n 4 k 2 ( 2 k + 1 ` ) 3 x
-> prob \Omega(\frac{k}{n^4}) to create x_new in one step.
-> exp. # steps to create all paths having up to \ell edges is O(\frac{n^4 log n}{k}).
Assume first only mutation upto length \sqrt{n log n}, then only crossover gives O(n^{3.5}\sqrt{log n}). ` k `
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation

15. Take Home Message: Thank you!
Crossover can provably help if there are sufficiently many possibilities for successful crossover.
Thank you!
Doerr, Happ, Klein: Crossover Can Provably Be Useful in Evolutionary Computation