Geometric Unification of Evolutionary Algorithms Alberto Moraglio EvoPhD 2006

By Unification I Mean… EAs: Algorithmically irrelevant differences: name/authorship/solution interpretation/domain of application EAs: Algorithmically relevant differences: solution representation/genetic operators Unification: A formal framework that applies to all representations

Contents I – Geometric Interpretation of Crossover II – Unification of Major Representations III – Crossover Principled Design IV – Unity of Evolutionary Search

I. Geometric Interpretation of Crossover

What is crossover? Crossover Is there any common aspect ? Is it possible to give a representation- independent definition of crossover and mutation?

Geometric Crossover Representation-independent generalization of traditional crossover Informally: all offspring are between parents Search space: all offspring are on shortest paths connecting parents

Geometric Crossover & Distance Search Space is a Metric Space: d(A,B) =length of shortest paths between A and B Metric space: all offspring C are in the segment between parents C in [A,B] d  d(A,C)+d(C,B)=d(A,B)

Example1: Traditional Crossover Traditional Crossover is Geometric Crossover under Hamming Distance Parent1: 011|101 Parent2: 010|111 Child: 011|111 HD(P1,C)+HD(C,P2)=HD(P1,P2) = 2

Example2: Blending Crossover Blending Crossover for real vectors is geometric under Euclidean Distance P1 P2 C ED(P1,C)+ED(C,P2)=ED(P1,P2)

Geometric definitions with probability distributions Uniform geometric crossover: Uniform geometric ε-mutation:

Representation independent and formal definition of crossover and mutation in the search space seen as a geometric space

II. Unification of Major Representations & Operators

Minkowski spaces – real vectors B((2, 2); 1) Euclidean space 2 B((2, 2); 1) Manhattan space Balls 2 2 B((2, 2); 1) Chessboard space [(1, 1); (3, 2)] 1 geodesic Euclidean space 1 3 [(1, 1); (3, 2)] = [(1, 2); (3, 1)] infinitely many geodesics Manhattan space Line segments [(1, 1); (3, 2)] infinitely many geodesics Chessboard space Representation: real vectors Neighbourhoods: continuous (3 types) Distances: Minkowski distances Implementation: algebraic manipulation of real vector (equation of line passing through two points) Pre-existing recombination operators: - both blend crossovers and discrete crossovers fit geometric definition - extended blend crossovers do not fit

Hamming spaces – binary strings B(00;1) Hamming space H(2,3) [00;11]=[01;10] 2 geodesics Hamming space H(2,3) B(000; 1) Hamming space H(3,2) [000; 011] = [001; 010] 2 geodesics Hamming space H(3,2) Representation: binary/multary strings Neighbourhoods: bit-flip/site substitution Distances: Hamming distances Implementation: symbolic manipulation of multary strings (mask-based crossovers) Pre-existing recombination operators: - all binary crossovers fit the geometric definition

Cayley spaces - permutations Representation: permutations Neighbourhoods: adj. swap, swap, reversal, insertion Distances: corresponding distances Implementation: “minimal permutation sorting by X move” algorithms: - adj. swap = bubble sort - swap = selection sort - insertion = insertion sort - reversal = approximated MPS by reversals (NP-Hard)) Pre-existing recombination operators: various pre-existing crossover operators are sorting algorithm in disguise (because sorting permutations is easier than sorting vectors of other items) abc bac acb bca cab cba B(abc; 1) Adjacent swap space abc bac acb bca cab cba [abc; bca] 1 geodesic Adjacent swap space B(abc; 1) Swap space & Reversal space abc bac acb bca cab cba abc bac acb bca cab cba [abc; bca] 3 geodesics Swap space & Reversal space B(abc; 1) Insertion space [abc; bca] 1 geodesic Insertion space abc bac acb bca cab cba abc bac acb bca cab cba

Syntactic tree spaces Representation: syntactic tree (lisp expression) Neighbourhood: weighted sub-tree neighbourhood Distance: structural distance Implementation: - sub-tree swap crossover - common region mask based crossover Pre-existing recombination operators: - traditional crossover (non-geometric) - homologous crossover - the geometric framework can help to clarify what is the landscape and distance related to homologous crossover and a distance connected with a geometric crossover which traditional crossover is an approximation + sin + xxx * * * yx * yy Parent 1Parent 2 y + sin x * * yy x Alignment Crossover Point Swap * * yy + xx Offspring 1 Offspring 2

Significance of Unification Most of the pre-existing crossover operators for major representations fit geometric definition Established pre-existing operators have emerged from experimental work done by generations of practitioners over decades Geometric crossover compresses in a simple formula an empirical phenomenon

IV. Crossover Principled Design

Crossover Principled Design Domain specific solution representation is effective Problem: for non-standard representations it is not clear how crossover should look like But: given a combinatorial problem you may know already a good neighbourhood structure Geometric Interpretation of Crossover  Give me your neighbourhood definition and I give you a crossover definition

+= ? Crossover Design Example

Non-labelled graph neighbourhood MOVE: Insert/remove an edge Fixed number of nodes

+ Offspring

V. Is Biological Recombination Geometric? Yes, come to my other presentation at EuroGP!

VI. Unity of Evolutionary Search

Example of evolutionary search

Abstract convex evolutionary search Main result: an evolutionary algorithm using geometric crossover with any probability distribution, any kind of representation, any problem, any selection and replacement mechanism, does the same search: convex search Proof based on abstract convexity (axiomatic geodesic convexity) and axiomatization of search process (abstract search process)

…Nearly Over!

Summary Unification (meaning): formally dealing with all representations at once Geometric Definition: unif. is possible by defining operators geometrically Unification: many interesting recombinations are geometric Crossover design: by specification of geometric definition to a new representation General theory: using formal definition only, all EAs do the same search: convex

Thanks to the Reviewers Franz: thanks for all your suggestions, I’d be glad to talk with you over a coffee… Mario?: thanks for the enthusiastic support A fan?: thanks for warning me that I may be victim of a geometric hallucination…

Questions?

Geometric Crossover & Path-relinking Meta-heuristic Path-relinking: searches on path between solutions in the neighbourhood structure (not necessarily on a shortest path) Geometric crossover can be understood as a formalized generalization (to metric spaces) of PR that elicits the dual relationship between distance and solution representation and gives a formal recipe to design new crossover operators Formalized: it allows theory Generalization: metric spaces are more general than graphs Elicits duality: syntactic recombination is equivalent to neighbourhood search Crossover design: tells how to build crossovers rather than how to search the search space Formal recipe: it defines exactly what crossover is for any representation