1. Geometric Unification of Evolutionary Algorithms Alberto Moraglio amoragn@essex.ac.uk EvoPhD 2006

2. By Unification I Mean… EAs: Algorithmically irrelevant differences: name/authorship/solution interpretation/domain of application EAs: Algorithmically relevant differences: solution representation/genetic operators Unification: A formal framework that applies to all representations

3. Contents I – Geometric Interpretation of Crossover II – Unification of Major Representations III – Crossover Principled Design IV – Unity of Evolutionary Search

4. I. Geometric Interpretation of Crossover

5. What is crossover? Crossover Is there any common aspect ? Is it possible to give a representation- independent definition of crossover and mutation? 100000011101000 100111100011100 100110011101000 100001100011100

6. Geometric Crossover Representation-independent generalization of traditional crossover Informally: all offspring are between parents Search space: all offspring are on shortest paths connecting parents

7. Geometric Crossover & Distance Search Space is a Metric Space: d(A,B) =length of shortest paths between A and B Metric space: all offspring C are in the segment between parents C in [A,B] d  d(A,C)+d(C,B)=d(A,B)

8. Example1: Traditional Crossover Traditional Crossover is Geometric Crossover under Hamming Distance Parent1: 011|101 Parent2: 010|111 Child: 011|111 HD(P1,C)+HD(C,P2)=HD(P1,P2) 1 + 1 = 2

9. Example2: Blending Crossover Blending Crossover for real vectors is geometric under Euclidean Distance P1 P2 C ED(P1,C)+ED(C,P2)=ED(P1,P2)

10. Geometric definitions with probability distributions Uniform geometric crossover: Uniform geometric ε-mutation:

11. Representation independent and formal definition of crossover and mutation in the search space seen as a geometric space

12. II. Unification of Major Representations & Operators

13. Minkowski spaces – real vectors 2 2 2 B((2, 2); 1) Euclidean space 2 B((2, 2); 1) Manhattan space Balls 2 2 B((2, 2); 1) Chessboard space 1 2 1 2 1 3 [(1, 1); (3, 2)] 1 geodesic Euclidean space 1 3 [(1, 1); (3, 2)] = [(1, 2); (3, 1)] infinitely many geodesics Manhattan space Line segments 1 2 1 3 [(1, 1); (3, 2)] infinitely many geodesics Chessboard space Representation: real vectors Neighbourhoods: continuous (3 types) Distances: Minkowski distances Implementation: algebraic manipulation of real vector (equation of line passing through two points) Pre-existing recombination operators: - both blend crossovers and discrete crossovers fit geometric definition - extended blend crossovers do not fit

14. Hamming spaces – binary strings 00 01 02 10 11 12 20 21 22 00 01 02 10 11 12 20 21 22 B(00;1) Hamming space H(2,3) [00;11]=[01;10] 2 geodesics Hamming space H(2,3) 000001 010011 100101 111110 B(000; 1) Hamming space H(3,2) 000001 010011 100101 111 110 [000; 011] = [001; 010] 2 geodesics Hamming space H(3,2) Representation: binary/multary strings Neighbourhoods: bit-flip/site substitution Distances: Hamming distances Implementation: symbolic manipulation of multary strings (mask-based crossovers) Pre-existing recombination operators: - all binary crossovers fit the geometric definition

15. Cayley spaces - permutations Representation: permutations Neighbourhoods: adj. swap, swap, reversal, insertion Distances: corresponding distances Implementation: “minimal permutation sorting by X move” algorithms: - adj. swap = bubble sort - swap = selection sort - insertion = insertion sort - reversal = approximated MPS by reversals (NP-Hard)) Pre-existing recombination operators: various pre-existing crossover operators are sorting algorithm in disguise (because sorting permutations is easier than sorting vectors of other items) abc bac acb bca cab cba B(abc; 1) Adjacent swap space abc bac acb bca cab cba [abc; bca] 1 geodesic Adjacent swap space B(abc; 1) Swap space & Reversal space abc bac acb bca cab cba abc bac acb bca cab cba [abc; bca] 3 geodesics Swap space & Reversal space B(abc; 1) Insertion space [abc; bca] 1 geodesic Insertion space abc bac acb bca cab cba abc bac acb bca cab cba

16. Syntactic tree spaces Representation: syntactic tree (lisp expression) Neighbourhood: weighted sub-tree neighbourhood Distance: structural distance Implementation: - sub-tree swap crossover - common region mask based crossover Pre-existing recombination operators: - traditional crossover (non-geometric) - homologous crossover - the geometric framework can help to clarify what is the landscape and distance related to homologous crossover and a distance connected with a geometric crossover which traditional crossover is an approximation + sin + xxx * * * yx * yy Parent 1Parent 2 y + sin x * * yy x Alignment Crossover Point Swap * * yy + xx Offspring 1 Offspring 2

17. Significance of Unification Most of the pre-existing crossover operators for major representations fit geometric definition Established pre-existing operators have emerged from experimental work done by generations of practitioners over decades Geometric crossover compresses in a simple formula an empirical phenomenon

18. IV. Crossover Principled Design

19. Crossover Principled Design Domain specific solution representation is effective Problem: for non-standard representations it is not clear how crossover should look like But: given a combinatorial problem you may know already a good neighbourhood structure Geometric Interpretation of Crossover  Give me your neighbourhood definition and I give you a crossover definition

20. += ? Crossover Design Example

21. Non-labelled graph neighbourhood MOVE: Insert/remove an edge Fixed number of nodes 0 1 2 1 2 3

22. + Offspring

23. V. Is Biological Recombination Geometric? Yes, come to my other presentation at EuroGP!

24. VI. Unity of Evolutionary Search

25. Example of evolutionary search

26. Abstract convex evolutionary search Main result: an evolutionary algorithm using geometric crossover with any probability distribution, any kind of representation, any problem, any selection and replacement mechanism, does the same search: convex search Proof based on abstract convexity (axiomatic geodesic convexity) and axiomatization of search process (abstract search process)

27. …Nearly Over!

28. Summary Unification (meaning): formally dealing with all representations at once Geometric Definition: unif. is possible by defining operators geometrically Unification: many interesting recombinations are geometric Crossover design: by specification of geometric definition to a new representation General theory: using formal definition only, all EAs do the same search: convex

29. Thanks to the Reviewers Franz: thanks for all your suggestions, I’d be glad to talk with you over a coffee… Mario?: thanks for the enthusiastic support A fan?: thanks for warning me that I may be victim of a geometric hallucination…

30. Questions?

31. Geometric Crossover & Path-relinking Meta-heuristic Path-relinking: searches on path between solutions in the neighbourhood structure (not necessarily on a shortest path) Geometric crossover can be understood as a formalized generalization (to metric spaces) of PR that elicits the dual relationship between distance and solution representation and gives a formal recipe to design new crossover operators Formalized: it allows theory Generalization: metric spaces are more general than graphs Elicits duality: syntactic recombination is equivalent to neighbourhood search Crossover design: tells how to build crossovers rather than how to search the search space Formal recipe: it defines exactly what crossover is for any representation