Coevolution Chapter 6, Essentials of Metaheuristics, 2013 Spring, 2014 Metaheuristics Byung-Hyun Ha R2R3

1 Outline  Introduction  1-Population competitive coevolution  2-Population competitive coevolution  N-Population cooperative coevolution  Niching: diversity maintenance methods  Summary

2 Introduction  Coevolution in biology  “The change of a biological object triggered by the change of a related object” (from Wikipedia)  In metaheuristics  Considering competitive or cooperative case, usually  To provide diversity in system  To discover not just high-quality but robust solutions  To solve complex and high-dimensional problems by breaking them along semi-decomposable lines a population another population competitive cooperative fitness evaluation by challenging fitness evaluation, jointly

3 Introduction  Examples (Talbi, 2009)  Predator-prey coevolution for constraint satisfaction Main population: potential solutions High-quality: those that satisfy a large number of constraints Secondary population: constraints High-quality: those that are violated by many solutions  Focusing on hard constraints by preferring them  Coevolution for function optimization Rosenbrock Populations Each one representing a decomposed function, f i (x i, x i+1 ) An individual corresponding to x i Competitive coevolution Each population finding local optimum Cooperative coevolution Iterative evolution of a population based on the others

4 Introduction  Types  1-Population competitive coevolution  2-Population competitive coevolution  N-Population cooperative coevolution  Diversity maintenance (Niching) ...  Fitness in coevolution  Assessing fitness Gathering test results of an individual in the context of others Assessing fitness of the individual based on the results  Internal vs. external fitness Relative vs. absolute  Possible problems Selection and breeding Determining progress of search ?

5 1-Population Competitive Coevolution  Optimizing solution designed to compete in some game, mostly  Each individual’s fitness is assessed by playing against other individuals.  e.g., checkers players, robot soccer team strategies  Intuition to internal fitness  Problems in improving learning gradient of search space c.f., situation of beginners evaluated by a guru  Alternatives Assessing how badly one loses Employing panels of various levels of skill Making them play each other Giving self-adjusting learning gradient ?

6 1-Population Competitive Coevolution  Assessing external fitness  Testing against guru  Testing against players from previous generations  Testing against some external system ;-)  An abstract procedure

7 1-Population Competitive Coevolution  Relative internal fitness assessment (n individuals)  c.f., cost of testing  Pairwise relative fitness assessment n/2 tests (problem?)  Complete relative fitness assessment n(n – 1)/2 tests (too many?)  K-fold relative fitness assessment kn tests (still many?) c.f., More precise K-fold relative fitness assessment  Single-elimination tournament relative fitness assessment (n – 1) tests Better individuals, more tests, but fairly noisy  Fitnessless selection  e.g., using tournament selection with size 2  c.f., not totally free from prematurity

8 2-Population Competitive Coevolution  Role of two populations  Primary population Looking for good (and robust) solutions  Alternative (or foil) population Searching for most challenging test cases  e.g., Finding sorting network that gives fewest comparisons  Primary: sorting network  Alternative: hard-to-sort arrays of numbers

9 2-Population Competitive Coevolution  Fitness assessment and breeding strategies  Sequential  Parallel Growing up together + reduced number of tests  Parallel Previous Improving gradient a bit

10 2-Population Competitive Coevolution  Fitness assessment  Specific way for each strategy c.f., number of tests, statistical dependency, using fittest individuals,...  Arms races and loss of gradient  P improves too rapidly than others All in P have the same fitness, so do all of Q Selection not working  Solution Pause evolution of too rapidly improving population until gradient recovers? Using Parallel Previous?

11 2-Population Competitive Coevolution  Example of container terminal operations  Container-grounding position determination by weighted sum of scores Solution as a list of weights

12 N-Population Cooperative Coevolution  Attacking a problem decomposable into subproblems  e.g., approaches robot soccer team with different players 1. a team as an individual 2. a player as an individual 3. middle ground?  Finding good subsolutions and integrating them  A population for each subproblem  Testing a solution by grouping individuals from populations  Assessing an individual’s fitness from test results of group including it  Coevolution strategies  Sequential  Parallel

13 N-Population Cooperative Coevolution  An abstract sequential procedure  Parallel procedure  Fitness assessment

14 N-Population Cooperative Coevolution  Pathological conditions in testing  Possibility of laziness e.g., fitness assessment with one excellent player  Relative overgeneralization c.f., fitness as average of test results, usually  Miscoordination

15 N-Population Cooperative Coevolution  Example of container terminal operations  Stacking container in a yard Two-level approach 1. determining a block by weighted sum of scores 2. determining a stack by weighted sum of scores Solution as a list of weights Employing cooperative coevolution approach

16 Other Variations  Cooperative-Competitive Coevolution ...  Co-adaptive...

17 Niching: Diversity Maintenance Methods  Preventing early convergence (revisited)  Increasing sample (population) size  Adding noise to Tweak procedure  Being less selective among individuals (picking less fit ones more often)  Adding random restarts to system  Adding explicit separation constraints in your population  Adding different individuals from the current ones in the population e.g., Scatter Search with Path Relinking  And more, here..  Punish individuals in some way for being too similar to one another Fitness sharing Crowding

18 Niching: Diversity Maintenance Methods  Fitness sharing  Encouraging diversity in individuals by reducing fitness for being too similar to on another  Sharing function s i receives punishment s(i, j) for j is near i where d(i, j): distance between i and j,  : neighborhood radius  Revised fitness f i = (r i )  /  j s(i, j), where r i is actual fitness  Crowding  Making similar individuals more likely to be picked for death in a steady- state system  Algorithms Restricted Tournament Selection, Deterministic Crowding,...

19 Niching: Diversity Maintenance Methods  Similarity  Three ways, at least Phenotypically: they behave similarly Genotypically: they have roughly the same makeup Individuals may have similar fitness (?)  Examples of measure (Metric) distance e.g., Euclidian distance, Hamming distance Proximity e.g., Jaccard coefficient, cosine similarity  Graphs? trees?

20 Summary  Coevolution  Internal vs. external fitness  Fitness assessment by testing  1-Population competitive coevolution  2-Population competitive coevolution  Fitness assessment and breeding strategies  Arms races and loss of gradient  N-Population cooperative coevolution  A problem decomposable into subproblems  Pathological conditions in testing  Diversity maintenance methods  Fitness sharing, crowding