Metaheuristics Lec - 7.

Metaheuristics Lec - 7

Coevolution

Meaning of Coevolution
The classic notion of Coevolution: different species acting as foils against one another and causing one another to adapt. Example: a particular fungus might develop an antibiotic to kill a particular bacterium, the bacterium then adapts to not be harmed by that antibiotic forcing the fungus to construct a tougher antibiotic.

Meaning of Coevolution
symbiotic relationships: close and often long-term interactions between different biological species. Coevolution could also include symbiotic relationships: leafcutter ants and the fungus they farm for food both co-adapting to work better as a team, so to speak.

Coevolution and Metaheuristics
Coevolution is also a common framework in metaheuristics, usually applied to population-based optimization methods. Coevolution generally refers to situations where the fitnesses of individuals in a population are affected by the presence of other individuals in the population(s). the question of whether individual A is superior to individual B is dependent on the presence or absence of some other individual or individuals C in the population.

Coevolution and Metaheuristics
Example: Assume that the fitness of an individual is based on competing with other individuals in the population in Tic-Tac-Toe. A usually wins more games than B does, so A has a higher fitness. But whenever A plays C, A loses badly. Curiously C always loses to B! So if C is in the population and playing, then A’s fitness may drop to less than B’s The fitness is context-sensitive.

Coevolution: Motivation
Coevolution can allow a system to gracefully ramp up in difficulty, It provides diversity in the system It can discover high quality and robust solutions, It can solve complex, high-dimensional problems.

Coevolution vs Evoltuion
Coevolutionary algorithms should have instead been called evolutionary algorithms, because the algorithms we call evolutionary algorithms really have little to do with evolution. Evolutionary algorithms are more like dog breeding: you select and breed the dogs you’re doing a form of artificial directed selection. In real natural selection fitness is context sensitive individuals survive based on the makeup of their particular populations (helping and/or competing with them) and the presence of certain predators or prey. So, coevolutionary algorithms basically follow natural evolutionary concepts.

Main Coevolution Techniques
1-Population Competitive Coevolution 2-Population Competitive Coevolution N-Population Cooperative Coevolution Diversity Maintenance (Niching)

1-Population Competitive Coevolution
Individuals in a single population base their fitness on games they play against one another. Commonly used to evolve good competitive strategies Example: for checkers or soccer

The population is broken into two subpopulations. The fitness of an individual in subpopulation 1 is based on how many individuals in subpopulation 2 it is able to defeat in some competition (and vice versa). Commonly subpopulation 1 contains the candidate solutions of interest to us, and subpopulation 2 contains foils meant to test them.

N-Population Cooperative Coevolution
The problem to be solved is divided into n subproblems Example: Say, the problem is to find soccer strategies for a team of n robots it’s divided into subproblems: Each subproblem finds a strategy for one of the robots. The task of finding each of these subsolutions is given to each of n subpopulations.

The fitness of an individual (in this case, a robot strategy) is assessed as follows: select individuals from the other subpopulations group them with the individual to form a complete n-sized solution determine the fitness of that solution. Commonly used to reduce the high dimensionality of big problems by decomposing them into multiple simpler problems.

Diversity Maintenance (Niching)
The goal is to impose diversity on the population as follows: Individuals in a single population are forced to spread portions of their fitness to one another or to be grouped into competitive niches (or species)

Variations: Cooperative-Competitive Coevolution
a solution is composed of multiple subpieces as in N-Population Cooperative Coevolution. But the solution is tested by playing it against other such solutions in a game, as in 1-Population Competitive Coevolution.

Variations: Cooperative-Competitive Coevolution
Example: imagine that we’re looking for soccer teams consisting of a goalie, forwards, midfielders, and defenders. We have four subpopulations An individual (a goalie, say) is assessed by selecting from these subpopulations to form a team which includes that individual. Then we select from the subpopulations to form an opposing team, and play a game of soccer

Co-adaption Coevolution is most commonly seen in population-based methods (notably Evolutionary Computation) But there is no reason why it can’t be applied in limited form to single-state metaheuristics. For example to use a hill-climber, just define your “population” as your current two individuals (parent and child). Usually, we call such algorithms co-adaptive rather than coevolved.

Fitness in Coevolution
We no longer have an absolute fitness. The fitness of an individual now is a relative fitness It is based on how it performed in the context of individuals in the same optimization process.

Fitness in Coevolution
For example if the individuals were competing against one another in a game, an individual in generation 1 might have a decent fitness because the other individuals in generation 1 are awful but if it was magically teleported to generation 100 its fitness would be terrible because others have improved

Two Issues First, it interacts with the dynamics of selection and breeding, This can result in problematic operation of the system Second, it’s a big problem if we want to assess how well the algorithm is doing. Previously we could just sample individuals out of each generation and see their fitnesses going up and up. But now it’s more likely that the individuals will be improving, but the fitnesses will be staying roughly the same because their opponents are improving as well.

2 Fitnesses: Internal and External
internal fitness is a measure used by the optimization system to determine selection. can be either relative or absolute, Relative internal fitness may change based on the tests performed in the context of other individuals in the population. Thus you may need to re-assess the relative internal fitness of all individuals each time around. We will use the function AssessInternalFitness(...)

2 Fitnesses: Internal and External
external fitness is a measure used to examine the quality of an individual in order to gauge the progress of the algorithm. external fitness should be absolute. We will use the function AssessExternalFitness(...)

Tests Sometimes fitness assessments are done by doing a collection of tests on your individual commonly the fitness is the sum or average of the test results.

Tests Why such Tests? This might be because you have a fixed set of test cases It can also be because you have a very large (often infinite) set of possible situations for which you’d like your individual to be optimal, or at least pretty good. Such tests thus would sample a lot of places in the situation space. You might also perform multiple tests with an individual to find a location in the space where it is particularly impressive, even if it’s not impressive everywhere. Robust Individual: An individual which is good in lots of situations, even if not optimal anywhere in particular.

Multiple Tests Fitness Assessment
Multiple-test fitness assessment shows up over and over in coevolution. It’s a natural fit because if you’re testing an individual against other individuals, you might as well test against a bunch of them to get a good sample. Each of the methods discussed here will employ fitness tests for different purposes. So the algorithms usually will have two stages to fitness assessment: first, gather some m tests for each individual (using Test(...)) in the context of other individuals, then assess the fitness of the individual based on those tests.

It is mostly used for optimizing candidate solutions designed to compete in some kind of games. The idea is simple: each individual’s fitness is assessed by playing that individual against other individuals in the population in the game of interest Example: search for good-quality checkers players search for robot soccer team strategies.

Intuition The intuition behind this idea is to improve the learning gradient of the search space.

Intuition Example: Say you’re trying to find a good poker player.
You may build a really good “guru” poker player by hand, then assess the fitness of individuals based on how many hands, out of n, in which they beat the guru. The problem is that the vast majority of random individuals are awful: they lose every single hand to the guru. Early on your population will consist entirely of these individuals and so the optimization system wouldn’t be able to tell which of the individuals were better than the others. The quality function would be flat until you find an individual who’s sometimes able to beat the guru, which is rare. If you are so lucky, then it’s easier going from there.

The Situation: Needle-in-a-Haystack
Basically your system can’t get started because there’s no way to differentiate among all the initial individuals because they’re so bad against the guru. Until you get players who can start beating the guru sometimes, it’s essentially a needle-in-a-haystack scenario.

A Fix A special way of assessing fitness among the individuals who always lose against the guru: how badly did they lose? Did they do something smart somewhere? Etc. But this is often quite complicated to do.

A Fix Alternatively you could create a panel of hand-built custom players: from very simple, stupid ones, all the way up to the guru. Individuals would be assessed based on how many of the custom players they beat. This would present a more gentle hill for the system to climb up.

Question??? Now the question is, if you’re smart enough to be able to construct a wide range of poker playing programs to test your individuals, why in the world would you be using stochastic optimization to find a good poker player?

A better Fix: 1-Population Competitive Coevolution
have your individuals play each other. Initially, an individual is bad, but the players he’s being tested against are bad too. Someone’s got to win the poker hand, so your players won’t have all zero fitness. Your optimization algorithm can tell which ones are (marginally) better. Later on, as the individuals improve, so do the players against which they are being tested. As your individuals get better, the problem naturally gets harder. We’ve created an automatically self-adjusting learning gradient.

Ways to Compute External Fitness
Test against a guru or against a panel of hand-created players. Test against a sample of players from previous generations (assuming they’re not as good). Here you make a big assumption that your later players are better than your earlier individuals. Essentially your external fitness wouldn’t be an absolute measure but relative to other players, which could create some odd results if the system’s not stable. Test against some external system in the real world.

Used for selection and breeding
Used to determine algorithm progress and Best

Relative Internal Fitness Assessment
There are a variety of ways that one might assess the fitness of individuals by putting them against other individuals in the population. The primary issue that you’re going to face is the number of tests (evaluations) involved in order to assess the fitness of the population. Many games, competitions, or similar things are stochastic: you might win one time, lose another. For example, in poker, you might need to play a bunch of hands to determine which player is doing better.

Relative Internal Fitness Assessment
Even if your game isn’t stochastic you still may face some issues: A common major pathology in 1-Population Competitive coevolution is the presence of cycles A beats B, and B beats C, but C beats A. Who is better? As another case, perhaps A beats B, but B beats far more individuals than A beats. Who’s better in that situation? It depends on what you’re looking for. Often the case is that you’re looking for the individual which wins against the most players; or against the most “good” players, or wins by the most total points on average. In such cases it’s common to require multiple tests to identify which individuals are really the better ones.

The Tradeoff The tradeoff here is how many tests you need to run.
Normally you have a fixed budget of tests. How to spend that budget is yet another parameter to deal with. you can spend them on more precisely determining fitness; or you can spend them on searching further.

One Simple Approach Pair off all the players,
have the pairs play each other, and use the results as fitness. tests two individuals (plays them against each other) and stores the results

Advantage: only requiring |P|/2 tests to assess the fitness of individuals in the population. Pair off all the players, have the pairs play each other, and use the results as fitness. tests two individuals (plays them against each other) and stores the results

Advantage: only requiring |P|/2 tests to assess the fitness of individuals in the population. Pair off all the players, have the pairs play each other, and use the results as fitness. Disadvantage: each individual only gets to be tested against one other individual, which is probably very noisy. tests two individuals (plays them against each other) and stores the results

A Second Approach test individuals against every other individual in the population. The fitness is then based on the various tests the individual received In total we have (|P| x |P| - 1) /2 tests!!! each individual is involved in |P| - 1 tests Using the results of all tests inolving P_i

A Third Approach: Middle Ground
testing each individual against some k other individuals in the population, chosen at random. Some individuals will ultimately get tested more than k times, but each individual will get at least k tests to assess its fitness. each individual is involved in at least k tests Using the results of all tests inolving P_i

A slight Variation keep almost exactly k tests for each individual
Keep total test <= k x |P| Hold individuals who haven’t have enough tests yet

R Holds individuals who haven’t have enough tests yet
For each individual, we want to ensure almost k tests P_i may already be involved in k tests and hence may be out of R. But normally, we need to remove P_i from R because we will be involving it in k tests now.

The main loop where we involve P_i in k tests

The idea is to take k random individual from R and pair each with P_i

The idea is to take k random individual from R and pair each with P_i Bur R may not contain enough individuals! So, choose k - |R| individuals from P-R and add all from R.

The idea is to take k random individual from R and pair each with P_i Bur R may not contain enough individuals! So, choose k - |R| individuals from P-R and add all from R. Test P_i against each individual in Q. So, we have k tests for P_i If a Q_i gets involved in k tests already, remove it from R

Another Approach: Single Elimination tournament
Even k x |P| may be too many tests If fitness is done simply by counting number of games won (as opposed to total score, etc.), we may involve the entire population in a big single elimination tournament. The fitness of an individual is how high it rises in the tournament

Q gets the individuals who were undefeated last iteration
R stores the individuals who are undefeated in current iteration

Q gets the individuals who were undefeated last iteration
The whole procedure is limited log_2 |P| times. This ensures that the total number of tests is |P| - 1. Q gets the individuals who were undefeated last iteration R stores the individuals who are undefeated in current iteration Doing the tests consecutive pairwise

2^0 + 2^ 1 + 2^2 + … + 2^{lg P - 1} = 1 – 2^{lg P}/ (1 - 2) =P - 1

Interesting Properties
individuals which are better are involved in more tests. In some sense this lets us be a bit more sharp in distinguishing among our better players. The disadvantage is that if the games are fairly noisy, then a good player might be accidentally lost in the rabble. However, Single-Elimination Tournament has often performed very well, if your desired test metric is simply based on games won or lost (rather than points).

Fitnessless Selection
Rather than use these methods to compute fitness, we might simply use them to determine the winner in Tournament Selection without ever computing a fitness at all! Example: Say, we’re using Tournament Selection with a tournament of size 2. We need to select an individual. To do so, we pick two individuals at random from the population and have them play a game right then and there. The winner is the one selected. if our tournament size n is a power of 2, we could select n unique individuals at random from the population, and put them through a little single-elimination tournament. The winner of the tournament is selected. We could hash the tournament results to avoid replaying individuals in the off-chance they’ve played against one another before.

Two Notes Improving the Gradient: We can improve even further by playing against ones in the previous generations. Local Optima: The whole population may get stuck in notorious local optima: Consider an optimal soccer team, your initial population might contain terrible arbitrary players, but one or two of them contain players which simply go to the ball and kick it into the goal. These rack up such huge scores against the terrible players that soon the entire population consists of teams of players who all go straight to the ball and try to kick it to the goal, and the population just won’t budge from that local optimum, forcing you to rethink how to assess fitness

The idea is simple and appealing: we construct not one but two populations (or if you like, subpopulations or demes). Population P will contain the individuals we’re trying to robustify. Population Q will contain test cases to challenge the individuals in P. The fitness of individuals in population P will be based on how well they perform against the individuals in population Q, and likewise, the individuals in Q will be assessed based on how well they perform against the individuals in population P.

2-Populations: P and Q P is the primary population
Q is the alternative or foil population. Ultimately we’re really just interested in the primary population. Thus, we only bother to assess the external fitness of population P, and likewise only maintain the best individuals from P. Rarely are the best of Q interesting. When we test one population in the context of the other (be it P or Q), the other population is known as the collaborating population.

An Example: Optimal Sorting Network
A sorting network is a series of comparisons on elements in the array which, when completed, results in the array being sorted. Each comparison compares two fixed elements in the array. If the upper item is smaller than the lower item, then the two items are swapped. Some comparisons can be done in parallel, and the whole mechanism is attractive because it can be done in hardware. The objective is not only to find a correct sorting network, but one which has the fewest comparisons possible.

A sorting network for an array of four numbers. Vertical lines indicate comparisons. Progresses left to right. Two of the comparisons can be done in parallel.

The idea is to coevolve a population of sorting networks against a competing population of hard-to- sort arrays of numbers. The fitness of a sorting network is the number of arrays it got correct; and the fitness of an array was the number of sorting networks it stumped. Thus while the sorting networks were improving themselves, the arrays were finding harder corner cases to challenge the sorting networks.

Interleaving Fitness Assessment and Breeding

Fitness Assessment and Breeding
2-Population Competitive coevolution introduces different ways to interleave fitness assessment and breeding. Sequential (or Serial) 2-Population Competitive Coevolution Parallel 2-Population Competitive Coevolution Parallel Previous 2-Population Competitive Coevolution

Sequential (or Serial) 2-Population Competitive Coevolution

Sequential (or Serial) 2-Population Competitive Coevolution
Assess the internal fitness of P in the context of Q and also assess P’s external fitness Then breed P, then assess the internal fitness of Q in the context of the new P, then breed Q, and so forth.

Used to determine algorithm progress and Best

Generation i

AssessInternalFitness(P,Q)
We need some way to assess the internal fitness of a population in the context of its collaborating population. The straightforward way to do it is to sample k individuals from the collaborating population to test against.

Disadvantages Sequential 2-Population Competitive Coevolution has two downsides First, Q must be tested against the new, improved P, always one step ahead of it. Second, the assessments are separated, which means you can’t combine AssessInternalFitness(P,Q) and AssessInternalFitness(Q, P). Probably when you were testing P, you got some fitness information about individuals in Q as well. Why throw that information away?

Parallel 2-Population Competitive Coevolution

Parallel 2-Population Competitive Coevolution
Here, each population is tested against the other, and then both of them breed. This solves both of the problems of Sequential (or Serial) 2-Population Competitive Coevolution: Neither population has a leg up on the other by design and we can group internal fitness assessment together for both populations:

Internal Assessments could be done simultaneously

Doing Internal Fitness Together
Assume that the two populations were the same size: we could shuffle P, then test each individual P_i against the corresponding individual Q_i. To do further tests, we might test each P_i against Q_{i+1}, then Q_{i+2}, and so on, wrapping around as necessary. But this creates statistical dependencies among the tests: for example, individuals P_i and P_{i+1} would be tested against almost exactly the same individuals, which is probably not good.

Doing Internal Fitness Together
We could shuffle the population P each time, but then we’d like to guarantee that in the shuffling certain individuals never get paired up again if they’ve been tested together before. So, shuffle P each time, then as long as there is a pair that’s already been tested before, we break up that pair

We want to do k tests for each P_i \in P
Test each P_i’ against Q_i Shuffle again! A previous tested pair is broken by swapping P_i’ with a random P_\ell’. Note thewhile loop.

One Problem for both Sequential and Parallel…
External fitness assessment can be a problem for both of these options You can’t test against Q per se because Q keeps changing (and ideally improving), So you won’t get a consistent, absolute fitness metric for P. How to solve this? Perhaps you could create a fixed sample drawn from the test-case space and test against that; or create a guru of some sort.

Parallel Previous 2-Population Competitive Coevolution

Main Idea Here we assess each population against the previous generation of the collaborating population. Why?? This might help improve the gradient a bit because each population is given a bit easier time. Except for the first generation, we’re back to testing populations separately again.

Internal Assessments could be done simultaneously for the first generation
We do the breeding first so that we can test the new generation against the old We do the test against previous generation: P’ against Q and Q’ against P

Internal Assessments done simultaneously for the first generation

Breeding of the first generation

Internal Assessments done separately for the second generation

Breeding of the second generation

Internal Assessments done separately for the third generation

Breeding of the third generation

Internal Assessments done separately for the fourth generation

Mix and Match Mix of the Parallel and Parallel Previous methods: test an individual against some k individuals from the current-generation collaborating population, and also against the n fittest individuals from the previous-generation collaborating population. For 1-Population Competitive Coevolution we can do the following: M (i.e., Population to be tested) is the current generation of the individuals C (i.e., Collaborative Population) is the previous generation of the same individuals.

Arms Races 2-Population Competitive Coevolution is often viewed as an abstract version of a biological arms race: one population learns a trick, forcing the second population to learn a new trick to beat the first one, and so on. In an ideal world, the arms race results in a natural, gradual build-up of gradient

Arms Races and Loss of Gradient
The following may happen… One population may have an easier optimization task, and so it improves so rapidly that it leaves the other population in the dust. At some point all the individuals in one population (say, Q) are so good that they all defeat the individuals in P soundly. When this happens, all the individuals in Q now basically have all the same fitness, because they all beat everyone in P. Likewise all the individuals in P have the same fitness because they all lose to everyone in Q.

Loss of Gradient… the selection operator no longer has anything to go on, So it starts picking individuals at random. This usually causes the external fitness to start dropping until the populations reestablish gradient again One population periodically improves so much that selection starts to fail, causing a drop in fitness.

How to Fix? if you can somehow detect that a population is improving too rapidly and loss of gradient is seeping in, you might pause the evolution of that population until gradient is reestablished.

Main Idea Competitive coevolution tries to improve individuals by pitting them against other individuals. In contrast, cooperative coevolution strives to find individuals who work well together. Why would you want to do that? Some optimization spaces are high-dimensional and gigantic. Cooperative coevolution simplifies those spaces by breaking them into multiple, much simpler, subspaces for different populations to search.

An Example… Let’s say you’re looking for an optimal robot soccer team.
Each team has eleven different players, Say, we have decided that each player must have unique robot behaviors Assume that there are 1000 different behaviors for each robot.

An Example…[Idea #1] Try to optimize the behavior of each of the 11 robots searching the whole search space. Here, you are trying to find an optimal setting in a space of 1000^11 possibilities!!!

An Example…[Idea #2] For each robot, create an arbitrary set of behaviors for everyone but that robot. Then use an optimization method to find the optimal robot behavior given this arbitrary team of fixed collaborating robots. At the end, take the optimal robots and put them together to form a final team. This requires 11 optimization runs, each of which is over a simple space of only 1000 possibilities. However there’s no guarantee that those robots will work together well at the end: after all they were not designed for one another but rather were designed to be the best robot given the arbitrary team they were forced to work with.

An Example: Middle Ground
N-Population Cooperative Coevolution strikes a middle- ground between these two situations. We perform 11 optimization runs at one time but individuals in each run are tested by grouping them with the current individuals from the other runs. We do this by creating 11 populations: a population of goalies, a population of left fullbacks, a population of right fullbacks, ... etc. Each population has its own optimization process. When an individual from a population needs to be tested as part of his fitness assessment, we group it with individuals selected from the other populations to form a complete soccer team. We then play a match, and the match results become the individual’s test results.

In short, each of the populations in N-Population Cooperative Coevolution is finding an optimal subsolution: a sub-part of the complete solution. This gives a huge reduction in complexity. In 2-Population Competitive Coevolution, there was one specific population of interest to us. But now all the populations are of interest to us because they are each solving a piece of the problem.

Rather than show Internal and External fitness assessment, we’ll just assess a joint fitness

We store in Best the fittest joint vector of individuals, one from each population, that we’ve found so far.

Optimization process for Population I: In the for loop, we assess some joint fitnesses but only apply them to the individuals in Population i. To test, we group it with individuals selected from the other populations We then do the fitness assessment.

How to do the fitness assessment?
Note that in the For-loop we assess some joint fitnesses but only apply them to the individuals in population P(i). We could do that as follows: For each individual in P(i) we perform some k tests by grouping that individual with randomly-chosen individuals from the other populations to form a complete solution:

Initially empty! We will fill it up with individuals
w keeps track of the number of tests l indexes the vector s!. Makes sure that s! shall contain the (current) individual P_j^(i) Other individuals of s! are chosen at random. \ell th entry of s! takes a random individual from P(\ell)

Sequential Approach Combination of Algorithms 88 and 89 is a sequential approach it’s wasteful because we do many tests but only use them to assess the fitness of a single individual—the collaborators are forgotten about. We could fix that by keeping around the previous tests and including them when we get around to testing the collaborating individuals for their fitness assessment. Or we could just do the Parallel approach.

Parallel Approach we can group all the joint fitnesses together at one time Thus, we can save some testing time by not doing further tests on collaborators who’ve been involved in a sufficient number of tests already. We could do this with a variation of Algorithm 85, but with N > 2 it might suffice to just pick collaborators at random, even if some by chance get tested more than others, hence:

Initially empty! We will fill it up with individuals
w keeps track of the number of tests l indexes the vector s!. Makes sure that s! shall contain the (current) individual P_j^(i) Other individuals of s! are chosen at random. \ell th entry of s! takes a random individual from P(\ell) Fitness assessment for all is done after the test is done for all

Pathological Conditions
Laziness relative overgeneralization miscoordination

Laziness If certain populations are doing impressively, other populations may just “be lazy”.

An Example of Laziness Say you’re trying to find an optimal team of basketball players. You’ve got a population of centers, of forwards, of guards, etc. Your guard population has converged largely to consist of copies of Michael Jordan. The Michael Jordans are so impressive that the population of (say) forwards doesn’t need to do any work for the team to be near optimal. In essence, all the forwards’ fitnesses look the same to the system: regardless of the forward selected, the team does really really well. So the system winds up selecting forwards at random and the forwards don’t improve. This condition is the cooperative equivalent of the Loss of Gradient pathology

Solution to Laziness The basic solution to this is to change your fitness function to be more sensitive to how the forwards are doing. For example, you might apply some kind of credit assignment scheme to assign the fitness differently to different cooperating individuals. Be careful: the system is now likely no longer cooperative, that is, coordinating individuals no longer receive the same fitness, and this can result in unexpected dynamics.

Relative Overgeneralization
Consider the question: How do you assess the fitness of a cooperative coevolutionary individual based on tests? One obvious choice is to average the test results with various collaborators from the other population(s) What is the problem here? Say, there is one optimal joint solution, but the hill leading to it is very small; whereas there’s a large suboptimal peak elsewhere See the next slide:

Relative Overgeneralization
If we tested individuals A1 and A2 with many individuals from Population B and took the average, A1 would appear fitter on average even though A2 was actually a collaborator in the optimum. A1 is a jack-of-all-trades-but-master-of none individual most of the time it’s involved in a joint solution that’s better than average. So, the populations converge to joint solutions which are suboptimal, but involve lots of jacks-of-all-trades.

Solution to Relative Overgeneralization
The way to fix this is to assess fitness as the maximum of the tests rather than their average. However to get good results you may need to do a lot of tests, perhaps even against the entire other population. It turns out that usually there are just a few “special” collaborators in the other population(s) which, if you tested just with them, would compute fitness orderings for your entire population in exactly the same way as testing against everyone. So, a great task would be to computes this archive of special collaborators, resulting in far fewer tests

Miscoordination Finally, if your fitness function has multiple global optima, or near-optima, you could also wind up victim to miscoordination.

Miscoordination Let’s say you have two cooperating populations, A and B, and two global optima, 1 and 2. The two optima are offset from one another Population A has discovered an individual A1 who is part of global optimum 1 (yay!), and likewise Population B has discovered an individual B2 who is part of global optimum 2 (yay!).

Miscoordination But neither of these individuals will survive,
because Population A hasn’t yet discovered individual A2 who, when collaborating with B2, would help B2 shine. Likewise Population B hasn’t yet found individual B1 who would make A1 look great. In the worst case, these populations are trying out A1 and B2 in combination, which winds up in a quite suboptimal region of the joint space. Thus, though A1 and B2 are optimal for their respective populations, the populations can’t tell: they look bad.

Metaheuristics Lec - 7.

Similar presentations

Presentation on theme: "Metaheuristics Lec - 7."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Metaheuristics Lec - 7.

Similar presentations

Presentation on theme: "Metaheuristics Lec - 7."— Presentation transcript:

Similar presentations

About project

Feedback