Download presentation
Presentation is loading. Please wait.
Published byCharleen Lewis Modified over 9 years ago
1
More on coevolution and learning Jing Xiao April, 2008
2
Coevolution Evaluation of individuals is based on interactions with other evolving individuals A potential for open-ended evolution can arise Ranking of individuals is dependent on the co-evolving individuals Inspired by biological coevolution: reciprocal evolutionary change in interacting species
3
Competitive Coevolution Competitive coevolution: individuals compete (e.g., evolution of game players). Test-based problem: the quality of a candidate solution is determined by its performance on a set of tests. Typically, two types of individuals are involved: candidate solutions (learners) and tests (e.g., predators and preys). Tests can also be evolved.
4
Pareto-Coevolution Test-based Problems as Multi- objective Problems Each test represents an objective. Solutions are non-dominated learners. Complete Evaluation Set (De Jong04) A test that makes a distinction between two learners belongs to CES Ideal evaluation is based on CES
5
Cooperative or Compositional Coevolution Solution requires coordinated combination of all types of components, where each component is a type of individual. A static fitness function is used as by standard EAs. An example of this type of problem is to find components (e.g. partial neural networks) that together form an adequate whole. Another example is the identification of combinations of strategies that work well together (such as a team of robots)
6
Competitive (test-based) vs. Cooperative (compositional) A lot of debate on the distinctions between the two Some consider the classification subjective (because the underlying algorithms can be similar) Some problems may involve both elements (e.g. a team sport) Substance is more important
7
Example 1 Potter, M and De Jong, K. (1994). A Cooperative Coevolutionary Approach to Function Optimization. Characterized by the following Initialize a separate population (species) for each numerical function variable a complete solution is composed of a representative from each species fitness of an individual in a species is determined from the function value of a complete solution (or the values of several solutions) involving this individual and individuals of other species, i.e., in terms of how well a individual “collaborates” with other species each species is evolved by a GA (or any evolutionary algorithm) using the best values of the other species
8
Example 1 (cont’d) The performance of this coevolution approach was compared to that of a standard GA on four multimodal functions. In all cases the coevolution approach significantly outperformed the standard GA both in minimum value found and in the speed of convergence. For one of the functions, the advantage of the coevolution approach was less because of inter-dependency of variables.
9
Example 2 Potter, M, De Jong, K and Grefenstette, J (1995). A Coevolutonary Approach to Learning Sequential Decision Rules. Evolve complex behaviors of simulated robots.
10
Example 2 (cont’d) Task environment: A robot (called “friendly robot”) must maneuver itself around an obstacle-free room where food pellets appear at random locations and times. The robot must consume these food pellets to replace lost energy and stay alive. Energy loss is a function of the speed and turning rate of the robot. There is a small energy drain even if the robot is stationary. Another robot (“enemy robot”) is controlled by a fixed set of hand-crafted rules to complete with the friendly robot for food pellets.
11
Example 2 (cont’d) Each robot has 10 sensors and 2 effectors (for speed and turning rate). The sensors consist of a food pellet detector, a clock indicating the amount of time passed since the appearance of the last food pellet, the robot’s current energy level, heading, and speed, the range and bearing to the center of the room, the range and bearing to the other robot, the range and bearing to the food pellet if it is present
12
Example 2 (cont’d) The goal is to evolve a set of behaviors (rules) to enable the friendly robot to survive indefinitely. Fitness of a rule set is the average energy level of the robot over a large number of training episodes for food gathering
13
Example 2 (cont’d) The hand-crafted rules for the enemy robot make it move with a random speed in a slightly wandering fashion towards food pellets when they appear and towards the room center otherwise.
14
Example 2 (cont’d) Two subsets of rules are considered for the friendly robot: Rules for the situation when food is present Rules for the situation when food is absent Initial subpopulations are seeded by dividing initial rules corresponding to a standard non-coevolutionary case (called the Samuel system) into the corresponding subsets.
15
Example 2 Results
16
Example 3 E. Uchibe and M. Asada. (2006) Incremental coevolution with competitive and cooperative tasks in a multi-robot environment. Proceedings of the IEEE, Vol. 94, Issue 7, pages 1412--1424, 2006.
17
Evolutionary Learning Two major approaches: Michigan approach: each individual represents a single rule. The whole pop. represents the complete system Pitt approach: each individual represents a complete system Credit assignment is difficult in the Michigan approach
18
Evolving Rule-Based Systems Encoding rules: many different ways Example: given a set of reactive rules: IF c11, c12, …, c1n THEN a1; IF c21, c22, …, c2n THEN a2; … Encode them in a single chromosome: c11 c12 … c1n a1 c21 c22 … c2n a2 … Order is not essential — set-based
19
Evolving Rule-Based Systems (cont’d) Recombination: Two parent recombination: select randomly two subset rules from two parents and swap them Global discrete recombination: rules in an offspring is selected randomly from all individuals Mutation: Swap subsets of conditions change subsets of conditions or actions
20
Evolving Rule-Based Systems (cont’d) Fitness evaluation: based on training error based on training error plus complexity Based on separate validation set
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.