1. Lecture 2. Co-Evolution (II)
학습목표
공진화와 관련된 다양한 방법론을 이해하고, 응용예를 통한 실제 적용가능성을 점검한다.

2. Outline Review of the last lecture Diploid gene representation
Parallel evolutionary algorithms (fine-grained): Local selection, Recombination
Different types of co-evolution
Inter-population co-evolution: An example in design, Knowledge discovery (data mining), Interactive evolution
Intra-population co-evolution: Co-evolving a backgammon player, Taking two interwound spirals apart by co-evolution, Iterated prisoner’s dilemma
Summary and overviews of other related work
Why co-evolutionary learning
More examples and open issues

3. Different Types of Co-Evolution
Based on the number of population involved:
Inter-population co-evolution
Two or more populations
Intra-population co-evolution
Within a single population
Based on the relationship among individuals
Competitive co-evolution
Individuals compete for higher fitness to solve a problem (often a dynamic problem)
Cooperative co-evolution
Individuals cooperate with other in order to solve a problem

4. Co-Evolution in Design (1)
Many design problems do not have a fixed goal or a fixed set of specifications
People do change minds
Current
Problem k
evolve
Current
Problem
k+1 … …
provide
fitness
provide
fitness
provide
fitness
Current
Solution
k+1
Current
Solution k
evolve … …

5. Co-Evolution in Design (2)
Task: floor plan design
A candidate solution
Adjacency graph (representing requirements)
kitchen
dining
room
bed1
bed2
bed3
lounge
hall
corridor wc
ensuite
kitchen
corridor
dining
lounge wc
bed ensuite
hall
bed3
bed2

6. Co-Evolution in Design (3)
Representation
Solution space
Problem (specification) space
genotype
phenotype
mapping 2
mapping 1 3 4 5
(pen movement)
(floor plan)
genotype A
mapping A B E
A B C D E BC CD E B C D
(adjacency) D C

7. Co-Evolution in Design (4)
Fitness evaluation
Solution space / population
Fitness = (Basic initial requirements) + (current best problem)
Problem space: Fitness = (current best solution)
What does “best” mean in this case?
Answer: How well an individual matches a floor plan (or adjacency graph)
solutions
(floor plans)
best
problems
(adjacency graphs)
provide
fitness

8. Knowledge Discovery (Data Mining)
One application: fraud detection
We want to discover all kinds of frauds, but we do not really know what they are (what kind of patterns they have)
In general, we want to find something interesting without knowing what “interesting” actually means
corporate
database
top performing
rules
rules
evolving
new
interestingness
analysis
human ranking
Interestingness is co-evolving with interesting rules

9. Interactive Evolution
Evolution with a human being in the loop
Often used in creative design or creative problem solving
May be time-consuming
Case1: no co-evolution
Case 2: co-evolution
population
fitness evaluation
by human
replacement
genetic operation
computer
programs
fitness
evaluation
human
beings

10. Intra-Population Co-Evolution
There is only one population
However, fitness of one individual depends on other individuals in the population
Example 1: Playing backgammon
TD-Gammon
a grand master level computer program based on NN
It learned by self-playing
Self-playing = Co-evolution
Is it because machine learning algorithms or co-evolution?

11. Co-Evolving Backgammon Players (1)
Simple neural network without any fancy learning algorithm except for hill-climbing
Simple EA with population size 1, hardly an EA!
Simple mutation with Gaussian noise
No recombination at all
Task = evolve an NN that plays backgammon

12. Co-Evolving Backgammon Players (2)
Task = evolve an NN that plays backgammon
feed-forward fully connected NN
Initial weights were 0’s
1. Let the initial NN be NNk, k  0
2. Generate a mutant challenger of NNk
w’ij = wij + G(0, s)
3. If NN’k is beaten by NNk, NNk+1 = NNk
Else NNk+1 = NNk* *NN’k
4. k  k+1, goto step 2
Performance: Winning 40% of the games against PUBEVAL after 100,000 generations
strong program trained by experts

13. Separating Interwound Two Spirals
Task: Given 194 training points, learn to separate two spirals
Very tough problem for machine learning algorithms, e.g., decision trees, neural networks, …
There was an attempt to evolve a solution to it. But the solution generalized poorly
Competitive co-evolution based on covering really helps
Fitness evaluation without co-evolution: number of test cases correctly classified
Fitness evaluation with co-evolution: based on pair-wise competition, it depends only on the number of test cases correctly classified but NOT covered by its opponent

14. Iterated Prisoner’s Dilemma
Non-zero sum, non-cooperative games
The 2 player version
The purpose here is not to find the optimal solution for some simplified conditions, but to study how to find it
Fitness evaluation
Entirely determined by the total payoff obtained through playing against each other
The initial population was generated at random
Player A C D 3 5 C 3
Player B 1 D 5 1

15. Why Co-Evolution We do not know the fitness function
There are too many cases to test in order to obtain a fitness value. Co-evolution can be used to FOCUS search in the most important area
The problem is inherently changing in time
Increase and maintain diversity
Self-learning

16. More Examples of Co-Evolution
Discovering CA rules (using coverage again)
Computer-aided learning (students + software tutors)
Robot morphology and control
Character recognition
Chess playing
International coffee market prices …

17. Open Issue
Forgetting (also known as the Red Queen effect): co-evolution does not have a good memory at present
Mediocre stable state: individuals learn to co-exist with each other and do not want to explore the search space any more
Incremental evolution: continuous improvement without forgetting
References
J. Poon and M.L. Maher, “Emergent behavior in co-evolutionary design,” Artificial Intelligence in Design’96, J. Gero (ed.), Kluwer Academic.
J.B. Pollack, A.D. Blair & M. Lund, “Coevolution of a backgammon player,” Proc. Of the Fifth Alife, May 1996.
H. Juille and J.B. Pollack, “Dynamics of co-evolutionary learning,” Proc. Of the 4th Int. Conf. on Simulation of Adaptive Behaviro, Sept. 1996, MIT Press, pp. 526~534

18. Homework #1 주제: Diversity 유지를 위한 speciation 방법 구현 및 실험 마감일: 9/30 내용:
5가지 평가함수(De Jong function 1~5)에 2가지 종분화 방법(Fitness sharing, Crowding)을 적용하여 진화동안의 diversity 변화를 조사한다
Bonus
Sorting algorithm/problem에 공진화와 종분화를 이용하기
Othello 게임에 공진화와 종분화를 이용하기