Lecture 2. Co-Evolution (II) 학습목표 공진화와 관련된 다양한 방법론을 이해하고, 응용예를 통한 실제 적용가능성을 점검한다.

Outline Review of the last lecture Diploid gene representation Parallel evolutionary algorithms (fine-grained): Local selection, Recombination Different types of co-evolution Inter-population co-evolution: An example in design, Knowledge discovery (data mining), Interactive evolution Intra-population co-evolution: Co-evolving a backgammon player, Taking two interwound spirals apart by co-evolution, Iterated prisoner’s dilemma Summary and overviews of other related work Why co-evolutionary learning More examples and open issues

Different Types of Co-Evolution Based on the number of population involved: Inter-population co-evolution Two or more populations Intra-population co-evolution Within a single population Based on the relationship among individuals Competitive co-evolution Individuals compete for higher fitness to solve a problem (often a dynamic problem) Cooperative co-evolution Individuals cooperate with other in order to solve a problem

Co-Evolution in Design (1) Many design problems do not have a fixed goal or a fixed set of specifications People do change minds Current Problem k evolve Current Problem k+1 … … provide fitness provide fitness provide fitness Current Solution k+1 Current Solution k evolve … …

Co-Evolution in Design (2) Task: floor plan design A candidate solution Adjacency graph (representing requirements) kitchen dining room bed1 bed2 bed3 lounge hall corridor wc ensuite kitchen corridor dining lounge wc bed1 ensuite hall bed3 bed2

Co-Evolution in Design (3) Representation Solution space Problem (specification) space genotype phenotype mapping 2 mapping 1 1 2 3 4 5 45 3 4 5 (pen movement) (floor plan) genotype A mapping A B E A B C D E BC CD E B C D (adjacency) D C

Co-Evolution in Design (4) Fitness evaluation Solution space / population Fitness = (Basic initial requirements) + (current best problem) Problem space: Fitness = (current best solution) What does “best” mean in this case? Answer: How well an individual matches a floor plan (or adjacency graph) solutions (floor plans) best problems (adjacency graphs) provide fitness

Knowledge Discovery (Data Mining) One application: fraud detection We want to discover all kinds of frauds, but we do not really know what they are (what kind of patterns they have) In general, we want to find something interesting without knowing what “interesting” actually means corporate database top performing rules rules evolving new interestingness analysis human ranking Interestingness is co-evolving with interesting rules

Interactive Evolution Evolution with a human being in the loop Often used in creative design or creative problem solving May be time-consuming Case1: no co-evolution Case 2: co-evolution population fitness evaluation by human replacement genetic operation computer programs fitness evaluation human beings

Intra-Population Co-Evolution There is only one population However, fitness of one individual depends on other individuals in the population Example 1: Playing backgammon TD-Gammon a grand master level computer program based on NN It learned by self-playing Self-playing = Co-evolution Is it because machine learning algorithms or co-evolution?

Co-Evolving Backgammon Players (1) Simple neural network without any fancy learning algorithm except for hill-climbing Simple EA with population size 1, hardly an EA! Simple mutation with Gaussian noise No recombination at all Task = evolve an NN that plays backgammon

Co-Evolving Backgammon Players (2) Task = evolve an NN that plays backgammon 197-20-1 feed-forward fully connected NN Initial weights were 0’s 1. Let the initial NN be NNk, k  0 2. Generate a mutant challenger of NNk w’ij = wij + G(0, s) 3. If NN’k is beaten by NNk, NNk+1 = NNk Else NNk+1 = NNk*0.95 + 0.05*NN’k 4. k  k+1, goto step 2 Performance: Winning 40% of the games against PUBEVAL after 100,000 generations strong program trained by experts

Separating Interwound Two Spirals Task: Given 194 training points, learn to separate two spirals Very tough problem for machine learning algorithms, e.g., decision trees, neural networks, … There was an attempt to evolve a solution to it. But the solution generalized poorly Competitive co-evolution based on covering really helps Fitness evaluation without co-evolution: number of test cases correctly classified Fitness evaluation with co-evolution: based on pair-wise competition, it depends only on the number of test cases correctly classified but NOT covered by its opponent

Iterated Prisoner’s Dilemma Non-zero sum, non-cooperative games The 2 player version The purpose here is not to find the optimal solution for some simplified conditions, but to study how to find it Fitness evaluation Entirely determined by the total payoff obtained through playing against each other The initial population was generated at random Player A C D 3 5 C 3 Player B 1 D 5 1

Why Co-Evolution We do not know the fitness function There are too many cases to test in order to obtain a fitness value. Co-evolution can be used to FOCUS search in the most important area The problem is inherently changing in time Increase and maintain diversity Self-learning

More Examples of Co-Evolution Discovering CA rules (using coverage again) Computer-aided learning (students + software tutors) Robot morphology and control Character recognition Chess playing International coffee market prices …

Open Issue Forgetting (also known as the Red Queen effect): co-evolution does not have a good memory at present Mediocre stable state: individuals learn to co-exist with each other and do not want to explore the search space any more Incremental evolution: continuous improvement without forgetting References J. Poon and M.L. Maher, “Emergent behavior in co-evolutionary design,” Artificial Intelligence in Design’96, J. Gero (ed.), Kluwer Academic. J.B. Pollack, A.D. Blair & M. Lund, “Coevolution of a backgammon player,” Proc. Of the Fifth Alife, May 1996. H. Juille and J.B. Pollack, “Dynamics of co-evolutionary learning,” Proc. Of the 4th Int. Conf. on Simulation of Adaptive Behaviro, Sept. 1996, MIT Press, pp. 526~534

Homework #1 주제: Diversity 유지를 위한 speciation 방법 구현 및 실험 마감일: 9/30 내용: 5가지 평가함수(De Jong function 1~5)에 2가지 종분화 방법(Fitness sharing, Crowding)을 적용하여 진화동안의 diversity 변화를 조사한다 Bonus Sorting algorithm/problem에 공진화와 종분화를 이용하기 Othello 게임에 공진화와 종분화를 이용하기