1. Evolution Programs (insert catchy subtitle here)

2. Evolution Programs Basic idea: use principles of evolution and heredity to “evolve” solutions to a problem

3. Rabbits –At any given time, there is a population of rabbits. Some are smarter/faster than the others. –Foxes eat most of the slower, dumber rabbits (but not all). –More of the fast, smart rabbits reproduce. –“Wild hare” mutations possible (think Monty Python here). –The next generation will be, on average, faster and smarter than the original population.

4. How do we apply this? This is NOT DNA-based computing as practiced in this department. Ideas? –Battlebots? (but how do robots breed?) –Blade Runner?

5. Outline of evolution program Maintain a population of individuals P(t) –Each individual represents a potential solution to the problem at hand Evaluate each solution for fitness. Form a new population P(t+1) –Select the most fit individuals –Some members undergo transformations by means of “genetic operators” to form new solutions

6. Basic evolution program t <- 0 Initialize P(t) Evaluate P(t) while not (termination condition) do t <- t+1 select P(t) from P(t-1) recombine P(t) evaluate P(t) end

7. Genetic Algorithms Subset of evolution programs Solutions represented as bit strings Each solution is a chromosome Each bit is a gene

8. Genetic Algorithms Components: –Genetic representation for potential solutions to the problem –Method for creating initial population –Evaluation function to rate relative “fitness” –Genetic operators that alter composition of solutions during reproduction –Values for parameters that algorithm uses (population size, probability of applying operators)

9. Genetic Operators Mutation –Arbitrary alteration of one or more genes of a selected chromosome by random change with probability equal to the mutation rate –Intuition: introduce some extra variability into the population

10. Genetic Operators Crossover –Combine the features of two parent chromosomes to form two similar offspring by swapping corresponding segments of the parents –Intuition: information exchange between different potential solutions

11. Example: Function optimization Problem: Maximize the function –f(x) = x sin (10  x) + 1.0 over the domain [-1..2]. Desired accuracy is six decimal places

12. Example: Function optimization Representation –Use a binary vector as a chromosome to represent real values of x. –Domain of length 3, so we need 3x10 6 values. –Can be represented as a 22-bit vector

13. Example: Function optimization Initial population –For a population of size n, create n 22-bit chromosomes with randomly initialized bit values. Evaluation function –eval(v) = f(x)

14. Example: Function optimization Genetic operators –Use classical operators: mutation and crossover. –Mutation flip a bit with probability equal to mutation rate –Crossover randomly select crossover point A crossover after the 5th bit of 00101|100 and 11010|011 yields two children –00101|011 and 11010|100

15. Example: Function optimization Parameters –population size = 50 –probability of crossover = 0.25 –probability of mutation = 0.01

16. Function optimization: Results

17. Example: Prisoner’s dilemma Problem: –Two prisoners are held in separate cells, unable to communicate with each other. –Each can choose either to defect and betray the other prisoner, or cooperate with the other prisoner by maintaining silence. –Rewards If both cooperate, moderate rewards for both If only one defects, defector is rewarded, cooperator is punished If both defect, both are tortured

18. Reward structure for PD

19. Representation for PD Population of “players”, each of whom has a particular strategy Initial strategy chosen at random Players play games against each other, scores are recorded

20. Representing the strategy for PD Deterministic strategy Use outcomes from 3 previous moves to determine next move 4 outcomes /move, 3 moves -> 64 possible histories 64-bit string to represent choice for each possible history Plus 6 bits to encode the three moves preceding the start of the game

21. Outline of algorithm Randomly initialize population Test each player by playing games. Score is average over all games played. Select players to breed. Mate players to produce offspring.

22. Results for PD Random start -- produced populations whose median member is as successful as best known heuristic Patterns evolved: –Don’t rock the boat: (CC)(CC)(CC) -> C –Be provokable: (CC)(CC)(CD) -> D –Accept apology: (CD)(DC)(CC) -> C –Forget: (DC)(CC)(CC) -> C –Accept a rut: (DD)(DD)(DD) -> D

23. A shot at TSP Traveling Salesman Problem –The salesperson must visit every city on route exactly once and return to start. Minimize the cost of travel over entire tour –Problem: how do we represent the problem? As a bit string? as an integer vector? If bit string, genetic operators may produce non-legal children Choose integer vector

24. Traveling Salesperson Problem Representation: –Integer vector –Example: (3 6 4 8 2 1 5 7) Initialization: –random –output from greedy TSP Evaluation: –calculate cost of tour

25. Traveling Salesperson Problem Genetic operator –Must preserve legality –Want to exploit similarities –Choose subsequence of one parent and preserve relative order of cities from other parent Parents –(1 2 3 4 5 6 7 8 9 10 11 12) –(7 3 1 11 4 12 5 2 10 9 6 8) Subsequence (4 5 6 7) Offspring: (1 11 12 4 5 6 7 2 10 9 8 3)

26. TSP: Results Results averaged over 20 random runs Applied to 100 randomly generated cities After 20k generations, value of whole tour is 9.4% above optimal

27. Characterization Genetic algorithms/evolution programs can be viewed as a form of search Like hill-climbing –But hill-climbing search get stuck in local maxima –Results depend on starting point GAs are probabilistic algorithms –But not random algorithms -- directed search Parallel search

28. One wag’s comment “Neural networks are the second best way of doing just about anything...” “... and genetic algorithms are the third.” Genetic algorithms/evolution programs can be used on a wide variety of problems

29. Cool looking stuff (see web pages)