Genetic Algorithms Chapter 9, Complexity: A Guided Tour

Evolution by Natural Selection Charles Darwin

Evolution by Natural Selection Charles Darwin Organisms inherit traits from parents

Evolution by Natural Selection Charles Darwin Organisms inherit traits from parents Traits are inherited with some variation, via mutation and sexual recombination

Evolution by Natural Selection Charles Darwin Organisms inherit traits from parents Traits are inherited with some variation, via mutation and sexual recombination Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring.

Evolution by Natural Selection Charles Darwin Organisms inherit traits from parents Traits are inherited with some variation, via mutation and sexual recombination Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring. This way traits producing adapted individuals spread in the population

Evolution by Natural Selection Organisms inherit traits from parents Traits are inherited with some variation, via mutation and sexual recombination Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring. This way traits producing adapted individuals spread in the population in computers Computer(e.g., programs) Charles Darwin

Evolution by Natural Selection Organisms inherit traits from parents Traits are inherited with some variation, via mutation and sexual recombination Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring. This way traits producing adapted individuals spread in the population in computers Computer(e.g., programs) Charles Darwin John Holland

Evolution by Natural Selection Organisms inherit traits from parents Traits are inherited with some variation, via mutation and sexual recombination Due to competition for limited resources, the organisms best adapted to the environment tend to produce the most offspring. This way traits producing adapted individuals spread in the population in computers Computer(e.g., programs) Charles Darwin John Holland Genetic Algorithms (GAs)

Some real-world uses of genetic algorithms

Used by GE to automate parts of aircraft design

Some real-world uses of genetic algorithms Used by GE to automate parts of aircraft design Used by pharmaceutical companies to discover new drugs

Some real-world uses of genetic algorithms Used by GE to automate parts of aircraft design Used by pharmaceutical companies to discover new drugs Used by the London Stock Exchange to automatically detect fraudulent trades

Some real-world uses of genetic algorithms Used by GE to automate parts of aircraft design Used by pharmaceutical companies to discover new drugs Used by the London Stock Exchange to automatically detect fraudulent trades Used to generate realistic computer animation in the movies Lord of the Rings: The Return of the King and Troy

Some real-world uses of genetic algorithms Used by GE to automate parts of aircraft design Used by pharmaceutical companies to discover new drugs Used by the London Stock Exchange to automatically detect fraudulent trades Used to generate realistic computer animation in the movies Lord of the Rings: The Return of the King and Troy Used to model and understand evolution in nature!

Example: Evolving Strategies for Robby the Robot

Input: Contents of N, S, E, W, C(Current)

Example: Evolving Strategies for Robby the Robot Input: Contents of N, S, E, W, C(Current) Possible actions: Move N Move S Move E Move W Move random Stay put Try to pick up can

Example: Evolving Strategies for Robby the Robot Input: Contents of N, S, E, W, C(Current) Possible actions: Move N Move S Move E Move W Move random Stay put Try to pick up can Rewards/Penalties (points): Picks up can: 10 Tries to pick up can on empty site: -1 Crashes into wall: -5

Example Strategy

Encoding a Strategy Code: MoveNorth = 0 MoveSouth = 1 MoveEast = 2 MoveWest = 3 StayPut = 4 PickUpCan = 5 MoveRandom = 6

Encoding a Strategy Code: MoveNorth = 0 MoveSouth = 1 MoveEast = 2 MoveWest = 3 StayPut = 4 PickUpCan = 5 MoveRandom = 6

Encoding a Strategy 0 Code: MoveNorth = 0 MoveSouth = 1 MoveEast = 2 MoveWest = 3 StayPut = 4 PickUpCan = 5 MoveRandom = 6

Encoding a Strategy 0 2 Code: MoveNorth = 0 MoveSouth = 1 MoveEast = 2 MoveWest = 3 StayPut = 4 PickUpCan = 5 MoveRandom = 6

Encoding a Strategy Code: MoveNorth = 0 MoveSouth = 1 MoveEast = 2 MoveWest = 3 StayPut = 4 PickUpCan = 5 MoveRandom = 6

Encoding a Strategy Code: MoveNorth = 0 MoveSouth = 1 MoveEast = 2 MoveWest = 3 StayPut = 4 PickUpCan = 5 MoveRandom = 6

Encoding a Strategy values Code: MoveNorth = 0 MoveSouth = 1 MoveEast = 2 MoveWest = 3 StayPut = 4 PickUpCan = 5 MoveRandom = 6

Genetic algorithm for evolving strategies for Robby

1.Generate 200 random strategies (i.e., programs for controlling Robby)

Random Initial Population

Genetic algorithm for evolving strategies for Robby 1.Generate 200 random strategies (i.e., programs for controlling Robby) 2.For each strategy, calculate fitness (average reward minus penalties earned on random environments)

Genetic algorithm for evolving strategies for Robby 1.Generate 200 random strategies (i.e., programs for controlling Robby) 2.For each strategy, calculate fitness (average reward minus penalties earned on random environments) 3.The strategies pair up and create offspring via “sexual recombination” with random mutations ― the fitter the parents, the more offspring they create.

Parent 1: Parent 2:

Parent 1: Parent 2:

Parent 1: Parent 2: Child:

Genetic algorithm for evolving strategies for Robby 1.Generate 200 random strategies (i.e., programs for controlling Robby) 2.For each strategy, calculate fitness (average reward minus penalties earned on random environments) 3.The strategies pair up and create offspring via “sexual recombination” with random mutations ― the fitter the parents, the more offspring they create. 4.Keep going back to step 2 until a good-enough strategy is found!

My hand-designed strategy:

“If there is a can in the current site, pick it up.”

My hand-designed strategy: “If there is a can in the current site, pick it up.” “Otherwise, if there is a can in one of the adjacent sites, move to that site.”

My hand-designed strategy: “If there is a can in the current site, pick it up.” “Otherwise, if there is a can in one of the adjacent sites, move to that site.” “Otherwise, choose a random direction to move in.”

My hand-designed strategy: “If there is a can in the current site, pick it up.” “Otherwise, if there is a can in one of the adjacent sites, move to that site.” “Otherwise, choose a random direction to move in.” Average fitness of this strategy: 346 (out of max possible  500)

My hand-designed strategy: “If there is a can in the current site, pick it up.” “Otherwise, if there is a can in one of the adjacent sites, move to that site.” “Otherwise, choose a random direction to move in.” Average fitness of this strategy: 346 (out of max possible  500) Average fitness of GA evolved strategy: 486 (out of max possible  500)

My hand-designed strategy: “If there is a can in the current site, pick it up.” “Otherwise, if there is a can in one of the adjacent sites, move to that site.” “Otherwise, choose a random direction to move in.” Average fitness of this strategy: 346 (out of max possible  500) Average fitness of GA evolved strategy: 486 (out of max possible  500) ???

One Run of the Genetic Algorithm Best fitness in population Generation number

Generation 1 Best average score =  81

Time: 0Score: 0

Time: 1Score: 0

Time: 2Score:  5

Time: 3Score:  10

Time: 4Score:  15

Generation 10 Best average score = 0

Time: 0Score: 0

Time: 1Score: 0

Time: 2Score: 0

Time: 3Score: 0

Generation 200 Fitness = 240

Time: 0Score: 0

Time: 1Score: 0

Time: 2Score: 0

Time: 3Score: 10

Time: 4Score: 10

Time: 5Score: 20

Time: 6Score: 20

Time: 7Score: 20

Time: 8Score: 20

Time: 9Score: 20

Time: 10Score: 20

Time: 11Score: 20

Time: 12Score: 20

Time: 13Score: 20

Time: 14Score: 30

Time: 15Score: 30

Time: 16Score: 40

Time: 17Score: 40

Time: 18Score: 50

Time: 19Score: 50

Time: 20Score: 60

Generation 1000 Fitness = 492

Time: 0Score: 0

Time: 1Score: 0

Time: 2Score: 10

Time: 3Score: 10

Time: 4Score: 20

Time: 5Score: 20

Time: 6Score: 20

Time: 7Score: 20

Time: 8Score: 20

Time: 9Score: 30

Time: 10Score: 30

Time: 11Score: 40

Time: 12Score: 40

Time: 13Score: 50

Time: 14Score: 50

Time: 15Score: 60

Time: 16Score: 60

Time: 17Score: 70

Time: 18Score: 70

Why Did The GA’s Strategy Outperform Mine?

My Strategy

The GA’s Evolved Strategy

Principles of Evolution Seen in Genetic Algorithms

Natural selection works!

Principles of Evolution Seen in Genetic Algorithms Natural selection works! Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation

Principles of Evolution Seen in Genetic Algorithms Natural selection works! Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation Best fitness in population Generation number

Principles of Evolution Seen in Genetic Algorithms Natural selection works! Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation Exaptation is common

Principles of Evolution Seen in Genetic Algorithms Natural selection works! Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation Exaptation is common Co-evolution speeds up innovation

Principles of Evolution Seen in Genetic Algorithms Natural selection works! Evolution proceeds via periods of stasis “punctuated” by periods of rapid innovation Exaptation is common Co-evolution speeds up innovation Dynamics and results of evolution are unpredictable and hard to analyze