Evolving New Strategies The Evolution of Strategies in the Iterated Prisoner’s Dilemma 01 / 25

What is the Prisoner’s Dilemma? There are two prisoners There are two prisoners Each one has taken part in the same criminal act Each one has taken part in the same criminal act The authorities are interrogating each one The authorities are interrogating each one Each prisoner can choose to keep their mouth shut or rat out their partner Each prisoner can choose to keep their mouth shut or rat out their partner If both prisoners stay quiet, they each get n months of jail time If both prisoners stay quiet, they each get n months of jail time If only one prisoner gets ratted out, that prisoner gets n + x months of jail time while the other prisoner gets n – y months of jail time If only one prisoner gets ratted out, that prisoner gets n + x months of jail time while the other prisoner gets n – y months of jail time If the prisoners rat each other out, they each get n + z months of jail time. If the prisoners rat each other out, they each get n + z months of jail time. In this case, n, x, y, and z are all greater than zero. In this case, n, x, y, and z are all greater than zero. In this case, x is greater than z. In this case, x is greater than z. 02 / 25

What is the Iterated Prisoner’s Dilemma? Prisoner’s Dilemma performed several times Prisoner’s Dilemma performed several times The two criminals have committed several crimes together The two criminals have committed several crimes together They are interrogated for each crime, with each set of interrogations being an instance of the original Prisoner’s Dilemma They are interrogated for each crime, with each set of interrogations being an instance of the original Prisoner’s Dilemma These interrogations are performed in sequence (or iteratively), and the jail time distributed to each prisoner is cumulative These interrogations are performed in sequence (or iteratively), and the jail time distributed to each prisoner is cumulative 03 / 25

How does the IPD relate to GAs? No optimal solution No optimal solution No real strategy No real strategy No clue No clue Hard problem Hard problem So back to the paper So back to the paper 04 / 25

What This Paper Shows GAs in a rich social setting GAs in a rich social setting Advantage of developing new strategies Advantage of developing new strategies One parent One parent Two parent Two parent Early commitments to paths Early commitments to paths Evolutionary processes optimal or arbitrary Evolutionary processes optimal or arbitrary 05 / 25

How Does It Show It? Simulation Simulation Multiple cases Multiple cases Comparative output Comparative output 06 / 25

The Simulation Specify the environment Specify the environment Specify the encoding Specify the encoding Testing the effects of random mutation Testing the effects of random mutation Run the simulation Run the simulation Analyze the results Analyze the results 07 / 25

The Environment Prisoner’s dilemma Prisoner’s dilemma Multiple prisoners Multiple prisoners Goal is to achieve mutual cooperation Goal is to achieve mutual cooperation Individuals may meet more than once Individuals may meet more than once 08 / 25

Initial Experiment Original strategies were submitted by fourteen people Original strategies were submitted by fourteen people Game Theory Game Theory Economics Economics Sociology Sociology Political Science Political Science Mathematics Mathematics Various levels of intricacy Various levels of intricacy 09 / 25

Initial Experiment Most complex strategy Most complex strategy Markov process Markov process Bayesian inference Bayesian inference Least complex strategy Least complex strategy TFT TFT TFT won TFT won 10 / 25

Second Experiment Sixty-two entries Sixty-two entries Six countries Six countries Computer hobbyists, professors Computer hobbyists, professors TFT was submitted again TFT was submitted again It won It won 11 / 25

The GA Population Population Encoding Encoding Generation Generation Crossover Crossover Mutation Mutation Fifty Generations Fifty Generations 12 / 25

Population Twenty chromosomes Twenty chromosomes Seventy genes Seventy genes 13 / 25

Encoding For each prisoner’s dilemma, there are four possibilities For each prisoner’s dilemma, there are four possibilities Each “player” has memory Each “player” has memory What each gene represents What each gene represents 14 / 25

A Single Generation Multiple games Multiple games Each game had one-hundred and fifty-one moves Each game had one-hundred and fifty-one moves Each chromosome played eight others Each chromosome played eight others Fitness was assigned Fitness was assigned Ratted out – Zero points Ratted out – Zero points Mutually ratted out – One point Mutually ratted out – One point Mutual cooperation – Three points Mutual cooperation – Three points You ratted, other person stayed quit – Five points You ratted, other person stayed quit – Five points 15 / 25

Crossover Fitness proportional selection Fitness proportional selection Involved standard deviation from mean Involved standard deviation from mean Strictly ten crossovers Strictly ten crossovers Single point Single point Two parents Two parents 16 / 25

Mutation Single gene flip Single gene flip One gene per two chromosomes One gene per two chromosomes 17 / 25

Results Median resultant member Median resultant member Just as good as TFT Just as good as TFT Resembled TFT Resembled TFT Five properties were found Five properties were found Don’t rock the boat Don’t rock the boat Be provocable Be provocable Accept apologies Accept apologies Forget Forget Accept a rut Accept a rut 18 / 25

Results ADJUSTER ADJUSTER Special chromosome which consistently seeks to exploit Special chromosome which consistently seeks to exploit TFT TFT Majority of other chromosomes Majority of other chromosomes 19 / 25

Results Twenty-five percent of runs Twenty-five percent of runs Median was better Median was better Exploit one chromosome Exploit one chromosome 20 / 25

Results: Why is this important? Chromosomes had to learn Chromosomes had to learn Discriminatory based on evidence Discriminatory based on evidence Self adjusting for exploitation Self adjusting for exploitation No alienation No alienation Break primary rule of first tournament Break primary rule of first tournament Be nice? I don’t think so Be nice? I don’t think so 21 / 25

Results: Misleading? Median Median Exploitative Exploitative Menacing Menacing A true criminal? A true criminal? Fixed size population and tournaments Fixed size population and tournaments Simulate real evolution Simulate real evolution 22 / 25

Results: A Slight Twist Asexual reproduction Asexual reproduction TFT TFT Less than half of the medians Less than half of the medians Changing environment Changing environment Play against everyone Play against everyone Everyone starts aggressive Everyone starts aggressive Fitness rapidly declines Fitness rapidly declines Fitness begins to even out Fitness begins to even out Fitness begins to rise Fitness begins to rise 23 / 25

Conclusions The GA is good for searching, large, multi- dimensional spaces The GA is good for searching, large, multi- dimensional spaces Multiple parent crossover helps Multiple parent crossover helps Arbitrary aspects of evolution Arbitrary aspects of evolution Hitch hikers Hitch hikers Exploration vs. Exploitation Exploration vs. Exploitation Selection Pressure Selection Pressure Evolutionary Commitments can be irreversible Evolutionary Commitments can be irreversible 24 / 25

Related Topics Mutation Mutation Crossover Crossover Inversion Inversion Coding principles Coding principles Dominant/Recessive Dominant/Recessive Rate of evolution Rate of evolution Population viscosity Population viscosity Speciation and niches Speciation and niches 25 / 25