1. Evolutionary Computation and Game Development Part 1: Background

2. Overview What is Evolutionary Computation (EC)? EC in the Real World EC and the Gaming Industry What is EC? (Continued) EC and Games Some Ideas Questions

3. What is Evolutionary Computation? Evolutionary Computation is the study of algorithms inspired by the natural process of evolution. o Genetic Algorithms (GA) o Evolution Strategies (ES) o Evolutionary Programming (EP) o Genetic Programming (GP) o Particle Swarm Optimization (PSO) o Ant Systems (AS) o Ant Colony Optimization (ACO)

4. EC in the Real World Scheduling o Scheduling of flights o Task scheduling Routing o FedEx o UPS o Disney World Tour Planning Design o Aircraft design o Factory layout

5. EC in the Gaming Industry IEEE CIG IEEE Transactions on Computational Intelligence and AI in Games Started in 2005 NERO with rtNEAT Using EC to create actual AI opponents Dynamic Content Creation Play Testing

6. NERO and rtNEAT NERO – Neuro-Evolving Robotic Operatives NEAT – Neuro-Evolution of Augmenting Topologies The NERO game was a sandbox where players evolved AI bots –Supervised learning Evolved Neural Network controllers with a Genetic Algorithm

7. EC for AI Opponents RTS games have recently become a test-bed for evolving AI opponents There is also interest in evolving support functions, units, and features in RTS Research and competition have also been held and are still being held for AI bots in FPS games However, most research still center around classic board games

8. Dynamic Content Creation Key questions –How can a game keep the player engaged? –How can a game keep players interested? –How can a game challenge a player? –How can a game satisfy a player? Two possible approaches –Adaptive difficulty or gameplay –Adaptive game content

9. Play Testing Use EC to find potential cheats or exploits Use EC to test feature balancing Use EC to possibly find better reproduction steps for isolating bugs/crashes One research done using Fifa ’99 –EA Burnaby

10. What is EC? (Continued) Evolutionary Computation is the study of algorithms inspired by the natural process of evolution. So, what does that mean? Here's the High Level Pseudo-code 1.Randomly Generate a Population (Generation) 2.Evaluate Initial Population (Evaluation) 3.Select Parents (Selection) 4.Perform Crossover (Crossover) 5.Apply Mutation (Mutation) 6.Evaluate Offspring (Evaluation) 7.Replace Parents (Replacement) 8.Goto 3.

11. Characteristics of an Evolutionary Algorithm (EA) "Any-time" Algorithms o The longer you run them, the better the solution (ideally) Stochastic in Nature o There is alot of randomness involved o This is the primary property Meta-heuristics o "a heuristic method for solving a very general class of computational problems by combining user-given black- box procedures in a hopefully efficient way" Flexibility Black Box operator that can be swapped out

12. Black Box Components Problem Encoding o How is the problem encoded/represented o Genotype and Phenotype Operators o Selection o Crossover o Mutation o Replacement o Evaluation Candidate Solution

13. Problem Encoding Genotype - the representation of the problem that is being manipulated by the EA Phenotype - the physical representation of the problem that the genotype maps to Encoding is a key component o is important no matter what EA you are using o improper encoding will cause bad results o too complicated an encoding runs the risk of bogging down the algorithm There are guidelines, known methods, and encoding requirements based on the EA you want to use However, there will always be some black magic involved

14. Example: Knapsack Problem Sword10 Shield20 Helmet4 Chain Mail14 Gauntlet6 Running Shoes2 Healing Potion1 Magic Wand3 Healing Potion1

15. Knapsack Problem Large number of items, limited space Simplified form of a inventory/resource management problem –Inventory management in RPG’s –Resource allocation in RTS’s

16. Phenotype vs. Genotype Sword10 Shield20 Helmet4 Chain Mail14 Gauntlet6 Running Shoes2 Healing Potion1 Magic Wand3 Healing Potion1 1 1 0 1 0 1 1 0 1

17. Some Known Encodings Bit Arrays Integer/Floating Point Arrays A mix of the above two Finite State Machines –First used in Evolutionary Programming Expression Tree –Mainly used in Genetic Programming

18. Evaluation Function Just as important if not more important than how the problem is encoded The quality of the evaluation function directly influences the results o EA's have no eyes o The evaluation function is an EA's guide dog There are also known good methods of formulating an evaluation function To get optimal results, the evaluation function must be customized for the problem There is a whole subfield of EC that doesn't use an evaluation function o Interactive Evolutionary Computation

19. Knapsack Problem (Cont.) How would we evaluate a candidate solution? –How do we judge which item combination is better than any other? –Do we just want to fill the sack as much as possible, or is there a reason to prefer certain items over others?

20. Knapsack Problem (cont.) Sword10 Shield20 Helmet4 Chain Mail14 Gauntlet6 Running Shoes2 Healing Potion1 Magic Wand3 Healing Potion1 100 70 10 54 35 27 49 5

22. Selection Natural Selection How do we select parents? Which parents should be paired up for crossover? Common selection methods o Random Selection o Rank-Based Selection o Roulette Wheel Selection o Tournament Selection o No Selection

23. Random Selection

24. Roulette Wheel Selection

25. ??? P1P4P3 ??? P6P2P3 Tournament Selection

26. Crossover How do we create offspring(s) with the parents? Completely dependent on problem representation and how many parents are involved Arrays of discrete values o Single Point or Multi-point Crossover o Uniform Crossover Array of real values o Discrete Crossover o Interpolation Trees or Finite State Machines o Pruning and Grafting

31. Mutation Also representation dependent Any form of random perturbation o random bit flips o random value replacements o random insertions or deletions

32. Replacement This is where Natural Selection kicks in a second time How will the offsprings replace the parents? Who will survive? Two common approaches o Offsprings replace parents completely o Offsprings and parents compete for survival

33. Operators Evolutionary Algorithms are meta-heuristics Operators are black boxes Doesn’t matter how they are implemented They just have to perform some specific task –The algorithm doesn’t care how task is accomplished

34. Additional Notes Crossover and mutation are sometimes considered optional o Some have found better results with only one of the two The truth is that there is alot of voodoo magic involved o Experience will tell you how best to encode a problem o But each problem type tends to have its own trick o There are known best practices for certain problem types  Traveling Salesman  Knapsack Problem o More often than not, your problem will fall into something totally new

35. Precautionary Warning EC is a tool –Used for exploratory purposes –Used when a pretty good solution is needed in a short period of time –Used when there is no known solution especially when there is no clear definition of what a solution is It will never perform better than a specialized algorithm

36. EC in Games Why is EC rarely used in real games? Complexity (Space and Time) –Which portion of the game do we want to evolve? –How would we encode the problem and where would we store the population? –Where will we find the clock cycles needed for the evolution? –What is a good evaluation function? –How would the producers tune this and what are behaviors they should expect? Uniform Experience –How do we control the overall user experience? –Can we even guarantee a predictable user experience? –Will predictability sacrifice algorithm performance?

37. Interactive Evolutionary Computation Replaces the evaluation function with some form of human response –Great for when the fitness cannot to quantitatively evaluated and qualitative measures are fuzzy at best The response can be a quantitative value or just a qualitative classification NERO is an example of an interactive EC

38. Adaptive Gaming Gameplay adapting to user action oDynamic scaling of difficulty (already been done) The game adapts to user actions –Dynamic content generation –“No one really knows what they really want” –“They know what they want when they see it”

39. Questions?