1. EVOLVING BETTER COMPUTER GAME ALGORITHMS WITH THE GPROTOOLKIT GENETIC PROGRAMMING SYSTEM By: Gregory Miranda University of Advancing Technology

2. Summary Introduction and Background Statement of the Problem Literature Review Methodology and Results Discussion, Implications, Recommendations Q&A

3. Introduction and Background Genetic Programming is a method for automatically creating computer software – Capable of solving complex problems using an iterative approach to find a solution – Can be used to create software functions that can solve new problems and to find new solutions to existing problems which have already been solved By applying genetic programming to an existing problem like pathfinding, it may be possible to find a more efficient solutions – Efficiency can be measured by comparing an A* algorithm to genetic programming’s best solution to see which performs the best and utilizes the least amount of computer resources

4. Introduction and Background (cont.) Genetic Programming still mainly an academic tool – Not used in mainstream software development – A number of genetic programming applications and libraries have been created to further genetic programming research The majority of the genetic programming applications are self contained, requiring the use of scripting languages to define the primitive set and the testing environment. The outputs of some of these applications can be saved in mainstream programming languages such as C++ and Java, but others require translating the output or digging through log files to find the final output of the system. Neither of these methods is easy to use and require a large learning curve before even attempting to use them. Having to recreate testing environment requires a lot of duplication

5. Statement of the Problem Despite the number of genetic programming applications available, none of them really address the needs of mainstream software developers – They are difficult to use and, in many cases, require duplication of effort to recreate the testing environment. To fix this gap, a genetic programming library was created that abstracted the majority of the genetic programming system and only exposed those areas that require integration into a developer’s software project. – Designed to be easy to integrate and use, the library should appeal to mainstream software developers in any industry.

6. Statement of the Problem (cont.) To prove that the library was capable of being used in the game industry, it was applied to creating a pathfinding solution that was comparable to the industry standard A* algorithm – Pathfinding is a basic component of many 2D and 3D computer games and simulations and is an area that receives a lot of attention as it can consume a large amount of computer resources when there are a large number of dynamic objects moving simultaneously which is becoming more common in today’s computer games

7. Literature Review Genetic Programming – Evolutionary computing technique for solving problems by automatically generating computer programs Developed by John Koza in 1992 Based on genetic algorithms (developed by John Holland, in 1992) Uses computer programs for genome instead of DNA or byte string – Modeled after several natural evolution principles Inheritance Mutation Fitness – Survival of the fittest

8. Literature Review (cont.) Genetic programming genome – Each program in the population must compile and execute in order to evaluate their fitness in solving the problem – Koza used a parse tree where the terminals and the functions make up the program’s components Nodes represents a function Leaves represents a terminal - an independent variable, zero- argument function, or a random constant Each of the branches coming out of a node represents an argument for the node’s function. The order of the components of the computer program is determined by the connections between the terminals and functions

9. Literature Review (cont.) Evolutionary Algorithm (Eggermont, 2007, p. 1) – Randomly create an initial population – Do Evaluate current population Select parents Generate offspring Replace current population with offspring – Until stop criterion met – Return best individual so far

10. Literature Review (cont.) Lisp was the first language used by Koza and others in the early research of genetic programming Since then genetic programming systems have been created with C, C++, Java, and even C#. Glaholt and Zhang (2004) believe that genetic programming is not used as a standard for solving complex problems because the tools are hard to use or do not produce useable output for programmers – Creators of GP-Lab

11. Literature Review (cont.) A number of genetic programming systems have been created in various programming languages – GP-Lab was Windows based application that could generate compiler ready C++ code from its genetic programming system and displayed it on the screen for the user to copy into their own applications – GAPS was easy to setup and run. It used an ASCII text file for input and a text log for output. The output did not produce any code that could be used by a programmer – lil-gp, which was flexible and comprehensive, containing all of the genetic programming necessary to evolve programs. To use, the function set and fitness functions need to be coded in C source files and compiled with the lil-gp source – Open BEAGLE is a genetic programming framework built with C++ with a set of genetic programming class libraries. To use Open BEAGLE, the programmer must override some classes and methods and compile their changes with the Open BEAGLE system. The output of the system is in XML and must be converted to source code by the user – Tiny-gp created for a competition. It was written in Java and only supports symbolic regression

12. Literature Review (cont.) Genetic Programming in Computer Games – Very little of the artificial intelligence research performed with games has been genetic programming, most have been with neural networks, genetic algorithms, and other machine learning concepts – It has been applied to a variety of strategy type games such as Othello, poker, draughts, chess, and backgammon along with some controller based games such as Ms. Pac-Man, robotic football, and radio controlled model car racing – Agapitos, Togelius, and Lucas, (2007) used both genetic programming and neural networks to create a controller for simulated car racing and compared the results against each other – Genetic programming was used to develop a high level strategy for training individual players in RoboCup soccer

13. Literature Review (cont.) Pathfinding in Computer Games – To figure out how to move a character from one point to another, the artificial intelligence system uses a pathfinding algorithm – The industry standard pathfinding algorithm used in computer games, and the basis for many newer algorithms, is A* A* is a simple algorithm that uses graph theory for finding the shortest path between two points The further away the goal is from the starting point, the longer it takes and the more memory is required Genetic Programming and Pathfinding – Strom (2006) created a pathfinding algorithm using a LISP based genetic programming system to be more aesthetic than A*

14. Methodology and Results GProToolkit Overview – Genetic Programming System incorporated into a C# library – C# vs Java or C++ – Object Oriented Design Primarily encapsulation – Tree-based representation – Variable number of automatically defined functions – Variable number of result producing branches – Strongly Typed GP Supports only primitive types (void, 32 bit integer, 64 bit integer, double, and boolean) – Variable number of arguments – Supports multiple objectives using Pareto optimality

15. Methodology and Results (cont.) GProToolkit Overview cont. – Application programmer defines the primitive set Uses reflection for discovering the function and terminal set defined by the application programmer in the client application. – Application programmer executes the Solution – Application programmer evaluates and sets Fitness values

16. Methodology and Results (cont.) Public Classes – Engine – Properties Internal Class – RNG

17. Methodology and Results (cont.) Internal Classes – Primitives – PrimitiveSet – PrimitiveList Public Class – PrimitiveBase Attributes – GPPrimitiveSet – GPConstant

18. Methodology and Results (cont.) Internal Classes – CodeElement – ConstantElement – MethodElement – PropertyElement – FieldElement – ArgumentElement – AdfElement

19. Methodology and Results (cont.) Internal Classes – Node – TreeInfo

20. Methodology and Results (cont.) Public Class – Solution Internal Classes – GPSolution – Program – FitnessResults – Result

21. Methodology and Results (cont.) Debugging Application (Unit Test) – nUnit Unit Test Framework – Symbolic Regression Test Create expression: x 2 + x + 1 Output: namespace Namespace1 { using System; public class Class1 { public virtual double ResultFunction1( GProToolkitUnitTest.MathFunctions functionBase) { return this.adf1(functionBase.ArgX(), functionBase.ArgX(), functionBase); } private double adf1(double arg0, double arg1, GProToolkitUnitTest.MathFunctions functionBase) { return functionBase.Subtract(functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0)), functionBase.Subtract(functionBase.Multiply(functionBase.Subtract(arg1, functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0))), functionBase.Subtract(arg0, arg0)), functionBase.Divide(functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0)), functionBase.Divide(functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0)), arg1)))); }

22. Methodology and Results (cont.) GPTester Application Demos – http://www.youtube.com/watch?v=Q3vxPygyFO4 http://www.youtube.com/watch?v=Q3vxPygyFO4 – http://www.youtube.com/watch?v=ShIw_Bxmsrg http://www.youtube.com/watch?v=ShIw_Bxmsrg Results from successful run – 50 generations – Population of 200 A* and GP Results A* Path Size (# tiles) A* Run Time (ms) GP Path Size (# tiles) GP Run Time (ms) Map 15826.8603.0 Map 25828.1583.0 Map 3435.1492.0 Map 4517.5553.0 Map 5413.0573.0 Map 61201.01227.0

23. Methodology and Results (cont.) A* (left) and GP Solution (right) for Tile Map 1 A* (left) and GP Solution (right) for Tile Map 2

24. Methodology and Results (cont.) A* (left) and GP Solution (right) for Tile Map 3 A* (left) and GP Solution (right) for Tile Map 4

25. Methodology and Results (cont.) A* (left) and GP Solution (right) for Tile Map 5 A* (left) and GP Solution (right) for Tile Map 6

26. Methodology and Results (cont.) Bad GP Solution for Tile Map 5 (but interesting)

27. Discussion, Implications, Recommendations Discussion – Easy to use – Hide GP details – Symbolic regression test solved a trivial problem domain – Sometimes a good solution could not be found in a single run Multiple runs using new initial populations were necessary – Adjusted the primitive set and the fitness results to find a ‘good’ solution – Issues with Pareto optimality Too many fitness results resulted in a large “best so far” pool that had a high proportion of unsuitable results being reused as parents. Too few fitness results though resulted in not having enough diversity and also produced unsuitable results. Through a lot of trial and error testing, a middle ground was found that resulted in good trial solutions being generated – The fact that the trial solution outperformed the A* algorithm in two-thirds of the test cases also showed that genetic programming was capable of making a more optimized pathfinding algorithm than the basic A* implementation. Given more time, and possibly some more adjustments to the fitness results and the primitive set, a trial solution could be generated that completely mimicked the A* algorithm path instead of just getting very close

28. Discussion, Implications, Recommendations Implications – Limitations of genetic programming Length of the genetic programming runs Difficulty in defining the proper primitive set – Primitive set and the fitness results are areas are highly customized to specific problems sets – Abstracting the genetic programming system into a library makes it easier integrate genetic programming – Can be used in game development to find new and novel solutions to standard algorithms

29. Discussion, Implications, Recommendations Recommendations – The primitive set should be able to include any library, both system and custom made Combination of both methods would lead to a system capable of developing a much wider range of computer software – Training a genetic program system can take a long time Most of the time is spent actually testing the solutions to measure their fitness – Allow for parallel processing and, if possible, distributed computing – Offer options with the structure of the genetic programming system Linear GP and graph GP in addition to tree-based GP – Expanded to support more than just primitive types – Support COM interop The library would be available in any language on a Microsoft Window system that supports COM, such as C++, Java, and Visual Basic

30. Q&A