Design of a real time strategy game with a genetic AI By Bharat Ponnaluri

Abstract Current strategy game AI Involved combinations of heuristics and constants. The combinations need to be optimized for the AI to be effective Too many possible constants&combinations-->difficult optimization Chess heuristic evaluations Moves available Is the king safe Pawn structure Material balance Genetic AI Can produce constants and combinations of heuristic algorithms quickly by eliminating suboptimal combinations.

Introduction Design of a simple real time strategy game –Build troops and try to take over cities –Like Risk in real time –Goal is to have an intelligent AI optimized with a Genetic Algorithm(GA). AI design –Coding of simple building block heuristic evaluation functions –Addition of constants(Agressiveness,stinginess) that determine behaviour –Chromosomes will be random combinations of Heuristic evalution functions –Used to evaluate the quality of a possible course of action based on game data

Personality Traits Goal: To make the AI more interesting and humanlike Traits –Agressiveness –Paranoia –Stinginess –Vengence –Trustworthiness

Observation AI will take advantage of other players' behavior Examples –Assertivness The AI will keep track of when a player retreats The AI will attack a player which retreats a lot –Attacking game leader One player is threatening to win AI will track who tries to stop that player Then the AI will let that player do the work

Game Theory Advantages –Not very complicated –Easily understandable by AI –Good model of human behavior

Game Theory: War of attrition Two players fighting over city Backing down=loss in percieved assertiveness Fighting=massive loss in strength t=time that conflict lasts Don't FightFight Don't Fight(0,0,0)(-t+1,-t,0) Fight(t,-t+1,0)(-t,-t,2*t)

Sample chromosome Chromosome enemyStrength()/2*myStrength()-3*fooA()/ power() Possible genes public int fooA().... public int enemyStrength().... public int myStrength()

Heuristic evaluation functions –Used to evaluate the quality of a possible course of action based on game data Example: –Public int getTrafficJamFactor() Return ( 1.001^(number of cars))/roadSize

How do GAs work Replicate evolution Steps 1.Determine random combinations of heuristics and constants(chromosomes) 2.Test chromosomes 3.Remove suboptimal chromosomes 4.Mutate surviving chromosomes 5.Swap data between surviving chromosomes 6.If chromosomes are optimal,end algorithm 7.Otherwise, repeat steps 1-7

Advantages and Disadvantages Advantages o Can function without user input o Can optimize a large number of constants o Usually is very good at optimizing stuff o Can determine unknown solutions Possible disadvantages o GAs are complicated o Take a significant amount of computing power o Networking will increase complexity

Evolutionary Stable Strategies(ESS) Gene pool of defectors means cooperating players will lose Advantages –Diverse AI personalities Disadvantages –May lead to suboptimal/boring AI's CooperateDefect Cooperate1,1-10,3 Defect3,-10-5,-5

ESS: Example Population of turtling AI's AI's only counterattack or attack weak players A more balanced AI will lose Playing against a group of turtling AI's is boring

New Heurstics Find Targets() –Average proximity –Income power factor –Troop power factor –Potential borders

Testing results Numbers greater than 1 are not good From last working version(before new heurstics) Number of Troops On Map Rendering Time Update Time AI

Generating a target number using a mathematical expression –Combination of numbers(1-9) and mathematical operators(-,+,*,/) adding up to a target number –Each number=4 bits –Chromosome *4.8568/ –Fitness value is the absolute value of (targetNumber)/(expressionValue-targetNumber) –Can be scaled for large numbers &decimals –No attempting to use brute force to test out random combinations.

Important Classes Classes –FormualaGenerator: Driver Program –GraphPanel: Draws the graph of the average fitness and maximum fitness. Makes the GA run periodically in synchornization with the graph. –Chromosome: Represents the chromosomes/expressions.

Static Mutation

Dynamic Mutation

Sample Expressions for 150 Static Mutation – (( / ) * * ) - (( ) / ) + ( * * ) = Dynamic Mutation – (( / ) * * ) - (( ) / ) + ( * * ) =

Analysis of results The dynamic mutation did not help because the standard deviations were 0 The GA generally works but occasionally crashes due to a bug which deletes all the chromosomes The graphs show rapid spikes in fitness followed by a rapid decline –Shows that the GA does not get stuck on local exterma. There are long period of nearly static fitness levels.

Important Methods RemoveFailures: –Finds all the chromosomes in the population with a fitness that is below average and removes them Crossover: –Picks random chromosomes to mate and produce children. –Once enough children are produced, the children are saved and the parents are deleted Mutations –Generates a mutation rate based on the standard deviation of the fitness values and average change of them –Calls the mutate method and gives it the mutation rate DrawGraph: Takes the saved data for the highest fitness rate, average fitness rate, standard deviation, and change of population and outputs the data visually.

Future Plan Debug implementation of dynamic mutation Make genetic algoritm generate formuals for input-output pairs based on common functions Create scenarios for my game with obvious good/bad choices for each player Run GA for game –Fitness rates will be based on how well the AI's evaluates the scenarios. –Running a full version of my game will take too much time.