StarCraft Learning Algorithms By Logan Yarnell, Steven Raines, and Dean Antel

Abstract  The opening or initial build order during a StarCraft match is an integral part of a player’s strategy. If an opening is naively chosen it makes a player vulnerable to tactics that can end the game very quickly. We examine three algorithms for choosing a “good” opener: Bayesian network, a genetic algorithm, and a pathfinding algorithm.

About StarCraft  Real-time strategy game  Similar to a complex digital version of chess but in real-time  Expert level gameplay is very complex, often requiring over 300 actions per minute.  Considered a national sport in South Korea

About StarCraft  Playing Starcraft requires simultaneously managing several distinct competencies:  Strategy  Production  Economy  Recon  Tactics

Basic Actions  Build buildings  Build units  Attack the enemy  Gather resources/manage economy

Genetic Algorithm  Modifies some set of “solutions” to a problem in a way similar to natural selection  Has mutation, crossover, and elitism functions  Solution = a build order  Each build order is given a fitness value determined by how successful it is  The number of mutations the build has gone through is also recorded  If a build order has gone through 10 mutations and it’s fitness < 5, elitism removes that build order

Genetic Algorithm  Most important part of genetic algorithm = mutate function  Sometimes the only part implemented if computational resources are limited  Fitness value determination is also important  Considered factors in determining a build’s fitness: time taken, amount of resources used, whether the match was won or lost, number of units/structures that survived…  In the end only win/loss rate was chosen to affect fitness

Genetic Algorithm  Mutate function randomly chooses to add a new unit, reorder the build, or remove a unit  isBuildValid method checks to make sure the mutate function doesn’t produce a useless build order by placing a unit before it’s prerequisite(s)  The newly mutated build is then written to a.txt file with NumOfMutations incremented  A match is run with that build, if it wins the fitness value is incremented, if it looses the fitness is not altered  There is an element of randomness, but complexity is more or less constant

Genetic Algorithm

Bayesian Network  Use knowledge from earlier games to predict what type of units the opponent is likely to have.  Requires mining a large amount of data from replays  Scouting is essential to make this work effectively  Difficult due to incomplete information available during matches

Pathfinding  From what is seen, create a shortest possible build. Relies on scouting.  Dijkstra’s algorithm O(n) complexity.  O(|V|^2)  O(|E|+|V|log|V|)  Intersection O(n^2)

Pathfinding (Continued)

Conclusions  Understanding the complexity of the environment is a key element to creating a learning algorithm for it.  Complementary not competitive.

Why 380?  There are many ways to implement a StarCraft bot that include various algorithms for the different aspects of gameplay:  Finite state machines  Scripting  Dynamic scripting  Probabilistic inference  Influence maps  Neural networks  Swarm intelligence  Potential fields  Genetic programming

Future Work  Enhance unit combat  Increase the amount of units considered in build orders  Optimize building efficiency to allow multiple buildings to be constructed at the same time.  Add dynamic scouting

Questions  What is the computational complexity (O(n)) of Dijkstra’s algorithm without optimization?  Is this a constraint satisfaction problem, or an optimization problem?  What probabilistic relationship is there between nodes in a Bayesian network that represents a StarCraft tech tree?

Answers  N^2  Optimization  Conditional dependence

Sources   model.pdf model.pdf   content/uploads/2014/12/making_and_acting_on_predictions_on_scbw_2014. pdf content/uploads/2014/12/making_and_acting_on_predictions_on_scbw_2014. pdf