Dynamic Task Allocation in a turn based strategy game Gilles Schtickzelle September 2012 ULB

Problem Statement Creating an intelligent player for a turn-based strategy game. Working Framework: Many possible challenges to meet: o Resource management o Adversarial planning o Spatial reasoning o …

A game of FreeCol Colonization of America Establish settlements, grow and develop them Victory : Declare independence & Beat the Royal Expeditionary Force

Colony Management Assigning tasks to units for optimal resources production

Division of labor in insect societies Ants and wasps colonies have efficient distributed task allocation mechanisms through stygmergy. Bonabeau, E., Theraulaz, G., & Deneubourg, J.-L. (1996).

Response Threshold Ants have probabilistic response to stimuli: Varying threshold θ induces specialization o Reduces switching costs o Increases individual efficiency

From insects to games FreeCol Colony Units Resources Expert units Ants/Wasps Colony Insects Tasks Specialization

Resources Dynamics Surplus: Extra workers. Shortage: Lose worker. Freedom. 50% required to win. Givesbonus or penalty to workers. required to make hammers Used to produce buildings or artileries required to make tools Used to produce buildings or artilleries

Allocation Mechanism One stimulus S r for each resource r = One set of dynamic thresholds θ ri per unit i

Stimuli and Thresholds Simple computation rules for each stimulus One set of dynamic thresholds θ ru per unit u Genetic Algorithm to find appropriate scale factors β r

Simple Scenario

AI goals 1.Reach the year 1776 with enough bells to be able to declare independence. 2. Have the best defense possible to resist the attack of the royal expeditionary force. 3.Allocate workers to 1.minimize famine 2.Keep the production modifier as high as possible

Results (Basic player) Freedom %SizeFamineMilitary EXPERT100%14024 MEAN (100 games)91.57 ± ± ± ± 0.62

Planning approach Suboptimal allocation: building too early Two planning methods: o Layered response threshold. o Rule-based planning.

Planning approach Layered response threshold : o Use two sets of scale factors: Optimized for growth Optimized for production Rule-based planning :

Planning Results (1) Layered AIRule-based AI

Planning Results (2) MilitarySoL %SizeFamine EXPERT MEAN (BASIC)15.99 ± ± ± ± 0.09 MEAN (LAYERED)17.49 ± ± ± ± 0.10 MEAN (RULE BASED)18.80 ± ± ± ± 0.09 Statistics for 100 games with the simple scenario.

Modified Threshold rule Unit u produces resource r Unit u does not produces resource r

“State of the art” player Modified Threshold update rule + rule-based planning

AI players comparison

AI goal completion 1.Reach the year 1776 with enough bells to be able to declare independence. 2. Have the best defense possible to resist the attack of the royal expeditionary force. 3.Allocate workers to 1.minimize famine 2.Keep the production modifier has high as possible Freedom %SizeFamineMilitary EXPERT100%14024 MEAN (100 games)100% ± ± ± ± 0.55

Conclusions Human-level performances can emerge from simple rules, without cheating. Easy to implement (compared to traditional rule-based only AI). Easy to tune down performances (if playing against non- expert). Hybrid system (with planning instructions) improves on basic RTM −Tendency to chaos with large number of stimuli −Difficult to extend to other game aspects (combat, spatial reasoning, diplomacy,…).