Player Modeling Chris Janneck 11/15/04

Overview Player Modeling for Adaptive Games –By Ryan Houlette –About the author The point of modeling –Regarding AI The simplest model A higher-level model More complex modeling Concerns with modeling and AI

Ryan Houlette Stottler Henke Associates – –Developer of AI systems Intelligent Tutoring, Scheduling, KM, Security, etc. –SimBionic – AI agent toolkit Lead Software Engineer –Co-developer of BrainFrame system AI SDK for entertainment

SimBionic (Include AVI from

BrainFrame (Include Figures 1 and 2 from “Putting AI in Entertainment…”)

The Point of Modeling Why model? –To create a representation of some item or phenomena that can be (more) easily explored, tweaked, observed and tested Where do we model? –Engineering, Design, Simulations, Manufacturing, Business, Mathematics –And, yes, in AI Model-based reasoning, instead of Case-based reasoning

Modeling in AI So far, discussed modeling of computer AI –FSMs, Behaviors and Knowledge Rep/Sharing, Maps and Movement, … Today, discuss modeling of human player –Why? Aren’t humans inferior to our AI? For entertainment, player must have fun Not fun  not played  bad rep  game bombs Fun(g) = Challenge(g) + Usability(g) + Replayability(g) + SenseTickling(g)

Goals of Player Modeling Maximizing Challenge() –Use player model to generate actions that exploit player’s weaknesses Maximizing Replayability() –Since model is dynamic, AI never acts the same way twice: always a different game Improving SenseTickling() –Player thinks AI is “good” or “smart” if it adapts well to the player –Tickling the brain-sense

The Simplest Model What is the simplest kind of model? –Model = {(attribute, value), (attr2, val2]…)} –Model is a class of action/trait “counters” A trait value is modified when observed –Value increase when action observed –Value may decrease over time, or to maintain normalization –Example traits: “UsesSmokeGrenades,” “AvoidsMovingThroughEnemyChokepoints,” “MeleeAttacks,” “AutomaticAttacks,” “RocketAttacks,” “Slipstreams,” “BreaksIntoTurns,” “AcceleratesThroughTurns,” “OverSpeedLimit,” “EatsPizza,” etc…

The Simplest Model: Code (include code from 559, 560)

The Simplest Model: Updating Recognize when update should be made –Can be easy “UsesLaser” – count number of times fired –Or more difficult “CanDoTrickyJumps” – if player successfully makes known difficult jumps on map “CanDoWallJumps” – if player successfully jumps >1x and lands on higher ground Instruct model to update –Least Mean Squares training traitValue =  * observedValue + (1-  ) * traitValue –Normalize values 0 <= x <= 1 –0 = never happens, 1 = all the time, 0.5 = don’t know

Evaluating Simple Models Benefits –Easy to code and use –Getting and updating values straightforward –Good for statistics-tracking and display Shortcomings –Difficult to model higher-level behavior –AI choices will have to use many model traits and computations Ex: “Which side to defend?” “What units to produce?”

A Higher-Level Model Hierarchical Modeling –Higher-level node value combination of children’s values –Abstract node combination of concrete traits

Hierarchical Player Model (include Figure from 10.1)

Hierarchical Issues Similar to other hierarchical structures –Like Troop-based AI tactics –Propagation If values of concrete, leaf-nodes change, must propagate values up the tree –Easier & faster model state checking Can scan abstract nodes quickly instead of revisiting all of the concrete counters

More Complex Modeling Extract and extrude hierarchical model –Can achieve wide range of modeling by tweaking: What is abstracted? How higher-levels are computed from lower? Of what do leaf nodes keep track? How are leaf nodes designed?

Possible Tweaks Leaf nodes – track state instead of counter –Represent each leaf as a FSM Abstract nodes – predict instead of report –Represent higher-level nodes as N-grams Probabilistic directed graphs of frequencies of sequences A node to monitor “force” locations –Leafs may return a TroopLocation class (containing (x,y)s of units) instead of a value

Modeling Issues Model complexity-time tradeoff –More complicated = more time spent –Still need to reserve time for AI decision- making, rendering, and less-important things Decouple model from game = good –If model is data-driven (i.e. from an external file), can be modified without re-compiling –If model is lightly-coupled to rest of AI, even better

Utilization of the Model AI queries model, uses result in decision-making –Examples: If determining which kind of weapon to use, check: –Player’s weapon proficiency/usage counts If determining what kind of attack to use (flanking, head-on, air-based, etc.), check: –Which attacks have been most often repelled –Which attacks the Player uses –Where are units located If determining how to allocate patrols, check: –If Player is more stealthy, put more in hidden paths, or detect better in shadow –If Player is more Rambo, put more in open areas, with heavier artillery

Design of the Model “Habits and Preferences” semantics –“UsesSmokeGrenades” means how often player uses this weapon “Knowledge and Proficiency” semantics –“UsesSmokeGrenades” means how good the player feels about using this weapon There is a subtle, but important difference –If “UsesSmokeGrenades” = 1, What does that mean? How should the AI respond?

Other Player Model Uses Allow exporting / transferring player model –If multiple games use same model (or similar nodes), a new game can more quickly adapt –Allow multiplayer games to build more generic model, based on several players Allow player to view model –Lets them see where they can/should improve “Level completed” statistics screen –May reveal some intricacies about AI

References Introduction to Learning in Games – n.htm