Gameplay Analysis through State Projection Erik Andersen 1, Yun-En Liu 1, Ethan Apter 1, François Boucher-Genesse 2, Zoran Popović 1 1 Center for Game Science Department of Computer Science University of Washington 2 Department of Education Université du Québec à Montréal FDG 2010 June 21 st, 2010

We want to know how people play

?

We want to find…

Player confusion

We want to find… Player confusion Player strategies

We want to find… Player confusion Player strategies Design flaws

Patterns in data SELECT * FROM replays WHERE location=x AND time>y AND attempt>3 AND death=“grenade”…

Patterns in data SELECT * FROM replays WHERE location=x AND time>y AND attempt>3 AND death=“grenade”… Confusion? Strategies?

Traditional Playtesting

Statistical Methods Surveys In-game statistics

Statistical Methods Surveys In-game statistics

Visual Data Mining Lets people see patterns in data Bungie (Halo 3)

Visual Data Mining Lets people see patterns in data Dynamic information? Bungie (Halo 3)

Visual Data Mining Lets people see patterns in data Dynamic information? Games with no map? Bungie (Halo 3)

But what about?

“Playtraces” GoalStart

“Playtraces” GoalStart

“Playtraces” GoalStart

“Playtraces” GoalStart Confusion? Distance to goal

Refraction

Massive educational data mining

2-D projection of points in high-dimensional space Clusters game states based on some distance function Classic Multidimensional Scaling

State Distance

Action Distance d a (s 1, s 2 )

State Distance GoalStart Confusion? Distance to goal

Distance to Goal d g (s 1, s 2 ) = abs(d g (s 1, s g ) - d g (s 2, s g ))

Distance Functions Action distanceCombinedDistance to goal

Refraction Distance Function d (s 1, s 2 ) = (d a (s 1, s 2 ) + d g (s 1, s 2 )) / 2

Playtracer Framework

Easy level

Difficult level

Failure

Chance To Win

Evaluation

35 children from K12 Virtual Academies

Evaluation 35 children from K12 Virtual Academies Mostly third and fourth-graders

Evaluation 35 children from K12 Virtual Academies Mostly third and fourth-graders About 15 levels

Evaluation 35 children from K12 Virtual Academies Mostly third and fourth-graders About 15 levels The game logged all player actions

Analysis

Player confusion

Analysis Player confusion Player hypotheses

Analysis Player confusion Player hypotheses Design flaws

Analysis Player confusion Player hypotheses Design flaws

Level 2

Level 2 Solution

Level 2 Visualization

Confusion: Hitting target from wrong side

Refinement

Confusion: Using pieces incorrectly

Analysis Player confusion Player hypotheses Design flaw

Level 4

Level 4 Solution

Level 4 Visualization

Hypothesis: Satisfy bottom target

Hypothesis: Get laser near targets

Hypothesis: Overload bottom target

Analysis Player confusion Player hypotheses Design flaws

Level 4 Visualization

Design flaw: Deadly state

Refinement

Limitations Difficult to find good distance function

Limitations Difficult to find good distance function

Limitations Difficult to find good distance function

Limitations Large game spaces

Conclusions Useful for game analysis

Conclusions Useful for game analysis We are expanding and refining Playtracer

Big Open Problems How to

Big Open Problems How to – specify distances between game states

Big Open Problems How to – specify distances between game states – differentiate types of confusion

Big Open Problems How to – specify distances between game states – differentiate types of confusion – classify player strategies

Acknowledgements Marianne Lee Emma Lynch Justin Irwen Happy Dong Brian Britigan Dennis Doan François Boucher-Genesse Seth Cooper Taylor Martin John Bransford David Niemi Ellen Clark Funding: NSF Graduate Fellowship, NSF, DARPA, Adobe, Intel, Microsoft

Cycles

Acyclic Paths

Player Tracking