Automating Content Analysis of Video Games T. Bullen and M. Katchabaw Department of Computer Science The University of Western Ontario N. Dyer-Witheford Faculty of Information and Media Studies The University of Western Ontario

Outline 1.Introduction 2.Automating Content Analysis 3.Prototype Implementation 4.Experiences and Discussion 5.Concluding Remarks

Introduction Content analyses of video games involve coding, enumerating, and statistically analyzing various elements and characteristics of games – This includes violence, offensive language, sexual content, gender and racial inclusiveness, and so on While content analysis has its limitations, it is invaluable in providing a quantitative assessment of games to go with more qualitative analyses – It can be an important tool to many people dealing with various aspects of games and the games industry

Introduction Problems arise, however, when one tries to apply traditional content analysis processes, for example from film or television, to games – Processes are manual and are consequently time consuming and labour-intensive – This tends to result in significantly reduced play times or limiting analyses to only a very few games – Traditional analyses also tend not to consider interactivity and non-linearity that occurs in games – The rapid rate at which games are released and the industry evolves makes keeping up difficult

Introduction In the end, with the limited time and resources often available, it is exceedingly difficult to perform thorough content analyses on even a reasonable portion of games To address these problems, our current work examines automating the process of content analysis for video games – Through automation, it is hoped that time and resources can be used more efficiently and effectively to permit more thorough studies

Automating Content Analysis To automate content analysis, we take advantage of the fact that, unlike other forms of media, video games are software executing on some kind of computing device This can permit two forms of automation: – Partial automation: software executing along side the game monitors game execution and collects and reports the data normally collected manually – Full automation: further software elements take the role of the player and generate gameplay experiences without the need for a human player

Automating Content Analysis: Instrumentation

Prototype Implementation As a proof of concept, we have used our instrumentation framework to instrument Epic’s Unreal Engine to enable automated content analyses of Unreal-based games – Unreal is a popular engine amongst professional and amateur developers, providing numerous possible games for content analysis experiments Instrumentation was implemented using the UnrealScript language – Source level access to the engine was not available

Prototype Implementation

Sensors have been developed to collect a wide variety of data useful for content analyses: – Death of characters, weapon use by characters, use of offensive language, gender and racial diversity in characters, and a variety of other game statistics – Data can be reported throughout a game or only as summaries at the end of games Sensors can be configured at run-time to tailor the data collected to the needs of the content analyses being conducted

Experiences and Discussion To validate our prototype implementation, we conducted several content analysis experiments on Unreal Tournament 2004 – This game is one of the flagship titles driven by the Unreal Engine It is a fairly popular First Person Shooter that has numerous gameplay options – Several different game types and rule sets – Individual and team-based games – Single player, multiplayer, and spectator modes

Experiences and Discussion: Deathmatch Game Level Info Level Name: Rrajigar Game Type: DeathMatch Total Players: 14 AI Players: 13 Human Players: 1 Spectators: 0 Male Players: 13 Female Players: 1 Level Loaded: 0:26:45 Game Finished: 0:30:29 Gameplay Elapsed (Seconds): AI Dialog: 28 Human Dialog: All Player Stats Total Deaths: 47 Total Suicides: 1 Total Kills: 46 Total AI Deaths: 46 Total Human Deaths: 1 Total Deaths Caused By AIs: 22 Total Deaths Caused By Humans: 25 Total Female Deaths: 12 Total Male Deaths: 35 Total Deaths Caused By Females: 6 Total Deaths Caused By Males: Local Player Stats Player Deaths: 1 Player Suicides: 0 Player Killed: 1 Deaths Caused By Player: 25 Player Killed By AI: 1 Player Killed By Human: 0 Player Killed By Male: 1 Player Killed By Female: 0 AI Deaths Caused By Player: 25 Human Deaths Caused By Player: 0 Female Deaths Caused By Player: 7 Male Deaths Caused By Player: 18 Deaths Witnessed By Player: Expletives ass:

Experiences and Discussion: Onslaught Game Level Info Level Name: Arctic Stronghold Game Type: Onslaught Total Players: 12 AI Players: 11 Human Players: 1 Spectators: 0 Male Players: 9 Female Players: 3 Level Loaded: 23:16:43 Game Finished: 23:32:3 Gameplay Elapsed (Seconds): AI Dialog: 216 Human Dialog: All Player Stats Total Deaths: 142 Total Suicides: 5 Total Kills: 137 Total AI Deaths: 138 Total Human Deaths: 4 Total Deaths Caused By AIs: 119 Total Deaths Caused By Humans: 23 Total Female Deaths: 26 Total Male Deaths: 116 Total Deaths Caused By Females: 8 Total Deaths Caused By Males: Local Player Stats Player Deaths: 4 Player Killed: 4 Deaths Caused By Player: 23 Player Killed By AI: 4 Player Killed By Human: 0 Player Killed By Male: 4 Player Killed By Female: 0 AI Deaths Caused By Player: 23 Human Deaths Caused By Player: 0 Female Deaths Caused By Player: 9 Male Deaths Caused By Player: 14 Deaths Witnessed By Player: Team Info Female Allies: 1 Male Allies: 4 Friendly Fire Deaths: 5 Allies Killed By Player: 0 Player Killed By Ally: 0

Experiences and Discussion Quality of data – Data collected through automation matched manual results, and in some cases was better Quantity of data – We found that we could collect massive amounts of data with no visible impact on gameplay, even when data was reported throughout a game Partial versus fully automated analyses – We found that results could be very different – Which is ultimately better?

Concluding Remarks Content analysis plays several important roles to the video games industry, but is unfortunately an arduous task to complete in a thorough fashion Our current work addresses this issue by providing an automated approach to content analysis based on software instrumentation Initial experimentation with a prototype implementation of this approach demonstrates its usefulness and shows great promise

Concluding Remarks Directions for future work include the following: – Conduct further experimentation and more detailed content analyses of Unreal Tournament 2004, and combine qualitative analyses with our results – Expand experimentation to other Unreal-based games – Investigate instrumentation of other popular game engines and conduct further analyses this way – Create sensors for measuring other content metrics – Further explore the issue of partial versus fully automated content analyses