Using Commonsense Reasoning in Video Games

There is a problem facing the video game industry: 3D environments are getting dramatically more realistic, but they are not getting any easier for artists to create The Dramatic Improvement in Video Game Realism

The Dramatic Improvement in Video Game Realism - Hardware (2003) (1998) In addition to looking more like jet engines, graphics cards are improving at an incredible pace

The Dramatic Improvement in Video Game Realism - Hardware NVIDIA is working under the philosophy of "Moore's Law Cubed"

The Dramatic Improvement in Video Game Realism - Software (1991)(1994)(1996) (1999) (2004) Wolfenstein 3D by id DOOM by idQuake by id Quake III by idDOOM 3 by id

The environments are more realistic (texture mapping, anti-aliasing, pixel and vertex shading, improved shadow generation, increased memory bandwidth, increased fill rate…) And they are also more complicated The Dramatic Improvement in Video Game Realism

The Dramatic Improvement in Video Game Realism – Complexity of Environments (lots of stuff) Morrowind by Bethesda Softworks, PC and Xbox

Morrowind took 3 years to develop in part because the massive game environment was generated by hand Result: Increased Development Time

The Complexity of Level Editors Editors to create 3D environments represent some of the most complex software applications ever created

The Complexity of Level Editors: A Sea of Direct Manipulation The Morrowind Construction Set

The Complexity of Level Editors Unfortunately, editors are not improving at a pace to compensate for the increased complexity of 3D environments

A CS100 Solution Some games automatically create their own environments, but the result is usually as simplistic as randomly generating a maze

A New Solution We are trying to use Common Sense Reasoning to automatically generate rich and immersive 3D environments

Rationale Save development time and money Create dynamic and “endless” worlds Expand the capabilities of Common Sense Reasoning Rationale

Project Goals Automatically place objects in a 3D world. Develop a set of common sense predicates that accurately describes object placement.

Human Computer Interaction Um, I want a table with a vase on it (X,Y,Z)? There is a disconnect between how people express object placement and how computers do. Common Sense knowledge will allow computers to render environments without requiring specific instructions for every action

System Architecture Level Editor with Common Sense Plug-in File Render the environment in the Morrowind Engine Algorithms to convert predicates to XYZ coordinates and intelligently place objects in a 3D world Knowledge about objects Natural Language Statements XML file with a list of objects XML file with RDF predicates about Objects (automatically generate)

Common Sense Predicates Develop a set of predicates that can describe object placement. E.g. On top of, Around, Beside, etc… How people describe object locations: –“the chairs around the table” –“the vase on top of the table”

Common Sense Predicates On Top ofBelow Center ofCorner of Edge ofAround InsideBeside In front ofBehind Against

Common Sense Predicates Dealing with Multiple Predicates Vase goes on top of a table. Vase goes in the center of a table. Place the vase on top of and in the center of a table.

Common Sense Predicates Dealing with Fuzzy Predicates What does “edge of” mean? How close to the edge? It depends on the object E.g. plates around edge of a table vs. frosting around edge of a cake Requires more common sense about object properties, like don’t want plates to fall off.

Our Proof of Concept The system makes mistakes, this is true with all applications that use common sense reasoning However, we have a working proof of concept

Example Scene These objects were automatically placed using Common Sense Reasoning

Future Work We are now exploring to what extent these algorithms can scale to more complicated environments