國立雲林科技大學 National Yunlin University of Science and Technology Building Reactive Characters for Dynamic Gaming Environments Peter Blackburn and Barry O’Sullivan, IEEE Symposium on Computational Intelligence and Games, Presenter : Wei-Shen Tai Advisor : Professor Chung-Chian Hsu 2006/2/15

N.Y.U.S.T. I. M. Outline Introduction Background Decision Tree Learning Integration with Tournament Evaluation Conclusions Comments

N.Y.U.S.T. I. M. Motivation Past “first person shooter” games Typically provide human players with opponents that simply acted as “cannon fodder”. Little effort was placed in developing complex behaviours for them. Intelligent opponents human-like decision making capabilities. sophisticated AI is a factor in ensuring a high replay rate for games.

N.Y.U.S.T. I. M. Objective Purely reactive non-player characters (NPC) Enables developers to incorporate AI in games with minimal development overhead. Building game opponents in this way yields natural reactive behaviour.

N.Y.U.S.T. I. M. Method Traditional finite-state machine (FSM) architecture the various states and the transitions from one state to another. Decision tree induced from the strategies that human players would employ as a basis for defining the conditions under which each state is reached. defines the complex relationships between each state giving rise to behaviour by re-evaluating the state (classification) suggested.

N.Y.U.S.T. I. M. Link to Unreal Tournament GameBot A Programming environment for controlling NPCs. the client-server architecture developed at the University Of Southern California’s information Science Institute. JavaBot Built on top of the GameBot infrastructure and control NPCs easily without having to worry about the underlying network protocols.

N.Y.U.S.T. I. M. Evaluation and results Evaluation way using decision trees compete against a collection of nine other standard Unreal Tournament NPCs in a death-match context. Competitive NPC consistently performing better than both the standard novice and average skilled characters.

N.Y.U.S.T. I. M. Introduction “first person shooter” Opponents (NPCs) that simply acted as “cannon fodder”. game players have grown bored due to the lack of sophistication of the opponents. AI (Artificial Intelligent) NPCs can evade attacks and plan retreats in order to obtain support or to find a better tactical vantage point.

N.Y.U.S.T. I. M. Sophisticated AI Intelligent opponents display human-like decision making capabilities. Purely reactive character that has skills based upon the strategies a human player would use.

N.Y.U.S.T. I. M. FSM and DT Traditional finite-state machine (FSM) architecture separating the various states and the conditions that determine the transitions from one state to another. Decision tree (DT) induced from the strategies that human players would employ in various gaming scenarios. defines the complex relationships between each state giving rise to behaviour by re-evaluating the state (classification) suggested.

N.Y.U.S.T. I. M. Contributions Proposed method building complex, but realistic, behaviour into game characters through the use of decision trees induced from human strategies; Cost reduces the software development burden required to build complex behaviours into computer games.

N.Y.U.S.T. I. M. Past works “behavioural cloning” Capture human sub-cognitive skills and incorporate them into a computer program. The learning program produces a set of rules that can reproduce the skilled behaviour. Artificial Neural Networks (ANN) could be used to train an NPC to learn a set of basic commands from an experienced player.

N.Y.U.S.T. I. M. DT advantages Common idea in last two techniques Training an NPC by monitoring actual game play over a long period of time. Decision tree building competitive NPCs based on human- provided responses to a number of gaming scenarios. with significantly lower training data requirements.

N.Y.U.S.T. I. M. Decision tree learning Approximating discrete-valued functions make classifications by sorting the features of an instance through the tree from the root to some leaf node. Constructed in a top-down fashion by adding a node that performs a test on a feature of the problem that best classifies the set of training examples.

N.Y.U.S.T. I. M. DT example Case 1

N.Y.U.S.T. I. M. Integration with Unreal Tournament Objective in the death match game Several characters compete to achieve a target number of “frags” within a specified time-frame (typically 30 minutes). A frag is a score computed as the number of opponents that are eliminated by the character.

N.Y.U.S.T. I. M. Architecture for Controlling an NPC GameBot controlling our NPCs that was first developed at the University Of Southern California’s Information Science Institute. server-side provides sensory information about the environment. For example, the location of other opponents that are currently visible, the positions of obstacles. client-side provides the capability to send messages to the server, thus specifying what action an NPC should take, for example move, turn, shoot, etc.

N.Y.U.S.T. I. M. Using DT to control NPCs Typically JavaBot implements a simple state changing method to control an NPC. Decision tree-based architecture built simple independent states that would be triggered by classifications obtained from the decision tree. These states provide very basic behaviours. provide our NPC with human-like behaviour and skill.

N.Y.U.S.T. I. M. Acquiring strategies from a human player Captured approach from skilled human player a form of user interview by creating a questionnaire-based system. The human selects one of several behavioural actions in response to each scenario.

N.Y.U.S.T. I. M. Inducing the decision tree Training set the human user response forms a training set from which a decision tree can be induced.

N.Y.U.S.T. I. M. Exploiting the Decision Tree Guide of NPCs using the decision tree to guide the behaviour of an NPC during the game. Categorical value converting Since decision trees do not work on numerical values directly, a preprocessor is required to discretise this data. For example, for health points a value of 0-40 might be considered ”Low”, might be ”Medium” and greater than 75 might be considered to be ”High”. these values usually need to be considered carefully for each particular game that this technique is being applied too. “refresh” time period can also affect the NPC’s performance quite dramatically. this time again will vary depending on the game context to which this technique is being applied too.

N.Y.U.S.T. I. M. Evaluation (1/2) Environment a character built using decision trees compete against a collection of nine other standard Unreal Tournament NPCs in a death-match context. Conditions varied the number of questions used to build the decision trees used in each case, (5 DTs) selecting values of 10, 50, 75 and 100, giving us different sized training sets. varied the skill level of the opponent characters, comparing against both novice and average skill levels.

N.Y.U.S.T. I. M. Evaluation (2/2) Other conditions Built 5 different decision trees for each training set size and used each in 50 game sessions. Each game session had a maximum time limit of 30 minutes and a target frag count of 20. For each game we recorded the score and position of the decision tree-driven character.

N.Y.U.S.T. I. M. Evaluation results Very competitive NPC consistently performing better than both the standard novice and average skilled characters in Unreal Tournament. our character is, on the average, placed in the top 3 positions in any configuration.

N.Y.U.S.T. I. M. Performance against novice skilled characters

N.Y.U.S.T. I. M. Performance against novice average characters

N.Y.U.S.T. I. M. Discussion More sample size, less performance? The objective is to eliminate as many opponents as possible. tradeoff survival and the risks in the centre of the battle to be killed or killing another. induction process with feedback about the performance of the resultant tree. Skilled opponents make less performance a slight degradation in performance as we encounter more skilled opponents.

N.Y.U.S.T. I. M. Conclusions (1/2) DT regards the choice of action that an NPC makes as a classification problem. easy to build and apply link these states together in a way that we have shown results in competitive games characters. save significant amounts of debugging only building a number of implementations of simple actions that a game character needs to perform.

N.Y.U.S.T. I. M. Conclusions (2/2) Character “personality” A developer can have some control over the personality that a character can have. This can be assisted by providing different responses to the scenarios presented on the questionnaire described earlier. Controlling a team of characters. The key idea is to build a “commander” decision tree which would determine how they behave as a group. For example, the team’s task might be to “capture the flag” of an opponent at a specified location. The commander tree initially informs the characters that they must capture the flag (their goal). Visualization tool and development environment Developers can use to build and modify decision tree based NPCs.

N.Y.U.S.T. I. M. Comments Fuzzy decision tree It is feasible to convert numerical data into linguistic terms. Acquiring strategies from real battle field Replaces the form of user interview. It is apt to acquire the respondence from skilled players with various situations in real Unreal Tournament. Evaluating through real skilled players Real-human with basic skilled should be better than NPCs.