Generating Diverse Opponents with Multi-Objective Evolution Generating Diverse Opponents with Multi-Objective Evolution Alexandros Agapitos, Julian Togelius, Simon M. Lucas, J¨urgen Schmidhuber and Andreas Konstantinidis Presented by Patoka Amir

Overview  Introduction  Objectives  Multi-objective evolutionary algorithms  Results  Future work

Introduction  F.E.A.R. winer of GameSpot Best Artificial Intelligence)  Easy construction of game AI (F.E.A.R. winer of GameSpot Best Artificial Intelligence) F.E.A.R.   Industry shifts focus to building interesting, divers and believable CI.

Introduction (Cont)  DIFFICULTY LEVELS:  Barbarians  Free Units  Research  Maintenance Costs  Health and Happiness  Artificial Intelligence Penalties  AI Freebies  Tribal Villages

Introduction (Cont)   Predictable AI  When Priorities Go Wrong!: NPC investigates a burning barrel that was thrown by the player and landed nearby. The barrel subsequently explodes while the NPC is nearby looking at it.

Introduction (Cont)   Propose a general approach to creating diverse and interesting NPC behaviors using Multi-objective evolutionary algorithms (MOEA) in combination with a number of partly conflicting behavioral fitness measures.

Overview  Introduction  Objectives  Multi-objective evolutionary algorithms  Results  Future work

Objectives  Optimize a genetically programmed car controller to exhibit:  Aggressiveness.  Opponent weakness exploitation.

Objectives (Cont.)  Environment:  A 2D simulator, modeling a radio controlled toy car (three possible drive and steering modes).  A track consisting of walls, a chain of waypoints and a set of staring points and directions (subject to random alteration).  A reasonable model of car dynamics, collisions.  A competitor (with an incrementally evolved general controller).

Objectives (Cont.)  Controller employ two expression trees representation (driving and steering) containing:  Standard arithmetic and trigonometric functions.  Formal parameters representing car state as viewed by first person sensors.

Objectives (Cont.)  Behavioral fitness measures:  Absolute progress.  Relative progress.  Maximum speed.  Progress variance.  # Steering changes.  # Driving changes.  Wall collisions.  Competitor proximity.  Max Car collisions.  Min Car Collisions.

Objectives (Cont.)  Algorithm:  Non-Dominated Sorting Genetic Algorithem (NSGA-II).  Tournament selection (starting with size 7 during final 10 generations increases by 20% each generation).  50 generations.  500 individuals.  Expression trees are limited to depth of 17 and created with a maximum depth of 8 through Ramped-half-and-half.

Overview  Introduction  Objectives  Multi-objective evolutionary algorithms  Results  Future work

MOEA Non-Dominated Sorting Genetic Algorithem  Pareto frontier:

Overview  Introduction  Objectives  Multi-objective evolutionary algorithms  Results  Future work

Results Aggressiveness – wall collisions avoidance  Fitness = max absolute progress + min wall collisions.

Results Aggressiveness – max speed & min steering  Fitness = max absolute progress + min wall collisions + min # steering changes.

Results Aggressiveness – max speed & min steering  Fitness = max absolute progress + min wall collisions + min # steering changes.

Results Aggressiveness – max speed & min driving  Fitness = max absolute progress + min wall collisions + min # driving changes.

Results Aggressiveness – max speed & min driving  Fitness = max absolute progress + min wall collisions + min # driving changes.

Results Aggressiveness – smoothness, avoidance and low speed  Fitness = max absolute progress + min wall collisions + max # driving changes.

Results Aggressiveness – smoothness, avoidance and low speed  Fitness = max absolute progress + min wall collisions + max # driving changes.

Results Aggressiveness – max car collisions  Fitness = max absolute progress + max car collisions + min car closeness + min # driving changes.

Results Aggressiveness – Car collisions  Fitness = max absolute progress + max car collisions + min car closeness + min # steering & driving changes.

Results Aggressiveness – Opponent weakness Exploitation  Fitness = max absolute progress + max speed + min car closeness + min # steering & driving changes.

Overview  Introduction  Objectives  Multi-objective evolutionary algorithms  Results  Future work

Future work   Prove concept on other game genres.

The End Any Questions ?

The End Thank you ;)