Mike Taks Bram van de Klundert

About Published 2005 Cited 286 times Kenneth O. Stanley Associate Professor at University of Central Florida Risto Miikkulainen Professor at the University of Texas at Austin Bobby D. Bryant Assistant Professor at university of Nevada

Contents Introduction NEAT rtNEAT NERO

Introduction real-time NeuroEvolution of Augmenting Topologies (rtNEAT) Adaption of the NEAT algorithm Create new genre of games requiring learning Black and white Tamagotchi

NEAT Neuro Evolution of Augmenting Topologies Growing neural network

Representation List of connection genes Innovation number Global counter In node Out node Weight Enabled

Initial population Uniform population of simple networks No hidden nodes Random weights

Mutation Weight mutation Structure mutation Add a connection between two nodes Replace a connection by a node Connection not removed only disabled Out connection inherits the value

Crossover terms Disjoint: gene is only in one network Excess: disjoint and outside of the range of innovations

Crossover shared genes: Uniform crossover Blend crossover Disjoint and excess genes Taken from most fit parent

Crossover example Equal fitness 9, 10 excess 6, 7, 8 disjoint

Speciation

Species assignment Check if there is a species close enough to the individual If not, create new species

Fitness

Selection

Trailer NERO

rtNEAT Differences Selection and replacement Removing agent

Differences Work real time Originally NEAT evaluates one complete generation of individuals, generates offspring “en masse”

Differences During a game, performance statistics are being recorded Replacing agents Perform actions every n game-ticks

Selection and replacement Calculate fitness Remove worst agent of sufficient age Choose parents among the best Create offspring, Reassign all agents to species

Removing agent Remove worst agent based fitness adjusted for species size New agents are continuously born, life time individually kept track of Possibility to just replace the neural network of an agent

NERO Player is a trainer Set up exercises Save and load neural networks

Training mode Place objects on the field (static enemies, turrets, rovers, flags,...) Adjust fitness rewards by sliders

Training mode Agents spawn in the “factory”

Sensors Radar to track enemy location Rangefinder Line-of-fire...

Evolved topology

Battle mode Assemble team of 20 agents Ends if one team is empty

Experiments Slightly nondeterministic game engine The same game is thus never played twice

Behaviors Different behaviors are trained Seek and fire by placing a single static enemy on the training field Firing and hitting a target was to slow to evolve. Aiming script was used

Behaviors Avoidance trained by controlling an agent manually Agent runs backwards facing the enemy and shooting at it

Behaviors Train agents to avoid turret fire

More complex behaviors Let agents attack enemy behind a wall Train agents to avoid hazardous corridors

More complex behaviors Train agents versus targets that are standing against a wall

More complex behaviors Incrementally add walls, agents will be able to navigate

Battling Paper rock scissors Seek vs avoidance

Battling

Questions?