1. Mike Taks Bram van de Klundert

2. About Published 2005 Cited 286 times Kenneth O. Stanley Associate Professor at University of Central Florida Risto Miikkulainen Professor at the University of Texas at Austin Bobby D. Bryant Assistant Professor at university of Nevada

3. Contents Introduction NEAT rtNEAT NERO

4. Introduction real-time NeuroEvolution of Augmenting Topologies (rtNEAT) Adaption of the NEAT algorithm Create new genre of games requiring learning Black and white Tamagotchi

5. NEAT Neuro Evolution of Augmenting Topologies Growing neural network

6. Representation List of connection genes Innovation number Global counter In node Out node Weight Enabled

7. Initial population Uniform population of simple networks No hidden nodes Random weights

8. Mutation Weight mutation Structure mutation Add a connection between two nodes Replace a connection by a node Connection not removed only disabled Out connection inherits the value

9. Crossover terms Disjoint: gene is only in one network Excess: disjoint and outside of the range of innovations

10. Crossover shared genes: Uniform crossover Blend crossover Disjoint and excess genes Taken from most fit parent

11. Crossover example Equal fitness 9, 10 excess 6, 7, 8 disjoint

12. Speciation

13. Species assignment Check if there is a species close enough to the individual If not, create new species

14. Fitness

15. Selection

16. Trailer NERO

17. rtNEAT Differences Selection and replacement Removing agent

18. Differences Work real time Originally NEAT evaluates one complete generation of individuals, generates offspring “en masse”

19. Differences During a game, performance statistics are being recorded Replacing agents Perform actions every n game-ticks

20. Selection and replacement Calculate fitness Remove worst agent of sufficient age Choose parents among the best Create offspring, Reassign all agents to species

21. Removing agent Remove worst agent based fitness adjusted for species size New agents are continuously born, life time individually kept track of Possibility to just replace the neural network of an agent

22. NERO Player is a trainer Set up exercises Save and load neural networks

23. Training mode Place objects on the field (static enemies, turrets, rovers, flags,...) Adjust fitness rewards by sliders

24. Training mode Agents spawn in the “factory”

25. Sensors Radar to track enemy location Rangefinder Line-of-fire...

26. Evolved topology

27. Battle mode Assemble team of 20 agents Ends if one team is empty

28. Experiments Slightly nondeterministic game engine The same game is thus never played twice

29. Behaviors Different behaviors are trained Seek and fire by placing a single static enemy on the training field Firing and hitting a target was to slow to evolve. Aiming script was used

30. Behaviors Avoidance trained by controlling an agent manually Agent runs backwards facing the enemy and shooting at it

31. Behaviors Train agents to avoid turret fire

32. More complex behaviors Let agents attack enemy behind a wall Train agents to avoid hazardous corridors

33. More complex behaviors Train agents versus targets that are standing against a wall

34. More complex behaviors Incrementally add walls, agents will be able to navigate

35. Battling Paper rock scissors Seek vs avoidance

36. Battling

37. Questions?