1. Evolution, Brains and Multiple Objectives
By Jacob Schrum

2. About Me B.S. from S.U. in 2006 Currently Ph.D. student at U.T. Austin
Majors: Math, Computer Science and German
Honors Thesis w/ Walt Potter: Genetic Algorithms and Neural Networks
Currently Ph.D. student at U.T. Austin
Received M.S.C.S. in 2009
Neural Networks Research Group: Genetic Algorithms and Neural Networks

3. Evolution Change in allele frequencies in population
Alleles = variant gene forms
Genes ⇨ traits
Traits affect:
Survival
Reproduction
Natural selection favors good traits

4. Genetic Algorithms Abstraction of evolution
Genes = bits, integers, reals
Natural selection = fitness function
Mutation = bit flip, integer swap, random perturbation, …
Crossover = parents swap substrings
Other representations, mutation ops, crossover ops, …

5. Boolean Satisfiability
Applications
Boolean Satisfiability
K. A. De Jong and W. M. Spears, “Using Genetic Algorithms
to Solve NP-Complete Problems” ICGA 1989

6. Applications Magic Squares
T. Xie and L. Kang, "An evolutionary algorithm for magic squares" CEC 2003

7. Applications Circuit Design
J. D. Lohn and S. P. Colombano, "A circuit representation
technique for automated circuit design" EC 3:3 Sep. 1999

8. Wing Design/Cost Optimization
Applications
Wing Design/Cost Optimization
J. L. Rogers and J. A. Samareh, "Cost Optimization with a Genetic Algorithm"
NASA Langley Research Center, RTA , October 2000

9. Traveling Salesman Problem
Applications
Traveling Salesman Problem
P. Jog, J. Y. Suh, and D. van Gucht. "The effects of population size,
heuristic crossover and local improvement on a genetic algorithm for
the traveling salesman problem" ICGA 1989.

10. Resource-Constrained Scheduling
Applications
Resource-Constrained Scheduling
S. Hartmann, "A competitive genetic algorithm for
resource-constrained project scheduling" NRL

11. Applications Lens Design
X. Chen and K. Yamamoto, "Genetic algorithm and its application in lens design", SPIE 1996

12. Weight Selection for Fixed Neural Networks
Applications
Weight Selection for Fixed Neural Networks
F.H.F. Leung, H.K. Lam, S.H. Ling and P.K.S. Tam, "Tuning of the structure and parameters of a neural network using an improved genetic algorithm" NN 14:1 Jan. 2003

13. What Are Neural Networks?
Applications
What Are Neural Networks?

14. Artificial Neural Networks
Brain = network of neurons
ANN = simple model of brain
Neurons organized into layers

15. What Can Neural Networks Do?
In theory, anything!
Universal Approximation Theorem
NNs are function approximators
In practice, learning is hard
Supervised: Backpropagation
Unsupervised: Self-organizing maps
Reinforcement Learning: Temporal-difference learning and Evolutionary computation

16. Neuro-Evolution Genetic Algorithms + Neural Networks
Many different network representations
Fixed length string Subpopulations for each Evolve topology and weights
hidden layer neuron [1] [2]
[1] F. Gomez and R. Miikkulainen, "Incremental Evolution Of Complex General Behavior" Adaptive Behavior 5, 1997.
[2] K. O. Stanley and R. Miikkulainen, "Evolving Neural Networks Through Augmenting Topologies" EC 10:2, 2002.

17. Constructive Neuroevolution
Population of networks w/ no hidden nodes
Random weights and connections

18. Constructive Neuroevolution
Evaluate, assign fitness
Select the fittest to survive

19. Constructive Neuroevolution
Fill out population
Crossover and/or cloning
Crossover
Clone

20. Constructive Neuroevolution
Random mutations
Perturb weight, add link, splice neuron
No mutation Perturb weight Add link Splice neuron

21. Constructive Neuroevolution
Can add recurrent links as well
Provide a form of memory

22. Neuroevolution Applications
Double Pole Balancing
F. Gomex and R. Miikkulainen, “2-D Pole Balancing With
Recurrent Evolutionary Networks” ICANN 1998

23. Neuroevolution Applications
Robot Duel
K. O. Stanley and R. Miikkulainen, "Competitive Coevolution
through Evolutionary Complexification" JAIR 21, 2004

24. Neuroevolution Applications
Vehicle Crash Warning System
N. Kohl, K. Stanley, R. Miikkulainen, M. Samples, and R. Sherony,
"Evolving a Real-World Vehicle Warning System" GECCO 2006

25. Neuroevolution Applications
Training Video Game Agents
K. O. Stanley, B. D. Bryant, I. Karpov, R. Miikkulainen, "Real-Time
Evolution of Neural Networks in the NERO Video Game" AAAI 2006

26. What I Do With Neuroevolution
Discover complex behavior
Multiagent domains
Simulations, robotics, video games
Support for multiple modes of behavior
Multiobjective optimization

27. Mutiobjective Optimization
Pareto dominance: iff
Assumes maximization
Want nondominated points
NSGA-II [3] used
Popular EMO method
Nondominated
[3] K. Deb, S. Agrawal, A. Pratap, T. Meyarivan, "A Fast Elitist Non-dominated sorting
genetic algorithm for multi-objective optimization: NSGA-II" PPSN VI, 2000

28. Non-dominated Sorting Genetic Algorithm II
Population P with size N; Evaluate P
Use mutation to get P´ size N; Evaluate P´
Calculate non-dominated fronts of {P È P´} size 2N
New population size N from highest fronts of {P È P´}

29. Evolve Game AI Game where opponents have multiple objectives
Inflict damage as a group
Avoid damage individually
Stay alive individually
Objectives are contradictory and distinct
Opponents take damage from bat
Player is knocked back by NPC

30. Intelligent Baiting Behavior

31. How to avoid stagnation
Some trade-offs are too easy to reach
Focus on difficult objectives
TUG: Targeting Unachieved Goals
Avoids need for incremental evolution
Evolution
Hard Objectives

32. Smaller Team w/ Expert Timing

33. Multitask Domains Perform separate tasks Predator/Prey
Prey: run away
Pred: prevent escape
Front/Back Ramming
Attack with ram on front
Attack with ram on back

34. Multimodal Networks One network, multiple policies
Multitask [4] = one mode per task
Mode mutation = network chooses mode to use
Multitask Mode Mutation
Two tasks, two modes Start with one mode, mutation adds another
Appropriate mode used for task Preference neurons control mode choice
[4] R. A. Caruana, "Multitask learning: A knowledge-based source of inductive bias" ICML 1993

35. Multimodal Predator/Prey Behavior
Learned with Mode Mutation
Runs away in Prey task Corralling behavior in Predator task

36. Multimodal Front/Back Ramming Behavior
Learned with Multitask
Efficient front ramming Immediately turn around to attack
with back ram

37. What about “real” domains?
Unreal Tournament 2004
Commercial video game
Basis for BotPrize competition: Bot Turing Test
Placed 2nd with our bot: UT^2

38. UT^2 Behavior/Judging Game

39. Summary Neural networks can represent complex behavior
Neuroevolution = way to discover this behavior
Multiobjective evolution needed in complex domains
Success in challenging designed/commercial domains

40. Questions. E-mail: schrum2@cs. utexas. edu Webpage: http://www. cs

41. Auxiliary Slides
Empirical results

42. Differences for Alternating and Chasing significant with p < .05