1. Neuro-evolving Maintain-Station Behavior for Realistically Simulated Boats Nathan A. Penrod David Carr Sushil J. Louis Bobby D. Bryant Evolutionary Computing Systems Lab Neuroevolution and Behavior Laboratory University of Nevada, Reno

2. Related Work

3. Maintain Station ● One or more boats follow a leader at specified offsets ● Allows groups of boats to move in formation ● Current work tests only the ability to maintain position ● Ultimate goal is to include following with obstacle avoidance

4. Maintain Station Continued ● Maintain station has two phases – First finding and moving toward the target point – Second staying on the target point by matching the heading and speed of the lead boat ● Boats have complex physics including friction and inertia ● Maintain behavior is difficult because of limited maneuverability ● Complex physics makes problem interesting

5. Lagoon ● Real-time 3D naval combat simulation game ● Provides complex physics and flexible control architecture ● Allows for easy integration of experimental control devices like neural networks

6. Approach ● Simple recurrent neural network (SRN) (add cite) ● Six inputs, two outputs, eight hidden ● ESP evolutionary algorithm (Gomez and Miikkulainen 1998) ● Neural network implicitly learns physics

7. Maintain Vector ● Method of representing the maintain station behavior ● All sensor and fitness values are based on the maintain vector ● Abstracts away from relative offset ● Following boat does not care about lead boat's position (is unaware if the relative vector position is changed to create new formations) ● Increases the generality of the behavior

8. Representation ● Three points evenly spaced around target point ● Egocentric vectors cast from boat to sensor points ● x and y components of normalized vectors provide sensor inputs ● Inputs are continuous ● Informs boat about direction to target and rotation relative to lead boat

9. Fitness ● vL – vector with the same heading as the maintain vector ● vH – vector with the same heading as the boat ● vP – vector cast from boat to target point ● dT – Euclidean distance from boat to target

10. Fitness continued ● Function 1: – Rewards moving towards target ● Function 2: – Rewards moving towards and away from target ● Function 3: – Rewards moving towards target while matching heading of lead boat ● Fitness values integrated over time

11. Experiment 1 ● Boats start in world with random positions and headings. ● Lead boat travels in a straight line at exactly half of its maximum speed ● Maintaining boats must acquire and then maintain position on their target points

12. Experiment 1 Results ● Results surprisingly similar for each fitness function ● Similar growth rates and maximum fitness values ● Results averaged over ten runs

13. Experiment 2 ● Boats start in world with random positions and headings. ● Lead boat travels in a straight line at constant speed chosen at random from between 25% and 75% ● Maintaining boats must acquire and then maintain position on their target points

14. Experiment 2 Results ● Again results are similar for each fitness function ● All functions learned the desired behavior ● Results averaged over ten runs

15. Qualitative Observations ● Boats trained with different fitness functions display subtle differences in behaviors ● Boats trained with Function 1 maintain well but can struggle in the approach ● Boats trained with Function 2 approach well but sometimes struggle to maintain ● Boats trained with Function 3 are better at balancing maintaining and approaching

16. Conclusions ● Neuroevolution is capable of producing the maintain station behavior ● All three fitness functions produced acceptable results ● A fitness function that considers both orientation relative to target point and orientation relative to target heading produces the most balanced results

17. Future Work ● Expand network to perform maintain station with obstacle avoidance ● Add new sensors to act as radar for detecting boats and land ● Alter fitness function to encourage avoidance behaviors

18. Acknowledgments This material is based upon work supported by the Office of Naval Research under contract number N00014-05-0709

19. Bibliography ● F. Gomez and R. Miikkulainen, “2-D pole-balancing with recurrent evolutionary networks,” ● in Proceedings of the International Conference on Artificial Neural Networks. ● Berlin; New York: Springer-Verlag, 1998, pp. 425–430. ● Available:http://nn.cs.utexas.edu/keyword?gomez:icann98