1 Incremental Evolution of Autonomous Controllers for Unmanned Aerial Vehicles using Multi-objective Genetic Programming Gregory J. Barlow, Choong K. Oh, and Edward Grant North Carolina State University U.S. Naval Research Laboratory

2 Overview Problem Unmanned Aerial Vehicle Simulation Multi-objective Genetic Programming Fitness Functions Experiments and Results Conclusions Future Work

3 Background We have previously evolved unmanned aerial vehicle (UAV) navigation controllers able to: Fly to a target radar based only on sensor measurements Circle closely around the radar Maintain a stable and efficient flight path throughout flight

4 Problem We are most interested in the more difficult radar types, particularly intermittently emitting, mobile radars Evolving controllers directly on the most difficult radars yields very low rates of success We would like to create controllers able to handle all of the radar types rather than having one controller for each type

5 Simulation To test the fitness of a controller, the UAV is simulated for 4 hours of flight time in a 100 by 100 square nmi area The initial starting positions of the UAV and the radar are randomly set for each simulation trial

6 Sensors UAVs can sense the angle of arrival (AoA) and amplitude of incoming radar signals

7 UAV Control Evolved Controller Autopilot UAV Flight Sensors Roll angle

8 Transference These controllers should be transferable to real UAVs. To encourage this: Only the sidelobes of the radar were modeled Noise is added to the modeled radar emissions The angle of arrival value from the sensor is only accurate within ±10°

9 Multi-objective GP We had four desired behaviors which often conflicted, so we used NSGA-II (Deb et al., 2002) with genetic programming to evolve controllers Each fitness evaluation ran 30 trials Each run had a population size of 500 Computations were done on a Beowulf cluster with 92 processors (2.4 GHz)

10 Functions and Terminals Turns Hard Left, Hard Right, Shallow Left, Shallow Right, Wings Level, No Change Sensors Amplitude > 0, Amplitude Slope 0, AoA Functions IfThen, IfThenElse, And, Or, Not,, >=, > 0, < 0, =, +, -, *, /

11 Fitness Functions Normalized distance UAV’s flight to vicinity of the radar Circling distance Distance from UAV to radar when in-range Level time Time with a roll angle of zero Turn cost Changes in roll angle greater than 10°

12 Normalized Distance

13 Circling Distance

14 Level Time

15 Turn Cost

16 Performance of Evolution Multi-objective genetic programming produces a Pareto front of solutions, not a single best solution. To gauge the performance of evolution, fitness values for each fitness measure were selected for a minimally successful controller.

17 Baseline Values Normalized Distance≤0.15 Determined empirically Circling Distance≤4 Average distance less than 2 nmi Level Time≥1000 ~50% of time (not in-range) with roll angle = 0 Turn Cost≤0.05 Turn sharply less than 0.5% of the time

18 Experiments Continuously emitting, stationary radar Simplest radar case Intermittently emitting, stationary radar Period of 10 minutes, duration of 5 minutes Continuously emitting, mobile radar States: move, setup, deployed, tear down In deployed over an hour before moving again Intermittently emitting, mobile radar Most difficult radar type for evolution

19 Direct Evolution Radar Type RunsControllers TotalSucc.RateTotalAvg.Max. Continuously emitting, stationary radar %3, Continuously emitting, mobile radar %2, Intermittently emitting, stationary radar %1, Intermittently emitting, mobile radar %

20 Incremental Evolution Environmental incremental evolution was used to improve the success rate for evolving controllers A population is evolved on progressively more difficult radar types

21 Incremental Evolution Radar Type RunsControllers TotalSucc.RateTotalAvg.Max. Continuously emitting, stationary radar %2, Continuously emitting, mobile radar %2, Intermittently emitting, stationary radar %2, Intermittently emitting, mobile radar %1,

22 Comparison

23 Intermittently emitting, mobile radar

24 Conclusions Autonomous navigation controllers were evolved to fly to a radar and then circle around it while maintaining stable and efficient flight dynamics Using incremental evolution dramatically increased the chances of producing successful controllers Incremental evolution produced controllers able to handle all radar types

25 Future Work We have successfully tested evolved controllers on a wheeled mobile robot equipped with an acoustic array tracking a speaker Controllers will be tested on physical UAVs for several radar types in field tests next year Distributed multi-agent controllers will be evolved to deploy multiple UAVs to multiple radars