1 Robustness Analysis of Genetic Programming Controllers for Unmanned Aerial Vehicles Gregory J. Barlow 1,2 and Choong K. Oh 2 1 The Robotics Institute, Carnegie Mellon University 2 The U.S. Naval Research Laboratory

2 Motivation Evolutionary robotics (ER) controllers may evolve in simulation or on real robots, but the true test of performance must happen in real-world conditions Testing unfit controllers may be dangerous or expensive for some robots

3 Transference For controllers evolved in simulation, evaluation in a noisy environment does not ensure good transference if simulated noise is not consistent with true noise If a controller performs well over a wide range of state and sensor noise conditions in simulation, prior work suggests that the controller should transfer well

4 Evolving controllers for unmanned aerial vehicles Unmanned aerial vehicles (UAVs) require assurance of off-design performance Even under noise not considered during evolution, controllers must still be able to efficiently accomplish the task Poorly performing controllers could cause crashes, possibly destroying the UAV

5 Overview Controller evolution (Barlow et al., 2004) Goals Robustness testing Results Conclusions

6 Controller evolution Evolve unmanned aerial vehicle (UAV) navigation controllers able to: Fly to a target radar based only on sensor measurements Circle closely around the radar Maintain a stable and efficient flight path throughout flight

7 Controller Requirements Autonomous flight controllers for UAV navigation Reactive control with no internal world model Able to handle multiple radar types including mobile radars and intermittently emitting radars Robust enough to transfer to real UAVs

8 Simulation To test the fitness of a controller, the UAV is simulated for 4 hours of flight time in a 100 by 100 square nmi area The initial starting positions of the UAV and the radar are randomly set for each simulation trial

9 Sensors UAVs can sense the angle of arrival (AoA) and amplitude of incoming radar signals

10 UAV Control Evolved Controller Autopilot UAV Flight Sensors Roll angle

11 Radars Stationary, continuously emitting Mobile, continuously emitting Stationary, intermittently emitting with regular period Stationary, intermittently emitting with irregular period Mobile, intermittently emitting with regular period

12 Transference To encourage good transference to real UAVs, during evolution: Modeled only the sidelobes of radars Added noise to the modeled radar emissions Set accuracy of the angle of arrival values to be within ±10° Evolved controllers were successfully tested on wheeled mobile robots (Barlow et al., 2005)

13 Multi-objective GP We had four desired behaviors which often conflicted, so we used NSGA-II (Deb et al., 2002) with genetic programming to evolve controllers Each evaluation ran 30 simulations Each of 50 evolutionary runs had a population size of 500 We used environmental incremental evolution to produce controllers evolved for a total of 1800 generations

14 Fitness Functions Normalized distance UAV’s flight to vicinity of the radar Circling distance Distance from UAV to radar when in-range Level time Time with a roll angle of zero Turn cost Changes in roll angle greater than 10°

15 Functions and Terminals Functions Prog2, Prog3, IfThen, IfThenElse, And, Or, Not,, >=, 0, =, +, -, *, /, X max, Y > max, Amplitude > 0, AmplitudeSlope > 0, AmplitudeSlope Arg, AoA < Arg Terminals HardLeft, HardRight, ShallowLeft, ShallowRight, WingsLevel, NoChange, rand, 0, 1

16 Considerations We have many acceptable controllers on the Pareto front, but we need to choose one “best” controller Controllers may be optimized to the simulation parameters, may not be robust to other noise levels or sources Fitness values are only measured over 30 trials

17 Goals Choose a single “best” evolved controller for future flight tests Evaluate the robustness of the best evolved controllers to sensor and state noise to assure off-design performance Compare evolved controllers to human designed controllers

18 Test functions 1. Flying to the radar Percent error in time to radar 2. Circling the radar Average circling distance 3. Efficient flight Percent error in flying with a roll angle of zero degrees 4. Stable flight Cost of sharp, sudden turns

19 Test functions

20 Baseline Values Flying to the radar≤0.2 Error in flight time to radar less than 20% Circling the radar≤2 Average distance less than 2 nmi Efficient flight≤0.5 ~50% of time (not in-range) with roll angle = 0 Turn Cost≤0.05 Turn sharply less than 0.5% of the time

21 Performance metrics Failures Percent of trials that don’t meet the baseline values Normalized maximum Magnitude of failure normalized by the baseline value Normalized mean Means for each test function normalized by the baseline value and then averaged Average rank Combination of first three performance metrics

22 Performance metrics

23 Selecting controllers for testing 1. GP produced 25,000 controllers 2. Based on prior work, 1,602 had acceptable mean fitness values 3. We ran 100 simulations on each of the five radar types for each of these 1,602 and chose ~ We cut these down to 10 using the normalized maximum performance metric

24 Designed controllers Hand-written Based on functions and terminals available to GP and knowledge of good GP strategies Proportional-derivative (PD) Takes AoA as input (approximates derivative) PID performed poorly with mobile radars, so integral term was not used

25 Robustness tests Robustness tests fell into five categories: AoA error, amplitude error, varied airspeed, heading error, and wind effects (position error) For every combination of radar type and controller, we performed 10,000 simulations, for a total of 50,000 simulations per controller per test

26 Robustness tests Angle of arrival error ±{10, 15, 20, 30}° Amplitude error {6, 12} dB UAV airspeed {50, 80, 100} knots Heading error {0, 0.5, 1, 1.5, 2}° Wind (position error) {0, 5, 10, 20, 30} knots

27 Results For each test, we ranked the 12 controllers based on the four performance metrics We combined these results into an overall ranking to determine the best controller The best evolved controller fails gracefully and compares well to the PD controller

28 Rankings Overall rankingbest failuresGDEFJHACBpdIhd norm maximumpdDIFGhdJEBAHC norm meanDEhdGFJHpdACBI average rankDGEpdFJHhdABCI Control case rankingbest failurespdABCDEFGHIJhd norm maximumpdGEJFIBADCHhd norm meanpdIJABCGEDFHhd average rankpdJGIEBAFCDHhd

29 Comparison Stationary, intermittently emitting Mobile, intermittently emitting Average for all five radar types Test typeDpdD D control case AoA= AoA= AoA= AoA= Amp= Speed= Speed= Stationary, intermittently emitting Mobile, intermittently emitting Average for all five radar types Test typeDpdD D Head= Head= Head= Head= Wind= Wind= Wind= Wind=

30 Conclusions Selected a single best controller for future flight tests Established the off-design performance of evolved UAV controllers; evolved controllers failed gracefully Performance of the best controller compares favorably with PD control Established the limits of performance for these evolved controllers

31 Acknowledgements Financial support was provided by Swampworks project office of the Office of Naval Research The U.S. Naval Research Laboratory (Code 5730) provided computation time on their Beowulf cluster Gregory J. Barlow is supported by a National Defense Science and Engineering Graduate Fellowship