1. 1 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Safety Analysis Group (GAS) Computer and Digital Systems Engineering Department (PCS) School of Engineering (Escola Politécnica - Poli) University of São Paulo (USP) São Paulo, Brazil Modeling and Simulation of Autonomous UAS for Risk Evaluation of Collision with Other Aircraft (Modelagem e Simulação de VANT Autônomo para Avaliação de Risco de Colisão com Outras Aeronaves)

2. 2 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Introduction About GAS (Safety Analysis Group). Safety issues of (non-military) UAS operation. –Focus on collision avoidance (sense and avoid). –Pilot responsibility: “Nothing in these rules shall relieve the pilot-in-command of an aircraft from the responsibility of taking such action (...)” (ICAO Rules of the Air [1]). –Control and Communication (C2) link failure. Academia, foreseeable future and autonomous UAS.

3. 3 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil The manned aerial vehicle paradigm Civil Aviation: only a Remotely Piloted Aircraft (RPA) “will be able to integrate into the international civil aviation system in the foreseeable future” (ICAO Circular 328 [3])  Human pilot in command.  Pilot skill, insight, experience.  Mood, stress.

4. 4 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil The pilot in UAS operation C2 link failure: research needed (FAA, 2013 [3]). Human factors and Ergonomics (Waraich et al., 2013 [4]). –E.g.: human pilot in “Automatic pilot” mode (Catino & Patriotta, 2013 [5]).

5. 5 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Granting autonomy to the UAV Safety premise: an additional autonomous UAV in the ATC system must not increase collision risk more than an additional manned aircraft would.

6. 6 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Hypotheses for an autonomous UAV Main hypothesis: Granting the UAV an autonomous piloting capability that ‘imitates’ the human pilot would make the autonomous operation to present safety levels similar to those observed in the operation of manned aircraft. Secondary hypothesis: If the UAV imitates only correct maneuvers, the safety levels of an autonomous operation would be greater as compared to human-piloted aircraft (including RPA).

7. 7 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil ‘Imitating’ - Learn from Demonstration Learn from Demonstration (Argall et al., 2009 [6]) –Subset of Supervised Learning. –Human pilot as a Demonstration Teacher. –Piloting Autonomous System (PAS, Gimenes et al., 2013 [7]) must ‘imitate’ the Demonstration Teacher. Learning culture is a characteristic of High Reliability Organizations such as the ATC (Brooker, 2013 [8]).

8. 8 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil UAV Model and CNS/ATM Human pilot → see and avoid. UAV → sense and avoid. Use of CNS/ATM technologies for Sense capability.  Automatic Dependent Surveillance – Broadcast (ADS-B).

9. 9 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil PAS Model

10. 10 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Model Evaluation Proof of concept –UAV assembly and training. –Qualitative evaluation. Simulation –Aircraft encounter models (Kochenderfer et al., 2010 [9]) Source: [9]

11. 11 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Model Evaluation (cont.) Simulation (cont.) –Adaptation of (trained) PAS to the simulation environment. –Monte Carlo simulation (quantitative evaluation): Trained PAS x Encounter models aircraft

12. 12 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Related Works Unmanned aircraft control:  Learning-based: Neural Networks, Support Vector Machines, Learning from Demonstration (Ross et al., 2013 [10]).  Focus on the computational problem only. Simulation-based collision risk estimation:  Aircraft encounter models (Kochenderfer et al., 2010 [9]).  Comparison of collision avoidance methods (Holt et al., 2013 [11] e por Alexopoulos et al., 2013 [12]).

13. 13 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Future Work Execution of training and proof of concept: –UAV assembly. –PAS training. –Verification of collision avoidance capability. Collision risk evaluation: –Selection of the encounter model. –Adaptation of PAS to the simulation environment. –Evaluation of the results.

14. 14 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Final Considerations Learn from Demonstration concept applied to autonomous UAS. Evaluation criterion: autonomous UAS safety levels must be equal or greater as compared to manned aircraft. Metrics for collision avoidance capability. Definition of next steps to obtain results. Computation problem vs air traffic problem.

15. 15 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil References [1] International Civil Aviation Organization, Annex 2 to the Convention on International Civil Aviation - Rules of the Air, no. July. 2005, pp. 1–74. [2] International Civil Aviation Organization, “ICAO Cir 328, Unmanned Aircraft Systems (UAS),” 2011. [3] Federal Aviation Administration, “Integration of Civil Unmanned Aircraft Systems (UAS) in the National Airspace System (NAS) Roadmap,” 2013. [4] Q. R. Waraich, T. a. Mazzuchi, S. Sarkani, and D. F. Rico, “Minimizing Human Factors Mishaps in Unmanned Aircraft Systems,” Ergon. Des. Q. Hum. Factors Appl., vol. 21, no. 1, pp. 25–32, Jan. 2013. [5] M. Catino and G. Patriotta, “Learning from Errors: Cognition, Emotions and Safety Culture in the Italian Air Force,” Organ. Stud., vol. 34, no. 4, pp. 437–467, Mar. 2013. [6] B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A survey of robot learning from demonstration,” Rob. Auton. Syst., vol. 57, no. 5, pp. 469–483, May 2009. [7] R. A. V. Gimenes, L. F. Vismari, V. F. Avelino, J. B. Camargo, J. R. Almeida, and P. S. Cugnasca, “Guidelines for the Integration of Autonomous UAS into the Global ATM,” J. Intell. Robot. Syst., vol. 74, no. 1–2, pp. 465–478, Sep. 2013. [8] P. Brooker, “Introducing Unmanned Aircraft Systems into a High Reliability ATC System,” J. Navig., vol. 66, no. 05, pp. 719–735, Jun. 2013. [9] M. J. Kochenderfer, M. W. M. Edwards, L. P. Espindle, J. K. Kuchar, and J. D. Griffith, “Airspace Encounter Models for Estimating Collision Risk,” J. Guid. Control. Dyn., vol. 33, no. 2, pp. 487–499, Mar. 2010. [10] S. Ross, N. Melik-Barkhudarov, K. S. Shankar, A. Wendel, D. Dey, J. A. Bagnell, and M. Hebert, “Learning monocular reactive UAV control in cluttered natural environments,” 2013 IEEE Int. Conf. Robot. Autom., pp. 1765–1772, May 2013. [11] J. Holt, S. Biaz, and C. A. Aji, “Comparison of Unmanned Aerial System Collision Avoidance Algorithms in a Simulated Environment,” J. Guid. Control. Dyn., vol. 36, no. 3, pp. 881–883, May 2013. [12] A. Alexopoulos, A. Kandil, P. Orzechowski, and E. Badreddin, “A Comparative Study of Collision Avoidance Techniques for Unmanned Aerial Vehicles,” in 2013 IEEE International Conference on Systems, Man, and Cybernetics, 2013, pp. 1969–1974.

16. 16 http://www.gas.pcs.poli.usp.br/ Universidade de São Paulo Brasil Thiago T. Matsumoto, M.Sc. student ttmatsumoto@usp.br Phone: +55 11 3091-5323 THANK YOU FOR YOUR ATTENTION For more information: www.gas.pcs.poli.usp.br