1. Cooperative Control of UAVs A mixed-initiative approach Ltn. Elói Pereira Portuguese Air Force Academy E-mail: etpereira@emfa.pt

2. Summary AFA project on UAVs; Cooperative Control of UAVs in mixed initiative environments; Military and Civil applications; Formalism for Allocation and Exchange vehicles within teams; –Example: Load Balancing between teams; Testbed description; Conclusions and future work.

3. ANTEX – Portuguese Air Force Project on UAVs Development of UAVs platforms to use as technologies demonstrators in several fields as: –Scientific Research; –Defense; –Civil applications… Give Air Force know how in operation of UAVs; To promote R&D initiatives with others organizations: –Faculty of Engineering of Porto University; –Technical Lisbon University; –University of California at Berkeley; –University of Victoria; –University FAF Munich; –Ecoles d'officiers de l'Armée de l'air (internship of two cadets)

4. Cooperative Control of UAVs Vehicles exchange information and commands in a network, changing their dependencies, states and mission roles to achieve a common goal; PlanPlan ExecuteExecute AssessAssess Cooperatively Communicate Sense Kill Process Source: [MICA Project]

5. Mixed Initiative Planning procedure and execution control must allow intervention by experienced human operators. –Essential experience and military insight of these operators cannot be reflected in mathematical models –It is impossible to design vehicle and team controllers that can respond satisfactorily to every possible contingency. In unforeseen situations, these controllers ask the human operators for direction. [Pravin et al.] Commander/ Operator Sensors Estimation Better status knowledge The Commander is an actuator Plant Measured status Better Info Decision Aids Better Decisions Courses of Action Better Performance Embedded Hierarchy Battlespace Decision & Control Source: [MICA Project]

6. Military and Civil Applications Research topic that has been attracting the attention of control, communications and computer science researchers; Possible applications with large societal impact are raising interest outside the scientific community; Military missions: –Combat; –Reconnaissance; –Surveillance; –Patrol; Civil missions: –Forest inspections; –Security; –Environmental applications;

7. Example: Strike Enemy Air Defenses (SEAD) mission MICA – Mixed-Initiative Control of Automata Teams (DARPA); Mission: Attack of the Blue force of UAV against Red's ground force of SAM sites and radars J. Borges de Sousa, T. Simsek e P. Varaiya, “Task planning and execution for UAV teams”, Proceedings of the Decision and Control Conference, Bahamas, 2004 sms14 Primary targets Blue base sms12 sls5sms13 sms15 sms17 sls6 sls8 sls7 sms11 Maneuvers: Follow_path Loitter Attack_jam

8. Execution control Example J. Borges de Sousa, T. Simsek e P. Varaiya, “Task planning and execution for UAV teams”, Proceedings of the Decision and Control Conference, Bahamas, 2004 sms14 Primary targets Blue base sms12 sls5sms13 sms15 sms17 sls6 sls8 sls7 sms11 Team A Team B Leg 6 Leg 1

9. Example J. Borges de Sousa, T. Simsek e P. Varaiya, “Task planning and execution for UAV teams”, Proceedings of the Decision and Control Conference, Bahamas, 2004 sms14 Primary targets Blue base sms12 sls5sms13 sms15 sms17 sls6 sls8 sls7 sms11 Leg 6 Attack segment Execution control Attack segment Leg 1

10. Example J. Borges de Sousa, T. Simsek e P. Varaiya, “Task planning and execution for UAV teams”, Proceedings of the Decision and Control Conference, Bahamas, 2004 sms14 Primary targets Blue base sms12 sls5sms13 sms15 sms17 sls6 sls8 sls7 sms11 Leg 6 Attack segment Safe path Leg 7 Execution control Attack segment Safe path Leg 1 Leg 2 Precedes

11. Execution control Example sms14 Primary targets Blue base sms12 sls5sms13 sms15 sms17 sls6 sls8 sls7 sms11 Leg 6Leg 7 Leg 8 Leg 1 Leg 2 Leg 3 Leg 4 Leg 5 Precedes Subtask 1 Subtask 2

12. Formalism for Allocation and Exchange vehicles within teams Matrix formalism: –Initial Allocation of vehicles to teams –Transition-vehicle incident matrix –Final Allocation of vehicles to teams The formalism could be used to design high level controllers in mixed-initiative environments Decision variables Team-transition incident matrix teams vehicles

13. Load-balancing algorithm Load-balance the number of vehicles within teams; Heterogeneous vehicles –Different fuel reserves; –Different number of weapons; –Different types of payloads; –… Performance Measure: Difference between the number of vehicles in the team and the number of vehicles initially planned for that team; Problem is solved as a Binary Integer Programming (BIP) optimization problem.

14. Example: Load-balancing Five teams with different necessities; Fuel constraints;

15. Actual UAV system configuration Ground Station Autopilot Avionics Sensors Payload Devices Neptus Command and Control Interface (FEUP) Servos Autopilot Avionics Sensors Payload Devices Servos

16. Advanced Configuration - Work in progress Autopilot manages low-level flight control PC-104 for higher-level tasks (vision processing, trajectory planning, coordinated between UAVs) Ground Station Aircraft Low level control and logging Payload High level Control and logging Autopilot Avionics Sensors Payload Devices Servos PC-104 Autopilot Avionics Sensors Payload Devices Servos PC-104

17. Vehicles ANTEX-X02 (AFA) Silver Fox (ACR) Lusitânia (FEUP) ANTEX-X03 (AFA) NOVA (AFA) Flying Wing (AFA)

18. Operation of UAVs and Cooperative control simulation

19. Conclusions and future work Cooperative control of UAVs is a research field with large margin of progression and with possible applications with societal impact (dull, dirty and dangerous missions); The intervention of the operator in the planning and execution control (mixed-initiative) is crucial in missions with large uncertainty, namely in military operations; ANTEX developments in a near short term: –Operation with several UAVs; –Track and follow structures (rivers, roads…) based on vision payloads; –Autonomous landing; Mid-term objective: –Operation with others types of unmanned vehicles (underwater, surface).

20. Questions? Thank you for your attention

21. Vehicles Characteristics Lusitânia UAV (FEUP) –Maximum Take Off Weight10 kg –Wing Span2.4 m –Payload5 kg –Endurance 0.75h –On board payload: wireless video camera Nova UAV (AFA) –Maximum Take Off Weight4 kg –Wing Span1.6 m –Payload0.5 kg –Endurance 0.75h Flying Wing UAV (AFA) –Maximum Take Off Weight3 kg –Wing Span1.6 m –Payload0.2 kg –Endurance 0.3h ANTEX-M X03 (AFA) –Wing Span6 m –Maximum Speed130 km/h –Stall Speed40 km/h –Maximum Take Off Weight100 kg –Payload30 kg –Engine22 hp –Endurance/Fuel Capacity0.3h/4L ANTEX-M X02 (AFA) –Maximum Take Off Weight10 kg –Wing Span2.4 m –Maximum Speed151 km/h –Payload4 kg –Endurance/Fuel Capacity0.3h/0.2L Silver Fox (ACR) –Maximum Take Off Weight 12.2 kg –Wing Span2.4 m –Maximum Speed 203 km/h –Payload 2.27 kg –Endurance/Fuel Capacity 10h/2.6L