MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Ricardo Bencatel ricardo.bencatel@fe.up.pt Pedro Almeida pinto.almeida@fe.up.pt Gil Manuel Gonçalves gil@fe.up.pt João Borges de Sousa jtasso@fe.up.pt

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Background Networked vehicles and systems AsasF Goals Requirements System breakdown structure Status Organization and control Vehicle control Multi-vehicle control Overview

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Background

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Motivation Forest fires in Portugal Oceanographic area Trend in military Less risk for the pilot Background Applications Rescue, civil protection & commercial applications Flying eye for rescue services Hazard inspections Surveillance Military & governmental applications Short range reconnaissance Search mission Critical area monitoring Fire prevention

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS AirCargo challenge Background

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS AirCargo challenge Background

Mixed Initiative Control of Automa-teams MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Mixed Initiative Control of Automa-teams Process Communicate Background Sense Execute To reiterate, MICA is all about controlling the cooperative behavior of unmanned vehicles operating as teams to prosecute multi-faceted tasks. Atackl Plan Cooperatively Assess Atack Process Sense Cooperative Battle Management of Teamed UAVs

Underwater Systems and Technology Lab Porto University Mission Design and deployment of innovative solutions for oceanographic and environmental applications Vehicles Autonomous submarines Remotely operated submarines Unmanned air vehicles Technologies Systems engineering Navigation and control Acoustic networks Networked control systems Power/computer systems Applications Monitoring sea outfalls Coastal oceanography Underwater archaeology Inspection and intervention Courtesy of Michael Incze, NUWC

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Cooperation United States University of California at Berkeley, USA C3UV Naval Postgraduate School, Monterey, CA, USA California Institute of Technology, CA, USA Portugal Academia da Força Aérea Portuguesa Instituto Superior de Engenharia do Porto Background

Cooperative UAV Program (2005-) MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Cooperative UAV Program (2005-) Low cost UAVs RC frame Shares electronics and control with AUVs Common CCC infrastructure Composite materials Long range Coordinated operation Formation flying Other cooperative missions Sensor networks and seagoing Vehicles Background

Networked vehicles and systems MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Networked vehicles and systems

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Networked operations Networked Control UAV UAV Surface buoy Autonomous surface vehicle Control station Navigation beacon Drifting sensors Oceanographic sensors AUV Control station Moored sensors AUV Moored sensors AUV

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Common thrusts on coordinated systems Development of user-friendly operator interface; To allow a single-user to control a whole fleet of vehicles Research on distributed task assignment Mixed-initiative (human operator in the control loop) environments Networked Control

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS AsasF

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Goals Integrate Piccolo with the Neptus framework for planning and control of autonomous vehicles Extend the Neptus framework to include UAV models and operational interfaces Evaluate mixed initiative operations AsasF

High-level Coordinated Control Veículos aéreos autónomos Low-level AsasF Take model airplane parameters Mount autopilot in each aircraft Perform tests carry out an autonomous flight demonstration High-level Coordinated Control Control structure modelled through the framework of dynamic networks of hybrid automata Tool to achieve coordinated control of multiple vehicles: Neptus framework.

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Requirements Minimum payload of 4Kg Modular, flexible, robust Low cost COTS components Easy control and operation Autonomous and remotely operated operation Air data (Altitude, Air pressure, airspeed, GPS, RPMs) Video acquisition and real time transmission Operational console capable of specifying new waypoints for coordinated flight AsasF

System Breakdown Structure MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS AsasF System Breakdown Structure

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Vehicles Large trainer Low cost, easy to fix or replace Stable platform Room and payload capacity UAV developed at Porto University AsasF

Veículos aéreos autónomos Brutus Super Telemaster Brutus v2 AsasF

Veículos aéreos autónomos AsasF

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Hardware AsasF Handset FF9 OS91FX engine Servos Electronic equipment (camera, Tx/Rx) Piccolo avionics Ground Station

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Status AsasF Building of platform Telemaster (commercial aircraft model) and final adjustments to Brutus v1 Brutus v2 development and building Study of Piccolo’s controller and SDK communications Study and implementation of HWIL simulation

Organization and control MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Organization and control

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Neptus at a glance Distributed command and control infrastructure for multi-vehicle systems with mixed-initiative interactions Supports mission life cycle Planning Simulation Supervision Review and analysis Data dissemination Organization and control

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS UAV mission planning Organization and control

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Organization and control

Piccolo Avionics System MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Piccolo Avionics System Miniature flight control system Radio link between ground station and avionics for command and parameter upload, telemetry, and DGPS Single ground station can control multiple units HIL simulation mode Organization and control Ground Station Piccolo Avionics Sensors Servos AIRCRAFT

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Organization and control

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Organization and control

HardWare-In-the-Loop Veículos aéreos autónomos HardWare-In-the-Loop Airplane Simulator & FlightGear Piccolos Organization and control CAN interface Operator Interface Computer Handset GroundStation

Platform configuration MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Platform configuration Piccolo manages low-level flight control PC-104 for higher-level tasks (vision processing, trajectory planning,…) Independent radio links Organization and control Sensors Piccolo Avionics Servos Ground Station Payload Devices PC-104 AIRCRAFT Similar to Berkeley’s architecture Aircraft Low level control / Logging Payload High level Control / Logging

Single vehicle controllers MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Single vehicle controllers UAV Platform Maneuver controller Vehicle supervisor State Messages Platform commands Abort Configurations Maneuver commands Configuration commands Execution events Mission supervisor External controller Vehicle maneuvers Individual Mission Team/task Link Control concepts Follow_path Loitter Attack_jam Organization and control Com mais detalhe (e só indicar isto)

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Prof. Fernando Lobo Pereira Prof. António Torres Marques USTL – Underwater Systems and Technology Laboratory Support NAAM – Núcleo de Aeronáutica, Aeroespacial e Modelismo PESC projects ISR – Instituto de Sistemas e Robótica INEGI – Instituto de Engenharia Mecânica e Gestão Industrial

MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Website: whale.fe.up.pt/asasf MULTI-UAV INTEGRATION FOR COORDINATED MISSIONS Ricardo Bencatel ricardo.bencatel@fe.up.pt Pedro Almeida pinto.almeida@fe.up.pt Gil Manuel Gonçalves gil@fe.up.pt João Borges de Sousa jtasso@fe.up.pt