1 Multi-agent Collaborative Flight Experiment Karl Hedrick UC Berkeley

2 CIRPAS, Camp Roberts, CA Operated by the Naval Post Graduate School

3 Collaborative UAV Flight Test  GOALS of the August, 2006 Experiment Thrust 1. Distributed collaboration with limited communications a.) multi-vehicle, multi-step task allocation b.) limited ground-air and air-air com c.) user task cancellation/reallocation d.) agent created tasks e.) task prioritization f.) no fly zone filter and re-planner g.) Simple human/UAV team interface Thrust 2. Vision-based river following. a.) Ability to identify and search for the desired structure (river). b.) Ability to accurately track the river once identified. c.) Ability to accurately map the boundaries of the river.

4 Collaboration Research Goals In General  Study distributed mechanisms for the collaboration of Unmanned Aerial Vehicles (UAV)  Generalize a large number of missions under one framework Surveillance/Mapping Border Patrol Search & Rescue Convoy Protection, etc.  Emphasis on Robustness rather than Optimization: Addition and Deletion of Tasks Addition and Removal/Failure of Agents Limited and/or Failed communication

5 C3UV Collaboration Software GOALS Transmit desired mission from user to agents Provide user with fused information from agents Decompose and assign tasks among agents in response to dynamic mission definition Accomplish tasks in an efficient and robust manner Agent in range of user Agent out of range User New tasks Cancel tasks Command station Mission state est. Mission state estimate

6 Communication Infrastructure User New tasks Cancel tasks Command station Piccolo Groundstation Piccolo Autopilot PC104 Piccolo Autopilot PC MHz radio 2.4 GHz ethernet

7 The Mission  User defines mission in terms of tasks  Philosophy “The user specifies what he or she would like accomplished. The system decides how to do so efficiently.”

8 Mission Commander Interface

9 Graphical User Interface  C:\Documents and Settings\Sonia\Desktop\CommanderSDK.exe C:\Documents and Settings\Sonia\Desktop\CommanderSDK.exe

10 Task types  Visit point Specify timing and priority Continuous visit is a circular orbit  Patrol segment Specify timing and priority Number of agents is either specified or calculated from desired scan time  Monitor area Specify timing, priority, and target velocity Number of agents is either specified or calculated from desired scan time  Guaranteed search Specify priority, target parameters, and whether to attempt if search cannot be guaranteed

11 Monitor Area Task Task details GoalDetect any velocity-bounded intruder that cannot leave a defined box TrajectoryUAV path depends on maximum speed of intruder; path becomes a “lawnmower” trajectory if maximum speed is zero

12 Guaranteed Search Task GoalDetect any intruder traveling from a known start point with a bounded velocity Simulation Result500 of 500 intruders detected in simulation Task details

13 System Features Distributed and dynamic task decomposition User specifies priority and timing options for tasks Agent-created sub-tasks UAVs avoid no fly zone and report when it effects the feasibility of a task Manage a heterogeneous fleet of Rascal and Bat aircraft with varying flight characteristics and sensors

14 Distributed Dynamic Task Decomposition Task decomposition and allocation is decided by agents in a decentralized manner Number of agents assigned to task depends on number of available agents and relative priorities of other tasks Task is divided into jobs for individual agents depending on the number of agents assigned Task can be re-divided in response to change in allocation Patrol area task for 3 agentsPatrol area task for 2 agents

15 Priorities and Timing Options Provide two priority types: Hard: Tasks with a high priority will be assigned before those with low priority Soft: Priority is one factor in an assignment function that also depends on others Provide three timing options: Once (collect a single image of a location) Continuous(monitor a location) Periodic(collect an image of a location every 30 minutes)

16 Avoiding a no fly zone: Pre-filter User interface does not allow creation of an impossible task No fly zone This task cannot be created due to conflict with no fly zone Some impossible tasks can be edited to resolve conflict with no fly zone No fly zone This task conflicts with the no fly zone. Should system resolve conflict? Yes No

17 Avoiding a no fly zone: Pre-filter User interface does not allow creation of an impossible task No fly zone This task cannot be created due to conflict with no fly zone Some impossible tasks can be edited to resolve conflict with no fly zone No fly zone Task edited automatically by user interface

18 Avoiding a no fly zone: active filter Task Controller No Fly Zone Filter Autopilot Desired autopilot command Safe autopilot command No fly zone Visit task

19 BLCC- Berkeley “Language” for Collaborative Control  Define the mission and communicate it to team members  Define the “state” of each agent  Define the mission “state”  Allow for faults  Allow for conflict resolution  Define the information to be communicated between agents.

20 Agents (UAVs)  Transition Logic: Governs transitions of tasks and subtasks  Communication: Deconflicts plans and synchronizes information between agents vs.  Planner(ex. path-planner): calculates cost, generates plan and chooses “todo”  Low-level Controller (ex. waypoint tracker)

21 Task-Point List  Every process and each agent communicates primarily through the task- point list  A task-point list exists for each task and is manipulated by each process to generate a desired mode/task/mission

22 Real-Time Task Allocation Algorithms Multiple agents/Multiple Tasks  Emphasis on real-time computation and robustness to communication limitations and agent failure.  Recent progress on optimal and sub-optimal algorithms

23 UC Berkeley C3UV Platforms  Modified Sig Rascal 110 model airframe  Bat-IV airframe, professionally built by MLB of Mountain View, CA

24 Task Allocation  Given n UAVs and m tasks, how do we assign tasks to UAVs? Assume that each task is simply a point to be visited, with some time spent at that point. Neglect UAV turn rate constraints – assume constant velocity For each UAV, let a tour be an ordered set of targets that it will visit Let the cost of tour be the total time required to complete. For a constant velocity UAV with no turn rate constraint, this time corresponds to distance.  Often this is posed as an instance of the multiple traveling salesman problem

25 Multiple Traveling Salesman  The Multiple Traveling Salesman Problems focuses on minimizing total cost. For n UAVs, with the cost of a tour for UAV j = T j  Our problem differs: we should focus on minimizing the max cost of any tour Given that we’re working with constant velocity UAVs, the cost in fuel of having a UAV circle is the same as having it do some work. For our problem, this corresponds to a minimum clock time problem. This problem is often referred to as the min-max Vehicle Routing Problem.

26 Min-Max Vehicle Routing  The min-max Vehicle Routing Problem is NP hard. There is no polynomial time solution.  We’d like to develop algorithms find near-optimal solutions to the min- max problem.  In practice, we’d like to build algorithms that are robust to communication losses; with perfect communication, we’d like these algorithms to achieve near optimal performance.

27 The Greedy Algorithm  In constructing a tour, let the UAV with the lowest cost function for its partial tour choose the next task.  This algorithm leads to balanced tours among UAVs: all UAVs perform tours of roughly equal cost. For the min-max VRP, optimal solutions will contain tours balanced to within the maximum distance between any two tasks.  This is a fast algorithm that creates balanced tours  Sub-optimal  2 questions: How well does this work? How do we implement this in a distributed system with limited communication?

28 Approximate Multi-step Distributed Min-Max Vehicle Routing Algorithm 0 Communication and 1 Computation

29 Example continued… 1 Communication and 1 Computation

30 Example continued… 1 Communication and 2 Computation

31 Example continued… 2 Communication and 2 Computation

32 Collaboration Architecture Mission State Estimate MSE A Internal State IS A Information Base CommunicationComputation BroadcastReceive Decision Functions Choose Redo Fault Complete MSE A MSE B Integrates MSE A and MSE B Retains most up-to-date information Agent A User Plan Task Execution

33 How we implement in software Piccolo Interface Datahub Payload Interface Controllers A2A Wireless Collaboration Sensors Telemetry Aircraft Commands High Level Telemetry User Commands Sensor Data Collaboration info AC Commands Mission Plan Mission Plan Task Execution Safe Flight Controller

34 Sig Rascal 110 airframe Balsa frame remote control aircraft kit with 110” wingspan Modifications: 32 cc gasoline engine with vibration isolation mounts Dual fuel tanks for 60 min flight time Carbon fiber reinforcement to support payload 26 lb takeoff weight Piccolo avionics system

35 Berkeley Air Force

36 PC104 stack and payload tray PC104 with 700 MHz Pentium III processor 2 GB flash memory (16 GB on vision plane) Bidirectional 1 Watt amplifier for b communication Vibration isolating suspension Wireless analog video transmitter

37 Infrastructure elements Piccolo avionics system Includes 900 MHz UHF radio and Piccolo Operator Interface Aircraft to aircraft 2.4 GHz ad-hoc wireless ethernet Custom user interfaces Mission Commander Interface and Process Monitor Interface

38 UC Berkeley UAV Platform: Autopilot, HIL Sim  Low Level Guidance and Control Provided by Cloud Cap Technology’s Piccolo Avionics Module and Corresponding Ground Station System  Capable of operating multiple vehicles. C3UV has successfully flown four vehicles simultaneously. Wireless Link Actuators and Sensors Signals Piccolo Avionics Ground Station Ground Station Computer Simulated States Hardware In Loop (HIL) Simulation Computer Manual Control Console

39 UC Berkeley UAV Platform: Communications Three Communications Channels:  Autopilot Avionics and Ground station: 900 MHz  Plane to Plane communication: 2.4 GHz  Video downlink: 1.2 GHz Ground Station

40 Piccolo avionics system Piccolo Ground station 900MHz UHF radio “Piccolo channel” Autopilot commands and telemetry Piccolo Operator Interface Send waypoint commands Monitor aircraft physical state Piccolo

41 User interfaces and Piccolo payload channel Piccolo Ground station 900MHz UHF radio “Piccolo channel” Autopilot commands and telemetry “Payload channel” Communication between PC104s and GUIs Piccolo Operator Interface Send waypoint commands Monitor aircraft physical state Piccolo PC104 Process Monitor Interface Activate or idle processes Monitor process states Mission Commander Interface Create tasks Monitor status of tasks and UAVs

42 Aircraft to aircraft communication Piccolo PC GHz b ad-hoc wireless ethernet Mission state estimates Task allocation data Piccolo PC104

43 Preview of collaboration experiment - Logistics Mission Commander Interface Video downlink from aircraft Piccolo Operator Interface Read-only display of Mission Commander Interface Process Monitor Interface Mission Commander Tent Status Monitor Tent Aircraft launch process 1.Aircraft takes off under manual control 2.Pilot gives control to Piccolo autopilot- aircraft flies on preloaded waypoint loop 3.Onboard software is activated using Process Monitor Interface ** At this point the aircraft is controlled only via the Mission Commander Interface. All other personnel serve only as monitors.**

44 Task and mission parameters Timing:once, continuous, periodic Priority:hard priorities = task allocation constraints soft priorities = weight on cost functions Number of agents:system estimates number of agents to meet desired scan time or user enters number of agents Task parameters Mission parameters No fly zone:UAV avoids no fly zone and cancels tasks that cannot be completed as a result Limited comm:Mission Commander Interface only communicates with UAVs within range set by user. Mission state information (ie, new tasks) propagates via aircraft to aircraft communication.

45 UC Berkeley UAV Platform: Camera Payload  Research Payload Consists of Multiple Cameras Fix-Mounted Cameras Capable of processing visual and near IR spectrum signals Onboard Processing or Downlink Fix-Mounted Camera

46 UC Berkeley UAV Platform: Computing  Onboard Computing 20 GB Hard Drive (optional: 2 GB Flash) 700 MHz Pentium III processor Consumer b card for inter-plane communications Video Capture card (1 or 2) Power Supply Vibration Isolation (critical for hard drive)

47 UC Berkeley UAV Platform: All Together Wing-Mounted Camera Onboard computer and wireless communication radio GPS Ant. Analog Video TX’s Air-Ground UHF Ant.

48 Single Unmanned Vehicle Hierarchical Architecture World Actuation: PiccoloSensing: Piccolo, Camera, IR, etc Hardware / Software / Simulation Interface Collision Avoidance and/or Path Planning Desired Turn Rate & Airspeed

49 Single Unmanned Vehicle Hierarchical Architecture World Actuation: PiccoloSensing: Piccolo, Camera, IR, etc Hardware / Software / Simulation Interface Collision Avoidance and/or Path Planning Desired Position, Orientation Task Specific Controller ( Tasker ) Example: Target Tracking Desired Turn Rate & Airspeed

50 Single Unmanned Vehicle Hierarchical Architecture World Actuation: PiccoloSensing: Piccolo, Camera, IR, etc Hardware / Software / Simulation Interface Collision Avoidance and/or Path Planning Desired Position, Orientation Task Specific Controller ( Tasker ) Example: Target Tracking Task Allocation and/or Scheduling Desired Task Desired Turn Rate & Airspeed

51 Single Unmanned Vehicle Hierarchical Architecture World Actuation: PiccoloSensing: Piccolo, Camera, IR, etc Hardware / Software / Simulation Interface Collision Avoidance and/or Path Planning Desired Position, Orientation Task Specific Controller ( Tasker ) Example: Target Tracking Desired Turn Rate & Airspeed Task Allocation and/or Scheduling Desired Task Vehicle Position and Obstacle Map Estimation Observations

52 Single Unmanned Vehicle Hierarchical Architecture World Actuation: PiccoloSensing: Piccolo, Camera, IR, etc Hardware / Software / Simulation Interface Collision Avoidance and/or Path Planning Desired Position, Orientation Task Specific Controller ( Tasker ) Example: Target Tracking Desired Turn Rate & Airspeed Task Allocation and/or Scheduling Desired Task Vehicle State and Obstacle Map Estimation Controller Specific Estimation Example: Target State Estimation Observations

53 Single Unmanned Vehicle Hierarchical Architecture World Actuation: PiccoloSensing: Piccolo, Camera, IR, etc Hardware / Software / Simulation Interface Collision Avoidance and/or Path Planning Desired Position, Orientation Task Specific Controller ( Tasker ) Example: Target Tracking Desired Turn Rate & Airspeed Task Allocation and/or Scheduling Desired Task Vehicle State and Obstacle Map Estimation Controller Specific Estimation Example: Target State Estimation Mission State Estimation Observations

54 Collaborative Unmanned Vehicle Hierarchical Architecture World Actuation: PiccoloSensing: Piccolo, Camera, IR, etc Hardware / Software / Simulation Interface Collision Avoidance and/or Path Planning Desired Position, Orientation Task Specific Controller ( Tasker ) Example: Target Tracking Desired Turn Rate & Airspeed Task Allocation and/or Scheduling Desired Task Vehicle State and Obstacle Map Estimation Controller Specific Estimation Example: Target State Estimation Mission State Estimation Observations Wireless Communication State Estimates

55 User Agent World Actuation Vehicle State and Obstacle Map Estimation Controller Specific Estimation Example: Target State Estimation Mission State Estimation Observations Wireless Communication State Estimates Commander ( GUI ) : CSL Generation Displayed for the User CSL Interpreter CSL InputHuman Control Note: CSL Input Includes prior knowledge such as target or obstacle map priors Centralized Computation

56 August, 2006 Experiment World Actuation: PiccoloSensing: Piccolo Hardware / Software / Simulation Interface No Fly Zone Desired Position, Orientation Task Specific Controller ( Tasker ) Example: Guaranteed Search Desired Turn Rate & Airspeed Task Allocation and/or Scheduling Desired Task Mission State Estimation Observations Wireless Communication State Estimates

57 New Bat 4 Airframe  Pusher design. Improved Payload, Duration, and Logistics  Weights and payload 45 lbs dry airframe 12 lbs fuel (2 gal) 25 lbs payload  Flight envelope Vmin 35 kts (40mph, 18 m/s) Vmax= 70 kts (80mph, 36 m/s) Turn rate <= 15 deg/sec (.26 rad/sec) deg bank)  Duration Nominal: 8 hrs Batt. Only: 2-3 hrs (with payload shut down)  Onboard power generation (150W)

58 Near Infrared Camera  Sensitivity in.7 to 1 micron range  An extension of ‘normal’ imaging  Primarily reflectance-based Relative reflectance of objects in different parts of spectrum is different May aid in discrimination

59 August ONR Demonstration

60 LESSONS LEARNED  Initially Air-Air com was so bad that mission performance was unacceptable.  Improved amplifier/antenna combined with robust architecture allowed us to accomplish complex missions successfully.  Ground-Air link needs to have a higher bandwidth for human/UAV interaction.  Required computation including task allocation and vision processing can be done on a Pentium III.  3 UAV’s with 4-5 tasks is already too complex for humans without autonomy software.  Complex interaction between task allocation and priority system needs further analysis.

61 The End