1 Cooperative and Noncooperative Operations of Swarms Hoam Chung, David Shim, Mike Eklund, Shankar Sastry University of California, Berkeley

SWARMS 2 Cooperative Operations of Swarms Heterogeneous formation flight of MD500s and UH-60s Various helicopter formations are now being used in many applications Some level of automation during formation flight can reduce pilot stress and fatigue Few research results on autonomous helicopter formation exist due to helicopter’s complicated dynamic properties, and technical difficulties

SWARMS 3 Mesh Controller Mesh Controller tasks: Obtain the leader and 2 neighboring helicopters’ current positions Compute mesh stable trajectories based on the acquired position information and send commands to the navigation computer Flight Computer Mesh Controller RS-232 Wireless Token Ring Neighbor 1 Neighbor 2 Leader On UAV

SWARMS Experiment

SWARMS Experiment Animation by A. Pant and X. Xiao Without leader info With leader info

SWARMS 6 Mesh Stable Controllers are OK, but… The use of leader information improves the performance of the autonomous formation flight For a heterogeneous mesh, an extension of mesh stability theory should be considered “Mesh Stability” does not mean the “Safety” It’s a starting point for autonomous formation flight

SWARMS 7 Model Predictive Control Computes control inputs using real-time optimization Shows better performance than non- predictive controls Can consider various safety constraints in on-line manner Easily accommodates adaptive disturbance rejection algorithms

SWARMS 8 Model Predictive Control Compute control inputs minimizing gap errors considering helicopter dynamics at every sampling time optimization can deal with various constraints Positions of neighboring vehicles Information of formation velocities desired gaps Structure of MPC Control inputs Weather conditions/Mission characteristics

SWARMS 9 Simulation Scenario 1 3DOF Point mass model Homogeneous formation Echelon right (45 deg. off lead) Forward flight at 67.5 mi/h Disturbance on 2 nd helicopter No safety constraints, no explicit disturbance rejection t n Heli0 Heli1 Heli2 Heli3 * Formation from FM Attack Helicopter Operations, Headquarters, Dept. of the Army

SWARMS 10 Animation Animation generated by MATLAB

SWARMS 11 Simulation Mesh stability is achieved without any explicit disturbance rejection algorithm

SWARMS 12 Bi-directional Information Flow For safer autonomous formation, the communication between neighbors should be bi-directional In case of mesh stability concept, it’s difficult to deal with bi- directional information What will happen if directions of disturbances are reversed? Information flow

SWARMS 13 Bi-directional Information Flow j  In case of MPC, simple redefinition of error signal can deal with bi-directional information flow  For example, we can redefine the error vector of j th helicopter so that - Keep the center between j-1 and j+1 in tangential direction - Keep the desired gap in normal direction j-1 j+1 t n  This flexibility of MPC allows various formations in 3D space with enhanced safety

SWARMS 14 Forming a Formation Adding vehicles one by one 1.Establish communication with vehicle A 2.Acquire variables about existing formation from vehicle A 3.Compute a merging trajectory and track it 4.Finish merging procedure if the gap error is within a certain bound 5.Engage formation controller Merging procedures on the vehicle B: A B

SWARMS 15 Forming a Formation Adding vehicles group by group 1.Vehicle b establishes communication with vehicle a in A 2.Vehicle b acquires variables about leading formation from vehicle a 3.Compute merging trajectory 4.Propagate acquired variables and computed trajectory through B 5.Track the merging trajectory 6.Finish merging procedure Merging procedures on the group B: A B + a b a b

SWARMS 16 Terminating a Formation Terminating a formation one by one 1.Compute a trajectory to get more gap from the existing formation 2.Notify termination schedule to vehicle A 3.Track the computed trajectory 4.Send “Separation Completed” to vehicle A and close the communication channel 5.Disengage formation controller and give control back to pilots Termination procedures on the vehicle B: A B

SWARMS 17 Terminating a Formation Terminating a formation group by group 1.Compute a trajectory to get more gap from the leading formation 2.Propagate the computed plan to followers 3.Notify termination schedule to vehicle a 4.Track the computed trajectory 5.Send “Separation Completed” to vehicle a and close the communication channel between a and b Termination procedures on b: A B a b a b

SWARMS 18 Modifying a Formation in the Air Modification of a formation MPC is basically a tracking controller By manipulating local formation variables(gap info), reconfiguration of a formation without reorganization can be easily achieved

SWARMS 19 Simulation Scenario 3 3DOF Point mass model Heterogeneous formation 3D Vee formation (45 deg. off lead, 5m gaps in n, t, and z) Forward flight at 67.5 mi/h, 5m(about 1.7 rotor radius) spacing Disturbances on the leader and the last follower in right wing No safety constraints, no explicit disturbance rejection t n Mass Ratio Heli0100% Heli1200% Heli2300% Heli3100% Heli4300% Heli5100% Heli6200% Heli0 Heli1 Heli2 Heli3 Heli6 Heli5 Heli4

SWARMS 20 Simulation of formation split and rejoin

SWARMS 21 Formation Rejoining Consider a situation that a vehicle is approaching to the existing 3D Vee formation for joining 1 2 Safe region

SWARMS 22 Formation Rejoining Objectives for a perfect formation rejoining A joining vehicle is positioned at predefined location in the formation When it finishes the procedure, its velocities and heading should be close enough with those of the entire formation During the procedure, the joining vehicle should remain in a safe region The motion of the future neighbor acts like a disturbance during the joining procedure For the vehicle in the formation, the first priority is maintaining the formation Disturbances deteriorate ideal navigation conditions always exist

SWARMS 23 Formation Rejoining The formation joining problem can be regarded as a differential gaming under input/state constraints Following question should be answered: Does RHC scheme guarantee reachability under disturbances? If so, how close is the reachable set rendered by RHC to that by infinite-horizon problem?

SWARMS 24 Finite-horizon Differential Game

SWARMS 25 Finite-horizon Differential Game The reachable set by the solution of FHODG problem is identical with that of a modified infinite-horizon problem As becomes small, the reachable set of RHC approaches to that of infinite-horizon solution with

SWARMS 26 Finite-horizon Differential Game This lemma plays important role in designing a receding horizon controller satisfying the condition

SWARMS 27 Finite-horizon Differential Game The reachable set can be enlarged by introducing longer prediction horizon These theorems and lemma tell us that, if the FHODG is feasible with some prediction length L, then it guarantees a successful formation rejoining

SWARMS 28 Works in Progress A RHC scheme will be designed for 3D nonlinear kinematics plus linear dynamics model Various numerical methods are now being investigated Continuation method – Ravio et al. Piecewise linear approximation and SQP – Fabien Lagrange multipliers method – Sutton and Bitmead For reducing computational burdens, the performance of open-loop and Stackelberg solutions under RHC scheme will be evaluated The algorithm will be implemented and tested on BEAR hardware-in-the-loop systems

SWARMS 29 Collision Avoidance using MPC Five helicopters are given destination points. The shortest (optimal) trajectory will lead to a collision. Each vehicle can detect other vehicles position only within the sensing/communication region. Can each vehicle fly safely and optimally? Unsafe Desired TrajectoryResolved by NMPTC with Collision Avoidance

SWARMS 30 Collision Avoidance using MPC Two UAVs are intentionally set on a head-on crash course Model-predictive control-based trajectory planner computes safe trajectories with sufficient clearance in real time Each vehicle’s current coordinate is used for MPC at each computation May 2003

SWARMS 31 Collision Avoidance using MPC

SWARMS 32 Obstacle Avoidance System Dynamic path planning: real-time path generation using model predictive control Sensing: onboard 3D laser scanner or preprogrammed obstacle maps Experiment system: Berkeley UAV architecture implemented on Yamaha industrial helicopter platform with 3D laser scanner

SWARMS 33 Urban Flight Experiment 10’ X 10’ Easy-up Canopy 6 canopies to simulate urban environment Secured by stakes at four corners Resistant to wind gust of rotor downwash Sufficient distances each other for helicopters to fly through Original path Adjusted path by MPC Vehicle Launching Pt. Ground Station Obstacles Richmond Field Station, UC Berkeley, Richmond, California

SWARMS 34 Urban Navigation Experiment

35 Non Cooperative Actions of Swarms: David Shim, Jongho Lee, Mike Eklund, Jonathan Sprinkle, Shankar Sastry

SWARMS 36 Aerial Pursuit-Evasion in MPC framework Pursuer wants to position itself in a good position to “shoot down” the evader, e.g., follow the target’s tail and align its heading with the relative position vector, X E -X P Evader wants to shake off the pursuer, e.g., get out of the hotspot Pursuer and evader avoid colliding into each other within a closed 3-D space State variables such as roll and pitch angles are constrained

SWARMS 37 Pursuer and Evader in a closed 3-D space with the additional cost Aerial Pursuit-Evasion in MPC framework Cost function also includes collision avoidance between aircraft and other obstacles including terrain Illustration by Mike Eklund and Jonathan Sprinkle

SWARMS 38 ANIMATION 2004

SWARMS 39 Multiplayer PEGs: Proposed Solution A close analogy is football: Multi player Initial (global) strategies well defined Limited (local) coordination after the snap What can we learn? How can we apply this? How far does the analogy go? Back

SWARMS 40 Multiplayer PEGs Preseason (Off-line precomputed strategy) Play book: Evaluate strategies and configurations that will maximize chance of success based on best estimate of other team’s tactics Practice and preseason games: Test playbook and find problems Game time (On-line adaptive strategy) Choose play based on best knowledge and experience Line up (in best detection configuration, not necessarily static) Execute the play Active and reactive actions (respond to detected evader) Local communication Adapt to evolving behavior Learn from experience, repeat as necessary (Learning by Doing) Back