1. 1 Cooperative Control of Distributed Autonomous Vehicles in Adversarial Environments 2.5 Year MURI Research Review November 18, 2003

2. 2 Team vs Team Siege Combat decisions Communication decisions Real-time computations Multivehicle (re)grouping Trajectory execution As well as… – Above: Resource allocation – Below: Servoloops – Parallel: Human intervention

3. 3 Lots of Issues & Approaches Hierarchy Distributed processing Distributed information Constrained communication Uncertain evolution Adversarial elements Multi-tier imposition Aesthetic aggregation Task decomposition Fixed structure imposition Discrete/continuous interaction Verification Stochastic estimation Hypothesis falsification Minimax allocation Game formulation Combinatorial optimization Embedded simulation Multi-hop communication Receding horizon implementation

4. 4 Core MURI Research Our approach: Extract & distill essential elements with well formulated subproblems. Develop core theory & understand limitations/trade-offs. Develop supporting computational algorithms. Illustrate & motivate new directions in test-bed examples. Recognize traceable and transportable implications.

5. 5 Dimensions of Cooperative Control Distributed control & computation The defining feature of cooperative control problems. Adversarial Interactions Uncertain Evolution Complexity Management

6. 6 Evolution of Dimensions Year 2 Distributed control & computation (Virtual) Hierarchy Adversary & Uncertainty Finite-state representations Proposal & Year 1 Scalability, modeling & reduction High level planning Low level execution Communications Year 2.5 Distributed control & computation Adversarial Interactions Uncertain Evolution Complexity Management

7. 7 Today’s Agenda Distributed Control & Computation Murray, Caltech & Klavins, UW Adversarial Interactions Speyer, UCLA Uncertain Evolution Dahleh, MIT Complexity Management D’Andrea, Cornell Discuss on-going work across universities in context of dimension. Explore multiple facets of research challenge. Recognize multiple dimensionality.

8. 8 Cross Dimensional Threads Explore multiple facets of research challenge. Recognize multiple dimensionality. Enemy Models Coordinating Actions Constructive Algorithms Roboflag Drill

9. 9 RoboFlag Drill problem with semi-intelligent targets Encode vehicle dynamics, obstacle avoidance, target intelligence, and group objective as a mixed integer linear program (MILP). Solving the MILP gives the optimal group strategy. Enemy Model 1/5 MILP Methods for Multi-Vehicle Systems

10. 10 Enemy Model 2/5 Linear-Programming-Based Multi-vehicle Path Planning with Adversaries Objective: Minimize the number of adversaries that enter a protected area. Explore the utility of Linear Programming for trajectory planning. Represent Enemy as “probabilistic diffusion” Potential Advantages: Reduce complexity with LP’s (versus mixed integer LP’s) Allow 2-sided optimization (versus “scripted” adversaries)

11. 11 Enemy Model 3/5 Probability Map of the Environment with Moving Opponents PDF of Each Opponent Map Building Path Planning - Find a sequence of cells connecting the origin and the destination using Dijkstra algorithm - Plan a path considering the centers of the sequence of cells as waypoints

12. 12 Enemy Model 4/5 Linear Quadratic Gaussian (LQG) Differential Games with Different Information Patterns Solution Evader Filter Solution has following structure.  A reduced-order dynamic filter in a subspace orthogonal to the pursuer’s control input.  Remaining estimated states are constructed via algebraic equations Control strategies appear similar to deterministic LQG Game Solution is linear and finite dimensional.  No linearity assumption on the strategies is made. Problem Two player zero-sum LQG pursuit-evasion game  Linear dynamics and Gaussian process and sensor noise  Quadratic cost function: Q(u,v)  Evader u makes noisy partial measurements of the state.  Pursuer v knows the state perfectly. Pursuer (Perfect measurement of Engagement states.) Evader (Partial measurement Of Engagement states.) State Space To describe Dynamic Motion of the Adversaries

13. 13 Conventional Methods: Agents “chase” other agent behaviors Alternative: Distributed feedback stabilization Enemy Model 5/5 Distributed Convergence to Nash Equilibria Can individual agents reach strategic equilibrium without declaration of their intentions? Utility: Strategic robustness …Adaptation vs fragile planning Game Theory Literature: It can’t be done (40yrs) Standard “Counterexample”: Anti-coordination Game P1 wants to deviate from P2 P2 wants to deviate from P3 P3 wants to deviate from P1 Each player only has 2 moves…all can’t be satisfied P1 P2P3

14. 14 Coordinating Actions 1/6 Consensus in Networks with Mobile Agents and Switching Topology Approach – Design cooperative control protocols for networks of mobile agents and analyze their convergence, performance, and robustness properties. Accomplishments – Theory for agreement protocols in networks of mobile agents with switching communications topology – Analysis of speed of reaching consensus in a group of vehicles/agents based on second eigenvalue of graph Laplacian Formation switching using balanced graphs Attitude Alignment for Large Collections of Vehicles

15. 15 Objective 1 - Objective 1 - Want to design : Objective 2 - Objective 2 - Given r q, the rate of q k, f ind suitable D and f such that: that parallels the classical Kalman filtering performance analysis, where: Given: Coordinating Actions 2/6 Mode Estimation of Switching Linear Systems

16. 16 Coordinating Actions 3/6 Observation of CCL-like Programs Problem : Determine state of communications protocol used by a group of robots given their physical movements. Assumptions : Protocol and motion control are described in CCL like language. Results : Definitions of observability, etc. for CCL programs Construction and analysis of an observer that converges when the system is "weakly" observable Construction of an efficient observer for Roboflag drill in particular.

17. 17 Coordinating Actions 4/6 Adaptive Languages in Uncertain Environments Elements: – Symbol grounding – Language learning – Language evolution

18. 18 Coordinating Actions 5/6 Adaptive Models in Interactive Markov Chains Start with two dynamically coupled systems with centralized objective. – Each subsystem makes simplified model of other. – Each subsystem designs local optimal controller based on modified cost. – After simulation/experience, subsystems revise models. Will it converge? What is performance? Anticipate FP proof Scheme CentralizedCoordinatedDecentralized Miss/Thousand 2159173

19. 19 Coordinating Actions 6/6 Communications under Bandwidth Limitations Central Command Wireless digital link Generates: references and control signals. Has access to information about the vehicles and adversarial environment. Strategy and path planning

20. 20 Constructive Algorithms 1/3 Flocking with Obstacle Avoidance  Stability of flocks is formalized.  A flock contains a, b, and g agents with specific tasks: a: maintains a distance d from an a agent. b: repels an a agent and exists if a exists. g: behaves like an a agent but is fixed.  Split/Rejoin and Squeezing maneuvers w/ local information.  Consensus under switching topology addressed for directed graphs.

21. 21 Constructive Algorithms 2/3 Decomposition Methods Decomposition We use trajectory generation and obstacle avoidance primitives to pose cooperative planning problems such as the target assignment problem (ex. RoboFlag Drill). Problems are effectively reduced to combinatorial optimization problems Complexity We show the target assignment problem is NP-hard. Greedy Branch & Bound algorithm Form a search tree and explore using upper and lower bounds to prune branches. Upper bound is computed using greedy cost to go algorithm thus you can stop at any point in your search and use the best feasible solution found from the greedy algorithm. Multi-level MPC algorithm For semi-intelligent targets. Run each level of the hierarchy in an MPC framework at rate governed by the complexity of the level. R TG > R OA > R BB Branch & Bound Steps J ub J opt

22. 22 P(k 1,k 2 ) := { initializers guard 1 :rule 1 guard 2 :rule 2... } S(k 1,k 2 ):=P(k 1,k 2 )+C(k 1 +1) sharing y,u "soup" of guarded commands composition = union non-shared variables remain local to component programs Constructive Algorithms 3/3 CCL: Computation and Control Language CCL Interpreter Formal programming language for control and computation. Interfaces with libraries in other languages. Automated Verification CCL encoded in the Isabelle theorem prover; basic specs verified semi-automatically. Investigating various model checking tools. Formal Results Formal semantics in transition systems and temporal logic. RoboFlag drill formalized and basic algorithms verified. CCL Protocol for Decentralized Target Allocation

23. 23 Roboflag Drill MILP Planning. Hierarchical decomposition. Model reduction. LP planning. Adaptive representations. CCL protocols. CCL observers.

24. 24 Dimensions as Specific Core Challenges Distributed Control & Communication Adversarial Interaction Uncertain EvolutionComplexity Management Decision makers (DM’s) are self- interested. Do not have access to other DM’s intentions. Do not have access to other DM’s information. Static utility functions for dynamic underlying systems. P1 P2P3

25. 25 Proposal: Expected Insights How to address scalability through modeling & decomposition. How to address computational complexity in hierarchical designs. How to develop reliable multi-layered cooperative strategies. How to counter adversarial actions with constrained communications. How to integrate local optimizations for collective performance. How to synchronize cooperating elements through modeling and ID. How to exploit neurological models to design cooperating elements. How to achieve reliable communications in hierarchical structures. How to derive adaptive languages for autonomous operations.

26. 26 Today’s Agenda Distributed Control & Computation Murray, Caltech & Klavins, UW Adversarial Interactions Speyer, UCLA Uncertain Evolution Dahleh, MIT Complexity Management D’Andrea, Cornell Discuss on-going work across universities in context of dimension. Explore multiple facets of research challenge. Recognize multiple dimensionality.