Cooperative Control of Distributed Autonomous Vehicles in Adversarial Environments AFOSR 2001 MURI Kickoff Caltech/Cornell/MIT/UCLA May 14, 2001

2 Agenda 8:00-8:30Continental Breakfast & Registration 8:30-8:45"Opening Remarks"Jacobs (AFOSR) 8:45-10:15"Overview"Shamma (UCLA) 10:15-10:30Break 10:30-12:00"Modeling & Planning with Robust Hybrid Automata"Feron (MIT) "Two-Tiered Distributed Control"D'Andrea (Cornell) "Real-time Trajectory Generation"Murray (Caltech) "Distributed Decision Making"Speyer (UCLA) 12:00-1:30Lunch 1:30-3:00"Logical Programming Environments"Hickey (Caltech) "Computational Complexity Management"Selman/Gomes (Cornell) "Distributed & Adaptive Communication Protocols"Pottie/Taylor (UCLA) "Neurological Modeling & Cooperation"Dahleh/Massaquoi (MIT) 3:00-3:15Break 3:15-4:00"Cornell Experimental Testbed"D'Andrea (Cornell) "UCLA Experimental Testbed"Chichka (UCLA) 4:00-5:00Open Discussion

3 Vision: Networks of (semi) Autonomous Vehicles Large scale operations Fault tolerance through redundant deployment Cost effectiveness through simple/specialized components “Complex collective behavior through simple local behavior”

4 Challenges Local information/decision making Constrained communications Large scale of operations Uncertain dynamic environment Hostile adversarial presence

5 Approach : Multidisciplinary: Multiscale Modeling + Hierarchical Planning + Logical Programming Environments + Complexity Management + Distributed Protocols + Language Adaptation + Biological Modeling Analytical & Constructive : Experimental: Case Study Simulations + Hybrid Hardware Realization

6 Hierarchical Formulation Fundamental challenge = lack of centralized decisions – Analytical difficulties – Computational complexity – Meaningful & tractable? Relevance of hierarchy: – Aggregation, distributed computation, reduced communication, etc. – Supporting social/biological evidence – Self similarity

7 Research Focus Scalability, modeling & reduction: Representation of distributed low level components in a manner amenable to high level planning with reduced complexity. High level planning: Development of analytical methods and computational algorithms for coordinated team strategies. Low level planning: Realization of team strategies through low level strategies and optimization. Communications: Investigation of communications issues within and among levels.

8 Case Studies Motivate & Illustrate Research Multi-vehicle tasking with obstacle and mutual avoidance (one-sided) Autonomous suppression of enemy defenses (two- sided)

9 Experimental Testbeds Two “hybrid facilities”: – HiFi virtual vehicles with hardware communications – Simple hardware vehicles with virtual communications/distribution Internal & open-platform investigations

10 Expected Outcomes Theory: Analytical understanding of achievable performance of distributed cooperative control systems. Computation: Algorithms & software tools for control design, testing, evaluation, and rapid prototyping. Experimentation: Application to simulated and hardware testbeds. Education: Multidisciplinary program with increased DoD visibility.

11 Expected Insights How to address scalability through modeling & decomposition. How to address computational complexity in hierarchical designs. How to develop reliable multi-layered cooperative strategies. How to counter adversarial actions with constrained communications. How to integrate local optimizations for collective performance. How to synchronize cooperating elements through modeling and ID. How to exploit neurological models to design cooperating elements. How to achieve reliable communications in hierarchical structures. How to derive adaptive languages for autonomous operations.

12 Team Strengths: People Caltech: Jason Hickey (CS) Richard Murray (CDS/ME) Cornell: Raffaello D’Andrea (MAE) Bart Selman (CS) Carla Gomes (CS) MIT: Munther Dahleh (EE) Eric Feron (AE) Steve Massaquoi (EE/Neuro) Brian Williams (AE) UCLA: David Chichka (AE) Greg Pottie (EE) Jeff Shamma (MAE) Jason Speyer (MAE) Charles Taylor (Bio)

13 Team Strengths: Experience Caltech: DARPA SEC DURIP Cornell: RoboCup AFRL/Cornell Intelligent Information Systems Institute MIT: DARPA JFACC NSF Natural motor control ONR Distributed cooperative languages UCLA: DARPA JFACC ONR Minuteman NSF Learning in natural & artificial systems

14 Agenda 8:00-8:30Continental Breakfast & Registration 8:30-8:45"Opening Remarks"Jacobs (AFOSR) 8:45-10:15"Overview"Shamma (UCLA) 10:15-10:30Break 10:30-12:00"Modeling & Planning with Robust Hybrid Automata"Feron (MIT) "Two-Tiered Distributed Control"D'Andrea (Cornell) "Real-time Trajectory Generation"Murray (Caltech) "Distributed Decision Making"Speyer (UCLA) 12:00-1:30Lunch 1:30-3:00"Logical Programming Environments"Hickey (Caltech) "Computational Complexity Management"Selman/Gomes (Cornell) "Distributed & Adaptive Communication Protocols"Pottie/Taylor (UCLA) "Neurological Modeling & Cooperation"Dahleh/Massaquoi (MIT) 3:00-3:15Break 3:15-4:00"Cornell Experimental Testbed"D'Andrea (Cornell) "UCLA Experimental Testbed"Chichka (UCLA) 4:00-5:00Open Discussion

15 Project Management Monthly seminar series Twice annual research meetings Graduate student exchanges

16 Technical Approach Scalability, Modeling & Reduction: Scalability, Modeling & Reduction: Representation of distributed low level components in a manner amenable to high level planning with reduced complexity. High level planning Low level planning Communications

17 Scalability, Modeling & Reduction Objective: Reduction of complexity for design. Example: Coordinated flight – Each vehicle represents control problem – Want real time coordinated flight planning – Answer: Quantization?

18 Quantization Naïve suggestion: Discretize state space? Issue: Reduction not related to control problem. Alternative: – Presume individual aircraft controllers. – Discretize via “primitives” of achievable maneuvers.

19 Robust Hybrid Automata Recent work at MIT for single vehicle: – Automata: Nodes (state subset) connected by trajectories – Hybrid: Trajectories executed by dynamical system – Robust: Tolerance set for node transitions Reduction directly related to control problem.

20 Quantization Consequences Complexity reduction. Low level limitations captured in admissible high level node transitions. Node transitions can be human inspired. Can define “primitive strings” to continue hierarchy. Discrete state !

21 Higher Order Primitives Simple example: String of primitives Need to derive coordinated primatives: – Will involve multiple RHA – Needed for complementary vehicle packages, e.g., maneuverability vs vulnerability – Draw upon existing expertise – Construct “play” sequences – Control interpretation: Feedforward

22 Encoding of Primitives Objective: Capture high level intent Previous example: Node sequences/Play sequences Alternative: Parameters in the low level optimization

23 Encoding via Objective Functions Recent work at Caltech for real-time trajectory generation: u [t o,t o +T] = arg min  L(x,u) dt +  (x(T),u(T)) dx/dt = f(x,u), g(x,u) < 0 Exploit structure of dynamics and “warm restart” to solve real-time.

24 Encoded Objectives, cont Can use penalty functions as encoded objectives Example: Cooperative trajectory planning “Node sequence” = terminal penalty sequence Objective function = mutual avoidance Different penalty functions reflect different cooperative roles, e.g., lead vs middle vs trailer

25 Encoded Objectives, cont Recent work at MIT (DARPA/JFACC) for hierarchical encoding Reduced order model AND Reduced order objective

26 Uncertain & Adversarial Conditions Uncertain environment motivated “robust” hybrid automaton Hostile adversary implies non-deterministic outcomes Approach: “non-deterministic” robust hybrid automata Examples: – Random outcome of battle engagement – Set-valued enemy actions

27 Model Reduction with Adversaries Deterministic case: “Hierarchical consistency” Random case: Averaged consistency (e.g., manufacturing systems) Adversarial case: Cannot “average” Approach: Encoding of low level adversarial encounters

28 Adversarial Encoding, cont Recent work at UCLA (DARPA/JFACC) for high-level adversarial modeling: Actual engagment: Sector-by-sector Model engagement: Single sector Allowed game-theoretic constructions

29 Technical Approach Scalability, Modeling & Reduction High level planning: High level planning: Development of analytical methods and computational algorithms for coordinated team strategies. Low level planning Communications

30 High Level Planning Hierarchical reduction likely leads to quantization. Increasing relevance of CS: – Combinatorial complexity management – Logical programming environment Control question: Find “coarseness” formulations that allow analytical insight.

31 High Level Planning, cont Conceptual approach is top-down or 2-way Opted for “designed behavior” vs “emergent behavior” Motivated by desire to meet specifications & understand performance trade-offs.

32 High Level Complexity Management Dimensionality reduction via higher order primitives Recent work at MIT with randomized algorithms: – Random waypoint selection in configuration space – Valuable in real-time obstacle avoidance Two-time scale complexity management: – Slow scale reinforcement learning/NDP – Fast scale randomized algorithms

33 Fast/Slow Scale Issues Computational obstacles persist despite dimensionality reduction Recent work at Cornell for complexity: – Critical problem formulation – Algorithm portfolios

34 Phase Transitions and Randomization Identify critical variable/constraint ratio Abrupt transition of “practical” complexity Understanding phase transition guides problem formulations Can approach sub-critical problem envelope Can bring in randomized algorithm “portfolio”

35 Logical Programming Environments Design questions become logical issues in discrete high level models Utility of logical programming environment: – Automated checking of design – Assistance in exploratory design – Verifiable re-use of existing design

36 LPE, cont High level planning can decompose into – Scheduling tasks – Executing tasks Example: – Formation – Avoidance – Regroup – Abort

37 LPE, cont Design question: Under what scenarios will vehicle collection successfully avoid obstacle? Formulate as logical question based on subtask and subgoals. Leads to multi-layered questions (within high level). Will benefit from complexity management research.

38 Team Strategies in Adversarial Environments Main team issue: Available information Major analytical difficulties with simple deviation from “centralized info” Example: Linear decentralized control Limited results for one & two sided team problems

39 Team Strategies, cont Approach: Exploit quantization & computation Impose “coarseness” until analytical solution structure emerges Use new insight to guide heuristic designs Example: UCLA DARPA/JFACC efforts

40 Low Level Planning Scalability, Modeling & Reduction High level planning Low level planning: Low level planning: Realization of team strategies through low level strategies and optimization. Communications

41 Local Optimization vs Global Objectives Low level optimization need not be consistent with high level intent Subtle issue: Same team becomes competitors Examples: Stadium viewing, collective feeding, voting mechanisms, prisoner’s dilemma

42 Conscientious Local Optimization Recent work at UCLA considers self-restraining local optimization: – Form model of interactions with team – Optimize based on interactive model – Self-impose conforming to model Leads to “handshaking” of model & achievable performance Interactive models both stochastic & deterministic

43 Uncertain & Adversarial Conditions “Traditional” receding horizon dilemma: – Short horizon for uncertainty (poor prediction) – Long horizon for stability Aggravated by existence of adversary Approach: Encode uncertainty/adversary in penalty functions

44 Cooperative Identification Local execution requires synchronized mode of operation Example: High order primitive of trajectory sequence Approach: Recent work at MIT (DARPA/JFACC) on “intent ID” applied to team members Communications viewpoint: Constrained consensus

45 Natural Cooperative Architectures Recent work at MIT on control architecture models of low level brain features Direct: How higher order primitives are assembled, recalled, & executed. Indirect: How internal models are resolved for prediction of future environmental interactions.

46 Communications What vs how to communicate? – Priorities – Quantity – QoS requirements Approach: Combine distributed mobile networks with controls communication requirements for new reliable protocols.

47 Adaptive Languages Controls/Communication intersection: – “Pre-programmed” vehicles with local information – Collective behavior determined by communicated information – Control decision = communications decision Approach: Adaptive languages/protocols as new perspective in highly autonomous operations

48 Agenda Scalability, modeling & reduction: Representation of distributed low level components in a manner amenable to high level planning with reduced complexity. High level planning: Development of analytical methods and computational algorithms for coordinated team strategies. Low level planning: Realization of team strategies through low level strategies and optimization. Communications: Investigation of communications issues within and among levels.

49 Agenda 8:00-8:30Continental Breakfast & Registration 8:30-8:45"Opening Remarks"Jacobs (AFOSR) 8:45-10:15"Overview"Shamma (UCLA) 10:15-10:30Break 10:30-12:00"Modeling & Planning with Robust Hybrid Automata"Feron (MIT) "Two-Tiered Distributed Control"D'Andrea (Cornell) "Real-time Trajectory Generation"Murray (Caltech) "Distributed Decision Making"Speyer (UCLA) 12:00-1:30Lunch 1:30-3:00"Logical Programming Environments"Hickey (Caltech) "Computational Complexity Management"Selman/Gomes (Cornell) "Distributed & Adaptive Communication Protocols"Pottie/Taylor (UCLA) "Neurological Modeling & Cooperation"Dahleh/Massaquoi (MIT) 3:00-3:15Break 3:15-4:00"Cornell Experimental Testbed"D'Andrea (Cornell) "UCLA Experimental Testbed"Chichka (UCLA) 4:00-5:00Open Discussion