Feedback Control and Multi-Agent Systems: Ubiquitous and Increasingly Interdependent Prof. Bill Dunbar Autonomous Systems Group Computer Engineering

Some familiar examples: How do we describe systems? … with math! What are Systems? … ANYTHING in Engineering, usually with Dynamics.

Math: Describing Diverse Engineering Systems in a Common Way Internet backboneCA power gridSan Fran ATC In these Examples:

Control Systems are Hidden Engineering Systems “A Control System is a device in which a sensed quantity is used to modify the behavior of a system through computation and actuation.”

My (and Potentially Your) Research Robotics – Exploration – Toys – Competition (soccer) Automated Freeways Supply Chain Management

Eventually…A Fully Autonomous Vehicle Off-Road Dessert Race

The Potential is Enormous Researchers at Caltech are working toward the math model of the “fruit fly system,” with the ultimate objective of making a micro- mechanical fly!

Distributed Optimization-Based Control of Multiagent Systems Ass. Prof. Bill Dunbar Autonomous Systems Group Computer Engineering

Multiagent Systems are Everywhere The Internet Air traffic control The Power Grid Autonomous Formations Control Problems with: Subsystem dynamics Shared resources (constraints) Communications topology Shared objectives

Multiagent Systems: Inherently Distributed and Cooperative Multiagent System: autonomous agents communication network Agent Environment output action sensor input Distributed: local decisions based on local information. Cooperative: agents agree on roles & dynamically coordinate.

A Relevant Decision Method: Receding Horizon Control (RHC) RHC uses optimization to find feasible/optimal plans for near future. Minimize (distance to pump & fuel) s.t. Car model (dynamics) Without hitting wall (constraint) objective To mitigate uncertainty, plan is revised after a short time. X computed actual

Mathematics of RHC is Finite Horizon Optimal Control Minimize (distance to pump& fuel) s.t. Car dynamics Without hitting wall (constraint) objective

Convergence of RHC Requires Appropriate Planning Horizon and Terminal Penalty Theoretical conditions sufficient & in absence of explicit uncertainty. * [Mayne et al., 2000]

RHC Compared to Other Control Techniques Gives planning & feedback with built-in contingency plans. Only technique that handles state and control constraints explicitly. Tradeoff: computationally intensive. zk()zk() state time t0t0 t0+t0+ z(t 0 ) z * (  ;t 0 ) T

RHC Successful in Applications: Process to Flight Control Caltech flight control experiment: Tracking ramp input of 16 meters in horizontal, step input of 1m in altitude. RHC updates at 10 Hz, trajectories generated by NTG software package. Movie

RHC Admits Cooperation Decoupled dynamics Avoid collision Get 1 to pump, 2 follow 1 & 3 follow 2. ok follow 123

RHC of Multiagent Systems: What’s Missing? Enables autonomy of single agent. Amenable to cooperation for multiple agents. Missing?…Distributed Implementation* Why not Centralized?…Local decision require Global information Parallelization**?…If you can, but sometimes not applicable. * [Krogh et al, 2000, 2001]**[Bertsekas & Tsitsiklis, 1997]

My Contribution: A Distributed Implementation of RHC Decoupled subsystem dynamics/constraints, Coupled cost L Decomposition Distributed: local decisions based on local information.

Solution of Sub-problems requires Assumed Plan for Neighbors Agent 3  state time t0t0 t0+t0+ z 3 (t 0 ) z 3 * (  ;t 0 )z3k()z3k() What 2 assumes What 3 does

Compatibility of Actual and Assumed Plans via Constraint state time tktk tk+tk+ z 3 (t k ) Bounds discrepancy Assumed plan Compatibility constraint

Distributed Implementation Requires Synchrony & Common Horizon T

Conditions for Theory are General

Convergence Conditions: Same as Centralized plus Bound on Update Period * [Dunbar & Murray, Accepted to Automatica, June, 2004]

Venue: Multi-Vehicle Fingertip Formation 2 4 q ref d 31

Decomposition of Coupled Cost

Simulation Parameters : Reference signal : Actual COM of {1,2,3} 2 4

Centralized RHC: Benchmark for Comparison

Centralized RHC Simulation

Distributed RHC is Comparable to Centralized RHC

Distributed RHC Simulation

Naïve Approach Produces Less Desirable Performance

Naïve Approach: Bad Overshoot

Summary of Contribution Distributed implementation of RHC is provable convergent, performs well, and is applicable to a class of Multiagent Systems: Distributed & cooperative structure: Local decisions based on local information Decomposition and incorporation of compatibility constraint Coordination via sharing feasible plans Applicable for: Heterogeneous nonlinear dynamics Generic objective function (need not be quadratic) Coupling constraints

Theory conservative; useful as guideline for implementation. Scalable: computational complexity independent of N a ; communication complexity independent of N a but dependent on N i (size of neighborhood). Communicating trajectories: more intensive than traditional decentralized control, but not too bad, given smoothness properties. Less communication than required by parallelization. Tradeoff: not recovering centralized solution to original problem, but that of a modified problem. Conclusions

Current and Ongoing Work Theoretically: Locally synchronous and asynchronous versions DONE: Coupled subsystem dynamics. Potential applications: Process control Supply chain management Reduced order contingency plans Connection with rollout algorithms in MDPs

Current and Ongoing Work Applications: Coordinated UAVs Mobile Sensor Networks Robots coordinating for toxin detection Intelligent Transportation Systems Automated freeways Semi-automated Air Traffic Control Interdisciplinary examples: Supply chain management (Business) Power/Water resource management