1. IPAM, 3 Mar 06R. M. Murray, Caltech1 Cooperative Control of Multi-Vehicle Systems Richard M. Murray Control and Dynamical Systems California Institute of Technology Domitilla Del Vecchio (U Mich) Bill Dunbar (UCSC) Alex Fax (NGC) Eric Klavins (U Wash) Reza Olfati-Saber (Dartmouth) Vijay Gupta Zhipu Jin Demetri Spanos Abhishek Tiwari Yasi Mostofi Cedric Langbort (CMI/UIUC)

2. IPAM, 3 Mar 06R. M. Murray, Caltech2 Arbiter computers for each vehicle Humans (2-3 per team) Example: RoboFlag (D’Andrea, Cornell) Robot version of “Capture the Flag” Teams try to capture flag of opposing team without getting tagged Mixed initiative system: two humans controlling up to 6-10 robots Limited BW comms + limited sensing D’Andrea & M ACC 2003

3. IPAM, 3 Mar 06R. M. Murray, Caltech3 RoboFlag Demonstration Integration of computer science, communications, and control Time scales don’t allow standard abstractions to isolate disciplines Example: how do we maintain a consistent, shared view of the field? Higher levels of decision making and mixed initiative systems Where do we put the humans in the loop? what do we present to them? Example: predict “plays” by the other team, predict next step, and react Hayes et al ACC 2003 Red Team view Flag carrier Tagged robot (blue) Obstacle

4. IPAM, 3 Mar 06R. M. Murray, Caltech4 Outline I.Cooperative Control subproblems II.Information Flow for Consensus and Formation Control III.Performance and Robustness IV.Distributed receding horizon control V.Protocols for cooperative control

5. IPAM, 3 Mar 06R. M. Murray, Caltech5 Context Cooperative control of large collections of vehicles Vehicles linked by task rather than dynamics Distributed computation  “consensus” required Point to point communications; limited BW, noisy links Dynamic and uncertain environments Avenues of research Stability and performance of interconnected systems Robustness and reconfiguration of vehicle teams Control over networks (“packet-based control theory”) Higher-level decision making and autonomy Model system for other applications Air traffic control Electric power networks Congestion and router control Integrated control, communications, computation

6. IPAM, 3 Mar 06R. M. Murray, Caltech6 RoboFlag Subproblems Goal: develop systematic techniques for solving subproblems Cooperative control and graph Laplacians Distributed receding horizon control Verifiable protocols for consensus and control 1.Formation control Maintain positions to guard defense zone 2.Distributed estimation Fuse sensor data to determine opponent location 3.Distributed consensus Assign individuals to tag incoming vehicles Implement and test as part of annual RoboFlag competition

7. IPAM, 3 Mar 06R. M. Murray, Caltech7 Information Flow in Vehicle Formations Example: satellite formation Blue links represent sensed information Green links represent communicated information Sensed information Local sensors can see some subset of nearby vehicles Assume small time delays, pos’n/vel info only Communicated information Point to point communications (routing OK) Assume limited bandwidth, some time delay Advantage: can send more complex information Topological features Information flow (sensed or communicated) represents a directed graph Cycles in graph  information feedback loops Question: How does topological structure of information flow affect stability of the overall formation?

8. IPAM, 3 Mar 06R. M. Murray, Caltech8 Sample Problem: Formation Stabilization Goal: maintain position relative to neighbors “Neighbors” defined by graph Assume only sensed data for now Assume identical vehicle dynamics, identical controllers? Example: hexagon formation Maintain fixed relative spacing between left and right neighbors Can extend to more sophisticated “formations” Include more complex spatia-temporal constraints relative position weighting factor offset 1 2 3 4 5 6 1 2 3 4 5 6

9. IPAM, 3 Mar 06R. M. Murray, Caltech9 Graph Laplacian Construction of (weighted) Laplacian A = adjacency matrix D = diagonal matrix, weighted by outdegree Properties of Laplacian Row sum equal 0 (stochastic matrix) All eigenvalues are non-negative, with at least one zero eigenvalue (from row sum) Multiplicity of 0 as an eigenvalue is equal to the number of strongly connected compo-nents of the graph All eigenvalues lie in a circle of radius one centered at 1 + 0i (Perron Frobenius) For bidirectional (eg, undirected) graphs, eigenvalues are all real, in [0,2]

10. IPAM, 3 Mar 06R. M. Murray, Caltech10 Mathematical Framework Analyze stability of closed loop Interconnection matrix, L, is the Laplacian of the graph Stability of closed loop related to eigenstructure of the Laplacian

11. IPAM, 3 Mar 06R. M. Murray, Caltech11 Stability Condition Theorem The closed loop system is (neutrally) stable iff the Nyquist plot of the open loop system does not encircle -1/ i (L), where i (L) are the nonzero eigenvalues of L. Example Fax and Murray IFAC 02, TAC 04

12. IPAM, 3 Mar 06R. M. Murray, Caltech12 Spectra of Laplacians Unidirectional tree CycleUndirected graph Periodic graph

13. IPAM, 3 Mar 06R. M. Murray, Caltech13 Example Revisited Example Adding link increases the number of three cycles (leads to “resonances”) Change in control law required to avoid instability Q: Increasing amount of information available decreases stability (??) A: Control law cannot ignore the information  add’l feedback inserted x x x x

14. IPAM, 3 Mar 06R. M. Murray, Caltech14 Improving Performance through Communication Baseline: stability only Poor performance due to interconnection Method #1: tune information flow filter Low pass filter to damp response Improves performance somewhat Method #2: consensus + feedforward Agree on center of formation, then move Compensate for motion of vehicles by adjusting information flow Fax and Murray IFAC 02

15. IPAM, 3 Mar 06R. M. Murray, Caltech15 Special Case: Consensus Consensus: agreement between agents using information flow graph Can prove asymptotic convergence to single value if graph is connected If w ij = 1/(in-degree) + graph is balanced (same in-degree for all nodes)  all agents converge to average of initial condition Variations and extensions (Jadbabaie, Leonard, Moreau, Morse, Olfati- Saber, Xiao, …) Switching (packet loss, dropped links, etc) Time delays, plant uncertainty Nearest neighbor graphs, small world networks, optimal weights Nonlinear: potential fields, passive systems, gradient systems x x x x

16. IPAM, 3 Mar 06R. M. Murray, Caltech16 Open Problems: Design of Information Flow (graph) How does graph topology affect location of eigenvalues of L? Would like to separate effects of topology from agent dynamics Possible approach: exploit for of characteristic polynomial x x x x

17. IPAM, 3 Mar 06R. M. Murray, Caltech17 Performance Look at motion between selected vehicles Jin and Murray CDC 04 G 1 - Control G 2 - Performance

18. IPAM, 3 Mar 06R. M. Murray, Caltech18 Robustness What happens if a single node “locks up” Different types of robustness (Gupta, Langbort & M) Type I - node stops communicating (stopping failure) Type II - node communicates constant value Type III - node computes incorrect function (Byzantine failure) Related ideas: delay margin for multi-hop models (Jin and M) Improve consensus rate through multi-hop, but create sensitivity to communcations delay Single node can change entire value of the consenus Desired effect for “robust” behavior:  x I =  /N x 1 (0) = 4 x 2 (0) = 9 x 3 (0) = 6x 4 (t) = 0 X 5 (t) = 6 x 6 (t) = 5 Gupta, Langbort and Murray CDC 06

19. IPAM, 3 Mar 06R. M. Murray, Caltech19 RoboFlag Subproblems Goal: develop systematic techniques for solving subproblems Cooperative control and graph Laplacians Distributed receding horizon control Verifiable protocols for consensus and control 1.Formation control Maintain positions to guard defense zone 2.Distributed estimation Fuse sensor data to determine opponent location 3.Distributed consensus Assign individuals to tag incoming vehicles Implement and test as part of annual RoboFlag competition

20. IPAM, 3 Mar 06R. M. Murray, Caltech20 Optimization-Based Control Task: Maintain equal spacing of vehicles around circle Follow desired trajectory for center of mass Parameters: Horizon: 2 sec Update: 0.5 sec Local MPC + CLF Assume neighbors follow straight lines Global MPC + CLF Dunbar and M IFAC 02

21. IPAM, 3 Mar 06R. M. Murray, Caltech21 Individual optimization: Theorem. Under suitable assumptions, vehicles are stable and converge to global optimal solution. Pf Detailed Lyapunov calculation (Dunbar thesis) Main Idea: Assume Plan for Neighbors state time t0t0 t 0 +d z 3 (t 0 ) z 3 * (t;t 0 )z 3 k (t) What 2 assumes What 3 does Compatibility constraint: each vehicle transmits plan to neighbors stay w/in bounded path of what was transmitted

22. IPAM, 3 Mar 06R. M. Murray, Caltech22 Example: Multi-Vehicle Fingertip Formation 2 4 q ref d 31

23. IPAM, 3 Mar 06R. M. Murray, Caltech23 Simulation Results

24. IPAM, 3 Mar 06R. M. Murray, Caltech24 RoboFlag Subproblems Goal: develop systematic techniques for solving subproblems Distributed receding horizon control Packet-based, distributed estimation Verifiable protocols for consensus and control 1.Formation control Maintain positions to guard defense zone 2.Distributed estimation Fuse sensor data to determine opponent location 3.Distributed consensus Assign individuals to tag incoming vehicles Implement and test as part of annual RoboFlag competition

25. IPAM, 3 Mar 06R. M. Murray, Caltech25 P(k 1,k 2 ) := { initializers guard 1 :rule 1 guard 2 :rule 2... } S(k 1,k 2 ):=P(k 1,k 2 )+C(k 1 +1) sharing y,u "soup" of guarded commands composition = union non-shared variables remain local to component programs CCL: Computation and Control Language Formal Language for Provably Correct Control Protocols CCL Interpreter Formal programming lang- uage for control and comp- utation. Interfaces with libraries in other languages. Automated Verification CCL encoded in the Isabelle theorem prover; basic specs verified semi-automatically. Investigating various model checking tools. Formal Results Formal semantics in transition systems and temporal logic. RoboFlag drill formalized and basic algorithms verified. CCL Protocol for Decentralized Target Allocation

26. IPAM, 3 Mar 06R. M. Murray, Caltech26 Example: RoboFlag Drill Klavins CDC, 03 Things that we can prove (so far) Semi-automated proofs (Isabelle) Avoidance: no two robots collide Self-stabilization: if attackers are far enough away, defenders self- stabilize before attackers arrive Next steps Implement CCL on MVWT Improved reasoning toolbox Sample drill N on N attack, w/ replenishment Random initial assignments Switching proto- col to avoid collisions

27. IPAM, 3 Mar 06R. M. Murray, Caltech27 Observation of CCL Programs Del Vecchio & Klavins, CDC'03 Problem: Determine state of communications protocol used by a group of robots given their physical movements. Assumptions: Protocol and motion control are described in CCL like language. Results: Defns of observability, etc. for CCL programs Construction and analysis of observer that converges when the system is "weakly" observable Construction of an efficient observer for Roboflag drill in particular Everything specified in CCL

28. IPAM, 3 Mar 06R. M. Murray, Caltech28 Summary: Cooperative Control of Multi-Vehicle Systems 1.Formation control Maintain positions to guard defense zone 2.Distributed estimation Fuse sensor data to determine opponent location 3.Distributed consensus Assign individuals to tag incoming vehicles Integration of computer science, communications, and control Mixture of techniques from computer science, communications, control Increased need for reasoning at higher levels of abstraction (strategy)