1. Distributed Control of Multiagent Systems: From Engineering to Economics Prof. William Dunbar Autonomous Systems Group Computer Engineering

2. 2 Some familiar examples: How do we describe (or predict) systems? … with math! What are Systems? … ANYTHING in Engineering, usually with Dynamics. (Images courtesy of http://www.cds.caltech.edu/~murray/cdspanel/ unless stated otherwise)http://www.cds.caltech.edu/~murray/cdspanel/

3. 3 Math: Describing Diverse Engineering Systems in a Common Way Internet backboneCA power gridSan Fran ATC In these examples:

4. 4 Control Systems are Hidden Engineering Systems “A Control System is a device in which a sensed quantity is used to modify the behavior of a system through computation and actuation.”

5. 5 Abstraction: Multiagent Systems The Internet Air traffic control Supply Chain Control Problems with: Subsystem dynamics Shared resources (constraints) Communications topology Shared objectives (SC image courtest of www.vipgroup.us)

6. 6 Multiagent Systems: Distributed and (presumed) Cooperative Multiagent System: autonomous agents communication network Agent Environment output action sensor input Distributed: local decisions based on local information. Cooperative: agents agree on roles & dynamically coordinate.

7. 7 A Relevant Decision Method: Model Predictive Control (MPC) MPC uses optimization to find feasible/optimal plans for near future. Minimize (distance to pump & fuel) s.t. Car model (dynamics) Without hitting wall (constraint) objective To mitigate uncertainty, plan is revised after a short time. X computed actual

8. 8 Mathematics of MPC is Finite Horizon Optimal Control Minimize (distance to pump& fuel) s.t. Car dynamics Without hitting wall (constraint) objective

9. 9 Convergence of MPC Requires Appropriate Planning Horizon Theoretical conditions sufficient & in absence of explicit uncertainty. * [Mayne et al., 2000]

10. 10 MPC Compared to Other Techniques Gives planning & feedback with built- in contingency plans. Only technique that handles state and control constraints explicitly. Tradeoff: computationally intensive. zk()zk() state time t0t0 t0+t0+ z(t 0 ) z * (  ;t 0 ) T

11. 11 MPC Successful in Applications: Process to Flight Control Caltech flight control experiment: Tracking ramp input of 16 meters in horizontal, step input of 1m in altitude. MPC updates at 10 Hz, trajectories generated by NTG software package. Movie

12. 12 MPC Admits Cooperation Decoupled dynamics Avoid collision Get 1 to pump, 2 follow 1 & 3 follow 2. ok follow 123

13. 13 MPC of Multiagent Systems: What’s Missing? Enables autonomy of single agent. Amenable to cooperation for multiple agents. Missing?…Distributed Implementation* Why not Centralized?…Local decision require Global information Parallelization**?…If you can, but sometimes not applicable. * [Krogh et al, 2000, 2001]**[Bertsekas & Tsitsiklis, 1997]

14. 14 My Contribution: A Distributed Implementation of MPC Decoupled subsystem dynamics/constraints, Coupled cost L Decomposition Distributed: local decisions based on local information.

15. 15 Solution of Sub-problems requires Assumed Plan for Neighbors Agent 3  state time t0t0 t0+t0+ z 3 (t 0 ) z 3 * (  ;t 0 )z3k()z3k() What 2 assumes What 3 does

16. 16 Compatibility of Actual and Assumed Plans via Constraint state time tktk tk+tk+ z 3 (t k ) Bounds discrepancy Assumed plan Compatibility constraint

17. 17 Distributed Implementation Requires Synchrony & Common Horizon T

18. 18 Conditions for Theory are General

19. 19 Convergence Conditions: Centralized plus Bound on Update Period * [Dunbar & Murray, Accepted to Automatica, June, 2004]

20. 20 Venue: Multi-Vehicle Fingertip Formation 2 4 q ref d 31

21. 21 Simulation Parameters : Reference signal : Actual COM of {1,2,3} 2 4

22. 22 Centralized MPC: Benchmark for Comparison

23. 23 Centralized MPC Simulation

24. 24 Distributed MPC is Comparable to Centralized MPC

25. 25 Distributed MPC Simulation

26. 26 Naive Approach Produces Less Desirable Performance

27. 27 Naïve Approach: Bad Overshoot

28. 28 Summary of Contribution Distributed implementation of MPC is provable convergent, performs well, and is applicable to a class of Multiagent Systems: Distributed & cooperative structure:  Local decisions based on local information  Decomposition and incorporation of compatibility constraint  Coordination via sharing feasible plans Applicable for:  Heterogeneous nonlinear dynamics  Generic objective function (need not be quadratic)  Coupling constraints and coupled dynamics

29. 29 Supply Chain Management (SCM) is an Attractive Venue for DMPC Dynamics (Linear/Nonlinear) s.t. constraints and moving set points. Forecasts of measurable inputs often available, which MPC can easily incorporate. Dynamic time scales and inter-stage communication BW are not limiting factors. Active research area. Why? Companies don’t compete - their supply chains do. Thus, SCM will make or break companies. Examples: Dell, Walmart. Challenge: distributed (asynchronous) coordination in the presence of time delays.

30. 30 Overview 1.Define three stage SCM problem from supply chain literature 2.Distributed Problem ==> Distributed MPC Implementation 3.Nominal decentralized feedback policy from supply chain literature 4.Numerical Experiments for Comparison 5.Conclusions and Extensions

31. 31 SCM: Information Flows Upstream (orders) and Material Flows Downstream (goods) Three Stages: Supplier S, Manufacturer M, Retailer R UP stream DOWN stream order rate demand rate shipment rate acquisition rate For each stage Control: order rate Measurable Exogenous Inputs: delay

32. 32 Bi-drectional Coupling in the Dynamics For each stage Dynamics: Constraints: Coupling: x depends on downstream order rate & upstream backlog Objective: Keep stock and unfulfilled order at desired levels

33. 33 DMPC: Parallel Updates Assuming Remainder of Previous Response for Neighbors Q-cost with move suppression DMPC Controller for each stage 1. A. B. C. 2. Move suppression ~ DMPC theory bounds

34. 34 Experiments Show Comparable Performance with Nominal Policy: Single Stage Case Nominal: Devised to match observed responses Response to initial stock offset Standard MPC (not DMPC)

35. 35 Step in Demand Rate: Comparable Performance + Advantage of Anticipation Add anticipation

36. 36 Three Stage with Pulse in Customer Demand: Comparable then Better with Anticipation NominalDMPC

37. 37 Conclusions and Extensions Realistic SCM problem (classic MIT “Beer Game”) DMPC comparable to validated nominal feedback policy. Clear advantage when customer demand can be reliably forecasted (anticipation). A detailed relative degree, controllability and stabilizability analysis to come. Unfulfilled order in stages M and R exhibited nonzero steady-state error. Next leap: multi-echelon chains - at least two (and possibly many) players operate within each stage, e.g., the S stage in Dell's ``build-to-order" supply chain management strategy might contain several chip suppliers such as Samsung, Intel and Micron. Extend theory asynchronous time conditions.