1. A Mechanism Design Approach for the Stabilization of Networked dynamical systems L. Galbusera, N. Gatti, C. Romani Dipartimento di Elettronica e Informazione – Politecnico di Milano e-mail: galbusera, ngatti, romani@elet.polimi.it 48th IEEE Conference on Decision and Control Shanghai, China December 16-18, 2009

2. 2 Networked control system (NCS) Elements:  N linear continuous-time subplants with unstable uncontrolled dynamics.  A bus communication medium.  N controllers designed to stabilize each subplant. Standing assumption: at each time instant, only one subplant is connected to its controller Control objective:  Synthesis of an effective dynamic scheduling policy (not preassigned).

3. 3 Networked control system (NCS) Previous literature on dynamic scheduling policies:  the scheduling is usually assigned in a centralized manner by comparing systems’ states and parameters (e.g., the CLS-ε policy in [Hristu-Varsakelis, CDC 2001]). Real-world applications:  the subplants can be modeled as strategic players in a game for having access to the communication medium. t AUCTION FOR ACCESSING THE MEDIUM AT TIME t* S1S1 S2S2 SNSN t* PLAYERS REPORT THEIR (NOT-NECESSARILY TRUE) CURRENT STATES

4. 4 Networked control system (NCS) t AUCTION FOR ACCESSING THE MEDIUM AT TIME t* S1S1 S2S2 SNSN t* Control objectives 1.Stability of the NCS 2.Efficient allocation of the communication medium 3.Avoiding strategic behaviors of the players PLAYERS ARE SELF-INTERESTED THEY REPORT THEIR (NOT-NECESSARILY TRUE) CURRENT STATES

5. 5 Preliminaries: stability in NCS t Dynamical model of subsystem i: Control law: Time: T S1S1 S2S2 j-th time interval of lenght T

6. 6 Preliminaries: stability in NCS Stability condition: Further assumption: Period T is discretized in M regular time intervals for executing the auctions. Lower bound to control subintervals t T 1 2 3 4 5 … M AUCTIONS

7. 7 if player i pays Groundings on mechanism design ALTERNATIVES (= possible outcomes of the game) PLAYER i MECHANISM TRUE EVALUATION of player i over the set of alternatives REPORTED EVALUATION of player i over the set of alternatives (other players) PAYMENT of player i MONETARY RESOURCES of player i A player can participate to the auction only if Reference: [Fudemberg & Tirole, Game theory, The MIT Press, 1991]

8. 8 Groundings on mechanism design WINNING ALTERNATIVE PLAYER 1 MECHANISM PLAYER N PLAYER 2 Maximization of the social welfare Reference: [Fudemberg & Tirole, Game theory, The MIT Press, 1991]

9. 9 Groundings on mechanism design WINNING ALTERNATIVE PLAYER 1 MECHANISM PLAYER N PLAYER 2 Maximization of the social welfare DEFINITION OF PAYMENTS Reference: [Fudemberg & Tirole, Game theory, The MIT Press, 1991]

10. 10 Groundings on mechanism design Key features:  Player i ’s utility:  Truthful mechanism: a mechanism in which each player cannot increase its utility by misreporting its true evaluation, i.e., a mechanism in which  VCG mechanisms (Vickrey, Clarke and Grove): a class of mechanisms which is guaranteed to be truthful by means of a suitable definition of the payment function:

11. 11 Groundings on mechanism design Key features:  Clarke’s pivot rule for specifying the payment: the winner’s payment equals the second-highest bid  VCG mechanisms are weakly budget-balanced, i.e.,  Therefore, the iterated application of the mechanism (non-strictly) decreases the players’ resources.  A solution: Cavallo’s pivot rule Cavallo’s pivot = Clarke’s pivot + redistribution mechanism > Truthfulness is preserved > Budget balancing is enhanced The second and third classified in the bid increase their resources

12. 12 A mechanism for NCS Two-layer structure:  First layer efficient allocation of the medium (with no stability guarantees);  Second layerfor ensuring stability. The allocation procedure is governed by two sets of monetary sources:  Standard resources (c i ) used at the first layer, in order to allocate the medium;  Stability-preserving resources (c si ) used at the second layer, in order to preserve stability. PLAYER i

13. 13 A mechanism for NCS What does the mechanism need to know in order to work?  The quantities  The period T  The standard resources and stability-preserving resources of the players  The true value of the state of each subsystem (=player) at the beginning of each period Set of alternatives: t a priori information online information

14. 14 A mechanism for NCS Evaluation function (common to both layers): if the subplant is choosen if the subplant is not choosen Remarks:  Subplant i has a positive evaluation only if it is chosen to be controlled.  The monetary resources do not directly affect the value of the evaluation function, but only enable the participation of the subplants to the mechanism. VCG mechanism (truthfulness) Depends on the state evolution of the closed-loop subsystem along the next time subinterval

15. 15 A mechanism for NCS Social-efficiency based selection criterion: Payment mechanism (related to standard resources) In view of truthfulness, the subplant i* with the highest evaluation value maximizes the social efficiency and is thus selected. Cavallo’s redistributions Limited communication requirements: each player only sends its own evaluation

16. 16 A mechanism for NCS Initialization of monetary resources  At the beginning of each period of length T, c i and c si are initialized as follows: Standard resources (c i ) depend on the state at the beginning of the same period Stability-preserving resources (c si ) equal the minimum number of subintervals subsystem i needs to be controlled in order to preserve stability

17. 17 A mechanism for NCS Update rules for monetary resources  Both standard resources and stability-preserving resources are updated at each execution of the mechanism during the period ( ). Standard resources (c i ) Stability-preserving resources (c si ) Each time the subplant is chosen, the resources are reduced by one unit until they reach zero. current resourcespaymentCavallo’s redistribution

18. 18 A mechanism for NCS Mechanism design switching rule  IDEA allocation based on standard resources (efficiency-based) until the stabilization requirement becomes critical.  At each time step both resources are updated;  Standard resources are used for the bid until the number of remaining time step before the end of the period are just enough to complete the stabilization of all subsystems (i.e., zeroing the stability-preserving resources).

19. 19 Simulation example  Three first-order unstable linear subplants, each of them associated with a controller that stabilizes the respective subplant. Open- and closed-loop eigenvalues:  A comparison between different allocation methods over a time period T:  (A) The proposed mechanism-based allocation method;  (B) A modified allocation method obtained by removing induced payments and standard monetary resources.  In order to emphasize the difference in the resulting control action, we assume that subplant S 1 reports the following altered evaluation function value: uncontrolled plants controlled plants

20. 20 Solution (A):  more marked alternation among subsystems in the scheduling;  penalization of the subsystem that “lies” (S 1 ), in view of the resource-exhaustion phenomenon. Simulation example (A) Proposed mechanism (B) Alternative solution

21. 21 Simulation example Solution (A):  Better overall state performance. (A) Proposed mechanism (B) Alternative solution

22. 22Conclusions Main features:  Application of mechanism design to the stabilization issue of networked control systems;  synthesis of a dynamic scheduling policy in a game-theoretical setting;  our scheme avoids strategic behaviors of the players and efficiently allocates the communication;  the mechanism needs limited information to properly work.

23. 23 Simulation example Stability tokens zeroed before the end of the period. Switching to the second layer does not occur in this example. Stability tokens Ordinary tokens (A) Proposed mechanism