1. Chess Review May 11, 2005 Berkeley, CA Closing the loop around Sensor Networks Bruno Sinopoli Shankar Sastry Dept of Electrical Engineering, UC Berkeley

2. Chess Review, May 11, 2005 2 Conceptual Issues Given a certain wireless sensor network can we successfully design a particular application? How does the application impose constraints on the network? Can we derive important metrics from those constraints? How do we measure network parameters?

3. Chess Review, May 11, 2005 3 What can you do with a sensor network? Literature provides key asymptotic results We are interested in answering different semantic questions, e.g.: At the algorithmic level: –How much packet loss can a tracking algorithm tolerate? At the network level: –How many objects can a particular sensor network reliably track?

4. Chess Review, May 11, 2005 4 Wireless Sensor Networks It’s a network of devices: –Many nodes: 10 3 -10 5 –Multi-hop wireless communication with adjacent nodes –Cheap sensors –Cheap CPU Issues w/ Sensor Networks and Data Networks ? –Random time delay –Random arrival sequence –Packet loss –Limited Bandwidth

5. Chess Review, May 11, 2005 5 Control Applications with Sensor Networks Pegs HVAC systems Power Grids Human body

6. Chess Review, May 11, 2005 6 Sensor net increases visibility Control and communication over Sensor Networks Observations y(t) Control inputs u(t) Computational unit

7. Chess Review, May 11, 2005 7 Experimental results: Pursuit evasion games

8. Chess Review, May 11, 2005 8 Problem Statement Given a control systems where components, i.e. plant, sensors, controllers, actuators, are connected via a specified communication network, design an “optimal “ controller for the system

9. Chess Review, May 11, 2005 9 Outline Problem Statement Optimal Estimation with intermittent observations Optimal control with both intermittent obs and control –TCP-like protocols –UDP-like protocols Conclusions

10. Chess Review, May 11, 2005 10 plant estimator + controller y(t) u(t) Network LGQ scenario for lossy network Bernoulli independent switches Modeling

11. Chess Review, May 11, 2005 11 Assumptions System: –Discrete time linear time invariant –Additive white gaussian noise on both the dynamics and the observation Communication network: –Packets either arrive or are lost within a sampling period following a bernoulli process. –A Delay longer than sampling time is considered lost. –Packet Acknowledgement depends on the specific communication protocol

12. Chess Review, May 11, 2005 12 Optimal estimation with intermittent observations Plant Aggregate Sensor State estimator Communication Network Main Results (IEEE TAC September 2004) Kalman Filter is still the optimal estimator We proved the existence of a threshold phenomenon:  max   min         c max cPt ct t PtMPE PPE |) 2 | ( 1 1 0condition initialany and 1for ][ 0condition initial some and 0for ][lim 0 0 0 Kalman Filter

13. Chess Review, May 11, 2005 13 Optimal control with both intermittent observations and control packets What is the minimum arrival probability that guarantees “acceptable” performance of estimator and controller? How is the arrival rate related to the system dynamics? Can we design estimator and controller independently? Are the optimal estimator and controllers still linear? Can we provide design guidelines? Plant Aggregate Sensor Controller State estimator Communication Network Communication Network

14. Chess Review, May 11, 2005 14 Control approach The problem of control is traditionally subdivided in two sub-problems: –Estimation Allows to recover state information from observations –Control Given current state information, control inputs are provided to the actuators The separation principle: – allows, under observability conditions, to design estimator and controller independently. If separation principle holds, optimal estimator (in the minimum variance sense) and optimal controller (LQG) are linear and independent

15. Chess Review, May 11, 2005 15 LQG control with intermittent observations and control Plant Aggregate Sensor Controller State estimator Communication Network Communication Network Ack is always present Ack is relevan t We’ll group all communication protocols in two classes: TCP-like (acknowledgement is available) UDP-like (acknowledgement is absent)

16. Chess Review, May 11, 2005 16 , – Bernoulli, indep. Minimize J_N subject to TCP – Transmission Control Protocol –PRO: feedback information on packet delivery –CONS: more expensive to implement UDP – User Datagram Protocol –PRO: simpler communication infrastructure –CONS: less information available ; LQG mathematical modeling

17. Chess Review, May 11, 2005 17 Estimator Design TCP UDP Prediction Step Correction Step

18. Chess Review, May 11, 2005 18 LQG Controller Design: TCP case Solution via Dynamic Programming: 1.Compute the Value Function t=N and move backward 2.Find Infinite Horizon by taking N  + 1 V t (x t ) – minimum cost-to-go if in state x t at time t

19. Chess Review, May 11, 2005 19 LQG Controller Design: TCP case We can prove that for TCP the value function can be written as: with: Minimization of v(t) yields:

20. Chess Review, May 11, 2005 20 Stochastic variable !! LQG Controller Design: TCP case

21. Chess Review, May 11, 2005 21 unbounded LQG averaged cost is bounded for all N if the following Modified Algebraic Riccati Equations exist: 1 1 bounded time-varying estimator gain estimator controller constant controller gain OPTIMAL LQG CONTROL Infinite Horizon: TCP case

22. Chess Review, May 11, 2005 22 Plant Aggregate Sensor Controller State estimator Communication Network Special Case: LQG with intermittent observations, unbounded =1 1 bounded 1

23. Chess Review, May 11, 2005 23 LQG Controller Design: UDP case Scalar system, i.e. x 2 R t=N t=N-1

24. Chess Review, May 11, 2005 24 LQG Controller Design: UDP case t=N-2 NONLINEAR FUNCTION OF INFORMATION SET It

25. Chess Review, May 11, 2005 25 prediction correction Without loss of generality I can assume C=I UDP controller: Estimator design Special case: C invertible, R=0

26. Chess Review, May 11, 2005 26 UDP special case: C invertible, R=0 It is possible to show that:

27. Chess Review, May 11, 2005 27 UDP Infinite Horizon: C invertible, R=0 It is possible to show that: unbounded 1 1 bounded Necessary condition for boundedness: Sufficient only if B invertible estimator/controller coupling

28. Chess Review, May 11, 2005 28 Conclusions Closed the loop around sensor networks General framework applies to networked control system Solved the optimal control problem for full state feedback linear control problems Bounds on the cost function Transition from state boundedness to instability appears Critical network values for this transition

29. Chess Review May 11, 2005 Berkeley, CA Thank you !!! For more info: sinopoli@eecs.berkeley.edu Related publications: Kalman Filtering with Intermittent Observations - IEEE TAC September 2004 Time Varying Optimal Control with Packet Losses - IEEE CDC 2004 Optimal Control with Unreliable Communication: the TCP Case - ACC 2005 LQG Control with Missing Observation and Control Packets - IFAC 2005