Stochastic optimal design for unknown networked control system over unreliable communication network Student: Hao Xu, ECE Department Faculty Advisor: Dr. Jagannathan Sarangapani, ECE Department Adaptive Observer Design for NCS In this work, TCP protocol is considered for NCS. Since TCP uses acknowledgements to indicate the acceptance of the packet, network information set can be defined as Based on TCP protocol, adaptive observer design can be separated into two steps: 1) innovation step and 2) correction step. Innovation step at time : Predicted future system states: where , and is independent and identically distributed white Gaussian noise (i.e. ) Prediction error: Correction step at time : Update law for adaptive observer: Convergence: when , and at the same time. Simulation Results Consider the linear time-invariant batch reactor dynamics After random delays and packet losses due to unreliable communication network, the original time-invariant system was discretized and represented as a time-varying system (Note: since the random delays and packet losses are considered, the NCS model is not only time varying , but also a function of time k) Performance evaluation of proposed optimal control with AO and AE Stability: Figure 5 Stability performance As shown in Figure 5, if we use a PID without considering effects of unreliable communication network (i.e. delays and packet losses), the NCS will be unstable(Figure 5-(a)). However, when we implement proposed AO and AE optimal control, the NCS can still maintain stable(Figure 5-(b)). Observability: Figure 6 Observer performance As shown in Figure 6-(a), proposed AO can force observed system stable to converge to actual system state quickly. In Figure 6-(b), the stability region for proposed AO is shown. Since system dynamics is unknown, the region of proposed method is tighter than region of AO with known system dynamics. OBJECTIVES Investigate the effects of unreliable communication network (e.g. TCP) on the stability of the NCS with unknown dynamics Develop an adaptive observer (AO) to estimate networked control system (NCS) states; Develop an adaptive estimator (AE)-based stochastic optimal control for unknown NCS under unreliable communication network by using observed system states. BACKGROUND Networked control can reduce the installation costs and increase productivity through the use of wireless communication technology The challenging problems in control of networked-based system are unreliable communication network effects (e.g. network delay and packet losses). These effects do not only degrade the performance of NCS, but also can destabilize the system. Approximate dynamic programming (ADP) techniques intent to solve optimal control problems of complex systems without the knowledge of system dynamics in a forward-in-time manner. Figure 1 Networked control system * The proposed approach for optimal controller design involves using a combination of adaptive observer (AO) and adaptive estimator (AE). The effects of unreliable communication work are incorporated in the dynamic model which will be used for observer and controller development. * L. Marschall, “Five Decades of Automation”, Siemens Technical Report,2005. AE-based Stochastic Optimal Control When NCS system states are observed, AE-based stochastic optimal control can be derived based on observed states. Set up stochastic Vale-function: where Using the adaptive estimator to represent Value-function: where and is the Kronecker product quadratic polynomial basis vector Define the update law to tune the adaptive estimator Represent residual error: where and Update law for adaptive estimator: where is a constant, and Determine the AE stochastic optimal control input based on observed states Convergence: when , and . CONCLUSIONS Effects of unreliable communication network such as random delays and packet losses degrade the performance of NCS and can cause instability unless they are explicitly accounted for. Proposed stochastic optimal control based on AO and AE for linear NCS renders acceptable performance without requiring system dynamics and unreliable communication network. Proposed optimal controller works forward in time and suitable for real-time control. Networked Control System Model Networked control system representation and Figure 2 depicts a block diagram representation: Figure 2 Block diagram of networked control system AE-based Stochastic Optimal Control (2) Figure 3 present the flowchart for the AE-based stochastic optimal regulator of NCS Figure 3 Stochastic optimal regulator flowchart FUTURE WORK Design a novel cross-layer communication network protocol to improve the effects of unreliable communication network. Joint optimize the NCS from both control part and unreliable communication network part. Extend the NCS work to developing theory for cyber physical systems