3. Applications Function approximation: t  y t: target, y: actual output Neural control r: reference u: control effort y: system output r  y Ming-Feng Yeh

Neural Control Schemes Supervised control Hybrid control Model reference control Internal model control Adaptive control Reference: C.L. Lin & H.W. Su, “Intelligent control theory in guidance and control system design: an overview,” Proc. Natl. Sci. Counc. ROC (A), pp. 15-30. Ming-Feng Yeh

Supervisory Control Neural network  inverse NN controller The neural controller in the system is utilized as an inverse system model. Ming-Feng Yeh

Hybrid Control Generalized learning (off-line learning) A rough approximation to the desired control law  Drive the plant over the operating range and without instability Specialized learning (on-line learning) Improve the control provided by the NN controller Ming-Feng Yeh

Model Reference Control Must define its input-output pair {r(t), yR(t)} in advance Attempts to make the plant output y(t) match the reference model output asymptotically. The error e(t) is used to adjust the weights of an neural controller. Ming-Feng Yeh

Internal Model Control The NN plant model is first trained off-line to emulate the controller plant dynamics directly. During on-line operation, the error is used as a feedback signal and passed to the NN controller. The effect of the NN controller is to subtract the effect of the control signal from the plant output, i.e., disturbances. The IMC plays a role as a feedforward controller and can cancel the influence due to unmeasured disturbances. Ming-Feng Yeh

Adaptive Control The tracking error cost is evaluated according to some performance index. The result is then used as a basis for adjusting the connection weights of the neural networks. The weights are adjusted on-line using basic backpropagation rather than off-line. Ming-Feng Yeh

Practical Stability Issues in CMAC Neural Network Control Systems Paper Study #1 Practical Stability Issues in CMAC Neural Network Control Systems Fu-Chuang Chen & Chih-Horng Chang IEEE Trans. on Control Systems Technology, Vol. 4, No. 1, pp. 86-91, 1996 Ming-Feng Yeh

Abstract CMAC is a practical tool for improving existing nonlinear control systems. CMAC can effectively reduce tracking error, but can also destabilize a control system which is otherwise stable. Quantitative studies are presented to search for the cause of instability in the CMAC control system. Ming-Feng Yeh

I. Introduction CMAC is basically a look-up table method, very easy to implement, and at the same time it is a powerful and practical tool for nonlinear control. There has been convergence result on the CMAC learning. Ming-Feng Yeh

Main Purpose of This Paper To introduce the CMAC control system from an industrial point of view. To describe the unstable phenomenon. To quantitatively study how the system parameters such as control gain, quantization, generalization, learning rate, etc., are related to the instability of the system. To suggest ways to improve system stability. To provide some experience evidence. Ming-Feng Yeh

II. CMAC Control System CMAC controller Plant proportional controller Y(k+1) = 0.5Y(k)+sin(Y(k))+U(k) Use a workable traditional controller to stabilize the plant and to help the CMAC learn to provide precise control. Ming-Feng Yeh

Functioning of CMAC Initially the CMAC table is empty. In each time step k, the CMAC involves a recall and a learning process. Ming-Feng Yeh

Recall Process Uses Yd(k+1) and Y(k) as the address to generate the control signal from CMAC table, where Yd(k+1) is the desired system output for the next time step. CMAC has two inputs and one output. Ming-Feng Yeh

Leaning Process U(k) is treated as the desired output to modified the content stored at location Y(k) and Y(k+1), where Y(k+1) is the actual system output for the next time step k+1. To speed up the initial learning and to achieve better generalization, the generalization technique is employed, i.e., each input vector to CMAC for recall and learning will map to a number of memory locations instead of only one memory location. Ming-Feng Yeh

Function Approximation How precisely the CMAC can approximate a function is mainly determined by the quantization in each dimension of the input vector. Reducing quantization would quickly increase the memory demand for storing the CMAC table. Ming-Feng Yeh

Table Update Mechanism Gradient-type learning rule Wi(k+1) = Wi(k) +   [ U(k)  Uc(k) ] / g g: the size of generation Wi: the content of the ith memory location, there being q locations to be updated : the learning rule U: the correct (desired) data Uc: the current (actual) data Ming-Feng Yeh

III. A Typical Simulation Study Proportional gain: P = 1.4 Learning rate:  = 0.1 Generalization = 50 Quantization = 5/500 (meaning five units divided into 500 divisions) Reference command = sin(2k/200) with each sinusoidal cycle consisting of 400 time steps Tracking error Number of cycles Ming-Feng Yeh

A Typical Simulation Study In the first five cycles, the system is solely controlled by the P controller. The CMAC is added at the 6th cycle, and then the error reduces quickly and significantly. The error remains small for some time, and then it diverges around the 143th cycle. Control output Number of time steps Ming-Feng Yeh

Discussion The CMAC can significantly reduce the tracking error. CMAC can destabilize a control system which is otherwise stable. The unstable phenomenon certainly comes from the interactions between the proportional controller and the CMAC network. Ming-Feng Yeh

Discussion The proportional controller can not be removed even when the magnitude of the proportional control is very small compared with that of the CMAC (i.e., when the system output error has been significantly reduced). Otherwise, the good tracking can not be maintained. Ming-Feng Yeh

Growth of Oscillation 8th cycle 90th cycle 155th cycle 40th cycle Ming-Feng Yeh

V. Method for Improving System Stability The continued learning of CMAC after the tracking error has reduced is the major cause of the instability. Stopping the CMAC learning has two drawbacks. First, it can be difficult to determine when to stop the CMAC learning. Second, if the CMAC stops learning, then the CMAC control system cannot respond to any change in the reference command. Ming-Feng Yeh

Modified Learning Rule To effectively stop the CMAC learning when the tracking error is small, but at the same time allow the system to respond to any change in the reference command, a deadzone is added to the CMAC updating rule. Wi(k+1) = Wi(k) +   D[ U(k)  Uc(k) ] / g where Ming-Feng Yeh

VI. Experiment Ming-Feng Yeh

Paper Study #2 Intelligent Controller Using CMACs with Self-Organized Structure and Its Application for a Process System T. Yamamoto, H. Yanagino & M. Kaneda Proceedings of 1997 IEEE , pp. 76-81 Ming-Feng Yeh

Abstract This paper describes a design scheme of intelligent system consists of some CMACs. Each of CMACs is trained for the specified command signal. A new CMAC is generated for unspecificed command signals, and the CMAC whose command is nearest for the new command signal, is eliminated. The proposed intelligent controller can be designed with relatively small memories. Ming-Feng Yeh

1. Introduction The CMACs included in the intelligent controller are trained in both off-line and on-line learning process for each of the specified command signals. For a unspecified command, a new CMAC is generated. The initial weights are set by employing the linear interpolation to the trained weights included in two CMACs whose command signals are nearest for the new command signal. The CMAC corresponding to the nearest command signal is eliminated. The proposed intelligent controller can be designed with relatively small memories. Ming-Feng Yeh

3. Intelligent Controller Design Reference Model CMAC1 CMACi CMACn  System controller Ming-Feng Yeh

Outline The input signals to each CMAC are the control error signal and the difference, that is, the two- dimensional CMACs are equipped in the intelligent control system. By including the command signal as one of input signals in the CMAC, the intelligent control system can be constructed by using only a three-dimensional CMAC. Ming-Feng Yeh

Off-line Learning Process CMAC  w(t): command signal u*(t): teacher signal Updated rule: h = 1, 2, …, K: total number of the selected weights g1(t): the gradient to update the weights a1, b1, c1: positive cont. Ming-Feng Yeh

On-line Learning Process The last weights obtained in the off-line learning are used as initial ones in the on-line learning. Updated rule: k: the time-delay of the system g2(t): the gradient to update the weights a2, b2, c2: positive cont. Ming-Feng Yeh

Off-line vs On-line In the off-line learning process, the teacher signal u*(t) is generated by a certain control law, e.g., PID control law and human experts. u*(t) is utilized in order to determine the initial weights in the on-line learning of the CMAC. In the on-line learning process, the teaching signal u*(t) can not be obtained. The desired reference model output ym(t) is introduced, and the on-line learning is performed so that the system output y(t) approaches to ym(t). Ming-Feng Yeh

Self-organized Structure A new CMAC is generated for a new command signal The initial weights includes in the new CMAC are set by employing the linear interpolation to the trained weights included two CMACs which are nearest for the new command signal The CMAC whose command signal is nearest for the new one is eliminated. Ming-Feng Yeh

4. Experimental Results Air pressure control system Control object: regulate the air pressure y to any desired values by manipulating the control value angle u. In order to obtain u*(t), PID control law is employed for this control system. Ming-Feng Yeh

Control Result Conventional PID control Off-line learning (20 iterations) Ming-Feng Yeh

Control Result (cont.) On-line learning (after 5 more iterations) Unspecified command signal Ming-Feng Yeh