1. Neuro-Fyzzy Methods for Modeling and Identification Part 2 : Examples Presented by: Ali Maleki

2. Presentation Agenda Introduction Tools for Fuzzy Identification and Modelling MATLAB – Fuzzy Logic Toolbox – ANFIS Fuzzy Modeling and Identification Toolbox Examples : Example 1: Hair Dryer Example 2: Static Function Example 3: Neutralization Process

3. Introduction Control Systems Competion Environment requirements Energy and material costs Demand for robust, fault-tolerant systems Extra needs for Effective process modeling techniques Conventional modeling? Lack precise and formal knowledg about the system Strongly nonlinear behavior, High degree of uncertainty, Time varying characteristics

4. Introduction (cont.) Solution: Neuro-fuzzy modeling A powerful tool which can facilitate the effective development of models by combining information from different source: Empirical models Heuristics Data Neuro-fuzzy models Describe systems by means of fuzzy if-then rules Represented in a network structure Apply algorithms from the area of Neural Networks

5. Zero-order TS fuzzy model Typical membership function Input-output equation Introduction (cont.)

6. System identification steps: Structure identification Parameter estimation choice of the model’s structure determines the flexibility of the model in the approximation of (unknown) systems model with a rich structure can approximate more complicated functions, but, will have worse generalization properties Good generalization means that a model fitted to one data set will also perform well on another data set from the same process.

7. Tools for Fuzzy Modeling & Identification Fuzzy Modelling and Identification Toolbox Develped by R. Babuska http://Lcewww.et.tudelft.nl/~Babuska Installation Version 3.03, April 2001 Fuzzy Logic Toolbox for MATLAB www.Mathworks.com/products/fuzzylogic Version 2.1.3, June 2004

8. Fuzzy Modeling and Identification Toolbox

9. Fuzzy Modeling and Identification Toolbox (cont.)

10. FMSIM function: Simulate a MIMO input-output fuzzy model Fuzzy Modeling and Identification Toolbox (cont.)

11. Structure of FM (fmstruct function): Fuzzy Modeling and Identification Toolbox (cont.)

12. Structure of FM (fmstruct function): Fuzzy Modeling and Identification Toolbox (cont.)

13. plotmfs function: Plot membership functions rms function: Root mean square between two signals vaf function: Percentile variance accounted for (VAF) between two signals Fuzzy Modeling and Identification Toolbox (cont.)

14. MATLAB - Fuzzy Toolbox anfis function: Training routine for Sugeno-type FIS anfisedit function: Open the ANFIS Editor GUI genfis1 function: Generate an FIS structure from data without data clustering genfis2 function: Generate an FIS structure from data using subtractive clustering

15. MATLAB - Fuzzy Toolbox (cont.) ANFIS Editor GUI

16. Example 1 : Hair Dryer (Fuzzy Logic Toolbox) Nonlinear dynamical system identification With use of ANFIS function in the Fuzzy Logic Toolbox Data set was obtained from a laboratory device called Feedback's Process Trainer PT 326, L. Ljung, "System Identification, Theory for the User", Prentice- Hall, 1987 – Chapter 17 The device's function is like a hair dryer: Input u(k) : Voltage over the mesh of resistor wires Output y(k) : Outlet air temperature

17. Input u(k) : Binary random signal shifting between 3.41 and 6.41 V Output y(k) : Outlet air temperature Sampling Time : 0.08 sec Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

18. Linear ARX model: y(k)+a 1 *y(k-1)+...+a m *y(k-m)=b 1 *u(k-d)+...+b n *u(k-d-n+1) a i and b j are linear parameters to be determined by least-squares methods This structure is exactly specified by three integers [m, n, d] Remind: System Identification : structure selection + parameter estimation Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

19. Remove the means from the data The data set was divided into a training set and a checking set Training data set : (k = 1 to 300) Checking data set : (k = 301 to 600) An exhaustive search was performed to find the best combination [m, n, d] each of the integer is allowed to changed from 1 to 10 independently Run through all different models: V = arxstruc(ze, zv, struc(1:10, 1:10, 1:10)); Find the best model: nn = selstruc(V, 0); The best ARX model : [m, n, d] = [5, 10, 2] Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

20. Training RMSE = 0.1122Checking RMSE = 0.0749 Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

21. Advantage of ARX model: Rapid model structure selection Rapid parameter identification The performance in the above plots appear to be satisfactory. If a better performance level is desired, we might want to resort to a nonlinear model. Neuro-fuzzy modeling approach, ANFIS Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

22. Use ANFIS for system identification: First step: input selection To determine which variables should be the input arguments to an ANFIS model. For simplicity, we suppose that there are 10 input candidates y(k-1), y(k-2), y(k-3), y(k-4), u(k-1), u(k-2), u(k-3), u(k-4), u(k-5), u(k-6) Two approaches for input selection: Sequential search Exhaustive search Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

23. Sequential search for input selection: can be done by the function seqsrch 10 + 9 + 8 =27 Selected inputs : y(k-1), u(k-3), and u(k-4) Training RMSE = 0.0609 Checking RMSE = 0.0604. Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

24. Exhaustive search on all possible combinations of the input candidates Can be done by function exhsrch We want to selects 3 inputs from 10 candidates, therefore, the total number of ANFIS models is Fortunately, for dynamical system identification, we do know that the inputs should not come from either of the following two sets of input candidates exclusively: Y = {y(k-1), y(k-2), y(k-3), y(k-4)} U = {u(k-1), u(k-2), u(k-3), u(k-4), u(k-5), u(k-6)} A reasonable guess: two inputs from Y and one from U Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

25. Exhaustive search Selected inputs : y(k-1), y(k-2), u(k-3) Training RMSE = 0.0474 Checking RMSE = 0.0485 Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

26. ARX model Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

27. ANFIS model Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

28. Comparision If fast modeling is the goal, then ARX is the right choice, If precision is the utmost concern, then we can go for ANFIS that is designed for nonlinear modeling and higher precision Example 1 : Hair Dryer (Fuzzy Logic Toolbox) (cont.)

29. Example 2 - Static Function ANFIS model with linear consequent function Number of rules: five rules Construction of initial model: Gustafson-Kessel algorithm Fit of the function with initial model – local models - membership functions

30. Example 2 - Static Function (cont.) this initial model can easily be interpreted in terms of the local behavior It is reasonably accurate (RMS= 0.0258) ANFIS method, 100 learning epochs anfis function of the MATLAB Fuzzy Logic Toolbox Fit of the function with fine-tuned model, local models, membership functions

31. RMS error is about 23 times better than the initial model Initial modelFine-tuned model RMS error = 0.0258RMS error = 0.0011 Example 2 - Static Function (cont.)

32. after learning, the local models are much further from the true local description of the function Initial modelFine-tuned model Fine-tuned model are thus less accurate in describing the system locally Example 2 - Static Function (cont.)

33. Example 3 – pH Neutralization Process Neutralization tank Effluent stream Acid Buffer Base Influent streams Neutralization tank pH in the tank Acid flowrate = cte Buffer flowrate = cte Base stream flowrate

34. Example 3 - pH Neutralization Process (cont.) Identification and validation data sets: Simulating the model by Hall and Seborg for random change of the influent base stream flow rate N = 499 samples with the sampling time of 15 s. The process is approximated as a first–order discrete-time NARX model

35. Membership functions Befor training After training Example 3 - pH Neutralization Process (cont.)

36. Rules: Initial Rules: Fine Tuned Rules: after 1000 epochs of hybrid learning using the ANFIS function of the MATLAB Fuzzy Logic Toolbox Example 3 - pH Neutralization Process (cont.)

37. Overtraining Problem: Comparision of RMS ERROR befor and after training Prediction befor and after training Example 3 - pH Neutralization Process (cont.)

38. References [1] Robert Babuska, “Neuro-Fuzzy Methods for Modeling and Identification”, Recent Advances in Intelligent Paradigms and Application, Springer- Verilag, 2002 [2] Robert Babuska, “Fuzzy Modeling and Identification Toolbox User’s Guide - For Use with MATLAB”, 1998. [3] MathWorks Inc., “Fuzzy Logic Toolbox – Users Guide – Version 2”, 2004. [4] L. Ljung, “System Identification, Theory for the User”, Prentice-Hall, 1987.

39. THANK YOU VERY MUCH For your Attention Presented by: Ali Maleki

40. Introduction - appendix Types of fuzzy models: (depending on the structure of if-then rules) Mamdani Model IF D 1 is low and D 2 is high THEN D is medium Takagi-Sugeno Model IF D 1 is low and D 2 is high THEN D=k (zero- order) IF D 1 is low and D 2 is high THEN D=0.7D 1 +0.2D 2 +0.1 (first- order)