1. Gradient Descent Rule Tuning See pp. 207-210 in text book

2. Rules Consider a rule base with M rules, r th rule has the form IF x 1 is T r,1 AND … AND x n is T r,n THEN y is y r (or y is y r + other stuff) TSK fuzzy system has mathematical form

3. Membership function parameters –Center, right-width, left-width –Consequent parameters 3 level (layer) structure of f(x) –Level (layer) 1: For each rule Compute all membership values for each term, compute product, store as z r –Level (layer) 2: Compute product of membership values and consequents, sum: n Sum membership values: d –Level (layer) 3: Compute quotient: f = n/d

4. Rule parameters Membership function parameters –Center, right-width, left-width –Consequent parameters Why not s, z and triangular membership functions? Why Gaussian membership functions?

5. Gradient Descent Choose parameters to minimize the error Corresponds to a blind person descending a mountain by finding the steepest descending slope and moving in that direction Slope is determined by differentiation (computing the “gradient”) Chain rule helps tremendously.

6. Gradient Descent Math Consider a sequence of input/output measurements: (x 0 p, y 0 p ) As each input/output measurement pair arrives (and before the next input/output measurement pair arrives), we want to adjust our model parameters to reduce the error e p = [f(x 0 p )-y 0 p ] 2 /2 Dropping the sub-and-super-scripts e = [f(x)-y] 2 /2 The gradient descent algorithm for any vector-valued parameter s is

7. Apply to:

8. For ybar Given x and y Modify for beta Modify for xbar Modify for sigma

9. Gradient Descent For a generic parameter For ybar, see previous slide For xbar For sigma Abstraction saves work.

10. One LV Example FL System LV X: Term set: Negative, Zero, Positive 3 rules Antecedent matrix, Consequent matrix Gaussian membership functions Super membership function Fuzzy function parameters TSK fuzzy function Gradient Descent parameter tuning

11. One LV Example FL System LV X: Negative5, Zero, Positive5 3 rules –If x is Negative5 then y is 25 –If x is Zero then y is 0 –If x is Positive5 then y is 25 Antecedent matrix and consequent matrix

12. One LV Example FL System LV X: Negative5, Zero, Positive5 Gaussian membership functions

13. One LV Example FL System Super membership function

14. One LV Example FL System TSK fuzzy function Gradient Descent parameter tuning

15. One LV Example FL System TSK fuzzy function, Gradient Descent parameter tuning ybar

16. One LV Example FL System TSK fuzzy function, Gradient Descent parameter tuning ybar Heart and soul of gradient descent algorithm to tune ybar using experimental data. Engineers derive these expressions. Computers compute with these expressions, often iteratively, to improve designs. Note interplay of theory and real-world data.

17. One LV Example FL System TSK fuzzy function, Gradient Descent parameter tuning xbar

18. One LV Example FL System TSK fuzzy function, Gradient Descent parameter tuning xbar

19. One LV Example FL System TSK fuzzy function, Gradient Descent parameter tuning xbar

20. One LV Example FL System TSK fuzzy function, Gradient Descent parameter tuning xbar

21. One LV: Gradient Descent Summary

22. We are now ready to do gradient descent

23. Two LV Example FL System Temperature term set: Cold, Comfortable, Hot Humidity term set: Wet, Dry 6 rules Antecedent matrix, Consequent matrix Gaussian membership functions Super membership function Fuzzy function parameters TSK Fuzzy Function Gradient descent parameter tuning

24. Two LV Example FL System Temperature term set: Comfortable, Warm, Hot Humidity term set: Wet, Dry 6 rules –If T is Comfortable and H is Wet then HI is –If T is Comfortable and H is Dry then HI is –If T is Warm and H is Wet then HI is –If T is Warm and H is Dry then HI is –If T is Hot and H is Wet then HI is –If T is Hot and H is Dry then HI is

25. Two LV Example FL System: Matrices –If T is Comfortable and H is Wet then HI is –If T is Comfortable and H is Dry then HI is –If T is Warm and H is Wet then HI is –If T is Warm and H is Dry then HI is –If T is Hot and H is Wet then HI is –If T is Hot and H is Dry then HI is

26. Two LV Example FL System Temperature term set: Cold, Comfortable, Hot Humidity term set: Wet, Dry Gaussian membership functions Super membership function

27. Two LV Example FL System TSK Fuzzy Function Gradient descent parameter tuning

28. Two LV Example FL System Gradient descent parameter tuning