Intelligent Data Analysis Introduction to Fuzzy Logic Michael R. Berthold

Fuzzy Logic Fuzzy Sets Fuzzy Numbers Fuzzy Operators Fuzzy Rules Fuzzy Inference

Types of Uncertainty Stochastic uncertainty Linguistic uncertainty example: rolling a dice Linguistic uncertainty examples : low price, tall people, young age Informational uncertainty example : credit worthiness, honesty Stochastic uncertainty an event occurs with a given probability lexical or linguistic uncertainty imprecise description of an object or concept 3. Informational uncertainty uncertainty caused by missing or incomplete information

Classical Sets young = { x  P | age(x)  20 } characteristic function: young(x) = 1 : age(x)  20 0 : age(x) > 20 young(x) A=“young” 1 x [years]

Fuzzy Sets Fuzzy Logic Classical Logic Element x belongs to set A with a certain degree of membership: (x)[0,1] Classical Logic Element x belongs to set A or it does not: (x){0,1} A(x) A(x) A=“young” A=“young” 1 1 x [years] x [years]

Fuzzy Set Definition : Fuzzy Set A = {(x, A(x)) : x  X,  A(x)  [0,1]} a universe of discourse X : 0  x  100 a membership function A : X  [0,1] A(x) A=“young” 1 =0.8 Membership function also called characteristic function x [years] x=23

Definitions : Support of a fuzzy set A Core of a fuzzy set A supp(A) = { x  X :  A(x) > 0 } Core of a fuzzy set A core(A) = { x  X :  A(x) = 1 } =0.6 -cut of a fuzzy set A A  = { x  X :  A(x)  } (x) 1 Alpha cuts usual for implementation in a computer Normalized fuzzy set A sup x  X  A(x) = 1 a b c d x Uni-modal fuzzy set only one “peak”

Types of Membership Functions Trapezoid: <a,b,c,d> Gaussian: N(m,s) (x) (x) 1 1 s a b c d x m x Triangular: <a,b,b,d> Singleton: (a,1) and (b,0.5) (x) (x) 1 1 a b d x a b x

Membership functions core: CA:={x | A(x)=1} = [b,c] A(x) 1 -cut: A:={x | A(x)= }  a b c d x support: SA:={x | A(x)>0} = (a,d)

Fuzzy Numbers Fuzzy Numbers have restricted fuzzy sets: X = 1 their core is a point: 1 x : (x)=1 left and right flank of  are monotonically increasing, respectively decreasing Common choices are triangular MS-functions  easy internal representation (a,b,c) (x) 1 a b c x

Computing with Fuzzy Numbers Addition: A+B(x) = max{A(y), B(z) | x=y+z} (x) A(x) B(x) A+B(x) 1 x Multiplication: AB(x) = max{A(y), B(z) | x=yz} x (x) 1 A(x) B(x) AB(x)

The Extension Principle Assume a fuzzy set A and a function f: How does the fuzzy set f(A) look like? For arbitrary functions f: f(A)(y) = max{A(x) | y=f(x)} f x A(x) y f(A)(y) f x A(x) y f(A)(y) max

Operators on Fuzzy Sets Union Intersection AB(x)=max{A(x),B(x)} AB(x)=min{A(x),B(x)} A(x) B(x) A(x) B(x) 1 1 x x AB(x)=min{1,A(x)+B(x)} AB(x)=A(x)  B(x) A(x) B(x) A(x) B(x) 1 1 x x

T-Norms and T-Conorms A function T : [0,1]x[0,1] => [0,1] is called a t-norm iff for any u,v,w  [0,1] T(u,1) = u 1 as unit u  v  T(u,w)  T(v,w) monotonicity T(u,v) = T(v,u) commutativity T(u,T(v,w)) = T(T(u,v),w) associativity A t-norm is strict if it is strictly increasing in [0,1]x[0,1] Bounded product Examples: min(u,v) minimum u•v algebraic product max(0,u+v-1) bounded product

T-Conorms or S-norms A function S : [0,1]x[0,1] => [0,1] is called a t-conorm or s-norm iff for any u,v,w  [0,1] S(u,0) = u 0 as unit u  v  S(u,w)  S(v,w) monotonicity S(u,v) = S(v,u) commutativity S(u,S(v,w)) = (S(u,v),w) associativity A s-norm is strict if it is strictly increasing in [0,1]x[0,1] Bounded sum Examples: max(u,v) maximum u+v-u•v algebraic sum min(1,u+v) bounded sum

Complement Negation: A(x)= 1 - A(x) Classical law does not always hold: AA(x)  1 AA(x)  0 Example : A(x) = 0.6 A(x) = 1 - A(x) = 0.4 AA(x) = max(0.6,0.4) = 0.6  1 AA(x) = min(0.6,0.4) = 0.4  0

T-norms and S-norms S(u,v) = 1 – T(1-u,1-v) : De Morgan Law T(u,v) = 1 – S(1-u,1-v) max(u,v) = 1 – min(1-u,1-v) u•v = 1 - (1-u)+(1-v) + (1-u)•(1-v) max(0,u+v-1) = 1 - min(1,1-u+1-v)

{ { Operator - Spectrum S-norms (T-conorms) T-norms drastic product bounded product algebraic product min max algebraic sum bounded sum drastic sum        u if v=1 drastic product : T(u,v)= v if u=1 0 otherwise { u if v=0 drastic sum : T(u,v)= v if u=0 1 otherwise {

Cylindrical Extension Imbed a lower dimensional fuzzy set in a higher dimensional space fuzzy set A with universe of discourse X1 and MSF mA and additional universe X2 Definition: CX1xX2 A is called the cylindrical extension of A to the universe X1x X2 iff (x1,x2)  X1x X2 : m CX1xX2 A(x1,x2) = mA (x1)

Cartesian Product fuzzy set A with universe of discourse X1 and MSF mA and fuzzy set B with universe of discourse X2 and MSF mB and Definition: AxB is called the cartesian product of A and B to the universe X1x X2 iff (x1,x2)  X1x X2 : m AxB (x1,x2) = min (mA (x1), mB (x2)) Notice: min can be replaced by any other t-norm

{ Fuzzy Relations classical relation fuzzy relation R  X x Y defined by mR(x,y) = 1 iff (x,y)  R 0 iff (x,y)  R { | fuzzy relation R  X x Y defined by mR(x,y)  [0,1] mR(x,y) describes to which degree x and y are related It can also be interpreted as the truth value of the proposition x R y

Fuzzy Relations Example: X = { rainy, cloudy, sunny } Y = { sun bathing, bicycling, camping, reading } X/Y sun bathing bicycling camping reading rainy cloudy sunny 0.0 0.2 0.0 1.0 0.0 0.8 0.3 0.3 1.0 0.2 0.7 0.0

Fuzzy Sets & Linguistic Variables A linguistic variable combines several fuzzy sets. example: linguistic variable : temperature linguistics terms (fuzzy sets) : { cold, warm, hot } x [C] (x) 1 cold warm hot 60 20

Fuzzy Rules causal dependencies can be expressed in form of if-then-rules general form: if <antecedent> then <consequence> example: if temperature is cold and oil is cheap then heating is high linguistic variables linguistic values/terms (fuzzy sets)

Fuzzy Rulebase Heating Temperature : cold warm hot Oil price: cheap normal expensive if temperature is cold and oil price is low then heating is high high high medium high medium low medium low low if temperature is hot and oil price is normal then heating is low

Fuzzy Knowledge-Base (x) cold warm hot 1 x [C] 20 60 Fuzzy Data-Base: Definition of linguistic input and output variables Definition of fuzzy membership functions (x) cold warm hot 1 x [C] 20 60 Fuzzy Rule-Base: List of fuzzy rules if temperature is cold and oil price is cheap then heating is high ….

Fuzzy Inference 1. Fuzzification Determine degree of membership for each term of an input variable : temperature : t=15 C oilprice : p=$13/barrel cold(t)=0.5 cheap(p)=0.3 1 1 0.5 0.3 t p 15C $13/barrel if temperatur is cold ... and oil is cheap ...

Fuzzy Inference 2. Combine the terms in one degree of fulfillment for the entire antecedent using a fuzzy AND: min-operator t 1 cold(t)=0.5 if temperatur is cold ... 15C p cheap(p)=0.3 and oil is cheap ... $13/barrel 0.5 0.3 ante = min{cold(t), cheap(p)} = min{0.5,0.3} = 0.3

Fuzzy Inference 3. Inference step: Apply the degree of membership of the antecedent to the consequent of the rule using a t-norm: min or prod operator consequent(h) ante =0.3 high(h) 1 ... min-inference: cons. = min{ante , high } h ... then heating is high ante =0.3 consequent(h) high(h) 1 ... prod-inference: cons. = ante • high h

Fuzzy Inference 4. Aggregate all the rules consequents using the max-operator for union ... then heating is high ... then heating is medium ... then heating is low 1 h

Defuzzification 5. Determine crisp value from output membership function for example using “Center of Gravity”-method: consequent(h) COG 1 h 73 Center of singletons is computationally easier Center of singletons defuzzification: mi = degree of membership fuzzy set i Ai = area of fuzzy set i ci = center of gravity of fuzzy set i h = Si mi • Ai • ci Si mi • Ai

Schema of a Fuzzy Decision Fuzzification Inference Defuzzification rule-base if temp is cold then valve is open cold warm hot open half close cold =0.7 0.7 0.7 if temp is warm then valve is half 0.2 0.2 warm =0.2 t v measured temperature if temp is hot then valve is close crisp output for valve-setting hot =0.0

Example Fuzzy Rule Base 1. If (temperature is cold) and (oilprice is normal) then (heating is high) 2. If (temperature is cold) and (oilprice is expensive) then (heating is medium) 3. If (temperature is warm) and (oilprice is cheap) then (heating is high) 4. If (temperature is warm) and (oilprice is normal) then (heating is medium) 5. If (temperature is cold) and (oilprice is cheap) then (heating is high) 6. If (temperature is warm) and (oilprice is expensive) then (heating is low) 7. If (temperature is hot) and (oilprice is cheap) then (heating is medium) 8. If (temperature is hot) and (oilprice is normal) then (heating is low) 9. If (temperature is hot) and (oilprice is expensive) then (heating is low)

Types of Fuzzy Rules Mamdani rule: Outputs are fuzzy sets B if X1 is A1 and X2 is A2 … Xn is An then Y is B Takagi-Sugeno-Kang (TSK)- rule: Outputs are functions f(xi) if X1 is A1 and X2 is A2 …. Xn is An then Y=f(x1,x2,..xn) 1. Order TSK- rule: Outputs are linear functions of xi if X1 is A1 and X2 is A2…Xn is An then Y=a0 + a1x1 … +anxn

Learning Fuzzy Rule Sets from Data Adjustable parameters in a Fuzzy Rule Set: Number, Distribution, and shape of membership functions for each attribute. consequent for each rule constructive or adaptive?

Dividing the Input Space Grid based Individual membership functions

Grid based Approaches equidistant grid, assign output of each tile from nearest example (Wang&Mendel 91) divide grid at point of largest error (Higgins&Goodman, 93) initialize grid equidistant, move membership functions similar to BackProp (Nauck&Kruse, 91-95)

Individual Rule Approaches Derive Fuzzy Rules from Radial Basis Function Network, using Gaussian membership functions (several authors) Extract Rules directly from the data (Simpson 91, Berthold&Huber95) Extract Rules from decision trees.

Finetuning Fuzzy Rule Sets Heuristics that move membership functions to minimize error (similar to Backprop) Convert Rules into Neural Network, then train Network and afterwards extract modified rules. Use Genetic Algorithms to fine tune rules...

Resources Web Fuzzy & Neuro-Fuzzy Repository at the University of Southampton http://www.isis.ecs.soton.ac.uk/resources/nfinfo NAFIPS: www.nafips.org Journals Fuzzy Sets & Systems IEEE Transactions on Fuzzy Systems Journal of Approximative Reasoning Intelligent Data Analysis (electronic) Southampton has links to Software, Journals,

Resources Software / Shareware Xfuzzy (fuzzy system design, verification, synthesis) NEFCON (Matlab Simulink) UNFUZZY (C / C++) FOOL & FOX (Xwindows) Commercial Matlab Fuzzy Logic Toolbox FuzzyTECH by Inform DataEngine by MIT SieFuzzy by SIEMENS Xfuzzy : Sevilla, Spain (World Championships) NEFCON : Braunschweig &Magdeburg Prof. Kruse UNFUZZY : Colombia FOOL&FOX Oldenburg