Fuzzy Interpretation of Discretized Intervals Author: Dr. Xindong Wu IEEE TRANSACTIONS ON FUZZY SYSTEM VOL. 7, NO. 6, DECEMBER 1999 Presented by: Gong Chen

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusions Answers for Final Exam

Concepts Review Induction: Generalize rules from training data Deduction: Apply generalized rules to testing data Three possible results of Deduction: –Single match –No match –Multiple match

Concepts Review Discretization of Continuous domains –Continuous numerical domains can be discretized into intervals –The discretized intervals can be treated as nominal values

Concepts Review Using Information Gain Heuristic for Discretization: (employed by HCV) –x = (x i + x i+1 )/2 for (i = 1, …, n-1) –x is a possible cut point if x i and x i+1 are of different classes –Use IGH to find best x –Recursively split on left and right –Stop recursive splitting when some criteria is met

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam

Overview Training Data Discretizaion induction rules Testing Data Deduction No match Single match Multiple match Fuzzy Borders

Outline Concepts Review Overview Problem Solution Several Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam

Problem Discretization of continuous domains does not always fit accurate interpretation! Recall, using Info Gain, --a kind of heuristic measure applying in training data, cannot accurately fit “data in real world”. Example

Problem Heuristic 1(e.g. Information Gain) Heuristic 2(e.g. Gain Ratio) young 49 old young 50 old 49.49

Problem Suppose after induction, we just get one rule: If (age=old) then Class=MORE_EXPERIENCE According to Heuristic 2, Instance(age=49.49) No match!

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam

Solution More safe way to describe age=49.49 is to say: To some degree, it is young; To some degree, it is old. Rather than using one assertion that definitely tells it is young or old. Thus, to some degree, it can get its rule and classification result other than no match. –No match  Single match or multiple match with some degree This is so-called fuzzy match!

Solution “Fuzziness is a type of deterministic uncertainty. It describes the event class ambiguity.” “Fuzziness works when there are the outcomes that belong to several event classes at the same time but to different degrees.” “Fuzziness measures the degree to which an event occurs.” –Jim Bezdek, Didier Dubois, Bart osko, Henri Prade

Solution “to some degree”? –Membership function describes “degree” –Membership function tells you to what degree, an event belongs to one class. –Membership function calculates this degree. Three widely used membership functions are employed by HCV. –Linear –Polynomial –Arctan

Solution Linear membership function x left x right l sl k = 1/2sl; a = -kx left + ½; b = kx right + ½ lin left (x) = kx + a lin right (x) = -kx + b lin(x) = MAX(0, MIN(1,lin left (x),lin right (x))) S: is user-specified parameter. e.g. 0.1 indicates the interval spreads out into adjacent intervals for 10% of its original length at each end.

Solution Polynomial Membership Function—using more smooth curve function instead of linear function. Arctan Membership Function Experimental results shows that no significant difference between three kinds of functions—so Polynomial Membership Function is chosen.

Solution poly side (x) = a side x 3 + b side x 2 + c side x + d side a side = 1/(4(ls) 3 ) b side = -3a side x side side  {left,right} c side = 3a side (x side 2 - (ls) 2 ) d side = -a(x side 3 -3x side (ls) 2 + 2(ls) 3 ) poly left (x),if x left -ls  x  x left + ls poly(x) = poly right (x),if x right -ls  x  x right +ls 1,if x left +ls  x  x right -ls 0,otherwise To what degree, x belongs to one interval

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam Problems

Related Techniques –No match Largest Class –Assign all no match examples to the largest class, the default class –Multiple match Largest Rule –Assign examples to the rules which cover the largest number of examples Estimate of Probability –Fuzzy borders can bring multiple match--conflicts, so hybrid method is desired for the whole progress

Related Techniques Estimate of Probability # of e.g.s in training set covered by conj The probability of e belongs to class c i Conj1 and Conj2 are two rules supporting e belongs to Ci

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam Problems

Algorithms Design in HCV HCV(Large) –No match: Largest Class –Multiple match: Largest Rule HCV(Fuzzy) –No match: Fuzzy Match –Multiple match: Fuzzy Match HCV(Hybrid) –No match: Fuzzy Match –Multiple match: Estimate of Probability

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam Problems

Experimental Results Data: –17 datasets from UCI Machine Learning Repository –Why select these: 1) Numerical data 2) Situations where no rules clearly apply Test conditions –68 parameters in HCV are all default except deduction strategy –Parameters for C4.5 and NewID are adopted as the one recommended by respective inventors

Experimental Results DatasetHCVHCV (large)HCVC4.5 NewID (hybrid)(fuzzy)(R 8)(R 5) Anneal98.00%93.00% 95.00%93.00%81.00% Bupa57.60%55.90% 71.20%61.00%73.00% Cleveland %68.10%73.60%71.40%76.90%67.00% Cleveland %56.00%52.70%51.60%56.00%47.30%  CRX 82.50%72.50%82.00%83.00%80.00%79.00% Glass (w/out ID)72.30%60.00% 71.50%64.60%66.00% Hungarian %85.00% 81.20%80.00%78.00% Hypothroid97.80%86.30%96.30%99.40% 92.00% Imports %59.30%61.00% 67.80%61.00% Ionosphere88.00%81.20% 86.30%85.50%82.00% Labor Neg76.50% 82.40% 65.00% Pima73.90%69.10% 73.50%75.50%73.00% Swiss % 97.00% Swiss %25.00%28.10%40.60%31.20%22.00% Va % 77.50%70.40%77.00% Va %25.40%29.60%31.00%26.80%20.00% Wine90.40%76.90% 90.40%90.00%90.40%

Experimental Results Predictive accuracy –HCV (hybrid) outperforms others in 9 datasets –HCV (large) 3 datasets –HCV (fuzzy) 2 datasets –C4.5 (R 8) 7 datasets –C4.5 (R 5) 6 datasets –NewID 3 datasets –HCV (hybrid)clearly and significantly outperforms other interpretation techniques (in HCV) for datasets with numerical data in “no match” and “multiple match” cases. C4.5 and NewID are included for reference, not for extensive comparison.

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam Problems

Conclusion Fuzziness is strongly domain dependent, HCV allows users to specify their own intervals and fuzzy functions. –An important direction to take with specific domains Fuzzy Borders design combined with probability estimation achieve better results in term of predicative accuracy. –Applicable to other machine learning and data mining algorithms

Outline Concepts Review Overview Problem Solution Related Techniques Algorithms Design in HCV Experimental Results Conclusion Answers for Final Exam Problems

Q1:When doing deduction on real world data, what are the three possible cases for each test example? –Single match –No match –Multiple match Q2: Of the three cases during deduction, which ones do the HCV hybrid interpretation algorithm use fuzzy borders to classify? –No match Q3: In the Hybrid interpretation algorithm used in HCV, –when are sharp borders set up? “Sharp borders are set up as usual during induction” –when are fuzzy border defined? In deduction, “only in the no match case, fuzzy borders are set up in order to find a rule which is closest to the test example in question”

Thank You!