1. Fuzzy Genetics-based Machine Learning Algorithms Presented by Vahid Jazayeri

2. outline Introduction to fuzzy sets and fuzzy rules Introduction to fuzzy sets and fuzzy rules Introduction to GBML algorithms Introduction to GBML algorithms GBML Algorithms GBML Algorithms MichiganMichigan PittsburghPittsburgh HybridHybrid

3. Fuzzy Sets A Fuzzy set includes a small number of linguistic values A Fuzzy set includes a small number of linguistic values {Small, Large, Medium}{Small, Large, Medium} {Small, Medium-small, Medium, Large, Medium-large}{Small, Medium-small, Medium, Large, Medium-large} Fuzzy values are assigned to variables instead of exact numerical values Fuzzy values are assigned to variables instead of exact numerical values Numerical values should be normalized into [0,1] Numerical values should be normalized into [0,1]

4. Cont. Each number in [0,1] can be each one of fuzzy values but with a degree of membership Each number in [0,1] can be each one of fuzzy values but with a degree of membership Example Example 0 is 100% small but 0% large.0 is 100% small but 0% large. 0.125 is 50% small, 50% medium-small, but 0% other values0.125 is 50% small, 50% medium-small, but 0% other values

5. Fuzzy Rules Answers to classification problems Answers to classification problems m classes existm classes exist n variable existsn variable exists A fuzzy rule determines a class according to n variable and with a certainty gradeA fuzzy rule determines a class according to n variable and with a certainty grade R j is the j -th ruleR j is the j -th rule X i is i-th variableX i is i-th variable A ji is the fuzzy value of X i in the j-th RuleA ji is the fuzzy value of X i in the j-th Rule C j is the class of the rule R jC j is the class of the rule R j CF j is the certainty factor of the rule RjCF j is the certainty factor of the rule Rj

6. Cont. Don’t-care is also a fuzzy value Don’t-care is also a fuzzy value Every number is 100% Don’t-care Every number is 100% Don’t-care For shortening the size of rules. For shortening the size of rules. If in a rule Xi is Don’t-care it means we can omit that checking from the rule, so the size of rule is shortened. If in a rule Xi is Don’t-care it means we can omit that checking from the rule, so the size of rule is shortened.

7. GBML Algorithms Genetics-Based Machine Learning algorithms are a simulation of natural genetic phenomena. Genetics-Based Machine Learning algorithms are a simulation of natural genetic phenomena. Each individual is a string of some meaningful units (characters, bits, …). Each individual is a string of some meaningful units (characters, bits, …). A population of individuals generates new populations by the operators: A population of individuals generates new populations by the operators: CrossoverCrossover The mixing of two strings and generating two other ones The mixing of two strings and generating two other ones MutationMutation A lowly probable change in some characters of a string A lowly probable change in some characters of a string Old, weak individuals are replaced by new, stronger individuals Old, weak individuals are replaced by new, stronger individuals

8. Fuzzy GBML Algorithms Individuals are fuzzy rules Individuals are fuzzy rules Fuzzy rules are coded as strings of the following characters: Fuzzy rules are coded as strings of the following characters: 1: small1: small 2: medium-small2: medium-small 3: medium3: medium 4: medium-large4: medium-large 5: large5: large #: don’t-care#: don’t-care

9. Cont. Example: Example: Says that if x1 is small, x2 is large, x3 is don’t- care and x4 is medium-small Says that if x1 is small, x2 is large, x3 is don’t- care and x4 is medium-small There is no need to encoding the class or certainty grade There is no need to encoding the class or certainty grade They can be calculated dynamicallyThey can be calculated dynamically Each encoded rule has a fitness value determined by a fitness function for discriminating good rules Each encoded rule has a fitness value determined by a fitness function for discriminating good rules 15#2

10. Heuristic Rule Generation Procedure An encoded rule is prepared. An encoded rule is prepared. It is wanted to figure out the class and certainty grade (CF) of the rule Rj according to m experienced predefined training patterns It is wanted to figure out the class and certainty grade (CF) of the rule Rj according to m experienced predefined training patterns 1) Calculate the compatibility of each training pattern X p with R j : 1) Calculate the compatibility of each training pattern X p with R j : M A ji if the membership function of A jiM A ji if the membership function of A ji

11. Cont. 2) Calculate the sum of compatibility grades for each class 2) Calculate the sum of compatibility grades for each class Among c classes of the problem, find the class Cj for Rule Rj that has the maximum value of B Class h (Rj) Among c classes of the problem, find the class Cj for Rule Rj that has the maximum value of B Class h (Rj) The certainty grade of Rj can be calculated as follows: The certainty grade of Rj can be calculated as follows:

12. Fuzzy Reasoning A set of Fuzzy rules are available A set of Fuzzy rules are available A set of normalized values for n variables of a pattern is in hand A set of normalized values for n variables of a pattern is in hand It is wanted to classify the pattern It is wanted to classify the pattern The winner rule of the rule-set is the rule Rj for which: The winner rule of the rule-set is the rule Rj for which: If Rj is the winner rule then it says that the pattern is in class C j with the probability of CF j If Rj is the winner rule then it says that the pattern is in class C j with the probability of CF j

13. Michigan Algorithm 1. Generate a set of N fuzzy rules as the initial population of individuals 2. Evaluate each rule with a fitness function. 3. Select N/2 pair of individuals 4. Each pair generate two new individual as the result of crossover 5. Each newly-born individual may change with a small probability by mutation. Now N new individuals are prepared 6. Evaluate the new individuals 7. Replace the best new individuals (with the highest fitness values) with the old bad individuals of the previous population (those with the lowest fitness values)  the new population is now ready 8. If stopping condition is not satisfied return to step 2

14. Example

15. Evaluation (fitness function) 1. Set the fitness of each rule to zero 2. For each training pattern find the winner rule for that pattern. If the class of the winner rule is the same as the class of the pattern, then add one score to the fitness of that rule Else give a negative score to its fitness or do nothing.

16. Pittsburgh Algorithm The procedure of Pittsburgh algorithm is the same as that of Michigan but with partial differences The procedure of Pittsburgh algorithm is the same as that of Michigan but with partial differences Each individual in Michigan is a rule and the population is a collection of rules Each individual in Michigan is a rule and the population is a collection of rules Each individual in Pittsburgh is a rule-set and the population is a collection of rule-sets Each individual in Pittsburgh is a rule-set and the population is a collection of rule-sets

17. Cont. The encoding of each rule-set is the concatenation of its encoded rules The encoding of each rule-set is the concatenation of its encoded rules Below is an encoded set of 3 rules. Each rule of 3 variables Below is an encoded set of 3 rules. Each rule of 3 variables The fitness of each rule-set is determined by the sum of the fitness of its rules The fitness of each rule-set is determined by the sum of the fitness of its rules Final answer in Michigan is the whole population but here is the best rule-set with the highest fitness Final answer in Michigan is the whole population but here is the best rule-set with the highest fitness

18. Comparing two methods

19. Hybrid The basis of the algorithm is Pittsburgh The basis of the algorithm is Pittsburgh A mutation in Pittsburgh means a probable change in an individual, that is a change in a rule-set. A change in a rule-set can be done by executing Michigan on that rule-set, So using Michigan as mutation operator in Pittsburgh is called Hybrid Algorithm A mutation in Pittsburgh means a probable change in an individual, that is a change in a rule-set. A change in a rule-set can be done by executing Michigan on that rule-set, So using Michigan as mutation operator in Pittsburgh is called Hybrid Algorithm Michigan yields good rules but not necessarily good rule-sets. Michigan yields good rules but not necessarily good rule-sets. Pittsburgh yields good rule-sets but not necessarily good rules. Pittsburgh yields good rule-sets but not necessarily good rules. But hybrid algorithm enjoys goods rule-sets with good rules

20. resources H. Ishibuchi, T. Nakashima, Data Mining techniques, chapter 40, techniques and applications of genetic algorithm-based methods for desiring compact fuzzy classification systems, 1999. H. Ishibuchi, T. Nakashima, Data Mining techniques, chapter 40, techniques and applications of genetic algorithm-based methods for desiring compact fuzzy classification systems, 1999. H. Ishibuchi, T. Nakashima, and T. Kuroda, A Hybrid Fuzzy Genetics-based Machine Learning Algorithm: Hybridization of Michigan approach and Pittsburgh approach, 2000. H. Ishibuchi, T. Nakashima, and T. Kuroda, A Hybrid Fuzzy Genetics-based Machine Learning Algorithm: Hybridization of Michigan approach and Pittsburgh approach, 2000.

21. Thanks Any question (?)