1. Learning Classifier Systems (Introduction) Muhammad Iqbal Evolutionary Computation Research Group School of Engineering and Computer Science Victoria University of Wellington New Zealand

2. Outline Examples Classification Problems Rules Format Overview of LCS Detailed XCS Process 2

3. 3-bit Boolean Classification Problem 3 ABC Y 0000 0010 0101 0111 1000 1011 1100 1111 0 0 #0 1 # 11 If A = 0 and B = 0, then Y = 0 If A = 0 and B = 1, then Y = 1 If A = 1 and C = 0, then Y = 0 If A = 1 and C = 1, then Y = 1 0 1 #1 1 # 00 3 Condition Action 0 # #1 0 1 11 Over-general Rule Over-fitted Rule Optimal Rules

4. 4 1 1 1 0 1010101 1 1 0 0 1010101 1 0 1 0 0110011 1 0 0 00110011 0 1 1 00001111 0 1 0 0 000 1111 0 0 1 11111111 0 0 0 00 000000 000000 100100 010010 110110 001001 101101 011011 111111 11###1 : 1 11###0 : 0 10##1# : 1 10##0# : 0 01#1## : 1 01#0## : 0 001### : 1 000### : 0 A B C FEDFED 6-bit Boolean Classification Problem

5. 3-bit Real-Valued Classification Problem 5 ABC Y 53.6202 109.8524 1512.3341 78.7152 ….. [3 7] [1.0 20.6] [13 25]2 If 3≤A ≤ 7 and 1.0 ≤ B ≤ 20.6 and 13 ≤ C ≤ 25, then Y = 2 All input attributes can be normalized, say, between 0.0 and 1.0. 5

6. Different Rich Encoding Schemes 6 Condition Action {0, 1, #} n Genetic Programming like Trees Constant Numeric Values {0, 1, #} n Finite State Machines Numeric Intervals Artificial Neural Networks etc

7. LCS Overview 7

8. Learning Classifier System (LCS) Population condition : action C 1 A 1 C 2 A 2 C 3 A 3... C N A N Evolutionary Computation Component rule selection, reproduction, mutation, recombination, and deletion Machine Learning Component rule evaluation, and action decision ENVIRONMENT problem instance action feedback 1 0 0 0 1 : 0 0 1 0 1 # : 1 1 # # 0 # : 1 # # 1 # 0 : 0 In LCS the ‘#’ sign, known as ‘don’t care’ symbol, can be either 0 or 1. 0 1 1 0 0 1 0 0 0 1 : 0 0 1 0 1 # : 1 1 # # 0 # : 1 # # 1 # 0 : 0 The XCS classifier system evolves a set of maximally general and accurate classifier rules that collectively solve the problem. 8 0 0

9. XCS Details 9

10. Rules in XCS 10 if condition then action; with certain attributes Attribute Description p Prediction: an estimate of the payoff that the classifier will receive if its action is selected. ε Prediction Error: which estimates the error between the classifier’s prediction and the received payoff. F Fitness: computed as an inverse function of the prediction error. exp Experience: which is a count of the number of times the classifier has been updated. n Numerosity: which is a count of the number of copies of each unique classifier. ts Time Stamp: keeps the time-step of the last occurrence of a GA in an action set to which this classifier belonged. as Action Set Size: which estimates the average size of the action sets this classifier has belonged to.

11. Source: Wilson, XCS tutorial ((43*99)+(27*3))/102 Action: 00 Action: 01 11

12. Applying GA in Action Set 0 0 1 0 1 0 conditionAF 0 0 1 # # 0190 0 # # 082 0 0 # 1 0 1150 0 # 1 0 1 0120 1 0 0 # 1 1043 0 0 1 0 # #170 conditionAF 0 0 1 # # 0190 0 # 1 0 1 0120 0 0 1 0 # #170 conditionAF 0 0 1 # # 0190 0 0 1 0 # #170 conditionAF 0 0 1 # # 0190 0 0 1 0 # #170 conditionAF 0 0 1 0 # 018 0 0 1 # # #18 conditionAF 0 # 1 0 # 008 # 0 1 # 1 #18 conditionAF 0 # 1 0 # 008 # 0 1 # 1 #18 ( (90+70)/2 ) × 0.1 population action set selected parents reproduced children crossed over children mutated children (niche mutation) final children subsumption deletion 12 if it is a well experienced and accurate classifier rule.

13. Thank You All! 13

14. Updating Procedures in XCS 14

15. Parameter List for XCS 15 Sr. No. ParameterDescription 1NPopulation size. 2βLearning rate for prediction, prediction error, and fitness updates. 3γDiscount factor in multistep problems. 4θ GA Threshold for GA application in the action set. 5ε0ε0 Threshold error in prediction under which a classifier is considered to be accurate. 6αControls the degree of decline in accuracy if the classifier is inaccurate. 7χProbability of crossover per invocation of the GA. 8μProbability of mutation per allele in an offspring. 9νFitness exponent. 10θ del Experience threshold for classifier deletion 11δFraction of mean fitness for deletion. 12θ sub Classifier experience threshold for subsumption 13P#P# Probability of a ‘#’ at an allele position in the condition of a classifier 14p I, ε I, and F I Prediction, prediction error, and fitness assigned to each classifier at the start.