1. Learning Classifier Systems

2. Learning Classifier Systems (LCS) The system has three layers: – A performance system that interacts with environment, – An apportionment of credit algorithm that rates rules as to usefulness, – A rule discovery algorithm that generates plausible new rules to replace less useful rules.

3. Performance System Cycles Message is posted in the message list from the input interface. Each rule is matched against the message list All matching rules compete to post in the next message list via bidding process; winning rule posts in the new message list The output interface checks the new message and produces an effector action. The new message list replaces the previous one. Repeat.

4. Overview of LCS

5. Rule format Rule – Condition = {0,1,#} k – Action = message to be posted in the message list – Strength = rule’s usefulness to the system

6. kindearsnum. of legssmartscream runawaykiss Example (Wolf or Grandmother?) teeth 1011#1 10 0100#0 01 1 1 Wolf GrandMa Encoding

7. Matching [M] ConditionActionStrength #1###010100 0#0#0011100 Message List 0100 ConditionActionStrength #1###010100 1#011###50 0#0#0011100 1###10101000 101#01111000 [N]

8. Bidding Process [M] Rule id ConditionActionStrength r1#1###010100 r30#0#0011100 β = 0.2 Bid(r1) = 0.2 × ¼ × 100 = 5 Bid(r3) = 0.2 × ½ × 100 = 10 r3 posts its message in the new message list. Bid(R,t) = β × specificity(R) × Strength(R,t) Specificity(R)= number of non # / k

9. Credit assignment: Bucket Brigade r3 Bucket 10 r5 Bucket 150 coupled Environment executed Reward 200

10. r3 Bucket 10 r5 Bucket 150 Environment Reward 200 Credit assignment: Bucket Brigade

11. Genetic Algorithms Fitness = rule strength Parents: Strong classifiers (best, roulette wheel, etc.) Mutation: alter parts of parent’s string Crossover: exchange parts of parents’ strings Offspring replaces a weak rule.

12. Genetic Algorithms (cont.) 001011## 10100100 Parent 1 Parent 2 00101100 101001## Crossover point 001011## 10100100 Parent 1 Parent 2 001010## 10100100 Crossover Mutation

13. Maze Environment A Environment Message List 40 5 f N 5 (1,2) GF ConditionActionStrengt h # >0 # # # #GF1000 # <0 # # # # ∧ TL TL1000 # <0 # # # # ∧ TR TR1000 (Signal smell-ahead bump heading score location)

14. References A Mathematical framework for Studying Learning in Classifier Systems, John H. Holland, Phsyca D, Vol 2, No 1-3, 1986, pp. 307-317 A Mathematical framework for Studying Learning in Classifier Systems A First Order Logic Classifier System, Drew Mellor Gecco ’05 A First Order Logic Classifier System