Learning Classifier Systems

Learning Classifier Systems (LCS) The system has three layers: – A performance system that interacts with environment, – An apportionment of credit algorithm that rates rules as to usefulness, – A rule discovery algorithm that generates plausible new rules to replace less useful rules.

Performance System Cycles Message is posted in the message list from the input interface. Each rule is matched against the message list All matching rules compete to post in the next message list via bidding process; winning rule posts in the new message list The output interface checks the new message and produces an effector action. The new message list replaces the previous one. Repeat.

Overview of LCS

Rule format Rule – Condition = {0,1,#} k – Action = message to be posted in the message list – Strength = rule’s usefulness to the system

kindearsnum. of legssmartscream runawaykiss Example (Wolf or Grandmother?) teeth 1011# # Wolf GrandMa Encoding

Matching [M] ConditionActionStrength #1### #0# Message List 0100 ConditionActionStrength #1### #011###50 0#0# ### # [N]

Bidding Process [M] Rule id ConditionActionStrength r1#1### r30#0# β = 0.2 Bid(r1) = 0.2 × ¼ × 100 = 5 Bid(r3) = 0.2 × ½ × 100 = 10 r3 posts its message in the new message list. Bid(R,t) = β × specificity(R) × Strength(R,t) Specificity(R)= number of non # / k

Credit assignment: Bucket Brigade r3 Bucket 10 r5 Bucket 150 coupled Environment executed Reward 200

r3 Bucket 10 r5 Bucket 150 Environment Reward 200 Credit assignment: Bucket Brigade

Genetic Algorithms Fitness = rule strength Parents: Strong classifiers (best, roulette wheel, etc.) Mutation: alter parts of parent’s string Crossover: exchange parts of parents’ strings Offspring replaces a weak rule.

Genetic Algorithms (cont.) ## Parent 1 Parent ## Crossover point ## Parent 1 Parent ## Crossover Mutation

Maze Environment A Environment Message List 40 5 f N 5 (1,2) GF ConditionActionStrengt h # >0 # # # #GF1000 # <0 # # # # ∧ TL TL1000 # <0 # # # # ∧ TR TR1000 (Signal smell-ahead bump heading score location)

References A Mathematical framework for Studying Learning in Classifier Systems, John H. Holland, Phsyca D, Vol 2, No 1-3, 1986, pp A Mathematical framework for Studying Learning in Classifier Systems A First Order Logic Classifier System, Drew Mellor Gecco ’05 A First Order Logic Classifier System