1. Alternative Representations for Artificial Immune Systems
James Marshall and Tim Kovacs
University of Bristol

2. Representational Bias
Representation introduces an inductive bias to a learning system
Affects what is learnable and how quickly it can be learned
Best representation may differ from task to task

3. Choosing representations
Q: how do we choose the best representation for our task?
A: we need:
Domain knowledge
Knowledge about representations
This work attempts to build up the latter

4. Learning representations
We can also ask the machine to learn which representation to use:
For a class of problems (easier)
On the fly for a particular problem (harder)
We’ve done some preliminary work where the learner has a choice of just two representations
But here we only study the two representations

5. Our learning task
Concept learning of Boolean functions from labeled examples
Examples are presented as binary strings of length n
Learn condition-action rules as either:
Hyperspheres (from the AIS literature)
Hyperplanes (from the LCS literature)

6. Hyperspheres Rule = condition, radius r, action
Condition: a binary string specifies a point
Rule matches if instance is within hamming distance r of condition
0000:2 =>0 matches 0000, 0001,
0010,0100,1000,0011,0110,1100,1001,1010,0101
Generality of rule is specified by radius
Some use in AIS literature
Related but more complex representations also used

7. Hyperplanes Rule = condition, action
Condition = ternary string from {0,1,#}
Rule matches if bitwise comparison succeeds: # in rules matches either 0 or 1 in string
00## matches 0000, 0001, 0010, 0011 only
Generality of rule is specified by number of #s
Widely used with Learning Classifier Systems
Related to AIS

8. Key difference
Generalisation of planes is conditioned on dimensions, spheres are not
0000 r =1 matches 0000,0001,0010,0100,1000
Planes select which dimensions to generalise over (#s), spheres only how many (radius)
This seems a fundamental difference: they belong to different classes of representation
We don’t know what classes there are!

9. Expressivity Spheres treat all dimensions equivalently
Greater need for exception rules than with planes
More spheres needed to express a given Boolean function, on average, than with planes
Size of minimal representation in a given representation correlates strongly with ease of learning
Consequently may be harder to learn with spheres

10. Example: suitability for multiplexers
A long-used testbed for LCS
Instances consist of address bits and data bits
Instance class given by value of addressed data bit
Defined for strings of length 3, 6, 11, 20…

11. Proofs It’s easy to show: 100% accurate planes exist for all n
100% accurate spheres exist for no n
Pairs of spheres in a default/exception relationship can achieve 100% accuracy
We expect planes to be much better for multiplexers

12. Enumeration of 11 mux 11 multiplexer has only 2048 instances
Let’s enumerate all the minimally general spheres (r=1) and planes (d = 2)
Compute accurate of each as percentage of matched instances belonging to majority class of matched instances

13. Accuracy of Enumerated Planes and Spheres
Planes Spheres
100% accurate classifiers are the mode for 2-d planes while no 100% accurate spheres exist.
Planes are also much better empirically (using XCS)

14. Information content Planes have more information: bits
where n is string length
Spheres have only
where x is the maximum radius
Again, spheres are less expressive
but also less complex, which suggests they’ll be easier to learn

15. Search space size Planes: Spheres: of radii between 0 and n inclusive
There are more planes than spheres for a given problem
Should be harder to find best solution with planes
But it should fit the target better (or need fewer rules)

16. Size of planes and spheres
Size = number of instances matched
Planes: where d is the dimensionality (number of #s) of the plane
This is independent of n
Spheres: where r is the radius
This depends on string length n
Consequence: as we increase string length spheres grow, planes don’t

17. Size of planes and spheres
Spheres grow more when we increment radius than planes do when we add a #
Spaces of spheres is less fine-grained

18. Size of planes and spheres

19. Example: adding sensors
We have a robot with n binary sensors
Suppose we add another
We have twice as many states
To make new input irrelevant we need twice as many spheres but same number of planes
Difficult to make concept conditional on new input using spheres, easy using planes
This must complicate feature construction/reduction using spheres

20. Conclusions
Spheres:
not well-suited to concepts which depend on particular dimensions
better when dimensions are equivalent
e.g. majority vote
whenever Hamming Distance is a useful distance metric!
have greater need for exception rules

21. Other approaches We’ve analysed representations:
syntactically: size, number of rules
on Boolean functions: how many rules are needed?
Other theoretical approaches:
COLT-type bounds
Schema theory, Markov chain analysis
More realistic tasks:
Encoding schemes used in the AIS literature
Higher-cardinality tasks
Empirical results on real-world problems

22. Non-binary problems Interval representations typical
Intervals reps. are between planes and spheres
Planes specify either 1 value or ignore dimension
Spheres with a centre in each dimension are intervals

23. Future work
Allow learning system to select which representation to use
Initial results good
Key issue: how efficient is this?
Fast: useful for one-off learning
Slow: useful for classes of problems?