1. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Dr Uwe Aickelin A Recommender System based on the Immune Network The Recommendation Problem The AIS Approach Algorithm Walkthrough Results and Discussion

2. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems The Recommendation Problem “What movies would you predict/recommend?” Prediction What rating would I give this film? Prediction quality can be assessed by absolute error Recommendation Give me a ‘top 10’ list of films I might like Recommendation quality can be assessed by a ranking ‘discordance’ metric

3. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems vs InnateAcquired vs Cell Mediated Humoral T Cell (CD-4, Helper) Binds to MHC-antigen complex Secretes cytokines to help… How do we protect the body against infection? (Antigens) B Cell Secretes Antibody which binds to antigen and recruits phagocytes (innate) T Cell (CD-8, Killer) Kills cell (viruses) The Biological Immune System

4. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems EachMovie database User profiles (3M votes 70k users) User Profile : set of tuples {movie, rating} Me: My user profile Neighbour: User profile of someone else Similarity metric: Correlation score between user profiles Neighbourhood: Group of neighbours similar to me Recommendations: generated from neighbourhood The Recommendation Problem

5. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems EachMovie database User profiles User Profile : set of tuples {movie, rating} Me: My user profile Neighbour: User profile of someone else Similarity metric: Correlation score between user profiles Neighbourhood: Group of neighbours similar to me Recommendations: generated from neighbourhood The AIS Approach Antigen Antibody Antibody – Antigen BindingAntibody – Antibody Binding Group of antibodies similar to antigen and dissimilar to other antibodies Stimulation Suppression

6. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Start with empty AIS Encode target user as an antigen Ag WHILE (AIS not full) && (More users) DO Add next user as an antibody Ab IF (AIS at full size) Iterate AIS FI OD Generate recommendations from AIS The AIS Algorithm Ab 4 Ab 1 Ab 3 Ab 2 Ag

7. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Algorithm walkthrough: Encoding DATABASE u 1 ={(m 1,v 11 ),(m 2,v 12 ),(m 3,v 13 )} u 2 ={(m 1,v 21 ),(m 2,v 22 ),(m 3,v 23 ),(m 4,v 24 )} u 3 ={(m 1,v 31 ),(m 2,v 32 ),(m 4,v 34 )} u 4 ={(m 1,v 41 ),(m 4,v 44 )} u 5 ={(m 1,v 51 ),(m 2,v 52 ),(m 3,v 53 ), (m 4,v 54 )} We do not have user votes for every film We want to predict the vote of user u 4 on movie m 3 Suppose we have 5 users and 4 movies

8. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Algorithm walkthrough (1) Start with empty AIS DATABASE u 1, u 2, u 3, u 4, u 5 AIS Encode user for whom to make predictions as an antigen Ag DATABASE u 1, u 2, u 3, u 4, u 5 u4u4 AIS Ag

9. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Algorithm walkthrough (2) Add antibodies until AIS is full… Ab 1 DATABASE u 1, u 2, u 3, u 4, u 5 u1u1 AIS Ag Add next user as an antibody Ab 1 Add users 2 and 3 … DATABASE u 1, u 2, u 3, u 4, u 5 u 2,u 3 AIS Ag Ab 1 Ab 2 Ab 3

10. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Algorithm walkthrough (3) After some more iterations… the AIS has filled up: Table of matching Scores between Ab and Ag MS 14, MS 24, MS 34 Table of matching Scores between Antibodies MS 12 = CorrelCoef(Ab 1, Ab 2 ) MS 13 = CorrelCoef(Ab 1, Ab 3 ) MS 23 = CorrelCoef(Ab 2, Ab 3 ) Ab 2 Ab 3 Ab 1 Ag

11. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Algorithm walkthrough (4) AIS is now at full size so begin iterations… Ab 1 Ab 2 Ag Ab 1 Ab 2 Ab 3 AIS Ag AIS Notice that antibody 3 has been eliminated. Calculate new CONCENTRATION for each Ab, considering interactions with Ag (STIMULATION) and other Ab (SUPPRESSION)

12. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Algorithm walkthrough (5) If AIS not yet full and more users available, repeat. Otherwise: GENERATE RECOMMENDATION from CONCENTRATION and ANTIGEN Correlation. Recommendation for user u 4 on movie m 3 will be highly based on vote on m 3 of user u 2 AIS Ab 1 Ab 2 Ab 1 Ag Ab 2

13. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Tested against EachMovie database (15000 users, 1628 films) Results compared to standard method (Pearson k-nearest neighbours) Prediction : Results of same quality Recommendation: Improved results, 4 out of 5 films correct versus 3 out of 5. Results

14. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems 1. Stimulation and suppression affect neighbourhood size and number of users looked at

15. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems 2. AIS matches Pearson for prediction

16. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems 3. AIS surpasses Pearson for Recommendation

17. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems General purpose recommendation tool (e.g. Bookmarks) Collaborative Filtering is a useful vehicle for examination of AIS dynamics: - Idiotypic effect for more varied population - Potential for distribution - Smaller neighbourhoods (vs computational cost) Wider applicability (e.g. online community formation) Evaluation

18. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Idiotypic effects alter nature of community How important is diversity? Are there other network effects that can be used? (hubs, routers etc) Distribution: the snowball effect What about interacting communities? Application areas: ad-hoc community formation, knowledge management, P2P routing… Speculation: online community formation

19. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Change detection (Checksums) ‘Self’ : files, network traffic, system calls Antibodies creation: positive vs negative selection Collaboration between different populations/sites Representation: binary string or symbolic (rules) Other IS features: activation thresholds (vs false positives) co-stimulation (vs false positives) memory detectors (secondary response) MHC masks to cover ‘holes’ (similar to self) AIS for Security

20. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Example: Hofmeyr & Forrest 2000

21. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Evaluation Applied to network intrusion, virus detection… Good results on test systems BUT… Negative Selection doesn’t scale Inefficient to map entire non self universe Changes over time Appropriate representation of self Appropriate matching Primary response requires infection? AIS for Security

22. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Traditional Self - Non Self Distinction An immune response is triggered when the body encounters something foreign. The difference between self and non-self is learnt early in life. E.g. eliminate those T- and B-cells that react to self. Problems: No reaction to foreign bacteria in gut No reaction to food we eat The human body changes over its life Auto-immune diseases Tumours / Transplants

23. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems The Danger Theory Need for discrimination: What should be responded to? Respond to Danger not to “foreignness”. No need to attack everything that is foreign. Danger is measured by damage / distress signals. Advantages: Can take care of non-self but harmless Can take care of self but harmful

24. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Danger Model Conclusions Self-Nonself discrimination still useful. Nonself does not cause immune response. Danger Signals trigger immune response. A question of semantics? Can this model help us build an AIS for security applications? What would be ‘danger signals’?

25. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Discussion Uwe Aickelin: http://www.aickelin.com/ http://www.aickelin.com/ Steve Cayzer: http://www-uk.hpl.hp.co.uk/people/steve_cayzer/ http://www-uk.hpl.hp.co.uk/people/steve_cayzer/

26. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Additional Slides

27. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems AIS Models - Idiotypic Antibody Antigen Antibody Epitope Paratope Farmer et al 1986 Paratope/Epitopes Lock and Key Interchangeable? Behaviour Matching Idiotypic (Memory, auto- immune)

28. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Jerne’s Big Idea (1974) Idiotype: specificity of antibody (epitopes to which it will bind) Idiotope: An idiotypic epitope Evidence: Antibodies produced against antibodies of same species (cf individual) Antigen P1P1 I1I1 P2P2 I2I2 Idiotypic Set P3P3 I3I3 Anti-Idiotypic Set Internal Image of Antigen + - AIS Models - Idiotypic

29. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems In Words… The idiotypic network hypothesis (Jerne 1974) builds on the recognition that antibodies can match other antibodies as well as antigens. A group of antibodies, which match an antigen, may be matched by other antibodies which may in turn be matched by yet other antibodies. This stimulatory effect will set up activation chains or loops. Matched antibodies are suppressed, and this effect will encourage diversity In Formulae… AIS Models - Idiotypic

30. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems For N antibodies, n antigens. x i is the concentration of antibody i p and e stand for ‘paratope’ and ‘epitope.’ s is the matching threshold. G is a rectifier function which outputs 0 for all negative input. k is the allowable overlap AIS Models - Idiotypic

31. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Simple user comparisons (Pearson, cosine, k- Nearest Neighbour) Problems: Sparsity, curse of dimensionality Memory vs Model based approaches Transformative and Transitive functions Default votes, Content based, Learning algorithms Challenge of distribution (vs centralization) Recommendation Approaches

32. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems System Description: Encoding Users are represented as a set of tuples which represent their votes:

33. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems We use the Pearson correlation measure System Description: Matching The measure is amended as follows

34. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Parameters: Matching Function

35. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Parameters: AIS

36. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems We predict a rating by using a weighted average over the neighbourhood of a user: System Description: Prediction

37. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Parameters: Prediction

38. click to edit master text Click to edit Master text styles Second level Third level Fourth level Fifth level Artificial Immune Systems Mean Absolute Error System Description: Evaluation Precision vsRecall Variance