1. 1 Collaborative Filtering: Latent Variable Model LIU Tengfei Computer Science and Engineering Department April 13, 2011

2. 2 Outline  Overview of CF approaches  Model based approach-latent variable model Probabilistic latent semantic analysis (PLSA) Other latent variable models  Summary

3. 3 Overview of CF Approaches

4. 4  CF Categories Memory-based CF  Conduct certain forms of nearest neighbor search in order to predict the rating for particular use-item pair. Model-based CF  Train a compact model that explains the given data so that ratings could be predicted via the model.

5. 5 Outline  Overview of CF approaches  Model based approach Probabilistic latent semantic analysis (PLSA) Other latent variable model  Summary

6. 6 Model based approach  Question: What is the shortcomings of memory based methods?  Reasons: Suboptimal solution problem Little knowledge learned from data Computationally expensive in local-neighbor search ……

7. 7 Probabilistic Latent Semantic Analysis  The Problem We want to predict the rating r that user u may assign to item i  Why latent variable model? Consider a simple case:  User x like/dislike item y “ because of ” some reason  The reason can not be observed, but may exist  We introduce a latent variable to model it

8. 8 Probabilistic Latent Semantic Analysis  Question: what rating that user u is likely to give to item i? Can we describe it with probability? The probability that the rating a user give to an item is decomposed into a sum of products.  z is the latent variable Probability that class z (can be seen as community in CF) would assign score r to item i. Mixing proportion

9. 9 Probabilistic Latent Semantic Analysis  Intuitive Graph Representation

10. 10 Probabilistic Latent Semantic Analysis  Model as a Gaussian distribution  Mixing proportion can be modeled as a categorical distribution

11. 11 Probabilistic Latent Semantic Analysis  To make predictions, we compute the expected rating

12. 12 Probabilistic Latent Semantic Analysis  Model parameters can be learnt by maximizing the following log likelihood of observed data  This can be readily solved using EM algorithm

13. 13 Probabilistic Latent Semantic Analysis  Question 1: how to learn the model parameters by EM algorithm?  Question 2: how to understand EM algorithm?

14. 14 Other latent variable models  Probabilistic latent preference analysis (PLPA)  Reference: NN. Liu et al, Probabilistic Latent Preference Analysis for Collaborative Filtering, CIKM ’ 09

15. 15 Outline  Overview of CF approaches  Model based approach-latent variable model Probabilistic latent semantic analysis (PLSA) Other latent variable models  Summary

16. 16 Summary  CF is popular  Memory based method Advantages and shortcomings  Model based method Latent variable model  Probabilistic latent semantic analysis  Other latent variable models

17. 17 Summary Thank you ! Questions?

18. 18 Reference  Thomas Hofmann, Collaborative Filtering via Gaussian Probabilistic Latent Semantic Analysis, SIGIR 2003  Thomas Hofmann, Latent Semantic Models for Collaborative Filtering, In ACM Transactions on Information Systems, 2004  Abhinandan Das et al, Google News Personalization: Scalable Online Collaborative Filtering, WWW 2007  NN. Liu et al, Probabilistic Latent Preference Analysis for Collaborative Filtering, CIKM ’ 09  Xiaoyuan Su et al, A Survey of collaborative Filtering Techniques, 2009