1. Tweetool (0. 1 100 version) Final Report Yilei Qian Computer Science University of Southern California qianyilei.usc@gmail.com A Twitter Recommend System based on Topic Modeling

2. Ideas Following too many points on Twitter Too many news every day Cannot find the interested and valued news Don’t know the name which user want to follow Need someone to recommend who to follow Need someone to recommend the hottest news Use topic modeling to re-rank all the user

3. Traditional Method

6. Topic Modeling

8. a topic model is a type of statistical model for discovering the abstract "topics" that occur in a collection of documents. Always used in natural language processing. Reference Papers: Steyvers,m. and Griffiths, T., “Probabilistic topic models,” Hand book of latent semantic analysis Blei, D.M and Ng, A.Y and Jordan, M.I, “Latent Dirichlet Allocation”, The Journal of Machine Learning Research 2003

9. Label based LDA Step: 1.Build the LDA Model 2.Train the model instance by train document 3.Run the LDA for all the data based on trained model instance Problem: 1.Punctuation marks. E.g. “”,.={}() … 2.Frequent words. E.g I, you…. 3.Other Noise

10. Result Generate

11. 13-Dimension Topics 1.Art & Design 2.Book 3.Business 4.Charity 5.Entertainment 6.Family 7.Fashion 8.Food & Drink 9.Health 10.Music 11.News 12.Science & Technology 13.Sports

12. Languages & Tools Web UI: HTML + AJAX(Unfinished) +CSS(unfinished)+Twitter REST API Android UI: Java, Android 2.1(unfinished) Server Side: Java 1.6, Servlet 2.0, Spring 3.0, Hibernate 3.3 Twitter API: Twitter4j 2.2.1 (300 request per hour) Server: Tomcat 7.08 Database: MySQL 5.5 Data Package: JSON Develop Platform: Eclipse 3.4 Total code lines: 2000(+) + 2421 + 462 = 5000(+) Subversion: http://tweetool-yilei.googlecode.com/svn/trunk/tweetool-yilei-read-only

13. Architecture DB Twitter fetch LLDA Tweetool Hibernate DAO Work Flow Servlets Work Flow Mobile Device HTML APPLICATION CONTEXT

14. Distributed Crawler & Computing

15. Problems(endless T_T) 1.High noise in topic model Few words, Odd marks, Abbreviation 2.Unfamiliar with Twitter API, A lot of bugs 3.Transaction Problems 4.The Ugly UI 5.Poor performance 6.Don’t have enough time. Many functions are unfinished 7.Tweetool system should be reconstructed !!! Environment: 7000+Users 22,0000+Tweets

16. Future Work 1.Try to finish it 2.Debug 3.Build a better train file 4.Add feedback function 5.Better topics classification

17. Web UI (Design Version)

18. Android UI Function Button Function Button Function Button Function Button Titile Main MenuNews Menu Title News