1. Expertise Finding for Question Answering (QA) Services March 5, 2014March 5, 2014 Department of Knowledge Service EngineeringDepartment of Knowledge Service Engineering Prof. Jae-Gil LeeProf. Jae-Gil Lee

2. The First Wednesday Multidisciplinary Forum23/5/2014 Table of Contents Community-based Question Answering (CQA) Services Background and Motivation Methodology Overview Evaluation Results Social Search Engines for Location-Based Questions Background and Motivation System Architecture and User Interface Evaluation Results

3. The First Wednesday Multidisciplinary Forum33/5/2014 Question Answering (QA) Services QA services are good at  Recently updated information  Personalized information  Advice & opinion [Budalakoti et al., 2010] Questions Answers Knowledge Base Search Experts

4. The First Wednesday Multidisciplinary Forum43/5/2014 Community-based Question Answering (CQA) Services Naver Knowledge-In Yahoo! Answers 50,000 questions per day 160,000 questions per day

5. The First Wednesday Multidisciplinary Forum53/5/2014 Motivation of Our Study Most contributions (i.e., answers) in CQA services are made by a small number of heavy users Recently-joined users are prone to leave CQA services very soon Only 8.4% of answerers remained after a year Making the long tail stay longer before they leave is of prime importance towards the success of the services

6. The First Wednesday Multidisciplinary Forum63/5/2014 Problem Setting To whom does the service provider need to pay special attention?  Recently-joined (i.e., light) users who are likely to become contributive (i.e., heavy) users Goal: estimating the likelihood of a light user becoming a heavy user (mainly by his/her expertise) Challenges: lack of information about the light user 어장관리 ?

7. The First Wednesday Multidisciplinary Forum73/5/2014 Intuition behind Our Methodology A person’s active vocabulary reveals his/her knowledge Vocabulary has sharable characteristics so that domain-specific words are repeatedly used by expert answerers SSD NAND ECC RAM Device Memory Computer NAND ECC RAM SSD Operation Data Drive Q&A 1 by Answerer 1 Q&A 2 by Answerer 2 Domain-Specific Vocabularies Common Vocabularies Level Difference Sharable Characteristics

8. The First Wednesday Multidisciplinary Forum83/5/2014 Estimated Expertise Heavy Users Words Light Users The more expert a user is, the higher the level of words he/she used is.

9. The First Wednesday Multidisciplinary Forum93/5/2014 Availability Simply measuring the number of a user’s answers with their importance proportional to their recency

10. The First Wednesday Multidisciplinary Forum103/5/2014 Answer Affordance Being defined as the likelihood of a light user becoming a heavy user if he/she is treated specially Considering both expertise and availability

11. The First Wednesday Multidisciplinary Forum113/5/2014 Data Set Collected from Naver Knowledge-In (KiN, 지식인 ) Spanning ten years (from Sept. 2002 to Aug. 2012) Including two categories: Computers and Travel Computers: factual information, Travel: subjective opinions The entropy was used for measuring the expertise of a user, working well especially for the categories where factual expertise is primarily sought after [Adamic et al., 2008] Statistics ComputersTravel # of answers3,926,794585,316 # of words191,502232,076 # of users228,36944,866

12. The First Wednesday Multidisciplinary Forum123/5/2014 Evaluation Setting (1/2) Finding the top-k users by Affordance() for light users  our methodology Retrieving the top-k directory experts managed by KiN  competitor Measuring the two measures for the next one month User availability: the ratio of the number of the top-k users who appeared on the day to the total number of users who appeared on that day Answer possession: the ratio of the number of the answers posted by the top-k users on the day to the total number of answers posted on that day

13. The First Wednesday Multidisciplinary Forum133/5/2014 Evaluation Setting (2/2) Ten year period Sept. 2002 July 2011 July 2012 Aug. 2012 Used for deriving the word levelsUsed for finding top-k experts by our methodology Picked up the top-k directory experts managed by KiN Monitored the user availability and answer possession

14. The First Wednesday Multidisciplinary Forum143/5/2014 The result of the answer possession The result of the user availability (a) Computers (b) Travel

15. See the paper for the technical details. Sung, J., Lee, J., and Lee, U., "Booming Up the Long Tails: Discovering Potentially Contributive Users in Community-Based Question Answering Services," In Proc. 7th Int'l AAAI Conf. on Weblogs and Social Media (ICWSM), Cambridge, Massachusetts, July 2013. This paper received the Best Paper Award at AAAI ICWSM-13.

16. The First Wednesday Multidisciplinary Forum163/5/2014 Table of Contents Community-based Question Answering (CQA) Services Background and Motivation Methodology Overview Evaluation Results Social Search Engines for Location-Based Questions Background and Motivation System Architecture and User Interface Evaluation Results

17. The First Wednesday Multidisciplinary Forum173/5/2014 Social Search (1/2) A new paradigm of knowledge acquisition that relies on the people of a questioner’s social network

18. The First Wednesday Multidisciplinary Forum183/5/2014 Social Search (2/2) If you want to get some opinions or advices from your online friends, what do you do? Not knowing whom to ask Knowing whom to ask Taking advantage of both approaches Social Search

19. The First Wednesday Multidisciplinary Forum193/5/2014 Location-Based Questions Informally defined as “search for a business or place of interest that is tied to a specific geographical location”[Amin et al., 2009] Very popular especially in mobile search and typically subjective Mobile search is estimated to comprise 10%∼30% of all searches  About 9∼10% of the queries from Yahoo! mobile search and over 15% of 1 million Google queries from PDA devices were identified as location-based questions In a set of location-based questions, 63% of them were non- factual, and the remaining 37% of them were factual  Mobile social search is the best way to process location-based questions

20. The First Wednesday Multidisciplinary Forum203/5/2014 Glaucus: A Social Search Engine for Location- Based Questions 1. Asking a question to Glaucus 2. Selecting proper experts 3. Routing the question to the experts 4. Returning an answer to the questioner 5. (Optional) Rating the answer Glaucus Social Search Engine User Database 1: Query Users 2: Selected Experts 3: Query Answer 4: Answer 5: Feedback Crawling Questioner

21. The First Wednesday Multidisciplinary Forum213/5/2014 User Interface An Android app has been developed and is under (closed) beta testing Questioner Answerer

22. The First Wednesday Multidisciplinary Forum223/5/2014 Data Collection Being able to collect who visited where and when on geosocial networking services such as Foursquare Users check-in to a venue and also may leave a tip Our crawler collects such information upon user approval

23. The First Wednesday Multidisciplinary Forum233/5/2014 Expert Finding Venue Location Category Time Misc. Venue Location Category Time Misc. Location Aspect Model Questioner Question Other Users Online Friend? Similarity Calculation Score Top-k

24. The First Wednesday Multidisciplinary Forum243/5/2014 Evaluation Setting Collected check-in’s and tips from Foursquare (foursquare.com) Confined to the places in the Gangnam District Ranging from April 2012 to December 2012 Statistics VariableValue # of users9,163 # of places (venues)1,220 # of check-in’s243,114 # of tips40,248

25. The First Wednesday Multidisciplinary Forum253/5/2014 Evaluation Results SocialTelescopeAardvarkGlaucus DCG Set 1 Set 2Set 3 3.94 3.99 4.07 6.61 6.31 6.68 8.25 8.82 7.78 2.37 1.97 Qualification of the Experts: Two human judges investigated the profiles of the experts selected by the three systems for 30 questions (distributed to 3 sets) and gave a score in 3 scales. Quality of the Answers: Two human judges examined the quality of the answers ― both from experts and non-experts ― and gave a score in 3 scales.

26. See the paper for the technical details. Choy, M., Lee, J., Gweon, G., and Kim, D., "Glaucus: Exploiting the Wisdom of Crowds for Location-Based Queries in Mobile Environments," In Proc. 8th Int'l AAAI Conf. on Weblogs and Social Media (ICWSM), accepted. To appear in June 2014.

27. Thank you very much! Any Questions? E-mail: jaegil@kaist.ac.kr Homepage: http://dm.kaist.ac.kr/jaegil@kaist.ac.krhttp://dm.kaist.ac.kr/