1. School of Information University of Michigan Expertise Networks in Online Communities: Structure and Algorithms Lada Adamic joint work with Jun Zhang and Mark Ackerman School of Information, University of Michigan NetSci May 24 th, 2007

2. Have you sought knowledge here? Knows Knowledge iN

3. Oozing out knowledge Knowledge In ``Knowledge search is like oozing out knowledge in human brains to the Internet. People who know something better than others can present their know-how, skills or knowledge'' NHN CEO Chae Hwi-young Largest search engine in Korea - 70% of search (Google: 2%) Comprehensive portal – integrated news, blogs, ‘knowledge search’ Knowledge-In had 60 million questions and answers as of Feb 2007 popular: why fingernails grow faster than toenails how fast a fly can fly why seagulls sit in the same direction

4. Ranking the contributors LevelRange of points Lowlife0-99 Commoner100-500 Citizen501-3000 Middle class3001-7000 Expert7001-15000 Hero15001-35000 Professional35001-65000 Superhuman65001-100000 (Gods)> 100000 Knowledge In

5. Culture of generosity “(It is) the next generation of search… (it) is a kind of collective brain -- a searchable database of everything everyone knows. It's a culture of generosity. The fundamental belief is that everyone knows something.” -- Eckart Walther (Yahoo Research) 90 million users worldwide

6. Limitations of Current Systems The Response Time Gap The Expertise Gap Difficult to infer reliability of answers Automatically ranking expertise may be helpful.

7. Related work Analysis of online communities NetScan (Smith, Fisher, et al. at Microsoft) Social network analysis (LiveJournal, blog communities) Motivations of online participation (Lakhani & Hippel) Graph-based ranking algorithms PageRank, HITS, etc. Expertise sharing studies Expertise recommenders ContactFinder (Krulwich et al.), Answer Garden (Ackerman) Small Blue (Lin) Automatic evaluating expertise levels Using different text resources (Kautz, et al, and a lot of others) Using email networks (Campbell et al.)

8. Overview Social network analysis Constructing Expertise Networks Finding meaningful metrics Empirical evaluation of ranking algorithms Human Rating vs. Algorithmic Ranking Simulation Understanding underlying dynamics Predicting performance of ranking algorithms in yet-unobserved community dynamics

9. Java Forum 87 sub-forums 1,438,053 messages community expertise network constructed: 196,191 users 796,270 edges

10. Constructing a community expertise network A BC Thread 1 Thread 2 Thread 1: Large Data, binary search or hashtable? user A Re: Large... user B Re: Large... user C Thread 2: Binary file with ASCII data user A Re: File with... user C A B C 1 1 A BC 1 2 A BC 1/2 1+1//2 A B C 0.9 0.1 unweighted weighted by # threads weighted by shared credit weighted with backflow

11. Not Everyone Asks/Replies Core: A strongly connected component, in which everyone asks and answers IN: Mostly askers. OUT: Mostly Helpers The Web is a bow tieThe JavaForum network is an uneven bow tie

12. Uneven participation number of people one replied to ‘answer people’ may reply to thousands of others ‘question people’ are also uneven in the number of repliers to their posts, but to a lesser extent

13. Who Answers Whom Degree-degree correlations between asker and helper

14. Summary of JavaForum Network Different types of participation Askers, ask-help-er, helpers Different levels of participation top helpers, others Who replied to whom Top repliers answer questions for everyone Other helpers help those with somewhat lower expertise

15. Relating network structure to Java expertise Human-rated expertise levels 2 raters 135 JavaForum users with >= 10 posts inter-rater agreement (  = 0.74,  = 0.83) for evaluation of algorithms, omit users where raters disagreed by more than 1 level (  = 0.80,  = 0.83) LCategoryDescription 5Top Java expertKnows the core Java theory and related advanced topics deeply. 4Java professionalCan answer all or most of Java concept questions. Also knows one or some sub topics very well, 3Java userKnows advanced Java concepts. Can program relatively well. 2Java learnerKnows basic concepts and can program, but is not good at advanced topics of Java. 1NewbieJust starting to learn java.

16. Structural Info Based Expertise Ranking Metrics # replies posted (# answers) experts can answer many questions # people replied to (# indegree) experts can answer questions from many different people z-score for the 2 above (observed –  )/  experts are above the mean in the above two metrics PageRank replying to people who reply to people higher level experts can answer mid-level experts HITS experts answer questions by people whose questions other experts have answered hubs point to good authorities

17. automated vs. human ratings # answers human rating automated ranking z # answers HITS authority indegree z indegree PageRank

18. Algorithm Rankings vs. Human Ratings simple local measures do as well (and better) than measures incorporating the wider network topology Top K Kendall’s  Spearman’s  # answers z-score # answers indegree z-score indegree PageRank HITS authority 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

19. Modeling community structure to explain algorithm performance Control Parameters: Distribution of expertise Who asks questions most often? Who answers questions most often? best expert most likely someone a bit more expert ExpertiseNet Simulator

20. Simulating probability of expertise pairing suppose: expertise is uniformly distributed probability of posing a question is inversely proportional to expertise p ij = probability a user with expertise j replies to a user with expertise i 2 models: ‘best’ preferred‘just better’ preferred j>i

21. Visualization Best “preferred”just better

22. Degree correlation profiles best preferred (simulation)just better (simulation) Java Forum Network asker indegree

23. The Simulation of JavaForum Settings: Distribution of expertise (skewed) Who asks questions most often? (novices) Who answers questions? (best expert most likely) Results Similar bow tie structure Similar degree distribution Slightly different correlation profiles Similar algorithm performance PageRank does not outperform simpler degree-based metrics

24. Different ranking algorithms perform differently In the ‘just better’ model, a node is correctly ranked by PageRank but not by HITS

25. It can tell us when to use which algorithms Preferred Helper: ‘ just better ’ Preferred Helper: ‘ best available ’

26. Summary Expertise Networks have interesting characteristics A set of useful metrics Ranking algorithms are affected by network structures Simulation as an analysis tool There are rich design opportunities Find experts with the help of structural information (and content analysis) Predict good answers Re-order questions/answers to match expertise working paper: “Expertise-Level based Interface Personalization for Online Help-seeking Communities”

27. Looking at diverse sets of question-answer forums (Yahoo Answers) Expertise across different topics Using explicit ratings for evaluation of automated expertise identification & incorporation into algorithms (battling spam) Users’ expertise change over time Continually developing and evaluating our systems built upon these findings Future Work cars & transportation maintenance & repairs beauty & style hair

28. for more info ExpertiseRank algorithms and evaluations Zhang, J., Ackerman, M.S., Adamic, L., Expertise Networks in Online Communities: Structure and Algorithms, WWW’07 Simulations of expertise networks Zhang, J., Ackerman, M.S., Adamic, L., CommunityNetSimulator: Using Simulations to Study Online Community Network Formation and Implications, C&T2007 Jun Zhang junzh@umich.edujunzh@umich.edu http://www-personal.si.umich.edu/~junzh Mark Ackerman ackerm@eecs.umich.eduackerm@eecs.umich.edu http://www.eecs.umich.edu/~ackerm/ Lada Adamic ladamic@umich.eduladamic@umich.edu http://www-personal.umich.edu/~ladamic NSF (IRI-9702904)

29. ads Jun Zhang is graduating and on the job market (junzh@umich.edu)junzh@umich.edu Lada is looking for a postdoc (ladamic@umich.edu)ladamic@umich.edu

30. Simplest models do not capture all ‘local’ interactions