1. Reputation Systems Guest Lecture Paul Resnick Associate Professor Univ. of Michigan School of Information presnick@umich.edu

2. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Learning Objectives n Understand –What a reputation system is –Theory about when and why it should work –Open research questions n Participate in design –Recognize situations when it might be helpful –Raise some of the difficult design challenges

3. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Outline n What is a reputation system? n Theory: when/why they should work n Empirical results n Design space n Case study: NPAssist recommender

4. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Definition n A Reputation System… –Collects –Distributes –Aggregates n …information about behavior

5. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Examples n BBB n Bizrate n eBay n Expertise sites –Epinions “top reviewers” –Slashdot karma system

6. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Why Reputation Systems n Interacting with strangers n Sellers (Exchange Partners) Vary –Skill –Effort –Ethics

7. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Other Trust-Inducing Mechanisms in E-commerce n Insurance n Escrow n Fraud Prosecution

8. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu How Reputation Systems Should Work n Information n Incentive n Self-selection

9. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Some Issues n Anonymity n Name changes n Name trades n Lending reputations n Eliciting evaluation n Honesty of evaluations

10. Anonymity Analysis

13. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu 1L Pseudonyms n Third-party issues pseudonyms –No cost –Not replaceable –Reveal name to third party –Don’t reveal mapping of name to pseudonym

15. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Empirical Results: eBay n Feedback is provided n It’s almost all positive n Reputations are informative n Reputation benefits –Effect on probability of sale –Effect on price

16. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Provision of Feedback n Negatives: paid but did not receive; seller cancelled; not as advertised; communication n Neutrals: slow shipping, not as advertised, communication

17. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Feedback Profiles of Buyers and Sellers

18. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Predicting Problematic Transactions n Logistic Regression f(0,0) = 1.91% f(100,0) =.18% f(100,3) =.53% N = 36233 Beginning Block Number 0. Initial Log Likelihood Function -2 Log Likelihood 2194.3468 -2 Log Likelihood 2075.420 Dependent Variable.. NEGNEUT ---------------------- Variables in the Equation ----------------------- Variable B S.E. Wald df Sig R Exp(B) LNNPOS.7712.1179 42.7907 1.0000.1363 2.1624 LNPOS -.5137.0475 116.8293 1.0000 -.2288.5983 Constant -3.9399.1291 931.3828 1.0000

19. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Predictive Value

20. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Some Recently Completed Work n Experiment: does reputation affect profit? –Many positives: Yes, but only a little (8.1%) –One or two negatives: No n Incentives for quality feedback provision –Can pay based on agreement among raters

21. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Studies Currently Underway n Feedback provision (empirical) –Reciprocation, altruism, and free riding n Dynamics: learning and selection (empirical) n Geography: trust and trustworthiness by state

22. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Design Space n Rating scales n Aggregation of ratings n Who rates? n Incentives for raters n Identification/Anonymity –Exchange partners –Evaluation providers

23. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Case Study n Goal: help Michigan non-profits select consultants and other service providers n Is this a good candidate for a reputation system?

24. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Case Study n Goal: help Michigan non-profits select consultants and other service providers n Is this a good candidate for a reputation system?  Interacting with strangers  Sellers (Exchange Partners) Vary  Skill  Effort  Ethics

25. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Case Study Design Choices n Rating scales n Aggregation of ratings n Who rates? n Incentives for raters n Identification/Anonymity –Exchange partners –Evaluation providers

26. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu

27. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu

28. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu

29. SCHOOL OF INFORMATION UNIVERSITY OF MICHIGAN si.umich.edu Summary n RS inform, incent, select n Opportunity for RS: interactions with strangers n Design space –Scales, aggregation, raters, incentives, anonymity