Coye Cheshire July 20, 2016 // Computer-Mediated Communication Reputation (Part II)

How might different types of reputation systems solve the lemons problem? 7/20/2016Computer-Mediated Communication1

7/20/2016Computer-Mediated Communication2 “Reputation” systems in further detail Explicit Implicit Behavior Ratings by others Derived from behavior Join Date: Mar 2004 Posts: 22 Direct experience

Yamagishi’s Experiment  All participants play role of buyer and seller in an open market  Participants create ‘goods’ of some value (x), and sell at price (y).  Production points and payouts come from the experimenter  Buyers receive 1.5 times the true value of the good. 7/20/2016Computer-Mediated Communication3 Buyer gets 60 x 1.5 = 90 points

7/20/2016Computer-Mediated Communication4

Yamagishi’s Experimental Conditions  Anonymity  Identity  Reputation 7/20/2016Computer-Mediated Communication5

7/20/2016Computer-Mediated Communication6 Yamagishi et al. (First Experiments) Conditions of Environment (Independent Variables) -Anonymity -Identity -Reputation (3 rd party only) Outcome (Dependent Variable) -Price? -Quality level? -Probability of item being sold? -Time to sale?

Quality Levels 7/20/2016Computer-Mediated Communication7

7/20/2016Computer-Mediated Communication8 Levels of “dishonesty”

7/20/2016Computer-Mediated Communication9 Yamagishi et al (Second Set of Experiments) Reputation System (Independent Variable) -Positive -Negative -Mixed Outcome (Dependent Variable) -Price? -Quality level? -Probability of item being sold? -Time to sale?

Comparing Quality Levels 7/20/2016Computer-Mediated Communication10

7/20/2016Computer-Mediated Communication11 Quality levels over time (3 rd Expt, Mock Auction)

Identity 7/20/2016Computer-Mediated Communication12 I also found the description of negative reviews interesting. In the experiment, and a basis for the experiment, they assumed that users had the freedom to change identities; and yet we just had a discussion about the persistence of information and how hard it is to "forget" in the digital era. As more mash-ups of data are created, it may eventually be possible to trace sellers who "quit" and reemerge as "new" sellers, there are also sites that restrict creation of separate accounts by IP. I am not saying that going "incognito" is impossible, it will just be harder. -Adam J.

Strategy and Reality 7/20/2016Computer-Mediated Communication13 Being someone who steals other people's candy, I can say that (1) I love candy, so watch out and (2) accountability is key. If you aren't being held accountable for your actions, exploiting others is a profitable strategy. I mean, look around you. -Galen “you better hide your candy” P. About whether honesty pays, I had the same question as Kimra about whether the results would change in the longer term. -Tanushree J. I wonder if that conclusion would continue to hold if the game continued over a longer time frame. For instance, over the long term, do the sellers with persistent identities and/or better reputations and greater honesty ultimately profit more because trust in them only increases over time and thus makes lower-priced but less reputable alternatives look less appealing? -Kimra M.

7/20/2016Computer-Mediated Communication14 Resnick et al Effect of strong reputations on revenues compared to those without reputation Effect of “negatives” in a brief reputation on outcome of revenue

7/20/2016Computer-Mediated Communication15 Matched Pairs by Different Sellers

7/20/2016Computer-Mediated Communication16 Reputation Effects  Do strong reputations matter?  What is the impact of negative reputation marks in a mixed reputation system such as eBay?

Complexities and Limits of Field Experiments 7/20/2016Computer-Mediated Communication17 First of all, they didn't believe in using the number of bids as a control variable. They claimed that it is "an endogenous indicator of the impact of reputation on price and should not be an independent variable in a simple regression model." I completely disagree with this idea. Personally (coming from a background in social psychology), I would argue that the number of bids serves as a form of social proof, which may very well be a moderating factor, thus muddying the effects they found. They also claim to have made "roughly similar web sites" for all sellers. They address this as one of the threats to validity, but they seem to discount the importance of the visual aesthetic for each seller. -Courtney C.

7/20/2016Computer-Mediated Communication18 Critiquing Methodological Approaches to Similar Problems

7/20/2016Computer-Mediated Communication19 Two approaches to studying online reputation system (Ebay)  Why study it in a lab?  Why study it in a field experiment?

7/20/2016Computer-Mediated Communication20 Putting Experimental Work in Context  Selection of subjects (i.e., what do they value?)  Task length and learning  Accounting for time in statistical analyses  Do not assume that an experiment is even trying to ‘recreate’ a specific real-life situation unless they explicitly say so– and even then you should always consider the scope of the experiment.

7/20/2016Computer-Mediated Communication21 The Validity Problem Internal Validity External Validity

7/20/2016Computer-Mediated Communication22 External Validity: Generalizing From Experiments ?

7/20/2016Computer-Mediated Communication23 Ecological Validity: Approximation of Real-Life Activity Yamagishi et al. Resnick et al.

Resnick et al /20/2016Computer-Mediated Communication24

Other Implicit ‘Reputation’ Information in CMC? 7/20/2016Computer-Mediated Communication25

7/20/2016Computer-Mediated Communication26 Another side of reputation… Internet markets also have significant advantages in establishing reputations … any information that is gleaned can be near costlessly tallied on a continuing basis … [and] that information can be near costlessly transmitted to millions of potential customers. — Resnick et al. 2006, p. 80 “ ”

7/20/2016Computer-Mediated Communication27 The CMC and ‘Offline’ Reputation Link: Emergent Reputation Systems and Identity

As designers, what can we learn from all of this?  What kind of community do you have (or are you trying to foster)?  When and Why to use Pos/Neg/Mixed/Hybrid Reputation Systems?  What behavior(s) do you want to encourage, reward, punish?  Consider the “unintended consequences” of implicit information  Just because you build a system to be interpreted a certain way doesn’t mean that the user will agree… 7/20/2016Computer-Mediated Communication28