1. Social Learning and Consumer Demand Markus Mobius (Harvard University and NBER) Paul Niehaus (Harvard University) Tanya Rosenblat (Wesleyan University and IAS) 28 April, 2006

2. Introduction We “seed” a known social network with information by distributing new products randomly to some members. Methodology: How can we measure the influence of treated agents on their friends?

3. Introduction We “seed” a known social network with information by distributing new products randomly to some members. Methodology: How can we measure the influence of treated agents on their friends? Results: How does social influence decline with distance?

4. Methodology We build a simple model to infer the “interaction probability” between a treated agent and any of her social neighbors. During an interaction the treated agent’s knowledge is transferred to the neighbor. Interaction probabilities vary by social distance. Our model has the advantage that it can be easily estimated and that it can deal with treatment “overlaps”.

5. Methodology Interaction probabilities are convenient to measure influence. Example: Assume that an agent has 10 direct friends and 60 indirect friends and the interaction probabilities are and. Then on average the agent transfers knowledge to 1 direct friends and 3 indirect friends. In this example the agent affects knowledge in the network mainly by influencing indirect friends rather than direct friends because the interaction probability decreases less strongly than the network grows.

6. Basic Design: Stage 1: Measure Social Network Stage 2: Baseline Survey Stage 3: Distribute Products Stage 4: Track Social Learning

7. 1. Measuring the Social Network

8. Measuring the Network Rather than surveys, agents play in a trivia game Leveraged popularity of www.thefacebook.com www.thefacebook.com  Membership rate at Harvard College over 90% *  95% weekly return rate * * Data provided by the founders of thefacebook.com

9. Markus His Profile (Ad Space) His Friends

10. Trivia Game: Recruitment 1. On login, each Harvard undergraduate member of thefacebook.com saw an invitation to play in the trivia game. 2. Subjects agree to an informed consent form – now we can email them! 3. Subjects list 10 friends about whom they want to answer trivia questions. 4. This list of 10 people is what we’re interested in (not their performance in the trivia game)

12. Trivia Game: Trivia Questions 1. Subjects list 10 friends – this creates 10*N possible pairings. 2. Every night, new pairs are randomly selected by the computer Example: Suppose Markus listed Tanya as one of his 10 friends, and that this pairing gets picked.

13. Trivia Game Example a) Tanya (subject) gets an email asking her to log in and answer a question about herself b) Tanya logs in and answers, “which of the following kinds of music do you prefer?”

15. Trivia Game Example (cont.) c) Once Tanya has answered, Markus gets an email inviting him to log in and answer a question about one of his friends. d) After logging in, Markus has 20 seconds to answer “which of the following kinds of music does Tanya prefer?”

17. Trivia Game Example (cont.) e) If Markus’ answer is correct, he and Tanya are entered together into a nightly drawing to win a prize.

18. Trivia Game: Summary Subjects have incentives to list the 10 people they are most likely to be able to answer trivia questions about. This is our (implicit) definition of a “friend” Answers to trivia questions are unimportant  ok if people game the answers as long as the people it’s easiest to game with are the same as those they know best.  Roommates were disallowed  20 second time limit to answer  On average subjects got 50% of 4/5 answer multiple choice questions right – and many were easy

19. Recruitment In addition to invitations on login,  Posters in all hallways  Workers in dining halls with laptops to step through signup  Personalized snail mail to all upper-class students  Article in The Crimson on first grand prize winner Average acquisition cost per subject ~= $2.50

20. Participation Consent: 2932 out of 6389 undergrads (46%), and 50% of upperclassmen 10 friends: 2360 undergraduates (37%) Participation by year of graduation: 200545% 200652% 200753% 200834%

21. Participation By residential house (upperclassmen) Adams42%Leverett50% Cabot52%Lowell48% Currier52%Mather57% Dunster60%Pforzheimer50% Eliot48%Quincy49% Kirkland48%Winthrop43%

22. Network Data 23,600 links from participants 12,782 links between participants 6,880 of these symmetric (3,440 coordinated friendships)  Similar to 2003 results Construct the network using “or” link definition  5576 out of 6389 undergraduates (87%) participated or were named One giant cluster Average path length between participants = 4.2 Cluster coefficient for participants = 17%  Lower than 2003 results – because many named friends are in different houses

23. Growth of Neighborhoods Average # of roommates:0.98 Average # of direct friends:7.68 Average # of SD=2 friends:57.91 Average # of SD=3 friends:347.14

24. Methods in Comparison 2003 House Experiment in 2 undergraduate houses Email-data: Sacerdote and Marmaris (2004) Mutual-friend methods with facebook data? (Glaeser, Laibson, Sacerdote 2000)

25. 2. Baseline Survey

26. Goals of Baseline We want to predict valuations of subjects for our products without telling them which products we will distribute. This allows us to test whether subjects with a higher valuations are more influenced. We treat a product as a vector of attributes which span a space containing the specific product.

27. Choice of Products 1. We want new products to maximize the potential for social learning.

28. Choice of Products 1. We want new products to maximize the potential for social learning. 2. We want some products where subjects have to talk to exchange information (such as newspaper subscription) and some products whose use is conspicuous (such as cell phone).

29. “Public Products” T-Mobile Sidekick II Philips Key019 Digital Camcorder Philips ShoqBox

30. “Private Products” Student Advantage Discount Card (1 year) Qdoba Meal Vouchers (5) Baptiste Studios Yoga Vouchers (5)

31. Configurators We identified 5 or 6 salient features for each of the six products. For example, a product might be a general type of discount card for students. Particular features of the card could be: (i) provides a discount on textbooks; (ii) provides a discount on Amtrak/ Greyhound; etc. We elicit a baseline valuation from subjects plus a valuation for each feature (assumes additive separability of valuations over features).

32. Feature descriptions Baseline bid Feature bids

33. Constructed Bids We constructed an implicit bid B from subjects responses: Bid=Baseline Value + Sum over Feature Values (for existing features) Subjects were told that they could submit a second in the followup survey and that either this bid or the followup bid would be entered with equal probability into a uniform-price auction.

34. Constructed Bids We constructed an implicit bid B from subjects responses: Bid=Baseline Value + Sum over Feature Values (for existing features) Subjects were told that they could submit a second in the followup survey and that either this bid or the followup bid would be entered with equal probability into a uniform-price auction. Subjects are provided with incentives for truth-telling.

35. ($20)($50)($35)($150) ($250)(Price)

36. Distributions of Imputed Bids Imputed valuations look sensible. In each case market prices lie between median bid and upper tail.

37. 3. Distribution of Products

38. Randomized Product Trials Private Products  1 year Student Advantage cards  5 yoga vouchers  5 meal vouchers Public products  Try out for approximately 4 weeks during end of term

39. Randomization Only subjects with imputed bids above the median were eligible. We then offered products to about 100 subjects for each product. Blocked by year of graduation, gender, and residential house. Email invitations to come pick up samples Invitation times varied to vary strength of exposure (April 26 th – May 3 rd )

40. Response Rates Overall: 57%

42. Info Treatments Varied information communicated verbally by workers doing distribution Information treatments correspond to product features in our configurators (5 or 6 features for each product). Reinforced this information treatment with reminder emails Each treatment given with 50% probability to each subject

44. Other Treatments We also provided randomized online and print ads to subjects who did not receive products (not reported in this talk).

45. 4. Track Social Learning

46. Followup Survey We measure both subjective and objective knowledge of all subjects.

47. Followup Survey We measure both subjective and objective knowledge of all subjects. Subjective Knowledge: Stated probability that subject can answer any Yes/No question correctly.

48. Followup Survey We measure both subjective and objective knowledge of all subjects. Subjective Knowledge: Stated probability that subject can answer any Yes/No question correctly. Objective Knowledge: Average number of actual correct Yes/No questions in subsequent quiz.

49. Eliciting Confidence Levels Meet “Bob the Robot” and his clones Bob 1 – Bob 100 Subjects are randomly paired with an (unknown) Bob Subjects indicated a “cutoff Bob” at which they are indifferent about who should answer the question If assigned Bob is better than the cutoff, Bob answers the question; otherwise we use subject’s answer Incentive-compatible mechanism to elicit subject’s belief that he/she will get the question right

52. Final Valuations Also asked for a second bid for each product. Asked subjects about the valuations of other randomly selected subjects.

55. Analysis Model

56. An untreated (uninformed) subject has a probability p of interacting with some treated (informed) subject. The interaction probability p depends on the social distance between uninformed and informed subject. We distinguish three types of social distances: room mates (M), direct friends (NW1) and indirect friends (NW2).

57. Model We define knowledge as the subjective or objective probability of answering a question about the product correctly. If an informed and uninformed subject interact the knowledge of the informed subject is transferred to the uninformed subject (informed = treated with a product).

58. Model We define knowledge as the subjective or objective probability of answering a question about the product correctly. If an informed and uninformed subject interact the knowledge of the informed subject is transferred to the uninformed subject (informed = treated with a product). After interacting the uninformed subject has the same probability of answering a question correctly as the informed subject.

59. Model Assume that the knowledge of an informed subject is and the knowledge of an uninformed subject is. Assume that the uninformed’s probability of interacting with some informed subject is X. Then we can express the final expected knowledge of the uninformed agent as:

60. What is X? Assume that the uninformed agent has room mates who were offered a product, direct friends and indirect friends. Then we can express X as:

61. What is X? Assume that the uninformed agent has room mates who were offered a product, direct friends and indirect friends. Then we can express X as: The probability of interacting with some informed subject is 1 minus the probability of interacting with none of them.

62. Model We obtain: We observe and in the followup survey.

63. Model We obtain: We observe and in the followup survey. We do not observe because we cannot do a baseline quiz without revealing the product.

64. Model We obtain expression (*): We observe and in the followup survey. However, we do not observe because we cannot do a baseline quiz without revealing the product. Moreover, we expect the information of uninformed agents to vary with the number of eligible neighbors (and hence the number of neighbors who were offered a treatment) due to selection.

65. We instead compare agents in similar “cells”:

66. We instead compare untreated agents in similar “cells”: We say the green subject lives in a (1,4+,4) cell to indicate that she has one treated room-mate, and four treated NW1 and NW2 friends AND she has at least one more eligible (but non-treated) NW1 friend (indicated by plus sign).

67. For example, compare a (1,4+,4) cell with a (1,5,4) cell:

68. The green agent on the right faces the same neighborhood as the agent on the left but the randomization turned one eligible, untreated agent into a treated agent.

69. Model By dividing expression (*) for all agents in cell (1,5,4) by expression (*) for all agents in cell (1,4+,4) we obtain the marginal impact of treating one more NW1 neighbor:

70. Model By dividing expression (*) for all agents in cell (1,5,4) by expression (*) for all agents in cell (1,4+,4) we obtain the marginal impact of treating one more NW1 neighbor: Since we only have finitely many observations per cell we get an estimate for p. For each marginal comparison between two neighboring cells we get a new estimate. From this we can construct an estimate for p and a confidence interval.

71. Model By dividing expression (*) for all agents in cell (1,5,4) by expression (*) for all agents in cell (1,4+,4) we obtain the marginal impact of treating one more NW1 neighbor: By comparing neighboring cells we are essentially differing out the unobserved knowledge of the uninformed agent.

72. Analysis Results

73. We are estimating the interaction probabilities separately for each product. We use both subjective knowledge (“What is the probability that you can answer a Yes/No question correctly?”) and objective knowledge (“Actual share of correctly answered questions in the quiz”).

74. Results - Card

75. SE (0.16)* (0.21)* (0.02)* (0.04)* (0.09) (0.03)

76. Results - Yoga SE (0.19)* (0.23)* (0.04)* (0.03)* (0.03) (0.05)*

77. Results – Restaurant SE (0.03)* (0.08)* (0.03)* (0.04)* (0.02) (0.01)

78. Results – Camcorder SE (0.02)* (0.02)* (0.02)* (0.03)* (0.02)* (0.02)*

79. Results – MP3 SE (0.06)* (0.07)* (0.03)* (0.04)* (0.02)* (0.01)*

80. Results – PDA SE (0.04)* (0.07)* (0.03)* (0.04)* (0.02) (0.02)

81. Results For “private products” the interaction probability for NW2 neighbors is usually insignificant. For “public products” the NW2 effect is small but significant. NW2 neighborhoods are also 7-times as large as NW1 neighborhoods! Therefore, the expected number of influenced NW2 agents can be large.

82. Results We would expect that agents with higher implied bids (from baseline survey) should have greater incentive to learn about the product higher probability of being talked to by treated guys (assuming that treated agents know the interests of their friends)

83. Results We would expect that agents with higher implied bids (from baseline survey) should have greater incentive to learn about the product higher probability of being talked to by treated guys (assuming that treated agents know the interests of their friends) We therefore repeat the previous analysis and only compare high-implicit-bid agents across cells.

84. Results - Card

85. Results - Yoga

86. Results – Restaurant

87. Results – Camcorder

88. Results – MP3

89. Results – PDA

90. Results Generally, the interaction probability is greater towards high-value subjects.

91. Results Generally, the interaction probability is greater towards high-value subjects. This is consistent with the idea that high-value agents either pay more attention to social learning or are talked to more often by product owners who know their interests.

92. Who is influenced the most by social learning (close or distant neighbors)? (expected number of interactions taking Nhood size into account; subjective knowledge and significant probabilities only) MNW1NW2TOTAL CARD0.501.121.62 YOGA0.601.601.20 FOOD0.240.801.04 CAM.0.651.122.854.62 SOUND0.500.642.283.42 PDA0.451.442.854.74

93. Who is influenced the most by social learning (close or distant neighbors)? (expected number of interactions taking Nhood size into account; subjective knowledge and significant probabilities only) MNW1NW2TOTAL CARD0.501.121.62 YOGA0.601.601.20 FOOD0.240.801.04 CAM.0.651.122.854.62 SOUND0.500.642.283.42 PDA0.451.442.854.74 Although there is a greater probability to interact with close agents the expected number of interactions increases with distance.

94. Summary Novel design  Hedonic analysis using configurators  Measure of confidence using the Bobs Simple model of social learning provides interpretable “interaction probabilities”.