Prediction Markets & Information Aggregation Yiling Chen, Harvard SEAS

Preference vs. Information  Preference  I prefer orange to apple  I’m willing to pay $50 for this item  Information  About some uncertain event  Information helps to update beliefs  Sometimes mixed together

Information Elicitation and Aggregation Problem  Events of interest  Will Democratic party win the Presidential election?  Will US economy still in recession in 2010?  Will there be a bird flu outbreak by August 2011?  Will sales of printers exceed 30K in July? ……  Information is dispersed among individuals  Want to aggregate dispersed information to make an informed prediction

We can ask experts, but  How to identify them?  How to ensure them to tell the truth?  Incentivize experts using proper scoring rules  Need to pay every expert  How to resolve conflicts among experts?  Impossibility results

Bet = Credible Opinion  Q: Will Pittsburgh Panthers win the NCAA tournament?  Betting intermediaries  Las Vegas, Wall Street, Betfair, Intrade,... Panthers will not win the NCAA. Info I bet $1000 Panthers will win the NCAA. Info

Prediction Markets  A prediction market is a betting intermediary that is designed for information aggregation and prediction.  Payoffs of the traded item is associated with outcomes of future events. $1 if Obama wins $0 Otherwise $1×Percentage of Vote Share That Obama Wins $1 if Panthers win $0 Otherwise $f(x)

 Speculation price discovery price  expectation of random variable | all information Value of ContractPayoff Event Outcome $P( Panthers win ) P( Panthers win ) 1- P( Panthers win ) $1 $0 Panthers win Panthers lose Equilibrium Price  Value of Contract  P( Panthers Win ) Market Efficiency $1 if Panthers win, $0 otherwise Why Markets? – Get Information ?

Does it work?  I.E.M. beat political polls 451/596 [Forsythe 1992, 1999][Oliven 1995][Rietz 1998][Berg 2001][Pennock 2002] Iowa caucus Super Tuesday

IEM 1992 [Source: Berg, DARPA Workshop, 2002]

Example: IEM

Does it work?  Microsoft Prediction Market  August 2004: Predict internal product ship date  Official, accepted schedule: mid-November 2004  25 $50, made up of testers, developers, etc.  Securities: Pre-NOV, NOV, DEC, JAN, FEB, Post-FEB  Within a few minutes of opening, NOV dropped to $0.012…

Does it work?  Yes, evidence from real markets, laboratory experiments, and theory  Racetrack odds beat track experts [Figlewski 1979]  Orange Juice futures improve weather forecast [Roll 1984]  HP markets beat sales forecast 6/8 [Plott 2000]  Google, GE, Elli Lily, and more all have positive evidence  Sports betting markets provide accurate forecasts of game outcomes [Gandar 1998][Thaler 1988][Debnath EC’03][Schmidt 2002]  Market games work [Servan-Schreiber 2004][Pennock 2001]  Laboratory experiments confirm information aggregation [Plott 1982;1988;1997][Forsythe 1990][Chen, EC’01]  Theory: “rational expectations” [Grossman 1981][Lucas 1972]

Predicting the CEO  Will Mr. Smith or Ms. Jones be the CEO of company X? $1 if Ms. Jones becomes CEO $1 if Mr. Smith becomes CEO Pr(Mr. Smith) Pr(Ms. Jones) $1

Predicting CEO Outcomes  How will CEO affect stock prices?  Alternatively, $1 if Mr. Smith becomes CEO & stock price goes up $1 if Mr. Smith becomes CEO & stock price goes down $1 if Ms. Jones becomes CEO & stock price goes down $1 if Ms. Jones becomes CEO & stock price goes up 1 share of stock, if Mr. Smith becomes CEO 1 share of stock, if Ms. Jones becomes CEO

CEO Decision Market  Should company X hire Mr. Smith or Ms. Jones as CEO? $1 if Mr. Smith becomes CEO Pr(stock up|Mr. Smith) Pr(Ms. Jones) $1 $1 if Ms. Jones becomes CEO $1 if Mr. Smith becomes CEO & stock price goes up $1 if Ms. Jones becomes CEO & stock price goes up Pr(Mr. Smith) Pr(stock up|Ms. Jones) Which one is higher? Pr(stock up|Mr. Smith)=Pr(Mr. Smith & Stock up)/Pr(Mr. Smith)

CEO Decision Market  Conditional market $1 if stock price goes up and Mr. Smith becomes CEO $0 if stock price goes down and Mr. Smith becomes CEO called off if Mr. Smith does not become CEO $1 if stock price goes up | Mr. Smith becomes CEO $1 if stock price goes up | Mr. Jones becomes CEO

Decision Markets Give E(O|C) C hoices  FED money policy  Next president  Health care regulation  School vouchers  Who is CEO  Which ad agency O utcomes  GDP per capita  War deaths  Lifespan  School test scores  Stock price  Product sales [Source: Hanson]

Does money matter? [Servan-Schreiber et. al. 2004] Head to Head Comparison  2003 NFL Season  Football prediction markets  NewsFutures (play $)  Tradesports (real $)  Online football forecasting competition  probabilityfootball.com  Contestants assess probabilities for each game  Quadratic scoring rule  ~2,000 “experts” Results:  Play money and real money performed similarly  6 th and 8 th respectively  Markets beat most of the ~2,000 contestants  Average of experts came 39 th

Desired Properties of Prediction Markets and Other Information Aggregation Mechanisms  Liquidity  People can find counterparties to trade whenever they want.  Truthfulness  Participants reveal their information honestly and immediately.  Expressiveness  There are as few constraints as possible on the form of bets that people can use to express their opinions.  Computational tractability  The process of operating a market should be computationally manageable.  Can handle situations where ground truth is not available

Desired Properties of Prediction Markets and Other Information Aggregation Mechanisms  Liquidity (Use automated market makers)  People can find counterparties to trade whenever they want.  Truthfulness  Participants reveal their information honestly and immediately.  Expressiveness  There are as few constraints as possible on the form of bets that people can use to express their opinions.  Computational tractability  The process of operating a market should be computationally manageable.  Can handle situations where ground truth is not available

Truthfulness: Manipulation Concerns I  Can forward looking traders get more profit by bluffing in prediction markets?

Truthfulness: Manipulation Concerns I  Can forward looking traders get more profit by bluffing in prediction markets?  Conditionally independent signals: Truthful betting is the only equilibrium  Independent signals: No finite equilibrium that involves truthful betting, but it’s possible to change the mechanism so that bluffing is discouraged [Chen et. al. 09]

Truthfulness: Manipulation Concerns II  Manipulate market price to influence decision making  How to make prediction markets to be manipulation resistant is an open question.  Manipulation in Intrade

Expressiveness and Computational Complexity: Combinatorial Prediction Markets  Things people can express today  Democrat wins the election (with probability 0.55)  No bird flu outbreak in US before 2011  Horse A will win the race  Things people can not express (very well) today  Democrat wins the election if he/she wins both Florida and Ohio  Oil price increases & A Democrat wins & Recession in 2009  Horse A beats Horse B

Expressiveness and Computational Complexity: Combinatorial Prediction Markets USC wins a third round game. USC beats Wisconsin if they meet. A beats C A or B will be at position 1. Obama wins Florida and Ohio Chen et. al. EC’07, EC’08, STOC’08

When there is no ground truth  No prediction market in theory can handle it now, but in practice some are in use  E.g. New product development

When there is no ground truth  We still have hope  Peer prediction [Miller et. al. 2005]  Proper scoring rule  Comparing with peer  Strong common knowledge of common prior assumption  Bayesian truth serum [Prelec 2004]  Ask for an answer and a prediction  Reward answers that are more common than collectively predicted

In short  Prediction market is an example in which competitive agents interact indirectly through the market mechanism to achieve some collaborative goal.  It’s a centralized mechanism.  Is the bigger problem information elicitation and aggregation?  How to approach the problem when we do not have full rationality, do not need absolute truthfulness, and do not have ground truth?