Information Markets John Ledyard April 13, 2005 Nancy Schwartz Memorial Lecture KGSM, Northwestern University

What would you like to know? What will Disney’s profits be in 2006? –Ask an accountant, a stock analyst, a consultant, the CEO, Mickey Mouse, … Who will the next Pope be? –Ask a pundit, a cardinal, take a poll, …… How stable will the middle-east be on 12/31/05? –Ask the CIA, the Mossad, the Defense Department, the President, a committee of experts, ……. Should you set up an Information Market?

What is an Information Market? This is not about markets for information. Kihlstrom (1974), Radner and Stiglitz (1984), Kamien and Tauman (1990), Keppo, Moscarini, Smith (2005) It is about using market forces to bring together disparate bits and pieces of information and add them up, or aggregate them, for use in predictions or decisions. Google hits Information Markets = 25,700,000 Prediction Markets = 1,550,000

How would it work? Familiar Question: Who will win an election? –Standard approach - Polls. In 1988, University of Iowa Business School securitized the Presidential election prediction on the internet.

The Iowa Election Market  In 1992, for $1.00, Iowa sold or bought a set of securities that covered all possible outcomes of the election; Bush, Clinton, Other (included Perot).

The Iowa Election Market  In 1992, for $1.00, Iowa sold or bought a set of securities that covered all possible outcomes of the election; Bush, Clinton, Other (included Perot).  Each security paid $1 times the percentage of the vote for that person. Securities were traded.

The Iowa Election Market  In 1992, for $1.00, Iowa sold or bought a set of securities that covered all possible outcomes of the election; Bush, Clinton, Other (included Perot).  Each security paid $1 times the percentage of the vote for that person. Securities were traded.  After the election tally, if you owned 100 shares of Bush and Bush received 38% of the vote then you would be paid $38.

The Iowa Election Market  In 1992, for $1.00, Iowa sold or bought a set of securities that covered all possible outcomes of the election; Bush, Clinton, Other (included Perot).  Each security paid $1 times the percentage of the vote for that person. Securities were traded.  After the election tally, if you owned 100 shares of Bush and Bush received 38% of the vote then you would be paid $38.  The actual result was Clinton 43%, Bush 38%, Perot 19%

How the IEM might work. BushClintonOther Prices$.50$.40$.10 U think You go to the IEM website and see You see a way to make some money.

How the IEM might work. BushClintonOtherCashE(V) Prices$.50$.40$.10 U Think U buy $1$1-$1 = 0

How the IEM might work. BushClintonOtherCashE(V) Prices$.50$.40$.10 U Think U buy $1$1-$1 = 0 U $.05-$.10

How the IEM might work. BushClintonOtherCashE(V) Prices$.50$.40$.10 U Think U buy $1$1-$1 = 0 U $.05-$.10 U have $ $1.10 You actually will make (0.38x2) = -$0.34. But you don’t know that when you make this transaction. You can only act on your beliefs.

How the IEM might work. BushClintonOtherCashE(V) Prices$.50$.40$.10 U Think U buy $1$1-$1 = 0 U $.05-$.10 U have $ $1.10  Other traders then adjust their beliefs in response to the price changes. And so on.  If all goes well, in equilibrium, prices will equate to the full- information beliefs of the traders.  And if all goes well, these will be the true vote-shares.

1992 U.S. Election Source: IEM (2005)

How Accurate Has IEM Been? Source: IEM (2005)

Also….. National election market in NY ( ) –[Rhode and Strumpf (2004)] –“Over $165 million (in 2002 dollars) was wagered in one election, and betting activity at times dominated transactions in the stock exchanges on Wall Street.” –“In only one case did the candidate clearly favored in the betting a month before Election Day lose.”

Is this a Free Lunch? Iowa pays nothing. On average, the traders earn nothing But, in the end, everyone is better, maybe even maximally, informed.

“The Next Killer App”? or “Too Good To Be True”? Evidence, mostly empirical, suggests Information Markets Work. Evidence, mostly theoretical, suggests Information Markets Can’t Work. Today we will explore – how “Information Markets” work – how to design and engineer viable and accurate “Information Mechanisms”

Why might an IM work? There are two of us in this scenario. –(Neither of us is a Game Theorist.) There are 2 coins: –Coin A comes up heads 80% of the time. –Coin B comes up heads 20% of the time. One is chosen with probability.5. –This is our common prior. The coin is flipped twice for each of us. –You see (H,T) and I don’t see that. –I see and you don’t see that.

Why might an IM work? Remember: Coin A =.8 heads, Coin B =.2 heads, you see (H,T), I see (?, ?). What is the probability that the coin is A? Based on only your information, the answer is 0.5. –This is your initial posterior. Suppose there is a Market Maker who posts prices and asks us whether we want to buy or sell an asset that pays $1 if the coin is A and $0 if it is B. He posts a price of You offer to sell and I offer to buy.

Why might an IM work? Remember: Coin A =.8 heads, Coin B =.2 heads, you see (H,T), I see (?, ?), I offered to buy at.6. What is the probability that the coin is A? Since you know that I must have seen (H,H), you know (H,T,H,H). This is everything. Your answer should be Of course, I only know that you are either (H,T) or (T,T), so I don’t know everything - yet.

Why might an IM work? Suppose the Market Maker still posts a price of.6. We both offer to buy. I now know that your current posterior is.94 which means you must have seen (H,T) So we both now know that the total information is (3H, 1T) and our posteriors are the same: The “market” has “aggregated” the “information”! The underlying theories are Rational Expectations Equilibrium and Common Knowledge Information. Green (1973), Lucas (1972), Grossman (1977) Aumann (1976), Geanakopolos and Polemarchakis (1982)

But Wait!!!! ?=?= Market Maker Maxwell’s Demon There is something fishy here!

Why might an IM not work? Let’s go back to the Market and get rid of the Market Maker. Remember: Coin A =.8 heads, Coin B =.2 heads, you see (H,T), I see (?, ?), the asset pays $1 if A. I offer to sell you 2 units of the asset for.30. What should you do now? Infer that I saw (T,T) and believe (1 H, 3T). So you now should believe that P(A) =.06

Why might an IM not work? I offer you 2 units of the asset for.30, you saw (H,T) and know I saw (T,T), so your P(A) =.06. You believe the expected value of the asset is.06. Obviously you should reject my offer. The full information is either (1H,3T) or (0H,4T). –If you had seen (H,H) you would accept my offer. If you bid to buy above.004, I also know it is 0.6. We will not trade! The underlying theory is No-Trade Theorems [Grossman and Stiglitz (1976), Milgrom-Stokey (1988)]

What About Empirical Evidence? There are many naturally occurring IM’s And, in direct comparisons, they beat other institutions.

Pari-mutuel betting

Racetrack odds beat track experts –Figlewski (1979)

Futures markets

OJ futures improve freeze forecasts –Roll (1984)

Stock markets

Stock prices beat the experts panel in the post-Challenger probe –Maloney & Mulherin (2003)

Less Positive Field Evidence The consensus forecast (median of about 30 economists) has as much predictive power as the Goldman-Sachs pari-mutual market. [Wolfers and Zieztwitz (2003)] Wide bid-ask spreads and thin trading on most Tradesports.com markets. –Politics(4/11/05) (contract, price, spread, vol.) 2008DemnomClinton RepnomJBush –Papacy (4/11/05) (contract, price, spread, vol.) –Italy –Nigeria –USA

The Experimental Evidence is Mixed A number of experimentalists have demonstrated convergence to the full information rational expectations equilibrium. –In laboratory asset markets with one asset Forsythe, Palfrey, Plott (1982) –In laboratory elections McKelvey & Ordeshook (1985) –In laboratory asset markets with 3 assets Plott and Sunder (1988)

The Experimental Evidence is Mixed A number of experimentalists have demonstrated that information mechanisms do not always work. –In laboratory asset markets, if preferences differ and there are incomplete markets, there is little aggregation. Plott and Sunder (1988) Risk or ambiguity aversion? –In iterative polls in the laboratory, there is incomplete information aggregation. McKelvey & Page (1990) Incomplete Bayesian updating? –In laboratory pari-mutuel betting, we observe mirages. Plott (2002) (Pari-mutuel) Information cascades?

One More Set of Data [Grus and Ledyard (2005)] The 2 coin example, 2 flips each N = 3, 7, 8, 12: Caltech subjects Market, Pari-mutuel 3 minutes of transactions per period 8 periods per mechanism 3 mechanisms per session (2.5 hours) Earnings approximate $33/subject

These get it! These are big mirages. kl(.8,.65)=.06 kl(.8,.50)=.22 kl(.8,.20)=.83 MP informational size =.58 for N = 3 =.20 for N = 7 =.02 for N = 12 Numbers Matter!

Summary Statistics 32 observations Pari-MutuelMarket Got it41%63% “Got it” means KL <.01 or | p-FI | <.05 When N > 6, the market “gets it” 80% and the pari-mutuel gets it 75%.

Summary Statistics 32 observations for PM, M, P Pari-MutuelMarket Got it30% (75%)55% (80%) % Early0%38% Early means in 1st 3 transactions. Not in time. N= 3, 12

Summary Statistics 32 observations for PM, M, P Pari-MutuelMarket Got it30% (75%)55% (80%) % Early0%38% Mirages25%22% Mirage means p is not the FI and is one of the other possibilities.

Summary Statistics 32 observations for PM, M, P Pari-MutuelMarket Got it30% (75%)55% (80%) % Early0%38% Mirages25%22% tickets or transactions per subject N = 3, rest 2.5, 7.8 N = 3, rest 1.6, 6.2

Based on the 2-coin experiments (and more complicated ones) Some aggregation occurs but it is not perfect. Markets are faster than pari-mutuels and get it more often. Both are subject to mirages. Neither perform well at low-scale (N = 3). There is evidence to support the no-trade hypothesis, particularly when N = 3. –But there is still some trading. Incomplete updating? Not here. Boredom - yes, lots of little trades Greater fool? - Possibly.

Summary to here There is theoretical, experimental, and field evidence that traditional IM’s may work. But there is also evidence that there are impediments to complete information aggregation especially in environments with informationally large agents. Should we worry about small numbers? Can we find better information mechanisms?

Numbers and Informationally Large Traders Probability of success in next year for drug A? Expected sales of SUV model X in 2006 contingent on gas prices above $5 on July 2006? The expected software shipping date contingent on retaining feature R? Expected benefits of a government program conditional on one of several possible actions? (better cost-benefit analysis?)

Example contract: Jordan is more politically stable in 4Q2005 conditional on US military activity down in Iraq in 3Q2005 and US$aid in 2Q2005 up in Iran. Remember PAM? 8 nations, 5 indices, 4 quarters -Political stability -Military activity -Economic growth -US $ aid -US military activity Up, Down, or Constant Implies 320 active markets.

320 questions and completeness implies 2^320 = 2 * 10^96 contracts. Remember PAM? 8 nations, 5 indices, 4 quarters -Political stability -Military activity -Economic growth -US $ aid -US military activity Up, Down, or Constant Implies 320 active markets.

Need Better Information Mechanisms To be useful in many potential applications, Information Markets need to perform well with small numbers and informationally large traders. Traditional markets and pari-mutuels are not up to this. Two possible approaches –Subsidize the action in the traditional designs. –Design new mechanisms.

Better Information Mechanisms? Let us first try to subsidize and modify the traditional IM’s. –Pari-mutuel: Add some tickets into the pot so that the expected payoff of spending $1 is larger than $1. –Market: Randomly accept bids and offers at “market.” Noise trading [ Grossman and Stiglitz (1976)]

Helps when N = 3 kl(.8,.65)=.06 kl(.8,.50)=.22 kl(.8,.20)=.83

Helps when N = 3 kl(.8,.65)=.06 kl(.8,.50)=.22 kl(.8,.20)=.83 Does this mean “Noise Creates Information”?

Better but Still not Great PMPM/SMM/S Got it41%31%63%66% % Early0%3%47%56% Mirages25%13%22%13% Tickets or actions

Design New Mechanisms With one agent, a Scoring Rule is a good mechanism to elicit beliefs. –Report r. Receive S*ln(r/2) in asset that pays $1 if A and S*ln((1-r)/2) in asset that pays $1 if B. Incentive compatible to report true beliefs Encourages participation –Provides an expected value greater than 0. –Brier (1950) Monthly Weather Review, Goode (1952), Savage (1971) JASA, Page (1988)(PSR  VCG) Let’s adapt this to multi-agent situations.

If it Works for One Person…… Market Scoring Rules may be good mechanisms. –Hanson (2003) –An automated market maker (MM) Any trader can access the MM, at any time and trade assets according to a scoring rule by announcing his belief. The scoring rule is seeded with the last trader’s announcement of their beliefs. Each new announced belief is publicly reported. –Incentive compatible to report true beliefs (somewhat) At one iteration, but not over all iterations. Possible to mislead early and gain later (particularly if N =3) –Encourages participation (somewhat) Positive expected value to the group.

An Old Standby Dressed Up A Poll with Incentives may be a good mechanism. –Basic design is standard. All are polled and asked their beliefs. Beliefs are averaged and publicly announced. Repeated m times (We did 5 and 3.) –But at the end, each agent is paid according to the same scoring rule. –Incentive compatible to report true beliefs (somewhat) True at “equilibrium” if there is full-information aggregation Open question whether it is true during iterations –Encourages participation Positive expected value to everyone in the group.

Pretty much the same. ave KL PMS.042 MSR.045 P.047 MS.051 kl(.8,.65)=.06 kl(.8,.50)=.22 kl(.8,.20)=.83

P and MRS get it (always) >> MS and PMS ave KLMirage MSR P PM PMS MS M kl(.8,.65)=.06 kl(.8,.50)=.22 kl(.8,.20)=.83

Summary Statistics 32 observations (all N) PMSRMSPMS Got it53%75%66%31% % Early22%56%47%3% Mirages6% 13% KL dist In MSR, first mover wins. In Pari-mutuel, last mover wins. In time lapsed, MSR hits it really fast.

Reprise: Do traditional IM’s work? YES: Large numbers of informationally small agents and reasonably simple environments seem to overcome some of the no-trade incentives in traditional markets and pari-mutuels. –I don’t think we yet know exactly why. (Open Puzzle!) NO: With informationally large agents, much less incomplete markets or complex decision problems, it does not look so good. – N = 3 creates serious problems for both markets and pari-mutuels, even in simple environments. –More research is needed here.

Reprise: Can we design IM’s that work? YES: For informationally small agents in simple environments, –The Market Scoring Rule gets it early and often. –Polling with incentive payments also works pretty well but is slower to get the job done. –Markets are mirage prone, even with noise traders. –Pari-mutuels are slow, even with subsidization MAYBE: For informationally large agents, –The four best (ms, msr, p, pms) are all off by.10 to.15. –All mechanisms we looked at can be improved upon. –Or is there an impossibility theorem? (Open Puzzle!)

The End I leave you with two questions that I believe are worth pursuing.

= Market Maker Maxwell’s Demon Is the best mechanism? MSR Is ?