Internet-Based Auctions and Markets David M. Pennock Principal Research Scientist Yahoo! Research - NYC

Auctions: 2000 View Going once, … going twice,... YesterdayToday (~2000) –eBay: 4 million; 450k new/day

Auctions: 2000 View YesterdayToday (~2000)

Auctions: 2000 View YesterdayToday (~2000)

Auctions: 2006 View Yesterday –eBay –200 million/month Today –Google / Yahoo! –6 billion/month (US)

Auctions: 2006 View YesterdayToday

Auctions: 2006 View YesterdayToday

Newsweek June 17, 2002 The United States of EBAY In 2001: 170 million transactions worth $9.3 billion in 18,000 categories that together cover virtually the entire universe of human artifactsFerraris, Plymouths and Yugos; desk, floor, wall and ceiling lamps; 11 different varieties of pockets watches; contemporary Barbies, vintage Barbies, and replica Barbies. Since everything that transpires on Ebay is recorded, and most of it is public, the site constitutes a gold mine of data on American tastes and preoccupations.

The United States of Search 6 billion searches/month 50% of web users search every day 13% of traffic to commercial sites 40% of product searches $5 billion 2005 US ad revenue (41% of US online ads; 2% of all US ads) Doubling every year for four years Search data: Covers nearly everything that people think about: intensions, desires, diversions, interests, buying habits,...

Outline Selected survey of Internet-based electronic markets –Auctions (e.g., eBay) –Combinatorial auctions –Sponsored search advertisement auctions (e.g., Google, Yahoo!) –Prediction markets (e.g., Iowa political markets, financial markets)

What is an auction? Definition [McAfee & McMillan, JEL 1987] : –a market institution with an –explicit set of rules –determining resource allocation and prices –on the basis of bids from the market participants. Examples:

Why auctions? For object of unknown value Flexible Dynamic Mechanized –reduces complexity of negotiations –ideal for computer implementation Economically efficient!

Taxonomy of common auctions Open auctions –English –Dutch Sealed-bid auctions –first price –second price (Vickrey) –Mth price, M+1st price –continuous double auction

English auction Open One item for sale Auctioneer begins low; typically with sellers reserve price Buyers call out bids to beat the current price Last buyer remaining wins; pays the price that (s)he bid

Dutch auction Open One item for sale Auctioneer begins high; above the maximum foreseeable bid Auctioneer lowers price in increments First buyer willing to accept price wins; pays last announced price less information

Sealed-bid first price auction All buyers submit their bids privately buyer with the highest bid wins; pays the price (s)he bid $150 $120 $90 $50

Sealed-bid second price auction (Vickrey auction) All buyers submit their bids privately buyer with the highest bid wins; pays the price of the second highest bid $150 $120 $90 $50 Only pays $120

Incentive Compatibility (Truthfulness) Telling the truth is optimal in second-price auction Suppose your value for the item is $100; if you win, your net gain (loss) is $100 - price If you bid more than $100: –you increase your chances of winning at price >$100 –you do not improve your chance of winning for < $100 If you bid less than $100: –you reduce your chances of winning at price < $100 –there is no effect on the price you pay if you do win Dominant optimal strategy: bid $100 –Key: the price you pay is out of your control

Vickrey-Clark-Groves (VCG) Generalization of 2nd price auction Works for arbitrary number of goods, including allowing combination bids Auction procedure: –Collect bids –Allocate goods to maximize total reported value (goods go to those who claim to value them most) –Payments: Each bidder pays her externality: Pays difference between sum of everyone elses value without bidder minus sum of everyone elses value with bidder Incentive compatible (truthful)

Collusion Notice that, if some bidders collude, they might do better by lying (e.g., by forming a ring) In general, essentially all auctions are subject to some sort of manipulation by collusion among buyers, sellers, and/or auctioneer.

Revenue Equivalence Which auction is best for the seller? In second-price auction, buyer pays < bid In first-price auction, buyers shade bids Theorem: –expected revenue for seller is the same! –requires technical assumptions on buyers, including independent private values –English = 2nd price; Dutch = 1st price

Mth price auction English, Dutch, 1st price, 2nd price: N buyers and 1 seller Generalize to N buyers and M sellers Mth price auction: –sort all bids from buyers and sellers –price = the Mth highest bid –let n = # of buy offers >= price –let m = # of sell offers <= price –let x = min(n,m) –the x highest buy offers and x lowest sell offers win

Mth price auction $150 $120 $90 $50 $300 $170 $130 $110 Buy offers (N=4)Sell offers (M=5) $80

$50 $80 $90 $110 $120 $130 Mth price auction $150 $170 $300 Buy offers (N=4)Sell offers (M=5) Winning buyers/sellers price = $120

$50 $80 $90 $110 $120 $130 M+1st price auction $150 $170 $300 Buy offers (N=4)Sell offers (M=5) Winning buyers/sellers 6 price = $110

Incentive Compatibility (Truthfulness) M+1st price auction is incentive compatible for buyers –buyers dominant strategy is to bid truthfully –M=1 is Vickrey second-price auction Mth price auction is incentive compatible for sellers –sellers dominate strategy is to make offers truthfully

Impossibility Essentially no auction whatsoever can be simultaneously incentive compatible for both buyers and sellers! –if buyers are induced to reveal their true values, then sellers have incentive to lie, and vice versa –the only way to get both to tell the truth is to have some outside party subsidize the auction

Impossibility Setup: 1 good, 1 buyer w/ value [a1,b1], seller w/ value [a2,b2], nonempty intersec. Desirable properties / axioms: –(1) incentive compatible –(2) individually rational –(3) efficient –(4) no outside subsidy (1) (4) are mutually inconsistent [M & S 83]

$50 $80 $90 $110 $120 $130 k-double auction $150 $170 $300 Buy offers (N=4)Sell offers (M=5) Winning buyers/sellers 6 price = $110 + $10*k

Continuous double auction k-double auction repeated continuously over time buyers and sellers continually place offers as soon as a buy offer > a sell offer, a transaction occurs At any given time, there is no overlap btw highest buy offer & lowest sell offer

Continuous double auction

Winners curse Common, unknown value for item (e.g., potential oil drilling site) Most overly optimistic bidder wins; true value is probably less

Combinatorial auctions E.g.: spectrum rights, computer system, … n goods bids allowed 2 n combinations Maximizing revenue: NP-hard (set packing) Enter computer scientists (hot topic)… Survey: [Vries & Vohra 02]

Combinatorial auctions (Some) research issues Preference elicitation [Sandholm 02] Bidding languages [Nissan 00] & restrictions [Rothkopf 98] Approximation –relation to incentive compatibility [Lehmann 99] and bounded rationality [Nisan & Ronen 00] False-name bidders [Yokoo 01] Winner determination –GVA (VCG) mechs, iterative mechs [Parkes 99]; smart markets –integer programming; specialized heuristics [Sandholm 99] FCC spectrum auctions Optimal auction design [Ronen 01] More: [Vries & Vohra 02] [Brewer 99]

search las vegas travel, Yahoo! Sponsored search Space next to search results is sold at auction las vegas travel auction

Sponsored Search Auctions Search engines auction off space next to search results, e.g. digital camera Higher bidders get higher placement on screen Advertisers pay per click: Only pay when users click through to their site; dont pay for uncliked view (impression)

Sponsored Search Sponsored search auctions are dynamic and continuous: In principle a new auction clears for each new search query Prices can change minute to minute; React to external effects, cyclical & non-cyc –flowers before Valentines Day –Fantasy football –People browse during day, buy in evening –Vioxx

Example price volatility: Vioxx

Sponsored Search Today 2005: ~ $7 billion industry –2004: ~ $4B; 2003: ~ $2.5B; 2002: ~ $1B $5 billion 2005 US ad revenue (41% of US online ads; 2% of all US ads) Resurgence in web search, web advertising Online advertising spending still trailing consumer movement online For many businesses, substitute for eBay Like eBay, mini economy of 3rd party products & services: SEO, SEM

Sponsored Search A Brief & Biased History Idealab GoTo.com (no relation to Go.com) –Crazy (terrible?) idea, meant to combat search spam –Search engine destination that ranks results based on who is willing to pay the most –With algorithmic SEs out there, who would use it? GoTo Yahoo! Search Marketing –Team w/ algorithmic SEs, provide sponsored results –Key: For commercial topics (LV travel, digital camera) actively searched for, people dont mind (like?) it –Editorial control, invisible hand keep results relevant Enter Google –Innovative, nimble, fast, effective –Licensed Overture patent (one reason for Y!s ~5% stake in G)

Sponsored Search A Brief & Biased History In the beginning: –Exact match, rank by bid, pay per click, human editors –Mechanism simple, easy to understand, worked, somewhat ad hoc Today & tomorrow: –AI match, rank by expected revenue (Google), pay per click/impression/conversion, auto editorial, contextual (AdSense, YPN), local, 2nd price (proxy bid), 3rd party optimizers, budgeting optimization, exploration exploitation, fraud, collusion, more attributes and expressiveness, more automation, personalization/targeting, better understanding (economists, computer scientists)

Sponsored Search Research A Brief & Biased History Weber & Zeng, A model of search intermediaries and paid referrals Bhargava & Feng, Preferential placement in Internet search engines Feng, Bhargava, & Pennock Implementing sponsored search in web search engines: Computational evaluation of alternative mechanisms Feng, Optimal allocation mechs when bidders ranking for objects is common Asdemir, Internet advertising pricing models Asdemir, A theory of bidding in search phrase auctions: Can bidding wars be collusive? Mehta, Saberi, Vazirani, & Vaziran AdWords and generalized on-line matching 1st & 2nd Workshop on Sponsored Search Auctions at ACM Electronic Commerce Conference

Allocation and pricing Allocation –Yahoo!: Rank by decreasing bid –Google: Rank by decr. bid * E[CTR] Pricing –Pay next price: Min price to keep you in current position –NOT Vickrey pricing, despite Google marketing collateral; Not truthful –Vickrey pricing possible but more complicated

Some Challenges Predicting click through rates (CTR) Detecting click spam Pay per action / conversion Number of ad slots Improved targeting

A prediction market Take a prediction question, e.g. Turn it into a financial instrument payoff = realized value of variable = 6 ? = 6 $1 if 6 $0 if I am entitled to: US08Pres = Clinton? 2007 CA Earthquake?

Aside: Terminology Key aspect: payout is uncertain Called variously: asset, security, contingent claim, derivative (future, option), stock, prediction market, information market, gamble, bet, wager, lottery Historically mixed reputation –Esp. gambling aspect –A time when options were frowned upon But when regulated serve important social roles...

Why? Reason 1 Get information price expectation of outcome (in theory, lab experiments, empirical studies,...more later) Do you have a prediction question whose expected outcome youd like to know? A market in uncertainty can probably help

Why? Reason 1: Information Information market: financial mechanism designed to obtain estimates of expectations of prediction outcomes Easy as 1, 2, 3: 1.Take a random variable whose expectation youd like to know 2.Turn it into a financial instrument (payoff= realized value of variable) 3.Open a market in the financial instrument price(t) E t [X] (in many cases,... more later)

Getting information Non-market approach: ask an expert –How much would you pay for this? A: $5/36 $ –caveat: expert is knowledgeable –caveat: expert is truthful –caveat: expert is risk neutral, or ~ RN for $1 –caveat: expert has no significant outside stakes = 6 $1 if 6 $0 if I am entitled to:

Getting information Non-market approach: pay an expert –Ask the expert for his report r of the probability P( ) –Offer to pay the expert $100 + log r if $100 + log (1-r) if It so happens that the expert maximizes expected profit by reporting r truthfully –caveat: expert is knowledgeable –caveat: expert is truthful –caveat: expert is risk neutral, or ~ RN –caveat: expert has no significant outside stakes = 6 6 logarithmic scoring rule, a proper scoring rule

Getting information Market approach: ask the publicexperts & non-experts alikeby opening a market: Let any person i submit a bid order: an offer to buy q i units at price p i Let any person j submit an ask order: an offer to sell q j units at price p j (if you sell 1 unit, you agree to pay $1 if ) Match up agreeable trades (many poss. mechs...) = 6 $1 if 6 $0 if I am entitled to: = 6

Getting information Market approach: ask the publicexperts & non-experts alikeby opening a market: If, at any time, for any bidder i and ask-er j, p i > p j, then i&j trade min(q i,q j ) units at price {p j,p i } In equilibrium (no trades) –max bid p i < min ask p j = bid-ask spread – bounds aggregate public opinion of expectation = 6 $1 if 6 $0 if I am entitled to:

Aside: Mechanism alternatives This is the continuous double auction (CDA) Many other market & auction mechanisms work: –call market –pari-mutuel market –market scoring rules –CDA w/ market maker –Vegas bookmaker, others Key: Market price = aggregate estimate of expected value [Hanson 2002]

Real predictions For dice example, no need for market: E[x] is known; no one should disagree Real power comes for non-obvious predictions, e.g. $1 if ;$0 otherwise I am entitled to: $x if interest rate = x on Jan 1, 2004 I am entitled to:

$1 if ;$0 otherwise I am entitled to: Bin Laden captured $max(0,x-k) if MSFT = x on Jan 1, 2004 I am entitled to: call option $f(future weather) I am entitled to: weather derivative $1 if Kansas beats Marq. by > 4.5 points; $0 otherw. I am entitled to:

IPOIPOIPOIPO

Play money; Real predictions

Cancer cured by 2010 Machine Go champion by

Does it work? Yes... Evidence from real markets, laboratory experiments, and theory indicate that markets are good at gathering information from many sources and combining it appropriately; e.g.: –Markets like the Iowa Electronic Market predict election outcomes better than polls [Forsythe 1992, 1999][Oliven 1995][Rietz 1998][Berg 2001][Pennock 2002] –Futures and options markets rapidly incorporate information, providing accurate forecasts of their underlying commodities/securities [Sherrick 1996][Jackwerth 1996][Figlewski 1979][Roll 1984][Hayek 1945] –Sports betting markets provide accurate forecasts of game outcomes [Gandar 1998][Thaler 1988][Debnath EC03][Schmidt 2002]

Does it work? Yes... E.g. (contd): –Laboratory experiments confirm information aggregation [Plott 1982;1988;1997][Forsythe 1990][Chen, EC-2001] –And field tests [Plott 2002] –Theoretical underpinnings: rational expectations [Grossman 1981][Lucas 1972] –Procedural explanation: agents learn from prices [Hanson 1998][Mckelvey 1986][Mckelvey 1990][Nielsen 1990] –Proposals to use information markets to help science [Hanson 1995], policymakers, decision makers [Hanson 1999], government [Hanson 2002], military [DARPA FutureMAP, PAM] –Even market games work! [Servan-Schreiber 2004][Pennock 2001]

Why? Reason 2 Manage risk If is horribly terrible for you Buy a bunch of and if happens, you are compensated = 6 $1 if 6 $0 if I am entitled to: = 6

Why? Reason 2 Manage risk If is horribly terrible for you Buy a bunch of and if happens, you are compensated $1 if$0 if I am entitled to:

The flip-side of prediction: Hedging (Reason 2) Allocate risk (hedge) –insured transfers risk to insurer, for $$ –farmer transfers risk to futures speculators –put option buyer hedges against stock drop; seller assumes risk Aggregate information –price of insurance prob of catastrophe –OJ futures prices yield weather forecasts –prices of options encode prob dists over stock movements –market-driven lines are unbiased estimates of outcomes –IEM political forecasts

Reason 2: Manage risk What is insurance? –A bet that something bad will happen! –E.g., Im betting my insurance co. that my house will burn down; theyre betting it wont. Note we might agree on P(burn)! –Why? Because Ill be compensated if the bad thing does happen A risk-averse agent will seek to hedge (insure) against undesirable outcomes

E.g. stocks, options, futures, insurance,..., sports bets,... Allocate risk (hedge) –insured transfers risk to insurer, for $$ –farmer transfers risk to futures speculators –put option buyer hedges against stock drop; seller assumes risk –sports bet may hedge against other stakes in outcome Aggregate information –price of insurance prob of catastrophe –OJ futures prices yield weather forecasts –prices of options encode prob dists over stock movements –market-driven lines are unbiased estimates of outcomes –IEM political forecasts

Examples I buy MSFT stock at s. Im afraid it will go down. I buy a put option that pays Max[0,k-s] – k is strike price. If s goes down below k, my stock investment goes down, but my option investment goes up to compensate Im a farmer. Im afraid corn prices will go too low. I buy corn futures to lock in a price today.

Examples I own a house in CA. Im afraid of earthquakes. I pay an insurance premium so that, if an earthquake happens, I am compensated. I am an Oscar-nominated actor. Im afraid Im going to lose. I bet against myself on an offshore gambling site. If I do lose, I am compensated. (Except that the offshore site disappears and refuses to pay… )

What am I buying? When you hedge/insure, you pay to reduce the unpredictability of future wealth Risk-aversion: All else being equal, prefer certainty to uncertainty in future wealth Typically, a less risk-averse party (e.g., huge insurance co, futures speculator) assumes the uncertainty (risk) in return for an expected profit

On hedging and speculating Hedging is an act to reduce uncertainty Speculating is an act to increase expected future wealth A given agent engages in a (largely inseparable) mixture of the two Both can be encoded together as a maximization of expected utility, where utility is a function of wealth

On hedging and speculating Why would two parties agree to trade in a prediction market? 1.Speculation. They disagree on expected values (probs) 2.Hedging. They differ in their risk attitude or exposure – they trade to reallocate risk 3.Both (most likely) Aside: legality is murky, though generally (2) is legal in the US while (1) often is not. In reality, it is nearly impossible to differentiate.

On computational issues Information aggregation is a form of distributed computation Agent level –nontrivial optimization problem, even in 1 market; ultimately a game-theoretic question –probability representation, updating algorithm (Bayes net) –decision representation, algorithm (POMDP) –agent problems computational complexity, algorithms, approximations, incentives some

On computational issues Mechanism level –Single market What can a market compute? How fast (time complexity)? Do some mechanisms converge faster (e.g., subsidy) –Multiple markets How many securities to compute a given fn? How many secs to support sufficient social welfare? (expressivity and representational compactness) Nontrivial combinatorics (auctioneers computational complexity; algorithms; approximations; incentives) some

On computational issues Machine learning, data mining –Beat the market (exploiting combinatorics?) –Explain the market, information retrieval –Detect fraud some

Catalysts Markets have long history of predictive accuracy: why catching on now as tool? No press is bad press: Policy Analysis Market (terror futures) Surowiecki's Wisdom of Crowds Companies: –Google, Microsoft, Yahoo!; CrowdIQ, HSX, InklingMarkets, NewsFutures Press: BusinessWeek, CBS News, Economist, NYTimes, Time, WSJ,...