Reconstructing Preferences from Opaque Transactions Avrim Blum Carnegie Mellon University Joint work with Yishay Mansour (Tel-Aviv) and Jamie Morgenstern (Penn) [LATA 2016]

A very classic problem Given random samples of some function, learn a good approximation to that function. Basically, all of (supervised) machine learning.        

A less-classic but still well-studied problem Given random observations of some agent’s actions, learn what the agent is trying to optimize. –Inverse reinforcement learning. [Ng-Russell00] [Abbeel-Ng04] [Syed-Schapire07]… –Learning from revealed preferences. [Beigman-Vohra06] [Zadimoghaddam-Roth12] [Balcan-Daniely-Mehta-Urner-Vazirani14]

Today’s problem Suppose you don’t have access to agents in isolation. Given only results of multi-agent interactions, can you reconstruct agents’ preferences? –Observing the outcomes of a series of auctions (just who won – e.g, which ads appeared on pages), reconstruct agent preferences. –From observing which agents end up happy in some allocation mechanism, reconstruct both the agent preferences and the mechanism. Will talk today about a few problems in this space, coming from Algorithmic Economics

Can also participate ourselves. Two models (high-level) Suppose you don’t have access to agents in isolation. Given only results of multi-agent interactions, can you reconstruct agents’ preferences?

Can also participate ourselves. Two models (high-level) Suppose you don’t have access to agents in isolation. Given only results of multi-agent interactions, can you reconstruct agents’ preferences? –Combinatorial auction. Agents deterministic but prefs unknown. Auctioneer mechanism also unknown. Goal: recover agent prefs and auctioneer’s priorities.

Consider the following scenario… There are n bidders (jobs), Some end up happy, some not. Bidders have demand sets. Mechanism has resources it allocates using some priority ordering unknown to us. Goal: reconstruct the unknowns from observations, or at least be able to predict well. and one auctioneer (allocator).

Mechanism has priority order: Consider the following scenario…

Special case: 1 item (wanted by all) Say each day two bidders arrive Mechanism has priority order: Like inverse comparison-based sorting (adversary picks pair to compare, you predict outcome, you pay 1 per mistake).

General case (single-minded) Idea: rather than trying to learn the subsets (since we can’t see the items anyway), instead learn the conflict graph. At the same time as we try to learn the ordering

General case (single-minded) Start by assuming all are in conflict. Start with all bidders at the top of the ordering. Predict using conflict graph + arbitrary linearization of our partial order.

General case (single-minded) If correct answer includes a pair connected by an edge, can delete it. Start with all bidders at the top of the ordering. Predict using conflict graph + arbitrary linearization of our partial order.

General case (single-minded) If correct answer includes a pair connected by an edge, can delete it. If not but still a mistake, argue can safely demote some bidder(s). Predict using conflict graph + arbitrary linearization of our partial order.

General case (single-minded) If not but still a mistake, argue can safely demote some bidder(s). Show: no bidder ever demoted past its correct level if update only 1 st mistake. Use fact that our conflict graph is superset of correct one.

General case (single-minded) E.g., subset of advertisers request to show banner ad on a given page. Don’t want to be shown with competitor from same industry. Can also be applied to cases where conflicts don’t come from items. (or can view each conflict as a virtual item).

How about unit-demand Bidders have subset of interest, but just want one. Have preference ordering and will take highest that’s available. Seller has unknown priority over bidders. Let’s say you see what each bidder exits with. Goal: predict what will happen (who gets what) on new set of bidders.

How about unit-demand Bidders have subset of interest, but just want one. Have preference ordering and will take highest that’s available. One way to solve: by reduction to the previous case. –Two “copies” in conflict if correspond to same bidder or same item.

How about unit-demand Bidders have subset of interest, but just want one. Have preference ordering and will take highest that’s available. Can do more efficiently by combining with “inverse-sorting” –Use “inverse-sorting” algorithm for learning each bidder’s ordering. –Combined with previous algorithm for learning auctioneer’s ordering.

Learning the presence of variable prices –Idea: again, maintain partial ordering of bidders where each bidder assigned to some level, and no bidder farther down than should be –On incorrect prediction, look at first mistake (in our ordering) and use to add new linear constraint for that bidder (separation oracle). –Combine with online LP alg (like Ellipsoid).

Learning the presence of variable prices

Now to Bayesian model Let’s make the problem easier: just one item, auctioneer sells to highest bidder. (say at 2 nd -highest price but won’t see prices)

Some difficulties Even if you get to see winning bid, it’s not a representative draw from bidder’s distribution.

A really useful tool: Kaplan-Meier estimator Doing a study of survival rates after a medical procedure. But patients keep dropping out of the study. Idea: still getting good estimates of Pr(survive one more day | still in study). Chain together.

A really useful tool: Kaplan-Meier estimator Doing a study of survival rates after a medical procedure. But patients keep dropping out of the study.

Back to the auction Here, we don’t get to see winning bid, but can participate ourselves. Want to use to simulate Kaplan-Meier.

Back to the auction Two key ingredients:

Back to the auction Two key ingredients: 2.Control the errors to make sure they don’t blow up as you chain from high prices down to low prices. –Key trick is viewing error as having both a multiplicative and additive component, which helps with the induction.

Back to the auction E.g., you don’t have the event of person 2 showing up being correlated with the event of person 1 having a high value for the item…

Extended models What if items have a common-value component?

Extended models

Open problems In first part: –Analyze other natural allocation mechanisms? –Analyze other natural valuation classes / add noise? In second part: –Add combinatorial component? –What if you can’t participate, and instead you see the price paid by the winner, for Vickrey (2 nd price) auction? –Better mistake-bound for learning unit-demand bidders?

References Blum, Mansour, Morgenstern, “Learning What’s Going On: Reconstructing Preferences and Priorities from Opaque Transactions,” Proc. ACM-EC Blum, Mansour, Morgenstern, “Learning Valuation Distributions from Partial Observation,” Proc. AAAI 2015.