1. 1 Mechanism Design for Computationally Bounded Agents Kate Larson Carnegie Mellon University Thesis Proposal April 8, 2002

2. 2 Introduction Fundamental problem in building open distributed systems: –There are users and agents acting in their own self interest. –Designer wishes to guarantee optimal system wide solutions while being unable to enforce particular behavior on users.

3. 3 Introduction Techniques from game theory and mechanism design have been very useful: –Routing in networks –Resource allocation in operating systems –Auction design in electronic markets –Bidding agents for electronic auctions –….

4. 4 Introduction However, perfect rationality is often assumed! But in real systems… Hard problems CostTime constraints

5. 5 Package Delivery and Vehicle Routing Chicago to Home Chicago to Toronto Home to Chicago Chicago Toronto Home Depot 2 3 4 5 1 The delivery route can be computed. Toronto to Home

6. 6 Package Delivery and Vehicle Routing Chicago to Home Chicago to Toronto Home to Chicago Chicago Toronto Home Depot Toronto to Home 1 2 3 4 5 The new package can be easily fit into the delivery route. The cost can be kept low, and thus the bid also.

7. 7 Computing of Valuations Agents may need to compute their valuations for (bundles of) goods –Vehicle routing problem in transportation exchanges –Manufacturing scheduling problem in procurement Value of a set of items = value of solution with items - value of solution without May involve solving multiple NP-complete problems –Infeasible to solve optimally

8. 8 Incentives Game theory handles incentives –System designer specifies the mechanism (auction rules) –Each agent picks its own strategy Computational issues have to be handled also! –Computing as part of an agent’s strategy Interaction between incentives and computing –Equilibrium for rational agents is not always one for computationally bounded agents!

9. 9 My Approach Resource Bounded Reasoning from AI Game Theory and Mechanism Design Normative model of bounded rationality. Mechanism design for computationally bounded agents.

10. 10 Thesis Statement It is possible to incorporate agents’ computational actions into a game theoretic model. Agents’ computational restrictions have a strong impact on the choice of strategies. It is possible to design mechanisms that take into account agents’ computational restrictions.

11. 11 Structure Model of bounded rationality Game theoretic formulation Impact on existing mechanisms Mechanisms for bounded agents Experiments

12. 12 Outline A model of bounded rationality –A normative model of deliberation control for computationally bounded agents Game theory for computationally bounded agents The impact of agents’ computation Mechanism design for computationally bounded agents Experiments Schedule

13. 13 Restricted Computational Resources Agents have limited computational resources –Deadlines All parcels must be delivered by Friday evening! Battery on my laptop only lasts 3 hours –Cost associated with computation and deliberation Sending jobs to rendering farms

14. 14 Anytime Algorithms Anytime algorithms can be used to approximate valuations –Anytime algorithms can return a solution at any time –Solution improves over time Allows a trade off between computing time and solution quality Examples –Iterative refinement algorithms: Local search, simulated annealing –Search algorithms: Depth first search, branch and bound

15. 15 Performance Profiles Performance profiles describe how computation changes the solution –Characterize the quality of an algorithm’s output as a function of computing time Performance profile representation is important –Earlier methods were not normative: They did not capture all possible ways an agent can control its deliberation

16. 16 Performance Profiles Computing time Solution quality Deterministic performance profile Solution quality Variance introduced by different problem instances Computing time [Horvitz 87, 89, Dean & Boddy 89] Optimum

17. 17 Conditioning on solution quality so far [Hansen & Zilberstein 96] Ignores conditioning on the path Table-based Representation of Uncertainty in Performance Profiles Computing time Solution quality [Zilberstein & Russell 91, 96].08.19.24.15.30.17.39.16.10.16.25.30.22.08.04.17.20.22.30.24.19.15.09.10.20.22.23.37.31.13.15.11.14.33.18.21.18.08.22.17.25.24.15.13.40.31.15.19.05.15.20.03

18. 18 Performance Profile Tree 0 4 2 4 5 10 7 15 20 A P(B|A) B 5 C P(C|A) Solution Quality Value Node Random Node P(1) P(2) P(3) [Larson & Sandholm 2001a] Stores value of solution so far Captures uncertainty from random algorithms, unknown algorithms

19. 19 Properties of Performance Profile Trees Normative - allows conditioning on everything including –Path of solution quality –Path of other solution features –Problem instance features Can model uncertainty from –Randomized algorithms –Lack of knowledge about what algorithms other agents use –Problem instances

20. 20 Outline A model of bounded rationality Game theory for computationally bounded agents –Roles of computation –Strategic computing –Deliberation equilibrium Effects of agents’ computation Mechanism design for computationally bounded agents Experiments Schedule

21. 21 Roles of Computing Improving: –Agents compute to improve valuations –Computation changes the valuation Example: Solving TSP’s to find cost of best route Refining: –Agents compute to refine valuations –Computation changes agents’ beliefs about valuations Example: Researching if an art work is a forgery

22. 22 Roles of Computation For computationally bounded agents, computing has several strategic roles: 1.Improves or refines the solution to the agent’s own problem 2.Reduces uncertainty as to what future computing steps will bring 3.Improves the agent’s knowledge about others’ valuations 4.Improves the agent’s knowledge about what problems others may have computed on

23. 23 Strategic Computing Good estimates of other agents’ valuations can allow an agent to tailor its strategy to achieve higher utility Strong strategic computing: Agent uses some of its deliberation resources to compute on others’ problems Weak strategic computing: Agent uses information from others’ performance profiles

24. 24 Strategic Computing How an agent should allocate its computation can depend on – how others allocate their computation – what actions they all plan on taking Incorporate computation actions into agents’ strategies. Analyze games for deliberation equilibria

25. 25 Deliberation Equilibria A (Nash, dominant, etc.) deliberation equilibrium for computationally bounded agents is a (Nash, dominant, etc.) equilibrium where: 1.The agents’ computing strategies form a (Nash, dominant, etc) equilibrium, and 2.The agents’ noncomputing strategies form a (Nash, dominant, etc) equilibrium.

26. 26 Outline A model of bounded rationality Game theory for computationally bounded agents Effects of agents’ computation –One-to-One Negotiation (Bargaining) –One-to-Many Negotiation (Auctions) –Many-to-Many Negotiation (Exchanges) Mechanism design for computationally bounded agents Experiments Schedule

27. 27 Bargaining A A B B AB A B BA

28. 28 Bargaining [Larson and Sandholm, 2001a] Take-it-or-leave-it: One agent makes an offer to the other who can accept or reject. –Determined under what settings agents have dominant strategies. –Determined under what settings agents will not split their computation. –Designed algorithms for determining offline, agents optimal online strategies

29. 29 Bargaining [Larson and Sandholm, 2002] Alternating Offers Model: Agents take turns making proposals and counter proposals Despite richer strategy space, alternating offers model is similar to take-it-or-leave it model –In (Bayes Nash) deliberation equilibrium, agents behave as though they were in take-it-or-leave-it setting

30. 30 Future Bargaining Work So far have only worked in settings where computation is free but limited –What happens if computation is costly? Conjecture: Costly computation will not change the results.

31. 31 Auctions and Strategic Computing yes no Generalized Vickrey On which  agent, bundle  pair to allocate next computation step ? Multiple items for sale noEnglish ( 1 st price ascending ) yes no Vickrey ( 2 nd price sealed bid ) yesDutch ( 1 st price descending) yes First price sealed-bidSingle item for sale Costly computation Limited computation Strong strategic computing Counter- speculation by rational agents ? Auction mechanism [Larson and Sandholm 2001b, Larson and Sandholm 2001c]

32. 32 Exchanges Exchanges are a generalization of auctions where there are potentially multiple buyers and multiple sellers –Does strategic computing occur in exchanges? –Does the type of restriction on agents’ computational capabilities determine the occurrence of strategic computation? Conjecture: Strategic computation can occur with either type of restriction (costly or limited computation).

33. 33 Outline A model of bounded rationality Game theory for computationally bounded agents Effects of agents’ computation Mechanism design for computationally bounded agents Experiments Schedule

34. 34 Mechanism Design So far have focused on standard mechanisms and determining their strengths and limitations if agents have computational restriction. Change focus: 1.Mechanisms for computationally restricted agents. 2.Measures for mechanisms.

35. 35 Issues for Mechanism Design There are several issues that must be addressed for computationally restricted agents –What should agents report to the mechanism? Valuations once computed? Valuations agents expect to obtain? Results of partial computation? –What is the role of the mechanism? To specify allocations only? To specify allocations and computation policies?

36. 36 Issues for Mechanism Design Does there exist a mechanism that is: –Strategy proof, –Efficient, –Individually rational, and –Limits wastage of computational resources.

37. 37 Measuring Mechanisms: Miscomputing Ratio How does allowing agents free choice of computing policies effect the outcome, o? Miscomputing Ratio: R=SW(o*)/SW(o(NE)) SW(o*)=Social welfare if a global controller enforces computing policies so as to maximize social welfare SW(o(NE))=Social welfare of worst Nash Eq.

38. 38 Results and Future Work Results (Vickrey auctions): –If agents have limited or costly computation then the miscomputing ratio can be unbounded. –The choice of appropriate cost functions can motivate bidders to choose strategies that maximize social welfare. Future Work: –Constraining deadlines or cost functions –Using performance profile trees to predict R –Constraining performance profile trees –Comparing mechanisms using miscomputing ratio, R

39. 39 Outline A model of bounded rationality Game theory for computationally bounded agents Effects of agents’ computation Mechanism design for computationally bounded agents Experiments Schedule

40. 40 Experiments Goals: –To demonstrate that the performance profile tree is a feasible model. –To demonstrate that the performance profile tree model is general. –To determine if strategic computing is an artifice of analysis or something that occurs in the “real world”. Tools for controlling computation and visualization of deliberation and negotiation

41. 41 Status Model of bounded rationality Game theoretic formulation Impact on existing mechanisms Mechanisms for bounded agents Experiments completed partially future work

42. 42 Schedule Sp02Sm02F02Sp03Sm03F03

43. 43 Related Publications Larson and Sandholm, 2001a, Bargaining with Limited Computation:Deliberation Equilibrium, Artificial Intelligence, 132(2), pp.183-217 Larson and Sandholm 2001b, Computationally Limited Agents in Auctions, Workshop on Agent based approaches to B2B, Autonomous Agents 2001 Larson and Sandholm, 2001c, Costly Valuation Computation in Auction, TARK VIII Larson and Sandholm, 2002, An Alternating Offers Bargaining Model for Computationally Limited Agents, AAMAS 2002 Larson and Sandholm, 2002, Miscomputing Ratio: The Social Cost of Selfish Computing, draft.

44. 44 Other Publications Larson and Sandholm, 2000, Anytime Coalition Generation: An Average Case Study, Journal of Experimental and Theoretical Artificial Intelligence, 12(2000),23-42. Sandholm, Larson, Andersson, Shehory, and Tohme, 1999, Anytime Coalition Structure Generation with Worst Case Guarantees, Artificial Intelligence, (111)1-2, pp209-238.

45. 45 Future Auction Work: Reverse Auctions Are they different from standard auctions if agents have bounded computational resources? –Maybe or maybe not! –Reverse auctions differ from auctions in: Complexity of winner determination Communication complexity –At first look, it does not appear as though bidding strategies are different.

46. 46 Related Work Parkes 2001, Iterated combinatorial auctions so that both the computational burden of agents and mechanism are reduced Persico 2000, Information acquisition in auctions Bergemann and Pesendorfer 2001, Model where the auctioneer controls what information agents have available to them Bergemann and Välimäki 2001, Costly information acquisition, but no clear model of how information is obtained

47. 47 Related Work Nisan and Ronen 2000: Algorithmic mechanism design. What happens if the mechanism is computationally limited? Investigates ways of introducing approximation algorithms to compute outcomes without loosing incentive compatibility. Kfir-Dahav, Monderer, Tennenholtz 2000

48. 48 Experiments To demonstrate that the performance profile tree is a feasible model: –Multi-item auction where agents submit bids where the value is determined by the solution to a TSP problem –Prebundling to simplify the winner determination problem and to restrict the number of optimization problems agents are faced with –Focus on having agents use performance profile trees, determining how to make the performance profile tree model feasible for actual problems.

49. 49 Experiments To demonstrate that the performance profile tree is general: –Single item auction where agents valuations of the item is based on schedules they compute. –Use off the shelf schedulers (Micro-Boss, or a stochastic scheduler from Steve Smith’s group) to show that the techniques I propose are general

50. 50 Experiments To determine if strategic computation is an artifice of analysis or something that occurs in the “real world”: –Multi-item reverse auction using data from actual delivery companies –Agents must solve multiagent vehicle routing problems –How does one tailor performance profile trees to actual data without losing their analytical power? –Does strategic computing occur?

51. 51 Game Theory Concepts Game theory: method of studying systems of agents in settings where there are strategic interactions Game has –Set of agents, A –Set of outcomes, O –Set of actions for each agent Strategy: contingency plan that determines what action the agent will take at any point in the game Equilibrium is fundamental concept in game theory –Stable point in space of agents’ strategies

52. 52 Game Theory Concepts Different equilibrium concepts –Dominant strategy equilibrium Every agent has a strategy that they are best off playing, independent of other agents actions –Nash equilibrium The strategy one agent plays is optimal given what strategies other agents are playing Mechanism Design: –Study of how to implement good system wide solutions that involve mulitple self-interested agents

53. 53 Mechanism Design Mechanism Design: –Study of how to implement good system wide solutions that involve multiple self-interested agents Mechanism: M=(S 1,…,S n,g(.)) is a set of strategies S i available to each agent i, and an outcome function g(s). –Defines strategies available to agents –Describes methods used to determine outcome based on agents’ strategies

54. 54 Game Theory Game theory studies phenomena that occur when decision makers interact –It is assumed that decision makers are strategic. If he bids $10 then I could bid $11 and win. If he bids $11 then I could bid $12 and win.

55. 55 Auctions with Individual Bidders In auctions where the bidders are individual consumers, valuations are often idiosyncratic $5 for the bear $100 for the bear

56. 56 The Two Extremes Hamlet: “What a piece of work is a man! How noble in reason! How infinite in faculties!” Hamlet, II.2.319 Puck: “Lord, what fools these mortals be!” Midsummer Night’s Dream, III.3.116

57. 57 One Solution? “It is evident that the rational thing to do is to be irrational, where deliberation and estimation cost more than they are worth.” Frank Knight, Risk, Uncertainty and Profit, 1921

58. 58 Example of Bounded Rationality According to game theory, chess is a trivial game. - finite, zero sum, two player game with full information Then why do we find it so challenging to play and why is it difficult to get computers to play well?