Distributed Multiagent Resource Allocation In Diminishing Marginal Return Domains Yoram Bachrach(Hebew University) Jeffrey S. Rosenschein (Hebrew University)

Outline Multiagent Resource Allocation (MARA)  General problem  Applications Centralized and decentralized mechanisms  Selfish behavior challenge Specific restricted domain  VCG solution in restricted domain Allocation by interaction  Market motivation behind method Allocation protocol and suggested strategies  Convergence to optimal allocation  Strategic and selfish behaviour  Expected time to convergence Conclusions and future research

Multiagent Resource Allocation Allocating resources to users  Scarce resources  Selfish agents with private information Both users and resource owners An allocation maps resources to users

MARA Applications Industrial procurement Satellite resources Tasks in manufacturing systems Grid computing RF spectrum and coverage …

MARA Domain Properties Divisible / Indivisible  Can parts of a single resource be allocated to several agents? Sharable / Non-Sharable  Can a resource be allocated to several agents simultaneously? Single-Unit / Multi-Unit  Are there bundles of identical resources? Transferable / Non Transferable Utility  Can agents compensate by transferring utility among them?

MARA Approaches Attempt to maximize social welfare  Other possible goals – Maximin, fairness, …  There may be more than one optimal allocation Centralized mechanisms  A central mechanism gets the agents’ preferences and chooses an outcome Decentralized approaches  Agents actively participate in choosing the outcome Problem – agents are selfish and attempt to maximize their own utility

Centralized Mechanisms The mechanism must elicit the agents’ private information about allocations  But agents may manipulate to increase their own utility We are interested in incentive compatible mechanisms  Agents reply truthfully, under a certain rational behavior  Rational behavior captured in a game theoretic solution concept Vickery-Clarke-Groves (VCG) approach  Tax agents to make truth telling is a dominant strategy  Strategyproof, allocatively efficient but only weakly budget balanced

Distributed Mechanisms Central mechanisms may not be appropriate in distributed environments  Hard to establish a trusted central authority  Scalability concerns – the central mechanism may be a performance bottleneck Have agents interact among themselves to choose the allocation  Need to define the protocol for interaction  Selfish agents may still manipulate

Specific Domain Set of identical agents  Each agent only requires a single resource, and does not benefit from being allocated more than one resource Set of resources  Cannot be divided among agents  Can be shared among agents Diminishing marginal production  The total utility of the agents who are allocated a certain resource drops as more agents use that resource

Diminishing Marginal Return

Diminishing Marginal Return Total production is 10

Diminishing Marginal Return Total production increases to 14

Diminishing Marginal Return Total production increased by 4 when adding a single agent Marginal production of 4

Diminishing Marginal Return Total production increased by 1 when adding a single agent Marginal production of 1

What needs to be decided? A mechanism must decide:  An allocation – which agent gets which resource We want to maximize the social welfare – total production  Utility transfers Agents gain utility due to the allocation  Resource owners receive nothing Resource owners hold the private information  Eliciting this information requires incentivizing the resource owners to report their production function  Requires giving resource owners some of the utility We assume the total production across all the resources can be redistributed in any way

VCG in Restricted Domain Easy to compute an optimal allocation  Resources report total production functions  Find maximal social welfare by a greedy algorithm Assign to the resource with maximal marginal production Induce truthfullness by VCG tax  Requires establishing a trusted central authority Trust and security issues, central bottleneck, … Weakly budget balanced – some of the total production is kept in the mechanism and not distributed

Allocation by Interaction Define a protocol for interaction between agents and resource owners  Simulate a market for services Interaction proceeds in discrete time rounds  Each round determines both an allocation and transfers Design protocol and suggest interaction strategies so that the optimal allocation is always reached Challenges  Achieve the optimal allocation despite selfishness  Make sure the optimal allocation is reached quickly

Interaction Protocol R1 R2 R3 Round Payment (5) Currently on R1, getting utility 5

Interaction Protocol R1 R2 R3 Resource Request Currently on R1, getting utility 5

Interaction Protocol R1 R2 R3 Payment Bid (10)

Interaction Protocol R1 R2 R3 Accept Switch to R2 with utility 10

Interaction Protocol R1 R2 R3 Decline Stay on R1, with utility 5

Interaction Protocol R1 R2 R3 Round Payment 10 Currently on R2 with utility 10

Interaction Protocol R1 R2 Payment Change (5) Currently on R2 with utility 5

The Resource Owner’s Perspective Production – 12 Payments – 10 Utility – 2 Production – 13 Payments – 12 Utility – 1

Chosen Allocation The interaction decides both the allocation and redistribution of the utility  Agents are allocated the last resource whose bid they accepted  Agents get the utility as in the last payment bid they accepted  Resource owners keep the reminder of the production on the resource not redistributed to the agents The allocation may change at the end of every round An allocation is stable if once reached it never changes  Depends on the strategies of the participants Agents and resource owners

Suggested Strategy - Agents Each round, randomly choose a resource and request using the resource  If the bid in that resource is better than the current bid, switch to that resource (accept)  If the bid is lower than the current resource offers, stay with current resource

Suggested Strategy – Resource Owners Keep the agents’ share of the utility in the level of the marginal production on the resource On round start, offer all the agents allocated to the resource the current last marginal production Answer resource requests with bid of the next marginal production on the resource  If accepted, set the bid for all the agents to the new marginal production by a Payment Change message  If declined – do nothing

Resource Owners - Example MP = 4MP = 1

Protocol Stable Allocation Given a set of strategies for the agents and resource owners, a protocol stable allocation is one that, once reached, never changes  Under these strategies, no interaction results in an agent switching to a different resource Protocol stable under the suggested strategies  No agent is ever given a bid higher than what he is currently getting on his current resource  Resource owners bid the next marginal production  There is no resource where the next marginal production is greater than the current marginal production on other resources Similar to greedily allocating agents to resources according to marginal production

Convergence to Optimum Under the suggested strategies, the chosen allocation always converges to the optimal allocation  Monotonic improvement If an agent switches resources, the social welfare increases  Stability in optimum The optimal allocation is protocol stable  No “local” optima – protocol stable is optimal If a non optimal allocation is chosen, there is a possible round where an agent switches resources What about strategic behavior?

Strategic Behavior Agents and resource owners have to follow the protocol, but not the suggested strategies  Might obtain higher utility by choosing a different strategy Agents may accept a bid lower than what they currently have Resource owners may suggest a bid different than the current marginal production  Higher, to attract more agents  Lower, to give a lower share of utility to the agents Is such strategic behavior rational for self interested agents?

Strategic Agents (Our domain) If an agent gained from strategic behavior, we still reach an optimal allocation  If a single agent has deviated from the suggested strategy and gained utility Gained utility: a protocol stable allocation has been reached, in which the agent gets a higher utility  Then the reached protocol stable allocation is also optimal

Strategic Resource Owners Resource owners who set too high a bid  Attract more agents but pay more and lose utility Resource owners who set too low a bid  Pay less, but lose agents to competing resources who offer higher bids  When the domain is competitive for resource owners, such a manipulation is irrational Highly competitive settings  Condition that occurs mostly in environments where there are many resources with similar marginal production values Similar resources or slight changes in marginal production

Strategic Resource Owners In our specific domain  Diminishing marginal return  Highly competitive for resource owners If a resource owner gained from strategic behavior, we still reach an optimal allocation  If a single resource owner has deviated from the suggested strategy and gained utility Gained utility: a protocol stable allocation has been reached, in which the resource owner gets a higher utility  Then the reached protocol stable allocation is optimal

Convergence Time When agents and resource owners behave rationally, we converge to an optimal allocation  But how quickly is the optimal allocation reached? Under the suggested strategies  Expected time to convergence:  Bound on convergence time: Quick polynomial convergence

Related Work TFG-MARA survey  Y. Chevaleyre, P. E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J. A. Rodríguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation. Distributed mechanism design approaches  J. Feigenbaum and S. Shenker. Distributed algorithmic mechanism design: Recent results and future directions. Scheduling domains  B. Heydenreich, R. Muller, and M. Uetz. Decentralization and mechanism design for online machine scheduling. Negotiations over resources  U. Endriss, N. Maudet, F. Sadri, and F. Toni. Negotiating socially optimal allocations of resources.  T. W. Sandholm. Contract types for satisficing task allocation.

Conclusions A distributed approach to resource allocation in a specific domain  Achieves optimal allocation (maximal social welfare)  No central authority required  All utility divided among agents and resource owners “Strongly budget balanced”  Quick convergence Can a similar approach be applied to other domains (or more general domains)?