Negotiating Socially Optimal Allocations of Resources U. Endriss, N. Maudet, F. Sadri, and F. Toni Presented by: Marcus Shea

Introduction Consider a society of independent agents Agents have an initial allocation of indivisible resources Agents can make deals with one another in order to increase their utility

What class of deals will encourage our system to eventually reach a socially optimal state?

Introduction We will examine different classes of deals –Identify necessary and sufficient classes that will allow our society to converge to an optimal allocation

Introduction We will examine different classes of deals –Identify necessary and sufficient classes that will allow our society to converge to an optimal allocation Examples –1-deals without side payments –Multilateral deals with side payments

Introduction We will consider at different measures of social welfare –Changes definition of an ‘optimal’ allocation

Introduction We will consider at different measures of social welfare –Changes definition of an ‘optimal’ allocation Examples –Measure social welfare based on average utility of a system –Measure social welfare based on lowest utility of a system

Introduction Distributed approach to multiagent resource allocation –Local negotiation

Introduction Distributed approach to multiagent resource allocation –Local negotiation Compare to the centralized approach –Single entity decides on final allocation based on agents preferences over all allocations –Combinatorial auctions –May be difficult to find an ‘auctioneer’

Outline Preliminaries Rational Negotiation with Side Payments Rational Negotiation without Side Payments Egalitarian Agent Societies Conclusions

Preliminaries

Negotiation Framework Finite set of agents A Finite set of resources R Each agent i in A has a utility function u i that maps every set of resources to a real number

Allocation of Resources An allocation of resources is a function A from A to subsets of R such that A(i)∩A(j) = for i ≠ j An allocation of resources is just a partition of resources amongst the agents

Deals A deal is a pair δ = (A,A’) where A and A’ are distinct allocations of resources –‘old’ allocation and ‘new’ allocation The set of agents involved in a deal δ = (A,A’) is given by A δ = { i in A : A(i) ≠ A’(i) } - everyone whose set of resources has changed The composition of two deals δ 1 = (A,A’) and δ 2 = (A’,A’’) is δ 1 ◦δ 2 = (A,A’’) - two deals are processed simultaneously

Independently Decomposable A deal δ is independently decomposable if there exist deals δ 1 and δ 2 such that δ= δ 1 ◦δ 2 and A δ 1 ∩ A δ 2 = δ is made up of two subdeals concerning disjoint sets of agents δ =

Independently Decomposable A deal δ is independently decomposable if there exist deals δ 1 and δ 2 such that δ= δ 1 ◦δ 2 and A δ 1 ∩ A δ 2 = δ is made up of two subdeals concerning disjoint sets of agents δ = δ1δ1 δ2δ2 δ = δ 1 ◦δ 2

Utility Functions We may restrict our attention to utility functions u i with particular properties: –Monotonic: for all R 1,R 2 R –Additive: for all R R –0-1 Function: Additive and for all r in R –Dichotomous: for all R R

Utility Functions We may restrict our attention to utility functions u i with particular properties: –Monotonic: for all R 1,R 2 R –Additive: for all R R –0-1 Function: Additive and for all r in R –Dichotomous: for all R R An agent’s utility of an allocation is just the utility of his set of resources u i (A) = u i (A(i))

Rational Negotiation with Side Payments

We consider the scenario where agents can exchange money as well as resources We define a payment function as a function p from agents to real numbers that, when summed over agents, equals zero:

Rational Negotiation with Side Payments Our goal is to maximize utilitarian social welfare Utilitarian social welfare is just the sum of all agents utility –Maximizing is equivalent to maximizing average utility –Useful in any market where agents act individually

Individually Rational We assume our agents are rational We say a deal is individually rational if there exists a payment function so that every involved agent’s increase in utility is strictly greater than their payment Formally: deal δ = (A,A’) is individually rational if there exists a payment function p such that u i (A’) – u i (A) > p(i) for all agents i, except possibly p(i) = 0 for agents with A(i) = A’(i)

1-deals A 1-deal is a deal involving reallocation of exactly one resource Question: If (rational) agents are permitted to perform 1-deals only, will we eventually reach an optimal allocation?

1-deals Consider a system with two agents and two resources, r 1 and r 2 We specify the utility functions: Initial allocation A: Agent 1 has both resources u 1 ({}) = 0u 2 ({}) = 0 u 1 ({r1}) = 2u 2 ({r1}) = 3 u 1 ({r2}) = 3u 2 ({r2}) = 3 u 1 ({r1,r2}) = 7u 2 ({r1,r2}) = 8

1-deals Consider a system with two agents and two resources, r 1 and r 2 We specify the utility functions: Initial allocation A: Agent 1 has both resources –sw u (A) = 7, optimal allocation has value 8 –1-deals are not sufficient to get to an optimal allocation u 1 ({}) = 0u 2 ({}) = 0 u 1 ({r1}) = 2u 2 ({r1}) = 3 u 1 ({r2}) = 3u 2 ({r2}) = 3 u 1 ({r1,r2}) = 7u 2 ({r1,r2}) = 8

First Result We are going to move toward showing that if we allow our agents to perform arbitrary individually rational deals, then we will reach an optimal allocation through negotiation

Lemma 1 Lemma 1: A deal δ = (A,A’) is individually rational iff sw u (A) < sw u (A’) Intuition: If an entire society gets a strict increase in utility, then those profiting can payoff those who are losing so that everyone shares the gain

Thm 1: Maximal Utilitarian Social Welfare Theorem 1: Any sequence of individually rational deals will eventually result in an allocation A that maximizes sw u (A)

Thm 1: Maximal Utilitarian Social Welfare Theorem 1: Any sequence of individually rational deals will eventually result in an allocation A that maximizes sw u (A) Proof:

Thm 1: Maximal Utilitarian Social Welfare Theorem 1: Any sequence of individually rational deals will eventually result in an allocation A that maximizes sw u (A) Proof: Termination Argument –A and R finite means that there are only finitely many allocations –Lemma 1 gives that any individually rational deal strictly increases social welfare

Thm 1: Maximal Utilitarian Social Welfare Theorem 1: Any sequence of individually rational deals will eventually result in an allocation A that maximizes sw u (A) Proof:

Thm 1: Maximal Utilitarian Social Welfare Theorem 1: Any sequence of individually rational deals will eventually result in an allocation A that maximizes sw u (A) Proof: Suppose terminal allocation A is such that sw u (A) < sw u (A’) for some A’

Thm 1: Maximal Utilitarian Social Welfare Theorem 1: Any sequence of individually rational deals will eventually result in an allocation A that maximizes sw u (A) Proof: Suppose terminal allocation A is such that sw u (A) < sw u (A’) for some A’ ≠ A Then deal δ = (A,A’) increases social welfare, and thus is individually rational by Lemma 1, contradicting termination

Thm 1: Maximal Utilitarian Social Welfare Implications of Theorem 1 –Not really surprising Class of individually rational deals allows for any number of resources to be moved between any number of agents

Thm 1: Maximal Utilitarian Social Welfare Implications of Theorem 1 –Not really surprising Class of individually rational deals allows for any number of resources to be moved between any number of agents –Difficulty in actually finding an individually rational deal

Thm 1: Maximal Utilitarian Social Welfare Implications of Theorem 1 –Not really surprising Class of individually rational deals allows for any number of resources to be moved between any number of agents –Difficulty in actually finding an individually rational deal –We will not get stuck in a local optimum, any sequence will bring us to optimum allocation

Thm 1: Maximal Utilitarian Social Welfare Implications of Theorem 1 –Not really surprising Class of individually rational deals allows for any number of resources to be moved between any number of agents –Difficulty in actually finding an individually rational deal –We will not get stuck in a local optimal, any sequence will bring us to optimum allocation –This sequence could, however, be very long

Do we need the entire class of individually rational deals to guarantee that negotiation will eventually reach a socially optimal allocation?

Thm 2: Necessary Deals w/ Side Payments Theorem 2: Fix A, R. For every deal δ that is not independently decomposable, there exist utility functions and an initial allocation so that any sequence of individually rational deals leading to an optimal allocation must include δ.

Thm 2: Necessary Deals w/ Side Payments Theorem 2: Fix A, R. For every deal δ that is not independently decomposable, there exist utility functions and an initial allocation so that any sequence of individually rational deals leading to an optimal allocation must include δ. This remains true if we restrict utility functions to be monotonic, or dichotomous

Thm 2: Necessary Deals w/ Side Payments Theorem 2: Fix A, R. For every deal δ that is not independently decomposable, there exist utility functions and an initial allocation so that any sequence of individually rational deals leading to an optimal allocation must include δ. This remains true if we restrict utility functions to be monotonic, or dichotomous Proof: Carefully define utility functions and initial allocation so that δ is the only improving deal

Thm 2: Necessary Deals w/ Side Payments Implications of Theorem 2 –Any negotiation protocol that puts restrictions on the structural complexity of deals will fail to guarantee optimal outcomes if the class of utility functions is unrestricted, monotone, or dichotomous

Thm 2: Necessary Deals w/ Side Payments Implications of Theorem 2 –Any negotiation protocol that puts restrictions on the structural complexity of deals will fail to guarantee optimal outcomes if the class of utility functions is unrestricted, monotone, or dichotomous What can we do?

Thm 2: Necessary Deals w/ Side Payments Implications of Theorem 2 –Any negotiation protocol that puts restrictions on the structural complexity of deals will fail to guarantee optimal outcomes if the class of utility functions is unrestricted, monotone, or dichotomous What can we do? –Restrict utility functions –Change notion of social welfare

Additive Scenario Consider the scenario where utility functions are additive (no synergy effects) Will we be able to reach an optimal allocation without needing such a broad class of deals?

Thm 3: Additive Scenario Theorem 3: In additive scenarios, any sequence of individually rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare

Thm 3: Additive Scenario Theorem 3: In additive scenarios, any sequence of individually rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare Proof:

Thm 3: Additive Scenario Theorem 3: In additive scenarios, any sequence of individually rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare Proof: We get termination since we are looking at individually rational deals

Thm 3: Additive Scenario Proof:

Thm 3: Additive Scenario Proof:

Thm 3: Additive Scenario Proof:

Thm 3: Additive Scenario Are 1-deals necessary to achieve an optimal allocation in the additive scenario?

Thm 3: Additive Scenario Are 1-deals necessary to achieve an optimal allocation in the additive scenario? –Paper does not address this question

Thm 3: Additive Scenario Are 1-deals necessary to achieve an optimal allocation in the additive scenario? –Paper does not address this question –Easy to see that they are necessary: Let δ be a 1-deal that moves resource r 1 from agent i to agent j Give all resources to agent j, except r 1 to agent i Set u k ({r}) = 0 for every resource r, every agent k≠j Set u j ({r}) = 1 for every resource r Only individually rational deal is 1-deal δ

Class of Deals Side Payments Utility Functions Measure of Social Welfare Nature of Optimality Necessary / Sufficient Individually Rational DealsYes Unrestricted Monotonic DichotomousUtilitarian Global Maximum Sufficient[1] & Necessary[2] Individually Rational 1-dealsYesAdditiveUtilitarian Global Maximum Sufficient[3] & Necessary Cooperatively Rational DealsNo Unrestricted Monotonic DichotomousUtilitarian Pareto Optimal Sufficient[4] & Necessary[5] Cooperatively Rational 1-dealsNo0-1 FunctionsUtilitarian Global Maximum Sufficient[6] & Necessary Equitable DealsNo Unrestricted DichotomousEgalitarian Global Maximum Sufficient[7] & Necessary[8] Simple Pareto- Pigou-Dalton DealsNo0-1 FunctionsMixed Lorenz OptimalSufficient[9] Summary of Results

Rational Negotiation without Side Payments

Rational Negotiation w/o Side Payments Now we consider the scenario where there are no side payments made

Rational Negotiation w/o Side Payments Now we consider the scenario where there are no side payments made The class of individually rational deals no longer allows us to achieve optimal social welfare: –Agent 1 has sole resource r u1({}) = 0u2({}) = 0 u1({r}) = 1u2({r}) = 2

Rational Negotiation w/o Side Payments Maximizing social welfare is no longer possible in general We will instead see if a Pareto optimal outcome is possible, and what types of deals are sufficient to guarantee this outcome

Pareto Optimal A Pareto optimal allocation is one in which there is no other allocation with higher social welfare that would be no worse for any of the agents in the system Formally: Allocation A is Pareto optimal if there is no allocation A’ such that sw u (A) < sw u (A’) and u i (A) ≤ u i (A’) for all agents i

Pareto Optimal Recall our previous example –Agent 1 has sole resource r –This is Pareto optimal since agent 1 is worse off by giving resource r to agent 2, even though it would increase social welfare u1({}) = 0u2({}) = 0 u1({r}) = 1u2({r}) = 2

Cooperative Rationality We say a deal is cooperatively rational if no agent’s utility decreases, but at least one agent’s utility strictly increases Formally: We say a deal δ = (A,A’) is cooperatively rational if u i (A) ≤ u i (A’) for all agents i and there is an agent j such that u j (A) < u j (A’) We examine the class of cooperatively rational deals for the scenario without side payments

Thm 4: Pareto Optimal Outcomes Theorem 4: Any sequence of cooperatively rational deals will eventually result in a Pareto optimal allocation of resources Very similar proof to Theorem 1

Thm 5: Necessary deals w/o side payments Theorem 5: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of cooperatively rational deals leading to a Pareto optimal allocation would have to include δ

Thm 5: Necessary deals w/o side payments Theorem 5: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of cooperatively rational deals leading to a Pareto optimal allocation would have to include δ Still holds if utility functions are restricted to be monotonic or dichotomous

Thm 5: Necessary deals w/o side payments Analogously to Theorem 3, we can restrict our utility functions to get a positive result about converging to an optimal solution under the class of cooperatively rational 1-deals

Thm 6: 0-1 Scenarios Theorem 6: If utility functions are 0-1 functions (additive and u i ({r}) = 0 or 1), any sequence of cooperatively rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare

Thm 6: 0-1 Scenarios Theorem 6: If utility functions are 0-1 functions (additive and u i ({r}) = 0 or 1), any sequence of cooperatively rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare Note that we actually get optimal social welfare in this case, not just Pareto optimal!

Thm 6: 0-1 Scenarios Theorem 6: If utility functions are 0-1 functions (additive and u i ({r}) = 0 or 1), any sequence of cooperatively rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare Note that we actually get optimal social welfare in this case, not just Pareto optimal! Proof is simple –If A is not optimal, must have a agents i and j and resource r where r is in A(i), u i ({r}) = 0 and u j ({r}) = 1 –That 1-deal is cooperatively rational

Class of Deals Side Payments Utility Functions Measure of Social Welfare Nature of Optimality Necessary / Sufficient Individually Rational DealsYes Unrestricted Monotonic DichotomousUtilitarian Global Maximum Sufficient[1] & Necessary[2] Individually Rational 1-dealsYesAdditiveUtilitarian Global Maximum Sufficient[3] & Necessary Cooperatively Rational DealsNo Unrestricted Monotonic DichotomousUtilitarian Pareto Optimal Sufficient[4] & Necessary[5] Cooperatively Rational 1-dealsNo0-1 FunctionsUtilitarian Global Maximum Sufficient[6] & Necessary Equitable DealsNo Unrestricted DichotomousEgalitarian Global Maximum Sufficient[7] & Necessary[8] Simple Pareto- Pigou-Dalton DealsNo0-1 FunctionsMixed Lorenz OptimalSufficient[9] Summary of Results

Egalitarian Agent Societies

Egalitarian Social Welfare Consider a new measure of social welfare called egalitarian social welfare

Egalitarian Social Welfare Consider a new measure of social welfare called egalitarian social welfare Measures the utility of the ‘weakest/poorest’ member of the society

Egalitarian Social Welfare Consider a new measure of social welfare called egalitarian social welfare Measures the utility of the ‘weakest/poorest’ member of the society Makes sense when the society is working together or trying to be fair with one another –Recall: Earth Observation Satellite Access

Equitable Deals A deal δ = (A,A’) is equitable if min{ u i (A) | i in A δ } < min{ u i (A’) | i in A δ }

Equitable Deals A deal δ = (A,A’) is equitable if min{ u i (A) | i in A δ } < min{ u i (A’) | i in A δ } Lowest utility of all agents involved in a deal increases

Equitable Deals A deal δ = (A,A’) is equitable if min{ u i (A) | i in A δ } < min{ u i (A’) | i in A δ } Lowest utility of all agents involved in a deal increases Note: we do not need the weakest member of society to improve –Would not be a local condition

Equitable Deals A deal δ = (A,A’) is equitable if min{ u i (A) | i in A δ } < min{ u i (A’) | i in A δ } Lowest utility of all agents involved in a deal increases Note: we do not need the weakest member of society to improve –Would not be a local condition Lemma 2: If A and A’ are allocations with sw e (A) < sw e (A’), then δ = (A,A’) is equitable

Thm 7: Maximal Egalitarian Social Welfare Theorem 7: Any sequence of equitable deals will eventually result in an allocation of resources with maximal egalitarian social welfare

Thm 7: Maximal Egalitarian Social Welfare Theorem 7: Any sequence of equitable deals will eventually result in an allocation of resources with maximal egalitarian social welfare Only difficulty of proof is showing termination, the rest comes from the definition of equitable

Thm 8: Necessary Deals in Egalitarian Systems Theorem 8: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of equitable deals leading to an allocation with maximal egalitarian social welfare would have to include δ.

Thm 8: Necessary Deals in Egalitarian Systems Theorem 8: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of equitable deals leading to an allocation with maximal egalitarian social welfare would have to include δ. Still holds if utility functions are restricted to be dichotomous

Class of Deals Side Payments Utility Functions Measure of Social Welfare Nature of Optimality Necessary / Sufficient Individually Rational DealsYes Unrestricted Monotonic DichotomousUtilitarian Global Maximum Sufficient[1] & Necessary[2] Individually Rational 1-dealsYesAdditiveUtilitarian Global Maximum Sufficient[3] & Necessary Cooperatively Rational DealsNo Unrestricted Monotonic DichotomousUtilitarian Pareto Optimal Sufficient[4] & Necessary[5] Cooperatively Rational 1-dealsNo0-1 FunctionsUtilitarian Global Maximum Sufficient[6] & Necessary Equitable DealsNo Unrestricted DichotomousEgalitarian Global Maximum Sufficient[7] & Necessary[8] Simple Pareto- Pigou-Dalton DealsNo0-1 FunctionsMixed Lorenz OptimalSufficient[9] Summary of Results

Class of Deals Side Payments Utility Functions Measure of Social Welfare Nature of Optimality Necessary / Sufficient Individually Rational DealsYes Unrestricted Monotonic DichotomousUtilitarian Global Maximum Sufficient[1] & Necessary[2] Individually Rational 1-dealsYesAdditiveUtilitarian Global Maximum Sufficient[3] & Necessary Cooperatively Rational DealsNo Unrestricted Monotonic DichotomousUtilitarian Pareto Optimal Sufficient[4] & Necessary[5] Cooperatively Rational 1-dealsNo0-1 FunctionsUtilitarian Global Maximum Sufficient[6] & Necessary Equitable DealsNo Unrestricted DichotomousEgalitarian Global Maximum Sufficient[7] & Necessary[8] Simple Pareto- Pigou-Dalton DealsNo0-1 FunctionsMixed Lorenz OptimalSufficient[9] Summary of Results

Conclusions We studied an abstract negotiation framework where members of an agent society arrange multilateral deals to exchange bundles of indivisible resources

Conclusions We studied an abstract negotiation framework where members of an agent society arrange multilateral deals to exchange bundles of indivisible resources We analyzed how the resulting changes in resource distribution affect society with respect to different social welfare orderings

Conclusions We see that convergence to an optimal allocation depends on:

Conclusions We see that convergence to an optimal allocation depends on: –the class of allowable deals

Conclusions We see that convergence to an optimal allocation depends on: –the class of allowable deals –the notion of optimality being considered

Conclusions We see that convergence to an optimal allocation depends on: –the class of allowable deals –the notion of optimality being considered –the restrictions on utility functions

Conclusions We see that convergence to an optimal allocation depends on: –the class of allowable deals –the notion of optimality being considered –the restrictions on utility functions –the availability of side payments

Conclusions We see that convergence to an optimal allocation depends on: –the class of allowable deals –the notion of optimality being considered –the restrictions on utility functions –the availability of side payments Natural question: Complexity results –How fast do we converge to the optimal allocation? [Endriss and Maudet (2005)]

Conclusions Authors are looking at welfare engineering –Application-driven choice of a social welfare ordering –Design of agent behaviour profiles and negotiation mechanisms that permit socially optimal outcomes

Questions?

Lorenz Domination Let A, A’ be allocations for a society with n agents. Then A is Lorenz dominated by A’ if and furthermore, that inequality is strict for at least one k. k = 1 gives egalitarian social welfare k = n gives utilitarian social welfare

Pigou-Dalton Transfer A deal δ = (A,A’) is called a Pigou-Dalton transfer if it satisfies: –2 agents involved –Mean-preserving: u i (A) + u j (A) = u i (A’) + u j (A’) –Reduces inequality: |u i (A’) – u j (A’)| < |u i (A) – u j (A)| A simple Pareto-Pigou-Dalton deal is a 1- deal that is either cooperatively rational, or a Pigou-Dalton transfer

Class of Deals Side Payments Utility Functions Measure of Social Welfare Nature of Optimality Necessary / Sufficient Individually Rational DealsYes Unrestricted Monotonic DichotomousUtilitarian Global Maximum Sufficient[1] & Necessary[2] Individually Rational 1-dealsYesAdditiveUtilitarian Global Maximum Sufficient[3] & Necessary Cooperatively Rational DealsNo Unrestricted Monotonic DichotomousUtilitarian Pareto Optimal Sufficient[4] & Necessary[5] Cooperatively Rational 1-dealsNo0-1 FunctionsUtilitarian Global Maximum Sufficient[6] & Necessary Equitable DealsNo Unrestricted DichotomousEgalitarian Global Maximum Sufficient[7] & Necessary[8] Simple Pareto- Pigou-Dalton DealsNo0-1 FunctionsMixed Lorenz OptimalSufficient[9] Summary of Results