Cooperative/coalitional game theory A composite of slides taken from Vincent Conitzer and Giovanni Neglia (Modified by Vicki Allan) 1

Collaboration Our focus this semester is on how agents collaborate. One view of collaboration is coalition formation. A coalition is simply a group of agents that work together to accomplish the task. Formation focuses on the process of forming such a coalition. 2

coalitional game theory There is a set of agents N Each subset (or coalition) S of agents can work together in various ways, leading to various utilities for the agents Cooperative/coalitional game theory studies which outcome will/should materialize Key criteria: –Stability: No coalition of agents should want to deviate from the solution and go their own way –Fairness: Agents should be rewarded for what they contribute to the group (“Cooperative game theory” is the standard name (distinguishing it from noncooperative game theory, which is what we have studied in two player games). However this is somewhat of a misnomer because agents still pursue their own interests. Hence some people prefer “coalitional game theory.”) 3

Example A set of twelve people all drive to work. They would like to form car pools. Some can pick up others on their way to work. Others have to go out of their way to pick up others. A car can only hold 5 people. Who should carpool together? How often should they each drive? 4

Transferable utility Suppose that utility is transferable: If you get more intrinsic benefit from a coalition, you can give some of your utility to another agent in your coalition (e.g. by making a side payment) Then, all that we need to specify is a value for each coalition, which is the maximum total utility for the coalition In the case of the carpool, how would we determine utility? –Value function is also known as characteristic function –Def. A game in characteristic function form is a set N of players together with a function v() which for any subset S of N (a coalition) gives a number v(S) (the value of the coalition) The payout is a vector of utilities that sums to the value is possible Can you think of other examples where a coalition has an associated utility? 5

Suppose we have the following characteristic function CoalitionValue of coalition a2 b3 c4 ab6 ac6 bc7 abc12 6 How do we decide how to pay each member for their contribution? We are assuming the final (grand) coalition is the one that is formed. Agents are labeled a, b, c

Some things we know CoalitionValue of coalition Possible payout a22 b33 c44 ab6 ac6 bc7 abc12 7 No one should do worse

Other things we know Coalitio n Value of coalition Possible payout Bad as doesn’t use all value What about? a222 b333 c447 ab6 ac6 bc7 abc12 8 Somebody should get all the utility for the grand coalition

Other things we know Coalitio n Value of coalition Possible payout Bad as doesn’t use all value What about? a222 b333 c447 ab6 ac6 bc7 abc12 9 unstable as ab can do better

Let’s try a different example CoalitionValue of coalition a2 b3 c4 ab10 ac10 bc7 abc14 10 How do we decide how to pay each member for their contribution?

Suppose we have the following characteristic function CoalitionValue of coalition Possibilit y a24 b34 c46 ab10 ac10 bc7 abc14 11 Who would pull out?

Suppose we have the following characteristic function CoalitionValue of coalition Possibility a26 b34 c44 ab10 ac10 bc7 abc14 12 Is this any better?

Core Outcome is in the core if and only if: every coalition receives a total utility that is at least its original value and no smaller coalition is tempted to pull out. –For every coalition C, v(C) ≥Σ i in C u(i) For example, –v({a,b,c}) = 12, –v({a,b}) = v({a,c}) = v({b,c}) = 8, –v({a}) = v({b}) = v({c}) = 0 Now the outcome (4, 4, 4) is possible; it is also in the core (why?) and is the only outcome in the core. 13

Emptiness & multiplicity No solution? many solutions? Example 2: –v({a,b,c}) = 11, –v({a,b}) = v({a,c}) = v({b,c}) = 8, –v({a}) = v({b}) = v({c}) = 0 Now the core is empty! Notice, the core must involve the grand coalition (giving payoff for each). Example 3: –v({a,b,c}) = 18, –v({a,b}) = v({a,c}) = v({b,c}) = 8, –v({a}) = v({b}) = v({c}) = 0 Now lots of outcomes are in the core – (6, 6, 6), (5, 5, 8), … 14

Issues with the core When is the core guaranteed to be nonempty? What about uniqueness? What do we do if there are no solutions in the core? What if many? 15

Superadditivity v is superadditive if for all coalitions A, B with A∩B = Ø, v(AUB) ≥ v(A) + v(B) Informally, the union of two coalitions can always act as if they were separate, so should be able to get at least what they would get if they were separate Informally – if agents did worse working together, there would be no motivation for coalitions. Usually makes sense Previous examples were all superadditive Given this, always efficient for grand coalition to form Without superadditivity, finding a core is not possible. 16

Convexity v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B) In other words, the amount A adds to B (in the union) is at least as much it adds to the intersection. One interpretation: the marginal contribution of an agent is increasing in the size of the set that it is added to. The term marginal contribution means the additional contribution. Precisely, the marginal contribution of A to B is v(AUB)- v(B) 17

Convexity a value system v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B) Previous examples were not convex (why?) v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)- v(A∩B). For example, Does this have a core solution? –v({a,b,c}) = 12, –v({a,b}) = v({a,c}) = v({b,c}) = 8, –v({a}) = v({b}) = v({c}) = 0 Let A = {a,b} and B={b,c} v(AUB)-v(B) = = v{a,b,c} – v{b,c} = 12 – 8 = 4 v(A)-v(A∩B) = v{a,b} – v{b} = = 8 Problem 4 < 8 18

Convexity In convex games, core is always nonempty. (Core doesn’t require convexity, but convexity produces a core.) One easy-to-compute solution in the core: agent i gets u(i) = v({1, 2, …, i}) - v({1, 2, …, i-1}) –Marginal contribution scheme- each agent is rewarded by what it adds to the union. –Works for any ordering of the agents 19

The Shapley value motivation So, if there is no core, what do we do? It would be nice to say, “This is what people should be paid” Example – no core –v({a,b,c}) = 2, –v({a,b}) = v({a,c}) = v({b,c}) =2, –v({a}) = v({b}) = v({c}) = 0 –If {a,b} decided to be a team, c is left out and may opt to work for less to attract a partner: –c says, “Hey b, how about working with me? You can have 1.1. I’ll take.9” –a says, “Hey b, come back. You can have 1.2. I’ll take.8” –c says to a, “Hey a, work with me. We can each have 1. –The solution isn’t stable 20

The Shapley value [Shapley 1953] In dividing the profit, sometimes agent is given its marginal contribution (how much better the group is by its addition) Example – no core –v({a,b,c}) = 2, –v({a,b}) = v({a,c}) = v({b,c}) =2, –v({a}) = v({b}) = v({c}) = 0 So, b could say, “Hey a, work with me. I should get 2 as you were worth nothing without me. I should get the valued added (marginal value).” 21

The Shapley value [Shapley 1953] In dividing the profit, sometimes agent is given its marginal contribution (how much better the group is by its addition) The simple marginal contribution scheme is unfair because it depends on the ordering of the agents One way to make it fair: average over all possible orderings Let MC(i, π) be the marginal contribution of i in ordering π Then i’s Shapley value is Σ π MC(i, π)/(n!) The Shapley value is always in the core for convex games … but not in general, even when core is nonempty, e.g. 22

23 Example: v({a,b,c})= v({a,b}) = v({a,c}) = v({b,c}= 2, v = 0 everywhere else abc abc 020 acb 002 bac 200 bca 002 cab 200 cba 020 avg 4/6 Compute the Shapley value for each. Is the solution in the core? Appealing as all appeared to have same characteristics

24 Example: v({a,b,c})= 15 v({a,b}) = 3 v({a,c}) = 4 v({b,c}= 9, v =2 everywhere else bc abc acb bac bca cab cba avg Compute the Shapley value for each. Is the solution in the core? Without doing the math, what would you guess?

25 Example: v({a,b,c})= 15 v({a,b}) = 3 v({a,c}) = 4 v({b,c}= 9, v =2 everywhere else abc abc 2112 acb 2112 bac 1212 bca 627 cab 2112 cba 672 avg 19/634/637/6 Compute the Shapley value for each. Is the solution in the core? Is this reasonable?

Briefly describe the characteristics of “core” and “shapley” 26

Axiomatic characterization of the Shapley value The Shapley value is the unique solution concept that satisfies: –(Pareto) Efficiency: the total utility is the value of the grand coalition, Σ i in N u(i) = v(N) –Symmetry: two equivalent players (add the same amount to coalitions they join) must receive the same utility –Dummy: if v(S  {i}) = v(S) for all S, then i must get 0 –Additivity: if we add two games defined by v and w by letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of its utilities in v and w 27

Computing a solution in the core Can use linear programming (a formal mathematical method): –Variables: u(i) –Distribution constraint: Σ i in N u(i) = v(N) –Non-blocking constraints: for every S, Σ i in S u(i) ≥ v(S) Problem: number of constraints exponential in number of players (as you have values for all possible subsets) … but is this practical? 28

Example: Minimum Spanning Tree game For some games the characteristic function (value of each coalition) representation is immediate – others it is more difficult Communities a,b & c want to be connected to a nearby power source –Possible transmission links & costs as in figure –Notice that distance does not determine cost –Maybe the soil is rocky in certain directions source a b c

Example: Minimum Spanning Tree game What links should be built to get to power source? What should each community pay? Can c say, “Hey b, I’m getting my power through you. Don’t expect any reimbursement because you already had to connect.” Is that fair? source a b c

Example: Minimum Spanning Tree game Can a say, “Hey guys, let’s share the cost equally. The total cost of all of us connecting is 100. Let’s each do a third.” Can c say, “No way. I should be the cheapest as I was the cheapest by myself. You are gaining more.” source a b c

Example: Minimum Spanning Tree game Communities a,b,&c want to be connected to a nearby power source v(void) = 0 v(a) = 0 v(b) = 0 v(c) = 0 v(ab) = = 60 v(ac) = = 60 v(bc) = = 30 v(abc) = = 90 Red: currently paying Black: cost of new arrangement Green: Improvement We show what is gained from the coalition. How to divide the gain? source a b c

The important questions Which coalitions should form? How should a coalition which forms divide its winnings among its members? Unfortunately there is no definitive answer Many concepts have been developed since 1944: –stable sets –core –Shapley value –bargaining sets –nucleolus –Gately point 33

The Core What about MST game? We use value to mean what is saved by going with a group. –v(void)= v(a) = v(b) = v(c)=0 –v(ab) = 60, v(ac) = 60, v(bc) = 30 –v(abc) = 90 Analitically, in getting to a group of three, you must make sure you do better than the group of 2 cases: a+b>=60, iff c<=30 a + c>=60, iff b<=30 b + c>=30, iff x1<=

The Core Let’s divide up the 90 in savings. Lots of choices Let’s choose a division of the profits in the core: x=(60,25,5) Is this fair compared to what their costs were without cooperation? The payoffs represent the savings, the costs (what they really pay) under x are –cost(a)=100-60=40, –cost(b)=50-25=25 –cost(c)=40-5=35 source a b c FAIR? 35

The Shapley value: computation Consider the players forming the grand coalition step by step –start from one player and add other players until N is formed As each player joins, award to that player the value he adds to the growing coalition The resulting awards give an value added Average the value added given by all the possible orders The average is the Shapley value k 36

The Shapley value: computation MST game –v(void) = v(a) = v(b) = v(c)=0 –v(a,b) = 60, v(a,c) = 60, v(b,c) = 30, v(a,b,c) = 90 abc abc acb bac bca cab cba avg Value added by Coalitions 37

The Shapley value: computation MST game –v(void) = v(a) = v(b) = v(c)=0 –v(a,b) = 60, v(a,c) = 60, v(b,c) = 30, v(a,b,c) = 90 abc abc acb bac bca cab cba avg 4025 Value added by Coalitions 38

The Shapley value: computation A faster way The amount player i contributes to coalition S, of size s, is v(S)-v(S-i) This contribution occurs for those orderings in which i is preceded by the s-1 other players in S, and followed by the n-s players not in S ki = 1/n!  S:i in S (s-1)! (n-s)! (v(S)-v(S-i)) 39

The Shapley value has been used for cost sharing. Suppose three planes share a runway. The planes a, b, and c require 1, 2, and 3 KM, repectively to land. Thus, a runway of 3 must be build, but what part of the total cost for a 3 KM runway should each pay? Instead of looking at utility given, look at how much increased cost was required. abc abc acb bac bca cab cba avg

The Shapley value has been used for cost sharing. Suppose three planes share a runway. The planes a, b, and c require 1, 2, and 3 KM, respectively to land. Thus, a runway of 3 must be build, but what part of the total cost for a 3 KM runway should each pay? Instead of looking at utility given, look at how much increased cost was required. abc abc 111 acb 102 bac 021 bca 021 cab 003 cba 003 avg 2/65/611/6