Cooperative/coalitional game theory A composite of slides taken from Vincent Conitzer and Giovanni Neglia (Modified by Vicki Allan) 1

coalitional game theory There is a set of agents N Each subset (or coalition) S of agents can work together in various ways, leading to various utilities for the agents Cooperative/coalitional game theory studies which outcome will/should materialize Key criteria: –Stability: No coalition of agents should want to deviate from the solution and go their own way –Fairness: Agents should be rewarded for what they contribute to the group (“Cooperative game theory” is the standard name (distinguishing it from noncooperative game theory, which is what we have studied in two player games). However this is somewhat of a misnomer because agents still pursue their own interests. Hence some people prefer “coalitional game theory.”) 2

Example Three agents {1, 2, 3} can go out for Indian, Chinese, or Japanese food u 1 (I) = u 2 (C) = u 3 (J) = 4 u 1 (C) = u 2 (J) = u 3 (I) = 2 u 1 (J) = u 2 (I) = u 3 (C) = 0 So the base utility agent 1 gets for Indian food is 4. Each agent gets an additional unit of utility for each other agent that joins her. HOWEVER, going out to eat alone is NOT allowed. If all agents go for Indian together, they get utilities (6, 2, 4) All going to Chinese gives (4, 6, 2), all going to Japanese gives (2, 4, 6) Hence, the utility possibility set for {1, 2, 3} is {(6, 2, 4), (4, 6, 2), (2, 4, 6)} For the coalition {1, 2}, the utility possibility set is {(5, 1), (3, 5), (1, 3)} (why?) Agent /food IndianChineseJapanese

Stability & the core u 1 (I) = u 2 (C) = u 3 (J) = 4 u 1 (C) = u 2 (J) = u 3 (I) = 2 u 1 (J) = u 2 (I) = u 3 (C) = 0 V({1, 2, 3}) = {(6, 2, 4), (4, 6, 2), (2, 4, 6)} V({1, 2}) = {(5, 1), (3, 5), (1, 3)} Suppose the agents decide to all go for Japanese together, so they get (2, 4, 6) 1 and 2 would both prefer to break off and get Chinese together for (3, 5) – we say (2, 4, 6) is blocked by {1, 2} –Blocking only occurs if there is a way of breaking off that would make all members of the blocking coalition happier The core [Gillies 53] is the set of all outcomes (for the grand coalition N of all agents) that are blocked by no coalition In this example, the core is empty (why?) In a sense, there is no stable (meaning people won’t change) outcome. There is no way to form coalitions. 4

Transferable utility Now suppose that utility is transferable: you can give some of your utility to another agent in your coalition (e.g. by making a side payment) Then, all that we need to specify is a value for each coalition, which is the maximum total utility for the coalition –Value function also known as characteristic function –Def. A game in characteristic function form is a set N of players together with a function v() which for any subset S of N (a coalition) gives a number v(S) (the value of the coalition) Any vector of utilities that sums to the value is possible Hence, the total for utility possibility set for {1, 2, 3} {(6, 2, 4)=12, (4, 6, 2)=12, (2, 4, 6)=12} Notice they totals wouldn’t all have to be equal in other examples. 5

Transferable utility Outcome is in the core if and only if: every coalition receives a total utility that is at least its original value –For every coalition C, v(C) ≥Σ i in C u(i) In above example, –v({1, 2, 3}) = 12, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 8, –v({1}) = v({2}) = v({3}) = 0 Now the outcome (4, 4, 4) is possible; it is also in the core (why?) and is the only outcome in the core. 6

Emptiness & multiplicity Example 2: Let us modify the above example so that agents receive no utility from being together (except being alone still gives 0) –v({1, 2, 3}) = 6, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 6, –v({1}) = v({2}) = v({3}) = 0 Now the core is empty! Notice, the core must involve the grand coalition (giving payoff for each). Conversely, suppose agents receive 2 units of utility for each other agent that joins –v({1, 2, 3}) = 18, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 10, –v({1}) = v({2}) = v({3}) = 0 Now lots of outcomes are in the core – (6, 6, 6), (5, 5, 8), … 7

Issues with the core When is the core guaranteed to be nonempty? What about uniqueness? What do we do if there are no solutions in the core? What if many? 8

Superadditivity v is superadditive if for all coalitions A, B with A∩B = Ø, v(AUB) ≥ v(A) + v(B) Informally, the union of two coalitions can always act as if they were separate, so should be able to get at least what they would get if they were separate Usually makes sense Previous examples were all superadditive Given this, always efficient for grand coalition to form Without superadditivity, finding a core is not possible. 9

Convexity v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B) In other words, the amount A adds to B (in the union) is at least as much it adds to the intersection. One interpretation: the marginal contribution of an agent is increasing in the size of the set that it is added to. The term marginal contribution means the additional contribution. Precisely, the marginal contribution of A to B is v(AUB)-v(B) Example, suppose we have three independent researchers. When we combine them at the same university, the value added is greater if the set is larger. 10

Convexity Previous examples were not convex (why?) v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B). Let A = {1,2} and B={2,3} v(AUB)-v(B) = 12 – 8 v(A)-v(A∩B) = In convex games, core is always nonempty. (Core doesn’t require convexity, but convexity produces a core.) One easy-to-compute solution in the core: agent i gets u(i) = v({1, 2, …, i}) - v({1, 2, …, i-1}) –Marginal contribution scheme- each agent is rewarded by what it ads to the union. –Works for any ordering of the agents 11

The Shapley value [Shapley 1953] In dividing the profit, sometimes agent is given its marginal contribution (how much better the group is by its addition) The marginal contribution scheme is unfair because it depends on the ordering of the agents One way to make it fair: average over all possible orderings Let MC(i, π) be the marginal contribution of i in ordering π Then i’s Shapley value is Σ π MC(i, π)/(n!) The Shapley value is always in the core for convex games … but not in general, even when core is nonempty, e.g. –v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1, –v = 0 everywhere else 12

Axiomatic characterization of the Shapley value The Shapley value is the unique solution concept that satisfies: –(Pareto) Efficiency: the total utility is the value of the grand coalition, Σ i in N u(i) = v(N) –Symmetry: two symmetric players (add the same amount to coalitions they join) must receive the same utility –Dummy: if v(S  {i}) = v(S) for all S, then i must get 0 –Additivity: if we add two games defined by v and w by letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w most controversial axiom 13

Computing a solution in the core Can use linear programming: –Variables: u(i) –Distribution constraint: Σ i in N u(i) = v(N) –Non-blocking constraints: for every S, Σ i in S u(i) ≥ v(S) Problem: number of constraints exponential in number of players (as you have values for all possible subsets) … but is this practical? 14

Theory of cooperative games with sidepayments It starts with von Neumann and Morgenstern (1944) Two main (related) questions: –which coalitions should form? –how should a coalition which forms divide its winnings among its members? The specific strategy the coalition will follow is not of particular concern... Note: there are also cooperative games without sidepayments 15

Example: Minimum Spanning Tree game For some games the characteristic form representation is immediate Communities 1,2 & 3 want to be connected to a nearby power source –Possible transmission links & costs as in figure source

Example: Minimum Spanning Tree game Communities 1,2 & 3 want to be connected to a nearby power source. We show costs for each combination of communities. v(void) = 0 v(1) = -100 v(2) = -50 v(3) = -40 v(12) = -90 v(13) = -80 v(23) = -60 v(123) = -100 A normalization can be done source

Example: Minimum Spanning Tree game Communities 1,2 & 3 want to be connected to a nearby power source v(void) = 0 v(1) = = 0 v(2) = = 0 v(3) = = 0 v(12) = = 60 v(13) = = 60 v(23) = = 30 v(123) = = 90 A strategically equivalent game. We show what is gained from the coalition. How to divide the gain? source

Example: Minimum Spanning Tree (MST) game Communities 1,2 & 3 want to be connected to a nearby power source v(void) = 0/90 = 0 v(1) = ( )/90 = 0 v(2) = ( )/90 = 0 v(3) = ( )/90 = 0 v(12) = ( )/90 = 2/3 v(13) = ( )/90 = 2/3 v(23) = ( )/90 = 1/3 v(123) = ( )/90 = 1 A strategically equivalent game source

The important questions Which coalitions should form? How should a coalition which forms divide its winnings among its members? Unfortunately there is no definitive answer Many concepts have been developed since 1944: –stable sets –core –Shapley value –bargaining sets –nucleolus –Gately point 20

Imputation Given a game in characteristic function form (N,v) an imputation is a payoff division... –i.e. a n-tuple of numbers x=(x1,x2,...,xn) with two reasonable properties –xi >= v(i) (individual rationality) –  xi >= v(N)(collective rationality) greater if it doesn’t make sense to all work together for superadditive games –  xi = v(N) 21

Dominance An imputation x dominates an imputation y if there is some coalition S, s.t. –xi > yi for all i in S x is more convenient for players in S –  i in S xi <= v(S) the coalition S must be able to enforce x 22

The Core The set of all undominated imputations, –i.e. the set of all imputations x s.t. for all S,  i in S xi >= v(S) 23

The Core What about MST game? We use value to mean what is saved by going with a group. –v(void)= v(1) = v(2) = v(3)=0 –v(12) = 60, v(13) = 60, v(23) = 30 –v(123) = 90 Analitically, in getting to a group of three, you must make sure you do better than the group of 2 cases: –x1+x2>=60, iff x3<=30 –x1+x3>=60, iff x2<=30 –x2+x3>=30, iff x1<=

The Core Let’s choose an imputation in the core: x=(60,25,5) The payoffs represent the savings, the costs under x are –c(1)=100-60=40, –c(2)=50-25=25 –c(3)=40-5=35 source FAIR? 25

The Shapley value Target: a fair and individually rational distribution k Individually rational means a self-interested agent would agree to it. Axioms –1) if i and j have symmetric roles in v(), then ki=kj –2) if v(S)=v(S-i) for all S, then ki=0 –3) if v and w are two games with the same player set and k(v) and k(w) the imputations we consider, then k(v+w)=k(v)+k(w) (Shapley value’s weakness) Theorem: There is one and only one method of assigning such an imputation to a game (Shapley value’s strength) 26

The Shapley value: computation Consider the players forming the grand coalition step by step –start from one player and add other players until N is formed As each player joins, award to that player the value he adds to the growing coalition The resulting awards give an value added Average the value added given by all the possible orders The average is the Shapley value k 27

The Shapley value: computation MST game –v(void) = v(1) = v(2) = v(3)=0 –v(1,2) = 60, v(1,3) = 60, v(2,3) = 30, v(1,2,3) = avg Value added by Coalitions 28

The Shapley value: computation MST game –v(void) = v(1) = v(2) = v(3)=0 –v(12) = 60, v(13) = 60, v(23) = 30, v(123) = avg Value added by Coalitions 29

The Shapley value: computation MST game –v(void) = v(1) = v(2) = v(3)=0 –v(12) = 60, v(13) = 60, v(23) = 30, v(123) = avg Value added by Coalitions 30

The Shapley value: computation A faster way The amount player i contributes to coalition S, of size s, is v(S)-v(S-i) This contribution occurs for those orderings in which i is preceded by the s-1 other players in S, and followed by the n-s players not in S ki = 1/n!  S:i in S (s-1)! (n-s)! (v(S)-v(S-i)) 31

The Shapley value: computation k = (40,30,20) the costs under x are –c(1)=100-40=60, –c(2)=50-30=20 –c(3)=40-20=20 source

An application: voting power A voting game is a pair (N,W) where N is the set of players (voters) and W is the collection of winning coalitions, s.t. –the empty set is not in W (it is a losing coalition) –N is in W (the coalition of all voters is winning) –if S is in W and S is a subset of T then T is in W Also weighted voting game can be considered The Shapley value of a voting game is a measure of voting power (Shapley-Shubik power index) –The winning coalitions have payoff 1 –The loser ones have payoff 0 33

An application: voting power The United Nations Security Council in 1954 –5 permanent members (P) –6 non-permanent members (N) –the winning coalitions had to have at least 7 members, –but the permanent members had veto power A winning coalition had to have at least seven members including all the permanent members The seventh member joining the coalition is the pivotal one: he makes the coalition winning 34

An application: voting power 462 (=11!/(5!*6!)) possible orderings Power of non permanent members –(PPPPPN)N(NNNN) –6 possible arrangements for (PPPPPN) –1 possible arrangements for (NNNN) –The total number of arrangements in which an N is pivotal is 6 –The power of non permanent members is 6/462 The power of permanent members is 456/462, the ratio of power of a P member to a N member is 91:1 In 1965 –5 permanent members (P) –10 non-permanent members (N) –the winning coalitions has to have at least 9 members, –the permanent members keep the veto power Similar calculations lead to a ratio of power of a P member to a N member equal to 105:1 35

Other approaches Stable sets –sets of imputations J internally stable (no imputations in J is dominated by any other imputation in J) externally stable (every imputations not in J is dominated by an imputation in J) –incorporate social norms Bargaining sets –the coalition is not necessarily the grand coalition (no collective rationality) Nucleolus –minimize the unhappiness of the most unhappy coalition –it is located at the center of the core (if there is a core) Gately point –similar to the nucleolus, but with a different measure of unhappyness 36

Nucleolus [Schmeidler 1969] Always gives a solution in the core if there exists one Always uniquely determined A coalition’s excess e(S) is v(S) - Σ i in S u(i) For a given outcome, list all coalitions’ excesses in decreasing order E.g. consider –v({1, 2, 3}) = 6, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 6, –v({1}) = v({2}) = v({3}) = 0 For payoff (2, 2, 2), the list of excesses is 2, 2, 2, 0, -2, -2, -2 (for coalitions {1, 2}{1, 3}{2, 3} {1, 2, 3} {1} {2} {3}, respectively) For payoff (3, 3, 0), the list of excesses is 3, 3, 0, 0, 0, -3, -3 (coalitions {1, 3}, {2, 3}; {1, 2}, {1, 2, 3}, {3}; {1}, {2}) 37

38 Nucleolus is the (unique) payoff that lexicographically minimizes the list of excesses –Lexicographic minimization = minimize the first entry first, then (fixing the first entry) minimize the second one, etc. –It is like “dictionary ordering”. So for each possible outcome, you make a list of excesses for each coalition, and sort them in order. Then, the one that lexicographically minimizes the list is selected. The idea is that you are trying to be fair – so that no group receives a lot more benefit than another.

Marriage contract problem [Babylonian Talmud, 0-500AD] A man has three wives Their marriage contracts specify that they should, respectively, receive 100, 200, and 300 in case of his death … but there may not be that much money to go around… Talmud recommends: –If 100 is available, each wife gets 33 1/3 –If 200 is available, wife 1 gets 50, other two get 75 each –If 300 is available, wife 1 gets 50, wife 2 gets 100, wife 3 gets 150 What is going on? Define v(S) = max{0, money available - Σ i in N-S claim(i)} –Any coalition can walk away and obtain 0 –Any coalition can pay off agents outside the coalition and divide the remainder Talmud recommends the nucleolus! [Aumann & Maschler 85] 39