Cooperative/coalitional game theory A composite of slides taken from Vincent Conitzer and Giovanni Neglia (Modified by Vicki Allan) 1

Prisoners Dilemma rules 1.Binding agreements are not possible. Note in Prisoners dilemma, if binding agreements were possible, there would be no dilemma 2.Utility is given directly to individuals as a result of individual actions. So, I am not worried about collective utility. Solution: cooperative game theory 2

Cooperative Games Coalitions – set of agents Grand coalition – all agents work together Characteristic function: v:2 Ag  R subsets of agents are assigned a value. Simple coalitional game: a coalition has value 0 or 1. A voting system can be thought of as a simple game (have enough votes to win or do not) 3

Three Parts of Cooperative Action 1.Coalition Structure Generation – who should work together. Know only characteristic function. Want to earn a lot, but so does everyone else. 2.Optimization Problem – given coalitions and tasks, which coalition should do each problem. 3.Dividing the utility- What is fair? What is stable? 4

In Multi-Agent Systems Agents create their own coalitions rather than have a centralized decision. 5

Scenario 1 – Bargain Buy (supply-demand) Store “Bargain Buy” advertises a great price 300 people show up 5 in stock Everyone sees the advertised price, but it just isn’t possible for all to achieve it

Scenario 2 – hiring a new PhD (strategy) Universities ranked 1,2,3 Students ranked a,b,c Dilemma for second tier university offer to “a” student likely rejected delay for acceptance “b” students are gone

Scenario 3 – selecting a spouse (agency) Bob knows all the characteristics of the perfect wife Bob seeks out such a wife Why would the perfect woman want Bob?

Scenario 4 Blowhard (trust) What if one person talks a good story, but his claims of skills are really inflated? He isn’t capable of performing. the task.

Scenario 5 Power in diversity You consult local traffic statistics to find a good route home from work But so does everyone else

The coalition is completed and rewards are earned. How are they fairly divided among agents with various contributions? If organizer is greedy, why wouldn’t others replace him with a cheaper agent?

Consider the Stable Marriage Problem – small coalition In a small town there are n men and n women who wish to be wed. Each person would be happy to be married to any of the people of the opposite sex but has a definite preference ranking of the possible marriage partners. If marriages are arranged arbitrarily, some of the marriages can be unstable in the following sense. Suppose Alice marries Bob and Carol marries Dave. If Alice prefers Dave to Bob and Dave prefers Alice to Carol, then Alice and Dave will leave their partners; in this situation, we say that the marriages of Alice and Dave are unstable. Goal is stability not just the most partners: the difference between local and global utility. 12

Core Ex: software engineering group project. Who would you choose to work with? Who would also choose you? Coalition doesn’t form unless everyone is happy with it. Stability is necessary but not sufficient for a coalition to form. If it isn’t stable, someone will defect. If there are multiple stable coalitions, another may form instead. core: set of feasible distributions of payoff to members of the grand coalition so none will defect. We require the outcome (payments) to be both feasible (able to pay) and efficient (all utility distributed). Pareto efficiency is the efficiency referred to. 13

We say a coalition objects to (or blocks) an outcome if every member of the coalition is strictly better off in some other feasible outcome. Asking if the grand coalition is stable means, “Is the core non-empty?” The point of the core is to study stability not fairness. 14

Example v{a} = 1 v{b} = 3 v{a,b} = 5 How should we divide the profit if they work together? Which are feasible, fair, efficient, stable? 15

So what are concerns? What if there are no outcomes in the core? What if there are multiple outcomes in the core, how do you pick? Looking at all possible distributions of utility is exponential as you have 2 n possible objecting subsets to consider. 16

Example v{1} = 2 v{2} = 2 v{3} = 2 v{1,2} = 5 v{2,3} = 5 v{1,2,3} = 6 How should we divide the profit if they work together? Which are feasible, fair, efficient, stable? 17

More Examples Emptiness & multiplicity 18 Example 1: Let us modify the above example so that agents receive no additional utility from being all together (and being alone gives 0) –v({1, 2, 3}) = 6, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 6, –v({1}) = v({2}) = v({3}) = 0

More Examples Emptiness & multiplicity 19 Example 1: Let us modify the above example so that agents receive no additional utility from being all together (and being alone gives 0) –v({1, 2, 3}) = 6, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 6, –v({1}) = v({2}) = v({3}) = 0 Now the core is empty! Notice, the core must involve the grand coalition (giving payoff for each).

More Examples Emptiness & multiplicity 20 Example 2: –v({1, 2, 3}) = 18, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 10, –v({1}) = v({2}) = v({3}) = 0

More Examples Emptiness & multiplicity 21 Example 2: –v({1, 2, 3}) = 18, –v({1, 2}) = v({1, 3}) = v({2, 3}) = 10, –v({1}) = v({2}) = v({3}) = 0 Now lots of outcomes are in the core – (6, 6, 6), (5, 5, 8), …

Fairness Could we get people to join a coalition by saying, “Will you agree to a fair solution even though it is not in the core?” In example v{1} =0 v{2} = 0 v{3} = 0 v{1,2} = 5 v{2,3} = 5 v{1,2,3} = 6 What would a fair but unstable solution be? If we had extra (that no one could really demand), how would we divide any surplus? 22

Terms Marginal Contribution (value added): μ i (C) = v(C U {i}) – v(C) sh i is what i is given (its share) Symmetry: Agents that make the same contribution should get the same payoff Dummy player – never increases the value of the coalition beyond what it could earn alone μ i (C) = v(C U {i}) – v(C) = v{i} for every C Alternate definition of Dummy: μ i (C) = 0 for every C Additivity: if we add two games defined by v and w by letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w most controversial axiom (for example, participant i’s cost-share of a runway and terminal is it’s cost-share of the runway plus his cost-share of the terminal) 23

Superadditivity 24 v is superadditive if for all coalitions A, B with A∩B = Ø, v(AUB) ≥ v(A) + v(B) Informally, the union of two coalitions can always act as if they were separate, so should be able to get at least what they would get if they were separate. There is a synergy – if not, coalitions make no sense. Usually makes sense Previous examples were all superadditive Given this, always efficient for grand coalition to form Without superadditivity, finding a core is not possible.

The Shapley value [Shapley 1953] 25 In dividing the profit, sometimes agent is given its marginal contribution (how much better the group is by its addition) The marginal contribution scheme is unfair because it depends on the ordering of the agents One way to make it fair: average over all possible orderings Let MC(i, π) be the marginal contribution of i in ordering π Then i’s Shapley value is Σ π MC(i, π)/(n!) The Shapley value is always in the core for convex games … but not in general, even when core is nonempty, e.g. –v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1, –v = 0 everywhere else

Example: v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1, v = 0 everywhere else avg 26 Compute the Shapley value for each. Is the solution in the core?

Axiomatic characterization of the Shapley value 27 The Shapley value is the unique solution concept that satisfies: –(Pareto) Efficiency: the total utility is the value of the grand coalition, Σ i in N u(i) = v(N) –Symmetry: two symmetric players (add the same amount to coalitions they join) must receive the same utility –Dummy: if v(S  {i}) = v(S) for all S, then i must get 0 –Additivity: if we add two games defined by v and w by letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w.

Additivity Example Game a 28 CoalitionValue /6 2/6 CoalitionValue /6 Game b

a+b 29 CoalitionValue /6 12/6 Notice how their value in the combined game is the sum of their values in the original games

Shapley Result Satisfies all three fairness axioms (the last three listed) If you came up with some other method that satisfied all fairness axioms, it would be Shapley. In other words, the Shapley value is the UNIQUE value that satisfies all fairness axioms. 30

Computing a solution in the core 31 How do you even represent the characteristic function which defines the problem? exponential in the number of agents. Can use linear programming: –Variables: u(i) –Distribution constraint: Σ i in N u(i) = v(N) –Non-blocking constraints: for every S, Σ i in S u(i) ≥ v(S) Problem: number of constraints exponential in number of players (as you have values for all possible subsets) … but is this practical?

Convexity 32 v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B) In other words, the amount A adds to B (in forming the union) is at least as much it adds to the intersection. One interpretation: the marginal contribution of an agent is increasing in the size of the set that it is added to. The term marginal contribution means the additional contribution. Precisely, the marginal contribution of A to B is v(AUB)-v(B) Example, suppose we have three independent researchers. When we combine them at the same university, the value added by A is greater if the set is larger.

Example v{1} = 0 v{2} = 0 v{3} = 0 v{1,2} = 8 v{2,3} = 8 v{1,2,3} = 12 Let A = {1,2} and B={2,3} v(AUB)-v(B) ≥ v(A)-v(A∩B) 33

Convexity 34 In convex games, core is always nonempty. (Core doesn’t require convexity, but convexity produces a core.) One easy-to-compute solution in the core: agent i gets u(i) = v({1, 2, …, i}) - v({1, 2, …, i-1}) –Marginal contribution scheme- each agent is rewarded by what it adds to the union. –Works for any ordering of the agents

Theory of cooperative games with sidepayments It starts with von Neumann and Morgenstern (1944) Two main (related) questions: – which coalitions should form? – how should a coalition (which forms) divide its winnings among its members? The specific strategy the coalition will follow is not of particular concern... Note: there are also cooperative games without sidepayments 35

Example: Minimum Spanning Tree game For some games the characteristic form representation is immediate Communities 1,2 & 3 want to be connected to a nearby power source – Possible transmission links & costs as in figure 36 source

Example: Minimum Spanning Tree game Communities 1,2 & 3 want to be connected to a nearby power source 37 v(void) = 0 v(1) = 0 v(2) = 0 v(3) = 0 v(12) = = 60 v(13) = = 60 v(23) = = 30 v(123) = = 90 A strategically equivalent game. We show what is gained from the coalition. How to divide the gain? source

The Core What about MST game? We use value to mean what is saved by going with a group. – v(void)= v(1) = v(2) = v(3)=0 – v(12) = 60, v(13) = 60, v(23) = 30 – v(123) = 90 Analitically, in getting to a group of three, you must make sure you do better than the group of 2 cases: – x1+x2>=60, iff x3<=30 – x1+x3>=60, iff x2<=30 – x2+x3>=30, iff x1<=

The Core Let’s choose an imputation in the core: x=(60,25,5) The payoffs represent the savings, the costs under x are – c(1)=100-60=40, – c(2)=50-25=25 – c(3)=40-5=35 39 source FAIR?

The Shapley value: computation MST game – v(void) = v(1) = v(2) = v(3)=0 – v(1,2) = 60, v(1,3) = 60, v(2,3) = 30, v(1,2,3) = avg Value added by Coalitions

The Shapley value: computation MST game – v(void) = v(1) = v(2) = v(3)=0 – v(12) = 60, v(13) = 60, v(23) = 30, v(123) = avg Value added by Coalitions

The Shapley value: computation MST game – v(void) = v(1) = v(2) = v(3)=0 – v(12) = 60, v(13) = 60, v(23) = 30, v(123) = avg 4025 Value added by Coalitions Discount per person

The Shapley value: computation A faster way The amount player i contributes to coalition S, of size s, is v(S)-v(S-i) This contribution occurs for those orderings in which i is preceded by the s-1 other players in S, and followed by the n-s players not in S ki = 1/n!  S:i in S (s-1)! (n-s)! (v(S)-v(S-i)) 43

The Shapley value has been used for cost sharing. Suppose three planes share a runway. The planes require 1, 2, and 3 KM to land. Let’s label those planes 1, 2, and 3. Thus, a runway of 3 must be build, but how much should each pay? Instead of looking at utility given, look at how much increased cost was required avg

The Shapley value has been used for cost sharing. Suppose three planes share a runway. The planes require 1, 2, and 3 KM to land. Thus, a runway of 3 must be build, but how much should each pay? Instead of looking at utility given, look at how much increased cost was required.

An application: voting power A voting game is a pair (N,W) where N is the set of players (voters) and W is the collection of winning coalitions, s.t. – the empty set is not in W (it is a losing coalition) – N is in W (the coalition of all voters is winning) – if S is in W and S is a subset of T then T is in W Also weighted voting game can be considered in which each player has w voting weight an a quota is required to win. The Shapley value of a voting game is a measure of voting power (Shapley-Shubik power index) – The winning coalitions have payoff 1 – The losing coalitions have payoff 0 46

All powers sum to 1. What is power? {99,99,1: 100} Any two win, so all have equal power 1/3. {6,4,2: 10} Having non-zero weight does not guarantee power. 2 has zero power. It is a dummy. If the same agent exists in all winning coalitions, the core is non-empty (as no subgroup can pull out and win). Simple game: any coalition is either winning (1) or losing (0) It seems clear that some simple games could not be formulated in these terms. (incomplete) 47

An application: voting power The United Nations Security Council in 1954 – 5 permanent members (P) – 6 non-permanent members (N) – the winning coalitions had to have at least 7 members, – but the permanent members had veto power A winning coalition had to have at least seven members including all the permanent members The seventh member joining the coalition is the pivotal one: he makes the coalition winning Is this a fair voting system? 48

An application: voting power 11! possible orderings Power of non permanent members – (PPPPPN)N(NNNN) – 5 ways to pick which N comes before and 4! possible arrangements for those that come after. (5! total choices) – 6! possible arrangements for (PPPPPN) – The total number of arrangements in which an N is pivotal is 6!5! – The power of all non permanent members is 6!5!/11!, each getting.0022 The power of each permanent members is.1974 The ratio of power of a P member to a N member is 91:2 In 1965 – 5 permanent members (P) – 10 non-permanent members (N) – the winning coalitions has to have at least 9 members, – the permanent members keep the veto power Similar calculations lead to a ratio of power of a P member to a N member equal to 105:1 (.19627: ) 49

How to represent the characteristic function (in condensed way)? Induced subgraphs Node represents agents. Arc represents added benefit if source/target agents are both in the same coalition 50

Example 51 V(ab) = 3 v(ac) = 2 v(abc) = 5 v(bd) = 6 v(d) = 5 v(cd) = 9 v(abc) = 7 How would we change the diagram?

Example 52 What do you think of this method? Any concerns? Enforces superadditivity. Clearly, in linear space, we can’t expect to represent an exponentially rich set of rewards. What if B added to either A or C adds 3, but the gain isn’t experienced twice? Do we need to worry about cases where B adds 3 to A, but if C is also present, the gain is actually negative?

We say the method is … succinct – as it is much easier to represent incomplete – as there are characteristic functions for which the method doesn’t work How would we use the weighted graph to compute Shapley value? It seems like each node should get half the weight of the arc it shares. We can prove that by breaking in into small problems and using Shapley’s additivity axiom. 53

Marginal Contribution Nets Extension of idea of induced subgraphs where total weight is sum of weight due to parts. We have a series of patterns (boolean expressions involving agents) and values. To determine the characteristic value for a subset, you add up the values (positive or negative) associated with the rule. 54

Example #1: a Λ b  5 marginal increase #2: b  2 #3: c  4 #4: b Λ ¬c  -2 so 55 Possible with induced graphs, right? CoalitionValueRules a0none b0#2,#4 c4#3 ab5#1,#2,#4 ac4#3 bc6#2. #3 abc11#1,#2,#3

Thought questions? Are the rules powerful enough? According to researchers, all characteristic functions can be represented, but we aren’t told how to generate the rules!!! Plus, is it really any simpler? Instead of showing all combinations, you now have all the original combinations PLUS the combinations with negative components. 56

Majority game A group of agents S has a value of 1 if |S| > n/2. Can we do this with Induced subgraph? Can we do this with marginal contribution nets? 57