Prisoners Dilemma rules

1. Binding agreements are not possible. Note in Prisoners dilemma, if binding agreements were possible, there would be no dilemma

2. Utility is given directly to individuals as a result of individual actions. So, I don’t need to be worried about collective utility.

Solution: cooperative game theory1

Cooperative Games

• Coalitions – set of agents• Grand coalition – all agents work together• Characteristic function: v:2Ag R subsets of agents are assigned a

value.• Simple coalitional game: a coalition has

value 0 or 1. A voting system can be thought of as a simple coalitional game (have enough votes to win or do not)

2

Example• Characteristic function:• v{1} = 1 v{2} = 3 v{1,2} = 5• How should we divide the profit if they work together? Which

are feasible (have enough utility to cover), fair (treats equals equally), efficient (all utility is used – none left on the table), stable (no one will opt out)?

• <5,0>• <4,1>• <3,2>• <2,3>• <1,4>• <0,5>• <2,2>• <2,4>

3

Core• Ex: software engineering group project. Who would you

choose to work with? Who would also choose you? Coalition doesn’t form unless everyone is happy with it.

• Stability is necessary but not sufficient for a coalition to form. If it isn’t stable, someone will defect, but if there are multiple stable coalitions, another may form instead.

• core: set of feasible distributions of payoff to members of the grand coalition so none will defect.

• We require the outcome (payments) to be both feasible (able to pay) and efficient (all utility distributed).

• Pareto efficiency is the efficiency referred to.

4

• Asking if the grand coalition is stable means is the core non-empty?

• The point of the core is to study stability not fairness.

5

So what are concerns?

• What if there are no outcomes in the core?• What if there are multiple outcomes in the

core, how do you pick?• Looking at all possible distributions of

utility is exponential as you have 2n possible objecting subsets to consider.

6

Example• v{1} = 2 v{2} = 2 v{3} = 2 • v{1,2} = 5 v{2,3} = 5 v{1,3}=4 v{1,2,3} = 6• How should we divide the profit if they work

together? Which are feasible, fair, efficient, stable?

• <2,2,2>• <3,3,0>• <1,2,3>• <1,2.5,2.5>• <2,2.5, 1.5>

7

More Examples Emptiness & multiplicity

8

• Example 1: Let us modify the above example so that agents receive no additional utility from being all together (and being

alone gives 0)– v({1, 2, 3}) = 6,

– v({1, 2}) = v({1, 3}) = v({2, 3}) = 6,– v({1}) = v({2}) = v({3}) = 0

• Now the core is empty! Notice, the core must involve the grand coalition (giving payoff for each).

• Example 2:– v({1, 2, 3}) = 18,

– v({1, 2}) = v({1, 3}) = v({2, 3}) = 10,– v({1}) = v({2}) = v({3}) = 0

• Now lots of outcomes are in the core – (6, 6, 6), (5, 5, 8), …

Fairness• Could we get people to join a coalition by

saying, “Will you agree to a fair solution even though it is not in the core?”

• In examplev{1} =0 v{2} = 0 v{3} = 0 v{1,2} = 5 v{2,3} = 5 v{1,3} = 5 v{1,2,3} = 6What would a fair but unstable solution be?• If we had too much (that no one could really

demand), how would we divide any surplus?

9

Terms• Marginal Contribution (value added): μi(C) = v(C U {i}) – v(C)

10

Superadditivity

11

• v is superadditive if for all coalitions A, B with A∩B = Ø, v(AUB) ≥ v(A) + v(B)

• Informally, the union of two coalitions can always act as if they were separate, so should be able to get at least what they would get if they were separate.

• There is a synergy – if not, coalitions make no sense.• Superadditivity usually makes sense• Previous examples were all superadditive• Given this, always efficient for grand coalition to form• Without superadditivity, finding a core is not possible.

The Shapley value [Shapley 1953]

12

• In dividing the profit, sometimes agent is given its marginal contribution (how much better the group is by its addition)

• The marginal contribution scheme is unfair because it depends on the ordering of the agents

• One way to make it fair: average over all possible orderings• Let MC(i, π) be the marginal contribution of i in ordering π

• Then i’s Shapley value is ΣπMC(i, π)/(n!)• The Shapley value is always in the core for convex (definition

to follow) games• … but not in general, even when core is nonempty, e.g.

– v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1,– v = 0 everywhere else

Example: v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1,v = 0 everywhere else

1 2 3123

132

213

231

312

321avg

13

What is a core solution?

Compute the Shapley value for each.

Is the solution in the core?

Example: v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1,v = 0 everywhere else

1 2 3123 0 1 0

132 0 0 1

213 1 0 0

231 1 0

312 1 0 0

321 1 1 0

avg 4/6 1/6 1/6

14

Compute the Shapley value for each.

Is the solution in the core?

What is a core solution?

Axiomatic characterization of the Shapley value

15

• The Shapley value is the unique solution concept that satisfies:

– (Pareto) Efficiency: the total utility is the value of the grand coalition, Σi in Nu(i) = v(N)

– Symmetry: two symmetric players (add the same amount to coalitions they join) must receive the same utility

– Dummy: if v(S {i}) = v(S) for all S, then i must get 0– Additivity: if we add two games defined by v and w by

letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w

• most controversial axiom (for example, participant i’s cost-share of a runway and terminal is it’s cost-share of the runway plus his cost-

share of the terminal)

Shapley Result• Satisfies all three fairness axioms• If you came up with some other method

that satisfied all three fairness axioms, it would be Shapley. In other words, the Shapley value is the UNIQUE value that satisfies all fairness axioms.

16

Computing a solution in the core

17

• How do you even represent the problem? exponential in the number of agents.

• Can use linear programming:– Variables: u(i)– Distribution constraint: Σi in Nu(i) = v(N)– Non-blocking constraints: for every S, Σi in Su(i) ≥ v(S)

• Problem: number of constraints exponential in number of players (as you have values for all possible subsets)

• … but is this practical?

Convexity

18

• v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B)• In other words, the amount A adds to B (in forming the union) is

at least as much it adds to the intersection.• One interpretation: the marginal contribution of an agent is

increasing in the size of the set that it is added to. The term marginal contribution means the additional contribution.

Precisely, the marginal contribution of A to B is v(AUB)-v(B) • So if A can scale a mountain, sing a song, and hop and B can

scale a mountain, dance, and yodel:• v(A∩B) = v(scale)

• v(A)-v(A∩B) = v(scale, sing, hop)-v(scale)• v(AUB)-v(B) = v(scale, sing, hop, dance, yodel) –

v(scale,dance,yodel)

Convexity

19

• v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B)

• Previous examples were not convex (why?) v{1} = 2 v{2} = 2 v{3} = 2

v{1,2} = 5 v{2,3} = 5 v{1,3}=4 v{1,2,3} = 6

• v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B). Let A = {1,2} and B={2,3}

• v(AUB)-v(B) = 6 – 5 = 1• v(A)-v(A∩B) = 5 - 2 = 3

Convexity

20

• In convex games, core is always nonempty. (Core doesn’t require convexity, but convexity produces a core.)

• Example:• v{1} = 5 v{2} = 2 v{3} = 1 v{1,2} = 8 v{1,3} = 7 v{2,3} = 4 v{1,2,3} = 11

• One easy-to-compute solution in the core: agent i gets u(i) = v({1, 2, …, i}) - v({1, 2, …, i-1})

– Marginal contribution scheme- each agent is rewarded by what it ads to the union of all previous agents in a given order.

– Works for any ordering of the agents

The important questions

• Which coalitions should form?• How should a coalition which forms divide its winnings

among its members?• Unfortunately there is no definitive answer• Many concepts have been developed since 1944:

– stable sets– core– Shapley value– bargaining sets– nucleolus– Gately point

21

The Shapley value has been used for cost sharing. Suppose threeplanes share a runway. The planes require 1, 2, and 3 KM to land.

Let’s label those planes 1, 2, and 3.Thus, a runway of 3 must be build, but how much should each pay?

Instead of looking at utility given, look at how much increased cost was required.

1 2 3123132213231312321avg

The Shapley value has been used for cost sharing. Suppose three

planes share a runway. The planesrequire 1, 2, and 3 KM to land.Thus, a runway of 3 must be

build, but how much should each pay?

Instead of looking at utility given, look at how much increased cost was required.

How to represent the characteristic function?

• Induced subgraphsNode represents agents arc represents

added benefit if source/target agents are both in the same coalition

24

Example

25

What is coalition ABC worth?

Who would get what share of the profit?

What do you think of this method? Any concerns?

What if B added to either A or C adds 3, but the gain isn’t experienced twice?

Do we need to worry about cases where B adds

3 to A, but if C is also present, the gain is actually negative?

We say the method…• is succinct – as it is much easier to represent• incomplete – as there are characteristic

functions for which the method doesn’t work• How would be use the weighted graph to

compute Shapley value? It seems like each node should get half the weight of the arc it shares.

• We can prove that by breaking in into small problems and using Shapley’s additivity axiom.

26

Coalition Structure Formation• Sometimes we split the problem into

– Deciding which agents will work together– Deciding what tasks they will do

• This is particularly appropriate if all agents are owned by same group so we are just trying to maximize social welfare.

• .

27

A concise representation based on synergies [Conitzer & Sandholm AIJ06]

• Assume superadditivity• Say that a coalition S is synergetic if there do not exist

A, B with A ≠ Ø, B ≠ Ø, A∩B = Ø, AUB = S, v(S) = v(A) + v(B)

• Value of non-synergetic coalitions can be derived from values of smaller coalitions

• So, only specify values for synergetic coalitions in the input

Other concise representations of coalitional games

• [Deng & Papadimitriou 94]: agents are vertices of a graph, edges have weights, value of coalition = sum of weights of edges in coalition

• [Conitzer & Sandholm 04]: represent game as sum of smaller games (each of which involves only a few agents)

• [Ieong & Shoham 05]: multiple rules of the form (1 and 3 and (not 4) → 7), value of coalition = sum of values of rules that apply to it– E.g., the above rule applies to coalition {1, 2, 3} (so it

gets 7 from this rule), but not to {1, 3, 4} or {1, 2, 5} (so they get nothing from this rule)

– Generalizes the above two representations (but not synergy-based representation)

Nucleolus [Schmeidler 1969]• Always gives a solution in the core if there exists one• Always uniquely determined• A coalition’s excess e(S) is v(S) - Σi in Su(i)• For a given outcome, list all coalitions’ excesses in decreasing

order• E.g., consider

– v({1, 2, 3}) = 6, – v({1, 2}) = v({1, 3}) = v({2, 3}) = 6,– v({1}) = v({2}) = v({3}) = 0

• For outcome (2, 2, 2), the list of excesses is 2, 2, 2, 0, -2, -2, -2 (coalitions of size 2, 3, 1, respectively)

• For outcome (3, 3, 0), the list of excesses is 3, 3, 0, 0, 0, -3, -3 (coalitions {1, 3}, {2, 3}; {1, 2}, {1, 2, 3}, {3}; {1}, {2})

• Nucleolus is the (unique) outcome that lexicographically minimizes the list of excesses– Lexicographic minimization = minimize the first entry first, then (fixing

the first entry) minimize the second one, etc.