Learning Cooperative Games

Maria-Florina Balcan, Ariel D. Procaccia and Yair Zick(to appear in IJCAI 2015)

Cooperative Games

Players divide into coalitions to perform tasks

Coalition members can freely divide profits.

How should profits be divided?

Cooperative Games

A set of players - Characteristic function - • – value of a coalition .

Imputation: a vector satisfying efficiency: And individual rationality:

Cooperative Games

A game is called simple if

is monotone if for any :

The Core

An imputation is in the core if

• Each subset of players is getting at least what it can make on its own. • A notion of stability; no one can deviate.

6

Learning Coalitional ValuesI want the

forest cleared of threats!

7

Learning Coalitional Values

I’ll pay my men fairly to do it.

8

Learning Coalitional Values

But, what can they do?

9

Learning Coalitional Values

I know nothing!

10

Learning Coalitional Values

0 100 50 150

Let me observe what the scouting

missions do

11

Learning Cooperative Games

We want to find a stable outcome, but the valuation function is unknown.

Can we, using a small number of samples, find a payoff division that is

likely to be stable?

12

PAC Learning

We are given samples from an (unknown) function

Given these samples, find a function that approximates .

Need to make some structural assumptions on (e.g. is a linear classifier)

14

PAC Learning

Probably approximately correct: observing i.i.d samples from a distribution , with probability (probably), I am going to output a function that is wrong on at most a measure of of sets sampled from (approximately correct).

15

PAC Stability

Probably approximately stable: observing i.i.d samples from a distribution , with probability (probably), output a payoff vector that is unstable against at most a measure of of sets sampled from (approximately stable),

… or output that the core is empty.

16

Stability via LearnabilityTheorem: let be an PAC approximation of ; if then w.p. ,

Some caveats:1. Need to still guarantee that (we often can)2. Need to handle cases where but .

17

Stability via LearnabilitySo, if we can PAC learn , we can PAC stabilize .Is there another way of achieving PAC stability? For some classes of games, the core has a simple structure.

18

Simple Games

19

PAC Stability in Simple Games

Simple games are generally hard to learn [Procaccia & Rosenschein 2006]. But, their core has a very simple structure

Fact: the core of a simple game is not empty if and only if has veto players, in which case any division of payoffs among the veto players is in the core.

No need to learn the structure of the game, just identify the veto players!

20

Simple Games

21

Simple Games

22

Simple Games

23

Simple Games

24

PAC Stability in Simple Games

Only Sam appeared in all observed winning coalitions: he is likely to be a veto player; pay him everything.

25

PAC Stability in Simple Games

Theorem: simple games are PAC stabilizable (though they are not generally PAC learnable).

What about other classes of games?

We investigate both PAC learnability and PAC stability of some common classes of cooperative games.

Network Flow Games

• We are given a weighted, directed graph

• Players are edges; value of a coalition is the value of the max. flow it can pass from s to t.

s

t3

7

5

10

1

3 6

1

3

1

4

57

2

Network Flow Games

Theorem: network flow games are not efficiently PAC learnable unless RP = NP.

Proof idea: we show that a similar class of games (min-sum games) is not efficiently learnable (the reduction from them to network flows is easy).

Network Flow Games

Min-sum games: the class of -min-sum games is the class of games defined by vectors

1-min-sum games: linear functions.

Network Flow Games

Proof Idea:It is known that -clause-CNF formulas (CNF formulas with clauses) are hard to learn if .We reduce hardness for -CNF formulas to hardness for -min-sum.

(𝐱 1 ,𝜙 (𝐱 1 )) ,…,(𝐱𝑚 ,𝜙 (𝐱𝑚 ))

Learn that PAC approximates

Construct -clause CNF from

Argue that PAC approximates

Network Flow Games

Network flow games are generally hard to learn. But, if we limit ourselves to path queries, they are easy to learn!

Theorem: the class of network flow games is PAC learnable (and PAC stabilizable) when we are limited to path queries.

Network Flow Games

s

t3

7

5

10

1

3 6

1

3

1

4

57

2

Proof idea: Suppose we are given the input Define for every

Network Flow Games

s

t

22 2

2

22

Proof idea: Suppose we are given the input Define for every

Network Flow Games

s

t

52 2

2

22

5 5

55

Proof idea: Suppose we are given the input Define for every

Network Flow Games

s

t

22

5 5

55

Proof idea: Suppose we are given the input Define for every

1

1

1

1

11

Threshold Task Games [Chalkiadakis et al., 2011]

Each agent has a weight

A finite set of tasks ; each with a value and a threshold .

A set can complete a task if .

Value of a set: most valuable task that it can complete.

Weighted voting games: single task of value 1.

Threshold Task Games

Theorem: let -TTG be the class of TTGs with tasks; then -TTG is PAC learnable.Proof Idea:

1. : class of TTGs with tasks whose values are known ().

First show that is PAC learnable

2. If after samples from TTG we saw the value set ; then w.p. ,

3. Combining these observations, we know that after enough samples we are likely to know the values of , we can then pretend that our input is from , and learn a game for it. That game PAC approximates .

Additional Results

Induced Subgraph Games [Deng & Papadimitriou, 1994]: PAC learnable, PAC stabilizable if edge weights are non-negative.

1 7

3

9

2

84

5

6

3

2

5 4

1

3 6

1

3

1

4

57

2

Additional ResultsCoalitional Skill Games [Bachrach et al., 2008]: generally hard to learn (but possible under some structural assumptions).

– a set of skills : the skills of agent : the skills required by task : the set of tasks that can complete. is a function of (we look at several variants).

Additional ResultsMC-nets [Ieong & Shoham, 2005]: learning MC-nets is hard (disjoint DNF problem).

A list of rules of the form

“if contains and , but does not contain , award it a value of ”Value of : sum of its evaluations on rules.

ConclusionsHandling uncertainty in cooperative games is important!

- Gateway to their applicability. - Can we circumvent hardness of PAC learning and

directly obtain PAC stable outcomes (like we did in simple games)?

- What about distributional assumptions?

Thank you! Questions?

Additional Slides

Shattering Dimension and LearningGiven a class of functions that take values in , and a set of sets, we say that shatters if for every vector , there is some function such that

Intuitively: is complex enough in order to label the sets in in any way possible.

Shattering Dimension and LearningClaim: we only need a number of samples polynomial in , and to -learn a class of boolean functions .

Shattering Dimension and LearningIf takes real values, we cannot use VC dimension. Given a set of sets of size , and a list of real values , we say that shatters if for every there exists some function such that

The pseudo-dimension of

Shattering Dimension and LearningClaim: we only need a number of samples polynomial in , and to -learn a class of real functions .

Reverse Engineering a GameI have a (known) game I tell you that it belongs to some class :- it’s a -vector WVG- It’s a network flow game- It’s a succinct MC netBut I’m not telling you what are the parameters!Can you recover them? Using active/passive learning