Upload
loraine-davis
View
221
Download
0
Tags:
Embed Size (px)
Learning Cooperative Games
Maria-Florina Balcan, Ariel D. Procaccia and Yair Zick(to appear in IJCAI 2015)
Cooperative Games
Players divide into coalitions to perform tasks
Coalition members can freely divide profits.
How should profits be divided?
Cooperative Games
A set of players - Characteristic function - • – value of a coalition .
Imputation: a vector satisfying efficiency: And individual rationality:
Cooperative Games
A game is called simple if
is monotone if for any :
The Core
An imputation is in the core if
• Each subset of players is getting at least what it can make on its own. • A notion of stability; no one can deviate.
6
Learning Coalitional ValuesI want the
forest cleared of threats!
7
Learning Coalitional Values
I’ll pay my men fairly to do it.
8
Learning Coalitional Values
But, what can they do?
9
Learning Coalitional Values
I know nothing!
10
Learning Coalitional Values
0 100 50 150
Let me observe what the scouting
missions do
11
Learning Cooperative Games
We want to find a stable outcome, but the valuation function is unknown.
Can we, using a small number of samples, find a payoff division that is
likely to be stable?
12
PAC Learning
We are given samples from an (unknown) function
Given these samples, find a function that approximates .
Need to make some structural assumptions on (e.g. is a linear classifier)
14
PAC Learning
Probably approximately correct: observing i.i.d samples from a distribution , with probability (probably), I am going to output a function that is wrong on at most a measure of of sets sampled from (approximately correct).
15
PAC Stability
Probably approximately stable: observing i.i.d samples from a distribution , with probability (probably), output a payoff vector that is unstable against at most a measure of of sets sampled from (approximately stable),
… or output that the core is empty.
16
Stability via LearnabilityTheorem: let be an PAC approximation of ; if then w.p. ,
Some caveats:1. Need to still guarantee that (we often can)2. Need to handle cases where but .
17
Stability via LearnabilitySo, if we can PAC learn , we can PAC stabilize .Is there another way of achieving PAC stability? For some classes of games, the core has a simple structure.
18
Simple Games
19
PAC Stability in Simple Games
Simple games are generally hard to learn [Procaccia & Rosenschein 2006]. But, their core has a very simple structure
Fact: the core of a simple game is not empty if and only if has veto players, in which case any division of payoffs among the veto players is in the core.
No need to learn the structure of the game, just identify the veto players!
20
Simple Games
21
Simple Games
22
Simple Games
23
Simple Games
24
PAC Stability in Simple Games
Only Sam appeared in all observed winning coalitions: he is likely to be a veto player; pay him everything.
25
PAC Stability in Simple Games
Theorem: simple games are PAC stabilizable (though they are not generally PAC learnable).
What about other classes of games?
We investigate both PAC learnability and PAC stability of some common classes of cooperative games.
Network Flow Games
• We are given a weighted, directed graph
• Players are edges; value of a coalition is the value of the max. flow it can pass from s to t.
s
t3
7
5
10
1
3 6
1
3
1
4
57
2
Network Flow Games
Theorem: network flow games are not efficiently PAC learnable unless RP = NP.
Proof idea: we show that a similar class of games (min-sum games) is not efficiently learnable (the reduction from them to network flows is easy).
Network Flow Games
Min-sum games: the class of -min-sum games is the class of games defined by vectors
1-min-sum games: linear functions.
Network Flow Games
Proof Idea:It is known that -clause-CNF formulas (CNF formulas with clauses) are hard to learn if .We reduce hardness for -CNF formulas to hardness for -min-sum.
(𝐱 1 ,𝜙 (𝐱 1 )) ,…,(𝐱𝑚 ,𝜙 (𝐱𝑚 ))
Learn that PAC approximates
Construct -clause CNF from
Argue that PAC approximates
Network Flow Games
Network flow games are generally hard to learn. But, if we limit ourselves to path queries, they are easy to learn!
Theorem: the class of network flow games is PAC learnable (and PAC stabilizable) when we are limited to path queries.
Network Flow Games
s
t3
7
5
10
1
3 6
1
3
1
4
57
2
Proof idea: Suppose we are given the input Define for every
Network Flow Games
s
t
22 2
2
22
Proof idea: Suppose we are given the input Define for every
Network Flow Games
s
t
52 2
2
22
5 5
55
Proof idea: Suppose we are given the input Define for every
Network Flow Games
s
t
22
5 5
55
Proof idea: Suppose we are given the input Define for every
1
1
1
1
11
Threshold Task Games [Chalkiadakis et al., 2011]
Each agent has a weight
A finite set of tasks ; each with a value and a threshold .
A set can complete a task if .
Value of a set: most valuable task that it can complete.
Weighted voting games: single task of value 1.
Threshold Task Games
Theorem: let -TTG be the class of TTGs with tasks; then -TTG is PAC learnable.Proof Idea:
1. : class of TTGs with tasks whose values are known ().
First show that is PAC learnable
2. If after samples from TTG we saw the value set ; then w.p. ,
3. Combining these observations, we know that after enough samples we are likely to know the values of , we can then pretend that our input is from , and learn a game for it. That game PAC approximates .
Additional Results
Induced Subgraph Games [Deng & Papadimitriou, 1994]: PAC learnable, PAC stabilizable if edge weights are non-negative.
1 7
3
9
2
84
5
6
3
2
5 4
1
3 6
1
3
1
4
57
2
Additional ResultsCoalitional Skill Games [Bachrach et al., 2008]: generally hard to learn (but possible under some structural assumptions).
– a set of skills : the skills of agent : the skills required by task : the set of tasks that can complete. is a function of (we look at several variants).
Additional ResultsMC-nets [Ieong & Shoham, 2005]: learning MC-nets is hard (disjoint DNF problem).
A list of rules of the form
“if contains and , but does not contain , award it a value of ”Value of : sum of its evaluations on rules.
ConclusionsHandling uncertainty in cooperative games is important!
- Gateway to their applicability. - Can we circumvent hardness of PAC learning and
directly obtain PAC stable outcomes (like we did in simple games)?
- What about distributional assumptions?
Thank you! Questions?
Additional Slides
Shattering Dimension and LearningGiven a class of functions that take values in , and a set of sets, we say that shatters if for every vector , there is some function such that
Intuitively: is complex enough in order to label the sets in in any way possible.
Shattering Dimension and LearningClaim: we only need a number of samples polynomial in , and to -learn a class of boolean functions .
Shattering Dimension and LearningIf takes real values, we cannot use VC dimension. Given a set of sets of size , and a list of real values , we say that shatters if for every there exists some function such that
The pseudo-dimension of
Shattering Dimension and LearningClaim: we only need a number of samples polynomial in , and to -learn a class of real functions .
Reverse Engineering a GameI have a (known) game I tell you that it belongs to some class :- it’s a -vector WVG- It’s a network flow game- It’s a succinct MC netBut I’m not telling you what are the parameters!Can you recover them? Using active/passive learning