Price of Anarchy Georgios Piliouras. Games (i.e. Multi-Body Interactions) Interacting entities...

Price of Anarchy

Georgios Piliouras

Games (i.e. Multi-Body Interactions)

• Interacting entities• Pursuing their own goals• Lack of centralized control

Prediction?

Games

n players Set of strategies Si for each player i Possible states (strategy profiles) S=×Si Utility ui:S→R Social Welfare Q:S→R Extend to allow probabilities Δ(Si), Δ(S) ui(Δ(S))=E(ui(S)) Q(Δ(S))=E(Q(S))

(review)

Zero-Sum Games & Equilibria

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

RockRock

Paper

Paper

Scissors

Scissors

Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

1/3 1/3 1/3

1/31/31/3

Existence, Uniqueness of Payoffs [von Neumann 1928](review)

General Games & Equilibria?

1, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

RockRock

Paper

Paper

Scissors

Scissors

Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

Borel conjectured the non-existence of eq. in general

Prediction in GamesIdea 1

Nash Equilibrium (1950): A strategy tuple (i.e. one for each agent) s.t. no agent can deviate profitably.

For finite games, it always exists when we allow agents to randomize

Proof on the board

Equilibria & Prediction

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

NE 0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

Implicit assumptions The players all will do their utmost to maximize

their expected payoff as described by the game. The players are flawless in execution. The players

have sufficient intelligence to deduce the solution. The players know (can compute) the planned

equilibrium strategy of all of the other players. The players believe that a deviation in their own

strategy will not cause deviations by any other players. There is common knowledge that all players meet these conditions, including this one.

Uniqueness

Games & Equilibria

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

RockRock

Paper

Paper

Scissors

Scissors

Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

1/3 1/3 1/3

1/31/31/3

Equilibria & Prediction

20, 20 0, 11, 0 1, 1

StagStag

Hare

Hare

Multiple Nash:Which one to choose?

Prediction in GamesIdea 2a

Koutsoupias and Papadimitriou (1999) If there exist several Nash Equilibria, then be

pessimistic and output the worst possible one. (worst case analysis)

Worst in terms of what? Social Welfare Examples of Social Welfare: Sum of utilities, maxmin utility, median utility

Metrics of Social WelfareExamples

Sum of latencies (sec)maxmin utility ($)

Throughput bottleneck (bit/sec)

Prediction in GamesIdea 2b

Koutsoupias and Papadimitriou (1999) If there exist several Nash Equilibria,

then be pessimistic and output the worst possible one. (worst case analysis)

Normalization

Social Cost (worst Equilibrium)

Social Cost (OPT)Price of Anarchy =

≥ 1

PoA = ≥ 1

Social Cost (worst Equilibrium)

Social Cost (OPT)

x10

10

0A

B

C

D

10 agentsA D

PoA = 4/3

PoA ≤ 5/2, for all networks

delay (x) = x

[Koutouspias, Christodoulou 05]

Price of Anarchy

Equilibria & Prediction

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

NE 0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

NE PoA

Advantages of PoA Approach

Simplicity Widely Applicable (conditions?) Allows for cross-domain comparisons (e.g.

routing game vs facility location game) Analytically tractable?

Several variants: Price of Stability, Price of Total Anarchy, Price of X,…

YES, 1000+ citations

BREAK

Q: Any other ways to make predictions in multi-body problems? How do you do it in real life situations?

Recap + Plan• Games + Worst Case Analysis +

Normalization

• PoA =

• To do: – PoA Analysis (when welfare = sum utility)– Beyond Nash equilibria

Social Cost (worst Equilibrium)

Social Cost (OPT)

PoA

Congestion Games• n players and m resources (“edges”)• Each strategy corresponds to a set of

resources (“paths”)• Each edge has a cost function ce(x) that

determines the cost as a function on the # of players using it.

• Cost experienced by a player = sum of edge costs x xx x

2x 2xx x

Cost(red)=6

Cost(green)=8

Potential Games• A potential game is a game that exhibits a

function Φ: S→R s.t. for every s ∈ S and every agent i,

ui (si,s-i) - ui (s) = Φ (si,s-i) - Φ (s) • Every congestion game is a potential game:

Why?• This implies that any such game has a pure

NE. Why?

PoA ≤ 5/2 for linear latencies[Koutouspias, Christodoulou 05], [Roughgarden 09]

Definition: A game is (λ,μ)-smooth if i Ci(s*

i,s-i) ≤ λcost(s*) + μ cost(s) for all s,s*

Then: POA (of pure Nash eq) ≤ λ/(1-μ)

Proof: Let s arbitrary Nash eq.cost(s) = i Ci(s) [definition of social cost]

≤ i Ci(s*i,s-i) [s a Nash eq]

≤ λcost(s*) + μ cost(s) [(λ,μ)-smooth]

PoA ≤ 5/2 for linear latencies[Koutouspias, Christodoulou 05], [Roughgarden 09]

Technical lemma: A linear congestion game is (5/3,1/3)-smooth.

Proof :Step 0: Matlab simulations to get a hint about

what is the best possible (λ,μ) s.t. game is (λ,μ)-smooth.

Step 1: Verify hypothesis (On the board).

Tight Example

• N agents, • 2N elements (x1, x2,…, xN) (y1, y2,…, yN)

c(x)=x for all of them• Each agent i has 2 strategies : (xi ,yi) or

(xi, yi-1, yi+1)

…

x1

…

y1xN yNx2 y2

BREAK 2

Q: What about PoA of mixed NE?

Recap + Plan• Games + Worst Case Analysis +

Normalization

• PoA =

• To do: – PoA Analysis (when welfare = sum utility)– Beyond Nash equilibria

Social Cost (worst Equilibrium)

Social Cost (OPT)

PoA

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

Rock Paper Scissors

RockPaperScissors

1/3

1/3

1/3

1/3 1/3 1/3

Other Equilibrium Notions

Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

Nash: A probability distribution over outcomes, that is a product of mixed strategiess.t. no player has a profitable deviating strategy.

Choose any of the green outcomes uniformly (prob. 1/9)

Rock Paper Scissors

RockPaperScissors

1/3

1/3

1/3

1/3 1/3 1/3

Other Equilibrium Notions

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

Nash: A probability distribution over outcomes,

s.t. no player has a profitable deviating strategy.

Rock Paper Scissors

RockPaperScissors

1/3

1/3

1/3

1/3 1/3 1/3

Coarse Correlated Equilibria (CCE):

Other Equilibrium Notions

A probability distribution over outcomes, s.t. no player has a profitable deviating strategy.

Rock Paper Scissors

RockPaperScissors

Coarse Correlated Equilibria (CCE):

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

Other Equilibrium Notions

A probability distribution over outcomes, s.t. no player has a profitable deviating strategy.

Rock Paper Scissors

RockPaperScissors

Coarse Correlated Equilibria (CCE):

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

Choose any of the green outcomes uniformly (prob. 1/6)

Other Equilibrium Notions

A probability distribution over outcomes, s.t. no player has a profitable deviating strategyeven if he can condition the advice from the dist..

Rock Paper Scissors

RockPaperScissors

Correlated Equilibria (CE):

0, 0 -1, 1 1, -11, -1 0, 0 -1, 1-1, 1 1, -1 0, 0

Choose any of the green outcomes uniformly (prob. 1/6)

Other Equilibrium Notions

Is this a CE? NO

Other Equilibrium Notions

Pure NE NE CE CCE

Smoothness bounds extend to CCE

Definition: A game is (λ,μ)-smooth if i Ci(s*

i,s-i) ≤ λcost(s*) + μ cost(s)for all s,s*

Then: POA (of pure Nash eq) ≤ λ/(1-μ)

Proof: Let s arbitrary Nash eq.cost(s) = i Ci(s) [definition of social cost]

≤ i Ci(s*i,s-i) [s a Nash eq]

≤ λcost(s*) + μ cost(s) [(λ,μ)-smooth]

Smoothness bounds extend to CCE

Definition: A game is (λ,μ)-smooth ifi Ci(s*

i,s-i) ≤ λcost(s*) + μ cost(s)for all s,s*

Then: POA (of pure Nash eq) ≤ λ/(1-μ)

Proof: Let s arbitrary CCE.E[cost(s)] = E[i Ci(s)] [definition of social cost]

≤ E[ i Ci(s*i,s-i)] [s a CCE]

≤ λ E[cost(s*)] + μ E[cost(s)]

Criticism of PoA Analysis

• What happens in we add 10^10 to the utilities of each agent?

• Tightness is achieved over classes.

• Holds only for sum of utilities

• Sensitive to noise

Open Questions

• Choose your favorite class of games. Attempt (λ,μ)-smoothness analysis.–Possible problems• Technique gives trivial upper bounds• Still need to identify lower bounds

• What about uncertainty?

• Other (hidden) assumptions?

[Balcan,Blum,Mansur’09] [Balcan,Constantin,Ehrlich‘11]

Recap

• Nash always exists (fixed point) but not unique

• PoA addresses non-uniqueness

• (λ,μ)-smoothness general technique for proving PoA bounds, extends to other notions, and can provide tight bounds

Thank You