1

Smooth Games and Intrinsic Robustness

Christodoulou and Koutsoupias,Roughgarden

Slides stolen/modified from Tim Roughgarden

2

Congestion Games• Agent i has a set of strategies, Si, each

strategy s in Si is a set of resources• The cost to an agent is the sum of the

costs of the resources r in s used by the agent when choosing s

• The cost of a resource is a function of the number of agents using the resource

fr(# agents)3

4

Price of AnarchyPrice of anarchy: [Koutsoupias/Papadimitriou

99] quantify inefficiency w.r.t some objective function.– e.g., Nash equilibrium: an outcome such that

no player better off by switching strategiesDefinition: price of anarchy (POA) of a

game (w.r.t. some objective function):

optimal obj fn valueequilibrium objective fn value

the closer to 1 the better

5

Network w/2 players:

s t2x 12

5x50

Atomic identical flowis a Congestion game

6

Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)

Formally: if cP(f) = sum of costs of edges of P (w.r.t. the flow f), then:

C(f) = P fP • cP(f)

s ts tx

1½½

Cost = ½•½ +½•1 = ¾

Costs: atomic and non-atomic flow

7

Def: linear cost fn is of form ce(x)=aex+be

Linear costs

8

Nash Equilibrium: To Minimize Cost:

Price of anarchy = 28/24 = 7/6.• if multiple equilibria exist, look at the worst

one

s t2x 12

5x5cost = 14+10 = 24

cost = 14+14 = 28

s t2x 12

5x500

Atomic identical flowLinear costs

9

Theorem: [Roughgarden/Tardos 00] for every non-atomic flow network with linear cost fns:

≤ 4/3 ×

i.e., price of anarchy non atomic flow ≤ 4/3 in the linear latency case.

cost of non-atomic Nash flow

cost of opt flow

POA non-atomic flow

10

Abstract Setup• n players, each picks a strategy si

• player i incurs a cost Ci(s)

Important Assumption: objective function is cost(s) := i Ci(s)

Key Definition: A game is (λ,μ)-smooth if, for every pair s,s* outcomes (λ > 0; μ < 1):

i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)

11

Smooth => POA BoundNext: “canonical” way to upper bound

POA (via a smoothness argument).• notation: s = a Nash eq; s* = optimal

Assuming (λ,μ)-smooth:

cost(s) = i Ci(s) [defn of cost]

≤ i Ci(s*i,s-i) [s a Nash

eq] ≤ λ●cost(s*) + μ●cost(s)

[(*)]

Then: POA (of pure Nash eq) ≤ λ/(1-μ).

12

“Robust” POABest (λ,μ)-smoothness parameters:

cost(s) = i Ci(s) ≤ i Ci(s*

i,s-i) ≤ λ●cost(s*) + μ●cost(s)Minimizing: λ/(1-μ).

Congestion games with affine cost functions are (5/3,1/3)-

smooth• Claim: For all non-negative integers y,

z :

13

y(z + 1) · 53y2 + 1

3z2:

Thus,

14

y(z + 1) · 53y2 + 1

3z2

) ay(z + 1) + by · 53(ay2 + by) + 1

3(az2 + bz)

Let s, s¤ be any two vectors of strategies in a congestion game,with loads x and x¤,in (s¤

i ;s¡ i ) the number of users of e is · xe + 1, we havekX

i=1Ci (s¤

i ;s¡ i ) ·X

e2E(ae(xe + 1) + be)x¤

e

·X

e2E

53(aex¤

e + be)x¤e +

X

e2E

13(aexe + be)xe

= 53C(s¤) + 1

3C(s):

y = x¤e; z = xe

POA 5 / 2 Any congestion game(includes atomic unit

flow)

a,b ≥0

15

Why Is Smoothness Stronger?

Key point: to derive POA bound, only needed

i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)

to hold in special case where s = a Nash eq and s* = optimal.

Smoothness: requires (*) for every pair s,s* outcomes.– even if s is not a pure Nash equilibrium

16

The Need for RobustnessMeaning of a POA bound: if the game is at

an equilibrium, then outcome is near-optimal.

17

The Need for RobustnessMeaning of a POA bound: if the game is at

an equilibrium, then outcome is near-optimal.

Problem: what if can’t reach equilibrium?• (pure) equilibrium might not exist• might be hard to compute, even

centrally– [Fabrikant/Papadimitriou/Talwar], [Daskalakis/

Goldberg/Papadimitriou], [Chen/Deng/Teng], etc.• might be hard to learn in a distributed

way

Worry: are POA bounds “meaningless”?

18

Robust POA BoundsHigh-Level Goal: worst-case bounds that

apply even to non-equilibrium outcomes!

• best-response dynamics, pre-convergence– [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05],

[Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08]• correlated equilibria

– [Christodoulou/Koutsoupias 05]• coarse correlated equilibria aka “price

of total anarchy” aka “no-regret players”– [Blum/Even-Dar/Ligett 06],

[Blum/Hajiaghayi/Ligett/Roth 08]

19

Lots of previous work uses smoothness Bounds

• atomic (unweighted) selfish routing [Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05], [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09]

• nonatomic selfish routing [Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05]

• weighted congestion games [Aland/Dumrauf/Gairing/Monien/Schoppmann 06],

[Bhawalkar/Gairing/Roughgarden 10]• submodular maximization games [Vetta 02], [Marden/Roughgarden 10]• coordination mechanisms [Cole/Gkatzelis/Mirrokni 10]

Beyond Pure Nash Equilibria (Static)

20

pureNash

mixed Nashcorrelated eq

CCE

For all s;s0i : Es» ¾[Ci (s)] · E s¡ i » ¾¡ i [Ci (s0

i ;s¡ i )]¾= ¾1 £ ¾2 £ ¢¢¢£ ¾k

For all s;s0i : Es» ¾[Ci (s)] · E s» ¾[Ci (s0

i ;s¡ i )]

For all s;s0i : E s» ¾[Ci (s)jsi ] · E s» ¾[Ci (s0

i ;s¡ i )jsi ]

Mixed:

Correlated:

Coarse Correlated:

¾6= ¾1 £ ¾2 £ ¢¢¢£ ¾k

Beyond Nash Equilibria (non-Static)

Definition: a sequence s1,s2,...,sT of outcomes is no-regret if:

• for each player i, each fixed action qi:– average cost player i incurs

over sequence no worse than playing action qi every time

– if every player uses e.g. “multiplicative weights” then get o(1) regret in poly-time

– empirical distribution = "coarse correlated eq"

21

pureNash

mixed Nashcorrelated eq

no-regret

An Out-of-Equilibrium Bound

Theorem: [Roughgarden STOC 09] in a (λ,μ)-smooth game, average cost of every no-regret sequence at most

[λ/(1-μ)] x cost of optimal outcome.

(the same bound we proved for pure Nash equilibria)

22

23

Smooth => No-Regret Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

24

Smooth => No-Regret Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

= t i [Ci(s*i,st

-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st

-

i)]

25

Smooth => No-Regret Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

= t i [Ci(s*i,st

-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st

-

i)]

≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]

26

Smooth => No-Regret Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

= t i [Ci(s*i,st

-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st

-

i)]

≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]

No regret: t ∆i,t ≤ 0 for each i.To finish proof: divide through by T.

Intrinsic RobustnessTheorem: [Roughgarden STOC 09] for every set

C, unweighted congestion games with cost functions restricted to C are tight:maximum [pure POA] = minimum [λ/(1-μ)]congestion games

w/cost functions in C(λ ,μ): all such gamesare (λ ,μ)-smooth

27

Intrinsic RobustnessTheorem: [Roughgarden STOC 09] for every set

C, unweighted congestion games with cost functions restricted to C are tight:maximum [pure POA] = minimum [λ/(1-μ)]

• weighted congestion games [Bhawalkar/ Gairing/Roughgarden ESA 10] and submodular maximization games [Marden/Roughgarden CDC 10] are also tight in this sense

congestion gamesw/cost functions in C

(λ ,μ): all such gamesare (λ ,μ)-smooth

28