1 Intrinsic Robustness of the Price of Anarchy Tim Roughgarden Stanford University

1

Intrinsic Robustness of the Price of

Anarchy

Tim RoughgardenStanford University

2

The Mathematical Model

• a directed graph G = (V,E)

• k source-destination pairs (s1 ,t1), …, (sk ,tk)

• a rate (amount) ri of traffic from si to ti

• for each edge e, a cost function ce(•)– assumed nonnegative, continuous,

nondecreasing

s1 t1

c(x)=x Flow = ½

Flow = ½

c(x)=1

Example: (k,r=1)

3

Routings of Traffic

Traffic and Flows:

• fP = amount of traffic routed on si-ti path P

• flow vector f routing of traffic

Selfish routing: what are the equilibria?

s t

4

Nash Flows

Some assumptions:• agents small relative to network (nonatomic

game)• want to minimize cost of their path

Def: A flow is at Nash equilibrium (or is a Nash flow) if all flow is routed on min-cost paths [given current edge congestion]

xs t

1Flow = .5

Flow = .5

s t1

Flow = 0

Flow = 1x

Example:

5

History + Generalizations

• model, defn of Nash flows by [Wardrop 52]

• Nash flows exist, are (essentially) unique– due to [Beckmann et al. 56]– general nonatomic games: [Schmeidler 73]

• congestion game (payoffs fn of # of players)– defined for atomic games by [Rosenthal 73]– previous focus: Nash eq in pure strategies exist

• potential game (equilibria as optima)– defined by [Monderer/Shapley 96]

6

The Cost of a Flow

Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)

s t

x

1½½

Cost = ½•½ +½•1 = ¾

7

The Cost of a Flow

Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate)

Formally: if cP(f) = sum of costs of edges of P (w.r.t. the flow f), then:

C(f) = P fP • cP(f)

s ts t

x

1½½

Cost = ½•½ +½•1 = ¾

8

Inefficiency of Nash Flows

Note: Nash flows do not minimize the cost • observed informally by [Pigou 1920]

• Cost of Nash flow = 1•1 + 0•1 = 1• Cost of optimal (min-cost) flow = ½•½ +½•1 = ¾• Price of anarchy := Nash/OPT ratio = 4/3

s t

x

10

1 ½

½

9

Braess’s Paradox

Initial Network:

s tx 1

½

x1½

½

½

cost = 1.5

10

Braess’s Paradox

Initial Network: Augmented Network:

s tx 1

½

x1½

½

½

cost = 1.5

s tx 1

½

x1½

½

½0

Now what?

11

Braess’s Paradox

Initial Network: Augmented Network:

s tx 1

½

x1½

½

½

cost = 1.5 cost = 2

s t

x 1

x10

12

Braess’s Paradox

Initial Network: Augmented Network:

All traffic incurs more cost! [Braess 68]

• see also [Cohen/Horowitz 91], [Roughgarden 01]

s tx 1

½

x1½

½

½

cost = 1.5 cost = 2

s t

x 1

x10

13

The Bad News

Bad Example: (r = 1, d large)

Nash flow has cost 1, min cost 0

Nash flow can cost arbitrarily more than the optimal (min-cost) flow– even if cost functions are polynomials

s t

xd

10

1 1-Є

Є

14

Linear Cost Functions

First: focus on special case.

Def: linear cost fn is of form ce(x)=aex+be

Theorem: [Roughgarden/Tardos 00] for every network with linear cost fns:

≤ 4/3 ×

i.e., price of anarchy ≤ 4/3 in the linear case.

cost of Nash flow

cost of opt flow

15

Sources of Inefficiency

Corollary of previous Theorem:• For linear cost fns, worst Nash/OPT ratio is

realized in a two-link network!

• simple explanation for worst inefficiency– confronted w/two routes, selfish users

overcongest one of them

s t

x

10

1 ½

½

• Cost of Nash = 1

• Cost of OPT = ¾

16

Simple Worst-Case Networks

Theorem: [Roughgarden 02] fix any class of cost fns, and the worst Nash/OPT ratio occurs in a two-node, two-link network.

• under mild assumptions• inefficiency of Nash flows always has simple

explanation; simple networks are worst examples

17

Simple Worst-Case Networks

Theorem: [Roughgarden 02] fix any class of cost fns, and the worst Nash/OPT ratio occurs in a two-node, two-link network.

• under mild assumptions• inefficiency of Nash flows always has simple

explanation; simple networks are worst examples

Proof Idea: Nash flows minimize potential function

• potential function “close” to total cost function

18

Computing the Price of Anarchy

Application: worst-case examples simple worst-case ratio is easy to calculate

Example: polynomials with degree ≤ d, nonnegative coeffs POA ≈ d/log d

• quartic functions: worst-case POA ≈ 2

• 10% extra "capacity": worst-case POA ≈ 2

19

But Are We at Equilibrium?

Since 2002: price of anarchy (i.e., worst Nash/OPT ratio) analyzed in many models.

Critique: Usual interpretation of a POA bound presumes players reach equilibrium.

.

20

But Are We at Equilibrium?

Since 2002: price of anarchy (i.e., worst Nash/OPT ratio) analyzed in many models.

Critique: Usual interpretation of a POA bound presumes players reach equilibrium.

Soln #1: Justify via convergence theorems.

Soln #2: [taken here] Prove bounds for much bigger sets than just Nash equilibria.

21

Weaker Equilibrium Concepts

pureNash

mixed Nash

correlated eq

no regret

22

Weaker Equilibrium Concepts

pureNash

mixed Nash

correlated eq

no regret

best-responsedynamics

23

Main Result (Informal)

Informal Theorem: [Roughgarden 09] under “surprisingly general” conditions, a bound on the price of anarchy (for pure Nash) extends automatically to all 5 bigger sets.

Example Application: selfish routing games (nonatomic or atomic) with cost functions in an arbitrary fixed set.

24

The Setup

• n players, each picks a strategy si

• player i incurs a cost Ci(s)

Important Assumption: objective function is cost(s) := i Ci(s)

Next: generic template for upper bounding price of anarchy of pure Nash equilibria.

• notation: s = a Nash eq; s* = an optimal

25

An Upper Bound Template

Suppose we have:

cost(s) = i Ci(s) [defn of cost]

≤ i Ci(s*i,s-i) [s a Nash

eq]

26

An Upper Bound Template

Suppose we have:

cost(s) = i Ci(s) [defn of cost]

≤ i Ci(s*i,s-i) [s a Nash

eq] ≤ λ●cost(s*) + μ●cost(s) [(*)]

Then: POA (of pure Nash eq) ≤ λ/(1-μ).

27

An Upper Bound Template

Suppose we have:

cost(s) = i Ci(s) [defn of cost]

≤ i Ci(s*i,s-i) [s a Nash eq]

≤ λ●cost(s*) + μ●cost(s) [(*)]

Then: POA (of pure Nash eq) ≤ λ/(1-μ).

Definition: A game is (λ,μ)-smooth if (*) holds for every pair s,s* outcomes.

• not only when s is a pure Nash eq!

28

Main Result #1

Examples: selfish routing, linear cost fns.• every nonatomic game is (1,1/4)-smooth• every atomic game is (5/3,1/3)-smooth

29

Main Result #1

Examples: selfish routing, linear cost fns.• every nonatomic game is (1,1/4)-smooth• every atomic game is (5/3,1/3)-smooth

Theorem 1: in a (λ,μ)-smooth game, expected cost of each outcomes in the 5 sets above is at most λ/(1-μ).– such a POA bound “automatically” far more

general

30

Illustration

worstcorrelated equilibium

worstno regretsequence

1

optimaloutcome

worstpureNash

worstmixedNash

λ/(1-μ)

So: in every (λ,μ)-smooth game with a sum objective, inefficiency of outcomes in the 5 sets looks like:

31

Main Result #2

Theorem 2 (informal): in sufficiently rich classes of games, smoothness arguments suffice for a tight worst-case bound (even for pure Nash equilibria).

correlated equilibium

no regretsequence

1

optimaloutcome

pureNash

mixedNash

λ/(1-μ)for tightestchoice of λ,μ

Special Case of Result #1

Definition: a sequence s1,s2,...,sT of outcomes is no-regret if:

• for each player i, each fixed action qi:– average cost player i incurs over sequence no

worse than playing action qi every time

– simple hedging strategies can be used by players to enforce this (for suff large T)

Result: in a (λ,μ)-smooth game, average cost of every no-regret sequence at most λ/(1-μ) cost of optimal outcome.

32

Take-Home Points

• guarantees on equilibrium quality possible in interesting problem domains

• the most common way of proving such bounds automatically yields a much more robust guarantee– and this technique often gives tight bounds

Future research agenda: broader understanding of performance guarantees for adaptive systems.

33