Introduction to Game Theory - Project Group DynaSearch · PDF fileForexample,inmanyboardgames(e.g.chess) ... eachplayerchoosesexactlyonestrategy Deﬁnition ... PPAD-complete Part2:

Introduction to Game TheoryProject Group DynaSearch

Introduction to Game TheoryProject Group DynaSearch

November 5th, 2013

Maximilian Drees

Source: Fotolia, Jürgen Priewe

Introduction to Game Theory Maximilian Drees 1

HEINZ NIXDORF INSTITUTEUniversity of PaderbornAlgorithms and Complexity

In many situations, the outcome does not only depend on the actions of a singleentity, but on those of multiple entities, each with its own personal agenda.

For example, in many boardgames (e.g. chess), the winner is determined by his ownmoves as well as by the moves of the other players.

This can be applied to other, more “serious” fields, e.g. the stock market.

Game Theory



Interactions of rational decision-makersdecision-makers: agents/players

interactions: multiple agents act simultaneously or consequently

rational: each agent has preferences over outcomes, choose action which mostlikely leads to best feasible outcome

Goalsunderstand/predict behavior of players

predict outcomes

know how to gain advantages

design systems

Game Theory



Part 1: Normal Form Games & Nash Equilibriageneral introduction into game theorybasic notionsfirst examplescomplexity issues

Part 2: Selfish Routing & Price of Anarchyspecial case of gamesmore examplesquality of outcomes

Agenda



Part 1: Normal Form Games & Nash Equilibria



DefinitionA normal form game is a triple (N, (Si)i∈N , (ci)i∈N), where

N is the set of players, |N| = nSi is the set of strategies of player ici : S1 × S2 × . . .Sn → R is the cost function of player i

Example: Prisoner’s Dilemma

Silent ConfessSilent 2/2 5/1Confess 1/5 4/4

Normal Form Games



DefinitionFor player i with strategy set Si = {S1

i , . . . ,Ski }, a pure strategy is a Sj

i ∈ Si .

each player chooses exactly one strategy

DefinitionA pure strategy profile S is a vector of pure strategies, i.e. S ∈ S1 × . . .× Sn.

the costs of each player depends on the whole (pure) strategy profile

Pure Strategy Profiles



DefinitionA pure Nash equilibrium (pure NE) is a pure strategy profile in which no player canimprove its costs by unilaterally changing its pure strategy.

a pure NE is optimal for each player, provided the strategies of the other playersare fixed

Example: Prisoner’s Dilemma

Silent ConfessSilent 2/2 5/1Confess 1/5 4/4

Pure Nash Equilibrium



Example: Rock-Paper-Scissors

Rock Paper ScissorsRock 0/0 0/1 1/0Paper 1/0 0/0 0/1Scissors 0/1 1/0 0/0

there is no pure Nash equilibrium!





Rock Paper ScissorsRock 0/0 0/1 1/0Paper 1/0 0/0 0/1Scissors 0/1 1/0 0/0

there is no pure Nash equilibrium!




remember: in a pure strategy profile, each player chooses exactly one of itsstrategies

DefinitionFor player i with strategy set Si = {S1

i , . . . ,Ski }, a mixed strategy is a vector

xi = (x1i , . . . , xk

i ) ∈ Rk≥0 with

∑ki=1 x

ki = 1

x ji denotes probability of player i playing strategy Sj

i

DefinitionA mixed strategy profile is a vector of mixed strategies, i.e. (x1, . . . , xk), s.t. xi is amixed strategy for player i.

Mixed Strategy Profiles



DefinitionA mixed Nash equilibrium (mixed NE) is a mixed strategy profile in which no playercan improve its expected costs by unilaterally changing its mixed strategy.

Example: Rock-Paper-ScissorsRock Paper Scissors

Rock 0/0 0/1 1/0Paper 1/0 0/0 0/1Scissors 0/1 1/0 0/0

player Aplays rock with probability 1

2plays paper with probability 1

3 mixed strategy(

12 ,

13 ,

16

)plays scissors with probability 1

6player B

plays rock with probability 13

plays paper with probability 13 mixed strategy

(13 ,

13 ,

13

)plays scissors with probability 1

3

Mixed Nash equilibrium




A/B Rock Paper ScissorsRock 0/0 0/1 1/0Paper 1/0 0/0 0/1Scissors 0/1 1/0 0/0

player A: mixed strategy ( 12 ,13 ,

16 )

player B: mixed strategy ( 13 ,13 ,

13 )

mixed strategy profile(( 1

2 ,13 ,

16),( 13 ,

13 ,

13))

Result for player A:

12

(13 · 0+

13 · 0+

13 · 1

)+

13

(13 · 1+

13 · 0+

13 · 0

)+

16

(13 · 0+

13 · 1+

13 · 0

)

=16 +

19 +

118 =

13





A/B Rock Paper ScissorsRock 0/0 0/1 1/0Paper 1/0 0/0 0/1Scissors 0/1 1/0 0/0

there is a mixed Nash equilibrium based on the mixed strategy profile((13 ,

13 ,

13

),

(13 ,

13 ,

13

))

expected payoff for both players is 13




Theorem (Nash)Every finite normal form game has a mixed Nash equilibrium.

John Forbes Nash, Jr (13.06.1928)movie: A Beautiful Mind28-page dissertation on non-cooperative games

Nash’s Theorem



How hard is it to compute Nash equilibria?not a decision problem, because we already know mixed NE existnot an optimization problem, because a mixed NE does not have to be ”good“

DefinitionThe binary relation P(x, y) is contained in TFNP if and only if

for every x, there exists a y such that P(x, y) holdsthere is a poly. time algo. to determine whether P(x, y) holds

TFNP contains search problems

DefinitionPPAD is the class of search problems from TFNP which can be reduced toEnd-of-a-Line.

PPAD stands for ”Polynomial Parity Argument, Directed Case“

Complexity of Nash Equilibria



DefinitionLet V be a set of solutions in which each v has at most one successor and onepredecessor. Given an initial solution v0 and two poly. time computable functions

S : V → V which computes the successor of a solutionP : V → V which computes the predecessor of a solution

the task for the problem End-of-a-Line is to find a final solution with no successor orone with no predecessor and different from v0.

Intuition behind the problem on the board.

End-of-a-Line



Theorem (Cheng and Deng, 2005)The problem of computing a mixed Nash equilibrium (NASH) is PPAD-complete.

Concept of proof:1 reduce NASH to End-of-a-Line⇒ NASH is part of PPAD

2 reduce End-of-a-Line to NASH⇒ NASH is at least as hard as End-of-a-Line⇒ NASH is PPAD-hard



Theorem (Cheng and Deng, 2005)The problem of computing a mixed Nash equilibrium (NASH) is PPAD-complete.

Concept of proof:1 reduce NASH to End-of-a-Line⇒ NASH is part of PPAD

2 reduce End-of-a-Line to NASH⇒ NASH is at least as hard as End-of-a-Line⇒ NASH is PPAD-hard



Part 2: Selfish Routing & Price of Anarchy



graph G = (V ,E) with latency functions le(x) on the edgesk commodities with demand riflow of volume ri is sent from source si to sink tilatency function maps the amount of flow on an edge to a cost value

NotationsPi set of paths from si to tiP =

⋃ki=1 Pi set of all relevant paths

flow fi defines a value fi(P) ∈ [0, ri ] for every P ∈ Pi satisfying∑

P∈Pifi(P) = ri

flow f contains all values defined by all fifP =

∑ki=1 fi(P) is the total flow along path P ∈ P (P /∈ Pi ⇒ fi(P) = 0)

flow along edge e ∈ E is fe =∑

P3e fPlatency on edge e ∈ E at flow f is le(f) = le(fe)latency on path P ∈ P at flow f is lP(f) = lP(fP) =

∑e∈P le(fe)

Wardrop’s Traffic Model



Definition (Equilibrium flow)A flow f is at an equilibrium if for every i ∈ [k] and for every pair of paths P1,P2 ∈ Piwith fP1 > 0, it holds that lP1(f) ≤ lP2(f).

imagine a flow to consist of infinitely many particles of infinitesimal sizea flow is at a equilibrium if no particle can choose a path with smaller latencyfor every i ∈ [k] and every pair of paths P1,P2 ∈ Pi which are actually used byf , lP1(f) = lP2(f)

Example: Braess Paradox (on the board)

Equilibrium Flow



DefinitionThe average latency of a flow f is

C(f) :=∑P∈P

fP · lP(f) =∑e∈E

fe · le(f)

A flow minimizing the average latency is called optimal flow.

the average latency is the sum of the individual cost of each particlethe average latency of a flow equilibrium can be greater than the average latencyof an optimal flow

Another example on the board.



DefinitionLet flow f be at an equilibrium and let every other flow g at an equilibrium have asmaller average latency, i.e. C(f) ≥ C(g). Let flow opt be an optimal flow.Then the price of anarchy is defined as

PoA :=C(f)

C(opt)

PoA compares to worst-case situation with the optimal oneit measures the need to introduce a centralized control instance or enforce a setof rules on the players

Theorem (Roughgarden and Tardos, 2001)The price of anarchy for routing games with linear latency functions is at most 4

3 .

Now: Proof

Price of Anarchy



DefinitionLet flow f be at an equilibrium and let every other flow g at an equilibrium have asmaller average latency, i.e. C(f) ≥ C(g). Let flow opt be an optimal flow.Then the price of anarchy is defined as

PoA :=C(f)

C(opt)

PoA compares to worst-case situation with the optimal oneit measures the need to introduce a centralized control instance or enforce a setof rules on the players


3 .

Now: Proof

Price of Anarchy




3 .

assume all latency functions to be monotone increasing

LemmaA flow f is at an equilibrium if and only if for every flow g the following holds:∑

P∈P

(fP − gP) · lP(fP) ≤ 0

LemmaFor every edge e ∈ E

g(e) · [le(fe)− le(ge)] ≤14 · fe · le(fe)

Price of Anarchy




3 .

assume all latency functions to be monotone increasing

LemmaA flow f is at an equilibrium if and only if for every flow g the following holds:∑

P∈P

(fP − gP) · lP(fP) ≤ 0

LemmaFor every edge e ∈ E

g(e) · [le(fe)− le(ge)] ≤14 · fe · le(fe)

Price of Anarchy



Part 1: Normal Form Games & Nash Equilibriageneral introduction into game theory - Normal Form gamesbasic notions - pure/mixed strategy profiles, Nash equilibriumfirst examples - Prisoner’s Dilemma, Rock-Paper-Scissorscomplexity issues - complexity class PPAD (search problems), NASH

PPAD-complete

Part 2: Selfish Routing & Price of Anarchyspecial case of games - routing games based on Wardrop’s traffic modelmore examples - Braess Paradoxquality of outcomes - Price of Anarchy compares optimal outcome with worst

equilibrium

Summary



Thank you for your attention!Thank you for your attention!

Maximilian Drees

Heinz Nixdorf Institute& Department of Computer ScienceUniversity of Paderborn

Address: Fürstenallee 1133102 PaderbornGermany

Phone: +49 5251 60-6434Fax: +49 5251 60-6482E-mail: [email protected]: http://wwwhni.upb.de/en/alg/staff/max

Source: Fotolia, Jürgen Priewe



mailto:[email protected]

http://wwwhni.upb.de/en/alg/staff/max

Documents

Introduction to Game Theory - Project Group DynaSearch · PDF fileForexample,inmanyboardgames(e.g.chess) ... eachplayerchoosesexactlyonestrategy Deﬁnition ... PPAD-complete Part2: