4. & 5. Game Theory - UCSB Department of Economicsecon.ucsb.edu/~oprea/176/Games.pdf · 4. & 5. Game Theory Economics 176 Games and Nash Equilibrium Prisoner’s Dilemmas Coordination

4. & 5. GameTheory

Economics 176

Games and NashEquilibrium

Prisoner’sDilemmas

CoordinationGames

Behavioral GameTheory

4. & 5. Game Theory

Ryan Oprea

University of California, Santa Barbara

Economics 176

4. & 5. GameTheory

Economics 176



CoordinationGames


• Individual choice experiments• Test assumptions about Homo Economicus

• Strategic interaction experiments• Test game theory

• Market experiments• Test classical notions of competitive equilibrium

4. & 5. GameTheory

Economics 176



CoordinationGames


Game theoryCompetitive markets and individual decision problems have a lot incommon:

• In each case you don’t have to spend too much time thinkingabout what others are going to do..

• ...or expend too much energy trying to control what choicesothers make.

Game theory is the branch of economics that studies strategicinteraction:

• People’s actions affect one another and

• the effect of one’s own actions depend on the actions of others.

In strategic settings your best choice might depend on what othersare choosing.

• In that case no sense talking about “best” choice

• Instead we talk about “stable” set of choices (aka equilibrium).

4. & 5. GameTheory

Economics 176



CoordinationGames


Game theory

To use game theory we start with a description of a game:

• Players• A list of agents who take an action

• Actions• A set of decisions available to players

• Payoffs• A description of how each player is benefitted by her own actions

and the actions of others.

• Information• A description of what players know.

4. & 5. GameTheory

Economics 176



CoordinationGames


Example: Bimatrix Game

Simplest type of game is a Bi-matrix game:

• Two players

• Two available actions

• Payoffs are known to everyone.

• Actions are simultaneous (Normal form)

Blue SubLeft Right

Su

b Top 2020

050

Red Bottom 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames



Generally we can’t just calculate out what each player’s best decisionis.

• In many games the best decision depends on what others choose!

• Instead we first find the best response:

• Best choice given each possible choice of counterpart(s).

Blue SubLeft Right

Su

b Top 2020

050

Red Bottom 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames



Generally we can’t just calculate out what each player’s best decisionis.

• In many games the best decision depends on what others choose!

• Instead we first find the best response for each player:

• Best choice given each possible choice of the player’scounterpart(s).

Blue SubLeft Right

Su

b Top 2020

050

Red Bottom 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames


Nash Equilibrium

Nash equilibrium: A set of strategies for all players that are all bestresponses at once.

• Idea: such a set of strategies is stable as

• nobody has a reason to unilaterally change actions.

Blue SubLeft Right

Su

b Top 2020

050

Red Bottom 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames


Nash Equilibrium




Blue SubLeft Right

Su

b Top 2020

050

Red Bottom 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames


Nash Equilibrium




Blue SubLeft Right

Su

b Top 2020

050

Red Bottom 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames


Nash Equilibrium and Experiments

Why might people play Nash equilibrium and when? Nash gave twoanswers

• Epistemic: People reason their way to equilibrium.

• Evolution: Groups of people mutually learn their way toequilibrium.

• Question: Do either of these mechanisms work? Do both?

Nash equilibrium: probably most often used hypotheses generatingtool in experimental economics.

• However, it is applied to many very different types of strategicproblems

• Price competition• Team cooperation• Warfare• Negotiations

• Question: Does one theory (Nash equilibrium) really makeaccurate predictions across varying domains?

4. & 5. GameTheory

Economics 176



CoordinationGames



Why might people play Nash equilibrium and when? Nash gave twoanswers

• Epistemic: People reason their way to equilibrium.

• Evolution: Groups of people mutually learn their way toequilibrium.

• Question: Do either of these mechanisms work? Do both?

Nash equilibrium: probably most often used hypotheses generatingtool in experimental economics.

• However, it is applied to many very different types of strategicproblems

• Price competition• Team cooperation• Warfare• Negotiations

• Question: Does one theory (Nash equilibrium) really makeaccurate predictions across varying domains?

4. & 5. GameTheory

Economics 176



CoordinationGames



Experimental economists test game theory in the lab using severaldistinct timing and matching protocols:

• One shot: Subjects play game once, get paid and go home.

• Random rematching (strangers matching): Subjects playover and over again but each time are randomly (andanonymously) matched with someone new.

• Repeated rematching (partners matching): Subjects arematched with the same counterpart(s) over and over again (andthey know it).

4. & 5. GameTheory

Economics 176



CoordinationGames







4. & 5. GameTheory

Economics 176



CoordinationGames







4. & 5. GameTheory

Economics 176



CoordinationGames



Which of these you choose radically changes what type of questionsyou can ask (and answer).

• One shot: Subjects must “reason” their way to a solution.

• Random rematching (strangers matching): Subjects canmutually learn how one another play and possibly converge on astable outcome (perhaps a Nash equilibrium).

• Repeated rematching (partners matching): Subjects canform reputations, tacitly collude with one another, and “punish”one another etc.

4. & 5. GameTheory

Economics 176



CoordinationGames


The Prisoner’s Dilemma

The most famous bimatrix game of them all

• The sum of payoffs are maximized by cooperating (Coop)

• but both would be tempted to break cooperation so

• both choose to defectBlue Sub

Defect Coop

Su

b Defect 2020

050

Red Coop 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames



The game is interesting because

• Individual rationality predicts an outcome totally different from

• what maximizes social efficiency.

• This isn’t just a Nash equilibrium, but one in dominantstrategies!

Blue SubDefect Coop

Su

b Defect 2020

050

Red Coop 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames



Turns out a huge number of important environments in the socialworld have this feature

• Team work, common pool resources, arms races, advertising,fashion, college eduction, price competition, public goodsprovision etc.

• The prisoner’s dilemma is the simplest setting in whichindividual rationality and social efficiency are so starkly apposed.

• Makes it a great testing ground for studying this fundamentaltension.

Blue SubDefect Coop

Su

b Defect 2020

050

Red Coop 50

030

30

4. & 5. GameTheory

Economics 176



CoordinationGames


One-Shot and Strangers Matching

What happens when subjects are essentially playing without thepossibility of building a reputation?

One-shot game

• Modest cooperation in low-temptation, high-reward settings.

Strangers matching

• Modest initial cooperation

• “Cooperative” decay in which subjects quickly learn to play Nash

4. & 5. GameTheory

Economics 176



CoordinationGames


One-Shot and Strangers Matching

What happens when subjects are essentially playing without thepossibility of building a reputation?

One-shot game

• Modest cooperation in low-temptation, high-reward settings.

Strangers matching

• Modest initial cooperation

• “Cooperative” decay in which subjects quickly learn to play Nash

4. & 5. GameTheory

Economics 176



CoordinationGames


Finitely Repeated Prisoner’sDilemma

In a finitely repeated prisoner’s dilemma nobody should cooperateeither.

• “Unravelling” and backwards induction.

• Only complete defection is a Nash equilibrium!

Famously, this is not how people behave in finitely repeated prisoner’sdilemmas! Typical pattern:

• Cooperation early in the game and

• eventual defection later in the game.

• Rate of initial cooperation and timing of defection depend on• Parameters (gains to cooperation, temptation to defect)• number of repetitions.

4. & 5. GameTheory

Economics 176



CoordinationGames


Finitely Repeated Prisoner’sDilemma

In a finitely repeated prisoner’s dilemma nobody should cooperateeither.

• “Unravelling” and backwards induction.

• Only complete defection is a Nash equilibrium!

Famously, this is not how people behave in finitely repeated prisoner’sdilemmas! Typical pattern:

• Cooperation early in the game and

• eventual defection later in the game.

• Rate of initial cooperation and timing of defection depend on• Parameters (gains to cooperation, temptation to defect)• number of repetitions.

4. & 5. GameTheory

Economics 176



CoordinationGames


Andreoni and Miller (1993)

Note difference between partners and strangers treatment.

4. & 5. GameTheory

Economics 176



CoordinationGames


ExplanationsTwo favorite types of explanations for this “failure” of standard gametheory (this is still an area of active research).

Reputations (Gang of Four):

• Suppose there is a tiny possibility that your counterpartcooperates until defected on

• then it can be rational to pretend to be this type of player inearly rounds and

• defect only later in the game.

Bounded Rationality ε−equilibrium:

• When the rewards to maintaining cooperation are high

• subjects are willing to accept minor deviations from bestresponse, causing them to cooperate.

• Potential losses from being preempted in defecting get smallrelative to benefits of cooperation the longer the game isrepeated.

4. & 5. GameTheory

Economics 176



CoordinationGames











4. & 5. GameTheory

Economics 176



CoordinationGames











4. & 5. GameTheory

Economics 176



CoordinationGames


“Infinitely” RepeatedStandard game theory predicts no cooperation in finitely repeatedprisoner’s dilemmas.

However if the discount rate is high enough, standard theory doespredict cooperation in infinitely repeated prisoner’s dilemmas.

• For instance, by threatening to punish by defecting forever (the“grim trigger strategy”)

• you create incentives for counterpart to cooperate.

Can’t run an experiment forever but you can reinterpret discountfactor as a per round probability that the period ends (Roth andMurnighan, 1978).

• For instance end period if you roll a 1 on a die• equivalent to a discount factor of 5/6!

What does the experimental evidence show?• Cooperation doesn’t emerge when discount rates are too low for

it to emerge in standard theory (Dal Bo and Frechette, 2011)• high discount factors tend to lead to higher rates of cooperation

(i.e. Dal Bo, 2005)!

4. & 5. GameTheory

Economics 176



CoordinationGames








What does the experimental evidence show?

• Cooperation doesn’t emerge when discount rates are too low forit to emerge in standard theory (Dal Bo and Frechette, 2011)

• high discount factors tend to lead to higher rates of cooperation(i.e. Dal Bo, 2005)!

4. & 5. GameTheory

Economics 176



CoordinationGames








What does the experimental evidence show?• Cooperation doesn’t emerge when discount rates are too low for

it to emerge in standard theory (Dal Bo and Frechette, 2011)• high discount factors tend to lead to higher rates of cooperation

(i.e. Dal Bo, 2005)!

4. & 5. GameTheory

Economics 176



CoordinationGames


The Stag HuntThe basic Prisoner’s Dilemma is simple in an important sense

• You don’t care what your counterpart does (your best responseis constant)

• cooperating is always a dominated strategy.

In coordination games this isn’t the case

• You care what your counterpart did

• and want to match their action.

• The Stag-Hunt game is a famous example.

Blue SubStag Hare

Su

b Stag 2020

100

Red Hare 0

103

3

4. & 5. GameTheory

Economics 176



CoordinationGames


The Stag HuntThe basic Prisoner’s Dilemma is simple in an important sense

• You don’t care what your counterpart does (your best responseis constant)

• cooperating is always a dominated strategy.

In coordination games this isn’t the case

• You care what your counterpart did

• and want to match their action.

• The Stag-Hunt game is a famous example.

Blue SubStag Hare

Su

b Stag 2020

100

Red Hare 0

103

3

4. & 5. GameTheory

Economics 176



CoordinationGames


The Stag Hunt GameThere are multiple equilibria to this game (and in coordination gamesmore generally)!

• Big problem for game theory as

• it doesn’t actually make a prediction here!

Game theorists have names for these equilibria

• The efficient outcome (20, 20) is payoff dominant

• the safer outcome (3, 3) is risk dominant

Blue Substag Hare

Su

b Stag 2020

100

Red Hare 0

103

3

It is traditional to assume that the payoff dominant equilibrium isfocal and will be chosen by decision makers.

4. & 5. GameTheory

Economics 176



CoordinationGames


The Stag Hunt GameThere are multiple equilibria to this game (and in coordination gamesmore generally)!

• Big problem for game theory as

• it doesn’t actually make a prediction here!

Game theorists have names for these equilibria

• The efficient outcome (20, 20) is payoff dominant

• the safer outcome (3, 3) is risk dominant

Blue Substag Hare

Su

b Stag 2020

100

Red Hare 0

103

3

It is traditional to assume that the payoff dominant equilibrium isfocal and will be chosen by decision makers.

4. & 5. GameTheory

Economics 176



CoordinationGames


Stag Hunt ExperimentsOne-shot implementations

• People tend to play the inefficient risk dominant equilibrium!

Strangers matching implementations• Depends strongly on what happens initially but• strong tendency for things to converge to risk dominant

equilibrium.• Depends also to some degree on the costs of mis-coordinating in

the specific version of the game.

Partners matching implementations• Coordination on the pareto dominant equilibrium is common• as subjects are often willing to sacrifice earnings to show they

are serious about the payoff dominant equilibrium• and wait for their counterparts to catch up!

What if you let people communicate (via chat) before makingdecisions?

• In many settings has a strong effect on improving coordinationon the payoff dominant equilibrium!

4. & 5. GameTheory

Economics 176



CoordinationGames


Stag Hunt ExperimentsOne-shot implementations

• People tend to play the inefficient risk dominant equilibrium!

Strangers matching implementations• Depends strongly on what happens initially but• strong tendency for things to converge to risk dominant

equilibrium.• Depends also to some degree on the costs of mis-coordinating in

the specific version of the game.

Partners matching implementations• Coordination on the pareto dominant equilibrium is common• as subjects are often willing to sacrifice earnings to show they

are serious about the payoff dominant equilibrium• and wait for their counterparts to catch up!

What if you let people communicate (via chat) before makingdecisions?

• In many settings has a strong effect on improving coordinationon the payoff dominant equilibrium!

4. & 5. GameTheory

Economics 176



CoordinationGames


A Familiar Game

Nash’s theorem: Every finite player, finite strategy has a Nashequilibrium.

Rock-paper-scissors is a famous game with no evident equilibrium.

Blue SubRock Paper Scissors

Rock 00

1−1

−11

Su

b Paper −11

00

1−1

Red Scissors 1

−1−1

10

0

4. & 5. GameTheory

Economics 176



CoordinationGames


A Familiar Game

The equilibrium of this game is in mixed strategies: that is, eachplayer plays each strategy probabilistically.

Idea: Look for a set of probabilities for each player and each actionsuch that neither would want to change the probabilities.


Rock 00

1−1

−11

Su

b Paper −11

00

1−1

Red Scissors 1

−1−1

10

0

You probably already know what the mixed equilibrium of this gameis.

4. & 5. GameTheory

Economics 176



CoordinationGames


A Familiar Game

The equilibrium of this game is in mixed strategies: that is, eachplayer plays each strategy probabilistically.

Idea: Look for a set of probabilities for each player and each actionsuch that neither would want to change the probabilities.


Rock 00

1−1

−11

Su

b Paper −11

00

1−1

Red Scissors 1

−1−1

10

0

You probably already know what the mixed equilibrium of this gameis.

4. & 5. GameTheory

Economics 176



CoordinationGames


Symmetric Matching Pennies

Lets study a game that is very similar but a bit simpler (only 2actions per subject.)

Blue SubLeft Right

Su

b Up 3672

7236

Red Down 72

3636

72

4. & 5. GameTheory

Economics 176



CoordinationGames



Plot the best responsefunctions. For red player forexample:

• You choose a probabilityp you play up that is abest response

• to the probability q theblue player plays right!

• Reverse for blue, andwhere these bestresponse functionsintersect, you get theMixed Strategy NashEquilibrium.

Blue SubLeft Right

Su

b Up 3672

7236

Red Down 72

3636

72

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Probability of Right

Pro

babi

lity

of U

p

4. & 5. GameTheory

Economics 176



CoordinationGames



Do people actually play like this?

In games like this

• Subjects on average play Mixed Strategy Nash strategies ineach period but

• individual subjects do not seem to actually randomize (chooseiid strategies) from period to period.

• Often individual subjects don’t change behavior at all butaverage behavior in the population is actually close to mixedequilibrium proportions (sometimes called purification).

But one strange prediction of the theory is that a change in yourpayoffs should only affect your counterpart’s decisions – not yours!

• Can this possibly be?

4. & 5. GameTheory

Economics 176



CoordinationGames



Do people actually play like this?

In games like this

• Subjects on average play Mixed Strategy Nash strategies ineach period but

• individual subjects do not seem to actually randomize (chooseiid strategies) from period to period.

• Often individual subjects don’t change behavior at all butaverage behavior in the population is actually close to mixedequilibrium proportions (sometimes called purification).

But one strange prediction of the theory is that a change in yourpayoffs should only affect your counterpart’s decisions – not yours!

• Can this possibly be?

4. & 5. GameTheory

Economics 176



CoordinationGames


Own Payoff Effect

Blue SubLeft Right

Su

b Up 3672

7236

Red Down 72

3636

72

Blue SubLeft Right

Su

b Up 36360

7236

Red Down 72

3636

72

In the bottom game, one ofred’s payoffs is 5 times largerthan in the top game.

• Theoretically this shouldchange only blue’sbehavior.

• It should have no effecton red!

4. & 5. GameTheory

Economics 176



CoordinationGames


Asymmetric Matching Pennies

Note that the effect ofincreasing red’s payoff is to

• Increase the equilibriumprobability blue plays“right” from 50 % to90%

• but red’s probability ofplaying “up” stays wereit was at 50%!

Is this really what happens?

Blue SubLeft Right

Su

b Up 36360

7236

Red Down 72

3636

72

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0


Pro

babi

lity

of U

p

4. & 5. GameTheory

Economics 176



CoordinationGames


Asymmetric Matching Pennies

Of course not! Instead, thepayoff change affects bothplayers.

• Blue is more likely toplay right (as predicted)

• but red is also a lotmore likely to play top!

How do we explain this?

Blue SubLeft Right

Su

b Up 36360

7236

Red Down 72

3636

72

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0


Pro

babi

lity

of U

p

4. & 5. GameTheory

Economics 176



CoordinationGames


Quantal Response Equilibrium

One way is to propose a behavioral alternative to Nash equilibrium thatallows for a plausible type of mistake.

• This research agenda is often called behavioral game theory.

Example is quantal response equilibrium.

• Suppose subjects sometimes make mistakes and

• mistakes more likely if payoff losses from mistakes are small.

Instead of choosing as a strict function of other players’ strategy, youchoose probabilistically. For example it is popular to assume yourprobability of playing up in a game like this is

Pr(Up) =eπup/µ

eπup/µ + eπdown/µ(1)

where πup is the earnings from choosing up and µ is a parameterdetermining the degree of mistake-making as a function of payoffs (to beestimated from the data).

4. & 5. GameTheory

Economics 176



CoordinationGames



One way is to propose a behavioral alternative to Nash equilibrium thatallows for a plausible type of mistake.

• This research agenda is often called behavioral game theory.

Example is quantal response equilibrium.

• Suppose subjects sometimes make mistakes and

• mistakes more likely if payoff losses from mistakes are small.

Instead of choosing as a strict function of other players’ strategy, youchoose probabilistically. For example it is popular to assume yourprobability of playing up in a game like this is

Pr(Up) =eπup/µ

eπup/µ + eπdown/µ(1)

where πup is the earnings from choosing up and µ is a parameterdetermining the degree of mistake-making as a function of payoffs (to beestimated from the data).

4. & 5. GameTheory

Economics 176



CoordinationGames



We can use these “quantalbest responses” to form a“quantal responseequilibrium” (QRE) bylooking for where bestresponses meet.

Note, in symmetric matchingpennies the prediction isidentical to the Nashequilibrium prediction!

Good thing – Nash predictionis correct in this case. Butthen why do we need QRE?

Blue SubLeft Right

Su

b Up 3672

7236

Red Down 72

3636

72

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0


Pro

babi

lity

of U

p

4. & 5. GameTheory

Economics 176



CoordinationGames



It turns out the equilibriumformed by noisy best responsefunctions, actually predicts an“own payoff” effect.

It therefore serves as anexplanation for the non-Nashbehavior in the AsymmetricMatching Pennies game!

• Note, it predicts thelarge effect on red’spayoff change by blue

• but also an increase inthe probability of up forred!

Blue SubLeft Right

Su

b Up 36360

7236

Red Down 72

3636

72

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0


Pro

babi

lity

of U

p

4. & 5. GameTheory

Economics 176



CoordinationGames


Guessing Game

Consider the following simple game:

• N players (i.e. any number) must each choose a numberbetween 0 and 100.

• whoever chooses closes to 2/3 of the average choice wins.

• What is the Nash equilibrium?

A favorite answer is 33

• Assume every choice is equally likely. Then

• average should be around 50 and you should choose 2/3 of this!

• Is this the Nash equilibrium?

No!

4. & 5. GameTheory

Economics 176



CoordinationGames


Iterated Elimination of DominatedStrategies

Idea: In many games, you can think your way to Nash equilibrium.

You think about what others will do

• Others think about what you think they will do• You think about what others think about what you think they

will do.• Others think about what you think about what others will think

about what you think they will do....

In dominance solvable games this kind of process of reasoning willlead to a Nash equilibrium (eventually).

4. & 5. GameTheory

Economics 176



CoordinationGames


Guessing Game Equilibrium

The guessing game is dominance solvable and we can use this to findNash equilibrium:

• You choose randomly, on average choosing 50.

• You guess others reason as above and so choose 33.3.


• You guess others reason as above and so choose 14.8

• and so on.

Where does this process end?

• At zero and this is the Nash equilibrium!

4. & 5. GameTheory

Economics 176



CoordinationGames


Guessing Game Equilibrium

The guessing game is dominance solvable and we can use this to findNash equilibrium:





• and so on.

Where does this process end?

• At zero and this is the Nash equilibrium!

4. & 5. GameTheory

Economics 176



CoordinationGames


Guessing Game DataPeople do not actually do this! Instead subjects tend to choose onaverage around 23.

Also behavior is multimodal, with spikes at distinct points.

What is going on here?

4. & 5. GameTheory

Economics 176



CoordinationGames


Guessing Game DataPeople do not actually do this! Instead subjects tend to choose onaverage around 23.

Also behavior is multimodal, with spikes at distinct points.

What is going on here?

4. & 5. GameTheory

Economics 176



CoordinationGames


Limited Depth of Reasoning

Idea: Notice that the spikes in the previous graph happen at the“steps” of reasoning described above:





• and so on.

We can view each these steps as a level of sophistication in decisionmaking.

4. & 5. GameTheory

Economics 176



CoordinationGames



Idea: Notice that the spikes in the previous graph happen at the“steps” of reasoning described above:

• You choose randomly, on average choosing 50. (Level 0)

• You guess others reason as above and so choose 33.3 (Level 1).

• You guess others reason as above and so choose 22.2 (Level 2).

• You guess others reason as above and so choose 14.8 (Level 3)

• and so on.

We can view each these steps as a level of sophistication in decisionmaking.

4. & 5. GameTheory

Economics 176



CoordinationGames



It seems subjects on average are about Level 2.

• Another strand of behavioral game theory studies this alternative toNash.

• Level-k model or Cognitive Hierarchy model: subjects have limiteddepth of reasoning that determines behavior.

4. & 5. GameTheory

Economics 176



CoordinationGames


One Theory, Many Paths

Game theory implicitly assumes one path to Nash equilibrium, but inpractice there are many.

Even in the simple games we’ve considered so far, there are manymodes of reasoning and adaptation that can lead towards or awayfrom equilibrium:

• Avoid dominated strategies

• Build reputations

• Find focal equilibria

• Look for emergent conventions

• Think about others’ thinking

The fact that framing and modes of interaction can radically changewhich of these paths is followed is a major reason that game theoryneeds an empirical component!

Documents

4. & 5. Game Theory - UCSB Department of Economicsecon.ucsb.edu/~oprea/176/Games.pdf · 4. & 5. Game Theory Economics 176 Games and Nash Equilibrium Prisoner’s Dilemmas Coordination