Cumbersome coordination in repeated games

Int J Game Theory (2000) 29:101±118

99992000

Cumbersome coordination in repeated games

Horst Ra¨, David Schmidt*

Department of Economics, Indiana University, Bloomington, IN 47405, USA(e-mail: hra¨@indiana.edu; [email protected])

Received January 1998 / ®nal version June 1999

Abstract. This paper examines the role communication between players mightserve in enabling them to reach an agreement on the future play of a repeatedgame. The property of the communication process that we focus on is theamount of time it takes to complete. We characterize the eëcts of such com-munication processes indirectly by determining the set of agreements they mayyield. A weak and a strong criterion are introduced to describe sets of agree-ments that are ``stable'' in the sense that players would follow the currentagreement and not seek to reach a new agreement. We show that as playersbecome extremely patient, strongly stable sets converge to Pareto e½cientsingletons. We apply the stability criteria to Prisoner's Dilemmas and showhow the unique strongly stable set re¯ects asymmetries in the players' stage-game payo¨s. Finally, we model the communication process as a Rubinsteinalternating-oër bargaining game and demonstrate that the resulting agree-ments help characterize the strongly stable set for a general class of commu-nication mechanisms.

Key words: Repeated games, coordination, renegotiation, Rubinstein bar-gaining

1. Introduction

In®nitely repeated games typically have many subgame perfect equilibria,which raises the question of how players coordinate their expectations about

* We wish to thank workshop participants at Indiana University and Purdue University forcomments on previous drafts of this paper. We are especially grateful to Bob Becker, AndreasBlume, Roy Gardner and two anonymous referees for their constructive suggestions.

how to play the game. One explanation for how these common expectationsare reached is that the players engage in some sort of pre-play communica-tion. But in a game where players interact repeatedly, communication maynot only take place prior to but also during the play of the game. Players maybe tempted to communicate during the game, if this helps them, for example,to avoid prolonged periods of punishment. Such intraplay communication hasbeen examined in the literature on ``renegotiation proofness'', which charac-terizes strategy pro®les that are immune to communication between players ateach stage of the game.1 By invoking a Pareto criterion, renegotiation proof-ness tends to reduce the set of equilibrium pro®les because communicationmakes commitment to Pareto-inferior future actions, especially punishments,non-credible.

The current paper takes a diërent approach to the problem of intraplaycommunication. In particular, we extend the repeated game to include acommunication mechanism and examine the eëcts that changes in the com-munication environment have on equilibrium behavior in the extended game.Real world communication processes typically do not proceed without com-plication. Two aspects that may contribute to these complications appear tobe of particular interest: (i) time lags in the communication process, and (ii)monetary communication costs. The present paper deals with time lags in thecommunication process ± a situation we refer to as cumbersome coordination.Coordination is cumbersome, for instance, if it takes time for players to de-velop and make proposals to other players on how to play the game, or if itis time-consuming to evaluate proposals received from other players and tocollect and verify pertinent information. The eëct of monetary costs, e.g. thecost of bringing players together to communicate, on equilibrium outcomeshas been studied by Blume (1994). Among other things, he shows that asym-metries in these costs can ®lter through to the equilibrium, in the sense that atleast one player can receive payo¨s strictly greater than in any subgame per-fect equilibrium, if that player's costs are relatively low.2

In addition to modeling a diërent type of communication friction, ourpaper has several features that distinguish it from Blume's. We characterizethe eëcts of a slow communication process indirectly by the set of prescribedpaths of play, or `àgreements,'' it may yield. Two criteria are introduced todescribe sets of agreements that are ``stable'' in the sense that players wouldfollow the current agreement and not seek to reach a new agreement. A``weakly stable set'' has the property that no player has an incentive to deviatefrom an agreement in the set, if it is anticipated that the coordination processwill be used to reach another agreement in the set. A ``strongly stable'' setsatis®es the additional condition that for any agreement outside the set atleast one player would want to deviate from the agreement if the player

1 See, for instance, the papers by Farrell and Maskin (1989), and Bernheim and Ray (1989). Agood discussion of these and other concepts of renegotiation proofness is provided by Abreu,Pearce, Stachetti (1993).2 The practical importance of communication costs has been stressed by McCutcheon (1997).Building on Blume's work, McCutcheon argues that such costs may help facilitate collusion be-tween companies. She explains that while the Sherman (Antitrust) Act prohibits companies fromcoordinating on prices, the actual penalties for collusion are too small to deter collusion entirely.However, she argues, the penalties are just large enough to prevent companies from renegotiatingshould the collusive agreement call for a punishment phase. In this sense, the communication costsimposed by the Sherman Act facilitate collusion.

102 H. Ra¨, D. Schmidt

anticipated the coordination process to lead players to reach an agreementinside the set. We demonstrate that, as players become extremely patient, astrongly stable set becomes arbitrarily small. An application of these criteriato Prisoner's Dilemmas is shown to yield a unique strongly stable set, and itis demonstrated how this set re¯ects asymmetries in the players' stage-game payo¨s. Finally, we model the communication process as a Rubinsteinalternating-oër bargaining game and demonstrate that the resulting agree-ments coincide with the boundaries of the ``strongly stable'' set of agreements.

The rest of the paper is organized as follows: Section 2 introduces thenotation and main assumptions. In Section 3 we present our stability conceptsand provide existence and characterization results for sets of agreements sat-isfying these stability criteria. Section 4 provides an application to symmetricand asymmetric Prisoner's Dilemmas; and Section 5 discusses the results inthe context of Rubinstein bargaining. Section 6 concludes.

2. Description of the game

The two building blocks of our in®nitely repeated game are the stage gameand a coordination process, which are both assumed to be common knowl-edge among all players. The stage game is simply a normal form game thatis to be repeated inde®nitely. The coordination process is a way for playersto reach a common expectation of how to play this repeated stage game.3 Animportant component of the coordination process is a coordination mecha-nism that suggests to the players a path of play in the repeated stage game. Wewill refer to these suggested paths of play as agreements. This mechanism mayinvolve actions by the players, if for instance, the players were to select a pathof play through bargaining. The mechanism may also be seen as an externalparty suggesting an agreement. Another important detail of the coordinationprocesses we consider is that any player can invoke this coordination mecha-nism following the resolution of any stage game. Players are free to follow ordisregard these agreements. A role for a coordination process arises only whenat least one player believes the mechanism will in¯uence the behavior of someplayer in the game. The stability concepts we introduce below will require anassumption concerning players' expectations about other players' responses toa new agreement. Before introducing these concepts, we will ®rst formallydevelop our model.

The set of players is I � f0; 1; 2; . . . ; Ig. Player 0 (nature) may be neededto include random moves in the model if, for instance, the other I playersuse correlated strategies or if the coordination mechanism involves some typeof random selection of an agreement. Let Ai be player i 's set of actionsin the stage game, A � �j AI Aj, and Aÿi � �j AIni Aj. Player i 's stage-game

payo¨ is given by the function ui : A! R. The actual actions taken by theplayers in period t will be referred to as at A A. The history in the stage gamesup through period t will be written as ht

a � �a0; a1; . . . ; at�. Throughout thispaper, we scale the expected present discounted value of a stream of stage-game payo¨s by multiplying it by �1ÿ d�, where d is the discount factor be-

3 The coordination process serves a role similar to that of the correlation device in standard cor-related equilibria. We thank an anonymous referee for pointing this out.

Cumbersome coordination in repeated games 103

tween periods.4 We will always limit our attention to discount factors that arestrictly positive and less than one. A player's payo¨ derived from an in®nitesequence of actions is Ui�a0; a1; . . .� � �1ÿ d�Py

s�0 dsui�as�.We shall only consider stage games with a unique Nash equilibrium; aN

denotes the equilibrium ``strategy'' pro®le.5 This assumption permits us toavoid possible coordination problems within a stage game that might arisefrom a multiplicity of equilibria in that game, and focus on the coordinationproblems created by the in®nite repetition of the stage game.

After players observe the outcome of any stage game, they select a mes-sage6 in the coordination process. The coordination process has two potentialstates. If the coordination mechanism is actively being employed to reach anew agreement, players may have messages available to them that in¯uencethe eventual agreement. If the mechanism is not currently selecting a newagreement, the players' only available messages are to invoke the mechanismor not. The outcome of the mechanism is a suggested agreement, a A A�A� � � � � Ay. The utility player i derives if this agreement is followed by allplayers is Ui�a�.

The messages available to a player at any point in time depend on thedetails of the coordination process as well as the history of previous messagesin the coordination process, but not upon the actions taken in the stage game.A player may choose diërent messages depending on the observed play inthe stage games, but the messages available to the players are unaëcted bythe stage game actions. Let mt

i be player i 's message in period t, mt be thevector of all players' messages in that period, and ht

m � �m0;m1; . . . ;mt� bethe history of messages up through period t. The set of admissible messagesfor player i following a history ht

m shall be written as Mi�htm�. If the coordi-

nation mechanism is not active in period t, the messages available to allplayers are simply that they can start the coordination mechanism or not, soMi�htÿ1

m � becomes finitiate; passg. If any player selects `ìnitiate,'' the coordi-nation mechanism is invoked.

The coordination mechanism may take many diërent forms. For instance,when coordination is achieved with the help of a mediator, the extensive formof the coordination mechanism is especially simple as it does not require anymoves by the ``players'' themselves. We could have M0�ht

m� represent all thepossible agreements the mediator may suggest. If coordination occurs viabargaining, one might imagine a process that suggests the Nash bargainingsolution. Alternatively, the extensive form may consist of a sequence of oërsand counter-oërs in the spirit of the well-known Rubinstein alternating-oërgame. We provide a full speci®cation of a particular coordination processbased upon this bargaining model in Section 5.

The focus of our analysis will be on the properties of the outcomes of thesecoordination processes, independent of the speci®c mechanism used to reachthese outcomes. No matter what form this mechanism takes, the result ofevery coordination process is a suggestion of how to play the game. We

4 This normalization is employed so that repeated game payo¨s are per period averages.5 The results of the paper are trivially extended to the case where this equilibrium involves mixedstrategies. We restrict our attention to pure strategies in the interest of notational convenience.6 These messages are simply actions available to the players in particular subgames, we refer tothem as messages to distinguish them from actions in the stage games.


investigate how properties of these coordination processes may in¯uence thetypes of agreements that may be followed by the players.

The key characteristic of the mechanisms we want to examine is that thetime frame in which coordination takes place may not correspond to the timeframe established by the succession of stage games. The length of a period isquite naturally described by the amount of time it takes a player to change hisaction, that is, the time between stage games. However, the time required toachieve coordination upon a path of play may be longer or shorter than theamount of time it takes an individual player to unilaterally decide on a newaction in the repeated game. For instance, if coordination takes the form ofbargaining, one might imagine that several stage games pass between oërsand counter-oërs, or alternatively that many oërs and counter-oërs canoccur in the time it takes to play a single stage game.

We denote the length of time it takes from the invocation of the coordi-nation mechanism to its resolution by t A �1;�y�, with t � 1 representingthe case in which coordination takes exactly as long as one play of the stagegame. We refer to the t periods during which the coordination mechanism isproducing a new agreement as a coordination phase. We focus on slow or``cumbersome'' coordination, meaning that the amount of time that passesbetween the start of the coordination mechanism and the revelation of itssuggested agreement is at least one period.7 Since t primarily serves to indi-cate the number of stage games that transpire during a coordination phase, weonly consider integer values of t.

We make the following structural assumptions about the coordinationprocess:

(A1) any player can unilaterally initiate the coordination mechanism;(A2) once the mechanism has been initiated, it cannot be restarted until after

it produces a suggested agreement;(A3) the coordination mechanism is independent of time and history (For-

mally, consider any two histories of messages, hm and hm, in which thecoordination mechanism is invoked in periods t and s respectively, andall messages following the calls for coordination are identical, ie.�mt�1; . . . ;mt�r� � �ms�1; . . . ; ms�r�. Then Mi�ht�r

m � �Mi�hs�rm � for any

rU t and every player i, regardless of the other potential diërencesbetween these histories.);

(A4) t is ®nite and common knowledge;(A5) the set of agreements that may be suggested by the mechanism is

common knowledge;(A6) players expect all other players to follow the agreements reached

through the coordination process, and to initiate coordination followingany deviation form an agreement.

Assumptions (A1) and (A2) are straightforward. Assumption (A3) statesthat each time the coordination mechanism is invoked, it takes on the sameextensive form regardless of any actions or messages that may have precededit. This assumption, for instance, would be satis®ed by the Nash bargainingsolution. (A4) states that the mechanism suggests an agreement in ®nite time.

7 It is straightforward to show that if no stage games occur between the time the mechanism isinitiated and the new agreement is reached, the repetition of the one-shot Nash equilibrium is theonly ``stable'' equilibrium of the game.


We make this assumption to avoid consideration of ineëctive mechanismsthat never reach an agreement. (A5) allows players to have diërent conjec-tures about which agreements the mechanism is likely to suggest, but requiresthat there is no dispute about which agreements are possible. We view thecoordination process as a way to build common expectations about futureplay of the game among all players, assumption (A6) rules out the uninter-esting case in which the suggested agreements are simply ignored. It alsorequires that if players observe actions inconsistent with the agreement, theythen expect the coordination mechanism to be invoked in order to reach a newagreement. These assumptions give the coordination process a role in formingcommon expectations of how the game is to be played, but does not placelimitations on the actual choices of the players.

3. Stable sets of agreements

Having introduced the extensive form game in the previous section, it wouldbe natural to solve for the subgame perfect equilibrium pro®les of the game.Roughly speaking, a strategy pro®le is subgame perfect if, given a commonunderstanding of the strategy pro®le to be played, no player ever has an in-centive to deviate from it. In our model this common understanding is formedby the coordination mechanism as the game progresses, so the application ofsubgame perfection is problematic. Furthermore, without specifying the de-tails of the coordination mechanism, we do not precisely know the set ofagreements that it may suggest. The stability concepts we develop are anattempt to apply criteria much in the spirit of subgame perfection to sets ofagreements that might be suggested by a coordination process satisfying theassumptions set out in the previous section. The connection between our sta-bility concepts and subgame perfection is that all agreements that satisfy evenour weakest stability concept will be supportable by a subgame perfect equi-librium strategy pro®le.

The ®rst place where we will employ the spirit of subgame perfection is indetermining what will happen following a deviation from an agreement. If adeviation from an agreement is observed, a player expects all others players toinitiate coordination according to assumption (A6). Since only one call is re-quired to start the mechanism, a player who expects all others to call has noincentive not to call. If coordination is cumbersome, i.e., tV 1, the intervalfollowing a call for coordination but preceding a new agreement represents at-period ®nitely repeated game within the in®nitely repeated game. Assump-tion (A3) guarantees that actions taken in these t periods have no impact onthe forthcoming suggested agreement. Since the stage game has a unique Nashequilibrium, players have no incentive in the last period of the coordinationphase to play anything but the one-shot Nash. The standard backward in-duction argument implies that they will play the one-shot Nash in all periodsof the coordination phase.8

8 Allowing for multiple equilibria in the stage game would create numerous subgame perfectequilibria in this ®nitely repeated game. For a speci®c stage game, this set of equilibria could becalculated, and agreements could specify coordination phase behavior that is consistent with thisset. For an arbitrary stage game, however, the set of subgame perfect equilibria cannot be deter-mined. While it would be feasible to extend the analysis in this way, we feel that this extensionwould distract from the focus of this paper.


Given that a deviation will result in a coordination phase, we can denotea player's expected utility from deviating from an agreement a in the tth

period of the agreement, if he expects aN to be played until the next agreementis reached and a to be the outcome of the next coordination phase, as

Di�a; a; t� � �1ÿ d� maxai AAi

ui�~ai; atÿi� � �dÿ dt�1�ui�aN� � dt�1Ui�a�: �1�

We will often want to compare this to the payo¨ player i would expect fromcontinuing to follow a starting in the tth period under the agreement, we de-note this continuation payo¨ as Ui�a; t� � Ui�at; at�1; . . .�. As we have speci-®ed the coordination mechanism, the players may not know what the outcomeof the next coordination phase will be. However they do know the set ofagreements that the mechanism may suggest. For this reason, the stabilitycriteria we develop are de®ned over sets of agreements.

We distinguish between two concepts of stability: ``weak stability'' and``strong stability''. A set of agreements is weakly stable, if and only if noplayer wants to deviate from any agreement in the set to reach any otheragreement in the set. In other words, agreements must pass an internal con-sistency check. A formal de®nition follows:

De®nition 3.1 (Weak Stability). A set of agreements SJAy is said to beweakly stable i¨

Ui�a; t�VDi�a; a; t� Ea; a A S; tV 0; and i A I: �2�So if the set of agreements that could be suggested by a particular mecha-

nism is weakly stable, no player would ever have an incentive to deviate froman agreement suggested by this mechanism, no matter what expectation theplayers may have about the outcomes of future coordination phases. We willat times want to determine the stability properties of a particular agreement.We refer to an agreement as being weakly (strongly) stable if it belongs tosome weakly (strongly) stable set.

In subgame perfection, it is important that players have no incentive todeviate from the equilibrium path given their common expectations of whatwill happen out of equilibrium. Here, we do not require players to have iden-tical conjectures about what may occur in future coordination phases. Weimpose a stronger condition that no matter what agreements players mayindividually anticipate following from future coordination phases, they haveno incentive to deviate from an agreement. Thus the connection betweenweakly stable agreements and subgame perfect equilibria is that every weaklystable agreement can be supported as the equilibrium path of play in a sub-game perfect equilibrium.

If an agreement is weakly stable, then necessarily no player must want todeviate from that agreement only to return to the same agreement after thecoordination phase. This property is useful in proving existence of a weaklystable set.

Theorem 3.1. A weakly stable set exists.

Proof: Consider a set of agreements that includes only an agreement thatspeci®es repeated play of the one-shot Nash equilibrium. This set of agree-


ments is weakly stable. Since players expect to play the one-shot Nash for allperiods of the coordination phase, and also expect to reach this same agree-ment thereafter, the only potential gain could be in the period of deviation.But since they are playing a one-shot Nash in the ®rst place, there is no prof-itable deviation. r

Weak stability has the drawback that as the discount factor approachesone and t approaches in®nity, every feasible payo¨ vector in which everyplayer receives a payo¨ higher than in the one-shot Nash equilibrium can beobtained from some weakly stable agreement. Friedman's (1971) proof ap-plies trivially to our version of the Folk Theorem:

Theorem 3.2. For any feasible vector u of per-period payo¨s with ui > ui�aN�for all players i, there exists a d and a t such that for all d > d and t > t there isa weakly stable agreement with payo¨s u.

A strongly stable set of agreements is a weakly stable set that satis®es theadditional condition that for all agreements outside the set, there is a point intime at which at least one player would want to deviate from the agreement.Strong stability hence also requires an external consistency check. Internalconsistency only addresses whether or not the agreements will be followed.External consistency requires that all of the agreements from which no onewould deviate are included in this set; hence if an agreement is not in the set, itwill not be followed.

De®nition 3.2 (Strong Stability). A set of agreements SJAy is said to bestrongly stable for a given d A �0; 1� and tV 1 i¨

i. S is weakly stable; andii. Ea B S; bi A I; a A S; tV 0; such that Di�a; a; t� > Ui�a; t�.

Solution concepts that are based on both internal and external criteria suchas this date back to von Neumann and Morgenstern stability for coalitionalgames [see von Neumann and Morgenstern (1944)].9 Our concept diërs inthat it is inherently non-cooperative and is de®ned completely in terms of therepeated game with cumbersome coordination. It has been shown that avN&M stable set may not exist in certain coalitional games [see Lucas (1969)].We encounter the same problem with our concept of strong stability. Exis-tence of an equilibrium satisfying internal and external consistency is alsotypically not guaranteed in the more recent literature on renegotiation proof-ness [see, for instance, Farrell and Maskin (1989) and Bernheim and Ray(1989)]. However, if a strongly stable set exists for every discount factor, weare able to prove the following result:

Theorem 3.3. For any tV 1, in the limit as d approaches one, each stronglystable set converges to a Pareto e½cient singleton. However, there is no guar-antee that all strongly stable sets will converge to the same singleton.

Proof: Suppose there are two agreements in a strongly stable set that yielddiërent payo¨ vectors. Without loss of generality, assume that player one

9 The authors would like to thank an anonymous referee for pointing out this similarity.


receives a strictly larger payo¨ in the second agreement than in the ®rst. Thenthere must exist a critical value of the discount factor, d < 1 such that for alld > d, player one will deviate from the ®rst to reach the second agreement ± aviolation of the internal consistency check for strong stability. Next, supposethere exists a feasible payo¨ vector that Pareto dominates the payo¨s from allagreements in the set. Then there must exist a critical value of the discountfactor, ~d < 1 such that for all d > ~d, no player wants to deviate from thePareto superior agreement to one inside the strongly stable set ± a violation ofthe external consistency condition. r

In general, a game may have more than one strongly stable set. However,in the examples discussed in the next section, the strongly stable set is shownto be unique.

4. Stable sets of agreements in the Prisoner's Dilemma

One important class of games, in which existence of strongly stable equilibriadoes not pose a problem and in which it is easy to illustrate the theoreticalresults just presented, is the Prisoner's Dilemma. We ®rst calculate weakly andstrongly stable sets of agreements for symmetric Prisoner's Dilemmas, andthen turn to the eëcts of asymmetries in the players' stage-game payo¨s.

4.1. Symmetric Prisoner's Dilemma

Consider the Prisoner's Dilemma game shown in Table 1. The unique Nashequilibrium of this (stage) game is �D;D�. In order to have a convex payo¨space as shown in Figure 1, we employ correlated strategies within each stagegame. This is a standard technique used in the repeated game literature, seeSorin (1986) and Fudenberg and Maskin (1990). Let o t be a publicly observ-able random variable that is drawn from a uniform distribution on the unitinterval before each stage game, and let xk, yk, and zk be non-negative num-bers satisfying xk, yk, zk U 1. A correlated strategy, mk�o t�, for the stage gamecan then be characterized as follows:

mk�o t� ��C;D� if o t < xk,

�D;C� if xk Uo t < xk � yk,

�D;D� if xk � yk Uo t < xk � yk � zk,

�C;C� if xk � yk � zk Uo t.

8>>><>>>: �3�

Table 1. The Prisoner's Dilemma

Player 2C D

Player 1 C 2; 2 0; 3

D 3; 0 1; 1


For example, to play �C;C� with probability two thirds and �D;C� withprobability one third would require that xk � 0, yk � 1=3, and zk � 0.

Following our earlier de®nition, an agreement speci®es a path of play.By using correlated strategies, we avoid the need to use strategies involvingcomplex sequences of stage-game play in order to guarantee players a certainexpected payo¨ from an agreement. We can simply specify an agreement asa correlated strategy that is to be played every period. To avoid introducingfurther notation, we refer to an agreement by its correlated strategy. To de-termine whether a player would follow such an agreement, we need to con-sider the consequences of deviating from the equilibrium path. As shown inTheorem 3.1, the agreement to always play the one-shot Nash equilibrium isweakly stable for any d and t. In any agreement in which at least one playeris ever asked to play C, a player would most like to deviate when that playeris supposed to play C. The deviation would result in a one-shot gain of oneregardless of the other player's action. Consider a period t in which o t V xk �yk � zk so that players are supposed to play �C;C�. Player i 's utility fromfollowing mk is

Ui�follow mk in period t� � �1ÿ d�2� dUi�mk�: �4�

Similarly, player i 's utility from deviating in period t is

Ui�deviate from mk in period t� � �1ÿd�3��dÿdt�1�1�dt�1Ui�mm�; �5�

where mm is the strategy to be played in an agreement reached in the subse-quent coordination process.

To characterize the union of weakly stable sets one can start by consider-ing sets that consist of a single agreement. If no player wishes to deviate fromthat agreement under the expectation that the next coordination phase willresult in the same agreement, this one agreement constitutes a weakly stableset. If any player wishes to deviate from the agreement in order to eventuallyget back to it, then it cannot belong to any weakly stable set. In order to seeif an agreement to always play mk is in the union of weakly stable sets, it isnecessary and su½cient to check the following condition for each player: set-

Fig. 1. Payo¨s in the Prisoner's Dilemma


ting mm � mk in (5), player i prefers to follow mk if

Ui�mk�V1ÿ dt�1

dÿ dt�1: �6�

As indicated by Theorem 3.2, we can obtain a Folk Theorem-like result, andwe can now see why just having d converge to one is not enough to obtain thisresult: as d converges to one, condition (6) becomes Ui�mk�V �t� 1�=t. As ttends toward in®nity, however, player i would follow an agreement to play mk

if Ui�mk�V 1, which reproduces our Folk Theorem result.We now turn our attention to strong stability. Let m1 be the correlated

strategy to be played in the agreement that yields player one his highest payoïn a strongly stable set of agreements. If player one does not wish to deviatefrom an agreement under the belief that m1 will be the outcome of the nextcoordination phase, he will not deviate to reach any other agreement in thatstrongly stable set. Replacing mm with m1 in (5) yields the minimal payo¨player one would be willing to accept from an agreement. Making this re-placement and using (4) and (5), we observe that following mk is optimal forplayer one if

U1�mk�V1ÿ dt�1

d� dtU1�m1�: �7�

The locus of payo¨ pairs from agreements for which (7) holds with equalityconstitutes an indiërence curve for player one. When mapped into payo¨space, this locus appears as a vertical line. An equivalent indiërence condi-tion can be derived for player two, where m1 would be replaced with m2, playertwo's best agreement in this strongly stable set. In payo¨ space, player two'sindiërence curves are horizontal lines. A representative pair of indiërencecurves is drawn in Figure 1 crossing point �D;D�. Player one would prefer todeviate from agreements yielding payo¨s to the left of the vertical line, andfollow agreements to the right of it. Player two would follow agreements withpayo¨s above the horizontal line, but deviate from those below.

Because player one's indiërence curve is a vertical line, the agreementmost favorable to player two, from which player one would not wish to devi-ate, is the one yielding the highest feasible payo¨ for player two on theindiërence curve determined by m1. Thus (7) must hold with equality whenmk � m2:

U1�m2� �1ÿ dt�1

d� dtU1�m1�: �8�

A parallel argument holds for player two, leading to the following equation:

U2�m1� �1ÿ dt�1

d� dtU2�m2�: �9�

A strongly stable set for any given values of d and t can now be calculated bydetermining where each player's indiërence curve crosses the e½cient frontierwhen m1 and m2 are jointly determined by (8) and (9).


Two cases need to be addressed. First, both players' indiërence curvescould cross the e½cient frontier on the line segment between �C;C� and�D;C�.10 Second, player one's indiërence curve could cross between �C;C�and �C;D�, and player two's between �C;C� and �D;C�.11 The ®rst case canbe ruled out easily. To show this, assume by way of contradiction that bothplayers' indiërence curves cross the frontier on the segment between �C;C�and �D;C� in Figure 1. The correlated strategies that yield this situation are:m1 speci®es x1 � z1 � 0 [so that �C;D� and �D;D� are never played] andy1 > 0 [so that �D;C� is played with positive probability], and m2 speci®esx2 � z2 � 0 and y2 > 0. Making the appropriate substitutions in (8) and (9),we obtain:

2� y2 �1ÿ dt�1

d� dt�2� y1�; and

2ÿ 2y1 �1ÿ dt�1

d� dt�2ÿ 2y2�:

These two equations jointly imply that y2 � 2y1 � dt�y1 � 2y2�, which re-quires that y2 > y1. But m1 is de®ned as the strongly stable agreement leadingto player one's highest payo¨, so U1�m1�VU1�m2� or 2� y1 V 2� y2, acontradiction.

The second case to consider is where m1 satis®es x1 � z1 � 0 and y1 V 0,and under m2; y2 � z2 � 0 and x2 V 0. Given these agreements, (8) and (9)become:

2ÿ 2x2 � 1ÿ dt�1

d� dt�2� y1�; and �10�

2ÿ 2y1 �1ÿ dt�1

d� dt�2� x2�: �11�

These two conditions jointly require that �2ÿ dt��y1 ÿ x2� � 0, or y1 � x2.Substituting this into (10) yields:

y1 � x2 � ÿ1� 2dÿ dt�1

d�2� dt� :

Since equations (10) and (11) yield a unique solution and all other possiblespeci®cations of m1 and m2 have been eliminated, this must identify the uniquestrongly stable set. For some values of d and t, x2 and y1 are not valid prob-abilities, but for any given tV 1, there exists a d such that for all dV d theseare valid correlated strategies. However, for low values of d the only stronglystable agreement is the one-shot Nash, because any potential one-shot devia-tion gain weighs very heavily in the players' expected utilities.

Figure 2 plots player one's expected utility from m1 and m2 for t � 10 over

10 Due to the symmetry of the game, this is equivalent to both indiërence curves crossing be-tween �C;C� and �C;D�.11 The opposite of this case is not considered because it would not satisfy U1�m1�VU1�m2�.


the full range of d. A graph of player two's payo¨s would be identical.Whenever player one's utility is the same under m1 and m2, it indicates thatevery agreement in the strongly stable set yields the same unique payo¨. Theone-shot Nash payo¨s that are obtained for small discount factors are oneexample of this. Also notice that the payo¨s associated with any stronglystable agreement converge to a unique payo¨ vector as d tends toward one inaccordance with Theorem 3.3. In this symmetric game, it can be shown thatthe limiting payo¨ vector is (2, 2) for any t, because y1 and x2 tend towardzero for large discount factors.

4.2. Asymmetric Prisoner's Dilemma

We now turn to an investigation of the role symmetry plays in the previousresults. We change the game so that one player receives a higher payo¨ thanthe other in the Nash equilibrium, but hold the potential gains from deviation®xed at one. This game is shown in Table 2, where g > 0 is the payo¨ diër-ential at the equilibrium.12

Under these new payo¨s, player one's indiërence condition previouslygiven by equation (8), is now given by

U1�m2� �1ÿ dt�1

d� �1ÿ dt�g� dtU1�m1�:

Player two's indiërence condition remains unchanged from equation (9).

Fig. 2. Symmetric PD: Player one's payo¨s at m1 and m2 for t � 10

12 We also examined a second type of asymmetry, namely changing one player's payo¨ fromdeviation while holding the Nash equilibrium payo¨s ®xed. We do not report the results here be-cause they are similar to the ones we obtain when we shift one player's Nash equilibrium payo¨.The similarity in results is due to the fact that both types of asymmetries increase the opportunitycost to one of the players of staying in an agreement.


As in the previous section, the players' decisions to follow or deviate froman agreement can be represented using vertical and horizontal linear indiër-ence curves. Once again, the strongly stable set of agreements is determined bywhere these indiërence curves intersect the e½cient frontier. It is easily shownthat they either cross on opposite sides of �C;C� on the e½cient frontier as inthe symmetric game, or both cross the frontier on the segment of the frontierbetween �C;C� and �D;C�. Which of these two cases holds depends on theparameters d; t, and g. We omit the details of this calculation due to the simi-larity with the symmetric game analysis, and focus upon the results.

Figure 3 plots player one's payo¨s in m1 and m2 for t � 10 and g � 0:90. Asin the symmetric game, for low discount factors, the only stable agreementsare those yielding the one-shot Nash payo¨s, where player one now receives1.90. An interesting distinction between this and Figure 2 is that for high dis-count factors, player one receives a payo¨ of greater than two even at m2.Since it is not possible for both players to achieve expected payo¨s of greaterthan two, it is clear that this ``shifting of the Nash payo¨s'' worked in playerone's favor. The intuition for this is that the coordination process is now lesscostly to player one, so she must receive greater compensation (relative towhen g � 0) in order to stay in an agreement.

It can be further noted that, as the discount factor tends to one, payo¨sassociated with the strongly agreements do not tend toward (2, 2) as they did

Table 2. The Prisoner's Dilemma: Shifted Nash

Player 2C D

Player 1 C 2; 2 g, 3

D 3; 0 1� g, 1

Fig. 3. Shifted Nash PD: Player one's payo¨s at m1 and m2 for t � 10, g � 0:9


in the symmetric game. Instead, as the strongly stable set shrinks, the agree-ments that survive include correlated strategies in which both �C;C� and�D;C� are played with positive probability. In the limit, payo¨s therefore tendto favor player one. To what extent the limiting payo¨ vector favors playerone depends on how asymmetric the game is. Figure 4 graphs both player'slimiting payo¨s for t � 10 as g goes from zero to one. Notice that for gamesthat are approximately symmetric, the limiting payo¨ vector is still (2, 2); butas g increases, it starts to favor player one.

5. Coordination through Rubinstein bargaining

The details of the coordination mechanism have gone unspeci®ed up to thispoint. One could model the coordination of players' beliefs in innumerableways. However there is no guarantee that a given mechanism would lead toeven a weakly stable set of agreements. We have shown in Theorem 3.1 thatthere is always a trivial mechanism that would suggest a weakly stable agree-ment. The question we raise in this section is whether there is a non-trivialcoordination mechanism that suggests only weakly stable agreements in thisrepeated Prisoner's Dilemma game. We con®rm that there is by constructinga version of the well-known Rubinstein alternating-oër bargaining model[see Rubinstein (1982)]. This mechanism, in fact, yields only agreements thatbelong to the strongly stable set calculated previously. Furthermore, theagreements suggested by this bargaining mechanism actually coincide with theboundaries of the strongly stable set.

We will now fully specify the coordination process. If the players are notcurrently bargaining over a new agreement, the set of messages available tothem in the coordination process is finitiate; passg. If following the stagegame in some period s, either player initiates coordination, a fair coin is¯ipped to determine which player gets to make the ®rst oër in the next

Fig. 4. Shifted Nash PD: Payo¨s under limiting stable agreement


period.13 At the start of period s� 1, players play the stage game, then the®rst mover oërs an agreement. As in the previous section, we consider agree-ments that consist of repeated play of a correlated strategy, so the ®rstmover could suggest any correlated strategy. Speci®cally, if player i is the ®rstmover, his set of admissible actions given the relevant information aboutthe history of messages, ie. that coordination was initiated in the previousperiod and player i was selected to make the ®rst oër, is Mi�hs�1

m � � fmk ��xk; yk; zk� : xk; yk; zk V 0g. The other player has no available messages inperiod s� 1. The coordination mechanism is taken to be cumbersome in thefollowing way: tÿ 1 stage games pass before the second mover can respondto this oër. During these tÿ 1 periods, the players continue to play thestage game, but neither player has any messages available in the coordinationprocess. After the stage game in period s� t, the second mover now has theoption of either accepting or rejecting the oër. If the second mover accepts,then both players expect to start playing the agreement immediately, ie. inperiod s� t� 1. If the second mover instead rejects the oër in period s� t,this player makes a counter-oër following the stage game play in periods� t� 1. Then tÿ 1 more stage games pass before the original ®rst movercan respond to the counter-oër. Players continue making oërs in this fash-ion until one is accepted.

This example provides an opportunity to revisit some of the assumptionswe made about the coordination process in Section 2. According to assump-tion (A3), the coordination mechanism is independent of time and history.In this context, this is satis®ed because none of the details of the bargainingmechanism (selection of ®rst mover, sets of admissible agreements, etc.) de-pends on any previous stage game actions or coordination process messages,nor do they depend on the period in which bargaining starts. Another impor-tant assumption (A4) was that the coordination mechanism would suggest anagreement in t periods. Given that players play Nash in the t stage games ofthe coordination phase and that players expect other players to follow thesuggested agreement from this mechanism, Rubinstein's (1982) result applieshere. Speci®cally, for a given ®rst mover, this bargaining mechanism has aunique subgame perfect equilibrium. In this equilibrium, an oër is accepted tperiods after coordination has been initiated. Assumption (A2) speci®ed thatattempts to initiate a coordination mechanism during a coordination phaseare ignored. This has the natural interpretation here that players cannotmanipulate the alternation order by restarting the mechanism.

We now turn to the calculation of the optimal oërs in a symmetric Pris-oner's Dilemma using this bargaining mechanism. Without loss of generality,suppose player one is the ®rst mover in the current coordination phase. Whenmaking an oër, m1, player one tries to maximize her own utility subject to theconditions that player two will accept and follow m1. Whether these conditionshold depends on what player two thinks will happen if he rejects the oër, or ifhe accepts it but deviates.

If player two is considering rejecting the oër and responding with acounter-oër m2 in the following period, he would expect to play t periods of

13 Our results are not dependent on this particular method of determining the ®rst mover. It isonly necessary that each player believes that either player could be the ®rst mover in the next co-ordination phase with positive probability. The players' probability assessments need not beidentical.


the stage game before player one could respond to this counter-oër. Assum-ing m2 is selected so that player one will accept and follow it, player two'sutility from rejecting m1 is

U2�reject m1� � �1ÿ dt�1� dtU2�m2�: �12�

Player two would hence accept m1 if

U2�m1�V �1ÿ dt�1� dtU2�m2�: �13�

But in selecting m1, player one must also be concerned with whether playertwo will actually follow the agreement after it has been accepted, since thecoordination process is not binding.

Whether player two would follow an agreement m1 was considered inequation (9), where it was shown that player two would follow m1 if

U2�m1�V1ÿ dt�1

d� dtU2�m2�: �14�

Notice that this assumes that player two will be the ®rst mover in the nextcoordination phase. We consider this case because the de®nition of strongstability requires that player two would follow m1 for any possible outcome offuture coordination phases, so we take the outcome most favorable to playertwo. It is easy to see that any m1 satisfying this condition would also satisfycondition (13).

So player one would want to oër the m1 that maximizes U1�m1� whilesatisfying (14). But this is exactly the same condition used to determine m1,player one's best agreement in the strongly stable set in the previous section.The alternating-oër bargaining procedure can yield one of two outcomes,depending on which player is the ®rst mover. These two outcomes are pre-cisely the two player's ``best'' agreements in the strongly stable sets we calcu-lated in the previous section. The Rubinstein mechanism hence not onlyproduces strongly stable agreements, it actually yields the boundaries of thestrongly stable set for any coordination mechanism satisfying our earlier as-sumptions!

Incidentally, our ®nding that the strongly stable set collapses to a symme-tric payo¨ vector as players become more patient has a parallel in the basicRubinstein model. There, the equilibrium payo¨s received by the ®rst andsecond movers converge as the players become more patient.

6. Conclusions

This paper introduced a coordination process into an in®nitely repeated gameas a means to allow players to form common expectations about future playof the game. Speci®cally, we considered coordination processes that are time-consuming or ``cumbersome,'' meaning that it takes players more time toreach these common expectations than it does for them to complete a round ofthe stage game. We de®ned and characterized two stability criteria for thisclass of games. Weakly stable sets of agreements have the property that noplayer has an incentive to deviate from any agreement in the set if that player


anticipates this deviation to result in the coordination process being employedto reach a new agreement that is also in the set. Strong stability adds thecondition that for any agreement outside the strongly stable set there must beat least one player who would like to deviate if by doing so he expects to reachan agreement inside the set.

As long as the coordination process is cumbersome, payo¨s associatedwith agreements in strongly stable sets converge to Pareto e½cient singletonsas the discount factor approaches one. We also applied the concept of strongstability to Prisoner's Dilemmas. In the symmetric Prisoner's Dilemma coop-eration by both players is the only strongly stable agreement when playersbecome very patient. In the asymmetric Prisoner's Dilemma, the set ofstrongly stable agreements is skewed toward the player who is favored by theasymmetry. If this asymmetry is su½ciently large, this bias will even be pres-ent in the limiting strongly stable agreement. The economic intuition for thisresult is that, in order to be strongly stable, agreements must compensateplayers for their (asymmetric) opportunity costs.

References

Abreu D, Pearce D, Stachetti E (1993) Renegotiation and symmetry in repeated games. Journal ofEconomic Theory 60:217±240

Bernheim BD, Ray D (1989) Collective dynamic consistency in repeated games. Games andEconomic Behavior 1:295±326

Blume A (1994) Intraplay communication in repeated games. Games and Economic Behavior6:181±211

Farrell J, Maskin E (1989) Renegotiation in repeated games. Games and Economic Behavior1:327±360

Friedman J (1971) A noncooperative equilibrium for supergames. Review of Economic Studies38:1±12

Fudenberg D, Maskin E (1990) On the dispensability of public randomizations in discountedrepeated games. Journal of Economic Theory 53:428±438

Lucas WF (1969) The proof that a game may not have a solution. Transactions of the AmericanMathematical Society 137:219±229

McCutcheon B (1997) Do meetings in smoke-®lled rooms facilitate collusion?. Journal of PoliticalEconomy 105:330±350

Rubinstein A (1982) Perfect equilibrium in a bargaining model. Econometrica 53:97±109Scherer FM (1980) Industrial market structure and economic performance. Houghton Mi¿in,

BostonSorin S (1986) On repeated games with complete information. Mathematics of Operations

Research 11:147±160von Neumann J, Morgenstern O (1944) Theory of games and economic behavior. John Wiley and

Sons, New York


Documents

Cumbersome coordination in repeated games