February 2, 2016 9:40 WSPC/S0218-1274 1650006

International Journal of Bifurcation and Chaos, Vol. 26, No. 1 (2016) 1650006 (13 pages)c© World Scientific Publishing CompanyDOI: 10.1142/S0218127416500061

Hopf Bifurcations in Two-Strategy DelayedReplicator Dynamics

Elizabeth WessonCenter for Applied Mathematics,

Cornell University, Ithaca, NY 14853, USAenw27@cornell.edu

Richard RandDepartment of Mathematics,

Department of Mechanical and Aerospace Engineering,Cornell University, Ithaca, NY 14853, USA

rhr2@cornell.edu

David RandDepartment of Psychology, Department of Economics,

School of Management, Yale University,New Haven, CT 06520, USA

david.rand@yale.edu

Received February 13, 2015; Revised July 19, 2015

We investigate the dynamics of two-strategy replicator equations in which the fitness of eachstrategy is a function of the population frequencies delayed by a time interval T . We analyzetwo models: in the first, all terms in the fitness are delayed, while in the second, only opposite-strategy terms are delayed. We compare the two models via a linear homotopy. Taking the delayT as a bifurcation parameter, we demonstrate the existence of (nondegenerate) Hopf bifurcationsin both models, and present an analysis of the resulting limit cycles using Lindstedt’s method.

Keywords : Replicator; delay; Hopf bifurcation; limit cycle; Lindstedt.

1. Introduction

Evolutionary game theory models the evolution ofcompeting strategies within a population by com-bining the classical economic tools of game theorywith differential equations [Hofbauer & Sigmund,1998]. The most common approach focuses on therelative frequencies of different strategies in a pop-ulation using the replicator equation,

xi = xi(fi − φ), i = 1, . . . , n (1)

where xi is the frequency of (fraction of the popu-lation using) strategy i, fi(x1, . . . , xn) is the fitness

of strategy i, and φ =∑

fixi is the average fitnessacross the population.

Hofbauer and Sigmund [1998] have shown thatthe replicator equation can be derived from theLotka–Volterra equation, the classic predator–preymodel of species abundances (rather than frequen-cies). The n-strategy replicator equation is equiva-lent to the Lotka–Volterra system with n−1 species.The derivation, however, requires a rescaling oftime, and the correspondence between species andstrategies is not one-to-one.

Furthermore, it has been shown [Taylor &Jonker, 1978; Hofbauer et al., 1979; Zeeman, 1981;

1650006-1

February 2, 2016 9:40 WSPC/S0218-1274 1650006

E. Wesson et al.

Weibull, 1995] that the replicator equation can bederived from the exponential growth model

ξi = ξigi, i = 1, . . . , n (2)

where ξi is the abundance of strategy i, andgi(ξ1, . . . , ξn) the fitness of strategy i. The equiv-alence simply uses the change of variables xi = ξi/pwhere p is the total population, with the assump-tion that the fitness functions depend only on thefrequencies, and not on the populations directly.

The game-theoretic component of the replica-tor equation lies in the choice of fitness functions.Take the payoff matrix A = (aij), where aij is theexpected payoff for strategy i when it interacts withstrategy j. Then the expected total fitness fi isthe expected payoff of strategy i when interactingwith each strategy, weighted by the other strategies’frequencies:

fi = (A · x)i, (3)

where

x = (x1, . . . , xn),∑

i

xi = 1. (4)

In the standard replicator equation, therefore,interactions are assumed to occur instantaneously:the fitness of a given strategy depends on the fre-quency of each strategy at the given moment. Inmany real-world contexts, however, the fitness con-sequences of interactions are not immediate, butinstead experience some amount of delay. In thiswork, we explore the consequences of such delay.Specifically, we consider the two-strategy replicatorsystem, generalized to models in which the fitnessesare functions of the strategy frequencies delayed bya time interval T .

For ease of notation, write

A =(

a b

c d

)(5)

and

x = (x1, x2) = (x, y) (6)

where x + y = 1.Two-strategy games with delay have previously

been investigated by Yi and Zuwang [1997]. Thisprevious work considered cases that have an equi-librium in which the two strategies coexist, andobtained conditions for the instability of this inte-rior equilibrium when all interactions are subject todelay. It did not, however, analyze the bifurcationsand resulting limit cycles that occur at the change

of stability. A nonlinear perturbation analysis ofHopf bifurcations in three-strategy (Rock-Paper-Scissors) replicator equations with delay appears in[Wesson & Rand, 2014], but it only considers a sin-gle model in which all interactions are delayed.

In the present paper, we extend the previouswork in two ways. First, we go beyond the linearstability analysis, which predicts the existence of aHopf bifurcation, and include the nonlinear terms ofthe replicator equation. This allows us to approx-imate the resulting limit cycles using Lindstedt’smethod. Second, we consider models in which notall interactions are equally subject to delay: inter-actions between agents with different strategiesmay be more likely to be delayed than interac-tions between agents with the same strategy. Specif-ically, we consider a range of models indexed by ahomotopy parameter γ that determines the relativeweights of delayed and nondelayed terms in the fit-ness functions.

At one extreme is the case considered previ-ously [Yi & Zuwang, 1997; Alboszta & Miekisz,2004; Wesson & Rand, 2014], in which all interac-tions are delayed by T time units. We refer to thisas the full delay model: If we write xi ≡ xi(t − T )and define

x ≡ (x, y) (7)

then the total expected payoff — i.e. the fitness —for strategy i is given by

fi = (A · x)i. (8)

That is,

f1 = ax + by, f2 = cx + dy. (9)

Thus each agent’s payoff at the current momentdepends on the frequencies of each strategy T timeunits ago (i.e. the expected payoff of that agent’sstrategy T time units ago). This could representa situation in which human learners preferentiallyimitate those with higher payoffs, but informationabout the payoff of each strategy is delayed by T ;or a situation in which organisms evolving in awell-mixed population consume resources producedby other organisms to determine their fitness, andresources take T time use to diffuse between oneorganism and the next.

At the other end of the spectrum, only inter-actions between agents with different strategies aredelayed, while interactions between agents with thesame strategy are instantaneous. We refer to this

1650006-2

February 2, 2016 9:40 WSPC/S0218-1274 1650006

Hopf Bifurcations in Two-Strategy Delayed Replicator Dynamics

as the off-diagonal delay model. In this case, if wedefine

x1 ≡ (x, y), x2 ≡ (x, y) (10)

then the fitness for strategy i is given by

fi = (A · xi)i. (11)

That is,

f1 = ax + by, f2 = cx + dy. (12)

Note that in this model, the fitness cannot beconsidered an expected payoff, as xi is not a unitvector. This model may be interpreted as the resultof a certain form of assortment. For example, humanlearners have immediate information about the pay-off consequences of interactions with those havingtheir own strategy, but information about the out-come of interactions with people using the otherstrategy is delayed by T . Alternatively, nonhumanorganisms might interact (e.g. affect each other’s fit-nesses) by exchanging resources such as nutrients ortoxins; and the resources produced by organisms ofone’s own type may be immediately available (per-haps because of spatial colocalization of agents ofthe same type) while it takes T time units for theresources produced by the other type of organismto reach one via diffusion.

In general, we can bridge the two extreme mod-els by introducing a linear homotopy (i.e. a convexcombination) between them: we define

xiγ ≡ γxi + (1 − γ)x (13)

and consider the fitness functions

fi = (A · xiγ)i. (14)

That is,

f1 = a(γx + (1 − γ)x) + by,

f2 = cx + d(γy + (1 − γ)y).(15)

When γ = 0, we are in the full-delay case; whenγ = 1, we are in the off-diagonal delay case. Forvalues of γ between 0 and 1, the system may beconsidered as a stochastic combination of the two:interactions with agents using the other strategy arealways delayed by T , but interactions with agentsusing one’s own strategy are instanteous with prob-ability γ and delayed with probability 1 − γ.

For any value of γ, the use of delayed frequen-cies in the fitness functions makes the replicator

equation into the delay differential equation (DDE)

xi = xi(fi − φ) (16)

where fi is given in Eq. (15), and

φ =∑

i

xifi = xf1 + yf2. (17)

As a system of ODEs, the standard replica-tor equation is an (n − 1)-dimensional problem,because

∑xi = 1. This means n − 1 of the xi are

required to specify a point in phase space. In partic-ular, the nondelayed two-strategy replicator systemreduces to a single autonomous ODE. The delayedtwo-strategy replicator equation reduces to a singleautonomous DDE which, by contrast, is an infinite-dimensional system [Erneux, 2009] whose solutionis a flow on the space of functions on the interval[−T, 0).

2. Derivation

We will analyze the replicator equation Eq. (16)without specifying the homotopy parameter γ —that is, the fitness is given by Eq. (15). The fulldelay and off-diagonal delay models correspond tothe special cases γ = 0 and γ = 1, respectively.Then

x = x(f1 − φ) and y = y(f2 − φ) (18)

where

f1 = a(γx + (1 − γ)x) + by and

f2 = cx + d(γy + (1 − γ)y),(19)

which means

φ = xf1 + yf2

= x[a(γx + (1 − γ)x) + by]

+ y[cx + d(γy + (1 − γ)y)]. (20)

Substituting in these values, and writing y = 1−x,the system is reduced to the single delay differentialequation

x = x(1 − x)[b − d + (a − b − c + d)x

+ (a + d)(x − x)γ]. (21)

At this point, we note that the number ofparameters may be reduced by defining p, q, r as

1650006-3

February 2, 2016 9:40 WSPC/S0218-1274 1650006

E. Wesson et al.

follows:

p ≡ a − c (22)

q ≡ d − b (23)

r ≡ a + d = tr A. (24)

Then Eq. (21) becomes

x = x(1 − x)(px + q(x − 1) + γr(x − x)). (25)

3. Analysis

The equilibrium points of Eq. (25) satisfy x = 0 andx = x. There are three equilibria:

x = 0, x = 1, x =q

p + q. (26)

The first two are the endpoints of the interval ofphysical relevance, since we require that x ∈ [0, 1].The third lies in the interval (0, 1) if and only ifp and q have the same sign. We will assume thatthis is the case. Notice that the equilibria do not

depend on the homotopy parameter γ. We examinethe stability of the three points.

Taylor expanding about x = 0 and x = 1,respectively, we obtain the linearized systems

dx

dt= −qx about x = 0 (27)

d

dt(x − 1) = −p(x − 1) about x = 1. (28)

These two linearizations do not depend on x, sothe stability of the endpoints depends only on thepayoff coefficients and not on the delay. The twoendpoints have the same stability, since by assump-tion p and q have the same sign. If p, q > 0, wefind that both endpoints are stable; if p, q < 0, thenboth endpoints are unstable.

Now consider the third equilibrium. To deter-mine its stability, we set

z = x − q

p + q. (29)

In terms of z, Eq. (25) is

z = −(p(z − 1) + qz)(z(p + q) + q)(z(p + q − γr) + γrz)(p + q)2

. (30)

We linearize about z = 0 to get

z =pq(z(p + q − γr) + γrz)

(p + q)2. (31)

First, note that if delay T = 0, the linearizationreduces to

z =pqz

p + q. (32)

Under the assumption that p and q have the samesign, we see that the stability of the point z = 0is opposite to that of the endpoints. If p, q > 0, wefind that the point z = 0 is unstable when T = 0; ifp, q < 0, then it is stable. See Fig. 1.

In general, however, the linearization ofEq. (31) has a nonzero z term, so it is reasonable to

0 5 10 15 20 25t

0.2

0.4

0.6

0.8

1.0x(t)

5 10 15 20 25t

0.2

0.4

0.6

0.8

1.0

x(t)

(a) p = −1, q = −1/2, r = 2 (b) p = 1, q = 1/2, r = 2

Fig. 1. The stability of the endpoints depends on the sign of p and q. For T < Tc, including T = 0, the stability of the interiorequilibrium is opposite to that of the endpoints. Shown: plots of x(t) versus t where T = 1, γ = 1/3. Numerical solutionsgenerated by NDSolve in Mathematica.

1650006-4

February 2, 2016 9:40 WSPC/S0218-1274 1650006

Hopf Bifurcations in Two-Strategy Delayed Replicator Dynamics

expect that the stability will depend on the delay T .Given that, we analyze the system for a Hopf bifur-cation, taking T as the bifurcation parameter.

Set z = eλt (and z = eλ(t−T )) in Eq. (31) toobtain the characteristic equation

λ =pq e−λT (p + q + γr(eλT − 1))

(p + q)2. (33)

At the critical value of delay for a Hopf bifurca-tion, the eigenvalues corresponding to the subspacein which the Hopf occurs are pure imaginary, so wetake T = Tc and λ = iω. Substituting this intothe characteristic equation and taking the real andimaginary parts, we obtain

cos ωTc = − γp2q2r(p + q − γr)(γpqr)2 + ω2(p + q)4

(34)

sin ωTc = −pqω(p + q)2(p + q − γr)(γpqr)2 + ω2(p + q)4

. (35)

Squaring these equations and adding them, we cansolve for the critical frequency ω:

ω = pq

√p + q − 2γr

(p + q)3. (36)

It can be shown that the frequency is real andnonzero if and only if (in addition to p and q hav-ing the same sign) p + q − 2γr has the same signas p + q. That is,

{p, q < 0, p + q < 2γr} or

{p, q > 0, p + q > 2γr}.(37)

Thus Eq. (37) is a necessary condition for a Hopfbifurcation to exist. See Fig. 2. We will assume thatthis condition is satisfied.

Notice that in the full delay case γ = 0, Eq. (37)is trivially satisfied whenever p and q have the samesign. That is, the Hopf occurs for any payoff matrixsuch that the equilibrium point lies in the intervalof relevance. If p and q are both negative, the equi-librium is stable and the limit cycle resulting fromthe Hopf is stable. However, if p and q are bothpositive, the equilibrium is unstable and the limitcycle resulting from the Hopf is unstable. In eithercase, the Hopf occurs and a limit cycle is born withfrequency given by Eq. (36) as:

ω =pq

|p + q| . (38)

For any nonzero value of γ, however, there existpayoff matrices for which the equilibrium point liesin the interval of relevance, but |r| is large enoughthat Eq. (37) does not hold and the Hopf does notexist.

Now, substituting Eq. (36) back into Eq. (34),we obtain the critical delay Tc:

Tc = cos−1

( −γr

p + q − γr

)1pq

√(p + q)3

p + q − 2γr. (39)

0.2 0.4 0.6 0.8 1.0γ

r = tr A

0.2 0.4 0.6 0.8 1.0γ

r = tr A

(a) (b)

Fig. 2. The Hopf bifurcation occurs in systems lying in the shaded regions, i.e. (a) below the curve p + q = 2γr if p + q > 0and (b) above the curve p + q = 2γr if p + q < 0.

1650006-5

February 2, 2016 9:40 WSPC/S0218-1274 1650006

E. Wesson et al.

This result agrees with the values of Tc and ω given by Rand and Verdugo [2007]. We also apply the resultsof [Rand & Verdugo, 2007] to obtain an approximation for the amplitude of the limit cycle generated bythe Hopf bifurcation. See Appendix A.

If T = Tc + µ, the amplitude R of the limit cycle produced by the Hopf bifurcation is given by

R =

õP

Q(40)

where

P = −4p7q7(5(p + q) − 9γr)(p + q − 2γr)(p + q − γr)3

(p + q)12(41)

Q = −p4q4

√p + q − 2γr

(p + q)3(p + q − γr)3

(p + q)8

×[(p2 − q2)2(p + q − 3γr)

√p + q − 2γr

(p + q)3+ [−3γr(2p2 − pq + 2q2)(p + q)

+ (3p2 − pq + 3q2)(p + q)2 + 3γ2r2(p − q)2] cos−1

(− γr

p + q − γr

)]. (42)

When γ = 0, this reduces to

P = − 20p7q7

(p + q)7, (43)

Q = −p4q4(π(3p2 − pq + 3q2)|p + q| + 2(p + q)(p − q)2)2(p + q)5

. (44)

Since R is real, µ must have the same sign asP/Q. This determines whether the Hopf bifurcationis sub- or supercritical. In particular, if the pointz = 0 is stable for delay T < Tc and µ > 0, thenthe limit cycle is stable and the bifurcation is super-critical. We will treat an example of this type in thenext section.

4. Example: Hawk–Dove Games

As an example, consider the Hawk–Dove systemdescribed by Nowak [2006]. There are two strategiescompeting for a resource with benefit b: “hawks,”who will escalate fights against other players, and“doves,” who will retreat from fights. So, if a hawkmeets a dove, the hawk always wins, receiving ben-efit b, while the dove receives nothing. If two dovesmeet, each is equally likely to win the resource, sothe expected payoff is b/2. If two hawks meet, theyfight over the resource; each expects to gain benefitb/2 and incur a cost of injury c/2, for an expectedpayoff of b−c

2 . Therefore the game is represented bythe payoff matrix

A =

b − c

2b

0b

2

(45)

where b and c are positive numbers. (Note that b, cin Eq. (45) are not the same as b, c in Eq. (5).)

In this case, we have

p =b − c

2, q = − b

2, r = b − c

2(46)

and Eq. (25) becomes

x =12x(x − 1)(cx − b + γ(2b − c)(x − x)). (47)

The equilibria of the system are

x = 0, x = 1, x =b

c. (48)

The condition for the third equilibrium to lie in theinterval of physical relevance (0, 1), which is that p

1650006-6

February 2, 2016 9:40 WSPC/S0218-1274 1650006

Hopf Bifurcations in Two-Strategy Delayed Replicator Dynamics

and q must have the same sign, reduces to

0 < b < c. (49)

Notice that this means there is an equilibrium pointwhere both x and y are nonzero (i.e. both strategiescoexist), if and only if the expected payoff 1

2(b − c)for a hawk versus another hawk is negative.

If we let

z = x − b

c(50)

then the linearization about z = 0 is

z =b(b − c)(cz − (c − 2b)(z − z)γ)

2c2. (51)

In the case of no delay (T = 0 and z = z) thisbecomes

z =b(b − c)z

2c. (52)

Therefore, if Eq. (49) holds — that is, if the thirdequilibrium lies in the interval of relevance — thenthe point z = 0 is stable for T = 0, for any valueof γ.

If there is a Hopf bifurcation, its critical fre-quency Eq. (36) is

ω =b(c − b)2c3/2

√4bγ − 2γc + c. (53)

From Eq. (37), we see that the condition for ω tobe real — i.e. for the point z = 0 to have a Hopfbifurcation — is

(2γ − 1)c < 4γb. (54)

If γ ≤ 1/2, then Eq. (54) is trivially true, so ω isreal for all Hawk–Dove games such that Eq. (49)

holds. If γ > 1/2, however, ω is real only if

c <4bγ

2γ − 1. (55)

It is instructive to define a new parameter k,such that

c = kb. (56)

Intuitively, k is the cost per unit of benefit that thehawks are willing to incur. We can enforce condi-tions Eqs. (49) and (54) by stipulating that k > 1,and if γ > 1/2,

k <4γ

2γ − 1. (57)

Then in terms of b and k, the frequency ω is

ω =b(k − 1)

√4γ − 2γk + k

2k3/2. (58)

The critical delay Eq. (39) is

Tc =2c3/2

b(c − b)√

4bγ − 2γc + c

× cos−1

(1 − c

2bγ − γc + c

)(59)

=2k3/2

b(k − 1)√

4γ − 2γk + k

× cos−1

(k

(γ − 1)k − 2γ+ 1

)(60)

and the amplitude of the limit cycle that is born inthis bifurcation is given by Eqs. (40)–(42):

R =

õP

Q(61)

where µ = T − Tc. The ratio P/Q can be writtenin terms of b and k as

P

Q= [2bk−5/2(k − 1)3(2γ(k − 2) − k)(9γ(k − 2) − 5k)]

/[√

k(k − 2)2(−k2 − 6γ2(k − 2)2 + 5γ(k − 2)k) +√

4γ − 2γk + k(k2(k(3k − 7) + 7)

+ 3γ2(k − 2)4 − 3γk(k(2k − 5) + 5)(k − 2)) cos−1

(k

(γ − 1)k − 2γ+ 1

)]. (62)

In the full delay case (γ = 0), the critical frequency and delay become

ω =b(k − 1)

2k(63)

1650006-7

February 2, 2016 9:40 WSPC/S0218-1274 1650006

E. Wesson et al.

and

Tc =πk

b(k − 1)(64)

and the ratio P/Q reduces to

P

Q=

20b(k − 1)3

k3(π(k(3k − 7) + 7) − 2(k − 2)2). (65)

In the off-diagonal case (γ = 1), these values are

ω =b√

4 − k(k − 1)2k3/2

(66)

and

Tc =2k3/2 cos−1

(1 − k

2

)b√

4 − k(k − 1)(67)

and

P

Q=

−2b√

4 − k(k − 1)3(2k − 9)k−5/2

(k − 3)√−(k − 4)k(k − 2)2 + (24 − (k − 3)k(2k − 11)) cos−1

(1 − k

2

) . (68)

Note that in terms of b and k, for all values of γ

ω ∝ b, Tc ∝ 1b,

P

Q∝ b. (69)

Therefore, we can divide each of these quantitiesby the appropriate power of b to obtain normalizedversions that depend only on the parameter k.

Observe (Figs. 3–5) that when k = 2, ω, Tc andP/Q do not change as γ varies. In fact, for k = 2they reduce to

ω =b

4, Tc =

2πb

,P

Q=

b

2π. (70)

γ =0

γ =1/3

γ =2/3

γ =1

2 3 4 5 6 7 8k

0.1

0.2

0.3

0.4

ω / b

Fig. 3. Normalized frequency ω/b, at the Hopf bifurcationin the Hawk–Dove system, as a function of k = c/b. Red: fulltype delay (γ = 0). Blue: off-diagonal delay (γ = 1). Orange:γ = 1/3. Green: γ = 2/3.

This is to be expected, since for this value of kthe Hawk–Dove replicator equation (47) does notdepend on γ. It reduces to

x =12b(x − 1)x(2x − 1). (71)

We see by plotting the normalized version ofEq. (62) (Fig. 5) that P/Q > 0, so for the ampli-tude R to be real, µ must also be positive. Thus theHopf bifurcation is supercritical, and the limit cycleis stable. Numerical simulation using NDSolvein Mathematica confirms the stability of the limit

γ =0

γ =1/3

γ =2/3γ =1

1 2 3 4 5 6 7 8k

5

10

15

20

25b T c

Fig. 4. Normalized critical delay bTc, for the Hopf, as afunction of k = c/b. Red: full type delay (γ = 0). Blue:off-diagonal delay (γ = 1). Orange: γ = 1/3. Green: γ = 2/3.

1650006-8

February 2, 2016 9:40 WSPC/S0218-1274 1650006

Hopf Bifurcations in Two-Strategy Delayed Replicator Dynamics

γ =0

γ =1/3

γ =2/3γ =1

2 3 4 5 6 7 8k

0.05

0.10

0.15

P / ( bQ )

Fig. 5. Normalized growth coefficient PbQ , for the Hopf, as

a function of k = c/b. Red: full type delay (γ = 0). Blue:off-diagonal delay (γ = 1). Orange: γ = 1/3. Green: γ = 2/3.

cycle. See Figs. 6 and 7, in which we show numer-ical solutions for the particular case b = 1 andc = k = 3, corresponding to the payoff matrix

A =

−1 1

012

(72)

for values of T above and below Tc.Finally, we compare the results of this pertur-

bation method to those obtained by continuationin DDE-BIFTOOL. (The latter method is outlinedby Heckman [2012].) As in Figs. 6 and 7, we takeb = 1, c = k = 3. We have plotted the results forγ = 0, γ = 1/3, γ = 2/3, and γ = 1. See Fig. 8.

0 50 100 150 200 250t

0.2

0.4

0.6

0.8

1.0x ( t )

0 50 100 150 200 250t

0.2

0.4

0.6

0.8

1.0x ( t )

(a) (b)

Fig. 6. Numerical solutions of the Hawk–Dove system with b = 1, c = k = 3 and γ = 1/3 for values of T above and belowTc. (a) Delay 5 = T < Tc. Interior equilibrium is stable and (b) delay 7 = T > Tc. Limit cycle is stable.

0.2 0.4 0.6 0.8 1.0x (t)

−0.20

−0.15

−0.10

−0.05

0.00

0.05

x ' (t)

0.2 0.4 0.6 0.8 1.0x (t)

−0.20

−0.15

−0.10

−0.05

0.00

0.05

0.10x ' (t)

(a) (b)

Fig. 7. Parametric (x versus x) representation of numerical solutions of Hawk–Dove system with b = 1, c = k = 3 andγ = 1/3 for values of T above and below Tc. (a) Delay 5 = T < Tc. Interior equilibrium is stable and (b) delay 7 = T > Tc.Limit cycle is stable.

1650006-9

February 2, 2016 9:40 WSPC/S0218-1274 1650006

E. Wesson et al.

4.7 4.8 4.9 5 5.1 5.2 5.3 5.4 5.50

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Delay T

Am

plitu

de 2

R (

max

−m

in)

5.7 5.8 5.9 6 6.1 6.2 6.30

0.1

0.2

0.3

0.4

0.5

Delay T

Am

plitu

de 2

R (

max

−m

in)

(a) Full delay (γ = 0) (b) γ = 1/3

7.5 7.6 7.7 7.8 7.9 8 8.1 8.2 8.30

0.1

0.2

0.3

0.4

0.5

0.6

Delay T

Am

plitu

de 2

R (

max

−m

in)

10.8 10.9 11 11.1 11.2 11.3 11.4 11.5 11.6 11.70

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Delay T

Am

plitu

de 2

R (

max

−m

in)

(c) γ = 2/3 (d) Off-diagonal delay (γ = 1)

Fig. 8. Amplitude of limit cycle versus T in the Hawk–Dove system with b = 1, c = k = 3 given by Lindstedt (upper curve,red) and continuation in DDE-BIFTOOL (lower curve, blue), for various values of γ.

Note that the amplitude given by DDE-BIFTOOL is the full width of the limit cycle, twicethe amplitude predicted by Lindstedt’s method,which is the average displacement from the equi-librium point. We observe from Fig. 8 that for alltested values of γ the results of the two methods arein good agreement for values of T reasonably closeto Tc.

5. Conclusion

We have investigated the dynamics of two-strategyreplicator systems with delay. We have consid-ered a range of models indexed by a homotopy

parameter γ, which determines the relative weightsof delayed and nondelayed terms when determin-ing the fitness arising from interactions with agentshaving the same strategy. (Interactions betweenagents with different strategies are always delayed.)At one extreme (γ = 0) the model describes a full-type delay in which the fitness of each strategy isthe delayed expected payoff of that strategy. At theother extreme (γ = 1) only the opposite-strategyterms in each fitness function are delayed.

It is well known that periodic motions can-not occur in nondelayed two-strategy replicatorsystems, since the phase space is one-dimensional.

1650006-10

February 2, 2016 9:40 WSPC/S0218-1274 1650006

Hopf Bifurcations in Two-Strategy Delayed Replicator Dynamics

The introduction of delay makes the system into aDDE, so the phase space is infinite-dimensional.

In this work, we have shown that, for all valuesof γ, there exist two-strategy games for which non-degenerate Hopf bifurcations and limit cycles occur.In the full delay case (γ = 0), Hopf bifurcationsoccur for all two-strategy games. For γ > 0, thereexist two-strategy games for which there is no Hopfbifurcation.

In particular, we have demonstrated a rangeof parameters for which Hawk–Dove systems withdelayed competition exhibit stable limit cycleswhich are born in Hopf bifurcations. That is, forsuch systems, the population dynamics converge toa periodic oscillation in the proportions of hawksversus doves. We have used Lindstedt’s method toapproximate the amplitude of the limit cycles, andwe have shown that this approximation agrees withthe results of numerical continuation.

If γ ≤ 1/2, i.e. closer to full delay than off-diagonal delay, the Hopf bifurcation occurs for allHawk–Dove systems in which the cost c of fightingis greater than the benefit b attained. If γ > 1/2,i.e. closer to off-diagonal delay, there is a maximumcost-benefit ratio above which the bifurcation doesnot occur, and limit cycles in the population fre-quencies are not observed.

This generalization of the replicator equationmay be useful in modeling a range of scenarios,given the ubiquity and heterogeneity of delay inreal-world applications. It is often the case that peo-ple have more accurate and up-to-date informationabout those who are more similar to themselves,and that organisms which have the same pheno-type are located closer to each other. Thus it maywell be that delay plays a bigger role in interac-tions between agents with different strategies thanagents with the same strategy. The framework weintroduce here demonstrates the important conse-quences such heterogeneous delays can have on thesystem’s evolutionary dynamics.

References

Alboszta, J. & Miekisz, J. [2004] “Stability of evolution-arily stable strategies in discrete replicator dynamicswith time delay,” J. Theor. Biol. 231, 175–179.

Erneux, T. [2009] Applied Differential Delay Equations(Springer Science+Business Media, NY).

Heckman, C. [2012] “Numerical continuation usingDDE-BIFTOOL,” Retrieved from http://www.math.cornell.edu/rand/randdocs/Heckman DDEBiftool/.

Hofbauer, J., Shuster, P. & Sigmund, K. [1979] “A noteon evolutionarily stable strategies and game dynam-ics,” J. Theor. Biol. 81, 609–612.

Hofbauer, J. & Sigmund, K. [1998] Evolutionary Gamesand Population Dynamics (Cambridge UniversityPress, Cambridge).

Nowak, M. [2006] Evolutionary Dynamics (Belk-nap Press of Harvard Univ. Press, Cambridge,MA).

Rand, R. & Verdugo, A. [2007] “Hopf bifurcation formulafor first order differential-delay equations,” Commun.Nonlin. Sci. Numer. Simul. 12, 859–864.

Taylor, P. & Jonker, L. [1978] “Evolutionarily stablestrategies and game dynamics,” Math. Biosci. 40,145–156.

Weibull, J. [1995] Evolutionary Game Theory (MITPress, Cambridge, MA).

Wesson, E. & Rand, R. [2014] “Hopf bifurcationsin delayed rock-paper-scissors replicator dynamics,”Dynamic Games and Applications, doi: 10.1007/s13235-015-0138-2.

Yi, T. & Zuwang, W. [1997] “Effect of time delay andevolutionarily stable strategy,” J. Theor. Biol. 187,111–116.

Zeeman, E. [1981] “Dynamics of the evolution of animalconflicts,” J. Theor. Biol. 89, 249–270.

Appendix A

Hopf Bifurcation Formulafor First-Order DDEs

We present the formula for the radius of a limitcycle that is born in a Hopf bifurcation in a first-order constant-coefficient differential delay equa-tion, derived by Rand and Verdugo [2007].

Consider a differential delay equation (DDE)

dx

dt= αx + βx + a1x

2 + a2xx + a3x2 + b1x

3

+ b2x2x + b3xx2 + b4x

3 (A.1)

where x = x(t) and x = x(t − T ). The associatedlinear DDE is

dx

dt= αx + βx. (A.2)

Assume that Eq. (A.1) has a critical delay Tc forwhich it has a pair of pure imaginary eigenvalues±iωc and the linearized problem Eq. (A.2) has thegeneral solution

x = c1 cos ωct + c2 sin ωct. (A.3)

1650006-11

February 2, 2016 9:40 WSPC/S0218-1274 1650006

E. Wesson et al.

Since the system is autonomous, we can withoutloss of generality choose the initial conditions sothat the periodic solution has only one term,

x = c1 cos ωct. (A.4)

The goal of Lindstedt’s method is to find a peri-odic solution of the full nonlinear DDE Eq. (A.1) forvalues of T close to Tc by detuning the frequencyoff ωc. First, we introduce a small parameter ε viathe scaling

x = εu. (A.5)

We detune T by a term of order ε2:

T = Tc + µ = Tc + ε2µ2. (A.6)

Next we stretch time by the change of variables

τ = ωt (A.7)

where ω is detuned from ωc:

ω = ωc + ε2k2 + · · · . (A.8)

This gives us

du

dt= ω

du

dτ= (ωc + ε2k2 + · · ·)du

dτ(A.9)

and

u = u(τ − ωT )

= u(τ − (ωc + ε2k2 + · · ·)(Tc + ε2µ2)) (A.10)

= u(τ − ωcTc − ε2(k2Tc + µ2ωc) + · · ·) (A.11)

= u(τ − ωcTc) − ε2(k2Tc + µ2ωc)u′

× (τ − ωcTc) + · · · . (A.12)

Finally, we expand u as a series in ε:

u(τ) = u0(τ) + εu1(τ) + ε2u2(τ) + · · · . (A.13)

Collecting similar orders of ε, we find that thelowest-order equation has the solution

u0(τ) = A cos τ. (A.14)

By substituting this into the next higher-orderequation and eliminating secular terms, we can findA as a function of µ2. Multiplying through by ε2, wethus find that the limit cycle solution of Eq. (A.1)can be approximated by

x = R cos ωt (A.15)

where the amplitude R = ε2A as a function ofµ = ε2µ2 is given by

R2 =µP

Q(A.16)

where

P = 4β3(4α − 5β)(β − α)(α + β)2 (A.17)

Q = 5b2Tcβ6 + 15b4Tcβ

6 + 15b1β5 + 5b3β

5 − 4a21Tcβ

5 − 3a22Tcβ

5 − 22a23Tcβ

5

− 7a1a2Tcβ5 − 14a1a3Tcβ

5 − 7a2a3Tcβ5 − 15αb1Tcβ

5 + αb2Tcβ5 − 15αb3Tcβ

5

+ 3αb4Tcβ5 − 18a2

1β4 − a2

2β4 − 4a2

3β4 − 9a1a2β

4 − 18a1a3β4 − 9a2a3β

4

+ 3αb1β4 − 15αb2β

4 + αb3β4 − 15αb4β

4 + 18αa21Tcβ

4 + 7αa22Tcβ

4

+ 12αa23Tcβ

4 + 19αa1a2Tcβ4 + 30αa1a3Tcβ

4 + 37αa2a3Tcβ4 − 3α2b1Tcβ

4

+ 6α2b2Tcβ4 − 3α2b3Tcβ

4 − 12α2b4Tcβ4 + 12αa2

1β3 + 11αa2

2β3 + 26αa2

3β3

+ 33αa1a2β3 + 30αa1a3β

3 + 19αa2a3β3 − 12α2b1β

3 − 3α2b2β3 + 6α2b3β

3

− 3α2b4β3 − 8α2a2

1Tcβ3 − 12α2a2

2Tcβ3 + 4α2a2

3Tcβ3 − 26α2a1a2Tcβ

3 − 16α2a1a3Tcβ3

− 20α2a2a3Tcβ3 + 12α3b1Tcβ

3 − 12α2a1a3β2 − 32α2a2a3β

2 + 12α3b2β2 + 2α3b3β

2

+ 2α3b2Tcβ3 + 12α3b3Tcβ

3 − 14α2a22β

2 − 8α2a23β

2 − 18α2a1a2β2

+ 12α3b4β2 + 8α3a2

2Tcβ2 + 8α3a1a2Tcβ

2 − 4α3a2a3Tcβ2 − 8α4b2Tcβ

2

+ 4α3a22β − 8α3a2

3β + 8α3a2a3β − 8α4b3β + 8α4a2a3. (A.18)

1650006-12

February 2, 2016 9:40 WSPC/S0218-1274 1650006

Hopf Bifurcations in Two-Strategy Delayed Replicator Dynamics

The critical delay Tc and frequency ωc may beexpressed in terms of α and β by considering thelinear equation (A.2). Substituting Eq. (A.4) intoEq. (A.2) and setting the coefficients of sinωct andcos ωct equal to zero give

β sinωcTc = −ωc, β cos ωcTc = −α. (A.19)

Squaring and adding these, and substituting theresult back in, yields

ωc =√

β2 − α2 (A.20)

and

Tc =cos−1

(−α

β

)√

β2 − α2. (A.21)

The system in question, the two-strategy repli-cator equation (30), can be written as

z = αz + βz + a1z2 + a2zz + a3z

2

+ b1z3 + b2z

2z + b3zz2 + b4z3 (A.22)

where

α =γpqr

(p + q)2(A.23)

β =pq(p + q − γr)

(p + q)2(A.24)

a1 =γr(p − q)

p + q(A.25)

a2 =(p − q)(p + q − γr)

p + q(A.26)

b1 = −rγ, (A.27)

b2 = −p − q + γr (A.28)

a3 = b3 = b4 = 0. (A.29)

We substitute these values into Eqs. (A.16)–(A.18) to obtain the amplitude of the limit cycle,Eqs. (40)–(42).

1650006-13