Takustr. 714195 Berlin

GermanyZuse Institute Berlin

STEFANIE WINKELMANN1, JAN-HENDRIK NIEMANN2, SARAH WOLF3,CHRISTOF SCHÜTTE

Agent-based modeling: Population limits, metastabilityand large deviations4

1 0000-0002-0114-78192 0000-0003-4535-30563 0000-0002-5161-03494to appear in: AIP Publishing, 2020

ZIB Report 20-06 (Februar 2020)

Zuse Institute BerlinTakustr. 714195 BerlinGermany

Telephone: +49 30-84185-0Telefax: +49 30-84185-125

E-mail: bibliothek@zib.deURL: http://www.zib.de

ZIB-Report (Print) ISSN 1438-0064ZIB-Report (Internet) ISSN 2192-7782

Agent-based modeling: Population limits, metastability and largedeviations∗

S. Winkelmann, J.-H. Niemann, S. Wolf, Ch. Schütte

Abstract

Modeling, simulation and analysis of interacting agent systems is a broad field of research, withexisting approaches reaching from informal descriptions of interaction dynamics to more formal,mathematical models. In this paper, a continuous-time stochastic agent-based model is formulated,the corresponding Markov jump process is defined and its approximation by ordinary and stochasticdifferential equations (ODEs and SDEs, respectively) is described. We provide a functional andtransparent framework which allows for rigorous analysis, avoids problems of ambiguity and deliversstraightforward connections to other modeling approaches. We demonstrate the advantages of anSDE model for different scenarios of interacting agent systems with medium or large populationsizes. In comparison to the ODE limit model, the SDE gives a higher order approximation of theunderlying Markov jump process, both on a pathwise level and regarding the process’ moments.In particular, the SDE approach is able to retain metastabilty in the dynamics, which is lost in adeterministic ODE description, and to capture the distribution of rare and unlikely extreme events.Here, we apply the theory of large deviations to show consistency of the distributions’ remote tails.

Keywords: Agent-based models, continuous-time Markov jump process, stochastic differentialequation, population scaling, metastability, large deviations

This article provides a lucid mathematical framework for describing and analyzingsocial interaction dynamics as described by agent-based models including standardmodels for opinion formation or spreading of infectious diseases on social networks.It is demonstrated that the large population limit of infinitely many agents is givenby the commonly used deterministic mean-field limit. For the setting of a moderatepopulation size (i.e. many but not infinitely many agents) an approximative stochasticmodel is presented which, in contrast to the mean field limit, reproduces the fluctu-ations of the agent-based model up to second order and in particular exhibits rarestochastic effects like metastable behavior and rare transitions. The theory of largedeviations is then used to characterize these rare events completely for both, theagent-based model as well as for its stochastic approximation. This abstract conceptis illustrated by means of some guiding examples based on which it is also shown

∗The following article has been submitted to AIP Publishing.

1

that, under appropriate conditions, the approximate stochastic model can appropri-ately capture the metastable behavior and rare events of the full agent-based modelfor moderate population size.

1 IntroductionSocial dynamics and collective social phenomena arise in systems of multiple agents that act andinteract, often based on incomplete information, within a social network embedded in a commonenvironment, which can be given by geographical conditions, infrastructure, resources, etc., aswell as by norms, rules, and narratives. An agent can represent an individual, a household, firm,political or administrative organization or any type of discrete entity. A wide range of applications,such as innovation spreading (e.g., [24]) or infection kinetics (e.g., [14]) is addressed, using a widespectrum of methods from data-based micro-simulation of synthetic populations (e.g., [35]) toabstract individual1- and agent-based models (ABM) for studying underlying dynamic mechanisms.Comparatively compact models of social dynamics are present especially in the fields of socio- andecononophysics, see, e.g. the recent special issue "From statistical physics to social sciences" [6].A prime application lies in opinion dynamics, a field that goes back to the introduction of thevoter model by Clifford and Sudbury [9] in the 1970s, and later on obtained its name by Holley[22]. The basic idea is that agents imitate the opinion of neighbors. Various modifications inthe representation of opinions, imitation details, and the interaction structures exist2, see e.g.[23, 33, 34] for recent overviews.

While a focus on the individuals in social systems and their interactions was proposed as earlyas 1957 by Orcutt [32], only in the 1990ies, motivated by the increasing computing power, computersimulations of the resulting dynamics became a commonly used tool. Also nowadays, many modelsare implemented with the help of (often object-oriented and discrete-time) software frameworksfor agent-based modeling (see, e.g., [1]); not always the model is precisely specified beforehand.The basics of an ABM are easily communicable, e.g. to stakeholders, as individual (inter)actionrules at the micro level and stepwise simulation of the overall system’s dynamics can be explainedwithout requiring mathematical expertise of the audience. However, the great freedom of themodeler in defining details can make model documentation a challenge [20]. One point that oftenremains ambiguous in ABMs with a discrete time step – which is the overwhelming majority – isscheduling, that is, which agent(s) influence(s) which other agent(s) in which order in each timestep. As Weimer et al. [37] point out, most textbooks do not deeply reflect upon this topic3 andmodels commonly use random interaction orders in each step. In this case, the question ariseswhether agents "see" the changes already made within the same time step by other agents actingearlier. For parallelisation of the ABM simulation, this may lead to problems, however. Whenconsidering an ABM to be a representation of a real-world social system, time discretization intosteps in which each agent acts once is anyhow somewhat artificial. In this paper, we therefore use a

1The term individual-based models is predominantly used in ecology, whereas social sciences rather refer to agent-based models.

2For example, with the AB-model [8] an intermediate opinion AB between two opinions A and B is introduced.3Weimer et al. propose the SAS (Synchrony, Actor type, Scale) classification to simplify and generalize discussions

and comparisons between schedules, allowing standardization and reproduction of ABMs.

2

continuous-time approach to model interacting agent dynamics, thus circumventing the schedulingproblem.

Beside the simulation-oriented and rather informal ABMs there also exist more precise math-ematical modeling approaches for interacting agent dynamics. These range from (very detailed)microscopic stochastic descriptions that follow spatial movement and neighbor interaction of eachindividual agent [10], over individual agent stochastic descriptions with given (temporally fixed orvariable) interaction network but without explicit spatial movement [5], as well as (discrete-time)Markov chain approaches for collective population dynamics [3] including metastability analysis[21], to macroscopic deterministic models for the mean dynamics in a spatially well-mixed sce-nario [30]. There are a few approaches using continuous-time Markov processes for analyzing socialdynamics, see e.g. [2, 10]; a corresponding software framework is proposed in [36]. For opiniondynamics with agents transition rates depending linearly on the fraction of neighbors sharing thesame opinion, exact expressions for expectation and variance are derived in [5], considering identi-cal agents, binary opinions and complete networks – called peer assembly – and using lumpabilityinto a continuous-time birth-death chain. The approximation of continuous-time interacting agentdynamics by means of stochastic differential equations, as proposed in the present work, however,has rarely been touched in the literature (but are discussed – without derivation – in terms ofmean-field approximations utilizing the corresponding Fokker-Planck equations for discrete time,see [23]).

Starting point in this paper is a continuous-time Markov jump process which describes thecollective population dynamics of a rather general ABM similarly to the discrete-time model givenin [3]. In analogy to the term interacting particle system which is used in the context of chemicalreaction networks, we refer to our model as an interacting agent system. For the basic case ofa complete interaction network and homogeneous agents, we show how methods and results thatare well-known in the chemical context can be useful for analyzing dynamics of social systems. Inour model, each agent can switch according to given transition rules between finitely many states(called types). The propensities for such transitions to occur depend on the total population state,which counts the number of agents for each of the available types. The temporal evolution of thispopulation state is given by the Markov jump process, called transition jump process, and canbe characterized by a master equation, in direct analogy to the reaction jump process consideredin the context of chemical reaction systems. This very general setting can be applied in variouscontexts: agents may have health states and transitions represent infection and recovery; agentsmay have chosen a certain technology and transitions represent innovation adoption that happens,for example, by imitation; and many more examples can be considered.

By applying Gillespie’s algorithm [16] for the numerical simulation of the transition jump pro-cess, the problems of scheduling described above are naturally avoided. However, these simulationsbecome computationally intensive for settings with numerous agents, which motivates to find ap-proximative modeling approaches. Also in this regard, there are well-known results within thetheory of stochastic chemical reaction kinetics which can directly be translated to systems of inter-acting agents: In case of a medium or large population size, the transition jump process, describ-ing the agents’ interaction dynamics, can be approximated by ordinary or stochastic differentialequations (ODEs and SDEs, respectively), which reduces the numerical effort and makes it inde-pendent of the number of agents [13, 25, 26, 27, 28]. While the ODE approximation reproduces the

3

time-dependent first order moments of jump process, the SDE is able to capture both first- andsecond-order moments and to retain the characteristic behavior of the jump process. For demon-stration, we consider metastable dynamics of interacting agents and show that the SDE modelreproduces metastability, while the corresponding ODE approximation does not. Moreover, theprocess defined by the SDE effectively approximates the jump process regarding the distribution ofremote tail events, i.e., those events which have a very small probability. Here, the theory of largedeviations provides a tool for analyzing the processes’ untypical behavior [11]. As such rare events– although being unlikely – still may be critical in the dynamics of social systems (e.g. outbreak ofpandemics), this appears to be a valuable property of the SDE approximation. In general, the SDEapproach delivers a functional and transparent framework which allows for rigorous analysis of theunderlying dynamics. The corresponding generator exists and is clearly defined, prominent resultsfrom probability theory, like the Feynman-Kac formula, can directly be applied, and connectionsto deterministic modeling approaches are straightforward.

Overall, the transition jump process (like an agent-based discrete-time model) can be viewedas the microscopic description of the dynamics on the scale of population size, while the ODE isthe macroscopic analogue, and the SDE can be seen as a mesoscopic variant in between. Whileagent-based models of social dynamics are commonly used to model heterogeneous agents in localinteractions, we view this work on a simpler, spatially well-mixed agent system as a first step towardsfurther development of approximation methods for more complex dynamics of social systems.

The paper is structured as follows: The interacting agent system and the transition jump processare introduced in Sec. 2. To provide examples, a reduced standard setting is defined which willreappear in the subsequent sections. In Sec. 3, we rescale the jump process and consider the limitprocesses given by ODEs and SDEs. Two relevant types of dynamics are compared by means ofillustrative examples. The remote tail events are analyzed in Sec. 4, where we apply the theory oflarge deviations for both the jump process and the SDE process.

2 Interacting agent systems as stochastic transition jump pro-cesses

In this section, it will be shown how to model a system of interacting agents as a continuous-timeMarkov jump process. We consider a population of N agents, each of whom can be in one ofn different states Si, i = 1, . . . , n. In order to avoid confusion with the state of the system, werefer to the Si as types in the following, so n is the number of types.4 In the course of time, theagents can randomly change their types by interacting with other agents. Instead of modeling eachagent individually, the system’s state will be characterized by a vector containing the number ofagents of each type. This approach is based on the fundamental assumption that agents of onetype are indistinguishable and that each agent can at anytime interact with all other agents, i.e.,

4Actually, this is not ideal either, as in more complex ABMs, the term type may indicate persons, households,firms, etc. In many contexts, one may refer to agent choices (and in fact, their state may involve more than just theircurrent choice), however, this does not carry over, e.g., to the health case, where people probably do not choose tobe infected. Adding in passing that in the chemical reaction context, where agents are molecules, type correspondsto the term species and in evolutionary game theory, where agents are called players, to strategies, we neverthelessstick to type here.

4

the corresponding interaction network is a complete (perhaps weighted) graph, which correspondsto peer assembly in [5].

2.1 The interacting agent system

The considered system of interacting agents consists of

• a fixed number N ∈ N of agents,

• a set {Si : i = 1, . . . , n} of types Si available to the agents, i.e., at any point in time eachagent owns one of these types,

• a set {R1, . . . , RK} of transition rules Rk defining possible changes of the agents’ types, i.e.,representing actions of and interactions between agents,

• a set of propensity functions specifying the rates at which transitions randomly occur depend-ing on the population state.

In the agent-based context, transition rules are mostly described from the perspective of asingle agent; they may also be referred to as update rules or decisions. To completely specify thedynamics, in this case also a description of the interaction patterns and update orders (scheduling)is needed. Here, we adopt the formulation used commonly in the chemical context that takes asystem level perspective, leading to a simpler and more transparent description: each transitionrule Rk is represented by an equation of the form

Rk : a1kS1 + . . . + ankSn → b1kS1 + . . . + bnkSn, (1)

with the coefficients alk, blk ∈ N0 denoting the numbers of agents of each type involved in thetransition as input and output interaction partners (see below for examples). In order to ensurethat the total number of agents is not changed by such a transition, we assume

∑ni=1 aik =

∑ni=1 bik

for each k. The associated vector νk = (ν1k, . . . , νnk) ∈ Zn is defined by

νik := bik − aik

and describes the net change in the number of agents of each type Si due to transition Rk. Notethat different transitions can lead to identical net changes, accounting, e.g., for different socialmechanisms inducing a specific change. The population state of the system (i.e., the system state)is given by a vector of the form

x = (x1, . . . , xn) ∈ Nn0 ,

where xi refers to the number of agents of type Si. As the total number of agents is assumed to beconstant, the population state space5 is given by

X :={

x = (xi)i=1,...,n ∈ Nn0 :

n∑i=1

xi = N

}.

5This system description corresponds to the partition of the space of all possible configurations on which Banischet al. then define the macrodynamics of their general opinion model, see Sec. 4.3 in [3].

5

Each transition Rk induces an instantaneous change in the population state of the form

x → x + νk,

which may occur at any point in time t ≥ 0. The probability for transition Rk to happen in aninfinitesimal time step dt is given by αk(x)dt, where αk : X → [0, ∞) is the propensity function ofthis transition. We assume the propensity αk(x) to be proportional to the number of combinationsof interacting agents in x, and, moreover, to scale with the total population size N . E.g., for asecond-order transition of the form Rk : Si + Sj → Si′ + Sj′ , the number of possibly interactingSi-Sj-pairs is xixj , and the propensity is given by αk(x) = γkxixj/N for a rate constant γk > 0.6The scaling by N reflects the fact that the probability for two agents to meet within a populationof size N is proportional to 1/N . This is directly translated from the chemical reaction context,where the reaction propensities scale with the volume of the system (classical mass-action kinetics).For first-order transitions Rk : Si → Sj , it analogously holds that αk(x) = γkxi. Generalizing thesearguments to transitions as given in (1) leads to

αk(x) =

γkN

n∏i=1

1Naik

(xi

aik

)if xi ≥ aik for all i = 1, . . . , n

0 otherwise.(2)

Remark 2.1. The above description has been limited to the case of a complete interaction graphfor the sake of simplicity. Generalizations are possible, e.g., by weighting the edges in the graph orby forming the interaction network out of some weakly connected, complete components or clusters.In these cases, the form of the propensity functions and perhaps the definition of the state spaceneed to be adapted, but the general setting remains the same.

2.2 The transition jump process

The temporal evolution of the system is described by a continuous-time stochastic process X(t)t≥0,

X(t) = (Xi(t))i=1,...,n ∈ X,

with Xi(t) denoting the number of agents of type Si at time t ≥ 0. The process X(t)|t≥0 is aMarkov jump process, i.e. it is piece-wise constant, with jumps of the form

X(t) → X(t) + νk (3)

occurring at random jump times. The waiting times between the jumps follow exponential distri-butions determined by the propensity functions. More precisely, given the current state X(t) = xof the process at any time t, the waiting time τ(x) for the next jump event to occur is exponentiallydistributed with mean 1/λ(x), where λ : X → [0, ∞), given by

λ(x) :=K∑

k=1αk(x),

6γkdt is the probability, to first order in dt, that a randomly selected combination of Rk-initiating agents willinteract accordingly within the next infinitesimal time interval [t, t + dt).

6

is the jump rate function. Then, at time t + τ(x), the process jumps according to one of thetransitions Rk, k = 1, . . . , K, to a new state x + νk, with the probability for transition Rk to occurgiven by αk(x)/λ(x) for each k = 1, . . . , K.

LetP (x, t) := P(X(t) = x | X(0) = x0)

denote the probability to find the system in state x at time t given some initial state x0. In orderto describe the time-evolution of the system we consider the Kolmogorov forward equation of thetransition jump process X(t)|t≥0, which is given by

dP (x, t)dt

=K∑

k=1

[αk(x − νk)P (x − νk, t) − αk(x)P (x, t)

], (4)

where we set αk(x) := 0 and P (x, t) := 0 for x /∈ Nn0 in order to exclude terms in the right-hand

side of (4) where the argument x−νk contains negative entries. In the context of chemical reactionnetworks, (4) is called chemical master equation [17].

Generally, the master equation (4) cannot be solved analytically. Instead, the process’ distri-bution is typically estimated by Monte Carlo simulations of the underlying Markov jump processX(t)|t≥0, which may be generated using Gillespie’s stochastic simulation algorithm [16]. This al-gorithm allows to construct statistically exact realizations of the jump process in continuous time,circumventing the problem of scheduling which naturally arises by temporal discretization. Itscomputational complexity, however, scales with the number N of agents (in particular, the av-erage timestep decreases drastically with N), which motivates to consider approximate modelingapproaches for the case of large populations, see Sec. 3.

2.3 Standard setting

As a standard exemplary setting, we consider two sorts of transition rules. Firstly, there are adaptivetransitions Rij , i 6= j, having the form

Rij : Si + Sj → 2Sj ,

which means that, given two agents of different types Si 6= Sj , one of the agents adopts the typeof the other agent. An example would be the imitation of another agent’s opinion or technologicalinnovation. Secondly, there are spontaneous transitions R′

ij , i 6= j, which have the form

R′ij : Si → Sj

and describe the change of an agent’s type independently of other agents. Note that we replacedthe general index k by tuples (i, j) because this appears intuitive here and simplifies the notation.

For these transitions, the stoichiometric vectors are given by νij = ej−ei, where ei ∈ Nn0 denotes

a vector whose elements are all zero except the entry i which is one. That is, each transition Rij

or R′ij induces an instantaneous change in the population state of the form

x → x + ej − ei,

7

which means that the entry xi of x (the number of agents of type Si) is reduced by one, while xj

(the number of agents of type Sj) increases by one.The propensity functions αij of the adaptive transitions Rij are given by

αij(x) = γij

Nxixj ,

where γij ≥ 0 are chosen rate constants. For the spontaneous transitions R′ij , on the other hand,

we haveα′

ij(x) = γ′ijxi

for some rate constants γ′ij ≥ 0, as we assume all agents to undergo such a transition independently

of the other agents, see (2).By means of this standard setting we can create the following two exemplary scenarios which

will be investigated in more detail in Sec. 3.2.

1. Purely adaptive dynamics with traps: For purely adaptive dynamics with γ′ij = 0 for all

i, j, the jump process becomes non-ergodic and almost surely ends up in an absorbing state(trap), which means that all agents will finally be of the same type and no transitions arepossible anymore. An example is given by the spreading of infectious diseases, with a trapreferring to the extinction of the disease by immunization of the entire population.

2. Metastable dynamics: There exist different metastable areas of the state space X, wherethe population process remains for a comparatively long period of time before it eventuallyswitches to another metastable region. An example in the context of social dynamics couldbe fashions, or the evolution of conventions as described by Young in [39].

3 Population limitsFor ABMs of social systems, population limits are not to be taken literally in the sense of consideringinfinitely many agents. However, especially when considering the shift from small abstract ABMsfor studying mechanisms, as outlined by [29], to large scale empirically based ones (see, e.g., [4]),large numbers of agents become more and more relevant. Therefore, a look at population limits,and what can be learned from particle systems, is useful.

With an increasing population size N , the jumps in the population process X(t)|t≥0 occur moreand more often, while the size of an individual jump relative to the agent numbers xi decreases.In particular, in social systems it is also obvious that in a larger system, more actions take placeand that generally the influence of the single agent diminishes. Stochastic simulations of thejump process using Gillespie’s algorithm become inefficient since they track each of the individualstochastic jump events, and thus time advance in each iteration becomes infinitesimally small. Forlarge N , an appropriate rescaling of the dynamics may lead to effective approximate dynamics givenby stochastic differential equations or ordinary differential equations, a fact which is well-knownin the context of chemical reaction systems and has been mentioned in the context of opiniondynamics in [23]. In what follows, we summarize the corresponding results and apply them in thecontext of interacting agent dynamics using the standard scenarios of Sec. 2.3. These results aremainly based on the law of large numbers and the central limit theorem.

8

3.1 Population scaling and limit processes

Let XN (t)|t≥0 denote the transition jump process as defined in Sec. 2.2 given the total number Nof agents. Also for the propensity functions we now use the notation αN

k in order to indicate theirdependence on N .

The rescaled process

We introduce the relative frequencies c = x/N and rewrite the master equation (4) of the interactingagent system in terms of the frequency-based probability distribution

ρN (c, t) := P(XN (t) = Nc) = P (Nc, t),

together with the propensitiesαk(c) := N−1αN

k (cN).

In order to adopt the notation typically used in multiscale asymptotics we introduce the smallnessparameter ε = N−1. Then, with adapted notation, the master equation (the forward Kolmogorovequation) reads

dρε(c, t)dt

= 1ε

K∑k=1

[αk(c − ενk)ρε(c − ενk, t) − αk(c)ρε(c, t)

]. (5)

The ODE limit model

It has been shown in the 1970s by Kurtz [28] that the rescaled jump process XN (t)/N |t≥0, whosedistribution follows the master equation (5), converges to the frequency process C(t)|t≥0 given bythe ODE

ddt

C(t) =K∑

k=1αk(C(t))νk (6)

with initial state C(0) = limN→∞ XN (0)/N . More precisely, it holds that7

limN→∞

supt≤T

∣∣∣XN (t)/N − C(t)∣∣∣ = 0 a.s. (7)

for every T > 0, with order of convergence given by O(N−1/2) [28, 13]. For the first-order momentof the process XN /N it has been shown that

E(XN (t)/N

)= C(t) + O(N−1)

for all t ≥ 0 [25, 26].This means that in case of large N , the continuous, deterministic process C defined by (6) both

gives a path-wise approximation of the stochastic jump process X/N on finite time intervals, see(7), and approximates its first order moment with an error of order N−1.

7| · | in (7) refers to the Euclidean norm (or any other norm) on Rn.

9

The SDE limit model

In order to capture also the second-order moment of the jump process, we consider the stochasticLangevin process CL(t)|t≥0 given by the SDE

dCL(t) =K∑

k=1αk(CL(t))νkdt +

K∑k=1

1√N

√αk(CL(t))dBk(t)νk, (8)

which, in the context of chemical reaction kinetics, is known as the chemical Langevin equation [18].Here, Bk, k = 1, . . . , K, are independent Brownian motion processes. The Langevin process yieldsa higher-order approximation of the jump process on the path-wise level, with the error scaling likeO(

ln(N)N

)[27]. For the first and second order moments it has been shown in [19] that

E(XN (t)/N

)= E(CL(t)) + O(N−2),

V(XN (t)/N

)= V(CL(t)) + O(N−2)

for every t. Consequently, the SDE process can give a meaningful approximation of the jump process(regarding average dynamics) already for moderate N , where the deterministic limit process wouldfail or produce significant errors.

The Fokker–Planck equation associated with the SDE limit system (8) has the form

∂

dtρε

L(c, t) = −n∑

i=1

∂

∂ci

(bi(c)ρε

L(c, t))

+ ε

2

n∑i,j=1

∂2

∂ci∂cj

(Σij(c)ρε

L(c, t)), (9)

with

b(c) :=K∑

k=1αk(c)νk,

Σ(c) :=K∑

k=1αk(c)νkν>

k , (10)

see [23] for the discrete time case. Just as the master equation of the Markov jump process, alsothe Fokker–Planck equation (9) can in general not be solved analytically. However, trajectories ofthe Langevin process may be generated using the standard Euler–Maruyama scheme. Here, theruntime (average stepsize) is independent of the population size N , which makes the model moreefficient than the jump process model if the number of agents is large.

3.2 Illustrative examples

We now compare the jump process to the two limit processes given by the ODE (6) and the SDE(8) for the two exemplary scenarios described in Sec. 2.3, and investigate the approximation qualityand convergence on finite time intervals.

10

3.2.1 Purely adaptive dynamics

The convergence of the rescaled process to the ODE limit is restricted to finite time intervals, seeEq. (7). In order to illustrate that the statement cannot be extended to infinite time horizons, weconsider the setting of purely adaptive dynamics where γ′

ij = 0 for all i, j. This means that thereare no spontaneous transitions, but an agent can only change its type by adaptive transitions ofthe form

Rij : Si + Sj → 2Sj .

In this case, all states of the form x = Nei for some i = 1, . . . , n are absorbing, i.e., after reachingsuch a state where all agents are of the same type, the process will never leave it again. (Note thatthere can be more absorbing states if γij = 0 for some i 6= j.) In the course of time, the jumpprocess X(t)|t≥0 will almost surely reach one of the absorbing states. Consequently, the processis not ergodic and a path-wise approximation by average dynamics will always fail on a long timehorizon, as we see in the following example.

Example 3.1 (Purely adaptive dynamics with two agent types). Consider two types S1 and S2and set γ12 = 1.1, γ21 = 1 as well as γ′

12 = γ′21 = 0. In this case, looking at the right-hand side

of the ODE (6), we see that ddtC1(t) < 0 and d

dtC2(t) > 0 for all t, i.e., C1(t) converges to zerowhile C2(t) = 1 − C1(t) converges to one for t → ∞. The stochastic jump process, on the otherhand, will in the long run reach one of the absorbing states Ne1 or Ne2 in which one of the twotypes has died out. This "trapping" cannot be approximated by the ODE process but by the SDEprocess, which can approach both absorbing states in the course of time. However, the SDE processwill never reach the absorbing states exactly and can always recover from states close to extinction.Both ODE and SDE approximate the jump process only on finite time intervals, with the length ofthese intervals increasing with N , see Fig. 1.

Remark 3.2 (Trapping states). In order to maintain the trapping of the process in absorbingstates also for the SDE, one would need adaptive models which switch to a discrete jump processdescription as soon as some process components reach threshold values close to zero. Alternatively,one sets the value of the SDE process artificially to zero as soon as it reaches a predefined thresholdvalue close to zero.

3.2.2 Metastable dynamics

For appropriately chosen rate constants γij and γ′ij the stochastic jump process X(t)|t≥0 of the

standard setting given in Sec. 2.3 can be metastable. E.g., given two constants γ, γ′ > 0 withγ′ � γ and γij ≈ γ as well as γ′

ij ≈ γ′ for all i, j we obtain metastable dynamics, where one typepredominates the population for a long period of time before another type may by chance take overand then dominates for another period of time. Such metastable dynamics can be reproduced bythe SDE process, while the ODE process only approximates the time-dependent first order moment.

Example 3.3 (Two agent types in metastable setting). As in Example 3.1 we consider two types S1and S2, this time setting γ12 = 1, γ21 = 1.1 and γ′

12 = 0.03, γ21 = 0.005. This means that adaptivetransitions are more likely to end up in favor of type S1, while spontaneous transitions have a higher

11

0 5 10 15 20 25

Time t

0.0

0.2

0.4

0.6

0.8

1.0

Con

cent

rati

on

N = 101

N = 102

N = 103

N = 104

ODE

(a) Jump process

0 5 10 15 20 25

Time t

0.0

0.2

0.4

0.6

0.8

1.0

Con

cent

rati

on

N = 101

N = 102

N = 103

N = 104

ODE

(b) SDE process

Figure 1: (a) Individual realizations of relative frequencies XN1 (t)/N of type-1-agents for the purely

adaptive dynamics of Example 3.1 in comparison to the ODE solution C1(t). The initial state isXN

1 (0)/N = C1(0) = 0.6. Rate constants γ12 = 1.1, γ21 = 1. In the long run, the trajectoryX1(t)/N of the jump process will almost surely reach value zero or one for any N . Still, forany finite time interval [0, T ], the approximation by the deterministic solution becomes better forincreasing N and converges according to (7). (b) Individual simulations of the SDE process (8) fordifferent N (plotting the first component of CL) in comparison to the ODE solution C1(t). Remark:The reader should be aware that each individual simulation of the transition jump or of SDE processis a random observation from a distribution on path space. Therefore, individual simulations ofthe jump and the SDE process may show characteristic behavior but cannot be compared directly.

rate for agents of type S1 than for S2. In this example, the ODE (6) has an asymptotically stablefixed point C∗ ≈ (0.72, 0.28), while neither for the SDE process nor the transition jump process thisfixed point is stable.For N = 60 (the number of agents used in Fig. 2) both, the jump process and the SDE process,exhibit metastable behavior with two metastable areas: One type dominates the population for aperiod of time, before the system eventually flips over to a predominance of the other type, see Fig.2. For other moderate N we make very similar observations.With increasing N and wrt the same finite window in time, this bistability vanishes and the systemremains close to the deterministic solution of the ODE while short escapes from a narrow cylin-der around it become exponentially unlikely. However, these statements do not characterize whathappens on longer time intervals, especially those ones that scale exponentially with N . On suchexponentially long time intervals metastability still prevails, see large deviations in Sec. 4.In Fig. 3 we see that the SDE delivers a suitable approximation of first and second order momentfor both N = 60 and N = 600 (of course, approximation quality increases with N), while the ODEapproximation of the mean fails for N = 60.

Remark 3.4. It is possible to choose the rate constants such that the ODE limit has several fixed

12

0 100 200 300 400 500

Time t

0

10

20

30

40

50

60

Num

ber

ofag

ents

S1S2

(a) Jump process

0 100 200 300 400 500

Time t

0

10

20

30

40

50

60

Num

ber

ofag

ents

(b) SDE process

0 100 200 300 400 500

Time t

0

10

20

30

40

50

60

Num

ber

ofag

ents

(c) ODE process

Figure 2: Independent simulations of (a) the jump process, (b) the SDE process and (c) the ODEprocess for the metastable dynamics of Example 3.3. Two types S1 and S2, N = 60 agents, γ12 = 1,γ21 = 1.1 and γ′

12 = 0.03, γ′21 = 0.005. The fixed point of the ODE process is N · C∗ ≈ (43.2, 16.8).

Clearly, the ODE process cannot approximate the jump process since it is not able to exhibit ametastable behavior. The fixed point is asymptotically stable.

0 50 100

Time t

0.2

0.4

0.6

0.8

Con

cent

rati

on

JPSDEODE

(a) N = 60 agents

0 50 100

Time t

0.2

0.4

0.6

0.8

Con

cent

rati

on

JPSDEODE

(b) N = 600 agents

Figure 3: Mean (solid lines) and standard deviation (dashed lines) of S1-frequency for the jumpprocess (JP) and the SDE process (estimated from 104 Monte Carlo simulations) in comparison tothe ODE solution, given the metastable dynamics described in Example 3.3 with (a) N = 60 and (b)N = 600 agents. Same parameter values as in Fig. 2. It can be seen that with increasing numberof agents N the ODE process approximates the expectation of the two stochastic processes. Alsothe approximation of the second-order moment by the SDE process improves with increasing N .Especially for small N , see (a), the SDE process gives the better approximation of the expectationthan the ODE process.

13

points and the jump process (as well as the SDE) remains for a certain time period close to one ofthese before switching to another. If we choose, e.g., γ12 = 1, γ21 = 1.1, γ′

12 = 0.03 and γ′21 = 0,

then the ODE limit model has two fixed points, where in contrast to Example 3.1 only one refersto a trapping state: the fixed point Ctrap

∗ = (0, 1). The second one is an asymptotically stable fixedpoint Cstable

∗ = (0.7, 0.3) that does not refer to a trapping state. As in Example 3.3, for appropriateinitial values, the ODE limit dynamics converges asymptotically for t → ∞ to the stable fixed pointCstable

∗ , while for very large time intervals (scaling exponentially with N), the jump process (andthe SDE process as well) will deviate from the narrow cylinder around the stable fixed point andend up in the trapping state.

4 Large DeviationsThe approximation results of Sec. 3 concern the major behavior of the stochastic jump process. Wenow investigate the unlikely tail events (extreme events) of the dynamics and their approximationby the SDE process. For social systems, such events can be of great interest because they should beavoided (e.g., an outbreak of pandemics or riots) or because, on the contrary, a transition is desirable(e.g., a shift to environmentally friendly technologies or healthy lifestyles). We show, for ourstandard setting of Sec. 2.3, that (under specific conditions outlined below) there is an approximateagreement of the tail probabilities when comparing the jump process to the corresponding SDEprocess.

The analysis is done in terms of the rescaled jump process Cε(t) := 1N XN (t), with the corre-

sponding master equation given in (5), where ρε(c, t) := P(Cε(t) = c) and ε = N−1.

4.1 Large deviation rate function

Let Cε(·) = Cε(t)|t∈[0,T ] be an (arbitrary, random) path from an appropriate path space C (e.g.,C = H1([0, T ], Rn)) starting in Cε(0) = c0, and let P denote the probability distribution generatedby the master equation (5), respectively by the Markov jump process associated with it, or by theSDE limit equation (8). Furthermore, let c(·) = c(t)|t∈[0,T ], c(0) = c0, denote a specific path fromthe selected path space C (for example, the solution trajectory of the ODE limit system (6)). Thenwe call I : C → [0, ∞] the large deviation rate function associated with the law P if

limδ→0

lim infε→0

ε ln P(

supt∈[0,T ]

‖Cε(t) − c(t)‖ < δ)

= limδ→0

lim supε→0

ε ln P(

supt∈[0,T ]

‖Cε(t) − c(t)‖ < δ)

= −I(c(t)|t∈[0,T ]

),

see [15]. This if often written in the following, more intuitive form

P(Cε(·) ≈ c(·)

)� exp

(−1

εI (c(·))

), (11)

where ≈ denotes δ-closeness of the paths Cε(·) and c(·), and � asymptotic equality or exponentialequivalence, that is,

ε ln P(Cε(·) ≈ c(·)

)= −I (c(·)) + o(1).

14

The statement says that the probability to find a solution path Cε(·) that is close to the specificpath c(·) is exponentially small in ε with an asymptotic rate given by the rate function I. In analternative way of presenting this, I is the path space measure introduced by the dynamics forsmall ε: Following [7], we can write the probability pε(t0, c0, t1, c1) to go from c0 at time t0 to c1at time t1 using the path integral formalism:

pε(t0, c0, t1, c1) �∫

exp(

− 1ε

I (c(·)))Dc(·),

where Dc(·) denotes integration over all paths c(·) = c(t)|t∈[t0,t1] with c(t0) = c0 and c(t1) = c1.That is, for small ε, the exponential factor exp

(− 1

ε I (c(·)))

works as a probability density in thespace of paths.

In many cases, expressions for the rate function can be found by constructing its pointwise formI : Rn × [0, T ] → [0, ∞], such that the intuitive equation (11) becomes

P(Cε(t) ≈ c

)� exp

(−1

εI(c, t)

)∀t,

meaning ε ln P(Cε(t) ≈ c

)= −I(c, t)+o(1) for all t. While I characterizes the asymptotic behavior

of the probability distribution P induced by the dynamics in the state space Rn, I characterizesthe associated probability distribution in the path space C. In what follows, we will see how I canbe computed for (5) and (8). The construction below yields a form of the rate functions that canalso be found via rigorous construction techniques like the Feng–Kurtz [15] or the Gärtner–Ellis[12] methods, see [31].

4.1.1 Large deviation rate function of the jump process

The solution of the master equation (5) can be represented in exponential form such that

ρε(c, t) � exp(

−1ε

I(c, t))

for a rate function I, see Eq. (20) in the appendix. Using multiscale asymptotics, it is possible toderive a Hamilton–Jacobi equation for I of the form

∂tI(c, t) + H(c, ∇S) = 0

with the Hamiltonian function

H(c, ξ) :=∑

k

αk(c)(

exp(ν>k · ξ) − 1

)(12)

for ξ ∈ Rn. In terms of the Lagrangian

L(c, v) := supξ

[v> · ξ − H(c, ξ)

],

the rate function can be characterized by

I(c(t)|t∈[0,T ]) = I(c0, 0) +∫ T

0L(c(t), c(t))dt (13)

for c(0) = c0. Unfortunately, in general L does not have an explicit form.

15

Mean first exit times

The generator of the Markov jump process Cε(t)|t≥0 underlying the master equation (5) is theinfinitesimal generator of the backward Kolmogorov equation and given by

Gεf(c) := 1ε

∑k

αk(c)[f(c + ενk) − f(c)

].

If τ εc0 denotes the exit time of the Markov jump process from a bounded domain D ⊂ Rn with

boundary ∂D starting in a state c0 ∈ D, the mean first exit time or mean first passage timeηε(c0) = E(τ ε

c0) satisfiesGεηε = −1, (14)

with boundary conditions ηε = 0 on ∂D.When interested in large deviations of the mean first exit time ηε(c) = E(τ ε

c0) from a boundeddomain D after starting in c0 ∈ D, we set

ηε(c0) � exp(1

εφ(c0)

),

for another rate function φ : Rn → [0, ∞], meaning

lim infε→0

ε ln ηε(c0) = lim supε→0

ε ln ηε(c0) = φ (c0) .

We insert this expression into (14) for ηε and repeat the multiscale asymptotics approach (seeappendix) which, in leading order, results in∑

k

αk(c)(

exp(ν>k · ∇φ(c0)) − 1

)= 0.

This can again be expressed via the Hamiltonian defined in (12):

H(c0, ∇φ(c0)) = 0.

That is, the curves H = 0 in the phase portrait of the Hamiltonian system associated with themaster equation determine the rate function φ : Rn → [0, ∞] for the mean first exit time. We willsee how to utilize this insight for computing φ explicitly for a specific example in Sec. 4.2.

Remark 4.1. These results show that the Hamiltonian H is key to characterize both the largedeviation rate function and the (exponentially large) mean first passage time.

4.1.2 Large deviation rate function of the SDE limit system

For the SDE process (8) with the density ρεL(c, t) we consider a rate function IL such that

ρεL(c, t) � exp

(−1

εIL(c, t)

).

16

Again, there is a Hamiltonian function HL, see Eq. (22) in the appendix, such that

∂tIL(c, t) + HL(c, ∇IL) = 0.

Both rate functions I (for the master equation) and IL (for the SDE limit system) have the sameminimum curve, given by the solution of the ODE limit system. Moreover, for small ξ, we have

H = HL + O(‖ξ‖3),

i.e., the associated Hamiltonian of the SDE limit system is the second-order accurate approximationof the Hamiltonian of the master equation around the ODE limit curve.

This time, the associated Lagrangian can be directly computed if the matrix Σ defined in (10)is invertible, see (23) in the appendix, and the large deviation rate function IL on path space ofthe SDE system takes the form

IL(c(t)|t∈[0,T ]

)= 1

2

∫ T

0(c(t) − b(c(t)))>Σ(c(t))−1(c(t) − b(c(t))) dt (15)

for paths c(t)|t∈[0,T ] from H1([0, t], Rd) that start in the initial frequency state c0. We have hereused IL(c0, 0) = 0.

Mean first exit times

The mean first exit timeηε(c0) � exp

(1ε

φL(c0))

of the SDE process from a bounded domain D after starting in c0 ∈ D fulfills GεLηε = −1, where

the generator is given by

GεLf(c) =

n∑i=1

bi(c) ∂

∂cif(c) + ε

2

n∑i,j=1

Σij(c) ∂2

∂ci∂cjf(c).

Using again multiscale asymptotics (see appendix, p. 25) we obtain

b(c)> · ∇φL(c) + 12∇φL(c)> · Σ(c)∇φL(c) = 0,

which can be expressed in terms of the Hamiltonian function by

HL(c, ∇φL(c)) = 0.

4.2 Explicit rate function for specific propensities

We consider the standard setting of Sec. 2.3 with two types S1, S2 and a total propensity function

aij(c) := αij(c) + α′ij(c) = γijcicj + γ′

ijci, (i, j) = (1, 2), (2, 1),

and vectors ν given by

ν12 =(

−11

), ν21 = −ν12.

17

Jump process. Then the Hamiltonian (12) associated with the master equation reads

H(c, ξ) = a12(c)(

exp(ξ2 − ξ1) − 1)

+ a21(c)(

exp(ξ1 − ξ2) − 1),

and the Lagrangian is defined as

L(c, v) = supξ

[v> · ξ + a12(c)

(1 − exp(ξ2 − ξ1)

)+ a21(c)

(1 − exp(ξ1 − ξ2)

)].

In this case, we have the conservation property c1 + c2 = 1 for all solutions of the master equation.This implies for the Lagrangian that

L(c, v) ={

L1(c1, v1), v1 + v2 = 0, c2 = 1 − c1∞, otherwise,

where the reduced Lagrangian is given by (∆ξ := ξ1 − ξ2)

L1(c, v) = sup∆ξ

[∆ξv + a12(c)

(1 − exp(−∆ξ)

)+ a21(c)

(1 − exp(∆ξ)

)],

with c and v now one-dimensional and aij(c) = aij((c, 1 − c)). It can be computed explicitly,resulting in

L1(c, v) = v ln[ 1

2a21(c)(v +√

v2 + 4a12(c)a21(c))]

+ a12(c) + a21(c) −√

v2 + 4a12(c)a21(c).

We find, as to be expected, that L1 = 0 along the ODE limit trajectory, the solution of the reducedODE

v = c = (γ21 − γ12)c(1 − c) + γ′21(1 − c) − γ′

12c. (16)

The rate function for all 1-dimensional paths c(t)|t∈[0,T ] is

I(c(t)|t∈[0,T ]

)= I(c0, 0) +

∫ T

0L1(c(t), c(t))dt.

The reduced Hamiltonian reads

H1(c, ∆ξ) = a12(c)(

exp(−∆ξ) − 1)

+ a21(c)(

exp(∆ξ) − 1).

Its curves H1(c, ∆ξ) = 0 are given by

∆ξ = 0, ∆ξ(c) = ln a12(c)a21(c) . (17)

The phase portrait of H1 and these two curves are shown in Fig. 4a.

18

0.1 0.3 0.5 0.7 0.9

c

−0.2

−0.1

0.0

0.1

0.2

∆ξ

(a) Jump process

0.1 0.3 0.5 0.7 0.9

c

−0.2

−0.1

0.0

0.1

0.2

∆ξ

(b) SDE model

Figure 4: Phase portraits of the reduced Hamiltonian H1 (blue ) with curves H1 = 0 (red, green)and of the reduced Hamiltonian HL,1 with curves HL,1 = 0 for the jump process and the SDEmodel respectively. The arrows indicate the direction of the temporal evolution.

SDE limit model. When we turn to the SDE large deviation rate function for this specific case,we find that

Σ = (a12 + a21)(

1 −1−1 1

)is not positive (it has an eigenvalue λ = 0). Again, this is a consequence of the conservationproperty c1 + c2 = 1, and we have to work with a reduced Lagrangian given by

LL,1(c, v) = max∆ξ

(v∆ξ + (a12(c) − a21(c))∆ξ − 1

2∆ξ2)

= 12(a12(c) − a21(c) + v)2.

Moreover, we again observe that LL,1 = 0 along the trajectory of the reduced ODE limit equation(16).

The reduced Hamiltonian reads

HL,1(c, ∆ξ) = (a21(c) − a12(c))∆ξ + 12(a12(c) + a21(c))∆ξ2,

so that the curves for HL,1(c, ∆ξ) = 0 are given by

∆ξ = 0, ∆ξ(c) = 2a12(c) − a21(c)a12(c) + a21(c) . (18)

The phase portrait of HL,1 and these two curves are shown in Fig. 4b.When comparing Figs. 4a and 4b, we observe that the phase portraits and the curves for

H = 0 of the jump process and the SDE limit dynamics are almost identical. This means that therespective large deviation rates for the mean first exit time are almost identical.

19

Calculation of the rate function. In this simple (one-dimensional) case the large deviationrate function can be computed explicitly. For the rate constants we chose γ12 = 1, γ21 = 1.1 andγ′

12 = 0.03, γ′21 = 0.005 as in Fig. 2. If we are interested in the rate of the mean first exit time

for passing from c0 = 0.1 (meaning 10% of the agents are of type S1) to c1 = 0.9 (meaning 90%of the agents are of type S1), we have to do the following: From c0 = 0.1 to the fixed point atc∗ = 0.72 we can follow the green curve ∆ξ = 0 (which corresponds to the trajectory of the ODElimit system), while from c∗ to c1 we will have to act against the limit dynamics and follow the redcurve in Fig. 4a. Since the system is one-dimensional the rate function reads

φ(c0 → c1) =∫ c∗

c0∆ξdc +

∫ c1

c∗∆ξdc ≈ 0 + 0.0102 = 0.0102,

which results from ∇φ = ∆ξ, see [7]. The value is the same for the jump process and the SDEmodel, using (17) and (18), respectively. This result has to be compared to the empirical rateε ln(ηε(c0 → c1)) that can be calculated from many realizations of the jump and the SDE process,respectively. Fig. 5 shows how the empirical rate converges to the large deviation rate φ(c0 → c1)for ε → 0, that is, N → ∞. In addition, it shows that the empirical rate of the jump and the SDEprocess are almost identical even far from the limit.

Remark 4.2. Along the green curve in Fig. 4a and 4b, where ∆ξ = 0, the temporal derivativeddtc = ∂H

∂∆ξ is positive when we are on the left hand side of c∗ = 0.72 and negative when we are onthe right hand side, while the temporal derivative d

dt∆ξ = −∂H∂c is constant 0. Thus, with the ODE

solution we always end up in the fixed point c∗ no matter from which side we are approaching.Along the red curve, for ∆ξ 6= 0, say positive, the derivatives d

dt∆ξ and ddtc are positive and we

follow the red line upwards to c1. The direction of the temporal evolution is indicated by arrowsin Fig. 4a and 4b.

Remark 4.3. In this example the Hamiltonian’s phase portraits of the jump process and of the SDEprocess are almost indistinguishable so that the resulting mean first exit times are almost identical.However, this is not true in general, since the Hamiltonian associated with the SDE process is onlyidentical with the one of the jump process to second order in ∆ξ. Whenever larger values of ∆ξ arerelevant for the computation, we will see deviations (that lead to exponential deviations in the exittimes). Therefore, the characterization of metastable behavior via the SDE process will in generalnot be enough to understand the metastability of the jump process quantitatively.

5 ConclusionIn this work, we have formulated an agent-based model by means of a continuous-time Markovjump process and analyzed its approximation via stochastic differential equations. It has beendemonstrated that the SDE process not only provides a higher-order approximation of the jumpprocess than the corresponding deterministic limit system given by an ODE, but also preservescharacteristic properties of the jump process such as metastability, which are lost in the ODEdescription. Moreover, we considered rare events, which become more and more unlikely withincreasing population size, and showed that under certain conditions the asymptotic behavior of

20

0 200 400 600 800

Number of agents N

0.0

0.1

0.2

0.3

0.4

0.5 ε ln(ηε(c0 → c1))

φ(c0 → c1)

401 407

0.010

0.016

(a) Jump process

0 200 400 600 800

Number of agents N

0.0

0.1

0.2

0.3

0.4

0.5 ε ln(ηε(c0 → c1))

φ(c0 → c1)

401 407

0.010

0.016

(b) SDE process

Figure 5: Mean time ηε(c0 → c1), respectively the Monte Carlo estimate of its rate ε ln(ηε(c0 → c1))by 103 simulations of the respective process, to pass from c0 = 0.1 to c1 = 0.9 for the jump process(master equation) and the SDE limit process considered in Sec. 4.2 in comparison to the valueφ(c0 → c1) = 0.0102 of the rate function. Parameter values as in Fig. 2. Dashed lines: confidenceinterval for confidence level 0.999.

remote tails of the probability distributions almost match when comparing the jump process andthe SDE process. Based on large deviation theory, we derived equations which characterize therate functions of both processes and observed a convincing agreement when looking at the adaptiveand spontaneous transition dynamics of our standard interaction system.

The numerical effort for stochastic simulations of the Markov jump process scales with the num-ber of interacting agents, which is not the case for the respective SDE process. Therefore, the SDEdescription can be a very efficient modeling approach for interacting agent dynamics, especiallyin the case of a comparatively large agent population, where the root model becomes computa-tionally intensive, but stochastic effects cannot be neglected such that the ODE approximation isinappropriate.

Our analysis is restricted to the setting of complete (possibly weighted) communication net-works with homogeneous transition rates. All three considered model approaches (transition jumpprocess, SDE, ODE) do not distinguish between individual agents, but count the number of agentsof each available type (or rather their concentrations), assuming that every agent may at any timeinteract with all other agents. In times of internet communication and around-the-clock informationavailability, this assumption is not unrealistic for some interaction systems of interest. However,for many applications there clearly arise limitations to the interaction opportunities of agents, asfor example for disease spreading dynamics where the transfer of the disease by infection requiresphysical contact of agents. Here, more detailed models are required, which take the agents’ en-vironment into account or define (time-homogeneous or time-dependent) incomplete interactionnetworks. A question of interest for future investigations is how to transfer the results of this work

21

to such incomplete or inhomogeneous interaction systems and more complex agent-based models.

Funding

This research has been funded by the Deutsche Forschungsgemeinschaft (DFG, German ResearchFoundation) under Germany’s Excellence Strategy MATH+ : The Berlin Mathematics ResearchCenter, EXC-2046/1 project ID: 390685689, and through DFG grant CRC 1114.

Availability of data

The data that support the findings of this study are available from the corresponding author uponreasonable request.

6 Appendix

6.1 Large deviation for the transition jump process

We represent the solution of the rescaled master equation (5) in an exponential form:

ρε(c, t) = 1√ε

exp(

−1ε

Iε(c, t))

, (19)

where the prefactor is required for normalization. Then, obviously,

ρε(c, t) � exp(

−1ε

I(c, t))

, (20)

where, formally, I is the zero-order term of Iε around ε = 0, i.e.,

Iε = I + O(ε).

Using multiscale asymptotics (see [38], Section A.5.1) we find that

√ε

dρε

dt= −1

ε(∂tI)

√ερε + O(1),

and

αk(c − ενk)ρε(c − ενk, t) − αk(c)ρε(c, t) = αk(c)(

exp(ν>k · ∇I(c, t)) − 1

)ρε(c, t) + O(

√ε).

By inserting this into the master equation (5), multiplying it with√

ε, and utilizing√

ερε = O(1),the leading order terms on both sides are the ones in 1/ε and have to be identical. This yields thefollowing Hamilton–Jacobi equation for I:

∂tI(c, t) = −∑

k

αk(c)(

exp(ν>k · ∇I(c, t)) − 1

).

This can be rewritten in the typical form

∂tI(c, t) + H(c, ∇I) = 0,

22

where the Hamiltonian function takes the form

H(c, ξ) :=∑

k

αk(c)(

exp(ν>k · ξ) − 1

).

That is, I is transported along the characteristic curves of the Hamilton–Jacobi equation. Theseare given by the trajectories of the Hamiltonian system associated with H,

ddt

c = ∇ξH =∑

k

νk exp(ν>k · ξ) αk(c)

ddt

ξ = −∇cH = −∑

k

(exp(ν>

k · ξ) − 1)∇αk(c).

The minimum of I is associated with ξ = 0. That is, it is transported along the characteristic curvegiven by

ddt

c =∑

k

νk αk(c).

This means that the solution of the ODE limit system (6) for C(0) = c0, with c0 being the uniquefrequency for which I(·, 0) is minimal, is identical with the curve along which the large deviationrate function I is minimal.

Multiscale asymptotics for the mean first exit time of the jump process

Let ηε(c) = E(τ εc ) denote the mean first exit time of the Markov jump process from a bounded

domain D ⊂ Rn with boundary ∂D starting in a state c ∈ D. We assume ηε(c) to be of the formηε(c) = exp

(1ε φε(c)

)with φε = φ0 + εφ1 + O(ε2), such that

ηε(c) = exp(1

εφ0(c) + φ1(c) + O(ε)

)= exp

(1ε

φ0(c) + φ1(c))

(1 + O(ε)).

Inserting this into Gεηε = −1, see (14), gives

1ε

∑k

αk(c)[

exp(1

εφ0(c + ενk) + φ1(c + ενk)

)− exp

(1ε

φ0(c) + φ1(c)) ]

(1 + O(ε)) = −1. (21)

Now, we write (Taylor expansion)

1ε

φ0(c + ενk) = 1ε

φ0(c) + ν>k · ∇φ0(c) + O(ε)

andφ1(c + ενk) = φ1(c) + O(ε)

and insert both into (21), which gives

1ε

∑k

αk(c) exp(1

εφ0(c) + φ1(c)

) [exp

(ν>

k · ∇φ0(c))

− 1](1 + O(ε)) = −1

23

and thus

1ε

∑k

αk(c)[

exp(ν>

k · ∇φ0(c))

− 1](1 + O(ε)) = − exp

(−1

εφ0(c) − φ1(c)

)= O(1).

Comparing terms of order 1ε yields∑

k

αk(c)[

exp(ν>

k · ∇φ0(c))

− 1]

= 0.

6.2 Large deviation for the SDE process

In analogy to Eq. (19), we set

ρεL(c, t) = 1

εexp

(− 1√

εIε

L(c, t))

and find an equation for the large deviation rate function IL, where IεL = IL + O(ε), via multiscale

asymptotics,∂

dtIL(c, t) = −b(c)> · ∇IL(c, t) − 1

2(∇IL(c, t))>Σ(c)∇IL(c, t),

where we used the Fokker–Planck equation (9). Again, this exhibits the form of a Hamilton–Jacobiequation

∂tIL(c, t) + HL(c, ∇IL(c, t)) = 0,

this time with the Hamiltonian

HL(c, ξ) := b(c)> · ξ + 12ξ>Σ(c)ξ =

∑k

αk(c)(ν>

k · ξ + 12(ν>

k · ξ)2). (22)

The associated Hamiltonian system reads

ddt

c = ∇ξHL = b(c) + Σξ

ddt

ξ = −∇cH = −∑

i

∇cbi(c)ξi − 12ξ>∇cΣ(c)ξ.

Again, the minimum of IL(·, t) is located in ξ(t) = 0, which means that it travels along the trajectory

ddt

c = b(c) =∑

k

αk(c)νk,

that is, on the solution given by the ODE limit system.That is, the two rate functions I (for the master equation) and IL (for the SDE limit system)

have the same minimum curve, given by the solution of the ODE limit system. Moreover, for smallξ, we have

H = HL + O(‖ξ‖3),

24

i.e., the associated Hamiltonian of the SDE limit system is the second-order accurate approximationof the Hamiltonian of the master equation around the ODE limit curve.

The associated Lagrangian

LL(c, v) = supξ

[v> · ξ − HL(c, ξ)

]can be easily computed if Σ is a positive matrix, i.e., if Σ > 0:

LL(c, v) = 12(v − b(c))>Σ−1(v − b(c)). (23)

Multiscale asymptotics for the mean first exit time of the SDE process

The generator of the Kolmogorov backward equation for the SDE process is given by

GεLf(c) =

n∑i=1

bi(c) ∂

∂cif(c) + ε

2

n∑i,j=1

Σij(c) ∂2

∂ci∂cjf(c).

For the mean first exit time ηε(c) we again have GεLηε = −1 (Dynkin’s formula).

Use again ηε(c) = exp(

1ε φ0(c) + φ1(c)

)(1 + O(ε)). Then,

∂

∂ciηε(c) = exp

(1ε

φ0(c) + φ1(c))(1

ε

∂

∂ciφ0(c) + O(1)

)and

∂2

∂ci∂cjηε(c) = exp

(1ε

φ0(c) + φ1(c))( 1

ε2∂2

∂ci∂cjφ0(c) + O

(ε−1

)).

Inserting into GεLηε = −1 and multiplying by exp

(−1

ε φ0(c) − φ1(c))

gives

1ε

n∑i=1

bi(c) ∂

∂ciφ0(c) + 1

2ε

n∑i,j=1

Σij(c) ∂2

∂ci∂cjφ0(c) + O(1) = exp

(−1

εφ0(c) − φ1(c)

)

such that, using exp(−1

ε φ0(c) − φ1(c))

= O(1) and comparing terms of order 1ε , we get

n∑i=1

bi(c) ∂

∂ciφ0(c) + 1

2

n∑i,j=1

Σij(c) ∂2

∂ci∂cjφ0(c) = 0.

This can be written asb(c)> · ∇φ(c) + 1

2∇φ(c)> · Σ(c)∇φ(c) = 0,

which in terms of the Hamiltonian function means

HL(c, ∇φ(c)) = 0.

25

References[1] S. Abar, G. K. Theodoropoulos, P. Lemarinier, and G. M. P. O’Hare. Agent based modelling

and simulation tools: A review of the state-of-art software. Computer Science Review, 24:13– 33, 2017.

[2] D. Aldous. Interacting particle systems as stochastic social dynamics. Bernoulli,19(4):11221149, Sep 2013.

[3] S. Banisch, R. Lima, and T. Araújo. Agent based models and opinion dynamics as Markovchains. Social Networks, 34(4):549–561, 2012.

[4] C. L. Barrett, J. Johnson, and M. V. Marathe. High performance synthetic information envi-ronments – an integrating architecture in the age of pervasive data and computing. Ubiquity.Association for Computing Machinery, 2018.

[5] P. Bolzern, P. Colaneri, and G. De Nicolao. Opinion influence and evolution in social networks:A Markovian agents model. Automatica, 100:219–230, 2019.

[6] J.-P. Bouchaud and J.-P. Nadal. Foreword. Comptes Rendus Physique, 20(4):241 – 243, 2019.

[7] F. Bouchet, K. Gawedzki, and C. Nardini. Perturbative calculation of quasi-potential in non-equilibrium diffusions: A mean-field example. Journal of Statistical Physics, 163(5):1157–1210,Jun 2016.

[8] X. Castelló, V. M. Eguíluz, and M. San Miguel. Ordering dynamics with two non-excludingoptions: bilingualism in language competition. New Journal of Physics, 8:591–646, December2006.

[9] P. Clifford and A. Sudbury. A model for spatial conflict. Biometrika, 60(3):581–588, 12 1973.

[10] N. Conrad Djurdjevac, L. Helfmann, J. Zonker, S. Winkelmann, and C. Schütte. Human mo-bility and innovation spreading in ancient times: a stochastic agent-based simulation approach.EPJ Data Science, 7(1):24, 2018.

[11] A. Dembo and O. Zeitouni. Large deviations techniques and applications, 2010.

[12] R. S. Ellis. Large deviations for a general class of random vectors. Annals of Probability,12.1:1–12, 1984.

[13] S. N. Ethier and T. G. Kurtz. Markov processes: characterization and convergence, volume282. John Wiley & Sons, 2009.

[14] S. Eubank, H. Guclu, V. S. A. Kumar, M. V. Marathe, A. Srinivasan, Z. Toroczkai,and N. Wang. Modelling disease outbreaks in realistic urban social networks. Nature,429(6988):180–184, 2004.

[15] J. Feng and T. G. Kurtz. Large Deviations for Stochastic Processes. AMS, 2006.

26

[16] D. T. Gillespie. A general method for numerically simulating the stochastic time evolution ofcoupled chemical reactions. Journal of computational physics, 22(4):403–434, 1976.

[17] D. T. Gillespie. A rigorous derivation of the chemical master equation. Physica A: StatisticalMechanics and its Applications, 188(1-3):404–425, 1992.

[18] D. T. Gillespie. The chemical langevin equation. The Journal of Chemical Physics,113(1):297–306, 2000.

[19] R. Grima, P. Thomas, and A. V. Straube. How accurate are the nonlinear chemical fokker-planck and chemical langevin equations? The Journal of chemical physics, 135(8):084103,2011.

[20] V. Grimm, U. Berger, D. L. DeAngelis, J. G. Polhill, J. Giske, and S. F. Railsback. The ODDprotocol: A review and first update. Ecological Modelling, 221(23):2760–2768, 2010.

[21] M. Hallier. Formalization and Metastability Analysis of Agent-Based Evolutionary Models.PhD thesis, Freie Universität Berlin, 2014.

[22] R. A. Holley and T. M. Liggett. Ergodic theorems for weakly interacting infinite systems andthe voter model. The Annals of Probability, 3(4):643–663, 1975.

[23] A. Jędrzejewski and K. Sznajd-Weron. Statistical physics of opinion formation: is it a spoof?Comptes Rendus Physique, 20(4):244–261, 2019.

[24] E. Kiesling, M. Günther, C. Stummer, and L. M. Wakolbinger. Agent-based simulation of inno-vation diffusion: A review. Central European Journal of Operations Research, 20(2):183–230,2012.

[25] T. G. Kurtz. Solutions of ordinary differential equations as limits of pure jump Markov pro-cesses. Journal of Applied Probability, 7(1):49–58, 1970.

[26] T. G. Kurtz. The relationship between stochastic and deterministic models for chemical reac-tions. The Journal of Chemical Physics, 57(7):2976–2978, 1972.

[27] T. G. Kurtz. Limit theorems and diffusion approximations for density dependent markovchains. In Stochastic Systems: Modeling, Identification and Optimization, I, pages 67–78.Springer, 1976.

[28] T. G. Kurtz. Strong approximation theorems for density dependent Markov chains. StochasticProcesses and their Applications, 6(3):223–240, 1978.

[29] D. Lane. Models and Aphorisms. SFI Working Paper: 1994-10-058, 1994.

[30] A. K. Misra. A simple mathematical model for the spread of two political parties. NonlinearAnalysis: Modelling and Control, 17(3):343–354, 2012.

[31] A. Montefusco. Dynamic Coarse-Graining via Large-Deviation Theory. PhD thesis, ETHZürich, 2019.

27

[32] G. H. Orcutt. A New Type of Socio-Economic System. The Review of Economics and Statistics,39(2):116–123, May 1957.

[33] S. Redner. Reality-inspired voter models: A mini-review. Comptes Rendus Physique, 20(4),2019.

[34] A. Sîrbu, V. Loreto, V. D. P. Servedio, and F. Tria. Opinion dynamics: models, extensions andexternal effects. In Participatory Sensing, Opinions and Collective Awareness, pages 363–401.Springer, 2017.

[35] S. Venkatramanan, B. Lewis, J. Chen, D. Higdon, A. Vullikanti, and M. V. Marathe. Usingdata-driven agent-based models for forecasting emerging infectious diseases. Epidemics, 22:43– 49, 2018. The RAPIDD Ebola Forecasting Challenge.

[36] T. Warnke, O. Reinhardt, A. Klabunde, F. Willekens, and A. M. Uhrmacher. Modelling andsimulating decision processes of linked lives: An approachbased on concurrent processes andstochastic race. Population Studies, 71:sup1:69–83, 2017.

[37] C. Weimer, J. O. Miller, R. Hill, and D. Hodson. Agent Scheduling in Opinion Dynamics:A Taxonomy and Comparison Using Generalized Models. Journal of Artificial Societies andSocial Simulation, 22(4):5, 2019.

[38] S. Winkelmann and C. Schütte. Stochastic Dynamics in Computational Biology. Springer, toappear 2020.

[39] H. Peyton Young. The Evolution of Conventions. Econometrica, 61(1):57–84, 1993.

28