Accepted Manuscript

A primer on stochastic epidemic models: Formulation, numerical simulation, andanalysis

Linda J.S. Allen

PII: S2468-0427(16)30049-5

DOI: 10.1016/j.idm.2017.03.001

Reference: IDM 20

To appear in: Infectious Disease Modelling

Received Date: 29 December 2016

Revised Date: 5 March 2017

Accepted Date: 7 March 2017

Please cite this article as: Allen L.J.S., A primer on stochastic epidemic models: Formulation, numericalsimulation, and analysis, Infectious Disease Modelling (2017), doi: 10.1016/j.idm.2017.03.001.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service toour customers we are providing this early version of the manuscript. The manuscript will undergocopyediting, typesetting, and review of the resulting proof before it is published in its final form. Pleasenote that during the production process errors may be discovered which could affect the content, and alllegal disclaimers that apply to the journal pertain.

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

A Primer on Stochastic Epidemic Models: Formulation,

Numerical Simulation, and Analysis

Linda J. S. Allen

Department of Mathematics and Statistics, Texas Tech University, Lubbock, TX79409-1042 USA

Abstract

Some mathematical methods for formulation and numerical simulation ofstochastic epidemic models are presented. Specifically, models are formu-lated for continuous-time Markov chains and stochastic differential equa-tions. Some well-known examples are used for illustration such as an SIRepidemic model and a host-vector malaria model. Analytical methods forapproximating the probability of a disease outbreak are also discussed.

Keywords: branching process, continuous-time Markov chain, minoroutbreak, stochastic differential equation2000 MSC: 60H10, 60J28, 92D30

1. Introduction

The intent of this primer is to provide a brief introduction to the formula-tion, numerical simulation, and analysis of stochastic epidemic models for anewcomer to this field. A background in modeling with ordinary differentialequations (ODEs) is assumed. The ODE epidemic models serve as a frame-work for formulating analogous stochastic models and as a source of com-parison with the stochastic models. This primer is restricted to two types ofstochastic settings, continuous-time Markov chains (CTMCs) and stochas-tic differential equations (SDEs). Some well-known examples are used forillustration such as an SIR epidemic model and a host-vector malaria model.For additional examples and information on stochastic epidemic models andstochastic modeling in general, consult the textbooks and papers listed inthe references, e.g., [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14].

Preprint submitted to Infectious Disease Modelling March 5, 2017

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Stochastic modeling of epidemics is important when the number of infec-tious individuals is small or when the variability in transmission, recovery,births, deaths, or the environment impacts the epidemic outcome. The vari-ability associated with individual dynamics such as transmission, recovery,births or deaths is often referred to as demographic variability. The variabil-ity associated with the environment such as conditions related to terrestrialor aquatic settings is referred to as environmental variability. Environmentalvariability is especially important in modeling zoonotic infectious diseases,vector-borne diseases, and waterborne diseases (e.g., Ebola, avian influenza,malaria, and cholera) [15, 16, 17]. In this primer, the emphasis is on demo-graphic variability.

In CTMCs and SDEs, the time variable is continuous, t ∈ [0,∞), butthe state variables are either discrete (CTMC) or continuous (SDEs). Inthe following sections, these two stochastic processes are formulated for thewell-known SIR (Susceptible-Infectious-Recovered) epidemic model and theRoss malaria host-vector model. The Gillespie algorithm and the Euler-Maruyama numerical method are described for the two types of stochasticprocesses. In addition, some analytical methods from branching processesthat are related to the CTMC models are used to approximate the probabilityof an outbreak. In the last section, some stochastic methods for modelingenvironmental variability are presented.

2. SIR Deterministic Epidemic Model

In the SIR deterministic model, S(t), I(t), and R(t) are the number ofsusceptible, infectious, and recovered individuals, respectively. In the sim-plest model, there are no births and deaths, only infection and recovery:

dS

dt= −βI S

NdI

dt= βI

S

N− γI

dR

dt= γI,

(1)

where the total population size is constant, S(t) + I(t) + R(t) = N . Thedisease-free equilibrium is S = N and I = R = 0. The basic reproductionnumber R0 = β/γ which is equal to the ratio of the transmission rate βand the recovery rate γ, determines the epidemic outcome when S(0) ≈ N .

2

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

If I(0) > 0 and R0S(0)/N > 1, then the number of infectious individualsincreases, an outbreak, and if R0S(0)/N < 1, the number of infectious indi-viduals decrease. As R(t) = N − S(t)− I(t), system (1) can be simplified totwo equations for S(t) and I(t).

The stochastic formulation of the CTMC and SDE models requires defin-ing two random variables for S and I whose dynamics depend on the prob-abilities of the two events: infection and recovery. For simplicity, the samenotation is used in the stochastic and the deterministic formulations.

3. SIR Continuous Time Markov Chain

3.1. Formulation

The discrete random variables for the SIR CTMC model satisfy

S(t), I(t) ∈ {0, 1, 2, . . . , N},

where t ∈ [0,∞). The lower case s and i denote the values of the discreterandom variables from the set {0, 1, 2, . . . , N}. The transition probabilitiesassociated with the stochastic process are defined for a small period of time∆t > 0:

p(s,i),(s+k,i+j)(∆t) = P((S(t+∆t), I(t+∆t)) = (s+k, i+j)|(S(t), I(t)) = (s, i)).

The transition probabilities depend on the time between events ∆t but noton the specific time t, a time-homogeneous process. In addition, given thecurrent state of the process at time t, the future state of the process at timet + ∆t, for any ∆t > 0, does not depend on times prior to t, known as theMarkov property. For comparison purposes, the transition probabilities aredefined in terms of the rates in the SIR ODE model:

p(s,i),(s+k,i+j)(∆t) =

βis

N∆t+ o(∆t), (k, j) = (−1,+1)

γi∆t+ o(∆t), (k, j) = (0,−1)

1−(βis

N+ γi

)∆t

+ o(∆t), (k, j) = (0, 0)o(∆t), otherwise.

(2)

Summarized in Table 1 are the changes, ∆S(t) = S(t + ∆t) − S(t) and∆I(t) = I(t + ∆t) − I(t), associated with the two events, infection andrecovery.

3

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Table 1: SIR CTMC Model Assumptions

Event Change (∆S,∆I) Probability

Infection (−1,+1) βis

N∆t+ o(∆t)

Recovery (0,−1) γi∆t+ o(∆t)

Given S(0) = N − i and I(0) = i > 0, the epidemic ends at time t, whenI(t) = 0. The states (S, I), where I = 0 are referred to as absorbing states;the epidemic stops when an absorbing state is reached. The absorbing statesare the states (s, i) with i = 0.

3.2. Kolmogorov Differential Equations

Differential equations for the transition probabilities can be derived from(2). These differential equations are often referred to as the forward or thebackward Kolmogorov differential equations. The forward equations, also re-ferred to as the “master equations”, are used to predict the future dynamics,whereas the backward equations are used to study the end of the epidemic,such as estimating the probability of reaching an absorbing state.

Note that there are (N + 1)(N + 2)/2 ordered pairs of states (s, i), e.g.,(s, i) ∈ {(N, 0), (N − 1, 1), . . . , (0, 0)}, where s + i ≤ N . The general formof the Kolmogorov differential equations can be expressed in a simple form,if the transition probabilities are denoted as pa,b(t), where a and b are twoordered pairs from the set of (N + 1)(N + 2)/2 ordered pairs. The generalform for the forward Kolmogorov differential equations are

dpa,b(t)

dt=∑k 6=a

pa,k(t)qk,b − qa,apa,b(t) (3)

and the backward Kolmogorov differential equations are

dpa,b(t)

dt=∑k 6=a

qa,kpk,b(t)− qa,apa,b(t), (4)

where the values of qk,b, qa,a, and qa,k are defined from the transition rates inequation (2).

In the forward equations, the transition rates depend on the future stateb = (s, i). If a is any state, for the process to be in state b = (s, i) at time

4

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

t+ ∆t, one of the following events occurs: (1) the process transitions from ato (s+ 1, i− 1) in time t and an infection occurs with transition probabilityβ(s+1)(i−1)∆t/N+o(∆t), or (2) the process transitions from a to (s, i+1)in time t and a recovery occurs with transition probability γ(i+1)∆t+o(∆t),or (3) the process transitions from a to (s, i) in time t and and no changeoccurs with probability 1− (βsi/N + γi)∆t+ o(∆t). That is,

pa,(s,i)(t+ ∆t) = pa,(s+1,i−1)(t)β(s+ 1)(i− 1)

N∆t

+pa,(s,i+1)(t)γ(i+ 1)∆t

+pa,(s,i)(t)[1− (βsi/N + γi)∆t] + o(∆t),

where all terms of order ∆t are included in the last term o(∆t). Subtractingpa,(s,i)(t) from both sides, dividing by ∆t, and letting ∆t → 0, leads to theforward Kolmogorov differential equations:

dpa,(s,i)(t)

dt= pa,(s+1,i−1)(t)

β

N(s+ 1)(i− 1) + pa,(s,i+1)(t)γ(i+ 1)

− pa,(s,i)(t)[β

Nsi+ γi

].

A similar derivation applies to the backward equations. However, thetransition rates depend on the initial state a = (s, i). A transition occurs intime ∆t, either infection, recovery or no change, and in the remaining timet, there is a transition to state b. The backward equations (4) are

dp(s,i),b(t)

dt=

βsi

Np(s−1,i+1),b(t) + γip(s,i−1),b(t)

−[β

Nsi+ γi

]p(s,i),b(t).

For a more thorough derivation of these equations, consult the references,e.g., [3, 13, 18].

If the (N + 1)(N + 2)/2 ordered pairs (s, i) are labeled in a specific orderfrom k = 1 to k = (N+1)(N+2)/2, then a matrix of transition rates Q can bedefined. Matrix Q has dimension (N+1)(N+2)/2×(N+1)(N+2)/2 and isknown as the infinitesimal generator matrix. Matrix Q is straightforward todefine for a single random variable whose states are already linearly orderedfrom 0 to N (e.g., [3, 13, 14, 18]). But for a bivariate process with states (s, i)

5

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

the form of matrix Q depends on how the set of ordered pairs are linearlyordered. In general, matrix Q has negative diagonal entries and nonnegativeoff-diagonal entries. In addition, matrix Q has the property that the rowsums are zero. It follows from equations (3) and (4) that the forward andthe backward Kolmogorov differential equations can be written as systemsof matrix differential equations, dP (t)/dt = P (t)Q and dP (t)/dt = QP (t),respectively, where P (t) is the matrix of transition probabilities. Matrix P (t)has diagonal entries pa,a(t), where a is one of the linearly ordered states.The formal solution of these equations is P (t) = eQt, where P (0) = I isthe identity matrix. Note that sometimes the transition probabilities arewritten in the reverse order, p(s+k,i+j),(s,i)(t) = P((S(t), I(t)) = (s + k, i +j)|(S(0), I(0)) = (s, i)). In this case, the transpose of these equations isapplied with matrix QT instead of Q [3, 4].

3.3. Branching Process Approximation

In this brief introduction, we study the stochastic behavior near thedisease-free equilibrium to determine whether an epidemic (major outbreak)occurs when a few infectious individuals are introduced into the popula-tion. The probability of no major outbreak (a minor outbreak) for theCTMC model near the disease-free equilibrium is approximated by applyingbranching process theory and techniques from probability generating func-tions (pgfs). Most important is the fact that when I hits zero, it stays instate zero. The state I has reached an absorbing state and disease transmis-sion stops. In the limit, of course, the infectious individuals in the stochasticand deterministic models always approach zero, I(t) → 0 as t → ∞. Butfinite time extinction of I occurs in the stochastic model. We are interestedin the stochastic dynamics at the initiation of an epidemic, when almost ev-eryone in the population is susceptible. (The duration of the epidemic, onceinitiated, is another interesting stochastic problem, not considered here, seee.g., [8, 19].)

The branching process is the linear approximation of the SIR stochasticprocess near the disease-free equilibrium. For a few initial infectious individ-uals, the branching process either grows exponentially or hits zero. Thesetwo phenomena are captured in the branching process approximation of theCTMC model near the disease-free equilibrium. If the number of infectiousindividuals increases substantially to a large number of cases, then there isa major outbreak. However, if there are only a few additional cases, abovethe initial number of cases, then there is a minor outbreak. The branching

6

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

process is a good approximation of the CTMC model, if the susceptible pop-ulation size is sufficiently large. Then the two outcomes, either a major orminor outbreak, are clearly distinguishable.

The branching process is a birth and death process for I; the variables Sand R are not considered in this approximation. The term βi is the infectionrate (birth) and γi is the recovery rate (death). The process begins withjust a few infectious individuals. The branching process approximation is aCTMC, but near the disease-free equilibrium, the rates are linear (Table 2).

Table 2: Branching Process Approximation of the SIR CTMC Model Nearthe Disease-Free Equilibrium

Change ∆I Probability+1 βi∆t+ o(∆t)−1 γi∆t+ o(∆t)

Three important assumptions underlie the branching process approxima-tion:

(1) Each infectious individual behavior is independent from other infectiousindividuals.

(2) Each infectious individual has the same probability of recovery and thesame probability of transmitting an infection.

(3) The susceptible population is sufficiently large.

Assumption (1) is reasonable if a small number of infectious individuals isintroduced into a large homogeneously-mixed population (assumption (3)).Assumption (2) is also reasonable in a homogeneously-mixed population withconstant transmission and recovery rates, β and γ.

Two probability generating functions (pgfs) are used in the study of theprobability of extinction. The first one applies to each infectious individual,known as the offspring pgf, and the second one applies to the entire infectiousclass I(t) at time t. For our purposes, the offspring pgf is the most importantone. In general, an offspring pgf has the form:

f(u) =∞∑j=0

pjuj, u ∈ [0, 1],

7

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

where pj is the probability of one individual generating j new individuals ofthe same type, e.g., one infectious individual generates j infectious individu-als. The pgf has some properties that are useful in the analysis. For example,f(1) = 1 and the “mean” number of offspring generated from one individualis defined as

f ′(1) =∞∑j=1

jpj.

The offspring pgf for one infectious individual, I(0) = 1, is defined fromTable 2:

f(u) =γ

β + γ+

β

β + γu2, u ∈ [0, 1]. (5)

The first term in f(u) is the probability that an infectious individual recoversand the coefficient of the second term is the probability that an infectiousindividual infects another individual. The power to which u is raised is thenumber of infectious individuals generated from one infectious individual. Ifan individual recovers, then no new infections are generated (u0) and if theinfection is transmitted to another individual, there are now two individualsinfectious (u2). This offspring pgf differs from a discrete-time branchingprocess, where the “parent” dies and is replaced by the offspring in the nextgeneration. The difference is due to the fact that in a small period of time,in a continuous-time process, the infectious individual that infects anotherperson is counted as still being infectious (two infectious individuals). Thisinfectious individual has the same probability of infecting another individualβ/(β + γ) and the same probability of recovery, γ/(β + γ).

The offspring pgf (5) satisfies f(1) = 1 and the mean value is

f ′(1) =2β

β + γ.

This latter expression is not the same as the basic reproduction number, theaverage number of infectious individuals generated by one infectious individ-ual during the period of infectivity. However, f ′(1) is a threshold parameter,similar to R0. In particular, f ′(1) > 1 if and only if R0 > 1.

It is well-known from the theory of branching processes that a fixedpoint of the offspring pgf yields the asymptotic probability of extinction[12, 20, 21, 22]. It is shown in Appendix B that the fixed points of f are thestationary solutions (time-independent solutions) of the branching processapproximation for the probability of extinction of the infectious class I(t).

8

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Solving for the fixed points of f in (5), f(u) = u for u ∈ [0, 1], yields twosolutions, namely, u = 1 (f(1) = 1), and u = γ/β = 1/R0 (if R0 > 1). Itis shown in Appendix B that if R0 < 1, then the fixed point u = 1 is stableand if R0 > 1, the fixed point u = 1/R0 is stable. When I(0) = i > 1, theassumption of independence of infectious individuals, implies the probabilityof no outbreak is either 1 or (1/R0)

i.The preceding results were first applied to the stochastic SIR epidemic

model by Whittle in 1955 [23]. More precisely, he used the terminology, “theprobability of a minor outbreak” is (1/R0)

i and “the probability of a majoroutbreak” is 1− (1/R0)

i. As these estimates for extinction (no outbreak) areasymptotic approximations from the branching process, they are more accu-rate for a large susceptible population size N and a few infectious individuals.The results are summarized below:

Pminor outbreak =

(

1

R0

)i, R0 > 1

1, R0 < 1,

Pmajor outbreak =

1−(

1

R0

)i, R0 > 1

0, R0 < 1.

3.4. Numerical Simulation

In general, for multivariate processes, it is difficult to find analytical so-lutions for the transition probabilities from the forward and backward Kol-mogorov differential equations. For multivariate processes, it is often simplerto numerically simulate stochastic realizations (sample paths) of the process.A numerical method for simulation of CTMC models was developed by Gille-spie [24]. This method is known as the Gillespie algorithm or the StochasticSimulation algorithm. To numerically simulate the change in state, two uni-form random numbers, u1, u2 ∈ U [0, 1], are required for each change, one forthe interevent time and a second one for the particular event.

The Markov property implies that the interevent time T has an exponen-tial distribution T ∼ λe−λt, where the parameter λ is the sum of the ratesfor all possible events. For the SIR CTMC model,

λ = βsi/N + γi,

9

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

where (s, i) is the particular value for the state of (S(t), I(t)) at a giventime t. To compute a value for the interevent time τ from the exponentialdistribution, the first uniform random number u1 yields

τ = − lnu1λ

. (6)

(See Appendix A for the derivation of the interevent time.) The secondrandom number u2 tells which particular event occurs. In general, given nevents, the interval [0, 1] is subdivided according to the probability of eachevent, [0, p1], (p1, p1 + p2], . . . , (p1 + · · · pn−1, 1],

∑ni=1 pi = 1. If u2 lies in the

kth subinterval, then the kth event occurs. For the SIR CTMC model, thereare only two events with corresponding probabilities

p1 =βsi/N

βsi/N + γiand p2 =

γi

βsi/N + γi.

If i = 0, an absorbing state has been reached and the process stops.Four sample paths of the SIR CTMC model are plotted in Figure 1 for

N = 100 (top two graphs) andN = 750 (bottom two graphs). The close-up ofthe dynamics on the right side illustrates exponential growth at the initiationof an outbreak, the region where the branching process approximation isapplicable.

4. SIR Stochastic Differential Equations

4.1. Formulation

Stochastic differential equations for the SIR epidemic model follow froma diffusion process. The random variables are continuous,

S(t), I(t) ∈ [0, N ].

Forward and backward Kolmogorov partial differential equations for the tran-sition probability density functions can be derived and they in turn leaddirectly to the SDEs, e.g., [1, 4, 25, 26, 27, 28]. The SDEs are useful insimulating sample paths of the continuous-state process . In addition, theSDEs are much easier to solve numerically than the Kolmogorov differentialequations and faster than simulating sample paths of the CTMC model. Aheuristic derivation of the SDEs corresponding to the SIR epidemic model isdescribed below.

10

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

0 20 40 600

20

40

t

I(t)

0 5 10 150

5

10

15

I(t)

t

0 50 1000

50

100

150

t

I(t)

0 5 10 150

5

10

15

I(t)

t

Figure 1: The dashed curve is the ODE solution of I(t) in the SIR model andthe other curves are four sample paths of the SIR CTMC model. Parametervalues are β = 0.3, γ = 0.15, and either N = 100 (top graphs) or N = 750(bottom graphs). The initial conditions are S(0) = 98 and I(0) = 2. Thegraphs on the right are a close-up view on the time interval [0, 15] of thegraphs on the left. The value P = 0.25 is the estimate for the probability ofa minor outbreak. On average, one sample path out of four hits zero, givenR0 = 2 and I(0) = 2.

11

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Divide the time interval [0, t] into small subintervals of length ∆t. Let∆X(t) = (∆S(t),∆I(t))T . Subdivide ∆t further into smaller subintervals oflength ∆ti = ti − ti−1, i = 1, . . . , n with t0 = t, tn = t+ ∆t, and

∑ni=1 ∆ti =

∆t.

∆X(t) =n∑i=1

∆X(ti).

For ∆ti sufficiently small, it is reasonable to assume that the random variables{∆X(ti)} on the interval ∆t are independent and identically distributed. Forn sufficiently large, the Central Limit Theorem implies that ∆X(t) has anapproximate normal distribution with mean E(∆X(t)) and covariance matrixCV (∆X(t)), e.g., [1, 25, 26, 27, 28]. Thus,

∆X(t)− E(∆X(t)) ≈ Normal(0, CV (∆X(t)),

where 0 is the zero vector. The expectation of ∆X to order ∆t is the changethat occurs (+1 or −1) times the probability:

E(∆X) ≈(−βSI/N

βSI/N − γI

)∆t = f∆t

and the covariance matrix of ∆X to order ∆t is

CV (∆X) ≈ E(∆X(∆X)T ) = E(

(∆S)2 ∆S∆I∆S∆I (∆I)2

)=

(βSI/N −βSI/N−βSI/N βSI/N + γI

)∆t = C∆t.

To write the SDEs for the SIR stochastic process, either the square rootof the covariance matrix C∆t is required, or alternately, a matrix G so thatGGT = C [1, 25]. The following matrix G has this latter property (G isnot unique) [1, 25]. Matrix G is straightforward to compute as each columnrepresents the square root of the rates as given in Table 1,

G =

(−√βSI/N 0√βSI/N −

√γI

).

Then ∆X(t) ≈ f(X(t))∆t+G(X(t))∆W (t), where ∆W (t) = (∆W1(t),∆W2(t))T

and ∆Wi(t) ∼ Normal(0,∆t). Letting ∆t→ 0, leads to the following systemof SDEs:

dX(t) = f(X(t))dt+G(X(t))dW (t),

12

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

where W (t) = (W1(t),W2(t))T is a vector of two independent Wiener pro-

cesses. That is, Wi(t) ∼ Normal(0, t) is a normally distributed random vari-able with mean zero and variance t or dWi(t) ∼ Normal(0, dt). The notationB(t) = W (t) is also employed, where B(t) denotes Brownian motion, e.g.,[29]. This stochastic differential equation is known as an Ito SDE becausethe right side is evaluated at time t [1, 25].

Rewriting the expression dX(t) in terms of the random variables S(t) andI(t) leads to the following system of Ito SDEs:

dS(t) = −[βS(t)I(t)/N ] dt−√βS(t)I(t)/N dW1(t)

dI(t) = [βS(t)I(t)/N − γI(t)] dt+√βS(t)I(t)/N dW1(t)

−√γI(t) dW2(t).

(7)

Note that if I(t) = 0, then the epidemic stops. The Ito SDEs reduce to theoriginal ODE model (1), if the two Wiener processes are neglected.

4.2. Numerical Simulation

The Euler-Maruyama method is a simple numerical method that canbe used to simulate sample paths of SDEs. The Euler-Maruyama methodis of order ∆t and follows directly from the derivation of the SDE, where∆W (t) = W (t + ∆t) −W (t) ∼ Normal(0,∆t). In general, for a system ofSDEs of the form,

dX(t) = f(X(t), t) dt+G(X(t), t) dW (t),

the Euler-Maruyama method is a finite-difference approximation,

X(t+ ∆t) = X(t) + f(X(t), t)∆t+G(X(t), t)η√

∆t,

t = 0,∆t, 2∆t, . . ., where ∆t is chosen sufficiently small to ensure conver-gence. For k independent Wiener processes W (t) = (W1(t), . . . ,Wk(t))

T , thevector η = (η1, . . . , ηk) is k independent standard normal random numbersηi ∈ Normal(0, 1). Sample paths of the SIR SDE model are illustrated in Fig-ure 2. The sample paths are continuous, but not differentiable (a propertyof the Wiener process).

Numerical simulation of sample paths for SDE models is faster and sim-pler than computing sample paths for CTMC models when the populationsize is large. The time step in SDE models is chosen small but it has a fixed

13

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

length, whereas in CTMC models the interevent time τ must be computedfor each change ∆X and the interevent time decreases as the population sizeincreases (see equation (6)). Numerical methods with greater accuracy thanthe Euler-Maruyama method are discussed in the references, e.g., [30, 31].In addition, methods have been developed to speed up the stochastic simu-lation (see, e.g., StochSS: Stochastic Simulation Service, www.StochSS.org,Petzold, UC Santa Barbara).

0 20 40 600

5

10

15

20

25

30

35

40

t

I(t)

0 50 1000

50

100

150

t

I(t)

Figure 2: The dashed curve is the ODE solution of I(t) in the SIR modeland the other solid curves are four sample paths of the SIR SDE model.Parameter values and initial conditions are the same as for the CTMC modelin Figure 1 for N = 100 (left) and N = 750 (right).

5. Malaria Deterministic Model

Malaria infection is caused by a Plasmodium parasite. An infectiousmosquito transmits the parasite to a susceptible host through a mosquitobite. It is the female mosquito that bites the host to acquire blood forreproduction. According to the website of the World Health Organization[32] there were approximately 214 million new cases of malaria and 438,000deaths worldwide in 2015. Most cases were reported in the African region.Sir Ronald Ross was one of the first scientists to formulate mathematicalmodels for the spread of malaria between insect vectors and human hosts[33, 34, 35] and for his work on malaria he was awarded the Nobel Prize inPhysiology or Medicine in 1902.

14

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

In the Ross malaria model, the total number of vectors M and hosts H areconstant. The variables S and I are the number of susceptible and infectioushosts, respectively, and U and V are the number of healthy and infectiousvectors, respectively. A female mosquito requires a certain number of bloodmeals for reproduction and it is assumed that a single mosquito takes k bitesper unit time to fulfill this blood requirement. Another important assumptionis that the total number of bites by the mosquito population is dependenton the total number of mosquitoes but it is not dependent on the numberof human hosts (only the proportion of human hosts). The probability perbite that an infectious mosquito transmits malaria is p and the probabilityper bite that a healthy mosquito acquires infection is q. Parameter α = kpis the transmission rate from an infectious mosquito to a human and β = kqis the acquisition rate from an infectious host to a healthy mosquito. Thehost recovery rate is γ and the mosquito death rate is µ. The birth rateand death rate of the mosquito population are equal. The natural birth anddeath rates of humans are negligible with respect to the modeling time frameand are assumed to be zero. With the preceding assumptions, the malariamodel takes the form:

dS

dt= −αV S

H+ γI

dI

dt= αV

S

H− γI

dU

dt= −βU I

H+ µM − µU

dV

dt= βU

I

H− µV

(8)

The disease-free equilibrium for this model is S = H, U = M and I =0 = V . Linearization of the differential equations for I and V about thedisease-free equilibrium yields the matrices from the next generation matrixapproach [36]:

F − V =

(0 α

βM

H0

)−(γ 00 µ

). (9)

The spectral radius of FV −1 is often defined as the basic reproduction number[36]:

ρ(FV −1) =

√αβ(M/H)

µγ.

15

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

An equivalent form in terms of the threshold of one, is defined as the productof the transmission from vector to host and from host to vector:

R0 =

(α

µ

)(β(M/H)

γ

)(10)

(e.g., [37, 38]). This latter expression is used in the following discussion ofthe basic reproduction number. For model (8), the disease dies out if R0 < 1and a stable endemic equilibrium exists if R0 > 1.

6. Malaria Continuous Time Markov Chain

6.1. Formulation

The malaria CTMC model is a time-homogeneous process with the Markovproperty. There are six events corresponding to transmission from host tovector and vector to host, recovery of humans, death of healthy and infec-tious mosquitoes, and birth of mosquitoes. Table 3 is a summary of theseevents and their probabilities.

Table 3: Ross Malaria CTMC Model Assumptions

(∆S,∆I) Probability (∆U,∆V ) Probability(+1,−1) γi∆t+ o(∆t) (−1, 1) βu(i/H)∆t+ o(∆t)(−1,+1) αv(s/H)∆t+ o(∆t) (+1, 0) µM∆t+ o(∆t)

(−1, 0) µu∆t+ o(∆t)(0,−1) µv∆t+ o(∆t)

In the malaria CTMC model, given (S(t), I(t), U(t), V (t)) = (s, i, u, v),the exponential distribution for the interevent time T ∼ λe−λt has parameter

λ = γi+ αv(s/H) + βu(i/H) + µ(M + u+ v).

A sample path of the CTMC model is graphed in Figure 3. A stable endemicequilibrium exists in the ODE model if R0 > 1 and the sample path of theCTMC model fluctuates around this endemic equilibrium.

16

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

0 500

50

100

CTMC

S, I

SI

0 500

500

1000

CTMC

U, V

UV

0 500

50

100

SDE

t

S, I

SI

0 500

500

1000

SDE

t

U, V

UV

Figure 3: The smooth curve in each figure is the solution of the host-vectorODE model. The other curves are one sample path of the CTMC model (toptwo figures) and one sample path of the SDE model (bottom two figures). Theparameters are H = 100, M = 1000, α = 0.2, γ = 0.1, β = 0.2, and µ = 0.5.Initial conditions are I(0) = 0, V (0) = 10, S(0) = 100 and U(0) = 990. Thevalue of R0 = 8 with q1 = 1/6 and q2 = 3/4. The probability of extinctionin the CTMC is approximately P ≈ 0.06. The stable endemic equilibrium inthe ODE model is (S, I , U , V ) = (16.7, 83.3, 750, 250).

17

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

6.2. Branching Process Approximation

Approximation of the CTMC near the disease-free equilibrium, leads toa multitype branching process in two variables I(t) and V (t). Table 4 is asummary of the changes in I and V near the disease-free equilibrium whenS ≈ H and U ≈M .

Table 4: Branching Process Approximation of the Malaria Model Near theDisease-Free Equilibrium

Change (∆I,∆V ) Probability(+1, 0) αv∆t+ o(∆t)(0,+1) βi(M/H)∆t+ o(∆t)(−1, 0) γi∆t+ o(∆t)(0,−1) µv∆t+ o(∆t)

Next, two offspring pgfs are defined, one for a host and one for a mosquito.Each offspring pgf has the general form:∑

j,k

P((I, V ) = (j, k))uj1uk2, ui ∈ [0, 1], i = 1, 2.

For one infectious host, there are only two events, either recovery of thehost or infection of a mosquito. The offspring pgf for I, given I(0) = 1 andV (0) = 0 is

f1(u1, u2) =γ

γ + β(M/H)+

β(M/H)

γ + β(M/H)u1u2. (11)

That is, one infectious host recovers with probability γ/(γ + β(M/H)) orinfects a mosquito with probability β(M/H)/(γ + β(M/H)). Note the termu1u2 in (11) means one infectious host generates one infectious mosquito (u2raised to the power one) and remains infectious (u1 raised to the power one).The offspring pgf for V , given V (0) = 1 and I(0) = 0, is

f2(u1, u2) =µ

µ+ α+

α

µ+ αu1u2. (12)

That is, one infectious mosquito dies with probability µ/(µ+ α) or infects ahost with probability α/(µ+ α).

18

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

To find the probability of extinction (no outbreak), we compute the fixedpoints of the system on the unit square [0, 1] × [0, 1], f1(u1, u2) = u1 andf2(u1, u2) = u2, e.g., [20, 21, 22, 39]. The two solutions of this system are(1,1) and (q1, q2), where

q1 =γ

γ + β(M/H)+

β(M/H)

γ + β(M/H)

(1

R0

)(13)

q2 =µ

µ+ α+

α

µ+ α

(1

R0

)(14)

(e.g., [40]). The fixed point (q1, q2) exists in the unit square provided R0 > 1.Equivalent expressions were derived by Bartlett in 1964, but they were notwritten in terms of R0 [41]. Another equivalent set of expressions for q1and q2 were derived by Lloyd et al. assuming geometric offspring pgfs [38].The expression for q1 in (13) has a biological interpretation. Beginning fromone infectious host, there is no outbreak if the infectious host recovers withprobability γ/(γ + β(M/H)) or if there is no successful transmission to asusceptible mosquito with probability β(M/H)/(γ + β(M/H))[1/R0].

The stability of the fixed points is determined from the spectral radiusof the Jacobian matrix of the offspring pgfs, evaluated at (1, 1), e.g., [20, 21,22, 39]. If the spectral radius is less than one, then (1, 1) is stable and if it isgreater than one, (1,1) is unstable and (q1, q2) is stable. (See Appendix C.)The Jacobian matrix of the offspring pgfs evaluated at (1, 1) is

M =

βM/H

γ + βM/H

βM/H

γ + βM/Hα

µ+ α

α

µ+ α

.

In general, it can be shown that

W(M− I) = F − V, (15)

where F and V are computed from the next generation matrix approach,equation (9), W is a diagonal matrix of interevent time parameters, W =diag(γ + βM/H, µ+ α), and I is the identity matrix [39]. In addition, thereis an important relation between R0 and matrix M:

R0 > 1 if and only if ρ(M) > 1

19

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

which follows from the identity (15) [39].Therefore, the probability of extinction (no outbreak) is one, if R0 < 1,

but less than one, if R0 > 1. Given I(0) = i and V (0) = v, it follows fromthe independent assumption in the branching process approximation that theprobabilities of a major or a minor outbreak are:

Pminor outbreak =

{qi1q

v2 , R0 > 1

1, R0 < 1,Pmajor outbreak =

{1− qi1qv2 , R0 > 1

0, R0 < 1.

Alternate forms for the offspring pgf have been proposed that differ fromthe assumptions in the underlying SIR CTMC epidemic model (Table 1) or inthe host-vector CTMC model (Table 3), e.g., [38, 42, 43, 44]. Offspring pgfshave been assumed to have geometric, Poisson, gamma, or negative binomialdistributions [38, 42, 43, 44]. But they are often applied in the discrete-timecase. The geometric distribution and the forms in (5), (11), and (12) preservethe Markov property of an exponentially distributed lifetime [21]. Branchingprocess approximations can be extended to more complex epidemic modelswith multiple groups [39, 40].

7. Malaria Stochastic Differential Equations

A derivation of the SDEs for the host-vector model follows in a similarmanner as for the SIR model. For simplicity, denote the change of the systemvariables as ∆X = (∆S,∆I,∆U,∆V )T and the right side of the host-vectorODE model as gi, i = 1, 2, 3, 4, i.e., dS/dt = g1. Then the Ito SDEs forthe malaria model are computed from the expectation and covariance of∆X based on the six events in Table 3. The system of SDEs has the formdX(t) = f(X(t)) dt + G(X(t)) dW (t), where G is a 4 × 6 matrix satisfyingGGT = C, where to order ∆t, C∆t is the approximate covariance matrix,

G =

−√αV S/H

√γI 0 0 0 0√

αV S/H −√γI 0 0 0 0

0 0 −√βUI/H

√µM −

õU 0

0 0√βUI/H 0 0 −

õV

,

and W (t) is a vector of six independent Wiener processes, corresponding tothe six events represented in Table 3 [1, 25]. More explicitly, the system of

20

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Ito SDEs for the host-vector model is

dS = g1 dt−√αV S/H dW1 +

√γI dW2

dI = g2 dt+√αV S/H dW1 −

√γI dW2

dU = g3 dt−√βUI/H dW3 +

√µM dW4 −

õUdW5

dV = g4 dt+√βUI/H dW3 −

õV dW6,

where the dependence on time t is omitted. For I(t) = 0 and V (t) = 0, theepidemic stops.

The Euler-Maruyama method is used to compute sample paths of thehost-vector SDE model (see Figure 3) and to approximate the probabilitydensity for large time (see probability histogram in Figure 4). The probabilitydensity appears to be approximately stationary as solutions fluctuate nearthe endemic equilibrium of the ODE model. How much fluctuation occurs inthe disease dynamics depends on the transmission processes (demographicvariability). In a more realistic host-vector model, the disease dynamicsare also affected by environmental conditions. Environmental variability isespecially important in a vector-borne disease, where the vector life cycle isintimately connected to temperature and moisture levels.

8. Environmental Variability

For the SIR epidemic model or the malaria host-vector model, changesin the environment may impact the parameters for birth, death, recovery, ortransmission. For example, if birth, death, or transmission rates fluctuatewith changes in the environmental conditions, then a stochastic differentialequation for the model parameter can be formulated as a mean-revertingprocess (fluctuation about some average value) [45].

For example, a mean-reverting Ornstein-Uhlenbeck (OU) process for theparameter α is modeled by the SDE:

dα(t) = r(α− α(t)) dt+ σdW (t),

where α is the mean value and σ2 is proportional to the variance. In partic-ular, for the OU process, α(t) has a normal distribution with mean

E(α(t)) = α + (α(0)− α)e−rt

21

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

60 80 1000

200

400

600

800

1000

I

Prob ×

104

200 250 3000

50

100

150

200

250

V

Prob ×

104

Figure 4: The two histograms (computed from 10,000 sample paths of thehost-vector SDE model) approximate the probability density for I and Vat t = 50. Parameter values are the same as in Figure 3 but with initialconditions sufficiently large such that the probability of hitting zero is closeto zero. The computed mean values and standard deviations are µI = 83.2,µV = 249.6, σI = 3.94, and σV = 17.8. The mean values are close tothe endemic equilibrium values of the host-vector ODE model: (I , V ) =(83.3, 250).

22

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

and variance

V ar(α(t)) =σ2

2r(1− exp(−2rt)) (16)

[29, 45]. In the limit, as t→∞, the mean is α and variance is σ2/(2r). Themean agrees with the solution of the ODE: dα(t)/dt = r(α− α(t)).

In another example, if the carrying capacity K ≡ K(t) varies both sea-sonally and randomly, a biologically reasonable model for K(t) has the form[46]:

dK(t) = r(K −K(t)) dt+ r cos(ωt) dt+ σ dW (t). (17)

The mean of K is

E(K(t)) = K +

[K(0)− K − rr

ω2 + r2

]e−rt +

r

ω2 + r2[ω sin(ωt) + r cos(ωt)]

and variance has the same expression as in (16). (See Figure 5.) The pa-rameter K can also be modeled as a discrete random variable. Other mean-reverting stochastic processes that have the additional desirable property ofbeing nonnegative are discussed in [45].

50 100 150 200 250 300 35070

80

90

100

110

120

130

Days

K(t)

Figure 5: The smooth curve is the ODE solution of K(t) (Wiener processneglected in (17)) and the random curve is one sample path of the SDE model(17). Parameter values are K = 100, r = 0.3, r = 4, ω = 2π/365, and σ = 5.The initial condition is K(0) = 110.

23

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

9. Summary

The intent of this primer is to introduce the topic of stochastic epidemicmodeling from the perspective of an ODE epidemic framework. The empha-sis is on continuous-time Markov chains and stochastic differential equations.Many topics of relevance to stochastic epidemic modeling have been omit-ted, e.g., epidemic duration, discrete-time Markov chains, stochastic spatialmodels, and parameter estimation for stochastic models. The references pro-vide a wealth of information on these and other topics in stochastic epidemicmodeling.

Appendix A. Interevent Time in the CTMC Model

Let F (t) = 1 − e−λt = P(T ≤ t) be the cumulative distribution for theinterevent time T . Let U denote the uniform distribution on [0, 1]. Propertiesof U and the fact that F is strictly increasing on [0,∞), yields

F (t) = P(U ≤ F (t)) = P(F−1(U) ≤ t).

But this implies T = F−1(U). Calculation of the inverse function and usingthe fact that 1 − U has the same distribution as U leads to the formula forτ in equation (6):

F−1(U) = − ln(1− U)

λ= − ln(U)

λ.

Appendix B. PGF for the SIR Branching Approximation

For the SIR CTMC model, the pgf Gi for the infectious class I(t), givenI(0) = i, is defined in terms of the transition probabilities, pi,j(t) = P(I(t) =j|I(0) = i):

Gi(u, t) = E(uI(t)|I(0) = i) =∞∑j=0

pi,j(t)uj = pi,0(t)+pi,1(t)u+pi,2(t)u

2 + · · · .

(B.1)Note that Gi(0, t) = pi,0(t) is the probability of disease extinction by time t:I(t) = 0, given I(0) = i. From the independent assumptions, it follows thatGi is the product of i generating functions for I(0) = 1:

Gi(u, t) = [G1(u, t)]i. (B.2)

24

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

The backward Kolmogorov differential equations for the branching processapproximation of I(t) are as follows:

dpi,kdt

= βipi+1,k + γipi−1,k − (β + γ)ipi,k. (B.3)

A differential equation is derived for p1,0(t) based on the identities (B.1)and (B.2), and the backward equations (B.3). Differentiate (B.2) with respectto t, solve for ∂G1/∂t, then replace ∂Gi/∂t with an equivalent expression fromdefinition (B.1):

∂G1

∂t=

1

i(G1)i−1∂Gi

∂t=

1

iGi−1

(∞∑k=0

dpi,kdt

uk

).

Substitution of dpi,k/dt from the backward Kolmogorov differential equations(B.3) leads to

∂G1

∂t=

1

Gi−1

∞∑k=0

[βpi+1,k + γpi−1,k − (β + γ)pi,k]uk

= βGi+1

Gi−1+ γ

Gi−1

Gi−1− (β + γ)

Gi

Gi−1

= (β + γ)

[β

β + γ(G1)

2 +γ

β + γ−G1

]= (β + γ) [f(G1)−G1] ,

where the offspring pgf f is defined in (5). Since the right side of the differ-ential equation is independent of u, the differential equation also applies toG1(0, t) = p1,0(t). That is,

dp1,0(t)

dt= (β + γ) [f(p1,0(t))− p1,0(t)] , p1,0(t) ∈ [0, 1]. (B.4)

From the theory of autonomous differential equations, the steady-states of(B.4) are the stationary solutions of the process: solutions u of f(u) = u.From the offspring pgf in (5), there are two possible steady-state solutions,either u = 1 or u = γ/β = 1/R0. It is easy to check if R0 < 1, then u = 1is the stable steady-state and if R0 > 1, then u = 1/R0 is the stable steady-state. If more than one individual is infectious, I(0) = i > 1, then theasymptotic steady-states are either 1 or (1/R0)

i.

25

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Appendix C. PGF for the Host-Vector Branching Approximation

In a manner similar to the derivation for the stochastic SIR model, dif-ferential equations for the probabilities of extinction can be derived for thebranching process approximation in the host-vector model. Let p(i,v),(l,m)(t)be the transition probabilities for I(t) and V (t) in the branching processapproximation. The generating function G(i,v) is defined as

G(i,v)(u1, u2, t) =∑l,m

p(i,v),(l,m)(t)ul1u

m2 .

In the branching process approximation, assume

G(i,v)(u1, u2, t) = [G(1,0)(u1, u2, t)]i[G(0,1)(u1, u2, t)]

v. (C.1)

Below are the backward Kolmogorov differential equations for I and V , wherethe transition probabilities pa,b(t) ≡ p(i,v),b(t), a = (i, v), and b = (l,m):

dp(i,v),bdt

= αvp(i+1,v),b + βiM

Hp(i,v+1),b + γip(i−1,v),b + µvp(i,v−1),b

−[αv + βi

M

H+ γi+ µv

]p(i,v),b.

Differentiate the identity in (C.1) with respect to t for two cases: v = 0 andi = 0.

∂G(1,0)

∂t=

1

i[G(1,0)]i−1∂G(i,0)

∂t=

1

iG(i−1,0)

∑b

dp(i,0),bdt

ul1um2

∂G(0,1)

∂t=

1

v[G(0,1)]v−1∂G(0,v)

∂t=

1

vG(0,v−1)

∑b

dp(0,v),bdt

ul1um2 .

Substitute the backward Kolmogorov differential equations into the rightside, apply the identity (C.1), and simplify. This leads to the followingdifferential equations for G(1,0) and G(0,1):

∂G(1,0)

∂t=

(γ + β

M

H

)[f1(G(1,0), G(0,1))−G(1,0)

]∂G(0,1)

∂t= (µ+ α)

[f2(G(1,0), G(0,1))−G(0,1)

].

26

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

Since p(1,0),(0,0)(t) = G(1,0)(0, 0, t) and p(0,1),(0,0)(t) = G(0,1)(0, 0, t), the preced-ing differential equations can be expressed in terms of the two probabilitiesof extinction. The steady-states of this system are the fixed points of the off-spring pgfs (11) and (12). The stability of these steady-states is determinedby the eigenvalues of the Jacobian matrix W(M− I). The steady-state (1,1)is stable if the eigenvalues of the Jacobian matrix have negative real part ifand only if ρ(M) < 1 if and only if R0 < 1 [39].

[1] E. Allen, Modeling with Ito Stochastic Differential Equations, Springer:Dordrecht, The Netherlands, 2007.

[2] L. J. S. Allen, Stochastic Population and Epidemic Models. Persistenceand Extinction, Springer International Pub., Cham, Switzerland, 2015.

[3] L. J. S. Allen, An Introduction to Stochastic Processes with Applicationsto Biology, 2nd Ed., CRC Press, Boca Raton, Fl., 2010.

[4] L. J. S. Allen, An Introduction to Stochastic Epidemic Models, in:F. Brauer, P. van den Driessche, J. Wu (Eds.), Mathematical Epidemi-ology. Lecture Notes in Mathematics, Vol. 1945, Springer, Berlin, 2008,Ch. 3, pp. 81–130.

[5] H. Andersson, T. Britton, Stochastic Epidemic Models and Their Sta-tistical Analysis, Lecture Notes in Statistics, No. 151, Springer-Verlag,New York, 2000.

[6] N. T. J. Bailey, The Mathematical Theory of Infectious Diseases and itsApplications, Griffin, London, 1975.

[7] T. Britton, Stochastic epidemic models: a survey, Mathematical Bio-sciences 225 (1) (2010) 24–35.

[8] D. J. Daley, J. Gani, Epidemic Modelling: An Introduction, CambridgeUniv. Press, New York, 1999.

[9] R. Durrett, Stochastic spatial models, SIAM Review 41 (4) (1999) 677–718.

[10] P. E. Greenwood, L. F. Gordillo, Stochastic Epidemic Modeling, in:G. Chowell, H. J. M., L. M. A. Bettencourt, C. Castillo-Chavez (Eds.),Mathematical and Statistical Estimation Approaches in Epidemiology,Springer, Dordrecht, 2009, Ch. 2, pp. 31–52.

27

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

[11] V. Isham, Stochastic Models for Epidemics, in: A. C. Davison,D. Yadolah, N. Wermuth (Eds.), Celebrating Statistics: Papers in Hon-our of Sir David Cox on the Occasion of his 80th Birthday, OxfordStatistical Science Series, No. 33, Oxford Univ. Press, Oxford, 2005,Ch. 1, pp. 27–53.

[12] P. Jagers, Branching Processes with Biological Applications, John Wiley& Sons, London, 1975.

[13] S. Karlin, H. Taylor, A First Course in Stochastic Processes, 2nd Ed.,Academic Press, New York, 1975.

[14] S. Karlin, H. Taylor, A Second Course in Stochastic Processes, AcademicPress, New York, 1981.

[15] S. Altizer, R. Ostfeld, P. T. J. Johnston, S. Kutz, C. D. Harvell, Climatechange and infectious diseases: From evidence to a predictive framework,Science 341 (6145) (2013) 514–519.

[16] A. Jutla, E. Whitcombe, N. Hasan, B. Haley, A. Akanda, A. Huq,M. Alam, B. S. Sack, R. Colwell, Environmental factors influencing epi-demic cholera, Am. J. Trop. Med. Hyg. 89 (3) (2013) 597–607.

[17] X. Wu, Y. Lu, S. Zhou, L. Chen, B. Xu, Impact of climate change onhuman infectious diseases: Empirical evidence and human adaptation,Environment International 86 (2016) 14–23.

[18] S. M. Ross, An Introduction to Probability Models, 11th Ed., AcademicPress, Oxford, UK, 2014.

[19] A. D. Barbour, The duration of the closed stochastic epidemic,Biometrika 62 (2) (1975) 477–482.

[20] K. B. Athreya, P. E. Ney, Branching Processes, Springer-Verlag,NewYork, 1972.

[21] K. S. Dorman, J. S. Sinsheimer, K. Lange, In the garden of branchingprocesses, SIAM Review 46 (2004) 202–229.

[22] T. E. Harris, The Theory of Branching Processes, Springer-Verlag,Berlin, 1963.

28

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

[23] P. Whittle, The outcome of a stochastic epidemic: A note on Bailey’spaper, Biometrika 42 (1955) 116–122.

[24] D. T. Gillespie, Exact stochastic simulation of coupled chemical reac-tions, J. Phys. Chem. 81 (1977) 2340–2361.

[25] E. J. Allen, L. J. S. Allen, A. Arciniega, P. Greenwood, Construction ofequivalent stochastic differential equation models, Stochastic Analysisand Applications 26 (2008) 274–297.

[26] T. G. Kurtz, Solutions of ordinary differential equations as limits of purejump Markov processes, Journal of Applied Probability 7 (1970) 49–58.

[27] T. G. Kurtz, Limit theorems for sequences of jump Markov processesapproximating ordinary differential processes, Journal of Applied Prob-ability 8 (1971) 344–356.

[28] T. G. Kurtz, The relationship between stochastic and deterministic mod-els for chemical reactions, Journal of Chemical Physics 57 (1972) 2976–2978.

[29] B. Oksendal, Stochastic Differential Equations An Introduction withApplications, 5th Ed., Springer-Verlag, Berlin, 2000.

[30] P. E. Kloeden, E. Platen, Numerical Solution of Stochastic DifferentialEquations, Springer-Verlag, New York, 1992.

[31] P. E. Kloeden, E. Platen, H. Schurz, Numerical Solution of SDE throughComputer Experiments, Springer-Verlag, Berlin, 1997.

[32] World Health Organization, Fact Sheet: World Malaria Report 2015.URL http://www.who.int/malaria/media/world-malaria-report-2015/en/

[33] R. Ross, Report on the Prevention of Malaria in Mauritius, Churchill,London, 1909.

[34] R. Ross, The Prevention of Malaria, Murray, London, 1911.

[35] D. L. Smith, K. E. Battle, S. I. Hay, C. M. Barker, T. W. Scott,F. McKenzie, Ross, Macdonald, and a theory for the dynamics and con-trol of mosquito-transmitted pathogens, PLoS Pathogens 8 (4) (2012)e1002588.

29

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

[36] P. van den Driessche, J. Watmough, Reproduction numbers and sub-threshold endemic equilibria for compartmental models of disease trans-mission, Mathematical Biosciences 180 (2002) 29–48.

[37] J. M. Heffernan, R. J. Smith, L. M. Wahl, Perspectives on the basicreproductive ratio, Journal of the Royal Society Interface 2 (2005) 281–293.

[38] A. L. Lloyd, Z. Zhang, A. Morgan Root, Stochasticity and heterogeneityin host-vector models, Journal of the Royal Society Interface 4 (2007)851–863.

[39] L. J. S. Allen, P. van den Driessche, Relations between deterministic andstochastic thresholds for disease extinction in continuous- and discrete-time infectious disease models, Mathematical Biosciences 243 (2013)99–108.

[40] L. J. S. Allen, G. E. Lahodny Jr, Extinction thresholds in deterministicand stochastic epidemic models, Journal of Biological Dynamics 6 (2)(2012) 590–611.

[41] M. S. Bartlett, The relevance of stochastic models for large-scale epi-demiological phenomena, Journal of the Royal Statistical Society, SeriesC 13 (1964) 2–8.

[42] R. Antia, R. Regoes, J. C. Koella, C. T. Bergstrom, The role of evolutionin the emergence of infectious diseases, Nature 426 (2003) 658–661.

[43] S. Blumberg, J. O. Lloyd-Smith, Inference of R0 and transmission het-erogeneity from the size distribution of stuttering chains, PLOS Comput.Biol. 9 (5) (2013) e1002993.doi:10.1371/journal.pcbi.1002993.

[44] J. O. Lloyd-Smith, S. J. Schreiber, P. E. Kopp, W. M. Getz, Super-spreading and the impact of individual variation on disease emergence,Nature 438 (2005) 355–359.

[45] E. Allen, Environmental variability and mean-reverting processes, Dis-crete and Continuous Dynamical Systems Series B 21 (7) (2016) 2073–2089.

30

MANUSCRIP

T

ACCEPTED

ACCEPTED MANUSCRIPT

[46] L. J. S. Allen, E. J. Allen, C. B. Jonsson, The impact of environ-mental variation on hantavirus infection in rodents, in: A. B. Gumel,C. Castillo-Chavez, R. E. Mickens, D. P. Clemence (Eds.), Contem-porary Mathematics Series 410, Modeling the Dynamics of Human Dis-eases: Emerging Paradigms and Challenges, Vol. 410, AMS, Providence,R. I., 2006, Ch. 1, pp. 1–15.

31