Stochastic dynamic modelling and statisticalanalysis of infectious disease spread and cancertreatmentTheresa Stocks

Academic dissertation for the Degree of Doctor of Philosophy in Mathematical Statistics atStockholm University to be publicly defended on Thursday 13 December 2018 at 10.00 in sal14, hus 5, Kräftriket, Roslagsvägen 101.

AbstractMathematical models have proven valuable for public health decision makers as they can provide insights into theunderstanding, control and, ultimately, the prevention of diseases. This thesis contains four manuscripts dealing withstochastic dynamic modelling and statistical analysis of infectious disease spread and optimization of cancer treatment.

Paper I is concerned with deriving a patient- and organ-specific measure for the estimated negative side effects ofradiotherapy using a stochastic logistic birth-death process. Our analysis shows that the region of a maximum tolerableradiation dose can be related to the solution of a logistic differential equation; we illustrate our results for brachytherapyfor prostate cancer.

Paper II and III deal with inference for stochastic epidemic models. Parameter estimation for this model class can bechallenging as disease spread is usually only partially observed, e.g. in the form of accumulated reported incidences withinspecified time periods. To perform inference for these types of models, a useful method for maximum likelihood estimationis iterated filtering which takes advantage of the fact that it is relatively easy to generate samples from the underlyingtransmission process while the likelihood function for the given data is intractable.

Paper II is an application-oriented introduction to iterated filtering via the R package pomp (King et al., 2016) whichcontains a wide collection of simulation-based inference methods for partially observed Markov processes. We reviewthe theoretical background of the method and discuss by two examples its performance and some associated practicaldifficulties.

Paper III is concerned with model selection for partially observed epidemic models that differ with respect to the amountof variability they allow for and parameter estimation of those models from routinely collected surveillance data. Weillustrate the model selection and inference framework via the R package pomp for rotavirus transmission in Germany,however, the method can be easily adapted to other diseases.

In Paper IV we develop a transmission model for hepatitis C virus (HCV) infection among people who inject drugs(PWIDs) to enable countries to monitor their progress towards HCV elimination. In the scope of the WHO’s commitmentto viral hepatitis elimination, this topic is highly relevant to public health since injection drug use is the main route oftransmission in many countries. From the model and using surveillance data, we derive estimates of four key HCV-indicators. Furthermore, the model can be used to investigate the impact of two interventions, direct-acting antiviral drugtreatment and needle exchange programs, on the disease dynamics. In order to make the model and its output accessibleto relevant users, it is made available through a Shiny app.

Keywords: Mathematical modelling, infectious disease spread, cancer treatment, statistical inference, populationdynamics, birth-death process, partially observed Markov process.

Stockholm 2018http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-161485

ISBN 978-91-7797-484-0ISBN 978-91-7797-485-7

Department of Mathematics

Stockholm University, 106 91 Stockholm

STOCHASTIC DYNAMIC MODELLING AND STATISTICALANALYSIS OF INFECTIOUS DISEASE SPREAD AND CANCERTREATMENT 

Theresa Stocks

Stochastic dynamic modellingand statistical analysis ofinfectious disease spread andcancer treatment 

Theresa Stocks

©Theresa Stocks, Stockholm University 2018 ISBN print 978-91-7797-484-0ISBN PDF 978-91-7797-485-7 Printed in Sweden by Universitetsservice US-AB, Stockholm 2018

To my family. 

List of Papers

The following papers, referred to in the text by their Roman numerals, areincluded in this thesis.

PAPER I: A stochastic model for the normal tissue complication prob-ability (NTCP) and applicationsStocks, T., Hillen, T., Gong, J., and Burger, M. (2017). Mathemat-ical Medicine and Biology: A Journal of the IMA, 34(4):469-492.DOI:10.1093/imammb/dqw013.

PAPER II: Iterated filtering methods for Markov process epidemicmodelsStocks, T. (2018). To appear in the Handbook of Infectious Dis-ease Data Analysis, Held, L., Hens, N., O’Neill, P.D. and Wallinga,J. (Eds.). Chapman & Hall/CRC.arXiv:1712.03058v3.

PAPER III: Model selection and parameter estimation for dynamic epi-demic models via iterated filtering: application to rotavirusin GermanyStocks, T., Britton, T., and Höhle, M. (2018). Biostatistics, kxy057.DOI:10.1093/biostatistics/kxy057.

PAPER IV: Dynamic modelling of hepatitis C transmission among peo-ple who inject drugs: A tool to support WHO eliminationtargetsStocks, T., Martin, L. J., Kühlmann-Berenzon, S., and Britton, T.(2018), Submitted.bioRxiv: DOI: 10.1101/460550.

Reprints were made with permission from the publishers.

Author’s contribution

Paper I: Theresa Stocks (TS) derived the mathematical proofs under super-vision of Martin Burger and Thomas Hillen. TS carried out the numericalsimulations. TS wrote the manuscript with contributions by Thomas Hillen.TS and Thomas Hillen edited the manuscript with contributions by MartinBurger.

Paper II: TS is the only author of the paper.

Paper III: TS, Tom Britton and Michael Höhle derived the model. TS imple-mented the inference procedure using the pomp package and conducted allanalyses including the simulation studies. TS wrote and revised the manuscriptwith contributions by Tom Britton and Michael Höhle.

Paper IV: TS and Tom Britton derived the model in collaboration with LeahMartin and Sharon Kühlmann-Berenzon. TS carried out the numerical sim-ulations and implementation of the Shiny app. TS wrote the manuscriptwith contributions by all co-authors.

General comment: Paper I, an earlier version of Paper III, as well as parts ofthe introduction were contained in the Licentiate thesis of Theresa Stocks(Stocks, 2017).

Acknowledgements

I would like to thank everybody who made this thesis possible, especially:My supervisor Tom Britton, for your continuous support and guidance, con-structive comments and your open door at any time that made me alwaysfeel welcome.My supervisor Michael Höhle, for your expertise in the interface of mod-elling and real-world public health applications, interesting discussions andfor being, despite the geographical distance, always supportive over Skypeand email.My co-authors and former supervisors Thomas Hillen and Martin Burger,for your guidance and introducing me to the thrilling field of mathematicalmodelling.My co-authors Leah Martin and Sharon Kühlmann-Berenzon, for makingme understand the importance of interdisciplinary research and sharingyour views from a public health perspective with me.All my colleagues at the Department of Mathematics and Mathematical Statis-tics at Stockholm University, for providing such a friendly and stimulatingatmosphere. The Swedish Research Council and the Department, for fund-ing and giving me the opportunity to continuously learn, explore and travelto conferences.Federica and Ka Yin, for your friendship, your mentorship and uncountablefikas. Abid, for sharing an office. Neshat and the whole administration team,for always guaranteeing a smooth work-flow outside research. All fellowPhD students, for turning days into nights and from colleagues to friends.Petra, for being my soul mate. Julia, for your strength and positivity that gobeyond words. Andrea, for not letting time or distance make a difference.Akvile and Monika for making me feel at home away from home. Alex, formaking my life brighter every day.My family, for your love, support and being such a big part of my life. Mama,for picking up your phone at any time of the day, your valuable advice andbeing my greatest role model. My team, Nicolas, Valerie, Constanze andRobin, for encouraging and inspiring me each in your own, amazing, indi-vidual way. Oma and Tati, for always believing in me. Papa, for your supportand giving me so many opportunities to grow.

Contents

List of Papers i

Author’s contribution iii

Acknowledgements v

1 Introduction 9

2 Compartment models 112.1 Ordinary differential equations . . . . . . . . . . . . . . . . . . . 122.2 CTMC with demographic noise . . . . . . . . . . . . . . . . . . . 132.3 CTMC with demographic and extra-demographic noise . . . . 152.4 A first glimpse at statistical inference . . . . . . . . . . . . . . . 19

3 Summaries of Papers 213.1 Summary of Paper I . . . . . . . . . . . . . . . . . . . . . . . . . . 213.2 Summary of Papers II-IV . . . . . . . . . . . . . . . . . . . . . . . 22

4 Sammanfattning 25

References 27

1. Introduction

Understanding the causes and dynamics of a particular disease to controland, ultimately, eradicate the disease from the population remains one ofthe most important issues of our time. The rapid spread of communica-ble diseases in a world that is becoming increasingly interconnected as wellas the rising invasion of non-communicable diseases in an ageing societywhich is progressively exposed to poor lifestyle choices is an ubiquitousthreat to human health worldwide. Moreover, from an economical point ofview a high disease burden imposes a constant clinical and financial pres-sure on national healthcare systems. Traditionally, disciplines such as epi-demiology, medicine, biology and public health are concerned with the un-derstanding and control of diseases. Depending on which approach is cho-sen such investigations may be very expensive in order to finance the nec-essary people and services or they may present risks for patients in time-consuming pilot studies. A long-standing research field, which can help toinform the traditional fields listed above in an efficient way, is mathemati-cal modelling; in epidemiology, for example, the first mathematical modeldates back to Bernoulli (1760). Mathematical models can provide both sig-nificant insights into disease progression as well as scenario analysis andrisk assessment of potential control measures. By describing an ideal worldin which distinctive determinants can be investigated ceteris paribus, math-ematical models offer a way to disentangle how various factors influence theprogression of a disease and predict future scenarios depending on whichmeasures are implemented (Keeling and Rohani, 2008). For a comprehen-sive introduction to the use of mathematical models in biology see e.g. Allen(2007) and Brauer and Castillo-Chavez (2012). Equally important is the eval-uation of a model with adequate data – without data a meaningful statisticalmodel assessment is not possible. A good introduction to statistical infer-ence with applications in medicine, epidemiology and biology can be foundin Held and Sabanés Bové (2013).1

This thesis demonstrates how stochastic dynamic modelling and statisticalanalysis can contribute to increase the understanding of infectious disease

1This paragraph is a slightly modified replication of the introduction of Stocks (2017).

9

spread and enhance cancer treatment, in four explicit examples. In Chapter2 we give a brief general introduction to mathematical disease modelling viacompartment models and model-calibration from data, followed by a morespecific introduction and summary of each of the four papers contained inthis thesis (Chapter 3).

In a nutshell, Paper I is concerned with the quality assessment of radio-therapy for cancer treatment. Paper II discusses a method for simulation-based inference for partially observed stochastic epidemic models. PaperIII uses this inference framework to focus on the methodological questionof the role of stochasticity in transmission models to explain infectious dis-ease surveillance data. Paper IV develops a model to monitor key indicatorsfor hepatitis C virus transmission among people who inject drugs and to in-vestigate the effect of potential control measures.

10

2. Compartment models

A useful way of modelling population dynamics (e.g. of cells, animals or hu-mans, hereafter referred to as individuals) is by dividing the population intodifferent categories assuming that all individuals in the same category havethe same characteristics. The way individuals change over time can then bedescribed as flows through those categories and is the essence of compart-ment models.

In order to prepare the ground for the manuscripts contained in this thesis,especially Papers II-IV, we will start by introducing the general formulationof compartment models and, thereafter, discuss three different interpreta-tions of this model class that will be relevant to us, namely as A. ordinarydifferential equations, B. continuous-time Markov chains (CTMC) with de-mographic noise and C. CTMC with demographic and extra-demographicnoise, following mainly the exposition and notation in Bretó et al. (2009);He et al. (2009). A good overview of compartment models can be foundin Jacquez (1996); Matis and Kiffe (2000). We will explain how to generatesamples from these models and briefly mention how to perform inferencefor them if only specific aspects of the models are observed – a detailed de-scription thereof is the topic of Paper II. We give a numerical illustration foreach of those three interpretations in Figure 2.1.Note that this section is meant as a general introduction in order to give thereader a better intuitive understanding of the models presented in this the-sis rather than providing a comprehensive overview of analytical results. Werefer the interested reader to the literature pointed out throughout this sec-tion for further reading.

Compartment models divide the population into k ∈N+ distinct categoriesand describe the movement dynamics as a vector-valued time-continuousprocess X (t ) = (X1(t ), . . . , Xk (t )), t ≥ 0 that counts the number of individu-als in each compartment Xi , i ∈ 1, . . . ,k at time t . The counts can be eitherinteger- (counting distinct individuals) or real-valued (describing the pop-ulation as a concentration or proportion). For the remainder of this sec-tion X (t ) is assumed to be non-negative, hence, Xi (t ) ≥ 0 should hold for

11

all i , t . The total population size at time t is P (t ) = ∑ki=1 Xi (t ). Let the non-

decreasing function Ni j (t ) ≥ 0 denote the cumulative number of individ-uals that have moved from compartment i directly to j up until time t . Bythe conservation of mass identity the number of individuals in compartmentXi , i ∈ 1, . . . ,k at that time is then given as

Xi (t ) = Xi (0)+ ∑i 6= j

N j i (t )− ∑i 6= j

Ni j (t ) (2.0.1)

where j ∈ 1, . . . ,k and Xi (0) denotes the initial population. Each flow Ni j (t )is associated with a rate function µi j (t , X (t )) which denotes the per capitarate at which an individual in compartment i moves to compartment j withi , j ∈ 1, . . . ,k. In this formulation, the population is closed in the sense thatno individuals leave or enter the population – this assumption can be cir-cumvented by introducing additional source and sink compartments of thepopulation accounting for birth, death and migration. In the following wepresent three different interpretations of a compartment model; for simplic-ity we write µi j =µi j (t , X (t )) for the remainder of this section.

2.1 Ordinary differential equations

We start by considering a deterministic system to be defined by differen-tial equations below. Assuming that the counts are continuous rather thaninteger-valued, one description of a compartment model X (t ), given therates µi j , is to write the flows Ni j (t ) as a system of coupled ODEs

d

d tNi j (t ) =µi j Xi (t )

for i 6= j and i , j ∈ {1, . . . ,k}. Then Equation (2.0.1) becomes

d Xi (t )

d t= ∑

i 6= jµ j i X j (t )− ∑

i 6= jµi j Xi (t ). (2.1.1)

This deterministic, continuous-time process is also known as the determin-istic skeleton (Coulson et al., 2004; Cushing et al., 1998; He et al., 2009). Usu-ally population dynamics are only suitably described by highly nonlinearprocesses which means that the solution of (2.1.1) is often not available inclosed form. However, there exists an extensive literature on how to solvenonlinear ODEs numerically, see e.g. Atkinson et al. (2011).In the model formulation (2.1.1) no uncertainty is involved in the evolutionof the states of the system and, as a consequence, the model will alwaysproduce the same output for a given parameter constellation and initial val-ues, i.e. there is no randomness in the transitions. Such models can have

12

the advantage that analysis and inference is easier compared to stochasticsystems making their use popular. Deterministic compartment models arepreferably used when the target population is large and randomness is be-lieved to be of less importance.However, the population dynamics that compartment models aim to de-scribe are often exposed to many sources of uncertainty and fluctuations.Hence, stochastic models might be more suitable to describe these inher-ently variable systems, however, their analysis is usually more complicated.We present a stochastic version of this model in the next section.

2.2 CTMC with demographic noise

We now consider a related but stochastic model by assuming that individu-als move from one compartment to another according to some probabilitydistribution rather than deterministically. The simplest version of this as-sumption is that Ni j (t ) is a Poisson process (Brémaud, 1999). For this, it isassumed that the probability of moving from one compartment to anotherduring an infinitesimal time interval (t , t+τ] does not depend on the historyof the process but only on the state of the system at time t and that the in-finitesimal probability of simultaneous transitions between compartmentsis negligible. The infinitesimal transition probabilities that define such aMarkov process are given as

P(∆Ni j (t ) = 0|X (t ) = x) = 1−µi j xiτ+o(τ)

P(∆Ni j (t ) = 1|X (t ) = x) =µi j xiτ+o(τ) (2.2.1)

P(∆Ni j (t ) > 1|X (t ) = x) = o(τ)

for∆Ni j (t ) = Ni j (t +τ)−Ni j (t ) and x = (x1, . . . , xk ). Note, that in this formu-lation the process X (t ) accounts for the integer nature of individuals in thecompartments. The implicit assumption in this Poisson system is that theindividual waiting time to move from compartment i to j is exponentiallydistributed to the respective per capita rate µi j .Compared to the deterministic model described by (2.1.1) where the statevector X (t ) is a point mass at the solution of the ODE system at time t , it canquickly become challenging to find the distribution of the state vector X (t )at a given time t for the stochastic model described by (2.2.1). A direct wayto obtain the distribution of X (t ) is to derive the differential equations thatgovern the state probabilities Px1,...,xk (t ) = P((X1(t ), . . . , Xk (t )) = (x1, . . . , xk ))with 0 ≤ x1, . . . , xk ≤ P (t ) and

∑ki=1 xi = P (t ) for the population size P (t ) and

given initial conditions, also known as the Kolmogorov forward equations,see e.g. Ross (1995). This system of differential equations can then be solved

13

numerically. However, the size of the system depends on the number ofpossible states and therefore population size. As a consequence, solving thesystem of equations becomes computationally very expensive for large pop-ulation sizes.A different approach from the Kolmogorov forward equations is to insteadapproximate the distribution of X (t ) by simulation-based techniques, e.g.using Monte Carlo methods. For this, one can directly simulate from the sys-tem with exact methods, e.g. using the Gillespie algorithm (Gillespie, 1977).Given the current state of the system, the algorithm simulates the time un-til the next event and determines from which to which compartment thejump occurred. Then, the number of individuals in each compartment isupdated and time is incremented accordingly. This step is repeated until apre-defined stopping time is reached. However, here as well, the methodscales badly if the population size is large because of the immense numberof events that can take place.One way to speed up the simulations is by using the so called τ-leaping al-gorithm (Gillespie, 2001), an approximate method based on Gillespies algo-rithm, where all rates are assumed constant for a small time interval. As aspecial case, the algorithm in Box 1 reduces to the τ-leaping algorithm whenVar(∆Γi j ) = 0 for all i , j . For further details about approximation methodssee e.g. Gillespie (2001), Tian and Burrage (2004), Cai and Xu (2007), Matisand Kiffe (2000) and Wilkinson (2006).

The stochasticity in the model described by (2.2.1) arises from the unpre-dictability of event times such as birth, death and interactions between in-dividuals. This kind of variability is also know as demographic stochastic-ity. It can be shown that demographic stochasticity for population fractionsdecreases as population size increases and approximates the deterministicskeleton as described in (2.1.1) (Andersson and Britton, 2000, Chapter 5).For a fixed finite population size, using that µi j Xi (t ) ≤ κ where κ is a uni-form bound for all i , j , it can be shown that the Poisson system as formu-lated in (2.2.1) is equi-dispersed (Bretó et al., 2009; Bretó and Ionides, 2011).This means that the infinitesimal mean of the increments ∆Ni j (t ) is equalto the infinitesimal variance of the increments, more precisely

E(∆Ni j (t )|X (t ) = x) =µi j xiτ+o(τ)

Var(∆Ni j (t )|X (t ) = x) =µi j xiτ+o(τ).

However, there is evidence that data is often over-dispersed, i.e. the infinites-imal variance is larger than the infinitesimal mean (McCullagh and Nelder,1989) and that this overdispersion can be explained as extra-demographicstochasticity in particular (Bjørnstad and Grenfell, 2001). In this section,

14

we use the term extra-demographic stochasticity is a collective term for allfluctuations that go beyond demographic stochasticity. Extra-demographicstochasticity can include environmental stochasticity arising from externalfluctuations, e.g. changes in temperature or precipitation, but can also arisedue to unmodelled processes or model misspecifications. We come to an ex-tension of the model presented here accounting for demographic and extra-demographic stochasticity in the next section.

2.3 CTMC with demographic and extra-demographic noise

In the following we give a brief introduction to a general framework for con-tinuous-time Markov processes for discrete populations that combine de-mographic noise (infinitesimal increment mean = infinitesimal incrementvariance) and extra-demographic noise (infinitesimal increment mean < in-finitesimal increment variance). Note that the mathematical theory behindthis model class is quite deep and goes beyond the scope of the thesis; werefer the interested reader to Bretó et al. (2009), Bretó and Ionides (2011)and Bretó (2018) for further reading. The idea is that extra-demographicvariability arises because the transition rates between compartments arenot deterministic, as was assumed previously, but are exposed to randomfluctuations. In order to model those fluctuations it is assumed that therates vary according to some multiplicative white noise process ξi j (t ) withmean one and hence the transition rates can be written as µi jξi j (t ). Here,ξi j (t ) denotes the derivative of an integrated noise process Γi j (t ) such thatξi j (t ) = d

d t Γi j (t ). The integrated noise processes {Γi j (t ),1 ≤ i ≤ k,1 ≤ j ≤ k}are required to fulfill the following properties (Bretó et al., 2009):

(P1): Independent increments: {Γi j (t2)−Γi j (t1),1 ≤ i ≤ k,1 ≤ j ≤ k} is inde-pendent of {Γi j (t4)−Γi j (t3),1 ≤ i ≤ k,1 ≤ j ≤ k} for all t1 < t2 < t3 < t4.

(P2): Stationary increments: {Γi j (t2) − Γi j (t1),1 ≤ i ≤ k,1 ≤ j ≤ k} has ajoined distribution which depends only on the time increment t2− t1.

(P3): Non-negative increments: {Γi j (t2)−Γi j (t1) ≥ 0,1 ≤ i ≤ k,1 ≤ j ≤ k} forall t2 > t1.

(P4): Unbiased multiplicative noise: E(Γi j (t )) = t .

(P5): Partial independence of noise processes: For each i , {Γi j (t )} is inde-pendent of {Γi k (t )} for all j 6= k.

Properties (P1) and (P2) ensure that the integral over ξi j (t ) is well definedeven when the sample paths of Γi j (t ) are not formally differentiable (Karlin

15

and Taylor, 1981). It is shown in (Bretó et al., 2009, Theorem A.1) that, ifconditions (P1)-(P5) hold, the implicitly formulated infinitesimal transitionprobabilities

P(∆Ni j (t ) = ni j ,∀1 ≤ i ≤ k,1 ≤ j ≤ k|X (t ) = (x1, . . . , xk )) (2.3.1)

= E

(k∏

i=1

((xi

ni 1 · · ·ni ,i−1ni ,i+1 · · ·ni k ri

)(1−∑

l 6=ipi l

)ri ∏j 6=i

pni j

i j

))

where ri = xi −∑l 6=i ni l and

( nn1···nk

)is a multinomial coefficient with

pi j = pi j ({µi j (t , x)}, {∆Γi j })

=(

1−exp

(−∑

lµi l∆Γi l

))µi j∆Γi j

/∑lµi l∆Γi l , (2.3.2)

with µi j = µi j (t , x),∆Ni j (t ) = Ni j (t + τ) − Ni j (t ),∆Γi j = Γi j (t + τ) −Γi j (t )specify a Markov process.A wide range of Lévy processes satisfy the properties (P1)-(P5) above (Sato,1999). A particularly convenient choice are processesΓi j (t ) such that margi-nally Γi j (t + τ)−Γi j (t ) ∼ Gamma(τ/σ2

i j ,σ2i j ), where Gamma(α,β) denotes

the gamma distribution with shape parameterα and scale parameterβwithcorresponding mean αβ and variance αβ2. In the special case when theover-dispersion parameters σ2

i j = 0 for all i and j , Gamma(τ/σ2i j ,σ2

i j ) isdefined to take the non random value τ and hence (2.3.1) reduces to thePoisson system (2.2.1). As a consequence, the model presented here is alsosometimes referred to as Poisson system with stochastic rates (Bretó et al.,2009). Additionally, if the processes with gamma distributed infinitesimalincrements as described above are fully independent, i.e. {Γi j (t )} is inde-pendent of {Γkl (t )} for all pairs (i , j ) 6= (k, l ), an explicit formulation of thetransition probabilities can be derived, see (Bretó et al., 2009, Theorem A.2).In this case, Gillespie’s exact algorithm can be used for simulation (Gillespie,1977).

16

Box 1. Euler scheme for a numerical solution of the Markov chain withstochastic rates cf. Bretó et al. (2009)Procedure:

1. Divide the time interval [0,T] into N intervals of width τ= T /N

2. Set the initial value X (0)

3. For n = 0 to N −1

4. Generate noise increments {∆Γi j = Γi j (nτ+τ)−Γi j (nτ)}

5. Generate process increments

6. (∆Ni 1(nτ), . . . ,∆Ni ,i−1(nτ),∆Ni ,i+1(nτ), . . . ,∆Ni k (nτ),Ri (nτ))

6. ∼Multinomial(Xi (nτ), pi 1, . . . , pi ,i−1, pi ,i+1, . . . , pi k ,1−∑l 6=i pi l )

6. where pi j = pi j ({µi j (nτ, X (nτ))}, {∆Γi j }) is defined as in Equation (2.3.2)

6. and Ri counts the number of individuals remaining in compartment i

6. during the current Euler increment.

6. Set Xi (nτ+τ) = Ri (nτ)+∑j 6=i ∆N j i (nτ)

7. End For

In the more general case where Γi j (t ) only fulfill properties (P1)-(P5), nu-merical solutions of (2.3.1) are available by the discrete-time Euler schemeas described in Box 1. The Euler scheme is a discrete-time approximation toa continuous-time stochastic process and can be made arbitrary exact forτ → 0. For a thorough analysis it is always good to investigate if decreas-ing the time step has an impact on the conclusion of the analysis. However,sometimes a very fine time discretization is not feasible with respect to com-putational power/time. In that case numerical solutions can still have somevalue – one can see the continuous model with a specific time discretizationas a model itself rather that treating the continuous-time model as the onlytrue model that everything must compare to (Bretó et al., 2009).

The model formulation (2.3.1) accounts for the possibility that multiple in-dividuals move simultaneously between compartments. In infectious dis-ease modelling this is reasonable: it can be transmission events such as in-fluenza spreading in a closed bus or cholera spreading when several peo-ple drink from the same contaminated water source. This more flexiblemodel formulation hence allows for the presence of super-spreaders (oneinfectious individual infects more than one individuals in an infinitesimaltime increment). He et al. (2009) and Bretó et al. (2009) found that it is im-portant to account for extra-demographic noise because otherwise stronglimitations are put on the noise structure which can lead to bias in the pa-rameter estimates. Hence, the model formulation in (2.3.1 )is an interestingmodel generalization and with the procedure in Box 1 it is convenient tosimulate from the model. This convenience, however, comes at the cost of

17

0

2500

5000

7500

10000

0

2500

5000

7500

10000

0

2500

5000

7500

10000

0 10 20 30 40

0 10 20 30 40

0 10 20 30 40

Time

Individuals

Individuals

Individuals

S(t)

I(t)

R(t)

A.

B.

C.

Figure 2.1: Numerical example of a compartment model X (t ) =(S(t ), I (t ),R(t )) as A. an ordinary differential equation as described in(2.1.1), B. a CTMC with demographic noise as in (2.2.1), and C. a CTMCwith demographic and extra-demographic noise as in (2.3.1). The rates areµSI = βI (t )/P = 1 and µI R = γ = 0.5 with population size P = 10000 andinitial values S(0) = 9990, I (0) = 10 and R(0) = 0. The integrated noise processin C. has gamma distributed increments with noise parameters σ2

SI = 0.5and σ2

I R = 0. For simulation of the 20 model realizations in B. and C. theEuler scheme as described in Box 1 was used. For an interpretation of thiscompartment model see Section 3.2.

difficulty in computing analytic properties of the continuous-time process(Bretó et al., 2009). Hence, simulation-based inference for this model e.g.regarding simulation and robustness studies are often limited by computa-tional time and power.The modelling framework we have presented here has already been used inmany applications, see e.g. Bhadra et al. (2011); Bretó et al. (2009); He et al.(2009); Martinez et al. (2016).

18

2.4 A first glimpse at statistical inference

Mathematical modelling of biological systems can be interesting on its ownbecause it increases the understanding of how different components of themodel are related or because it poses mathematically interesting questions.However, if the model is supposed to inform decision making in other dis-ciplines such as clinical medicine or public health, it is crucial to evalu-ate the model with data. Statistical inference provides the tools to link amodel to the available data in order to estimate model parameters with cor-responding uncertainty measures, evaluate the performance of the modelby comparing simulations to the available data and to assess which of com-peting models might be preferable (Keeling and Rohani, 2008). However,model evaluation can be a difficult endeavor because sufficient data is notalways present, mathematical parameters are not easily measurable or cal-culations are computationally not feasible. The restricted availability of dataapplies especially for infectious diseases since data is rarely the result ofa planned experiment and mostly disease progression is only partially ob-served (O’Neill, 2010). Hence, traditional statistical approaches are not al-ways directly applicable.2

In a compartment model context, one is often interested in estimating theunknown transition rates µi j or over-dispersion parameters, e.g. σ2

i j . How-ever, instead of observing the full transition process usually only a specificaspect of the system is observed. The observations can be connected withthe underlying Markov process via an observation model; this model class iscalled partially observed Markov processes (POMP). Given the observations,a likelihood function can be formulated. Inference for this model class canbe challenging as transition probabilities might not be available in closedform and the likelihood of the given data quickly becomes intractable. Infer-ence for this model class is an active research area and many different meth-ods, which all have advantages and shortcomings have been proposed, e.g.iterated filtering (Ionides et al., 2006, 2015), simulated moments (Kendallet al., 1999), nonlinear forecasting (Sugihara and May, 1990), synthetic like-lihood (Wood, 2010) or Bayesian approaches such as approximate Bayesiancomputations (Toni et al., 2009) and particle MCMC (Andrieu et al., 2010).

2This paragraph is a slightly modified replication of Section 1.3 of Stocks (2017).

19

20

3. Summaries of Papers

3.1 Summary of Paper I

Paper I is concerned with the mathematical modelling of quality measuresfor radiotherapy and their optimization.

In this century, cancer has become one of the leading causes of death (WHO,2016). Common treatment strategies to eradicate the tumorous tissue fromthe human body are surgery, chemotherapy and radiation therapy (NationalCancer Institute, 2018). In this paper we focus on radiation therapy which isan efficient and often used treatment strategy- more than 50% of all patientsreceive a radiation therapy during their treatment (Delaney et al., 2005).During treatment, energetic beams from an external or internal source aresent through the body. The beams have the ability to harm a cell structurallyso it cannot divide any longer and dies (Symonds et al., 2012). The goal ofradiotherapy is to achieve a high probability of cure while the surroundinghealthy tissue is left functionally and structurally competent. A commonquality measure for radiotherapy treatments is the tumor control probabil-ity (TCP) which is the probability that no cancer cell has survived the treat-ment in the irradiated domain (Munro and Gilbert, 1961). Mathematicallyspeaking, if Pi (t ) denotes the probability that a number i of cancer cells are‘alive’ in an irradiated domain at time t , then the tumor control probabilityat that time is defined as

TCP(t ) = P0(t ),

where all cells are assumed to be identical. Depending on the model, theterm of a cell being ‘alive’ would have to be thoroughly defined (Zaider andMinerbo, 2000). Clearly, a treatment aims at obtaining a value for TCP whichis close to one, however, there is a natural bound to how much radiationcan be absorbed by the surrounding healthy tissue before the patient ex-periences some serious side effects. A not so well explored research areais the mathematical modelling of these unwanted byproducts of radiother-apy which are however crucial to assess for finding an optimal treatmentschedule. A quality measure for the damage on healthy tissue is the so callednormal tissue complication probability (NTCP) which is the probability that

21

a fatal number of healthy tissue cells has been damaged. Two early ap-proaches are described in Lyman (1985) and Niemierko and Goitein (1993).For example, if we now assume that Qi (t ) denotes the probability that anumber i of healthy tissue cells are ‘alive’ in an irradiated domain at timet and the organ works properly if more than L healthy cells are ’alive’ thenormal tissue complication probability at that time is defined as

NTCP(t ) =L∑

i=0Qi (t ).

In Paper I we provide a short review of relevant existing model-based TCPconcepts and from there derive a stochastic, dynamical model for NTCPwhich is organ, patient and treatment specific. In the model, cell growthis modelled as a stochastic birth and death process. The resulting solutionfor the NTCP is numerically feasible only for a small number of cells. Wehence derive an asymptotic limit for a large carrying capacity and relate theNTCP to the solution of an ordinary differential equation. We conclude thata deterministic solution of the NTCP is a good indicator of where radiationexposure of the normal tissue becomes critical. This framework is based onsimple modelling assumptions and aims at preparing a framework for clini-cal practice. We illustrate our findings for brachytherapy of prostate cancer.3

3.2 Summary of Papers II-IV

Papers II-IV are concerned with the dynamic modelling and statistical anal-ysis of infectious disease spread.

Mathematical modelling of infectious diseases aims at understanding, con-trolling and, ultimately, preventing infectious disease outbreaks or reduc-ing endemic incidence and to inform public health decision makers aboutthe possible impact of interventions such as treatment, vaccination, travelbans or other control measures. A comprehensive overview for mathemat-ical modelling of infectious disease transmission is given e.g. in Andersonand May (1991), Keeling and Rohani (2008), Vynnycky and White (2010) andDiekmann et al. (2013).A common approach to model the spread of infectious diseases is via com-partment models as described in Section 2, where individuals belong tothe different compartments dependent on their health status (Kermack andMcKendrick, 1927). Depending on the complexity of the models these can

3Section 3.1 is a slighly modified replication of Section 1.4 of Stocks (2017).

22

also account for heterogeneities such as age of the individuals or spatial dis-tributions. In the simplest case, individuals are grouped into three differentcategories: they are either susceptible to the disease (S), infected and infec-tious (I), or recovered and immune (R). A susceptible individual can acquirean infection from an infectious individual and then recovers after a certaintime, staying immune to the disease for the rest of its life. With the notationin Section 2 the vector-valued process counting susceptible, infectious andrecovered individuals at time t can be written as X (t ) = (S(t ), I (t ),R(t ), t ≥0). Assuming that the population is closed, i.e. S(t )+ I (t )+R(t ) = P , withP constant, for all t ≥ 0, the per capita transition rates can be defined asµSI = βI (t )/P and µI R = γ where β denotes the average number of infec-tious contacts an infected individual has with susceptible individuals pertime unit and γ denotes the recovery rate. If X (t ) follows the model interpre-tation of (2.2.1) in Section 2, hence being a CTMC with demographic noise,this model is called the general stochastic epidemic (Bartlett, 1949). For anumerical example of this compartment model see Figure 2.1. In real-worldapplications, the population size is often not closed and models might, de-pendent on the disease and the question the model is supposed to answer,include demography, seasonality, environmental factors, over-dispersion orfurther compartments e.g. maternal and latent states.An important mathematical indicator of an infectious disease is its basic re-production number R0. The basic reproduction number is defined as theexpected number of secondary cases caused by a typical infectious individ-ual during the early stage of an epidemic when everyone is susceptible (An-dersson and Britton, 2000, Chapter 1, p. 6). It is a useful parameter becausein many epidemic models R0 = 1 is a threshold value indicating whether ornot a infectious disease may spread through a population, which is the casefor R0 > 1 or R0 < 1, respectively.

As touched upon briefly in Section 2.4, inference for stochastic (epidemic)compartment models can be challenging because often only certain aspectsof the disease spread are observed. Public health surveillance systems, forexample, often routinely collect the reported number of newly infected caseswithin a given time interval. In the model presented above, the accumulatednumber of newly infected individuals in a time period (tm−1, tm] with m ∈1, ..., M is given as HSI (tm) = ∫ tm

tm−1NSI (s)d s. This quantity can then be re-

lated to the observations via an observation model, resulting in a POMP. Ad-ditional challenges might arise because of reporting delays, under-reporting,case miss-classifications etc. (Gibbons et al., 2014); usually, these complica-tions become more prominent the less severe the disease is.

23

The inference method we focus on in Papers II and III is called iterated fil-tering (Ionides et al., 2006, 2015) and is a simulation-based method to findthe maximum likelihood estimate for a POMP where the underlying trans-mission process can, in particular, be of the form presented in (2.3.1) in Sec-tion 2.3. The method uses the fact that is it simple to generate sample pathsfrom the model, as described in Box 1 in Section 2.3, while the transitionprobabilities and the likelihood for given data are analytically intractable.More specifically, Paper II gives a thorough introduction to inference forpartially observed Markov process epidemic models via iterated filtering. Itdiscusses potential challenges of the method and demonstrates its perfor-mance by two concrete examples using the R packages pomp (King et al.,2016) which contains a wide selection of simulation-based inference algo-rithms. Paper III uses this framework for model selection and parameterestimation focusing on the question which choice of underlying transmis-sion model ((2.1.1), (2.2.1) or (2.3.1) from Section 2) and which observationmodel is best suited to explain available surveillance data. We illustratethis approach for rotavirus in Germany, however the method can be eas-ily adapted for other infectious diseases. In both papers, we assume thatthe increments of the integrated noise process are gamma distributed (cf.Section 2.3) and that the most variable transitions are the infection events,hence setting all other extra-demographic noise parameters σ2

i j to zero.

In Paper IV hepatitis C virus (HCV) transmission among people who injectdrugs is modeled based on a model interpretation as described in (2.1.1) inSection 2.1. From the model and based on routinely collected surveillancedata, we derive four key HCV-indicators recommended to be monitored bythe WHO. This topic is highly relevant to public health as in many coun-tries injection drug use is the main route of transmission and the WHO hascommitted to eliminate viral hepatitis by 2030. The model is intended tobe used by public health professionals in order to monitor their country-specific disease situation and investigate how intervention strategies mightaffect the transmission dynamics. Hence, we have converted the model intoan accessible, web-based tool which can be used for resource allocationand treatment planning to prevent potentially costly but inefficient inter-ventions and support evidence-based public health decision-making.

24

4. Sammanfattning

Matematiska modeller har visat sig vara värdefulla för beslutsfattare inomfolkhälsa då de, förutom att ge en fördjupad kunskap om olika sjukdomar,också kan ge en förståelse av hur sjukdomar kan kontrolleras och förebyg-gas. Denna avhandling består av fyra manuskript, tre av dem behandlarstokastiska dynamiska modeller och statistisk analys av spridningen av in-fektionssjukdomar, och ett manuskript behandlar optimering av cancerbe-handling.

Studie I syftar till att härleda ett patient-och organspecifikt mått för skat-tningen av de negativa bieffekterna av strålbehandling, detta genom att an-vända en stokastisk logistisk födelse-och dödsprocess. Vi finner att den max-imala tolererbara strålningsdosen kan approximeras av lösningen till en lo-gistisk differentialekvation. Våra resultat illustreras för brakyterapi som an-vänds vid prostatacancer.

Studie II och III behandlar inferens för stokastiska epidemimodeller. Pa-rameterskattningar för denna typ av modeller försvåras av det faktum attspridningen av en sjukdom vanligtvis endast partiellt observeras, t.ex. genomackumulerad incidens under en given tidsperiod. En användbar metod föratt kunna utföra inferens för denna typ av modeller, med maximum likelihood-metoden, är itererad filtrering. Denna metod använder sig av att det är rel-ativt enkelt att generera stickprov från den underliggande spridningspro-cessen även om likelihood-funktionen för observerade data är svårhanterlig.

Studie II är en applikationsorienterad introduktion till itererad filtreringvia R-paketet pomp (King et.al., 2016), som innehåller en bred samling simu-leringsbaserade inferensmetoder för partiellt observerade Markovprocesser.Vi ger en översikt av metodens teoretiska bakgrund och med hjälp av två ex-empel för vi en diskussion om hur väl metoden presterade samt går igenomnågra praktiska svårigheter.

Studie III behandlar modellval för partiellt observerade epidemimod-eller, där epidemimodellerna skiljer sig med avseende på hur stor variabilitetde tillåter, och parameterskattningar av dessa modeller med rutinmässigtinsamlade övervakningsdata. Vi illustrerar det statistiska ramverket och hurmodellvalet görs via R-paketet pomp, illustrationen görs för spridningen avrotavirus i Tyskland även om metoderna likväl kan tillämpas på andra sjuk-

25

domar.I Studie IV utvecklar vi en spridningsmodell för hepatit C-virusinfektion

(HCV) bland personer med intravenöst drogmissbruk, detta i syfte att un-derlätta för länder att övervaka förloppet av att eliminera HCV. Intravenöstdroganvändande utgör den vanligaste smittvägen av hepatit C i många län-der. Mot bakgrund av WHOs åtagande att eliminera hepatit är modeller avdenna typ av stort intresse för folkhälsomyndigheter. Med vår modell ochövervakningsdata härleder vi skattningar för fyra nyckelindikatorerna förHCV. Modellen kan dessutom användas för att studera effekten av två typerav intervention på smittspridningen: direktverkande antivirala substanseroch sprututbytesprogram. För att göra modellen lättförståelig för relevantaanvändare har vi även gjort den tillgänglig via en Shiny app.

26

References

Allen, L. J. S. (2007). An Introduction to Mathematical Biology. Pearson.

Anderson, R. M. and May, R. M. (1991). Infectious Diseases of Humans: Dynamicsand Control. Oxford Science Publications.

Andersson, H. and Britton, T. (2000). Stochastic Epidemic Models and Their Statis-tical Analysis. Springer Lecture Notes in Statistics, 151.

Andrieu, C., Doucet, A., and Holenstein, R. (2010). Particle Markov chain MonteCarlo methods. Journal of the Royal Statistical Society, 72(3):269–342.

Atkinson, K. E., Han, W., and Stewart, D. (2011). Numerical Solution of OrdinaryDifferential Equations. John Wiley & Sons.

Bartlett, M. S. (1949). Some evolutionary stochastic processes. Journal of the RoyalStatistical Society. Series B (Methodological), 11(2):211–229.

Bernoulli, D. (1760). Essai d’une nouvelle analyse de la mortalité causée par la petitevérole. Mém Math Phys Acad Roy Sci Paris, 1:1–45.

Bhadra, A., Ionides, E. L., Laneri, K., Pascual, M., Bouma, M., and Dhiman, R.(2011). Malaria in northwest India: Data analysis via partially observed stochas-tic differential equation models driven by Lévy noise. Journal of the AmericanStatistical Association, 106(494):440–451.

Bjørnstad, O. N. and Grenfell, B. T. (2001). Noisy clockwork: Time series analysis ofpopulation fluctuations in animals. Science, 293(5530):638–643.

Brauer, F. and Castillo-Chavez, C. (2012). Mathematical Models in Population Biol-ogy and Epidemiology. Springer.

Brémaud, P. (1999). Markov Chains: Gibbs Fields, Monte Carlo Simulation, andQueues. Springer-Verlag New York.

Bretó, C. (2018). Modeling and inference for infectious disease dynamics: alikelihood-based approach. Statistical Science, 33(1):57–69.

Bretó, C., He, D., Ionides, E. L., and King, A. A. (2009). Time series analysis viamechanistic models. The Annals of Applied Statistics, 3(1):319–348.

27

Bretó, C. and Ionides, E. L. (2011). Compound Markov counting processes and theirapplications to modeling infinitesimally over-dispersed systems. Stochastic Pro-cesses and Their Applications, 121(11):2571–2591.

Cai, X. and Xu, Z. (2007). K-leap method for accelerating stochastic simulation ofcoupled chemical reactions. The Journal of Chemical Physics, 126(7):074102.

Coulson, T., Rohani, P., and Pascual, M. (2004). Skeletons, noise and populationgrowth: the end of an old debate? Trends in Ecology & Evolution, 17(7):359–364.

Cushing, J. M., Costantino, R. F., Dennis, B., Desharnais, R. A., and Hensona, S. M.(1998). Nonlinear population dynamics: Models, experiments and data. Journalof Theoretical Biology, 194(1):1–9.

Delaney, G., Jacob, S., C.Featherstone, and Barton, M. (2005). The role of radio-therapy in cancer treatment: estimating optimal utilization from a review ofevidence-based clinical guidelines. Cancer, 104(6):1129–1137.

Diekmann, O., Heesterbeek, H., and Britton, T. (2013). Mathematical Tools for Un-derstanding Infectious Disease Dynamics. Princeton Series in Theroretical andComputational Biology.

Gibbons, C. L., Mangen, M. J., Plass, D., Havelaar, A. H., Brooke, R. J., Kramarz,P., Peterson, K. L., Stuurman, A. L., Cassini, A., Fèvre, E. M., and Kretzschmar,M. E. (2014). Measuring underreporting and under-ascertainment in infectiousdisease datasets: a comparison of methods. BMC Public Health, 14(147).

Gillespie, D. T. (1977). Exact stochastic simulation of coupled chemical reactions.The Journal of Physical Chemistry, 81(25):2340–2361.

Gillespie, D. T. (2001). Approximate accelatered stochastic simulation of chemicallyreacting systems. Journal of Chemical Physics, 115(4):1716–1733.

He, D., Ionides, E. L., and King, A. A. (2009). Plug-and-play inference for diseasedynamic: measles in large and small populations as a case study. Journal of theRoyal Society Interface, 7(43):271–283.

Held, L. and Sabanés Bové, D. (2013). Applied Statistical Inference. Springer.

Ionides, E. L., Bretó, C., and King, A. A. (2006). Inference for nonlinear dynamicalsystems. PNAS, 103(49):18438–18443.

Ionides, E. L., Nguyen, D., Atchadé, Y., Stoeva, S., and King, A. A. (2015). Infer-ence for dynamic and latent variable models via iterated, perturbed bayes maps.PNAS, 112(3):719–724.

Jacquez, J. A. (1996). Compartmental Analysis in Biology and Medicine. 3rd ed.BioMedware, Ann Arbor, MI.

Karlin, S. and Taylor, H. M. (1981). A Second Course in Stochastic Processes. Aca-demic Press.

28

Keeling, M. J. and Rohani, P. (2008). Modeling Infectious Diseases in Humans andAnimals. Princeton University Press.

Kendall, B. E., Briggs, C. J., Murdoch, W. W., Turchin, P., Ellner , S. P., McCauley, E.,Nisbet, R. M., and Wood, S. N. (1999). Why do populations cycle? A synthesis ofstatistical and mechanistic modeling approaches. Ecology, 80(6):1789–1805.

Kermack, W. O. and McKendrick, A. G. (1927). A contribution to the mathematicaltheory of epidemics. Proceedings of the Royal Society A: Mathematical, Physicaland Engineering Sciences, 115(772):700.

King, A. A., Nguyen, D., and Ionides, E. L. (2016). Statistical inference for partiallyobserved Markov processes via the R package pomp. Journal of Statistical Soft-ware, 69(12):1–43.

Lyman, J. T. (1985). Complication probability as assessment from dose- volumehistograms. Radiation Research, Supplement 8, 104:S13–S19.

Martinez, P. P., King, A. A., Yunusd, M., Faruqued, A. S. G., and Pascual, M. (2016).Differential and enhanced response to climate forcing in diarrheal disease due torotavirus across a megacity of the developing world. PNAS, 113(15):4092–4097.

Matis, J. H. and Kiffe, T. R. (2000). Stochastic Population Models: A CompartmentalPerspective. Springer-Verlag New York.

McCullagh, R. and Nelder, J. (1989). Generalized Linear Models. 2nd ed. Chapmanand Hall, London.

Munro, T. R. and Gilbert, C. W. (1961). The relation between tumour lethaldoses and the radiosensitivity of tumour cells. British Journal of Radiology,34(400):246–251.

National Cancer Institute (2018). Cancer treatment. https://www.cancer.gov/about-cancer/treatment, [Accessed November 9, 2018].

Niemierko, A. and Goitein, M. (1993). Modeling of normal tissue response to radia-tion: the critical volume model. Int. J. Rad. Oncol. Biol. Phys., 25(1):135–145.

O’Neill, P. D. (2010). Introduction and snapshot review: relating infectious diseasetransmission models to data. Statistics in Medicine, 29(20):2069–2077.

Ross, S. (1995). Stochastic Processes, 2nd Edition. John Wiley & Sons.

Sato, K. (1999). Lévy Processes and Infinitely Divisible Distributions. CambridgeUniversity Press.

Stocks, T. (2017). Dynamic Modelling of Communicable and Non-CommunicableDiseases. Licentiate thesis, Stockholm University, diva2:1064666.

Sugihara, G. and May, R. M. (1990). Nonlinear forecasting as a way of distinguishingchaos from measurement error in time series. Nature, 344(6268):734–741.

29

Symonds, R. P., Deehan, C., and Mills, C. M. J. A. (2012). Walter and Miller’s Textbookof Radiotherapy: Radiation Physics, Therapy and Oncology, 7th Edition. Elsevier.

Tian, T. and Burrage, K. (2004). Binomial leap methods for simulating stochasticchemical kinetics. The Journal of Chemical Physics, 121(21):10356.

Toni, T., Welch, D., Strelkowa, N., Ipsen, A., and Stumpf, M. P. (2009). ApproximateBayesian computation scheme for parameter inference and model selection indynamical systems. Journal of the Royal Society, 6(31):187–202.

Vynnycky, E. and White, R. (2010). An Introduction to Infectious Disease Modelling.Oxford University Press.

WHO (2016). Cancer. http://www.who.int/cancer/en/, [Accessed December 7, 2016].

Wilkinson, D. (2006). Stochastic Modelling for Systems Biology. Chapman &Hall/CRC Mathematical & Computational Biology.

Wood, S. N. (2010). Statistical inference for noisy nonlinear ecological dynamicsystems. Nature, 466:1102–1104.

Zaider, M. and Minerbo, G. N. (2000). Tumor control probability: a formulationapplicable to any temporal protocol of dose delivery. Physics in Medicine andBiology, 45(2):279–293.

Department of Mathematics, Stockholm University, 106 91 StockholmE-mail address: theresa.stocks@math.su.se

30