Stochastic modelling of epidemic spread · Introduction Stochastic processes SIS model DTMC CTMC Topological spaces and Borel -algebras Deﬁnition 3 (Topology) Let ⌦ 6= ; be an

Introduction Stochastic processes SIS model DTMC CTMC

Stochastic modelling of epidemic spread

Julien Arino

Department of Mathematics

University of Manitoba

Winnipeg

Julien [email protected]

19 May 2012


1 Introduction

2 Stochastic processes

3 The SIS model used as an example

4 DTMC

5 CTMC


1 Introduction



4 DTMC

5 CTMC


Deterministic SIR

Use the standard KMK as in Fred’s lectures:

S

0 = ��SII

0 = �SI � ↵I

R

0 = ↵I

Basic reproduction number

R0

=�

↵S

0

• If R0

< 1, no outbreak

• If R0

> 1, outbreak

Use ↵ = 1/4, N = S + I + R = 100, I (0) = 1, R0

= {0.5, 1.5}and � so that this is true


R0

= 0.5 R0

= 1.5

0 50 100 1500

10

20

30

40

50

60

70

80

90

100

Time (days)

SIR

0 50 100 1500

10

20

30

40

50

60

70

80

90

100

Time (days)

SIR

0 50 100 1500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Time (days)

Num

ber i

nfec

tious

0 50 100 1500

1

2

3

4

5

6

7

8

Time (days)

Num

ber i

nfec

tious


Stochastic DTMC SIR

Consider S(t) and I (t) (R(t) = N � S(t)� I (t)). Definetransition probabilities

p

(s

0,i 0),(s,i)(�t) =

P{(S(t +�t), I (t +�t)) = (s + s

0, i + i

0)|(S(t), I (t)) = (s, i)}

by

p

(s

0,i 0),(s,i)(�t) =

8>>>><

>>>>:

�si�t (s 0, i 0) = (�1, 1)↵i�t (s 0, i 0) = (0,�1)1� [�si + ↵i ]�t (s 0, i 0) = (0, 0)

0 otherwise

Parameters as previously and �t = 0.01


R0

= 0.5 R0

= 1.5

0 50 100 1500

1

2

3

4

5

6

7

Time (days)

Num

ber i

nfec

tious

0 50 100 1500

5

10

15

20

25

Time (days)

Num

ber i

nfec

tious

0 50 100 1500

0.2

0.4

0.6

0.8

1

1.2

1.4

Time (days)

Num

ber i

nfec

tious

0 50 100 1500

0.5

1

1.5

2

2.5

3

3.5

Time (days)

Num

ber i

nfec

tious


Number of outbreaks

0 10 20 30 40 50 60 70 80 90 1000

100

200

300

400

500

600

700

800

900

1000

Value of I0

Num

ber o

f sim

ulat

ions

with

an

outb

reak

(/10

00)

R0=0.5

R0=1.5

outbreak: 9t s.t. I (t) > I (0)


Why stochastic models?

Important to choose right type of model for given application

Deterministic models

• Mathematically easier

• Perfect reproducibility

• For given parameter set,one sim su�ces

Stochastic models

• Often harder

• Each realization di↵erent

• Needs many simulations

In early stage of epidemic, very few infective individuals:

• Deterministic systems: behaviour entirely governed byparameters

• Stochastic systems: allows “chance” to play a role


In the context of the SIR model

Deterministic

• R0

strict threshold

• R0

< 1 ) I (t)! 0 monotonically

• R0

> 1 ) I (t)! 0 after a bump (outbreak)

Stochastic

• R0

threshold for mean

• R0

< 1 ) I (t)! 0 monotonically (roughly) on average butsome realizations have outbreaks

• R0

> 1 ) I (t)! 0 after a bump on average (outbreak) butsome realizations have no outbreaks


1 Introduction



4 DTMC

5 CTMC


Theoretical setting of stochastic processes

Stochastic processes form a well defined area of probability, asubset of measure theory, itself a well established area ofmathematics

Strong theoretical content

Very briefly presented here for completeness, will not be used inwhat follows

Used here: A. Klemke, Probability Theory, Springer 2008

DON’T PANIC!!!


�-algebras

Definition 1 (�-algebra)

Let ⌦ 6= ; be a set and A ⇢ 2⌦, the set of all subsets of ⌦, be aclass of subsets of ⌦. A is called a �-algebra if:

1 ⌦ 2 A2 A is closed under complements, i.e., Ac := ⌦ \ A 2 A for any

A 2 A3 A is closed under countable unions, i.e., [1

n=1

A

n

2 A for anychoice of countably many sets A

1

,A2

, . . . 2 A

⌦ is the space of all elementary events and A the system ofobservable events


Probability space

Definition 21 A pair (⌦,A), with ⌦ a nonempty set and A ⇢ 2⌦ a

�-algebra is a measurable space, with sets A 2 Ameasurable sets. If ⌦ is at most countably infinite and ifA = 2⌦, then the measurable space (⌦, 2⌦) is discrete

2 A triple (⌦,A, µ) is a measure space if (⌦,A) is ameasurable space and µ is a measure on A

3 If in addition, µ(⌦) = 1, then (⌦,A, µ) is a probability space

and the sets A 2 A are events. µ is then usually denoted P(or Prob)


Topological spaces and Borel �-algebras

Definition 3 (Topology)

Let ⌦ 6= ; be an arbitrary set. A class of sets ⌧ ⇢ ⌦ is a topology

on ⌦ if:

1 ;,⌦ 2 ⌧

2

A \ B 2 ⌧ for any A,B 2 ⌧

3 ([A2FA) 2 ⌧ for any F 2 ⌧

(⌦, ⌧) is a topological space, sets A 2 ⌧ are open and A 2 ⌦with A

c 2 ⌧ are closed

Definition 4 (Borel �-algebra)

Let (⌦, ⌧) be a topological space. The �-algebra B(⌦, ⌧)generated by the open sets is the Borel �-algebra on ⌦, withelements A 2 B(⌦, ⌧) the Borel sets


Polish spaces

Definition 5A separable topological space whose topology is induced by acomplete metric is a Polish space

Rd , Zd , RN, (C ([0, 1]), k · k1) are Polish spaces


Stochastic process

Definition 6 (Stochastic process)

Let (⌦,F ,P) be a probability space, (E , ⌧) be a Polish space withBorel �-algebra E and T ⇢ R be arbitrary (typically, T = N,T = Z, T = R

+

or T an interval).A family of random variables X = (X

t

, t 2 T ) on (⌦,F ,P) withvalues in (E , E) is called a stochastic process with index set (ortime set) T and range E .


OKAY, DONE WITH HARD THEORY!!!


Types of processes

So a stochastic process is a collection of random variables (r.v.)

{X (t,!), t 2 T ,! 2 ⌦}

The index set T and r.v. can be discrete or continuous

•Discrete time Markov chain (DTMC) has T and X

discrete (e.g., T = N and X (t) 2 N)•Continuous time Markov chain (CTMC) has T continuousand X discrete (e.g., T = R

+

and X (t) 2 N)

In this course, we focus on these two. Also exist Stochasticdi↵erential equations (SDE), with both T and X (t) continuous


Markov property

Definition 7 (Markov property)

Let E be a Polish space with Borel �-algebra B(E ), T ⇢ R and(X

t

)t2T an E -valued stochastic process. Assume that

(Ft

)t2T = �(X ) is the filtration generated by X . X has the

Markov property if for every A 2 B(E ) and all s, t 2 T withs t,

P[Xt

2 A|Fs

] = P[Xt

2 A|Xs

]

Theorem 8 (The usable one)

If E is countable then X has the Markov property if for all n 2 N,all s

1

< · · · < s

n

< t and all i

1

, . . . , in

, i 2 E with

P[Xs

1

= i

1

, . . . ,Xs

n

= i

n

] > 0, there holds

P[Xt

= i |Xs

1

= i

1

, . . . ,Xs

n

= i

n

] = P[Xt

= i |Xs

n

= i

n

]


1 Introduction



4 DTMC

5 CTMC


SIS model

We consider an SIS model with demography. In ODE,

S

0 = b � dS � �SI + �I

I

0 = �SI � (� + d)I

Total populationN

0 = b � dN

asymptotically constant (N(t)! b/d =: N⇤ as t !1). Canwork with asymptotically autonomous system where N = N

⇤ (andS = N

⇤ � I )


S

0 = b � �SI � dS + �I

I

0 = �SI � (� + d)I

DFE: (S , I ) = (b/d , 0). EEP: (S , I ) =⇣�+d

� , bd

� �+d

�

⌘

VdD&W: F = D

I

[�S ]|DFE

= �N⇤, V = D

I

[(� + d)I ]|DFE

= � + d

) R0

= ⇢(FV�1) =�

� + d

N

⇤

• If R0

< 1, limt!1(S(t), I (t)) = (N⇤, 0)

• If R0

> 1, limt!1(S(t), I (t)) =

⇣N

⇤

R0

,N⇤ � N

⇤

R0

⌘


Reduction to one equation

As N is asymptotically constant, we can write S = N

⇤ � I and soonly need

I

0 = �(N⇤ � I )I � (� + d)I

We can then reconstruct dynamics of S from that of I


1 Introduction



4 DTMC

5 CTMC


Discrete time Markov chain

Definition 9 (DTMC – Simple definition)

An experiment with finite number of possible outcomes S1

, . . . ,SN

is repeated. The sequence of outcomes is a discrete time Markov

chain if there is a set of N2 numbers {pij

} such that theconditional probability of outcome S

j

on any experiment givenoutcome S

i

on the previous experiment is pij

, i.e., for 1 i , j N,t = 1, . . .,

p

ji

= Pr(Sj

on experiment t + 1|Si

on experiment t).

The outcomes S1

, . . . ,SN

are the states, and the p

ij

are thetransition probabilities. The matrix P = [p

ij

] is the transition

matrix.


Homogeneity

One major distinction:

• If pij

does not depend on t, the chain is homogeneous

• If pij

(t), i.e, transition probabilities depend on t, the chain isnonhomogeneous


Markov chains operate on two level:

• Description of probability that the system is in a given state

• Description of individual realizations of the process

Because we assume the Markov property, we need only describehow system switches (transitions) from one state to the next

We say that the system has no memory, the next state dependsonly on the current one


States

The states are here the di↵erent values of I . I can take integervalues from 0 to N

The chain links these states

I=0 I=1 I=2 I=3 I=N-1 I=N

Recoveryor Death

Recoveryor Death

Recoveryor Death

Recoveryor Death

Infection Infection Infection


Probability vector

Letp

i

(t) = P(I (t) = i)

p(t) =

0

BBB@

p

0

(t)p

1

(t)...

p

N

(t)

1

CCCA

is the probability distribution

We must haveP

i

p

i

(t) = 1

Initial condition at time t = 0: p(0)


Transition probabilities

We then need description of probability of transition

In general, suppose that Si

is the current state, then one ofS1

, . . . ,SN

must be the next state. If

p

ji

= P{X (t + 1) = Sj

|X (t) = Si

}

then we must have

p

i1

+ p

i2

+ · · ·+ p

iN

= 1, 1 i N

Some of the p

ij

can be zero


Transition matrixThe matrix

P =

0

BB@

p

11

p

12

· · · p

1N

p

21

p

22

· · · p

2N

p

N1

p

N2

· · · p

NN

1

CCA

has

• nonnegative entries, pij

� 0

• entries less than 1, pij

1

• column sum 1, which we write

NX

i=1

p

ij

= 1, j = 1, . . . ,N

or, using the notation 1l = (1, . . . , 1)T ,

1l

T

P = 1l

T


In matrix form

p(t + 1) = Pp(t), t = 1, 2, 3, . . .

where p(t) = (p1

(t), p2

(t), . . . , pN

(t))T is a (column) probabilityvector and P = (p

ij

) is a N ⇥ N transition matrix,

P =

0

BB@

p

11

p

12

· · · p

1N

p

21

p

22

· · · p

2N

p

N1

p

N2

· · · p

NN

1

CCA


Stochastic matrices

Definition 10 (Stochastic matrix)

The nonnegative N ⇥N matrix M is stochastic ifP

N

i=1

a

ij

= 1 forall j = 1, 2, . . . ,N. In other words, 1lTM = 1l

T

Theorem 11Let M be a stochastic matrix M. Then the spectral radius

⇢(M) 1, i.e., all eigenvalues � of M are such that |�| 1.Furthermore, � = 1 is an eigenvalue of M

M has all column sums 1 , 1l

T

M = 1l

T

� e.value of M with associated left e.vector v , v

T

M = �vT

) 1l

T

M = 1l

T : left e.vector 1lT and e.value 1..


Long “time” behaviorLet p(0) be the initial distribution (column) vector. Then

p(1) = Pp(0)

p(2) = Pp(1)

= P(Pp(0))

= P

2

p(0)

Iterating, we get that for any t 2 N,

p(t) = P

t

p(0)

Therefore,

limt!+1

p(t) = limt!+1

P

t

p(0) =

✓lim

t!+1P

t

◆p(0)

if this limit exists (it does if P is constant and chain is regular)


Regular Markov chainRegular Markov chains not covered here, as most chains in thedemographic context are absorbing. For information, though..

Definition 12 (Regular Markov chain)

A regular Markov chain is one in which P

k is positive for someinteger k > 0, i.e., Pk has only positive entries, no zero entries.

Theorem 13If P is the transition matrix of a regular Markov chain, then

1

the powers P

t

approach a stochastic matrix W ,

2

each column of W is the same (column) vector

w = (w1

, . . . ,wN

)T ,

3

the components of w are positive.

So if the Markov chain is regular,

limt!+1

p(t) = Wp(0)


Interesting properties of matrices

Definition 14 (Irreducibility)

The square nonnegative matrix M is reducible if there exists apermutation matrix P such that PT

MP is block triangular. If M isnot reducible, then it is irreducible

Definition 15 (Primitivity)

A nonnegative matrix M is primitive if and only if there is aninteger k > 0 such that Mk is positive


Directed graphs (digraphs)

Definition 16 (Digraph)

A directed graph (or digraph) G = (V,A) consists in a set V ofvertices and a set A of arcs linking these vertices

1 2

3

5

4


Connecting graphs and matrices

Adjacency matrix M has mij

= 1 if arc from vertex j to vertex i ,0 otherwise

1 2

3

5

4

) M =

0

BBBB@

0 1 0 0 01 0 0 0 01 0 0 1 00 1 0 1 00 0 0 0 0

1

CCCCA

Column of zeros (except maybe diagonal entry): vertex is a sink

(nothing leaves)


Paths of length exactly 2

Given adjacency matrix M, M2 has number of paths of lengthexactly two between vertices

1 2

3

5

4

) M

2 =

0

BBBB@

1 0 0 0 00 1 0 0 00 2 0 1 00 0 0 1 00 0 0 0 0

1

CCCCA

Write m

k

ij

the (i , j) entry in M

k

m

2

32

= 2 since 2! 1! 3 and 2! 4! 3..


Connecting matrices, graphs and regular chains

Theorem 17A matrix M is irreducible i↵ the associated connection graph is

strongly connected, i.e., there is a path between any pair (i , j) ofvertices

Irreducibility: 8i , j , 9k , mk

ij

> 0

Theorem 18A matrix M is primitive if it is irreducible and there is at least one

positive entry on the diagonal of M

Primitivity: 9k , 8i , j , mk

ij

> 0

Theorem 19Markov chain regular , transition matrix P primitive


Strong connectedness (1)

4

5

7

8

2

6

3

1

Not strongly connected


Strong connectedness (2)

4

5

7

8

2

6

3

1

Strongly connected


Strong components

4

5

7

8

2

6

3

1

Strong component 1 Strong component 2


Absorbing states, absorbing chains

Definition 20A state S

i

in a Markov chain is absorbing if whenever it occurs onthe n

th generation of the experiment, it then occurs on everysubsequent step. In other words, S

i

is absorbing if pii

= 1 andp

ij

= 0 for i 6= j

Definition 21A Markov chain is said to be absorbing if it has at least oneabsorbing state, and if from every state it is possible to go to anabsorbing state

In an absorbing Markov chain, a state that is not absorbing iscalled transient


Absorbing strong components

4

5

7

8

2

6

3

1

Absorbingstrong component Transient states

Here, we focus on absorbing strong components containing onlyone vertex


Some questions on absorbing chains

1 Does the process eventually reach an absorbing state?

2 Average number of times spent in a transient state, if startingin a transient state?

3 Average number of steps before entering an absorbing state?

4 Probability of being absorbed by a given absorbing state,when there are more than one, when starting in a giventransient state?


Reaching an absorbing state

Answer to question 1:

Theorem 22In an absorbing Markov chain, the probability of reaching an

absorbing state is 1


Standard form of the transition matrix

For an absorbing chain with k absorbing states and N � k

transient states, the transition matrix can be written as

P =

✓Ik

0

R Q

◆

with following meaning,

Absorbing states Transient statesAbsorbing states I

k

0

Transient states R Q

with Ik

the k ⇥ k identity matrix, 0 an k ⇥ (N � k) matrix of zeros,R an (N � k)⇥ k matrix and Q an (N � k)⇥ (N � k) matrix.


The matrix IN�k

� Q is invertible. Let

•F = (I

N�k

� Q)�1 be the fundamental matrix of theMarkov chain

•T

i

be the sum of the entries on row i of F

•B = FR

Answers to our remaining questions:

2

F

ij

is the average number of times the process is in the jthtransient state if it starts in the ith transient state

3

T

i

is the average number of steps before the process enters anabsorbing state if it starts in the ith transient state

4

B

ij

is the probability of eventually entering the jth absorbingstate if the process starts in the ith transient state


Back to the SIS model

ODE:I

0 = �(N⇤ � I )I � (� + d)I

This equation has 3 components:

1 �(N⇤ � I )I new infection

2 �I recovery

3

dI death

In the DTMC, want to know what transitions mean and what theirprobabilities are

I=0 I=1 I=2 I=3 I=N-1 I=N

Recoveryor Death

Recoveryor Death

Recoveryor Death

Recoveryor Death

Infection Infection Infection


Consider small amount of time �t, su�ciently small that only oneevent occurs: new infection, loss of infectious individual, nothing..Let

B(i) := �(N⇤ � i)i D(i) := (� + d)i

be rates of “birth” and “death” of infectious. Then

p

ji

(�t) = P {I (t +�) = j |I (t) = i}

=

8>>>><

>>>>:

B(i)�t j = i + 1

D(i)�t j = i � 1

1� [B(i) + D(i)]�t j = i

0 otherwise

I=0 I=1 I=2 I=3 I=N-1 I=N

Recoveryor Death

Recoveryor Death

Recoveryor Death

Recoveryor Death

Infection Infection Infectionβ(N−1)Δ t β(N−2)2Δ t β(N−1)Δ t

(γ+d )Δ t (γ+d )2Δt (γ+d )3Δ t (γ+d )N Δ t


Transition matrix

0

BBBBB@

1 D(1)�t0 1� (B(1) + D(1))�t0 B(1)�t...

. . .D(N)�t

0 0 0 · · · B(N � 1)�t 1� D(N)�t

1

CCCCCA

Note that indices in the matrix range from 0 to N here


Properties

p

0

is an absorbing state

For any initial distribution p(0) = (p0

(t), p1

(0), . . . , pN

(0)),

limt!1

p(t) = (1, 0, . . . , 0)T limt!1

p

0

(t) = 1

Exists quasi-stationary distribution conditioned on non-extinction

q

i

(t) =p

i

(t)

1� p

0

(t), i = 1, . . . ,N


Consequence of absorption

We have

limt!1

p(t) = (1, 0, . . . , 0)T limt!1

p

0

(t) = 1

but time to absorption can be exponential


1 Introduction



4 DTMC

5 CTMC


Several ways to formulate CTMC’s

A continuous time Markov chain can be formulated in terms of

• infinitesimal transition probabilities

• branching process

• time to next event

Here, time is in R+


For small �t,

p

ji

(�t) = P {I (t +�) = j |I (t) = i}

=

8>>>><

>>>>:

B(i)�t + o(�t) j = i + 1

D(i)�t + o(�t) j = i � 1

1� [B(i) + D(i)]�t + o(�t) j = i

o(�t) otherwise

with o(�t)! 0 as �t ! 0


Forward Kolmogorov equations

Assume we know I (0) = k . Then

p

i

(t +�t) = p

i�1

(t)B(i � 1)�t + p

i+1

(t)D(i + 1)�t

+ p

i

(t)[1� (B(i) + D(i))�t] + o(�t)

Compute (pi

(t +�t)� p

i

(t))/�t and take lim�t!0

, giving

d

dt

p

0

= p

1

D(1)

d

dt

p

i

= p

i�1

B(i � 1) + p

i+1

D(i + 1)� p

i

[B(i) + D(i)] i = 1, . . . ,N

Forward Kolmogorov equations associated to the CTMC


In vector form

Write previous system as

p

0 = Qp

with

Q =

0

BBBBBB@

0 D(1) 0 · · · 00 �(B(1) + D(1)) D(2) · · · 00 B(1) �(B(2) + D(2)) · · · 0

D(N)�D(N)

1

CCCCCCA

Q generator matrix. Of course,

p(t) = e

Qt

p(0)


Linking DTMC and CTMC for small �t

DTMC:p(t +�t) = P(�t)p(t)

for transition matrix P(�t). Let �t ! 0, obtain Kolmogorovequations for CTMC

d

dt

p = Qp

where

Q = lim�t!0

P(�t)� I�t

= P

0(0)


Going ODE to CTMC

Easy to convert compartmental ODE model to CTMC!!


The general setting: flow diagram

S Ib

βSI

γ I

dS dI


ODE approach

S Ib

βSI

γ I

dS dI

Focus on compartments

Sb

βSI

γ I

dS

I

βSI

γ I

dI


CTMC approach

S Ib

βSI

γ I

dS dI

Focus on arrows (transitions)

S Ib

βSI

γ I

dS dI


Focus on transitions

S Ib

βSI

γ I

dS dI

•b: S ! S + 1 birth

•dS : S ! S � 1 death of susceptible

• �SI : S ! S � 1, I ! I + 1 new infection

• �I : S ! S + 1, I ! I � 1 recovery

•dI : I ! I � 1 death of infectious

Will use S = N

⇤ � I and omit S dynamics


Gillespie’s algorithm (SIS model)

1: Set t t

0

and I (t) I (t0

)2: while t t

f

do

3: ⇠t

�(N⇤ � i)i + �i + di

4: Draw ⌧t

from T s E(⇠t

)5: v [�(N⇤ � i)i/⇠

t

,�(N⇤ � i)i/⇠t

+ �i/⇠t

, 1]6: Draw ⇣

t

from U([0, 1])7: Find pos such that ⇣

t

� v(pos)8: switch (pos)9: case 1: New infection, I (t + ⌧

t

) = I (t) + 110: case 2: End of infectious period, I (t + ⌧

t

) = I (t)� 111: case 3: Death, I (t + ⌧

t

) = I (t)� 112: end switch

13: t t + ⌧t

14: end while


IMPORTANT: in Gillespie’s algorithm, we do not consider pii

, theevent “nothing happens”


Drawing at random from an exponential distributionWant ⌧

t

from T s E(⇠t

), i.e., T has probability density function

f (x , ⇠t

) = ⇠t

e

�⇠t

x

1

x�0

Use cumulative distribution function F (x , ⇠t

) =Rx

�1 f (s, ⇠t

) ds

F (x , ⇠t

) = (1� e

�⇠t

x)1x�0

which has values in [0, 1]. So draw ⇣ from U([0, 1]) and solveF (x , ⇠

t

) = ⇣ for x :

F (x , ⇠t

) = ⇣ , 1� e

�⇠t

x = ⇣

, e

�⇠t

x = 1� ⇣

, ⇠t

x = � ln(1� ⇣)

, x =� ln(1� ⇣)

⇠t

Documents

Stochastic modelling of epidemic spread · Introduction Stochastic processes SIS model DTMC CTMC Topological spaces and Borel -algebras Deﬁnition 3 (Topology) Let ⌦ 6= ; be an