Stochastic Representations for Jump Processes in …anderson/RecentTalks/2013...Stochastic Representations for Jump Processes in Biology with Applications to Numerical Methods David

Stochastic Representations for Jump Processes in Biologywith Applications to Numerical Methods

David F. Anderson∗

∗[email protected]

Department of Mathematics

University of Wisconsin - Madison

IMA Workshop: Stochastic Modeling of Biological Processes

May 16th, 2013

Outline

1. Will quickly develop representations for relevant models –Gillespie/chemical master equation.

I Used as models of biochemical components (Gene networks, viral infection)or numbers of open ion channels, etc.

2. Will consider how to best numerically approximate (via Monte Carlo)

Ef (X (t))

∇θEf (θ,X (θ, t)). (If time.)

Key issues are

I bias (inherent error in numerical method)

I statistical error (confidence intervals)

3. Focus is on utilizing mathematical representations to betterunderstand/develop computational methods.

Outline

I This talk is quite different from rest of talks: Methodological, MonteCarlo, not really focussed on neural networks.

I This means my most exciting graphic is a chart listing computationalcomplexities!

I HOWEVER, there is a true miracle in the talk – Multi-level Monte Carlo,

I and the hopeful take-home message: structure can be utilized incomputing, and biological problems have lots of structure! (And“coupling” is very useful)

I Alternate hope (of course): someone has a hard computational problemthat could be attacked with MLMC.

Outline – the whyThere is a long history, with a huge literature, for Monte Carlo methods in

I Discrete event simulation (operations research, Generalizedsemi-Markov processes, queuing models)

I Mathematical finance

I SDEs driven by Brownian motion

Overall goal of this line of research is to develop methods specifically forbiological applications that are just as sophisticated as methods for fieldslisted above.

Above referenced literature is very large:I Ef (X (t)): Discrete event simulation literature (HUGE: Glynn, Haas, etc.),

Gillespie, Petzold, Khammash, Burrage, Giles, Tempone, Erban,Isaacson,....

I ∇θEf (θ,X (θ, t)): Glynn, Arkin, Glassermann, Hu, Fu, Khammash,...

and will utilize ideas found there (variance reduction, etc.), together with newones for our specific setting.

Biological example: transcription-translation

Gene transcription & translation:(with negative auto-regulation)

Gκ1→ G + M transcription

Mκ2→ M + P translation

Mκ3→ ∅ degradation

Pκ4→ ∅ degradation

G + Pκ5�κ−5

B Binding/unbinding of Gene

Cartoon representation:

1

1J. Paulsson, Physics of Life Reviews, 2, 2005 157 – 175.

Another example: Viral infectionLet

1. T = viral template.2. G = viral genome.3. S = viral structure.4. V = virus.

Reactions:

R1) T + “stuff”κ1→ T + G κ1 = 1

R2) Gκ2→ T κ2 = 0.025

R3) T + “stuff”κ3→ T + S κ3 = 1000

R4) Tκ4→ ∅ κ4 = 0.25

R5) Sκ5→ ∅ κ5 = 2

R6) G + Sκ6→ V κ6 = 7.5× 10−6

I R. Srivastava, L. You, J. Summers, and J. Yin, J. Theoret. Biol., 2002.I E. Haseltine and J. Rawlings, J. Chem. Phys, 2002.I K. Ball, T. Kurtz, L. Popovic, and G. Rempala, Annals of Applied Probability, 2006.I W. E, D. Liu, and E. Vanden-Eijden, J. Comput. Phys, 2006.

Morris-Lecar

Voltage satisfying

dVdt

= f (V (t),N(t))

=1C(Iapp − gCam∞(V (t))(V (t)− VCa)− VK )− gL(V − VL)− go

K N(t)(V (t)).

with number of open potassium channels following

0

(Ntot)α−→←−β

1

(Ntot − 1)α−→←−2β

2 · · · (k−1)

(Ntot − k + 1)α−→←−kβ

k

(Ntot − k)α−→←−

(k + 1)β

(k+1) · · · (Ntot−1)

α−→←−

(Ntot)β

Ntot

Modeling

1. These models (and much more complicated ones) have historically beenpredominantly modeled using ODEs.

2. However: there are often low numbers of molecules, which makes timingof reactions more random (less averaging),

3. Researchers (mostly) lived with these shortcomings until the late 1990sand early 2000s when it was shown ODE models can not captureimportant qualitative behavior of certain models:

I Green fluorescent protein.

ODEs were often the wrong modeling choice.

Modeling

Q: What is a better modeling choice? Should be

1. discrete space, since counting molecules, and

2. stochastic dynamics, to account for random reaction times.

Model development for a simple model

For example, consider the simple system

A + B → C

where one molecule each of A and B is being converted to one of C.

Simple book-keeping says: if

X (t) =

XA(t)XB(t)XC(t)

gives the state at time t , then

X (t) = X (0) + R(t)

−1−11

,

whereI R(t) is the # of times the reaction has occurred by time t andI X (0) is the initial condition.

Goal: represent R(t) in terms of Poisson process.

Models of interest

Recall that for A + B → C our intuition is to specify infinitesimal behavior

P{reaction occurs in (t , t + ∆t ]∣∣Ft} ≈ κXA(t)XB(t)∆t ,

and that for a non-homogeneous Poisson process with intensity/propensityλ(t) we have

P{Yλ(t + ∆t)− Yλ(t) ≥ 1|Ft} = 1− P{Yλ(t + ∆t)− Yλ(t) = 0|Ft}

= 1− exp{∫ t+∆t

tλ(s)ds

}≈ λ(t)∆t .

This suggests we can model

R(t) = Y(∫ t

0κXA(s)XB(s)ds

)where Y is a unit-rate Poisson process.

Models of interest: A + B → C

Hence XA(t)XB(t)XC(t)

≡ X (t) = X (0) +

−1−11

Y(∫ t

0κXA(s)XB(s)ds

).

This equation uniquely determines X for all t ≥ 0.

Build up model: Random time change representation of Tom Kurtz• Now consider a network of reactions involving d chemical species,

S1, . . . ,Sd :d∑

i=1

νik Si −→d∑

i=1

ν′ik Si

Denote reaction vector as

ζk = ν′k − νk ,

so that if reaction k occurs at time t

X (t) = X (t−) + ζk .

• The intensity (or propensity) of k th reaction is λk : Zd≥0 → R.

• By analogy with before:

X (t) = X (0) +∑

k

Rk (t)ζk ,

with

X (t) = X (0) +∑

k

Yk

(∫ t

0λk (X (s))ds

)ζk ,

Yk are independent, unit-rate Poisson processes.

Example: ODE Lotka-Volterra predator-prey model

Think of A as a prey and B as a predator.

Aκ1→ 2A, A + B

κ2→ 2B, Bκ3→ ∅,

with κ1 = 2, κ2 = .002, κ3 = 2.

Deterministic model. Let x(t) = [#prey,#predators]T

x(t) = x(0) + κ1

∫ t

0x1(s)ds

[10

]+ κ2

∫ t

0x1(s)x2(s)ds

[−11

]+ κ3

∫ t

0x2(s)ds

[0−1

]

Stochastic model. Let X (t) = [#prey,#predators]T

X(t) = X(0) + Y1

(κ1

∫ t

0X1(s)ds

)[10

]+ Y2

(κ2

∫ t

0X1(s)X2(s)ds

)[−11

]

+ Y3

(κ3

∫ t

0X2(s)ds

)[0−1

]

Morris-Lecar

Voltage satisfying

dVdt

=1C(Iapp − gCam∞(V (t))(V (t)− VCa)− VK )− gL(V − VL)− go

K N(t)(V (t)).

with number of open potassium channels following

0

(Ntot)α−→←−β

1

(Ntot − 1)α−→←−2β

2 · · · (k−1)

(Ntot − k + 1)α−→←−kβ

k

(Ntot − k)α−→←−

(k + 1)β

(k+1) · · · (Ntot−1)

α−→←−

(Ntot)β

Ntot

N(t) = N(0)− Yclose

(∫ t

0β(V (s))N(s)ds

)+ Yopen

(∫ t

0α(V (s))(Ntot − N(s))ds

).

Other ways to understand model

We can also describe the model as a continuous time Markov chain withinfinitesimal generator

Af (x) =∑

k

λk (x)(f (x + ζk )− f (x)).

And we have Dynkin’s formula (See Ethier and Kurtz, 1986, Ch. 1)

Ef (X (t))− f (X0) = E∫ t

0Af (X (s))ds,

Letting f (y) = 1x (y), above so that

E[f (X (t))] = P{X (t) = x} = px (t),

gives Kolmogorov forward equation (chemical master equation)

p′t (x) =∑

k

λ(x − ζk )pt (x − ζk )− pt (x)∑

k

λk (x)

Simulation

Simulating random time change representation is called the next reactionmethod.

There is also the Gillespie algorithm: simulating the embedded discretetime Markov chain.

We need to simulate at each step

1. A holding time: exponential with parameter

λ0(X (t)) =∑

k

λk (X (t)).

2. Which reaction: k th with probability

λk (X (t))

λ0(X (0)).

Computation of expectations using Monte Carlo – motivated by JohnYin at U. of Wisconsin

First problem for today: Approximate Ef (X (T )) to some desired tolerance,ε > 0.

Straightforward:

I Simulate the CTMC exactly (Gillespie’s algorithm, Next ReactionMethod),

I generate independent paths, X[i](t), use the unbiased/consistentestimator

µn =1n

n∑i=1

f (X[i](t))n→∞→ Ef (X (t)).

I stop when desired confidence interval is ±ε.

What is the computational cost?

Recall,

µn =1n

n∑i=1

f (X[i](t)).

Thus,

Var(µn) =1n

Var(f (X (t))).

So, if we wantσn = O(ε),

we need σf (X(t))√n

= O(ε) =⇒ n = O(σf (X(t))ε

−2).

If N gives average cost (steps) of a path using exact algorithm:

Total computational complexity = (cost per path)× (# paths)

= O(Nσf (X(t))ε−2).

Can be bad if (i) N · σf (X(t)), and/or (ii) ε is small.

Benefits/drawbacks

Benefits:

1. Easy to implement.

2. Estimator

µn =1n

n∑i=1

f (X[i](t))

is unbiased and consistent.

Drawbacks:

1. The cost of O(Nσf (X(t))ε−2) could be prohibitively large.

2. For our models, we sometimes have that N is very large.

Let’s try an approximate method.

Tau-leaping: Euler’s method

Explicit tau-leaping 2 or Euler’s method, was first formulated by Dan Gillespiein this setting.

Tau-leaping is essentially an Euler approximation of∫ t

0λk (X (s))ds:

Z (h) = Z (0) +∑

k

Yk

(∫ h

0λk (Z (s)) ds

)ζk

≈ Z (0) +∑

k

Yk

(λk (Z (0)) h

)ζk

d= Z (0) +

∑k

Poisson(λk (Z (0)) h

)ζk .

2D. T. Gillespie, J. Chem. Phys., 115, 1716 – 1733.

Euler’s method

Path-wise representation for Z (t) generated by Euler’s method is

Z (t) = X (0) +∑

k

Yk

(∫ t

0λk (Z ◦ η(s))ds

)ζk ,

where

η(s) =⌊ s

h

⌋h, =⇒ η(s) = tn if tn ≤ s < tn+1 = tn + h

is a step function giving left endpoints of time discretization.

Return to approximating Ef (X (T ))Let Z` denote an approximate processes generated with time discretizationstep of h` . Let

µn =1n

n∑i=1

f (Z`,[i](t)).

We note

Ef (X (t))− µn =[Ef (X (t))− Ef (ZL(t))

]+ Ef (ZL(t))− µn

= bias + statistical error.

Suppose have an order one method

Ef (X (t))− Ef (ZL(t)) = O(hL).

We need:1. hL = O(ε).2. n = ε−2.

Suppose a path costs O(ε−1) steps. Then

Total computational complexity = (# paths)× (cost per path)

= O(ε−3).

Benefits/drawbacks

Benefits:

1. Can drastically lower the computational complexity of a problem ifε−1 � N.

CC of using exact = Nε−2

CC of using approximate = ε−1ε−2.

Drawbacks:

1. Convergence results usually give order of convergence. Can’t give aprecise hL. Bias is a problem.

2. Tau-leaping has problems: what happens if you go negative?(Alvaro Moraes - Chernoff bounds)

3. Gone away from an unbiased estimator.

Recap

I Exact simulation is unbiased/consistent but potentially unusably slow.

I Tau-leaping/Euler’s method is fast, but has a difficult to quantify bias.

I We would like:

I Unbiased

I As fast as tau-leaping (on a large class of models) with crude step-size.

I For this, we use multi-level Monte Carlo.

5/15/13 5:26 PMMike Giles - Multilevel Monte Carlo method

Page 1 of 5http://people.maths.ox.ac.uk/gilesm/mlmc_community.html

Multilevel Monte Carlo research

In this page I attempt to list the research groups working on multilevel Monte Carlo methods, andthe main papers and presentations that I am aware of, grouped by topic.

Research groups

Abo Academi (Avikainen) -- numerical analysisBasel (Harbrecht) -- elliptic SPDEs, sparse gridsBath (Kyprianou, Scheichl, Teckentrup, Ullmann) -- elliptic SPDEsDuisburg (Belomestny) -- Bermudan and American optionsEPFL (Abdulle) -- stiff SDEs and SPDEsETH Zürich (Barth, Jentzen, Lang, Schwab) -- SPDEsFrankfurt (Gerstner, Kloeden) -- numerical analysisFraunhofer ITWM (Iliev) -- SPDEs in engineeringHong Kong (Chen) -- Brownian meanders, nested simulation in financeIIT Chicago (Hickernell) -- SDEs, infinite-dimensional integration, complexity analysisKaiserslautern (Heinrich, Korn, Ritter) -- finance, SDEs, parametric integration, complexityanalysisKAUST (Tempone, von Schwerin) -- adaptive time-steppingKiel (Gnewuch) -- randomized multilevel QMCKTH (Szepessy) -- adaptive time-steppingMannheim (Neuenkirch) -- numerical analysisMarburg (Dereich) -- Lévy-driven SDEsMunich (Hutzenthaler) -- numerical analysisNottingham (Cliffe, Park) -- elliptic SPDEsOxford (Giles, Hambly, Reisinger, Szpruch) -- SDEs, SPDEs, numerical analysis, financeapplicationsPassau (Müller-Gronbach) -- infinite-dimensional integration, complexity analysisPurdue (Gittelson) -- SDPEsStanford (Glynn) -- numerical analysisStrathclyde (Higham, Mao) -- numerical analysis, exit times, stochastic chemical modellingTexas A&M (Efendiev) -- SPDEs in engineeringUCLA (Caflisch) -- Coulomb collisions in physicsUNSW (Kuo, Sloan) -- multilevel QMCUTS (Baldeaux) -- multilevel QMCWarwick (Stuart) -- MCMC for SPDEsWIAS (Schoenmakers) -- Bermudan and American optionsWisconsin (Anderson) -- numerical analysis, stochastic chemical modelling

Brownian SDEs

A. Abdulle, A. Blumenthal. 'Stabilized multilevel Monte Carlo method for stiff stochasticdifferential equations'. Preprint, 2013. linkM.B. Alaya, A. Kebaier. 'Central limit theorem for the multilevel Monte Carlo Euler method and



applications to Asian options'. HAL preprint, 2012. linkR. Avikainen. 'On irregular functionals of SDEs and the Euler scheme'. Finance andStochastics, 13(3):381-401, 2009. linkD. Belomestny, F. Dickmann, T. Nagapetyan. 'Pricing American options via multi-levelapproximation methods'. Preprint, 2013. linkD. Belomestny, J. Schoenmakers. 'Multilevel dual approach for pricing American stylederivatives'. To appear in Finance and Stochastics, 2013. linkS. Burgos, M.B. Giles. `Computing Greeks using multilevel path simulation'. pp.281-296 inMonte Carlo and Quasi-Monte Carlo Methods 2010. Springer, 2012. linkN. Chen, Z. Huang. `Brownian meanders, importance sampling and unbiased simulation ofdiffusion extremes'. Operations Research Letters, 40(6):554-563, 2012. linkT. Gerstner, S. Heinz. 'Dimension- and time-adaptive multilevel Monte Carlo methods'.pp.107-120 in Sparse Grids and Applications, Lecture Notes in Computational Science andEngineering, Volume 88, 2013. linkT. Gerstner, M. Noll. 'Randomized multilevel quasi-Monte Carlo path simulation'. In RecentDevelopments in Computational Finance, World Scientific / Imperial College Press, 2013. linkM.B. Giles. `Multi-level Monte Carlo path simulation'. Operations Research, 56(3):607-617,2008. linkM.B. Giles. `Improved multilevel Monte Carlo convergence using the Milstein scheme'.pp.343-358, in Monte Carlo and Quasi-Monte Carlo Methods 2006, Springer, 2008. linkM.B. Giles, D.J. Higham, X. Mao. 'Analysing multilevel Monte Carlo for options with non-globally Lipschitz payoff'. Finance and Stochastics, 13(3):403-413, 2009. linkM.B. Giles, B.J. Waterhouse. 'Multilevel quasi-Monte Carlo path simulation'. pp.165-181 inAdvanced Financial Modelling, in Radon Series on Computational and Applied Mathematics,de Gruyter, 2009. linkM.B. Giles. `Multilevel Monte Carlo for Basket Options'. Winter Simulation Conference, 2009.linkM.B. Giles. `Multilevel Monte Carlo methods'. To appear in Monte Carlo and Quasi-MonteCarlo Methods 2012, Springer, 2013. linkM.B. Giles, L. Szpruch. 'Antithetic multilevel Monte Carlo estimation for multi-dimensionalSDEs without Lévy area simulation'. Preprint, 2012. linkM.B. Giles, L. Szpruch. 'Multilevel Monte Carlo methods for applications in finance'. In RecentDevelopments in Computational Finance, World Scientific / Imperial College Press, 2013. linkM.B. Giles, K. Debrabant, A. Roessler. 'Numerical analysis of multilevel Monte Carlo pathsimulation using the Milstein discretisation'. Preprint, 2013. linkD.J. Higham, X. Mao, M. Roj, Q. Song, G. Yin. 'Mean exit times and the multi-level MonteCarlo method'. SIAM Journal on Uncertainty Quantification, 1(1):2-18, 2013. linkD.J. Higham, M. Roj. 'Computing mean first exit times for stochastic processes using multi-level Monte Carlo'. Proceedings of the 2012 Winter Simulation Conference, 2013. linkH. Hoel, E. von Schwerin, A. Szepessy, R. Tempone. 'Adaptive multilevel Monte Carlosimulation'. Numerical Analysis of Multiscale Computations, 82:217-234, 2012. linkH. Hoel, E. von Schwerin, A. Szepessy, R. Tempone. 'Implementation and analysis of anadaptive multilevel Monte Carlo algorithm'. Preprint, 2010. linkM. Hutzenthaler, A. Jentzen, P. Kloeden. 'Divergence of the multilevel Monte Carlo method'.Preprint, 2011. linkA. Kebaier. 'Statistical Romberg extrapolation: a new variance reduction method andapplications to option pricing'. Annals of Applied Probability, 15(4):2681-2705, 2005. link



P. Kloeden, A. Neuenkirch, R. Pavani. 'Multilevel Monte Carlo for stochastic differentialequations with additive fractional noise'. 189(1):255-276, Annals of Operations Research,2011. linkT. Primozic. 'Estimating expected first passage times using multilevel Monte Carlo algorithm'.MSc dissertation, 2012. linkC. Rhee, P. Glynn. 'A new approach to unbiased estimation for SDEs'. Preprint, 2012. link

Lévy-driven SDEs

S. Dereich. 'Multilevel Monte Carlo algorithms for Lévy-driven SDEs with Gaussiancorrection', Annals of Applied Probability, 21(1):283-311, 2011. linkS. Dereich, F. Heidenreich. 'A multilevel Monte Carlo algorithm for Lévy-driven stochasticdifferential equations', Stochastic Processes and their Applications, 121(7):1565-1587, 2011.linkA. Ferreiro-Castilla, A.E. Kyprianou, R. Scheichl, G. Suryanarayana. 'Multi-level Monte-Carlosimulation for Lévy processes based on the Wiener-Hopf factorisation'. Preprint, 2012. linkH. Marxen. 'The Multilevel Monte Carlo method used on a Lévy driven SDE'. Monte CarloMethods and Applications, 16(2):167-190, 2010. linkY. Xia, M.B. Giles. `Multilevel path simulation for jump-diffusion SDEs', pp.695-708 in MonteCarlo and Quasi-Monte Carlo Methods 2010, Springer, 2012. link

Complexity analysis

J. Baldeaux, M. Gnewuch. 'Optimal randomized multilevel algorithms for infinite-dimensionalintegration on function spaces with ANOVA-type decomposition'. Preprint, 2012. linkJ. Creutzig, S. Dereich, T. Müller-Gronbach, K. Ritter. 'Infinite-dimensional quadrature andapproximation of distributions'. Foundations of Computational Mathematics, 9(4):391-429,2009. linkJ. Dick, M. Gnewuch. 'Infinite-dimensional integration in weighted Hilbert spaces: anchoreddecompositions, optimal deterministic algorithms, and higher order convergence'. Preprint,2012. linkM. Gnewuch. 'Infinite-dimensional integration on weighted Hilbert spaces'. Mathematics ofComputation, 81(280):2175-2205, 2012. linkM. Gnewuch. 'Lower error bounds for randomized multilevel and changing dimensionalgorithms'. Preprint, 2012. linkS. Heinrich. 'Monte Carlo complexity of global solution of integral equations'. Journal ofComplexity, 14(2):151-175, 1998. linkS. Heinrich, E. Sindambiwe. 'Monte Carlo complexity of parametric integration'. Journal ofComplexity, 15(3):317-341, 1999. linkS. Heinrich. 'Monte Carlo approximation of weakly singular integral operators'. Journal ofComplexity, 22(2):192-219, 2006. linkF.J. Hickernell, T. Müller-Gronbach, B. Niu, K. Ritter. 'Multi-level Monte Carlo algorithms forinfinite-dimensional integration on '. Journal of Complexity, 26(3):229-254, 2010. linkT. Müller-Gronbach, K. Ritter. 'Variable subspace sampling and multi-level algorithms'.pp.131-156 in Monte Carlo and Quasi-Monte Carlo Methods 2008, Springer, 2009. linkB. Niu, F.J. Hickernell, T. Müller-Gronbach and K. Ritter. 'Deterministic multi-level algorithms



for infinite-dimensional integration on '. Journal of Complexity, 27(3-4):331-351, 2010. linkB. Niu. 'Monte Carlo simulation of infinite-dimensional integrals'. PhD thesis, Illinois Instituteof Technology, 2011. link

SPDEs

A. Abdulle, A. Barth, C. Schwab. 'Multilevel Monte Carlo methods for stochastic ellipticmultiscale PDEs'. Preprint, 2012. linkA. Barth, A. Lang. 'Multilevel Monte Carlo method with applications to stochastic partialdifferential equations'. Int Journal of Computer Mathematics, 89(18):2479-2498, 2012. linkA. Barth, C. Schwab, N. Zollinger. 'Multi-level Monte Carlo finite element method for ellipticPDEs with stochastic coefficients'. Numerische Mathematik, 119(1):123-161, 2011. linkJ. Charrier, R. Scheichl, A. Teckentrup. 'Finite element error analysis of elliptic PDEs withrandom coefficients and its application to multilevel Monte Carlo methods'. SIAM Journal onNumerical Analysis, 51(1):322-352, 2013. linkK.A. Cliffe, M.B. Giles, R. Scheichl, A.L. Teckentrup. 'Multilevel Monte Carlo methods andapplications to elliptic PDEs with random coefficients'. Computing and Visualization inScience, 14(1):3-15, 2011. linkY. Efendiev, O. Iliev, C. Kronsbein. 'Multi-level Monte Carlo methods using ensemble levelmixed MsFEM for two-phase flow and transport simulations'. Fraunhofer ITWM report 217,2012. linkY. Efendiev, C. Kronsbein, F. Legoli. 'Multi-level Monte Carlo approaches for numericalhomogenization'. Preprint, 2013. linkM.B. Giles. 'Multilevel Monte Carlo simulation', 2009. linkM.B. Giles, C. Reisinger. 'Stochastic finite differences and multilevel Monte Carlo for a classof SPDEs in finance', SIAM Journal of Financial Mathematics, 3(1):572-592, 2012. linkC.J. Gittelson, J. Konno, C. Schwab and R. Stenberg. 'The multi-level Monte Carlo finiteelement method for a stochastic Brinkman problem'. Preprint, 2012. linkS. Graubner. 'Multi-level Monte Carlo Methoden für stochastische partielleDifferentialgleichungen'. Diplomarbeit, TU Darmstadt, 2008. linkH. Harbrecht, M. Peters, and M. Siebenmorgen. 'On multilevel quadrature for ellipticstochastic partial differential equations'. Preprint 2011-01, Mathematisches Institut, UniversitätBasel, 2011. linkV.H. Hoang, C. Schwab, A.M. Stuart. 'Sparse MCMC gpc finite element methods for Bayesianinverse problems'. Preprint, 2012 linkY.T. Hou, M. Cheng, M. Yan. 'A Data-driven Stochastic Multiscale Method', 2010. linkC. Ketelsen, R. Scheichl, A.L. Teckentrup. 'A hierarchical multilevel Markov Chain MonteCarlo algorithm with applications to uncertainty quantification in subsurface flow'. Preprint,2013. linkC. Kronsbein. 'On selected efficient numerical methods for multiscale problems withstochastic coefficients'. PhD thesis, 2012. linkF.Y. Kuo, C. Schwab, I. Sloan. 'Multi-level quasi-Monte Carlo finite element methods for aclass of elliptic partial differential equations with random coefficients'. Preprint, 2012. linkS. Mishra, C. Schwab, J. Sukys. 'Multi-level Monte Carlo finite volume methods for nonlinearsystems of conservation laws in multi-dimensions', Journal of Computational Physics,231(8):3365-3388, 2012. link



F. Müller, P. Jenny, D.W. Meyer. 'Multilevel Monte Carlo for two phase flow and Buckley-Leverett transport in random heterogeneous porous media'. Journal of ComputationalPhysics, in press, 2013. linkN.H. Risebro, C. Schwab, F. Weber. 'Multi-level Monte Carlo front tracking for random scalarconservation laws'. Preprint, 2012. linkA.L. Teckentrup, R. Scheichl, M.B. Giles, E. Ullmann. 'Further analysis of multilevel MonteCarlo methods for elliptic PDEs with random coefficients', to appear in NumerischeMathematik, 2013. linkA.L. Teckentrup. 'Multilevel Monte Carlo methods for highly heterogeneous media'. WinterSimulation Conference, 2012. link

Other applications

D. Anderson, D.J. Higham. 'Multi-level Monte Carlo for continuous time Markov chains, withapplications in biochemical kinetics'. SIAM Multiscale Modelling and Simulation, 10(1):146-179, 2012. linkA. Brandt, M. Galun, D. Ron. 'Optimal multigrid algorithms for calculating thermodynamiclimits'. Journal of Statistical Physics, 74(1-2):313-348, 1994. linkA. Brandt, V. Ilyin. 'Multilevel Monte Carlo methods for studying large scale phenomena influids'. Journal of Molecular Liquids, 105(2-3):245-248, 2003. linkK. Bujok, B. Hambly, C. Reisinger 'Multilevel simulation of functionals of Bernoulli randomvariables with application to basket credit derivatives'. Preprint, 2012. linkA.M. Dimits, B.I. Cohen, R.E. Caflisch, M.S. Rosin, L.F. Ricketson. 'Higher-order and multi-level time integration of stochastic differential equations and application to Coulombcollisions'. Journal of Computational Physics, 242:561-580, 2013. linkS. Heinrich. 'Multilevel Monte Carlo methods'. Lecture Notes in Computer Science, 2179:58-67, 2001. linkS. Heinrich. 'The multilevel method of dependent tests'. In Advances in Stochastic SimulationMethods, Springer, 2000. linkD.J. Higham. 'Stochastic ordinary differential equations in applied and computationalmathematics'. IMA Journal of Applied Mathematics, 76(3):449-474, 2011. linkA.L. Speight. 'A multilevel approach to control variates'. Journal of Computational Finance,12:1-25, 2009. linkA.L. Speight. 'Multigrid techniques in economics'. Operations Research, 58(4):1057-1078,2010. link

BiBTeX file

Download

Please tell me of any errors in this bib file, or any references which need to be updated.

Multi-level Monte Carlo: Main idea (Mike Giles)I Suppose I want

EX ≈ 1n

n∑i=1

X[i],

but realizations of X are expensive.

I Suppose X ≈ ZL, and ZL is cheap.

I Suppose X , ZL can be generated simultaneously (built on sameprobability space) so that

Var(X − ZL)

is small.

I Then use

EX = E[X − ZL] + EZL ≈1n1

n1∑i=1

(X[i] − ZL,[i]) +1n2

n2∑i=1

ZL,[i].

I MLMC idea: keep going

EX = E(X − ZL) + EZL = E(Z − ZL) + E(ZL − ZL−1) + EZL−1 = · · ·

Multi-level Monte Carlo: Main idea (Mike Giles)

First recall notation:

X (t) ∼ exact process, Z` ∼ tau-leaping with step size of h` =1

M`.

So, we can think of multiple versions of tau-leaping:

ZL, ZL−1, ZL−1, . . . , Z`0 .

Now simply make sequence of control variates:

Ef (X (t)) = E[f (X (t))− f (ZL(t))] + Ef (ZL(t))

= E[f (X (t))− f (ZL(t))] + E[f (ZL(t))− f (ZL−1(t))] + Ef (ZL−1(t))

...

= E[f (X (t))− f (ZL(t))] +L∑

`=`0+1

E[f (Z`(t))− f (Z`−1(t))] + Ef (Z`0 (t)).

Multi-level Monte Carlo: Main idea (Mike Giles – Oxford)

Ef (X (t)) = E[f (X (t))− f (ZL(t))] +L∑

`=`0+1

E[f (Z`(t))− f (Z`−1(t))] + Ef (Z`0 (t)).

For appropriate choices of n0, n`, and nE , we define the estimators for thethree terms above via

QEdef=

1nE

nE∑i=1

(f (X[i](T )− f (ZL,[i](T ))),

Q`def=

1n`

n∑i=1

(f (Z`,[i](T ))− f (Z`−1,[i](T ))), for ` ∈ {`0 + 1, . . . , L}

Q0def=

1n0

n0∑i=1

f (Z`0,[i](T )),

and note that

Q def= QE +

L∑`=`0

Q`

is an unbiased/consistent estimator for Ef (X (T )).

So what is coupling and the variance? NOTE THAT THIS IS THE WHOLEGAME!

How do we generate processes simultaneously – coupling

Suppose I want to generate:I A Poisson process with intensity 13.1.I A Poisson process with intensity 13.

I We could let Y1 and Y2 be independent, unit-rate Poisson processes,and set

Z13.1(t) = Y1(13.1t),

Z13(t) = Y2(13t),

Using this representation, these processes are independent and, hence,not coupled.

The variance of difference is large:

Var(Z13.1(t)− Z13(t)) = Var(Y1(13.1t)) + Var(Y2(13t))

= 26.1t .

How do we generate processes simultaneously

Suppose I want to generate:I A Poisson process with intensity 13.1.I A Poisson process with intensity 13.

I We could let Y1 and Y2 be independent unit-rate Poisson processes, andset

Z13.1(t) = Y1(13t) + Y2(0.1t)

Z13(t) = Y1(13t),

The variance of difference is much smaller:

Var(Z13.1(t)− Z13(t)) = Var (Y2(0.1t)) = 0.1t .

How do we generate processes simultaneously

Suppose we want

1. non-homogeneous Poisson process with intensity f (t) and

2. non-homogeneous Poisson process with intensity g(t).

We can can let Y1, Y2, and Y3 be independent, unit-rate Poisson processesand define

Zf (t) = Y1

(∫ t

0f (s) ∧ g(s)ds

)+ Y2

(∫ t

0f (s)− (f (s) ∧ g(s)) ds

),

Zg(t) = Y1

(∫ t

0f (s) ∧ g(s)ds

)+ Y3

(∫ t

0g(s)− (f (s) ∧ g(s)) ds

),

where we are using that, for example,

Y1

(∫ t

0f (s) ∧ g(s)ds

)+ Y2

(∫ t

0f (s)− (f (s) ∧ g(s)) ds

)d= Y

(∫ t

0f (s)ds

),

where Y is a unit rate Poisson process.

Back to our processes

X (t) = X (0) +∑

k

Yk

(∫ t

0λk (X (s))ds

)ζk ,

Z (t) = X (0) +∑

k

Yk

(∫ t

0λk (Z ◦ η(s))ds

)ζk .

Now couple

X (t) =X (0) +∑

k

Yk,1

(∫ t

0λk (X (s)) ∧ λk (Z` ◦ η`(s))ds

)ζk

+∑

k

Yk,2

(∫ t

0λk (X (s))− λk (X (s)) ∧ λk (Z` ◦ η`(s))ds

)ζk

Z`(t) =Z`(0) +∑

k

Yk,1

(∫ t

0λk (X (s)) ∧ λk (Z` ◦ η`(s))ds

)ζk

+∑

k

Yk,3

(∫ t

0λk (Z` ◦ η`(s))− λk (X (s)) ∧ λk (Z` ◦ η`(s))ds

)ζk

For approximate processes

Z`(t) = Z`(0) +∑

k

Yk,1

(∫ t

0λk (Z` ◦ η`(s)) ∧ λk (Z`−1 ◦ η`−1(s))ds

)ζk

+∑

k

Yk,2

(∫ t

0λk (Z` ◦ η`(s))− λk (Z` ◦ η`(s)) ∧ λk (Z`−1 ◦ η`−1(s))ds

)ζk

Z`−1(t) = Z`−1(0) +∑

k

Yk,1

(∫ t

0λk (Z` ◦ η`(s)) ∧ λk (Z`−1 ◦ η`−1(s))ds

)ζk

+∑

k

Yk,3

(∫ t

0λk (Z`−1 ◦ η`−1(s))− λk (Z` ◦ η`(s)) ∧ λk (Z`−1 ◦ η`−1(s))ds

)ζk ,

Multi-level Monte Carlo: chemical kinetic setting

Can prove:

Theorem (A., Higham 2011)Suppose (X ,Z`) satisfy coupling. Then, there exist positive constantsC1(T ),C2(T ) > 0, such that

supt≤T

E|X (t)− Z`(t)|2 ≤ C1(T )N−ρh` + C2(T )h2` .

Theorem (A., Higham 2011)Suppose (Z`,Z`−1) satisfy coupling. Then, there exist positive constantsC1(T ),C2(T ) > 0, such that

supt≤T

E|Z`(t)− Z`−1(t)|2 ≤ C1(T )N−ρh` + C2(T )h2` .

2David F. Anderson and Desmond J. Higham, Multi-level Monte Carlo for continuous time Markovchains, with applications in biochemical kinetics, SIAM: Multiscale Modeling and Simulation, Vol. 10,No. 1, 146 - 179, 2012.

Multi-level Monte Carlo: an unbiased estimatorFor well chosen n0, n`, and nE . We want

Var(Q) = Var

QE +L∑

`=`0

Q`

=

1nE

CE (N−ρh` + h2`) +

L∑`=`0+1

1n`

C`(N−ρh` + h2`) + C`0

1n`0

want= O(ε2),

while minimizing

Comp. cost = nE N +L∑

`=`0+1

n`h−1` + n`0 h−1

`0.

Solving yields

Comp. cost ≈[(N−ρ

hL + h2L)]ε−2N

+ ε−2(

h−1`0

+ ln(ε)2N−ρ

+ ln(ε−1)1

M − 1

)

Multi-level Monte Carlo: an unbiased estimator

Some observations:

1. Weak error of the method plays no role in analysis.

2. Common problems associated with tau-leaping

I Negativity of species numbers,

does not matter. Just define process in a sensible way.

3. The method is unbiased/consistent. This is different from otherapplications!

Example

Consider a model of gene transcription and translation:

G 25→ G + M,

M 1000→ M + P,

P + P 0.001→ D,

M 0.1→ ∅,

P 1→ ∅.

Suppose:

1. initialize with: G = 1, M = 0, P = 0, D = 0,

2. want to estimate the expected number of dimers at time T = 1,

3. to an accuracy of ± 1.0 with 95% confidence.

Example

Method: Exact algorithm with crude Monte Carlo.

Approximation # paths CPU Time # updates3,714.2 ± 1.0 4,740,000 149,000 CPU S (41 hours!) 8.27 ×1010

Method: Euler tau-leaping with crude Monte Carlo.

Step-size Approximation # paths CPU Time # updatesh = 3−7 3,712.3 ± 1.0 4,750,000 13,374.6 S 6.2× 1010

h = 3−6 3,707.5 ± 1.0 4,750,000 6,207.9 S 2.1× 1010

h = 3−5 3,693.4 ± 1.0 4,700,000 2,803.9 S 6.9× 109

h = 3−4 3,654.6 ± 1.0 4,650,000 1,219.0 S 2.6× 109

Example


Approximation # paths CPU Time # updates3,714.2 ± 1.0 4,740,000 149,000 CPU S (41 hours!) 8.27 ×1010

Method: unbiased MLMC with `0 = 2, and M and L detailed below.

Step-size parameters Approx. CPU Time # updatesM = 3, L = 6 3,713.9 ± 1.0 1,063.3 S 1.1 ×109

M = 3, L = 5 3,714.7 ± 1.0 1,114.9 S 9.4 ×108

M = 3, L = 4 3,714.2 ± 1.0 1,656.6 S 1.0 ×109

M = 4, L = 4 3714.2 ± 1.0 1,334.8 S 1.1 ×109

M = 4, L = 5 3,713.8 ± 1.0 1,014.9 S 1.1 ×109

I the exact algorithm with crude Monte Carlo demandedI 140 times more CPU time

than our unbiased MLMC estimator.

Some conclusions about this method

1. Gillespie’s algorithm is by far the most common way to computeexpectations:

1.1 Means.

1.2 Variances.

1.3 Probabilities.

2. The new method (MLMC) also performs this task with no bias (exact).

3. Can be orders of magnitude faster than Gillespie.

4. Much harder to implement.

5. Makes no use of any specific structure or scaling in the problem.

Another example: Viral infectionLet

1. T = viral template.

2. G = viral genome.

3. S = viral structure.

4. V = virus.

Reactions:

R1) T + stuffκ1→ T + G κ1 = 1

R2) Gκ2→ T κ2 = 0.025

R3) T + stuffκ3→ T + S κ3 = 1000

R4) Tκ4→ ∅ κ4 = 0.25

R5) Sκ5→ ∅ κ5 = 2

R6) G + Sκ6→ V κ6 = 7.5× 10−6

I R. Srivastava, L. You, J. Summers, and J. Yin, J. Theoret. Biol., 2002.I E. Haseltine and J. Rawlings, J. Chem. Phys, 2002.I K. Ball, T. Kurtz, L. Popovic, and G. Rempala, Annals of Applied Probability, 2006.I W. E, D. Liu, and E. Vanden-Eijden, J. Comput. Phys, 2006.

Another example: Viral infection

Stochastic equations for X = (XG,XS ,XT ,XV ) are

X1(t) = X1(0) + Y1

(∫ t

0X3(s)ds

)− Y2

(0.025

∫ t

0X1(s)ds

)− Y6

(7.5× 10−6

∫ t

0X1(s)X2(s)ds

)

X2(t) = X2(0) + Y3

(1000

∫ t

0X3(s)ds

)− Y5

(2∫ t

0X2(s)ds

)− Y6

(7.5× 10−6

∫ t

0X1(s)X2(s)ds

)

X3(t) = X3(0) + Y2

(0.025

∫ t

0X1(s)ds

)− Y4

(0.25

∫ t

0X3(s)ds

)

X4(t) = X4(0) + Y6

(7.5× 10−6

∫ t

0X1(s)X2(s)ds

).


Reactions:

R1) T + stuffκ1→ T + G κ1 = 1

R2) Gκ2→ T κ2 = 0.025

R3) T + stuffκ3→ T + S κ3 = 1000

R4) Tκ4→ ∅ κ4 = 0.25

R5) Sκ5→ ∅ κ5 = 2

R6) G + Sκ6→ V κ6 = 7.5× 10−6

If T > 0,I reactions 3 and 5 are much faster than others.I Looks like S is approximately Poisson(500× T ) ≈ 500 · T .

Can average out to get approximate process Z (t).


Approximate process satisfies.

Z1(t) = X1(0) + Y1

(∫ t

0Z3(s)ds

)− Y2

(0.025

∫ t

0Z1(s)ds

)− Y6

(3.75× 10−3

∫ t

0Z1(s)Z3(s)ds

)Z3(t) = X3(0) + Y2

(0.025

∫ t

0Z1(s)ds

)− Y4

(0.25

∫ t

0Z3(s)ds

)Z4(t) = X4(0) + Y6

(3.75× 10−3

∫ t

0Z1(s)Z3(s)ds

).

(1)

Now useEf (X (t)) = E[f (X (t))− f (Z (t))] + Ef (Z (t)).


X(t) = X(0) + Y1,1

(∫ t

0min{X3(s), Z3(s)}ds

)ζ1 + Y1,2

(∫ t

0X3(s)− min{X3(s), Z3(s)}ds

)ζ1

+ Y2,1

(0.025

∫ t


)ζ2 + Y2,2

(0.025

∫ t

0X1(s)− min{X1(s), Z1(s)}ds

)ζ2

+ Y3

(1000

∫ t

0X3(s)ds

)ζ3

+ Y4,1

(0.25

∫ t

0min{X3(s), Z3(s)}(s)ds

)ζ4 + Y4,2

(0.25

∫ t

0X3(s)− min{X3(s), Z3(s)}(s)ds

)ζ4

+ Y5

(2∫ t

0X2(s)ds

)ζ5

+ Y6,1

(∫ t

0min{λ6(X(s)), Λ6(Z (s))}ds

)ζ6 − Y6,2

(∫ t

0λ6(X(s))− min{λ6(X(s)), Λ6(Z (s))}ds

)ζ6

Z (t) = Y1,1

(∫ t


)ζ1 + Y1,3

(∫ t

0Z3(s)− min{X3(s), Z3(s)}ds

)ζ1

+ Y2,1

(0.025

∫ t


)ζ2 + Y2,3

(0.025

∫ t

0Z1(s)− min{X1(s), Z1(s)}ds

)ζ2

+ Y4,1

(0.25

∫ t

0min{X3(s), Z3(s)}(s)ds

)ζ4 + Y4,3

(0.25

∫ t

0Z3(s)− min{X3(s), Z3(s)}(s)ds

)ζ4

+ Y6,1

(∫ t

0min{λ6(X(s)), Λ6(Z (s))}ds

)ζ6 − Y6,3

(∫ t

0Λ6(Z (s))− min{λ6(X(s)), Λ6(Z (s))}ds

)ζ6,


Suppose wantEXvirus(20)

Given T (0) = 10, all others zero.


Approximation # paths CPU Time # updates13.85 ± 0.07 75,000 24,800 CPU S 1.45× 1010

Method: Ef (X (t)) = E[f (X (t))− f (Z (t))] + Ef (Z (t)).

Approximation CPU Time # updates13.91 ± 0.07 1,118.5 CPU S 2.41× 108

Exact + crude Monte Carlo used:

1. 60 times more total steps.

2. 22 times more CPU time.

Next problem: parameter sensitivities.

Motivated by Jim Rawlings.

We have a family of models indexed by vector of parameters θ

X (θ, t) = X (θ, 0) +∑

k

Yk

(∫ t

0λk (θ,X (θ, s))ds

)ζk .

and we define

J(θ) = E f (θ,X (θ, t)).

We want∇θJ(θ) = ∇θEf (θ,X (θ, t)).

There are multiple methods. We first consider finite differences:

J ′(θ) ≈ J(θ + ε)− J(θ)

ε+ O(ε).


Noting that

J ′(θ) =ddθ

Ef (X θ(t)) =Ef (X θ+ε(t))− Ef (X θ(t))

ε+ o(ε).

The usual finite difference estimator is

DR(ε) =1R

R∑i=1

f (X θ+ε[i] (t))− f (X θ

[i](t))

ε

If generated independently, then

Var(DR(ε)) = O(R−1ε−2).


Couple the processes.

X θ+ε(t) = X θ+ε(0) +∑

k

Yk,1

(∫ t

0λθ+ε

k (X θ+ε(s)) ∧ λθk (X θ(s))ds)ζk

+∑

k

Yk,2

(∫ t

0λθ+ε

k (X θ+ε(s))− λθ+εk (X θ+ε(s)) ∧ λθk (X θ(s))ds

)ζk

X θ(t) = X θ(0) +∑

k

Yk,1

(∫ t

0λθ+ε

k (X θ+ε(s)) ∧ λθk (X θ(s))ds)ζk

+∑

k

Yk,3

(∫ t

0λθk (X θ(s))− λθ+ε

k (X θ+ε(s)) ∧ λθk (X θ(s))ds)ζk ,


Theorem (Anderson, 2012)Suppose (X θ+ε,X θ) satisfy coupling. Then, for any T > 0 there is a CT ,f > 0for which

E supt≤T

(f (X θ+ε(t))− f (X θ(t))

)2≤ CT ,f ε.

This lowers variance of estimator from

O(R−1ε−2),

toO(R−1ε−1).

Lowered by order of magnitude (in ε).

2David F. Anderson, An efficient Finite Difference Method for Parameter Sensitivities ofContinuous Time Markov Chains, SIAM Journal on Numerical Analysis, Vol. 50, Issue 5, 2237 -2258, 2012

Parameter Sensitivities

G 2→ G + M,

M 10→ M + P,

M k→ ∅,

P 1→ ∅.

Want∂

∂kE[X k

protein(30)], k ≈ 1/4.

Method # paths Approximation # updates CPU TimeLikelihood Ratio 689,600 -312.1 ± 6.0 2.9× 109 3,506.6 S

Exact 246,200 -318.8 ± 6.0 2.1× 109 3,282.1 SCRP (RTC 1) 26,320 -320.7 ± 6.0 2.2× 108 410.0 SCFD (RTC 2) 4,780 -321.2 ± 6.0 2.1× 107 35.3 S

Analysis

TheoremSuppose (X θ+ε,X θ) satisfy coupling. Then, for any T > 0 there is a CT ,f > 0for which

E supt≤T

(f (X θ+ε(t))− f (X θ(t))

)2≤ CT ,f ε.

Proof:Key observation of proof:

X θ+ε(t)− X θ(t) = Mθ,ε(t) +

∫ t

0F θ+ε(X θ+ε(s))− F θ(X θ(s))ds,

where “most” of the jumps have vanished.

Now work on Martingale and absolutely continuous part.

We can now do even better: returning to the likelihood method (Glynn,Arkin)

Basic idea via example: Let Wλ ∼ exp(λ). Suppose I want (via Monte Carlo)

ddλ

E[W 3λ

].

Noting that

EW 3λ =

∫ ∞0

x3λe−λx dx

we see (assuming we can pass the limit)

ddλ

(EW 3

λ

)=

∫ ∞0

x3(e−λx − λxe−λx )dx

=

∫ ∞0

x3(λ−1 − x)λe−λx dx

= E[W 3λ(λ−1 −Wλ)

].

Can do the same for paths and find

ddθ

Ef (θ,X (θ, t)) = E [f (θ,X (θ, t))M(θ, t)] .

Thanks!

Collaborators: Des Higham (MLMC) and Masanori Koyama.

References:

1. Mike B. Giles. Multi-level Monte Carlo path simulation, OperationsResearch, 56(3):607-617, 2008.

2. David F. Anderson and Desmond J. Higham, Multi-level Monte Carlo forcontinuous time Markov chains, with applications in biochemical kinetics,SIAM: Multiscale Modeling and Simulation, Vol. 10, No. 1, 146 - 179,2012.

3. David F. Anderson, Efficient Finite Difference Method for ParameterSensitivities of Continuous time Markov Chains, SIAM Journal onNumerical Analysis, Vol. 50, Issue 5, 2237 - 2258, 2012.

4. David F. Anderson and Thomas G. Kurtz, Continuous Time MarkovChain Models for Chemical Reaction Networks, in Design and Analysisof biomolecular circuits, Springer, 2011, Eds Heinz Koeppl et al.

Funding: NSF-DMS-1009275.

Documents

Stochastic Representations for Jump Processes in …anderson/RecentTalks/2013...Stochastic Representations for Jump Processes in Biology with Applications to Numerical Methods David