SPATIALLY ADAPTIVE STOCHASTIC NUMERICAL METHODSFOR INTRINSIC FLUCTUATIONS IN REACTION-DIFFUSION

SYSTEMS

PAUL J. ATZBERGER ∗

Abstract. Stochastic partial differential equations are introduced for the continuum concentra-

tion fields of reaction-diffusion systems which account for fluctuations arising from finite number of

particle effects. Spatially adaptive stochastic numerical methods are developed for approximation

of the stochastic partial differential equations. The methods allow for adaptive multilevel meshes,

Neumann and Dirichlet boundary conditions, and domains having geometries with curved bound-

aries. A key issue addressed by the methods is the formulation of consistent discretizations for the

stochastic driving fields at coarse-refined interfaces of the mesh and at boundaries. As a demon-

stration of the methods, investigations are made of the role of fluctuations in a biological model for

microorganism direction sensing based on concentration gradients. Also investigated, a mechanism

for spatial pattern formation induced by fluctuations. The discretization approaches introduced for

SPDEs have the potential to be widely applicable in the development of numerical methods for the

study of spatially extended stochastic systems.

Key words. Adaptive Methods, Stochastic Numerical Methods, Stochastic Partial Differ-

ential Equations, Reaction-Diffusion, Multilevel Meshes, MAC Discretization, Statistical Physics,

Fluctuation-Dissipation Principle, Pattern Formation, Gray-Scott Reactions, Gradient Sensing.

PREPRINT NOTE:. This is a preprint under review which is being activelychecked and revised. As such, the materials presented here may contain errors. Pleasefeel free to send any comments, errors, or suggestions to atzberg@math.ucsb.edu.

Last updated: Saturday 5th September, 2009; 9:08pm.

1. Introduction. In many systems a fundamental role is played by the spatialdistribution of molecular species which undergo diffusive migrations while participat-ing in chemical reactions. Examples include the synthesis and processing of materials,intracellular signaling in biology, and morphogenic processes in the development oftissues [54; 67–71; 90; 93]. For many reaction-diffusion systems, the most interestingfeatures are exhibited only in a sub-region of the spatial domain, such as in a chemi-cally active front or in a layer near boundaries. An important role is also played byconditions at the boundaries or by the geometry of the boundaries [47; 69; 70]. Acommonly used approach to model such reaction-diffusion systems is to use contin-uum field descriptions at the mean-field level for the local concentration of a molecularspecies. Such models are often expressed in terms of deterministic partial differentialequations (PDEs). While this approach works well for many problems, at sufficientlysmall length-scales fluctuations arise in such continuum field descriptions as a con-sequence of neglected microscopic positional and momenta degrees of freedom of theindividual molecules.

To account for such fluctuations, we formulate stochastic partial differential equa-tions (SPDEs) which introduce Gaussian stochastic fields into the PDE description ofreaction-diffusion systems. We consider contributions from the intrinsic density fluc-tuations arising primarily from the finite number of molecules undergoing diffusivemigrations as opposed to fluctuations arising from the chemical reactions. The fluc-

∗University of California, Department of Mathematics, Santa Barbara, CA 93106; e-mail:

atzberg@math.ucsb.edu; phone: 805-893-3239; Work supported by NSF Grant DMS-0635535.

1

2 P.J. ATZBERGER

tuations are modeled by the stochastic fields using fluctuation-dissipation principlesof statistical mechanics.

When numerically approximating SPDEs, a number of issues arise which arenot present in the corresponding deterministic setting. Numerical approximationof SPDEs requires both discretization of the partial differential equations and dis-cretization of the stochastic driving fields. As a consequence of the stochastic driv-ing fields, solutions of SPDEs are often not as smooth as in the corresponding de-terministic PDE. Solutions of SPDEs often exist only in a generalized sense in aspace of non-differentiable functions or in a space of linear functionals (distribu-tions) [62; 65; 66; 72]. Caution must be taken when formulating discretizations forsuch solutions. For example, traditional approaches such as finite difference methodsoften rely on the Taylor Theorem which requires smoothness to ensure accuracy. Asan alternative, spectral methods can be formulated for SPDEs which rely on less-stringent results from approximation theory to ensure accuracy (cite). While spectralmethods are useful for many SPDEs, they are typically restricted to domains havingperiodic boundaries or rather simple geometries and are often not readily amenableto adaptivity [79; 81].

We shall introduce an approach for the derivation of discretizations based on finitedifference methods for the approximation of SPDEs. To obtain accurate methods, theapproach approximates solutions of the SPDEs by stochastic field values which cor-respond to a spatial averaging of solutions on length-scales comparable to the latticespacing of the discretization mesh. Stochastic numerical methods are formulated al-lowing for adaptive multilevel meshes, Neumann and Dirichlet boundary conditions,and domains having geometries with curved boundaries. A key issue addressed bythe methods is the development of consistent discretizations of the stochastic drivingfields at coarse-refined interfaces of the mesh and at boundaries. As a demonstrationof the issues encountered at coarse-refined interfaces simulation studies are performedshowing results for different discretization choices at such interfaces. For the deriveddiscretizations, analysis is carried out which shows convergence of the methods as theunderlying mesh is refined.

As a demonstration of the developed stochastic numerical methods, simulationstudies are carried out for two applications. The first application studies the effectof fluctuations in microorganism direction sensing based on concentration gradients.The case investigated concerns a single cell which senses concentration gradients in anenvironment exhibiting a shallow gradient obscured by fluctuations. The biologicalcell is represented by a region having a disk-like geometry with Neumann bound-ary conditions. A gradient is induced in the concentration of an external signalingmolecule by specifying at two walls the concentrations through Dirichlet boundaryconditions. The stochastic numerical methods are utilized on a domain having a ge-ometry defined by the two walls and region exterior to the disk. Results are reportedfor the role of fluctuations in a biological model recently proposed for cell gradientsensing [47].

The second application studies fluctuation-induced pattern formation in spatiallyextended systems. A variant of the Gray-Scott chemical reactions is considered in aregime where the deterministic reaction-diffusion system only exhibits a localized sta-tionary pattern. When introducing fluctuations, a rich collection of patterns emergeover time, in which spotted patterns migrate, combine, and replicate. The adaptivefeatures of the stochastic numerical methods are used to track at high resolution thedynamically evolving regions where the reactions are chemically active.

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 3

In summary, the proposed SPDEs give a model for intrinsic concentration fluctua-tions in reaction-diffusion systems. At the level of the continuum concentration fields,the model captures fluctuations arising from the diffusive migrations of molecules.The stochastic numerical methods allow for adaptive approximation of solutions ondomains having fairly general geometries and boundary conditions. Many of the ideasintroduced in the approaches for discretization of the SPDEs and for development ofthe numerical methods are expected to be widely applicable in the study of spatiallyextended stochastic systems.

2. Reaction-Diffusion Systems with Intrinsic Concentration Fluctua-tions. Reaction-diffusion systems are often modeled by partial differential equationswhich account for the evolution of the continuum concentration fields as the molec-ular species diffusively migrate and undergo chemical reactions. At sufficiently smalllength-scales fluctuations arise in continuum field descriptions as a consequence ofneglected microscopic positional and momenta degrees of freedom of the individualmolecules. To account for such fluctuations in reaction-diffusion systems we considerstochastic partial differential equations (SPDEs) of the form

∂c(x, t)

∂t= ∇x · D∇xc(x, t) + F [c] + n(x, t)(2.1)

〈n(x, t)nT (x′, t)〉 = Λ(x,x′)δ(t − t′).(2.2)

In the notation, c denotes the composite vector of concentration fields for the chemicalspecies. The term ∇x · D∇xc accounts for diffusion of the chemical species and isbased on a generalization of Fick’s Law allowing for non-isotropic diffusion. The tensorD characterizes the rate at which chemical species undergo diffusive migrations andis assumed to be symmetric and positive semidefinite. Throughout, we assume thatthe chemical species diffuse independently, which corresponds to D being a matrixwhich is block diagonal. The block matrices D(i) correspond to the diffusion of theith chemical species and are of size d×d, where d is the number of spatial dimensions.The term F accounts for the chemical reactions. In general, F denotes a non-linearfunctional of the concentration fields which can be either stochastic or deterministic.In the present work we shall consider only the case where F is deterministic. Theterm n accounts for fluctuations and is a Gaussian stochastic field δ-correlated in timewith mean zero and spatial covariance Λ. In the notation, 〈·〉 denotes expectationwith respect to the probability distribution of a random variable.

To derive a specific form for the Gaussian stochastic field n accounting for con-centration fluctuations we make a number of simplifying assumptions. We considerthe physical regime where fluctuations are small relative to the mean concentrationand where fluctuations are dominated by contributions from the diffusive migrationsof the molecular species. These assumptions correspond to the fluctuations of theconcentration field at thermodynamic equilibrium having covariance [6; 7; 9–11]

〈(c(x) − c)(c(x′) − c)〉 = cδ(x − x′)(2.3)

where c denotes the mean concentration. To determine the spatial covariance struc-ture of n we shall use a variant of the fluctuation-dissipation principle of statisticalmechanics.

The fluctuation-dissipation principle maintains that at thermodynamic equilib-rium and within the regime of linear reponses of the system, relaxation from a pertur-bated state caused by an external field occurs in the same manner as relaxation from a

4 P.J. ATZBERGER

perturbated state caused by fluctuations. As a consequence, the dissipative operatorsof the dynamics and equilibrium covariance can be related to the covariance structureof the fluctuations driving the system. This can be expressed as, see [4; 7; 9; 10],

Λ = −AC − C∗A∗(2.4)

where A∗,C∗ denote the adjoint of the operators. From equation 2.1 and equation 2.3we have

A = ∇x ·D∇x(2.5)

C = cδ(x − x′).(2.6)

Since the operators in our case are self-adjoint the covariance structure of the drivingfluctuations can be expressed as

Λij(x,x′) = −2ciδij∇x ·D(i)∇xδ(x − x′)(2.7)

where we use that Λ = −2AC and for the ith molecular species ci = 〈ci〉 denotes themean concentration. This determines the stochastic driving field n in equation 2.1since n is Gaussian. The stochastic partial differential equations provide a model fornear equilibrium fluctuations in the concentration fields of the molecular species.

3. Discrete Approximation of the Reaction-Diffusion System. In orderto discretize equation 2.1 in space, we divide the spatial domain Ω into a partition ofcells Ωm where Ω = ∪M

m=1Ωm and the field values are averaged over the volume ofeach cell

cm(t) =1

|Ωm|

∫

Ωm

c(x, t)dx.(3.1)

We shall approximate the dynamics of cm(t) by an Ito Stochastic Differential Equa-tion [10; 15] of the form

dct = Lctdt + dgt(3.2)

where ct denotes the composite vector of concentrations over all the sets Ωm at timet and L is a discrete approximation of the term ∇ · D∇.

Motivated by equation 2.2 of the continuum system, we shall assume that gt isa mean zero Gaussian stochastic process which is δ-correlated in time and which canbe represented in terms of increments of Brownian motion as

dgt = QdBt.(3.3)

In the notation, dBt are increments of a vector valued Brownian motion with nindependent components and Q is an m × n matrix. Under these assumptions, thestochastic field gt is determined by its covariance

⟨

dgtdgTt′⟩

= 〈QdBtdBTt′Q

T 〉 = QIδ(t − t′)dtdt′QT = Γδ(t − t′)dtdt′(3.4)

where we let Γ = QQT . We have used the formal identity of Ito Calculus that〈dBtdB

Tt′〉 = Iδ(t− t′)dtdt′, which in our notation corresponds to Ito’s Isometry [15].

In discretizing the equations, we must approximate the stochastic componentsof the system corresponding to equations 2.2 and ??. For the equilibrium of the

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 5

discretized system we shall impose that cm have fluctuations consistent with averagingthe continuum system over the partition. From equation 3.1 this requires

⟨

(cm − cm) (cn − cn)T⟩

=C

|Ωm|δm,n(3.5)

where Cij = ciδij . For the stochastic field of the discrete system we shall derive acovariance structure which is consistent with the equilibrium fluctuations given inequation 3.5. Let Ct = 〈(ct − c) (ct − c)

T 〉 denote the covariance of the fluctuationsof the field at time t and Γ = QQT denote the covariance structure of the stochasticfield gt. From Ito’s Lemma [15] we have

dCt =(

LCt + CtLT + Γ

)

dt.(3.6)

At statistical steady-state this has dCt = 0. This shows that in order for equation 3.2to have the equilibrium fluctuations given by equation 3.5, the stochastic field musthave covariance

Γ = −(

LC + CLT)

.(3.7)

In the case that LC is symmetric this can be reduced to

Γ = −2LC.(3.8)

This gives for any choice of partition Ωm and discretization L the correspondingcovariance structure for the stochastic field gt to be used in equation 3.2. In thecase of diffusion of a single non-reacting species, this choice of Γ formally approachesΛ, giving Γ → Λ as the partition is refined |Ωm| → 0 and when making the formalassociations L → ∆x and δm,n/|Ωm| → δ(x − x′).

In order for this discretization to be useful in practice it is important that efficientnumerical methods can be developed for the stochastic differential equation 3.2. Acentral issue in developing practical numerical methods is to generate efficiently thestochastic field gt with the covariance structure given in equation 3.7. This willdepend on the spatial partition used to discretize the system and on how the term∇ ·D∇ is approximated.

For discretizations corresponding to uniform meshes with periodic boundary con-ditions the Fast Discrete Fourier Transform can be used [5]. In this case the term∇ ·D∇ can be approximated spectrally and Γ diagonalized in Fourier space allowingfor the stochastic fields to be generated efficiently [13]. However, these methods are nolonger applicable for more general problems on non-periodic domains with Dirichlet orNeumann boundary conditions or for meshes that arise in adaptive numerical meth-ods, which are no longer uniformly discretized in space. In this paper we shall addressthe challenge of developing efficient numerical methods for non-periodic non-uniformmeshes having Neumann or Dirichlet boundary conditions.

3.1. Discrete Approximation of the Operator ∇ ·D∇. Since the chemicalspecies are assumed to diffuse independently, the diffusion tensor D has diagonalblocks of size d× d, where d is the spatial dimension of the system. This allows for adecomposition into uncoupled operators ∇x ·D(k)∇x for each chemical species, whereD(k) is the block of the kth chemical species. The block D(k) is symmetric and canbe diagonalized by D(k) = PTD(k)P for some unitary P . A linear change of variablex = Rx has ∇x = R∇x and ∇x· = ∇x · RT . This gives

∇x · D(k)∇x = ∇x · RTD(k)R∇x.(3.9)

6 P.J. ATZBERGER

In the case that D(k) is positive definite, we have by letting R = P (D(k))−1/2 that

∇x ·D(k)∇x = ∆x.(3.10)

In the case that D(k) is only semidefinite we set the entries to be zero which areundefined from division by zero in (D(k))−1/2. With this definition of (D(k))−1/2 weagain have equation 3.10.

This shows that for any positive semidefinite D(k) the operator ∇x · D(k)∇x

can always be obtained from the Laplacian by an appropriate change of variable.Since these operators are decoupled, we introduce for the concentration field of eachchemical species a separate coordinate system x(k) = R(k)x and let ck = ck(x(k), t).Throughout, we take as the approximation for the operator ∇·D∇ the discretizationwhich is obtained by making the appropriate change of variable and approximatingthe Laplacian.

3.2. Discrete Approximation of the Laplacian on Uniform and Multi-level Meshes in 2D. Throughout, we shall consider the particular case of a spatialpartition consisting of square cells which differ with their neighbors in the lengths oftheir sides by at most a factor of two. For convenience in describing the discretiza-tions we shall define values both at the cell centers and at the faces of each cell, whichwe collectively refer to as the mesh, see Figure 3.1. To obtain a discrete operatorapproximating the Laplacian we define for the mesh a discrete gradient operator anda discrete divergence operator [19; 20; 23].

The discrete gradient operator computes from cell centered values an approxi-mation to the gradient at each face center in the component corresponding to thedirection of the face relative to the cell center. Given that neighboring cells can varyin size there are two cases for such meshes. In the case that a cell is the same size asits neighbor the discrete gradient component is defined on the shared face by

(Gc)(i)m+hi

= sign(hi)cm+qi

− cm

∆xm

.(3.11)

In the notation, the cell centered values are indexed by m = (m1, m2) and the facecentered values correspond to indices of the form m′ = (m1 + d1, m2 + d2), wheredk ∈ 0,± 1

2 with the constraint that only one index of k may have dk 6= 0. Forexample, the mesh cell centered at index m has face centered value in the direction ofthe negative x-axis given by m′ = (m1− 1

2 , m2). The superscript notation (i) refers tothe vector component of the gradient, the face is specified by hi = ± 1

2ei with ei thestandard basis vector which has component i equal to one and all other componentszero, and qi = 2hi. The notation ∆xm refers to the side length of the cell with indexm.

In the case that a neighbor is of a different size, as shown in Figure 3.1, thegradient at the shared faces is defined by

(Gc)(1)B =

c(1)A − 1

2

(

c(1)B + c

(1)C

)

34∆xA

(3.12)

(Gc)(1)C =

c(1)A − 1

2

(

c(1)C + c

(1)B

)

34∆xA

(3.13)

(Gc)(1)A =

1

2

(

(Gc)(1)B + (Gc)

(1)C

)

.(3.14)

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 7

Fig. 3.1. Coarse-refined interface in 2D. The symbol + denote locations of cell centered values.The symbol denote locations of the face centered values. Throughout, the face centered value ofthe coarse cell is assumed to be the average of the two neighboring cells.

Here we have given the expressions only for the special case depicted in Figure 3.1.The other cases are defined similarly, see [19; 20; 23].

A discrete divergence operator is now defined which computes from face centeredvalues an approximation of the divergence at the cell center. For all cells the discretedivergence operator is defined by

(Db)m =

2∑

i=1

b(i)m+hi

− b(i)m−hi

∆xm

(3.15)

where b is the composite vector of face centered vector values of the mesh. At aninterface between cells of different sizes we shall always assume that the face centeredvalue of the coarse cell is the average of the face centered values of the two neighboringcells.

A discrete Laplacian for any such spatial partition can then be obtained from

L = DG.(3.16)

This operator maps cell centered values to cell centered values. This discrete Laplaciancan be shown to be locally second order accurate for cells having neighbors of the samesize and first order accurate for cells having neighbors of a different size [19; 20; 23].We shall refer to this as the ”MAC Discretization” of the Laplacian.

3.3. Discrete Approximation of the Laplacian at Curved Boundariesin 2D. We now discuss the case where the domain has a non-rectangular geometrywith curved boundaries. To approximate the Laplacian at such boundaries we shallintroduce cells into the mesh which are cut by the boundary. The Laplacian is dis-cretized in these regions using a finite volume method. Our approach follows closelythe methods referred to as Embedded Boundary Methods, Cartesian Grid Methods,Cut-Cell Methods, see [35–41].

Dividing the spatial domain into a partitioning of regions Ωm we have from theDivergence Theorem [85] on each partition element

∫

Ωm

∆cdx =

∫

∂Ωm

∇c · ndAx.

8 P.J. ATZBERGER

Fig. 3.2. Cut Mesh Cells Near a Curved Boundary: The symbols + denote locations of cellcentered values. The symbols denote locations of the face (edge) centered values. The light regiondenotes the part of the cell inside the domain and the dark region the part of the cell outside thedomain.

By approximating the expressions for the integrals on the left and right sides we obtaina finite volume discretization of the form

[Lc]m =1

|Ωm|∑

j∈Nm

J(n)m,j |∂Ωm,j |

where |Ωm| is the area of the partition element, Nm is an indexing set correspondingto the common edges shared with this partition element, |∂Ωm,j | is the length of

the jth edge of the partition element, and J(n)m,j approximates the normal component

of the concentration flux corresponding to ∇c · n. Throughout, we shall use as ourdiscrete degrees of freedom in the scheme the concentration field defined on the cellcenters of the regular mesh and flux components defined on the center of each edge,

see Figure 3.2. For the fluxes we shall use J(n)m,j = ∇u(xm,j) · nm,j where for each

edge the u(x) denotes the linear product interpolation of the concentration field at xusing the four nearest neighbors and xm,j denotes the location of the center of thejth edge for mesh cell m.

To obtain such a partition, space is initially discretized by regular mesh cells whichare then intersected with the domain to determine sub-regions in each cell which areeither inside or outside the domain. It is assumed that the boundary geometry issmooth, at least twice differentiable, so that at sufficient resolution the boundary canbe approximated well by piecewise-linear segments intersecting exactly two edges ofany cut mesh cell. Under these assumptions, only relatively few cases arise in thegeometry of the cut cells and the areas of the sub-regions and lengths of the edges canbe readily computed. It can be shown that this scheme yields a first order accurateapproximation to the Laplacian at such curved boundaries [35; 36; 40; 41]. Away fromthe boundary the scheme corresponds to the standard second order accurate centraldifference approximation of the Laplacian.

In practice when using such schemes for time dependent problems an importantissue arises from the small elements which introduce stringent stability constraints on

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 9

the permissible time steps, as expressed in the Courant-Friedrichs-Lewy Condition ofthe scheme [37; 38; 86]. This can be most directly seen by noting that the stabletime step size is bounded by the minimum area of the partition elements to algebraicorder. This presents a difficulty in practice since small elements can in principle arisein any mesh since the boundary is arbitrary and the domain could intersect only smallregions of a cut cell, for example, such as only very near to one of the cell corners.To reduce this source of instability in the scheme, we introduce a merging procedureinto the discretization routines which fuse cut cell regions with a neighboring cellwhen such regions are smaller than some threshold percentage of the regular meshcell size. For instance in Figure 3.2, if the split cell on the lower right were deemedtoo small it would be fused with the largest neighboring mesh cell, which in this caseis the cell above it. The degree of freedom associated with the center of the regularcell containing the small split cell is then eliminated and all dependencies on thisvalue in the scheme are replaced by a quadratic interpolation from the neighboringconcentration values in the direction of the merged neighboring cell. All fluxes intoto the small cut cell region are directed into the merged cell. This merging procedurecan be shown to increase the stability of the scheme while retaining at least first orderaccuracy [35–41].

In order to obtain sufficient resolution at curved boundaries another issue encoun-tered in practice concerns the relatively fine mesh which may be required while such aresolution may not be required globally. To achieve greater computational efficiencywe shall use the approach in conjunction with multilevel meshing to adapt the level ofspatial resolution in regions near the curved boundaries. We shall discuss how thesemethods can be used in the context of a specific application in Section 6.2.

4. Methods to Generate Increments of the Stochastic Field g(t). Weshall now discuss approaches to discretize the stochastic driving field g(t) consistentwith equations 3.2 – 3.8. Discretizations will be derived for a variety of differentmeshes. For uniform rectangular meshes discretizations for g will be derived in thecase that Neumann or Dirichlet boundary conditions are imposed. For multi-levelmeshes discretizations for g will be derived for the coarse-refined interfaces of themesh. In the case of non-rectangular meshes discretizations will be derived for g inthe regions near the curved boundaries. In order for these discretizations to be usefulin practice it is important that stochastic realizations of g can be computed efficiently.In each case, we shall discuss specific approaches by which to generate efficiently theincrements of the stochastic field g(t).

4.1. Uniform Meshes with Dirichlet or Neumann Boundary Conditionson Rectangular Domains. In the case of a uniform mesh the Laplacian given byequation 3.16 corresponds to a central finite difference approximation of the Laplacianand yields the standard five point stencil. In this case C = σ2I with σ2 = c/|A| where|A| = ∆x2 is the volume of a cell of the uniform mesh. The covariance structure forthe stochastic field becomes

Γ = −2LC(4.1)

since the product LC is symmetric. For uniform meshes the gradient is the negativetranspose of the divergence operator G = −DT . This allows for the covariance to beexpressed as

Γ = 2σ2DDT = (√

2σD)(√

2σD)T .(4.2)

10 P.J. ATZBERGER

This shows that increments of the stochastic field g can be generated using equation3.3 with Q =

√2σD and increments of independent Brownian motions centered at

the cell faces of the mesh.

Physically this approach corresponds to introducing ”random fluxes” for thestochastic transport of conserved quantities between mesh cells, similar approacheshave been discussed in the context of fluctuating hydrodynamics [7; 9]. The keypoint here is that the random fluxes arise naturally for the discretization of the physi-cal system by the requirement that the equilibrium concentration fluctuations satisfyequation 3.7. We shall discuss in Section ?? the consequences of taking a less rigor-ous but more intuitive physical approach to obtain a discretization of the stochasticfield g. Such approaches can lead to numerical methods with significant errors whichare manifested in correlations being introduced into the equilibrium concentrationfluctuations of the system.

We now discuss how various boundary conditions can be handled. The case ofDirichlet boundary conditions can be handled by introducing a set of ”ghost cells”which comprise the boundary of the mesh. Since equation 3.2 is linear we can considerthe case of homogeneous zero Dirichlet boundary conditions without loss of generality.A modified discrete divergence operator D consistent with the homogeneous Dirichletboundary conditions can be obtained by setting in the matrix representation D allentries to zero in rows corresponding to the ”ghost cells”. A modified gradient oper-ator G consistent with the boundary conditions can be obtained by setting in G allentries to zero in columns corresponding to the ”ghost cells”. The extended covari-ance matrix C then has zero entries for the ”ghost” values since these remain fixedin time and do not fluctuate. A Laplacian consistent with the boundary conditions isthen given by L = DG. Since G = −DT the covariance can be factored similarly toequation 4.2. This allows for increments of the stochastic field g to be generated withthe factor Q =

√2σD using independent increments of Brownian motion for each face

centered value in equation 3.3.

The Neumann boundary conditions are imposed by setting the gradient compo-nents at the face centered values of the mesh for cells on the boundary. Since equation3.2 is linear we can consider the case of homogeneous zero Neumann boundary condi-tions without loss of generality. In this case a modified discrete divergence operatorD can be obtained by setting in the matrix representation of D all entries to zeroin columns corresponding to the boundary faces. A modified gradient operator Gis obtained from the matrix representation G by setting all entries to zero in rowscorresponding to the boundary faces. The covariance matrix C is uneffected by theboundary conditions in this case. A modified Laplacian consistent with the boundaryconditions is obtained by L = DG. Since the modified operators have the relationshipG = −DT the covariance can be factored similarly to equation 4.2. This allows for in-crements of the stochastic field g to be generated with Q =

√2σD using independent

increments of Brownian motion for each face centered value in equation 3.3.

A mixture of Dirichlet and Neumann boundary conditions can also be handled bythis approach. In this case the required factor Q is obtained by applying the abovemodifications individually to the matrix representations of the divergence operator Dand gradient operator G on the extended mesh for each row or column correspondingto a cell centered or face centered value fixed by the boundary condition. Since therelationship G = −DT is preserved by these modifications the factor can be obtainedin a manner similar to equation 4.2.

An important feature of the matrix factors Q obtained by the methods above

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 11

is that they are always sparse, with the number of non-zero entries proportional tothe number of mesh cells. As a consequence the cost of generating increments of thestochastic field g has a computational complexity which scales only linearly O(N)in the number of mesh points N . This is in contrast to other methods, such as theCholesky algorithm for correlated variates [87]. The Cholesky algorithm requires twodistinct computations. The first a computation scaling as the cube of the number ofmesh points O(N3) in order to determine the factor Q. The second a computation togenerate the increments of the stochastic field g scales quadratically in the numberof mesh points O(N2) since the Cholesky factor Q is typically not sparse [61; 82; 87].

4.2. Multilevel Meshes in 2D. We now discuss the case of multilevel mesheswhich contain mesh cells of varying levels of refinement. In this case the discretegradient operator is no longer proportional to the transpose of the divergence operator,G 6= −DT as a consequence of the coarse-refined interfaces of the mesh. This preventsdirectly obtaining a factorization as in equation 4.2.

We shall instead consider a more general approach to factor Γ. Let the stochasticfield be expressed as a sum of two random variables g = g1 + g2, then the covariancecan be expressed as

Γ = Γ1 + Γ2 + Γ(1,2) + Γ(2,1)(4.3)

where Γk = 〈gkgTk 〉, Γ(k,j) = 〈gkg

Tj 〉, k, j ∈ 1, 2, k 6= j. Let g1 = Q1n1 and g2 =

Q2n2 with n1 and n2 each being vectors having in each component an independentstandard Gaussian with mean zero and variance one. We then have Γ1 = Q1Q

T1 ,

Γ2 = Q2QT2 , and Γ(1,2) = Γ(2,1) = 0 which gives

Γ = Γ1 + Γ2.(4.4)

This shows that a decomposition of Γ into the sum of any two positive semidefinitematrices corresponds to expressing g as the sum of two independent Gaussian randomvariables with the corresponding covariances Γ1 and Γ2.

To use this decomposition to obtain a method to generate g requires findingfactors Q1 and Q2 which satisfy equation 4.4 where Γ is determined from equation3.7. For this purpose, we shall associated a composite vector n1 of independentstandard Gaussian random variables with the face centered values of the mesh andn2 with cell centered values of the mesh.

To obtain Γ1 we consider the modified divergence operator D′ with all entries incolumns corresponding to faces of coarse-refined interfaces set to zero. We similarlymodify the gradient operator G′ to have all entries in rows corresponding to facesof the coarse-refined interfaces set to zero. This corresponds to setting Neumannboundary conditions at the boundaries of cells at each coarse-refined interface, whichform the border regions between different levels of refinement of the mesh. Since themodified divergence operator and gradient operator satisfy G′ = −D′T we can factorΓ1 associated with the modified operators as in equation 4.2 to obtain a sparse factorQ1. This can be used to generate g1 = Q1n1.

Now by equation 4.4 we must take Γ2 = Γ− Γ1. However, for the decompositionto be valid we must have that Γ2 is positive semidefinite. Furthermore, for the de-composition to give an efficient method in practice for generating variates, the factorQ2 should be as sparse as possible. An important feature of Γ2 is that it consistsof uncoupled blocks corresponding to small clusters of cells along the coarse-refinedinterface. This in part, motivated our choice of Γ1. This reduces the problem to

12 P.J. ATZBERGER

considering each individual block matrix corresponding to a cluster. Another featureis that the block matrices are all of a similar form. For brevity in the exposition weshall only discuss in detail the specific case of the cluster depicted in Figure 3.1. Theother cases for the clusters follow similarly.

In the case depicted in Figure 3.1 the cluster associated with the coarse cell labeledby A has the covariance block matrix given by

MA =8CAA

3∆x2A

1 −2 −2−2 4 4−2 4 4

(4.5)

where the row and column ordering is A,B, C and CAA denotes the correspondingmatrix entry of A in the covariance in equation 3.5. The eigenvectors and eigenvaluesof the matrix are

v1 =1√5

210

,v2 =1

3√

5

2−4

5

,v3 =1

3

1−2−2

(4.6)

with eigenvalues λ1 = 0, λ2 = 0, λ3 = 9 respectively. An especially important featureof these block matrices M is that they are positive semidefinite which shows that thedecomposition chosen above for the covariance matrix is indeed valid and yields amethod to generate the field by g = g1 + g2. For this purpose random variates areused at cell centered values of each cluster to generate a variate with the requiredcovariance structure M

rA = η1

√

λ1v1 + η2

√

λ2v2 + η3

√

λ3v3(4.7)

where ηk are independent standard Gaussian random variables of mean zero andvariance one. Of course this can be simplified since λ1 = λ2 = 0. The other clustercases can be handled in a similar manner. To obtain g2 the appropriate componentsare set by independently generating rA for each of the clusters. If we associate η1, η2, η3

with the n2 cell centered values of the mesh corresponding to A,B, C then equation4.7 defines a sparse factor Q2 with the number of non-zero entries proportional tothe number of mesh cells bordering coarse-refined interfaces. The above procedure isthen equivalent to generating g2 = Q2n2.

We now have a method to generate increments of the stochastic random field gon multilevel meshes with covariance satisfying equations 3.7. An important featureof this method is that the factors Q1 and Q2 can be constructed with linear com-putational complexity in the number of mesh cells O(N). Also, since the factors areeach sparse the increments of the stochastic random field g can be generated withonly a linear computational complexity in the number of mesh cells O(N). Dirichletand Neumann boundary conditions on multilevel meshes on rectangular domains canalso be handled by this method by modifying the discrete gradient and divergenceoperators in the same manner as described for the uniform mesh case in Section 4.1.

4.3. Meshes with Curved Boundaries in 2D. We now discuss an approachto generate the required increments of g satisfying equation 3.8 for meshes on domainswith curved boundaries. Let G denote the operator which computes at the center ofeach edge the approximate normal component of the gradient of the concentration

field used in J(n)m,j . Let D denote the operator which yields the mesh centered values

by acting on the flux values at the edge centers. The Laplacian can then be expressed

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 13

as L = DG. It can be shown that LC is symmetric so that the required covariance forg is given by Γ = −2LC. Since in general D 6= −GT directly factoring the covariancematrix is challenging. Instead, we shall again make use of the decomposition discussedin Section 4.2, in which Γ = Γ1 + Γ2, where g = g1 + g2 for independent g1 and g2

with Γ1 and Γ2 the respective covariances.We shall find it useful to classify mesh cells into two distinct sets, a boundary set

B and an interior set I. The boundary set B consists of all mesh cells which are cutby the domain boundary along with all immediate neighboring cells. The interior setI consists of all of the remaining cells of the mesh. For the mesh cells in I, let G′ bethe operator obtained from the matrix representation of the gradient operator G bysetting all rows to zero which correspond to edges which are shared with mesh cellsin B. Also, set D′ to be the operator obtained from the matrix representation of thedivergence operator D by setting all the columns to zero which correspond to thesesame edges. To define G′ and D′ on the entire mesh all rows and columns are takento be zero which correspond to cells not in I.

Now to obtain operators corresponding to the set B of mesh cells, we define G′′ =G − G′ and D′′ = D − D′. By the definition of the operators, D′G′′ = D′′G′ = 0, sothat L = L′+L′′ where L′ = D′G′ and L′′ = D′′G′′. We can further set C = C′ +C′′,where C′, C′′ are obtained by setting all entries outside the corresponding sets ofmesh cells to zero. With these definitions we obtain the decomposition Γ = Γ1 + Γ2

with Γ1 = −2L′C′ and Γ2 = −2L′′C′′. With this decomposition, the increments of g1

can be readily generated since Γ1 corresponds to the case of a mesh with Neumannconditions on a domain with a stair-case boundary, which falls into the previouslydiscussed cases in Sections 4.1 and 4.2.

We now discuss a method to generate increments of the field g2. Obtaining afactorization of Γ2 is made challenging by the following issues: D′′ 6= −G′′T , thematrix entries are irregular with dependencies on the local geometry of the boundary,and the covariance matrix is only positive semi-definite. The Cholesky factorizationalgorithm is often used to generate correlated variates, however, for matrices which areonly semi-definite it can not be used directly [82; 87]. To cope with these difficultieswe shall generate the variates by using a decomposition of the matrix Γ2 into itseigenvalues and eigenvectors. The increments of g2 can then be generated using

g2 =

M∑

k=1

ηk

√

λkvk

where the kth eigenvalue and orthonormal eigenvector are denoted respectively by λk

and vk, and the ηk denote independent Gaussian random variables with mean zeroand variance one.

It is important to mention that while computing the eigenvalue decompositionhas computational complexity O(M3) in the worse case [61], the number of mesh cellsM corresponds only to the collection B of cells needed to capture the one dimensionalgeometry of the boundary and can be performed independently for each separate un-connected curved boundary region. For problems where the boundaries do not changein time this decomposition of Γ2 need only be computed once. To generate realiza-tions of the stochastic field g2 each time step incurs a computational cost of O(M2)since the eigenvectors will have in general have a full set of non-zero components.

The presented decomposition of Γ allows for the stochastic driving field g to begenerated for domains having curved boundaries. While additional costs are intro-duced by the curved boundaries the numerical methods still utilize the previously

14 P.J. ATZBERGER

developed explicit factorization expressions for the interior part of the mesh awayfrom the boundaries. We discuss how this approach can be used in practice in thecontext of a specific application in Section 6.2.

5. Convergence of the Stochastic Numerical Methods for the LinearizedEquations. As a validation of the stochastic numerical methods we show formallythat the methods converge in the case when the system is near steady-state and thefluctuations are small relative to the mean concentration. As discussed in Section 1,an important issue is that the solutions of equation 2.1 are not well-defined in termsof classical functions with point-wise values, but are instead represented by distri-butions [62; 64; 66]. To simplify the discussion and to avoid delving into too manytechnical issues, we shall formally demonstrate a form of weak convergence of thestochastic numerical methods semi-discretized in space.

The form of weak convergence we shall consider corresponds to convergence ofthe moments of linear functionals of the form

a(x, t) = A[c] =

∫

Ω

∫ t

0

α(x,y, s)c(y, s)dsdy(5.1)

when numerically approximated by

a(x, t) = A[c] =∑

m

∫ t

0

α(x,ym, s)cm(s)ds∆xd(5.2)

where α(x,y, s) is a bounded compactly supported smoothly varying function in spaceand time. We shall consider the following form of weak convergence

∥

∥

∥M

(n)

A1,A2,...,An

− M(n)A1,A2,...,An

∥

∥

∥

∞→ 0(5.3)

as ∆x → 0. The nth moment is defined by

M(n)A1,A2,...,An

(x1, t1,x2, t2, . . . ,xn, tn) = 〈a1(x1, t1)a2(x2, t2) · · · an(xn, tn)〉.(5.4)

This convergence is required for each moment n ≥ 1 and choice of functionalsA1, A2, . . . , An of the form of equation 5.1 approximated by equation 5.2. In thenotation ‖f(x1, t1,x2, t2, . . . ,xn, tn)‖∞ = supx1,t1,x2,t2,...,xn,tn

|f |. This is one of manydifferent types of convergence which can be defined for stochastic processes, see [13].

As an intuitive motivation for this form of weak convergence, the functionals Acan be thought of as being analogues of physical observations as would be obtainedfrom experimental measurements of an underlying fluctuating concentration field.In experiments any measured quantity is almost always averaged in space and timeto some extent. Such averaging is represented in the functional by integrating theconcentration field against the function α. Weak convergence then corresponds tothe situation where the statistics of any measurement of the underlying concentrationfield can be reproduced in simulations up to a specified precision by using a sufficientlyrefined discretization mesh.

A number of simplifications can be made by utilizing linearity of the functionalA and properties of c. From linearity and the smoothness of α we have that a(x, t) isa Gaussian random field with well-defined point-wise values. This has the importantconsequence that statistics of the random field are completely determined by the thefirst two moments. This requires only n ≤ 2 need be considered in equation 5.3.

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 15

For the system close to statistical steady-state and for sufficiently small fluctu-ations relative to the mean concentration it is sufficient to consider the lineariza-tion of equations 2.1. This corresponds in equation 2.1 to a functional of the formF[c] = Fc + f0 where F denotes a linear functional and f0 a constant field indepen-dent of c. In this regime, averaging equation 2.1 and equation 3.2 give for the firstmoment a deterministic reaction-diffusion equation. For the first moments the conver-gence follows straightforwardly from the deterministic convergence theory developedfor MAC discretizations (cite). We shall focus on demonstrating convergence of thesecond moments, which arise from the fluctuations.

When working with the second moments it is helpful to consider the covariance

function R(x1, t1,x2, t2) = M(2)A1,A2

−M(1)A1

M(2)A2

, which can be expressed more explic-itly as

(5.5)

R(x1, t1,x2, t2) =

∫

dy1dy2

∫

ds1ds2 α1(x1,y1, s1)q(y1, s1,y2, s2)α2(x2,y2, s2)

q(y1, s1,y2, s2) = 〈(c(y1, s1) − c)(c(y2, s2) − c)〉where α1 and α2 correspond to the linear functionals A1 and A2 represented in theform of equation 5.1. The integrals in y1, y2 and s1, s2 are taken over the boundeddomain (y1,y2, s1, s2) ∈ Ω × Ω × [0, t] × [0, t]. Similarly for the semi-discretized

system we have the covariance function R(x1, t1,x2, t2) = M(2)

A1,A2

−M(1)

A1

M(2)

A2

, which

can be expressed explicitly as

R(x1, t1,x2, t2) =∑

m1

∑

m2

∫

ds1ds2 α1(x1,ym1, s1) ·(5.6)

·q(ym1, s1,ym2

, s2)α2(x2,ym2, s2)∆xd

m1∆xd

m2

q(ym1, s1,ym2

, s2) = 〈(cm1(s1) − c)(cm2

(s2) − c)〉 .

Since c is a solution of equation 2.1 we have formally that c(y, s) = e(s−r)Lc(y, r)when s > r, where L denotes the Laplacian a negative semidefinite linear differentialoperator and etL denotes the solution operator over the time interval [0, t] from thesemi-group associated with equation 2.1, see [64; 72]. By the choice of stochasticdriving field n in equation 2.2 we have 〈(c(y1, s1) − c)(c(y2, s2) − c)〉 = e(s1−s2)LC,where s1 ≥ s2 and

C(y1,y2) = cδ(y1 − y2).(5.7)

Substituting this into equation 5.5 yields

R(x1, t1,x2, t2) =

∫

dy1dy2

∫

s1>s2

ds1ds2 α1(x1,y, s1)e(s1−s2)LCα2(x2,y, s2)

+

∫

dy1dy2

∫

s2>s1

ds1ds2 α2(x2,y, s2)e(s2−s1)LCα1(x1,y, s1).

By a similar argument for the semi-discretized equation 3.2 we have c(s) = e(s−r)Lc(r)when s > r, where L denotes the MAC-Laplacian represented by a negative semidefi-nite matrix and etL denotes the matrix exponential, see [80]. By the choice of stochas-tic driving field gt in equation 3.7 we have

⟨

(c(s1) − c)(c(s2) − c)T⟩

= e(s1−s2)LC,where s1 ≥ s2 and

[C]m1,m2= cδm1,m2

/∆xdm1

(5.8)

16 P.J. ATZBERGER

where δm1,m2is the Kronecker δ-function. In the notation [·]m1,m2

denotes the(m1,m2) matrix entry. Substituting this into equation 5.6 yields

R(x1, t1,x2, t2) =∑

m1,m2

∫

s1>s2

ds1ds2 α1(x1,ym1, s1) ·

·[e(s2−s1)LC]m1,m2α2(x2,ym2

, s2)∆xdm1

∆xdm2

+∑

m1,m2

∫

s2>s1

ds1ds2 α2(x2,ym1, s2) ·

·[e(s2−s1)LC]m1,m2α1(x1,ym2

, s1)∆xdm1

∆xdm2

.

To show convergence it is useful to let

β1(y, t) =

∫

e(t−s2)LC(y,y2)α2(x2,y2, s2)dy2(5.9)

β1(ym, t) =∑

m2

[e(t−s2)LC]m,m2α2(x2,ym2

, s2)∆xdm2

(5.10)

with similar definitions for β2, β2. From the definitions of the operators e(t−s2)L ande(t−s2)L we have that β1 and β1 solve the following differential equations with specifiedinitial values

∂β1/∂t = Lβ1, for t > s2

β1(y, s2) =∫

C(y,y2)α2(x2,y2, s2)dy2 for t = s2

(5.11)

and

dβ1/dt = Lβ1, for t > s2

β1(ym, s2) =∑

m2[C]m,m2

α2(x2,ym2, s2)∆xd

m2, for t = s2

.(5.12)

The β2 and β2 solve similar differential equations. Using the specific form of C andC given in equations 5.7 and equation 5.8 we have that β1(y, s2) = α2(x2,y, s2) andβ1(ym, s2) = α2(x2,ym2

, s2). From the determinstic convergence theory developedfor the MAC-Lapacian for such differential equations (cite), we have

‖β − β‖∞ → 0(5.13)

as ∆x → 0. We remark that while our analysis is only formal, such uniform conver-gence results are subject to technical conditions such as smooth initial conditions, abounded domain Ω, time-independent boundary conditions, and a bounded steady-state solution, which we shall assume throughout [63].

The difference of the covariance functions of the discretized system and continuumsystem can be bounded using the triangle inequality by

‖R− R‖∞ ≤ I1 + I2 + I3 + I4.(5.14)

where

(5.15)

I1 =

∥

∥

∥

∥

∥

∑

m

∫

s1>s2

ds1ds2 α1(x1,ym, s1)(

β1(x1,ym, s1) − β1(x1,ym, s1))

∆xd

∥

∥

∥

∥

∥

∞

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 17

I2 =

∥

∥

∥

∥

∥

∑

m

∫

s2>s1

ds1ds2 α2(x2,ym, s2)(

β2(x2,ym, s2) − β2(x2,ym, s2))

∆xd

∥

∥

∥

∥

∥

∞

I3 =

∥

∥

∥

∥

∥

∑

m

∫

s2>s1

ds1ds2 α2(x2,ym, s2)β1(x1,ym, s1)∆xd

−∫

dy

∫

s2>s1

ds1ds2 α2(x2,y, s2)β1(x1,y, s1)

∥

∥

∥

∥

∞

I4 =

∥

∥

∥

∥

∥

∑

m

∫

s1>s2

ds1ds2 α1(x1,ym, s1)β2(x2,ym, s2)∆xd

−∫

dy

∫

s1>s2

ds1ds2 α1(x1,y, s1)β2(x2,y, s2)

∥

∥

∥

∥

∞

.

Using properties of the infinity norm ‖ · ‖∞ the integral and sum can be bounded by

I1 ≤∥

∥

∥

∥

∥

∑

m

∫

s1>s2

ds1ds2 |α1(x1,ym, s1)|∆xd

∥

∥

∥

∥

∥

∞

∥

∥

∥β1 − β1

∥

∥

∥

∞.(5.16)

An important property of the estimate is that the first term remains bounded as∆x → 0. This follows since α1 is compactly supported and the domains in space andtime are bounded. Using this fact and equation 5.13 we have I1 → 0 as ∆x → 0. By asimilar argument I2 → 0. For I3 we have from equation 5.15 that the first term is theRiemann sum approximation of the second term. Since α1 is compactly supportedand the domains in space and time are bounded this implies I3 → 0. By a similarargument I4 → 0.

These estimates formally show that

‖R− R‖∞ → 0.(5.17)

This along with convergence of the first moments implies

‖M(2)

A1,A2

− M(2)A1,A2

‖∞ → 0.(5.18)

Since the random fields are Gaussian and completely determined by the first twomoments this analysis formally establishes that the stochastic numerical methodsweakly converge. An important feature of the form of weak convergence consideredis that the stochastic methods produce statistics convergent not only for individualobservables represented by A. The stochastic methods are also convergent for anycross-correlation statistics for observables represented by A1 and A2 which referencethe same underlying concentration field.

To obtain convergent stochastic numerical methods the analysis indicates that itis not only required that the discretization of the differential operator L be consis-tent, but that the discretized system have equilibrium fluctuations with a covariancestructure consistent with C of the continuum system. An important issue in prac-tice is that the equilibrium covariance structure is not discretized directly but ratherarises from the fluctuations induced by the discretized stochastic driving field gt, asin equation 3.2. To control the discretization errors introduced in the fluctuations ofthe discretized system as the mesh is refined we utilized a variant of the Fluctuation-Dissipation Principle of statistical mechanics, see Section 3. This was used to ensure

18 P.J. ATZBERGER

0 500 10000

200

400

600

800

1000

x

y

’White Noise’

0 500 1000−400

−200

0

200

400

600

800

1000

y

Cov

aria

nce

Mesh Cross Section

0 500 10000

200

400

600

800

1000

x

y

’Random Fluxes’

0 500 1000−400

−200

0

200

400

600

800

1000

y

Cov

aria

nce

Mesh Cross Section

0 500 10000

200

400

600

800

1000

x

y

’Adjusted Stochastic Forcing’

0 500 1000−400

−200

0

200

400

600

800

1000

y

Cov

aria

nce

Mesh Cross Section

Fig. 5.1. Spatial Covariance of Equilibrium Fluctuations at a Coarse-Refined Interface. Thetop row shows the covariance of the equilibrium fluctuations of the system for each choice of thestochastic driving field at a cell at the coarse-refined interface on the refined side (see + symbol).In the second row, the covariance with this cell is plotted as a function of y at cross sections of themesh near the interface.

that the stochastic numerical methods exhibit equilibrium fluctuations with the speci-fied covariance C, which was chosen to be consistent with C of the continuum system.This approach is especially important at coarse-refined interfaces of adaptive meshesand cut mesh-cells near the domain boundaries to ensure consistent discretizationsfor the stochastic driving field gt.

5.1. Equilibrium Fluctuations at Coarse-Refined Interfaces. We now dis-cuss how the choice of covariance structure for the stochastic field g influences theequilibrium fluctuations of the system. For dynamics given by equation 3.2, the gen-eral relationship in equation 3.8 shows how the covariance of the equilibrium fluctua-tions C are related to the covariance of the stochastic driving field Γ. In the case whenLC is symmetric and L is invertible, the equilibrium covariance C can be expresseddirectly in terms of Γ

C = −1

2L−1Γ.(5.19)

This expression shows that while discretization errors in approximating the covarianceof the stochastic driving term may be localized the errors can have a non-local influenceon the covariance of the equilibrium fluctuations of the system.

To illustrate how the choice of covariance influences the equilibrium fluctuationswe shall compare three choices for the driving stochastic field g. The first is a variantof ’white noise’ where Γm,n = (c/|Am|)δm,n. The second is an ”intuitive physicallymotivated” random flux for the multilevel mesh which at coarse-refined interfaces uses

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 19

0 500 1000

0

50

100

150

200

y

Cov

aria

nce

Mesh Cross Section

0 500 1000

0

50

100

150

200

y

Cov

aria

nce

Mesh Cross Section

0 500 1000

0

50

100

150

200

y

Cov

aria

nce

Mesh Cross Section

0 500 10000

200

400

600

800

1000

x

y

’White Noise’

0 500 10000

200

400

600

800

1000

x

y

’Random Fluxes’

0 500 10000

200

400

600

800

1000

x

y

’Adjusted Stochastic Forcing’

Fig. 5.2. The top row shows the covariance of the equilibrium fluctuations of the system foreach choice of the stochastic driving field at a cell at the coarse-refined interface on the coarse side(see + symbol). In the second row, the covariance with this cell is plotted as a function of y at crosssections of the mesh near the interface.

for coarse-cells the average of the random fluxes of the neighboring refined mesh cells.The third is the choice derived from equation 3.5–3.7.

We shall make the comparisons on a periodic mesh. In this case an importanttechnicality arises in that the operator L is in general not invertible, since the constantconcentration field is in the null space of L, in particular, Lq = 0 with qm = 1 forall m. This means that the covariance C is only defined by equation 3.7 up to anarbitrary rank one matrix of the form λqqT . To obtain an invertible operator L weshall restrict ourselves to a subspace which corresponds to concentration fields whichsum to a given fixed value qT c = c0, which determines λ.

The equilibrium fluctuations at the coarse-refined interface associated with eachchoice of the stochastic driving field are shown in Figures 5.1 and 5.2. We find thatusing a ’white noise’ stochastic driving field introduces long-range correlations intothe equilibrium fluctuations. Using the phenomenological discretization of averaged’random fluxes’ at the coarse-refined interface also results in long-range correlationsin the equilibrium fluctuations of the system. The choice using the covariance derivedfrom 3.7 is found to have only localized uncorrelated fluctuations, corresponding tothe equilibrium fluctuations given in equation 3.5. This shows that numerical errorscan be controlled to be localized if statistical features of the stochastic equations aretaken into account when discretizing the stochastic driving term.

6. Applications. We now discuss how the stochastic numerical methods canbe used in practice. We shall first discuss some important issues in modeling thechemical reaction terms of the stochastic reaction-diffusion system, which arise fromthe stochastic partial differential equation 2.1 only admitting generalized solutions.

20 P.J. ATZBERGER

We then demonstrate how the methods can be used in the context of two specificapplications. The first concerns how biological cells sense concentration gradientsin the environment. The stochastic numerical methods are used to investigate therole of concentration fluctuations in a proposed mechanism for cell gradient sensing.The second application concerns the Gray-Scott chemical reactions in a spatiallyextended system. The stochastic numerical methods are used to investigate the roleof concentration fluctuations in patterns exhibited in the system.

6.1. Modeling the Chemical Reactions. Since the stochastic partial differ-ential equations 2.1–2.7 do not admit classical solutions the point-wise values of thestochastic concentration field, which are often used in modeling chemical reactions,may not be meaningful. To cope with this issue many studies of stochastic reaction-diffusion systems tacitly regularize the stochastic differential equations by discretizingthe system on a regular lattice. As a consequence, many of the results reported inthe literature have a direct dependence on the numerical scheme and discretizationparameters, such as the discrete lattice spacing [8; 73; 74; 76–78]. In many of the nu-merical schemes the magnitude of the concentration field fluctuations, which play animportant dynamic role through the non-linear chemical reaction terms, even divergein magnitude as the numerical mesh is refined [77; 78]. However, as a consequenceof the finite number of molecules undergoing the diffusive migrations and chemicalreactions one would expect on physical grounds that observations of the continuumconcentration field representation of the system would exhibit fluctuations havingcontributions to the chemical reactions of only a finite size.

As with almost any field description arising in continuum mechanics, the contin-uum field description and governing equations are not expected to yield physicallymeaningful results beyond some small characteristic length scale of the system. Forinstance as a rather extreme example, one would not expect a continuum field de-scription to accurately capture features of the physical system on length scales smallerthan the distance of the mean free path of the molecular collisions. Similar issues arisefor other descriptions such as the reaction-diffusion master’s equations in which spa-tial discretization parameters must be carefully tuned not to be too large or too smallrelative to the distance molecules migrate between chemical reactions (reaction mean-free path) in order to obtain physically reasonable results [8; 73; 74; 76; 96]. We shallavoid such fine tuning of the numerical methods by explicitly introducing an addi-tional physical parameter in the parameterization of our models for use in an explicitregularization of the stochastic concentration field in order to appropriately accountfor the role of fluctuations in the chemical reactions of the system.

Many regularization procedures can be considered for the stochastic fields. Ideally,such a procedure would be motivated from studies of molecular dynamics simulations,particle based simulations, or quantum field theoretic models [73; 75; 88; 89; 91; 92;96; 98–100]. Here we shall take a more phenomenological approach and use for thechemical reactions regularizations of the general form

F[c](x, t) =

∫

α(x,y, η(y, t))dy

η(y, t) =

∫

β(y, z)c(z, t)dz

∫

β(y, z)dy = 1.

In these expressions, α accounts for the rate at which molecules of each chemical

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 21

species are introduced or removed at location x by the reactions. The β term modelsthe fraction of molecules at location z which migrate to participate in chemical reac-tions associated with location y. These terms have the role of effectively smoothingthe concentration field over some length scale ℓ. The expression can be conceptuallymotivated by thinking about the individual molecules for a given configuration repre-sented by the continuum field when projecting the microscopic degrees of freedom ofthe diffusive migrations and reactions to obtain the rate of change of the continuumconcentration field. We remark that such a conceptual approach could also be used inprinciple to derive additional terms from a microscopic model to account for stochasticcontributions arising from the chemical reactions of the system, if these are of signif-icant magnitude [99]. We now discuss how this general form relates to a commonlyused model for the chemical reactions in reaction-diffusion master’s equations.

One approach used in modeling the chemistry of reaction-diffusion systems isbased on extending the theory developed for homogeneous macroscopic chemical re-actors to spatially extended systems by assuming that the system can be treatedlocally as homogeneous and well-mixed. In formulating continuum reaction-diffusionequations the chemical kinetics are modeled by a(c) using the same functional formas for the chemistry in the homogeneous case but instead using the local concentra-tion value. In stochastic models, such as the reaction-diffusion master’s equations, alength scale appears in the discretization of space into control volumes of size ℓ. Inthe discretization it is tacitly assume that ℓ is larger than the distance of the meanfree path of the molecular collisions but small enough that molecular species can beapproximated locally as well-mixed and homogeneous [73; 76; 96]. From these consid-erations the reactions are approximated as occurring nearly independently in volumesof size V = ℓd, where d is the spatial dimension of the system.

In the case of a single molecular species, let s denoting the local number ofmolecules in the control volume V . The reactions are then parameterized by a micro-scopic kinetic rate a(s) for the number of molecules introduced or removed from thevolume per unit time [73; 94; 95]. This rate is related to the macroscopic kinetic ratea(c) used in continuum reaction-diffusion equations by

a(c) = a(s)/V(6.1)

which uses that the macroscopic rate is given in terms of the concentration of moleculesintroduced or removed per unit time per unit volume. In the current setting, the terms is given in terms of the continuum field description of the system by

s(x, t) =

∫

V(x)

c(y, t)dy.(6.2)

This gives for the chemical reactions a functional of the form

F[c](x, t) = a(s(x, t))/V.(6.3)

This corresponds to taking η = s/V , α(x,y, η) = (a(ηV )/V )δ(y − x), β(y, z) =χV(y)(z)/V where V(y) is the control volume centered about y and χV is the char-acteristic function over V . There are many possible choices for V(x) and χV , suchas choosing a set V(x) which is a disk or square centered at x and a function χV

which decays smoothly to zero over the set V . A similar expression can be obtainedfor the case of multiple species c(x, t) and spatially varying rates a(x, c). We remarkthat the functional yields fluctuations in concentration which contribute to the chem-ical reactions which are finite and are explicitly controlled by our choice of α, β, as

22 P.J. ATZBERGER

opposed to being controlled through features of the numerical discretization. Manyother forms for the functionals can of course be introduced to model the contributionsof the chemical reactions.

A key point to be emphasized is that when using stochastic partial differentialequations to account for fluctuations in reaction-diffusion systems the particular func-tional used for the chemical reactions should be regarded as part of the modeling andparameterization of the system at a coarse-grained level. Ideally when modeling at thecontinuum level such functionals should be obtained from microscopic models of thephysical system. We shall discuss some additional choices for the chemical reactionterms in the context of specific applications in Sections 6.2 and 6.3.

6.2. Gradient Sensing in Cell Biology (Concentration Fluctuations inNon-Rectangular Geometries). In cell biology the spatial distribution of chem-ical species plays a fundamental role in many cellular processes. For example, thebacterium Escherichia coli detects gradients in the concentration of important nutri-ents in the environment which bias the direction a cell moves toward regions whichare more nutrient rich. Individual Dictyostelium discoideum bacterium cells respondto spatial and temporal features of concentration fields of signaling molecules, suchas cAMP generated by other cells, to coordinate collective movements which resultin the formation of complex multicellular structures, such as fruiting bodies andspores [55; 56]. In the developmental biology of multicellular organisms concentrationfields of many signaling molecules are used to determine cell differentiation withintissues [1; 93; 106; 107]. The means by which cells detect local concentrations andgradients of external chemical species and determine a response is a fundamentalprocess in cell biology.

Features of the external concentration field are typically detected by cells throughbinding of the external molecules to receptor proteins which reside in the outer cellmembrane. Receptor proteins upon binding undergo conformational changes whichtrigger local chemical reactions which produce products which diffuse along the cellmembrane or into the cytoplasm [1; 106; 107]. While many proteins and metabo-lites involved in these processes are known, there are many questions concerning theparticular interactions and mechanisms by which concentration field signals gener-ate a cellular response. This is a rather active area of experimental and theoreticalresearch [47; 101; 103–105; 107; 109; 110].

Here we formulate a basic model and investigate the role played by intrinsicfluctuations of the concentration field in the detection of concentration gradients. It isillustrative to characterize the concentration fluctuations on the length and time scalesof individual cells. A typical cellular environment can have signaling chemical species,which are present in concentrations which can range from as small as picomolar (pM)to as large as molar (M), see [1; 49; 108; 109].

As an illustration consider an intermediate concentration of 1mM, and the lengthscale of a 100nm cubic box. One millemolar corresponds to mM = 10−3NA/litre= 6.022 × 1023 molecules/m3 with NA Avogadro’s number. On the length scale of100nm there is on average only 6.022 × 102 molecules per box. For a rough estimateof the time scale of the fluctuations we note that typical signaling molecules, such ascAMP, have diffusion coefficients on the order of 108nm2/s, see [43]. For a box withedge length ℓ = 100nm the amount of time required for a particle to diffuse out ofthe box is of the order τD = ℓ2/D = 10−4s. This provides a rough estimate of thetime scale on which fluctuations are expected to be correlated. Given that cells areobserved to change course in chemotaxis in very shallow concentration gradients on the

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 23

Fig. 6.1. Gradient Sensing Schematic: (Top Left) Environmental concentration gradient in-teracting with a single cell. (Top Right) A schematic of the cellular gradient sensing mechanism.Molecules of the external concentration field bind to receptor proteins embedded in the cell mem-brane. Receptor binding events cause conformational changes of receptor proteins which activatechemical reactions which release product molecules which diffuse in the cell membrane and cyto-plasm. (Bottom Left) The adaptive numerical mesh on which the stochastic concentration equation6.7 is solved for a cell having the geometry of a disk. The gradient is not imposed but rather arisesfrom sources and sinks at either end of the rectangular domain (Dirichlet Boundary Conditions).The obtained gradient profile is the solution of the concentration equations on the domain exterior tothe disk shaped cell, on which no-flux boundary conditions are imposed (Neumann Boundary Condi-tions). (Right Bottom) Closer view of the adaptive mesh which includes cut-cells in the vicinity ofthe curved boundary of the biological cell which is adapted to obtain an accurate numerical solution.

time scale of seconds or faster, fluctuations in concentration may play an importantrole [50]. Here we show how equations 2.1 – 2.7 and the proposed numerical methodscan be used to investigate the role of fluctuations in concentration of the signalingchemical species.

To model how a cell initially processes a signal detected by membrane receptorswe shall consider a system of three basic chemical species which originate and diffusewithin the cellular membrane. The chemical species are (i) an activator molecularspecies denoted by A, (ii) an inhibitor molecular species denoted by I, and (iii) areporter molecular species denoted by Q. The reporter species Q is meant to accountfor how the receptor binding events result in an internal chemical signal which feedsinto further cellular reactions which give products within the cell membrane and cyto-plasm, such as signals for cell motility (local actin polymerization), cell polarization,or calcium release from local buffers / internal stores [1; 47; 107; 109].

We shall consider each of the molecular species as being in one of two possibleforms: active or inactive, which are denoted by P ∗ and P , respectively. Transitions

24 P.J. ATZBERGER

0 0.5 1 1.5 2 2.5 3

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

θ

Con

cent

ratio

n

Receptor Input (S)

0 0.5 1 1.5 2 2.5 3

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

θ

Con

cent

ratio

n

Activator Response (E)

0 0.5 1 1.5 2 2.5 30.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1

θ

Con

cent

ratio

n

Inhibitor Response (I)

0 0.5 1 1.5 2 2.5 3

0.96

0.97

0.98

0.99

1

θ

Con

cent

ratio

n

Resporter Output (Q)

Fig. 6.2. Gradient Sensing Simulation Results: The gradient sensing cell model was simulatedwith parameters chosen in a physical regime corresponding to typical rates of chemical reactions anddiffusion reported in the cell biology literature, see Table 6.2. The mean concentration is plottedwith estimated error bars corresponding to three standard deviations. For ease of comparison theconcentration levels are scaled by the maximum mean concentration value for each chemical species.(Top Left) Shows the receptor activation level for a shallow external concentration gradient withfluctuations large relative to the gradient. The maximum mean concentration was 1.5mM. (TopRight) The concentration profile of the reporter chemical species which diffuses in the cell mem-brane. The maximum mean concentration was 9.3mM. (Bottom Left) The concentration profile ofthe inhibitor. The maximum mean concentration was 1.5mM. (Bottom Right) The concentrationprofile of the activator chemical species. The maximum mean concentration was 1.5mM.

between inactive and active can occur, for example through phospholyation or methy-lation of the individual proteins. We shall generically refer to this as the productionof the active species or deactivation of the active species. In the model, we posit thatthe cell processes the external signal to form the reporter products Q by two com-peting processes. The first involves increases in the concentration of species A whichincreases the local production of the active reporter species Q∗ → Q. The secondinvolves increases in the concentration of species I which increases the local deactiva-tion of the reporter species Q → Q∗. The external concentration field influences theseprocesses through the receptor binding events which locally produce active species ofA and I. More precisely, the model for the chemical species inside the biological cellis given by the following system of reaction-diffusion equations:

∂E

∂t= DE∆E − κdeE + κreS(6.4)

∂I

∂t= DI∆I − κdiI + κriS(6.5)

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 25

∂Q

∂t= DQ∆Q + κqeE(QT − Q) − κqiIQ,(6.6)

where QT = Q∗ + Q. We shall consider for the geometry of the cell a disk of radiusR in two dimensional space. In this case, the cell membrane corresponds to the circleof radius R. The equations 6.4 - 6.6 should be considered to reside on this circularmembrane with periodic boundary conditions.

The external concentration field C(x, t) is obtained as the solution of

∂C

∂t= DC∆C + η(6.7)

〈η(x, t)η(x′, t′)〉 = −2DC∆δ(x − x′)δ(t − t′)(6.8)

where η satisfies the conditions in equation 2.1. The external concentration field c isobtained by solving equation 6.7 on a domain which has sources-sinks at the left andright edges which impose a fixed concentration (Dirichlet Boundary Conditions) andin a region exterior to the biological cell, which is treated as being impermeable to theexternal chemical species (no-flux Neumann Boundary Conditions), see Figure 6.1.

As we shall discuss the external concentration gradient is no longer strictly linearwhen taking into account the geometry and no-flux boundary of the biological cell.This potentially overlooked feature could have important implications in interpretingbiological experiments. We remark that the mesh is adaptive in space and includescut-cells near the curved surface of the biological cell, see Figure 6.1. More complexgeometries in two and three dimensions can also be considered with a fairly straight-forward extension of the methodology proposed here.

The external concentration field influences the production rates of internal chem-ical species by the receptor binding events, which in the model produce activator andinhibitor at the local rates κreS and κriS. The receptor binding events are modeledat a coarse-grained level by considering the local number of molecules which are inthe vicinity of a receptor cluster. In our model we take Si(x, t) = αniδ(x − xi) foreach cluster, where ni denotes the number of molecules in the vicinity of cluster i atlocation xi. This corresponds to molecules diffusing near the receptor cluster bindingat the rate α.

To obtain ni, a local integration of the concentration field is performed by convo-luting with a kernel Λ(x,xi) centered at the ith receptor cluster at location xi. Usinga radial symmetric kernel we have

ni =

∫

Λ(|x − xi|)c(x, t)dx.

We remark that in the case that Λ(r) = 1 for r < a and zero otherwise, the ni wouldcorrespond to the number of molecules within distance r∗ of the receptor cluster. Inpractice, we will use a smooth kernel which decays rapidly for r > a to help reduceartifacts which arise from shifts of xi relative to the discretization mesh.

We now discuss how the model can be used to investigate the effect of fluctuationson the cellular processing of an external concentration gradient. We consider the casewhen the fluctuations are large relative to the gradient around the cell perimeter. Toparameterize the model to be in an appropriate physical regime, we use kinetic ratesand diffusion coefficients taken from the cell biology literature. Our specific choice ofparameters can be found in Table 6.2.

Signaling molecules have diffusion coefficients on the order of 108nm2s−1. Forexample, the signaling molecule cAMP is reported to have a diffusion coefficient of

26 P.J. ATZBERGER

Table 6.1

Cell Gradient Sensing Model: Description of the Parameters

Parameter DescriptionR Radius of the disk shaped cell.DS Diffusion coefficient of external signaling chemical species.a Receptor sensor associated length scale.DE Diffusion coefficient of excitatory chemical species.κre Rate of receptor initiated activation of excitatory chemical species.κde Rate of degradation/deactivation of excitatory chemical species.DI Diffusion coefficient of inhibitory chemical species.κri Rate of receptor initiated activation of inhibitory chemical species.κdi Rate of degradation/deactivation of inhibitory chemical species.QT Total concentration of reporter chemical species.DQ Diffusion coefficient of reporter chemical species.κqe Rate of production of active reporter aided by the excitatory chemical species.κqi Rate of degradation/deactivation of active reporter aided by the inhibitory chemical species.

Table 6.2

Cell Gradient Sensing Model: Values of the Parameters

Parameter DescriptionR 2 × 104 nm.DS 108 nm2s−1.a 100 nm.DE 106 nm2s−1.κre 104 s−1.κde 104 s−1.DI 5 × 107nm2s−1.κri 1 s−1.κdi 1 s−1.QT 100 mM.DQ 106 nm2s−1.κqe 10−1 mM−1s−1.κqi 1 mM−1s−1.

2.7± 0.2 108 nm2s−1, see [43]. The diffusion coefficients of molecules diffusing in thecell can range from 105 nm2s−1 – 107 nm2s−1, see [44; 45]. In cell gradient sensingit is observed that cells are able to respond to changes in the external concentrationfield on the order of seconds [47; 55]. The rates of the biochemical chemical reactionsare taken to occur in the model with first order rates ranging from 1s−1 – 104s−1 andsecond order rates 10−1 mM −1s−1 – 1 mM−1s−1.

Simulations of the cell gradient sensing mechanism were carried out by numer-ically integrating in time the discretized reaction-diffusion system using the Euler-Maruyuma Method [13]. In the discretizations used near the curved boundaries ofthe disk shaped cell in order to reduce stiffness in the discretized equations cut-mesh-cells were fused to form larger composite cells when smaller than 10% of thecorresponding full mesh-cell size, as discussed in Section 3.3. For the receptor binding

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 27

events and internal production of reaction products the kernel Λ which is integratedover the mesh was taken to be a gaussian with standard deviation a = 100nm. A timestep of 2.5 × 10−5s was used and the model was simulated for 1.6 × 106 time stepscorresponding to a physical time scale of 40s.

In Figure 6.2 the results of the simulation for the concentrations fluctuations ofthe signaling and intracellular chemical species are reported. It is found that evenwhen the concentration fluctuations are large to the gradient the proposed sendingmechanism is capable of producing a reporter concentration profile which robustlyindicates the external mean concentration gradient provided observations occur overa sufficient duration of time.

6.3. Gray-Scott Reaction-Diffusion (Fluctuation Induced Patterns). Wenow consider a variant of the Gray-Scott reaction-diffusion system given by

∂u

∂t= D1∆u + f [u, v] + η1(6.9)

∂v

∂t= D2∆v + g[u, v] + η2(6.10)

where η1, η2 account for the concentration fluctuations and are Gaussian randomfields with mean zero and covariance

〈η1(x, t)η1(x′, t′)〉 = −2uD1∆xδ(x − x′)δ(t − t′)(6.11)

〈η2(x, t)η2(x′, t′)〉 = −2vD2∆xδ(x − x′)δ(t − t′)(6.12)

〈η1(x, t)η2(x′, t′)〉 = 0.(6.13)

The D1, D2 are the diffusion coefficients of the chemical species. This corresponds tothe physical regime in which concentration fluctuations are small relative to the meanconcentration and are dominated by density fluctuations not depending directly onthe chemical reactions. The chemical reactions of the molecular species are accountedfor by the functionals f , g of the general cubic form:

f [u, v](x, t) = α6nun2v + α5n

2u + α4n

2v + α3nunv + α2nu + α1nv + α0(6.14)

g[u, v](x, t) = β6nun2v + β5n

2u + β4n

2v + β3nunv + β2nu + β1nv + β0(6.15)

with

nu(x, t) =

∫

Λ(y − x)u(y, t)dy(6.16)

nv(x, t) =

∫

Λ(y − x)v(y, t)dy(6.17)

where the kernel is taken to be Λ(z) = (1/√

2πσ2) exp(

−|z|2/2σ2)

. This has theeffect of explicitly regularizing the concentration field over the length scale σ.

To obtain a discrete approximation to the integrals in equations 6.16 – 6.17 onthe multilevel mesh we shall use nu,m =

∑

m Λm,num∆ym with Λm,n = 1Zm

Λ(xm −xn)∆xd

n where Zm normalizes the kernel to sum to one in the index n and is givenby Zm =

∑

n Λ(xm − xn)∆xdn. We remark that as ∆x → 0 it follows that Zm → 1

and we have nu,m → nu giving a well-defined limit as the mesh is refined.We now discuss the reaction-diffusion system for a particular choice of parameter

values, see Table 6.3. For this choice, the phase portrait of the basic non-spatial two

28 P.J. ATZBERGER

0.9 0.92 0.94 0.96 0.98 1 1.02 1.04 1.06 1.08 1.1

0.92

0.94

0.96

0.98

1

1.02

1.04

1.06

1.08

1.1

1.12

u

v

f−nullcline

g−nullcline

g−nullcline

f−nullcline

g−nullcline

g−nullcline

Fig. 6.3. Phase Portrait of the Gray-Scott Local Dynamics

dimensional dynamical system corresponding to the chemical reactions f, g is given inFigure 6.3. In the system corresponding to the local dynamics there is only one stablesteady-state at (nu = 1.1, nv = 1.0). However, an interesting feature of the phasespace is that there is a region in which the two nullclines pass in close proximity (thesystem is close to undergoing a bifurcation [111]). An interesting behavior of the sys-tem is that even relatively small perturbations of the dynamical system in this regioncan cause the system state to cross the nullclines. Such back and fourth switchingcould in principle effectively stabilizing states in this region of the phase plane. Inthe reaction-diffusion system, perturbations are introduced, from this perspective, tothe local dynamical system through the internal concentration fluctuations and thespatial extent of the system in which diffusion and the regularization couples laterallythe local dynamical states of the system. To investigate the behavior of the spatiallyextended system subject to fluctuations we shall use equations 2–2.7 approximatedby the proposed adaptive stochastic numerical methods on a multilevel mesh.

In the simulations an initial perturbation is introduced into the system in whicha small square region centered at the origin of edge length 17.6 is set to (u = 1.07, v =1.03), everywhere else is set to (u = 1.1, v = 1.0). Since similar such states could arisespontaneously from fluctuations in the system, introduction of this initial perturbationavoids having to simulate the potentially long waiting time for a rare ”nucleatingevent” which breaks the translational symmetry of the homogeneous state. In orderto make sure this initial perturbation alone is not responsible for the observed results,simulations were performed in the absence of fluctuations, and it was found thata simple symmetric pattern of only four spots is obtained and appears stable forlong times, see Figure 6.4. To investigate the role of stochastic effects, the systemwith the same initial condition was simulated subject to intrinsic fluctuations, seeTable 6.3. Unlike the deterministic case, a rich pattern of spot-like patterns emergeswhere the spots continually migrate and replicate, see Figure 6.5. Movies of the fullevolution process of the emerging pattern can be found on-line [102]. We find thatthe fluctuations may play a non-negligible role in pattern formation processes and

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 29

Fig. 6.4. Gray-Scott Reaction-Diffusion System in the Deterministic Case

Table 6.3

Gray-Scott: Values of the Parameters

Parameter Description[α0, α1, α2, α3, α4, α5, α6] [1.100605,−2.2,−1.10055, 2.2, 1.1, 0.0,−1.1]× 106.[β0, β1, β2, β3, β4, β5, β6] [−0.998845, 1.998845, 1.0,−2.0,−1.0, 0.0, 1.0]× 106.Du 5.5 × 103.Dv 5.0 × 103.L 5.632 × 102.u 1.1.v 1.0.

may introduce additional structures not observed in the corresponding deterministicsystem.

30 P.J. ATZBERGER

Fig. 6.5. Gray-Scott Reaction-Diffusion System with Internal Concentration Fluctuations

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 31

7. Conclusions. In the context of reaction-diffusion systems we have shown howto approximate a class of stochastic partial differential equations using adaptive finitedifference methods traditionally applied to deterministic equations. A key feature ofthe approach was to discretize the stochastic driving terms in a manner respectingfundamental statistical features of the equations. Our approach uses results for thestochastic differential equations which relate the covariance of the stochastic drivingterms to the covariance of the equilibrium fluctuations to ensure that the Gaussianstochastic driving terms are discretized in a manner that at statistical steady-state ex-hibit fluctuations which agree with the averaged fluctuations of the continuum system.The stochastic numerical methods allow for reaction-diffusion systems with intrinsicfluctuations dominated by density fluctuations to be simulated on multilevel meshesand on domains which may have geometries with curved boundaries. The numericalmethods potentially allow for significant gains in computational efficiency throughadaptive resolution of phenomena occurring in spatially extended stochastic systems.

8. Acknowledgments. The author would like to acknowledge support from theresearch grant NSF DMS-0635535 while at UCSB. The author would like to thankAlexander Roma and Boyce Griffith for helpful suggestions concerning adaptive nu-merical methods. The author would also like to thank Samuel Isaacson for stimulatingdiscussions concerning reaction-diffusion systems. The work on biological cell gradientsensing benefited from talks and discussions at the May 2008 workshop QuantitativeApproaches to Cell Motility at the Institute for Mathematics and its Applications(IMA) at the University of Minnosota.

References.[1] B. Alberts, A. Johnson, J. Lewis, M. Raff, K. Roberts, and

P. Walker, Molecular Biology of the Cell, Garland Publishing, (2002).[2] L. C. de Hoog and M. Mann, Proteomics, Annu. Rev. Genomics Hum. Genet.,

5:267–293, (2004).[3] C. P. Fall, E. S. Marland, J. M. Wagner, and J. J. Tyson, eds., Compu-

tational cell biology, vol. 20 of Interdisciplinary Applied Mathematics, Springer-Verlag, New York, (2002). ??

[4] L. E. Reichl, A modern course in statistical physics, John Wiley & Sons Inc.,New York, second ed., (1998).

[5] J. W. Cooley and J. W. Tukey, An algorithm for the machine calculationof complex Fourier series, Math. Comput. 19: 297?301. (1965).

[6] H. Ruben, Generalised Concentration Fluctuations under Diffusion Equilibrium,J. Appl. Prob. 1, 47-68 (1964).

[7] R. F. Fox and G. E. Uhlenbeck, Contributions to Non-Equilibrium Thermo-dynamics: I. Theory of Hydrodynamic Fluctuations, Phys. Fluid 13, 8, (1970).

[8] F. Baras and M. M. Mansour, Reaction-diffusion master equation: A com-parison with microscopic simulations, Phys. Rev. E, Vol. 54, No. 6, Dec. (1996).

[9] L. D. Landau and E. M. Lifshitz, Course of theoretical physics. Vol. 9:Statistical physics, Pergamon Press, Oxford, (1980).

[10] C. W. Gardiner, Handbook of stochastic methods, Springer Series in Synerget-ics, 1985

[11] N.G. van Kampen, Stochastic Processes in Physics and Chemistry, North-Holland Publishing, 1981

[12] M. Holmes, Introduction to Perturbation Methods, Springer-Verlag, (1995).[13] P. E. Kloeden and E. Platen, Numerical solution of stochastic differential

equations, Springer-Verlag, 1992.

32 P.J. ATZBERGER

[14] L. D. Landau and E. M. Lifshitz, Course of theoretical physics. Vol. 9:Statistical physics, Pergamon Press, Oxford, 1980, ch. IX.

[15] B. Oksendal, Stochastic Differential Equations: An Introduction with Applica-tions. Springer, 2000

[16] G. Oster and A. Mogilner, Force generation by cellular polymers.Supramolecular Polymers, A. Ciferri (ed.), New York:Dekker (in press), 2005.

[17] P. J. Atzberger, P, R. Kramer, and C. S. Peskin, A Stochastic ImmersedBoundary Method for Fluid-Structure Interactions at Microscopic Length Scales,J. Comp. Phys., Vol. 224, Iss. 2, (2007).

[18] P, R. Kramer, C. S. Peskin, and P. J. Atzberger, On the Foundationsof the Stochastic Immersed Boundary Method, (to appear in Computational Me-chanics).

[19] B. E. Griffith, Simulating the blood-muscle-valve mechanics of the heart byan adaptive and parallel version of the immersed boundary method, PhD thesis,Courant Institute of Mathematical Sciences, New York University, 2005.

[20] A. M. Roma, A multilevel self adaptive version of the immersed boundarymethod., PhD thesis, Courant Institute of Mathematical Sciences, New YorkUniversity, 1996.

[21] C. S. Peskin, The immersed boundary method, Acta Numerica, 11 , pp. 1–39,(2002).

[22] P. R. Kramer and C. S. Peskin , Incorporating thermal fluctuations intothe immersed boundary method, Proceedings of the Second MIT Conference onComputational Fluid and Solid Mechanics, K. K. Bathe, ed. Elsevier, 2, pp. 1755–1758, (2003).

[23] M. L. Minion, A projection method for locally refined grids., Journal of Com-putational Physics, 127(1):158–178, 1996.

[24] M. J. Berger and I. Rigoutsos, An algorithm for point clustering and gridgeneration. IEEE Transactions on Systems. Man and Cybernetics, 21(5):1278–1286, 1991.

[25] W. L. Briggs, V. E. Henson, and S. F. McCormick, A Multigrid Tutorial,Second Edition. Society for Industrial and Applied Mathematics, Philadelphia,PA, USA, 2000.

[26] A.J. Chorin, A numerical method for solving incompressible viscous flow prob-lems, J. Comput. Phys. 2 (1) (1967) 12?26.

[27] A. J. Chorin, Numerical solution of the Navier-Stokes equations, Mathematicsof Computation, 22(104):745–762, 1968.

[30] S. F. McCormick Multilevel Adaptive Methods for Partial Differential Equa-tions, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA,1989.

[29] S. F. McCormick, Multilevel Projection Methods for Partial Differential Equa-tions, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA,1992.

[30] S. F. McCormick, S. M. McKay, and J. W. Thomas, Computational com-plexity of the fast adaptive composite grid (FAC) method. Applied NumericalMathematics, 6(3):315–327, 1989.

[31] C.S. Peskin, Flow patterns around heart valves: a numerical method, J. Comput.Phys. 10 (1972) 252?271.

[32] C.S. Peskin, Numerical analysis of blood flow in the heart, J. Comput. Phys. 25(1977) 220?252.

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 33

[33] A.M. Roma, C.S. Peskin, M.J. Berger, An adaptive version of the immersedboundary method, J. Comput. Phys. 153 (1999) 509?534.

[34] D.L. Brown, R. Cortez, M.L. Minion, Accurate projection methods for theincompressible Navier?Stokes equations, J. Comput. Phys. 168 (2001) 464?499.

[35] P.McCorquodale, P.Collela, and H.Johansen, A Cartesian Grid Embed-ded Boundary Method for the Heat Equation on Irregular Domains, J. Comput.Phys. 173, 620–635 (2001).

[36] T. Ye, R. Mittal,H. S. Udaykumar,and W. Shyy, An Accurate CartesianGrid Method for Viscous Incompressible Flows with Complex Immersed Bound-aries, J. of Comput. Physics 156, 209?-240 (1999)

[37] J. J. Quirk, An Alternative to Unstructured Grids for Computing Gas DynamicFlows Around Arbitrarily Complex Two-Dimensional Bodies, Computers FluidsVol. 23, No. I, pp. 125–142, (1994).

[38] M. Berger, M. Aftosmis and J. Melton, Accuracy, Adaptive Methods andComplex Geometry, Proc. First AFOSR Conference on Dynamic Motion CFDRutgers University, L. Sakell and D. Knight, editors, June, 1996.

[39] D. M. Ingram, D .M. Causon, and C. G. Mingham, Developments in Carte-sian cut cell methods Mathematics and Computers in Simulation 61, 561-?572(2003) .

[40] W. J. Coirier and K. G. Powell, An Accuracy Assessment of Cartesian-Mesh Approaches for the Euler Equations, J. Comput. Physics 117, 121–131(1995).

[41] M. S. Day, P. Colella, M. J. Lijewski, C. A. Rendleman and D. L. Mar-

cus, Embedded Boundary Algorithms for Solving the Poisson Equation on Com-plex Domains, Lawrence Berkeley National Laboratory. Paper LBNL-41811.http://repositories.cdlib.org/lbnl/LBNL-41811, (May 13, 1998).

[42] M. Postma, P. J. M. Van Haastert, A Diffusion?Translocation Model forGradient Sensing by Chemotactic Cells, Biophysical Journal, Vol. 81, pg. 1314?-1323, September 2001.

[43] C. Chen,T. Nakamura, and Y. Koutalos, Cyclic AMP Diffusion Coefficientin Frog Olfactory Cilia. , Biophysical Journal, Vol. 76, pg. 2861-?2867, May 1999.

[44] Niv, H., O. Gutman, Y. I. Henis, and Y. Kloog, Membrane interactionsof a constitutively active GFP-Ki-Ras 4B and their role in signaling. Evidencefrom lateral mobility studies., J. Biol. Chem. 274: 1606?1613, (1999).

[45] Arrio-Dupont, M., G. Foucault, M. Vacher, P. F. Devaux, and S.

Cribier, Translational diffusion of globular proteins in the cytoplasm of culturedmuscle cells., Biophys. J. 78:901?907, 2000.

[46] J. Krishnan and P. A. Iglesias, Analysis of the Signal Transduction Proper-ties of a Module of Spatial Sensing in Eukaryotic Chemotaxis, Bulletin of Math-ematical Biology, 65, 95?128, (2003).

[47] P. A. Iglesias and P. N. Devreotes, Navigating through models of chemo-taxis Current Opinion in Cell Biology 20:35?40, (2008).

[48] M. I. Monine, A. M. Berezhkovskii, E. J. Joslin,H. S. Wiley,

D. A. Lauffenburger,and S. Y. Shvartsman, Ligand Accumulation in Au-tocrine Cell Cultures, Biophysical Journal Volume 88, 2384-?2390 April (2005).

[49] D. Ishii, K. L. Ishikawa, T. Fujita, M. Nakazawa, Stochastic modellingfor gradient sensing by chemotactic cell, Science and Technology of AdvancedMaterials 5, 715?718 (2004).

[50] H. C. Berg and E. M. Purcell, Physics of Chemoreception, Biophysical

34 P.J. ATZBERGER

Journal VOl. 20, 1977.[51] T. S. Shimizu, S. V. Aksenov and D. Bray, A Spatially Extended Stochastic

Model of the Bacterial Chemotaxis Signalling Pathway, J. Mol. Biol. 329, 291-?309 (2003).

[52] A. Narang, K. K. Subramanian, and D. L. Lauffenburger, A mathe-matical model for chemoattractant gradient sensing based on receptor-regulatedmembrane phospholipid signaling dynamics., Ann. of Biomed. Eng. 29, 677-691(2001).

[53] H. Levine, D. A. Kessler, and W. J. Rappel, Directional sensing in eu-karyotic chemotaxis: A balanced inactivation model, PNAS vol. 103, no. 26,9761-?9766, June 27, (2006).

[54] M. D. Onsum, K. Wong, P. Herzmark, H. R. Bourne, and A. P. Arkin,Morphology matters in immune cell chemotaxis: membrane asymmetry affectsamplification, Phys. Biol. 3,190?-199 (2006).

[55] C. L. Manahan, P. A. Iglesias, Y. Long, and P. N. Devreotes, Chemoat-tractant Signaling in Dictostelium Discoideum, Annu. Rev. Cell Dev. Biol.20:223?53, (2004).

[56] H. Zhang, M. ?R. Gomez-Garcia, M. R. W. Brown, and A. Kornberg,Inorganic polyphosphate in Dictyostelium discoideum: Influence on development,sporulation, and predation, PNAS, Feb. 22, vol. 102, no. 8, pp. 2731-?2735,(2005).

[57] M. Onsum and C. V. Rao, A Mathematical Model for Neutrophil GradientSensing and Polarization, PLoS Comp. Bio. Vol. 3 Iss. 3, March (2007).

[58] T. M. Keenan and A. Folch, Biomolecular gradients in cell culture systems,Royal Soc. of Chem. Lab Chip, 8, 34-?57 (2008).

[59] N. Andrew and R. H. Insall, Chemotaxis in shallow gradients is mediatedindependently of PtdIns 3-kinase by biased choices between random protrusions,Nature Cell Biology, Vol. 9, No. 2, February (2007).

[60] Ian C. Schneider and Jason M. Haugh, Quantitative elucidation of a distinctspatial gradient-sensing mechanism in fibroblasts, J. of Cell Bio., Vol. 171, No. 5,883?-892. December 5, (2005).

[61] N. Mastronardia, M. Van Barel, E. Van Camp, and R. Vandebril, Oncomputing the eigenvectors of a class of structured matrices, J. of Comp. andApplied Math. 189, 580?-591, (2006).

[62] G. Da Prato and J. Zabczyk, Stochastic equations in infinite dimensions,Cambridge University Press, (1992).

[63] J. C. Strikwerda, Finite Difference Schemes and Partial Differential Equa-tions, Society for Industrial and Applied Mathematics, (2004).

[64] Analysis, E. H. Lieb and M. Loss, American Mathematical Society, (1997).[65] H. Holden, B. Oksendal, J. Uboe, and T. Zhang, Stochastic Partial Dif-

ferential Equations: A Modeling, White Noise Functional Approach, Birkhauser,(1996).

[66] H. Kunita, Generalized Solutions of a Stochastic Partial Differential Equation,J. of Theoretical Probability, Vol. 7, No. 2, (1994).

[67] A. M. Turing, The Chemical Basis of Morphogenesis, Phil. Trans. R. Soc.London B 237 pp 37-72, (1952).

[68] W. C. Carter, J. E. Taylor, and J. W. Cahn, Variational Methods forMicrostructural-Evolution Theories, JOM of the The Minerals, Metals & Mate-rials Society (TMS), December, (1997).

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 35

[69] J. D. Murray, Mathematical Biology, 2nd Corrected Edition, Springer, (1993).[70] J. Keener and J. Sneyd, Mathematical Physiology, Springer-Verlag, (1993).[71] M. Mimura, Reaction-Diffusion Systems Arising in Biological and Chemical

Systems: Application of Singular Limit Procedures, Mathematical Aspects ofEvolving Interfaces, ed. Ambrosio, L., Deckelnick, K., Dziuk, G., Mimura, M.,Solonnikov, V., Soner, H.M., Springer-Verlag, (2000).

[72] L. C. Evans, Partial Differential Equations, American Mathematical Society,(1998).

[73] S. A. Isaacson and C. S. Peskin, Incorporating diffusion in complex geome-tries into stochastic chemical kinetics simulations, SIAM J. Sci. Computing, Vol.28, No. 1, pp. 47?74, (2006).

[74] S. A. Isaacson, The reaction-diffusion master equation as an asumptotic ap-proximation of diffusion to a small target, preprint, (2008).

[75] S. A. Isaacson, Relationship between the reaction?diffusion master equationand particle tracking models, J. of Phys. A, 41, 065003, (2008).

[76] J. Elf, A. Doncic. and M. Ehrenberg, Mesoscopic reaction-diffusion inintracellular signaling, Proceedings of SPIE Vol. 5110, (2003).

[77] F. Lesmes, D. Hochberg, F. Mora, and J. Perez-Mercader, Noise-Controlled Self-Replicating Patterns, Phys. Rev. Lett., Vol. 91, No. 23, 5, Dec.(2003).

[78] A. Munteanu and R. V.Sole’, Pattern Formation in Noisy Self-ReplicatingSpots, International Journal of Bifurcations and Chaos 16(12), 3679-3685, (2006).

[79] Q. Du and T. Zhang, Numerical Approximation of Some Linear StochasticPartial Differential Equations Driven by Special Additive Noises, SIAM J. Num.Anal., Vol. 40, No. 4, pp. 1421?1445, (2002).

[80] G. Strang, Linear Algebra and Its Applications, Harcourt Brace and Company.[81] M. A. Katsoulakis, G. T. Kossioris, and O. Lakk, A finite element method

via noise regularization for the stochastic allen-cahn problem, Sussex Mathemat-ics Research Reports, SMMR-2005-07,(2005).

[82] W. H. Press and Saul A. Teukolsky and W. T. Vetterling and B. P.

Flannery, Numerical Recipes, Cambridge University Press, (2002).[83] P. Gray and S. K. Scott, Sustained Oscillations and Other Exotic Patterns

of Behavior in Isothermal Reactions, J. Phys. Chem, Vol. 89, pp.22–32, (1985).[84] K. Lee, W. D. McCormick, J. E. Pearson, and H. L. Swinney, Experimental

observation of self-replicating spots in a reaction-diffusion system,Nature, Vol. 369, May (1994).

[85] J. Stewart, Calculus, Thomson Brooks/Cole (2009).[86] R. Courant, K. Friedrichs and H. Lewy, On the partial difference equations

of mathematical physics (translation from German), IBM Journal, March, pp.215-234, (1967).

[87] L. N. Trefethen and D. Bau, Numerical Linear Algebra, Society of Industrialand Applied Mathematics, (1997).

[88] G. .A. Bird, Molecular Gas Dynamics and the Direct Simulation of Gas Flows,(Oxford: Clarendon), (1994).

[89] D. A. McQuarrie, Stochastoc Approach to Chemical Kinetics. J. Appl. Prob.4, 413-478, (1967).

[90] T. Leppanen, M. Karttunen, R. A. Barrio, and K. Kaski, Turing systemsas models of complex pattern formation, Brazilian J. of Physics, Vol. 34, no2, pp.368-372, (2004).

36 P.J. ATZBERGER

[91] S. S Andrews and D. Bray, Stochastic simulation of chemical reactions withspatial resolution and single molecule detail, Phys. Biol. 1, pp.137?151, (2004).

[92] F. Zaera and I. Rusinek, Monte Carlo Simulation for Inhomogeneous Chem-ical Kinetics: Application to the Belousov-Zhabotinskii Reaction, J. of Comp.Chem., Vol. 2, No. 4,402-409, (1981).

[93] T. Tabata and Y. Takei, Morphogens, their identification and regulation, De-velopment 131, 703-712 (2004).

[94] D. T. Gillespie, A General Method for Numerically Simulating the StochasticTime Evolution of Coupled Chemical Reactions, J. of Comp. Phys. 2, 403–434,(1976).

[95] D. Gillespie, Exact Stochastic Simulation of Coupled Chemical Reactions, J. ofPhys. Chem., Vol. 8, 1, No. 25, (1977).

[96] D. Gillespie A rigorous derivation of the chemical master equation, Physica A,188,404-425, (1992).

[97] H. Wang, Z. Fu, X. Xu, and Q. Ouyang, Pattern Formation Induced byInternal Microscopic Fluctuations, J. Phys. Chem. A, 111, 1265-1270, (2007).

[98] K. Takahashi, S. N. Arjunan, M. Tomita Space in systems biology of signal-ing pathways ? towards intracellular molecular crowding in silico, FEBS Letters,579, 1783?1788, (2005).

[99] A. J. Chorin, O. H. Hald, and R. Kupferman, Optimal prediction and theMori-Zwanzig representation of irreversible processes, PNAS, March 28, ,vol. 97,no. 7, (2000).

[100] D. Hochberga, M. P. Zorzano, and F. Moran, Spatiotemporal patternsdriven by autocatalytic internal reaction noise, J. of Chem. Phys, 122, 214701,(2005).

[101] S. Funamoto,R. Meili, S. Lee, L. Parry, and R. A. Firtel, Spatial andTemporal Regulation of 3-Phosphoinositides by PI 3-Kinase and PTEN MediatesChemotaxis, Cell, Vol. 109, 611?623, May 31, (2002).

[102] P. J. Atzberger, http://www.math.ucsb.edu/∼atzberg/index.html[103] C. Bouzigues, M. Morel, A. Triller, and M. Dahan, Asymmetric re-

distribution of GABA receptors during GABA gradient sensing by nerve growthcones analyzed by single quantum dot imaging, PNAS, July 3, vol. 104, no. 27,11251?-11256, (2007).

[104] C. Y. Chung, S. Funamoto and R. A. Firtel, Signaling pathways con-trolling cell polarity and chemotaxis, TRENDS in Biochemical Sciences, Vol.26,No.9, September (2001).

[105] N. Andrew and R. H. Insall, Chemotaxis in shallow gradients is mediatedindependently of PtdIns 3-kinase by biased choices between random protrusions,Nature Cell Biology, Vol. 9, No. 2, Feb. (2007).

[106] B. Shilo and N. Barkai, EGF receptor signaling ? a quantitative view, Curr.Bio. Vol 17 No 24, (2007).

[107] S. Y. Shvartsman, H. S. Wiley, and D. A. Lauffenburger, EpidermalGrowth Factor Receptor Signaling in Tissues, IEEE Control Systems Magazine,Aug, (2004).

[108] M. I. Monine, A. M. Berezhkovskii, and al., Ligand Accumulation inAutocrine Cell Cultures, Biophysical Journal, Vol. 88, April, 2384?2390, (2005).

[109] I. I. Moraru and L. M. Loew, Intracellular Signaling: Spatial and TemporalControl, Physiology, 20, 169?-179, (2005).

[110] T. M. Keenan and A. Folch, Biomolecular gradients in cell culture systems,

SPATIALLY ADAPTIVE METHODS FOR REACTION DIFFUSION SYSTEMS 37

Lab on a Chip, 8, 34?57, (2008).[111] J. Guckenheimer and P. Holmes, Nonlinear Oscillations, Dynamical Sys-

tems, and Bifurcations of Vector Fields, Springer-Verlag, New York, (1983).