Embed Size (px)
Citation preview
Numerical solutions of stochastic PDEs driven by arbitrary type of noise1
Tianheng Chen2, Boris Rozovskii3 and Chi-Wang Shu4
Abstract
So far the theory and numerical practice of stochastic partial differential equations
(SPDEs) have dealt almost exclusively with Gaussian noise or Levy noise. Recently, Mikule-
vicius and Rozovskii (2016) [22] proposed a distribution-free Skorokhod-Malliavin calculus
framework that is based on generalized stochastic polynomial chaos expansion, and is com-
patible with arbitrary driving noise. In this paper, we conduct systematic investigation on
numerical results of these newly developed distribution-free SPDEs, exhibiting the efficacy
of truncated polynomial chaos solutions in approximating moments and distributions. We
obtain an estimate for the mean square truncation error in the linear case. The theoreti-
cal convergence rate, also verified by numerical experiments, is exponential with respect to
polynomial order and cubic with respect to number of random variables included.
Keywords: distribution-free, stochastic PDE, stochastic polynomial chaos, Wick product,
Skorokhod integral
1Research supported by ARO grant W911NF-16-1-0103.2Division of Applied Mathematics, Brown University, Providence, RI 02912. E-mail:
tianheng [email protected] of Applied Mathematics, Brown University, Providence, RI 02912. E-mail:
boris [email protected] of Applied Mathematics, Brown University, Providence, RI 02912. E-mail: [email protected]
1
1 Introduction
Stochastic partial differential equations (SPDEs) and stochastic ordinary differential equa-
tions (SODEs) describe functions driven by finite or infinite dimensional stochastic processes.
SPDEs have found a broad range of applications, including mathematical biology, financial
engineering and nonlinear filtering, to quantify the intrinsic uncertainty in these models (see
e.g., Lototsky and Rozovskii [16]). Since analytical solutions to SPDEs are rarely avail-
able, numerical methods have to be designed to solve them. The most popular approach
is the Monte Carlo method, which generates independent random sample paths via direct
discretization or averaging over characteristic lines (see e.g., Milstein and Tretyakov [23]).
However, it is well known that Monte Carlo simulation suffers from slow rate of convergence
and often requires millions of samples to reach a desired accuracy level. Various techniques
have been established to accelerate convergence and alleviate computational burden, e.g.,
importance sampling and quasi Monte Carlo sampling. For more detailed discussion on nu-
merical methods of SPDEs, we refer readers to the books [24, 37] and the references therein.
An alternative numerical approach is based on the truncation of stochastic polynomial
chaos expansion, which is well-developed for SPDEs driven by Gaussian randomness (see
e.g., reference books [8, 7]). Such methodology starts with the orthogonal representation
of Gaussian white noise by a series of i.i.d standard Gaussian random variables. According
to the Cameron-Martin theorem [3], products of Hermite polynomials of these Gaussian
variables constitute a complete L2 basis of the probability space (also known as Hermite
chaos or Wiener chaos in the literature). Moreover, the multiple Ito integral formula [10]
helps recast stochastic integrals into the Wick product form [29], leading to a system of
deterministic PDEs, called propagator (see e.g., Lototsky and Rozovskii [14]), satisfied by
the expansion coefficients. Then standard numerical solvers can be applied to the propagator
system. The Wiener chaos expansion approach separates the stochastic part (basis functions)
and the deterministic part (coefficients), and is usually recognized as the stochastic Galerkin
method (see e.g., Xiu [33]) due to its evident resemblance with the spectral Galerkin method
in numerical analysis.
For linear parabolic SPDEs, the propagator system has lower triangular structure and
hence can be solved sequentially [19, 12, 20, 14]. The authors in [12] also derived an estimate
for the truncation error, showing that the mean square error converges exponentially with
respect to expansion order and linearly with respect to number of random variables included,
but grows exponentially as time grows. A recursive multistage modification was then de-
veloped in [12] to enable long time integration and investigated numerically in [34]. As a
counterpart, a recursive version of the stochastic collocation method was thoroughly studied
2
in [35, 36] as well. On the other hand, the propagator system for nonlinear SPDEs is fully
coupled and requires substantially more computational power [9, 17]. Wick-Malliavin ap-
proximation has been proposed in [21] as a decoupling technique. Numerical simulations were
performed for nonlinear problems and elliptic SPDEs with random coefficients in [28, 27] to
validate the efficacy of Wick-Malliavin approximation for polynomial nonlinearity and small
noise.
Although the stochastic polynomial chaos expansion is mostly restricted to Gaussian
random noise, the extension towards arbitrary type of noise has been explored over the last
two decades. Xiu and Karniadakis [30, 31] constructed the correspondence between types
of random variables and orthogonal polynomials in the Askey scheme [1]. Their generalized
polynomial chaos (gPC) technique turned out to be very successful for problems with random
initial/boundary condition and/or random coefficients [31, 32, 33]. However, gPC expansion
is not naturally compatible with stochastic integrals due to the lack of some vital connections,
e.g., Wick product, Skorokhod integral [26] and Malliavin calculus [18]. These building blocks
are indeed available for Levy randomness [8, 7], and Liu [11] presented some numerical
results of SPDEs driven by Poisson noise. Recently, Mikulevicius and Rozovsky built the
distribution-free Skorokhod-Malliavin calculus framework [22] directly upon gPC expansion,
giving rise to a new family of SPDEs under their arbitrary noise paradigm. For linear SPDEs,
the propagator system is independent of the type of randomness involved, and so are the
first and second moments (this is already noticed by Benth and Gjerde [2] for Poisson noise);
meanwhile, for nonlinear problems, the propagator system does vary from one type of noise
to another.
The main objective of this paper is to numerically testify the stochastic polynomial chaos
approach to solving the distribution-free SPDEs in [22]. The rest of this paper is structured
as follows. In Section 2 we summarize the distribution-free stochastic analysis infrastructure
in [22], and provide propagator systems for linear and nonlinear model problems. In Section 3
we improve the truncation error estimate in [12] for linear problems, proving that convergence
rate of mean square error is actually cubic with respect to the number of random variables.
Theorem 3.1 is a major contribution of this paper. Section 4 performs numerical experiments
on linear and nonlinear SPDEs. We will study numerical orders of convergence and the
claims in Section 3 are verified. Exact solutions or reference solutions are obtained through
some alternative approaches, including Monte Carlo simulation, moment equations and the
Fokker-Planck equation. We conduct comparisons with reference solutions, and between
different types of driving noise. Concluding remarks and possible future research directions
are given in Section 5. The computation of interaction coefficients appeared in the nonlinear
propagator system is shown in the appendix.
3
2 Distribution-free stochastic analysis
Let (Ω,F , P) be a probability space and H = L2([0, T ]) for some T > 0. We define the
following driving noise N(t):
N(t) =
∞∑
k=1
mk(t)ξk (2.1)
and the stochastic process
N(t) =
∫ t
0
N(s)ds =
∞∑
k=1
( ∫ t
0
mk(s)ds)ξk (2.2)
where Ξ = ξk∞k=1 is a sequence of uncorrelated random variables with E[ξk] = 0 and
E[ξ2k] = 1 for each k ≥ 1, and mk∞k=1 is a complete orthonormal basis in H . The term
distribution-free arises from the fact that each random variable can be of any distribution.
They are not required to be identically distributed or independent. Then we aim to construct
an orthogonal basis in L2(Ω, σ(Ξ), P), in a sense that is similar to the Wiener chaos expansion
for Gaussian noise.
2.1 Polynomial chaos expansion
Denote by J the set of multi-indices α = (αk)∞k=1 of finite length |α| =
∑∞k=1 αk:
J = α = (αk)∞k=1 : αk ≥ 0, |α| <∞
Each multi-index α with |α| = n can be uniquely identified by its characteristic set Iα =
(i1α, i2α, · · · , inα), which is a vector of length n and given by
imα = k if and only if
k−1∑
j=1
αj < m ≤k∑
j=1
αj, for each 1 ≤ m ≤ n
For instance, if α = (0, 1, 0, 2, 3, 0, · · ·), Iα = (2, 4, 4, 5, 5, 5). Particularly, we let d(α) = inα,
the position of the rightmost nonzero entry in α. We also define polynomials and factorials
of multi-indices:
ξα :=∞∏
k=1
ξαk
k , α! :=∞∏
k=1
αk!
The following two assumptions are required for stochastic polynomial chaos expansion.
A1. For each finite dimensional random vector (ξi1, ξi2, · · · , ξid), the moment generating
function E[exp(θ1ξi1 + θ2ξi2 + · · ·+ θdξid)] exists for all (θ1, θ2, · · · , θd) in some neighborhood
of 0 ∈ Rd.
4
A2. We have an orthogonalization Kα, α ∈ J of the system of polynomials ξα, α ∈ J such that for each n ≥ 1, Kp, |p| ≤ n spans the same linear subspace Hn as ξp : |p| ≤ n,and for each |α| = n + 1,
Kα = ξα − projectionHnξα
We rescale the basis functions Φα = cαKα so that E[ΦαΦβ ] = δαβ(α!). Obviously Φε0= 1
and Φεk= ξk where ε0 is the multi-index whose entries are all zero, and εk is the multi-index
such that its k-th entry is 1 and all other entries are zero for k ≥ 1. It is demonstrated in
[22] that Φα, α ∈ J is indeed a complete Cameron-Martin type basis.
Proposition 2.1. [22] Assume A1 and A2 hold. Then Φα, α ∈ J is a complete orthog-
onal basis of L2(Ω, σ(Ξ), P). For each η ∈ L2(Ω, σ(Ξ), P), its stochastic polynomial chaos
expansion is
η =∑
α∈J
ηαΦα, ηα =E[ηΦα]
α!
and the Parseval’s identity holds:
E[η2] =∑
α∈J
(α!)η2α
In such manner we separate the random part and the deterministic part.
For the special case where ξk∞k=1 are i.i.d. random variables, if there exists an orthog-
onal set of univariate polynomials ϕn(ξ1)∞n=0 such that E[ϕn(ξ1)ϕm(ξ1)] = δmnn!, and the
moment generating function of ξ1 is finite near zero, the polynomial chaos basis functions
are simply products of these univariate polynomials
Φα =
∞∏
k=1
ϕαk(ξk) (2.3)
Then A1 and A2 are satisfied and Φα, α ∈ J forms a complete orthogonal basis. Table
2.1 shows some common types of random distributions and their corresponding polynomial
chaos basis functions (see e.g., [30]). It may be necessary to shift and rescale ξk∞k=1 to
achieve zero mean and unit variance. Two specific examples are provided below.
Example 2.1.1. If ξk∞k=1 are i.i.d. standard Gaussian random variables, ϕn(ξ1)∞n=0
are probabilists’ Hermite polynomials Hen(x)∞n=0. Here E[ϕ2n(ξ)] = n! is automatically
satisfied. The driving noise N(t) =∑∞
k=1 mk(t)ξk is indeed the Gaussian white noise W (t),
and the driving process N(t) is the standard Wiener process W (t) (see e.g., [8]).
5
Table 2.1: Correspondence between random distribution and polynomial chaos basis func-tions for an i.i.d. sequence of random variables.
Distribution of ξ1 Polynomial chaos basisContinuous Gaussian Hermite
Gamma LagurreBeta JacobiUniform Legendre
Discrete Poisson CharlierBinomial KrawtchoukNegative binomial MeixnerHypergeometric Hahn
Example 2.1.2. If ξk∞k=1 are i.i.d. uniformly distributed on [−√
3,√
3], ϕn(ξ1)∞n=0 are
the rescaled version of Legendre polynomials Ln(x)∞n=0:
ϕn(ξ1) =√
(2n + 1)n!Ln
( ξ1√3
)(2.4)
to ensure E[ϕ2n(ξ1)] = n!. Analogous to the Wiener process, the driving stochastic process
N(t) has zero mean and uncorrelated increments. However, N(t) is non-Gaussian as its
characteristic function is
E[iθN(t)] = E
[exp
(iθ
∞∑
k=1
( ∫ t
0
mk(s)ds)ξk
)]=
∞∏
k=1
sin(√
3θ( ∫ t
0mk(s)ds
))
√3θ
( ∫ t
0mk(s)ds
)
Remark 2.1. We intentionally use the weaker assumption of uncorrelated random variables
to incorporate Levy randomness, whose chaos expansion basis functions are not polynomials
of simple random variables. Such complexity is beyond the scope of this paper, and we will
always consider i.i.d. random variables in our numerical experiments.
2.2 Wick product and Skorokhod integral
This subsection briefly explains the languages to construct distribution-free stochastic inte-
grals. For the opposite direction, i.e. the stochastic (Malliavin) derivative, we refer interested
readers to [22] for more details. We will work on the space of generalized random variables
written as formal chaos expansion series:
D′(E) :=
u =∑
α∈J
uαΦα : uα ∈ E
and the space of square integrable general random variables:
D(E) :=u =
∑
α∈J
uαΦα : uα ∈ E,∑
α∈J
α!‖uα‖2E <∞
6
where E is a given Hilbert space. For instance, if E = H = L2([0, T ]), D′(H) consists of
generalized stochastic processes u = u(t) =∑
α∈J uα(t)Φα such that each uα ∈ L2([0, T ]),
andD(H) equals D′(H)∩L2(Ω×[0, T ]), the subspace of square integrable stochastic processes
in D′(H).
Let us first introduce the Wick product ⋄, a convolution type binary operator on expan-
sion coefficients:
Φα ⋄ Φβ = Φα+β, u ⋄ v =∑
α∈J
∑
β∈J
uαvβΦα+β for u, v ∈ D′(R)
Then for u = u(t) ∈ D′(H), its Skorokhod integral δ(u) ∈ D′(R) is denoted by
δ(u) :=
∫ T
0
u(t) ⋄ N(t)dt =∑
α∈J
∞∑
k=1
(∫ T
0
uα(t)mk(t)dt)Φα+εk
(2.5)
Especially, if u = u(t) ∈ H is a deterministic function,
δ(u) =∞∑
k=1
(∫ T
0
u(t)mk(t)dt)ξk, E[δ(u)2] =
(∫ T
0
u(t)mk(t)dt)2
= ‖u‖2H (2.6)
Therefore, δ can be regarded as an isometric embedding from H into L2(Ω).
Proposition 2.2. Suppose u ∈ C([0, T ]) is continuous. Consider a sequence of partitions
of [0, T ]:
∆i = [ti,j−1, ti,j] : 1 ≤ j ≤ i− 1, ti,0 = 0, ti,i = T
such that |∆i| → 0. We have
i∑
j=1
u(ti,j−1)(N(ti,j)−N(ti,j−1))→ δ(u)
in L2(Ω). In other words, δ(u) is the limit of discrete sums in Ito’s sense.
Proof. In fact,
i∑
j=1
u(ti,j−1)(N(ti,j)−N(ti,j−1)) =i∑
j=1
u(ti,j−1)( ∞∑
k=1
( ∫ ti,j
ti,j−1
mk(t)dt)ξk
):= δ(ui)
where
ui(t) =i∑
j=1
1[ti,j−1,ti,j)(t)u(ti,j−1)
and 1 is the indicator function. As |∆i| → 0, it is easy to verify that ui → u in H . Hence
according to the isometric property, δ(ui) converges to δ(u) in L2(Ω).
7
There is an equivalent way to characterize the Skorokhod integral in terms of multiple
integrals. For nonnegative n, let Hn := L2([0, T ]n) and Hn be the family of symmetric
functions in Hn. We use t(n) as the short hand notation of (t1, t2, · · · , tn). For each multi-
index α with |α| = n, we set
Eα(t(n)) =∑
σ∈Gn
mi1α(tσ(1))mi2α
(tσ(2)) · · ·minα(tσ(n)) (2.7)
where Gn is the permutation group on 1, 2, · · · , n. Notice that Eα, |α| = n is a complete
orthogonal basis of Hn and ‖Eα‖2Hn = n!α!. Then we are ready to establish multiple integrals
on Hn. For f =∑
|α|=n fαEα(t(n)) ∈ Hn, let
In(f) := n!∑
|α|=n
fαΦα ∈ D(R)
In(f) is square integrable due to the fact that
E[In(f)2] =∑
|α|=n
α!(n!fα)2 = n!‖f‖2Hn <∞
Now for u ∈ D(R), we can write down its chaos expansion with regard to multiple integrals:
u =∞∑
n=0
In(fn(t(n))), fn(t(n)) :=1
n!
∑
|α|=n
uαEα(t(n)) (2.8)
By the same token, for u = u(t) ∈ D(H),
u(t) =
∞∑
n=0
In(fn(t, t(n))), fn(t, t(n)) :=1
n!
∑
|α|=n
uα(t)Eα(t(n)) (2.9)
Definition 2.1. u = u(t) ∈ D(H) with expansion (2.9) is called adapted if
supp fn(t, ·) ∈ [0, t]n, for each t ∈ [0, T ]
The proposition below points out the close connection between Skorokhod integral and
the classic Ito integral, which is crucial to deriving deterministic propagators for Ito type
SPDEs. Proofs can be found in [7].
Proposition 2.3. For u = u(t) ∈ D(H) with expansion (2.9), if δ(u) ∈ D(R), the multiple
integral expansion of δ(u) is
δ(u) =∞∑
n=0
In+1(fn(t(n+1))) (2.10)
8
where the standard symmetrization g for g ∈ Hn is taken to be
g(t(n)) :=1
n!
∑
σ∈Gn
g(tσ(1), tσ(2), · · · , tσ(n)) (2.11)
Furthermore, in the case that ξk∞k=1 are i.i.d. standard Gaussian variables, if u is also
adapted, the Skorokhod integral coincides with the Ito integral:
δ(u) =
∫ T
0
u(t) ⋄ W (t)dt =
∫ T
0
u(t)dW (t) (2.12)
2.3 SPDE model problems
In this subsection we will look into examples of linear and nonlinear distribution-free SPDEs.
We will take linear parabolic SPDE as the linear model problem, and stochastic Burgers
equation as the nonlinear model problem. In both examples, the solution u = u(t, x) lives in
D(L2([0, T ]×D)), where D ∈ Rd is a smooth finite domain equipped with periodic boundary
condition. The polynomial chaos expansion of u(t, x) reads
u(t, x) =∑
α∈J
uα(t, x)Φα, uα ∈ L2([0, T ]2 ×D)
Example 2.3.1. Consider the following homogeneous linear parabolic SPDE:
∂tu(t, x) = Lu(t, x) +Mu(t, x) ⋄ N(t), (t, x) ∈ (0, T ]×D
u(0, x) = u0(x), x ∈ D(2.13)
where
Lu(t, x) =
d∑
i=1
d∑
j=1
aij(x)∂i∂ju(t, x) +
d∑
i=1
bi(x)∂iu(t, x) + c(x)u(t, x) (2.14)
Mu(t, x) =d∑
i=1
αi(x)∂iu(t, x) + β(x)u(t, x) (2.15)
and ∂i is the i-th spatial partial derivative. If ξk∞k=1 are i.i.d. standard Gaussian variables,
according to Proposition 2.3, (2.13) is equivalent to the Ito type SPDE
du(t, x) = Lu(t, x)dt +Mu(t, x)dW (t), (t, x) ∈ (0, T ]×D (2.16)
or the Stratonovich type SPDE
du(t, x) = Lu(t, x)dt +Mu(t, x) dW (t), (t, x) ∈ (0, T ]×D (2.17)
where Lu = Lu − 12MMu. We assume that the coefficients in L and M are smooth and
bounded, L is uniformly elliptic, and u0(x) is deterministic and bounded. These assumptions
9
are sufficient for a unique square integrable solution u ∈ D(L2([0, T ]×D)) (see e.g., [20, 15]).
As we will see later, the propagator system is independent of the type of noise. Therefore
these well-posedness requirements remain the same in the distribution-free setting.
Recall the definition of the Skorokhod integral (2.5). We come up with the propagator
system by comparing the expansion coefficients on both sides of (2.13):
∂tuα(t, x) = Luα(t, x) +∑
εk≤α
Muα−εk(t, x)mk(t), (t, x) ∈ (0, T ]×D
uα(0, x) = u0(x)1α=ε0, x ∈ D
(2.18)
It is a system of linear parabolic deterministic PDEs, with a lower-triangular and sparse
structure, i.e. a multi-index with order n only talks to itself and multi-indices with order
n − 1. As a result, we can solve the system sequentially, and coefficients with the same
order can be updated in parallel. Additionally, the system does not depend on the type
of randomness involved, which implies the computational overhead from changes of noise is
almost negligible. The propagator is solved once and for all.
Numerical discretization of (2.18) usually follows the method of lines technique, in which
we start with standard spatial discretization schemes, transforming the propagator system
into a larger system of ODEs. Suppose that the spatial discretization has M degrees of
freedom, and A and B are M ×M difference matrices of L andM. The ODE system is
u′α(t) = Auα(t) +
∑
εk≤α
mk(t)Buα−εk(t), t ∈ (0, T ]
uα(0) = u01α=ε0
(2.19)
where uα(t) ∈ RM is the vector uα(x, t) evaluated at those degrees of freedom. Then Runge-
Kutta type ODE solvers can be directly adopted.
In practice, it is impossible to handle the infinite system (2.18) or (2.19), and a finite
truncation is always necessary. For K, N ≥ 0, define the truncated multi-index set
JN,K := α ∈ J : |α| ≤ N, d(α) ≤ K, #(JN,K) =
(N + K
N
)
That is, JN,K contains multi-indices whose polynomial order is no more than N , and number
of random variables is no more than K. The size of JN,K grows rapidly with respect to both
N and K. The truncated solutions of (2.18) and (2.19) are
uN,K(t, x) :=∑
α∈JN,K
uα(t, x)Φα, uN,K(t) :=∑
α∈JN,K
uα(t)Φα (2.20)
Approximation error introduced by finite truncation will be further analyzed in Section 3.
10
Example 2.3.2. Here we restrict our interest to the one-dimensional case such that D ⊂ R.
Consider the stochastic Burgers equation with additive noise [9]:
∂tu(t, x) + ∂x
(u2
2
)= µ∂2
xu + σ(x)N(t), (t, x) ∈ (0, T ]×D
u(0, x) = u0(x), x ∈ D(2.21)
where µ is a positive constant and σ(x) is a periodic forcing function. The corresponding
Ito type SPDE in the case of i.i.d standard Gaussian noise is
du(t, x) =(µ∂2
xu− ∂x
(u2
2
))dt + σ(x)dW (t), (t, x) ∈ (0, T ]×D
u(0, x) = u0(x), x ∈ D(2.22)
By assuming that u0(x) is deterministic and σ, u0 ∈ L2(D), we make sure that (2.21) has a
unique square integrable solution (see e.g., [6]). However, such result does not generalize to
the distribution-free setting as the propagator system varies for different driving noises.
In order to figure out the propagator system, we have to expand u2 as stochastic poly-
nomial chaos expansion:
u2 =∑
α∈J
( ∑
β∈J
∑
p∈J
B(α, β, p)uβup
)Φα (2.23)
where
B(α, β, p) =E[ΦαΦβΦp]
E[Φ2α]
=E[ΦαΦβΦp]
α!(2.24)
are interaction coefficients. Hence the propagator equations are
∂tuα(t, x) +1
2
∑
β∈J
∑
p∈J
B(α, β, p)∂x(uβup) = µ∂2xuα + σ(x)
∞∑
k=1
1α=εkmk(t), (t, x) ∈ (0, T ]×D
uα(0, x) = u0(x)1α=ε0, x ∈ D(2.25)
It is a fully coupled system of nonlinear PDEs, whose interaction coefficients B(α, β, p)
depend on the type of driving noise. Compared with the linear case, (2.25) lacks sparsity,
and must be recalculated each time we change distribution. Both features make the nonlinear
problem more challenging and expensive to simulate. In Appendix A we will present the
generic procedure to calculate interaction coefficients, and explicit formulas for some special
types of distribution.
For a truncated multi-index set JN,K , the approximated solution is
uN,K(t, x) :=∑
α∈JN,K
uα(t, x)Φα
11
where uα : α ∈ JN,K satisfies the truncated propagator system
∂tuα(t, x)+1
2
∑
β∈JN,K
∑
p∈JN,K
B(α, β, p)∂x(uβup) = µ∂2xuα+σ(x)
∞∑
k=1
1α=εkmk(t), (t, x) ∈ (0, T ]×D
(2.26)
We note that uα is not uα due to aliasing error. The method of lines technique can also be
applied to the numerical discretization of (2.26).
Remark 2.2. In principle, the propagator system can be determined explicitly as long as
we only have polynomial nonlinearity. We expand power functions as tensor products in
a way that is similar to (2.23). The expansion of nonpolynomial functions is much more
challenging. Several methods are presented in [5] to perform general function evaluations on
polynomial chaos series.
3 Error estimate
For the sake of simplicity, we will focus on the truncation error analysis of the ODE system
(2.19), which can be viewed as the propagator system of the following linear SODE system:
u′(t) = Au(t) + Bu(t) ⋄ N(t), t ∈ (0, T ]
u(0) = u0
(3.1)
where u ∈ D(H) and H = (L2([0, T ]))M . A and B are constant M ×M matrices. The
reason for such simplification is twofold. Since (3.1) is the spatial discretization of (2.13), it
is the equation we are actually dealing with in numerical simulations. Besides, all arguments
in this section can be generalized to (2.13) in a straightforward manner, but with more
technical considerations. We simply replace the Euclidean norm with appropriate Sobolev
norms and impose regularity assumptions on L andM (see e.g., [36]).
Theorem 3.1. Suppose that mk∞k=1 is the trigonometric basis
m1(t) =
√1
T, mk(t) =
√2
Tcos
((k − 1)πt
T
), k ≥ 2 (3.2)
Let λA = ‖A‖2 and λB = ‖B‖2 be the matrix norms. Then the mean square error of the
truncated solution uN,K(T ) from (2.20) is bounded by the estimate
E[|uN,K(T )−u(T )|2] ≤ e(2λA+λ2B)T
((λ2BT )N+1
(N + 1)!+
16λ2Aλ2
BT 3
π4(K − 12)3
(5+3λ2AT 2+6λ4
BT 2))|u0|2 (3.3)
12
Remark 3.1. From (3.3), we conclude that the mean square truncation error converges at
an exponential rate with respect to N , and at a cubic rate with respect to K. We improve
the error estimate in [12] where the authors only proved linear rate with respect to K. Cu-
bic convergence result can also be found in [9] for a special example of stochastic Burgers
equation. However, the approximation error increases exponentially in time. Long time sim-
ulation might be impractical. At least more expansion coefficients are required to balance out
error growth.
The proof is primarily along the lines in [12]. We first prove a lemma to extract the
analytical solution of the propagator system.
Lemma 3.1. Suppose uα(t), α ∈ J solves the propagator system (2.19). For each n ≥ 0
and α ∈ J with |α| = n, the explicit formula of uα(t) is
uα(t) =1
α!
∫ (t,n)
Fn(t, t(n))Eα(t(n))dt(n) (3.4)
where Eα(t(n)) is from (2.7) and
Fn(t, t(n)) := e(t−tn)ABe(tn−tn−1)AB · · ·Bet1Au0
∫ (t,n)
g(t(n))dt(n) :=
∫ t
0
∫ tn
0
· · ·∫ t2
0
g(t(n))dt1 · · · dtn−1dtn
Notice that Fn is not related to the fn in (2.9) as Fn is not symmetric.
Proof. We prove by induction on n. If n = 0, uε0(t) = etAu0. (3.4) is obviously correct. Now
for n ≥ 1 and |α| = n, we assume that (3.4) holds for all β ∈ J with |β| < n. By Duhamel’s
principle,
uα(t) =
∫ t
0
e(t−s)A( ∑
εk≤α
mk(s)Buα−εk(s)
)ds
=1
α!
∫ t
0
e(t−s)A( ∑
εk≤α
αkmk(s)B
∫ (s,n−1)
Fn−1(s, t(n−1))Eα−εk
(t(n−1))dt(n−1))ds
=1
α!
∫ (t,n)
Fn(t, t(n))( ∑
εk≤α
αkmk(tn)Eα−εk(t(n−1))
)dt(n)
=1
α!
∫ (t,n)
Fn(t, t(n))Eα(t(n))dt(n)
where we use the identity
Eα(t(n)) =∑
εk≤α
αkmk(tn)Eα−εk(t(n−1))
Hence (3.4) is satisfied by any α.
13
Proof of Theorem 3.1. According to Parseval’s identity, we decompose the truncation error
as
E[|uN,K(T )− u(T )|2] =∞∑
n=N+1
∑
|α|=n
α!|uα(T )|2 +∞∑
k=K+1
N∑
n=1
∑
|α|=n,d(α)=k
α!|uα(T )|2
We only need to show the two inequalities below:
∞∑
n=N+1
∑
|α|=n
α!|uα(T )|2 ≤ e(2λA+λ2B
)T (λ2BT )N+1
(N + 1)!|u0|2 (3.5)
∞∑
k=K+1
N∑
n=1
∑
|α|=n,d(α)=k
α!|uα(T )|2 ≤ e(2λA+λ2B)T 16λ2
Aλ2BT 3
π4(K − 12)3
(5 + 3λ2AT 2 + 6λ4
BT 2)|u0|2 (3.6)
As for (3.5), set Fn(T, ·) to be the standard symmetrization of Fn(T, ·) (Fn is extended
with zero value outside the simplex t(n) : 0 ≤ t1 ≤ · · · ≤ tn ≤ T), we have
uα(T ) =1
α!
∫ (T,n)
Fn(T, t(n))Eα(t(n))dt(n) =1
α!
∫
[0,T ]nFn(T, t(n))Eα(t(n))dt(n)
Since Eα, |α| = n is an orthogonal basis of Hn,
∑
|α|=n
α!|uα(T )|2 =∑
|α|=n
1
α!
∣∣∣∫
[0,T ]nFn(T, t(n))Eα(t(n))dt(n)
∣∣∣2
= n!‖Fn(T, ·)‖2Hn = (n!)2
∫ (T,n)
|Fn(T, t(n))|2dt(n)
=
∫ (T,n)
|Fn(T, t(n))|2dt(n)
(3.7)
For any given t(n),
|Fn(T, t(n))| ≤ eλA(T−tn)λBeλA(tn−tn−1) · · ·λBeλAt1 |u0| = eλAT λnB|u0|
Plugging this into (3.7) yields
∞∑
n=N+1
∑
|α|=n
α!|uα(T )|2 ≤( ∞∑
n=N+1
e2λAT (λ2BT )n
n!
)|u0|2 ≤ e(2λA+λ2
B)T (λ2
BT )N+1
(N + 1)!|u0|2
which exactly recovers (3.5). Here we use the mean-value form of the remainder term of
Taylor’s expansion:∞∑
n=N+1
xn
n!= eθx xN+1
(N + 1)!for some θ ∈ [0, 1]
14
Proof of (3.6) is more involved. For any α with |α| = n and d(α) = k,
∫ (T,n)
Fn(T, t(n))Eα(t(n))dt(n) =
n∑
j=1
∫ (T,n−1) (∫ tj+1
tj−1
Fn(T, t(n))mk(tj)dtj
)Eα−εk
(t(n\j))dt(n\j)
=n∑
j=1
∫ (T,n−1) (∫ tj
tj−1
Fn(T, t(n\j,s))mk(s)ds)Eα−εk
(t(n−1))dt(n−1)
(3.8)
where t(n\j) is the short hand notation of (t1, · · · , tj−1, tj+1, · · · , tn) and t(n\j,s) is the short
hand notation of (t1, · · · , tj−1, s, tj, · · · , tn−1). We also adopt the convention t0 = 0, tn+1 = T .
Define
M1k (t) :=
∫ t
0
mk(s)ds =
√2T
(k − 1)πsin
((k − 1)πt
T
)
M2k (t) :=
∫ t
0
M1k (s)ds =
√2T 3
(k − 1)2π2
(1− cos
((k − 1)πt
T
))
and
Fjn(T, t(n)) :=
∂Fn
∂tj(T, t(n)) = e(T−tn)AB · · · e(tj+1−tj)A(BA−AB)e(tj−tj−1)A · · ·Bet1Au0
Fjjn (T, t(n)) :=
∂2Fn
∂t2j(T, t(n)) = e(T−tn)AB · · · e(tj+1−tj)A(A2B+BA2−2ABA)e(tj−tj−1)A · · ·Bet1Au0
The following estimates are right at hand:
|Fjn(T, t(n\j,s))| ≤ 2eλAT λAλn
B|u0|, |Fjjn (T, t(n\j,s))| ≤ 4eλAT λ2
AλnB|u0|, ∀1 ≤ j ≤ n and t(n\j,s)
(3.9)
M1k (0) = M1
k (T ) = M2k (0) = 0 , |M2
k (T )| ≤√
8T 3
(k − 1)2π2,
∫ T
0
(M2k (t))2dt =
3T 4
(k − 1)4π4
(3.10)
Then we perform integration-by-parts twice on the inner integral of (3.8) and obtain
∫ (T,n)
Fn(T, t(n))Eα(t(n))dt(n) :=
∫ (T,n−1)
Gn,k(T, t(n−1))Eα−εk(t(n−1))dt(n−1)
:=
∫ (T,n−1)
(G1n,k + G2
n,k + G3n,k)(T, t(n−1))Eα−εk
(t(n−1))dt(n−1)
where
Gn,k(T, t(n−1)) = G1n,k(T, t(n−1)) + G2
n,k(T, t(n−1)) + G3n,k(T, t(n−1))
and
G1n,k(T, t(n−1)) =
n∑
j=1
(Fn(T, t(n\j,s))M1
k (s)∣∣∣s=tj
s=tj−1
)
15
G2n,k(T, t(n−1)) = −
n∑
j=1
(Fj
n(T, t(n\j,s))M2k (s)
∣∣∣s=tj
s=tj−1
)
G3n,k(T, t(n−1)) =
n∑
j=1
( ∫ tj
tj−1
Fjjn (T, t(n\j,s))M2
k (s)ds)
Since M1k (0) = M1
k (T ) = 0 and Fn(T, t(n\j,s); s = tj) = Fn(T, t(n\(j+1),s); s = tj) for 1 ≤ j ≤n− 1, G1
n,k(T, t(n−1)) = 0. By (3.9) and (3.10), the other two terms are bounded by
|G2n,k(T, t(n−1))| ≤ 2eλAT λAλn
B|u0|(2
n−1∑
j=1
|M2k (tj)|+ |M2
k (T )|)
≤ 4eλAT λAλnB|u0|
( n−1∑
j=1
|M2k (tj)|+
√2T 3
(k − 1)2π2
)
|G3n,k(T, t(n−1))| ≤ 4eλAT λ2
AλnB|u0|
(∫ T
0
|M2k (t)|dt
)≤ 4eλAT λ2
AλnB|u0|
√
T
∫ T
0
(M2k (t))2dt
= 4eλAT λ2Aλn
B|u0|√
3T 5
(k − 1)2π2
Similar to the idea in (3.7),
∑
|α|=n,d(α)=k
α!|uα(T )|2 =∑
|α|=n,d(α)=k
1
α!
∣∣∣∫ (T,n)
Fn(T, t(n))Eα(t(n))dt(n)∣∣∣2
=∑
|α|=n,d(α)=k
1
α!
∣∣∣∫ (T,n−1)
Gn,k(T, t(n−1))Eα−εk(t(n−1))dt(n−1)
∣∣∣2
≤∑
|β|=n−1
1
β!
∣∣∣∫ (T,n−1)
Gn,k(T, t(n−1))Eβ(t(n−1))dt(n−1)∣∣∣2
=
∫ (T,n−1)
|Gn,k(T, t(n−1))|2dt(n−1) ≤∫ (T,n−1) (
|G2n,k(T, t(n−1))|+ |G3
n,k(T, t(n−1))|)2
dt(n−1)
≤16e2λAT λ2Aλ2n
B
∫ (T,n−1) ( n−1∑
j=1
|M2k (tj)|+
√2T 3
(k − 1)2π2+ λA
√3T 5
(k − 1)2π2
)2
dt(n−1)
≤48e2λAT λ2Aλ2n
B
∫ (T,n−1) ((n− 1)
( n−1∑
j=1
(M2k (tj))
2)
+2T 3
(k − 1)4π4+ λ2
A
3T 5
(k − 1)4π4
)dt(n−1)
The remaining part of proof is clear. Since∑n−1
j=1 (M2k (tj))
2 is a symmetric function,
∫ (T,n−1) ( n−1∑
j=1
(M2k (tj))
2)dt(n−1) =
1
(n− 1)!
∫
[0,T ]n−1
( n−1∑
j=1
(M2k (tj))
2)dt(n−1)
=n− 1
(n− 1)!
3T n+2
(k − 1)4π4
16
Therefore
∑
|α|=n,d(α)=k
α!|uα(T )|2 ≤ 48e2λAT λ2Aλ2n
B
(n− 1)!(k − 1)4π4
(3(n− 1)2T n+2 + 2T n+2 + 3λ2
AT n+4)
Summing over n and k yields:
∞∑
k=K+1
N∑
n=1
∑
|α|=n,d(α)=k
α!|uα(T )|2
≤e2λAT 48λ2A
π4
( ∞∑
k=K+1
1
(k − 1)4
)( N∑
n=1
λ2nB (3(n− 1)2T n+2 + 2T n+2 + 3λ2
AT n+4)
(n− 1)!
)|u0|2
≤e(2λA+λ2B
)T 16λ2Aλ2
BT 3
π4(K − 12)3
(5 + 3λ2AT 2 + 6λ4
BT 2)|u0|2
where we use the inequalities
∞∑
k=K+1
1
(k − 1)4≤
∞∑
k=K
∫ k+ 1
2
k− 1
2
1
x4dx =
∫ ∞
K− 1
2
1
x4dx =
1
3(K − 12)3
N∑
n=1
xn−1
(n− 1)!≤
∞∑
n=0
xn
n!= ex
N∑
n=1
(n− 1)2xn−1
(n− 1)!= x +
N−3∑
n=0
(n + 2)xn+2
(n + 1)!≤ x +
∞∑
n=0
2xn+2
n!≤ ex(1 + 2x2)
We have finished the proofs of (3.5) and (3.6). Then (3.3) immediately follows.
Remark 3.2. The proof of (3.5) is independent of the choice of mk∞k=1, the convergence
rate with respect to N is always exponential. The proof of (3.6) relies on the trigonometric
basis assumption. The crucial property is (3.10), which enables cubic convergence. In fact,
the proof will work for any orthonormal basis such that
M1k (T ) = 0, M2
k (T ) = O(k−2), ‖M2k‖H = O(k−2), ∀k ≥ 2
For example, consider the scaled Legendre basis
mk(t) =
√2k − 1
TLk−1
(2t
T− 1
)(3.11)
We are able to show that
M1k (t) =
1
2
√T
2k − 1
(Lk
(2t
T− 1
)− Lk−2
(2t
T− 1
))
17
M2k (t) =
1
4(2k + 1)
√T 3
2k − 1
(Lk+1
(2t
T− 1
)− Lk−1
(2t
T− 1
))
− 1
4(2k − 3)
√T 3
2k − 1
(Lk−1
(2t
T− 1
)− Lk−3
(2t
T− 1
))
where Lk is taken to be 0 for negative k. Then M1k (T ) = 0 for any k ≥ 2 and M2
k (T ) = 0 for
any k ≥ 3. We also have ‖M2k‖H = O(k−2). Therefore for Legendre basis, the convergence
rate with respect to K is still cubic.
Remark 3.3. If A and B commute such that AB = BA, then
Fn(T, t(n)) = e(T−tn)Ae(tn−tn−1)A · · · et1ABnu0 = eTABnu0
is a constant vector that does not depend on t(n). Consequently, for any |α| = n,
uα(T ) =eTABnu0
α!
∫ (T,n)
Eα(t(n))dt(n) =eTABnu0
n!α!
∫
[0,T ]nEα(t(n))dt(n)
=eTABnu0
α!
∫
[0,T ]nmi1α
(t1) · · ·minα(tn)dt(n) =
eTABnu0
α!M1
i1α(T ) · · ·M1
inα(T )
For trigonometric basis (and Legendre basis), M1k (T ) = 0 for any k ≥ 2. That is, uα(T ) = 0
whenever d(α) = inα ≥ 2. It is enough to fix K = 1 and only consider the truncation on N .
The resulting error estimate is simply
E[|uN,1(T )− u(T )|2] ≤ e(2λA+λ2B
)T (λ2BT )N+1
(N + 1)!|u0|2 (3.12)
4 Numerical experiments
We first introduce some post-processing techniques for the numerical solution written as
polynomial chaos expansion:
uN,K(t, x) =∑
α∈JN,K
uα(t, x)Φα
Moments can be computed directly, the first and second moments are
E[uN,K(t, x)] = uε0(t, x), E[u2
N,K(t, x)] =∑
α∈JN,K
α!(uα(t, x))2 (4.1)
For linear problems, the first two moments remain the same for all kinds of randomness since
the propagator system does not change. The third and fourth moments are given by
E[u3N,K(t, x)] =
∑
α∈JN,K
α!uα(t, x)( ∑
β∈JN,K
∑
p∈JN,K
B(α, β, p)uβ(t, x)up(t, x))
(4.2)
18
E[u4N,K(t, x)] =
∑
α∈J2N,K
α!(( ∑
β∈JN,K
∑
p∈JN,K
B(α, β, p)uβ(t, x)up(t, x))2
(4.3)
In the computation of fourth moment, we use the fact that the expansion order of u2N,K(t, x)
is at most 2N . Due to the emergence of interaction coefficients B(α, β, p), higher moments
always depend on the type of randomness. Other statistics can be computed via random
sampling. We simply generate L i.i.d. copies of (ξ1, · · · , ξK), denoted by ξ(l)1 , · · · , ξ(l)
K for
each 1 ≤ l ≤ L. The sample points of uN,K(t, x) are:
u(l)N,K(t, x) :=
∑
J∈JN,K
uα(t, x)Φα(ξ(l)1 , · · · , ξ(l)
K )
Then for any function f , the expectation E[f(uN,K(t, x))] is approximated by sample mean.
We can also plot the normalized histogram of these sample points to visualize empirical
distribution.
In this section, we always assume that ξk∞k=1 are i.i.d. random variables. To be more
specific, we will test three types of randomness: Gaussian noise with Hermite chaos (Ex-
ample 2.1.1), uniform noise with Legendre chaos (Example 2.1.2), and Beta(12, 1
2) noise with
Chebyshev chaos. In the last situation, ξk∞k=1 are supported on [−√
2,√
2] with probability
density function
ρ(ξ1) =
√2− ξ2
1
2π, ξ1 ∈ [−
√2,√
2]
The univariate chaos basis functions are rescaled Chebyshev polynomials Tn(x)∞n=0:
ϕ0(ξ1) = 1, ϕn(ξ1) =√
2n!Tn
( ξ1√2
), n ≥ 1 (4.4)
Now we proceed to solve distribution-free linear and nonlinear SPDEs numerically. Prop-
agator systems are integrated in time with fourth order Runge-Kutta method. Time step is
small enough so that error from temporal discretization is negligible. mk(t)∞k=1 is taken to
be the trigonometric basis (3.2). The results of the scaled Legendre basis (3.11) are almost
indistinguishable from trigonometric basis and will not be reported.
For the purpose of validation and comparison, we also adopt several techniques to com-
pute reference solutions.
• Moment equations: the ODE of first few moments, available to linear Ito type SPDEs.
• Fokker-Planck equation: the PDE of probability density function, available to low-
dimensional Ito type SODEs.
• Monte Carlo simulation: the most commonly used and least restrictive approach, avail-
able to additive noise and/or Ito type SPDEs. Main drawbacks are high computational
cost and low accuracy.
19
The first two methods are free from sampling error, and will be selected whenever possible.
Monte Carlo simulation serves as a backup option.
Example 4.3 (Linear SODE). Suppose that u = u(t) ∈ D(H) satisfies the following linear
SODEu′(t) = u(t) + 1 + u(t) ⋄ N(t), t ∈ [0, T ]
u(0) = 1(4.5)
For Gaussian noise, it is equivalent to the Ito type SODE
du(t) = (u(t) + 1)dt + u(t)dW (t), t ∈ [0, T ] (4.6)
By Ito’s formula, we are able to derive its moment equations
dE[u(t)]
dt= E[u(t)] + 1
dE[u2(t)]
dt= 3E[u2(t)] + 2E[u(t)]
dE[u3(t)]
dt= 6E[u3(t)] + 3E[u2(t)]
dE[u4(t)]
dt= 10E[u4(t)] + 6E[u3(t)]
(4.7)
The analytical solution to (4.7) is
E[u(t)] = 2et − 1
E[u2(t)] =7
3e3t − 2et +
2
3
E[u3(t)] =37
15e6t − 7
3e3t +
6
5et − 1
3
E[u4(t)] =38
15e10t − 37
15e6t +
4
3e3t − 8
15et +
2
15
(4.8)
For uniform and Beta noise, the first and second moments are the same, but we do not have
explicit formulas for the third and fourth moments.
We evolve the propagator ODE system up to end time T = 1 with time step δt = 10−4.
In order to examine the convergence rates of mean square truncation error, ideally we should
compute E[|uN,K − u∞,K|2] to single out the error induced by N , and E[|uN,K − uN,∞|2] to
single out the error induced by K. In practice we use u20,K to approximate u∞,K, and uN,50
to approximate uN,∞. Figure 4.1 contains the semi-log plot of E[|uN,K(1)−u20,K(1)|2] versus
N with K = 1, and the log-log plot of E[|uN,K(1) − uN,50(1)|2] versus K with N = 1, 2, 3.
The numerical rate of convergence with respect to N is evidently exponential. The plot with
respect to K has a zigzag shape (especially for N = 1), but the average slope is close to 3.
20
1 2 3 4 5 6 7 8 9 10
N
10−7
10−6
10−5
10−4
10−3
10−2
10−1
100
101
[|u
N,1(1)−u20,1(1)|2
]
(a) Convergence with respect to N
1 2 3 4 5 6 7 8 9 10 11 12
K
10−5
10−4
10−3
10−2
10−1
100
[|u
N,K
(1)−uN,5
0(1)|2
]
1 : 3
N=1
N=2
N=3
(b) Convergence with respect to K
Figure 4.1: Example 4.3: Plots of mean square truncation error with respect to N and K.Left panel shows the semi-log plot of E[|uN,1(1) − u20,1(1)|2] versus N for N = 1, · · · , 10.Right panel shows the log-log plot of E[|uN,K(1) − uN,50(1)|2] versus K for N = 1, 2, 3 andK = 1, · · · , 12.
Table 4.1: Example 4.3: Values and numerical convergence orders of eN,K :=E[|uN,K(1) − uN,50(1)|2] for N = 1, 2, 3 and K = 2, 4, · · · , 12. The orders are given bylog(
eN,K
eN,K+2)/ log(K+2
K).
KN = 1 N = 2 N = 3
error order error order error order2 7.997e-3 - 2.080e-2 - 2.677e-2 -4 9.659e-4 3.049 2.201e-3 3.241 2.831e-3 3.2416 2.816e-4 3.040 6.102e-4 3.164 7.799e-4 3.1808 1.173e-4 3.044 2.480e-4 3.129 3.158e-4 3.14310 5.939e-5 3.050 1.238e-4 3.114 1.572e-4 3.12612 3.398e-5 3.062 7.018e-5 3.113 8.895e-5 3.123
The cubic convergence rate is more clearly seen in Table 4.1, where we only compare even
values of K to average out the zig-zag profile.
Next we compute moments of the truncated solution. Table 4.2 lists the first four central
moments of uN,K(1) with all three types of randomness, by taking N = K = 4, N = K = 6
and N = K = 8. For Gaussian noise, moments of u(1) are also included according to
(4.8), while for uniform noise and Beta(12, 1
2) noise, third and fourth moments of u(1) are not
available. Two conclusions can be drawn from the table. Comparing the central moments
of uN,K(1) and u(1), we see that the variance can be approximated well with relatively few
chaos expansion terms, but more terms are needed to resolve higher moments. Comparing
among types of driving noise, we notice the large discrepancy in third and fourth moments,
21
despite the fact that they share the same mean and variance. It is probably related to
the kurtosis of different distributions. The fourth moment of ξ1 is 3 for standard Gaussian
distribution, 95
for uniform distribution, and 32
for Beta(12, 1
2) distribution. Higher kurtosis
in ξk∞k=1 leads to higher kurtosis in uN,K .
Table 4.2: Example 4.3: Comparison of central moments of uN,K(1) and u(1). We takeN = K = 4, N = K = 6 and N = K = 8. Higher moments of u(1) are only available toGaussian noise.
Type of noise Type of moment u(1) u4,4(1) u6,6(1) u8,8(1)Gaussian Variance 22.413 22.313 22.410 22.413
Third central moment 565.548 487.838 558.223 565.138Fourth central moment 41759.97 22914.36 37479.48 41233.50
Uniform Variance 22.413 22.313 22.410 22.413Third central moment - 208.415 220.604 221.203Fourth central moment - 3080.52 3446.20 3474.01
Beta Variance 22.413 22.313 22.410 22.413Third central moment - 150.739 159.566 160.011Fourth central moment - 1789.49 1957.34 1970.63
Empirical distribution is a more intuitive way to describe random variables. Figure 4.2
demonstrates the empirical probability densities of u4,4(1), u6,6(1) and u8,8(1) for all three
types of randomness. All densities are estimated by normalized histograms with 103 bins
out of 107 i.i.d samples of uN,K(1). For Gaussian noise, we can also find out the probability
density function of u(t), denoted by ρ(u, t). The governing PDE of ρ(t, u), i.e. Fokker-Planck
equation, follows from the Ito form (4.6):
∂tρ(t, u) = −∂u((u + 1)ρ) + ∂2u
(u2
2ρ), (t, u) ∈ (0, 1]× (0,∞)
ρ(0, u) = δ(u− 1), u ∈ (0,∞)(4.9)
where δ is the Dirac delta function. Substituting v = log u, we simplify (4.9) and get
∂tρ(t, v) =(1
2− e−v
)∂vρ +
1
2∂2
vρ, (t, v) ∈ (0, 1]× R
ρ(0, v) = δ(v), v ∈ R
(4.10)
(4.10) is a standard convection-diffusion equation. We choose the local discontinuous Galerkin
(LDG) method [4] as the numerical solver. The computational domain is [−7, 7] with zero
boundary condition, and divided into 501 quadratic elements. The numerical solution of
Fokker-Planck equation at t = 1 is also displayed in Figure 4.2. The empirical densities of
u4,4(1) and u6,6(1) slightly deviates from the Fokker-Planck solution, and the empirical den-
sity of u8,8(1) agrees with the Fokker-Planck solution very well. As for the comparison among
22
three types of noise, their density patterns are qualitatively different. The distributions with
Gaussian noise spread out and have long tails. The density profiles with Beta(12, 1
2) noise
are mostly constrained in a narrow region, and the density profiles with uniform noise lie
somewhere in between. This also explains the difference of higher moments in Table 4.2.
0 5 10 15 20
u
0
0. 05
0. 1
0. 15
0. 2
0. 25
Density
N=4, K=4
N=6, K=6
N=8, K=8
Fokker-Planck
(a) Gaussian noise
0 5 10 15 20
u
0
0. 1
0. 2
0. 3
0. 4
0. 5
Density
N=4, K=4
N=6, K=6
N=8, K=8
(b) Uniform noise
0 5 10 15 20
u
0
0. 2
0. 4
0. 6
0. 8
1
1. 2
Density
N=4, K=4
N=6, K=6
N=8, K=8
(c) Beta(1
2, 1
2) noise
Figure 4.2: Example 4.3: Normalized histograms of uK,N(1) out of 107 i.i.d samples. Wetake N = K = 4, N = K = 6 and N = K = 8. For Gaussian distribution, the black dashedline represents the numerical solution of Fokker-Planck equation (4.10). Values larger than20 are discarded. Number of bins is 103.
In summary, this example studies a very simple linear SODE so that reference solutions
of moments and density function can be acquired without much effort. By linearity, we only
need to solve the propagator once and save the expansion coefficients for post-processing.
The mean square truncation error results in Figure 4.1 and Table 4.1 indicate exponential
convergence with respect to N and cubic convergence with respect to K, as predicted by
23
Theorem 3.1. Table 4.2 and Figure 4.2 highlight the contrast of higher moments and empir-
ical distributions with different noises. We also observe that although the mean square error
converges rapidly, high order chaos expansion terms are beneficial to the approximation of
higher moments and distribution.
Example 4.4 (Linear parabolic PDE). We solve the linear parabolic PDE in Example 2.3.1.
Consider the one-dimensional space region D = [0, 2π] with periodic boundary condition.
The initial data is u0(x) = cos x. Differential operators L andM are set to be [35, 36]
Lu = 0.145∂2xu + 0.1 sin x∂xu, Mu = 0.5∂xu
Fourier collocation method with M = 32 collocation points is used for spatial discretization.
Let xiMi=1 be the set of equidistant collocation points such that xi = 2π(i−1)M
. Then u(t, ·) is
identified by the vector evaluated at xiMi=1:
u(t) =[u(1)(t) . . . u(M)(t)
]T:=
[u(t, x1) · · · u(t, xM)
]T
After spatial discretization, L andM turn into difference matrices A and B. We only need
to work on the linear SODE system (3.1).
For Gaussian noise, it is possible to obtain moment equations of (3.1). Direct application
of Ito’s formula yields
dE[u(i)u(j)]
dt=
M∑
l=1
(AilE[u(j)u(l)] + AjlE[u(i)u(l)]) +
M∑
l=1
M∑
r=1
BilBjrE[u(l)u(r)], 1 ≤ i, j ≤ M
(4.11)
It is a M2-dimensional ODE system describing the evolution of covariance matrix. The
covariance matrix is not influenced by the type of driving noise, as a result of linearity.
Higher moment equations are written in a similar fashion, but they are computationally
formidable in that for d-th moment, we have to tackle the full tensor product system with
Md dimensions.
The propagator system is run up to T = 5 with Runge-Kutta time step δt = 10−3. Figure
4.3 depicts the mean square truncation error with respect to N and K. Same as the previous
example, we compute E[‖uN,K(5, ·)− u20,K(5, ·)‖2l2] as the proxy of error induced by N , and
E[‖uN,K(5, ·)−uN,50(5, ·)‖2l2] as the proxy of error induced by K, where the discrete L2 norm
for v ∈ H is defined by
‖v‖2l2 :=2π
M
M∑
i=1
(v(i))2 =2π
M
M∑
i=1
(v(xi))2
Once again, the N -version convergence shows exponential rate, and the K-version conver-
gence shows cubic rate.
24
1 2 3 4 5 6 7 8 9 10
N
10−5
10−4
10−3
10−2
10−1
100
[‖(
uN,1−u20
,1)(5,·)‖
2 l2]
(a) Convergence with respect to N
1 2 3 4 5 6 7 8 9 10 11 12
K
10−7
10−6
10−5
10−4
10−3
10−2
[‖(
uN,K
−uN,5
0)(5,·)‖
2 l2]
1 : 3
N=1
N=2
N=3
(b) Convergence with respect to K
Figure 4.3: Example 4.4: Plots of mean square truncation error in discrete l2 norm withrespect to N and K. Left panel shows the semi-log plot of E[‖uN,1(5, ·)−u20,1(5, ·)‖2l2] versusN for N = 1, · · · , 10. Right panel shows the log-log plot of E[‖uN,K(5, ·) − uN,50(5, ·)‖2l2]versus K for N = 1, 2, 3 and K = 1, · · · , 12.
The first four central moments of uN,K(5, ·) with all three noises are plotted in Figure
4.4. We consider N = 4, 8 and K = 4. Variance of u(5, ·) is also exhibited as reference by
solving the moment equation (4.11) through fourth order Runge-Kutta method with time
step δt = 10−3. For Gaussian noise, higher moments can be approximated by Monte Carlo
sampling. We use the second order weak scheme (see Chapter 2 of [24]) to discretize the
Ito integral and generate sample paths. Let up be the sample of u at p-th time step. The
update rule is
up+1 = up + δtAup +δt2
2A2up +
√δtζpBup +
δt
2(ζ2
p − 1)B2up +
√δt3
2ζp(AB + BA)up
where ζp are i.i.d. standard Gaussian random variables. Third and fourth central moments
of the Monte Carlo solution with 106 samples and time step δt = 10−3 are also shown in
Figure 4.4. We observe that both u4,4 and u8,4 predict the variance sufficiently well, but
only u8,4 succeeds in resolving higher moments. As for the comparison among three types
of noise, their third central moments have different structures. Fourth central moments look
similar in shape but different in magnitude.
Empirical distributions at the second collocation point x2 are illustrated in Figure 4.5.
For Gaussian noise, we plot normalized histograms of u4,4(5, x2) and u8,4(5, x2) out of 107
samples in the left panel, as well as the histogram of 106 Monte Carlo samples for reference.
Number of bins is 103. Here solving Fokker-Planck equation is not feasible as it depends on
M spatial variables. We underline that the distribution of u(5, x2) is supported in [−1, 1], as
25
0 π/2 π 3π/2 2π
x
0
0. 1
0. 2
0. 3
0. 4
Variance
N=4, K=4
N=8, K=4
Moment equation
(a) Variance
0 π/2 π 3π/2 2π
x
−0. 2
−0. 1
0
0. 1
0. 2
Third central moment
Gaussian: N=4, K=4
Gaussian: N=8, K=4
Gaussian: Monte Carlo
Uniform: N=4, K=4
Uniform: N=8, K=4
Beta: N=4, K=4
Beta: N=8, K=4
(b) Third central moment
0 π/2 π 3π/2 2π
x
0
0. 1
0. 2
0. 3
0. 4
Fourth central moment
Gaussian: N=4, K=4
Gaussian: N=8, K=4
Gaussian: Monte Carlo
Uniform: N=4, K=4
Uniform: N=8, K=4
Beta: N=4, K=4
Beta: N=8, K=4
(c) Fourth central moment
Figure 4.4: Example 4.4: Central moments of uK,N(5, ·). We take N = 4, 8 and K = 4. Blackdashed lines are reference solutions. Variance is computed via moment equation (4.11) (forall types of noise), and higher moments are approximated by Monte Carlo method with 106
samples (only for Gaussian noise).
a result of averaging over characteristic lines [23]. The empirical distribution of Monte Carlo
sampling is indeed inside [−1, 1] and highly rightly skewed. Almost all samples of u4,4(5, x2)
and u8,4(5, x2) fall into [−1, 1] as well. We discard the few outlier samples in the figure.
The empirical density function of u4,4 underestimates the position of right peak, and the
empirical density function u8,4 is almost on top of the reference solution. For uniform noise
and Beta(12, 1
2) noise, we plot the empirical density functions of u10,4(5, x2) and u12,4(5, x2)
in the right panel. The profiles of u10,4 and u12,4 nearly coincide with each other, which
suggests that we achieve reasonable approximations of the true density functions. These
density patterns look dramatically different from the Gaussian noise case, in that they are
26
neither supported in [−1, 1] nor rightly skewed. Such distinction is consistent with Figure
4.4, where the skewness (third central moment) at x2 is negative for Gaussian noise, and
positive for other two noises.
−1 −0. 5 0 0. 5 1
u
0
1
2
3
4
5
6
7
8
Density
N=4, K=4
N=8, K=4
Monte Carlo
(a) Gaussian noise
−0. 5 0 0. 5 1 1. 5
u
0
0. 5
1
1. 5
2
2. 5
Density
Uniform: N=10, K=4
Uniform: N=12, K=4
Beta: N=10, K=4
Beta: N=12, K=4
(b) Uniform and Beta(1
2, 1
2) noise
Figure 4.5: Example 4.4: Normalized histograms of uN,K(5, x2) out of 107 i.i.d samples. Inthe left panel, we take N = 4, 8 and K = 4. Values outside [−1, 1] are discarded, and theblack dashed line represents the normalized histogram of Monte Carlo simulation with 106
samples. In the right panel, we take N = 10, 12 and K = 4. Number of bins is 103.
In this example, we analyze a one-dimensional linear parabolic SPDE with relatively long
evolution time. Here solving moment equations is only practical for the second moment. We
resort to Monte Carlo simulation to produce other reference solutions. Our observations
are roughly parallel to the previous example. The truncation error of the second moment
converges at rates predicted by Theorem 3.1. Higher moments and empirical distributions
are more difficult to characterize, and highly depend on the type of underlying randomness.
Example 4.5 (Passive scalar equation). We move on to two-dimensional linear transport
type SPDE. Consider the following distribution-free passive scalar equation, driven by two
independent noises N1(t) and N2(t), and equipped with periodic boundary condition.
∂tu(t, x, y) =1
2(M2
1 +M22)u +M1u ⋄ N1(t) +M2u ⋄ N2(t), (t, x, y) ∈ [0, T ]× [0, 2π]2
u(0, x, y) = u0(x, y) = sin(2x) sin(y), (x, y) ∈ [0, 2π]2
(4.12)
where
M1u = cos(x + y)(∂xu− ∂yu), M2u = sin(x + y)(∂xu− ∂yu)
N1(t) =
∞∑
k=1
mk(t)ξ1k, N2(t) =
∞∑
k=1
mk(t)ξ2k
27
and ξdk : d = 1, 2, k ≥ 1 is an i.i.d. sequence of random variables. We introduce the
Cartesian product of multi-index sets:
J 2 := α = (α1, α2) : α1, α2 ∈ J , J 2N,K := α = (α1, α2) : α1, α2 ∈ JN,K
The polynomial chaos basis functions are
Φα :=∞∏
k=1
ϕα1k(ξ1
k)∞∏
k=1
ϕα2k(ξ2
k)
Under the extended nomenclature, the propagator system has the form
∂tuα(t, x, y) =1
2(M2
1 +M22)uα +
∑
ε1k≤α
M1uα−ε1kmk(t) +
∑
ε2k≤α
M2uα−ε2kmk(t), (t, x, y) ∈ [0, T ]× [0, 2π]2
uα(0, x, y) = sin(2x) sin(y)1α=ε0, (x, y) ∈ [0, 2π]2
(4.13)
where ε1k = (εk, ε0) and ε2
k = (ε0, εk). The truncated solution is still defined as
uN,K(t, x, y) :=∑
α∈J 2N,K
uα(t, x, y)Φα (4.14)
We employ Fourier collocation method with M = 64 collocation points in each dimension for
the spatial discretization of (4.13). Equidistant collocation points are denoted by xi = yi =2π(i−1)
M. The propagator system is then computed up to T = 0.2 with time step δt = 5×10−4.
For Gaussian noise, (4.12) reduces to the passive scalar equation in Stratonovich version
[13, 34]
du(t, x, y) =M1u dW1(t) +M2u dW2(t), (t, x, y) ∈ [0, T ]× [0, 2π]2 (4.15)
We are able to work out the analytical solution based on tracing back characteristic lines
[23]. u(T, x, y) has the following representation
u(T, x, y) = u0(Xx,y(0), Yx,y(0)) (4.16)
where Xx,y(t) and Yx,y(t) satisfy the system of backward (characteristic) SODEs
dXx,y(t) = cos(Xx,y + Yx,y)←−−dW1(t) + sin(Xx,y + Yx,y)
←−−dW2(t), t ∈ [0, T ]
dYx,y(t) = − cos(Xx,y + Yx,y)←−−dW1(t)− sin(Xx,y + Yx,y)
←−−dW2(t), t ∈ [0, T ]
Xx,y(T ) = x, Yx,y(T ) = y
(4.17)
The definition of backward Ito integral←−−dW (t) can also be found in [23]. Summing the two
equations, we realize that Xx,y(t) + Yx,y(t) is constant over time. Therefore
Xx,y(0) = x− cos(x+ y)η1− sin(x+ y)η2, Yx,y(0) = y +cos(x+ y)η1 +sin(x+ y)η2 (4.18)
28
where η1 = W1(T ) and η2 = W2(T ) are independent Gaussian variables. Then the analytical
solution is
u(T, x, y) = sin(2(x−cos(x+y)η1−sin(x+y)η2)) cos(y+cos(x+y)η1 +sin(x+y)η2) (4.19)
We note that the distribution of u(T, x, y) is again supported in [−1, 1]. Since η1 =√
Tξ11
and η2 =√
Tξ21 , the solution just depends on ξ1
1 and ξ21 . In terms of truncation, we can
only take K = 1 and adjust the value of N . Such simplification is actually a direct result of
Remark 3.3. It is easy to show thatM1 andM2 commute with each other, so that they also
commute with 12(M2
1 +M22). Commutativity is the reason why K = 1 is enough. Monte
Carlo sampling of (4.19) is trivial. Moments of u(T, x, y) can be computed very accurately
through Gauss-Hermite quadrature rule. We pick 50 quadrature points in each dimension
to establish reference solutions.
The N -version convergence of mean square truncation error is displayed in Figure 4.6.
We plot values of E[‖uN,1(0.2, ·, ·)−u(0.2, ·, ·)‖2l2] in logarithm scale for N = 1, · · · , 10, where
the discrete L2 norm is
‖v‖2l2 :=4π2
M2
M∑
i=1
M∑
j=1
(v(xi, yj))2
The convergence rate is clearly exponential.
1 2 3 4 5 6 7 8 9 10
N
10−5
10−4
10−3
10−2
10−1
100
101
[‖(
uN,1−
u)(0.
2,·,
·)‖2 l2]
Figure 4.6: Example 4.5: Semi-log plot of mean square truncation error E[‖uN,1(0.2, ·, ·)−u(0.2, ·, ·)‖2
l2] for N = 1, · · · , 10.
Next we fix N = 8 and pay attention to higher moments. Figure 4.7 presents contour
plots for the third and fourth central moments of u8,1(0, 2, ·, ·). For Gaussian noise, we also
provide third and fourth central moments of u(0.2, ·, ·) using Gauss-Hermite quadrature. The
agreement between the truncated solution and the reference solution is quite satisfactory. For
29
uniform noise and Beta(12, 1
2) noise, the corresponding contour plots have different patterns,
especially for the third central moment.
We can also detect the impact of driving noise by checking empirical distributions. In
Figure 4.8 we demonstrate normalized histograms of uN,1 at the collocation point (x6, y6)
out of 107 samples. For Gaussian noise, we choose N = 4, 8, together with the reference
distribution generated by 107 samples of (4.19). Samples outside [−1, 1] are discarded.
Similar to Figure 4.5, u8,1 outperforms u4,1 in approximating the highly rightly skewed true
distribution. For uniform noise and Beta(12, 1
2) noise, we consider N = 8, 10. The empirical
density functions of u8,1 and u10,1 are mostly overlapping, so that they can be thought as
credible approximations. Once again we notice the fact that different driving noises lead to
strikingly different empirical density profiles.
Example 4.6 (Stochastic Burgers equation). We consider the stochastic Burgers equation
in Example 2.3.2. The space region is D = [0, 1] with periodic boundary condition. The
parameters are µ = 0.005 and σ(x) = 12cos(4πx), and the initial data is u0(x) = 1
2(e(2πx) −
1.5) sin(2π(x + 0.37)). We apply the Fourier collocation method with M = 128 collocation
points for spatial discretization. The end time is set to be T = 0.8 with Runge-Kutta time
step δt = 10−3.
For such additive noise, Monte Carlo simulation is well suited for any type of distribution.
We generate sample paths of N(t) by truncating the infinite sum up to K = 50. For a fixed
sample path, we solve the resulting deterministic Burgers equation using Fourier collocation
method and fourth order Runge-Kutta time stepping, with M = 128 collocation points and
time step δt = 10−3. Moments and empirical distributions of Monte Carlo samples will be
chosen as reference solutions. We take 106 samples.
The convergence of mean square truncation error is given in Figure 4.9. We again plot
E[‖uN,K(0.8, ·) − u20,K(0.8, ·)‖2l2] to represent N -version convergence, and E[‖uN,K(0.8, ·) −
uN,50(0.8, ·)‖2l2] to represent K-version convergence. Due to nonlinearity, these truncation
errors rely on the underlying randomness. For the N -version convergence, we only plot results
with uniform and Beta(12, 1
2) noise as the interaction coefficients of Hermite polynomials grow
exponentially with respect to N , causing the numerical computation to blow up for large N .
For the K-version convergence, the plots are nearly identical for three noises, so that we only
present the plot with Gaussian noise. We emphasize that the numerical convergence rate is
still exponential with respect to N and cubic with respect to K, even though Theorem 3.1
is only proved for the linear case.
Then we fix K = 8. Third and fourth central moments of u2,8(0.8, ·) and u5,8(0.8, ·)are drawn in Figure 4.10. As the profiles with different noises are close to each other, we
only show the zoomed-in view between the 21-st collocation point and the 50-th collocation
30
0 π/2 π 3π/2 2π0
π/2
π
3π/2
2π
−0.072
−0.048
−0.024
0.000
0.024
0.048
0.072
(a) Third central moment: Gaussian noise(reference solution)
0 π/2 π 3π/2 2π0
π/2
π
3π/2
2π
0.004
0.020
0.036
0.052
0.068
0.084
0.100
0.116
(b) Fourth central moment: Gaussian noise(reference solution)
0 π/2 π 3π/2 2π0
π/2
π
3π/2
2π
−0.072
−0.048
−0.024
0.000
0.024
0.048
0.072
(c) Third central moment: Gaussian noise0 π/2 π 3π/2 2π
0
π/2
π
3π/2
2π
0.008
0.024
0.040
0.056
0.072
0.088
0.104
0.120
(d) Fourth central moment: Gaussian noise
0 π/2 π 3π/2 2π0
π/2
π
3π/2
2π
−0.036
−0.024
−0.012
0.000
0.012
0.024
0.036
(e) Third central moment: uniform noise0 π/2 π 3π/2 2π
0
π/2
π
3π/2
2π
0.003
0.015
0.027
0.039
0.051
0.063
0.075
0.087
(f) Fourth central moment: uniform noise
0 π/2 π 3π/2 2π0
π/2
π
3π/2
2π
−0.0300
−0.0225
−0.0150
−0.0075
0.0000
0.0075
0.0150
0.0225
0.0300
(g) Third central moment: Beta(1
2, 1
2) noise
0 π/2 π 3π/2 2π0
π/2
π
3π/2
2π
0.004
0.016
0.028
0.040
0.052
0.064
0.076
0.088
0.100
(h) Fourth central moment: Beta(1
2, 1
2) noise
Figure 4.7: Example 4.5: Third and fourth central moments of u8,1(0.2, ·, ·). First two plotsare reference solutions with Gaussian noise. 30 equally spaced contour levels are used for allplots.
31
−1 −0. 5 0 0. 5 1
u
0
5
10
15
20
25
Density
N=4
N=8
Monte Carlo
(a) Gaussian noise
−0. 5 0 0. 5 1
u
0
1
2
3
4
Density
Uniform: N=8
Uniform: N=10
Beta: N=8
Beta: N=10
(b) Uniform and Beta(1
2, 1
2) noise
Figure 4.8: Example 4.5: Normalized histograms of uN,1(0.2, x6, y6) out of 107 i.i.d samples.In the left panel, we take N = 4, 8. Values outside [−1, 1] are discarded, and the blackdashed line represents the normalized histogram of 107 Monte Carlo samples of (4.19). Inthe right panel, we take N = 8, 10. Number of bins is 103.
1 2 3 4 5 6 7 8 9 10
N
10−11
10−10
10−9
10−8
10−7
10−6
10−5
10−4
10−3
[‖(
uN,1−
u20
,1)(0.
8,·)‖
2 l2]
Uniform noise
Beta(12, 12) noise
(a) Convergence with respect to N
1 2 3 4 5 6 7 8 9 10 11 12
K
10-6
10-5
10-4
10-3
10-2
10-1
[‖(
uN,K
−uN,5
0)(0.
8,·)‖
2 l2]
1 : 3
Gaussian noise: N=1
Gaussian noise: N=2
(b) Convergence with respect to K
Figure 4.9: Example 4.6: Plots of mean square truncation error in discrete l2 norm withrespect to N and K. Left panel shows the semi-log plot of E[‖uN,1(0.8, ·) − u20,1(0.8, ·)‖2l2]versus N with uniform and Beta(1
2, 1
2) noise for N = 1, · · · , 10. Right panel shows the
log-log plot of E[‖uN,K(5, ·)− uN,50(5, ·)‖2l2] versus K with Gaussian noise for N = 1, 2 andK = 1, · · · , 12.
point. Central moments of Monte Carlo solution are also plotted for comparison. We note
that for all noises, u2,8 results in inaccurate approximations, and u5,8 leads to much better
performance. The plots of empirical density functions of u2,8 and u5,8 out of 107 samples at
x30 are provided in Figure 4.11. From the figure we can also see how u5,8 is superior to u2,8
32
in agreeing with the reference distributions.
0. 2 0. 25 0. 3 0. 35
x
−0. 01
0
0. 01
Third central moment
Gaussian: N=2, K=8
Gaussian: N=5, K=8
Gaussian: Monte Carlo
Uniform: N=2, K=8
Uniform: N=5, K=8
Uniform: Monte Carlo
Beta: N=2, K=8
Beta: N=5, K=8
Beta: Monte Carlo
(a) Third central moment
0. 2 0. 25 0. 3 0. 35
x
0
0. 01
0. 02
Fourth central moment
Gaussian: N=2, K=8
Gaussian: N=5, K=8
Gaussian: Monte Carlo
Uniform: N=2, K=8
Uniform: N=5, K=8
Uniform: Monte Carlo
Beta: N=2, K=8
Beta: N=5, K=8
Beta: Monte Carlo
(b) Fourth central moment
Figure 4.10: Example 4.6: Third and fourth central moments of uN,K(0.8, ·) between x21 andx50. We take N = 2, 5 and K = 8. Dashed lines are reference solutions computed by MonteCarlo simulation with 106 samples.
5 Concluding remarks
In this paper, we explore the numerical solutions of SPDEs by truncating the stochastic
polynomial chaos expansion series under the distribution-free framework. We generalize the
definition of Wick product and Skorokhod integral to arbitrary driving noise. Then Wick-
Skorokhod type SPDEs are not limited to Gaussian randomness or Levy randomness. More
importantly, for linear SPDEs, the propagator system, and even the first two moments or
the solution, are the same for different noises. The computational burden of solving the
propagator system is purely off-line. The only on-line work is post-processing. However,
the propagator system of nonlinear SPDE changes with noise as interaction terms come into
play.
Analysis of the mean square truncation error is carried out for linear problems. We
prove exponential convergence with respect to polynomial order, and cubic convergence with
respect to the number of random variables. The cubic rate arises from repeated integration-
by-parts and special properties of the orthonormal basis mk(t)∞k=1. We need to assume
trigonometric basis or Legendre basis.
We conduct systematic investigation on the numerical results of linear and nonlinear
SPDEs with different driving noises. Numerical rates of convergence are consistent with
our theoretical analysis. Higher moments and density function can also be approximated
33
−1 −0. 5 0 0. 5 1
u
0
1
2Density
N=2, K=8
N=5, K=8
Monte Carlo
(a) Gaussian noise
−1 −0. 5 0 0. 5 1
u
0
0. 5
1
1. 5
Density
N=2, K=8
N=5, K=8
Monte Carlo
(b) Uniform noise
−1 −0. 5 0 0. 5 1
u
0
0. 5
1
1. 5
Density
N=2, K=8
N=5, K=8
Monte Carlo
(c) Beta(1
2, 1
2) noise
Figure 4.11: Example 4.6: Normalized histograms of uN,K(0.8, x30) out of 107 i.i.d samples.We take N = 2, 5 and K = 8. Black dashed line represents the normalized histogram ofMonte Carlo simulation with 106 samples. Number of bins is 103.
effectively with sufficiently many polynomial chaos expansion terms. However, we should also
recognize some drawbacks and unsolved problems, which gives hints on our future research.
(i) To the best of our knowledge, the limiting procedure of distribution-free Skorokohd
integral is unclear. Proposition 2.2 is only for deterministic processes, and Proposition
2.3 is only for Gaussian (and Levy) noise. Then we can only come up with reference
solutions for Ito type SPDEs with Gaussian (and Levy) noise and / or SPDEs with
additive noise. Further work is required for better understanding of the distribution-
free stochastic analysis.
(ii) We do not focus on long time integration in this paper, but the exponential growth of
34
error with respect to time is seen both theoretically and numerically. Proper techniques
should be devised to mitigate the impact of time evolution. We remark that direct
generalization of the multi-stage methodology in [12, 11, 36] is specious as the driving
process N(t) may not have independent increments.
(iii) The propagator system usually consists of PDEs of the same type but with different
data (e.g. different coefficients, different given right hand sides, etc.). It can be ex-
pected that for a large number of expansion terms, the application of reduced basis
method [25] may significantly reduce the computational cost while maintaining desired
accuracy.
Acknowledgement. The authors would like to thank Michael Tretyakov and Zhongqiang
Zhang for helpful discussions on the relation between commutativity and K-version conver-
gence.
A Interaction coefficients B(α, β, p)
We still assume that ξk∞k=1 are i.i.d. random variables. Then the interaction coefficient
B(α, β, p) can be decomposed into
B(α, β, p) =E[ΦαΦβΦp]
α!=
∞∏
k=1
E[ϕαk(ξk)ϕβk
(ξk)ϕpk(ξk)]
αk!:=
∞∏
k=1
b(αk, βk, pk) (A.1)
It suffices to compute b(i, j, l) for any i, j, l ≥ 0. According to orthogonality,
ϕj(ξ)ϕl(ξ) =∞∑
i=0
E[ϕi(ξ)ϕj(ξ)ϕl(ξ)]
i!ϕi(ξ) =
∞∑
i=0
b(i, j, l)ϕi(ξ) (A.2)
Hence b(i, j, l) is the i-th expansion coefficient of ϕj(ξ)ϕl(ξ) in terms of ϕn(ξ)∞n=0. In
particular, for the three types of noises and corresponding orthogonal polynomials considered
in Section 4, there are explicit formulas for these expansion coefficients.
• For Gaussian noise and Hermite chaos. ϕn(ξ) = Hen(ξ). Since
Hej(x)Hel(x) =
minj,l∑
r=0
j!l!
(j − r)!(l − r)!r!Hej+l−2r(x) (A.3)
we have
b(i, j, l) =
j!l!
(j−r)!(l−r)!r!if i = j + l − 2r and r ≤ mini, j
0 otherwise(A.4)
35
• For uniform noise and Legendre chaos, ϕn(ξ) =√
(2n + 1)n!Ln(ξ/√
3). Define
λn :=Γ(n + 1/2)
n!Γ(1/2)=
∏n−1m=0(m + 1/2)
n!
Then the expansion of Lj(x)Ll(x) is
Lj(x)Ll(x) =
minj,l∑
r=0
2(j + l − 2r) + 1
2(j + l − r) + 1
λrλi−rλj−r
λi+j−r
Lj+l−2r(x) (A.5)
Thus
b(i, j, l) =
√(2i+1)(2j+1)(2l+1)
2(j+l−r)+1
√j!l!i!
λrλi−rλj−r
λi+j−rif i = j + l − 2r and r ≤ mini, j
0 otherwise
(A.6)
• For Beta(12, 1
2) noise and Chebyshev chaos, ϕn(ξ) =
√cnn!Tn(ξ/
√2) where c0 = 1 and
cn = 2 for n ≥ 1. Since Chebyshev polynomials are essentially cosine functions,
Tj(x)Tl(x) =1
2Tj+l(x) +
1
2T|j−l|(x) (A.7)
Thus
b(i, j, l) =
1 if i = j, l = 0 or i = l, j = 0
12
√cjcl
ci
√j!l!i!
if j, l > 0 and i = j + l or i = |j − l|0 otherwise
(A.8)
Here the expansion coefficients have a sparse pattern. For fixed j and l, there are at
most two values of i such that b(i, j, l) is nonzero.
In general, we compute b(i, j, l) by matching the monomial coefficients on the both sides
of (A.2) (see e.g., [38]). Suppose that
ϕn(ξ) =n∑
m=0
Pm,nξm
According to (A.2), for i > j + l, b(i, j, l) = 0, and b(i, j, l) : 0 ≤ i ≤ j + l satisfies the
following linear system
j+l∑
i=0
b(i, j, l)Pm,i =
mini,j∑
r=max0,i−l
Pr,jPi−r,l (A.9)
It is very easy to solve (A.9) as Pm,nj+lm,n=0 is a upper triangular matrix. This procedure is
applicable to any set of orthogonal polynomials.
36
References
[1] R. Askey and J. A. Wilson. Some Basic Hypergeometric Orthogonal Polynomials that
Generalize Jacobi polynomials. American Mathematical Society, 1985.
[2] F. E. Benth and J. Gjerde. A remark on the equivalence between Poisson and Gaussian
stochastic partial differential equations. Potential Analysis, 8:179–193, 1998.
[3] R. H. Cameron and W. T. Martin. The orthogonal development of non-linear functionals
in series of Fourier-Hermite functionals. Annals of Mathematics, 385–392, 1947.
[4] B. Cockburn and C.-W. Shu. The local discontinuous Galerkin method for time-
dependent convection-diffusion systems. SIAM Journal on Numerical Analysis, 35:2440–
2463, 1998.
[5] B. J. Debusschere, H. N. Najm, P. P. Pebay, O. M. Knio, R. O. Ghanem and O. P.
Le Maıtre. Numerical challenges in the use of polynomial chaos representations for
stochastic processes. SIAM Journal on Scientific Computing, 26:698–719, 2004.
[6] G. Da Prato, A. Debussche and R. Temam. Stochastic Burgers’ equation. Nonlinear
Differential Equations and Applications NoDEA, 1:389–402, 1994.
[7] G. Di Nunno, B. Øksendal and F. Proske. Malliavin Calculus for Levy Processes with
Applications to Finance. Springer, 2009.
[8] H. Holden, B. Øksendal, J. Ubøe and T. Zhang. Stochastic Partial Differential Equa-
tions. Springer, 1996.
[9] T. Y. Hou, W. Luo, B. Rozovskii and H. Zhou. Wiener chaos expansions and numerical
solutions of randomly forced equations of fluid mechanics. Journal of Computational
Physics, 216:687–706, 2006.
[10] K. Ito. Multiple Wiener integral. Journal of the Mathematical Society of Japan, 3:157–
169, 1951.
[11] H. Liu. On spectral approximations of stochastic partial differential equations driven by
Poisson noise. Ph.D. thesis, University of Southern California, 2007.
[12] S. Lototsky, R. Mikulevicius, and B. Rozovskii. Nonlinear filtering revisited: a spectral
approach. SIAM Journal on Control and Optimization, 35:435–461, 1997.
37
[13] S. Lototsky and B. Rozovskii. Passive scalar equation in a turbulent incompressible
Gaussian velocity field. Russian Mathematical Surveys, 59:297–312, 2004.
[14] S. Lototsky and B. Rozovskii. Stochastic differential equations: a Wiener chaos ap-
proach. In From Stochastic Calculus to Mathematical Finance, Springer, 2006, pp.433–
506.
[15] S. Lototsky and B. Rozovskii. Wiener chaos solutions of linear stochastic evolution
equations. The Annals of Probability, 638–662, 2006.
[16] S. Lototsky and B. Rozovskii. Stochastic Partial Differential Equations. Springer, 2017.
[17] W. Luo. Wiener chaos expansion and numerical solutions of stochastic partial differen-
tial equations. Ph.D. thesis, California Institute of Technology, 2006.
[18] P. Malliavin. Stochastic calculus of variation and hypoelliptic operators. In Proc. Intern.
Symp. SDE Kyoto 1976, Kinokuniya, 1978, pp.195–263.
[19] R. Mikulevicius and B. Rozovskii. Separation of observations and parameters in nonlin-
ear filtering. In Decision and Control, Proceedings of the 32nd IEEE Conference, IEEE,
1993, pp.1564–1569.
[20] R. Mikulevicius and B. Rozovskii. Linear parabolic stochastic PDE and Wiener chaos.
SIAM Journal on Mathematical Analysis, 29:452–480, 1998.
[21] R. Mikulevicius and B. Rozovskii. On unbiased stochastic Navier–Stokes equations.
Probability Theory and Related Fields, 154:787–834, 2012.
[22] R. Mikulevicius and B. Rozovskii. On distribution free Skorokhod–Malliavin calculus.
Stochastics and Partial Differential Equations: Analysis and Computations, 4:319–360,
2016.
[23] G. Milstein and M. Tretyakov. Solving parabolic stochastic partial differential equations
via averaging over characteristics. Mathematics of Computation, 78:2075–2106, 2009.
[24] G. Milstein and M. Tretyakov. Stochastic Numerics for Mathematical Physics. Springer,
2013.
[25] A. Quarteroni, A. Manzoni and F. Negri. Reduced Basis Methods for Partial Differential
Equations: an Introduction. Springer, 2015.
[26] A. V. Skorokhod. On a generalization of a stochastic integral. Theory of Probability &
Its Applications, 20:219–233, 1976.
38
[27] D. Venturi, X. Wan, R. Mikulevicius, B. Rozovskii, and G. Karniadakis. Wick–Malliavin
approximation to nonlinear stochastic partial differential equations: analysis and simu-
lations. In Proc. R. Soc. A, The Royal Society, 2013, pp.1–20.
[28] X. Wan and B. Rozovskii. The Wick–Malliavin approximation of elliptic problems with
log-normal random coefficients. SIAM Journal on Scientific Computing, 35:A2370–
A2392, 2013.
[29] G. C. Wick. The evaluation of the collision matrix. Physical Review, 80:268–272, 1950.
[30] D. Xiu and G. Karniadakis. The Wiener–Askey polynomial chaos for stochastic differ-
ential equations. SIAM Journal on Scientific Computing, 24:619–644, 2002.
[31] D. Xiu and G. Karniadakis. Modeling uncertainty in flow simulations via generalized
polynomial chaos. Journal of Computational Physics, 187:137–167, 2003.
[32] D. Xiu and G. Karniadakis. Supersensitivity due to uncertain boundary conditions.
International Journal for Numerical Methods in Engineering, 61:2114–2138, 2004.
[33] D. Xiu. Numerical Methods for Stochastic Computations: a Spectral Method Approach.
Princeton University Press, 2010.
[34] Z. Zhang, B. Rozovskii, M. Tretyakov and G. Karniadakis. A multistage Wiener chaos
expansion method for stochastic advection-diffusion-reaction equations. SIAM Journal
on Scientific Computing, 34:A914–A936, 2012.
[35] Z. Zhang, M. Tretyakov, B. Rozovskii and G. Karniadakis. A recursive sparse grid col-
location method for differential equations with white noise. SIAM Journal on Scientific
Computing, 36:A1652–A1677, 2014.
[36] Z. Zhang, M. Tretyakov, B. Rozovskii and G. Karniadakis. Wiener chaos versus stochas-
tic collocation methods for linear advection-diffusion-reaction equations with multiplica-
tive white noise. SIAM Journal on Numerical Analysis, 53:153–183, 2015.
[37] Z. Zhang and G. Karniadakis. Numerical Methods for Stochastic Partial Differential
Equations with White Noise. Springer, 2017.
[38] M. Zheng, B. Rozovskii and G. Karniadakis. Adaptive Wick–Malliavin approxima-
tion to nonlinear SPDEs with discrete random variables. SIAM Journal on Scientific
Computing, 37:A1872–A1890, 2015.
39
