Monte Carlo Methods for Uncertainty Quanti cation

Monte Carlo Methods for Uncertainty Quantification

Abdul-Lateef Haji-AliBased on slides by: Mike Giles

Mathematical Institute, University of Oxford

Contemporary Numerical Techniques

Haji-Ali (Oxford) Monte Carlo methods 1 / 27

Lecture outline

Lecture 4: PDE applications

PDEs with uncertainty

examples

multilevel Monte Carlo

Extensions


PDEs with Uncertainty

Looking at the history of numerical methods for PDEs, the first stepswere about improving the modelling:

1D → 2D → 3D

steady → unsteady

laminar flow → turbulence modelling → large eddy simulation→ direct Navier-Stokes

simple geometries (e.g. a wing) → complex geometries(e.g. an aircraft in landing configuration)

adding new features such as combustion, coupling to structural /thermal analyses, etc.

. . . and then engineering switched from analysis to design.



The big move now is towards handling uncertainty:

uncertainty in modelling parameters

uncertainty in geometry

uncertainty in initial conditions

uncertainty in spatially-varying material properties

inclusion of stochastic source terms

Engineering wants to move to “robust design” taking into account theeffects of uncertainty.

Other areas want to move into Bayesian inference, starting with an a prioridistribution for the uncertainty, and then using data to derive an improveda posteriori distribution.



Range of applications: parabolic, elliptic, hyperbolic, numericalhomogenization, reduced basis approximation.Examples:

Long-term climate modelling:

Lots of sources of uncertainty including the effects of aerosols,clouds, carbon cycle, ocean circulation(http://climate.nasa.gov/uncertainties)

Short-range weather prediction

Considerable uncertainty in the initial data due to limitedmeasurements



Engineering analysis

Perhaps the biggest uncertainty is geometric due to manufacturingtolerances

Nuclear waste repository and oil reservoir modelling

Considerable uncertainty about porosity of rock

Astronomy

“Random” spatial/temporal variations in air density disturbcorrelation in signals received by different antennas

Finance

Stochastic forcing due to market behaviour



In the past, Monte Carlo simulation has been viewed as impractical due toits expense, and so people have used other methods:

stochastic collocation

polynomial chaos

Because of Multilevel Monte Carlo, this is changing and there are nowseveral research groups using MLMC for PDE applications

The approach is very simple, in principle:

use a sequence of grids of increasing resolution in space (and time)

as with SDEs, determine the optimal allocation of computationaleffort on the different levels

the savings can be much greater because the cost goes up morerapidly with level (higher dimensionality).


PDEs with Uncertainty: Example

Elliptic PDE coming from Darcy’s law:

∇·(κ(x)∇p

)= 0

where the permeability κ(x) is uncertain, and log κ(x) is often modelled asbeing Normally distributed with a spatial covariance such as

R(log κ(x1), log κ(x2)) = σ2 exp(−‖x1−x2‖/λ)

Modelling of oil reservoirs and groundwater contamination in nuclearwaste repositories (Giles, Scheichl, Cliffe, 2011).


Stochastic Field

A typical realisation of κ for λ = 0.01, σ = 1.


Stochastic Field

Samples of log κ are provided by a Karhunen-Loeve expansion:

log κ(x , ω) =∞∑n=0

√θn ξn(ω) fn(x),

where θn, fn are eigenvalues / eigenfunctions of the correlation function:∫R(x , y) fn(y) dy = θn fn(x)

and ξn(ω) are standard Normal random variables.

Numerical experiments truncate the expansion.

(Latest 2D/3D work uses an efficient FFT construction based on acirculate embedding.)


Stochastic FieldDecay of 1D eigenvalues

100

101

102

103

10−6

10−4

10−2

100

n

eig

en

va

lue

λ=0.01

λ=0.1

λ=1

When λ = 1, can use a low-dimensional polynomial chaos approach, butit’s impractical for smaller λ.


PDEs with Uncertainty: Results

Boundary conditions for unit square [0, 1]2:– fixed pressure: p(0, x2)=1, p(1, x2)=0– Neumann b.c.: ∂p/∂x2(x1, 0)=∂p/∂x2(x1, 1)=0

Output quantity – mass flux: −∫

k∂p

∂x1dx2

Correlation length: λ = 0.2

Coarsest grid: h = 1/8 (comparable to λ)

Finest grid: h = 1/128

Karhunen-Loeve truncation: mKL = 4000

Cost taken to be proportional to number of nodes. ∼ h2`


2D Results

0 1 2 3 4−12

−10

−8

−6

−4

−2

0

2

level l

log

2 v

aria

nce

Pl

Pl− P

l−1

0 1 2 3 4−12

−10

−8

−6

−4

−2

0

2

level l

log

2 |m

ea

n|

Pl

Pl− P

l−1

V[P`−P`−1] ∼ h2` E[P`−P`−1] ∼ h2`


MLMC General Theorem

If there exist independent estimators Y` based on N` Monte Carlo samples,each costing C`, and positive constants α, β, γ, c1, c2, c3 such thatα≥ 1

2 min(β, γ) and

i)∣∣∣E[P`−P]

∣∣∣ ≤ c1 2−α `

ii) E[Y`] =

E[P0], ` = 0

E[P`−P`−1], ` > 0

iii) V[Y`] ≤ c2N−1` 2−β `

iv) E[C`] ≤ c3 2γ `


MLMC General Theorem

then there exists a positive constant c4 such that for any ε<1 there existL and N` for which the multilevel estimator

Y =L∑`=0

Y`,

has a mean-square-error with bound E[(

Y − E[P])2]

< ε2

with a computational cost C with bound

C ≤

c4 ε−2, β > γ,

c4 ε−2(log ε)2, β = γ,

c4 ε−2−(γ−β)/α, 0 < β < γ.


2D Results

0 1 2 3 410

2

103

104

105

106

107

108

level l

Nl

10−3

10−2

100

101

102

accuracy ε

ε2 C

ost

Std MC

MLMCε=0.0005

ε=0.001

ε=0.002

ε=0.005

ε=0.01


Complexity analysis

Relating things back to the MLMC theorem:

E[P`−P] ∼ 2−2` =⇒ α = 2

V` ∼ 2−2` =⇒ β = 2

C` ∼ 2d` =⇒ γ = d (dimension of PDE)

To achieve r.m.s. accuracy ε requires finest level grid spacing h ∼ ε1/2and hence we get the following complexity:

dim MC MLMC

1 ε−2.5 ε−2

2 ε−3 ε−2(log ε)2

3 ε−3.5 ε−2.5


Non-geometric multilevel

Almost all applications of multilevel in the literature so far use a geometricsequence of levels, refining the timestep (or the spatial discretisation forPDEs) by a constant factor when going from level ` to level `+ 1.

Coming from a multigrid background, this is very natural, but it is NOTa requirement of the multilevel Monte Carlo approach.

All MLMC needs is a sequence of levels with

increasing accuracy

increasing cost

increasingly small difference between outputs on successive levels


Applying MLMC to your problems

First, identify the approximation parameter(s) in the problem to builda hierarchy of approximations. We also need to make sure that wecan to sample correlated approximations.

Try to find estimators that increase the variance convergence rate, β,as much as possible to get the optimal complexity.

The MLMC/MIMC theory depends on a set of assumptions.Checking these assumptions numerically is straightforward butproving them can be challenging.


Multilevel Monte Carlo: Summary

Let ∆`P = P` − P`−1 and ∆0 = P0 and observe that

E[P] =∞∑`=0

E[∆`P] ≈L∑`=0

E[∆`P].

Then, the MLMC estimator

L∑`=0

1

N`

N∑m=1

∆`P(`,m)`

with optimal choice of L and N` has complexity

O(ε−2−max(0, γ−βα )

),

when γ 6= β and O(ε−2|log ε|2

)otherwise.


Multilevel Monte Carlo for high-dimensional problems

The problem is that the cost usually grows exponentially as the number ofdiscretization parameters, d , increases, i.e., the cost is O

(2dγ`

).

On the other hand, the weak error, O(2−α`

), and variance, O

(2−β`

), do

not decrease with increasing number of discretization parameters.

Leading to a complexity that suffers from the curse of dimensionality

O(ε−2−max(0, dγ−βα )

).

Can we leverage some mixed-regularity between these discretizationparameters to increase the convergence rate of the weak error and thevariance?


Multi-Index Monte Carlo




Multi-Index Monte CarloLet ` = (ì )

di=1 ∈ Nd be a “multi-index” for d discretization parameters

and denote by P` the corresponding approximation.Then, define the i th-dimension difference operator

∆i ,`P =

{P` if ì = 0,

P` − P`−e iotherwise

and ∆`P =d⊗

i=1

∆i ,`P.

Observe, again, that

E[g ] =∑`∈Nd

E[∆`P] ≈∑`∈I

E[∆`P],

for some I ⊂ Nd . Then, estimate the expectations with independentMonte Carlo samplers to yield the MIMC estimator

∑`∈I

1

N`

N∑m=1

∆`P(m).



Assuming that the cost is O(2γ|`|

),

|E[∆`g ]| = O(

2−α|`|),

and V[∆`g ] = O(

2−β|`|),

Decreasing ε

`1

`2

where |`| = `1 + `2 + . . . `d . Then, there is an optimal choice of N` and I(total degree or simplex) so that the complexity of MIMC is

O(ε−2−max(0, γ−βα )|log ε|d−1

),

when γ 6= β and O(ε−2|log ε|2d+max(0,d−3)

)otherwise.


Other MLMC extensions

Unbiased MLMC: Based on the observation

E[P] =∞∑`=0

E[∆`P] = E[∆`P/p`],

where, in the RHS, ` is a random variable with P[` = i ] = pi , chosenoptimally.Useful when β > γ to simplify the analysis of some compositemethods but has the same complexity as MLMC.

Multilevel Richardson-Romberg extrapolation (ML2R): ApplyingRichardson extrapolation to the weak error estimate, find optimal wìn

E[PL] =L∑`=0

wÈ[∆`P]

to have a smaller number of required levels, L. ML2R is useful whenβ ≤ γ and improves the computational complexity.


Other hierarchical samplers

The same multilevel/multi-index ideas (and extensions) can be easilyapplied (and laboriously analysed) to other samplers such as:

Stochastic Collocation and Quasi-Monte Carlo: When we have highermixed regularity between the stochastic variables. Leads to bettercomplexities.

Random L2 projection: To produce surrogate models of the quantitiesof interest instead of computing their expectations. Useful forregression and Machine Learning.

Markov Chain Monte Carlo or Sequential Monte Carlo: for inverseproblems where the expectation is with respect to a posteriordistribution depending on a set of observed data points.


Final comments

Uncertainty Quantification is a hot topic, with its own conferencesand journals

Monte Carlo methods are a powerful approach to handle uncertaintyin a number of different settings

Multilevel Monte Carlo greatly reduces the cost in a lot of settings,particularly when dealing with PDEs

for more details, see (Giles, 2015) in Acta Numerica.


Documents

Monte Carlo Methods for Uncertainty Quanti cation