Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Sparse Grid Methods for Uncertainty Quantification
Michael Griebel
University of Bonn and Fraunhofer SCAI
Joint work with Alexander Rüttgers
1. Sparse grids - Construction principles and properties - Optimal sparse grids - Adaptive combination method 2. Application - Multi-scale viscoelastic flows
Motivation
• Numerical methods in uncertainty quantification:
- Galerkin approach - Collocation technique - Discrete projection
• Needed on stochastic/parameter domain: - Approximation of integrals - Interpolation, especially for collocation
• Simple domains with product structure: • Issue: high- or even infinite-dimensional problems
dd Raa ,],[ I
Curse of dimension • • Bellmann ´61: curse of dimension
• Find situations where curse can be broken ? • Trivial: restrict to
but practically not very relevant • In any case: some smoothness changes with or importance of coordinates decays successively (e.g. after suitable nonlinear transformation)
,)(rVf ,: )( Rf d r isotropic smoothness
)(||)(|||| // dr
H
dr
HM MOfMdCff rss
)(dOr
)()(|||| / cdcd
M MOMOff
d
dof#M
I
Sparse grid approach • Basic principles:
– 1-dim multilevel series expansion with proper decay – d-dim product construction – Trunctation of resulting multivariate expansion
• Effect:
– reduction of cost complexity – nearly same accuracy as „full“ product – necessary: certain smoothness requirements – adaptivity for detection of lower-dimensional manifolds
Simple example: Hierarchical basis
parabola in [-1,1]
conventional coefficients no decay from level to level
hierarchical coefficients decay by ¼ from level to level
1l
2l
3l
1W
2W
3W
3V
)1)(1()( xxxf
Tensor product hierarchical basis Generalization to higher dimension by tensor product
decay in x- and y-direction by 1/4 decay in diagonal direction by 1/16
Idea: Omit points with small associated hierarchial coefficient values
Table of subspaces
11 l 21 l 31 l
12 l
22 l
32 l
21llW
Regular sparse grids
Properties of regular sparse grids
Cost:
Accuracy:
Mitigates the curse of dimension of conventional full grids Note: Higher regularity in mixed derivative, ~d For wavelets, general stable multiscale systems:
instead of
-norm
Sparse grids Full grids
Smoothness:
:
Space, seminorm:
)( dNO
)( 2NO
))log(( 1dNNO
))log(( 12 dNNO
cx
fd
i i
||1
2
2
cxx
f
d
d
|
...|
22
1
2
2
2 ||, fHmixmix fH ,
2 ||, 2
))(log( 2/)1(2 dNNO
nN 2
2L
Re-invented several times: 1957 Korobov, Babenko 1963 Smolyak 1970 Gordon 1980 Delvos, Posdorf 1990 Zenger, G. 2000 Stromberg, deVore 2010 ????
hyperbolic cross points blending method Boolean interpolation sparse grids hyperbolic wavelets
Application areas include: • solution of PDEs • integral equations • eigenvalue problems
• quadrature • interpolation • data compression
History of regular sparse grids
ll WVPP d
ljl
d
jl
d
j jj
)(
111
:)(:
Basic principles of sparse grids • 1-dim multilevel sequence of operators and spaces
• Sequence of differences, telescopic approach
• d-dim. product construction
• Appropriate truncation of resulting multivariate expansion
lVll VVP )1(:
llllll WVVVPP ::)(: 1
)1(
1
lll Wff )(
dd NN
d
d Nlll ),,,( 21 l
Nl
l
ll
l PPdN
I
I
II
• Integration:
– Sequence of nested or non-nested point sets and weights, size: or
=> various sparse grid quadrature rules • Interpolation , approximation
– Local piecewise polynomials, multiscale expansion: hierarchical basis, interpolets, wavelets, multilevel basis, size:
=> sparse grid finite element spaces – Global polynomials: Fourier series, Chebyshev, Legendre,
Hermite, Bernoulli polynomials size or or
=> total degree / hyperbolic cross approximation
Examples of multiscale expansions, 1d
lnl 12 l
ln
12 l
ln
lnl 12 l
ln
RVVQP lll )1(:
lll VVIP )1(: lll VVAP )1(:
I
1|| lW
12|| l
lW
12|| l
lW
Regular sparse grid approach • Index sets
• The hierarchical representation is then
• Other representations:
– generating system – Lagrange system over SG points – semi-hierarchical – combination method
d
j
j
d
n dnlN1
1
sparse 1|:| ll
nlN jdj
d
n
,,1
full max|:|
ll
1||
sparse
1 dn
nPl
l
1||
sparse
1
)()(dn
n ffPl
l
I
I
• A simple alternative representation is [G., Schneider, Zenger 91],
– Involves just the (anisotropic) full grid discretizations on different levels and linearly combines them
• 2D example
The combination technique
1|| 1
1|combi
1
1
1||
1)1(
dnn
dn
n Pd
Pl
l|l
l jl
d
jPP
1:
l
lP
2||1||
combi
11 dndn
n PPPl
ll
l
4n
5 indices, level n
• Redundant representation but allows the simple
reuse of existing code • Completely parallel computation of the subproblems • Corresponds to a certain multivariate extrapolation
method [Rüde 91]
• Necessary: Existence of a pointwise error expansion. – Euler-Maruyama of stochastic ODE: additive expansion
(leading error term) of mean square error • Multilevel-Monte Carlo is just 2-d combination method
– Variance and bias for the two dimensions and a proper refinement rule which reflects the MC and the Euler-Maruyama rates [Gerstner12, Harbrecht,Peters,Siebenmorgen13]
The combination technique
lP
• In general: Given
– a class of functions and an error norm – an associated bound for the benefit of – a bound for the cost of
• We can a-priori derive a (quasi-) optimal sparse grid by solving a binary knapsack problem [Bungartz+G.03]
and setting • Boils down to just sorting the quotients of the
benefit versus cost according to its size and taking the largest indices into account
A priori construction of sparse grids
l
)(/)( ll cb
)(lc
)(lb
l
dN
bl
l l)(max fix
N
Ccd
l
l l)(thatsuch 1,0l
1: ll d
C NI
• Representation • Cost per subspace • Benefit for accuracy
• Choice of best subspaces ? Knapsack problem ! => local benefit2/cost ratio
-norm-based sparse grids in )()( xx
ll ff ),..,( 1 dxxx
),..,( 1 dlllll x Wf )(
1||2)dim()(
1lll Wc
)2( 1||2 lO
ll
WVdn
optd
n
1||
),(
1
2l
1l
1|| 1 dnl isoline
n
2L
mix
d fbf ,
||2
2 ||23)(|||| 1
2l
l l
regular sparse grid space 1
1
1
||5
||
||42 2
2
2)(/)(
ll
l
ll
cb
2
mixH
Anisotropic sparse grids
• Non-equal directions
– Weighted Sobolev spaces [Sloan+Wozniakowski93]
– Anisotropic smoothness spaces [Gerstner+G. 98, G.+Zung15]
– Different dimensions for different directions [G.+Harbrecht 11]
• Via knapsack problem:
– A priori construction of optimal anisotropic sparse index sets
– log-terms disappear
r
mixH ,
)()()( 21
,,, 2121
d
ssssss
mix IHIHIHH dd
)()()( 2121
d
sss dHHH
Generalized sparse grids • General index sets
• Downward closed set, no holes
• Associated sparse grid operator
• Associated space and associated function
dN
ll
WV
l
lP
l
ll
l fffP )(
dje j ,,1 ll1l
2lI
• Can also be generalized to a given downward closed
index set
• Combination coefficient
with characteristic function on the index set • Again: just (anisotropic) full grid discretizations
on different levels get linearly combined • Note: many coefficients on the lower levels are zero
The combination technique
l
llPcP
)()1( 1|| zl1
0z
zl
c
lP
Tensor product sparse grids • Examples:
– space time, , parabolic problems – space parameters but smooth in parameter variables – space stochastics but analytic in stochastic variables
• Main result: Curse of dimension only w.r.t. the larger dimension and/or the lower smoothness [G.+Harbrecht11], [G.+Zung15]
• Time, parametrization and stochastic coordinates disappear in the overall complexity rate
=> just space discretization matters
1,3 21 dd
2010,3 21 dd
21 ,3 dd
)),0(()(),0()( 22210
2infTHH
n
TLHnVu
ucuunn
Sparse space-time grids
)),0(()( 22 THHu
space dimension 2, space-time sparse grid, Cranck-Nicolson case, n=4,5:
space dimension 1, space-time sparse grid, Euler case
• Approximation error and necessary regularity [G.+Oeltz07]
– Classical regularity theory (Ladyzenskaja, Wloka)
– Sparse space-time grids posses same approximation rate as full space-time grids but just cost complexity of space problem
– In each time slice there is a conventional full grid
• Solutions of stochastic/parametric PDEs live on product
• of temporal domain • of spatial domain with • and stochastic/parametric domain with large or
even infinity. • Often: Very high smoothness in -part
– Here: especially weighted analyticity for the different coordinates, decay in covariance [Cohen,Devore,Schwab10,11]
– Then, even infinite-dimensional become treatable • Sparse grid not only within stochastics but also
between spatial, temporal and stochastic domain
Stochastic and parametric PDEs
),,(),,(),(),,( yxyxyxyx trtfAtut
YXyx Tt ),,(
3,2,11 dX2dY
y
Y
T
• Analytic regularity in polydisc with radii • Sequence of smoothness indices • With global polynomials: • Accuracy with respect to the involved #dof
[Beck,Nobile,Tamellini,Tempone12,14], [Tran,Webster,Zhang15], [G.+Oettershagen15]
• For the infinite-dimensional case:
– Logarithmic growth => algebraic rate [Todor,Schwab07], [Cohen,Devore,Schwab10,11]
– Linear growth => subexponential rate [G.+Oettershagen15], [Tran,Webster,Zhang15]
Sparse grids and analytic functions
)...( 11|)(| ddkakaecf
k
)log(),( 1 ra daa
)( /)1()()( /1 ddMdgm MeOd a
M
1/
1
1
ja je
dd
j
jagm
/1
1
)(
a eddd d /)!()( /1
)( )1( MO1
ja j 0 ))log(( 2/141)log(
8
3
MMMOM
),(: 1 drr r
Stechkin´s Lemma
Stechkin´s Lemma can not show this rate but gives only an algebraic bound
Dimension-adapted sparse grids • So far: function class known,
– a-priori choice of best subspaces by optimization – size of benefit/cost ratio indicated if subspace is relevant => sparse grid patterns for
• Now: for given single function – adaptively build up a set of active indices – benefit , i.e. local error-indicator of – cost for subspace , – benefit/cost indicator – refinement strategy to build new index set, – global stopping criterion => sparse grid pattern
• Directions with product of different smoothness
lW
YXT
f
2||)(||:)( fb ll f
||)( ll Wc
)(/)(:)( lll cb
The adaptive combination algorithm
simple extension to dimension-adaptive version exists => UQ14
1l2l
refinement rule
downward closedness
• Evolution of the algorithm:
• As any adaptive heuristics: may terminate too early • If mixed regularity not present, refinement to the usual full grid
index sets:
corresponding grids:
Example
Application: Non-Newtonian fluids
• Classical Newtonian fluids: Obey Newton´s law of
viscosity, stress tensor is proportional to load/force
• But various complex fluids show strange behavior which is not correctly described
Weissenberg effect tubeless siphon effect Barus effect
Application: Non-Newtonian fluids
• Non-Newtonian fluids contain microstructures which
are the reason for their unusual properties • Examples: paint, toothpaste, shampoo, blood, oils
• Polymeric fluids are a subset of non-Newtonian fluids • Long-chained molecules in a Newtonian solvent • Viscoelasticity due to interaction of elastic molecules and
drag forces in basic flow • A macroscopic model like the Navier Stokes equations
+ macrosopic extensions is no longer sufficient • Needs to be augmented by model on the micro scale => Two scale modelling
Mathematical modelling
• The conservation equations for polymeric fluids are
the same as for the Newtonian case, but the presence of polymer molecules contributes a polymeric extra-stress tensor and an additional polymeric viscosity such that the viscosity ratio
• The Navier-Stokes equations are now
+ b.c., with Reynolds number and viscosity ratio
pτ
ppt
τuuuu
Re
1
Re
1
0 u
conservation of momentum
solvent viscosity
Re
ps
s
polymeric viscosity s
p
1p
• On the microsocopic scale, a polymer chain is
modelled by a spring chain of K+1 beads
• Position in physical space/flow domain • Orientations in configuration space • Probability to find chains at time with position in
. and orientations in
Microscopic modelling
x 3R
Kqq ,...,1KR3
t xxx d, KKK dd qqqqqq ,...., 111
),,...,,(),,...,,(,,0: 11 ttRT KK qqxqqx
I
I
I
• The function is a pdf, i.e. • The application of Newton´s 2nd law to the forces
acting on chain leads to the Fokker-Planck equation
with Rouse matrix
• Describes evolution of under chain´s spring forces
• Various models for spring force: Hooke:
FENE: , FENE-P:
Fokker-Planck equation
i
TK
iit
quu xqx )()(1
ji
K
i
K
j
iji
K
j
ij ADe
ADe
qqqF
1 11 4
1)(
4
1
1,0
)(),..,( 1 KqFqF
KA ]121[
qF(q)
bqb
2||||,1 /||q||
qF(q) 2b
b
2
2,
1q
/qqF(q)
Deborah number
• represents polymeric configurations of micro-system • Expectation in configuration space
• Coupling of internal configurations of micro system to macroscopic stress tensor via Kramer´s expression
Constant C depends on model, Deborah number, viscosity ratio
• Issues with the Fokker-Planck equation – becomes more singular for higher values of [Suli, Knezevic08]
=> extremely fine numerical resolution needed [Lozinski, Owen 03]
– -dimensional + time-dependent => curse of dim.
Coupling to the macro scale
Kdd qq ...1
K
i
iip C1
)( IdqFqτ
De
)1(333 KK
• There is a formal equivalence between the Fokker-
Planck equation and stochastic partial differential eq. – Describes evolution of random fields that
represent the configuration vector – Brownian forces on the beads are modelled by the 3-dim.
Wiener processes – The vector consists of the component-wise differences
Stochastic microscopic modelling
T
K ),...,( 1 qqq
1,...,1),( KitiW
T
K ),...,( 1 QQQ
),()(),()(),( tttd xQuxQuxQ
)(2
1)),((
4
1td
DedttQA
DeUxF
K
)(tU
Kittt iii ,...,1),()())(( 1 WWU
Deborah number
• Brownian configuration fields (BCF) [Hulsen97]
Random field for configuration • Discretization of x-space: the grid cells make from
the parabolic SPDE a system of SODEs (MoL) • Discretization of SODE-system: Put configuration
fields in each of the space grid cells and evolve their configuration discretely over time, i.e. all configuration fields have fixed spatial positions (Eulerian view).
Stochastic microscopic simulation
),( txQ
BM
GM
BG MM
GM
• In each grid cell with center we solve/
integrate the stochastic DE for a number of stochastic realizations
• They are distributed according to the known equilibrium density for
• But we do not know for . Thus, we approximate the first moments in Kramer´s relation as
i.e. we replace the integral by Monte Carlo quadrature
Stochastic microscopic simulation
BM
Bk
j Mjt ,...,1),,()( xQ
K
i
M
j
k
j
ik
j
i
B
K
i
kikikp
B
ttM
C
ttCt
1 1
)()(
1
)),((),(1
)),((),(),(
IdxQFxQ
IdxQFxQxτ
)),((),( tt kiki xQFxQ
0t
0t
GMk ,,1 kx
• Navier Stokes equations:
– Uniform grid cells, staggered grid, cell centers , , cell faces – WENO for convective terms, 2nd order scheme for other terms – Euler or Crank-Nicolson in time, CFL-condition – Chorin-like projection method
• Microscale stochastic equations: – stochastic samples for each grid cell => samples – QUICK for convective terms – Explicit Euler-Maruyama, semi-implicit Euler for FENE – Same time step size as for NS equations – Variance reduction scheme with equilibrium control variates
Numerics
ppτ u
BM BG MM
• Code works as expected • But: Huge memory requirements and
huge computing times due to large number of realizations in each cell
• Example for 3D multi-scale problem – Flow domain with
• = 100x100x100 grid cells • = 10.000 stochastic realizations in each grid cell
– Total memory requirements: • 8 MB for the pressure field • 24 MB for the velocity field • 48 MB for the six independent components of • 75 GB*N for all the stochastic variables
– Some months of computing time
Issues
p
u
pτ
GM
BM
BM
BG MM
Newtonian
non-Newtonian
• Consider our multiscale flow problem in more detail. • We have the problem parameters:
mesh width, time step size, stochastic realizations, springs • How can we improve on computational complexity ?
– Instead of MC use QMC – Multilevel-MC, MLQMC for stochastic ODEs (time + stoch.) This is just a certain 2d combination technique/ sparse grid approach [Gerstner 12] [Harbrecht,Peters,Siebenmorgen13]
– Combination technique in all 3 discretization parameters i.e. for space x time x stochastics, and for model parameter K, i.e. …. x number of springs
– If the optimal combination formula is not a priori known: run the (dimension)-adaptive algorithm
Sparse grid approach
Coordinates for the combination method
we use only an isotropic grid in our NS solver
• Approximation of the vector and the tensor • Compute benefits and costs componentwise • One index set for all components
• Weighted and scaled benefit/cost indicator
Scaling with initial level not necessary if or
Indicators for the combination method u pτ
)(lb )(lc
2,
2,
2,2
2,2
||))(1(||)(
||))((||)1(,
||))(1(||))((
||))((||max)(
Fp
Fp
bc
b
bc
b
ll
uulul
l
0 1)1(b
• Non-Newtonian fluid in a 2D channel.
– Fluid is at rest at initial time t = 0, – Shearing of fluid over time with rate – Linear spring force model (dumbbell, K=1) – Probability density function
1d in space, 2d in configuration space and time-dependent
• Discretization: – Initial level – Refinement from level to level by factor *2 – Error indicator , we are after error in
Example 1: Couette flow
RtxRtx ),,(),,(: 4 qq
dydu /
256)16,(4,samples),/1,/1( tx
1
time
velocity
space
u
I I
5.0De
Example 1 Couette flow
• Behaviour of adaptive combination technique
• We asymptotically observe an anisotropic sparse grid structure
• Comparison:
– Full grid error
– Cost (dof) full grid sparse grid
Example 1 Couette flow
1u2L
04.0)E(u6,6,6
01.0)E(u7,7,7
8
6,6,6 104.5)C(u 9
7,7,7 103.4)C(u
7C 106.4)C(u
• Relative error of
3 time 2 l 5 time 2 l
3l
• Non-Newtonian fluid in a 3D domain.
– Steady uniaxial extensional flow, – Stress tensor is aimed for – FENE force model, K-spring chain – We vary the number K of springs up to 5 – Probability density function 3N-dimensional in configuration, time-dependent, number of springs, no space
• Discretization – Initial level – Refinement for time and samples from level to level by
factor *2, refinement for springs by +1 – Error indicator , we are after error in
Example 2: Steady extensional flow
RtRRt K ),(),(: 3 qq
)2
,2
,( zyx
u
1) 2, (1024,springs),/1 samples,( t
0
pτ
pτ
I I I
0.1De
Example 2: Steady extensional flow • Behaviour of adaptive
combination technique • We observe:
– a sparse grid structure for all indices
– plus a nearly full grid between time and springs for the smallest sample size
– Different refinement: *2 versus +1
• Relative error for of adaptive combination technique
xx2L
• Convergence of model for rising number K of springs
• All results are computed on fine level with 2 million samples. • Fixed stochastic time step width
Example 2: Steady extensional flow
2048/1t
• Basic principles of sparse grids • Optimization by knapsack problem • Dimension-adaptive combination method
– Solution of subproblems on levels – Sparse grid approximation by linear combination – Refinement with hierarchical contributions and local cost
• Application to non-Newtonian flow – Two-scale problem, stochastic microscale
• Adaptive combination method works on discretization directions (space x time x samples) and also for model parameters (… x springs)
=> Allows to couple discretization and modelling errors
Concluding remarks
lP l
l
The C library HCFFT G.+Hamaekers
• Hierarchical sparse grid interpolation based on:
- Fast Fourier transform (FFT), fast Sine and Cosine transform - Fast Chebyshev transform, Fast Legendre transform - Various other polynomial transforms
• Different hierarchical bases for different dimensions • Dyadic and arbitrary, non-dyadic refined grids • Several types of general sparse grids • Dimension-adaptive sparse grids • For high precision: possible use of long double • Freely available at www.hcfft.org
• Code NAST3DGPF which is freely available at http://www.nast3dgpf.de/
The flow solver