Transported Probability and Mass Density Function (PDF… · Transported Probability and Mass Density Function (PDF/MDF) Methods for Uncertainty Assessment and Multi-Scale Problems

Transported Probability and Mass Density Function (PDF/MDF)

Methods for Uncertainty Assessment and Multi-Scale Problems

Institute of Fluid Dynamics

Patrick Jenny!!

Institute of Fluid Dynamics!Swiss Federal Institute of Technology; ETH Zürich

Institute of Fluid DynamicsPatrick Jenny

Course OutlineMotivation: For the simulation of fluid flows, probability density function (PDF) methods have advantageous properties compared to moment-based approaches or purely deterministic methods and are applicable in different fields. For example, PDF methods are used to model turbulent reactive flows, to quantify uncertainty of transport in the subsurface, and for simulations of multi-phase flows or rarefied fluids. !Goals: In this short course, the theory behind PDF methods is outlined and solution algorithms, which typically are Lagrangian particle Monte Carlo methods, are described. It will become clear, for which problems this approach is attractive, how it can be implemented and employed, and how the associated mathematical theory can help to develop and analyze models. Finally, as an illustrative example, a PDF method for rarefied gas flow simulations is discussed in more details. !Recommended Books: Statistical Mechanics of Turbulent Flows. Stefan Heinz. Springer, 2003.

http://www.springer.de !Turbulent Flows. Stephen B. Pope. Cambridge University Press, 2000.

http://eccentric.mae.cornell.edu/~pope/TurbulentFlows.html

http://www.springer.de

http://eccentric.mae.cornell.edu/~pope/TurbulentFlows.html


Course Outline

Part I: The first part provides an introduction, in which illustrative examples from various fields are presented, including uncertainty assessment of contaminant transport, turbulent combustion and rarefied gas flow. This is helpful to become familiar with the basic ideas of PDF modeling. !Part II: In the second part, the necessary mathematical background is introduced, whereas the main focus lies on the relation between Fokker-Planck (FP) equations for PDF transport, and continuous stochastic processes for individual realizations. This rigorous mathematical theory allows to derive deterministic PDF evolution equations from stochastic models, which is extremely powerful, even if one does not intend to employ a PDF method; for example to derive stochastic moment equations.


Course OutlinePart III: In part three, the concepts and advantages of PDF methods are demonstrated in more depth with the help of an illustrative example. Starting with a simple stochastic model for molecular motion in a monatomic gas, the mathematical theory provided in the second part of this course is employed to first derive the corresponding PDF equation for molecule locations and velocities and then the consistent set of conservation laws for mass, momentum and energy. While the Navier-Stokes-Fourier system only holds in the continuum limit, that is, if the mean free path length is extremely small compared to the geometrical length scale, the PDF equation can also describe rarefied gas flow. In fact, it proved to be a very good approximation of the Boltzmann equation, but provides crucial computational advantages for solution algorithms. !Part IV: Part four is devoted to related numerical solution methods. In most cases, the probability space is high dimensional and therefore particle Monte Carlo methods are favored. Sometimes it is possible to find good closures based on moment equations, especially if the PDF shape can be presumed, but such approaches are not further discussed. Instead, various components of particle Monte Carlo methods are described with focus on accurate particle path integration.


Part I: Basic Ideas of PDF Modeling

• introduction and motivation

• illustrative examples from various fields: - mixing - turbulent reactive flows - uncertainty assessment of contaminant transport - rarefied gas flow

• general structure of PDF/MDF transport equation


• in many problems a statistical description is preferable example: uncertainty assessment of contaminant transport in the sub-surface !

• complex joint PDF shapeexample: turbulent mixing !

• non-linearities example: turbulent reactive flows !

• complex temporal or spatial correlation behavior example: subsurface flow !

• wide range of length and timescales example: dynamics of rarefied gases !!

• long tradition in turbulent combustion (pioneered by Stephen B. Pope)

Part I - Introduction and Motivation


Turbulent MixingPart I - Illustrative Examples



(D. W. Meyer, P. Jenny)





Turbulent Reactive FlowsPart I - Illustrative Examples

The chemical reaction source term in a turbulent reactive flow, Q�(Y, T ), is

typically a highly non-linear function of the chemical composition vector Y =

(Y1, Y2, . . . , Yn)

Tand the temperature T . Accordingly, it is generally not suit-

able to estimate hQ�(Y, T )i appearing in the mean conservation equation of Y�

by Q�(hYi, hT i). Alternatively, a solution method is required that provides the

full joint composition–temperature PDF f(Y, T ) since

hQ�(Y, T )i =

Z

Rn

Z 1

0Q�(Y, T )f(Y, T ) dT dY.


Temperature Mixture fraction

(M. Hegetschweiler, P. Jenny)Turbulent Reactive FlowsPart I - Illustrative Examples


(B. Zoller, P. Jenny)Turbulent Reactive FlowsPart I - Illustrative Examples


Uncertainty Assessment of Sub-Surface FlowPart I - Illustrative Examples


simulator

„Oil

Cut

“

time

Monte Carlo:

Uncertainty Assessment of Sub-Surface FlowPart I - Illustrative Examples


Uncertainty Assessment of Contaminant TransportPart I - Illustrative Examples

Monte Carlo:

transport of tracer particles



Monte Carlo vs. PDF method:











publications


Dynamics of Rarefied GasesPart I - Illustrative Examples

(H. Gorji and P. Jenny, Journal of Computational Physics, 2014)

•Kn: 0.05 •speed of lid: 100 m/s •wall temperature: 300 K •all the walls are diffusive

lid-driven micro cavity



(H. Gorji, M. Torrilhon and P. Jenny, J. Fluid Mech., 680, 2011)

micro channel; Knudsen paradox


(P. Jenny, M. Torrilhon and S. Heinz, Journal of Computational Physics, 229, 2010)

PDF at left wall

PDF at center

Kn = 0.044

Kn = 5.3


micro channel: velocity slip



micro channel; Kn = 0.2




temperature contours for hypersonic nitrogen flow over a wedge

(H. Gorji and P. Jenny, Phys. Fluids, 25, 2013)

vibrational T translational T


General Structure of PDF/MDF Transport Equation Part I - Illustrative Examples

⇥�

⇥t+

⇥

⇥xi(Ui�) = S(�,x, t)

�fX(x; t)�t

+�

�xi

✓⌧DXi

Dt

��x; t�

fX(x; t)◆

= S(fX(x; t),x, t)

vector of random variables: X = (X1, X1, ...,Xn)

T

PDF of X in n-dimensional x-space: fX(x; t)

PDF transport equation:

compare with classical mass balance equation:

Homework: demonstrate for n=3 that

�fX(x; t)�t

=?

PDF/MDF methods

deterministic quantitative predictions are not always feasible


Part I - Summary

deterministic quantitative predictions are not always desired

either due to huge scale differences, “chaos” or uncertainty in the input data

one may rather be interested in joint PDFs

determine probability space, evolution equation and closure thereof


Part II: Necessary Mathematical Background

• illustration with Brownian motion !

• general PDF evolution equation - Kramers Moyal expansion - theorem of Pawula - Fokker-Planck equation

!• relation between PDF equation and stochastic processes for individual realizations !

• derivation of stochastic moment equations from a PDF equation

!!

• extremely powerful theory, even if one does not intend to employ a PDF method


Part II - Illustration with Brownian Motion

• Brownian motion was discovered in 1827 by the botanist Robert Brown

• first phenomenon modeled with stochastic processes; Einstein (1905), Langevin (1908)

• probabilistic quantification of the distance traveled by a pollen grain over time



Particle evolves according to Brownian motion:

dXi = (2�)

1/2dWi,

where Wi(t) is a Wiener process with dWi = Wi(t+dt)�Wi(t) being independent

normal distributed random variables with

hdWii ⌘ 0 and hdWidWji = dt�ij .

A statistically exact integration of the position is achieved with

�Xi = (2��t)1/2⇠i,

where ⇠i are independent normal distributed random variables and �t is the

time step size.


Sample trajectory of the stochastic model !!!for Brownian motion:

⇥Xi =p

2�⇥t �i,

How does the PDF fX(x; t) of the particle position X evolve?




Answer:

@fX

@t

=

@

2

@xi@xi(�fX) ,

where xi is the sample space coordinate of the stochastic variable Xi.

If a huge number M of particles is considered, then the local particle number

density C represents MfX , i.e. for constant M on obtains

@C

@t

=

@

2

@xi@xi(�C) .


Part II - Kramers Moyal ExpansionNext: we derive the general form of an evolution equation for fX(x; t), where

x 2 R:

fX(x; t) = h�(X(t)� x)ifX(x; t + �t) = h�(X(t)� x + �X)i

= fX(x; t) +

1X

k=1

1

k!

*✓� @

@x

◆k

�(X(t)� x) �X

k

+

= fX(x; t) +

1X

k=1

✓� @

@x

◆k ⇢h�X

k|x; tik!

fX(x; t)

�

from which follows the Kramers Moyal equation:

@fX(x; t)

@t

=

1X

k=1

✓� @

@x

◆k

8>><

>>:lim

�t!0

h�X

k|x; tik!�t| {z }

D(k)

fX(x; t)

9>>=

>>;.

Problem: 1 many terms!

Next: show that only two terms (k = 1, 2) are required, if lim�t!0 �x/�t

is bounded.


Part II - Theorem of Pawula

Theorem 1 If 9m > 1 : D

(2m) = 0, then 8k > 2 : D

(k) = 0.

Proof 1 Consider the two random variables ↵ = �X

aand � = �X

bwith

a, b 2 N ^ a, b � 1.Schwarz inequality )

h↵�|x; ti2 h↵2|x; tih�2|x; tih�X

a+b|x; ti2 h�X

2a|x; tih�X

2b|x; ti

�t ! 0 ^ b = a + k )

⇣(2a + k)!D(2a+k)

⌘2 (2a)!(2a + 2k)!D(2a)

D

(2a+2k)

If D

(2a) = 0 ) D

(2a+k) = 0 8k � 1If D

(2a+2k) = 0 ) D

(2a+k) = 0 8k � 1

) if 9m > 1 : D

(2m) = 0, then 8k > 2 : D

(k) = 0.⇤

lim�t!0

h�X

k|x; tik!�t| {z }

D(k)


Part II - Fokker-Planck Equation

Kramers Moyal equation:

@fX(x; t)

@t

=

1X

k=1

✓� @

@x

◆k

8>><

>>:lim

�t!0

h�X

k|x; tik!�t| {z }

D(k)

fX(x; t)

9>>=

>>;

There exist two possibilities:

1. only D

(1)and D

(2)are unequal zero, or

2. D

(2k) 6= 0 for all k � 1.

Gradiner showed that option one is true, if lim�t!0 �X/�t is bounded.


Part II - Fokker-Planck EquationRelation Between PDF Equation and Stochastic Processes

From the theorem of Pawula it follows that the Fokker-Planck equation

@fX(x; t)

@t

= �@D

(1)fX(x; t)

@x

+

@

2D

(2)fX(x; t)

@x

2

with

D

(1)= lim

�t!0

h�X|x; ti�t

D

(2)= lim

�t!0

h�X

2|x; ti2�t

describes the evolution of PDF’s based on continuous processes. Note that the

PDF equation allows to ”link” stochastic processes (or rules) with a determin-

istic description.



More general for high dimensional probability (sample) spaces with X(t) being

a realization in the x-space at time t:

@fX(x; t)

@t

= �@D

(1)i fX(x; t)

@xi+

@

2D

(2)ij fX(x; t)

@xi@xj

with with

D

(1)i = lim

�t!0

h�Xi|x; ti�t

D

(2)ij = lim

�t!0

h�Xi�Xj |x; ti2�t

.



Remember Brownian motion example with

�Xi = (2��t)

1/2⇠i

from which follows that

D

(1)i = lim�t!0

1�t (2��t)

1/2h⇠ii = 0

and D

(2)ij = lim�t!0

12�t2��th⇠i⇠ji = ��ij .

and therefore

@f

@t

=

@

2�f

@xi@xi

as presented earlier.



Remember Brownian motion example with

�Xi = (2��t)

1/2⇠i

from which follows that

D

(1)i = lim�t!0

1�t (2��t)

1/2h⇠ii = 0

and D

(2)ij = lim�t!0

12�t2��th⇠i⇠ji = ��ij .

and therefore

@f

@t

=

@

2�f

@xi@xi

as presented earlier.

Verification of Pawula’s theorem; e.g.:

D(4)ijkl = lim

�t!0

14!⇥t

(2�⇥t)2h⇥i⇥j⇥k⇥li = �ijkl�2

6lim

�t!0⇥t = 0

2prop.


Part II - Summary

stochastic model for individual realizations (e.g. Lagrangian fluid elements)

structure of joint PDF/MDF transport equation: Fokker-Planck equation

coefficients in Fokker-Planck equation from stochastic model

moment equations from PDF equation

rigorous mathem

atical theory

typically involves additional closure assumptions, but useful for comparison, analysis and model development


Part III: Explanation of Concepts with Concrete Example

• concepts and advantages discussed in more depth !

• illustrative example: gas dynamics stochastic model for molecular motion in a monatomic gas => corresponding MDF equation for molecule locations and velocities => consistent set of conservation laws for mass, momentum and energy => discussion and results !

• goal: understand how different descriptions (stochastic model, MDF equation, moment equations) are linked and develop some intuition

dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt


considered: monatomic gas flow

position of a molecule: X(t)velocity of a molecule: M(t)

physical space: x

velovity space: V

gas velocity: U(x, t) = �M(t)|x⇥

model: continuous stochastic process

Part III - Stochastic Model for Molecular Dynamics


Part III - Stochastic Model for Molecular Dynamics


dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt


Part III - PDF/MDF Equation


PDF: fX,M (x,V ; t)

total mass: M(t)

MDF: F(x,V , t) =M(t)fX,M (x,V ; t)

gas density: �(x, t) =

RR3 F(x,V , t)dV

gas velocity: U(x, t) =

1�(x,t)

RR3 V F(x,V , t)dV

sensible energy: es(x, t) =

12�(x,t)

RR3(Vi � Ui(x, t))2F(x,V , t)dV

dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt


Part III - PDF/MDF Equation


�F�t

= ��Dx

i

F�x

i

� �DV

i

F�V

i

+�2Dxx

ij

F�x

i

�xj

+�2DV V

ij

F�V

i

�Vj

+�2DxV

ij

F�x

i

�Vj

specifically:

MDF equation:

⇥F⇥t

+ Vi⇥F⇥xi

+⇥FiF⇥Vi

=⇥

⇥Vi

✓1�

(Vi � Ui)F◆

+⇥2

⇥Vi⇥Vi

✓2es

3�F

◆

| {z }S(F)

dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt


with the weights

conserved quantities:

MDF equation (Fokker-Planck):

Part III - Constraints

⇥F⇥t

+ Vi⇥F⇥xi

+⇥FiF⇥Vi

=⇥

⇥Vi

✓1�

(Vi � Ui)F◆

+⇥2

⇥Vi⇥Vi

✓2es

3�F

◆

| {z }S(F)


Part III - Moment Equations

independent of S


Part III - Moment Equations

independent of S

dependent on S


Chapman-Enskog Expansion => Navier-Stokes

Part III - Consistency with Continuum Equations


(H. Gorji and P. Jenny, Journal of Computational Physics, 2014)

•Kn: 0.05 •speed of lid: 100 m/s •wall temperature: 300 K •all the walls are diffusive

Part III - Results: Micro Cavity

temperature contours together with heat flux vectorsDSMC FP



Knudsen paradox

Part III - Results: Micro Channel


(P. Jenny, M. Torrilhon and S. Heinz, Journal of Computational Physics, 229, 2010)

velocity slip PDF at left wall

PDF at center

Kn = 0.044

Kn = 5.3



profiles for Kn = 0.2




Part III - Results: Shock in N2

FP (H. Gorji and P. Jenny, Phys. Fluids, 25, 2013)

DSMC (I. D. Boyd, Phys. Fluids, 3, 1991)

vibrational T translational T


Part III - Summary

intuitive stochastic model for molecules; honor constraints!

conservation of mass, momentum, energy; independent of S

limit => Navier-Stokes

good results despite approximations; much more general than NS

improvements possible by adding more DOFs

continuous process to model discontinuous behavior


Part IV: Numerical Solution

• numerical difficulties and simplifications - why particle methods

• particle in cell method - estimation of moments and interpolation - particle time stepping

• hybrid FP/DSMC algorithm and further topics

• examples

Part IV - Dimensionality => Particle Method

⇥F⇥t

+ Vi⇥F⇥xi

+⇥FiF⇥Vi

=⇥

⇥Vi

✓1�

(Vi � Ui)F◆

+⇥2

⇥Vi⇥Vi

✓2es

3�F

◆

| {z }S(F)


model: dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt


model: dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt

Part IV - Dimensionality => Particle Method


x

Estimation and Interpolation of Moments in Physical Space

in many applications it is not required to explicitly “extract” the MDF one is rather interested to “extract” expectations of functions of xs at locations xp => grid and kernel function need to be 1D, 2D or 3D (and not nD)

�̂

Part IV - Particle in Cell Method

⇥̄(xp, t) ⇡ 1�̂

npX

k=1

⇣m(k)�̂

⇣X

p(k)(t)� x

p⌘⌘

eQ(xp, t) ⇡ 1�̂ ⇥̄(xp, t)

npX

k=1

⇣Q

⇣X

(k)(t), t⌘

m(k)�̂⇣X

p(k)(t)� x

p⌘⌘

⇥̄�+1(xp) ⇡ µ⇥̄�(xp) +1� µ

�̂

npX

k=1

⇣m(k)�̂

⇣X

p(k)(t)� x

p⌘⌘

(⇥̄ eQ)�+1(xp) ⇡ µ(⇥̄ eQ)�(xp) +1� µ

�̂

npX

k=1

⇣Q

⇣X

(k)(t), t⌘

m(k)�̂⇣X

p(k)(t)� x

p⌘⌘

eQ�+1(xp) ⇡ (⇥̄ eQ)�+1(xp)⇥̄�+1(xp)


Statistical and Bias Error Reduction

reduction of statistical and bias errors can be achieved by employing exponentially weighted moving time averaging, i.e. use

instead of:


⇥̄(xp, t) ⇡ 1�̂

npX

k=1

⇣m(k)�̂

⇣X

p(k)(t)� x

p⌘⌘

eQ(xp, t) ⇡ 1�̂ ⇥̄(xp, t)

npX

k=1

⇣Q

⇣X

(k)(t), t⌘

m(k)�̂⇣X

p(k)(t)� x

p⌘⌘


Particle Time SteppingPart IV - Particle in Cell Method

model for rarefied gas dynamics:

dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt


Particle Time Stepping

dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt


model for rarefied gas dynamics:

problem: unphysical cooling! (P. Jenny, M. Torrilhon, S. Heinz)


Particle Time SteppingPart IV - Particle in Cell Method

dXi = Midt

dMi = �1�

(Mi � Ui) dt +✓

4es

3�

◆1/2

dWi + Fidt

homogeneous adiabatic test case:

2nd order discretization:(P. Jenny, M. Torrilhon, S. Heinz)



goal: statistically exact scheme for constant U and es for any time step size !without loss of generality: derivation for Ui=Fi=0, i.e. consider solution of

dXi = Midt

dMi = �1�

Midt +✓

4es

3�

◆1/2

dWi

first for the velocity only:

with


=>


Part IV - Particle in Cell MethodParticle Time Stepping

for velocity:


internal energy:


(P. Jenny, M. Torrilhon, S. Heinz)






for position:


=>

=>



for position:







correlation:

=>




correlation:










performance:



flow around cylinder (Kn = 0.1):Knudsen paradox (Kn=5.1157):




Part IV - Particle in Cell MethodHybrid FP/DSMC Algorithms - Further Topics

Chapter 2. Theory

2. Theory

2.1. The Range of Applicability of Mathematical Models for Rarefied Gas Flows

Gas flows can be described mathematically either in terms of continuous fields of macroscopicvariables—i.e., stream velocity, temperature, density, etc.—or on the molecular level. The con-ventional mathematical model of gas flows based on continuous fields is provided by the Navier-Stokes (NS) equations. However, the Navier-Stokes equations become invalid as soon as gradientsbecome so steep that their characteristic lengths are of the order of the mean free path, � [4,§1.2]. The range of applicability of the NS equations is thus best described in terms of thelocal Knudsen number Kn, defined as Kn = �/L, where L is a reference length implied by themagnitude of local gradients, e.g. L = ⇥ [d⇥/dx]�1. Figure 2.1 illustrates the range of applica-bility of di✏erent mathematical models for gas flows. Any simulation method applicable to thewhole range of the Knudsen number must therefore be based on the molecular description. Themathematical model based on this description is the Boltzmann equation, which describes thestatistical evolution of the fraction of molecules in a given element of the position-velocity phasespace. Consequently, the new hybrid algorithm is based on the Boltzmann equation. Here andhenceforth we only treat monatomic, simple gas flows.

Figure 2.1.: The Knudsen number limits on the mathematical models [4, Fig. 1.1].

2

accuracy:

efficiency:

FPDSMC [G. A. Bird, 1963, 1994 (NTC)]

FP originally suggested by [C. Cercignani, 1988]

DSMC

(H. Gorji, S. Küchlin, P. Jenny)



(H. Gorji, P. Jenny)




Part IV - Summary

particle MC scale linearly with dimensions

challenges: noise, bias error, time stepping, number control

hybrid algorithms cover whole Kn-range

errors: spatial discr., extract., interpol., part. time step., statistical and bias, number control

statistically steady state is much easier

PDF/MDF methods

deterministic quantitative predictions are not always feasible


Final Summary

deterministic quantitative predictions are not always desired

either due to huge scale differences, “chaos” or uncertainty in the input data

one may rather be interested in joint PDFs

determine probability space, evolution equation and closure thereof

General Motivation


stochastic model for individual realizations (e.g. Lagrangian fluid elements)

structure of joint PDF/MDF transport equation: Fokker-Planck equation

coefficients in Fokker-Planck equation from stochastic model

moment equations from PDF equation

rigorous mathem

atical theory

typically involves additional closure assumptions, but useful for comparison, analysis and model development

Final SummaryMathematical Theory


intuitive stochastic model for molecules honor constraints!

conservation of mass, momentum, energy independent of S

limit =>Navier-Stokes

good results despite approximations much more general than NS

improvements possible by adding more DOFs

continuous process to model discontinuous behavior

Final SummaryIllustration: Rarefied Gas Dynamics


Thanks!!!

Questions?!!

Contact: [email protected]!

Documents

Transported Probability and Mass Density Function (PDF… · Transported Probability and Mass Density Function (PDF/MDF) Methods for Uncertainty Assessment and Multi-Scale Problems