View
214
Download
1
Category
Tags:
Preview:
Citation preview
DATA-DRIVEN COMPUTATIONAL DATA-DRIVEN COMPUTATIONAL STATISTICS AND STOCHASTIC STATISTICS AND STOCHASTIC
TECHNIQUES FOR THE ROBUST DESIGN TECHNIQUES FOR THE ROBUST DESIGN OF CONTINUUM SYSTEMSOF CONTINUUM SYSTEMS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Nicholas Zabaras
Materials Process Design and Control LaboratorySibley School of Mechanical and Aerospace Engineering
188 Frank H. T. Rhodes HallCornell University
Ithaca, NY 14853-3801
Email: zabaras@cornell.eduURL: http://www.mae.cornell.edu/zabaras/
Keynote Lecture: Keynote Lecture: Inverse Problems, Design and Optimization SymposiumInverse Problems, Design and Optimization Symposium , Rio de Janeiro, Brazil, March 17-19, 2004, Rio de Janeiro, Brazil, March 17-19, 2004
Event –
Titanic, the worlds largest luxury liner sank on April 14, 1912
Cause – Lack of knowledge about the damaging properties of extra sulphur in steel
Phase transition in steel to a more brittle beta-phase
Inference –
Uncertainty in material properties can lead to catastrophic results
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
UNCERTAINTY IN MATERIALS – TITANIC PLIGHTUNCERTAINTY IN MATERIALS – TITANIC PLIGHT
COLUMBIA SPACE-SHUTTLE DISASTERCOLUMBIA SPACE-SHUTTLE DISASTER
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Uncertainty in modeling Uncertainty in modeling
• Oblique impact of debris and subsequent collision with Oblique impact of debris and subsequent collision with the shuttlethe shuttle
• Material properties at the operating conditions not well Material properties at the operating conditions not well investigatedinvestigated
RESULT –RESULT –
• CatastropheCatastrophe
UNCERTAINTY IN MATERIALS PROCESSESUNCERTAINTY IN MATERIALS PROCESSES
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Modeled as flow in media with variable porosity.
Only a statistical description is possible.
Macroscopically, thus need to have a stochastic
framework for analysis
Meso-scale (dendritic structures seen)
Structure of dendrites affect macroscopic quantity like porosity
Dendritic structure is a strong function of initial process conditions
Small perturbation in initial material concentrations, temperature, flow profile can significantly alter the dendritic profiles
“ Can we employ a multiscale stochastic formulation to model initial uncertainty and provide a statistical characterization for porosity”?
Typical dendritic structures obtained due to small perturbations in
initial conditions
Approaches:
Probabilistic approaches
--- e.g. Spectral stochastic methods
Statistical approaches
I) Frequentist approach
II) Bayesian approach
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Stochastic optimization and uncertainty quantification --- importance and requirements
process uncertainty
Driving force
DAQ uncertainty
initial uncertainty
Result (Y) – to - cause (θ) ?
or performancerequirements
Mathematical representation
Y = F(θ, ωi, ωs) + ωm
process uncertainty
Why stochastic optimization & uncertainty quantification ?• need for sensitivity of inverse solutions to the system uncertainties• robustness, reliability and system performance requirements• uncertainties are unavoidable and may change the dynamics of the system• need complete probabilistic description of uncertainties and design solutions
What needs to be done?• probabilistic modeling of uncertainties• uncertainty propagation (forward sensitivity)• techniques for robust design• stochastic optimization techniques• probabilistic description of inverse solution• higher order statistics computation• prediction under uncertainties
ASPECTS OF DESIGN UNDER UNCERTAINTYASPECTS OF DESIGN UNDER UNCERTAINTY
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Are PDFs of other unspecified
process conditions (due to interactions)
feasible?
Required product with desired material properties and shape with specified confidence (output PDFs)
Can we obtain the
PDFs by existent testing?
Update model PDFs and database (digital library)
Reference material data and
process conditions PDFs
Are PDFs of design variables
technically feasible?
Digital library for accessing previous simulation, experimental data
Information about uncertainly levels in materials processes
High performance computing
environment for “robust design”
PDFs of design variables and other process conditions
Yes
Yes
Yes
Interface with digital library and expert advice to modify design objectives, material models, process models
No
No
No
Digital library
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
PRESENTATION ORGANIZATION
• A spectral stochastic approach to the A spectral stochastic approach to the analysis and robust design of continuum analysis and robust design of continuum systems (work with Velamur A. Badri systems (work with Velamur A. Badri Narayanan)Narayanan)
• A Bayesian inference approach to A Bayesian inference approach to inverse/design problems in continuum inverse/design problems in continuum systems (work with Jingbo Wang)systems (work with Jingbo Wang)
• Conclusions and discussionsConclusions and discussions
Part APart A
A Spectral Stochastic Approach to the A Spectral Stochastic Approach to the Analysis and Robust Design of Analysis and Robust Design of
Continuum SystemsContinuum Systems
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
UNCERTAINTY QUANTIFICATION - PRIMERUNCERTAINTY QUANTIFICATION - PRIMER
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Probabilistic characterization of uncertaintyProbabilistic characterization of uncertainty
Models: Black box or Models: Black box or PDE-basedPDE-based
Model uncertaintyModel uncertainty
• Uncertainty in constitutive Uncertainty in constitutive modelsmodels
• Uncertainty in governing Uncertainty in governing equation parameters, equation parameters, process conditionsprocess conditions
Characterization by joint Characterization by joint probability distribution functionsprobability distribution functions
• For each uncertainty input, a For each uncertainty input, a probability distribution is probability distribution is attributedattributed
• Statistical description of Statistical description of uncertaintyuncertainty
• Suitable for most engineering Suitable for most engineering systemssystems
Uncertain quantities modeled as Uncertain quantities modeled as random variablesrandom variables and and stochastic stochastic processesprocesses
RANDOM VARIABLES - MOTIVATIONRANDOM VARIABLES - MOTIVATION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Motivating example – Tossing of a pair of dice (experiment)Motivating example – Tossing of a pair of dice (experiment)
• Sample space Sample space = set of all possible outcomes = all integers i and j such that = set of all possible outcomes = all integers i and j such that
• Define a function X = i + jDefine a function X = i + j
• X takes values inside the set X takes values inside the set BB = {2,…,12} = {2,…,12}
• Further for each value X takes we can assign a probabilityFurther for each value X takes we can assign a probability
}6,1:),{( jiji
Definition : X in the above example is an example of a real valued random variable. It is a Definition : X in the above example is an example of a real valued random variable. It is a function that maps elements in sample space to the real line with an associated probabilityfunction that maps elements in sample space to the real line with an associated probability
• If sample space is continuousIf sample space is continuous
• And the values X takes are also continuous And the values X takes are also continuous
• Then X is a continuous random variable with the following relationThen X is a continuous random variable with the following relation
)(1
)(][AX A
X dyyfdPAXP
X taking X taking values inside values inside an interval Aan interval A
Probability Probability measuremeasure
PDF of XPDF of X
STOCHASTIC PROCESSES - MOTIVATIONSTOCHASTIC PROCESSES - MOTIVATION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Definition – Probability spaceDefinition – Probability space
The sample space The sample space , the collection of all possible events in a sample space , the collection of all possible events in a sample space FF and the and the probability law probability law P P that assigns some probability to all such combinations constitute a that assigns some probability to all such combinations constitute a probability space (probability space (, , FF, , P P ))
• Some factsSome facts
A real value random variable maps the probability space to the real lineA real value random variable maps the probability space to the real line
• Stochastic processStochastic process
What if we have a system property with spatial uncertainty?What if we have a system property with spatial uncertainty?
Porous mediumPorous medium
• Random porosityRandom porosity
• Each point in space Each point in space corresponds to a random corresponds to a random porosity valueporosity value
• Porosity of a point is very close Porosity of a point is very close to the porosity of nearby point – to the porosity of nearby point – notion of correlationnotion of correlation
RANDOM VARIABLES – EXPECTATION, NORMSRANDOM VARIABLES – EXPECTATION, NORMS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• The statistical average of a function of a random variable is defined asThe statistical average of a function of a random variable is defined as
A
X dyyfygXgE )()()]([
Provided the integral exists. ‘Provided the integral exists. ‘AA’ is the subset of real line where the probability density function ’ is the subset of real line where the probability density function ffXX(y) (y) is is
positive. positive.
• Few important statistical definitionsFew important statistical definitions
• For a stochastic process For a stochastic process W(x,t,W(x,t,) )
Covariance kernelCovariance kernel
LL22 norm norm
)],','(),,([)',',,( txWtxWtxtx
Convergence in LConvergence in L22 (convergence in mean square sense) forms the backbone of spectral stochastic methods (convergence in mean square sense) forms the backbone of spectral stochastic methods
),()],,([),,(2
1
)(2txydytxWEtxW
TDTDL
DIMENSION REDUCTION - ISSUESDIMENSION REDUCTION - ISSUES
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• A majority of uncertain inputs (material data, process conditions) exhibit spatial uncertaintyA majority of uncertain inputs (material data, process conditions) exhibit spatial uncertainty
• Modeling spatial uncertainty with stochastic processes involves representing each point as Modeling spatial uncertainty with stochastic processes involves representing each point as a random variable – Computationally impossiblea random variable – Computationally impossible
• Techniques to reduce the dimensionality (represent stochastic processes with as few Techniques to reduce the dimensionality (represent stochastic processes with as few random variables as possible) ?random variables as possible) ?
Karhunen-Loeve expansionsKarhunen-Loeve expansions
Covariance kernel of the Covariance kernel of the uncertain inputsuncertain inputs
Eigen- decompositionEigen- decomposition
Use first few eigen modes Use first few eigen modes to represent the to represent the
stochastic processstochastic process
These eigen modes are These eigen modes are enough to capture around enough to capture around
90% of the covariance 90% of the covariance kernelkernel
(Visualize as 90% of (Visualize as 90% of energy in the system energy in the system since covariance is a since covariance is a
measure of kinetic measure of kinetic energy)energy)
KARHUNEN-LOEVE – BRIEF EQUATIONSKARHUNEN-LOEVE – BRIEF EQUATIONS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
n
iiii txtxWEtxW(n)
1
)(),()],,([),,(
Random Random variablesvariables
Stochastic Stochastic processprocess
MeanMean
Approximate the stochastic process as a summation involving independent random variablesApproximate the stochastic process as a summation involving independent random variables
Eigen-decomposition of covariance kernelEigen-decomposition of covariance kernel
)','(');,(')','()',',,( txytxdytxtxtx ii
TD
i
Convergent in mean square senseConvergent in mean square sense
)(),,(),,()(2
NntxWtx(n)WTDL
Most optimal of all possible lower dimensional representations of a stochastic processMost optimal of all possible lower dimensional representations of a stochastic process
KARHUNEN-LOEVE – SHORTCOMINGSKARHUNEN-LOEVE – SHORTCOMINGS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Needs apriori knowledge of Covariance KernelNeeds apriori knowledge of Covariance Kernel
- Available for uncertain inputs- Available for uncertain inputs
- Available for certain model for uncertainties (constitutive relations)- Available for certain model for uncertainties (constitutive relations)
• Covariance kernels of output response of a physical system not availableCovariance kernels of output response of a physical system not available
• Covariance kernels of certain non-Gaussian inputs highly complicatedCovariance kernels of certain non-Gaussian inputs highly complicated
Can we construct uncertainty representation Can we construct uncertainty representation schemes for output and other non-Gaussian inputs schemes for output and other non-Gaussian inputs
similar to the Karhunen-Loeve expansion?similar to the Karhunen-Loeve expansion?
n
iii
n txWtxW0
)( )(),(~
),,(
Generalized Polynomial Chaos expansionGeneralized Polynomial Chaos expansion
Approximation Approximation of stochastic of stochastic
processprocess
Chaos Chaos polynomialspolynomials
(random (random variables)variables)
POLYNOMIAL CHAOS – GAUSSIAN INPUTSPOLYNOMIAL CHAOS – GAUSSIAN INPUTS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Inputs are Gaussian random variablesInputs are Gaussian random variables
• Best representation of outputs is obtained by choosing the chaos polynomials from an Best representation of outputs is obtained by choosing the chaos polynomials from an
Hermite-chaos familyHermite-chaos family
Hermite chaosHermite chaos
First few polynomialsFirst few polynomials
n
ii
n
nn
nnH1
2
11
2
1exp)1(),,(
Related one-to-one with Related one-to-one with ii(())
1
1
212
11
0
One uncertain inputOne uncertain input
POLYNOMIAL CHAOS – NON-GAUSSIAN INPUTSPOLYNOMIAL CHAOS – NON-GAUSSIAN INPUTS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• For non-Gaussian inputs, the chaos polynomials are chosen from the Askey series of hypergeometric For non-Gaussian inputs, the chaos polynomials are chosen from the Askey series of hypergeometric
orthogonal polynomialsorthogonal polynomials
• This choice leads to optimal mean square convergenceThis choice leads to optimal mean square convergence
Chaos polynomialChaos polynomial Support spaceSupport space Random variableRandom variable
LegendreLegendre [[]] UniformUniform
JacobiJacobi [[]] BetaBeta
HermiteHermite [-[-∞,∞,∞∞]] Normal, LogNormalNormal, LogNormal
LaguerreLaguerre [0, [0, ∞]∞] GammaGamma
• Combinations of uncertain inputs can use combination of these polynomials for uncertainty Combinations of uncertain inputs can use combination of these polynomials for uncertainty representationrepresentation
• Number of chaos polynomials used to represent output uncertainty depends on Number of chaos polynomials used to represent output uncertainty depends on
- Type of uncertainty in input- Type of uncertainty in input- Distribution of input uncertainty- Distribution of input uncertainty- Number of terms in KLE of input- Number of terms in KLE of input- Degree of uncertainty propagation desired (first order, second order …)- Degree of uncertainty propagation desired (first order, second order …)
USING SPECTRAL STOCHASTIC METHODS IN FEM USING SPECTRAL STOCHASTIC METHODS IN FEM
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
g
h
DNeumann conditions specified
Temperature specified
TRANSIENT STOCHASTIC HEAT CONDUCTIONTRANSIENT STOCHASTIC HEAT CONDUCTION
)},0{,(),,(),(),0,(
),,(),,(),,(
),,(),,(),,(),,(
),,(),,()(
0
DtxxTxT
TtxtxqnT
k
TtxtxTtxT
DtxTktT
C
h
gg
Diffusion equationDiffusion equation
Dirichlet BCDirichlet BC
Neumann BCNeumann BC
Initial conditionsInitial conditions
ABSTRACT WEAK FORM IN STOCHASTIC SPACES ABSTRACT WEAK FORM IN STOCHASTIC SPACES
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Recall - Basic assumption in any spectral stochastic representation of uncertainty is that of finite varianceRecall - Basic assumption in any spectral stochastic representation of uncertainty is that of finite variance
• Heat equation requires twice differentiable temperature solutions – Strong conditionHeat equation requires twice differentiable temperature solutions – Strong condition
• Strong condition translates to once differentiable temperature field – Weak conditionStrong condition translates to once differentiable temperature field – Weak condition
Definition: Definition: The stochastic function space HThe stochastic function space H11(D) x L(D) x L22((TT) x L) x L22(() is used to denote ) is used to denote
all stochastic processes that have square integrable spatial derivatives, are all stochastic processes that have square integrable spatial derivatives, are square integrable in time and have finite variancesquare integrable in time and have finite variance
D T
dtdPxdtxu
LTLDHtxu
),,(
)()()(),,(
2
22
1
WEAK FORMULATION WEAK FORMULATION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
dtdPxdqwwb
dtdPxdwTkwt
TCwTa
where
Ttxtxw
TtxtxTtxT
wbwTa
LTLDHtxw
thatsuchLTLDHtxTFind
h T
D T
g
gg
)(
),(
)(),,(0),,(
)(),,(),,(),,(
)(),(
)()()(),,(
)()()(),,(
221
221
• Trial functions are stochastic and have a finite varianceTrial functions are stochastic and have a finite variance
• QuestionsQuestions- How to choose finite element weighting functions that are stochastic ?- How to choose finite element weighting functions that are stochastic ?- Will the resulting FE representation satisfy the basic assumptions (finite variance) ?- Will the resulting FE representation satisfy the basic assumptions (finite variance) ?
FE FORMULATION DETAILS - INTERPOLATIONFE FORMULATION DETAILS - INTERPOLATION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Spatial interpolationSpatial interpolation
Divide the domain Divide the domain into element into element
(regions)(regions)
TT11
TT22
TT33
TT44
4
1
)(),(),,(i
ii xNtTtxT
Stochastic representationStochastic representation
P
rriri tTtT
0
)()(),(
Nodal unknownsNodal unknowns Chaos polynomialsChaos polynomials
Tensor product of basisTensor product of basis
4
1 0
)()()(),,(i
P
rriir xNtTtxT
• Consider each node to be comprised of (P) unknowns in stochastic caseConsider each node to be comprised of (P) unknowns in stochastic case
FE FORMULATION DETAILS – MATVEC SYSTEMSFE FORMULATION DETAILS – MATVEC SYSTEMS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
)(
)(
)(
)()(),,(
)()()()(),,(][
)()()()(),,(][
][][
1
1
1
e
e
e
dPxdxNtxqAf
dPxdxNxNtxkAK
dPxdxNxNtxCAM
fTKTM
r
Nel
em
D
sr
Nel
emn
D
sr
Nel
emn
• Deterministic case – Each node has 1 DOFDeterministic case – Each node has 1 DOF
• Here – Each node has P+1 DOFsHere – Each node has P+1 DOFs
• m = (m = (-1)(P+1)+r, n=(-1)(P+1)+r, n=(-1)(P+1)+s-1)(P+1)+s
• The matrix system is thus (P+1) times larger than a deterministic FEM systemThe matrix system is thus (P+1) times larger than a deterministic FEM system
EXAMPLE – 2D STOCHATIC HEAT CONDUCTIONEXAMPLE – 2D STOCHATIC HEAT CONDUCTION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
T0
T0
L
(0,0)
(1,1)
)1.0,1(0 ~)1.0,1(~)( NeLogNormalT Non-Gaussian BCNon-Gaussian BC
Boundary temperature Boundary temperature specified as a random specified as a random variable distributed as variable distributed as LogNormalLogNormal
Thermal conductivityThermal conductivity )1,0(~,5.01~)( 22 Nk
Output representationOutput representation • Two uncertain inputs modeled as random variablesTwo uncertain inputs modeled as random variables
- Two-dimensional KLE for inputs- Two-dimensional KLE for inputs- Two-dimensional third order Hermite-Chaos for output- Two-dimensional third order Hermite-Chaos for output
TEMPERATURE STATISTICSTEMPERATURE STATISTICS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
MeanMean Standard deviationStandard deviation
• Mean solution reaches steady state faster than standard deviationMean solution reaches steady state faster than standard deviation
• Uncertainty has a direct effect on the arrival to steady stateUncertainty has a direct effect on the arrival to steady state
• Though input uncertainty variation in temperature is small (10% of mean) output variation in Though input uncertainty variation in temperature is small (10% of mean) output variation in temperature temperature
is about 50% of mean !!is about 50% of mean !!
Known flux specification 0
hD
Unknown flux ?
I
Temperature readings from sensor available with complete statistics along the internal boundary I
STOCHASTIC INVERSE HEAT CONDUCTIONSTOCHASTIC INVERSE HEAT CONDUCTION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
),,(),,(),,();,,(
),,(),,(),,(
),,(),,(),,(
)},0{,(),(),(),0,(
),,(),,()(
I0
00
h
in
TtxtxYqtxT
TtxtxqnT
k
TtxtxfnT
k
DxxTxT
TDtxTktT
C
Diffusion equationDiffusion equation
Initial conditionInitial condition
Known flux BCKnown flux BC
Unknown flux BCUnknown flux BC
MeasurementsMeasurements
Definition of the objective function as a norm in an appropriate function space
Define gradient of objective from directional derivative
Sensitivity of temperature with
respect to perturbation in flux
ddd)],,();,,([2
1
),,();,,(2
1)(
)T()()ˆ(
T
20
2
)T(00
02000
2
ttxYqtxT
txYqtxTqJ
LqqJqJ
I
IL
DEFINITION OF STOCHASTIC OBJECTIVE FUNCTIONDEFINITION OF STOCHASTIC OBJECTIVE FUNCTION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
)T(00'
)T(0000
02
20
)),((
))],,,();,,(([)()(
L
Lq
qqJ
TDtxYqtxTqJqqJI
)();,,();,,(2
)T(2000000
Lq qTDqtxTqqtxT
Definition of temperature sensitivity
GRADIENT OF OBJECTIVE – ADJOINT FORMULATIONGRADIENT OF OBJECTIVE – ADJOINT FORMULATION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Gradient of the objective function can be evaluated only in an distributional sense
Simplify above using integration by parts to get adjoint equations
• Solution of adjoint problem at the boundary with unknown flux specification is defined as the gradient of the objective function
• Adjoint problem is driven by the temperature residual = difference between estimated temperature and sensor measurements
0 2
2 0
0 0 0 ( T )
'0 0 ( T )
( ) ([ ( , , ; ) ( , , )], )
( ( ), )
Iq L
L
J q q T x t q Y x t D T
J q q
2 2
0
(D T ) (D T )( *( ), ) ( , ( )) 0L L
qD T
L L
Definition of the adjoint variable for evaluating gradient of the objective function
Adjoint definitionAdjoint definition
Temperature SensitivityTemperature Sensitivity
0 0 0( ) ( , , ; ), ( , , ) TJ q x t q x t
CONTINUUM STOCHASTIC SENSITIVITY METHOD- CSSMCONTINUUM STOCHASTIC SENSITIVITY METHOD- CSSM
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Definition of stochastic parameter sensitivity –Gateaux differential of the parametric stochastic field w.r.t perturbations in parameters
Observations
• Mathematically rigorous definition of a continuum stochastic sensitivity field
• Deterministic sensitivity information is contained in the direct simulation. Stochastic sensitivity is concerned with perturbations in the PDF
0 0 00 0
0
( , , ; ) ( , , ; )( , , ; , ) lim
T x t q q T x t qx t q q
Temperature Temperature sensitivity sensitivity parameterparameter
Perturbation Perturbation in PDF of in PDF of
fluxflux
CSSM – PHYSICAL INTERPRETATIONCSSM – PHYSICAL INTERPRETATION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Original PDF
Output
PDF obtained after perturbing the
design parameters
How does perturbation in the Joint probability distribution of design parameters affect the solution?
How to characterize perturbations in PDFs?
CSSM – a summary
•An accurate differentiate-then-discretize approach is used here
• The temperature field is design differentiated to obtain the sensitivity field. Since we assume continuous, finite-variance stochastic processes, this approach is valid
• Sensitivities are essentially linear, since we are interested only in infinitesimal perturbations in PDF of design variables
Key questionsKey questions
INVERSE STOCHASTIC HEAT CONDUCTIONINVERSE STOCHASTIC HEAT CONDUCTION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
g
0
DBoundary condition unknown
Temperature specified
Body under thermal loading
K,C
Unknown material properties – modeled as
stochastic fields
Temperature measurements taken in certain locations in the
body
• Measurement errors
• Environment fluctuations
Given: temperature measurements polluted with uncertainty
Estimate: The unknown boundary condition with uncertainty limits
Constraints: thermal properties of the solid are random
ROBUST STOCHASTIC HEAT CONDUCTIONROBUST STOCHASTIC HEAT CONDUCTION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
g
0
DBoundary condition unknown
Temperature specified
Body under thermal loading
K,C
Unknown material properties – modeled as
stochastic fields
Desired temperature at specific locations
with tolerable uncertainty limits
Given: desired temperature in parts of the body
Estimate: How well we should know the unknown boundary condition
Constraints: Tolerable uncertainty limits in desired temperature
• Uncertain material properties
Points to measurement and
experimental techniques to obtain
the boundary condition within
desired uncertainty limits
DEFINITION OF SUBPROBLEMSDEFINITION OF SUBPROBLEMS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
in
h
0 0
0 0
h
( ) ( , , ) (D T )
( ,0, ) ( , ) ( , ) (D {0} )
( , , ) ( , , ) ( T )
( , , ) ( , , ) ( T )
( ) ( , , ) (D T )
( ,0, ; , ) 0 ( , ) (D {0} )
0 ( , , ) (
TC k T x t
tT x T x x
Tk f x t x t
nT
k q x t x tn
C k x tt
x q q x
k x tn
I
0 0
max max
0 I
T )
( , , ) ( , , ) ( T )
( ) ( , , ) (D T )
( , , ) 0 ( , ) (D { } )
0 ( , , ) ( T )
( , , ; ) ( , , ) ( , , ) ( T )
k q x t x tn
C k x tt
x t x t
k x tnT
k T x t q Y x t x tn
h
0
DGuess flux q0
applied
Known flux applied
Direct Direct problemproblem
h
0
DFlux
perturbation
Insulated Sensitivity Sensitivity problemproblem
h
0
DInsulated
Insulated Adjoint Adjoint problemproblem
I
TRIANGULAR FLUX PROBLEMTRIANGULAR FLUX PROBLEM
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Non-Gaussian thermal conductivity
• A chi-square distribution is chosen for thermal conductivity, this ensures that k > 0
Triangular heat flux profile is used as a basic template on which unknown flux probability distributions are built
Non-dimensional time
flux
0 0.25 0.5 0.75 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
• FEM solution at x = 0.3 is used to model sensor mean temperature readings
• Measurement error (Gaussian noise) is used to pollute the solution at x=0.3
• The unknown flux is probabilistically reconstructed
• Using the estimated flux predictions are made at x=0.5
tri
2.5 , 0 0.4
2 2.5 , 0.4 0.8
0, 0.8 1
t t
q t t
t
Unknown flux
X = 0
X = L
InsulatedTemperature
sensor readings
2( ) 1 ( ( ) 1), ( ) (0,1)k N Shifted chi-squareShifted chi-square
SENSOR DATA – SOLUTION OF DIRECT SSFEM PROBLEMSENSOR DATA – SOLUTION OF DIRECT SSFEM PROBLEM
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Sensor data at x = 0.3 obtained as a solution of a direct stochastic heat conduction problem using SSFEMSensor data at x = 0.3 obtained as a solution of a direct stochastic heat conduction problem using SSFEM
• No measurement error is added to the solutionNo measurement error is added to the solution
+ + + + + + + + + + + + + ++
++
++
++
++
++
++
++
++
++
Non-dimensional time
Mea
no
ptim
alflu
x
0 0.25 0.5 0.75 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
11 iter5 iter20 iter40 iter60 iter80 iterExact
+
+ + + + + + + + + + + + + ++
++
++
++
++
++
++
++
++
++
Non-dimensional time
Fir
sto
rder
term
inP
CE
ofo
ptim
alflu
x
0 0.25 0.5 0.75 10
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.11 iter5 iter20 iter40 iter60 iter80 iterExact
+
+ + + + + + + + + + + + + + + + + ++
++
++
++
++
++
++ +
+
Non-dimensional time
Sec
on
do
rder
term
inP
CE
ofo
ptim
alflu
x
0 0.25 0.5 0.75 1-0.01
-0.008
-0.006
-0.004
-0.002
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014 1 iter5 iter20 iter40 iter60 iter80 iterExact
+
Mean optimal fluxMean optimal flux
• Converges in 10 Converges in 10 iterationsiterations
• Exact mean is Exact mean is reconstructedreconstructed
First order PCE termFirst order PCE term
• Converges in 30 Converges in 30 iterationsiterations
• captures most of captures most of standard deviation in standard deviation in fluxflux
Second order PCE termSecond order PCE term
• Converges in 80 Converges in 80 iterationsiterations
• Nearly zero – optimal Nearly zero – optimal flux is also nearly flux is also nearly GaussianGaussian
SENSOR DATA – DETERMINISTIC SOLUTION + NOISESENSOR DATA – DETERMINISTIC SOLUTION + NOISE
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Sensor data at x = 0.3 obtained as a solution of a deterministic heat conduction problemSensor data at x = 0.3 obtained as a solution of a deterministic heat conduction problem
• Gaussian measurement noise is added to the temperature solution to obtain simulated temperature Gaussian measurement noise is added to the temperature solution to obtain simulated temperature readingsreadings
0 0.25 0.5 0.75 10
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0 0.1 0.2-0.05
-0.025
0
0.025
0.05
0 0.1 0.2-0.05
-0.025
0
0.025
0.05
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Non-dimensional timeM
ean
flux
and
first
PC
Ete
rm0 0.25 0.5 0.75 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 MeanFirst PCE term+
+
+
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Non-dimensional time
Mea
nflu
xan
dfir
stP
CE
term
0 0.25 0.5 0.75 1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 MeanFirst PCE term+
Mean Mean temperature temperature
readingsreadings
Large noise levelLarge noise level Small noise levelSmall noise level
• Estimation is closely related to the accuracy of temperature readings
• Unlike deterministic inverse problems, large error in measured data does not affect estimation of mean flux
TEMPORALLY CORRELATED MEASUREMENTSTEMPORALLY CORRELATED MEASUREMENTS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Sensor data at x = 0.3 obtained as a solution of a direct stochastic heat conduction problem using SSFEMSensor data at x = 0.3 obtained as a solution of a direct stochastic heat conduction problem using SSFEM
• Measurement is considered to be a Gaussian process with mean equal to deterministic solutionMeasurement is considered to be a Gaussian process with mean equal to deterministic solution
• The covariance function of the Gaussian process is defined asThe covariance function of the Gaussian process is defined as
1 2 1 2Cov( , ) max 1 / ,0 , 0t t t t b b
• Covariance function is by definition positiveCovariance function is by definition positive
• Individual sensor readings are generated as realizations of the Gaussian processIndividual sensor readings are generated as realizations of the Gaussian process
•The realizations constitute the available dataThe realizations constitute the available data
• The sensor readings are now used to reconstruct the Karhunen-Loeve expansion The sensor readings are now used to reconstruct the Karhunen-Loeve expansion
mean1
( , , ) ( ) ( ) ( )N
i ii
T d t T t T t
Sensor Sensor locationlocation
Mean Mean sensor sensor
readingsreadings
Instrument Instrument standard standard deviationdeviation
I.I.D I.I.D GaussiansGaussians
TEMPORALLY CORRELATED MEASUREMENTS - RESULTSTEMPORALLY CORRELATED MEASUREMENTS - RESULTS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
t
q+ tp
0 0.25 0.5 0.75 1-0.1
-0.05
0
0.05
0.1
Order 1Order 2Order 3Order 4
t
q+ tp
0 0.25 0.5 0.75 1
0
0.25
0.5
0.75
1
t
q+ tr
i
0 0.25 0.5 0.75 1
-0.01
0
0.01
0.02
0.03
0.04 Order 1Order 2Order 3Order 4
t
q+ tr
i
0 0.25 0.5 0.75 1
0
0.25
0.5
0.75
1
Higher order PCE termsHigher order PCE termsMean Mean fluxflux
Mean fluxMean flux Higher order PCE termsHigher order PCE terms
tri
0, 0 0.1
2.5 0.25, 0.1 0.5
2.25 2.5 , 0.5 0.9
0, 0.9 1
t
t tq
t t
t
pulse
0, 0 0.1
1.0, 0.1 0.4
0, 0.4 0.6
1.0, 0.6 0.9
0, 0.9 1
t
t
q t
t
t
STOCHASTIC DESIGN – DIFFERENT PROBLEMSSTOCHASTIC DESIGN – DIFFERENT PROBLEMS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
High density regions
Low density regions - tails
Reliability type optimization
• Optimization to prevent rare probability events
• Failure implies a catastrophe e.g.- Titanic, Columbia shuttle.
• Requires definition of a failure surface – highly nonlinear
• Optimization in bounded possibly non-convex surfaces
Robust optimization
• Optimization to prevent performance deterioration due to fluctuations in inputs
• Failure implies loss of performance
•Requires definition of a robust objective based on physics of the problem
• Completely stochastic optimization
FINITE Vs INFINITE DIMENSIONAL OPTIMIZATIONFINITE Vs INFINITE DIMENSIONAL OPTIMIZATION
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Finite dimensional
• Introduce parametric representations for design variables
• Robust design statement now is a parametric optimization problem in finite dimensional space
• Extent of parameterization governs accuracy
Infinite dimensional
• Design variables are considered as functions and are not parameterized priory
• Leads to a function space optimization problem
• Corresponds to an infinite dimensional parameterization, thus require regularizationApproaches for
robust design of continuum systems
Parameter spaceFinite-dimensional optimization framework can be derived as a
special case.
Part BPart B
A Bayesian Computational Statistics A Bayesian Computational Statistics Approach to Stochastic Optimization and Approach to Stochastic Optimization and Uncertainty Quantification in Continuum Uncertainty Quantification in Continuum
System Design/ControlSystem Design/Control
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Fundamentals of Bayesian statistical inferenceFundamentals of Bayesian statistical inference
• Classical statistics and Bayesian statistics Classical: study the probability of a hypothesis in restricted circumstances Bayesian: study the probability of a hypothesis both unconditionally (its prior probability) and given some “evidence” (its posterior probability)
)|( YθP)(
)()|(
YP
θPθYP
• Essence of Bayesian inference - interested in values of unknown quantities θ=[θ1, θ2, … θk]T
- suppose have some ‘a priori belief’ about P(θ) - obtained some data Y=[Y1, Y2, … Yn]T relevant to θ Find out an expression to incorporate both prior beliefs and the data
)|()(1
)|( YppC
Yp
Priori pdf
Likelihood Posterior pdf
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
• Bayes’ formula
• An example: Y ~ N(θ, σ2), θ ~ N(θo, σo2)
θ|Y ~ N(θ1, σ12)
22
2
22
2
1o
o
oo y
22
2221
o
o
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
The LikelihoodThe Likelihood
The role of likelihood is to incorporate distribution information from measurementdata
Y = F(θ) + ω
For a typical system as,
the likelihood is determined by the distribution of ω, e.g. whenω ~ N(0, σ2) :
FYθp T
2))((
2
1exp{)|(
Y FY ))}((
Symbolically, the likelihood tries to “filter” the prior belief of θ.
Likelihood is the conditional probability of observation Y given the parameter θ.
It is important to realize likelihood as an interface to data (Bayesian inference is a data driven model)
little data some data more data
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
The prior distributionThe prior distribution
• role of a prior pdf --- incorporate known to a priori information --- regularize the likelihood• a prior can be a normal pdf or an “improper” pdf• techniques of prior distribution modeling --- accumulated distribution information --- conjugate prior distributions --- physical constraints --- local uniforms --- spatial statistics models
A prior distribution is the unconditional belief of certain unknowns (hypothesis)before the related observations (evidence) are achieved.
A decision should be based on what is known and the evidence --- the priordistribution makes Bayesian the most rational inference approach
An example of conjugate prior:if Y|θ,σ ~ N(θ, σ), then θ ~ N(θo, σo), σ ~ inv-Gamma (a,b).
A class π of prior distributions is said to form a conjugate family if the posterior density p(θ|X) is in the class for all X whenever the prior density is in π.
spatial models are of special importance for stochastic continuum systemse.g. Gauss random fields (GRF), Markov random fields (MRF), …
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Various Bayesian formulationsVarious Bayesian formulations
* A hierarchical structure
)()(),|()|,( ppYpYp
)()|(),|()|,( ppYpYp
)()|( pYp )|( p Ym Yo, )|( Yop
usually no knowledge of hyper-parameters, this formulation can diminish theeffect of poor knowledge on hyper-parameters
* An augmented formulation
provides a complete probabilistic description to system uncertainties
* An Expectation-Maximization formulation
more robust formulation (iterative regularization) when there are missing data
Bayesian is an adaptive model --- posteriors can be treated as priors for new data
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Spatial statisticsSpatial statistics
A popular model: Markov Random Field (MRF)
• has an exponential form• explores the spatial and temporal dependence• closely related to Tikhonov regularization
}))((exp{)(~
ji jiijWp
Some basic facts:• analyzing spatially indexed data• exploiting the spatial locations of data• studying spatial phenomena existing throughout continuous spatial regions but are only observed at a finite sample of locations (originates from statistical physics)
Advantages for prior distribution in continuum system• regularize the inverse problem by exploiting the spatial dependence --- a physically more rational approach• data are usually available at a few locations• efficient multi-scale modeling
Updating of spatial distribution or exploiting of the posterior distribution requires efficient and accurate numerical tools
All joint distributions can be represented as an MRF, so do MCMC algorithms
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Markov Chain Monte Carlo simulation (MCMC)Markov Chain Monte Carlo simulation (MCMC)
What’s attractive: The Metropolis algorithm, an instance of MCMC, has been ranked among the ten algorithms that have had the greatest influence on the development and practice of science and engineering in the 20th century (Beichl &Sullivan, 2000)
MCMC motivationsMCMC motivations::Integration and optimization problems in large dimensional spaces, which Integration and optimization problems in large dimensional spaces, which play a fundamental role in machine learning, physics, statistics…play a fundamental role in machine learning, physics, statistics…* Bayesian inference and learnin* Bayesian inference and learningg (a) (a) NormalizatioNormalization: to obtain the posterior p(n: to obtain the posterior p(x x |y) given the prior p(x) and likelihood |y) given the prior p(x) and likelihood
p(p(y y |x)|x) (b) (b) MarginalizatioMarginalization: given the joint posterior of x and z, to obtain marginal posteriorn: given the joint posterior of x and z, to obtain marginal posterior (c) (c) Conditional expectationConditional expectation* Statistical mechanics* Statistical mechanics* Optimization* Optimization* Penalized likelihood model selection* Penalized likelihood model selection
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Markov Chain Monte Carlo simulation (MCMC)Markov Chain Monte Carlo simulation (MCMC)
Monte Carlo PrincipleMonte Carlo Principle1. draw an i.i.d. set of samples {1. draw an i.i.d. set of samples {x x (i)} I=1:(i)} I=1:NN from a target density p(x) from a target density p(x)
2. approximate the target density with the following empirical point-mass function2. approximate the target density with the following empirical point-mass function
3. approximate the integral (expectation) I(3. approximate the integral (expectation) I(ff) with tractable sums I) with tractable sums INN( ( f f ))
N
ixN x
Nxp
i1
)(1
)(
N
iX
NiN dxxpxffIxf
NfI
1
)()()()(1
)(
Sampling strategiesSampling strategies1. Rejection sampling:1. Rejection sampling:
sample from a distribution p(x) by sampling from
another easy-to-sample proposal distribution q(x) when
p(x) <=Mq(x), M is a constant (Robert &Casella, 1999)
2. Importance sampling:2. Importance sampling:
3. MCMC3. MCMC
)()()(ˆ1
iN
i
iN xxwxp
N
i
ii xwxffI1
)()()(ˆ
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Markov Chain Monte Carlo simulation (MCMC)Markov Chain Monte Carlo simulation (MCMC)MCMC is a strategy for generating samples MCMC is a strategy for generating samples xxii while exploring the state space X using a Markov while exploring the state space X using a Markov chain mechanism. This mechanism is constructed so that the chain spends more time in chain mechanism. This mechanism is constructed so that the chain spends more time in the most important regions. the most important regions.
Concept of Markov ChainConcept of Markov Chainr.v. r.v. xx є є X ={xX ={x11 x x22 ..., ...,xxss }. The stochastic process }. The stochastic process
xxii is called a Markov chain if is called a Markov chain if p(xp(xii| | xxi-1i-1 ,..., ,..., xx11) = ) = T(xT(xii| | xxi-1i-1). ).
The chain is homogeneous if T remains invariant for The chain is homogeneous if T remains invariant for
all i, with sum of all i, with sum of T(xT(xii| | xxi-1i-1) for all i is one. A chain is ) for all i is one. A chain is
stable if p(x) converges to a invariant distribution in stable if p(x) converges to a invariant distribution in
spite of initial state. A chain is stable as long as the spite of initial state. A chain is stable as long as the
transition (T) satisfies transition (T) satisfies irreducibilitirreducibility and y and aperiodicitaperiodicity.y.
MCMC samplers are irreducible and aperiodic Markov chains that have the
target distribution as the invariant distribution. To design these samplers, the
detailed balance has to be satisfied.
)|()()|()( 111 iiiiii xxTxpxxTxp
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Metropolis – Hastings (MH) algorithm --- Metropolis – Hastings (MH) algorithm --- Basic from of MCMCBasic from of MCMC
An MH step of invariant distribution p(x) and proposal distribution q(An MH step of invariant distribution p(x) and proposal distribution q(xx**|x) involves|x) involves
sampling a candidate value sampling a candidate value xx** given the current value given the current value x x according to q(according to q(xx**|x). |x).
Initialize xInitialize x00
For i=0:N-1For i=0:N-1
sample u~U(0,1)sample u~U(0,1)
sample sample xx** ~ ~ q(q(xx**|x|xii) )
if u < A(xif u < A(xii, , xx**)=min)=min{1, p(x{1, p(x**)q(x)q(xii|x|x**)/(p(x)/(p(xii)q(x)q(x**|x|xii))}))}
xxi+1i+1=x=x**
else xelse xi+1i+1=x=xii
Some properties of MH
(a) The normalizing constant of the target distribution is not required.
(b) It is easy to simulate several independent chains in parallel.
(c) The success or failure of the algorithm often hinges on the choice of proposal distribution.
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Extensions of MH algorithmExtensions of MH algorithm(a) Independent sampler: q(x*|xi) =q(x*). (b) Metropolis algorithm: q(x*|xi) =q(xi|x*).
(c) Simulated annealing
Initialize x0 and set T0=1For i=0:N-1 sample u~U(0,1) sample x* ~ q(x*|xi)
if u < A(xi, x*)=min{1, p1/Ti(x*)q(xi|x*)/(p1/Ti(xi)q(x*|xi))}
xi+1=x*
else xi+1=xi
set Ti according to cooling design
(d) Mixtures and cycles of MCMC kernels
Initialize x0
For i=0:N-1 sample u~U(0,1) if u<a apply MH with proposal K1 (global proposal) else apply MH with proposal K2 (local proposal)
(e) Cycles of kernels
Initialize x0
For i=0:N-1 - sample the block xi+1
b1 according to proposal distribution q1(xi+1
b1|xi+1-b1, xi
b1) and target distribution p(xi+1
b1|xi+1-b1)
- sample the block xi+1b2 according to proposal
distribution q1(xi+1b2|xi+1
-b2, xib2) and target
distribution p(xi+1b2|xi+1
-b2) . . - sample the block xi+1
bs according to proposal distribution q1(xi+1
bs|xi+1-bs, xi
bs) and target distribution p(xi+1
bs|xi+1-bs)
(f) Gibbs sampler
)|(~ 11
ijj
ij xxpx
Initialize x0
For i = 0:N-1 For j = 1:m sample
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Stochastic inverse problemStochastic inverse problem
• Parameterization of unknown quantity q
m
iii txwq
1
),( discrete unknown θ
• System input and output relation
Input qInput q Direct numericalSolver F
Direct numericalSolver F Measurement YMeasurement Y
Y = F(θ,ωs) + ωm
)()|()(
)()|()|( pY pY p
pY pY p
)()(),|()|,( ppYpYp
)()|(),|()|,( ppYpYp
simple form
augmented model
hierarchical model
)()|( pYp )|( p Ym Yo , )|( YopEM model
(missing data)
• Bayesian formulation
• Prior distribution modeling
• provides estimate with associated probability bounds• quantifies uncertainty in noise• provides solution at various accuracy levels• captures modes of nonlinear problems• collects more samples from higher density regions• works for implicit likelihood
• allows for various prior models• accumulated knowledge• exploits spatial relation of unknowns
Advantages of Bayesian approachAdvantages of Bayesian approach
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Bayesian formulation and regularization theory
A simple example:
--- Deterministic estimator with zeroth order Tikhonov regularization
--- Bayesian interpretation
MAP estimator
})(21{minˆ 22
2 pLLS YFaug
TWFYYp T2 )
21
exp(}))((2
1exp{)|(
FY ))(( PPDF
}2
])([])([21{minˆ
2 WYFYFaug TT
MAP
LS and MAP estimates are identical taking α=λσ2/2 and W identity matrix
Some key literatures on Bayesian inference of continuum systems
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• C. Ferrero, K. Gallagher, Stochastic thermal history modeling. 1. Constraining heat flow histories
and their uncertainty, Marine and Petroleum Geology 19 (2002) 633-648.
• B. DeVolder, J. Glimm, J.W. Grove, Y. Kang, Y. Lee, K. Pao, D.H. Sharp, K. Ye, Uncertainty
quantification for multiscale simulations, Journal of Fluids Engineering 124 (2002) 29-41.
• D. Higdon, H. Lee, Z. Bi, A Bayesian approach to characterizing uncertainty in inverse problems
using coarse and fine-scale information, IEEE Transactions on Signal Processing 50(2) (2002)
389-399.
• H. K. H. Lee, D. M. Higdon, Z. Bi, M. A.R. Ferreira and M. West, Markov random field models for
high-dimensional parameters in simulations of fluid flow in porous media, Technometrics 44 (3)
(2002).
• A.F. Emery, Stochastic regularization for thermal problems with uncertain parameters, Inverse
Problems in Engineering 9 (2001) 109-125.
• T. D. Fadale, A. V. Nenarokomov, A. F. Emery, Uncertainties in parameter estimation: the inverse
problem, Int. J. Heat Mass Transfer 38(3) (1995) 511-518.
• Anna M. Michalak, Peter K. Kitanidis, A Method for Enforcing Parameter Nonnegativity in
Bayesian Inverse Problems with An Applicaiton to Contaminant Source Identification, {\em Water
Resour. Res.}, 39(2), 1033, doi:10.1029/2002WR001480, 2003.
Γo
Γg
Γh
* ***
****
** unknown heat flux
known temperature
known heat flux
thermocouples
),( TktTC P
in ,
,),( gTtxT ,g
],0[ maxtt
,),(
hqn
txTk
on
,h),()0,( 0 xTxT
,),(
0qn
txTk
)(unknown ,0
],0[ maxtt
on ],0[ maxtt
in
on ],0[ maxtt
Inverse Heat Conduction Problem (IHCP)
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Bayesian formulation for IHCP --- A parametric approach
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• Parameterization of unknown heat flux q0
m
iii txwq
10 ),( Unknown vector θ
Input θInput θ direct numericalsolver F
direct numericalsolver F
Measurement YMeasurement Y
simulationnoise
numerical error
Y = F(θ) + ω
• System input and output relation
random
• Likelihood function
FYθp T2
))((2
1exp{)|(
Y FY ))}(( --- known σ
--- unknown σ FYθ,p T2
))((2
1(σ2)-n/2exp{σ2)|(
Y FY ))}((
Assumptions• numerical error much less then measurement noise• ω iid ~ N(0, σ2)
Bayesian formulation for IHCP --- A parametric approach
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Markov Random Field (MRF)
}))((exp{)(~
jijiijWp
}2
exp{)( 2/ Wp Tm
2
21
)( uu
else
ji
jin
Wi
ij ~
,0
,1
,
Prior distribution modeling
--- Single layer posterior:
--- Augmented posterior:
TWFYYp T2 )
2
1exp(}))((
2
1exp{)|(
FY ))((
)exp(
(σ2)-n/2exp{)|,(
Yp
TW
FY T2
)2
1m/2exp(
}))((2
1
FY ))((
• Gibbs sampler
Y = Hθ + Yi + ω
• Modified Gibbs sampler --- use Gibbs sampler for θ --- take full conditional maximum of σ2
Gibbs sampler and modified Gibbs sampler
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
1
2i
i ii i
b
a a
2
2 21 1
2N N
si s sii ii i p
s s
H Ha W b
s s st t p ji j ik kt i j i k i
Y H W W
),(~| 2iiii N
)|(~ 11
ijj
ij xxpx
Initialize x0
For i = 0:N-1 For j = 1:m sample
xq
dL
Y (d,iΔt)
--- True q in simulationq
0 0.4 0.8
1.0
--- Normalized governing equation
2
2
xT
tT
1t 0 ,0 1x
0),0( xT 1x 0
0
LxxT
)(0
tqxT
x
,
,
, 1t 0
1t 0,
t1.0
--- Discretization of q(t)
θiθi-1 θi+1
dt
1D IHCP example
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
--- Case 1, 2, 3 d = 0.3 Δt = 0.02 (n=50) dt = 0.04 (m=26) σ = 0.001, 0.005, 0.010 (2.5% Tmax)
MLE estimate Posterior mean
1D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
1D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
--- Effect of --- Effect of σσ
True σ 12
0.21 0.31 0.41 0.51 0.610
4
8
q (t = 0.16)
0.80 0.90 1.000
4
8
12
1.10
q (t = 0.40)
0.25 0.35 0.45 0.550
4
8
12
q (t = 0.64)
Marginal PDFs
True 95% probability bounds
1D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Guess of 2σ
Marginal PDFs
0.1 0.2 0.3 0.40.5 0.6 0.70
2
4
6
q (t = 0.16)
0.6 0.7 0.8 0.9 1.0 1.1 1.20
2
4
6
q (t = 0.40)
0.10.20.3 0.40.50.6 0.70
2
4
6
q (t = 0.64)
Gets wider
1D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
--- Temperature prediction at d=0.5
Unknown σ
95% probability bounds using augmented model
• Normalized governing equations
,2
2
2
2
fy
T
x
T
t
T
,10 x ,10 y ,0t
,1 xx
TT ,
1 yyTT
,0t
,0
xx
qx
T
,
0
y
y
qy
T
,0t
,0TT t = 0.
• What is the problem
Know f, α, T0, Tx and Ty, reconstruction qx and qy through temperature measurements
2D IHCP example
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
T(t=0)=T0
T=
0
T=0
• True quantities in simulation
f = 0, α = 1, Tx = 0, Ty = 0, T0 = 2•sin(πx)•sin(πy),
qx = 2•π•sin(πy)•exp(-2π2t), qy = 2•π•sin(πx)•exp(-2π2t), This problem has an analytical solution in the form of T = 2•sin(πx)•sin(πy)•exp(-2π2t)
• Temperature measurements arecollected at 13 sites *
**
*
**
* * * * ** *
2D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
true qx
--- Case I ! 13 thermocouples ! 25 measurement steps ! σ = 5.0e-3 (1% ΔTmax) ! α = 5.0e-5
MLE estimate of qx Posterior mean estimate of qx
true qyMLE estimate of qyPosterior mean estimate of qy
Relative error is 28.76%
2D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Relative erroris 4.63%
--- Case I ! 13 thermocouples ! 25 measurement steps ! σ = 5.0e-3 (1% ΔTmax) ! α = 5.0e-5
-1.0 0.0 1.00.0
1.0
2.0
q (0, 0, 0.005) -2.5 -1.5 -0.5
0.0
0.6
1.2
q (0, 0.0625, 0.005) -4.0 -3.0 -2.0
0.0
0.8
1.6
q (0, 0.1875, 0.005)
-5.5 -4.5 -3.50.0
0.8
1.6
q (0, 0.25, 0.005) -6.5 -5.5 -4.5
0.0
0.8
1.6
q (0, 0.375, 0.005)
q (0, 0.125, 0.005) -3.5 -2.5 -1.5
0.0
0.4
0.8
q (0, 0.3125, 0.005) -5.5 -3.5 -1.5
0.0
0.4
0.8
q (0, 0.4375, 0.005) -7.2 -5.7 -3.2
0.0
0.4
0.8
q (0, 0.5, 0.005) -7.5 -5.5 -3.5
0.0
0.4
0.8
q (0, 0.5625, 0.005) -7.5 -5.5 -3.5
0.0
0.4
0.8
-6.5 -5.5 -4.00.0
0.8
1.6
q (0, 0.625, 0.005)
-6.5 -4.5 -2.50.0
0.4
0.8
q (0, 0.6875, 0.005) -5.1 -4.1 -3.1
0.0
0.8
1.6
q (0, 0.75, 0.005) -4.2 -3.2 -2.20.0
0.8
1.6
q (0, 0.8125, 0.005)
-3.2 -2.2 -1.20.0
0.8
1.6
q (0, 0.875, 0.005) -2.5 -1.5 -0.5 0.5
0.0
0.8
1.6
q (0, 0.9375, 0.005) -1.0 0.0 1.0
0.5
1.5
2.5
q (0, 1.0, 0.005)
Marginal PDFs of
qx at t=0.005
2D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
true qxMLE estimate of qx Posterior mean estimate of qx
true qyMLE estimate of qyPosterior mean estimate of qy
Relative erroris 5.45%
Relative error is 33.19%
--- Case II ! 13 thermocouples ! 25 measurement steps ! σ = 1.0e-2 (2% ΔTmax) ! α = 2.5e-4
2D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
true qxMLE estimate of qx Posterior mean estimate of qx
true qyMLE estimate of qyPosterior mean estimate of qy
Relative erroris 5.73%
Relative error is 35.92%
--- Case III ! 13 thermocouples ! 25 measurement steps ! σ = 2.0e-2 (4% ΔTmax) ! α = 5.0e-4
2D IHCP example …
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Inverse Heat Radiation Problem
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
S(gray
boundary)thermocouple
participatingmedia
Vheat source
*
**
*
*
What g(t) causes measured T?
2 ( ) ( )p r
TC k T g t G x x y y z zq
t
4( ) ( )
4 bs I I I r d Is
4b
b
TI
4
14 ( ( ) )
4br I I r s dq
0
1( ) ( ) 0b n s
I r s I n I r d n ss s
wT T
Bayesian inverse formulation
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
θi-1
dt
neighbors of θi
t
ghi
θi θi+1
• discretization of g(t)
• Likelihood
• Prior
• Posterior
1
ˆ ( ) ( )m
i ii
g t h t
2 2
1 ( ( )) ( ( ))( ) exp{ }
(2 ) 2
T
n nT T
Y F Y Fp Y
2 1( ) exp( )
2m Tp W
TWFYYp T2 )
2
1exp(}))((
2
1exp{)|(
FY ))((
Direct simulation
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
A finite element (FE) + S4 method framework
• Ordinate discretization
• Weak formulation of FEM
• algorithm
0
10
j
j ii b j jn s
I I n w I ns s
,
*)*,*,()(
)1(
)()(
WdvTC
Wdvzzyyxxtgqt
dvWTktWdvTC
i
p
rV
i
V
i
pV
24
1
.~
4
~
~~
ijjVbV
iViiV
dvWwIdvWI
dvWIdvWIs
24
1
( ) ( )4
i i i j j bj
I I I r w Is
1. Set T(i)guess = T(I-1);
2. Substitute T(i)guess to compute Ib ;
3. Solve intensity eq for I(i);4. Compute ;5. Solve temperature eq to update T(i)
guess;6. If the solution converged, set T(i)
guess
as T(i) and save I(i); otherwise, go to 2;7. Go to the next time step.
rq
Reduced order modeling --- A POD based approach
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• homogeneous part of the direct problem
• inhomogeneous part of the direct problem
• eigenfunction problem
• reduced order models
2 0Ik T
4( ) ( )
4I I I I
bs I I I r d Is
4( )II bb
TI
0
1( ) 0I I
b n sI I n I r d n ss s
IwT T atS
2 ( ) ( )h
hp r
TC k T g t G x x y y z zq
t
4( ) ( )
4h h h I
b bs I I I r d I Is
0
1( ) 0h h
n sI n I r d n ss s
0hT atS
( ) ( )
1
1 eNi i
Vie
U U dvN
1
( ) ( ) ( )TK
h Ti i
i
T t r a t r
1
( ) ( ) ( )IK
h Ii i
i
I t r s b t r s
1
( ) 1TK
jj ji i j j T
i
daM H a S Q g t j K
dt
1 1
1I IK K
ji i ji i j Ii i
A b B b D j K
2( )T
j p jVM C dv
T Tji j iV
H k dv
Tj jrV
S dvq
( )Tj jV
Q G x x y y z z dv 4
{ ( ) }I I I Iji i j i jV
A s d dv
4 4{( ) }I I
ji i jVB d d dv
4
( )I Ij b b jV
D I I d dv
MCMC algorithm --- a cycle design of single component update
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
• implicit likelihood MH sampler• increasing acceptance probability single component update
Algorithm:
mimic the structure of Gibbs sampler
A cycle of symmetric MH samplers
symmetric MH sampler( ) ( 1) ( ) ( ) ( )
2
1 1( ) exp{ ( )}
22i i i
j j j j j jqjqj
q
},...,,,...,,{ )()(1
)1(1
)1(2
)1(1
)1( im
ij
ij
iiij
Initialize Initialize θθ00
For i=0:N-1For i=0:N-1
For j=1:mFor j=1:m
sample u~U(0,1)sample u~U(0,1)
sample sample θθ**jj ~ ~ qqjj((θθ **
jj | | θθ ii
-j-j , , θθ ii
jj ) )
if u < A(if u < A(θθ iijj , , θθ **
jj ))
θθi+1i+1==θθ**
else else θθi+1i+1==θθii
A testing example
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
x
z y
1m
1m
1m
g(t)(0.5m, 0.5m, 0.5m)
800K
800K
800K
800K
800K
O
800K
***
12
3
o t
g(t)
400kW/m3
0.05s0.01s 0.04s
80kW/m3
o t
g(t)
0.02s 0.04s 0.05s
160kW/m3
80kW/m3
g1(t)
g2(t)
Schematic of the example
Profile of testing heat sources
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Basis fields
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004 Frame 001 09 Mar 2004
1st, 3rd and 6th basis of Th
1st, 3rd and 6th basis of Ih
along direction s =[0.9082483 0.2958759 0.2958759]
1st, 3rd and 6th basis of Ih
along direction s =[-0.9082483 0.2958759 0.2958759]
Homogeneous temperature solution
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
Th computed by full model Th computed by reduced order model
Comparison of reduced order solutions atthermocouple locations
Heat source reconstruction
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Materials Process Design and Control Laboratory
MAP estimates of g1 at different magnitude of noise
MAP estimates of g2 at different magnitude of noise
Posterior mean of g1 when σT =0.005
Posterior mean of g2 when σT =0.005
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Computational (Bayesian & spatial) statistics approach --- advances and obstacles
Design/control variablesor unknown parameters (g) • thermal conditions (heat flux)• mechanical force• external field force• chemical reactions• processing speed•other mechanisms
Outliers
Get rid of polluted data
Data
min
ing
Uncertainty modeling
Continuum system:
• thermal transfer• material deformation• fluid flow …
Observed data orstate variable requirements
distribution estimate
Statistical formulationof objective:• direct simulation• likelihood modeling• statistical approach• loss function • stochastic regularization• computational considerations
point estimate
Bayesian & spatial
• estimates with probabilities• quantify uncertainty in noise• solution at various accuracy level• complete probabilistic description
Advantages:• data driven in nature• simulation in deterministic space• stochastic regularization (spatial models)• global optimization
Obstacles:• lack of accurate distribution information• computation cost
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
probabilistic modeling• uncertainty nature• physical constraints• statistical learning• model selection/validation
Prior Learning
Bayesian formulation
Posterior exploration
prior distribution modeling• accumulated information• conjugate family• non-informative priors• physical constraints• spatial statistical models MRF, kringing …
)()|()(
)()|()|( pY pY p
pY pY p
)()(),|()|,( ppYpYp
)()|(),|()|,( ppYpYp
simple form
augmented model
hierarchical model
likelihood computation• system equations• boundary conditions• numerical simulation (FEM, FD, FV, SN, …)• multi-length scale simulation• parallel implementation
asymptotic study• simulated annealing• genetic algorithms
gradient optimization• conjugate gradient• steepest descent gradient sampling strategies
• importance sampling & rejection sampling• MCMC - Metropolis-Hasting - symmetric sampler - hybrid & cycle - reverse jump - sequential MCMC
reduced-order modeling(POD)• Galerkin formulation• eigenvalue problem
)()|( pYp )|( p Ym Yo , )|( YopEM model
(missing data)
system uncertainty propagation study
key uncertainties measurement data mining• outlier detection• instrument illustration• probabilistic modeling•hypothesis testing
• perfect samplingdigital library
Machine learning(SVM)
model selection
A typical Bayesian framework on stochastic inverse problems
Conclusions and discussionConclusions and discussion
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
ASPECTS OF ANALYSIS UNDER UNCERTAINTYASPECTS OF ANALYSIS UNDER UNCERTAINTY
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Statistical information
available based on prior experiments
No prior statistical
information
Assume prior distributions for
input PDFs
Bayesian Analysis
Assume PDFs of input based on
analysis of physics of the problem and
by experience
Input PDFs
Update prior PDFs based on
experimental data
Karhunen-Loeve Karhunen-Loeve decompositiondecomposition
High High performance performance computingcomputing
Digital Digital databasedatabase
User User interfaceinterface
Analysis Analysis toolboxtoolbox
Refine the input Refine the input stochastic modelstochastic model
Stochastic continuum models governed by partial
differential equations
Discrete data based “Black-Box” type input-output
models
SSFEM based analysis Monte Carlo analysis
accelerated by spectral stochastic
expansions
Input resolution enough
to attain output convergence?
ExitExit
ROBUST DESIGN – DATA TO DESIGN MODELSROBUST DESIGN – DATA TO DESIGN MODELS
Materials Process Design and Control LaboratoryMaterials Process Design and Control Laboratory
CCOORRNNEELLLL U N I V E R S I T Y
CCOORRNNEELLLL U N I V E R S I T Y
Stochastic design toolbox
Data from tests and experiments
Filtering data in two-pass Bayesian approach, Spectral stochastic input modeling
Stochastic forward
uncertainty propagation
Highly correlated data
Bayesian, MCMC, Markovian models
Loosely correlated data
Spectral stochastic, VMS, Support space models
Completely data driven model
Bayesian inverse statistical inference,
MCMC, MH algorithms
Analytical design model, robust design objectives
Spectral stochastic design optimization, Continuum
stochastic sensitivity method
Meta models
Partially data driven models with spectral
stochastic design methodologies
Robust design solutions
Optimal material data, process conditions with
PDFs
Testing design under expected operating
conditions
Duty cycle feedback
Bayesian post design inference
Duty cycles, operational feedback
Post design
concurrent updation
Point to optimal testing and data collection
Point to model inadequacies, correct input stochastic PDFs
Design feedback
Recommended