Stochastic Response Surface Approximation Using (Bayesian ... · Stochastic Response Surface Approximation Using (Bayesian and Fuzzy) Hierarchical Models Dr. Dan M. Ghiocel Email:

DOWNLOAD FROM GP TECHNOLOGIES, INC., www.ghiocel-tech.com1

HCF Conference, New Orleans, LO, March 8HCF Conference, New Orleans, LO, March 8--11, 200511, 2005

Stochastic Response Surface Approximation Stochastic Response Surface Approximation Using (Bayesian and Fuzzy) Hierarchical ModelsUsing (Bayesian and Fuzzy) Hierarchical Models

Dr. Dan M. GhiocelDr. Dan M. GhiocelEmail: dan.ghiocel@ghiocelEmail: [email protected]

Phone: 585Phone: 585--641641--03790379

Ghiocel Predictive Technologies Inc.Ghiocel Predictive Technologies Inc.


To discuss some of advanced stochastic response approximation tools that can useful for engine applications.

The proposed hierarchical models are capable of accurately approximating complex stochastic response problems.

The use of hierarchical approximation models is very efficient when employed in conjunction with (i) stochastic ROM and for (ii) RBDO analysis.

Objective of this PresentationObjective of this Presentation


1. Basic Computational Stochastic Mechanics Issues

2. Stochastic Response Approximation- Two-Level Hierarchical Stochastic Models- Three-Level Hierarchical Stochastic Models

3. Application Examples

4. Concluding Remarks

Content PresentationContent Presentation


Two Major Stochastic Modeling Aspects:Two Major Stochastic Modeling Aspects:

1. Develop Accurate Stochastic Approximation Models for High-Complexity Behavior Given Sample Dataset (Data-based Stochastic ROM) - Global and Local Accuracy in Statistical Data Space – ROM in Data Space- For both System Inputs and Outputs

2. Develop Fast Stochastic Simulation Models Given the Physics (PDE) and Stochastic Inputs (Physics-based Stochastic ROM) - Global and Local Accuracy in Physical Space – ROM in Physical Space- For System Outputs

3. Combine 1) and 2) to build efficient Stochastic ROMs for stress predictions

1.1. Basic Stochastic Mechanics IssuesBasic Stochastic Mechanics Issues


2. Stochastic Response Surface Approximation2. Stochastic Response Surface Approximation

Implicit Formulation: Using joint PDF estimation of

y

jx −

y

jx −

==)(f),y(f)y(f

xxx

T],y[z x=

dy)y(yf]y[E xx ∫∞

∞−

=

--- Least-square fitting ( is explicit)

--- Stochastic Neural Networks ( is implicit)Decomposes overall complex JPDF in localized simple JPDFs.

y

y

Limited

Refined

)()(ry xxx ε+=

Solution is obtained by stochastic interpolation

Explicit Formulation: Using function approximation via nonlinear regression

Convergence: Minimizing Mean Square Error (in Mean Square sense)Causal relationship. Implicit assumption of Gaussian variations.

)(r]y[E xx =

Convergence: Using Maximumum Likelihood Function (in Probability sense)Non causal relationship. No implicit assumption of Gaussian variations.

SecondSecond--Order (SO) Approximation of Stochastic FieldsOrder (SO) Approximation of Stochastic Fields

HighHigh--Order (HO) Approximation of Stochastic FieldsOrder (HO) Approximation of Stochastic Fields

Defined by stochastic vector or field


))(z)(u(g)(z)(u),(ui

iii

ii ∑∑ θ=θ=θ xxx

SO Stochastic Field ExpansionSO Stochastic Field ExpansionA Non-Gaussian (translation) stochastic field can be expanded:

(1) Original Space Expansiondx ),x(u )x(u)(z

Dii ∫ θ=θCompute Non-Gaussian Variables:

(2) Transformed Space Expansiona. Transform Original Field in A Gaussian Imageb. Perform Expansion in Gaussian Image Space c. Back-Transform to Non-Gaussian Original Space

Non-Gaussian GaussianStaticMappingCase (1) Case (2)

)(z)(u)(z)(),(u i

n

0iiii

n

0ii θ=θΦλ=θ ∑∑

==

xxx

Example: KL/PCA Expansion KL/PCAEigenDecompositionREDUCED STOCHASTIC SPACE


TwoTwo--Level (Bayesian) Hierarchical ModelsLevel (Bayesian) Hierarchical Models

1L denoted one computational layer ( stochastic FEA)

1L 1L 1L

Symbolic Representation

Input Output

∑=k

kkji )h(fw)x,x(f

jxix1h

2h

jxix1h

2h

∑=k

kklji )h(fw)x,x,x(f3D

2DLocal JPDF ModelsLocal JPDF Models


Comparison between RBF and Local PPCA ExpansionsComparison between RBF and Local PPCA Expansions

Local PPCA ExpansionLocal PPCA ExpansionRBF ExpansionRBF Expansion


Best Fitted Fitted

Best Fitted, Simpler Best Fitted, More Refined

Model Fitting (Estimation Problem)Model Fitting (Estimation Problem)

Model Selection (Evidence Problem)Model Selection (Evidence Problem)

Optimize ModelParameters Given Data

Select Most Plausible ModelGiven Data

Stochastic Model Fitting and Selection Stochastic Model Fitting and Selection


-1-0.5

00.5

11.5

-1

0

1

20

100

200

300

400

X1

Mean Surface actual

X2

F(X

1,X

2)

-1

0

1

2 -1-0.5

00.5

11.5

0

100

200

300

400

500

600

X2

2D Surface at 90% Probability after smoothing

X1

F(X

1,X

2)

-1-0.5

00.5

1

0100

200300

400500

0

2

4

6

X1

F(Y) at a given X=0

Y

F(Y

)

50% Probability Surface50% Probability Surface

IntegrationIntegration)G(f x

),G(f x

90% Probability Surface90% Probability Surface

Decomposition in Local Stochastic ModelsDecomposition in Local Stochastic Models Joint and Conditional Joint and Conditional PDFsPDFs ComputationComputation

Computation of ProbabilityComputation of Probability--Level Response SurfacesLevel Response Surfaces


Stochastic vs. Fuzzy Approximation Stochastic vs. Fuzzy Approximation

Stochastic ApproximationStochastic Approximation

Priors are assumed to be independent standard Gaussian PDF

Stochastic 2L HM Approximation

∑∑∏

∏∑

=

= =

=

=

==M

1jM

1ji

ji

n

1ii

ij

i

n

1ii

j

M

1jjj

)x(fa

)x(fay)x(by)x(f

Fuzzy BF ApproximationSingleton fuzzifier, Gaussian membership functions, center average defuzzifier and product-inference rule

Fuzzy ApproximationFuzzy Approximation

∑∑∏

∏∑

=

=

=

==M

1jM

1iiii

i

jjii

j

M

1jjj

)s(f)sx(f

)s(f)sx(f)(y)(h)(yy[E xxxx]


SubstractiveSubstractive ClusteringClustering GustafsonGustafson--KesselKessel ClusteringClustering

Smoothed Response SurfaceSmoothed Response Surface Summation of Summation of UnweightedUnweighted Local Local JPDFsJPDFs for Xfor XFuzzy ClusteringFuzzy Clustering--Based Local BF ModelsBased Local BF Models


EuristicEuristic Stochastic Field Interpolation SchemesStochastic Field Interpolation Schemes

Weighted Average Constant Interpolation (WACI)Weighted Average Constant Interpolation (WACI)

[ ] )(yyE xx = =

∑

∑

=

=NC

1iii

ii

NC

1ii

)s(p)s(f

)s(p)s(f]y[E

x

x=

∑

∑

=

=NC

1iiii

iii

NC

1ii

p),(f

p),(fy

xx

xx

Weighted Average Linear Interpolation (WALI)Weighted Average Linear Interpolation (WALI)

[ ] )(yyE xx = =

∑

∑

=

=NC

1iiii

iii

NC

1ii

p),(f

p),(f)(y

xx

xxx=

∑

∑

=

=+

NC

1iiii

iii

NC

1ii

Ti

p),(f

p),(f)b(

xx

xxxa

[ ]Tp,i2,i1,i1p,i

xiy

ii ,...,,11

φφφφ

−=φ

−=+

Φa

iTiy

ii

1b zΦφ

=

ia

ib

= slopes= slopes

= intercept= intercept

[ ] )(yyE xx =∑∑

∑∑

= =

= =

µ

µn

1i

m

1jj,ij,i

n

1ij,ij,i

m

1jj,i

)(f

)(f]y[E

x

xx= )(hy j,i

n

1i

m

1jj,i x∑∑

= =

[ ] )(yyE xx =∑∑

∑∑

= =

= =

µ

µn

1i

m

1jj,ij,i

n

1ij,ij,i

m

1jj,i

)(f

)(f]y[E

x

xx= )(h)b( j,i

n

1i

m

1jj,i

Tj,i xxa∑∑

= =+


01

23

45

0

2

45

-10

0

10

20

30

40

X1X2

F(X

1,X

2)

2D Nelson Plot

NelsonNelsonSurfaceSurface

10 Local JPDF with WACI10 Local JPDF with WACI 10 Local JPDF with WALI10 Local JPDF with WALI


13 Local JPDFs plus WALI 7 Local JPDFs plus WALI

8 Local JPDFs plus WALI 14 Local JPDFs plus WALI

SubstractiveSubstractive Clustering with WALIClustering with WALINelsonNelson

RosenbrockRosenbrock


2L HM2L HM

TwoTwo--Level Hierarchical Model Versus Level Hierarchical Model Versus KriggingKrigging with 10% Noisewith 10% Noise50% Probability Surface50% Probability Surface

KriggingKrigging95% Probability Surface 95% Probability Surface


Hamiltonian MCMCHamiltonian MCMC

Dynamic MC for Stochastic RS ApproximationDynamic MC for Stochastic RS Approximation

No particle inertiaNo particle inertia With particle inertiaWith particle inertia


MCMCMCMC--Based Response Surface ApproximationBased Response Surface ApproximationSimulated Conditional Mean SurfacesSimulated Conditional Mean SurfacesData and Local ModelsData and Local Models

Particles Particles with inertiawith inertia

Remark:Remark: Confidence Intervals Depend on Local Data Density and ParticleConfidence Intervals Depend on Local Data Density and Particle Inertia! Inertia!

Larger Particle InertiaLarger Particle InertiaLower Particle InertiaLower Particle Inertia


Comparison of 2L and 3L (Bayesian) HM ModelsComparison of 2L and 3L (Bayesian) HM Models

MCMCMCMC--based 2L HM 10 based 2L HM 10 ClustClust BFBF 3L HM (10 3L HM (10 ClustClust BF with Point BF) BF with Point BF)

2L HM Point BF 2L HM Point BF ANFIS 10 Subs ANFIS 10 Subs ClustClust BFBF


3. Application Examples 3. Application Examples 72 Blade Mistuned 72 Blade Mistuned BliskBlisk Response SurfaceResponse Surface

Blade 22 Max

Max of All Blades

Blade 17 Max

DOEDOE

2L (Bayesian) Hierarchical Models

DOEDOE


5D Highly5D Highly--Nonlinear Response SurfaceNonlinear Response SurfaceSurface Sections As Functions of X1, X2 (fixed values for X3, X4Surface Sections As Functions of X1, X2 (fixed values for X3, X4, X5), X5)

Engine Compartment Pressure Response Surface Engine Compartment Pressure Response Surface

2L (Bayesian) Hierarchical Models


4. Concluding Remarks4. Concluding Remarks

1.1. Hierarchical stochastic approximation models, such as 2L and 3L Hierarchical stochastic approximation models, such as 2L and 3L HMsHMs,,are accurate tools for response surface modeling for complex stoare accurate tools for response surface modeling for complex stochasticchasticproblems. problems.

2. Using 2. Using HMsHMs, probability, probability--level response surfaces can be easily computed.level response surfaces can be easily computed.They save large computational efforts in RBDO analyses.They save large computational efforts in RBDO analyses.

3. The best stochastic approximation results were obtained using3. The best stochastic approximation results were obtained using thetheproposed 3L HM that combines a pair of two 2L proposed 3L HM that combines a pair of two 2L HMsHMs. .

4. The current practice approaches based using quadratic regress4. The current practice approaches based using quadratic regression withion withDOE sampling rules can be inadequate for complex stochastic respDOE sampling rules can be inadequate for complex stochastic responses,onses,as illustrated herein (slide 20).as illustrated herein (slide 20).

Documents

Stochastic Response Surface Approximation Using (Bayesian ... · Stochastic Response Surface Approximation Using (Bayesian and Fuzzy) Hierarchical Models Dr. Dan M. Ghiocel Email: