PhD Preliminary Oral Exam CHARACTERIZATION AND PREDICTION OF CFD SIMULATION UNCERTAINITIES

Ph.D Preliminary Oral Exam, Aerospace and Ocean Engineering Department, February 27 th 2002 1

Characterization and Prediction of CFD Simulation Uncertainties

PhD Preliminary Oral Exam

CHARACTERIZATION AND PREDICTION OF CFD SIMULATION UNCERTAINITIES

by

Serhat Hosder

Chair: Dr. Bernard Grossman

Committee Members:

Dr. Raphael T. Haftka Dr. William H. Mason

Dr. Reece Neel Dr. Rimon Arieli

Department of Aerospace and Ocean Engineering

Virginia Tech.

Blacksburg, VA



Outline of the Presentation

• Introduction

• Classification of CFD Simulation Uncertainties

• Objective of the Present Work

• Previous Studies

• Transonic Diffuser Case

• Results, findings and discussion about different sources of uncertainty

• Conclusions



Introduction (1)• The Computational Fluid Dynamics (CFD) as an

aero/hydrodynamic analysis and design tool

• Increasingly being used in multidisciplinary design and optimization (MDO) problems

• Different levels of fidelity (from linear potential solvers to RANS codes)

• CFD results have a certain level of uncertainty originating from different sources

• Sources and magnitudes of the uncertainty important to assess the accuracy of the results



Introduction (2)

Drag Polar Results for DLR F-4 Wing at M=0.75, Rec=3x106 (taken from 1st AIAA Drag

Prediction Workshop (DPW), Ref. 1)



Classification of CFD Simulation Uncertainties

• Physical Modeling Uncertainty– PDEs describing the flow (Euler, Thin-Layer N-S, Full N-S, etc.)– Boundary and initial conditions (B.C and I.C)– Auxiliary physical models (turbulence models, thermodynamic

models, etc.)

• Uncertainty due to Discretization Error– Numerical replacement of PDEs and continuum B.C with

algebraic equations– Consistency and Stability of PDEs– Spatial (grid) and temporal resolution

• Uncertainty due to Iterative Convergence Error



Definition of “Uncertainty” and “Error”

• Oberkampf and Trucano (Ref. 2) defined

Uncertainty as a potential deficiency in any phase or activity of modeling process that is due to the lack of knowledge (uncertainty of turbulence models, geometric dimensions, thermo-physical parameters, etc.)

Error as a recognizable deficiency in any phase or activity of modeling and simulation

• Discretization errors can be estimated with certain methods by providing certain conditions

• In this work, we’ll refer the inaccuracy in the CFD simulations due different sources as “uncertainty”



Objective of the Present Work

• Characterize different sources of CFD simulation uncertainties– Consider different test cases

– Apply different grids, solution schemes/parameters, and physical models

• Try to quantify/predict the magnitude and the relative importance of each uncertainty

• Compare the magnitudes of CFD simulation uncertainties with other sources of uncertainty (geometric uncertainty, uncertainty in flow parameters, etc.)



Previous Studies

• Previous CFD related studies mainly focused on discretization and iterative convergence error estimations– Grid Convergence Index (GCI) by Roache (Ref. 3)– Discretization Error of Mixed-Order Schemes by C. D. Roy (Ref. 4)

• Trucano and Hill (Ref. 5) proposed statistical based validation metrics for Engineering and Scientific Models

• Hemsch (Ref. 6) performed statistical analysis of CFD solutions from 1st AIAA DPW.

• Kim (Ref. 7) made statistical modeling of simulation errors (from poorly converged optimization runs) and their reduction via response surface techniques



Description of Transonic Diffuser Test Case (1)

• Known as “Sajben Transonic Diffuser” case in CFD Validation studies

• Top wall described by an analytical equation

• Although geometry is simple, the flow-field is complex.

• The Shock strength and the location determined by exit-pressure-to-inlet-total pressure ratio Pe/P0i

• Pe/P0i=0.72 (Strong shock case), Pe/P0i=0.82 (Weak shock case),



Description of Transonic Diffuser Test Case (2)

Mach contours for the weak shock case

Mach contours for the strong shock case



Simulation Code, Solution Parameters, and Grids (1)

• General Aerodynamic Simulation Program (GASP)– 3-D, structured, multi-block, finite-volume, RANS code

• Inviscid fluxes calculated by upwind-biased 3rd (nominal) order spatially accurate Roe-flux scheme

• All viscous terms were modeled (full N-S)

• Implicit time integration to reach steady-state solution with Gauss-Seidel algorithm



Simulation Code, Solution Parameters, and Grids (2)

• Flux-Limiters – Van Albada’s limiter

– Min-Mod limiter

• Turbulence Models– Spalart-Allmaras (Sp-Al)

– k-w (Wilcox, 1998 version)

• Grids Generated by an algebraic mesh generator – Grid 1 (g1): 41x26x2

– Grid 2 (g2): 81x51x2

– Grid 3 (g3): 161x101x2

– Grid 4 (g4): 321x201x2

– Grid 5 (g5): 641x401x2 (Used only for Sp-Al, Min-Mod, strong shock case)

• y+= 0.53 (for g2) and y+= 0.26 (for g3) at the bottom wall



Output Variables (1)

Nozzle efficiency, neff

H0i : Total enthalpy at the inlet

He : Enthalpy at the exit

Hes : Exit enthalpy at the state that would be reached by isentropic

expansion to the actual pressure at the exit

esi

eieff HH

HHn

0

0

ydyhyuyH oi

y

i

i

)()()(0

0

ydyhyuyH e

y

e

e

)()()(0

ydyhyuyH es

y

es

e

)()()(0

1

0

)()(

i

eoipes P

yPTcyh

th

yy

Throat height



Orthogonal Distance Error, En

A measure of error in wall pressures between the experiment and the curve representing the CFD results

Output Variables (2)

exp

1

exp

N

dE

N

ii

n

2exp

2 ))()(()(min icixxxi xPxPxxdexitinlet

Pc : Wall pressure obtained from CFD calculations

Pexp: Experimental Wall Pressure Value

Nexp: Total number of experimental points (Nexp=36)

di: Orthogonal distance from the ith experimental data point to Pc(x) curve



Uncertainty due to iterative convergence error (1)

• Normalized L2 Norm Residual of the energy equation for the case with Sp-Al turbulence model, Van-Albada and Min-Mod limiters at the strong shock case.

• Same convergence behavior with respect to the limiters observed for the k-w case.




Poor L2 norm convergence does not seem to effect the convergence of the neff results at different grid levels




nexacteff

neff

neff ntnn )()(

ntn e

n

neff

nneff

exacteff

nnn

1)(

1

1

1

neff

neff

neff

neffn

nn

nn

n

neff

neffn

nn

1

1

1

1

100%neff

nneff

neff

neffn

eff nn

nnnoferror

Roy and Blottner (Ref. 8) proposed a method to estimate, the iterative convergence error at time level (cycle) n

Assuming exponential decrease for

Need three time levels in the exponential region

where

ntn e

ntn e



Uncertainty due to discretization error (1)

For each case with a different turbulence model, grid level (resolution) and the flux-limiter affect the magnitude of the discretization error

• The effect of the limiter observed at grid levels g1 and g2

• At grid levels g3 and g4, the effect is much smaller




• Richardson’s extrapolation method:

)( 1 ppexactk hOhff

h: a measure of grid spacing

p: The order of the method.

• Assumptions needed to use Richardson’s method:

– Grid resolution is in the asymptotic region

– The order of the spatial accuracy, p should be known. Usually observed order of spatial accuracy is different than the nominal value. The observed order should be determined.

– Monotonic grid convergence. Mixed-order schemes can cause non-monotonic convergence. Roy (Ref. 4) proposed a method for for the discretization error estimate of mixed-order schemes.



Uncertainty due to discretization error (3)turb.

model limiter Pe/P0i p (neff)exact

grid level

discretization error (%)

g1 9.820

g2 4.505

g3 1.562 Sp-Al Van Albada 0.72 1.528 0.71830

g4 0.542

g1 14.298

g2 6.790

g3 2.716 Sp-Al Min-Mod 0.72 1.322 0.71590

g4 1.086

g1 6.761

g2 3.507

g3 1.528 Sp-Al Van Albada 0.82 1.198 0.80958

g4 0.666

g1 8.005

g2 3.539

g3 1.185 Sp-Al Min-Mod 0.82 1.578 0.81086

g4 0.397

g1 3.514

g2 1.459

g3 0.370 k-w Van Albada 0.82 1.980 0.82962

g4 0.094

g1 4.432

g2 1.452

g3 0.461 k-w Min-Mod 0.82 1.656 0.82889

g4 0.146




• “p” values are dependent on the grid levels used

• However the difference between the (neff)exact values are small

compared to overall uncertainty

turb. model

limiter Pe/P0i grid

levels used

p neff_exact grid level

discretization error (%)

g1 14.405734

g2 6.940532

g3 2.812698

g4 1.139865

Sp-Al Min-Mod 0.72 g2, g3, and g4

1.303 0.715235

g5 0.729023

g1 13.728739

g2 6.307712

g3 2.204303

g4 0.5413695

Sp-Al Min-Mod 0.72 g3, g4, and g5

2.026 0.719492

g5 0.1329585



• The uncertainty due to discretization error is bigger for the cases with strong shock compared to the weak shock results at each grid level. The flow structure has significant effect on the discretization error.

• For the monotonic cases, largest errors occur for the Sp-Al, Min-Mod, strong shock case and the smallest errors are obtained for the k-w , Van-Albada, weak shock case

• Non-monotonic convergence behavior for the cases with k-w and the strong shock as the mesh is refined








• Noise due to discretization error observed at grid levels 1 and 2.

• Noise error small compared to the systematic discretization error between each grid level. However, this can be important for gradient-based optimization.

• Kim (Ref. 7) successfully modeled the the noise error due to poor convergence of the optimization runs by fitting a probability distribution (Weibull) to the error.

• The noise error can be reduced via response surface modeling.



Uncertainty due to turbulence models (1)

• Uncertainty due to turbulence modeling (in general physical modeling) should be investigated after estimation of the discretization and iterative convergence error.

• Difficult to totally separate physical modeling errors from discretization errors

• “Validation” of the Engineering and Scientific Models deals with accuracy of the physical model

• Need high-quality experimental data



• Orthogonal distance error, En is used for comparison

of CFD results with the experiment


En for each case is scaled with the maximum value

obtained for k-w , Min-Mod, strong shock case




For each case (strong shock or weak shock), best match with the experiment is obtained with different turbulence models at different grid levels




• Experimental uncertainty should be considered

• With the experimental geometry, a perfect match with CFD and experiment can be observed upstream of the shock

• Upstream of the shock, discrepancy between CFD simulations and experiment is most likely due to the experimental uncertainty



Uncertainty due to turbulence models (5)• A better way of using En for this example would be to

evaluate it only downstream of the shock

• The discretization and iterative convergence error should be estimated for En in a similar way used for the nozzle

efficiency

• An estimate of exact value of (En ) can be used for

approximating the uncertainty due to turbulence models

• The relative uncertainty due to the selection of turbulence models can also be investigated by using (neff)exact values

obtained by Richardson’s extrapolation



Uncertainty due to turbulence models (6)• Hills and Trucano (Ref. 5) proposed a Maximum Likelihood based

model validation metric to test the accuracy of the model predictions

• Uncertainty in the experimental measurements and the model parameters are considered– Model parameters:

• Material properties• Geometry• Boundary or Initial Conditions

• This method requires prior knowledge about the measurement and the model parameter uncertainty (modeling with probabilistic distributions)

• Looks for statistically significant evidence that the model validations are not consistent with the experimental measurements




• PDF(d) : PDF of measurement vector occurrencePDF(p) : PDF of model parameter vector occurrence

• PDF(d, p) = PDF(d) x PDF(p)

• Find the maximum likely values for the mode of the measurements d and the model parameters p– Find the maximum value of Joint PDF via optimization

• Evaluate the probability of obtaining a smaller PDF assuming that the model is correct

• If this value is bigger than the level of significance that we assumed for rejecting a good model, than the model predictions are consistent with the experiment




• Possible application to test the accuracy of the turbulence models – Takes into account the experimental uncertainty

– Requires prior knowledge of uncertainty in the measurements and the model parameters

– Selection of model parameters

– No simple relationship with the model parameters and the output quantities. Using response surface techniques may be needed to find a functional form.



Additional Test Cases

• Need more cases to generalize the results obtained in Transonic Diffuser Case

• Next possible case : Steady, turbulent, flow around an airfoil (RAE2822 or NACA0012)– Consider transonic and subsonic cases

– Consider a range of AOA

– Output quantities to monitor: Cl, Cd, Cp distributions

– Orthogonal distance error may be used for characterizing Cp distributions

• Consider a case with a more complex geometry



Conclusions (1)

• Different sources of uncertainty in CFD simulations should be investigated separately.

• Discretization and iterative convergence errors can be estimated by certain methods in certain conditions

• Limiters affect the iterative convergence and the discretization error. – L2 norm convergence affected by the use of different limiters

– Poor L2 norm convergence do not seem to affect the neff results

• Asymptotic Grid convergence hard to obtain

• Flow structure has a strong effect on the magnitude of the discretization error.

• Iterative convergence error small compared to the discretization error

• Uncertainty due to turbulence model should be investigated after the estimation of discretization and iterative convergence error.



• Comparison with the experiment is needed to determine the accuracy of the turbulence models

• Experimental uncertainty should be considered possibly by using a statistical method

• More cases need to be analyzed to generalize the results

Conclusions (2)



References

1. Levy, D. W., Zickuhr, T., Vassberg, J., Agrawal S., Wahls. R. A., Pirzadeh, S., Hemsch, M. J., Summary of Data from the First AIAA CFD Drag Prediction Workshop, AIAA Paper 2002-0841, January 2002

2. Oberkampf, W. L. and Trucano, T. G., Validation Methodology in Computational Fluid Dynamics. AIAA Paper 2000-2549, June 2000

3. Roache, P. J. Verication and Validation in Computational Science and Engineering.Hermosa Publishers, Albuquerque, New Mexico, 1998.

4. Roy, C. J., Grid Convergence Error Analysis for Mixed-Order Numerical Schemes, AIAA Paper 2001-2606, June 2001

5. Hills, R. G. and Trucano, T. G., Statistical Validation of Engineering and Scientific Models: A Maximum Likelihood Based Metric, Sandia National Loboratories, SAND2001-1783

6. Hemsch, M. J., Statistical Analysis of CFD Solutions from the Drag Prediction Workshop, AIAA Paper 2002-0842, January 2002

7. Kim, H., Statistical Modeling of Simulation Errors and Their Reduction Via Response Surface Techniques, PhD dissertation, VPI&SU, June 2001

8. Roy, C. J. and Blottner F. G., Assesment of One-and Two-Equation Turbulence Models for Hypersonic Transitional Flows, Journal of Spacecraft and Rockets, Vol.38, No. 5, September-October 2001

Documents

PhD Preliminary Oral Exam CHARACTERIZATION AND PREDICTION OF CFD SIMULATION UNCERTAINITIES