Overview of CFD Verification and Validation

ALI ELRASHID ALI BADI

CFD Verification and Validation

Introduction

The overall objective is to demonstrate the accuracy of CFD codes One should first understand the distinctions between a code, simulation, and model.

Code: A code is a set of computer instructions and data inputs and definitions. A code typically has three stages related to the level of validation completed: research, pilot, and production. The production code fully validated for the intended design applications, including a system-level validation.

Model: A model is defined as a representation of a physical system or process intended to enhance our ability to understand, predict, or control its behavior.

Essentially, one implement a model into a computer code and then uses the code to perform a CFD simulation, which yield values, used in the engineering analysis. Verification and validation examines the errors in the code and simulation results.

Credibility (An improvement of credibility is considered to be the same as confidence building or providing quality to the customer) is obtained by demonstrating acceptable levels of uncertainty and error (A recognizable deficiency in any phase or activity of modeling and simulation that is not due to lack of knowledge).

The levels of uncertainties and errors are determined through verification assessment and validation assessment.

Verification assessment determines if the programming and computational implementation of the conceptual model is correct. It examines the mathematics in the models through comparison to exact analytical results. Verification assessment examines for computer programming errors.

Simulation: A simulation is defined as the exercise or use of a model. (That is, a model used in a simulation). For a CFD analysis, the application or run of the CFD code is a simulation

Use of CFD Results

The level of accuracy required from a CFD analysis depends on the desired use of the results. A conceptual design effort may be content with general shock structure information, whereas a detailed design may require accurate determination of the pressure recovery. Each quantity to be determined generally has its own accuracy requirement. Levels of credibility may vary according to the information required.

The application of CFD for design and analysis may be catagorized into three levels according to increased levels of required accuracy: 1) provide qualitative information, 2)

2

provide incremental quantities, and 3) provide absolute quantities. This discussion follows the ideas of Benek et al.

Provide qualitative information. CFD provides details on the entire flow field not possible with experimental methods. This is useful in understanding on a qualitative level the behavior of the flow field. Accuracy requirements are low.

Provide incremental quantities. Corrections to experimental observations can be provided by CFD at a higher accuracy than existing with the CFD method. This is due to cancellation of part of the error when taking the differences. For example, if there is a design change from a baseline, for which the quanity is known, Pbaseline, the quantity P for the design change can be expressed as

P = Pbaseline + dP

Where dP is the increment in P corresponding to the design change. If two CFD simulations are performed, the first with the baseline geometry and the second with the modified geometry, then the increment in P due to the modified geometry can be estimated as,

dP = ( P + E )2 - ( P + E )1 = dPactual + dE.

The E is the error associated with the quantity P obtained from the CFD simulation. As can be seen, the error to the increment is dE, which cancels out some of the error.

Provide absolute quantities. This level involves determining absolute values of the quantity P and requires the highest level of accuracy. The accuracy required is usually stated as part of the design process. The accuracy observed from the CFD simulation varies according to the character of the quantity, and so, it is not possible to state an accuracy or error band that applies to all quantities obtained from the CFD simulation. The verification methods discussed below regarding a grid convergence study will provide the error band for the calculations.

Verification Assessment

Verification is “The process of determining that a model implementation accurately represents the developer's conceptual description of the model and the solution to the model”

Verification assessment examines 1) if the computational models are the correct implementation of the conceptual models, and 2) if the resulting code can be properly used for an analysis. The strategy is to identify and quantify the errors in the model implementation and the solution. The two aspects of verification are the verification of a code and the verification of a calculation. The objective of verifying a code is error evaluation, which is, finding and removing errors in the code. The objective of verifying a calculation is error estimation, which is determining the accuracy of a calculation. Each is discussed below.

3

Verification has also been described as solving the equations right. It is intended to concern itself more with mathematics rather than engineering. It is intended to look for errors in the programming and implementation of the models.

Roache considers two aspects of verification: 1) verification of a code and 2) verification of a calculation. These two aspects are described below:

Verification of a code involves error evaluation, which is, looking for bugs, incorrect implementations of conceptual models, errors in inputs, and other errors in the code and usage. This is typically done by the developers prior to release of the code. First, consistency checks are performed which examine basic relationships expected in the solutions (i.e. mass conservation). Then the code is used to simulate a suite of ``highly accurate'' verification cases. These cases should be analytic or numeric solution to ordinary and partial differential equations. Verification should not be performed with experimental data. A grid refinement study should be conducted to bring out potential errors. All the options of the code should be examined. This becomes more complicated as the number of options available within a CFD code increase. Identifying and quantifying each type of error is important because errors can interact and cancel each other - leading to erroneous conclusions in the validation process. One potentially useful method of verification is comparing the results of two codes. However, verification is not a democratic activity and one should watch for comparing with an inaccurate code. The comparison is strengthened when the two codes use differing numerical methods. The following paragraphs discuss specific checks that can be performed as part of a code verification process.Verification of a calculation involves error estimation, which is determining the accuracy of a single calculation and putting an error band on the final value. The approach involves performing a grid convergence analysis and determines the observed order of convergence, error bands, and grid convergence indices (GCI).

Verification Assessment Process

The process for Verification Assessment of a CFD code and / or simulation can be summarized as:

1. Examine the Computer Programming of the Code. One of the most basic tasks of verification assessment is the review of the computer programming or coding to check for and identify computer programming errors or "bugs". This is done by visually checking the coding and by computationally running subprograms using a test code. This is aided by complete and clear documentation, both internal and external. This step is to directly detect computer programming errors.

2. Examine Iterative Convergence. Verification assessment requires that a simulation demonstrates iterative convergence.

3. Examine Consistency. One should check for consistency in the CFD solution. For example, the flow in a duct should maintain mass conservation through the duct. Further total pressure recovery in an inlet should stay constant or decrease through the duct.

4

4. Examine Spatial (Grid) Convergence. The CFD simulation results should demonstrate spatial convergence.

5. Examine Temporal Convergence. The CFD simulation results should demonstrate temporal convergence.

6. Compare CFD Results to Highly Accurate Solutions The veracity of a code can be examined by comparing the CFD simulation results to highly accurate solution to the models used within the CFD code. This can include analytical solutions, benchmark numerical solutions to ordinary differential equations (ODEs), and benchmark numerical solutions to partial differential equations (PDEs).

Validation Assessment Validation is defined as “The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model”.

Validation has also been described as "solving the right equations". It is not possible to validate the entire CFD code. One can only validate the code for a specific range of applications for which there is experimental data. Thus one validates a model or simulation. Applying the code to flows beyond the region of validity is termed prediction.

Validation examines if the conceptual models, computational models as implemented into the CFD code, and computational simulation agree with real world observations. The strategy is to indentify and quantify error and uncertainty through comparison of simulation results with experimental data. The experiment data sets themselves will contain bias errors and random errors which must be properly quantified and documented as part of the data set. The accuracy required in the validation activities is dependent on the application, and so, the validation should be flexible to allow various levels of accuracy.

The approach to Validation Assessment is to perform a systematic comparison of CFD simulation results to experimental data from a set increasingly complex case.

Each CFD simulation requires verification of the calculation as specified in the discussion of Verification Assessment.

Validation Assessment Process

The process for Validation Assessment of a CFD simulation can be summarized as:

1. Examine Iterative Convergence. Validation assessment requires that a simulation demonstrates iterative convergence.

2. Examine Consistency. One should check for consistency in the CFD solution. For example, the flow in a duct should maintain mass conservation through the duct. Further total pressure recovery in an inlet should stay constant or decrease through the duct.

5

3. Examine Spatial (Grid) Convergence. The CFD simulation results should demonstrate spatial convergence.

4. Examine Temporal Convergence. The CFD simulation results should demonstrate temporal convergence.

5. Compare CFD Results to Experimental Data. Experimental data is the observation of the "real world" in some controlled manner. By comparing the CFD results to experimental data, one hopes that there is a good agreement, which inreases confidence that the physical models and the code represents the "real world" for this class of simulations. However, the experimental data contains some level of error. This is usually related to the complexity of the experiment. Validation assessment calls for a "building block" approach of experiments which sets a hierarchy of experiment complexity.

6. Examine Model Uncertainties. The physical models in the CFD code contain uncertainties due to a lack of complete understanding or knowledge of the physical processes. One of the models with the most uncertainty is the turbulence models. The uncertainty can be examined by running a number of simulations with the various turbulence models and examine the affect on the results.

Building-Block Approach for Experiments

A building-block approach is followed in performing the validation assessment for a complex system. The approach consists of phases involving successively more complex flow physics, geometry, and interactions.

Requirements for Experimental Data

The experimental data likely has uncertainties and error associated with it. In comparing the CFD simulation results to experimental data, one should discuss the experimental errors. Plots comparing CFD results and experimental data should include a visual display of the error bars on the experimental data.

6