NCHRP Project 22-24 Development of Verification and Validation Procedures for Computer Simulation use in Roadside Safety Applications SURVEY OF PRACTITIONERS

NCHRP Project 22-24

Development of Verification and Validation Procedures for Computer Simulation use in

Roadside Safety Applications

SURVEY OF PRACTITIONERS

INTENT

A survey was sent out to practitioners who use LS-DYNA in roadside safety analysis To assimilate useful and practical information for

the project team To determine state of the art practices for

building, verifying and validating FEM Increase likelihood of developing models that run

without errors

APPLICATION Practitioners were asked about what types of

techniques they use and what range of variation they considered acceptable when performing typical roadside safety simulations.

A total of 41 questions asked to practitioners in following topics:

1. Background information on practitioners2. Details about model building3. Model verification4. Model validation

RESPONSE TO SURVEY Background Information

33 practitioners out of 45 identified responded to survey

Average LS-DYNA experience 8.5 years Average years of experience in roadside safety

also 8.5 years Approximate number of LS-DYNA projects

used to evaluate roadside hardware 23.

RESPONSE TO SURVEY Details about Model Building


Where do you generally obtain your vehicle models:

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

NCAC Library

Build Own Models

Other Public Domain Sources

Automotive Industry

Proprietary Models

Self Modified Public Domain Vehicles

Source Used

Primary Source


Rate the importance of the following model features for Roadside Safety Analysis

Average Score

1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3

Articulated Suspension

Rolling Tires

Detailed Geometry

Fast Run Time

Vehicle Component Failure

Failing Tires

lowest highest


Continued at next page

Which of the following NCAC Vehicle Models have you used?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Reduced Geo Metro

Reduced C2500

Reduced Single Unit Truck

Detailed C2500

Detailed Ford Taurus

Detailed S10

Detailed F250

Detailed Ford Explorer

Without Changes

Self Modified


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Bolts

Rivets

Welds

Soil

Road Surface

Detailed

Approximated

Which of the following do you model in detail and which are modeled using approximate techniques?


Sources for Material Properties

0% 20% 40% 60% 80% 100% 120%

Laboratory Experiments

Other FE Models

ASTM/AASHTO/ANSI

Source Used

Primary Source



How are Welded Connections Modeled?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Merged Nodes

Spotweld w/ Failure

Tied Contact

Tied Contact w/ Failure

Technique Used

Primary Technique


How do you model post-soil interaction?

0% 10% 20% 30% 40% 50% 60%

Solid Lagrangian

Non-Linear Springs

SPH

Eulerian Mesh

Technique Used

Primary Technique

RESPONSE TO SURVEY Model Verification



Comments on the definition:

The ASME definition can be more generic: “The process of determining that a model implementation correctly represents the modeler’s conceptual description of the model and the solution to the model.”The verification checks should ensure that the model is build correctly. It should include more than that the simulation should run. The verifications would include the material model selection and properties, element formulation selection and properties, mesh quality and geometry, contacts and friction coefficients, boundary and initial conditions, and connections. I think all verifications checks could be made before running the simulation to ensure the model is free of error and is a correct representation of actual structure. Validations are performed by running the simulation and comparing to test data (iterative).Somewhat Agree with the above statement, but is not sufficient. Model verification is building the model RIGHT. The verification process should deal with accuracy assessments that ensure correct models are created which are free of errors and provide adequate representation of the actual systems."Verification" is usually understood as a demonstration that the model behaves according to the theory. "Validation" is usually understood as a demonstration that the model behaves as the physical reality.I would add that the models do NOT produce non-physical behaviors aside from numerical instabilities.Verification of a model should also include checking that the results are reasonable. We propose that this item, rather than "Verification", is called "Computational Verification", in order to make clearer that it deals with problems related to the mathematical implementation and calculation of the problem.I agree more closely with the DoD/AiAA statement. For developing material models, verification is a check on the coding of the material model - making sure it was implemented according to the theory that was intended to be implemented.


Comments on the definition:

There is no need to create a specific definition for roadside safety simulations. More general definition can be used such as: “Verification evaluates the accuracy with which the computational (FE, discrete) model depicts the mathematical model”. The mathematI share the ASME definition (Check Len Schwer committee) "Verification is the process of determining that a computational model accurately represents the underlying mathematical model and its solution." This puts the responsibility of verification on the code developers, except for user's verification activities like mesh sensitivity and convergence of solution.….What you want to know is whether or not the model simulates real life conditions. Verification, in my opinion, should read model Validation. The simulated results should be correlated with testing to verify the model….……I had not considered this attempt to distinguish between verification and validation before. Assuming that it is a sensible distinction (of which I am not fully convinced), there appear to be two different statements offered. I assume that a polished statement should amount to the model including all the relevant parts of the system modeled in a way that satisfactorily deploys the capabilities of the simulation code. Verification is about the model "working" in the intended way, and validation about the accuracy of its representation of reality. (This is YOUR project - you sort it out!)I would NOT limit it to the collective experience of the roadside safety simulation community


Average Size Reported (mm)

0 10 20 30 40 50 60 70 80 90 100

Contact othercomponents

Large deformations / Nocontact

Small Deformation / NoContact

Average Size Reported23 mm

44 mm

93 mm

Mesh Sensitivity and Quality Determination:

What is the largest shell element size used for steel parts that:


21

33

56

Average Size Reported (mm)

0 10 20 30 40 50 60

Post expected to fracture

Large rotation insoil / Nofracture

Small Deformation / NoFracture

Average Size Reported21 mm

33 mm

56 mm

Mesh Sensitivity and Quality Determination:

What is the largest solid element size used for wood posts where:



In general, what is the maximum warpage angle that you try to limit the elements to in your models:

0% 10% 20% 30% 40% 50% 60%

0 - 5%

5% - 10%

10%-15%

15%-20%

N/A

48.3%

13.8%

13.8%

3.4%

20.7%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

3 to 1

5 to 1

2 to 1

N/A


I generally keep the aspect ratio of my mesh smaller than:

19.4%

45.2%

22.6%

12.4%

RESPONSE TO SURVEY Model Validation


Average Score

3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4

Barrier Damage (qualitative)

Crash Sequence (qualitative)

Displacement (quantitative)

Vehicle Trajectories (qualitative)

Accelerations (quantitative)

Velocities (quantitative)

Vehicle Damage (qualitative)

Rank the following statements in order of importance in validating a finite element simulation with physical test (1=least important, 5 = most important):


Which qualitative comparisons do you use to compare your simulation to the physical tests?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Hardware Deformations

Location / number offailed components

Visual Methods (i.e.,pictures / movies)

Phenomological (i.e.,same general response)

Parameter Used

Primary Parameter Used


Which quantitative time history comparisons do you use to compare your simulation to the physical tests?

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Displacements andRotations

Velocity

Accelerations

Energy

Stress/Strain

Technique Used

Primary Technique


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

TRAP (i.e., OIV, ORA,THIV, ASI, etc.)

Geer's Parameters

NARD validation manual

Ray's T-statistic

Technique Used

Primary Technique

Which of the following parameters or metrics do you use to compare your simulation results with experimental results?



Summary The survey provided information regarding the

current state of the art practices for building, verifying and validating FEM

This information will provide some practical guidance for the project team when developing V&V procedures

The information could also be synthesized into a “Best Practices” document. BUT, probably doesn’t provide enough information to do

one justice May be out of scope for this project

LIST OF PRACTITIONERS RESPONDED

Documents

NCHRP Project 22-24 Development of Verification and Validation Procedures for Computer Simulation use in Roadside Safety Applications SURVEY OF PRACTITIONERS