NAFEMS

Analysis Planning & Simulation Data Management, complementary techniques

to assure confidence in simulation results

Mark Norris, theSDMconsultancyAlthea de Souza, Quesada Solutions

Steve Howell, Abercus

NAFEMS

Mark Norris CEng MBA theSDMconsultancyPLM and SPDM Consultant• Engineering background: Simulation for machine dynamics, 

crack propagation in aircraft, FE, MBS, CAD/CAM• 30 years experience of designing and delivering Leading Edge 

Engineering IT solutions• Wide PLM expertise, industry sectors, solutions• 7 years in the SDM domain focusing on value, value analysis,

best practices, system selection, POCs, program governance• Delivery of SDM value analyses and SDM proofs of concept• Author of the NAFEMS white paper on the Business Value of

Simulation Data Management• Delivers NAFEMS SPDM Training Workshop (2 Days)

Mark Norris

NAFEMS

Agenda

• Introduction to assuring confidence• Simulation Data Management for assuring confidence• SDM adoption challenges• Analysis Planning for assuring confidence• Conclusion

NAFEMS

Introduction

• Confidence in Results is enhanced by Confidence in the Processes used to obtain them– CMMI scale is widely recognised for overall Process Maturity

for Integrated Product Development– ISO9001 certification is a necessity for certain industries

• PLM systems are extensively used to manage Change and Document Processes in regulated industries

• Simulation Process Data Management (SPDM) solutions have been available since 2000 but usage is limited

NAFEMS

Simulation is Challenging to Manage

• The simulation domain poses unique challenges in the assurance of confidence in results– Multi-dimensional problem– Many process steps– Every step impacts the results

• It’s so unique that a new type of data management had to be developed to manage Simulation Data-sets

• Simulation is continuously evolving– Into new domains– New methods (Benchmark January 2017)

NAFEMS

Requirements

Structure Definition + Geometry

Material models

Load‐cases, Regulatory & Performance

Process Definitions/Templates

System Definition & Parameters

Loads by Calculation or Measurement

Input data, intermediate data, process data & results data

Physical Test Results

Simulation Process Audit Trail

Discrete Solution

Discrete Models

Math Models

© 2017 theSDMconsultancy

NAFEMS - The International Association for the Engineering Analysis Community Essential Principles of SDM eL079 Session V1.0 Page 9

A Simulation Process is a chain of complex stepsDecisions & Assumptions made at each stage must be:– consistent – recorded

IDEALISATION• Physics• Geometry• Allowables

DISCRETISATION• De-featuring• Mesh• Loading

SOLUTION• Numerical

analysis

Discrete Solution

Discrete Models

Math Models

Physical System

© 2017 theSDMconsultancy

NAFEMS

Searchfor data

Submit & Solve

Extract & Post‐Process

Create ReportM

esh

Assemble

SDM: Directed Acyclic Graph to record simulation and process datawhereas core PLM data model = hierarchical product structures 

© 2017 theSDMconsultancy

NAFEMS

CMMI Maturity Levels

Managed L2

Optimizing L5

Initial L1

Defined L3

QuantitativelyManaged L4

Processes continually improved through innovative & incremental technological improvements

Processes are measured & controlled, timing is predictable. Quantitative quality goals are set & met.

Processes are well characterised & understood. Specifications, peer review, version control

Processes are planned, documented, performed, monitored & controlled at the project level. Often reactive

Processes are ad-hoc, success depends on competencies & heroics of people. The person is responsible for the result.

L0Simulation to be deployed in this domain for the first time. Competencies, Methods, Processes need to be developed©theSDMconsultancy

CMMI = Capability Maturity Model Integration

NAFEMS

unreadable

12

NAFEMS

CMMI Maturity Levels

Managed L2

Optimizing L5

Initial L1

Defined L3

QuantitativelyManaged L4

Processes continually improved through innovative & incremental technological improvements

Processes are measured & controlled, timing is predictable. Quantitative quality goals are set & met.

Processes are well characterised & understood. Specifications, peer review, version control

Processes are planned, documented, performed, monitored & controlled at the project level. Often reactive

Processes are ad-hoc, success depends on competencies & heroics of people. The person is responsible for the result.

L0Simulation to be deployed in this domain for the first time. Competencies, Methods, Processes need to be developed©theSDMconsultancy

SPDM Progression• Optimise

frequently-used Processes

• Refine as Best Practices

• Identify Standard Processes for Use

• Identify Processes in Regular Use

• Record what the Analyst did as painlessly as possible

NAFEMS

• I’m confident this simulation is valid.

• For what it does, its representations are accurate enough.

• It does the right things, its representations are complete enough 

• It represents the right entities and attributes.

• It works, trust me.• We have no idea of the 

maturity

Mapping Maturity Models: H&Y(Harmon and Youngblood)

Managed L2

Optimizing L5

Initial L1

Defined L3

QuantitativelyManaged L4

Processes continually improved through innovative & incremental technological improvements

Processes are measured & controlled, timing is predictable. Quantitative quality goals are set & met.

Processes are well characterised & understood. Specifications, peer review, version control

Processes are planned, documented, performed, monitored & controlled at the project level. Often reactive

Processes are ad-hoc, success depends on competencies & heroics of people. The person is responsible for the result.

L0Simulation to be deployed in this domain for the first time. Competencies, Methods, Processes need to be developed

NAFEMS

Sandia Predictive Capability Maturity Model

• Six process steps to achieve confidence in Simulation Results for High Consequence Engineering Systems

• Underpinned by an SDM solution

15Oberkampf & Imbert NAFEMS NWC17

NAFEMS

CMMI Maturity of Simulation

Managed L2

Optimizing L5

Initial L1

Defined L3

QuantitativelyManaged L4

Processes continually improved through innovative & incremental technological improvements

Processes are measured & controlled, timing is predictable. Quantitative quality goals are set & met.

Processes are well characterised & understood. Specifications, peer review, version control

Processes are planned, documented, performed, monitored & controlled at the project level. Often reactive

Processes are ad-hoc, success depends on competencies & heroics of people. The person is responsible for the result.

L0Simulation to be deployed in this domain for the first time. Competencies, Methods, Processes need to be developed

SME Members, Small S&M Teams

High Maturity Auto OEM’s running 50 Virtual Prototypes daily or more often

High Maturity Auto Tier 1s running 500 Virtual Prototypes daily or more often

Members wideningUse of Simulation 

© 2017 theSDMconsultancy

NAFEMS

SDM is a Proven, Robust Technology

• Pioneers are particularly successful• NAFEMS EUROPEAN SPDM CONFERENCE 2015: CELEBRATING 15

YEARS OF SDM IN PRODUCTION IN MUNICH• 2,000,000 complete vehicle data and results sets at BMW • 2,500 managed vehicle simulations per day• 1,800,000 simulations @ Audi• Successes in all sectors

– Auto, Aero, Nuclear, Rail, Oil,…• OEMs, tier 1 suppliers SMEs

BMW GroupMarc HertleinPage: 18

SheerDriving Pleasure

SPDM @ BMWContinuous Integration of SPDM

May 2013

Integrated simulation processes.

Skriptlibrary Skriptlibrary Report TemplatesHPC Interface

Configuration Assembly Solve Post process Reporting

~20000coresSpread over

Europe

Current Applications and Architecture.

NAFEMS

SDM for ISO9001 at• Abercus is a Consultancy specialising in Advanced

Engineering Simulation for the Energy Industries• Abercus developed an SDM called JET to retain precise records of every

simulation undertaken• JET, their in-house SDM tool automatically creates the simulation scripts,

post-processes and then stores the results • JET has saved time and effort at the point of use – and also provides the

information required for the Quality Management System (QMS) record, minimising duplication of effort

19Achieving ISO 9001 accreditation and the role of the NAFEMS QSS

NAFEMS

Saving time/effort at point of useEnsuring Quality and Traceability

JET – AN SDPM TOOL

*******************

**************

NAFEMS

SPDM adoption is limited

• SDM has very limited adoption by the 750,000 analysts worldwide, only 1 to 2% of engineers

• Whereas PDM or PLM adoption is nowv 90%+ in any industrial company using simulation

• Information system usage & deployment is new for Modelling & Simulation organisations (NAFEMS Survey), most Analysts still use the file system & logbook

• Digital Transformation like SDM and PLM is difficult, especially the first time

NAFEMS

Digital Transformation like SDM Requires• Executive Sponsorship

– The SDM program must deliver a Solution to the Key Initiatives/needs of the business

– Compatible with the Constraints of the IT landscape• Commitment and Time from Senior Staff

– To enable their key processes better, faster, cheaper• A Change Champion & Program Manager

– Transformation has to be evangelised and driven• High Level Technical SDM/IT Resources

– Specific Process and Productivity improvement require high level skills

• Funding; 100k€ to 1M€

NAFEMS

SPDM Deployment

• The first step in SDM deployment is to understand the planning and reporting requirements

• Valeo presented their PEN approach Plans d’ExpérienceNumérique in 2016 (Plans for a Numerical Experiment) – “Contract” between Engineering and Modelling & Simulation– Stored in a digital repository– Intellectual Property from Simulation can be searched/retrieved

• Stepping stone to SPDM

NAFEMS

Techniques to improve Capability Maturity

Managed L2

Optimizing L5

Initial L1

Defined L3

QuantitativelyManaged L4

Processes continually improved through innovative & incremental technological improvements

Processes are measured & controlled, timing is predictable. Quantitative quality goals are set & met.

Processes are well characterised & understood. Specifications, peer review, version control

Processes are planned, documented, performed, monitored & controlled at the project level. Often reactive

Processes are ad-hoc, success depends on competencies & heroics of people. The person is responsible for the result.

L0Simulation to be deployed in this domain for the first time. Competencies, Methods, Processes need to be developed

NAFEMS PSE certification

Analysis  Planning

Simulation Process Data Management

© 2017 theSDMconsultancy

NAFEMS

Engineering

How approaches fit together

Experimentation Domain

Numerical Experimentation

L2‐L5 Simulation Data ManagementProcess Record + Data Record

Physical Experimentation(Test Data Management) Test Peporting

Analysis Record: Results & 

Conclusions

L1‐ Analysis Management

Test Planning

Version Control is mandatory for L3 Maturity. Managing the Process Record  Enables Process Improvement 

L0‐ Prof. Sim. Eng. (PSE) Certification 

Analysis Plan: Requirements & Record

NAFEMS

ANALYSIS PLANNING AND REPORTING

27

NAFEMS

Introduction – to Althea

• Background in mechanical engineering• PhD applying CFD to an industrial application• First encountered NAFEMS in 1998 and joined the CFD

working group• Elected as chairman of the group in 2002• PSE assessor• Commissioning editor of NAFEMS magazine, Benchmark

NAFEMS

Background• Post-PhD job in industry as the sole analyst, working within the

engineering design department• Short handover from outgoing engineer

– Computer full of simulation files, grouped by project– Verbal description of each project, noted into paper logbook

• Took over existing projects• Asked to provide new results from previous projects

– External drag with simplified external geometry– Downstream flow conditions at domain outlet– Pressure values on curved surfaces where no geometry

surface exists• Frequently moving goalposts during a project

NAFEMS

Two Problems

• Finding previous simulation data

• Understanding what had been done, and why

NAFEMS

Formalised processes• Back in 1998, no-one had heard of SDM • Even PDM systems were relatively uncommon

• As an isolated analyst working in a design department, there were no processes for simulation to inherit

• Part-Solution: set of ‘analysis request’ forms to form the basis of a documented contract between the person requesting the simulation (usually a design engineer) and the simulation engineer (me).

• Different levels of detail depending on the simulation knowledge of the ‘customer’

NAFEMS

Analysis Planning BookAround this time I became involved in the NAFEMS CFD working group and proposed a book on planning a CFD analysis

NAFEMS

Why Plan Simulations?

• Purpose of the simulation• Required output data and locations• Required accuracy • Most suitable models for the physics • Simplifications and assumptions• Simulation quality activities• GIGO

NAFEMS

What happens without a plan?“If you fail to plan, you are planning to fail!”

– Benjamin Franklin

“A goal without a plan is just a wish.” – Antoine de Saint-Exupéry

“Every minute you spend in planning saves 10 minutes in execution”

– Brian Tracy

NAFEMS

Overview of CFD Process

Analysis Request

Request Assessment

Analysis Specification

Geometry Creation

Model Set-up

Solution Process

Solution Verification

Analysis Validation

Post Processing

Report Writing

Data Storage

Non-ConformingAnalyses

Stop if use of CFDis not Justified

Grid Generation

NAFEMS

How to Plan a Simulation 1• Contract with the ‘customer’ and any other stakeholders

– Understand requirements and uses • Physical variables of interest• Agree boundary conditions• Early design assessment or final design confirmation• Subsequent compatibility with other simulations (multi-physics, systems)

– Understand constraints • time available• Accuracy• available input data and quality of that data

– Resource holders• Software and IT suppliers

NAFEMS

How to Plan a Simulation 2

• Consider analysis approach– What assumptions– What models and numerical approach (simulate or not?)– Acceptable convergence

• Consider post processing – Can it be automated– Do surfaces need to be created– How best to communicate the results to the customer

• Record the analysis plan

NAFEMS

What happens if the planning isn’t recorded?

Inappropriate interpretation of simulation resultsInappropriate use of simulation results

Customers and simulation engineers disagree

The wrong simulation might be carried out

NAFEMS

Implementing Analysis Planning

Five levels of metadata management• Scanned paper

– Referencing file names• Electronic Paper

– Pointers to files• Intelligent, modular templates

– Adaptable to particular simulation • Metadata in a database input through/visible in a form

– Pointers to files• Full SDM with Metadata and File management

39

NAFEMS

Comparison of SPDM & Analysis PlanningProject  SPDM Analysis PlanningProject Approval Department/Enterprise  Team/AnalystSenior Management Attention Essential Not neededSimulation Skill set  Team leader with “Process” skills  Senior AnalystIT resource Very specialised SDM IT Skills minimalCost €100k to €1M Man‐hours

Value SPDM Analysis Planning

Planning Access to past work Stimulates Thought before Calc. 

Reporting Complete, Automated Ensures completeness & rigor

Other ILM, Automation, Global Working, Prepares for SPDM deployment

Management of Complexity, ………

NAFEMS

Audi reported a 35% productivity gain, initially from : – model documentation

– automation of results extraction, 10,000/day – report generation – documentation by creation of 200,000 objects/day

Curve comparison

Report

NAFEMS

Conclusions

• Analysis Planning and Reporting is a valuable technique to improve Governance and Confidence in Results– Brings rigour and traceability to Simulation– Can be deployed quickly with limited effort– Paves the way for an SPDM deployment

• Simulation Data Management has bought complete traceability & instant visibility of complex models, how results were obtained, but implementation can prove a challenge– NAFEMS can help, Body of Knowledge and Training

42

NAFEMS

Q&A

althea@quesadasolutions.co.ukmark.norris@theSDMconsultancy.comsteve.howell@abercus.comprankul.middha@abercus.com

43

NAFEMS

How NAFEMS can help in SPDM

• Confrences– European SPDM Conference 28-29 Novembre 2018 in Munich

• Know-How(BOK) – more than 300 présentations• Books

– Business Value White Paper– Considerations for the Implementation of SPDM (2018)

• WorkingGroup SDMWG – 2nd Thursday of the month Webex• Training

– Introduction to SPDM– Two day Workshop: Essentials of SPDM

NAFEMS

Conclusion: Value & Know-How are Key to SPDM Success

Thank you for your attention

Enterprise’s Needs

Engineer’s Needs

Capabilities of SPDM

SuccessfulDeployment

€ Hours

CAE Data Process

mark.norris@theSDMconsultancy.comyour partner for SDM success