A review of Commercial MPA Capability – 2005 update€¦ · A review of Commercial MPA Capability – 2005 update Mark Cross and Avril Slone School of Engineering University of

FENET THEMATIC NETWORKCOMPETITIVE AND SUSTAINABLE GROWTH(GROWTH) PROGRAMME

University of

Wales SwanseaBudapest Hungary 2005

A review of Commercial MPA Capability – 2005 update

Mark Cross and Avril SloneSchool of Engineering

University of Wales SwanseaSwansea Wales UK


University of


INTRODUCTION

• September 2002 – overview of emerging multi-physics technologies

• Over the following 2.5 years what has been progress at the commercial level?

• What do we mean by multi-physics?

• What is the need or role for high performance computing clusters?

• What of user environments?


University of


What is multi-physics simulation?

Most CAE analysis software tools developed in the context of a single discipline:• CFD (fluid flow, heat transfer, combustion)• CSM (structures, dynamics, contact, heat transfer)• CEM (electro-magnetics)• Acoustics

What of their interactions? - Until recently unless they were natural (e.g. thermo-mechanics or thermo-fluids) we mostly cheated or ignored them


University of


Challenge of multi-physics

• Multi-physics - closely coupled interactions amongst the separate component of continuum physics phenomena

• CAE modelling and analysis software is essentially phenomena specific :- CFD uses FV techniques with segregated iterative solvers- CSM uses FE techniques with direct solvers- CAA & CEM uses either FE/FV techniques

plus the heritage software approaches that go with each.

• Must ensure accurate filtering and mapping of data for volume source and boundary data

• Must ensure that where needed, the mesh and geometries deform compatibly


University of


Levels of physical couplingtemperature

CFD CSM

pressure

CFD CSM

velocity

CFD CSM

deformation

pressure

Meshadaptation

• Very Low - 1 way- simple via a file between codes

• Medium - 2 way- much trickier, mesh compatibility& time step constraints

• High- time & space accurate- very challenging in every respect!


University of


Multi-Physics solversDirectly Coupled SolutionDirectly Coupled Solution

KaKa = = ffKKff KKfsfs KKftft

KK == KKsfsf KKss KKstst

KKtftf KKtsts KKtt

ffff

ff == ffss

fftt

aaff

aa == aass

aatt


University of


Multi-physics solver approach

Staggered Solution:Staggered Solution:

Explicit or implicitExplicit or implicit

KKffnnaaff

nn = = ffffnn -- gg11((aass

nn--11,,aattnn--11))

KKssnnaass

nn = = ffssnn -- gg22((aaff

nn,,aattnn--11))

KKttnnaatt

nn = = ffttnn -- gg33((aass

nn,,aattnn))


University of


Classifying multi-physics

• What most vendors advertise is multi-physics

• What most vendors offer is multi-disciplinary

• Multi-disciplinary – using data generated by one code as input into another – loose or one way coupling (e.g. electric field loading a thermal calculation)

• Multi-physics – two way exchange of information, which could involve implicit convergence within a time-step (e.g. thermo-mechanical)

• Closely coupled multi-physics – time and space accurate exchange of data (e.g. dynamic fluid-structure interaction)


University of


Parallel multi-physics solver strategy

KKffnnaaff

nn = = ffffnn -- gg ((aass

nn,,aattnn))

KKssnnaass

nn = = ffssnn -- gg22((aaff

nn,,aattnn))

KKttnnaatt

nn = = ffttnn -- gg33((aass

nn,,aattnn))

P1P1P2P2P3P3

• The challenge here is to ensure parallel mapping minimises the movement of data between Processors for the solution of each component

• Natural to put each component solver on a separate Processor (or processor group), BUT this maximisesthe movement of data around system & compromisesScalability, so . . . have to be smarter


University of


PRE / POST Processing

Need for read and write data in different formats for each of the simulation tools

Multi-Disciplinary EnvironmentStructural Analisys Module

Computational Chemistry

Module

Thermal Analysis Module

CFD

Module PRE / POST PROCESS

Electromagnetics Module

Multi-user environment for the development and application of different numerical simulation software tools.


University of


What else do you need to do multi-physics simulation?

• Necessities:- phenomena specific solver software that can accept boundary data, volume source data and modifications to property data fromother codes- good filters to exchange boundary and volume source data from one solver module to another- solver strategies which are compatible

• Practical demands:- Compatibility of the mesh structure- Very good filters for mapping numerical information from one solver to another- Avoid opening and closing files – read numerical information directly from one solver by another; a common memory database is desirable - Parallel scalability is necesary for the large problems


University of


Key issues in closely coupled multi-physics simulation

Phys-A Phys-B

• Interpolation from one setof variables to another =>compatibility of mesh

• Single database of mesh data & simulation variables

• Solver strategy- direct vs iterative- Eulerian vs Lagrangian

• Is coupling strategy compatiblewith scalable parallelism, EVEN if software components are parallel


University of


Multi-physics: coupling issues

Coupling options:

a) All solvers written in one software environment with common numerical approach, with regard to discretisation, meshing, solver strategy, and parallelisation strategy, where data all held in one common database

Not practical if wish to use separate well established commercial codes for each phenomena, so . . . .

b) Need a filter structure to enable phenomena specific codes to exchange information directly from each others database without opening closing files –effectively need interoperability of separate codes.

Options here derive from tools for developed for parallelisation – especially PVM, MPI and specifically MpCCI

Challenge that remains is the parallel scalability of interoperable codes – each code may run in parallel but their coupling may severely compromise scalability


University of


MpCCI –a tool for code interoperability

• Emerged from an EU project – public domain OPEN SOURCE tools

• www.scai.fraunhofer.de/mpcci.0.html

• Conference in Feb ’05 showing a number of MpCCI facilitated code couplings:- ABAQUS + FLUENT for DFSI- STAR-CD + NASTRAN for DFSI- Computational aero-acoustics (CFD + acoustics)

• MpCCI here to stay and potentially facilitates genuine coupling across the board – initial experiences have not been trouble free, but . .

http://www.scai.fraunhofer.de/mpcci.0.html


University of


Industry consolidation

• Additional physics capabilities being bought in through industry consolidation.

• Examples include:a) ANSYS acquiring CFXb) ESI acquiring CFD-ACEc) ESI acquiring PROCASTd) Century Dynamics/AUTODYN acquired by ANSYS


University of


Sector Specific ‘multi-physics’ Software

Castings• PROCAST• http://www.ues-software.com• MAGMASOFT • http://www.magmasoft.com

Forgings• DEFORM• http://www.deform.com• SUPERFORGE • http://www.mscsoftware.com• FORGE3 • http://www.transvalor.com

Temperature Profiles – MAGMASoft: Hansen et-al Numerical Simulation

of Casting Solidification in Automotive Applications, Pub TMS

Not much change in 3 years – they do what they do!


University of


Sector Specific ‘multi-physics’ Software

• PolymersC-Moldhttp://www.moldflow.com

• Joining ProcessesSYSWELD (Welding software) http://www.esi-group.com

• Electronic coolingFlothermhttp://www.flomerics.com

Some additions here:e.g. Flomerics now has a solid mechanics module

In the main - similar


University of


Commercial CAE analysis technologies - a web survey

Tools claiming multi-physics capabilities:

• ANSYS/Multi-physics - http://www.ansys.com/• ABAQUS - http://www.abaqus.com• ADINA - http://www.adina.com• ALGOR - http://www.algor.com• AUTODYN - http://centurydynamics.com• CFD-ACE - http://www.esi-group.com• DYNA - http://www.ls.com• FEMLAB - http://www.comsol.com/• INFOLYTICA - http://www.infolytica.com• MSC- NASTRAN - http://www.mscsoftware.com• PHYSICA+ - http://www.multi-physics.com• SmartCAE - http://www.smartcae.com• STAR-CCM+ - http://www.adapco.com

http://www.ansys.com/

http://www.abaqus.com/

http://www.adina.com/

http://www.algor.com/

http://centurydynamics.com/

http://www.esi-group.com/

http://www.ls.com/

http://www.comsol.com/

http://www.infolytica.com/

http://www.mscsoftware.com/

http://www.multi-physics.com/

http://www.smartcae.com/

http://www.adapco.com/


University of


ANSYS• http://www.ansys.com• The ANSYS Multi-physics solution• Two methods to couple multiple physics together, • Direct and Sequential:

• Direct - solves all DOFs at the FEA coefficient matrix level. • Sequential - solves DOFs for one physics then passes results as

loads and boundary conditions to the second physics.• At least two iterations, one for each physics, in sequence, are

needed to achieve a coupled response.

How close is the coupling? When does it matter?

Examples of MpCCI interfacing of ANSYS with CFX for DFSI


University of


ANSYS• Available physics include

• Structural• Thermal• CFD (now CFX)• Acoustic• Electromagnetic

• Couplings include• Thermo-mechanical• Thermo-fluid• Piezo-thermal-structural-electric • Fluid structure interaction (thru MpCCI)

Good deal of multi-disciplinary, but more restricted multi-physics


University of


ANSYS multi-physics• Example:Multi-physics contact

• Current and resultant Joule heating in a switch contact is modelled as the switch is actuated.

• Mechanical, thermal and current flow are modelled using direct coupled field elements

• Good example of natural solver ‘fit’


University of


ABAQUS

von Mises stress

Temperature

• Very strong on non-linear solid mechanics analysis• Coupled problems• Thermo - mechanical

• sequentially or fully coupled• Disc Brake – fully coupled

• Thermo-electrical • Pore fluid flow-mech• Stress - mass diffusion

• sequentially coupled• Piezoelectric (linear only) • Acoustic - mechanical (linear only)• Coupling all with potential solvers• ABAQUS announces technical

collaboration with FLUENT for DFSI using MpCCI


University of


ADINA

• Well established software for thermal-fluid-structural problems

• Can use a variety of coupling levels

• Example - fuel system:- Large structural motion- Fluid• Various turbulence models• Incompressible, slightly/fully compressible- Different meshes for fluid & structure

• Parallel?• User environment?


University of


ALGORhttp://www.algor.com

• Professional Multi-physics

• Core technology includes: • Static stress• Mechanical Event Simulation (MES)• Linear and nonlinear material models• Linear dynamics• Steady-state and transient heat transfer• Steady and unsteady fluid flow and • Electrostatics• Simultaneously replicates the dynamic flexing behaviour of a

component or mechanism to predict stresses that may result from motion or from the interaction of the part with other independent objects – claim for DFSI, but . . .

http://www.algor.com/


University of


ALGOR

• Thermo-mechanical analysis of diesel engine cylinder head• Elastic analysis• Maximum stresses occur near

exhaust port• Matches experimental results

which indicate leakage in this region


University of


AUTODYN

• Owned by Century Dynamics Ltd, Horsham, UK• Acquired by ANSYS recently

• Mixed FV (velocity), FE (displacement) and DEM methods plus any mixture for fluid structure interaction

• Fully 3D and parallel

• Focus upon extreme impact events – collapse, explosions, etc


University of


Collision- mixed ALE and Eulerian


University of


CFD-ACE

• http://www.cfdrc.com• CFD-ACE+ • Coupled simulations of

• Fluid, • Thermal• Chemical• Biological • Electrical • Mechanical phenomena.

• Issues• A solver environment – not a solver• Uses PVM as the means of interoperability


University of


CFD-ACEGeometry of IGBT moduleCFD-ACE+ simulation

of wire deformation in IGBT module

Stress distribution in the wires


University of


CFD-ACE

CFD-ACE+ Simulation of SEMATECH Benchmark Case with radiative heating of a wafer via a quartz window with water cooling

Now owned by ESI_Group, France


University of


LS-DYNAHuman-clothing interactionDynamic non-linear FE

• Contact interaction with the body• Large displacements

Associated techniques• airbags and seatbelts in cars

Stresses ~ bra cups and strapsJogging - Heat transfer?Human tissue

• Solid mechanics or• Non-Newtonian fluid?

Dynamic, but is it MDA or MPA?


University of


FEMLAB• Built upon MATLAB• Uses FE methods in both 2D & 3D

• Nice user environment• Good for student use

• Capabilities:- Heat transfer- Solid mechanics- Electro-magnetics- Flow (NS, incompressible, 2-eqn turbulence)


University of


FEMLAB• Model exemplifying fluid-

structure interactions modelled using FEMLAB.

• Viscous forces and the system’s pressure impose forces to the surface of a structure.

• The deformation in the soft structure is not small and the fluid regime will therefore dynamically change.

• This means that change in the structure is coupled back to the fluid dynamics.


University of


FEMLABhttp://www.comsol.com/

Built upon MATLAB- FE approach 2D/3D- Nice User Env- Good student tool

For Fluid-Structure Interactions (FSI)• Linear Static Analysis• Momentum Transfer & CFD• Solvers used: Non-linear• Time-dependent• Products used:

• FEMLAB • MEMS ModuleT

Adding constituent physics – how easy is this?

http://www.comsol.com/


University of


MSC- NASTRAN et alAir Launched Cruise Missile showcase MP example!

• Subject to engulfing fuel fire (temp 1273 K) • Finite element model ~ MSC Patran• Radiantly heated -MSC Patran Thermal

Loosely Coupled Interface MSC Patran EXODUS PCLExternal Sandia software

• Coyote (Thermo-Chemical-Fire)• Pronto (Dynamic)• Jaq (Structural)• Toro (Electro-magnetic)

Definitely MDA, but MP?

Major change since 2002:Interface with STAR-CD for DFSI thru MpCCI


University of


STAR-CCM+ formerly COMET

• COMET based upon FV-UM approach - 3D• ICCM, out of TU Hamburg, Germany• Claims flow, heat transfer and solid mechanics

including fluid-structure interaction• Good flow and heat transfer web examples• No web examples of solid mechanics or FSI• Is it actually MDA or MP?

• Sept 2002 - BOUGHT OUT BY STAR-CD COMPANY & WITHDRAWN FROM MARKET

• Re-appears as STAR-CCM+ late in 2004


University of


STAR – CCM+• Owned by Adapco

• Launched in recent months

• To facilitate closely coupled simulation

•Vertex based polyhedral mesh based solver technology

• Runs in parallel

• Currently limited multi-physics –a framework for it though


University of


PHYSICA• Spin off from University of Greenwich,

London, and licensed through PHYSICA Ltd, Croydon - (http://www.physica.co.uk)

• Uses primarily FV-UM methods• Demonstrates:

• NS flows (turbulence, free surfaces)• Heat transfer (phase change, reactions)• Solid mechanics (non-linear)• Electro-magnetics• Close coupling of all in examples

• Parallel design from outset• Software targets multi-physics simulation,

especially in the context of process manufacturing (e.g. casting, forming, welding)

• Core solver technology – hooks into ANSYS, FEMGV, NASTRAN, etc


University of


Extrusion through U-shaped die

4.76 mm

41.27 mm

47.62 mm

Mesh details

63220 elements

69507 nodes

•• Workpiece = 470Workpiece = 470°°CC

•• Die =Die = 450450°°CC

•• Air = 30Air = 30°°CC


University of


Parallel results

Processors

Run time(hours)

Speed-up

1 81.9 1

4 18.3 4.48

8 10.2 8.03

12 7.5 10.92

16 6.1 13.43

• Itanium IA 64 cluster running Linux OS• Eight nodes, two 733MHz processors per node• Each node with 2 Gb memory & 2Gb swap space

SCALABLE PARALLEL CLOSELY COUPLED MULTI-PHYSICS – VERY HARD!

Single phase mesh partitions on 16 processors


University of


New Players• New companies appearing on web in the last 4 years

• Infolytica – limited physics

• SmartCae – focus upon FSI and code coupling –cannot tell whether its an MpCCI tool or not

• ZEUS-MP – academic collaboration, flow dominated.


University of


Smart CAE

FSI in Smart CAE

Facilitated by MpCCI?


University of


Smart CAE


University of


INFOLYTICA• Infolytica Corporation • E-M software for high and low

frequency devices.• Company founded in 1978,

MagNet software

• http://www.infolytica.com/• Gas Insulated Switch packages

used• MagNet determines AC current flow

through the switch in the closed position.

• ElecNet to analyze the potential distribution in the open positions.

• ThermNet used in two modes

VERY LIMITED MULTI-PHYSICS!


University of


Conclusions• Consolidation a major driver for enabling multi-disciplinary and multi-

physics simulation

• Some genuine progress over the last couple of years, but maybe not as much as might be expected

• Reason – its hard work!

• PVM and MpCCI play their role in enabling code interoperability – the first and necessary stage of coupling

• Most commercial tools demonstrate some coupling capability – quite a lot still a one-off, not productised

• Market segmentation – some working towards genuine MPA and others optimising what they have

• A key issue is cluster based scalable parallel multi-physics simulation


University of


Exploiting parallel cluster technology: the challenge

• MpCCI and other filter technologies

• Upside – enables interaction at the code d’base level

• Downside – all data exchange must go via filter and is a compute bottleneck wrtscalability on parallel clusters

Code A (CFD)

Code B(FEA)

Map onto Parallel cluster

Map ontoParallelcluster

Filter tech:MpCCI


University of


Parallel results - revisited

Processors

Run time(hours)

Speed-up

1 81.9 1

4 18.3 4.48

8 10.2 8.03

12 7.5 10.92

16 6.1 13.43

• Itanium IA 64 cluster running Linux OS• Eight nodes, two 733MHz processors per node• Each node with 2 Gb memory & 2Gb swap space

SCALABLE PARALLEL CLOSELY COUPLED MULTI-PHYSICS – VERY HARD!

Single phase mesh partitions on 16 processors

Documents

A review of Commercial MPA Capability – 2005 update€¦ · A review of Commercial MPA Capability – 2005 update Mark Cross and Avril Slone School of Engineering University of