Software Integration 04052006 - The EU Provenance … · (NASTRAN) Wrapper Aero elastic Wrapper ... ANSYS and MSC/NASTRAN for heat transport and deformation in ... Software_Integration_04052006.ppt

Folie 1 > Software Integration and Workflow Management > A. SchreiberMay 4, 2006

Numerical simulation software integration with TENTSoftware Integration and Workflow ManagementAndreas Schreiber, DLR Simulation and Software TechnologyJoint ESTEC-DLR Workshop, ESA/ESTEC, Noordwijk, May 3-5, 2006


IntroductionWorkflows

Today, many problems require complex numerical simulations. Examples:

Re-entry simulation of space vehiclesAero dynamical and aeroelastical analysis of flight maneuvers

Performing such simulations is software technologically complexInvocation of many different codes in correct orderUsage of high performance computersTransfer of data between the different codesCollaboration with colleagues

Complex simulations are Workflows of many codes:

SimulationPostprocessing/

VisualisationPreprocessing

Data Data


What is TENT?Answer for Users

Essential features:Easy setup and configuration of simulation workflows with integrated applicationsUsage of distributed computing resourcesOnline steering and visualizationProject-based data managementwith support for cooperative workingMultidisciplinary coupledsimulations

TENT is an integration and simulation environment for engineering applications.


What is TENT?Answer for Computer Scientists

Essential characteristics:Component based

CORBA Peer-to-peer modelExtensible Java-GUIDistributed computing (Grids)Flexible integration of existing tools

Large development libraryData management

Open solution (WebDAV & XML)

Development based only on accepted standardsCORBA, LDAP, FTP, HTTP, WebDAV, XML, ...

Development based only on accepted standardsCORBA, LDAP, FTP, HTTP, WebDAV, XML, ...

TENT is an open and extensible framework for tool integration and workflow management.


CORBA

GUI

Plug-In . . . Plug-In

System components

Data Server

Factories

Nam

e Server

Co

up

ling

Scriptin

g

Applications

Wrapper

CFD

Wrapper

CSM

Wrapper

Visu

ali-satio

nWrapper

Filter

System Architecture


Architecture and Deployment

Fact

ory

Visuali-zation

Nam

ing

Ser

vice

Factory

Pre-processor Simulation

Co

mp

ute

Ser

ver

Front-end Machine

CORBA

Key:

Java

other

Factory

Post-processor Adaption

Co

mp

ute

Ser

ver

Dat

a Se

rver

Workflow Visualization (GUI)

Workflow

Control


Application Application browserbrowser

Property editorProperty editorand plugand plug--insins

WorkflowWorkflowDesignDesign

Help, Logging, GUI plugHelp, Logging, GUI plug--insins

ControlControl


PlugPlug--In: Panel In: Panel

PlugPlug--In: ToolbarIn: Toolbar

Meta dataMeta data

Project browserProject browser


Example GUI Plug-In2D-Plot


Monitoring


CORBA

Factory

GUI

FactoryFactory

TENT-Factory: (CORBA-)Server for starting programs

Grid

Cluster (LSF, NQS, ...)

Workstation / PC

Distributed Computing

Divisions-/Companies

Local Server/Workstation

Worldwide


Integration of ApplicationsWrapper

StartControl, Properties

CORBA-Interface

Control

Oth

er T

ENT

com

po

nen

t

Oth

er T

ENT

com

po

nen

t Application

Wrapper (application specific)

CORBA


Integration of ApplicationsWrapper Library (SDK)

Wrapper development in Java or PythonSome features

Generation of input filesTemplating (replacement of parameters)

Parsing of result filesExtraction of valuesGeneral filter mechanism

ExecutionRemote executionApplication control via pipes, TCP/IP sockets, CORBA, …

Wrapper code can be generated…


Integration of ApplicationsAutomatic Generation of Wrapper Code

To reduce the effort and minimize coding errorsTENT specific Eclipse plug-ins for:

Generating new wrapper codeExtending existing code


Control flow(method invocation, properties)

Control flowEvents and method invocations through CORBA

Communication

CORBA-Interface CORBA-Interface

Data flow(FTP, GASS, ...)

Data flowSmall data set as parameter through CORBAMassive data through GASS, FTP, GridFTP, Socket, ...

CORBA


Data ManagementProject and Workflow Structure

Free hierarchical structure of projects and data:

Cooperative workingUser management and access controlData exchange via data serverSimilar to PDM systems

Check-Out, Check-In

Components

Simulations

Projects

Configuration & data files


TENT Data server

Data ManagementXML & WebDAV

ConceptStorage of project data in XML format on a serverAccess to data using the standard protocol WebDAV

TENT

Other applications

InternetFile Server

DatabaseHTTP/WebDAV

Web

DA

V

HTTP/WebDAV

WebDAV: Web Distributed Authoring & VersioningExtension of the HTTP protocol (locking, meta data, versioning, search, and access control)


Data ManagementData Provenance

Provenance is information in addition to the standard logging which helps answer questions about origins of dataTypical such questions are

Given some data item, what was the simulation case?Given some parameter, in what simulation(s) has it been used?What data has been recorded in a simulation with a specific parameter?What simulations have been run using a given model (aircraft design)?Given two/more simulations with the same setup, what is the result and the difference in provenance?

TENT has integrated a provenance storing and querying service(developed in the EU GRID PROVENANCE project)


ScriptingIntegration, Control and Automation

Applications do not have everything, so scripting is needed.Usage of scripting in TENT

Integration of applicationsWorkflow control (loop constructs, conditional constructs, coupling control)Test scripts for quality assurance automationDebuggingCommand journaling

TechniqueEmbedded Python interpreterScript interpreter in the GUIScript console in the GUIScripting block in workflows (control and code integration)


Application Systems

Virtualaircraft

Space reentry vehicles

AutomobiledesignFlight maneuvers

Turbine engines


Example Application 1 Flight Maneuver Simulation

Interactive simulation environment for the simulation of a freely flying, fully configured, elastic warplaneProject SikMa: X-31


Flight Maneuver SimulationModel

Simplified model for wind tunnel experimentsSimulation for validation of experiments


Flight Maneuver SimulationResults

Coupled simulation: Aerodynamic – flight mechanics – aeroelasticityComputing times: ~ 7 days on large computing cluster (> 64 procs.)Important: Online monitoring and restart capability


Coupling Control

Structure solver(NASTRAN)

WrapperAero elastic

Wrapperflight

mechanics

CFD solver(TAU code)

Wrapperfluid

dynamics

CO

RB

A

Files

Files

TENT GUI

Flight mechanics(SIMULA)

Files

Aero elastic(MATLAB)

Files

HPCmachine

Visualization

Flight Maneuver SimulationIntegration in TENT

Filetransfer

CORBA

Data

IPC

Host/Filesystem


Filetransfer

Programmstart

TENT-Daten/Monitoring

Host/Filesystem

Netzwerk

SIMULASIMULAServer

TAUCodeTAUCode

Desktop machine

GUI VisualizationVisualization

WrapperCFD

WrapperCFD

Cluster

CouplingControl

CouplingControl

WrapperFlight mech.

WrapperFlight mech.

Data server

Batch

-SystemB

atch-System

Flight Maneuver Simulation - Deployment



Flight Maneuver Simulation Online Monitoring of Results


Example Application 2Space Re-Entry Simulation

Simulation of thermal heavily loaded parts of the X-38 space re-entry vehicleFluid-Structure-Thermal coupling


Space Re-Entry Simulation Model

Simplified model for wind tunnel experimentsSimulation for validation of experimentsFocus on flaps and gap


Space Re-Entry Simulation Results

Coupled simulationDLR-TAU code for aerodynamicsANSYS and MSC/NASTRAN for heat transport and deformation in the structure


Coupling with TENTUse of coupling library MpCCI

MpCCI/MPIMpCCI/MPI

TENT Coupling

Configuration

Resource selectionand start

Structuresolver

Co-ProcessStructure

WrapperCo-Process Structure

Couplingcontrol

Wrappercouplingcontrol

CFDsolver

Co-ProcessFlow

WrapperCo-Process

Flow

CO

RB

A

Mp

CC

IM

pC

CI

Files

Files

TENT GUI

Visualization

Filetransfer

CORBA

MpCCI

IPC

Host/Filesystem



New TechnologiesTechnology Upgrade for TENT

TENTGUI with Plug-Ins (proprietary development)Single role for all usersSingle communication protocol (CORBA)Authentication with user name & passwordFile-based data managementModularity in sub systems

RCE (Project SESIS)GUI Plug-Ins Framework (Eclipse)Multiple user rolesVarious protocols (Web Services, Java RMI, CORBA, …)Authentifizierung with certificates (single-sign-on)Data management with complex data structure (and files)Everything is modular

Project SESIS

TENT used modern technologies of the past 10 yearsTechnology upgrade for the next 10+ years in project SESIS


Project SESISGoals

Design in 7 Days in a virtual organization

Early ship design for creation of offersReduction of design timesDistributed design und simulation

Shipyards organizing ship building processIntegration of supplier in design process

Quelle: Flensburger Schiffbau-Gesellschaft mbH & Co. KG


Project SESISSystem Design

System designed with a Plug-In architecture based on the OSGi platform

RCE (Reconfigurable Computing Environment)Advantages

Consistent modularizationGood extensibility und scalabilitySpecificity by configurationUse of many existing Plug-Ins (see Eclipse!)


HardwareGrid Middleware

Project SESISBase-System Layer

Usage of databases and Grid middleware solutionsSESIS supports

CommunicationUpdatesPrivilege managementServices discovery

SESIS

Installation of identical base system on every machine

Specificity by loading additional plug-ins

Installation of identical base system on every machine

Specificity by loading additional plug-ins


Usage ScenarioVirtual Organization

RCESESIS

RCESESIS

DBRCESESIS


SESIS Graphical User InterfaceEclipse RCE w/Plug-Ins

Documents

Software Integration 04052006 - The EU Provenance … · (NASTRAN) Wrapper Aero elastic Wrapper ... ANSYS and MSC/NASTRAN for heat transport and deformation in ... Software_Integration_04052006.ppt