MuPIF Training
27.-28. March, 2017Czech Technical University, Faculty of Civil Engineering, Prague
AgendaMar 27
13:30-15:15 MuPIF: Introduction, design and installation (BP)
15:45-17:30 MuPIF: Network communication, examples (VS, ALL)
18:30 Dinner in “Bernard PUB Cesta Časem”, Na Kocínce 210/3
Mar 28
8:30-10:00 MuPIF: Coupled/linked models
10:30-12:15 MuPIF: API Development.
12:30-13:30 Lunch
13:30-15:15 Practical session, Discussion
2
Part 1: Introduction to MuPIF platform
Bořek Patzák, Vít Šmilauer, Martin Horák
Czech Technical University, Faculty of Civil Engineering, Prague
3
Content of Part 1• MuPIF Platform design – an overview• Distributed aspects• Platform installation• Basic platform operations, interactive session
– Connecting applications– Fields, field mapping– Visualization
4
MuPIF integrative role in Composelector
DEC
ISIO
N S
UP
PO
RT A
PP
SLC
E,
Ligh
t W
eig
hti
ng,
Vis
ualiz
ati
on
Cost
ing,
Res
Subs
Ass
ess
ment.
etc
…
DEC
ISIO
N S
UP
PO
RT A
PP
SLC
E,
Ligh
t W
eig
hti
ng,
Vis
ualiz
ati
on
Cost
ing,
Res
Subs
Ass
ess
ment.
etc
…
The simulation pipeline in MuPIF consists of top-level script in Python language linking and steering the individual models.
Distributed computing
Database
MaterialModel #3
MaterialModel #2,
Visualization
MaterialModel #5
MaterialModel #4
MuPIF● Steering scripts
for material and process modelling
● Database interrogation
● Integrates computational facilities
● Seamless data flow using VPN
● Material and process modelling
● Business model and optimization
SimulationChain
Validation
ProcessModel #1
BusinessModel,
Optimization
MaterialModel #1
Validation
Consider car as example• Complex system
• Need for simple interface hiding complex details allowing to operate the car
Introduction to OO design
Introduction to OO design
Consider car as example• Complex system
• Need for simple interface hiding complex details allowing to operate the car
• Different car types: need for standard interface allowing to drive all car types
Car Interface
Consider car as example• Object is encapsulation
of attributes and methods• Methods define interface
to interact with object• Classes can form
hierarchies to express similarity (inheritance)
CarEngineGearWheels[4]PlateID...
start()stop()turnLeft (angle)turnRight (angle)accelerate (rate)decelerate (rate)
Bus
NumberOfSeats
openDoor()closeDoor()
Introduction to OO design
10
Summary
• Objects represent abstraction of (any) entity in the modeling space. • Class is definition of object. Object is an instance of a class.• Object is encapsulation of data and methods • Objects can be manipulated and data exchanged using methods only• Classes can form hierarchical structure allowing to describe similarity or
a-kind-of relation (Inheritance) between classes• Abstract classes define public interface to be implemented by derived
classes
• Interface concept allows to hide internal data storage formats (natural supports for multiple data formats, design is data format independent)
Introduction to OO design
11
MuPIF simulation workflows
Design based on interacting, distributed objects (components)
• MuPIF is trying to make an abstraction of models and complex data to enable the idea of plug-and-play work-flows
• Models and complex data represented as software components, implementing common interfaces
• Top level classes for specific component group define common interface shared by all derived classes
• Python language extended by introduction of high level classes
MuPIF Platform – Design ideas
Advantages of design based on interacting objects (components)
• Internal data and storage formats naturally hidden behind common interface, native support for multiple data formats (objects manipulated using the same interface)
• Instead of trying to standardize data structures, MuPIF is focused on identification and standardization of methods on similar components (models, fields, microstructures,...)
• These methods are defined by interfaces defined by top-level classes and to be implemented by derived classes
•• Interface concept allows to manipulate similar components using same
generic interface – abstraction• Models will get data and operations on data in one consistent package,
do not have to interpret data themselves• Meta data can be attached to any component• Components in MuPIF can represent local as well as remote data, can be
distributed
MuPIF Platform - Design
Application
getField (id)getProperty (id)…
Internal Data
Interface
Field
evaluateAt(pos)getMesh ()…
Internal Data
Interface
Application Class representing an external application – Model or Postprocessor. Provides methods for getting and registering properties, fields, and functions, processing specific solution step, update state, termination, etc.Derived classes: • micress (acsrvappmic1.access.rwth-aachen.de),• mmpraytracer (mmpserver.erve.vtt.fi), • LAMPS
Field Field is a scalar, vector, or tensor quantity defined on a spatial domain (represented by the Mesh class).
Property Characteristic value of a problem, which has no spatial variation.
Function Function is an object defined by mathematical expression and can be a function of spatial position, time, and other variables.
Mesh Representation of a computational domain as a collection of Cells and VerticesDerived classes:• UnstructuredMesh
MuPIF Platform – Base classes
Vertex Represents Vertex.
Cell Interpolation Cell, whose geometry is defined using verticesDerived classes:2D: Triangle_2d_lin, Quad_2d_lin3D: Tetrahedron_3d_lin, Brick_3d_lin
TimeStep Represents solution time step.
Units Representation of units. Allows to convert units and check for unit compatibility.
MuPIF Platform – Base classes
Service Description__init__ (self, file) Constructor. Initializes the application.
ARGS:• file (str): path to application initialization file.
getField(self, fieldID, time) Returns the requested field at given time. Field is identified by fieldID.ARGS:• fieldID (FieldID): identifier• time (double): target timeReturns: Returns requested field (Field).
setField(self, field) Registers the given (remote) field in application.ARGS:• field (Field): remote field to be registered by the applicationReturns: None
getProperty(self, propID, time, objectID=0)
Returns property identified by its ID evaluated at given time.ARGS:• propID (PropertyID): property ID• time (double): time when property to be evaluated• objectID (int): identifies object/submesh on which property is evaluated
(optional)Returns: Returns representation of requested property (Property).
setProperty(self, property, objectID=0) Register given property in the applicationARGS:• property (Property): the property class• objectID (int): identifies object/submesh on which property is evaluated
(optional)Returns: None
Example: Application class interface
Service DescriptionsolveStep(self, tstep, stageID=0, runInBackground=False)
Solves the problem for a given time step. Evaluates the solution from actual state to given time. ARGS:• tstep(TimeStep): solution step• stageID(int): optional argument identifying solution stage• runInBackground(bool): if set to True, the solution will run in background (in
separate thread), if supported.
Returns: NonefinishStep(self, tstep) Called after a global convergence within a time step.
ARGS:• tstep(TimeStep): solution stepReturns: None
getCriticalTimeStep(self) Returns the actual (related to the current state) critical time step increment (double).Returns: critical time step (double)
getAssemblyTime(self, tstep) Returns the assembly time related to a given time step. The registered fields (inputs) should be evaluated in this time.ARGS:• tstep (TimeStep): solution stepReturns: assembly time (double)
terminate(self) Terminates the application.Returns: None
getAPIVersion(self) Returns the supported API version.Returns: API version (int)
Note: Full API specification part of public D1.1: Application Interface specification
Example: Application class interface
Native API implementation • Direct calls to application's
function• Python interface (python code,
wrapper-Swig, Boost)
Indirect API implementation• Wrapper around I/O streams or
files, more common
MuPIF MuPIF
Application 1
API
Application.lib
Application 1
API
Application.exe
Application interface implementation
ApplicationInit()solveAt (timeStep)getField (id)setField (field, id)
All components of the same type can be manipulate using the same interface defined by abstract class, internal details (data storage, functions) hidden
FieldevaluateAt (pos, time)
Simulation Scenario
CFD Celsian
Micress
FEM Field
HDF5 Field
Platform user Component developer
Key features of the platform
23
Transparent, distributed object system fully integrated into Python using so-called proxies.
A proxy is a special kind of object that acts as if it were the actual object, forwarding the calls to the remote objects, and pass the results back to the calling code.
Optional SSL based encryption for security
Local scenario
Distributed scenario
App1 App2
App1
App2
App2 Proxy
Name server
Pyro
Internet
Distributed environments
24
obj1 obj2
A proxy is local representation of remote object. • It intercepts the method calls you would normally do on
an object• transfers the call with arguments to the computer that
contains the remote object • transfers the results back to the caller
Calling code doesn’t have to know if it’s dealing with a normal or a remote object, because the code is identical.
obj1 obj2obj2 Proxy
Distributed environments, remote objects
Local/Distributed environmentsDistributed scenario Transparent, distributed object system
fully integrated into Python using so-called proxies.
Name server for resolving remote objects locations
Optional SSL based encryption for security
App1 App2 App1 App2App2 Proxy
Pyro name server
Pyro
Local scenario Communication between objects
by direct method calls
25
Communication with remote objects
• Pyro will automatically take care of any registered objects that you pass around through remote method calls, replace them by a proxy automatically.
• The receiving side obtains representation of remote object instead of a local copy.
• This feature can be enabled or disabled (AUTOPROXY config item)
26
Communication with large remote objects - possible scenarios
• Direct communication with remote object
• Direct aggregated communication
• Communication with local copy
ff Proxy
Pyro
c = []for p in points: c.append(p.getCoordinates())f.evaluateAt(coords)
fCopy of ffor p in points: f.evaluateAt(getCoordinates())
ff Proxy
Pyro
for p in points: f.evaluateAt(p.getCoordinates())
27
Communication with large remote objects - possible scenarios
Benchmark of remote Field Evaluation – 2D unstructured mesh (10k cells)– 1000 random field evaluations
Coping the remote field took 0.39 [s]
Evaluation method Time [s]
Separate remote field evaluation 7.52
Batched remote field evaluation 3.15
Separate local field evaluation 2.50 + 0.39
Batched local field evaluation 2.57 + 0.39
28
MuPIF installation
MuPIF Platform – how to access
• From the MuPIF pages (mupif.org)• Directly on SourceForge
http://sourceforge.net/projects/mupif/– Access/browse the code– Register for notifications/updates– Get support (Forum, Bug tracker)
• Getting the code directly via GITgit clone git://git.code.sf.net/p/mupif/code mupif.git
30
Setting up the platform
• General dependences:– Python (2.7.x recommended), – Python modules (Pyro4, pyvtk, NumPy)– ssh client (ssh, putty), ssh server
• Recommended: – Pip (tool for installing and managing Python
packages)– Paraview (vtk based visualization tool), – GIT client (distributed version control system)
31
Recommended setup on Windows
1. Install Anaconda scientific python package: https://store.continuum.io/cshop/anaconda/
2. Install mupif package
3. Recommended● VPN client or putty (ssh client, http://www.putty.org/)● Paraview (http://www.paraview.org/)
32
$ pip install mupif
Recommended setup on Unix/Linux
1. Install Python 2.x using your package manager or download installation package https://www.python.org/downloads/
2. Install pip (package manager or https://pip.pypa.io/en/latest/installing.html)
3. Install required python modules using pip:
4. Install VPN or ssh clients
33
$ pip install mupif
MuPIF directory structure
34
MuPIF_TOP_DIR - contains source code and other files of the MuPIF package +--mupif - contains source code of the MuPIF package | |--doc - documentation (reference manual and User guide) | |--examples - examples and tests | |--tests – unit tests | |--APIs – available application interfaces | |--Physics - module for units | |--tools - various supportive tools | |--*.py - MuPIF classes |--__init__.py - description of MuPIF module |--README.txt - general description |--setup.py - support for setuptools |--MANIFEST.in - support for setuptools
Typical MuPIF_TOP_DIR locations● C:\Users\user\Anaconda2\Lib\site-packages\mupif (Anaconda + pip install mupif)● /usr/local/lib/python2.7/dist-packages/mupif (Linux + pip install)
35
•Web interface http://sourceforge.net/projects/mupif/
MuPIF homepage – Git, Tickets
•Tickets – bug reporting, wishes
36
•mupif/examples/testExamples.sh – routine tests for Example01-Example07, Example09, Example10•Common configuration in mupif/examples/conf.py•These tests are routinely run after each commit on github.com•Test both for python 2.7.* and 3.0.*•Email to the developers if any problem
MuPIF routine tests
37
Overview of MuPIF tests
Local apps
Distributed apps
Direct access
Indirect access
Interpolation, VTK, serialization
Job Manager
Units
Example01
Example02
Example03
Example04
Example05
Example06
Example06-JobMan
Example07
Example09
Example10
• Standard tests with used features / extensions
MuPIF interactive session
Interactive session
• You can start interactive Python session and type commands directly
• Perhaps more convenient approach is to create a new simple script in editor (notepad++) and then run it using python
$ python>>>
$ edit demo.py…$ python demo.py
39
Python basics
• No need to declare variables
• Code blocks are defined by their indentation. "code block" means functions, if statements, for loops, while loops, and so forth. Indenting starts a block and unindenting ends it. There are no explicit braces, brackets, or keywords.
$ python>>> value=1.0
>>>for i in range(10):>>> print i>>>
40
Python basics
• To create an instance of a class (object):
• To invoke a method on object:
$ python>>> import sys # import sys module >>> from mupif import Vertex # import Vertex class>>> v = Vertex.Vertex (1, 1, (10., 10., 0.)) # create an instance of Vertex
>>> print v.getCoordinates()
Demo01.pyDemo01.py
Demo01.pyDemo01.py
41
• Simple example to illustrate Filed evaluateAt,
vtk export, and data exchange between apps.• Simple mesh generator on rectangular domain• Field mapping, result in vtk
42
Simple example – interactive session
Interactive sessionimport sys # import sys module to modify system pathsys.path.append('../..') # add path to mupif
from mupif import FieldID # include definition of some mupif classesfrom mupif import Fieldfrom mupif import ValueType
import demoapp # import a simple demo applicationimport meshgen # import a simple mesh demo module
43
Interactive sessionBasic field operations
app = demoapp.demoapp(None) # create instance of demo applicationprint app.getApplicationSignature() # print application signature
f = app.getField(FieldID.FID_Temperature, 0.0) # get sample field defined by demo appprint f.evaluate((5.0, 5.0, 0.0)) # evaluate field at given position
#evaluating field outside domain -> error#correct waytry: print f.evaluate((5.0, -5.0, 0.0)) #evaluate field outside domainexcept ValueError as e: # catch ValueError exception print "Warning: point is outside the domain“ print 0.0
#make some graphicsf.field2Image2D(title='Temperature', fileName='thermal.png')#export field to vtkf.field2VTKData().tofile('example1')
Demo02.pyDemo02.py
44
Interactive sessionSimple field mapping
#generate target mesh using meshgen: meshgen(origin, size, nx, ny)mesh2 = meshgen.meshgen ((0.0, 4.5), (10.0, 1.5), 50, 10);
values=[]; #list to store interpolated vertex valuesfor i in mesh2.vertices(): #loop over target mesh vertices val = f.evaluate(i.getCoordinates()) #evaluate source field in target mesh vertices values.append(val)
#create new field on target mesh using interpolated valuesf2=Field.Field(mesh2, FieldID.FID_Temperature, ValueType.Scalar, None, 0.0, values);#output vtk representation of mapped fieldf2.field2VTKData().tofile('example2')
Demo03.pyDemo03.py
45
Interactive sessionSimple field mapping – visualization
Visualize the vtk file(s) in paraview– File->Open, select vtk file– Filters->Alphabetical->Warp by scalar
46