slide 1

Visualization Infrastructure and Services at the MPCDF

Markus Rampp & Klaus ReuterMax Planck Computing and Data Facility (MPCDF)

(visualization@mpcdf.mpg.de)

Interdisciplinary Cluster Workshop on Visualization

Garching, Nov 4, 2015

slide 2

Outline

Topics

. overview remote visualization services

· hardware & software infrastructure

· project support

. challenges and outlook

MPCDF Visualization Team

. people involved (part-time, main focus is HPC)

Elena Erastova (visualization projects)

Klaus Reuter (software and hardware coordination, consulting, projects, training)

Markus Rampp (consulting, projects, training)

slide 3

Visualization infrastructure for the Max-Planck-Society

MPCDF visualization services:

. provide central software and hardware infrastructure for remote visualization

. target: interactive data exploration & analysis, presentation

. support for adaptation and instrumentation of simulation codes

. guidance for selection, adoption and usage of analysis & visualization software

. dedicated support for individual (particularly demanding) visualization projects

Main conceptual challenges:

. broad range of disciplines in MPG: Plasmaphysics, Astrophysics, . . . , comp. Biology

y many different scientific contexts

y variety of simulation codes: ”home-grown”, commercial, open-source, third-party, . . .

y non-standardized, heterogeneous data structures and formats

y ”legacy” analysis pipelines, . . .

. massive datasets from HPC simulations:(massive: amount of raw data, memory requirements, complexity)

◦ multidimensional (3D + time), multi-variate data

◦ ”unusual” grids: meshless data, special curvilinear coordinates, . . .

slide 4

Visualization infrastructure for the Max-Planck-Society

Status

. consulting & dedicated project support since 2008

. MPG visualization cluster operational since Sep. 2010

. open to all MPG scientists and collaboration partners

. many projects supported (some highlights by K. Reuter)

. broad userbase (beyond Garching campus)

Rationale for centralized visualization in the MPG:

. a necessity for a HPC centre rather than an optional service

· huge amounts of output data produced by HPC simulations

· transfer of raw data for local analysis & visualization no more possible

· even dumping the RAM is becoming prohibitive due to I/O constraints

y in-situ visualization (not covered here)

· visualization requires HPC-like resources (specialized hardware, housing, . . . )

· requires substantial expertise on methods, software, . . .y sustainability

. Technological prerequisites

· efficient and transparent remote rendering solution via WAN: VirtualGL/TurboVNC

slide 5

Central visualization infrastructure: technical prerequisites

traditional ”X over ssh” (e.g. ssh -X)

. 3D data are transfered to the client

. fails to deliver interactive frame rates

. uses X-server/graphics card of the client

y not suited for 3D applications

server-side rendering

. only (compressed) image stream is transferred

. delivers interactive frame rates with moderate WAN bandwidth

. uses X-server/graphics card(s) of the server

. generic solution (OpenGL)

. mature software solutions/products:

· VirtualGL/TurboVNC (Open Source, ex SUN)

(original illustrations by L. Scheck, by courtesy of LRZ)

slide 6

Remote-visualization cluster

Focus:

. enable our (geographically dispersed) scientific users to perform complex visualization tasks with-out special technical prerequisites (software, hardware)y remote visualization

Hardware overview (HP cluster)

. 5 ”standard” visualization nodes each equipped with:

· 2 Intel quadcore CPUs: 8 cores, 144 GB RAM

· 2 NVidia FX 5800 graphics cards

. 1 ”high-end” visualization node:

· 4 Intel hexacore CPUs, 24 cores, 256 GB RAM

· 2 NVidia FX 5800 graphics cards

. 1 login node: viz00.rzg.mpg.de

. dedicated disk system (GPFS, ' 30 TB)

. GPFS filesystem /ptmp of HPC system Hydra mounted

. 2 graphics workplaces (active stereo) in MPCDF offices

Software stack

. SLES 11 (MPCDF standard cluster setup), VizStack middleware (GPUs, X-servers, . . . )

. web-based reservation system (HP, MPCDF)

. remote rendering solution: VirtualGL/TurboVNC (free clients for Linux, MS Windows, MAC)

slide 7

User interface

Remote desktop (via TurboVNC)

. a standard desktop in a separate window

. application agnostic

. desktop icons for main applications

. preconfigured according to session properties(number of GPUs, CPUs)

. linux terminal: vglrun <command>

slide 8

Software for Visualization

Software for interactive data visualization and analysis

. VisIt: main workhorse for 3D analysis

. Paraview: main workhorse for 3D analysis

. VAPOR: large-scale data (requires preprocessing)

. Voreen: volume rendering

Tools and libraries

. GNU R, IDL, MATLAB, gnuplot, . . .

. VTK, HDF5, SILO, . . .

. mencoder: scripts for x264 encoding of movies

Special-purpose software

. Splotch: a (non-interactive), parallel ray tracer for SPH data.

. VMD (Visual Molecular Dynamics): a molecular graphics software.

. POV-Ray: a freeware multi-platform ray-tracing package.

. Blender: an open source, cross platform suite of tools for 3D creation.

slide 9

Application support

Documentation

. http://www.rzg.mpg.de/services/visualisation/

Training

. courses (http://www.rzg.mpg.de/services/visualisation/scientificdata/presentations)

· K. Reuter: RZG-Services zur Visualisierung wissenschaftlicher Datensatze, DV-Treffen der Max-Planck-Institute,Gottingen, Sep 15, 2010

· K. Reuter: Scientific Visualisation Services at RZG, Seventh GOTiT High Level Course, Garching, Oct 19, 2010

· M. Rampp: Introduction to VisIt, LRZ course on ”Visualisation of Large Data Sets on Supercomputers”, 2010 –2011

· M. Rampp: Introduction to VisIT, 11th Summer school on scientific visualization, CINECA Bologna/Italy, Jun 13,2012

· M. Rampp: Visualization of HPC simulation data: overview and tutorial, ISSS-12, Prague (2015)

· overview talks at Max-Planck-Institutes: MPA, Garching (2009), FHI, Berlin (2011), MPI f. Biophysics, Frankfurt(2014), . . .

Project support

. dedicated support for visualization projects at different levels:

· from basic ”first level” support to comprehensive visualization and analysis tasks

· requires (considerable) insight to scientific domain

· several completed and ongoing projects, in close collab. with the users/scientists:http://www.rzg.mpg.de/services/visualisation/scientificdata/projects

. contact: visualization@rzg.mpg.de

slide 10

MPG/MPCDF reference applications

Projects with MPCDF support (in close collab. with research groups)

. application domains:

· Plasmaphysics: MHD turbulence simulations for nuclear fusion research (IPP)

· Stellar astrophysics: Supernova simulations, NS mergers (MPA)

· Cosmology : Structure and star formation (MPE)

· Molecular dynamics: Materials research for plasma-wall-interaction (IPP), DFT (FHI)

· CFD: DNS simulations of turbulent Taylor-Couette flows (MPI-DS)

. data structures/grids:

· regular: cartesian, polar (2D, 3D), block-structured (”Yin-Yan”)

· irregular: (mapped) point clouds

. data sizes, dimensions:

· up to 20483 (cartesian), 1000× 180× 360 (polar), 2048× 769× 1153 (cylindrical)

· up to ' 106 (particles in 3D), ' 107 (nodes in 3D unstructured mesh)

· all: multi-variable (scalar, vector), time-dependent

see also:

http://www.rzg.mpg.de/services/visualisation/scientificdata/projects

Presentation by K. Reuter

slide 11

MPG/MPCDF reference applications

slide 12

MPG/MPCDF reference applications

slide 13

MPG/MPCDF reference applications

slide 14

MPG/MPCDF reference applications

slide 15

MPG/MPCDF reference applications

slide 16

Challenges & outlook

Technological

. hitting the limits of general-purpose software tools (VisIT, Paraview): interactivity, memory

demands: O(10003) data

y use GPUs in HPC system, e.g. MPG Hydra with Nvidia K20x GPUs

y enables in-situ visualization: a big buzz or something interesting to watch ?

◦ basic technique: implement library calls in simulation code(APIs for C, FORTRAN)

◦ mediates callbacks to visualization tool

◦ supported by Paraview (”catalyst”), VisIT (”libsim”)

slide 17

Challenges & outlook

Organizational

. dedicated project support is of key importance (but not scalable)

. beyond scientific data analysis and insights

· ever increasing standards and expectations for public understanding of science

· are we the right people to ”direct” professional animations (TV documentaries) ?

· do we need ”real” scientific data for this at all ?

· credits ? y Max-Planck Visualization Award

. highly efficient and innovative algorithms often don’t make it into usable software

slide 18

Challenges & outlook

Organizational

. dedicated project support is of key importance (but not scalable)

. beyond scientific data analysis and insights

· ever increasing standards and expectations for public understanding of science

· are we the right people to ”direct” professional animations (TV documentaries) ?

· do we need ”real” scientific data for this at all ?

· credits ? y Max-Planck Visualization Award

. highly efficient and innovative algorithms often don’t make it into usable software

Outlook

. remote visualization on HPC system Hydra

(to replace visualization cluster in early 2016)