CONFERENCE DEVELOPER INTEL HPC - Intel | Data ......Software-Defined Visualization: Getting the most...

INTEL® HPC DEVELOPER CONFERENCE

Software-Defined Visualization: Getting the most out of ParaView w/ OSPRay

▶ Intel HPC Developer ConferenceNovember, 2016

Paul A. Navrátil, Ph.D.TACC

David E DeMarleKitware

Software-Defined Visualization

Performant Visualization across Hardware Platforms

3

Agenda

▶ Part I-A: Visualization Rendering – A Brief Overview

▶ Hardware configurations and considerations

▶ Rasterization summary

▶ Ray tracing summary

▶ Part I-B: Obtaining SDVis Components (sdvis.org)

▶ OpenSWR – openswr.org

▶ OSPRay – ospray.github.io

▶ ParaView– tacc.github.io/pvOSPRay

▶ Part II: Demonstrations – ParaView + OSPRay

▶ Wavelet data source

▶ Meningioma

▶ FIU ground core sample

▶ Remote Visualization to TACC

4

Part II : Hands On

5

❏ What you need❏ SSIS Intel_Connect pwd “saltlake16”

paraview (download 5.2 RC4)data:

www.paraview.org/Wiki/ Intel_HPC_Dev_Con_ParaView_and_OSPRay_Tutorial

USB’s (please return them)local SMB fileshare:

“afp://172.17.4.105” or “smb://172.17.4.105”.

Workflow Overview – HPC vs. Vis

6

Typical HPC Workflow

Initial Conditions01001101011001

11001010010101

00101010100110

11101101011011

00110010111010

αβγ

ParametersSimulation

Timestep 101001101011001

11001010010101

00101010100110

11101101011011

00110010111010

Timestep n01001101011001

11001010010101

00101010100110

11101101011011

00110010111010

Results

7

Typical Visualization Workflow

Timestep 101001101011001

11001010010101

00101010100110

11101101011011

00110010111010

Timestep n01001101011001

11001010010101

00101010100110

11101101011011

00110010111010

Visualization Algorithms

Geometry

Rendering Images Display

Iterate! 8

Original Community Solution: Move All Interaction to Client for Vis

File Size

100Gbps

10 Gbps 1 Gbps 300

Mbps 54

Mbps

1 GB < 1 sec 1 sec 10 sec 35 sec 2.5 min

1 TB ~100 sec ~17 min ~3

hours~10

hours~43

hours

1 PB ~1 day ~12 days

~121 days >1 year ~5

years

Files too large to move out of data center where they were generated 9

Visualization Rendering Methods

10

Vis Algorithm Renderer

Scientific Visualization Process

▶ the primary goal of visualization is insight

▶ a picture really is worth 1,000 words or (potentially) tera-/peta-/exa-bytes of data

▶ as dataset sizes increase so does the need for scientific visualization

0100110101100111001010010101001010101001101110110101101100110010111010

Raw Data Geometry Pixels

11

Rasterization and Ray-Tracing

▶ Rasterization – project 3D data onto 2D image plane, step across image plane to determine color

▶ Often accelerated by purpose-built hardware

▶ Performance complexity a function of total geometry

▶ Shading typically not physically-based

▶ Ray Tracing – sample 3D data through 2D image plane, simulating light ray travel

▶ Efficient with modern many core, wide vector CPUs

▶ Performance complexity a function of visible data

(acceleration structure build a function of total data)

▶ Physically-based shading “easy” (performance varies)

12

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Starts with data or models that are described in (x,y,z) coordinates. The data usually defines solids or boundaries in terms of their skins. Transform from data coordinates to something called world space.

Object Space

World Space

(tessellated)

13

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Attempt to eliminate any objects that can not possibly be seen. Check endpoints of a triangle, line segment or point and store a flag for inside or outside. Perform a logical and of the endpoints to determine whether the entity is wholly in or out of the view.

14

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Compute normals at the endpoints for lighting calculation:

Ambient Diffuse Specular

15

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Change coordinate systems such that the eye sits at the origin and the viewing plane is defined. This is called eye space.

-X

Y

Z

Right Handed Viewing System

16

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Eliminate any triangles or points that are not within the view frustum (outside the scene) so that we don’t have to draw them.

-X

Y

Right Handed Viewing System

Z

17

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Project 3D objects to 2D Viewing plane. This is a projection from eye space to screen space.

-X

Y

Z

Right Handed Viewing System

18

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Convert the object into pixels using some sort of interpolation (usually linear).

19

Rasterization Pipeline

Modeling Transformatio

ns

Rasterization Display

Trivial Rejection

IlluminationViewing

Transformations

Clipping Projection

Starts with data or models that are described in (x,y,z) coordinates. The data usually defines solids or boundaries in terms of their skins. Transform from data coordinates to something called world space

20

OpenGL Pipeline

Vertex operations such as lighting, clipping, projection and viewport mapping

Series of framebuffer addresses and values.

Series of conditional tests such and modifications such as blending and z-buffering

21

Recursive Ray Tracing Example

A

C

B

DA

CB

D

ray tree

E

E

…

22

Recursive Ray Tracing Example

A

C

B

DA

B C

D

ray tree

primary ray

secondary rays

23

Acceleration Structures

▶ First ray tracers tested every object for intersectionCreated acceleration structures to reduce object tests

▶ Focus object tests on those that are likely to be hit

▶ Bound a complex object with a simpler one if simple test fails, no need for complex test

▶ Partition space to group objects, only test objects when a ray enters their partition

▶ Acceleration structure creates groups of scene data

▶ These groups determine which data are accessed together

▶ Nearby objects accessed together only if they occupy the same part of the acceleration structure

24

Bounding Volume Hierarchy

A

CE

B

D

A

B C

D E

• Subdivides object space

• Bounded regions can overlap

• Must test all regions per level to find nearest intersection

boxes for B and C overlapC and E are nearby in the scene but occupy different parts of acceleration structure

25

K-D Tree

▶ Subdivides scene space

▶ Objects may be duplicated or split to fit

▶ Structure can be unbalanced and deep

A

CE

B

D

A

C

ED

B,C

C overlaps C duplicated in tree

26

Why Ray Tracing?

Jeffrey Howard, Intel Corporation

• Images compare “easy” rasterization with “easy” ray tracing

• Advanced rasterization can simulate some ray tracing effects, but ray tracing produces higher quality and is generally easier to implement

intereflection

caustic

soft shadows anti-aliasing

27

Realistic Optic Effects

Motion Blur Depth of Field

R. Cook, T. Porter, L. Carpenter T. Kim

28

Accurate Simulation of Light Travel

Copyright Audi AG

29

Better Lighting for Visualization

C. Gribble and S. Parker

local shading ambient occlusion

shadows diffuse interreflections

30

Better Lighting for Visualization

C. Gribble and S. Parker

31

Richtmyer-Meshkov InstabilityCarson Brownlee, Aaron Knoll, Paul Navrátil, TACC

Ingo Wald, Carsten Benthin, Sven Woop, Intel

OpenGL version – flat lighting, constant shadows, limited depth perception 32

Richtmyer-Meshkov InstabilityCarson Brownlee, Aaron Knoll, Paul Navrátil, TACC

Ingo Wald, Carsten Benthin, Sven Woop, Intel

Embree RT version – rich lighting, ambient occlusion, improved depth perception 33

Richtmyer-Meshkov InstabilityCarson Brownlee, Aaron Knoll, Paul Navrátil, TACC

Ingo Wald, Carsten Benthin, Sven Woop, Intel

Embree RT version with ‘glass’ planes – integrated, realistic material behavior 34

South Florida Ground Core SampleSade Garcia, Michael Sukop (Florida International University),

Kevin Cunningham (US Geological Survey), Carson Brownlee, Aaron Knoll (TACC)

OpenGL version – flat lighting, constant shadows, limited depth perception35

South Florida Ground Core SampleSade Garcia, Michael Sukop (Florida International University),

Kevin Cunningham (US Geological Survey), Carson Brownlee, Aaron Knoll (TACC)

Embree RT version – rich lighting, ambient occlusion, improved depth perception36

South Florida Ground Core SampleSade Garcia, Michael Sukop (Florida International University),

Kevin Cunningham (US Geological Survey), Carson Brownlee, Aaron Knoll (TACC)

Embree RT version – rich lighting, ambient occlusion, improved depth perception37

Implicit Surface Intersection

▶ Ray tracing can intersect surfaces directly without explicit tessellation

▶ Example: molecular data from materials simulations

▶ “ball and stick” surfaces + potential field volumes

20× – 100× memory savingsand faster image generation! 38

A. KnollA. Knoll

Simulation of N-Acetyl-L-tryptophanamide through a Cellular Membrane

Alfredo Cardenas, Ron Elber (ICES UT Austin)Aaron Knoll, Anne Bowen (TACC), Ingo Wald (Intel)

39

tryptophan locations clearly expressed

Simulation of N-Acetyl-L-tryptophanamide through a Cellular Membrane

Alfredo Cardenas, Ron Elber (ICES UT Austin)Aaron Knoll, Anne Bowen (TACC), Ingo Wald (Intel)

40

Software-Defined Visualization Libraries

41

SDVis Rasterization

▶ Mesa 3D Graphics Library – mesa3d.org

▶ Open-source general OpenGL implementation

▶ Often included in Linux distros

▶ Convenient, if not always performant, default for CPU

▶ OpenSWR – openswr.org

▶ Open-source OpenGL implementation tuned for Intel CPUs

▶ Performant rendering using wide-vector instructions

▶ OpenGL coverage for visualization and more

(integrated into Mesa3D 12.0)

42

SDVis Ray Tracing

▶ GLuRay – tacc.github.io/GLuRay/

▶ Provides basic ray tracing via OpenGL intercept

▶ Fallback to rasterized OpenGL (OpenSWR, Mesa, HW)

▶ Supported in part by NSF award ACI-1339863

▶ GraviT – tacc.github.io/GraviT/

▶ Provides efficient memory management and work scheduling for distributed-memory ray tracing

▶ Uses GLuRay for basic interface, advanced interface for improved performance and feature support

▶ Designed to generalize for use in simulation codes

▶ Supported in part by NSF award ACI-1339863

43

SDVis Ray Tracing▶ Manta – mantawiki.sci.utah.edu

▶ Open-source, generally performant ray tracer

▶ ParaView plug-in available

▶ OSPRay – ospray.github.io

▶ Open-source ray tracer tuned for Intel hardware

▶ ParaView plug-in available (tacc.github.io/pvOSPRay/)

▶ OptiX – developer.nvidia.com/optix

▶ Closed-source ray tracer tuned for NVIDIA hardware

▶ Supports CPU execution (though less performant)

▶ IndeX – nvidia-arc.com/products/index.html

▶ Closed-source DVR tuned for NVIDIA hardware

▶ Requires license to operate

44

Part II : Hands On

45

❏ What you needparaview (download 5.2 RC4)data:

www.paraview.org/Wiki/ Intel_HPC_Dev_Con_ParaView_and_OSPRay_Tutorial

USB’s (please return them)local SMB fileshare:

“afp://172.17.4.105” or “smb://172.17.4.105”.

Part II : Hands On

46

▶ What is ParaView?An application and architecture for display and

analysis of massive* scientific datasets.Client/Server architecture lets it runs on variety of

platforms from netbooks to the largest machines in the world

Open Source, commercially friendly BSD licenseLarge user/developer baseCommercial support available through Kitware

▶ http://www.paraview.org/download/

▶ http://www.paraview.org/Wiki/The_ParaView_Tutorial Presented Monday @ SC

1.1 Trillion Cells on 1/19’th of DOE Trinity & OpenSWR

ParaView

HistoryStarted in 2000 as collaborative effort between Los

Alamos National Laboratories and Kitware Inc. (lead by James Ahrens).

ParaView 0.6 released October 2002.September 2005: collaborative effort between

Sandia National Laboratories, Kitware Inc. and CSimSoft to rewrite user interface to be more user friendly and develop quantitative analysis framework.

3.0 May 2007, Qt, Python, 2d Plots5.0 October 2015 - SWR, TACC pvOSPRay plugin5.1 May 2016 - integrated OSPRay for surfaces5.2 November 2016 - add OSPRay volumes

47

▶ Based on VTKVisualization Pipeline

OSPRay integrated into VTK, so any VTK app can usehttps://blog.kitware.com/ray-traced-rendering-revisited/

48

User Interface

Menu Bar

Toolbars and Shortcuts

Pipeline Browser

Properties PanelApply ButtonSearch FieldShow Advanced

48

▶ http://www.paraview.org/Wiki/The_ParaView_Tutorial Presented Monday @ SC

File->Open “disk_out_ref.ex2”

Filters->Contour

Click on disk_out_ref.ex2 to make it active again

Filters->Extract Surface

Filters->Clip SHOW

HIDE

OSPRay Rendering

Wavelet : procedurally generated volumetric data

Sources->Wavelet

Outline, Surface, Solid Color -> “RTData”, Volume

View Section at bottom of Properties -> Enable OSPRay

Volume->Wireframe, zoom in compare

50

Not colormapped?

Too thin? Thick?

OSPRay OpenGL

Part II : Hands On

51

Filters->Contour isosurfaces

Turn on Shadows

Notes:* scene and self shadowing* speckle shadow bug with

line/cylinders and point/spheres

* “LightKit” 6 lights in scene, mostly above and from camera better controls soon

Part II : Hands On

52

Volumes Cast Shadows w/ OSPRay

Level Of Detail - not at all helpful

53

Unlike rasterization framerate doesn’t really depend on how many primitives in scene. Also switching back and forth is slower than with GL.

Glyph Mode : Uniform Spatial -> All PointsGlyph Type : 2D Vertex -> Sphere

Disable ParaView’s LODEdit->Settings (OSX: ParaView->Preferences) -> Render View -> LOD Threshold to max

Implicit Spheres and Cylinders

54

Filters->Glyph Traditional: Glyph Type = Sphere Ray traced implicit : 2D Glyph, Vertex

Part II : Hands On

55

▶ Scaled Implicits, finer control than “Point Size”

Filters-> Python Calculator : expression = RTData/max(RTData)uncheck “Interpolate Scalars before mapping”check “OSPRay Use Scale Array on”change from “OSPRay Scale Array = RTData” to “= result”Hit “OSPRay Scale, Edit” to change value->Radius transfer function “

Part II : Hands On

Real Data: Meningioma

longitudinal study

Load MRBrainTumor1,2

isocontour

slice

extract VOI, volume render

56

Part II : Hands On

Real Data:

FIU

57

Part II : Hands On

▶ Demonstration : remote visualization

▶ https://portal.tacc.utexas.edu/user-guides/maverick#visapps-paraview

▶ possibly File->Connect (internal) or TACC startup with desktop forwarding

▶ Run parallel job

▶ Explain secondary rays problem, GraviT, Intel’s MPI DMP volumes

58

TACC Remote Visualization

59

Why Visualize Remotely?

- Avoid moving data (especially TB or PB!)

- Access many nodes for enhanced analysis powerHundreds to thousands of compute nodes available

- Visualize together with simulation computation- in situ / co-processing- computational steering

TACC Remote Visualization

60

Two Connection Models:

- TACC Visualization Portalhttps://vis.tacc.utexas.edu/

- SLURM Job on Login Nodessh stampede.tacc.utexas.edu ORssh login-knl1.stampede.tacc.utexas.edusbatch -A <acct> /share/doc/slurm/job.vnc <opts>

- Both use VNC remote desktop

TACC Visualization Portal

61

- Which machine- Project to be billed- Session Type

- VNC- Rstudio- iPython / Jupyter Notebook

- VNC desktop resolution- Number of compute nodes- Number of processes per node- Click < Start Job >

- Before first run, must set VNC Password(different from login, 6-8 char)

Need TACC or XSEDE login credentials

62

63

Enter VNC password here

TACC Visualization Portal

64

- VNC job blocks on this xterm (black background)Close it to end job

- Closing the web browser DOES NOT end the VNC job

- Allows you to reconnect without losing work

- If not expressly ended, job will run until time limit reached

65

Select Jobs tab for running jobs

Click Terminate Session to end job

VNC SLURM Job

66

After connecting to login node:vncpasswd # only needed before first executiontouch vncserver.out # only needed if file does not existsbatch -A <acct> /share/doc/slurm/job.vnc <opts>

for example:sbatch -A A-ccvis /share/doc/slurm/job.vnc -geometry 19020x1080

monitor job status by following output file:tail -f vncserver.out

VNC SLURM Job

67

When job runs, displays VNC port connection info:Created reverse ports on Stampede KNL loginsYour VNC server is now running!To connect via VNC client: SSH tunnel port 36602 to

login-knl1.stampede.tacc.utexas.edu:36602 Then connect to localhost::36602

To tunnel port from localhost (for secure connection):ssh -f -N -L 36602:login-knl1.stampede.tacc.utexas.edu:36602 \

login-knl1.stampede.tacc.utexas.edu

Then connect VNC viewer to localhost:<VNC port>vncviewer localhost:36602

FIU Dataset Demo

68

Groundwater flow through karst limestone core sampledata courtesy of Florida International University and U.S. Geological Survey

Data available at /work/00401/pnav/data/fiu_nhdr

We will visualize: the karst substrate (rho, ~0.9 GB)the fluid flow (u, ~1.4 GB)

69

Load nhdr data

70

Create stream tracer

71

Enable ambient occlusion (AO)

72

Create contour

73

Create clip plane

74

▶ What’s next?▶ In-situ optimizations▶ Progressive Refinement, Path Tracer, Lights, Materials▶ Secondary Rays in DMP

▶ Want to know more?▶ The ParaView Tutorial, Monday at SC16▶ Come see us at the TACC, Kitware and Intel booths▶ Sunday talk by Paul▶ Sunday talk by Dave

Thank you!

INTEL® HPC DEVELOPER CONFERENCE