Scalable Visualization Toolkits for Bays to Brains

NPACI: National Partnership for Advanced Computational Infrastructure

SDSC

The Scripps Research Institute

U. Texas, Austin

U.C.L.A.

SIO/MIT

U. California, Davis

Mississippi State U.

Scalable Visualization Toolkits for Bays to Brains


Motivation• Support analysis, filtering, and compositing

– Larger-than-core (and swap) volumetric data sets

– Multi-modal and time-varying data

– Multiple data sets simultaneously

• And...– Do efficient data movement

– Execute well on parallel architectures

– Integrate easily w/existing applications & toolkits

– Provide interactive user interfaces for volume exploration

CT

Cryosection

Classification


Motivating Applications• Develop state-of-the-art visualization of:

– Brain data• 1789 TIF color images

• 1472 x 1152 x 1789 x 3 (bytes) = 8.5 GB

– Ocean data• 2160 x 960 x 30 x 4 (bytes) = 0.2 GB

• 237.3 (MB) x 128 (time-steps) = 29.7 GB

• Surface height, salinity, temperature, velocity

• Demonstrate use beyond these fields


Toolkits

Function libraries providing scalable visualization features for new and existing volumetric applications


Layered Toolkit Architecture• Lowest layers provide access

to larger-than-core data sets– Automatic paging of

N-dimensional mesh data

– Local and remote (SRB) files

• Higher layers providelarge-data visualization– Optimized for best data movement

– Multi-data set scenes

– On-the-fly isocontouring during rendering

Volume Scene Toolkit

SRB & Local I/O

Isocontouring Toolkit

Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Mesh Toolkit• Manage a paged mesh (array-like)

– An N-dimensional grid of values

– Spatial data & time-series

– Arbitrary data content

– Multiple file formats

– Volume metadata

– Operations like filtering, sub-sampling, ...

• Layered atop local I/O and SRB


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Mesh Toolkit• Intelligent pre-fetch by encapsulating common

access patterns– Random access (slow)

– Structured access (faster)

– Storage-order access (fastest)

• Neighborhood-aware paging– Page-in nearby data in N-dim.

– Convolution filtering, ...


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

5 13 24679 8

51 32 4 6 7 8 9

51 32 4 6 7 8 9

0 1 2 3 4

5 6 7 8 9

10 11 12 13 14

15 16 17 18 19

20 21 22 23 24

Current center Current center elementelement

Keep neighboring Keep neighboring elements paged-in as wellelements paged-in as well

Filter Filter windowwindow


Mesh Toolkit• Status

– C++ and Java implementations

– Fully documented

– SRB integration in progress

– In active use for large data manipulation

– Software release at 2001 All-Hands (with SRB)


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Volume Scene Toolkit• Define a scene containing multiple data sets

– Leaf nodes are data sets, functions, ...

– Interior nodes composite, filter, ...

– Transforms align overlapping data sets

• Evaluate it to generate results– Client defines the scene

– Server on big iron evaluates it

– Layered atop Mesh Toolkit

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


ClientClient

ServerServer


Volume Scene Toolkit• Cut, Segment, Filter, and Combine data

– Manage multiple data sets of arbitrary data content

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Toolkit• Status

– C++ prototype implementation

– In active use for multiple data sets• Head and brain data

• Tissue and cell data (Science cover)

• Other (Orion, ...)

– Software release in 2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



VolRen Toolkit• Data-efficient isocontouring

– Isocontours produce surfaces at chosen data values• Large data produces very large isocontour polygon sets

– Toolkit uses a fast seed-based approach• Small amount of pre-computation speeds up

isocontouring on later user requests

– Works from compressed data• Parallel visualization of compressed streams

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


VolRen Toolkit• Merged isocontouring and rendering

– Isocontour extraction during volume rendering• Avoids creation & storage of large polygon sets

– Raycasting rays intersect volume &isosurfaces, properly mixing colors and shading

• Parallel out-of-core algorithm– Image-space partitioning

– Master-slave structure

– Low data communication cost due todata pre-processing and reorganization


VolRen Toolkit• Available as a toolkit and a stand-alone system

– Can integrate into applications, or

– Client-server system for rendering of high-resolution videos

– Client-server system for interactive exploration of time-dependent data

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


ClientClient

ServerServer

Param


VolRen Toolkit• Status

– C++ implementation

– Additional features in development (incorporation of streaming images and geometry- progressive)

– In active use for multiple data sets (structured and unstructured data)

– will layer atop mesh-toolkit

– Software release 1st quarter 2001 (toolkit)

– remote access (client-server) release 2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Rendering Toolkit• High-performance perspective volume renderer

– Optimized for parallel architectures

– Efficient data movement for larger-than-core rendering

• Available as a toolkit and a stand-alone renderer– Can integrate into applications, or

– Client sends parameters

– Server on big iron renders usingserver-side high-resolution data

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


ClientClient

ServerServer

VISTA


Rendering Toolkit• Render volume meshes

– Perspective ray-casting of individual volume data sets

– Scenes voxelized first at arbitrary resolution

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

Volume Scene Toolkit VISTA


Rendering Toolkit• Render volume scenes

– Perspective ray-casting of volume scenes

– Only evaluates scene where needed

– Particularly good for time-varying scenes

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Rendering Toolkit• Status

– C++ prototype implementation (based on Orion renderer)

– In active use for multiple data sets

– Software release mid-2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



3D GUI Toolkit• Common GUI components for 3D

– Generic parts for quick GUI development

– 3D drawing windows

– 3D manipulators (shape handles)

– 3D input device handlers

– Point cloud & textured volume shapes

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



3D GUI Toolkit• Status

– Java partial implementation

– Additional features in development

– In active use in multiple 3D viewers


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Tools

Stand-alone interactive applications and serversbuilt using the scalable visualization toolkits


Multi-Mesh Viewer• Interactive multi-mesh viewer

– Built atop Mesh & 3D GUI Toolkits

– Client-server access to Rendering Toolkit

– Uses standard 3D graphics hardware

– Single and multiple volume viewing

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Interactive

Rendered


Multi-Mesh Viewer• Data-efficient visualization

– Adjustable viewing resolution

– Movable region-of-interest

– Client-side proxy volumes forhigh resolution data

– Client-side interaction withserver-side rendering

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Interactive Rendered


Multi-Mesh Viewer• Status

– Java implementation

– Portable: PC, SGI, …

– SRB integration in progress

– In active use to interactively view meshes

– Software release at 2001 All-Hands (with SRB)

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


Python Volume Viewer• Hardware accelerated visualization

– Built atop Mesh Toolkit +DejaVu, VLI, & Python language

– Uses VolumePro rendering hardware• Renders 2563 volume in real time

– Interactively adjust color, lighting, ...

PVV


Python Volume ViewerSRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications


• Data-efficient multi-resolution visualization– Interactive zoom between low & high resolution data

PVV


Python Volume Viewer• Status

– Python using C++ toolkit implementation

– Portable: PC, Sun, SGI

– In active use to interactively visualize meshes and explore lighting, color-table, opacity and viewing parameters

– Software release in 2nd quarter 2001

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

Volume Scene Toolkit PVV


Java Mesh Visualizer• Integrated visualization tool

– Built atop Mesh & 3D GUI toolkits +Shastra collaboration suite

– Client-server access to data servers,renderers, and isocontouring

– Web-based applet orstand-alone application

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications

Volume Scene Toolkit JVisTool


Java Mesh Visualizer• Flexible visualization

– Adjustable cutting planes

– Isocontouring control

– Integrated collaboration features

– Client-side proxy volumes forhigh resolution data

– Client-side interaction withserver-side rendering & isocontouring

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Java Mesh Visualizer• Status

– Java implementation

– Portable: PC, SGI, …

– In active use to visualize meshes and isocontours


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Editor• Interactive volume scene editor

– Built atop Mesh, Scene, & 3D GUI Toolkits

– Client-server access to Rendering Toolkit

– Diagramatic scene structure editing

– Client-side interaction and editing withlow-resolution proxy volumes

– Server-side rendering withhigh-resolution volumes

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Editor• Data-efficient multi-mesh visualization

– Efficient compositing of multiple large meshes

– Multi-modal & time-varying visualization

– Interactive volume scene layout

– Interactive control of filters, compositing, lighting, …

– Scene and viewpoint animation control

SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Volume Scene Editor• Status

– C++ early prototype complete

– Prototype in active use

– Java re-design in progress


SRB & Local I/O


Rendering Toolkit

3D GUI Toolkit

Mesh Toolkit

Applications



Visualization Results

New techniques to better visualize large, complex data sets - from Bays to Brains and beyond...


Brain Visualization• Cleaning cryosection image data

– Slices show ice and clutter around the brain

– Must remove this clutter before visualization• No real surface (cavities!)

– Auto-segmentation uses YIQ color model, filtering, region growing, dilation, segmentation

Remove exterior Detect cavities


Brain Visualization• Compressing data using wavelets

– Enables multi-resolution visualization with same data size

– Each step separates low and high frequency components

original (256 x 256) Step 1 Step 2


Brain Visualization• Visualizing the cleaned brain data set

– Cutting, segmenting, classifying, lighting, rendering, ...


Oceanography Visualization• Visualizing temperature

– Temperature mapped to color

– Isocontours + volume rendering


Oceanography Visualization• Visualizing velocity

– Vector fields atop volume rendering


Oceanography Visualization• Visualizing velocity

– Vector direction mapped to color

– Vector magnitude mapped to intensity


SDSC

The Scripps Research Institute

U. Texas, Austin

UCLA

SIO/MIT

U. California, Davis

Mississippi State U.

Scalable Visualization Toolkits for Bays to Brains

Documents

Scalable Visualization Toolkits for Bays to Brains