NPACI: National Partnership for Advanced Computational Infrastructure
SDSC
The Scripps Research Institute
U. Texas, Austin
U.C.L.A.
SIO/MIT
U. California, Davis
Mississippi State U.
Scalable Visualization Toolkits for Bays to Brains
NPACI: National Partnership for Advanced Computational Infrastructure
Motivation• Support analysis, filtering, and compositing
– Larger-than-core (and swap) volumetric data sets
– Multi-modal and time-varying data
– Multiple data sets simultaneously
• And...– Do efficient data movement
– Execute well on parallel architectures
– Integrate easily w/existing applications & toolkits
– Provide interactive user interfaces for volume exploration
CT
Cryosection
Classification
NPACI: National Partnership for Advanced Computational Infrastructure
Motivating Applications• Develop state-of-the-art visualization of:
– Brain data• 1789 TIF color images
• 1472 x 1152 x 1789 x 3 (bytes) = 8.5 GB
– Ocean data• 2160 x 960 x 30 x 4 (bytes) = 0.2 GB
• 237.3 (MB) x 128 (time-steps) = 29.7 GB
• Surface height, salinity, temperature, velocity
• Demonstrate use beyond these fields
NPACI: National Partnership for Advanced Computational Infrastructure
Toolkits
Function libraries providing scalable visualization features for new and existing volumetric applications
NPACI: National Partnership for Advanced Computational Infrastructure
Layered Toolkit Architecture• Lowest layers provide access
to larger-than-core data sets– Automatic paging of
N-dimensional mesh data
– Local and remote (SRB) files
• Higher layers providelarge-data visualization– Optimized for best data movement
– Multi-data set scenes
– On-the-fly isocontouring during rendering
Volume Scene Toolkit
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
NPACI: National Partnership for Advanced Computational Infrastructure
Mesh Toolkit• Manage a paged mesh (array-like)
– An N-dimensional grid of values
– Spatial data & time-series
– Arbitrary data content
– Multiple file formats
– Volume metadata
– Operations like filtering, sub-sampling, ...
• Layered atop local I/O and SRB
Volume Scene Toolkit
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
NPACI: National Partnership for Advanced Computational Infrastructure
Mesh Toolkit• Intelligent pre-fetch by encapsulating common
access patterns– Random access (slow)
– Structured access (faster)
– Storage-order access (fastest)
• Neighborhood-aware paging– Page-in nearby data in N-dim.
– Convolution filtering, ...
Volume Scene Toolkit
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
5 13 24679 8
51 32 4 6 7 8 9
51 32 4 6 7 8 9
0 1 2 3 4
5 6 7 8 9
10 11 12 13 14
15 16 17 18 19
20 21 22 23 24
Current center Current center elementelement
Keep neighboring Keep neighboring elements paged-in as wellelements paged-in as well
Filter Filter windowwindow
NPACI: National Partnership for Advanced Computational Infrastructure
Mesh Toolkit• Status
– C++ and Java implementations
– Fully documented
– SRB integration in progress
– In active use for large data manipulation
– Software release at 2001 All-Hands (with SRB)
Volume Scene Toolkit
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
NPACI: National Partnership for Advanced Computational Infrastructure
Volume Scene Toolkit• Define a scene containing multiple data sets
– Leaf nodes are data sets, functions, ...
– Interior nodes composite, filter, ...
– Transforms align overlapping data sets
• Evaluate it to generate results– Client defines the scene
– Server on big iron evaluates it
– Layered atop Mesh Toolkit
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
ClientClient
ServerServer
NPACI: National Partnership for Advanced Computational Infrastructure
Volume Scene Toolkit• Cut, Segment, Filter, and Combine data
– Manage multiple data sets of arbitrary data content
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
Volume Scene Toolkit• Status
– C++ prototype implementation
– In active use for multiple data sets• Head and brain data
• Tissue and cell data (Science cover)
• Other (Orion, ...)
– Software release in 2001
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
VolRen Toolkit• Data-efficient isocontouring
– Isocontours produce surfaces at chosen data values• Large data produces very large isocontour polygon sets
– Toolkit uses a fast seed-based approach• Small amount of pre-computation speeds up
isocontouring on later user requests
– Works from compressed data• Parallel visualization of compressed streams
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
VolRen Toolkit• Merged isocontouring and rendering
– Isocontour extraction during volume rendering• Avoids creation & storage of large polygon sets
– Raycasting rays intersect volume &isosurfaces, properly mixing colors and shading
• Parallel out-of-core algorithm– Image-space partitioning
– Master-slave structure
– Low data communication cost due todata pre-processing and reorganization
NPACI: National Partnership for Advanced Computational Infrastructure
VolRen Toolkit• Available as a toolkit and a stand-alone system
– Can integrate into applications, or
– Client-server system for rendering of high-resolution videos
– Client-server system for interactive exploration of time-dependent data
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
ClientClient
ServerServer
Param
NPACI: National Partnership for Advanced Computational Infrastructure
VolRen Toolkit• Status
– C++ implementation
– Additional features in development (incorporation of streaming images and geometry- progressive)
– In active use for multiple data sets (structured and unstructured data)
– will layer atop mesh-toolkit
– Software release 1st quarter 2001 (toolkit)
– remote access (client-server) release 2001
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
Rendering Toolkit• High-performance perspective volume renderer
– Optimized for parallel architectures
– Efficient data movement for larger-than-core rendering
• Available as a toolkit and a stand-alone renderer– Can integrate into applications, or
– Client sends parameters
– Server on big iron renders usingserver-side high-resolution data
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
ClientClient
ServerServer
VISTA
NPACI: National Partnership for Advanced Computational Infrastructure
Rendering Toolkit• Render volume meshes
– Perspective ray-casting of individual volume data sets
– Scenes voxelized first at arbitrary resolution
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit VISTA
NPACI: National Partnership for Advanced Computational Infrastructure
Rendering Toolkit• Render volume scenes
– Perspective ray-casting of volume scenes
– Only evaluates scene where needed
– Particularly good for time-varying scenes
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit VISTA
NPACI: National Partnership for Advanced Computational Infrastructure
Rendering Toolkit• Status
– C++ prototype implementation (based on Orion renderer)
– In active use for multiple data sets
– Software release mid-2001
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit VISTA
NPACI: National Partnership for Advanced Computational Infrastructure
3D GUI Toolkit• Common GUI components for 3D
– Generic parts for quick GUI development
– 3D drawing windows
– 3D manipulators (shape handles)
– 3D input device handlers
– Point cloud & textured volume shapes
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
3D GUI Toolkit• Status
– Java partial implementation
– Additional features in development
– In active use in multiple 3D viewers
– Software release in 2001
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
Tools
Stand-alone interactive applications and serversbuilt using the scalable visualization toolkits
NPACI: National Partnership for Advanced Computational Infrastructure
Multi-Mesh Viewer• Interactive multi-mesh viewer
– Built atop Mesh & 3D GUI Toolkits
– Client-server access to Rendering Toolkit
– Uses standard 3D graphics hardware
– Single and multiple volume viewing
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
Interactive
Rendered
NPACI: National Partnership for Advanced Computational Infrastructure
Multi-Mesh Viewer• Data-efficient visualization
– Adjustable viewing resolution
– Movable region-of-interest
– Client-side proxy volumes forhigh resolution data
– Client-side interaction withserver-side rendering
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
Interactive Rendered
NPACI: National Partnership for Advanced Computational Infrastructure
Multi-Mesh Viewer• Status
– Java implementation
– Portable: PC, SGI, …
– SRB integration in progress
– In active use to interactively view meshes
– Software release at 2001 All-Hands (with SRB)
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
Python Volume Viewer• Hardware accelerated visualization
– Built atop Mesh Toolkit +DejaVu, VLI, & Python language
– Uses VolumePro rendering hardware• Renders 2563 volume in real time
– Interactively adjust color, lighting, ...
PVV
NPACI: National Partnership for Advanced Computational Infrastructure
Python Volume ViewerSRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
• Data-efficient multi-resolution visualization– Interactive zoom between low & high resolution data
PVV
NPACI: National Partnership for Advanced Computational Infrastructure
Python Volume Viewer• Status
– Python using C++ toolkit implementation
– Portable: PC, Sun, SGI
– In active use to interactively visualize meshes and explore lighting, color-table, opacity and viewing parameters
– Software release in 2nd quarter 2001
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit PVV
NPACI: National Partnership for Advanced Computational Infrastructure
Java Mesh Visualizer• Integrated visualization tool
– Built atop Mesh & 3D GUI toolkits +Shastra collaboration suite
– Client-server access to data servers,renderers, and isocontouring
– Web-based applet orstand-alone application
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit JVisTool
NPACI: National Partnership for Advanced Computational Infrastructure
Java Mesh Visualizer• Flexible visualization
– Adjustable cutting planes
– Isocontouring control
– Integrated collaboration features
– Client-side proxy volumes forhigh resolution data
– Client-side interaction withserver-side rendering & isocontouring
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit JVisTool
NPACI: National Partnership for Advanced Computational Infrastructure
Java Mesh Visualizer• Status
– Java implementation
– Portable: PC, SGI, …
– In active use to visualize meshes and isocontours
– Software release in 2001
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit JVisTool
NPACI: National Partnership for Advanced Computational Infrastructure
Volume Scene Editor• Interactive volume scene editor
– Built atop Mesh, Scene, & 3D GUI Toolkits
– Client-server access to Rendering Toolkit
– Diagramatic scene structure editing
– Client-side interaction and editing withlow-resolution proxy volumes
– Server-side rendering withhigh-resolution volumes
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
Volume Scene Editor• Data-efficient multi-mesh visualization
– Efficient compositing of multiple large meshes
– Multi-modal & time-varying visualization
– Interactive volume scene layout
– Interactive control of filters, compositing, lighting, …
– Scene and viewpoint animation control
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
Volume Scene Editor• Status
– C++ early prototype complete
– Prototype in active use
– Java re-design in progress
– Software release in 2001
SRB & Local I/O
Isocontouring Toolkit
Rendering Toolkit
3D GUI Toolkit
Mesh Toolkit
Applications
Volume Scene Toolkit
NPACI: National Partnership for Advanced Computational Infrastructure
Visualization Results
New techniques to better visualize large, complex data sets - from Bays to Brains and beyond...
NPACI: National Partnership for Advanced Computational Infrastructure
Brain Visualization• Cleaning cryosection image data
– Slices show ice and clutter around the brain
– Must remove this clutter before visualization• No real surface (cavities!)
– Auto-segmentation uses YIQ color model, filtering, region growing, dilation, segmentation
Remove exterior Detect cavities
NPACI: National Partnership for Advanced Computational Infrastructure
Brain Visualization• Compressing data using wavelets
– Enables multi-resolution visualization with same data size
– Each step separates low and high frequency components
original (256 x 256) Step 1 Step 2
NPACI: National Partnership for Advanced Computational Infrastructure
Brain Visualization• Visualizing the cleaned brain data set
– Cutting, segmenting, classifying, lighting, rendering, ...
NPACI: National Partnership for Advanced Computational Infrastructure
Oceanography Visualization• Visualizing temperature
– Temperature mapped to color
– Isocontours + volume rendering
NPACI: National Partnership for Advanced Computational Infrastructure
Oceanography Visualization• Visualizing velocity
– Vector fields atop volume rendering
NPACI: National Partnership for Advanced Computational Infrastructure
Oceanography Visualization• Visualizing velocity
– Vector direction mapped to color
– Vector magnitude mapped to intensity
NPACI: National Partnership for Advanced Computational Infrastructure
SDSC
The Scripps Research Institute
U. Texas, Austin
UCLA
SIO/MIT
U. California, Davis
Mississippi State U.
Scalable Visualization Toolkits for Bays to Brains