Upload
melchior-magda
View
31
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Roy van Pelt. Illustrative Volume Rendering on Consumer Graphics Hardware. Exam Committee: dr. A. Vilanova (BMT) dr. ir. H.M.M. van de Wetering (CSE) dr. ir. M. Chaudron (CSE). Project motivation. Improve comprehensibility Useful for context visualisations - PowerPoint PPT Presentation
Citation preview
Illustrative Volume Rendering
onConsumer Graphics
HardwareRoy van Pelt
Exam Committee:
dr. A. Vilanova (BMT)
dr. ir. H.M.M. van de Wetering (CSE)
dr. ir. M. Chaudron (CSE)
2/40
Project motivation
Improve comprehensibility Useful for context visualisations Similar to illustrations in physiology
books
3/40
Introduction: Illustrative Volume Rendering Volume Rendering
3D dataset, i.e.: CT / MRI
Images by kind courtesy of R. Brecheisen (2007) (MVR framework)
4/40
Introduction: Illustrative Volume Rendering Illustrative Volume Rendering
Depict dataset as a drawing/sketch
Stippling Hatching Contours
5/40
Introduction: Illustrative Volume Rendering Existing framework: VolumeFlies
Traditional illustration techniques Based on particle systems
6/40
Project objectives
Design and implement VolumeFlies, using consumer graphics hardware, such that real-time interaction with the data is possible.
Integrate VolumeFlies with a direct volume rendering approach.
7/40
Background Info: GPU pipeline
• Massively parallel– Adjust algorithms to use parallelism– Unified shader design
• Programmability– Stream programming model using shaders
RasterizationRasterizationGeometry Geometry
ProcessingProcessing
FragmentFragment
OperationsOperations
VertexVertex
ShaderShader
GeometryGeometry
ShaderShader
FragmentFragment
ShaderShader
8/40
Background Info: GPU pipeline
• Massively parallel– Adjust algorithms to use parallelism– Unified shader design
• Programmability– Stream programming model using shaders
RasterizationRasterizationGeometry Geometry
ProcessingProcessing
FragmentFragment
OperationsOperations
VertexVertex
ShaderShader
GeometryGeometry
ShaderShader
FragmentFragment
ShaderShader
9/40
Background Info: General GPU approach
RasterizationRasterizationGeometry Geometry
ProcessingProcessing
FragmentFragment
OperationsOperations
VertexVertex
ShaderShader
GeometryGeometry
ShaderShader
FragmentFragment
ShaderShader
Proxy geometry = “Array index”
Source buffer = “Input array” Destination buffer = “Output array”
Transform feedbackDiscard fragment shader
10/40
Background Info: Particle system Particle: an element described by
propertiesFor example particle position in the volume
p
Movie source: http://www.runevision.com
11/40
Framework modules
Initialiser1
2
3
4
Behaviour
Filter
Visualiser
Initialises particle set
Alters particle set
Removes unwanted particles
Illustratively depicts particles
12/40
Framework modules
Initialiser1
Initialize particles near the ISO-surface
Related to marching cubes
13/40
Framework modules
Initialiser1
VertexVertex
ShaderShader
GeometryGeometry
ShaderShader
FragmentFragment
ShaderShader
3D TextureVolume
Proxy geometry
Particle positions (x,y,z) = (r,g,b)
14/40
Framework modules
Initialiser1
2
3
4
Behaviour
Filter
Visualiser
15/40
Framework modules
2 Behaviour
0.2 0.4 0.6 0.8 1
20
40
60
ij i jr p p
( , , )i i ii
i i i
E E Ev
x y z
Redistribute particles evenly on the surface
Energy minimisation
ip1,
(| |)m
i ij ijj j i
E E r
jp
ijr| |ijriv
(| |)ij ijE r
| |ijr
“Robust particle systems for curvature dependent sampling of implicit surfaces” – Meyer et al. (SMI 2005)
Minimise the total energy within the particle system
1
m
i
i
E E
16/40
iv in
Framework modules
2 Behaviour
Redistribute particles evenly on the surface
Two step particle displacement
iso-surface
tangent plane
“Robust particle systems for curvature dependent sampling of implicit surfaces” – Meyer et al. (SMI 2005)
17/40
Framework modules
2 Behaviour
1. Sort particles by their bin numberOdd-even merge sort
2. Create a lookup table for the binsBinary search0 4 7 10
3. Redistribute particles by repulsionEnergy minimisation approach
X X X X X X X X X X X X
Y Y Y Y Y Y Y Y Y Y Y Y
Z Z Z Z Z Z Z Z Z Z Z Z
Bin 0 Bin 1 Bin 0 Bin 0 Bin 0 Bin 1 Bin 2 Bin 1 Bin 3 Bin 2 Bin 3 Bin 2
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
10
10
10
11
11
11
12
12
12
X X X X X X X X X X X X
Y Y Y Y Y Y Y Y Y Y Y Y
Z Z Z Z Z Z Z Z Z Z Z Z
Bin 0 Bin 0 Bin 0 Bin 0 Bin 1 Bin 1 Bin 1 Bin 2 Bin 2 Bin 2 Bin 3 Bin 3
1
1
1
3
3
3
4
4
4
5
5
5
2
2
2
6
6
6
8
8
8
7
7
7
10
10
10
12
12
12
9
9
9
11
11
11
Redistribution hard to port to GPU:
Addressing all neighbours is
computationally expensive
Solution:
Create a bin-structure to compute
energy minimisation locally.
18/40
Framework modules
2 Behaviour
VolumeFlies applied a fixed number of iterations
A new stop-criterion is introduced
Verify the change of the total energy
Apply reduction operation on the GPU
8E7E6E5E4E3E2E1E
'4E
'3E
'2E
'1E
'4E
'3E
'2E
'1E
''2E
''1E
'''1E Total system energy
+
+
+
1
m
i
i
E E
19/40
Framework modules
2 Behaviour
Odd-Even Merge Sort
Binary Search
Energy Minimization
Verify stop-criterion
GPU approach more elaborate.
Still much faster than CPU approach.
Particle system is generic.
20/40
Framework modules
Initializer12
3
4
Behaviour
Filter
Visualizer
21/40
Framework modules
3 Filter
a) Create the ISO surface
Cone splatting:
Cones directed to the viewing plane
Cones are scaled to prevent edge overlap
22/40
Framework modules
3 Filter
b) Hidden surface removal
Off-screen buffer:
Splat rgb = Particle xyz
Visible when particle position occurs as a colour
23/40
Framework modules
Initialiser1
2
3
4
Behaviour
Filter
Visualiser
24/40
Density-based stippling
4 Visualiser
Vary particle density to change tone
Initial density resembles darkest tone
Per particle random threshold
Based on basic diffuse lighting
Executed by vertex shader
User can control contrast
25/40
Scale-based stippling
4 Visualiser
Vary point size to change tone
Larger points create darker areas
Scale point representation
Based on basic diffuse lighting
Executed by vertex shader
User can control contrast
26/40
Direction-based hatching
4 Visualiser
Hatch tracing in a single direction
Single and cross hatching
Single hatches
Cross hatches
27/40
Direction-based hatching
4 Visualiser
iso-surface
Hatch tracing in a single direction
Segments projected in tangent plane
28/40
Direction-based hatching
4 Visualiser
VertexVertex
ShaderShader
GeometryGeometry
ShaderShader
FragmentFragment
ShaderShader
Particle positions
Generate hatches in fixed direction
3D Texture: Volume
Proxy geometry
Hatch segments
1
VertexVertex
ShaderShader
GeometryGeometry
ShaderShader
FragmentFragment
ShaderShader
Visualise generated hatchesProxy geometry
Hatch segments
2
29/40
Direction-based hatching
4 Visualiser
Again apply basic diffuse lighting
Two-level threshold
2nd level
1st level
30/40
Curvature-based hatching
4 Visualiser
Emphasize curvature with hatches
Fast curvature calculation approach
2. Optimize for GPU by using interpolation capabilities
Differential structure in volume required
“Fast Third-Order Texture Filtering” – Sigg, Hadwiger (GPU Gems 2)
1. Filtering approach for derivative reconstruction
3. Compute principal curvature
Re-implemented in high-level shading language
Based on general GPU-approach
31/40
Curvature-based hatching
4 Visualiser
Emphasize curvature with hatches
Defining the principal curvature
Image by Eric Gaba
Variation of the normal vector when moving a small distance
iso-surface
tangent plane
in
Maximum and minimum change describe the principal
curvature
1ik
2ik
32/40
Curvature-based hatching
4 Visualiser
Emphasize curvature with curvature
Calculate principal curvature
Measure first partial derivates: gradient
/ | |n g g<<<<<<<<<<<<<<<<<<<<<<<<<< <<
“Curvature-Based Transfer Functions for Direct Volume Rendering Methods and Applications” – Kindlmann et al. (IEEE Visualization 2003)
g<<<<<<<<<<<<<<
g<<<<<<<<<<<<<<
n
iso-surface
tangent plane
Measure second partial derivates: Hessian H Extract curvatures by eigen analysis on the shape operatorS
/ | |
TP I nn
S PHP g
<<<<<<<<<<<<<<
1k
33/40
Curvature-based hatching
4 Visualiser
1D Transfer function
-1 0 1
1 2
1k
2k
34/40
Curvature-based hatching
4 Visualiser
Messy hatch results using curvature directions
Smooth the field directions and weight the trace
Fixed direction when
surface is unreliable
35/40
Curvature-based hatching
4 Visualiser
Principal curvature can be calculated in real-time
Demonstrated by real-time curvature colour mapping
1
2
36/40
Contours
4 Visualiser
Draw ‘hatches’ for particles near the contour
Segments trace the direction of the contour
Contour: locations where normal
is perpendicular to the view
37/40
Results: Performance
0 2 4 6 8 10 12 14 16 18 20
Profiling time (seconds)
VolFliesGPU VolumeFlies
253.34
52.65
53.05
Load Volume
Brute-force initialiser
Redistribution
Smooth Field
Hatch generation (Direction)
Hatch generation (Scale)
Hatch visualisation
Contours
38/40
Results: Demonstration movie
39/40
Conclusions & Future work Completely GPU-based General GPU approach Two generic components:
Particle system Real-time curvature estimator
Integration with direct volume rendering More elaborate memory management Improvement of modules / Zooming New applications and styles: DTI, animation, ghosting,
exploded view…
40/40
Thank you for your attention!
Questions
?
41/40
Particle repulsion
Energy minimisation
Two step particle displacement
ij i jr p p
( , , )i i ii
i i i
E E Ev
x y z
1,
(| |)m
i ij ijj j i
E E r
1
m
i
i
E E
2
( )
( )
( )| |
Ti i i i i
i i
ii i i
i
p p I n n v
g f p
gp p f p
g
<<<<<<<<<<<<<<
<<<<<<<<<<<<<<
<<<<<<<<<<<<<<
42/40
Stippling
Density-based
Scale-based
( ) max( ,0)d i iL i n e
( ( ))ci dv L i
( ) max( ,0)d i iL i n e
3( (1 ( )))2
ci dS L i
43/40
Hatching Smooth field
2 1
2 1
2arctans
1 2
1 2
0, | | | |( , ) 1
1 | 2(| | ) |,2
if and
s otherwise
1( ) (1 )
( )
jji TT T
j
jT
w ks w w s
w
wb
44/40
Hatching Tracing
0
1 1 21
2 1 2
(1 ) , | | | |
(1 ) , | | | |
j j j
j j j
i
j j jj j jj
j j jj j j
j
h s
w h w k k h k hh
w h w k k h k h
jw
s
45/40
Contours
2 2 1 1 1 2( ) ( )d k e k k e k <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Image by D. DeCarlo
46/40
Curvature estimation 1/3
Measure partial derivatives by convolution filtering with cubic B-spline
47/40
Curvature estimation 2/3
Optimise for the GPU by using interpolation
xi i+1i-1 i+2
Weights lookup texture
Colour texture
48/40
Curvature estimation 3/3
Compute principal curvature
Eigen analysis on S
/ | |
/ | |
T
n g g
P I nn
S PHP g
<<<<<<<<<<<<<<<<<<<<<<<<<< <<
<<<<<<<<<<<<<<
1 11
2 22
Sk k
Sk k