Medical Image Visualization in 3D

8/6/2019 Medical Image Visualization in 3D

1/56

Medical Image Visualization in 3D

Abhishek Mitra

Ph.D in Medical Imaging


2/56

3D Imaging Technique

Scan & Step

scanning

Helical Scanning

Reconstruction


3/56

Data preparation

1.5

0.5

2.8

0.3

10.7

2

3D volume data are represented by a finite number of cross

sectional slices (a stack of images)

On each volume element (voxel), stores a data value

For simplicity sake, we assume that the dataset is a regular 3D grid,

and each grid point has a data value.


4/56

Surface

Rendering

Iso value = 80 Iso value = 200


5/56

1.5

0.5

2.8

0.3

10.7

2

Surface Rendering:

Cont.


6/56

Surface Rendering:

Cont.


7/56

1.5

0.5

2.8

0.3

10.7

2

Surface Rendering:

Cont.

Iso value = 80

Iso value = 200


8/56

Volume

Rendering


9/56

Volume Rendering


10/56


11/56


12/56

Trace rays from eye instead

Start from the image and follow the ray

until ray finds (or fails to find) a light source

This is what most people mean by ray casting.


13/56


14/56


15/56


16/56


17/56


18/56


19/56


20/56


21/56


22/56


23/56


24/56


25/56


26/56

Physical World : Photons shoot out from light sources, reflect off surfaces and into the eye.

Problem : Only small fraction reaches the eye (or image plane). Difficult

to simulate.

Alternative way to simulate :

Reverse the process! Trace the path backwards i.e from the eye (or pixels) back to the light

sources.

Ray Tracing & Ray Casting


27/56

Shadow Test

Check against other objects to see if point is shadowed

Cast shadow ray from surface point to light

If shadow ray hits opaque object, no contribution

Eye

Shadowing

Object


28/56


29/56


30/56

ContributionFrom Primary Ray

(0.5,1.0,

0.5)

Recursive ray tracing

Eye


31/56

(0.5,1.0,

0.5)

V2

V1

L (0.8,0.8,0.0)


ContributionFrom Secondary Ray

Eye


32/56

Final Pixel Color

(0.5,1.0,0.5)

V2

V1

L (0.8,0.8,0.0)+ =


Eye


33/56


34/56

Eye

Shadowing

Object


35/56

Basic Illumination / Shading Technique


36/56

An illumination modelin computer graphics

also called a lighting modelor a shading model used to calculate the color of an illuminated position

on the surface of an object

An illumination modelcomputes the lighting effects for

a surface using the various optical properties-Degree of transparency, color reflectance, surface texture

The reflection (phong illumination) modeldescribes

the way incident light reflects from an opaque surface-Diffuse, ambient, specular reflections

-Simple approximation of actual physical models


37/56

Shading

- Use a phong Illumination/Shadding model

- Itsimulates rough and shinysurfaces

For each Sample evaluate color :

I = Iambdiff+ Idiffused+ Ispecular


38/56

Ambient Light

I = Iambdiff+ Idiffused+ Ispecular It simulates the indirect lighting in a scene,

Multiple reflection of nearby (light-reflecting) objectsyields a uniform illumination, which is constant for an object

Eye

Light


39/56

Diffuse Light

It simulates direct lighting on a rough surface

Incident light is scattered with equal intensity in all directions

Viewer independent, depend on light vector and surface normal

Paper, rough wood, brick, etc...

I = Iambdiff

+ Idiffused

+ Ispecular

N

L

V


40/56

Specular Light

It simulates direct lighting on a smooth surface

Perfect reflector (mirror) reflects all lights to the direction

where angle of reflection is identical to the angle of incidence

Depend on both light vector and view direction

Plastic, metal, polished wood, etc...

I = Iambdiff

+ Idiffused

+ Ispecular

N

L

VR


41/56

I = Iambdiff+ Idiffused+ Ispecular

Shading: Phong Illumination Model


42/56


43/56


44/56


45/56

Volume Rendering Operation

Classification

Gradient Computation

Image Compositing

Shading

Preparing

Direct Volume Rendering Technique


46/56

Data preparation

Classification


Image Compositing

Shading

Preparing

1.5

0.5

2.8

0.3

10.7

2

3D volume data are represented by a finite number of cross

sectional slices (a stack of images)

On each volume element (voxel), stores a data value

For simplicity sake, we assume that the dataset is a regular 3D grid,and each grid point has a data value.


47/56

Classification


Image Compositing

Shading

Preparing

L

R

EyeN

Why compute the gradient ?

Gradient of densities provides information on surface:

- Density changes at surface boundaries:

change also appears in their gradient- Presence of surface is qualified by magnitude of

gradient

- Gradient vector will be used as surface normal in the

shading process

- Gradient vectors norm is used in classification


48/56

To f

ind the normal at po

int P:

Consider the six adjacent data points.

Let the value at the next point along the x axis be

denoted as fx+1.

Similar notation for the other five points.

Then, the normal at point P is N =

(fx+1-fx-1,fy+1-fy-1,fz+1-fz-1).

1.5

0.5

2.8

0.3

10.7

2

Gradients computation: Central Difference

Classification


Image Compositing

Shading

Preparing

Cl ifi ti l


49/56

Classification


Image Compositing

Shading

Preparing

Classification goal- Make sense of a given data set

- Group voxels of same substance

- Assign color/opacity value that guides visualization

- ColorTransfer Function-> Maps data to a given color

- Opacity Trans. Fn.-> Maps data to a given opacity

Translucent surfaceTranslucent surface

f(xi) C(xi), a(xi)

C l T f F ti


50/56

Color Transfer FunctionThe color transfer function allows to make a

simple classification or segmentation

Classification


Image Compositing

Shading

Preparing

21.05 27.05

24.03 20.05


51/56

Classification


Image Compositing

Shading

Preparing

Classificationassignsto each voxelanopacityav

Opacity describeshowtransparenta voxelsis:- av=1: completely opaque, cant see through- av=0; transparent, no impact on rendered picture

Bycontrollingtheopacity,wecan:

- Show surfaces by setting opacity to 0 or 1

- Show both exterior and interior regions by grading

opacity from 0 to 1

Regionofinterest: highopaicity (moreopaque)Rest: translucentortransparent

For example,ifthe value for boneisaround 0.4,and thevalue forskinisaround 0.1,the usermay usethe following

opacitymaptoseethe bonethroughtranslucentskin

Opacity Transfer Function

f(xi) C(xi),a(xi)

O i f i


52/56

Opacity

Opacity Transfer FunctionOpacity

0.1 0.4Volume

Data

1.0

Tissue:

Opaque

Volum

Data0.1

Tissue:

Semi transparent

0.4

1.0

Bone:Opaque

Opacity

Volume

Data1.00.1 0.4

Bone:

Opaque

1.0

Skull + Skin

Sh di


53/56

Apply light to volume

Phong lighting model will give the light reflected to the viewer at any

point in volume if we know the local normal

Imagine an isosurface shell through each data point surface normal is

provided by gradient vector

Thus we can get color reflected at each data point

Shading

Classification


Image Compositing

Shading

Preparing

L

R

Eye

Light not attenuated in

passage to voxel

Light attenuated in

passage from voxel

N

Compositing


54/56

Compositing

Classification


Image Compositing

Shading

Preparing

For every pixel in output image

Treat volume as a translucent object shoot ray into volume at least one per pixel at evenly spaced ray locations, obtain color and opacity by

interpolation

Accumulate color and opacities of voxels from back to front

Stop when ray exits grid or reaches full opacity


55/56

EEEE

EE

!

!

)1(

)1(

inout

inout CCC

Compositing- Illustration

color

Eye

Vol me Rendering: S


56/56

Volume Rendering:Summary

Classification


Compositing

Shading

Preparing

1.0

Volume

Data0.1

Tissue:

Semi transparent

0.4

Bone:

Opaque

Opacity

L

R

EyeN

Documents

Medical Image Visualization in 3D