52
image data processing image filtering and segmentation in 2D and 3D

image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

image data processing

image filtering and segmentation in 2D and 3D

Page 2: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

aims and objectives

•  visualization •  data optimization •  data segmentation •  feature recognition •  measuring •  object reconstruction •  ...

Page 3: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

filte

ring

image data processing and visualization: workflow

patient/object data

image data convolution

feature extraction

data set registration

2D (image) visualization

3D (volume) visualization

measuring

fused imaging

2.5D (surface) visualization

Page 4: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

what is an image?

-  a distribution of a variable I in space (x,y) or (x,y,z) -  a scalar field (= an array/matrix/volume of numbers)

image image “landscape” = imagescape

Page 5: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

what is an image filter?

input image I output image O

a mathematical function that transforms an imput image into an output image: I(x,y,z) à O(x,y,z)

Page 6: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

extract/enhance “relevant” information remove/reduce“irrelevant” information

why image filters?

input image I output image O

Page 7: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

1. Visualizing 2D biomedical image data

Page 8: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

windowing function di

spla

y un

its

(8 b

it/pi

xel)

input

output 255

0

à see neural response function

example: visualizing a CT cross section

-1000 (air)

3095 (maximum)

-0 (water)

window width

window center

Hounsfield density units (12bit/pixel)

Page 9: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

windowing: CT data example

low contrast window

soft tissue window “inverted” window

bone threshold

Page 10: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example of a false-color scale

R

G

C M

B

Y

path through color space

Page 11: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example of a false color scale

input

output

255

1791

Page 12: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

false color application in 3D

bone thickness map"

Page 13: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

discrete false-color scale

→ see LUT, multiple thresholding, data segmentation, enhanced reality

Page 14: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

2. Image processing with 2D and 3D filters

using data from many input elements (pixels/voxels) to calculate

the value of one output elements (pixel/voxel)

Page 15: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

what is an image filter?

input image I output image O

filter function = kernel filtering = convolution ⊗

mathematical function with a spatial structure (“kernel”) that transforms an input image I into an output image O

Page 16: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

principle of filtering

input output

input image output image convolution: apply a function (an operator)

to all image elements (pixels or voxels)

kernel function (size n x n)

Page 17: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

principle of filtering

convolution

=

Page 18: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

some examples

•  the maths of simple filters •  mean and Laplace filter •  statistical filters: Gauss, median, histogram •  edge detection

Page 19: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

the maths of simple filters

Ix-1,y+1 Ix,y+1 Ix+1,y+1

Ix-1,y Ix,y Ix+1,y

Ix-1,y-1 Ix,y Ix+1,y-1

input image elements: 3x3 region around pixel (x,y)

3x3 filter kernel (weight matrix)

Ox,y = Ix+i,y+ j !wi, j( )j="1

1

#i="1

1

#

Ox,y

output image element: new value at pixel (x,y)

⊗ = w-1,1 w0,1 w1,1

w-1,0 w0,0 w1,0

w-1,-1 w0,-1 w1,-1

… … …

… … …

Page 20: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

mean filter („low-pass“)

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

3x3 kernel

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

1/25 1/25 1/25 1/25 1/25

output images

input image

5x5 kernel

Page 21: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

Laplace filter („high-pass“)

0 -1/4 0

-1/4 1 -1/4

0 -1/4 0

see: http://www.wiley.com/go/virtualreconstruction

Page 22: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

statistical filter 1: noise reduction with Gauss filter

random noise from image data acquisition device

Gauss filter

Page 23: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

statistical filter 2: impulse noise reduction with median filter

median filter (a statistical filter): evaluate median value in nxn neighborhood of each pixel

Page 24: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

statistical filter 3: histogram filter

output intensity is a function of the frequency of input intenstiy

Page 25: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

edge detection

•  identify object boundaries

Page 26: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

edge detection in the retina

principle of edge detection: •  center: positive weight •  periphery: negative weight

+- - - -

-

-

+-

Page 27: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

edge detection: computer

Laplace filter kernel: 2nd spatial derivative of the image

Page 28: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

spatial derivatives

I=f(x) Ii-1 Ii Ii+1

xi xi+1 xi-1

(1⋅Ii-1-2⋅Ii+1⋅Ii+1)

Ii- Ii-1 Ii+1-Ii ΔI/Δx

Δ2I/Δx2

1 -2 1

function

1st deriv.

2nd deriv.

Laplace filter

position x

Page 29: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example image: I = f(x,y)

x

y

Page 30: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example image: I = f(x,y)

x

y

I

Page 31: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example image: first spatial derivatives

Ix’ =dI/dx

Iy’=dI/dy

Page 32: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example image: gradient [Ix’ Iy’]

vector components are first derivatives along x and y

Page 33: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example image: Laplacian Ix’’ + Iy’’

sum of second derivatives along x and y

Page 34: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example image: Laplacian Ix’’ + Iy’’

sum of second derivatives along x and y

edges

Page 35: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example image: I = f(x,y)

x

y

I edges

Page 36: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

edge detection with Laplace filter

input output

Page 37: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

concatenating filters

Gauss: noise reduction

Laplace: edge detection

input 1 output 1 = input 2 output 2

Page 38: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

3. image data segmentation

filters for data classification

transformation of bitmap graphics into vector graphics

Page 39: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

data segmentation

principal aim: classification

•  attribute specific properties (labels) to each voxel of the data set

•  create explicit object representations •  methods:

–  automated: thresholding –  interactive: feature extraction with gradient-based

and morphological filters

Page 40: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example 1: thresholding attribute one output value to a range of input values

thresholding multiple thresholding (à lookup tables)

Page 41: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example 2: morphological operators (filters)

•  extract pre-defined features from 2D/3D images

•  reconstruct explicit contours, surfaces, volumes in the form of geometric descriptions

à transform bitmap graphics into vector

graphics

Page 42: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

geometric „primitives“

Page 43: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

contour reconstruction in 2D with line primitives

marching squares: morphological 2D filter kernel

Page 44: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

contour data

Page 45: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

surface reconstruction in 3D

marching cubes: morphological 3D filter kernel

etc... 28=256 variants)

Page 46: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

surface data

Page 47: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

free-form surface representation

1 cm

Page 48: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

example 3: deformable templates

•  define a-priori forms (templates) based on anatomical knowledge

•  adapt (deform) templates to 2D/3D image data

•  extract 2D/3D features •  measure difference between actual feature

and template

Page 49: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

freeform 2D-deformable templates

•  use an a-priori model as a template to classify image data

•  deform the template until it optimally fits the image data

•  label the image data accordingly

Page 50: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

freeform 3D-deformable templates: brain extraction tools Ventrice (PhD thesis, 2011)

Page 51: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

cylindrical template (cylindrical surface parameterization)

D E

l a m p l

0⁰ 90⁰ 180⁰ 270⁰ 360⁰

a l

m

0

1

c = 1

ma

z

F

0⁰ 90⁰

180⁰ 270⁰

Morimoto et al. (2011)

Page 52: image data processing - UZHimage data processing and visualization: workflow patient/object data image data convolution feature extraction data set registration 2D (image) visualizationwindowing

spherical template (spherical surface parameterization)

chimpanzee human

Specht et al. (2007)