Deformable Contours

1

Dr. E. Ribeiro

Classical Methods

2

Thresholding Edge detectionAn image of blood vessel

Slide by: Chunming Li, Vanderbilt University

An Advanced Method: Active Contour Model

3Slide by: Chunming Li, Vanderbilt University

Edges vs. boundaries

Edges useful signal to indicate occluding boundaries, shape.

Here the raw edge output is not so bad…

…but quite often boundaries of interest are fragmented, and we have extra “clutter” edge points.

Images from D. Jacobs

Deformable contours

Given: initial contour (model) near desired object

a.k.a. active contours, snakes

(Single frame)

Fig: Y. Boykov[Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987]

Deformable contours

Given: initial contour (model) near desired object

a.k.a. active contours, snakes

(Single frame)

Fig: Y. Boykov

Goal: evolve the contour to fit exact object boundary

[Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987]

Why do we want to fit deformable shapes?

• Non-rigid, deformable objects can change their shape over time, e.g. lips, hands.

Figure from Kass et al. 1987

Why do we want to fit deformable shapes?

• Some objects have similar basic form but some variety in the contour shape.

Deformable contours: intuition

Image from http://www.healthline.com/blogs/exercise_fitness/uploaded_images/HandBand2-795868.JPG

Figure from Shapiro & Stockman

Deformable contours

initial intermediate final

a.k.a. active contours, snakesHow is the current contour adjusted to find the new contour at each iteration?

• Define a cost function (“energy” function) that says how good a possible configuration is.

• Seek next configuration that minimizes that cost function.

Snakes energy functionThe total energy (cost) of the current snake is defined as:

externalinternaltotal EEE +=

A good fit between the current deformable contour and the target shape in the image will yield a low value for this cost function.

Internal energy: encourage prior shape preferences: e.g., smoothness, elasticity, particular known shape.

External energy (“image” energy): encourage contour to fit on places where image structures exist, e.g., edges.

10))(),(()( ≤≤= ssysxsν

Parametric curve representation(continuous case)

Fig from Y. Boykov

External energy: intuition• Measure how well the curve matches the image data• “Attract” the curve toward different image features– Edges, lines, etc.

How do edges affect “snap” of rubber band?

Think of external energy from image as gravitational pull towards areas of high contrast

External image energy

Magnitude of gradient- (Magnitude of gradient)

( )22 )()( IGIG yx +−22 )()( IGIG yx +

External image energy

• Image I(x,y)• Gradient images and

• External energy at a point v(s) on the curve is

• External energy for the whole curve:

),( yxGx ),( yxGy

)|))((||))((|())(( 22 sGsGsE yxexternal ννν +−=

∫=1

0

))(( dssEE externalexternal ν

10))(),(()( ≤≤= ssysxsν

Internal energy: intuition

http://www3.imperial.ac.uk/pls/portallive/docs/1/52679.JPG

A priori, we want to favor smooth shapes, contours with low curvature, contours similar to a known shape, etc. to balance what is actually observed (i.e., in the gradient image).

Internal energyFor a continuous curve, a common internal energy term is the “bending energy”. At some point v(s) on the curve, this is:

The more the curve bends the larger this energy value is.

The weights α and β dictate how much influence each component has.

Elasticity,

Tension

Stiffness,

Curvature

sdd

dsd

sEinternal 2

2))((

22

ννβαν +=

∫=1

0

))(( dssEE internalinternal νInternal energy for whole curve:

Dealing with missing data

• The smoothness constraint can deal with missing data:

[Figure from Kass et al. 1987]

Total energy(continuous form)

// bending energy

// total edge strength

under curve

externalinternaltotal EEE γ+=

∫=1

0internal ))(( dssEEinternal ν

∫=1

0

))(( dssEE externalexternal ν

Parametric curve representation(discrete form)

• Represent the curve with a set of n points

10),( −== νiyx iii Kν …

Discrete energy function:external term

21

0

2 |),(||),(| iiy

n

iiixexternal yxGyxGE ∑

−

=

+−=

Discrete image gradients

),( yxGx ),( yxGy

• If the curve is represented by n points

Summary: elastic snake• A simple elastic snake is defined by– A set of n points,– An internal elastic energy term– An external edge based energy term

• To use this to locate the outline of an object– Initialize in the vicinity of the object– Modify the points to minimize the total

energy

Level Set Representation of Curves

24

zero level

zero level

Slide by: Chunming Li, Vanderbilt University

Level Set Method (Osher and Sethian, 1988)

25

• Curve evolution:

where F is the speed function, N is normal vector to the curve C

• Level set formulation:

N

Slide by: Chunming Li, Vanderbilt University

26

Geodesic active contour

27

Geodesic active contour

Geodesic active contour

28

29

Segmentation using statistical models (Rousson and Deriche, 2002)

Energy functional

Probability describing the pixel values inside region i

30

Segmentation using statistical models (Rousson and Deriche, 2002)

Energy functional

Probability describing the pixel values inside region i

The energy functional can be rewritten as:

Two-phase case

31

Two-phase case

32

Energy functional

Level set function

Two-phase case

33

Energy functional

Heaviside step function

Smooth approximation

Length term

Indicator functions (partitions)

34

Indicator functions (partitions)

35

Distance function (level set function)

36

Two-phase case

37

Estimating the Parameters of the Gaussian densities

Two-phase case

Experiments

39

Gray level

40

Color and Texture

41

Results

42Slide by: Chunming Li, Vanderbilt University

3D Segmentation of Corpus Callosum

43Slide by: Chunming Li, Vanderbilt University

Result

44

Synthetic noisy image

Slide by: Chunming Li, Vanderbilt University

2D Segmentation of Real Color Images

45

A real image of potatoes

Slide by: Chunming Li, Vanderbilt University

2D Vessel Segmentation

46Slide by: Chunming Li, Vanderbilt University

Segmentation of White Matter in MR images

47Slide by: Chunming Li, Vanderbilt University

Effect of the Level Set Regularization

48

Final zero level contour Final level set function

Without level set regularization

Slide by: Chunming Li, Vanderbilt University

3D Vessel Segmentation

49

MRA Vessel Segmentation

Slide by: Chunming Li, Vanderbilt University

3-D Ultrasound

50http://mrcas.mpe.ntu.edu.sg/research/imgpro/3d_level_set.htm

3-D Ultrasound

51