Image Segmentation and Active Contour

Drawing by some archeologist

Segment out the vase from the image

With what we have learned so far, what would (could ) you do?

Active Contour (aka snake, deformable contour)

Better idea: try to fit the boundary with some curve (closed contour).

Contour no longer belong to a small family such as ellipse or circles. The shape conforms to the object boundary.

Vase

Armadillo

Details

1. How to represent contours?

1. Explicit scheme

2. Implicit scheme

2. How to characterize the desired contour

1. As the (global) minimum of the a cost function.

3. How to deform the contour?

1. Discrete optimization (typically for explicit scheme)

2. Partial differential equation (for implicit scheme).

Contour Representation (explicit scheme)

A contour is represented as

1. A collection of control points

2. An interpolating scheme

• Linear interpolation gives a piecewise linear curve

• Other interpolation scheme such as splines give smoother curve.

MATLAB has several routines on splines and polynomial interpolations.

Contour Representation (implicit scheme)

Represent a contour as the zero levelset of a continuous function. No need to keep track of the control points.

Given a contour, one such continuous function is the signed distance function from the contour.

-

+The value at each point is the distance to the given contour. Exterior points are assigned positive values and interior points have negative values.

What is the desired contour?

Variational framework: Express the desired contour as the minimum of a energy functional (cost function).

s is the arc-length parameterization of the contour C=C(s).

Each term in the energy functional correspond to forces acting on the contour.

control the relative strength of the corresponding energy term and they vary along the contour.

The first two terms, continuous and smoothness constraints, are internal to the contour. The third term, an external force, accounts for the edge attraction, dragging the contour toward the closest image edge.

in discrete form

(minimizes contour length)

or better to prevent clusters

The second term minimizes the curvature to avoid zigzags or oscillations.

Curvature is related to the second derivatives and using central difference, the second derivatives can be approximated by

What is curvature? Curvature measures how fast the tangents of the contour vary.

More on curvature

is the tangent angle

If (x, y) gives the parameterization

Edge Attraction term

This terms depend only on the contour and independent of its derivatives.

In general, are set to constants and their values will influence the result.

Deforming the contour

Staring from a collection of points, find the contour which fits the target image contour best, by minimizing the energy functional

Move each control point iteratively.

Deforming Contour

1. Greedy minimization

1. Find point in a small neighborhood that minimizes the energy functional

2. Small neighborhood means low computational load

2. Corner Elimination

1. Search for corners as curvature maxima along the contour. Set i = 0 for points with maximal curvature. Neglecting contribution from these points keep the contour piecewise smooth.

Algorithm Snake

1. For each i= 1, .. , N, find the location in U(pi) for which the energy functional is minimum and move the control point pi to that point.

2. For each i=1, …, N, estimate the curvature and look for local maxima.

• Set i= 0 for all pi at which the curvature has a local maximum or exceeds some user-defined value.

3. Update the value of the average distance, d.

Inputs: Image I and a chain of points on the image

f the minimum fraction of snake points that must move in each iteration and U(p) a small neighborhood of p and d (the average distance).

Keeping track of control points make can be cumbersome. In particular, topological change is difficult to execute correctly.

Main problem with explicit scheme

Implicit Scheme is considerably better with topological change.

•Transition from Active Contours:–contour v(t) front (t)

–contour energy forces FA FC

–image energy speed function kI

•Level set:–The level set c0 at time t of a function (x,y,t) is the set of arguments { (x,y) , (x,y,t) = c0 }

–Idea: define a function (x,y,t) so that at any time,

(t) = { (x,y) , (x,y,t) = 0 }•there are many such

• has many other level sets, more or less parallel to

•only has a meaning for segmentation, not any other level set of

Need to figure out how to evolve the level set function!

Level Set Framework

Usual choice for : signed distance to the front (0)

- d(x,y, ) if (x,y) inside the front

(x,y,0) = 0 “ on “

d(x,y, ) “ outside “

0

0

0

0 0 0 0

0 0

0

0

0

0

0

0

0

0

000

0

0

000

0

-1-1

-1

-1

-1

-1

-1

-1

-1-1-1

-1

-1

-1-1-1

-1

-1

-1 -1 -1 -1

-2 -2 -2 -2

-2

-2

-2

-2-2-2

-2

-2

-2 -2 -3 -3

-3

-3

-3

-3

1

1

1

1

1

1

1

1

111

1

1

111

1

1

1

1

1

1

1 1 1 1

1 1

1

1

2

2

2

2

2

2

2

22

2

222

2

2

2

2

2

2

2

2

2 2 2 2 2

2

2

2

3

3

3

3

3

3

3

3

3

33

333

3

3

3

3

3

3

3 4

4

4

4

4

4

4

4444

4

4

4

4

4

4

7

5

5

5

5

55

5

5

5

5

6

6

6 6

(t)(x,y,t)

(x,y,t)

0

-2

5

Level Set

0

0

0

0 0 0 0

0 0

0

0

0

0

0

0

0

0

000

0

0

000

0

-1-1

-1

-1

-1

-1

-1

-1

-1-1-1

-1

-1

-1-1-1

-1

-1

-1 -1 -1 -1

-2 -2 -2 -2

-2

-2

-2

-2-2-2

-2

-2

-2 -2 -3 -3

-3

-3

-3

-3

1

1

1

1

1

1

1

1

111

1

1

111

1

1

1

1

1

1

1 1 1 1

1 1

1

1

2

2

2

2

2

2

2

22

2

222

2

2

2

2

2

2

2

2

2 2 2 2 2

2

2

2

3

3

3

3

3

3

3

3

3

33

333

3

3

3

3

3

3

3 4

4

4

4

4

4

4

4444

4

4

4

4

4

4

7

5

5

5

5

55

5

5

5

5

6

6

6 6

(x,y,t)

(x,y,t+1) = (x,y,t) + ∆(x,y,t)

1

-1

-1

-1

-1

-1

-1

-1

-1-1

-1

-1

-1-1-1

-1

-1 -1

1

1

1

-1

1

1

1

1

1

1

1

1

1

111

1

111

1

1

1

1

1

1 1

1 1

1

-2

-2

-3

-2 -2 -2 -2

-2

-2

-2

-2-2-2

-2

-2

-2 -2 -3 -3

-3

-3

-3

-3

2

2

2

2

2

2

2

2

22

2

222

2

2

2

2

2

2

2

2

2 2 2

0

0

0

0

0

0

0

0

0

0

0

0

00

0

000

0

0

0

0

0

0

0 0

0 0

0

2

2

2

2

3

3

3

3

3

3

3

33

333

3

3

3

3

3

3

3 4

4

4

4

4

4

4

4444

4

4

4

4

4

4

7

5

5

5

5

55

5

5

5

5

6

6

6 6

• no movement, only change of values

• the front may change its topology

• the front location may be between samples

Level Set

Segmentation with LS:• Initialise the front (0)• Compute (x,y,0)• Iterate:

(x,y,t+1) = (x,y,t) + ∆(x,y,t)

until convergence

• Mark the front (tend)

Here is the equation … (more about it later)Front Speed !!

• Equation 17 p162:

t

ˆ k I FA FG () 0

link between spatial and temporal derivatives, but not the same type of motion as contours!

div

constant “force”

(balloon pressure)

(x,y,t+1) - (x,y,t)

extension of the speed function kI (image influence)

smoothing “force” depending on the local curvature (contour influence)

spatial derivative of

product of influences