Segmentation and Perceptual Grouping The problem Gestalt Edge extraction: grouping and completion...

Segmentation and Perceptual Grouping The problem Gestalt Edge extraction: grouping and completion Image segmentation

Camouflage

Kanizsa Triangle

The image of this cube contradicts the optical image

Perceptual Organization

Atomism, reductionism: Perception is a process of decomposing an

image into its parts. The whole is equal to the sum of its parts.

Gestalt (Wertheimer, Köhler, Koffka 1912) The whole is larger than the sum of its parts.

Mona Lisa

Mona Lisa

Gestalt Principles

Proximity

Gestalt Principles

Proximity Similarity

Gestalt Principles

Proximity Similarity Continuity

Gestalt Principles

Closure Proximity Similarity Continuity

Gestalt Principles

Proximity Similarity Continuity

Closure Common Fate

Gestalt Principles

Proximity Similarity Continuity

Closure Common Fate Simplicity

Smooth Completion

Isotropic Smoothness Minimal curvature Extensibility

Elastica

Elastica is not scale invariant

2( ) min ( )E k s ds

,

1( ) ( )

kl l k

E E

Elastica

Scale invariant measure

Approximation

2 21 2 1 2( ) 4( )iE

2( ) min ( )iE l k s ds

1 2

Finding lines from points

Parametric methods: RANSAC

RANSAC

RANdom SAmple Concensus Complexity:

Need to go over all pairs: O(n2) For each pair check how many more points are

consistent: O(n) Total complexity: O(n3 )

RANSAC

Another application of RANSAC: Find transformation between images

Example: compute homography Compute homography for every 4 pairs of

corresponding points Choose the homography that best explains the

image m4n4 sets should be tested

Another example: compute epipolar lines How many correspondences are needed?

Hough Transform

Hough Transform

Linear in the number of points Describe lines as

Or better

Prepare a 2D table

y mx n

cos sinx y c

θ

c

Hough Transform

θ

c

+1+1

+1+1

+1

Hough Transform

θ

c

1316

What if we want to find circles?

Curve Salience

Saliency Network

Encourage Length Low curvature Closure

Saliency Network

Tensor Voting

Every edge element votes to all its circular edge completions

Vote attenuates with distance: e-αd

Vote attenuates with curvature: e-βk

Determine salience at every point using principal moments

Tensor Voting

Stochastic Completion Field

Random walk:

In addition, a particle may die with probability:

2

cos

sin

(0, )

x

y

N

1/ re

Stochastic Completion Fields

Stochastic Completion Fields

Most probable path:

with

Can be implemented as a convolution

2

2

( )

1

21log( 2 )

k s ds ds

r

Stochastic Completion Fields

Stochastic Completion Fields

Snakes

Given a curve Г(s)=(x(s),y(s)), define:

with

1

0

image int ext

image

22 2

int 2

ext

( ( ))

( ( ))

( , )

( ) ( )

...

E s ds

E s E E E

E I x y

E s ss s

E

Extremum: Calculus of Variation Given a functional

A condition for a local extrimum is obtained using the Euler-Lagrange equation

Curve evolution is defined

Solution obtained when

( , )0

x s t E d E

t x ds x

0

( )

x E d E

x s x ds x

0

( , )T

x s E x x ds

( , )0

x s t

t

Curve evolution

Level Set Methods

( , ; )S x y t

( , ; ) 0S x y t Curve defined implicitlyby

Curve Evolution

Curve Evolution

Shortest Path

Image Segmentation: Thresholding

Histogram

0 50 100 150 200 250

0

200

400

600

800

1000

1200

Thresholding

Thresholding

125

15699

S-T Min-Cut/Max Flow

S-T Min-Cut/Max Flow

S

t

Normalized Cuts

Given a graph G=(V,E), define W = {wij} weights

D = diag{di},

L = D - W Laplacian

Let , we seek to solve

( , ) ( , )

( , ) ( , )minV A B

cut A B cut A B

assoc A V assoc B V

i ijj

d w

V A B

Normalized Cuts

This can be show to be equivalent to

with

With these constrains the problem is NP-hard. Without the constraint the solution is obtained

through the generalized eigenvalue problem

minT

Tu

u Lu

u Du

Lu Du

1and 1 0

1

iT

ii

v Au u Dk

v Bk

Normalized Cuts

Dividing into two segments: Partition determined by the eigenvector with the

second smallest eigenvalue We need to pick a threshold

Dividing into more than two segments: Pick several thresholds. Divide each segment recursively. Pick the best few eigenvectors and then perform

k-means.

Texture Examples

Filter Bank

Textons

image textons

textonassignment

Normalized Cuts

Mean Shift Segmentation

Mean Shift Segmentation

Given an image, convert it to a function that is inversely related to edgeness

Perform mean shift from every pixel Cluster pixels that lead to the same peak

Mean Shift Segmentation

Summary

Local processing is often insufficient to separate objects

We reviewed several approaches for curve extraction, completion region segmentation

Preattentive: Parallel

Preattentive: Parallel

Attention: Serial