Embed Size (px)
Citation preview
Segmentation and Perceptual Grouping The problem Gestalt Edge extraction: grouping and completion Image segmentation
Camouflage
Kanizsa Triangle
The image of this cube contradicts the optical image
Perceptual Organization
Atomism, reductionism: Perception is a process of decomposing an
image into its parts. The whole is equal to the sum of its parts.
Gestalt (Wertheimer, Köhler, Koffka 1912) The whole is larger than the sum of its parts.
Mona Lisa
Mona Lisa
Gestalt Principles
Proximity
Gestalt Principles
Proximity Similarity
Gestalt Principles
Proximity Similarity Continuity
Gestalt Principles
Closure Proximity Similarity Continuity
Gestalt Principles
Proximity Similarity Continuity
Closure Common Fate
Gestalt Principles
Proximity Similarity Continuity
Closure Common Fate Simplicity
Smooth Completion
Isotropic Smoothness Minimal curvature Extensibility
Elastica
Elastica is not scale invariant
2( ) min ( )E k s ds
,
1( ) ( )
kl l k
E E
Elastica
Scale invariant measure
Approximation
2 21 2 1 2( ) 4( )iE
2( ) min ( )iE l k s ds
1 2
Finding lines from points
Parametric methods: RANSAC
RANSAC
RANdom SAmple Concensus Complexity:
Need to go over all pairs: O(n2) For each pair check how many more points are
consistent: O(n) Total complexity: O(n3 )
RANSAC
Another application of RANSAC: Find transformation between images
Example: compute homography Compute homography for every 4 pairs of
corresponding points Choose the homography that best explains the
image m4n4 sets should be tested
Another example: compute epipolar lines How many correspondences are needed?
Hough Transform
Hough Transform
Linear in the number of points Describe lines as
Or better
Prepare a 2D table
y mx n
cos sinx y c
θ
c
Hough Transform
θ
c
+1+1
+1+1
+1
Hough Transform
θ
c
1316
What if we want to find circles?
Curve Salience
Saliency Network
Encourage Length Low curvature Closure
Saliency Network
Tensor Voting
Every edge element votes to all its circular edge completions
Vote attenuates with distance: e-αd
Vote attenuates with curvature: e-βk
Determine salience at every point using principal moments
Tensor Voting
Stochastic Completion Field
Random walk:
In addition, a particle may die with probability:
2
cos
sin
(0, )
x
y
N
1/ re
Stochastic Completion Fields
Stochastic Completion Fields
Most probable path:
with
Can be implemented as a convolution
2
2
( )
1
21log( 2 )
k s ds ds
r
Stochastic Completion Fields
Stochastic Completion Fields
Snakes
Given a curve Г(s)=(x(s),y(s)), define:
with
1
0
image int ext
image
22 2
int 2
ext
( ( ))
( ( ))
( , )
( ) ( )
...
E s ds
E s E E E
E I x y
E s ss s
E
Extremum: Calculus of Variation Given a functional
A condition for a local extrimum is obtained using the Euler-Lagrange equation
Curve evolution is defined
Solution obtained when
( , )0
x s t E d E
t x ds x
0
( )
x E d E
x s x ds x
0
( , )T
x s E x x ds
( , )0
x s t
t
Curve evolution
Level Set Methods
( , ; )S x y t
( , ; ) 0S x y t Curve defined implicitlyby
Curve Evolution
Curve Evolution
Shortest Path
Image Segmentation: Thresholding
Histogram
0 50 100 150 200 250
0
200
400
600
800
1000
1200
Thresholding
Thresholding
125
15699
S-T Min-Cut/Max Flow
S-T Min-Cut/Max Flow
S
t
Normalized Cuts
Given a graph G=(V,E), define W = {wij} weights
D = diag{di},
L = D - W Laplacian
Let , we seek to solve
( , ) ( , )
( , ) ( , )minV A B
cut A B cut A B
assoc A V assoc B V
i ijj
d w
V A B
Normalized Cuts
This can be show to be equivalent to
with
With these constrains the problem is NP-hard. Without the constraint the solution is obtained
through the generalized eigenvalue problem
minT
Tu
u Lu
u Du
Lu Du
1and 1 0
1
iT
ii
v Au u Dk
v Bk
Normalized Cuts
Dividing into two segments: Partition determined by the eigenvector with the
second smallest eigenvalue We need to pick a threshold
Dividing into more than two segments: Pick several thresholds. Divide each segment recursively. Pick the best few eigenvectors and then perform
k-means.
Texture Examples
Filter Bank
Textons
image textons
textonassignment
Normalized Cuts
Mean Shift Segmentation
Mean Shift Segmentation
Given an image, convert it to a function that is inversely related to edgeness
Perform mean shift from every pixel Cluster pixels that lead to the same peak
Mean Shift Segmentation
Summary
Local processing is often insufficient to separate objects
We reviewed several approaches for curve extraction, completion region segmentation
Preattentive: Parallel
Preattentive: Parallel
Attention: Serial
