Image Search Using Deformable Contours

By : Preeyakorn Tipwai

Advisor : Suthep Madarasmi, Ph.D

Computer Vision Laboratoy, Computer Engineering Department

King Mongkut’s University of Technology Thonburi

Problem

A target is assumed to be a scaled, rotated

version of the template with edges distorted

Methodology

InspirationJain et al [1] , “Object Matchin

g Using Deformable Templates”

Our Methodogy Finding Hypotheses : MGH

T Peak Clustering :

Watershed Method Contour Matching : Smo

oth Membrane Fitting

Preprocessing

Given a sketched template Find tangent direction

Given a target image Calculte edge map : Canny Edge Detection Find tangent direction

MGHTMGHT2

r

L,

xc,yc

= 0

A line at the contour edge is extended in the direction until it meets the other end of the contour

r1, 1, 1, l1 r2, 2, 2, l2 r3, 3, 3, l3 r4, 4, 4, l4

0...19 15,180,195,99 9,179,219,101 8,177,216,102 9,176,198,100

20...39 17,160,23,5 14,159,38,7 18,161,175,62 15,162,195,95

30…49 19,165,31,53 20,170,8,52 22,167,15,52 18,159,158,12

… … … … …

340...359

23,105,346,11 24,103,165,11 21,102,346,18 22,104,195,24

R-Table

MGHTMGHT Invariant rotation

and scale of

2

L,

= 0 2

L,= 0

= 30-200 = -170 = 190 = 300-110 = 190

MGHTMGHT

θθβ c

L

L S c

Rotation Factor:

Scaling Factor :

New ref. Point :

xc = x + S r cos (

yc = y + S r sin (

Watershed for Peak Clustering

1. Shed, by labeling, at the first level, calculate peaks of each label2. Increase to higher level, shed again

2.1 Meet an area of previous level, shed to that area2.2 Not meet any area of previous level, make a new area ,

calculate a new peak

Deformation : Contour Matching

Parameter : xyor (u,v)

Grid Matching : Data and Smoothness ConstraintsInter-grid Matching: Consistency between adjacent grids

Coarse and Fine Matching

Coarse and Fine Matching

Inter-grid Matching: Example

label at coarser level label at finer level

-1

0

1

-4, -3 or –2

-1, 0 or 1

2, 3 or 4

Matching Algorithm

Update (u,v) : Gibbs Sampler with simulated annealing to minimize energy function

Template

Target Edge

Experiment on Contour Matching

TemplateTemplate

TargetTargetEdgeEdge

ResultResult

Experiment on Contour Matching

TemplateTemplate

TargetTargetEdgeEdge

ResultResult

Experiment on Image Search

Template Target Edge Map ResultHough Space

Experiment on Image Search

Target Edge Hypotheses 1st Match

2nd Match 3rd Match 4th Match

Experiment on Image Search

Template Target Edge Map The Best Match Hough Space

Experiment on Image Querying

Database Search for Circle shape Search for bulb shape

Conclusion

A deformation model

Contour Matching

A method for image search

Future work: large image database, efficient method for minimizing energy, coarse-and-fine approach to computer vison modules

Similarity Retrieval Effectiveness

0

20

40

60

80

100

120

#1 #2 #3 #4 #5

Agreement

Average

0

20

40

60

80

100

#1 #2 #3 #4

0

20

40

60

80

100

#1 #2 #3 #4

circle shapecircle shape heart shapeheart shape bulb shapebulb shape

max : 100, min : 96ave : 98

max : 100, min : 92ave : 95

max : 96, min : 8ave : 75

Experimental Result

3.986274 0.929011 2.705226Template Target

EdgeHypotheses

Threshold : 1.0 - 2.6

Experimental Result

2.165488 0.965049 1.755835

Threshold : 1.0-1.6

Template Target Edge Hypotheses

Experimental Result

1.799267 1.114566 5.074061

Threshold : 1.2-1.6

Template Edge Map Hypotheses

Experimental Result

Template TargetEdge

Hypotheses 0.868600 0.879799 3.799124

Threshold : 0.9-3.6

Experimental Result

Template Target Edge Hypotheses

1.293034 1.452130 3.364521 4.4185782

Threshold : 1.5-3.2

Energy Threshold

0

0.5

1

1.5

2

2.5

3

3.5

4

accepted rejected

guitar1

guitar 2

vase

bottle

lamp