View
266
Download
0
Tags:
Embed Size (px)
DESCRIPTION
High retrieval precision in content-based image retrieval can be attained by adopting relevance feedback mechanisms. In this paper we propose a weighted similarity measure based on the nearest-neighbor relevance feedback technique that the authors proposed elsewhere. Each image is ranked according to a relevance score depending on nearest-neighbor distances from relevant and non-relevant images. Distances are computed by a weighted measure, the weights being related to the capability of feature spaces of representing relevant images as nearest-neighbors. This approach is proposed to weights individual features, feature subsets, and also to weight relevance scores computed from different feature spaces. Reported results show that the proposed weighting scheme improves the performances with respect to unweighed distances, and to other weighting schemes.
Citation preview
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 1
Pattern Recognition and Applications GroupDepartment of Electrical and Electronic EngineeringUniversity of Cagliari, Italy
PhD Program in Electronic and Computer EngineeringPhD School in Information Engineering
Neighborhood-Based Feature
Weighting for Relevance
Feedback in Content-Based
Retrieval
Luca [email protected]
R AP
Pattern Recognition and
Applications Group
Giorgio Giacinto
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 2
Outline
• Relevance Feedback
• Image representation
• Weighted similarity measures
• State of the art: Estimation of Feature Relevance
• Neighborhood-Based Feature Weighting
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 3
Aim of this work
• Exploiting neighborhood relations to weight
feature sets
• Weight designed to improve Relevance
Feedback based on Distance weighted kth-
Nearest Neighbor
• Dw k-NN estimate the relevance of an image
according to the (non-)relevant one in its
nearest neighborhood
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 4
Distance weighted kth-Nearest
Neighbor
relevanceNN
I( ) =p
NN
rI( )
pNN
rI( ) + p
NN
nrI( )
=I !NN
nrI( )
I !NNr
I( ) + I !NNnr
I( )
where pNN
rI( ) =
1
N
V I !NNr
I( )( )
and V I !NN I( )( )" I !NN I( )
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 5
Images
database
System
Image
Retrieval
Relevance Feedback
User
• Query by examples
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 6
Images
database
System
Image
Retrieval
Relevance Feedback
User
k best ranked images
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 7
Images
database
System
Image
Retrieval
Relevance Feedback
User
image labelling
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 8
Image representation
color textureshape
col. hist. layout color moments co-occurrence texture
I(F)
F = [ f1 … fi … fF ]
f1,1 … f1,i fFi …
f1,1,1 . . .f1,1,j . . .f1,1,32
f1,i,1 . . .f1,i,j . . .f1,i,9
fF,i,1 . . .fF,i,j . . .fF,i,16
fi,1… fi,j
level
image
feature
representation
components
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 9
Image representation
color histogram layout
f1,1 = [g1,1,1, g1,1,2, g1,1,3, g1,1,4 ]
g1,1,1 = [f1,1,1, …, f1,1,8]
g1,1,2 = [f1,1,9, …, f1,1,16]
g1,1,3 = [f1,1,17, …, f1,1,24]
g1,1,4 = [f1,1,25, …, f1,1,32]
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 10
Image similarity
S fi, j( ) = IA fi, j ,k( ) ! IB fi, j ,k( )p
k=1
N
"#$%&'(
1p
S fi( ) = S fi, j( )j
!
S = S fi( )i
! feature (higher)
representation
components (lower)
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 11
Weighted similarity measures
In order to have good performance into images
retrieval systems
• Relevant images should be considered as
neighbors each others.
• Non-relevant images should not be in the
neighborhood of relevant ones.
• Weighted similarity measures.
• Weights related to the capability of featurespaces of representing relevant images as
nearest-neighbors
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 12
Weighted similarity measures
S fi, j( ) = wi, j ,k IA fi, j ,k( ) ! IB fi, j ,k( )p
k=1
N
"#$%&'(
1p
S fi, j( ) = wg idpgIA , IB( )
g=1
G
!
S fi( ) = wi, j iS fi, j( )j
!
S = wi iS fi( )i
! feature (higher)
representation
components (lower)
component subset
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 13
State of the art
• Inverse of standard deviationRui, Huang, Mehrotra. Int. Conf. on Image Processing , 1997
wfj=1
! j
• Probabilistic learning (PFRL)Peng, Bhanu, Qing. Computer Vision and Image Understanding, 1999
wfj=
eT irf j z( )( )
eT irl z( )( )
l=1
F
!
fj is the j-th feature, !j is its standard deviation
rfj(z) is the measure of relevance of the j-th
feature for the query z
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 14
Neighborhood-Based
Feature Weighting
• “Relevance” of different feature space is
estimated in terms of their capability of
representing relevant images as Nearest
Neighbors
• Relevance of an image is estimated according
to the relevant and non-relevant images in its
nearest nieghborhood
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 15
Neighborhood-Based
Feature Weighting
wfx
=p
NN
r fx( )
pNN
r fx( ) + p
NN
nr fx( )
=
dmin
fx I
i,N( )
i!R
"
dmin
fx I
i,R( )
i!R
" + dmin
fx I
i,N( )
i!R
"
where pNN
r fx( ) =
1
VNN
r fx( )
and VNN
r fx( )! 1
card(R)d
min
fx I
i,R( )
i"R
#
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 16
Neighborhood-Based
Feature Weighting
• Evaluation of capability to exploit neighborhood
relations in terms of weighted similarity measures
and in terms of weighted relevance score :
– Components level
– Component subset level
relevance
NNfi, j( )S fi, j( )
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 17
Why better?
• Inverse of standard deviation
– Doesn’t use information about neighborhood of
relevant images
• Probabilistic learning (PFRL)
– It considers only relevant images
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 18
Dataset
• Corel 19511 images
• 43 classes (min: 96 - max: 1544 images)
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 19
Feature sets
• 4 feature sets
– Co-Occurrence Texture (4x4 subsets)
• 4 directions x 4 values
– Color Moments (3x3 subsets)
• first 3 moments x (H, S, V)
– Color Histogram (4x8 subsets)
• 8 ranges of H x 4 ranges of S
– Color Histogram Layout (4x8 subsets)
• 4 sub-images x 8 color
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 20
Experiment setup
• 500 queries
• 9 iterations
• 20 images retrieved each iteration
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 21
Legend
*Dw 2-NN no weight
SVM no weight
Dw 2-NN Probabilistic learning
Dw 2-NN Inverse of standard deviation
Dw 2-NN Neighborhood-Based
Dw 2-NN N-Based component subset
Dw 2-NN N-Based Score component subset
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 22
Experimental Results
Color Histogram
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 23
Experimental Results
Color Histogram
F =1
1
2 ! prec+
1
2 !recall
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 24
Experimental Results
Color Histogram (PFRL)
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 25
Conclusions
• Reported results show that a weighted measure
improve the performance of the NN technique
• Weighted distance metric based on feature
subset provided the best results
• Neighborhood-Based weights provide similar or
better results with respect to PFRL but without
annoying tuning operations