SVD-SIFT FOR WEB NEAR-DUPLICATE IMAGE DETECTIONImage Processing (ICIP), 2010 17th IEEE International Conference on September 26-29, 2010

Outline• Near-duplicate image• Scale Invariant Feature Transform (SIFT) & SURF• Introduction• Singular Value Decomposition (SVD) Theorem • Low-rank matrix approximation• SVD-SIFT• Experiment Result• Conclusion

Near-duplicate image

Near-Duplicate Image Detection

Web near-duplicate image

Near-duplicate image type demonstration and Detection effect based on SIFT descriptor

SIFT & SURF

593 feature points

782 feature points 34 pairs of feature points match

Introduction• Two kinds of Methods for near-duplicate image

detection :• Global features• Performance good when image format, size and

quality change, but poor for the complex edition techniques such as inserting, forging, etc.

• Local features• It has good detection effect on the near-duplicate

images processed with different formats, size transformation, complex edition, post processing, etc. However, the disadvantage of local descriptor methods is the high computational cost.

Introduction

Near-duplicate image type demonstration and Detection effect based on SIFT descriptor

• SIFT is the kind of method based on local features.

• In this paper, we propose SVD-SIFT feature, it uses the property of SIFT feature itself to improve the matching speed.

SVD Theorem• The matrix SVD theorem can be described as

follows:• If M ∈ Rm×n (m > n), rank(M) = r, then there exists two

orthogonal matrices U, V , and a diagonal matrix makes the establishment of the following equation:

Where U = [u1, u2, u3, ..., um] ∈ Rm×m, UT = U-1, UUT

= UT U = I

; V = [v1, v2, v3, ..., vn] ∈ Rn×n , VT = V-1, VVT

= VT V = I ;

Σ = diag(σ1, σ2, ..., σr, 0, ..., 0) ∈ Rm×n, σ1 ≥ σ2 ≥ ... ≥ σr .

=

mxn mxm mxn nxn

SVD Theorem

=

• The singular value decomposition and the eigenvalue decomposition are closely related. Namely:• The left singular vectors of M are eigenvectors of MMT .• The right singular vectors of M are eigenvectors of MTM .• The non-zero singular values of Σ are the square roots of

the non-zero eigenvalues of MTM or MMT .

mxn mxn

The left singular vectors of MThe right singular vectors of M

u1 u2 ... um

v1

vn

v2...

SVD Theorem• The matrix singular value has the following

characteristics:• Characteristic 1: Transposition and replacement

invariance• That is to say, after transposition or row-column

replacement operation of the matrix, its singular value is invariable.

• Characteristic 2: Energy concentricity• The matrix M can be approximately restructured by the

first k largest singular values of M. It can be proved that the matrix corresponding to the first k largest singular values of A is the closest to matrix M under the Frobenius norm. (Low-rank matrix approximation)

Low-rank matrix approximation• Consider a matrix M ∈ Rm×n and its singular value

decomposition M = UΣVT , then the matrix M in the Frobenius norm of rank k(k ≤ min(m, n)), the best approximation matrix can be expressed using the following formula:

Where

U = [u1, u2, u3, ..., um] ∈ Rm×m

V = [v1, v2, v3, ..., vn] ∈ Rn×nΣ = diag(σ1, σ2, ..., σr, 0, ..., 0) ∈ Rm×n

U = [u1, u2, u3, ..., um] ∈ Rm×m

V = [v1, v2, v3, ..., vn] ∈ Rn×n = diag(σ1, σ2, ..., σk, 0, ..., 0) ∈ Rm×n

Low-rank matrix approximation

‖𝑈 ~Σ𝑉 𝑇 −~𝑀‖𝐹 ‖𝑈 𝑇𝑈 Σ𝑉 𝑇𝑉 −𝑈 𝑇~𝑀𝑉‖𝐹

SVD-SIFT• The image SIFT feature is one set that contains many local

feature points, each feature point is described by a 128-dimensional vector. Therefore, the SIFT feature points set of one image can be represented with a matrix.

SVD-SIFT extraction and matching based on SIFT feature points set.

n

• According to Characteristic 1: • The singular value of the image SIFT feature matrix is not

related to the position of SIFT feature point. Then, we do not need to care about the position of the SIFT points when matching.

• According to Characteristic 2:• We can use the energy concentricity of the first k largest

singular values of the image SIFT feature matrix to greatly reduce the matching time cost.

SVD-SIFT

SVD-SIFT• Suppose that A and B represent two images

containing m and n SIFT feature points respectively, the matching algorithm has the following four steps :• Step 1:• Matrix A128×m = (A1,A2, ...,Am) represents the feature point set of

image A and matrix B128×n = (B1,B2, ...,Bn) represents the feature points set of image B, respectively.

• Step 2:• A d-dimensional linear subspace of A and B is represented

by an orthonormal basis matrix PA ∈ A128×d and PB ∈ B128×d

respectively subject to AATPAΛAPATand BBTPBΛBPBT, where ΛA and ΛB are the eigenvalue diagonal matrices of the d largest eigenvalues, PA and PB are the eigenvector matrices of the d largest eigenvalues.

SVD-SIFT• Step 3:• Using the first k largest values of ΛA and ΛB respectively

to constitute the SIFT singular value feature vector of images A and B.

• Step 4: • Two measurement methods can be chosen when

matching:1) The L1 distance is used to compute the similarity of k

dimensional SIFT singular value characteristic;2) The singular value decomposition is carried out for PATPB ∈

Rd×d, so PATPB = USVT , then the similarity between point sets A and B can be measured by the trace of the singular value matrix S, sim(A,B) = trace(S).

Experiment Result• Ordinal Intensity Signature(OIS)• Global features• L1 distance matching

• SIFT• Local features• Best-Bin-First-Tree matching

• SVD-SIFT• Global features• L1 distance matching

Experiment Result• Precision (P)

• Recall (R)

Experiment Result• The detection time

includes the time for feature extraction and pre-processing of the query image.

Conclusion• The computational costs of directly matching two

SIFT feature points sets is high. In this paper, SVD-SIFT has been proposed and it has been theoretically proven.

• According to energy concentricity of matrix singular value, the first k largest singular values of image SIFT feature matrix maintains the original characteristic of the matrix well, experimental results demonstrate that the method can obtain a better tradeoff between the effectiveness and efficiency for detection.