Image Matching with Distinctive Visual Vocabulary

Hongwen Kang Martial HebertSchool of Computer ScienceCarnegie Mellon University

{hongwenk, hebert, tk}@cs.cmu.edu

Takeo Kanade

Abstract

In this paper we propose an image indexing and match-ing algorithm that relies on selecting distinctive high dimen-sional features. In contrast with conventional techniquesthat treated all features equally, we claim that one can ben-efit significantly from focusing on distinctive features. Wepropose a bag-of-words algorithm that combines the featuredistinctiveness in visual vocabulary generation. Our ap-proach compares favorably with the state of the art in imagematching tasks on the University of Kentucky RecognitionBenchmark dataset and on an indoor localization dataset.We also show that our approach scales up more gracefullyon a large scale Flickr dataset.

1. IntroductionMuch progress has been achieved in image matching

in recent years [25, 26, 27]. Despite considerable ad-vances, achieving good performance for applications in-volving large image databases remains challenging [13, 34].There are obvious computational issues because, as thedatabase size increases, finding exact nearest neighbors be-comes inefficient, especially for high-dimensional imagefeatures. More importantly, as the database size increases,the number of similar feature points in a unit distance in-terval increases exponentially [3, 36], i.e., the curse of di-mensionality starts to affect performance (Figure 1). There-fore, distinguishing distinctive features from indistinctivefeatures is important for both efficiency and, more impor-tantly, matching accuracy.

Much of the literature has focused on the computationalissues. For example, [33] proposed a framework based onthe bag of words (BOW) model from the text retrieval do-main. To further improve the efficiency, a tree data structurewas adopted in [29]. The limitations of the vector quantiza-tion approaches have been addressed and the proposed im-provements include adding local soft distance [15, 30], im-proving the symmetry among nearest neighbors [16], min-

Hash and spatial verification [4, 31], fast search with metriclearning [14], and other variations. Advances in approxi-mate nearest neighbor search also contributed to improvingefficiency [1, 28].

In this paper we explore the feasibility of addressing thesecond issue, i.e., distinguishing distinctive features fromindistinctive features. Intuitively, we note that the conceptof feature distinctiveness only makes sense when we con-sider the feature vector in the context of its nearest neigh-bors in the feature space (Figure 1). When all of its nearestneighbors are similarly close, a feature is indistinctive andhas little or even negative contribution to image matching.From the literature, we found these heuristics can be sum-marized well by a formal definition of feature distinctive-ness based on statistics and information theory [11], whichhas been successfully applied in content-based image re-trieval as a retrieval quality criteria, after the retrieval is ac-complished [19]. In this work, we explore its effectivenessin feature selection and indexing large image databases. Weshow that feature distinctiveness is strongly correlated withimage matching performance. This observation is espe-cially important, given the tendency to use larger and largernumber of high dimensional features in the image matchingproblem [29, 31, 16]. We propose a bag-of-words algorithmthat combines the feature distinctiveness in visual vocabu-lary generation.

Some of the published approaches used other strategiesfor selecting features. For example, in [33], a visual vocab-ulary was generated using only features that could be stablytracked through consecutive video frames, which eliminatesa large number of features that are not repeatable. However,this tracking based process is not applicable to static imagedatasets that are not extracted from the same video stream.Also, in [25], an empirical method was used to prune thecandidate feature matches by requiring that a good matchmust have distance less than 0.6 times that of the secondbest match. Although this criteria is intuitive, it is difficultto formulate the underlying theory and this approach is notapplicable to large scale image matching problems where

1

(a) Distinctive feature (b) Indistinctive feature

Figure 1. An example comparing distinctive features (left) and indistinctive features (right). Each row shows: 1) the query and retrievedimages; 2) the feature that was used as the query and its nearest neighbors; and, 3) their corresponding distances. When all of its nearestneighbors are similarly close, a feature is indistinctive and has little or even negative contribution to image matching.

the same and or similar objects can appear in multiple im-ages.

Our approach is related to the supervised feature selec-tion approaches in the literature. For example, [35] and [23]use information theory and conditional entropy. [24] alsoshowed an interesting idea on reducing the number of fea-tures used in location representation through optimizing theweighting of features to maximize the posterior probabilityof a location. [32] is most similar to our approach in thatthey generate a visual vocabulary by maximizing the infor-mation gain in distinguishing different locations. The ma-jor difference between these approaches and ours is that ourdistinctive feature selection process is fully unsupervised,based on the non-uniformity of high dimensional featurespace and feature distinctiveness. Beyond the obvious scal-ability advantage of unsupervised techniques, our approachis also more appropriate for the image matching problemitself. For example, in Figure 2, our approach discoversthe distinctive “eye” type of features that are important forimage matching. However, since this feature also appearson several different kinds of objects, they would be down-weighed by the supervised approaches due to the confusionfor classification.

The use of local nearest neighbor statistics in our ap-proach has additional connections to estimating distancefunctions for recognition and retrieval tasks [9, 10, 37].Although our primary goal is for specific instance match-ing, our general approach is also similar to approachesin object categorization that try to identify important fea-tures/locations for category recognition [7, 22, 38]. Alsonote that our distinctiveness measure is based on localnearest neighbor information, which is fundamentally dif-ferently from the dimensionality reduction literature suchas [8, 20]. These are global approaches that seek to lowerthe dimensionality of the whole feature space, without tak-ing advantage of the non-uniformity in the data distribution[6, 17].

Our approach is generally applicable to any high di-mensional features used in image matching. In this pa-

per, we use the Scale Invariant Feature Transformations(SIFT) [25]. We use the standard benchmark of [29] in or-der to validate the effectiveness of our approach. Since ourapproach is particularly well-suited for situations in whichthe database images are very confusing, we also evaluate theperformance on a public dataset from an indoor localizationtask [18]. In this task, the images differ in relatively smalldetails which cannot be captured without accounting explic-itly for the distinctiveness of features. Finally, we demon-strate that our approach scales up more gracefully on a largescale Flickr dataset.

2. Distinctiveness of high dimensional features2.1. Definition of feature distinctiveness

We first introduce the measure of feature distinctivenessand we show its connection with the concept of intrinsicdimensionality [19]. In a high dimensional space, assum-ing uniform distribution, the number of feature points in aunit ball increases exponentially with respect to the dimen-sionality (the “curse of dimensionality” [3].) This is indeedwhat makes information retrieval applications, such as im-age search, extremely challenging. As shown in [11, 19],in a high dimensional space, the expected ratio of the dis-tance between a query feature point to the (K+1)th nearestneighbor and its distance to the Kth nearest neighbor is:

E{d(K+1)NN}E{dKNN}

≈ 1 +1

Kn, (1)

where n is the dimension of the feature space. The ratio de-creases monotonically as the dimensionality increases. Fora given n, the maximum ratio is achieved between the near-est and second nearest neighbors of the query feature point,which is:

E{d2NN}E{d1NN}

≈ 1 +1

n. (2)

For example, for a 128 dimensional feature, this expecteddistance ratio is merely 1.008. Features with such high

Example All features Database Distinctive features

Figure 2. Given an image and the features detected, it is insufficient to determine which feature is more informative by itself; however,after putting this image in the context of the given database, our algorithm is able to select the distinctive features based on the statistics oftheir nearest neighbors. (Features shown here only illustrate the locations of the original MSER regions.)

dimensionality are therefore very unreliable for similaritysearch, because any small disturbance to the feature space,e.g., lighting or viewpoint, could change the nearest neigh-bor ordering. Fortunately, in real-world applications, 1) fea-ture spaces are rarely uniform, and 2), locally, the featureintrinsic dimensionality is much lower than the actual di-mensionality n of the parent feature space.

Based on these observations, we can draw a direct con-nection between feature distinctiveness and its intrinsic di-mensionality. Then our goal is to select the features thathave low intrinsic dimensionality (distinctive) and filter outthe ones that have high intrinsic dimensionality (indistinc-tive). To this end, [19] suggested a generative model forestimating the data likelihood for a given intrinsic dimen-sionality, by simply counting the number of nearest neigh-bors appearing within some range from the feature point ofinterest. It was shown that given an intrinsic dimensionalityn′, if the nearest neighbor of a feature x is at distance dNN ,the likelihood of observing more than Nc data points in thedistance range of Rp × dNN (Rp ≥ 1.0) is:

P (Nc|n′, Rp) = (1− 1

(Rp)n′ )

Nc , (3)

note that P (Nc|n′, Rp) is independent of the absolute valueof dNN . In this paper, we use this likelihood definition asour measure of the feature distinctiveness. Intuitively, themore features (larger Nc) observed in this distance range theless distinctive x is. For example, the query feature of Fig-ure 1(b) is less likely to have a low intrinsic dimensionalitythan that of Figure 1(a), because many of the other neigh-bors have almost the same distance as its nearest neighbor,therefore Nc will be quite large. In practice, we will choosea desired intrinsic dimensionality n′, we will estimate Nc

for each feature and select the features for which P (Nc..) islarger than certain threshold. In our experiments, we calcu-late the values of Rp for each n′ in the same way exactly asin [11, 19].

2.2. Correlation between feature distinctiveness andimage matching performance: a case study

From a public image dataset [29] that we used in our ex-periments, we generate some empirical statistics of feature

Figure 3. Statistics of feature distinctiveness estimated from thedataset of [29]. A 128 dimensional MSER-SIFT feature is used;we assume that the intrinsic dimensionality is 6. In each graph,statistics are drawn from images categorized based on their match-ing scores (1 − 4) resulted from a baseline algorithm (N.S. [29]),which does not use distinctiveness. The graphs show a strongcorrelation between the number of distinctive features and the re-trieval performance, i.e., the more distinctive features exist in thequery image, the better it will get matched to the correct databaseimages.

distinctiveness (Figure 3). We set n′ = 6 and Rp = 2.77for this empirical study. To see the strong correlation be-tween feature distinctiveness and image matching perfor-mance, we categorize query images based on their match-ing performance, using a baseline algorithm that does notuse feature distinctiveness [29]. We notice that the moredistinctive features the query image has, the better it willget matched to the correct database images.

3. Image matching with distinctive featuresWe propose an integration of the feature distinctiveness

with the bag-of-visual-words framework [33]. The majorsteps of our approach are: distinctive visual vocabulary gen-eration, distinctive feature selection for database image in-dexing and query image representation, vector quantizationand retrieving, as shown in Figure 4.

3.1. Distinctive visual vocabulary generationThe purpose of visual vocabulary generation is to cluster

a large set of high dimensional features to a finite set thatare representative for the visual patterns in the database.

Figure 4. Diagrams of the distinctive bag-of-words (Distinc-tive BOW) approach.

This visual vocabulary essentially compresses the datasetbecause it is usually much smaller than the original featureset [12, 33]. The K-means algorithm is frequently used forthis purpose. The original K-means algorithm can be for-mulated into the following minimization problem,

argminC

K∑i=1

∑xj∈Ci

‖xj − ci‖2, (4)

where C = ∪Ci. Ci is a cluster consists of a set of datapoints with their averaged center at ci.

This clustering approach, however, is sensitive to thenon-uniformity or bias in the dataset [21]. This is especiallyproblematic for the K-means algorithm, since the calcula-tion of a cluster center can be skewed by a few data pointsthat are far away from the actual cluster centers [11, 12].In this paper, we adopt a weighted clustering algorithm thatweighs feature points with their distinctiveness. The weigh-ing strategy has two advantages. First, it leads to clustercenters that are closer to regions where features with higherdistinctiveness are located at; and second, the skewing ef-fect of indistinctive features far away from the cluster cen-ters will be alleviated by their low distinctiveness. We de-fine the clustering problem as the following,

argminC

K∑i=1

∑xj∈Ci

wj ∗ ‖xj − ci‖2, (5)

where wj is the weight for each data point xj . In our exper-iments, we set wj to the distinctiveness measure of xj . Theoutput of the distinctiveness weighted clustering process isa set of cluster centers. This set is called distinctive visualvocabulary, and a member of it is called a distinctive visualword.

3.2. Database image indexing and retrieving withbag-of-distinctive-words

After the visual vocabulary is constructed, we assign adiscrete digit to represent a high dimensional feature vector.This digit represents the visual word that is the closest to thefeature vector, under the Euclidean distance. This processis normally known as vector quantization [12]. Convention-ally, any given feature vector is forced to be assigned to atleast one of the clusters, regardless of the specific distanceconfiguration between this feature vector and all the clustercenters. In this paper, however, instead of quantizing everyfeature vector in the image, we propose to select only a sub-set of features that are most distinctive, then quantize anduse them for indexing and searching.

The distinctiveness of an image feature is calculated inthe same way as described in Section 2. But this time, thereference feature set that an image feature vector comparewith is the distinctive visual vocabulary generated from theprevious step, instead of using the original features from thedatabase. Because the distinctive visual vocabulary is rela-tive small, we can efficiently calculate the distinctiveness ofeach feature vector. Also, since one needs to find the near-est neighbor for the purpose of vector quantization any way,using the visual vocabulary for distinctiveness calculationadds negligible computational cost.

After the distinctiveness of the query features is calcu-lated, we apply a threshold (0.9) to select the distinctiveones and vector quantization is used to transform the origi-nal feature vectors to discrete visual words indexes, we notethe generated image representation by bag-of-distinctive-words. Based on these bag-of-distinctive-words, each im-age is now represented by a term frequency inverse docu-ment frequency vector (TF-IDF). More details of TF-IDFformulation and its application in image search can befound in [2, 33]. Distances between a query image and thedatabase images are then calculated by the L1 distances ofthese TF-IDF vectors [29].

Instead of distinctive feature selection, one can use thedistinctiveness of a feature as the soft weight in the TF-IDFvectors. We choose not to implement this strategy in thispaper for the consideration of computational expense. Cal-culating the precise distinctiveness requires searching for alarge number of nearest neighbors (Nc), which can be in-efficient. Instead, for a distinctiveness threshold (0.9), todetermine whether a feature is distinctive or not, the maxi-mum number of nearest neighbors that one needs to retrieveis less than a hundred. This is especially useful for the sce-narios where early stopping can decrease the computationalexpense significantly [5, 19].

4. ExperimentsThe University of Kentucky Recognition Benchmark

[29] provides a suitable baseline to demonstrate and analyze

the effectiveness of our proposed approach. This dataset in-cludes 2550 objects and scenes, e.g. Figure 2. Each ob-ject is captured 4 times from different viewpoints, distancesand illumination conditions. For evaluation, each image isused as query and a score between 1 and 4 is calculated forthe retrieval results, corresponding to the number of rele-vant images returned by the algorithm among the top 4 (4 isthe highest achievable score, meaning that all four databaseimages matching the query image have been found). Thisscore divided by 4 is the value that the precision and re-call measurements are equal at. In addition to this bench-mark dataset, we collected a larger image dataset that con-sists of about 500 thousand high resolution images down-loaded from the Flickr website, using the public API for thedaily list of “interesting” photos1. In our experiments, wedenote the University of Kentucky Recognition Benchmarkitself by “UKBench”, and the combined dataset by “UK-Bench+Flickr”.

4.1. Object instance recognition

General object recognition has been a difficult researchproblem in the literature. Using image search techniques,however, one can achieve promising results in recognizingspecific instances of objects [29]. The University of Ken-tucky Recognition Benchmark dataset has been commonlyused for benchmarking the performance of various imagesearch algorithms in this application [16, 29, 31]. In thisexperiment, we use the standalone UKBench dataset. To beconsistent with the other baselines, here we use the origi-nal 7 million SIFT features computed from MSER regions[25, 26]. We quantitatively evaluate our approaches andcompare them with the state of the art.

We vary the selections of the intrinsic dimensionality n′

in the distinctive visual vocabulary generation process (n′V )

and the feature selection process for the bag-of-distinctive-words (n′

I ). We compare our performance with severalstate-of-the-art techniques (Table 1). The first baseline al-gorithm is the Nister and Stewenius algorithm [29] that usesa hierarchical K-means algorithm for vocabulary generationand a multi-level scoring strategy. The second baseline al-gorithm is proposed by Jegou et. al. [16] which showedthat improving the nearest neighbor symmetry with a con-textual similarity measure could improve the matching per-formance.

The third baseline algorithm [31] is most similar to ourapproach, which used approximate K-means algorithm forlarge vocabulary generation. In fact, it is exactly the same asour algorithm when n′

V = +∞ and n′I = +∞, i.e., every

feature has distinctiveness 1.We pick the parameters by 10-round cross validation on

a hundred samples randomly selected from the database,and choose the combination that is favored the most, i.e.,

1http://www.flickr.com/services/api/flickr.interestingness.getList.html

Algorithm ScoreNister and Stewenius[29] 3.29

Jegou, et.al.[16] 3.38Philbin, et.al.[31] 3.45Distinctive BOW 3.51

Table 1. Quantitative comparison of our approach to other base-line bag-of words approaches that do not use distinctive featureselection in vocabulary generation.

N.S.[29] R.S.[18] Distinctive BOWClean set 0.996 0.999 1.0

Confusing set 0.843 0.905 0.98Table 2. Quantitative comparison of our approach with the N.S. al-gorithm [29] and the R.S. algorithm [18] on an image based indoorlocalization task.

n′V = 25, n′

I = 25 and vocabulary size 1M . Although,the performance does not vary much for other choices ofn′, e.g. n′

V = 25 and n′V = 30, and vocabulary sizes,

e.g., 500K and 1M (Figure 5). Considering the potentialvariance due to K-means initialization, for all the experi-ments with the 1M vocabulary, we used fixed initial clustercenters that are randomly initialized for once; and for eachof the other experiments, we used independently randomlyinitialized cluster centers. We found that the initializationdoes not cause significant variance to the algorithm. Figure5 shows the performance for different parameters and thecomparison to other baseline algorithms is shown in Table1.

Figure 6 shows a representative example comparing ourresult (Distinctive BOW) and that of the baseline algorithm[29], since this is the only publicly available feature set withquantized visual words. We implemented the flat weightingscheme and we verified that our implementation of the base-line got exactly the same score as published.

4.2. Image based indoor localizationIn this section, we analyze the performance of the pro-

posed image matching approach in the context of an indoorlocalization application. In this application, the location ofa user is estimated by matching an image taken by the userwith a large database of position-tagged images. The chal-lenge in this task stems from the high degree of repeata-bility in man-made environments. The resulting ambigu-ity complicates the matching process dramatically. For thistask, [18] proposed an iterative algorithm (Re-Search, orR.S.) that refines a similarity function based on the visualwords distribution in a small neighborhood. However, thisapproach is unstable and sensitive to quantization noises.Therefore, [18] resorted to an intermediate solution thatcombines the local and global similarity functions. In con-trast, our approach fits very well into this scenario because

(a) (b) (c)

Figure 5. The effects of different parameter choices on recognition performance, (a), different intrinsic dimensionality choices in distinc-tive visual vocabulary generation; (b), different sizes of distinctive visual vocabularies; (c), different intrinsic dimensionality choices indistinctive feature selection for indexing and retrieving.

query rank2 rank3 rank4 rank5 rank6 rank7 rank8

N.S

.O

urs

Figure 6. Comparison of the performance of our approach and the N.S. algorithm [29] in some extremely challenging situations that involvesevere illumination, view point changes and cluttered background.

it emphasizes the effect of feature distinctiveness for robustimage matching.

For evaluation, we use the same publicly availabledataset as [18]. The database consists of around 8.8 thou-sand images (each associated with a location label). Twosets of testing images are used, one has rich and distinctivevisual structures, called the “clean set”, the other “confus-ing set” is composed of some much more challenging im-ages that captured more detailed parts of the scene, or ob-jects that are common across images, such as doors, posters,etc. Most of the features in those images are ambiguoustherefore techniques that do not emphasize on distinctive-ness performs poorly. Both testsets are composed of 80 im-ages with location ground-truth. For each testing image,the 8 most similar pre-captured images are retrieved. A ma-jority vote scheme is used for location prediction and theperformance is measured the same way as [18].

The provided dataset has 6.4 million SIFT features ex-tracted with the HESAFF [27] region detector. Throughcross validation on a hundred of images from the database,we set the intrinsic dimensionality n′ = 6 and the distinc-tiveness threshold 0.9, which reduced the number of fea-tures down to 4.2M. The baseline for comparison here arethe two algorithms developed in [18].

We measure the performance with precision-recall andmean average precisions (mAP)(Table 2). On the clean-set, there was no surprise that all four algorithms reachedalmost perfect performance. On the much more challeng-ing confusing-set, our approach significantly outperform

the Nister and Stewenius algorithm (N.S.) [29] and the R.S.algorithm proposed by [18]. Some qualitative comparisonsare illustrated in Figure 7, notice that our proposed approachperforms much better than the baseline algorithms in theconfusing environment and the extreme cases where veryfew non-ambiguous features are available.

4.3. Scaling up image search with distinctive fea-tures

In addition to the performance on the small scale bench-mark dataset, one important performance measurement ofan image search algorithm is its scalability to large scaledatasets. We evaluate the performance of our approach onthe UKBench+Flickr dataset. In this evaluation, images inthe original UKBench dataset are used as query. The searchscore is measured the same way using the top 4 returns asmentioned earlier. A retrieval is correct if the retrieved im-age is from the original UKBench dataset and contains thesame object as in the query image.

In this experiment, we use a fixed visual vocabulary thathas been generated using the approach proposed in this pa-per, and we focus on the scalability of our approach andevaluate the added benefit of the distinctive feature selectionapproach in large scale applications. Two algorithms are un-der comparison, one is the standard bag-of-words algorithmthat uses all the features for indexing and retrieving (Stan-dard BOW); the other uses the proposed distinctive featureselection to select and quantize only a part of the features(Distinctive BOW).

query rank 1 rank 2 rank 3 rank 4 rank 5 rank 6 rank 7 rank 8

R.S

.O

urs

Figure 7. Some qualitative examples comparing our approach (Distinctive BOW) and the R.S. algorithm [18] on the localization task.Our approach significantly outperforms these baseline algorithms in extremely confusing environment, where very few visual features areavailable. (Red boxes indicate incorrectly matched images.)

(a) (b)

Figure 8. Performance evaluation on scaling up the object instancerecognition to large scale dataset (UKBench+Flickr). In the exper-iment, we vary the size of the distraction Flickr dataset and moni-tor the changes in the recognition performance when the databasescales up. The two measurements are, (a), the absolute value ofrecognition accuracy; and (b), the recognition score decreasingrate with respect to sizes of the database.

For this experiment, we extract the HESAFF SIFT fea-tures from the UKBench+Flickr dataset using a publiclyavailable package [27]. Using the default parameters, theHESAFF SIFT feature extractor generates on average 2000features per image, and in total about 20M for the UK-Bench dataset. We randomly sampled 10M of them forthe distinctive visual vocabulary generation. We use thesame parameters as we used in the previous experiment, i.e.,n′V = 25, n′

I = 25 and vocabulary size 1M .In addition to the absolute value of the object instance

recognition score (Figure 8(a)), a better measurement forscalability is the changing rate of the recognition scoreswith respect to the database size. We measure this as the de-crease of the recognition score for every unit number (1M )of distraction images that are added, i.e., the slope of thecurve in Figure 8(a). Figure 8(b) shows the score decreas-ing rate when more and more distraction images are addedto the database.

The changing rate is high at the beginning when new dis-traction images are added and then it starts to decrease. Thisphenomenon is due to the way that we measure the recog-nition rate, i.e., all the images from the Flickr distractiondataset are considered equally. Therefore, newly added dis-traction images do not add a significant distraction effect tothe ones that are already in the database.

The ratio between the score decreasing rates of the two

approaches under comparison is 0.7 ∼ 0.9, i.e., applyingfeature selection is 10% ∼ 30% better in scalability com-pared to the standard bag-of-words approach, which is ben-efited directly from the selection of distinctive features forindexing and retrieving since the same distinctive visual vo-cabulary is used in both approaches.

5. Conclusions and future workIn this paper, we explored an approach for image match-

ing that builds on the distinctiveness of high dimensionalfeatures, reflected in their relationship with their nearestneighbors. This approach compares favorably with the stateof the art in image matching tasks such as the Universityof Kentucky Benchmark dataset and an indoor localizationdataset, also our approach scales up more gracefully on alarge scale Flickr dataset

There are several directions that remain to be explored.First, the distinctiveness we rely on right now assumes a sin-gle intrinsic dimensionality across the dataset, without fullytaking advantage of the non-uniformity property of high di-mensional space. Second, we would also like to evaluate thegeneralizability of our visual vocabulary to other datasets.

6. AcknowledgementsThe authors would like to thank anonymous reviewers

for helpful and constructive suggestions. This material isbased upon work partially supported by the National Sci-ence Foundation under Grant No. EEC-0540865.

References[1] S. Arya and D. M. Mount. Approximate nearest neigh-

bor queries in fixed dimensions. In SODA ’93, 1993.1

[2] R. Baeza-Yates and B. Ribeiro-Neto. Modern Infor-mation Retrieval. Addison Wesley, May 1999. 4

[3] R. Bellman. Adaptive Control Processes: A GuidedTour. Princeton University Press, 1961. 1, 2

[4] O. Chum, J. Philbin, and A. Zisserman. Near duplicateimage detection: min-hash and tf-idf weighting. InBMVC, 2008. 1

[5] C. Faloutsos. Searching Multimedia Databases byContent. 1996. 4

[6] C. Faloutsos and I. Kamel. Beyond uniformity andindependence: Analysis of r-trees using the conceptof fractal dimension. In PODS. ACM Press, 1994. 2

[7] R. Fergus, P. Perona, and A. Zisserman. Weakly su-pervised scale-invariant learning of models for visualrecognition. Int. J. Comput. Vision, 2007. 2

[8] I. K. Fodor. A survey of dimension reduction tech-niques. LLNL technical report, 2002. 2

[9] A. Frome, Y. Singer, and J. Malik. Image retrieval andclassification using local distance functions. In NIPS.2007. 2

[10] A. Frome, Y. Singer, F. Sha, and J. Malik. Learningglobally-consistent local distance functions for shape-based image retrieval and classification. In ICCV,2007. 2

[11] K. Fukunaga. Introduction to statistical pattern recog-nition (2nd ed.). 1990. 1, 2, 3, 4

[12] A. Gersho and R. M. Gray. Vector quantization andsignal compression. Kluwer Academic Publishers,Norwell, MA, USA, 1991. 4

[13] J. Hays and A. A. Efros. Scene completion using mil-lions of photographs. SIGGRAPH, 2007. 1

[14] P. Jain, B. Kulis, and K. Grauman. Fast image searchfor learned metrics. In CVPR, 2008. 1

[15] H. Jegou, M. Douze, and C. Schmid. Hamming em-bedding and weak geometric consistency for largescale image search. In European Conference on Com-puter Vision, 2008. 1

[16] H. Jegou, C. Schmid, H. Harzallah, and J. Verbeek.Accurate image search using the contextual dissimi-larity measure. PAMI, 2009. 1, 5

[17] F. Jurie and B. Triggs. Creating efficient codebooksfor visual recognition. In ICCV ’05, 2005. 2

[18] H. Kang, A. A. Efros, M. Hebert, and T. Kanade. Im-age matching in large scale indoor environment. InCVPR: Workshop on Egocentric Vision, 2009. 2, 5, 6,7

[19] N. Katayama and S. Satoh. Distinctiveness-sensitivenearest neighbor search for efficient similarity re-trieval of multimedia information. In ICDE, 2001. 1,2, 3, 4

[20] Y. Ke and R. Sukthankar. Pca-sift: A more distinc-tive representation for local image descriptors. CVPR,2004. 2

[21] K. Kerdprasop, N. Kerdprasop, and P. Sattayatham.Weighted k-means for density-biased clustering. In InData Warehousing and Knowledge Discovery, pages488–497, 2005. 4

[22] G. Kim, C. Faloutsos, , and M. Hebert. Unsupervisedmodeling of object categories using link analysis tech-niques. In CVPR, 2008. 2

[23] S. Lazebnik and M. Raginsky. Supervised learningof quantizer codebooks by information loss minimiza-tion. IEEE Trans. PAMI, 2009. 2

[24] F. Li. Probabilistic location recognition using reducedfeature set. In ICRA, 2006. 2

[25] D. G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 2004. 1, 2, 5

[26] J. Matas, O. Chum, M. Urban, and T. Pajdla. Robustwide-baseline stereo from maximally stable extremalregions. Image and Vision Computing, 2004. 1, 5

[27] K. Mikolajczyk and C. Schmid. Scale & affine invari-ant interest point detectors. IJCV, (1), 04. 1, 6, 7

[28] M. Muja and D. G. Lowe. Fast approximate nearestneighbors with automatic algorithm configuration. InVISSAPP (1), 2009. 1

[29] D. Nister and H. Stewenius. Scalable recognition witha vocabulary tree. In CVPR, 2006. 1, 2, 3, 4, 5, 6

[30] J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisser-man. Lost in quantization: Improving particular objectretrieval in large scale image databases. In CVPR08.1

[31] J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisser-man. Object retrieval with large vocabularies and fastspatial matching. In CVPR, 2007. 1, 5

[32] G. Schindler, M. Brown, and R. Szeliski. City-scalelocation recognition. CVPR, 2007. 2

[33] J. Sivic and A. Zisserman. Video Google: A textretrieval approach to object matching in videos. InICCV, 2003. 1, 3, 4

[34] A. Torralba, R. Fergus, and W. T. Freeman. 80 milliontiny images: A large data set for nonparametric objectand scene recognition. PAMI, 2008. 1

[35] M. Vidal-Naquet and S. Ullman. Object recognitionwith informative features and linear classification. InICCV ’03, 2003. 2

[36] Z. Wu, Q. Ke, M. Isard, and J. Sun. Bundling featuresfor large scale partial-duplicate web image search.2009. 1

[37] H. Zhang, A. C. Berg, M. Maire, and J. Malik. Svm-knn: Discriminative nearest neighbor classification forvisual category recognition. In CVPR, 2006. 2

[38] S. Zhang, Q. Tian, G. Hua, Q. Huang, and S. Li. De-scriptive visual words and visual phrases for image ap-plications. In MM ’09. ACM, 2009. 2