Towards a High Dimensional Data Management System

Dong Deng† Lei Cao‡ Jiachen Liu? Runhui Wang†

†Rutgers University ‡MIT ?University of Michigan

dong.deng@rutgers.edu lcao@csail.mit.eduamberljc@umich.edu rw545@scarletmail.rutgers.edu

ABSTRACTHigh dimensional data is ubiquitous nowadays and plays animportant role in many artificial intelligence (AI) applicationssuch as face recognition, image retrieval, and knowledge baseconstruction. The semantics of images, video, documents,and knowledge can be captured by meaningful high dimen-sional feature vectors extracted from deep learning models.Though there are many research in querying and indexinghigh dimensional data, most of them are point solutions.The high dimensional data, such as feature vectors, is stilllargely managed by the application developers individually.In this paper, we propose to manage the high dimensionaldata in a systematical way and present the design of Saber,an end-to-end high dimensional data management system.Saber features scalability, high performance, and ease ofuse and configure. It consists of several modules, includingdata ingestion, storage management, index management, andpolystore query processing. We aim at integrating Saberwith the mature AI ecosystem and helping AI practitionersreduce the burden of managing high dimensional data.

1. INTRODUCTIONHigh-dimensional data is ubiquitous nowadays and plays an

important role in many artificial intelligence applications suchas image retrieval [2], face recognition [4], and knowledge baseconstruction [9]. The semantics of images, video, documents,and knowledge graphs (KG) can be captured by meaningfulhigh dimensional feature vectors extracted from deep learningmodels [13]. Semantic related objects, such as synonyms andedges of the same type/relationship in a KG tend to havesimilar feature vectors [9]. Similarity search (a.k.a., nearestneighbor search) is one of the most important query types onhigh dimensional data, which has been extensively studiedin the last twenty years [1, 10, 13–15,22]. Various techniqueshave been developed, such as the locality sensitive hashing(LSH) [10,14], the product quantization [1,11], the proximitygraph [7,15], the sketch-based methods [12], etc. However,most of them are point solutions and nowadays the high

This article is published under a Creative Commons Attribution License(http://creativecommons.org/licenses/by/3.0/), which permits distributionand reproduction in any medium as well allowing derivative works, pro-vided that you attribute the original work to the author(s) and CIDR 2019.9th Biennial Conference on Innovative Data Systems Research (CIDR ‘19)January 13-16, 2019 , Asilomar, California, USA.

dimensional data is still largely managed by applicationdevelopers individually. Without careful implementation,querying high dimensional data can be extremely inefficientand miss the high performance requirement in many real timeapplications. Thus it is necessary to build an end-to-end highdimensional data management system to ease the burden ofapplication developers, which is exactly the purpose of thispaper.

High dimensional data has the following characteristics.

• First, high dimensional data is big and grows fast.For example, Youtube-8M1, one of the largest pub-licly available datasets today, contains 1.9 billion 1024-dimensional feature vectors extracted from 350,000hours of video using the Inception network. In compar-ison, as of May 2019, 720,000 hours of new video wereuploaded to YouTube per day2. The data is indeed inbig volume. As another example, BIGANN3, anotherlarge publicly available dataset, contains 1 billion 128-dimensional Scale-Invariant Feature Transform (SIFT)feature vectors extracted from 1 million images, whileInstagram has more than 34 billion photos4. A coarseestimation gives more than 34 trillion SIFT featurevectors and many petabytes storage in raw. Thus it iscritical for our system to have good scalability.

• Second, there are many kinds of high dimensional datawith different sparsity, number of dimensionality, andvalue domain. For example, the high dimensional fea-ture vectors extracted by deep learning techniques areusually dense, in real values, and ranges from tensto thousands of dimensions. In contrast, the TF-IDF weighed documents in information retrieval aresparse vectors and in hundreds of thousands of dimen-sions to millions of dimensions. Their value domainis positive value. In recommendation systems, suchas the friend relationship in social networks and thepurchase/rate/click history, data can be representedas ultra high dimensional (in hundreds of millions di-mensions to billions of dimensions), extremely sparse,binary vectors. In addition, various metrics are de-signed to measure the similarity/distance between highdimensional data. Among them, Euclidean distance(Lp-Norm in general), cosine similarity, inner product,

1https://research.google.com/youtube8m/2https://www.statista.com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute/3http://corpus-texmex.irisa.fr/4https://www.statisticbrain.com/instagram-company-statistics/

and Jaccard similarity are the most widely used ones.Note that, the cosine similarity can be inferred fromthe Euclidean distance by normalizing all vectors tounit vectors.

• Third, the high dimensional data is usually coupledwith other metadata information. For example, inonline e-commercial websites, feature vectors are ex-tracted from product images, while these products alsohave other structured information like names, price, rat-ings, and inventory, as well as unstructured informationlike descriptions and reviews.

Based on the characteristic of high dimensional data, wedesign Saber, an end-to-end high dimensional data manage-ment system, which features scalability, high performance,and ease of use and configure. It consists of data inges-tion, storage management, index management, and polystorequery processing components. The data ingestion moduleextracts high dimensional feature vectors from the raw inputdata. The high dimensional data, along with their metadata,are stored in multiple different data stores. The polystorequery processing component coordinates these data storesto process multi-modal queries [17]. Saber uses scalableand ease to configure indexes. An index is designed for thedense vectors. Saber also supports data and index updates.Saber is designed to support the following three query typesfor the high dimensional data.

1. Similarity Search. This is the same as the nearest neigh-bor (NN) search query. Specifically, given a collectionof high dimensional vectors, a distance (or similarity)metric, a query vector, and an integer k, similaritysearch returns k vectors whose distance to the queryis smallest (or whose similarity to the query is largest)under the given metric. A commonly used variant isthe range similarity search query where a threshold isgiven instead of the integer k and all the vectors whosedistance to the query is smaller than the threshold(or whose similarity to the query is larger than thethreshold) are returned.

2. Similarity Join. This is the analogy to the join query inSQL. Given two collections of vectors (or one collectionin the self-join case), similarity join query returns allthe similar vector pairs under a given metric. Thesame criteria as the similarity search query is used tojudge whether two vectors are similar. This can beused to join two collections of objects (e.g., images anddocuments).

3. Similarity GroupBy. This is the analogy to the groupbyquery in SQL. Intuitively, this query deduplicates thehigh dimensional vectors and returns clusters of sim-ilar/duplicate vectors. This is useful when the userswant to deduplicate or aggregate the high dimensionaldata (e.g., removing the duplicate products in productsearch). Formally, given a collection of vectors, similar-ity groupby query returns clusters of vectors such thatin any two clusters, none of their vectors are similar.This definition guarantees the transitivity closure: if Xand Y are duplicate and Y and Z are duplicate, X areZ must also be duplicates. For ease of understand, wecan build a graph where each vertex corresponds to avector and there is an edge if and only if two vectors on

PolystoreSystem

PreprocessingPQIndex

Storage

DataIngestion

Metadata+Vectors

RPC

TileDB

PostgreSQL

ElasticSearch

File System

InvertedIndex

QueryProcessingIndexManagement

RPC

RPC

RPC

QueryResults

Data&Query

Figure 1: System architecture

the two ends are similar. Then each cluster correspondsto a connected component in this graph.

Saber deals with sparse dense, real-value high dimen-sional vectors and sparse, positive-value ultra high dimen-sional vectors. These are two most interesting types of highdimensional data. Many feature vectors (especially thoseextracted from deep learning models) are dense, real-valuevectors while TF-IDF weighted documents (and ratings inrecommendation systems) are sparse, positive-value vectors.For dense vectors, Euclidian distance and cosine similarityare supported, while Jaccard similarity and cosine similarityare supported for sparse data. We expect to support moremetrics and data types in the future.

The rest of the paper is organized as follows. In Section 2,we present the architecture of Saber. Section 3 introducesthe data ingestion module and Section 4 discuss our novelindex structure for dense vectors. We discuss the ploys-tore query processing in Section 5, review related work inSection 6, and conclude the paper in Section 7.

2. SYSTEM ARCHITECTUREThe system architecture is shown in Figure 1. It consists

of the following components.

Data Ingestion. The application developers use RemoteProcedure Call (RPC) services to interact with the system,such as the Google gRPC and the Apache Thrift. The dataand query (e.g., images, documents, video, knowledge graphs)are first processed by a data ingestion module, which wediscuss in details in Section 3. This data ingestion module isresponsible for extracting high dimensional feature vectorsas well as the corresponding metadata (e.g., the locationand timestamp in a photo and the object class of an imagedetermined by a classifier) from the input. We also acceptplain high dimensional vectors as the input data and query.

Storage Management. The metadata and vectors gener-ated from the data ingestion module are forwarded to thestorage module. As discussed before, the high dimensionaldata is usually coupled with other structured and unstruc-tured metadata. Saber utilizes multiple systems to store thehigh dimensional vectors and the metadata. For example,Saber stores the structured data in the relational databasemanagement system (RDBMS) like PostgreSQL while thesemi-structured and unstructured data are stored in the full-text search engine ElasticSearch. For the raw multimediadata, a file system is employed to store them. Saber uses

Figure 2: Illustration of the multi-modal query [17].

TileDB to store the dense and sparse vectors. These sys-tems provide fast, reliable, and scalable data read and writeservices.

Index Management for High Dimensional Data. Forthe structured, semi-structured, and unstructured data, theircorresponding data stores provide powerful indexing mecha-nism and Saber use them as it is. For the high dimensionalvectors, Saber builds and manages its own index. Manyfeature vectors (especially those extracted from deep learningmodels) are dense, real-value vectors. We leverage productquantization to compress the original dense vectors and buildnovel (minimum spanning) tree index for the compresseddata. We also design strategies to deal with data and indexupdates. The details will be given in Section 4.

For the sparse, positive-value vectors, we leverage theinverted index and the filter-and-refine framework [5], suchas the prefix filtering [3], to process queries. A huge advantageof these techniques is the ease of use and configure. Usuallythere is few to no hyper-parameters to tune. These aremature techniques and we leave out the details.

Polystore Query Processing. Our system has a poly-store query processing module to coordinate the multi-modalquery against the data in multiple stores. Multi-modal queryinvolves data of different types, which is very common inthe real world. Figure 2 shows product search examples inthe work [17] from WalmartLab. We can issue the similar-ity search query over the high dimensional feature vectorsextracted from the product photos in conjunction with thekeyword search query over the textual data. As anotherexample, surveillance applications usually only interests inobjects (such as people and cars) that appear in a specificregion or within a specific period of time. It can be expressedas a multi-modal query across the structured data and thehigh dimensional feature vectors. We discuss the details ofpolystore query processing in Section 5.

3. FAST DATA INGESTIONThe system provides a rich set of tools to facilitate the

users to ingest the input data in various types of formatsinto high dimensional vectors that effectively represent thekey features of the raw input data. Based on the types of theinput that users can provide, the tools can be categorizedinto the following types:

• Direct data loading. If the input data is already in the

format of high dimensional feature vectors, then oursystem simply efficiently loads the data into data tableleveraging the existing tools ingesting structured data.

• Extract features from supervised model. If the inputdata is in complex format, such as image or video,simply treating each raw image file/video frame as ahigh dimensional feature vector is not effective in sup-porting similarity search. This is because even a smallrotation of image will dramatically change its represen-tation. However, when the users are able to provide aConvolutional Neural Network (CNN) that effectivelyclassifies the given image datasets/videos, our systemfully leverages the representation learning capability ofCNN and provides tools to extract features from theneural network. Instead of directly extracting featuresfrom a single layer close to the output, we design amechanism that aggregates the states of different layersto better represent the image.

• Semi-supervised feature extraction. However, usersare not always able to provide a well trained classifica-tion model due to the lack of abundant training data,etc. Therefore, we design a semi-supervised featureextraction method that uses a small set of examples tolearn the high dimensional representations of the rawimages/videos. The key idea is to leverage the observa-tion that Generative Adversarial Networks (GAN) caneffectively capture the data manifold during the battleof data generator and data discriminator [8]. Besidesrequiring the GAN to separate the fake examples andreal examples, our semi-supervised method also usesthe small set of examples to guide the GAN to learna representation that can separate different classes ofreal data, therefore more effective in representing theraw input.

• Unsupervised feature extraction. In the case that theusers cannot provide any labeled example, we providemultiple unsupervised tools to extract features fromthe raw data. These tools either leverage the manifoldlearning capacity of GAN similar to the semi-supervisedtool, or use Autoencoder that is shown to be effectivein learning representation without relying on any label.

• Feature extraction for text data. Besides the image/videodata, our system also provides tools to effective ex-tract features from text data including TF-IDF andword2vector [16], etc.

Leveraging the progress of AutoML, these tools save usersthe efforts of tuning parameters. In most of cases, the usersonly need to specify the dimension of the feature vector theyprefer. For the structured and unstructured metadata, weuse existing tools and the corresponding data store in thesystem to digest them.

4. INDEX FOR DENSE VECTORSIn this section, we present our novel index for the dense,

real-value high dimensional vectors. We first introduce thelimitations of existing approaches in Section 4.1. Section 4.2presents our novel tree-based index and Section 4.3 discusseshow to support distributed query processing. We designstrategies to deal with data and index updates in Section 4.4.

Part 1 · · · Part MCentroid 1 0.45 · · · 1.24· · · · · · · · · · · ·Centroid k 0.88 · · · 0.82

Table 1: An example distance lookup table

(8,6,10,23,1,39,28,65)

(8,6,10,56,1,39,28,65)

PQcodeA:

PQcodeB:

Distance(Q,B)=Distance(Q,A)+Table[23][4]- Table[56][4]

Distance(Q,A)=Table[8][1]+Table[6][2]+…+Table[64][8]

Figure 3: Sharing computation between similar PQ codes.

4.1 Limitation of Existing ApproachesSimilarity queries, including similarity search (a.k.a. near-

est neighbor search), similarity join, and similarity groupby,play an important role in managing high dimensional dataand have enormous applications. However, it is rather chal-lenge to process similarity queries at large scale and existingapproaches would all fail. Specifically, it is well known thatexact methods that guarantee to return completely accurateresults do not competitive with the brute-force linear scan ofthe entire dataset due to the “curse of dimensionality” [10].For example, the R-tree and K-D tree index become evenslower than the simple linear scan when data dimensional-ity is larger than 10 [18]. Existing approximate algorithms(including Locality Sensitive Hashing (LSH) based, proximity-graph based, and vector quantization based methods) canscale to higher dimension. However, they have problemswhen the data cardinality is at large scale.

LSH based methods. In general, LSH based methodsemploy many hash functions in a specific hash family topartition high dimensional data such that nearby/similardata points are more likely to be hashed to the same bucketthan faraway/dissimilar data points [10, 22]. However, toachieve high accuracy, LSH based methods usually involveexcessively large number of hash tables and incur huge indexsize and suboptimal efficiency.

Proximity-graph based methods. In proximity graphs,each vertex corresponds to a data point. Nearby data pointsare connected such that for any data point, one of its neigh-bors must be closer to the query than the data point itself [15].Then the best first search (such as the A* search) on theproximity graph can find the nearest neighbor to a query.However, building such a graph is very expensive and storingthe graph needs large space.

Another noticeable disadvantage of proximity-graph basedmethods and LSH based methods is that they both needto access the original data vectors to calculate their realdistance to the query. This incurs many expensive randomIOs and/or network communication (in distributed queryprocessing).

Quantization based methods. Vector quantization clus-ters the data vectors into k clusters (using k-means for ex-ample) and approximate each vector by its nearest centroid.Product quantization evenly divides the D dimensional space

A

B

C

D

<4, 23, 56>

E

A

B

C

D

<4, 23, 56>

E

A

B

C

D

<4, 23, 56>

E

Q

Figure 4: A minimum spanning tree generated from the fullgraph

into M disjoint D/M dimensional subspaces and applies vec-tor quantization to every subspace independently [11]. Theneach vector is compressed to a Mlog k bit code to indicate theM corresponding nearest centroids in the M subspaces. ForD=128, M=16, and k=64, the compression rate is 64. Moreimportantly, the distance to the query can be estimated usingonly the code; i.e., the original vectors are never accessedin query processing. However, existing product-quantizationbased methods need to exhaustively scan all the codes orsearch in an exponentially large space. This is too expensivefor large scale dataset where even the codes cannot fit inmemory.

4.2 Tree-based Index and Query ProcessingThe query processing engine for large scale high dimen-

sional data in Saber is built based on a novel tree indexstructure. In a high level, we build a (wide-and-flat) treeindex for all the product quantization codes. By traversingthe tree, we can find the nearest neighbors to the query. Thecomputation will be shared between all parent and childnodes. Moreover, for some tree nodes that satisfy certainconditions, we can skip traversing the entire subtrees rootedat these nodes. Same as other tree indexes like B-tree, we canstore the lower level nodes on disk or in different machines forlarge scale dataset. The pruned branches will never be visited.Thus, random IO accesses and/or network communicationin distributed computing setting will be reduced.

Sharing computation in similar PQ codes. More specif-ically, as shown in Figure 1(a), PQ-based methods pre-compute a lookup table containing the distances from allcentroids to the query in the M subspaces. The code of avector contains the indexes of the nearest centroids in the Msubspace. Then the distance from a vector to the query canbe estimated using M lookups. For example, in Figure 1(b),the distance from a data vector A to the query Q can beestimated by adding up the M=8 table lookups. We observethat the distance computation between “similar” codes canbe shared. For example, as shown in Figure 1(b), the codesA and B only differ in the 4-th part. Thus Distance(Q, B)can be calculated by removing the differences in the 4-th partfrom the previously computed Distance(Q, A). In this way,only two lookups are needed instead of M=8 to calculateDistance(Q, B).

The minimum spanning tree index. To share morecomputation, we can (virtually) build a full graph for all thecodes, where the edge weight is the number of differencesbetween two codes. For example, as shown in Figure 2(a),the edge between codes A and B has a weight of one as they

only differ in the 4-th part. Then the minimum spanningtree generated from the graph, as shown in Figure 2(b), hasthe minimum total number of differences. By traversing thistree and using the differences to calculate distances, we canget the nearest neighbor to the query.

Pruning computation by filtering subtrees. We ob-serve that some subtrees can be skipped during tree travers-ing. Specifically, for any node X in the subtree rooted atnode A, based on the triangle inequality

Distance(Q,X) ≥ Distance(Q,A) − sdDistance(A,X)

holds. Note Distance(A, X) is independent to the queryQ and can be pre-computed in the offline indexing phase.Moreover, when A is visited during traversing, Distance(Q,A) is calculated. Thus we have a lower bound of the distancefrom any node in the subtree rooted at A to the query. Ifthis lower bound is no smaller than the threshold (i.e., thedistance from the query to the nearest neighbor visited sofar), the entire subtree rooted at A can be safely pruned asit cannot yield any node closer to the query than the currentnearest one.

Similarity join and similarity group query. At thispoint, Saber simply decomposes the similarity join andsimilarity groupby queries into a bunch of similarity searchqueries. In the future, we will design more sophisticatedalgorithms to process these queries more efficiently.

Same as the inverted index, product quantization has fewto no hyper parameters to tune and is ease to use and con-figure. In contrast, the LSH-based methods and proximity-graph based methods involves multiple hyper parametersand their performance is sensitive to those parameters.

4.3 Distributed Similarity Query ProcessingTo further improve the scalability of similarity query pro-

cessing, we propose to support distributed query processing.A simple solution evenly distributes the data to all the ma-chines. A coordinate machine is assigned to each similaritysearch query. The query is processed in each individual ma-chine and the results are returned to and aggregated by thecoordinate machine.Saber supports another alternative solution which parti-

tions the tree index. Specifically, we first determine a fewinner nodes as the boundary nodes. All the ancestor nodesof these boundary nodes consist a tree, which we name asthe filter tree. The filter tree is copied to every machine.The subtrees rooted at these inner nodes are partitionedand evenly distributed to all the machines. Each query israndomly assigned a coordinate machine. The query is firstprocessed by the filter tree to prune unqualified subtrees asdiscussed before. Then the query is sent to all those machineswith at least one qualified subtree. Finally, the results arereturned to and aggregated by the coordinate machine.

4.4 Data and Index UpdateDealing with continuously growing large scale datasets has

many applications. For example, the news searching systemthat updates on hourly/daily basis, recommendation systemsbased on users’ most recent behavior, and object detection invideo surveillance. However, existing methods for similaritysearch are designed for static datasets, which lead to highcomputational cost for adjusting indexes to accommodatenewly arrived data.

��������

��������

��������� �

��������� �

��

��

��

����

��

�� �� �� ��

�� ��

������� ������������� ��� ������

Figure 5: Hierarchical clustering for data and index updates.

Hierarchical clustering. To support efficient and accu-rate index updates in Saber, we leverage the hierarchicalclustering (hierarchical k-means more specifically) for vectorquantization. Hierarchical k-means decomposition representsan input dataset as a tree. The root of the tree points to thecentroid of the whole dataset. The decomposition processiteratively divides the dataset into k sub-clusters using k-means. Each sub-cluster is then associated to a child node ofthe current tree node. The decomposition process continuesto the sub-clusters until the size of each sub-cluster is smallenough.

Index updating. Index updating is achieved by mergingand splitting specific sub-clusters whenever necessary. In ahigh level, we merge two clusters when deleting data andsplit clusters when inserting data if certain condition is met.When merging clusters, we directly merge two clusters thatshare the same parent tree node and update the centroid.When splitting a cluster, we create two child nodes under thespecific tree node, split the specific cluster into two clusters,and calculate the centroids.

If the number of points in a cluster is below certain thresh-old, we merge it with its neighbor. In contrast, if the varianceof one cluster exceeds some threshold and the number ofpoints exceeds certain number, we perform split operationsince the quality of the cluster is low.

For example, as shown in Figure 5, the space is originallysplit into four clusters. After inserting some new points (inred color), since the variance inside cluster C3 barely changes,the corresponding index remains untouched. In contrast, thevariance inside cluster C2 hugely changed. Thus we split thiscluster and update the index for both old and new data.

5. POLYSTORE QUERY PROCESSINGTo support multi-modal query, our system uses consistent

IDs across different data stores, i.e., the same object has anunique ID for information stored in different places. Thequery is first decomposed into several sub-queries where eachsub-query involves data only in a single store. Then thesesub-queries are processed by their corresponding data stores.Next, the results of the sub-queries are aggregated. Eachresult corresponds to a set of object IDs. The results are in-tersected for conjunction queries and unioned for disjunctionqueries. Finally, the raw data corresponds to the object IDsin the final result set is returned from the file system withthose projected attributions.

Joint Query Optimization. For disjunction multi-modalqueries like “find similar objects within a specific period of

time”, it may not be optimal to process all the sub-queriessimultaneously and aggregate the results. This is becausethe similarity search sub-query on high dimensional data istime-consuming and usually becomes the bottleneck. Thusit may be much more efficient if the other sub-queries firstefficiently identify a small set of objects as candidates andthen the similarity search sub-query processes the smallset of candidates right after. Based on this observation,Saber designs a joint query optimization module. It uses acost model to estimate the cost of different sub-queries anddetermines the optimal execution order of the sub-queries.

Development Plan. We are still in the initial stage ofsystem implementation. The plan is to get a prototype readyby next January. If this paper is accepted, we will demoSaber during the conference presentation.

6. RELATED WORKBigDAWG [6] is a polystore system. Silva et al. [19] propose

SimDB to support similarity query processing in relationalDBMS. Mu et al. [17] propose to index high dimensional datain ElasticSearch so as to support multi-modal query. How-ever, all these systems have poor performance and scalabilityin processing high dimensional data. Silva et al. [20] andTang et al. [21] give several similarity groupby definitions onnumerical and multi-dimensional data, which is different fromour similarity groupby definition on high dimensional data.Xu et al. [23] proposed an index structure that can handleindex updates. However, it only updates the index of thenewly arrived data and has constraints on the distributionof new data to guarantee the accuracy of the updated index,which is unstable with large change of data distribution.

7. CONCLUSIONIn this paper, we present the design of our end-to-end high

dimensional data management system Saber. It consists ofdata ingestion, storage management, index management, andpolystore query processing components. The data ingestionmodule extracts high dimensional feature vectors from theraw input data. The high dimensional data, along withtheir metadata, are stored in multiple different data stores.The polystore query processing component coordinates thesedata stores to process the multi-modal query. We quantizethe dense high dimensional vectors into codes using productquantization and index the codes using a tree index structure.For the sparse high dimensional vectors, we employ theinverted index structure and prefix filtering techniques toprocess queries. We also design strategies to deal with dataand index updates.

8. REFERENCES[1] F. Andre, A. Kermarrec, and N. L. Scouarnec. Cache

locality is not enough: High-performance nearestneighbor search with product quantization fast scan.PVLDB, 9(4):288–299, 2015.

[2] A. Babenko, A. Slesarev, A. Chigorin, and V. S.Lempitsky. Neural codes for image retrieval. In ECCV,pages 584–599, 2014.

[3] R. J. Bayardo, Y. Ma, and R. Srikant. Scaling up allpairs similarity search. In WWW, pages 131–140, 2007.

[4] Q. Cao, Y. Ying, and P. Li. Similarity metric learningfor face recognition. In ICCV, pages 2408–2415, 2013.

[5] D. Deng, G. Li, H. Wen, and J. Feng. An efficientpartition based method for exact set similarity joins.PVLDB, 9(4):360–371, 2015.

[6] J. Duggan, A. J. Elmore, M. Stonebraker,M. Balazinska, B. Howe, J. Kepner, S. Madden,D. Maier, T. Mattson, and S. B. Zdonik. The bigdawgpolystore system. SIGMOD Record, 44(2):11–16, 2015.

[7] C. Fu, C. Xiang, C. Wang, and D. Cai. Fastapproximate nearest neighbor search with thenavigating spreading-out graph. PVLDB,12(5):461–474, 2019.

[8] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu,D. Warde-Farley, S. Ozair, A. C. Courville, andY. Bengio. Generative adversarial nets. In NIPS, pages2672–2680, 2014.

[9] A. Grover and J. Leskovec. node2vec: Scalable featurelearning for networks. In SIGKDD, pages 855–864,2016.

[10] P. Indyk and R. Motwani. Approximate nearestneighbors: Towards removing the curse ofdimensionality. In STOC, pages 604–613, 1998.

[11] H. Jegou, M. Douze, and C. Schmid. Productquantization for nearest neighbor search. IEEE Trans.Pattern Anal. Mach. Intell., 33(1):117–128, 2011.

[12] P. Li, A. B. Owen, and C. Zhang. One permutationhashing. In NIPS, pages 3122–3130, 2012.

[13] Y. Liu, H. Cheng, and J. Cui. PQBF: i/o-efficientapproximate nearest neighbor search by productquantization. In CIKM, pages 667–676, 2017.

[14] Q. Lv, W. Josephson, Z. Wang, M. Charikar, and K. Li.Multi-probe LSH: efficient indexing forhigh-dimensional similarity search. In VLDB, pages950–961, 2007.

[15] Y. A. Malkov and D. A. Yashunin. Efficient and robustapproximate nearest neighbor search using hierarchicalnavigable small world graphs. CoRR, abs/1603.09320,2016.

[16] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, andJ. Dean. Distributed representations of words andphrases and their compositionality. In NIPS, pages3111–3119, 2013.

[17] C. Mu, J. Zhao, G. Yang, J. Zhang, and Z. Yan.Towards practical visual search engine withinelasticsearch. In SIGIR 2018 Workshop On eCommerce,2018.

[18] H. Samet. Foundations of multidimensional and metricdata structures. Morgan Kaufmann series in datamanagement systems. Academic Press, 2006.

[19] Y. N. Silva, A. M. Aly, W. G. Aref, and P. Larson.Simdb: a similarity-aware database system. InSIGMOD, pages 1243–1246, 2010.

[20] Y. N. Silva, W. G. Aref, and M. H. Ali. Similaritygroup-by. In ICDE, pages 904–915, 2009.

[21] M. Tang, R. Y. Tahboub, W. G. Aref, M. J. Atallah,Q. M. Malluhi, M. Ouzzani, and Y. N. Silva. Similaritygroup-by operators for multi-dimensional relationaldata. In ICDE, pages 1448–1449, 2016.

[22] Y. Tao, K. Yi, C. Sheng, and P. Kalnis. Quality andefficiency in high dimensional nearest neighbor search.In SIGMOD, pages 563–576, 2009.

[23] D. Xu, I. W. Tsang, and Y. Zhang. Online productquantization. TKDE, 30(11):2185–2198, 2018.