Embed Size (px)
Citation preview
Hardware Accelerated Similarity Search
George Williams
Who Am I?
Director, GSI Technology Previously, Chief Data Scientist
Senior Data Scientist AI Research Scientist
Software Engineer
Recent Headlines
Convergence and Integration
That was then...
This is Now: Technology Disintegration
More Innovation Around The Corner
?
GSI’s Similarity Search Accelerator
Agenda
● Chip Explosion ● GSI Technology ● What is Vector Similarity Search? ● GSI’s Similarity Search Accelerator ● Integration Case Studies: Bio, Database ● Early Adopters Program
Who Is GSI Technology?
Aerospace, Government, R&D
High Performance SRAM and DRAM
GSI Vector Similarity Search Accelerator Chip
What We Do
What Is Vector Similarity Search?
What is Vector Similarity Search?
Numeric Representation
Bit-vector 0110000100
Coordinates (2.3, 5.6)
Numeric Representation
Simple “Distance” Function
d = Func (a, b)
What is Vector Similarity Search?
K = 5
K = 3Numeric Representation
Simple “Distance” Function
K Nearest Neighbor (Top-K)
What is Vector Similarity Search?
K = 5
K = 3
Numeric Representation
Simple “Distance” Functions
K Nearest Neighbor (Top-K)
Search is Computational
E-Commerce, Bioinformatics
What is Vector Similarity Search?
E-Commerce: Visual Search
Visual Search
Binary Codes, Continuous Embeddings
Euclidean, L1, Hamming, Cosine
>1 Billion Images
Visual Search: Embedding Space
Bioinformatics: Molecule Similarity
Fingerprints
Tanimoto
Many Large DBs
100s GB
Molecule Similarity: Tanimoto
Jaccard
Intersection / Union
Bioinformatics: Molecule Similarity
Drug Discovery of Novel Molecules
Virtual Screening
Activity (Toxicity) Prediction
Programming Interfaces
Idiomatic SQL
Integrate Into Data Pipelines
Leverage Skills of Data Eng & Scientists
Many Domains and ApplicationsE-Commerce / Recommendations
Bioinformatics / Genomics
Healthcare / Medical Records
Cybersecurity / Malware Detection
Computer Vision / Video Surveillance
GSI’s Similarity Search Accelerator
Computational Memory
“In-Place” Associative Processing
Bit Logic ● Programmable ● 2 million
Consumer Board Solution
16GB Memory
PCIe Card
128Mb 128Mb
2 Chips Per Board
On Board DDR4 Main Memory
SRAM Cache Per Chip
1 Chassis (4U) Solution
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
4 Boards Per Chassis
Multiple Chassis Solution
One Chassis Is The
Master
Network Attached Storage
RDMA support (NAS As Data Source)
Network
..16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
Segmentation by Clustering
Offline Clustering / K-Means
Avoids Full DB “Scan”
Faster Performance
Availability
Q2, 2019
Q1, 2019demo boards
mass production
chip Q4, 201816GB Memory
PCIe Card
128Mb 128Mb
Weizmann Institute Case Study
“GSI’s [Accelerator] can dramatically reduce the time required to search our small molecules database...”
- Dr. Efrat Ben-Zeev, Computational Chemist
Weizmann Institute Case Study
Molecule Similarity Search Biovia Pipeline Pilot Application Query of 34M Molecule DB Takes 10 Minutes !
Using GSI Accelerator (estimated) Query Latency Reduced To 300ms 400 Queries In 1 Second
Biovia Application Integration Application
Python Library
C Library
DRIVERS
GSI Accelerator
In-Memory Database Integration
Database
C Library
DRIVERS
GSI Accelerator
In-Memory: Expected Performance
Memory Speed Vector Size Throughput (imgs/sec)
MemSQL ~50GB / sec 4 KB 12.5 million images/sec
GSI ~100GB / sec 4 KB 25 million images/sec
Early Adopters Program
Consult With Our Hardware and AI Experts
Co-Development and App Integration
Access to simulator and test hardware
Co-Marketing Opportunity
GSI Upcoming Events
Nov, Open Data Science Panel, Visual Search
Nov, PyData ( Washington DC )
Dec, GSI Similarity Search Accelerator Workshop
Coming Soon, GSI’s Tech Meetup
2019, First Chips and Boards Available
Contact Us
Twitter: @gsitechnology @cgeorgewilliams
Blogs: gsitechnology.com medium.com/gsitechnology
Email: [email protected]
The End. Thanks !
Query
Option 1: INFERENCE IN PRODUCTION ● Image2Vec (VGG, Resnet) ● NLP ● LSTN
Option 2: Query Vector done by External Application ● Fingerprint Query ● In Memory Vector (Cosine)
Option 3: 3rd Party Inference Vector
Single Board
Small Database
Fit All Data Into Cache For Lowest Latency
If Larger, Paging Occurs To Memory
Cluster Techniques
16GB Memory
PCIe Card
128Mb 128Mb
Large Database
Large Database (<1TB, Flat) Pharma, Drug Search, Weizmann Molecule Search, In-Memory
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
Multi Board Solutions
For Huge Databases ( ~ 1TB )
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
For Throughput: Batch Queries Split Across Boards
Chassis Host Master Merges The Results
Offline Data Preparation
Training InferenceOptimize For Cache and
Memory
Clustering For Large DatabasesOffline Clustering
Centroids List <16GB
Reduces Storage
Only Centroids Are Kept Local
For Real-Time Performance
Huge Database
Exact Nearest Neighbors For Small Vectors
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
Approx For Large Vectors (Quantization)
Large Scale Sim Search, FAISS
Biovia Application Integration
Biovia Application Used By Thousands of Bio-Tech Companies
Weizmann: Load A Database
Weizmann: 3rd Party Search
Weizmann: Select Search Method
Weizmann: Search
Weizmann: Define Parameters
Weizmann: Run Protocol
