Efficient Retrieval of Recommendationsin a Matrix Factorization Framework

Noam Koenigstein Parikshit Ram Yuval ShavittSchool of Electrical

Engineering

Tel Aviv University

Computational Science &Engineering

Georgia Institute ofTechnology

School of ElectricalEngineering

Tel Aviv University

Motivation• In the field of Recommender System, Matrix Factorization

(MF) models have shown superior accuracy for recommendation tasks.E.g., The Netflix Prize, KDD-Cup’11, etc.

• Training is fast. Computing test scores is fast.But… Retrieval of Recommendations (RoR) is s--l--o--w !

• This problem is well known in the industry, yet never been approached before in academia!

22

45

3 32

. . . .

4 12

25

I T E M S

USERS

Yahoo! Music:

1M Users625K Items

6 Tera elements ~300 multiplications ~5 days CPU

Naïve Multithreading: High latency + wasteful

Yahoo! Music:

1M Users625K Items

6 Tera elements ~300 multiplications ~5 days CPU

Reduction to Inner Product

.

.

.

1

.

.

.

�̂�𝑢𝑖=𝐩𝑢𝑇 𝐪𝑖=‖𝐩𝑢

❑‖‖𝐪𝑖‖cos𝜃𝑢𝑖

Core problem:

Given a user vector and a set of item, find an item vector that will maximize

Best Matches Algorithms

• Metric Space

• Cosine Similarity

• Locality Sensitive Hashing

Metric TreesR

R

Branch-and-bound Algorithm

Bounding Inner Product Similarity

Approximate Solution

Users vectors can be normalized Users can be clustered based on their spherical angle!

Relative Error Bound

What is the error when recommendations are retrieved based on an approximate user vector?

|𝑒𝑟𝑟𝑜𝑝𝑡|=|𝑝𝑐❑𝑇𝑞𝑖

❑−𝑝𝑢❑𝑇𝑞𝑖

❑||𝑝𝑐❑

𝑇 𝑞𝑖❑|

≤1−cos (𝜃𝑝𝑐𝑞 𝑖

+Δ )cos (𝜃𝑝𝑐𝑞 𝑖 )

Adaptive Approximate Solution

Experimentations Set-up MovieLens Netflix Yahoo!

MusicRatings 1,000,206 100,480,507 252,800,275

Users 6,040 480,189 1,000,990

Items 3,952 17,770 624,961

Sparsity 95.81% 98.82% 99.96%

Yahoo! Music Recommendations: Modeling Music Ratings with Temporal Dynamics and Item TaxonomyGideon Dror, Noam Koenigstein, Yehuda Koren(RecSys-11`)

Exact Alg. Speedup

Approximate Alg. Speedup

Speedup vs. Precision

Speedup vs. MedianRank

Conclusions

• We introduce a new and relevant research problem

• An exact solution with limited speedup

• An approximate solution with a tradeoff between accuracy and speedup

• Much room for further research…