Upload
others
View
20
Download
7
Embed Size (px)
Citation preview
Deep Learning for Recommender Systems
Marcel Kurovski Karlsruhe, October 25th 2017
2
About Me§ Industrial Engineer (M.Sc.)§ Data Scientist at inovex§ Machine Learning – focus on Deep Learning§ Masterthesis:
Deep Learning for Recommender Systems:Joint Learning of Preference and Similarity
Marcel Kurovski
3
Agenda
1. Motivation
2. State-of-the-Art
3. VehicleRecommendationswith Deep Learning
4. ACM RecSysConference 2017
5. Discussion
4
Annual Data Sphere increases exponentially
International Data Corporation: Data Age 2025 study, April 2017
Informationà Humans
Processing Capacity
5
Information Overload
https://www.linkedin.com/pulse/its-information-overload-filter-failure-productivity-industry-zayats/
“It‘s not information overload.It‘s filter failure."
- Clay Shirky
7
Collaborative Filtering
? 1 1 1
? 1 ? ?
1 1
m
Users
n Items
3
1
2
3
1
2
2
1
3
4
1 2 3 4
8https://www.slideshare.net/MrChrisJohnson/algorithmic-music-recommendations-at-spotify/10-Collaborative_Filtering10HeyI_like_tracks_P
Collaborative Filtering
9https://buildingrecommenders.wordpress.com/2015/11/18/overview-of-recommender-algorithms-part-2/
Matrix Factorization
10
Cold Start
http://www.yusp.com/wp-content/uploads/2015/07/cold-start-problem-recommender-systems-1.jpg
11
Recommender Systems for IF
SPARSITY
12adapted from http://www.kdnuggets.com/2016/02/nine-datasets-investigating-recommender-systems.html
Sparsity Comparison
13adapted from http://www.kdnuggets.com/2016/02/nine-datasets-investigating-recommender-systems.html
Sparsity ComparisonMovieLens 1M: 4.26% MovieLens 20M: 0.53%
Last.fm: 0.28% Vehicles All: 0.0046%
14
Content-based Filtering
? 1 1 1
? 1 ? ?
1 1
m
Users
n Items
age
gender
history
mileagemodelcolor
3
1
2
3
1
2
2
1
3
4
1 2 3 4
15
“Deep Learning becomes a general-purpose solution fornearly all learning problems."
- Covington et al.
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
Google Trends forDeep Learning
16
Motivation: Deep Learning for RecSys
Information Overload
Information Filtering
RecommenderSystems
Learning Problem
Deep Learning
17
Agenda
1. Motivation
2. State-of-the-Art
3. VehicleRecommendations withDeep Learning
4. ACM RecSysConference 2017
5. Discussion
18
Recommendations are everywhere
19
„The company reported a 29% salesincrease to $12.83 billion [...]Amazon has integratedrecommendations into nearly everypart of the purchasing process fromproduct discovery to checkout.“
http://fortune.com/2012/07/30/amazons-recommendation-secret/
20
„Our recommender system is usedon most screens of the Netflixproduct beyond the homepage, andin total influences choice for about80% of hours streamed at Netflix. The remaining 20% comes fromsearch [...]“
Gomez-Uribe, Carlos A. and Hunt, Neil: The Netflix Recommender System: Algorithms, Business Value, and Innovation (2015)
Suche
EmpfehlungenRecommendations
Search
21DLRS: Deep Learning based Recommender Systems
Domains and Types for DLRS
DNNs
CNNs
RNNs
AEs
Sonst.
Sonst.
2013
2016
2017
2015
2009
2015 2015 2015
2017
2015
2016
2016
22
DNNs for Video-Recommendations (1)
23Covington, Paul, Jay Adams, and Emre Sargin: Deep neural networks for youtube recommendations (2016)
DNNs for Video-Recommendations(2)
24Covington, Paul, Jay Adams, and Emre Sargin: Deep neural networks for youtube recommendations (2016)
DNNs for Video-Recommendations(3)
Deep Candidate Generation Deep Ranking
25Cheng, Heng-Tze et al.: Wide and Deep Learning for Recommender Systems (2016)
https://research.googleblog.com/2016/06/wide-deep-learning-better-together-with.html
Wide and Deep Learning for App-Recos (1)
26
Wide and Deep Learning for App-Recos (2)
Cheng, Heng-Tze et al.: Wide and Deep Learning for Recommender Systems (2016)
https://research.googleblog.com/2016/06/wide-deep-learning-better-together-with.html
DeepComponent Wide
ComponentEmbeddings
27
Agenda
1. Motivation
2. State-of-the-Art
3. VehicleRecommendationswith Deep Learning
4. ACM RecSysConference 2017
5. Discussion
28
Vehicle Recommendations: End-to-End Approach
CandidateGeneration
Serving Ranking
Preprocessing ClassifierTrainingData
29
Vehicle Recommendations: Technologies
Locally OptimizedProduct Quantization
HardwareGPU-Server
NVIDIA Tesla K804x Intel Xeon 3.5 GHz64GB RAM, 850GB Disk
AWS Instances
30
Vehicle Recommendations: Data
Users & InteractionsRegistered UsersSample Size: 100,000 UsersEvents: View, Bookmark, Contact
Time-basedTrain-Test-Split
CW14
CW15
CW16
CW17
CW18
April 2017 May
Training Test
85 : 15
31https://medium.com/towards-data-science/deep-learning-4-embedding-layers-f9a02d55ac12
What does ‘Embedding‘ actually mean?
0. blue 0
1. green 0
2. red 0
3. yellow 0
4. orange 0
5. black 1
6. white 0
7. brown 0
1
0
1
binaryEmbedding
One-Hot-Encoding
32
33
categorical features
one-many-encoding one-hot-encoding
feature valuesucat icat
eclimatisation
icont
embeddinguser
consumption first_reg price...
embeddingi, cont
ucont
embeddingu,cont
...
outlier removal
z-normalisation
ELU (256)
ELU (128)
ELU (64)
Deep Component
Wide Component
cross user-item transformations
embeddingitem
...
...
climatisation color
ecolor etransmission
transmission
OutputProbability that user ulikes vehicle i
meanconsumption meanprice
stddevconsumption stddevprice
...
concatenate concatenate
outlier removal
z-normalisation
Pre
pro
cess
ing
Em
bed
din
gW
ide
and
Dee
p
34
Vehicle Recommendations: End-to-End Approach
CandidateGeneration
Serving Ranking
Preprocessing ClassifierTrainingData
✓ ✓
35
Vehicle Recommendations: Ranking
Target: Rank Candidates descendantly by interaction probability
user0
...
userm
itemm,0
...itemm,T
user-specificCandidate Lists
user-specifick-Rankings
𝑘 ≤ 𝑇
itemm,0
...itemm,k
36
Vehicle Recommendations: End-to-End Approach
CandidateGeneration
Serving Ranking
Preprocessing ClassifierTrainingData
✓ ✓
✓ ✓
37
Results: DLRS Recommendation Relevance
0,25%
0,35%
0,45%
0,55%
0,65%
0,75%
0,85%
k = 1 k = 5 k = 10
MA
P@
k
CF (⍺=0.03, d=100)
Hybrid CF-CBF (⍺=0.03, d=100)
Hybrid CF-CBF (⍺=0.03, d=700)
DL (multi-cos)
+20%
+65%
38
Agenda
1. Motivation
2. State-of-the-Art
3. VehicleRecommendationswith Deep Learning
4. ACM RecSysConference 2017
5. Discussion
39
40Twitter: @domonkostikk
ACM RecSys Conference 2017
627 Participants
43 Countries
„Accuracy doesn‘t matter – impact does!“
„Try to not useMovieLens“
„People are most curiousabout themselves“
41Quadrana, Massimo et al.: Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks (2017)
RNNs for Video and Job Recommendations
42
"We can only see a short distanceahead, but we can see plentythere that needs to be done."
- Alan Turing
43
References[1] Quadrana, Massimo, Karatzoglou, Alexandros, Hidasi, Balázs, Cremonesi, Paolo. “Personalizing Session-based Recommendations with Hierarchical
Recurrent Neural Networks“ Proceedings of the 11th ACM Conference on Recommender Systems. 2017
[2] Wang, Hao, Wang, Naiyan, Yeung, Dit-Yan. “Collaborative Deep Learning for Recommender Systems“ Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2015
[3] Cheng, Heng-Tze, et al. "Wide & deep learning for recommender systems." Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 2016.
[4] Covington, Paul, Jay Adams, and Emre Sargin. "Deep neural networks for youtube recommendations." Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016.
[5] Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016.
[6] Heaton, Jeff. Artificial Intelligence for Humans: Deep Learning and Neural Networks. 2015.
[7] Ricci, Francesco and Rokach, Lior and Shapira, Bracha. Recommender Systems Handbook. Springer-Verlag. 2015
[8] Abadi, Martín, et al. "Tensorflow: Large-scale machine learning on heterogeneous distributed systems." arXiv preprint arXiv:1603.04467 (2016).
[9] Loni, Babak, et al. "Bayesian Personalized Ranking with Multi-Channel User Feedback." Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016.
[10] Kalantidis, Yannis, and Yannis Avrithis. “Locally optimized product quantization for approximate nearest neighbor search.“ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.
[11] Reinsel, David, Gantz, John, Rydning, John. “Data Age 2025: The Evolution of Data to Life-Critical Don't Focus on Big Data; Focus on the Data That's Big“ International Data Corporation (IDC). 2017
Thank You
Marcel Kurovski
Big Data Scientist
inovex GmbH
Kupferhütte 1.13,
Schanzenstr. 6-20
51063 Cologne
0173 3181 088