Temporal Diversity in Recommender SystemsNeal Lathia, Stephen Hailes, Licia Capra, and Xavier AmatriainSIGIR 2010

April 6, 2011Hyunwoo Kim

2

Outline Introduction Why Temporal Diversity? Evaluating for Diversity Promoting Temporal Diversity Conclusion

3

Introduction Collaborative Filtering [Kim, ECRA2010]

4

Introduction

in 2006

in 2011Alice

User’s interest changes over time [Zheng, ESWC2011]

baby health

education

5

Introduction A problem with current evaluation techniques

– No temporal characteristics of the produced recommen-dations

In this work,– Diversity of top-N lists over time

6

Why Temporal Diversity? Two perspectives

– Changes that CF data undergoes over time– How surveyed users respond to recommendations with vary-

ing levels of diversity

Changes over time– Continuous rating of content– Recommender systems have to make decisions based on

INCOMPLETE and CHANGING data– A list at any particular time is likely to be different with pre-

vious list

– Do these changes translate into different recommendations over time?

7

Why Temporal Diversity? User survey

– Popular movies from

8

Why Temporal Diversity? User survey

– S1: popular movies with no diversity– S2: popular movies with diversity– S3: randomly selected movies

In S3, some users com-mented:

“appeared to very random”“varied widely”

“avoided box office hits”…

In S1, some users com-mented:

“lack of diversity persisted”“too naïve”

“not working”“decreased interest”

…Users are responding to the im-pression of the recommender system!!

9

Why Temporal Diversity? Qualities in recommendations

– ACCURATE recommendations– CHANGE OVER TIME– NEW recommendations

10

Evaluating for Diversity How diverse CF algorithms are over time

– Baseline: item’s mean rating– Item-based k-Nearest Neighbor (kNN)– Matrix factorization approach based on Singular Value De-

composition (SVD)

Dataset– Netflix prize dataset

To improve the accuracy of predictions about how much some-one is going to enjoy a movie based on their movie preferences

$1,000,000 grand prize on September 21, 2009

11

Evaluating for Diversity Diversity and novelty

Last week’s list

This week’s list

Diversity = 1/5

12

Evaluating for Diversity Diversity and novelty

Previous recommen-dations

This week’s list

Novelty = 2/5

13

Evaluating for Diversity Diversity results and analysis

– Baseline produces little to no diversity– Factorization and nearest neighbor approaches increment di-

versity

14

Evaluating for Diversity Novelty results and analysis

– Novelty values are lower than diversity values– When different a recommendation appears, it is a recom-

mendation at some point in the past

15

Evaluating for Diversity How diversity relates to accuracy

– RMSE: Root Mean Squared Error– Different algorithms often overlap and kNN CF is sometimes less

accurate than the baseline

16

Promoting Temporal Diversity Diversity comes at the cost of accuracy When promoting diversity, we must continue to take

into account users’ preferences

Three methods– Temporal switching– Temporal user-based switching– Re-ranking frequent visitors’ lists

17

Promoting Temporal Diversity Temporal switching

Temporal user-based switching

kNN SVD SVDkNN kNN

kNN SVD SVDkNN kNN

user login user login user login

1st 2nd 3rd 4th 5th

18

Promoting Temporal Diversity Temporal switching from a system

19

Promoting Temporal Diversity Temporal user-based switching

20

Promoting Temporal Diversity Re-ranking frequent visitors’ lists

Full listTop-5 list Re-ranking list

Diversity 40%

21

Promoting Temporal Diversity Re-ranking frequent visitors’ lists

– Only a single CF algorithm is used

22

Conclusion What we found

– State-of-the-art CF algorithms produce low temporal diversity– They repeatedly recommend the same top-N items to users

What we did– A metric to measure temporal diversity– A fine-grained analysis of the factors that may influence di-

versity

Future work– How novel items find their way into recommendations– How user rating patterns can be used to improve recom-

mender system’s resilience to attack