[UMAP 2015] Integrating Context Similarity with Sparse Linear Recommendation Model

Integrating Context Similarity

with Sparse Linear

Recommendation ModelYong Zheng, Bamshad Mobasher, Robin Burke

Center for Web Intelligence, DePaul University, Chicago, USA

The 23rd Conference on User Modeling, Adaptation and Personalization,

Dublin, Ireland, June 29 – July 3, 2015 (UMAP 2015)

Agenda

• RecSys and Context-aware RecSys

• Contextual Modeling

• SLIM and Contextual SLIM

• Modeling Context Similarity

• Experimental Evaluations

• Conclusions and Future Work

Agenda







RecSys and Context-aware RecSys

• Recommender Systems (RS)

The data is usually a 2D rating matrix: User × Item ―> Ratings

Task-1: Rating Predictions for <user, item> pair

Task-2: Top-N Recommendations for a specific user, i.e., provide a

list of ranked items to the user

• Context-aware RecSys (CARS)

Context dimension: the variable, e.g., time, location, companion

Context condition: values in dimension, e.g., weekend and weekday

Context situation: a set of conditions, e.g., <weekend, home, sister>

The data is represented in a multi-dimensional rating space.

Task-1: Rating Predictions for <user, item, contexts>

Task-2: Top-N Recommendations for a user in specific contexts,

RecSys and Context-aware RecSys

Agenda







Contextual Recommendations

• How to build CARS algorithms?

Contextual Recommendations

• Contextual ModelingThere are usually two ways for contextual modeling:

1). Independent Contextual Modeling

Tensor Factorization, ACM RecSys 2010

2). Dependent Contextual Modeling

2.1). Deviation-Based Modeling

Context-aware Matrix Factorization, ACM RecSys 2011

Contextual Sparse Linear Method, ACM RecSys 2014

2.2). Similarity-Based Modeling

The proposal in this paper, UMAP 2015

Contextual Modeling

• Independent Contextual ModelingTensor Factorization (TF), ACM RecSys 2010

Assumption: context is independent with user/item dimension.

But usually, there are dependencies involved.

Contextual Modeling

• Dependent Contextual ModelingContext-aware Matrix Factorization (CAMF), ACM RecSys 2011

Contextual Sparse Linear Method (CSLIM), ACM RecSys 2014

Global average

rating

User bias Item bias

Matrix Factorization:

CAMF:

Item bias in contexts

Contextual Modeling

• Dependent Contextual ModelingContext-aware Matrix Factorization (CAMF), ACM RecSys 2011

Contextual Sparse Linear Method (CSLIM), ACM RecSys 2014

Those approaches are named as deviation-based modeling,

since they tried to incorporate contextual rating deviations into

recommendation algorithms by modeling dependencies or

correlations between contexts and user/item dimensions.

Any other alternatives? How about the dependencies or correlations

among contexts? We name this approach of context modeling as

similarity-based modeling.

Agenda







SLIM and Contextual SLIM

• Why SLIMSLIM = Sparse Linear Method, which is an effective top-N

recommendation algorithm in traditional RS.

In this paper, we choose SLIM as the base algorithm, and introduce how

to build contextual SLIM algorithms by incorporating context similarity.

SLIM was demonstrated as the most effective top-N recommendation

algorithms in previous work. Here, we focus on top-N contextual

recommendation. Other algorithms, such as matrix factorization, can

also be chosen as base algorithm.

• SLIM in Traditional RecSys

Matrix R = rating matrix; W = coefficient matrix

SLIM aggregates users’ ratings by coefficients between items.

It learns item coefficients by minimizing the ranking score.

Sparse Linear Method (SLIM)

• CSLIM in Context-aware RecSys

P is multidimensional contextual rating space; W is item coefficient matrix;

Matrix D estimates the rating deviation from one context to another.

1). By Deviation-Based Contextual Modeling, RecSys 2014, CIKM 2014

Contextual SLIM (CSLIM)


Previous dependent contextual modeling approaches mainly focused on

modeling the correlations between context and user/item dimensions,

but ignore the correlation between contexts themselves;

Context similarity = similarity between two contexts, measuring inner

similarities or correlations between two contextual situations;

We propose and believe that modeling context similarities is another important

way to develop dependent contextual modeling approaches, rather than

modeling contextual rating deviations!!!

2). By Similarity-Based Contextual Modeling, UMAP 2015



Original SLIM:

Deviation-Based CSLIM:

Similarity-Based CSLIM:

2). By Similarity-Based Contextual Modeling, UMAP 2015

Deviation term

Similarity term


Agenda







Modeling Context Similarity

• Context Similarity

Context similarity can be obtained in the following ways:

1). Semantics

But, it is hard to for <holiday, cinema> & <weekday, home>;

Semantics is more useful for hierarchical or tree-based categorical data;

2). Calculation based on co-ratings in different contexts

However, contextual rating data is usually sparse, which results in unreliable

calculations for context similarity.

3). Learning methods

Instead, we can learn the similarity directly by minimizing ranking errors.

error = ranking score – predicted ranking score

Minimizing this ranking error by gradient descent in CSLIM


• Context SimilarityLearning methods

Instead, we can learn the similarity directly by minimizing ranking errors.

error = ranking score – predicted ranking score

Minimizing this ranking error by gradient descent in CSLIM

However, the performance may directly depend on how we represent

and model context similarity. In this paper, we discuss 4 modeling:

1). Independent Context Similarity (ICS)

2). Latent Context Similarity (LCS)

3). Weighted Jaccard Context Similarity (WJCS)

4). Multidimensional Context Similarity (MCS)


• 1).Independent Context Similarity (ICS)Similarity-Based CSLIM:

Independent Context Similarity (ICS) can be represented as follows:

For example: Ck = {Time = Weekend, Location = Home}; Cm = {Time = Weekday, Location = Office}

is: Similarity(Weekend, Weekday) × Similarity (Home, Office)

Assumption: contextual variables are assumed as independent.

What to be learnt: each individual similarity between two conditions;


• 2).Latent Context Similarity (LCS)Similarity-Based CSLIM:

Latent Context Similarity (LCS) is an improvement over ICS.

For example: Ck = {Time = Weekend, Location = Home}; Cm = {Time = Weekday, Location = Office}

is: Similarity(Weekend, Weekday) × Similarity (Home, Office)

Each condition is represented by a vector;

What to be learnt: the weights in vectors for each contextual condition.

Training: <weekend, weekday> <weekday, holiday>

Testing: <weekend, holiday>Context Sparsity


• 3).Weighted Jaccard Context Similarity (WJCS)

Weighted Jaccard Context Similarity refers to similarity between two strs.

Assume those three context dimensions are equally weighted, w1 = w2 = w3 = 1.

= # of matched dimensions / # of all dimensions = 2/3

What to be learnt: the weight for each context dimension.

Similarity is measured by Weighted Jaccard similarity

User Movie Time Location Companion Rating

U1 Titanic Weekend Home Girlfriend 4

U2 Titanic Weekday Home Girlfriend 5

U3 Titanic Weekday Cinema Sister 4

U1 Titanic Weekday Home Sister ?


• 4).Multidimensional Context Similarity (MCS)Similarity-Based CSLIM:

Multidimensional context similarity utilizes the distance metric.

(NA, Home, Weekday)

(NA, Home, Weekday)

(Kids, Home, NA) (Kids, Home, NA)


• 4).Multidimensional Context Similarity (MCS)Similarity-Based CSLIM:

Key points in MCS:

1). Each contextual variable is represented as an axis;

2). Each contextual condition is one position in corresponding axis;

3). Thus a contextual situation is mapped as a point in the space;

4). The distance between two points is viewed as dissimilarity;

Any distance metric can be applied; here we use Euclidean distance.

What to be learnt: the positions of each condition in axises.


• Summary

Similarity-Based CSLIM:

What to be learnt in each context similarity model:

ICS LCS

The correlation (real value) for each

individual pair of context conditions

The vector representation (weights in

factors) for each contextual condition

WJCS MCS

The weights for each context

dimension.

The positions (real values) for each

contextual condition

Agenda







Experimental Evaluations

• Data Sets

Note: The number of context-aware data sets is really limited!!

We use 5-folds cross validation for evaluation purposes.

We use Precision and Mean Average Precision (MAP) as metrics:

- Precision: measuring the hit ratio towards relevant items;

- MAP: additional taking the rankings of items into account.


• Algorithms for Comparison

1). Baseline Algorithms

CASA = Context-aware Splitting Approaches (a pre-filtering approach)

TF = Tensor Factorization (independent contextual modeling)

CAMF = Context-aware MF (dependent contextual modeling)

Deviation Model = CSLIM using deviation-based contextual modeling

2). New Algorithms

Four algorithms using different context similarity representations:

Similarity-ICS Model, Similarity-LCS Model

Similarity-WJCS Model, Similarity-MCS Model

Note: all those models were built on SLIM.



• Summary of the results

1). Which algorithm is the best?

Answer: Similarity-Based CSLIM using Multidimensional Context Similarity

2). Which one is better? Deviation or similarity-based modeling?

Answer: we can always find a similarity-based contextual modeling outperforming the

deviation-based modeling; but, the appropriate representation for context similarity

should be selected.

3). Which representation is the best?

Generally speaking, latent context similarity always outperforms independent context

similarity; and multidimensional context similarity is the best choice. Weighted

Jaccard context similarity shows non-stable recommendation performance in the

experiments.

Agenda







Conclusions & Future Work

• Conclusions We propose a new way to build dependent contextual modeling – similarity-

based contextual modeling;

We choose SLIM as the base algorithm and incorporate context similarity into

SLIM to formulate new contextual SLIM algorithms;

We discuss different representations to model context similarity;

We demonstrated the advantages of similarity-based CSLIM by experimental

evaluations over multiple context-aware data sets.

• Future WorkMultidimensional Context Similarity (MCS) is the best representation to model

context similarity; but it increases computational costs at the same time. In our

future work, we’d like to explore how to reduce the computational costs for MCS,

e.g., reducing context dimensions, merging contextual conditions, etc.

Conclusions & Future Work

• Stay TunedContext similarity can also be incorporated into matrix factorization.

Yong Zheng, Bamshad Mobasher, Robin Burke. "Incorporating Context

Correlation Into Context-aware Matrix Factorization". Workshop on Intelligent

Personalization @ IJCAI 2015

Yong Zheng, Bamshad Mobasher, Robin Burke. "Correlation-Based Context-

aware Matrix Factorization". In DePaul CDM School of Computing Research

Symposium, 2015 (Best Paper Award)

• Survey: Context-aware Movie RatingsWelcome to fill out it: http://depaul.qualtrics.com/SE/?SID=SV_4TrIZbAnQtzaHsx

Short URL: http://tinyurl.com/surveycars

http://depaul.qualtrics.com/SE/?SID=SV_4TrIZbAnQtzaHsx

http://tinyurl.com/surveycars

Integrating Context Similarity

with Sparse Linear

Recommendation ModelYong Zheng, Bamshad Mobasher, Robin Burke

Center for Web Intelligence, DePaul University, Chicago, USA

The 23rd Conference on User Modeling, Adaptation and Personalization,

Dublin, Ireland, June 29 – July 3, 2015 (UMAP 2015)

Engineering

[UMAP 2015] Integrating Context Similarity with Sparse Linear Recommendation Model