Towards context-sensitive collaborative mediarecommender system

Mohammed F. Alhamid & Majdi Rawashdeh &

Hussein Al Osman & M. Shamim Hossain &

Abdulmotaleb El Saddik

Received: 25 January 2014 /Revised: 16 June 2014 /Accepted: 14 August 2014# Springer Science+Business Media New York 2014

Abstract With the rapid increase of social media resources and services, Internet users areoverwhelmed by the vast quantity of social media available. Most recommender systemspersonalize multimedia content to the users by analyzing two main dimensions of input:content (item), and user (consumer). In this study, we address the issue of how to improvethe recommendation and the quality of the user experience by analyzing the contextual aspectof the users, at the time when they wish to consume multimedia content. Mainly, we highlightthe potential of including a user’s biological signal and leveraging it within an adaptedcollaborative filtering algorithm. First, the proposed model utilizes existing online socialnetworks by incorporating social tags and rating information in ways that personalize thesearch for content in a particular detected context. Second, we propose a recommendationalgorithm to improve the user experience and satisfaction with the use of a biosignal in therecommendation process. Our experimental results show the feasibility of personalizing therecommendation according to the user’s context, and demonstrate some improvement on coldstart situations where relatively little information is known about a user or an item.

Multimed Tools ApplDOI 10.1007/s11042-014-2236-3

M. F. Alhamid :H. Al Osman :A. El SaddikMultimedia Communications Research Laboratory (MCRlab), School of Electrical Engineering andComputer Science, University of Ottawa, Ottawa, Canada

H. Al Osmane-mail: halos072@uottawa.ca

A. El Saddike-mail: elsaddik@uottawa.ca

M. F. Alhamid (*) :M. S. Hossain : A. El SaddikCollege of Computer and Information Sciences (CCIS), King Saud University, Riyadh, Saudi Arabiae-mail: malha016@uottawa.ca

M. S. Hossaine-mail: mshossain@ksu.edu.sa

M. RawashdehDivision of Engineering, New York University Abu Dhabi, Abu Dhabi, United Arab Emiratese-mail: majdi@nyu.edu

Keywords Personalized search . Context media search . Context-aware recommendation .

Collaborative context . Retrieval model . Information filtering

1 Introduction

The prevalence of smart devices has brought more accessibility to different multimediacontents through the social web. Also, the ability to embed our living environment with smalland cheap sensors enables and accelerates the development of context-aware recommendationsystems that can obtain the most desirable content in a specific context. The combination ofreal-time access to different multimedia contents and the overwhelming content optionsavailable on the social web increases the challenge of building a personalized search model.For these reasons, this field of study, related to helping users browse and consume multimediacontents, has been a challenging issue for researchers over the past few years [33, 40].

Different users have different personalities [26]; likewise, a user’s choice of which multi-media content to consume varies with different parameters, which include emotions andvarious health conditions [23]. This necessitates the integration of user preferences andphysiological context in the development of a recommendation algorithm. Using today’stechnology, we can obtain a user’s various physiological functions and monitor them usingbiomedical sensors, with the help of computer systems.

In this paper, we focus on recommending multimedia content to users based on theirinstantaneous requirements, which can be represented by categorizing their contexts intophysiological and environmental parameters, in order to enhance their experience and comfortlevel. Although a broad number of contextual parameters can be considered, we only showcertain context conditions in this paper. Particularly, we highlight the importance of thephysiological aspect of the user’s context during the recommendation process. However, theproposed model can also be applied to accommodate other contextual information that willhelp personalize the search.

As people share their preferences and information on different popular social media sites, theyshare important information about their consumedmultimedia contents including their descriptiveexperience. A huge number of users on Last.fm, Youtube, and Facebook upload media contentwith annotation tags that describe their uploaded type of media. Our method benefits from theavailability of such social tags by exploring context-based item recommendations and by avoidingthe unrealistic need to analyze innumerable items on the Internet in order to detect the context.Accordingly, our recommendation technique offers a feasible solution to find relevant mediacontent, based on the detected contextual information, and benefiting from the large number ofexisting online users who contribute to the classification and annotation of different media, whichis more effective than just relying on feature extraction.

Inspired by our earlier work [2], our proposed recommendation model makes the followingcontributions toward recommendation techniques. First, we utilize existing online social networks byincorporating social tags and rating information in ways that personalize the search for the rightcontent, for a particular detected context. Second, we propose a recommendation algorithm toimprove the user experience and satisfaction by the use of a biosignal in the recommendation process.

The rest of this paper is organized as follows: Section 2 briefly reviews the existing workrelated to context-aware recommendations. Section 3 presents the detection of the userphysiological context. Section 4 provides details concerning the proposed context-awarerecommendation model. Section 5 highlights some scenarios and the usability of the proposedmodel by presenting an implemented recommender application. Finally, we present theexperimental method and the related results and conclusion in Sections 6 and 7.

Multimed Tools Appl

2 Related work

In this section, we briefly bring attention to the different existing recommendation techniques,and particularly to the ones based on context-awareness. We also review existing approachesthat explore context detection and context-aware computing during the recommendationprocess.

2.1 Recommendation techniques

In recent years, the development of recommender systems has attracted considerable researchinterest in an attempt to discover the relation between users and the items they consume. Thestudies analyzed the hidden relationships between users and items, and proposed severalmethodologies that became today’s ways of establishing recommendations. The first tradition-al recommendation approach is Content-Based Filtering (CBF). The CBF approach analyzesthe item content and creates a profile for each by assigning features such as type, category, andcontent special attributes. It then relies on profile matches, which contain user-content explicitratings of other item profiles. Such an approach is commonly used in building music or movierecommender systems, and needs the user’s historical selection of content in order to predictthe ranking of new items.

Another traditional method is Collaborative Filtering (CF), which recommends items basedon the relationship of users rather than only items. This approach relies on the identification ofsimilarities between users who share the same interests. As opposed to CBF, CF gives moreattention to social aspects and the feedback of similar users to unknown user-item rating.

At a later time, researchers combined CBF and CF in order to build much improvedversions of these traditional methods. A hybrid method is proposed to overcome a problemthat exists in CF, where they only consider users to be close to each other based on thecalculated distance, when in reality they might have completely different tastes. Anotherproblem is that new users do not have enough history to feed the recommender system andallow it to find relevant items. This is a well-documented problem known as the cold-startproblem [37].

Contextual information boosts the effectiveness of the traditional recommender systems,which consider only the user, items, or their related information to find the next expected itemto be selected. For instance, the user may like to listen to a particular type of music whilerunning in the morning and not at other times of the day. Accordingly, if the context of the userhas been well detected, the recommendation results might be completely different from thetraditional ones, and better suited to the user’s needs [33].

Some studies combine the existing recommendation approach with some contextual infor-mation. For instance, Woerndl et al. [36] apply a combination of content-based filtering,collaborative filtering, and a hybrid method on the dataset of mobile application recommendersystems. The authors present the effectiveness of applying the hybrid approach as a successfulway of accommodating a contextual type of information. Campos et al. [9] presented therecommendation model that combines CBF and CF. Context is also considered as a contentfeature and it is assumed that context elements are independent, so they can apply the Bayesiannetwork algorithm on them. Another hybrid approach is presented by Lekakos and Caravelas[22], which is to recommend movies by analyzing the content and the collaborative relationbetween users. Bogers [4] proposed a Markov random walk algorithm over a contextual graphon a movie dataset. The user rating and tags, the movie genre, and the actors are all consideredas contextual information in the graph. Cai et al. [6] extracted emotion tags available on a webdocument in order to match them with music lyrics. In a similar focus, Hyung et al. [18] use

Multimed Tools Appl

textual inputs from the user to find similar documents using a Latent Semantic Analysisapproach (LSA) and a Probabilistic Latent Semantic Analysis (PLSA), which leads to a betterrecommendation of music. Another study using music extracted features as a context for musicrecommendation is proposed by Yoon el al. [38]. Han et al. [13] propose a music recommen-dation model that transits the user’s emotional state from one to the other using the recom-mended music. Authors use a proposed ontology to model the relationship between musicitems and the related emotion.

More recently, recommender systems rely on social networks in order to collect vast amounts ofinformation about the user, their multimedia choices and their behavior. For instance, Zangerleet al. [42] present a facilitation approach to collect a user’s music dataset from Twitter, assumingthe user’s music player shares such information on Twitter. Pessemier et al. [29] discussed anactivity detection in a mobile environment as supportive information for the recommendationprocess. By using an attached accelerometer on a mobile device, the authors were able to detectfour basic activities: running, walking, standing and cycling. Based on the detected activity and onother contextual information such as location and weather, the recommender system can fetchdifferent categories of information such as train schedules and restaurants in the surrounding area.In another study, Kim et al. [20] employed a Hidden Markov Model on a contextual informationdataset to recommend the most appropriate menu for healthcare services.

2.2 Physiological-related recommendation

In their study, Liu et al. [23] addressed the effects of music playlist recommendations on theheart, while flying on an airplane. Since the airplane environment causes discomfort and stress,authors used a heart sensor embedded in the aircraft’s seat to monitor the heart activities andparticularly to measure the stress. This study [23] focuses on analyzing the airplane environ-ment and the effects of music on the user’s level of stress, which differs from the focus of thispaper. Oliver and Flores-Mangas [28] presented MPTrain system that uses a smartphone-basedaccelerometer and heart sensor to analyze the user activities to select appropriate music bychanging the user selection based on the music rhythm. MPTrain is proposed to help usersachieve their exercise goals by speeding up or slowing down their activity. In another study,Nirjon et al. [27] proposed a context-aware recommendation system that uses an earphonedevice equipped with a heart rate detection sensor. The sensor collects continuousElectrocardiography (ECG) signals, and extracts the peaks in order to calculate the averageheart rate instantaneously. Similarly, the system extracts offline beats from the music signal orwhat is known as the tempo of the music, as well as the frequency of the sound, and the energyof the signal.

Our work differs from all the presented studies in this section in such a way that ourproposed context-aware recommendation model considers the physiological condition of theuser as a useful context parameter to enhance the recommendation results. Another differenceis that our aim is to recommend items that match the detected context and not to transfer thephysiological condition from one user to the other. In addition, our model does not require theanalysis of the content of the media item itself, for example the content of the image or thevideo. Instead, we rely on the user’s available context.

3 Detecting a User’s physiological context

In order to detect the user’s physiological context, we have to record and monitor the user’sbiological signals to determine their physiological or emotional (e.g. mental stress) situation.

Multimed Tools Appl

Even though we collect biological signals, we are not concerned with diagnosing illnesses. Ourgoal is to use the collected physiological data to better understand the physiological status ofusers. In fact, in this paper we focus on one particular emotion: mental stress. The best tool todetect the targeted physiological conditions is to monitor the heart using an ECG signal thatcan measure the Heart Rate Variability (HRV) [7].

Some of the least obtrusive commercial ECG devices are in the form of a chest strap fittedwith two electrodes and an electronic circuit. This obliges the user to wear the ECG sensor atall times in order for the application to benefit from physiological or psychological contextualinformation. Since these devices are wearable, they might be associated with a certain level ofdiscomfort over prolonged periods of use. In this paper, we used a much more convenient andnon-invasive device as shown in Fig. 1. The ECG sensor used is a product from the AliveCorCompany, and is connected to a “Samsung Galaxy III” smartphone device. By having a sensorthat just looks like a protection cover attached to the smart device, an installed androidapplication can collect the ECG signal. We use the ECG signal to extract the HRV information.

Since the ECG signal represents the electrical activity of the heart, we can extract the HRVsignal by calculating the series of time intervals between two consecutive heart beats or R-waves [27]. Fig. 2 shows a portion of a recorded ECG signal for a couple of minutes. The HRVis known to shed light on the mental stress and infer some conclusions about the conditions ofmonitored individuals [3, 11, 14, 17].

To detect mental stress, the latest measurement collected is compared to a previous recordedbenchmark. The benchmarks are typically previous measurements taken during a neutral stressstate. The length of the measurement is a key constraint here, since the longer the measurementis, the more reliable the conclusions are. Therefore, it is widely recommended to use ECGmeasurement records of at least three minutes [14]. Accordingly, the stress detection algorithmassesses the recorded ECG signal and compares the HRV calculated values to the benchmarkmeasurement. Nonetheless, since the reason for measuring the physiological parameters in thisstudy is to recommend suitable multimedia content, such a long measurement reduces thesystem’s usability and applicability. In other words, asking a user to have a three minute ECGmeasurement before the system can recommend a song is unrealistic. Consequently, wedecided to minimize the ECG recording time needed to evaluate the physiological parametersin order to increase the application’s practicability. The use of shorter ECG time measurementswhen analyzing mental stress is still a relatively new concept. Therefore, more research isneeded in order to assess the accuracy of such an approach compared to longer accurate

Multimediarecommendationapplication

Single-lead ECG

Fig. 1 An image of the single-lead ECG sensor attached to an android device in our developed prototypeapplication

Multimed Tools Appl

measurement methods [30]. As we have previously argued, the proposed model is nottargeting medical applications and therefore we are more at liberty to use such an approach.As a result, we are willing to potentially sacrifice some accuracy in the goal of increasingusability. Therefore, for the physiological context detection, we simply use the short termanalysis of the heart rate variability proposed by [30] to determine if there is the potential of theuser being in an elevated stress state, a neutral state, or a relaxed state, as in Fig. 3.

After collecting the required biosignal, the extracted physiological information is sent alongwith other contextual parameters to the recommender system. Then, the recommender systemrecommends content such as music and movies, which consider the detected physiologicalstate of the user.

3.1 Detecting physiological context experiment

We attempted to determine how the short ECG measurement is able to distinguish between thethree main physiological contexts: elevated stress, neutral, and relax states. We invited subjectsto measure their HRV parameters while asking them to perform three types of activities. Thesessions were composed of stress, neutral, and relax exercises. For the stress exercise, subjectswere asked to perform a Stroop color-word test. The Stroop test has been used in differentphysiological and psychological studies [21, 25, 34] since it involves sympatho-adrenalactivation that is reflected in the subject’s heart and respiration rates [34]. For the neutralexercise, subjects were asked to sit comfortably and try to read an article. We will use this

Fig. 2 A portion of the original ECG signal recorded using the single-lead ECG sensor

Elevated StressState

NeutralState

RelaxedState

Fig. 3 Three different physiological states are proposed as possible user physiological contexts

Multimed Tools Appl

exercise as a benchmark to analyze stress and relaxation situations. For the relaxed exercise,subjects were asked to sit comfortably, close their eyes, and listen to relaxing music. Eachexercise lasted three minutes and contained 3 min worth of ECG data, but only the last 30 swere fed to the physiological detection algorithm to evaluate the short ECG measurementperformance.

Ten adult subjects participated in the physiological context detection experiment: 6 malesand 4 females. The average age was 28.7 years. The experiment was carried in an office spacein our laboratory. An office desk, a laptop and a headphone were provided for each participant.Subjects were seated in front of the laptop while the ECG sensor sent the data recorded to aJava based computer program.

After collecting the ECG signal, three HRV parameters of interest were extracted tomeasure the mental stress:

& Low frequency band of the HRV (LF).& High frequency band of the HRV (HF).& The ratio of low frequency to high frequency HRV (LF/HF).

Numerous related studies have shown the effectiveness of these HRV parameters tomeasure mental stress [1, 30]. Particularly, during a stressful situation, the HF component isnoticed to decrease while the LF ratio to HF increases. The resulting HRV parameters for the10 subjects during the last 30 s of the three exercises of the experiment are presented inTable 1.

The results obtained have shown, on average for the 10 subjects, an expectedincrease in LF/HF component triggered by a stressor activity (with respect to theneutral state). Such an increase in the LF/HF was also observed in earlier documentedstudies using a similar Stroop test, such as in [32]. In addition, for the same stressoractivity, a decrease in the HF component (with respect to the neutral state) wasnoticed from the experiment results on average for the 10 subjects. Such a decrease

Table 1 The results of detecting the physiological context of the users

Subjectnumber

Neutral Exercise Session (N) Stress Exercise Session (S) Relax Exercise Session (R)

LF (N)1 HF (N)1 LF/HF(N)

LF (S)1 HF (S)1 LF/HF(S)

LF (R)1 HF (R)1 LF/HF(R)

1 2,179.69 605.20 3.60 2,884.29 566.49 5.09 1,421.07 906.34 1.57

2 5,457.56 4,715.32 1.16 2,871.76 2,041.03 1.41 2,755.02 6,388.11 0.43

3 2,548.07 3,136.71 0.81 2,467.03 1,443.93 1.71 2,976.48 4,119.28 0.72

4 1,648.41 922.55 1.79 1,939.05 864.21 2.24 1,399.98 563.22 2.49

5 4,972.69 964.01 5.16 3,898.86 863.92 4.51 1,579.52 1,128.10 1.40

6 1,646.78 267.89 6.15 2,468.13 251.07 9.83 1,340.22 263.30 5.09

7 6,518.59 2,158.91 3.02 12,489.29 1,642.77 7.60 1,792.80 1,577.55 1.14

8 25,890.21 150,127.87 0.17 49,184.93 108,881.39 0.45 66,522.72 142,255.49 0.47

9 5,306.72 2,970.81 1.79 11,629.48 4,229.37 2.75 9,693.36 7,317.30 1.32

10 3,164.43 1,854.81 1.71 4,436.91 1,791.26 2.48 5,809.44 3,210.19 1.81

Average 5,933.32 16,772.41 2.53 9,426.97 12,257.54 3.81 9,529.06 16,772.89 1.64

STDEV 7,227.36 46,876.27 1.93 14,487.92 33,968.30 2.97 20,201.45 44,158.70 1.36

1 Values are expressed in milliseconds (ms2 )

Multimed Tools Appl

has also been confirmed in [8, 17]. On the other hand, for the relaxation activity, wenoticed, on average, an expected decrease in the LF/HF component with respect to theneutral state. Nonetheless, we did not find a significant change in the HF componentwith respect to the neutral state. Therefore, we will use the LF/HF exclusively todifferentiate between the various physiological (stresses, neutral, or relaxed) states.

Subjects were asked to evaluate the three exercises subjectively by giving a rating value thatindicates how stressful each exercise was. A rating range from 0 to 10 was used, with 10indicating the exercise was stressful, and 0 indicating that the exercise was relaxing. Ourexperimental design of three different exercises distinguishing the required three physiologicalstatuses (stressed, neutral, and relaxed) has been reflected in the results reported in Table 2.Accordingly, the HRV analysis results verify the correlation between the physiological statusof subjects performing the three experimental exercises and the observed subjectiveassessment.

4 Preliminaries

Before describing the details of the recommendation model, we have to introduce aset of definitions to formalize our recommendation problem. In this paper, bold upper-case letters, such as A, are used to denote matrices; whereas the corresponding lower-case italic with two subscript indices, such as ax,y represent the entries of thematrices. Capital italicized letters represent sets, such as U, and an upper-case italicletter with one subscript index such as Ux represents an entry element x from set U.We also formalize the matrix elements in the recommendation model to follow thefollowing format: Matrix=[(element entry)Set A, Set B]|A|×|B|. For example, the context-item matrix in the model is represented in the following format: A=[ac,i]|C|×|I| . Inaddition, for simplifying the recommendation problem, we refer to the term “users” or(U) to represent the set of users, and “items” or (I) to denote a set of media resourcesthat can be recommended such as music or movies. Table 3 summarizes the notationsemployed in the rest of this paper.

Table 2 The subjective results ofuser experience during theexperiment

Subjectnumber

* (0: relaxed, 10: stressed)

Neutral ExerciseSession (N)

Stress ExerciseSession (S)

Relax ExerciseSession (R)

1 4.00 7 2.00

2 6.00 9 3.00

3 3.50 6.5 3.00

4 4.00 7 0.00

5 6.00 9 1.00

6 5.00 9 2.00

7 4.00 7 2.00

8 4.00 6 3.00

9 2.50 8 0.50

10 3.00 9 0.00

Average 4.20* 7.75* 1.65*

Multimed Tools Appl

4.1 Problem definition

Combining contexts is an important challenge in our recommendation model. Our researchproblem is to identify different contextual dimensions and deliver media content that best fitsthe user’s detected context. Suppose we have a group of users who share music content. Auser’s information, including their history of rating and their textual annotations given tocertain tracks describing their experience as “love”, “relaxing”, “for gym”, etc., are available.We also have other kinds of users who have not previously recorded any annotations towardany music or have not shared their ratings publicly on the internet.

For a list of possible detected conditions represented as context dimensions CN, whereCN={n1,n2,n3,…n|N|}, a list of users U={u1, u2,…,u|U|}, a list of context parameters C={c1,c2,…,c|C|} - extracted from CN- and a list of available item I={i1, i2,…,i|I|}, a recommendationmodel that predicts the suitability or interest for user (u) in item (i) given a set of context (c)can be built. Then, the context attributes are used in the recommendation process to filter anduncover items that are probably of interest to the user in such a context, personalized accordingto the user’s preferences.

4.2 Constructing the required matrices

To build the required matrices needed for our recommendation model, we use the exampleillustrated in Fig. 4. As shown in Fig. 4, we have a list of usersU={u1, u2,…,u5}, a list of itemsI={i1, i2,…,i12}, and a list of contexts C={c1,c2,…,c5}. Then, we construct three mainmatrices to build the base for our recommendation model as follows: context-user matrixT|C|×|U|, user-item matrix R|U|×|I|, and context-item matrix E|C|×|I|.

To build the matrix T|C|×|U|, let t(cy,ux) be the number of times a user ux consumed items incontext cy. Similarly, in the matrix E|C|×|I|, let e(cy,ix); be the number of times item ix has beenconsumed in context cy. If a user has not consumed any items in a given context, or if an itemhas never been consumed by any user in a particular context, then the t(cy,ux) and e(cy,ix)=0respectively. In addition, if we only consider the frequency of usage for a particular contextwithin the users or items scope, then the accuracy of the recommendation results might beaffected by the number of users who repeatedly use items in a large variety of contexts.Accordingly, we would neglect the importance of how many users have consumed items

Table 3 A summary of the nota-tions meaning Notations Meaning

U Sets of users.

I Sets of items

C Sets of contexts

E|C|×|I| Context-item matrix

T|C|×|U| Context-user matrix

R|U|×|I| User-item matrix.

T Normalized matrix T.

E Normalized matrix E.

S|U|×|U| User-user similarity matrix.

B|I|×|I| Item-item similarity matrix.

Sk Top k most similar users.

Bk Top k most similar items.

Multimed Tools Appl

within that context as the opposite of small number of users who consumed many items in aparticular context. Therefore, we normalized the frequency values in a range between 0 and 1by the following formulas:

t cy; ux� � ¼ ncu cy; ux

� �Ncy;u

� �ð1Þ

e cy; ix� � ¼ nci cy; ix

� �Ncy;i

� �ð2Þ

where ncu(cy,ux) is the number of occurrences of context cy in the list of consumed items by ux,nci(cy,ix) is the number of occurrences of context cy in the list of contexts an item has beenconsumed in (fy,x) as in Eq. 3. Ncy,u and Ncy,i represent the number of times the context cy isused with all items, and the number of times the context cy is used by all users, respectively.

nci cy; ix� � ¼X

y;x

δy; x f y; x� �

δy;x ¼ 1 cy occured in ix0 otherwise

� ð3Þ

Note that, we also examined the usage of a binary version of values for T and E, but in thiscase, we would not be able to show how often a particular item is being used in a specificcontext. That’s because in the binary case, we would only be able to know that an item hasbeen consumed in that specific context.

The construction of the three matrices (T, R, and E) leads to the discovery of the latentassociation of items toward a particular context, and the latent association of users towardcontexts, and accordingly, leverage relevant items for a user in a particular context.

5 The recommendation algorithm

In this section, we present the algorithm used to construct a context-aware recommendationmodel. The inceptive idea is that users sharing an item such as music are likely to also share

u1

u2 u3

u4

i11

i11

i8i12

i1

i4i13

i5 i10

i9i7

i2

i6

c1

c1

c1

c1

c5 c2

c3

c1

c3

c4

c5

c2c4 c5 c2c5

c3

c2

c4

c3

u5 Items consumed incontext ( )c3

item

Context of use

user

Fig. 4 A representation of the three dimensions of the recommendation problem

Multimed Tools Appl

some hidden contextual information. Such contextual information is able to effectivelydescribe the user’s preferences toward their selected items. The analysis of the availablecontextual information associated with consumed items enables the analysis of items con-sumed in similar contexts. In general, users who consume certain items in a given list ofcontexts are more likely to form a contextual pattern to bridge the information gaps betweenusers and new items.

5.1 User-based collaborative filtering

Before we analyze the user’s context, using collaborative filtering technique [5], the proposedrecommendation algorithm identifies the user’s neighbors. The concept behind relying on thedetection of similar users who share some items is to exploit the list of items consumed bygiven users to find other interesting items consumed by similar users (also called nearestneighbors). To determine the similarity between two users, we used cosine-based similarity.The cosine-based similarity is a widely used approach that takes two vectors of shared items ofuser ux, and uy, and quantifies their similarity according to their angle, as in Eq. 4. To minimizethe computational cost, we consider top k nearest neighbors for each user. Accordingly, weeliminate the computed similarities of those users who share few items with others, and assigna zero similarity value if the similar user is not among the top k nearest neighbors. We employthe matrix (S), where S=Sk, to form the user-user similarity matrix. Figure 5 shows an exampleof constructing the similarity matrix S.

sux ;uy ¼ cos ux; uy� � ¼ ux :uy

j uxj jj2:j uy j2

!ð4Þ

5.2 Item-based collaborative filtering

As we employ collaborative filtering to observe the user-user similarities, we employthe user-item matrix R to observe the item-item similarities. According to [10], theitem similarity can be computed using collaborative filtering. The idea here is that auser is likely to consume items that are similar to some of what they have alreadyseen in the past. The similarity values can be obtained by measuring the cosine angle

User-user similarity matrix(S)

ux,i1

user

uy,i1

:. :.

...

ux,i|I|

similarity

item

:. :.

... ...

...

...

...

ux,ik

uy,ik

ux

user

uy

:.

...

:.

...

...

...

user

ux...

...

...

...

uy

sx,y

User-item matrix(R)

Fig. 5 A representation of the user-user similarity matrix S

Multimed Tools Appl

between the two column vectors in the matrix R, similar to finding the user-usersimilarity according to Eq. 5.

bix ;iy ¼ cos ix; iy� � ¼ ix:iy

j ixj jj2:j iy j2

!ð5Þ

With regard to the resulted similarities, we can form the item-item similarity matrix B. Thesimilarity value of bix ;iy is only considered if it is greater than the top k nearest item neighbors,otherwise the similarity value is set to zero. Figure 6 shows an illustrated example ofcomputing the item-item similarity.

5.3 Extracting latent context-item preferences

Before recommending items to a user, we need to find the latent preferences of that usertoward their current context. This can be done by analyzing the hidden preferences of userstoward items in a given context. Accordingly, by finding the latent context-item, we capturehow a particular context has occurred with the user’s selection of items that are similar to agiven particular item. We utilize the matrix T and the transpose of matrix B, which weconstructed earlier to form the new context-item matrix Z. Formally, the matrix Z representsthe matrix multiplication results of both T and B, as in Eq. 6:

Z ¼ T Bk� �T ð6Þ

Where the matrix T denotes a normalized version of the matrix T, and the matrix (Bk)T

denotes the top k nearest items as explained in Section 4.2. The multiplication of the c-th rowby the i-th column implies finding the latent preferences of context c, on item i with respect tothe items’ k nearest neighbor. Figure 7 shows the details of constructing the new matrix Z.

The reason for having normalized values in the matrix T is to reduce the effect of items thatwere consumed by many users. Hence, the first type of items contributes more in estimatingthe context-based prediction score than the second type of items. Using normalization, we canminimize the effects of those items in regards to the detected contexts.

Item-item similarity matrix(B)

u1,ix

user

uk,ix

:. :.

similarity

item

:. :.

...

...

... u1,iy

uk,iyix

item

iy

:.

...

:.

...

...

...

itemix...

...

...

...

iy

bx,y

User-item matrix(R)

......

...

u|U|,iy

...

...

Fig. 6 A representation of the item-item similarity matrix B

Multimed Tools Appl

5.4 Extracting latent item-user preferences

In this step, we capture the user latent preferences toward an item. The main idea isthat users in a context consume certain items, and that when they are in the samecontext in the future, they will likely consume items that are either similar to theirpreferences or similar to the choice of their nearest neighbors. We denote matrix Y torepresent the latent item preferences to a given user ux, which also includes the itempreferences of his/her similar users. We build the matrix Y according to Eq. 7 asfollows:

Y ¼ R� �T

Sk� �T ð7Þ

Where R� �T

is the transpose of the original normalized rating matrix R (denoted as D) and

Sk is the top k user-user similarity matrix. The product of the two matrices R� �T

Sk� �T

bringsthe user and their nearest neighbors’ preferences to a given item.

It is also important at this point to consider the issue of having some users that aremore active in rating and consuming different items than other inactive users. Thisleads to more contributions in the recommendation model from the active userscompared to the less active users. Therefore, we normalize the values in matrix D,before the multiplication step, in order to reduce such contribution effects. Figure 8shows the details of constructing the new matrix Y.

c1,,i1 c1,i|I|

c|C|,i1 c|C|,i|I|

×

=

c1,,i1 c1,i|I|

c|C|,i1 c|C|,i|I|

context ...

...

...

...

item

...

...

...

...

(E)_

item

...

...

item

i2,,i1

ik,i1 Item-item similarity matrix(B )k

Latent context-item matrix (Z)

c ont ext ...

...

...

...item

...

...

...

...

|I| k×

|C| |I|×

T

k 1×

:.

...

:.

...

...

...

...

...

...

b1,2

i1

i1

iki2

context-item matrix

i2

i|k|

Fig. 7 An illustration of the process of computing the latent context-item matrix Z

Multimed Tools Appl

5.5 Employing the latent collaborative models for recommendation

We take into consideration the user’s previous items, their nearest neighbors, as well as thesimilarity of items that have previously been consumed in the detected context. Items with ahigh ranking score will be recommended to the user. In addition, the recommended itemsreflect the user’s context within which it is recommended. In order to compute the items’ finalrating scores, we use the two previously described models: the latent item-context model, andthe latent item-user model. The association of these two models builds the required contextualbridge between users and items. Specifically, the proposed recommendation produces itemrecommendations relevant to a given context by extracting latent preferences. The calculationof the user-item ranking score is computed by:

Ranku;c ið Þ ¼ ∑c∈m

α Zc;i � Yi;u ð8Þ

Where Zc,i is the matrix entry of the c-th context row and the i-th item column of the Zmatrix. The Yi,u is the entry value of the i-the row and the u-th column in the Y matrix. Theparameter α is an attenuation factor, where α (0…1) for reducing the weight factor of a lesssensitive context. The tuning of the α value is set after some experimental results. Details ofthe parameter α are presented in the parameter tuning of the performance evaluation inSection 7.5. Items with higher ranking scores are recommended to the user. Figure 9 presentsan illustration of the user-item ranking score computation step. In Eq. 8, we show that the

i1,,u1

i|I|,u1 i|I|,u|U|

×

=

i1,u|U|

i|I|,u|U|

item

...

...

...

...

user

...

...

...

...

(D)user

...

...

user

u2,,u1

uk,u1 User-user similarity matrix(S )k

Latent item-user matrix (Y)

ite m

...

...

...

...user

...

...

...

...

|U| k×

|I| |U|×

T

k 1×

:.

...

:.

...

...

...

...

...

...

s1,2

u1

u1

uku2

item-user matrix

u2

uk

i1,u1,

i|I|,u1

i1,u|U|

_

Fig. 8 An illustration of the process of computing the context-user matrix Y

Multimed Tools Appl

ranking score is not limited to a single context; if multiple contexts are in the user’s query, thenthe summation of the multiplication will represent the ranking score for that item. With regardto the example illustrated in Fig. 4, the final matrix does not have to be physically stored in thedatabase; its ranking score can be computed by executing a query on the two matrices Z and Yto reduce the model’s complexity.

6 Architecture and system design

6.1 Application scenario

Today’s smartphone ecosystem accelerated the development of different context-awarerecommender systems, particularly in observing the user’s contexts such as weather,location, activity, and time. Smartphones also enable enormous access to multimediacollections and act as a bridge between different sensors and the recommendationengine. Hence we developed a smartphone application that can be effectively used ina home environment. Let us assume a user who usually listens to music whilestudying and who has an exam the next morning. At home, the user used theapplication’s ECG interface to capture their heart signal. Depending on the user’s

c1ux

user context

i1,u|U|

i|I|,u|U|Latent item-user matrix (Y)

item

...

...

...

...user

...

...

...

...

i1,ux,

i|I|,ux

c1,,i1 c1,i|I|

c|C|,i1 c|C|,i|I|

Latent context-item matrix (Z)

context ...

...

...

...item

...

...

...

...

user

uy,i1

:.

...item

:.

... ...

...

ux,i3

uy,i3

Final user-item matrix

...

...

:.

:.

ux,i2

uy,i2

ci

context

ux,i1

Query

+

Items with higher ranking score

Recommended

Fig. 9 The final user-item ranking score

Multimed Tools Appl

stress level, the application will suggest relaxing music according to his/her prefer-ences. Nonetheless, as we explained in the design of the recommendation algorithm,the application explores the user’s social friends and their preferences as well. In thisscenario, the application can not only recommend music, but it can also adapt therecommendation result to fit a user’s context.

The application prototype is developed in an android environment. The applicationcan detect the date, time, location, song being played, the play count, and the heartsignal, and has the ability to customize other parameters into the collection, as shownin Fig. 10.

6.2 Design and architecture

The architecture of the introduced recommendation system consists of four mainlayers: the input/output interface layer, the context management layer, the client-local resources layer, and the server-cloud resource layer, as shown in Fig. 11. Theclient side is separated into client interfaces and user-local server. Identifying theuser’s context and all the input parameters are processed within the user-local side. Inaddition, the processing of the recommendation algorithm including all required

Fig. 10 A screenshot of the context-aware recommendation prototype interface

Multimed Tools Appl

recommendation agents is kept in user-local server. The server side has a cloud-baseddesign to store additional multimedia resources and social profiles. Figure 12 showsthe sequence of main functionality interactions for a user requesting a song. As aproof of concept, the prototype application synchronizes any media content storedwithin the user’s Dropbox account. The input/output interface layer handles thecollection of the required contextual data and interacts with the user, which includesdelivering the recommendation results. The context-management layer identifies thecontext of the user by analyzing the retrieved sensory data. The local resources layerstores the user behavior, and selects and evaluates the different recommendationparameters needed for the recommendation algorithm to function. The EntityRelationship Diagram (ERD) of the client-local resources layer is demonstrated inFig. 13. The resource contents and the available social profiles are stored in a cloud-based repository.

7 Experimental evaluation

In this section, we investigated whether the proposed recommendation technique improves theitem prediction accuracy or not. The goal of the experiments is to evaluate the accuracy of theproposed approach: the utilization of the user’s context to recommend a different number ofitems. We changed the number of items retrieved each time to measure their relevance to theuser’s request.

7.1 Dataset

To find a publicly available dataset that carries some contextual information is acrucial challenge. Such lack of availability challenges any design of a context-awarerecommendation algorithm [40]. Therefore, we crawled our dataset from an onlinesocial music database: last.fm. Specifically, music information and annotations dataare extracted from the last.fm website. Last.fm is an online social music radioresource that enables their users to subscribe, listen and tag their favorite albums,tracks, and artists. The crawled dataset contains 164 users, 626 tracks, 251 contextualtags, and 10,711 overall item-context assignments. Users of last.fm annotate differentalbums and tracks with textual tags. Due to the fact that users can give any textualdescription to their favorite tracks and albums, different words can be used to describethe same meaning. For instance, four different users may tag item i1 with fourdifferent tags with the same meaning: “relaxing music”, “for relax”, “relax”, and“relaxation”. These tags can be grouped together under one annotation: “relaxing”.

We also tested the performance of our proposed system on a bigger dataset. Weused the publicly available dataset from MovieLens1 (www.movielens.org), which wasused in [15]. The MovieLens dataset consists of 943 users, 1,682 movies, and100,000 user-item ratings. This dataset does not have explicit contextual data, butcontains some information that we consider a context in our experiment for a proof ofconcept experiment: genre (romance, action, adventure etc.), and user’s information(age, gender, occupation and zip code).

1 The MovieLens dataset can be downloaded from: http://www.grouplens.org/node/73.

Multimed Tools Appl

7.2 Comparison with other methods

In our evaluation, we present the detailed experimental results of our ranking method incomparison with other benchmark methods.

& Popular Items (PopItems): As described in [19], the PopItems technique gives moreprediction weight to items with a higher count value within a specific context cy.

& Item Rank (ItemRank): This technique is a random walk scoring algorithm proposed in[12]. Using a user-item relationship graph, the algorithm estimates the probability of user uvisiting item i in a random walk.

& uMender: This recommendation technique is proposed by J. Su et al. [33]. The algorithmsfirst create a sub-matrix to find users and items that are similar in the same contextcondition. Then the algorithm obtains the positive and the negative preferences based onthe available rating values. Finally, the algorithm finds frequencies in the negative andpositive item sets and computes the related user-item prediction.

Evaluating contextual and jointpreferences

History

Multimedia Contents

Client-Local Resources Layer

CloudManager

Web Services

User

Input/output interfacesLayer

Biosensors

User Profile

EvaluatingInputs

Context Management Layer

Identifycontext

Context matching

Profile exploitation

ContextualLog

Elevated Stress State

Neutral State

Relaxed State

Time, Date

Social

Location

Physiological

Social ProfilesServer-Cloud Resources Layer

Recommendation Engine

Service

Fig. 11 The architecture of the context-aware recommendation model

Multimed Tools Appl

& Collaborative and Content-based Technique (CCbT): Lops et al. [24] proposed a collab-orative content-based tag recommendation algorithm. Since we are dealing with context byutilizing social tags, we considered comparing this technique with our proposed modelsince the tags annotation is also analyzed here.

7.3 Evaluation methodology

The evaluation procedure is divided into two parts: the first part measures the accuracy of thecontext-based recommendation prediction, using offline experiments on different datasetscrawled from online multimedia databases; the second part evaluates the user’s satisfactionwith the resulted context-based recommendations after using the proposed prototypeapplications.

7.3.1 Offline experiment

For the experiments in this section, we similarly follow the experimental procedure proposedin [41]. We randomly divided the two datasets into two groups: a training set that represents80 % of the original dataset, and the remaining 20 % used as a test set. The training set is usedto train the recommendation model, while the test set has items withheld randomly and theirassociated contexts are used as test-queries for each user. Since the algorithm performancemight be sensitive to particular items chosen in the training or the test set, we repeated the runof the algorithms 5 times with different portioning. In addition, the values represented in theexperimental results section represent the average of those different 5 runs with the standarddeviation.

User

Request a song

Client-Local Handler

Evaluate inputs

submit identified context, user profile

context/Item Analyzer

context matching

Recommender Engine

Identified current andprevious contextual

parametersMatching preferences

Client Application Server

Profile and resourcesexploitation

Server-Cloud Handler

Required resources

items ranking scores

list of recommendations identified top-k relevant items

Compute rankingscores

Context Provider

GetCurrentContext

current context

Fig. 12 Interaction diagram of a user requesting a song using the proposed application

Multimed Tools Appl

7.3.2 Online experiment

In addition to the offline dataset experiments, we also conducted a subjective user evaluationusing the android prototype application mentioned earlier. Providing users with some contex-tual knowledge, we computed the suitable items to be recommended according to the givencontext and displayed them to the user. In order to measure the effectiveness of our approach,the recommendation algorithm presents the recommended items retrieved using anotheralgorithm that is non-personalized to the user’s context. We used the collaborative filteringalgorithm presented in [31] as the second recommendation technique used in the application.The goal behind this experiment is to know whether or not our context-aware predictionmethod can capture items more relevant to the user’s preferences than the non-context-awaretechnique.

Each user in the online experiment is asked to evaluate the ten pieces of music recom-mended in a certain context. Subjects go through different context scenarios, thus they evaluatethe recommendation for different context conditions. Subjects browse the recommended musicand answer if they like or dislike the music in such a context.

7.4 Evaluation metrics

To measure the algorithm’s retrieval accuracy, we adopted precision and recall, which arewidely used evaluation parameters to measure the effectiveness of the retrieved

Fig. 13 Client-Local Resources Layer Database ERD

Multimed Tools Appl

recommendations in our offline experiment. Precision can be calculated by finding the ratio ofthe recommended items to the items already identified as relevant to the user, as in Eq. 9 [35].Recall can be calculated by finding the amount of relevant contents among all recommendedcontents as in Eq. 10 [35].

precision ¼ recommended∩ relevant

recommendedð9Þ

recall ¼ recommended∩ relevant

relevantð10Þ

For the comparison of our method to the other four benchmark algorithms, we report theMean Average Precision (MAP) by using Eq. 11, in addition to precision and recall.

MAP ¼ 1

Uj jXUj j

u¼1

1

tu

Xn¼1

tu

Pn � Rn ð11Þ

Where tu is the number of test cases for user u, and Pn is the precision at top n and Rn is abinary variable that equals to 1 if the item is relevant at rank n [16]. The MAP reports theaverage precision at each top k result [39]. Note that we varied the number of items retrieved(top k values) to measure the ranking positions of each recommended item. For instance, theprecision values are reported for each top k (k=1, k=5, and k=10), which show the number ofrelevant items at top 1, top 5, and top 10.

7.5 Parameter tuning

The proposed recommendation model uses α, an attenuation factor, where α ∈ (0…1) is usedto reduce the weight factor of the contextual effects on the prediction scores. Prior to startingthe experiments, we gave α equal values in all contexts in order to run an empirical study. Bytuning this parameter, we may increase or decrease the influence of the context on the finalscoring value given to an item. Hence, it is critical to correctly set the value of α to improve therecommendation performance.

We first measure the MAP, and the Mean Reciprocal Rank (MRR) using different values ofα on the Last.fm dataset and then on the Movielens dataset, as shown Table 4. The MRRmeasure is computed using Eq. 12:

MRR k¼nð Þ ¼ 1

Uj jXUj j

u¼1

∑i∈ tu∩Rnuð Þ

1

r ið Þ� �

ð12Þ

Where tu is the test case for user u, Run is the top n returned records, and the value r(i) ranges

between: 1≤r(i)≤n. The results in Table 4 show the best MAP andMRR values obtained whenα=0.3 on Last.fm dataset and α=0.5 on Movielens dataset.

7.6 The effect of normalization

Prior to running the comparison experiments, we investigated the impact of matrix normali-zation on the evaluation metrics. We measured MAP and MRR after running the algorithm onnormalized and non-normalized matrices. Specifically, we employed two different versions ofthe context-user matrix D and context-item matrix E. Tables 5 to 6 reports the improvement on

Multimed Tools Appl

MAP and MRR using normalized matrices over non-normalized matrices of D and E in eachdataset. In addition, we conducted a statistical analysis to measure the significance of theimprovement of the normalized approach over the non-normalized one. Specifically, weconducted the two-tailed paired t-tests under the same conditions of D and E and for thesame dataset. The results confirmed that the normalized approach positively impacted thesignificance for both MAP and MRR values Table 6.

7.7 Experimental results

7.7.1 Offline experiment

In this section, we present the results of the comparison between the performance of ourapproach and the recommendation techniques introduced in Section 7.2. As explained inSection 7.3, we computed the performance of each recommendation approach in retrievingaccurately relevant items, as well as their ranking positions in the recommendation list. Firstly,we evaluated the recommendation performance by calculating precision and recall, obtainedby our approach and the other four alternative approaches, on Last.fm and Movielens datasets,as in Figs. 14 and 15.

Figures 14 and 15 depict the precision-recall curves showing how our proposed approachoutperforms the other baseline algorithms on both datasets. The number of retrieved items for auser’s quest is plotted on data points of the graph curves; the curves start from left denoting thetop k=1, whereas the last point on the right denotes the top k=10. Our approach obtainedapproximately a 2 %, 3 %, 10 %, and 12 % precision improvement on the Last.fm datasetcompared to CCbT, uMender, ItemRank, and PopItems respectively. Our approach also

Table 5 Effect of normalization on the Last.fm dataset

Non-Normalization Normalization

D, E MAP@10 MRR@10 MAP@10 MRR@10

25 0.118±0.009 0.288±0.001 0.129±0.007** 0.294±0.008**

50 0.125±0.008 0.311±0.010 0.138±0.008** 0.299±0.009*

100 0.166±0.019 0.385±0.007 0.149±0.007* 0.302±0.004**

150 0.179±0.004 0.402±0.008 0.169±0.016** 0.335±0.008*

All 0.186±0.002 0.422±0.001 0.199±0.016** 0.579±0.004**

* Significant at p<0.01, ** Significant at p<0.001

Table 4 The different weightsassigned to α to measure its sensi-tivity in each dataset

Dataset Number of Items

α MAP@10 MRR@20

Last.fm 0.1 0.120±0.007 0.214±0.008

0.3 0.225±0.016 0.584±0.004

0.5 0.153±0.004 0.412±0.008

Movielens 0.1 0.131±0.008 0.177±0.007

0.3 0.106±0.011 0.325±0.016

0.5 0.152±0.011 0.369±0.09

Multimed Tools Appl

obtained approximately a 3 %, 1 %, 3 %, and 6 % precision improvement on Movielensdataset compared to CCbT, uMender, ItemRank, and PopItems respectively.

When comparing the reported results, we observed that finding latent assignments ofcontext to items and latent relations of users towards contexts reveals more relevant itemsthan non-context-aware approaches. We also noticed that our approach is relatively affected bythe number of context parameters attached to an item, which indicates that the more usersconsume an item in different contexts, the better the recommendation would be.

We continue to determine if the types of users in the dataset affect the sensitivity of therecommendation algorithm. The reason is that in both datasets there are users who only rated afew items in different contexts, as well as other users who rated many items. Accordingly, weinvestigated the size of each user’s rating history, which is used for context discovery. Then,we divided the users in the datasets into three different groups denoted as Active, Normal, andPassive users. Each user is assigned to one of these three groups depending on the number ofrated items. If a user rated at least 11 items or more, they are considered to be active users, ifthey rated from five to ten items, they are considered normal users. Passive users or (cold startusers) are those who rated less than 5 items. We measure the algorithm’s sensitivity to thenumber of items rated per user. The resulted performances for each recommendation approachin the two datasets are shown in Figs. 16 and 17. As we expected, all the algorithms weresensitive to the number of items rated by each user. Particularly, the recommendation accuracy

Table 6 Effect of normalization on the Movielens dataset

Non-Normalization Normalization

D, E MAP@10 MRR@10 MAP@10 MRR@10

25 0.0915±0.003 0.158±0.002 0.159±0.001* 0.169±0.001*

50 0.119±0.004 0.160±0.009 0.174±0.001** 0.197±0.007*

100 0.152±0.008 0.177±0.007 0.198±0.003* 0.201±0.006*

150 0.189±0.006 0.225±0.0012 0.239±0.001* 0.267±0.003**

All 0.202±0.002 0.281±0.014 0.286±0.001** 0.318±0.014**

* Significant at p<0.05, ** Significant at p<0.01

0%

5%

10%

15%

20%

25%

0% 10% 20% 30% 40% 50% 60%

Prec

ision

Recall

Our Approach

CCbT

uMender

IRank

PopItem

Fig. 14 Precision and recall obtained by the 5 algorithms on the Last.fm dataset

Multimed Tools Appl

increased when the number of items rated by a user increased. Note also that all algorithmsachieved considerably lower MAP for the cold-start problem, due to the fact that thealgorithms do not have enough history to feed the recommendation process. However,detecting the user’s context indeed facilitates the improvement of the recommendations insuch cold start cases.

7.7.2 Online experiment

To provide insight into the performance of our approach on real users, we conductedan online experiment on invited subjects. According to the experimental setup intro-duced in Section 7.3.2, we invited 15 subjects to participate in evaluating our context-aware recommender prototype application. The subjects were adult, 7 males and 8females. The average of their age was 23.2 years. To eliminate the effects of othercontextual parameters such as the user’s age, education level, mother language,culture, and other; we carefully selected participants that shared the same parametersthat are not included as a context in our application. The experiment was performedoutside the laboratory and each user was given an android phone to use. All phonesneeded access to the Internet in order to save the data collected on our server cloud.

0%

5%

10%

15%

20%

25%

0% 10% 20% 30% 40% 50% 60% 70%

Prec

ision

Recall

Our ApproachCCbTuMenderIRankPopItem

Fig. 15 Precision and recall obtained by the 5 algorithms on the Movielens dataset

0.0987 0.0999

0.12365

0

0.05

0.1

0.15

PU NU AU

MAP

User's groupsPopItem IRank uMender CCbT Our Approach

Fig. 16 A comparison of MAP according to the variation number of items rated by a user in the Last.fm dataset

Multimed Tools Appl

We used a smaller online crawled dataset, which contained 40 popular artists and 419musical tracks. Then, we asked the subjects to rate at least 20 tracks and tag eachtrack with some contextual tags according to their preferences. From these surveys,we obtained from 343 item ratings and 1,747 context-item associations. Each subjectwas asked to listen to a track for one minute and then tag it with single or multiplewords. We followed the same evaluation protocol that was used in similar studiessuch as the one by Cai et al. [6]. The collected rating and contextual data are used asground truth dataset for the proposed model to generate a recommendation list for agiven invited subject. Afterwards, for each user, we detected six different scenarios ofcontext and ran the recommendation algorithm to produce 10 recommended itemssuitable for that particular context. Three of the scenarios reflect the three physiolog-ical context dimension introduced in Section 3. We repeated asking the subjects toperform three types of activities (perform a Stroop color-word test, sit comfortablyand try to read an article, and sit comfortably, close their eyes, and listen to relaxingmusic). After performing each activity, subjects were asked to provide their feedbackabout the resulted recommendation list. As for the rest of the three other contextualscenarios, the application detects them based on the information available as well asthe information provided by the user. For instance, date, time, weather, are detectedright away by the android application while the user has to specify whether he/she isalone, with a partner, or with the family member. Subjects can optionally select acontextual condition such as: studying for an exam, or having a romantic date, etc.Based on the information collected, the application chooses randomly three differentcontexts to as a test query for evaluation.

Users could listen to each recommended item and answer whether they liked ordisliked the recommended music in such a context. As a result, we received 900responses telling us whether a user ux liked or disliked the retrieved item iy in a givencontext cz for each recommendation algorithm. We then computed the average preci-sion of a user obtained from the 6 different contextual scenarios. The results arebriefly summarized in Fig. 18. In addition, we conducted a statistical analysis tomeasure the significance of the improvement of our approach using the two-tailedpaired t-test. Our approach achieved statistical significant at 1 % level where p<0.01over the non-context-aware approach.

0.15030.1698

0.2135

0

0.05

0.1

0.15

0.2

0.25

PU NU AU

MAP

User's groupsPopItem IRank uMender CCbT Our Approach

Fig. 17 A comparison of MAP according to the variation number of items rated by a user in the Movielensdataset

Multimed Tools Appl

We further analyzed the subjects’ evaluation of each recommendation list in regard to thethree physiological contextual queries. We conducted a two-tailed paired t-test on the re-sponses of subjects in relaxed, stressed and neutral situations. Both tests on relaxed and neutralconditions agreed that there are certain preferences of users toward these two conditions.However, there is no significant differences with the resulted obtained in the neutral conditiontest. Hence, we conclude that other contextual information should be included in the querywhere the physiological condition of the subject does not differ from their normal benchmarks.

8 Conclusion

Sensors attached to smartphones are becoming widely used to support the interactionsbetween the user and the different context-aware recommendation systems. In thispaper, we identify the user’s context and explore the use of physiological data toenhance the recommendation process. We also demonstrate the importance of usingcontextual information to provide enhanced recommendation quality and to increasethe level of interactions between the user and their preferred multimedia contents.Moreover, the advantage of our proposed recommendation model is that it considersthe contextual information by reflecting the online available social tags to explore thelatent contexts assigned to items, as well as applying CF to find latent contextspreferences from similar users. Additionally, the proposed model can search and rankitems without the need to analyze the item’s content, such as analyzing the musiclyrics or voice signal, to predict the associated context. The experimental resultsdemonstrate that the proposed context-aware recommendation technique offers favor-able advantages in enhancing the accuracy of the prediction and providing suitableitems for the user’s context.

To improve the recommendation quality, in future work we will address the issue ofproviding recommendations to a group of users rather than only to individuals. Additionally,we intend to investigate the proposed recommendation approach with additional contextparameters in larger datasets. For instance, we can extend this work for different type ofresources such as movies, books, and news.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Non context-aware Context-awarePr

ecisi

on

Subjects

Fig. 18 Average precision of recommendation retrieval

Multimed Tools Appl

References

1. Al Osman H, Eid M, and El Saddik A (2013) U-biofeedback: a multimedia-based reference model forubiquitous biofeedback systems. Multimed. Tools Appl

2. Alhamid M F, Rawashdeh M, Osman H Al, and El Saddik A (2013) Leveraging biosignal and collaborativefiltering for context-aware recommendation. In Proceedings of the 1st ACM international workshop onMultimedia indexing and information retrieval for healthcare. ACM

3. Bernardi L, Wdowczyk-Szulc J, Valenti C, Castoldi S, Passino C, Spadacini G, Sleight P (2000) Effects ofcontrolled breathing, mental activity and mental stress with or without verbalization on heart rate variability.J Am Coll Cardiol 35(6):1462–9

4. Bogers T (2010) Movie recommendation using random walks over the contextual graph. In Proc. of the 2ndIntl. Workshop on Context-Aware Recommender Systems

5. Breese J, Heckerman D, and Kadie C (1998) Empirical analysis of predictive algorithms for collaborativefiltering. Proc. Fourteenth Annu. Conf. Uncertain. Artifcial Intell, pp. 43–52

6. Cai R, Zhang C, Wang C, Zhang L, and Ma W (2007) MusicSense: Contextual Music Recommendationusing Emotional Allocation Modeling. In Proc. MULTIMEDIA’07 Proceedings of the 15th internationalconference on Multimedia, pp. 553–556

7. Camm DHSAJ, Malik M, Bigger JT, Breithardt G, Cerutti S, Cohen RJ, Coumel P, Fallen EL, Kennedy HL,Kleiger RE, Lombardi F, Malliani A, Moss AJ, Rottman JN, Schmidt G, Schwartz PJ (1996) Heart ratevariability: standards of measurement, physiological interpretation and clinical use. Task force of theEuropean society of cardiology and the North American society of pacing and electrophysiology.Circulation 93(5):1043–1065

8. Colombo R, Mazzuero G, Sofffiantino F, Ardizzoia M, Minuco G (1989) A comprehensive PC solution to heartrate variability analysis in mental stress. Comput. Cardiol. 1989, Proceedings. IEEE, Jerusalem, pp. 475–478

9. de Campos LM, Fernández-Luna JM, Huete JF, Rueda-Morales MA (2010) Combining content-based andcollaborative recommendations: a hybrid approach based on Bayesian networks. Int J Approx Reason 51(7):785–799

10. Deshpande M, Karypis G (2004) Item-based top- N recommendation algorithms. ACM Trans Inf Syst 22(1):143–177

11. Egelund N (1982) Spectral analysis of heart rate variability as an indicator of driver fatigue. Ergonomics25(7):663–72

12. Gori M, Pucci A, Roma V, and Siena I (2007) Item rank: A random-walk based scoring algorithm forrecommender engines. In the 20th international joint conference on Artificial Intelligence, pp 2766–2771

13. Han B, Rho S, Jun S, Hwang E (2009) Music emotion classification and context-based music recommen-dation. Multimed Tools Appl 47(3):433–460

14. Healey JA, Picard RW (2005) Detecting stress during real-world driving tasks using physiological sensors.IEEE Trans Intell Transp Syst 6(2):156–166

15. Herlocker J and Konstan J (1999) An algorithmic framework for performing collaborative filtering. Proc.22nd Annu. Int. ACM SIGIR Conf. Res. Dev. Inf. retrieval. ACM

16. Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommendersystems. ACM Trans Inf Syst 22(1):5–53

17. Hjortskov N, Rissén D, Blangsted AK, Fallentin N, Lundberg U, Søgaard K (2004) The effect ofmental stress on heart rate variability and blood pressure during computer work. Eur J ApplPhysiol 92(1–2):84–9

18. Hyung Z, Lee K, and Lee K (2013) Music recommendation using text analysis on song requests to radiostations. Expert Syst. Appl, no. October

19. Jäschke R, Marinho L, Hotho A, Schmidt-Thieme L, Stumme G (2008) Tag recommendations in socialbookmarking systems. AI Commun 21(4):231–247

20. Kim J, Lee D, and Chung K.-Y (2014) Item recommendation based on context-aware model for personalizedu-healthcare service. Multimed Tools Appl 71(2):855–872

21. Lattimore PJ (2001) Stress-induced eating: an alternative method for inducing ego-threatening stress.Appetite 36(2):187–8

22. Lekakos G, Caravelas P (2006) A hybrid approach for movie recommendation. Multimed ToolsAppl 36(1–2):55–70

23. Liu H, Hu J, and Rauterberg M (2010) iHeartrate: a heart rate controlled in-flight music recommendationsystem. Proc. 7th Int Conf Methods Tech. Behav. Res. ACM

24. Lops P, de Gemmis M, Semeraro G, Musto C, Narducci F (2012) Content-based and collaborativetechniques for tag recommendation: an empirical evaluation. J Intell Inf Syst 40(1):41–61

25. Narducci F, Snape WJ, Battle WM, London RL, Cohen S (1985) Increased colonic motility during exposureto a stressful situation. Dig Dis Sci 30(1):40–4

Multimed Tools Appl

26. Nasoz F, Lisetti CL, Vasilakos AV (2010) Affectively intelligent and adaptive car interfaces. Inf Sci (Ny)180(20):3817–3836

27. Nirjon S, Dickerson R, Li Q, Asare P, Stankovic J, Hong D, Zhang B, Shen G, Jiang X, and Zhao F (2012)MusicalHeart: A Hearty Way of Listening to Music. ACM, Sensys

28. Oliver N and Flores-Mangas F (2006) MPTrain: a mobile, music and physiology-based personal trainer.Proc. 8th Conf. Human-Comput. Interact. Mob. Devices Serv. (Mobile HCI 2006)

29. Pessemier T, Dooms S, and Martens L (2013) Context-aware recommendations through context and activityrecognition in a mobile environment. Multimed. Tools Appl., Jul

30. Salahuddin L, Cho J, Jeong MG, Kim D (2007) Ultra short term analysis of heart rate variability formonitoring mental stress in mobile settings. Conf Proc IEEE Eng Med Biol Soc 2007:4656–9

31. Shin D, Lee J, Yeon J, and Lee S (2009) Context-Aware Recommendation by Aggregating User Context.Commerce and Enterprise Computing, 2009. CEC'09. IEEE, Vienna, pp 423–430

32. Siska E (2002) The stroop colour-word test in psychology and biomedicine. Univ Palacki OlomucGYMNICA 32(1):45–52

33. Su J, Yeh H, Yu P, Tseng V (2010) Music recommendation using content and context information mining.Intell Syst IEEE 25(1):16–26

34. Tulen JH, Moleman P, van Steenis HG, Boomsma F (1989) Characterization of stress reactions to the stroopcolor word test. Pharmacol Biochem Behav 32(1):9–15

35. van Rijsbergen CJ (1979) Information retrieval, 2nd edn. Butterworths, London36. Woerndl W, Schueller C, Wojtech R, and Gmbh U (2007) A Hybrid Recommender System for Context-

aware Recommendations of Mobile Applications, Data Eng. Work. 2007 I.E. 23rd. Int Conf on, pp. 871–87837. Yeung KF (2011) A context-aware framework for personalized recommendation in mobile environments.

University of Portsmouth, Portsmouth, United Kngdom38. Yoon K, Lee J, Kim M (2012) Music recommendation system using emotion triggering low-level features.

Consum Electron IEEE Trans 58(2):612–61839. Yu K, Zhang B, Zhu H (2012) Towards Personalized Context-Aware Recommendation by Mining Context

Logs. Adv Knowl Discov Data Mining, Lect Notes Comput Sci 7301:431–44340. Yujie Z and Licai W (2010) Some challenges for context-aware recommender systems. 2010 5th. Int Conf

Comput Sci Educ, pp. 362–36541. Zanardi V and Capra L (2008) Social ranking: uncovering relevant content using tag-based recommender

systems. In Proceeding of 2008 ACM conference on Recommender systems, pp 51–5842. Zangerle E, Gassler W, and Specht G (2012) Exploiting Twitter’s Collective Knowledge for Music

Recommendations. Proc. WWW Work. #MSM2012 (2nd Work. Mak. Sense Microposts), pp. 14–17

Mohammed F. Alhamid Received his M.A.Sc in Computer Science from University of Ottawa, Ottawa,Ontario, Canada in 2010. He is currently working towards his PhD degree with the Multimedia CommunicationResearch Laboratory (MCRLab), School of Electrical Engineering and Computer Science (EECS) at theUniversity of Ottawa. His research interests include ambient intelligence, service-oriented architectures, socialmedia mining and filtering, and recommender systems.

Multimed Tools Appl

Majdi Rawashdeh Received his Bachelor degree in Computer Science from Yarmouk University, Jordan in1994. He then received his master degree in Computer Science from University of Ottawa, Canada in 2009. Healso received his PhD degree in Computer Science with the Multimedia Communication Research Laboratory(MCRLab), School of Electrical Engineering and Computer Science (SEECS) at the University of Ottawa. Hisresearch interests include Social media analysis, user modeling, and recommender systems.

Hussein Al Osman Is an Assistant Professor in Software Engineering at the School of Electrical Engineering andComputer Science at the University of Ottawa. He completed his B.A.Sc in Computer Engineering with highestdistinction (Summa Cum Laude) in 2007, M.A.Sc in Electrical Engineering in 2009 and Ph.D. in ElectricalEngineering in 2014, all at the University of Ottawa. Over the course of his academic journey, he has receivednumerous scholarships and awards (3X NSERC Scholarships, Queen Elizabeth II Graduate Scholarship, PartTime Professor Award…). His current research interests include: health informatics, serious games for health,affective computing, human computer interaction and active biometrics.

M. Shamim Hossain Is an Associate Professor of CCIS, at King Saud University, Riyadh, KSA. Dr. ShamimHossain received his Ph.D. degree in Electrical and Computer Engineering from the University of Ottawa,Canada. His research interests include serious games, cloud and multimedia for healthcare, big data formultimedia, social media, and biologically inspired approach for multimedia and software system. He hasauthored and co-authored more than 65 publications including refereed IEEE/ACM/Springer/Elsevier journals,conference papers, books, and book chapters. He has served as a member of the organizing and technicalcommittees of several international conferences and workshops. Recently, he received outstanding paper award

Multimed Tools Appl

from an IEEE Conference. He served as a co-chair of the 1st, 2nd, 3rd and 4th IEEE ICME workshop onMultimedia Services and Tools for E-health MUST-EH 2011, MUST-EH 2012, MUST-EH 2013 and MUST-EH2014. He served as a co-chair of the 1st workshop on Cloud-Based Multimedia Services and Tools for E-health(CBMST-EH 2012) with ACM Multimedia 2012 (ACM MM’12). He serves/served as a guest editor of IEEETransactions on Information Technology in Biomedicine, Springer Multimedia tools and Applications (MTAP),Springer Cluster Computing and Hindawi International Journal of Distributed Sensor Networks. Dr. Shamim is aSenior Member of IEEE and a member of ACM.

Abdulmotaleb El Saddik Is Distinguished University Professor and University Research Chair in the School ofElectrical Engineering and Computer Science at the University of Ottawa. He is an internationally-recognizedscholar who has made strong contributions to the knowledge and understanding of multimedia computing,communications and applications.

He has authored and co-authored four books and more than 450 publications. Chaired more than 40conferences and workshop and has received research grants and contracts totaling more than $18 Mio. He hassupervised more than 100 researchers. He received several international awards among others ACM Distin-guished Scientist, Fellow of the Engineering Institute of Canada, and Fellow of the Canadian Academy ofEngineers and Fellow of IEEE and IEEE Canada Computer Medal.

Multimed Tools Appl