10
“Improvise - Creating playlists according the user activity” Ricardo Antunes Lopes da Cunha [email protected] Instituto Superior T´ ecnico, Lisboa, Portugal May 2014 Abstract The number of songs available in Internet has grown steadily since its appearance. As a result of this growth, it is extremely difficult for users to find the appropriate music that suit their needs. To allow users to listen to the music they like, two techniques are quite used, searching and recommendation. While recommender systems rely more actively on the users needs, the search engines receive their needs more passively. In the present work we developed a solution for music recommendation, whose objective is to search and recommend songs in tune with the activity that the user is performing. With this objective in mind, we developed a hybrid approach reflecting the integration of a recommendation system based on context and content. Regarding context, we use information about the user’s activity. In what concerns the content, we took into account the information about the acoustic features of each song. The recommendation algorithm starts by recommending the same songs for all users, we call this the generic model. The user’s interaction with the system leads to the adjustment of the generic model, to its own musical preferences changing it into a personalized model. A personalized recommendation model for each user will then be obtained after some interactions with the system, which will better reflect their tastes. Keywords: Music Recommendation System, Hybrid recommendation, Recommendation based on con- text, User activity, Association between music and activities, Generic and personalized model 1. Introduction Nowadays there are numerous resources at our dis- posal for the consumption of digital music, whether through online stores or streaming services. Thus we have at our disposal thousands of songs that can be listen anywhere, anytime, and on multiple devices like MP3 players, smartphones and tablets. However, with the access to such a vast collec- tion of songs, the main problem arises when the user wants to choose songs he wants to listen. This prob- lem is even worse when trying to create a playlist to listen to while performing a certain task, such as practicing sports, driving or studying. So creating playlists becomes a tiresome and frustrating process to users. To solve this problem in the past years several proposals have been developed for the generation and creation of playlists. In this work we describe and analyze the main techniques used in these ap- proaches, which include the use of recommendation methods like collaborative filtering, content-based, context-based and hybrid approaches. Despite the good results that such techniques can achieve, they cannot be adapted to the specific ac- tivity the user is performing. In fact, this topic has not yet been well explored, therefore an effective so- lution for this type of recommendation is missing. With this problem in mind we aim to design a solution for playlist recommendation based on the activity the user is performing (context). In addi- tion to the activity performed by the user our goal is to use information about the songs (content) to help in the recommendation process, to detect a possible parallelism between music features and activities. Our solution is based on a hybrid recommenda- tion system that considers the context in which the music is being listen to and the musical features of each song. Initially the same set of songs is recommended to all users. This we call the generic model where each song is recommended taking into account some of the tests performed with real users in order to re- alize the songs predominantely preferred for each activity they perform. The songs are recommended taking into account a set of intervals that define how music should be recommended for each activ- ity. These intervals represent the musical features of each song. With the interaction of the user’s with the generic model, this is adapted to a personalized model that takes in account the previously choices made by the user’s. In the next section we discuss the related work, 1

Improvise - Creating playlists according the user activity · \Improvise - Creating playlists according the user activity" Ricardo Antunes Lopes da Cunha [email protected]

  • Upload
    others

  • View
    30

  • Download
    0

Embed Size (px)

Citation preview

“Improvise - Creating playlists according the user activity”

Ricardo Antunes Lopes da [email protected]

Instituto Superior Tecnico, Lisboa, Portugal

May 2014

Abstract

The number of songs available in Internet has grown steadily since its appearance. As a result of thisgrowth, it is extremely difficult for users to find the appropriate music that suit their needs. To allowusers to listen to the music they like, two techniques are quite used, searching and recommendation.While recommender systems rely more actively on the users needs, the search engines receive theirneeds more passively. In the present work we developed a solution for music recommendation, whoseobjective is to search and recommend songs in tune with the activity that the user is performing. Withthis objective in mind, we developed a hybrid approach reflecting the integration of a recommendationsystem based on context and content. Regarding context, we use information about the user’s activity.In what concerns the content, we took into account the information about the acoustic features of eachsong. The recommendation algorithm starts by recommending the same songs for all users, we call thisthe generic model. The user’s interaction with the system leads to the adjustment of the generic model,to its own musical preferences changing it into a personalized model. A personalized recommendationmodel for each user will then be obtained after some interactions with the system, which will betterreflect their tastes.

Keywords: Music Recommendation System, Hybrid recommendation, Recommendation based on con-text, User activity, Association between music and activities, Generic and personalized model

1. Introduction

Nowadays there are numerous resources at our dis-posal for the consumption of digital music, whetherthrough online stores or streaming services. Thuswe have at our disposal thousands of songs thatcan be listen anywhere, anytime, and on multipledevices like MP3 players, smartphones and tablets.

However, with the access to such a vast collec-tion of songs, the main problem arises when the userwants to choose songs he wants to listen. This prob-lem is even worse when trying to create a playlistto listen to while performing a certain task, such aspracticing sports, driving or studying. So creatingplaylists becomes a tiresome and frustrating processto users.

To solve this problem in the past years severalproposals have been developed for the generationand creation of playlists. In this work we describeand analyze the main techniques used in these ap-proaches, which include the use of recommendationmethods like collaborative filtering, content-based,context-based and hybrid approaches.

Despite the good results that such techniques canachieve, they cannot be adapted to the specific ac-tivity the user is performing. In fact, this topic hasnot yet been well explored, therefore an effective so-

lution for this type of recommendation is missing.

With this problem in mind we aim to design asolution for playlist recommendation based on theactivity the user is performing (context). In addi-tion to the activity performed by the user our goal isto use information about the songs (content) to helpin the recommendation process, to detect a possibleparallelism between music features and activities.

Our solution is based on a hybrid recommenda-tion system that considers the context in which themusic is being listen to and the musical features ofeach song.

Initially the same set of songs is recommended toall users. This we call the generic model where eachsong is recommended taking into account some ofthe tests performed with real users in order to re-alize the songs predominantely preferred for eachactivity they perform. The songs are recommendedtaking into account a set of intervals that definehow music should be recommended for each activ-ity. These intervals represent the musical featuresof each song. With the interaction of the user’s withthe generic model, this is adapted to a personalizedmodel that takes in account the previously choicesmade by the user’s.

In the next section we discuss the related work,

1

presenting research made in music recommendersystems. Section 3 describes our solution for rec-ommendation. Section 4 describes the tests madewith users and in section 5 we present the resultsand findings of our work. Finally in section 6 weconclude our work and provide future direction.

2. Related Work

Given the enormous amount of songs that existnowadays and the continuous growth of internetand the power of mobile devices, managing andsearching songs has become more difficult. In thissection we present the state-of-art approaches inmusic recommendations.

2.1. Music Recommendation Methods2.1.1. Collaborative filtering

Collaborative filters are the most common ap-proaches in recommender systems. This techniqueis based on user-generated content (eg: ratings orimplicit feedback) where items are recommended toa user if they are appreciated by similar users.

In the music domain, collaborative filters worksby constructing a matrix M with n songs and musers, which contains the user interaction with theitems (eg ratings, number of views, etc.). Each linerepresents a profile of a user and the columns repre-sent songs. The value Mua,ij is the rating that theuser ua assigned to the item ij .

Although this method is one of the most used formusic recommendation, it presents some problemslike the early-rater and cold-start problems[1].

2.1.2. Content-Based

Recommendations based on music content havebeen used considerably less than the collabora-tive filtering method. This is probably due to thefact that the techniques based on content requiringknowledge about the data, and music is notoriouslydifficult to describe and classify.

In a content-based recommendation system, in-formation describing songs is gathered; this infor-mation relies on music features such as genre[2],loudness, energy, rhythm or melody. This approachis not based on assessments made by users, but onthe features of the songs.

Because this type of recommenders is focused onfinding similarities between the various character-istics of the songs using only own music featureslike rhythm, energy and genre, the user’s personalopinion is not been taken into account when therecommendation is being made.

2.1.3. Context-Based

In the field of recommender systems, the contextrefers to the user situation such as time, mood oractivity that he is performing while he is searchingfor a recommendation[3].

Advanced technologies such as wearable comput-ers and smart devices, have created the opportunityfor researchers to collect and use data about thecontext to enrich the interaction between users andcomputers. Situational information as time, emo-tional status, presence of others, past and futureevents can help the system to better understandthe current needs of the user.

The positive aspect of these systems is that theusers needs may vary depending on the context, andthese recommendation systems can be adapted toprovide the most relevant services.

2.1.4. Demographic filteringDemographic filtering is one of the most basic meth-ods for identifying the type of users that like a cer-tain item[4].

This technique classifies users profiles into groupsaccording to some personal data (age, marital sta-tus, gender, etc.), geographic data (city, country)and psychographics (interests, lifestyle, etc.). Themain drawback of this method resides in the factthat the recommendation is too generic since allusers with the same demographic profile will receivethe same recommendations, which is incorrect sincepeople of same gender or age can have very differenttastes.

2.1.5. Hybrid approachesThe major problems of collaborative filters and con-tent based approaches rely on new users or itemsand in modeling the user preferences, respectively.The main purpose of a hybrid method is to achievebetter recommendations combining some of thetechniques described earlier.

According to Burke[5] there are several meth-ods to integrate different approaches into a hybridrecommender where we emphasize the weighted,switching, mixed and cascade methods.

Hybrid systems perform better than other ap-proaches presented, since they can alleviate someof the drawbacks that suffer a single technique in-tegrating multiple approaches.

2.2. Music recommendation works2.2.1. Context-Aware Mobile Music Recommen-

dation for Daily ActivitiesThis work presents a novel approach to employcontextual information collected with mobile de-vices for satisfying user’s short-term music playingneeds[6].

This work presents a novel approach to employcontextual information collected with mobile de-vices for satisfying users short-term music playingneeds. The presented idea is to create a music sys-tem that is able to detect automatically and in realtime, which activities the user is performing andplaying music appropriate to these activities. Theactivities that are automatically detected by this

2

work are walking, working, running, relaxing, sleep-ing and shopping.

For each one, the sensor data includes time of day,accelerometer data, and audio from a microphone.The recommendation problem is formulated as atwo-step process: (1) infer the users current contextcategory using the features collected by the mobiledevice, and (2) find a song matching the contextthat was inferred. The first step is called contextinference and the second step music content analy-sis.

2.2.2. Lifetrak: Music In Tune With Your Life

The essential idea behind Lifetrak[7] is that sev-eral sensing modalities influence the music thatis played. Essentially, Lifetrak enables a context-aware playlist that automatically chooses music inreal-time based upon the location, the pace of move-ment, the current time, and various other phenom-ena in the users environment. Furthermore, Life-trak uses a simple learning mechanism to adjustthe ratings of songs for a particular context basedon users feedback when a song is being played.

The system does not autonomously analyze songsand connect them to corresponding contexts. In-stead, Lifetrak relies on the user to tag the songdatabase with the basic context constructs. Life-trak then figures out the users current context andranks the song database according to this informa-tion.

There are basically four main components in-volved in the software architecture for Lifetrak.Thefirst component is the user space document whichcontains a list of all songs and the contexts thatapply for each song. Then there is the contextengine that actually gets information from varioussensing modalities and categorizes them into spe-cific tags. There is also the rating generator whichtakes the current con- text and the user definedsong database, which contains the individual con-text tags for each song, and then generates a rankedplaylist. Finally, the music player provides an in-terface for the end user.

2.2.3. Nextone Player: A Music RecommendationSystem Based On User Behavior

In this paper the authors designed a system thatallows recommending songs that are favored by theuser, new to his ear and that correspond to hispreferences[8]. To this end the authors use user’sbehavior as feedback to decide what song should beplayed at that particular moment.

The authors determine whether a song is to berecommended as the next one in the playlist fromfive perspectives: genre, year, favor, freshness andtime pattern. Time series analysis of genre and yearare used to predict these attributes to the next songrather than to select the song with similar genre

and year to the current song. According to theauthors a similar song relatively to a current onecannot be assumed as a reasonable good choice ofrecommendation. Prediction using time series anal-ysis caters better to a users likes. The numberof times a song has been actively played and howmany times the same has been completely playedare used to infer the strength of songs favor. Otherfeature introduced by the authors was to considerthe musics freshness in relation to a user. The fresh-ness of a song may be considered as the strength ofstrangeness or the amount of experience forgotten.In a different period of a day or a week, users tend toselect different styles of songs. For example, in theafternoon, a user may like a soothing kind of musicfor relaxation and may switch to energetic songs inthe evening. This paper uses a Gaussian MixtureModel to represent the time pattern of listening andcompute the probability of playing a song at thattime.

At the time of the recommendation, the obtainedresults from the different perspectives presentedabove are integrated into a final result. Giventhat different users have different tastes, differentweights are attributed to the five factors at the timeof integration of results. This system has interest-ing techniques such as the concept of freshness andthe use of users interaction with the system to findout the year and gender of the next song to play.However this approach does not take into accountthe users context which we consider relevant to amore effective recommendation.

2.2.4. Foxtrot: A Soundtrack for Where You AreIn this article[9], the authors propose a mobile ap-plication named Foxtrot, that allows people to sharemusic they associate to a given location. To providean engaging experience, but at the same discretetime, the Foxtrot mixes tags assigned by users, thegeographical location and music, to create a kind ofradio channel where people can hear music associ-ated with a given location.

The Foxtrot revolves around two primary objects,users, and songs. Users have their current loca-tion, speed, hearing profile, history, and friends list.Each song focuses on a geographical location, dis-tance visibility, a type and a set of tags specified byusers. The presented approach to create a playlistconsists of three steps. In a first step, the applica-tion collects songs that are close to the user’s loca-tion, listed in ascending order of distance. To everysong is associated a score that indicates their rele-vance to the user. This score is a combination ofthree factors, namely: preference, geographical rel-evance and freshness. In the last step, the tracks arearranged to achieve a pleasant-sounding mix for agood user experience.

One drawback in this solution is that anyone can

3

assign a song to a given location, however this as-signment may have been made in a completely dif-ferent context from that in which the person (user)hears the music which might lead one to think thatthe allocation made is meaningless in his context.

3. Developed Solution3.1. Association between songs and activitiesTo develop an effective music recommendation sys-tem according to the activity that each user is per-forming it is necessary to understand users prefer-ences in accordance with the task being performed.This section will therefore describe the method usedto capture these tastes as well as the implementa-tion decisions made during the process. We willalso present the application used to collect the datato be used in the generic model, explaining how itworks.

In the first phase of the recommendation processour solution recommends an equal set of songs forall users and each activity. This procedure cor-responds to the interaction of the users with thegeneric model. This recommendation model is to beused by all users and his aim is to be framed takinginto account the users interaction with the system.With the first interaction results the personalizedmodel that is shaped according to the choices madeby the user when using the generic model. Since itis from this model that we will generate the multi-ple custom models, it is important therefore that itcaptures general likings for all potential users.

In order to gather songs that all users might pre-fer, for each activity we developed a web applicationthat will be explained in detail in the next section.

Due to the paradox of choice of Schwartz [10]which indicates that the difficulty of a choice isgreater, the greater the range of options we haveat our disposal, it was important to find a way tofilter the vast amount of musics and artists avail-able.

There are currently dozens of genres and theirtaxonomy is not well defined. Because of this diffi-culty of identifying music genres, we decided to usethe one accepted by the majority of digital musicservices. Therefore we chose to represent 15 dif-ferent genres. The number and genres were chosenbased on the division of genres made by some ser-vices known in the field of digital music like last.fm,musicbox and musicovery.com. Further the identi-fication of the main musical genres there was theneed of filtering by artist. This filtering considersthree factors: the number of available music genres,the genres chosen for each activity during the asso-ciation process and the current popularity of eachexisting artist. The number of artists displayed is30, this is twice the number of genres. We considerthis an ideal number of artists since it’s not a verylarge number and ensures that in the worst case ce-

nario the user has at least two possible choices foreach genre, which happens when the user choosesall the 15 genres for a given activity. Since there aremillions of songs available, it was necessary to filterthe number of songs to be presented. To achievethis objective filterings by genre and artists wereimportant. Filterings by genre, allows us to choosethe music genre that best fits the user needs de-pending on current activity. Filtering by artists inturn allows us to choose our favorite artists, reduc-ing the number of possible songs shown. Yet foreach activity we set the limit of 100 songs to bepresented. These 100 songs are then divided by thetotal number of artists previously selected.

Having taken the previous decisions, we startedto develop the web application whose purpose wasto collect information on the preferred songs foreach different activity supported by the system.

3.2. Data collection applicationTo gather information about the songs preferred foreach activity, we developed a web-application thatconsiders the decisions we presented in the previoussection. The collected data will then be used in thecreation of the generic model.

The usage of this application, for the creation ofthe generic model, can be divided into three majorphases. First an activity is chosen randomly fromthose for which the user has not yet selected anymusic. Selected the current activity, a set of genresis shown to the user and he will have to choose atleast one that conforms with his interests to theassigned activity (figure 1).

Figure 1: Genre selection.

In the second phase are presented artists that re-flect the choices made in the previous step. Theseartists are obtained through the API of the echonestservice 1, being these artists the most relevant foreach of the chosen genres. Here the user must selectat least one to proceed to the last stage (Figure 2).

The last phase presents a set of 100 songs relatedto the artists previously chosen. For all songs shownthe user must select at least 20 that he considersappropriate for the current activity (figure 3).

The web application was based on HTML, CSSand JavaScript technologies. For the collection of

1echonest.com

4

Figure 2: Artist selection.

Figure 3: Music selection.

artists and songs we used the API provided byechonest service, as well as the service provided bydeezer 2 that lets us play the songs previously cho-sen by echonest service. To store the informationprovided by the users we used PHP which save, thesongs selected for each activity.

3.3. Recommendation ModelThe generic model is fairly comprehensive and ini-tially we intended it to be the same for all users.With this intuition we needed the input from alarge range of listeners in order to attain a widerange of musical interests for the different activi-ties supported by the system. This enabled us todetermine the existence of dominant characteristicswhen choosing specific musics for specific activities.The choices collected from several users led to theidentification of the top 100 songs for the various ac-tivities. Using then the echonest API we extractedthe songs content information.

After collecting the music features it was neces-sary to evaluate any possible relationship betweenthose and each users activity.

We used the Weka[11] software in order to betterunderstand the influence of each feature, during theselection of a song in a given activity. Weka3 allowsus to use various learning algorithms such as C4.5decision trees and clustering algorithms such as theExpectation Maximization algorithm. In additionto these capabilities Weka can be used to evaluatethe attributes present in a dataset in order to un-derstand the most discriminatives.

2deezer.com3http://www.cs.waikato.ac.nz/ml/weka/

This last feature was used in our dataset toidentify the most important musical features. Forthis we used the CfsSubsetEval attribute evaluatoralong with the bestfit search method. After thisevaluation we where able to conclude that accous-ticness, energy, loudness and tempo where the mostimportant features capable of discriminating the re-lationship between users chosen music and activi-ties. These are the features that we are going touse in the recommendation process of our solution.

Having chosen the most important features weused Weka to test different types of classifiers inorder to understand if it would be easy distin-guish songs using these features. We concluded thatthe Random Tree and Random Forest[12] classifiersproduced the best results when verifying the qual-ity of the features, and from now on all the testsmade with classifiers will use these two.

Our results showed that the chosen features aregood, in general they are sufficiently discriminativeto predict what music should belong to each activ-ity.

a b c d e fa = Walking 70 5 0 0 5 20b = Working 20 40 5 20 0 15c = Relaxing 5 10 65 0 0 20d = Running 15 15 0 60 0 10e = Sleeping 5 0 0 0 95 0

f = Shopping 20 10 0 5 0 65

Table 1: Confusion matrix.

However through the confusion matrix shown intable 1 , we were able to see that the working ac-tivity is the only one where not at least 50% ofthe songs belonging to that activity were detected.One of the reasons may be related to the fact of thisactivity being the most general among the six activ-ities given in our solution, since the work that eachperson is undergoing might require both differentphysical and mental effort.

3.3.1. Activities RefinementWith data collected through the classifier, we no-ticed that there were some difficulties in distinguish-ing what songs should belong to the working activ-ity, since these were often confused with runningand walking activities (see table 1).

Due to the results obtained and to the fact thatthe working activity was the most subjective, wechose to exclude this activity from our solution.Thus we recreated the classifier in the same manneras explained above but with only 100 songs becausethe 20 songs belonging to the working activity wereremoved. The confusion matrix resulting from theremoval of working activity can be found in table 2.

The removal of this activity improved the number

5

a b c d ea = Walking 60 5 5 10 20

b = Relaxing 10 80 0 0 10c = Running 10 0 75 0 15d = Sleeping 0 0 0 100 0

e = Shopping 15 10 5 0 70

Table 2: Confusion matrix with 5 activities.

of correctly classified songs in all activities, with theexception of walking activity. However overall thetotal number of songs correctly classified is higherthan in the previous case. The results obtained sofar lead us to exclude the working activity from oursolution as it was the hardest to distinguish from allthe others. We believe that this option will certainlyimprove the recommendation received by the user.

3.3.2. Generic Model

After having at our disposal the most importantsong features, as well the most chosen songs for eachactivity, it was necessary to understand how to usethis information in order to achieve an effective rec-ommendation.

A recommendation system is often seen as a sug-gestion of items similar to a feature vector thatrepresents the users tastes. That is, the recom-mendation is based at a point in n-dimensionalspace where n represents the number of dimen-sions/features. However, the use of the echonestAPI doesn’t allow us to search a set of songs givena single point in space. To overcome this problem,instead of a single point it was used a set of intervalsdelimiting the value that each feature could take, inspace these group of intervals defines a hyper rect-angle.

Taking into account the decisions taken earlierthese hyper rectangles are made of 4 dimensionswhich are defined by the intervals calculated withmaximum and minimum values that each featureaccouticness, energy, loudness and tempo can takewithin each activity. The size of these rectanglesdiffer between activities and users. In our solutionsince we had five activities, we will have not one butfive hyper rectangles, one for each activity.

To find the best way to generate intervals thatrepresent each song, was the next step in the modeldevelopment.

We started by testing two different methods inorder to calculate the intervals of each feature. Thefirst proposed method used the mean minus thestandard deviation for finding the minimum of theinterval and the mean plus the standard deviationto find its maximum. The second method reliedon the use of the 10% percentile as the minimumvalue of the interval and the 90% percentile for itsmaximum.

The intervals created using the two methods de-scribed above, were used to search songs througha website developed using the API echonest. Theminimum and maximum values of the intervals cre-ated for each musical feature were used for this pur-pose.

Each query performed using those intervals re-turned 100 songs that were used with the Ran-dom Tree and Random Forest classification algo-rithms. With these tests we concluded that themethods used to generate the intervals were not ad-equate since the intervals generated were too largeand there was a considerable overlap between them.The resulting recommendation was poor being quitesimilar for all activities.

Considering that the two tested methods hadgiven poor results it was decided to use two newmethods based on the first one, however we decidedto use only a percentage of the standard deviation,namely 15, 20, 25 and 30%. In addition the result-ing variant of the first method, another method wastested similar to the previous one but with the dif-ference of using the median instead of the average.

The new methods were once again tested, usingthe 100 songs selected by the intervals of each fea-ture generated for each activity as the test sets ofthe classification algorithms.

This time limits were generated in two differentways, one using the top 100 songs and the otherusing the top 20 selected by the users when usingthe data collection application presented on section3.2. With this new variant a set of 16 different datasets were created (figure 4).

Figure 4: Generated data sets.

The top 20 and 100 songs selected previously bythe users were used as training data sets of the clas-sification algorithm. Taking this into account andsince we had two different classification algorithmsto test our intervals, we made a total of 64 test inorder to determine the best method to calculate theintervals to be used in the recommendation process.The use of the top 20 and 100 songs for the intervalcreation was related to the need to understand if itwould be beneficial or not to have a wide historicalof songs (20 or 100) for the generation of the inter-vals, in order to obtain the highest percentage ofsongs considered correct for each activity.

6

For the 64 tests made the best result for the num-ber of songs that deemed correct occurred for themethod consisting on the average ± 15% of thestandard deviation. However it should be noticedthat in this case the number of songs used to testthe classifier (i.e. recommended) were only 331 in-stead of the 500 used in most of the other tests.

Given the low number of songs collected by themethod mentioned above, we decided to opt for thesecond best method corresponding to the usage ofthe median ± 20% of the standard deviation. Inthis case the median and standard deviation werecalculated from the top 100 songs chosen by theusers. The training instances used by the classifierwere the top 20 songs chosen by the users for eachactivity. Tables 3 and 4 correspond to the confusionmatrices for the two tests mentioned above (mean± 15% of the standard deviation and median ± 20%of standard deviation).

a b c d ea = Walking 88.5 11.5 0 0 0

b = Relaxing 0 100 0 0 0c = Running 40 0 57 0 3d = Sleeping 0 5 0 95 0

e = Shopping 19 81 0 0 0

Table 3: Confusion matrix of mean ± 15% standarddeviation using Random Forest algorithm.

a b c d ea = Walking 53 11 30 0 6

b = Relaxing 0 85 0 0 15c = Running 2 0 95 0 3d = Sleeping 0 0 0 100 0

e = Shopping 52 30 3 0 15

Table 4: Confusion matrix of median ± 20% stan-dard deviation using Random Forest algorithm.

In addition to the problem mentioned earlierabout the number of songs returned using themethod of the average ± 15% of the standard devia-tion, when analyzing the confusion matrix shown intable 3 we can verify that no songs of the shoppingactivity were assigned as belonging to that sameactivity.

In table 4 we can check an higher mess than theone presented in table 3, however in this case a to-tal of 493 songs were returned using this method. Anumber of songs that were quite close to the desiredvalue of 500 songs. Excepting for the shopping ac-tivity, in this matrix we can see that all activitieshad a success rate of over 50% of songs consideredas belonging to each activity. Since the number ofsongs considered correct for the shopping activitywere not significantly lower when compared with

those obtained in other tests, we decided to use themethod that uses the top 100 songs for the calcula-tion of the median ± 20% of the standard deviationas the method to determine the intervals of the hy-per rectangle (figure 5).

Figure 5: Method used to generate the recommen-dation intervals.

4. User EvaluationAfter the best calculation method for the intervalswas found, that information was used to create thegeneric model that will do the same recommenda-tions for all users. In order to validate this model,we developed a web page where users could choosethe songs they consider most appropriate for eachactivity. Some of the results obtained during thetests made with the generic model were then usedto train the personalized model using the selectedsongs in the previous test. The web page used totest the personalized model was the same that wasused to test the generic model.

The application for the evaluation of the genericmodel was built in with the aim to recommend aset of songs according to the activity that each useris performing. Users begin by answering a smallquestionnaire with their demographic information.Afterwards, a set of 5 activities supported by thesystem are displayed to the user from which he mustchoose the one that he is performing (Figure 6).

Figure 6: Activity selection.

7

To test the quality of the generic model we show50 songs to the users that are within the intervalscalculated according to the options taken in theprevious section. Although a total of 50 songs aregiven to the users, in each moment only one song ispresent to the user.

In order that users have an idea of the musicsuggested, we play a sample of 30 seconds for eachsong, in detriment of the total length of each songto decrease the time spending during the process ofchoice.

In the developed application the user should se-lect only the songs that he considers appropriatefor each activity. Whenever a song does not suithis tastes, he should press the next button to moveon to the next song (see figure 7).

Figure 7: Music selection.

For each suggested music, only a 30 seconds sam-ple is played in order to decrease the time spendingduring the process of choice. An example of an in-teraction with the application used in the genericmodel can be seen in figure 7.

Our solution is based on the construction of ageneric model that is initially equal for all users.The generic model will evolved to a personalizedmodel as a result of the interaction with the user,adapting constantly to the choices made by him.

The evaluation was divided into two steps. Thefirst step was to evaluate the generic model in orderto understand if this is flexible enough to recom-mend music that comply with some of the prefer-ences of potential users.

In the second step, some users who had previouslyinteracted with the generic model, were asked tointeract again with the recommender application.However this time the new songs are recommendedtaking into account the choices made by the usersduring their interaction with the generic model.

The new songs presented were chosen using thesame method of calculation used in the creation ofthe intervals for the generic model. The aim of thisprocedure was to evaluate whether the model is ad-justed according to user preferences. In this casean increase in the number of songs that each userwould consider appropriate for each activity shouldbe expected.

5. Results and FindingsThe generic model evaluation was performed intwo different ways. An online evaluation where wespread the web page described in section 5.1 viasome social networks such as facebook and google+,and presencial tests, where we asked users to testthe generic model.

The online evaluation yielded a total of 21 re-sponses and presential tests were carried out withthe help of 10 users. The generic model was testedmainly by male users. In fact 15 of the online testswere made by male users, and 8 of the presentialtests were also made with male users. The majorityof the user’s representing 81% of the total aged be-tween 22 and 29 years old,. Five users correspond-ing to a total of 16% had ages between 16-21 years,and the remaining 3% were represented by just 1user for the class age between 30 and 39 years old.

When comparing online tests with presentialones, there are significant differences in the num-ber of songs chosen by each user. Online users onlyconsider in average 4 to 5 songs as correct for eachactivity. This value increases for the presential usersto an average between 11 and 12 songs per activity.

This difference may be related to the fact thatwhen facing the users during the presential tests,they might probably be intimidated and as a resulttend to be more concentrated on the tests respond-ing therefore more attentively. The results for theaverage number of songs selected by activity arerepresented in figure 8.

Figure 8: Average selected songs by activity.

Once selected the songs for each activity, usershad to say whether or not they agreed with therecommendation made. The possible answers were:strongly disagree, disagree, neither agree nor dis-

8

agree, agree and strongly agree. This is a subjectiveevaluation, however the result allowed us to under-stand, approximately, the satisfaction level of therecommendation made for each user.

Using the scale to rate the degree of users satis-faction for each of the recommendations made, wecan classify the first and second options as a nega-tive feedback, the third as neutral and last two asa positive feedback. Using this division, we foundan increase of 13% in the positive feedback by theusers who made the presential tests, the percent-ages relating to the feedback provided by users areshown in figure 9.

Figure 9: User feedback about the recommendation.

Although the users who had completed the onlinetest in average selected a small number of songs, wecould verify that the overall satisfaction with therecommended songs is positive. The same can beseen with the users who performed the tests in per-son, these results demonstrate that in general therecommendation meets the interests of users how-ever for some reason in average the online userschoose a much narrower number of songs.

In order to check if the generic model can betransformed into a personalized model, new testswere conducted with the 10 users who responded tothe previous tests in person with the generic model.To test the personalized model, the intervals of thedifferent music features were modified taking intoaccount the choices made by each user in their in-teraction with the generic model.

The results obtained during the tests used withthe personalized model can be seen in figure 10.

In the graph shown in figure 10 we can see thatthere was on average an improvement in the rec-ommendation when using the personalized model,which can recommend on average more than 15songs per activity.

After the evaluation made to all users using thepersonalized model, we made another test using thechosen songs during the first interaction with thepersonalized model. This test was intended to checkif there was an improvement in the number of songs

Figure 10: Personalized model results.

deemed appropriate for each user using the person-alized model. Unfortunately it was not possible toperform these tests with all the 10 users who hadexperienced their personalized model, having beenperformed only by 5 of them.

figure 11 shows the evolution of the number ofsongs chosen by each of the users who used the per-sonalized model for the second time.

Figure 11: 2nd iteration using the personalizedmodel.

From the obtained data we found that in bothcases where there were fewer songs chosen duringthe use of the generic model there was a continuousincrease in the recommendation using the person-alized model. In the other cases a continuous growon the recommendation was not found, but in onlyone case the personalized model was not able torecommend more songs than the generic model.

6. Conclusions and Future WorkWith the arrival of digital music and Internet tech-nologies, a great ammount of musical content isavailable to millions of users around the world.With the millions of artists and songs on market,begins to be difficult for users to search songs thatplease them. Furthermore, the large amount of mu-sic available has opened opportunity for the emer-gence of new recovery and music recommendationtechniques.

With this work it was possible to perceive thechallenges and the inherent complexity of the rec-

9

ommendation systems. Knowledge related to thisarea, namely in what respects the types of recom-mendation techniques and the challenges presentedby them were acquired. At the same time it wasimportant to realize the state of the art in recom-mender systems focused on music.

In our solution, we verified that the use of ageneric model for all users can be beneficial for aninitial recommendation. The capacity of this modelfor an initial recommendation is important becauseit allows fulfilling one of the great problems found inrecommendation known as the Cold Start Problem.

The use of intervals to distinguish songs to rec-ommend in each activity revealed to be a capablemethod of molding to the likings of the differentusers. This was visible with the improvement ver-ified in the number of songs recommended duringthe passages of the generic model to the person-alized one and through the second iteration madeusing the songs recommended by the personalizedmodel.

These intervals thus allow to recommend musicsbased on their musical features and overcome one ofthe problems of some recommendation techniquesknown as popularity bias. In such a way we at-tained the intended objectives, in presenting a so-lution that has no difficulty in recommending songsto new users and that is capable to recommend anytype of music be it known or unknown to him. How-ever, the concept of freshness is not included in oursolution, since the number of times that a musicwas already recommended to each user, as well asthe last time that the recommendation was madewas not taken into account. Given that we are notusing the concept of freshness in our solution, theinclusion of this concept would be an important as-pect to consider in the future. With this conceptwould be possible to assign a freshness score to eachsong, that would serve to have a more diverse rec-ommendation, and promote the discovery of newsongs.

It could also be interesting to choose randomlytwo new points within the current interval to rep-resent a new maximum and minimum. These newpoints would be always within the current interval,the restriction of some values inside of the interval,would allow to verify more rapidly in which direc-tion are the musical tastes of each user.

Finally, information concerning the names ofartists whose songs were commonly chosen by userscould also be used in the recommendation process.This allows us to recommend new songs of the user’spreferred artists which had not yet been recom-mended. With the artists identity it should alsobe possible to look for similar ones. This could beinteresting as it gives the user the opportunity offinding new songs. Such as the artists name, an-

other possibility is information of each song genrefor each activity.

References[1] Christopher Avery and Richard Zeckhauser.

Recommender systems for evaluating com-puter messages. Commun. ACM, pages 88–89,1997.

[2] N. Scaringella, G. Zoia, and D. Mlynek. Au-tomatic genre classification of music content:a survey. Signal Processing Magazine, IEEE,pages 133–141, 2006.

[3] Anind Kumar Dey. Providing architecturalsupport for building context-aware applica-tions. 2000.

[4] Elaine Rich. Readings in intelligent user inter-faces. pages 329–342, 1998.

[5] Robin Burke. Hybrid recommender systems:Survey and experiments. User Modelingand User-Adapted Interaction, pages 331–370,2002.

[6] Xinxi Wang, David Rosenblum, and Ye Wang.Context-aware mobile music recommendationfor daily activities. Proceedings of the 20thACM international conference on Multimedia- MM ’12, page 10, 2012.

[7] Jeff Mascia. Lifetrak : Music In Tune WithYour Life Categories and Subject Descriptors.1st ACM International Workshop on Human-Centered Multimedia, pages 25–34, 2006.

[8] Yajie Hu and Mitsunori Ogihara. Nextoneplayer: A music recommendation system basedon user behavior. Proceedings of the 12th In-ternational Society for Music Information Re-trieval Conference, pages 103–108, 2011.

[9] Anupriya Ankolekar and Thomas Sandholm.Foxtrot: a soundtrack for where you are. Pro-ceedings of Interacting with Sound Workshop:Exploring Context-Aware, Local and Social Au-dio Applications, pages 26–31, 2011.

[10] B. Schwartz. The Paradox of Choice: WhyMore Is Less.New York. page 304, 2005.

[11] Geoffrey Holmes Bernhard Pfahringer Pe-ter Reutemann Ian H. Witten Mark Hall,Eibe Frank. The weka data mining software:An update; sigkdd explorations, volume 11, is-sue 1. 2009.

[12] Leo Breiman. Random forests. Machine learn-ing, pages 1–33, 2001.

10