Data Fusion for Dealing with the Recommendation Problem

DataFusionforDealingwiththeRecommendation

ProblemDenisParra,PUCChile

KeynoteforIFUP

WorkshoponMulti-dimensionalInformationFusionforUserModelingandPersonalization

UMAP2016,Halifax,Canada

Inthistalk

• Recommendationofarticleswithuser-controlledfusion

• Fusingdatainthemusicdomain

• Fusionfore-marketplacesinvirtualworlds

• Howtointegratetimeintocollaborativefiltering?

7/17/16 D.Parra,IFUPkeynote,UMAP2016 2

Part1:RecommendationofArticleswithUser-ControlledFusion


RecommendationofArticles

• Problem:a)Traditionaluserfeedbackis(was?)difficulttoobtain,b)Sparsity

• Thereareseveralpotentialsourcesofrecommendation,butmostlyfromtheitems:

• Content• Co-citations,co-authorship• Etc.

• Ourapproach:giveuserscontroloverwhattofuse.• Would itwork?• Howmuchdatacombination istheoptimum?• Doesvisual representationaffectthebehavior/accuracy?


References• Verbert,K.,Parra,D.,Brusilovsky,P.,&Duval,E.(2013).Visualizingrecommendationstosupportexploration,transparencyandcontrollability.InProceedingsofthe2013internationalconferenceonIntelligentuserinterfaces (pp.351-362).ACM.

• Parra,D.,Brusilovsky,P.,&Trattner,C.(2014).Seewhatyouwanttosee:visualuser-drivenapproachforhybridrecommendation.InProceedingsofthe19thinternationalconferenceonIntelligentUserInterfaces (pp.235-240).ACM.

• Verbert,K.,Parra,D.,&Brusilovksy,P.(2014).Theeffectofdifferentset-basedvisualizationsonuserexplorationofrecommendations.InProceedingsoftheJointWorkshoponInterfacesandHumanDecisionMakinginRecommenderSystems (pp.37-44).


TalkExplorer

• ImplementedinitiallyforauserstudyinACMHypertext2012forConferenceNavigator.

• Mainquestiontoaddress:Dousersconsiderthefusionofseveralsourcesofdatawhenchoosingrelevantitems?


Recap– ConferenceNavigator

Program Proceedings Author List Recommendations


TalkExplorer Interface


TalkExplorer - Entities

EntitiesTags,RecommenderAgents,Users


TalkExplorer – CentralCanvas

RecommenderRecommender

Cluster with intersection of entities

Cluster (of talks) associated to only one entity

• CanvasArea:IntersectionsofDifferentEntities

User


TalkExplorer - Articles

ItemsTalksexploredbytheuser


TalkExplorer StudiesI&II

• StudyI• ControlledExperiment:Userswereaskedtodiscoverrelevanttalksbyexploringthethreetypesofentities:tags,recommenderagentsandusers.

• ConductedatHypertextandUMAP2012(21users)• SubjectsfamiliarwithVisualizationsandRecsys

• StudyII• FieldStudy:Userswereleftfreetoexploretheinterface.

• ConductedatLAK2012andECTEL2013(18users)• Subjectsfamiliarwithvisualizations,butnotmuchwithRecSys


Evaluation:Intersections&Effectiveness

• Whatdowecallan“Intersection”?

• Weused#explorationsonintersectionsandtheireffectiveness,definedas:


ResultsofStudiesI&II

• Effectivenessincreaseswithintersectionsofmoreentities

• Effectivenesswasn’taffectedinthefieldstudy(study2)

• …butexplorationdistributionwasaffected


SetFusion

• Mainmotivationwasinvestigatingasimplerwaytovisualizerecommendationsfromseveralsources.Wouldthatimprove“effectiveness”?

• 3studieswereconducted• FieldstudyinCSCW2013• ControlleduserwithiConference series• FieldstudyinUMAP2013


SetFusion


SetFusion I

Traditional Ranked List

Papers sorted by Relevance. It combines 3 recommendation approaches.


SetFusion - IISlidersAllow the user to control the importance of each data source or recommendation method

Interactive Venn DiagramAllows the user to inspect and to filter papers recommended. Actions available:- Filter item list by clicking on an area- Highlight a paper by mouse-over on a circle- Scroll to paper by clicking on a circle- Indicate bookmarked papers


SetFusion ControlledStudy

• 40users,within-subjectsstudy,simulatediConference attendance


ControlledStudyMainResults

• Controllingandfusingsourcesofrelevancyproducesmorebookmarks:

• 58.44%ofbookmarksafterusingsliders• 28.08%ofbookmarksafterusingVenndiagram


ControlledStudyMainResults

• Userspreferarticlesrecommendedbyafusionofmethods,inbothconditions,buttheeffectisstrongerwiththevisualization


SetFusion – UMAP2013

• FieldStudy:letusersfreelyexploretheinterface

- ~50% (50 users) tried the SetFusion recommender

- 28% (14 users) bookmarked at least one paper

- Users explored in average 14.9 talks and bookmarked 7.36 talks in average.


TalkExplorer Vs.SetFusion

Clustermap Venndiagram


TalkExplorer vs.SetFusion

• Comparingdistributionsofexplorations

In studies 1 and 2 over TalkExplorer we observed an important change in the distribution of explorations.


TalkExplorer vs.SetFusion

• Comparingdistributionsofexplorations

Comparing the field studies:- In TalkExplorer, 84% of

the explorations over intersections were performed over clusters of 1 item

- In SetFusion, was only 52%, compared to 48% (18% + 30%) of multiple intersections, diff. not statistically significant


Take-aways

• Weshowedthatintersectionsofseveralcontextsofrelevancehelptodiscoverrelevantitems.

• Thevisualparadigmusedcanhaveastrongeffectonuserbehavior:weneedtokeepworkingonvisualrepresentationsthatpromoteexplorationwithoutincreasingthecognitiveloadovertheusers.


Part2:FusingDataintheMusicDomain


References

Parra-Santander,D.,&Amatriain,X.(2011).WalktheTalk:Analyzingtherelationbetweenimplicitandexplicitfeedbackforpreferenceelicitation.ProceedingsofUMAP2011,Girona,SpainParra,D.,Karatzoglou,A.,Amatriain,X.,&Yavuz,I.(2011).Implicitfeedbackrecommendationviaimplicit-to-explicitordinallogisticregressionmapping.ProceedingsoftheCARSWorkshop,RecSysChicago,IL,USA,2011.


Introduction(backin2011)

• Mostofrecommendersystemapproachesrelyonexplicitinformationoftheusers,but…

• Explicitfeedback:scarce(peoplearenotespeciallyeagertorateortoprovidepersonalinfo)

• Implicitfeedback:Islessscarce,but(Huetal.,2008)There’snonegativefeedback …andifyouwatchaTVprogramjust

onceortwice?

Noisy …butexplicit feedbackisalsonoisy(Amatriainetal.,2009)

Preference&Confidence …weaimtomaptheI.F.topreference(our maingoal)

Lackofevaluationmetrics …ifwe canmapI.F.andE.F.,wecanhaveacomparableevaluation


Introduction(Today)

• Isitpossibletomapimplicitbehaviortoexplicitpreference(ratings)?Thesedatacaneventuallybefusedintoasinglecompactmodel.

• OURAPPROACH:StudywithLast.fmusers• PartI:Askuserstorate100albums(howtosample)• PartII:Buildamodeltomapcollectedimplicitfeedbackandcontexttoexplicitfeedback


WalktheTalk(2011)

Albumstheylistened toduringlast:7days,3months,6months,year,overall Foreachalbuminthe listwe

obtained: #userplays(ineachperiod),#ofglobal listeners and#ofglobalplays


WalktheTalk- 2

• Requirements:18y.o.,scrobblings >5000


QuantizationofDataforSampling• Whatitemsshouldtheyrate?Item(album)sampling:

• ImplicitFeedback(IF):playcount forauseronagivenalbum.Changedtoscale[1-3],3meansbeingmorelistenedto.

• GlobalPopularity(GP):globalplaycount forallusersonagivenalbum[1-3].Changedtoscale[1-3],3meansbeingmorelistenedto.

• Recentness (R):timeelapsedsinceuserplayedagivenalbum.Changedtoscale[1-3],3meansbeinglistenedtomorerecently.


RegressionAnalysis

• IncludingRecentness increasesR2inmorethan10%[1->2]• IncludingGPincreasesR2,notmuchcompared toRE+IF[1->3]• NotIncludingGP,butincluding interactionbetween IFandREimproves thevarianceoftheDVexplained bytheregressionmodel.[2->4]

M1:implicit feedback

M2:implicitfeedback&recentness

M4:Interactionofimplicitfeedback&recentness

M3:implicitfeedback,recentness,globalpopularity


RegressionAnalysis

• Wetestedconclusionsofregressionanalysisbypredictingthescore,checkingRMSEin10-foldcrossvalidation.

• Resultsofregressionanalysisaresupported.

Model RMSE1 RMSE2Useraverage 1.5308 1.1051M1:Implicit feedback 1.4206 1.0402M2:Implicitfeedback +recentness 1.4136 1.034M3:Implicitfeedback + recentness +globalpopularity 1.4130 1.0338M4:Interaction ofImplicitfeedback *recentness 1.4127 1.0332


PartII:ExtensionofWalktheTalk

• ImplicitFeedbackRecommendationviaImplicit-to-ExplicitOLRMapping(Recsys 2011,CARSWorkshop)

• Considerratingsasordinalvariables• Usemixed-modelstoaccountfornon-independenceofobservations

• Comparewithstate-of-the-artimplicitfeedbackalgorithm


Recallingthe1st study(5/5)

• PredictionofratingbymultipleLinearRegressionevaluatedwithRMSE.

• ResultsshowedthatImplicitfeedback (playcountofthealbumbyaspecificuser)andrecentness(howrecentlyanalbumwaslistenedto)wereimportantfactors,globalpopularityhadaweakereffect.

• Resultsalsoshowedthatlisteningstyle(ifuserpreferredtolistentosingletracks,CDs,oreither)wasalsoanimportantfactor,andnottheotherones.


...but

• LinearRegressiondidn’taccountforthenestednatureofratings

• Andratings weretreatedascontinuous,whentheyareactuallyordinal.

User1

13530452215432

Usern

32104525432135

...


So,OrdinalLogisticRegression!

• ActuallyMixed-EffectsOrdinalMultinomialLogisticRegression

• Mixed-effects:Nestednatureofratings• Weobtaina distributionoverratings(ordinalmultinomial)pereachpairUSER,ITEM->wepredict theratingusingtheexpectedvalue.

• …Andwecancomparetheinferredratings with amethodthatdirectlyusesimplicitinformation(playcounts)torecommend (byHu,Koren etal.2007)


OrdinalRegressionforMapping

• Model

• Predictedvalue


Datasets

• D1:users,albums,if,re,gp,ratings,demographics/consumption

• D2:users,albums,if,re,gp,NORATINGS.


Results


Conclusions(after5years)

• FusionofImplicitfeedback(scrobbles)andrecencycanhelptomakemorepreciserecommendations

• ModelsliketheonebyGurbanov andRiccipresentedthisyearatUMAPofferamorecompactwaytoworkwiththesedata:

“ModelingandPredictingUser Actionsin Recommender Systems”byTural Gurbanov, FrancescoRicci,Meinhard Ploner

• Evaluationisstillachallenge!


Part3:DataFusionforVirtualWorlds


References

Lacic,E.,Kowald,D.,Eberhard,L.,Trattner,C.,Parra,D.,&Marinho,L.B.(2015).Utilizingonlinesocialnetworkandlocation-baseddatatorecommendproductsandcategoriesinonlinemarketplaces.InMining,Modeling,andRecommending'Things'inSocialMedia (pp.96-115).SpringerInternationalPublishing.Trattner,C.,Parra,D.,Eberhard,L.,&Wen,X.(2014,April).Whowilltradewithwhom?:Predictingbuyer-sellerinteractionsinonlinetradingplatformsthroughsocialnetworks.In Proceedingsofthe23rdInternationalConferenceonWorldWideWeb (pp.387-388).ACM.


SecondLife


SocialNetwork

Marketplace

VirtualWorld

Christoph TrattnerKnow-CenterGraz,Austria

Dataset(Task:Itemrecommendation)


RecommendationApproaches

• User-basedCollaborativeFiltering,where

• Hybridapproaches(combinefeatures)


SimilarityFeatures- I


SimilarityFeaturesII


Hybrids


DifferentTask:PredictBuyer-Seller


PredictBuyer-Sellers:AUCResults


Summary

• Thesestudiesshowthatsocialnetworkdataisveryimportantforcertaintypesofrecommendations.

• Duetothelackofavailablecross-servicedataintherealworld,usingdatafromSecondLifehasthepotentialofaProxytobuildmodelsfortherealworld.


Part4:FusionofTimeintoCF


References

Larrain,S.,Trattner,C.,Parra,D.,Graells-Garrido,E.,&Nørvåg,K.(2015).GoodTimesBadTimes:AStudyonRecency EffectsinCollaborativeFilteringforSocialTagging.In Proceedingsofthe9thACMConferenceonRecommenderSystems (pp.269-272).ACM.


Time-AwareCollaborativeFiltering

• CollaborativeFiltering(UserandItem-based)considersalltransactionsequallyimportant

• Buttransactionswhichhappenedtoolongagomightbelessimportantshapingtheusermodel…


5

4

2

1

54

Active user

User_1

User_2

2

3

4

Item 1

Item 2

consumed2yearsago

consumed1monthago

TwoConceptsforTime-AwareCF

• Itemsconsumedrecentlymightbemoreimportantthanitemsconsumedlongtimeago.

•When andhow toincorporatetimeinuser-anditem-basedcollaborativefiltering?


WhenandHowinUB-CF


Item1 Item2 … Itemj Itemm

User1 1 5 2

User2 5 1 4 2

…

User i 3 4

…

Usern 2 5 5

Step1:Findsimilarusers.Weighttransactionsbasedonrecency difference

WhenandHowinUB-CF



User1 1 5 2

User2 5 1 4 3

…

User i 3 4

…

Usern 2 5 4

Step2:Similarusersfound.Recommenditemswithhighratingsandconsumedrecently.

WhenandHowinIB-CF



User1 1 5 2

User2 5 1 4 2

…

User i 3 4

…

Usern 2 5 5

Step1:Findsimilaritemssim(items(user i)).Weightitemsbasedonrecency.

Consu-med1weekago

Consu-med1yearago

WhenandHowinIB-CF



User1 1 5 2

User2 5 1 4 2

…

User i 3 4

…

Usern 2 5 5

Step2:FindsimilaritemsItem1.Weightitemsbasedonrecency difference.

Decayfunctions

• Exponential

• Power

• Linear

• Logistic

• BLL


Parametersandfitting


Daysfrombookmark

Median=50days

Evaluation:Datasets


Evaluation:ResultsI


Evaluation:ResultsII


Summary

• Bestresults:Post-filteringcombinedwithpowerdecaygivesthebest

• Pre- andPost-filteringproduceastrongeffect,butUB-CFismoresusceptiblethanIB-CFtotheeffectoffilteringspeciallypre-filtering.

• ThehybridizationofUBandIBimprovesmakestherecommendationmorerobust.

• Futurework:fitparametersonauserbasisratherthandatasetbasis.


Wrappingup


• Visualapproachesforuser-controllabledatafusioncanwork,butthere’sroomtofindeffectivevisual-interactivecombinations.

• Inthemusicdomainandotherdomains,timeandrecency canworkverywellforrecommendation.

• …butusingtimerequiresanadequatemodelingofthedecayfunctions.

• InformationfromVirtualworldscouldmaybeusedasproxytobuildmodelsandusethemfortransferlearning.


PromisingworksinthisUMAP2016

• UsingSemanticInformation:ExtendtheworkofMusto etal.(UMAP2016)tosupportbettermodelsandmoreexplainablemodels.

• Combinetaxonomieswithimplicit/explicitfeedbackusingcompactgraphicalmodels(co-authoredbyg.Guo)

• Extendmodelswithtimeandothersourcesoffeedback(Turgnov etal.)


IdeasforDataFusion

• Combinemultimodalinformationwithinthesameembeddingusingdeeplearninghasgivengreatresultsinvisualprocessing+NLPfields:

• VisualQ&A• AutomaticCaptioningofPictures


Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Lawrence Zitnick, C., & Parikh, D. (2015). Vqa: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2425-2433).

Thanks!

[email protected]://dparra.sitios.ing.uc.cl/


Technology

Data Fusion for Dealing with the Recommendation Problem