A Federated Search and Social Recommendation Widget

A FEDERATED SEARCH AND SOCIAL RECOMMENDATION

WIDGETSten Govaerts, Sandy El Helou, Erik Duval and Denis Gillet.

Saturday 19 March 2011

WHAT’S A WIDGET ?!?










CONTEXT

• Personal Learning Environment:

• customizable• re-use, creation & mashup of tools, resources

• enable users to access content

• in different contexts


CONTEXT






CONTEXT






ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


ARCHITECTURE


DEMO...


DEMO...


DEMO...


DEMO...


PAGE RANKHYPERLINK

WEBSITE

A node is important if and only if many other important nodes point to it

Rank of node i:


friend

connection

saved/shared

liked

liked

disliked

friend

connection

saved/shared

R1

R2

R3

R4

R5

Sten

Sandy

Erik

OUR CASE


NOW FOR MULTI-DIRECTIONAL, PERSONALIZED & CONTEXTUAL RANKING

A node is important to a particular set of nodes (representing the target user and the context) if and only if many important nodes connected to this root set, via important relation types, point to it


EVALUATION

• 15 PhD students at K.U. Leuven and EPFL.

• What?

• usability

• user satisfaction

• usefulness


FIRST PHASE

• current media search tool: Google & YouTube

• understanding recommendations: 6/15 from like/dislike


FIRST PHASE




FIRST PHASE




FIRST PHASE




FIRST PHASE




FIRST PHASE




FIRST PHASE




SECOND PHASE


SECOND PHASE

• only 14 participants (one less)

• open questions

• usefulness of recommendations: 11/14 pro.

• user satisfaction: System Usability Scale (SUS) & MS Desirability Toolkit

• SUS score: 66,25%

• 2 groups


SECOND PHASE

• only 14 participants (one less)

• open questions

• usefulness of recommendations: 11/14 pro.

• user satisfaction: System Usability Scale (SUS) & MS Desirability Toolkit

• SUS score: 66,25%

• 2 groups K.U. Leuven: high (75%)EPFL + one K.U.Leuven: low (50%)


WHY THE DIFFERENT SUS?

• 1st phase by 2 interviewers









• issues:

• distracts of unrelated widget’s UI updates.

• layout too dense

• height of widgets too small

• KULeuven student had prior experience with iGoogle.

• not evaluating the widget but the whole experience...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


DESIRABILITY...


RECOMMENDATIONS EVALUATION

• compare recommendations to their favourite tool: Google

• 2 groups with different queries





Precision# relevant items returned

# total items returned=





in top N list# relevant items returned


Precision at N





•Google: more relevant results

•Google: avg. prec. at 10 = 65%

• widget: avg. prec. at 10 = 50%



Precision at N





•Google: more relevant results

•Google: avg. prec. at 10 = 65%

• widget: avg. prec. at 10 = 50%



Precision at N

less variation in results


FIXES...


FIXES...


FIXES...


FIXES...


FUTURE WORK

• evaluation in larger scale real-world situation (university + business)

• evaluate user satisfaction of widget and not container

• evaluate the recommendations further (based on use).

•make recommendations transparent


MORE VISUAL SEARCH...


THANK YOU!QUESTIONS?...


Technology

A Federated Search and Social Recommendation Widget