Personalizing Media Interaction on the (Semantic & Social) Web

Personalizing Media

Interaction on the Web

Lora Aroyo

© Kierstin Shaylee

1Wednesday, October 3, 12

http://www.flickr.com/photos/65381383@N07/

http://www.flickr.com/photos/65381383@N07/

“interface is the message”

Aaron Koblin: Artfully visualizing our humanity, TED Talk, 2011

Lora Aroyo twitter: @laroyo


it’s complicated



HCI challengessemanticsuser data

interaction

personalization Lora Aroyo twitter: @laroyo

Flickr: elkabong 4Wednesday, October 3, 12

semantic HCI challenge

• explicit semantics (often from open sources, e.g. LOD) used for system decisions and results

• use facetted presentation, searching and browsing of information

• use typically classifications, typologies or other structures of concepts

• integrate & aggregate data from different sources






user data HCI challenge



• search, e.g. keyword, faceted

• browse, story lines, narratives through collections

• annotations of multimedia, e.g. (collaborative) tagging, professional annotation of text, images and video, tagging games

• explanations, hints, user feedback, e.g. explanation of recommendation results, explanation of autocompletion suggestions

interaction HCI challenge



interaction with semantics



© BBC MMVIIIhttp://twitpic.com/il1w/full


http://twitpic.com/il1w/full

http://twitpic.com/il1w/full

http://www.bbc.co.uk/programmes/b00c06n2.rdf

Lora Aroyo twitter: @laroyo11Wednesday, October 3, 12



• data challenges multiply with Web openness

• social recommendations are preferred

• it is not about (single) algorithms any more

• it is about the metrics

personalization challenge



















applications


© Ugli. on the Walls of BBC London

personalized TV


User Perspective

• what is the role of social recommendations?• how do people watch TV together?• how devices influence watching?• what are perceived trade-offs for privacy vs. personalization?


Technology Perspective

• what is the role of metadata in TV & Apps?• what are ways to share information in real time?• what are ways to sync TV & other metadata?• what is an easy way for devices to find & talk to each other?


choices, choices ...

© VickyBuser, BBC


lost in space© RedBee Slides at MIPCube2012


always missing something© RedBee Slides at MIPCube2012


demand for experience23Wednesday, October 3, 12

© RedBee Slides at MIPCube2012


Social Recommendations© Ericsson ComsumerLab: TV & Web Consumer Trends 2011


Social Recommendations

I usually find NEW programmes to watch by:

RESULTS OF NOTUBE SOCIAL WEB & TV SURVEY 2012

© NoTube Studies @ BBC


Social Recommendations

I usually find NEW programmes to watch by:


Social TV


© NoTube Studies @ BBC







interface explorations






Linking Web Data to User Profiles


TV Preferences• Inferred from the Social Web

• tweets, FB likes, last.fm listened, etc.• weighted interests

• Represented using Linked Data web identifiers

• record-linkage:

• NLP:

facebook.com/pages/ dbpedia.org/

http://dbpedia.org/resource/Matt_Lucashttp://dbpedia.org/resource/David_Walliams

#littlebritain


http://dbpedia.org/resource/Matt_Lucas


http://dbpedia.org/resource/David_Walliams


https://twitter.com/#!/search?q=%23littlebritain


TV Preferences• Inferred from the Social Web

• tweets, FB likes, last.fm listened, etc.• weighted interests

• Represented using Linked Data web identifiers

• record-linkage:

• NLP:

facebook.com/pages/ dbpedia.org/

Brilliant british humor by Matt Lucas & David Walliams - whole range of facinating characters portraying diversity of british society

http://dbpedia.org/resource/Matt_Lucashttp://dbpedia.org/resource/David_Walliams

#littlebritain








http://xmlns.notu.be/aair/33Wednesday, October 3, 12

http://xmlns.notu.be/aair/


http://xmlns.notu.be/aair/33Wednesday, October 3, 12



http://xmlns.notu.be/wi33Wednesday, October 3, 12

http://xmlns.notu.be/wi


http://xmlns.notu.be/wi33Wednesday, October 3, 12




TV Program Enrichment

SemanticPattern-based

Recommendation Strategy

RDF GraphTV

Programs

Semantic ContentPatterns for

TV Programs

HybridRecommendation

Strategy

StatisticalSimilarity-based

Recommendation StrategyUser Ratings &

Demographics(BBC EPG

Data)

EPG Metadata(BBC)

Recommendation Service

SimilarityClusters

of Programs

User Data Analysis

End-UsersEnd Users


NoTube’s Beancounteraggregate, analyze & profile


NoTube’s User Activitiesaggregate


NoTube’s User Interestsprofile


NoTube’s User Interestsanalyze


NoTube’s User Explanationscontrol


Recommendations in NoTube

• surface interesting, new & relevant programs to individual and group users• combine in a complementary way different statistical & semantic approaches• define metrics for, e.g. serendipity, diversity, relevance

© Dan Brickley


From Predictability to Serendipity



Paul Merton looks at

Alfred Hitchcock

SecretGardens

Make ‘em laugh




Alfred Hitchcock

SecretGardens

Make ‘em laugh

documentaryformat

format

format



homes and gardens

genre

Grammy Awards

genre

genre art culture and the media

award

award


Alfred Hitchcock

SecretGardens

Make ‘em laugh

Basic patternsdocumentaryformat

format

format



homes and gardens

genre

Grammy Awards

genre


award

award

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

actor


Alfred Hitchcock

SecretGardens

Make ‘em laugh

Basic patternsHomogeneous patterns

documentaryformat

format

format

comedy

genre



homes and gardens

genre

Grammy Awards

genre


award

award

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

actor

SpikeMilligan

influence

The adventuresof Barry

McKenzie

actor

film

genr

e form

at


Alfred Hitchcock

SecretGardens

Make ‘em laugh

Basic patternsHomogeneous patterns

documentaryformat

format

format

comedy

genre



homes and gardens

genre

Grammy Awards

genre


award

award

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

actor

SpikeMilligan

influence


McKenzie

actor

film

genr

e form

at


Alfred Hitchcock

SecretGardens

Make ‘em laugh

CharlieChaplin

synopsisenrichment

ShanghaiKnights

form

at

action andadventure

genre

JackieChan

actor

influence

Basic patternsHomogeneous patternsHeterogeneous patterns

documentaryformat

format

format

comedy

genre



homes and gardens

genre

Grammy Awards

genre


award

award

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

actor

format

synopsisenrichment

Alfred Hitchcock

director

Suspicion

AcademyAwards

awar

d

Thriller

genre

SpikeMilligan

influence


McKenzie

actor

film

genr

e form

at


Alfred Hitchcock

SecretGardens

Make ‘em laugh

CharlieChaplin

synopsisenrichment

ShanghaiKnights

form

at

action andadventure

genre

JackieChan

actor

influence

Basic patternsHomogeneous patternsHeterogeneous patterns

documentaryformat

format

format

comedy

genre


Weak Semantics & Strong Statistics



Paul Merton

looks at Alfred Hitchcock

PaulMerton

anchor partnerCarolineQuentin

Life ofRiley

act

ordocumentary

form

at

art culture and the media

genre43Wednesday, October 3, 12


Paul Merton


PaulMerton


Life ofRiley

act

ordocumentary

form

at


genre

woman25 years old1 child

Userprofile



Paul Merton


PaulMerton


Life ofRiley

act

ordocumentary

form

at


genre

25 year old women with

1 child


Userprofile



Paul Merton


PaulMerton


Life ofRiley

act

ordocumentary

form

at


genre

25 year old women with

1 child

Similar users


Userprofile


NoTube’s N-Screen

N-Screendrag & drop, sharing & TV control

drag & drop, shuffle, sharing & TV control


NoTube’s N-Screensimilar programs


• means of surfacing content buried in the video collection• when users might reach a dead-end with the recommendations • adds an extra element of serendipity

NoTube’s N-Screenrandom selection


NoTube’s N-Screenexplanations

• depending on recommendation strategies• depending on the user’s task


Focus Groupshttp://notube.tv


http://notube.tv

http://notube.tv

♥

INTERESTING PROGRAMMES

watching with friends

suggestions based on programmes others have watched

linking programmes by guests, actors, presenters etc.

sharing suggestions with others in the same room

?

watching 'together apart' in different locations

changing programmes on someone else's TV

♥♥

♥

✔

✘

suggestions based on your past 'Social Web' activities

controlling the TV

drag & drop

swapping suggestions with specific friends

sharing them

✔

♥

✔

♥

sharing to watch immediately

http://notube.tv

? explaining recommendations

'shuffle' option for programme selection

?

finding

USER TESTING RESULTS December 2011, BBC R&D London10 participants : 5 male, 5 female, aged between 20 to 64

drag & drop to change what's playing on TV

♥ ✔ ✘ ? loved liked disliked unsure

For more details:


data: open, shared & accessible

infrastructures: interoperable

smart services: relevant content in right context

connected heritage


enriching collections



discovering links

style: BaroqueteacherOf: Gerrit Dou

studentOf: Pieter Lastman

self-‐portrait

mili<a place: Amsterdam, 1625 to 1650

The Night Watch

http://chip-project.org Lora Aroyo twitter: @laroyo


http://www.getty.edu/vow/ULANFullDisplay?find=rembrandt&role=&nation=&page=1&subjectid=500032894

http://www.getty.edu/vow/ULANFullDisplay?find=rembrandt&role=&nation=&page=1&subjectid=500032894

enriching with events



discovering links



personalized experience






http://chip-project.org59Wednesday, October 3, 12

http://agora.cs.vu.nl


http://chip-project.org59Wednesday, October 3, 12














Michiel Hildebrand

Lotte Belice Baltussen

Jacco van Ossenbruggen

Marteen Brinkerink

Johan OomenGuus Schreiber

Riste Gligorov

Marieke van Erp

Lourens van der Meij

Roxane Segers

Piek Vossen

Susan Legêne

Thomas Ploeger

Chiel van den Akker Frank de Bakker

Bibiana Armenta

Iina Hellsten

Geertje Jacobs

Geert-Jan Houben

Chris Welty

Natalia Stash

Yiwen Wang


Questions?

@laroyohttp://lora-aroyo.org


http://lora-aroyo.org

http://lora-aroyo.org