Personalizing Media Interaction on the (Semantic & Social) Web

Personalizing Media

Interaction on the Web

Lora Aroyo

1Wednesday, October 3, 12

“interface is the message”

Aaron Koblin: Artfully visualizing our humanity, TED Talk, 2011

Lora Aroyo twitter: @laroyo

it’s complicated

HCI challengessemanticsuser data

interaction

personalization Lora Aroyo twitter: @laroyo

Flickr: elkabong 4Wednesday, October 3, 12

semantic HCI challenge

• explicit semantics (often from open sources, e.g. LOD) used for system decisions and results

• use facetted presentation, searching and browsing of information

• use typically classifications, typologies or other structures of concepts

• integrate & aggregate data from different sources

user data HCI challenge

• search, e.g. keyword, faceted

• browse, story lines, narratives through collections

• annotations of multimedia, e.g. (collaborative) tagging, professional annotation of text, images and video, tagging games

• explanations, hints, user feedback, e.g. explanation of recommendation results, explanation of autocompletion suggestions

interaction HCI challenge

interaction with semantics

http://www.bbc.co.uk/programmes/b00c06n2.rdf

Lora Aroyo twitter: @laroyo11Wednesday, October 3, 12

• data challenges multiply with Web openness

• social recommendations are preferred

• it is not about (single) algorithms any more

• it is about the metrics

personalization challenge

applications

personalized TV

User Perspective

• what is the role of social recommendations?• how do people watch TV together?• how devices influence watching?• what are perceived trade-offs for privacy vs. personalization?

Technology Perspective

• what is the role of metadata in TV & Apps?• what are ways to share information in real time?• what are ways to sync TV & other metadata?• what is an easy way for devices to find & talk to each other?

choices, choices ...

lost in space© RedBee Slides at MIPCube2012

always missing something© RedBee Slides at MIPCube2012

demand for experience23Wednesday, October 3, 12

Social Recommendations© Ericsson ComsumerLab: TV & Web Consumer Trends 2011

Social Recommendations

I usually find NEW programmes to watch by:

RESULTS OF NOTUBE SOCIAL WEB & TV SURVEY 2012

Social Recommendations

I usually find NEW programmes to watch by:

Social TV

interface explorations

Linking Web Data to User Profiles

TV Preferences• Inferred from the Social Web

• tweets, FB likes, last.fm listened, etc.• weighted interests

• Represented using Linked Data web identifiers

• record-linkage:

• NLP:

facebook.com/pages/ dbpedia.org/

http://dbpedia.org/resource/Matt_Lucashttp://dbpedia.org/resource/David_Walliams

#littlebritain

TV Preferences• Inferred from the Social Web

• tweets, FB likes, last.fm listened, etc.• weighted interests

• Represented using Linked Data web identifiers

• record-linkage:

• NLP:

facebook.com/pages/ dbpedia.org/

Brilliant british humor by Matt Lucas & David Walliams - whole range of facinating characters portraying diversity of british society

http://dbpedia.org/resource/Matt_Lucashttp://dbpedia.org/resource/David_Walliams

#littlebritain

http://xmlns.notu.be/aair/33Wednesday, October 3, 12

http://xmlns.notu.be/wi33Wednesday, October 3, 12

TV Program Enrichment

SemanticPattern-based

Recommendation Strategy

RDF GraphTV

Programs

Semantic ContentPatterns for

TV Programs

HybridRecommendation

Strategy

StatisticalSimilarity-based

Recommendation StrategyUser Ratings &

Demographics(BBC EPG

EPG Metadata(BBC)

Recommendation Service

SimilarityClusters

of Programs

User Data Analysis

End-UsersEnd Users

NoTube’s Beancounteraggregate, analyze & profile

NoTube’s User Activitiesaggregate

NoTube’s User Interestsprofile

NoTube’s User Interestsanalyze

NoTube’s User Explanationscontrol

Recommendations in NoTube

• surface interesting, new & relevant programs to individual and group users• combine in a complementary way different statistical & semantic approaches• define metrics for, e.g. serendipity, diversity, relevance

From Predictability to Serendipity

Paul Merton looks at

Alfred Hitchcock

SecretGardens

Make ‘em laugh

Alfred Hitchcock

SecretGardens

Make ‘em laugh

documentaryformat

format

homes and gardens

Grammy Awards

genre art culture and the media

Alfred Hitchcock

SecretGardens

Make ‘em laugh

Basic patternsdocumentaryformat

format

homes and gardens

Grammy Awards

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

Alfred Hitchcock

SecretGardens

Make ‘em laugh

Basic patternsHomogeneous patterns

documentaryformat

format

comedy

homes and gardens

Grammy Awards

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

SpikeMilligan

influence

The adventuresof Barry

McKenzie

e form

Alfred Hitchcock

SecretGardens

Make ‘em laugh

Basic patternsHomogeneous patterns

documentaryformat

format

comedy

homes and gardens

Grammy Awards

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

SpikeMilligan

influence

McKenzie

e form

Alfred Hitchcock

SecretGardens

Make ‘em laugh

CharlieChaplin

synopsisenrichment

ShanghaiKnights

action andadventure

JackieChan

influence

Basic patternsHomogeneous patternsHeterogeneous patterns

documentaryformat

format

comedy

homes and gardens

Grammy Awards

PaulMerton

anchor

partner

CarolineQuentin

Life ofRiley

format

synopsisenrichment

Alfred Hitchcock

director

Suspicion

AcademyAwards

Thriller

SpikeMilligan

influence

McKenzie

e form

Alfred Hitchcock

SecretGardens

Make ‘em laugh

CharlieChaplin

synopsisenrichment

ShanghaiKnights

action andadventure

JackieChan

influence

Basic patternsHomogeneous patternsHeterogeneous patterns

documentaryformat

format

comedy

Weak Semantics & Strong Statistics

Paul Merton

looks at Alfred Hitchcock

PaulMerton

anchor partnerCarolineQuentin

Life ofRiley

ordocumentary

art culture and the media

genre43Wednesday, October 3, 12

Paul Merton

PaulMerton

Life ofRiley

ordocumentary

woman25 years old1 child

Userprofile

Paul Merton

PaulMerton

Life ofRiley

ordocumentary

25 year old women with

1 child

Userprofile

Paul Merton

PaulMerton

Life ofRiley

ordocumentary

25 year old women with

1 child

Similar users

Userprofile

NoTube’s N-Screen

N-Screendrag & drop, sharing & TV control

drag & drop, shuffle, sharing & TV control

NoTube’s N-Screensimilar programs

• means of surfacing content buried in the video collection• when users might reach a dead-end with the recommendations • adds an extra element of serendipity

NoTube’s N-Screenrandom selection

NoTube’s N-Screenexplanations

• depending on recommendation strategies• depending on the user’s task

Focus Groupshttp://notube.tv

INTERESTING PROGRAMMES

watching with friends

suggestions based on programmes others have watched

linking programmes by guests, actors, presenters etc.

sharing suggestions with others in the same room

watching 'together apart' in different locations

changing programmes on someone else's TV

♥♥

suggestions based on your past 'Social Web' activities

controlling the TV

drag & drop

swapping suggestions with specific friends

sharing them

sharing to watch immediately

http://notube.tv

? explaining recommendations

'shuffle' option for programme selection

finding

USER TESTING RESULTS December 2011, BBC R&D London10 participants : 5 male, 5 female, aged between 20 to 64

drag & drop to change what's playing on TV

♥ ✔ ✘ ? loved liked disliked unsure

For more details:

data: open, shared & accessible

infrastructures: interoperable

smart services: relevant content in right context

connected heritage

enriching collections

discovering links

style: BaroqueteacherOf: Gerrit Dou

studentOf: Pieter Lastman

self-‐portrait

mili<a place: Amsterdam, 1625 to 1650

The Night Watch

http://chip-project.org Lora Aroyo twitter: @laroyo

enriching with events

discovering links

personalized experience

http://chip-project.org59Wednesday, October 3, 12

http://agora.cs.vu.nl

Michiel Hildebrand

Lotte Belice Baltussen

Jacco van Ossenbruggen

Marteen Brinkerink

Johan OomenGuus Schreiber

Riste Gligorov

Marieke van Erp

Lourens van der Meij

Roxane Segers

Piek Vossen

Susan Legêne

Thomas Ploeger

Chiel van den Akker Frank de Bakker

Bibiana Armenta

Iina Hellsten

Geertje Jacobs

Geert-Jan Houben

Chris Welty

Natalia Stash

Yiwen Wang

Questions?

@laroyohttp://lora-aroyo.org

Personalizing Media Interaction on the (Semantic & Social) Web

Technology

The Effect of Semantic Interaction on Foraging in …infovis.cs.vt.edu › sites › default › files › effect-semantic...The Effect of Semantic Interaction on Foraging in Text

PERSONALIZING NEGLIGENCE LAW

The Interaction of Morphosyntactic and Semantic …¨€語研究（Gengo Kenkyu）149: 43–59（2016） The Interaction of Morphosyntactic and Semantic Processing in Japanese Sentence

SWAHA08 - Personalizing Human Interaction through Hybrid Ontological Profiling: Cultural Heritage Case Study

Personalizing Your Printing

Interaction Design for the Semantic Webhomepages.cwi.nl/~lynda/courses/USI07/IntDesSW-IntroUSI07.pdf · Interaction Design for the Semantic Web Lynda Hardman CWI, Semantic Media Interfaces

Personalizing SAP BusinessObjects Explorer …...Personalizing SAP BusinessObjects Explorer Information Spaces Applies to: SAP BusinessObjects Explorer and personalizing the display

On the interaction of structural and semantic casewdl/Synsem.pdf · On the interaction of structural and semantic case! Dieter Wunderlich, Renate Lakämper Seminar für Allgemeine

Semantic Interaction for Visual Text Analyticsinfovis.cs.vt.edu/sites/default/files/PDF_2.pdf · 2012-07-13 · Semantic Interaction for Visual Text Analytics Alex Endert Virginia

Personalized Interaction With Semantic Information Portals Eric Schwarzkopf DFKI (Eric.Schwarzkopf@dfki.de)

Personalizing Actions in Context for Risk ... - Semantic Web · represented separately by a graph consisting of a set of semantic entities and binary relations between entities, all

Collecting Interaction Traces in Distributed Semantic Wikis · 2018-07-23 · Collecting Interaction Traces in Distributed Semantic Wikis Anh-Hoang Le, Marie Lefevre, and Amélie

Personalizing Education

How motion verbs are special: The interaction of semantic · PDF file · 2005-11-25How motion verbs are special: The interaction of semantic and pragmatic information in aspectual

Personalizing the Interaction in a Web-based Educational ...hermes.di.uoa.gr/inspire/UMUAI-Papanikolaou.pdf · ways: curriculum sequencing, adaptive navigation support, adaptive presentation,

Culatta Personalizing Learning

Semantic Web Interaction

DeepSI: Interactive Deep Learning for Semantic Interaction

Syntactic-Semantic Interaction in Israeli Sign Language ...sandlersignlab.haifa.ac.il/irit_articles/Meir_SLL_1998.pdfSyntactic-Semantic Interaction in Israeli Sign Language Verbs The

A Platform for Object-Action Semantic Web Interaction