User Profiling by Social Curation By GENG Xue Supervised by
Prof. Chua Tat-Seng
Slide 2
A Crowd of Social Network Platforms
Slide 3
Changes in Concerts & Pope Inauguration 19902010
Slide 4
Exponential Multimedia 2013 Internet Trends
http://www.kpcb.com/insights/2013-internet-trends
http://www.kpcb.com/insights/2013-internet-trends The Visual nature
of the web increases exponentially
Slide 5
One of Kind Big Multimedia
Slide 6
How to Deliver Meaningful Contents to the Right Person ? User
Profiling
Slide 7
Definition A process to establish user profiles by extracting
& representing the characteristics and preferences of users.
Better Service Better Experience
Slide 8
Recommendation Recommend ? Similar A B Recommend !
Slide 9
Recommendation Similar Basic info & Social relationships A
B
Slide 10
So, User Profiling by Multimedia Analysis
Slide 11
However, Multimedia data are very diverse & unorganized.
Traditional approaches fail.
Slide 12
Solutions Structured multimedia & Social Intelligence.
Flickr GalleriesFacebook Like
Slide 13
Social Curation Modern Funnel: Social Curation. People select
organize keep track of items they like.
Slide 14
Social Curation Services (Pinterest)
Slide 15
Boards to which image are re-pinned Board name and holder
Why SCSs good? Content-centric network Social intelligence to
refine content models
Slide 19
Framework Profile Structure (Ontology) Learning Profile
Structure Refinement User Profiles
Slide 20
Ontology-based user profiles (e.g., Fashion Domain)
Slide 21
Profile Ontology Construction
Slide 22
An example (pegged pants)
Slide 23
Profile Ontology Learning Idea : sibling samples are more
visually similar, classifiers should be more distinct. V dresses,
Strapless dresses, Halter dress, One Shoulder dress
Slide 24
Failure Cases Features may be Wrong
Slide 25
Refinement of user profiles Organized contents without social
intelligence (content-centric network). Social intelligence to
refine user profiles.
Slide 26
If images are shared by more users/boards simultaneously, they
more likely belong to the same preference. User/Board-level
Connection
Slide 27
Observation: Shopping Sites Recommendation T shirt People also
see these
Slide 28
Observation: Movie Recommendation Recommendation
Slide 29
If images are shared by more users/boards simultaneously, they
more likely belong to the same preference. User/Board-level
Connection
Slide 30
Content-level Connection Similar images share similar visual
cues and semantics. More Similar
Slide 31
Mathematical Social Intelligence
Slide 32
Refinement of User Profiles Multi-level connections are
incorporated into the low-rank method Before refinement After
refinement User-level connection Bundle-level connection
Visual-level connection Semantic-level connection
Slide 33
Visualization of User Profiles It is a vector: (, 0.13, , 0.23,
, 0.3, )
Slide 34
Experimental Data Data collection 1,239 users, 1,538,658
images. Profile learning and refinement We split labeled images
equally into training/testing sets as the ground truths. Image
recommendation We split the dataset by pin-time for
training/testing We added half noisy data out of fashion domain to
simulate real world recommendation system.
Slide 35
Evaluation of Profile Learning Failure Cases: a)Some image
samples are too fine-grained. b)Some concepts tend to co-occur in
the same image frequently.
Slide 36
Evaluation of Profile Refinement Failure Cases: a)Sparse &
noisy connections from some outdated items. b)Some items are
co-repined leading to similar multi-level connections
Slide 37
Evaluation of Image Recommendation
Slide 38
Conclusion Social Curation is NEW! It has Well-organized
Contents Social Intelligence We test it on Pinterest (fashion
domain).