Upload
ruth-clark
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
1
Time-Sensitive Web Image Ranking and Retrieval via Dynamic Multi-Task
Regression
Gunhee Kim Eric P. Xing
School of Computer Science, Carnegie Mellon University
February 6, 2013
Image Ranking and Retrieval
Goal: Find the images for a given query
ex. Cardinal Text-based image retrieval
2
Image Ranking and Retrieval
ex. Cardinal
northern_cardinal_glamour.jpgFile name
http://www.allaboutbirds.org/guide/Northern_Cardinal/id
Url
3
Text-based image retrieval
• Ambiguity and noise due to mismatch.
• Scalable and successful so far
Goal: Find the images for a given query
Recent Image Ranking and Retrieval
Various efforts to improve text-based image search
User relevance feedback [Wang et al. CVPR 11]
Text-based search by apple
chosen by a user
Reranking on visual features
Pseudo-relevance feedback[Liu et al. CVPR 11]
Human labeled training data[Yang et al. MM10]
Image click data [Jain et al. WWW11]
4
5
Time-Sensitive Image Ranking and Retrieval
From experiments of 7.5 millions of Flickr images of 30 topicswe found three good reasons …
Discovery of temporal patterns of Web image collections
• [D08] Dakka et al. CIKM 2008• [M09] Metzler et al. SIGIR 2009• [K10] Kulkani et al, WSDM 2011
• [V11] Amodeo et al, CIKM2011• [R12] Radinsky et al, WWW 2012• …..
No previous work using temporal info on image retrieval
[Related work] Exploring temporal dynamics of Web queries
• Popular search keywords and relevant documents change over time.
• ex) Keyword search, Product search, News recommendation
6
Why Time-Sensitive Image Retrieval? (1/3)
1. Knowing when search takes place is useful to infer users' implicit intents.
Cardinal: (1) the red bird in America.
Fall to Winter (Sep. ~ Feb.)
Bing
(2) Arizona cardinals (football) (3) St. Louis cardinals (baseball)
Spring to Fall (Mar. ~ Oct.)
• Severely redundant. Almost identical all year long.
7
Why Time-Sensitive Image Retrieval? (1/3)
1. Knowing when search takes place is useful to infer users' implicit intents.
at May 4, 2009
at Feb. 7, 2009 Football
Bing
Ourresults
baseball
• Diversity can make search interesting.
8
Why Time-Sensitive Image Retrieval? (2/3)
2. Timing suitability can be used as a complementary attribute to relevance.
Bing
at May 4, 2009
at Feb. 7, 2009
Ourresults
• There are so many almost equally good images.Background: snow
Background: Green Baby birds or eggs
9
Why Time-Sensitive Image Retrieval? (3/3)
3. Temporal information is synergetic in personalizedimage retrieval.
Louisville Men's College Basketball
At Nov. 7, 2009 for user 30033302
Each user’ term usages are relatively stationary, and predictable once they are learned.
10
Algorithm
Regularized multi-task regression on multivariate point process
• Goal: Scalably learn temporal models for each topic keyword.
• Multi-task framework: allows multiple image descriptors.
• Several regularization schemes
• Personalization by locally-weighted learning
12
Multivariate Point Process Models
Given a stream of hornet pictures up to T
Clustering by descriptor 1 Clustering by descriptor 2
Time t1 t2 t3 t5 t6 t7 t9 t10
1st descriptor (v1) 2nd descriptor (v2)
13
Regularized GLM on Point Processes
Given a stream of hornet pictures up to T
Time t1 t2 t3 t5 t6 t7 t9 t10
Formulate a regression between occurrence rates and covariates.
Covariates: any likely factors to be associated with image occurrence (ex. Time, season, and other external events)
Compute sparse regularized MLE solutions
For each visual cluster, we select only a small number of strong factors.
(v1,v2) (3, 2) (3, 2) (3, 2) (2, 1) (1, 3) (1, 4) (2, 3) (2, 1)