The Art and Science of Image Retrieval

Jonathon Harejsh2@ecs.soton.ac.uk

School of Electronics & Computer ScienceUniversity of Southampton

The Art and Science of Image Retrieval

Jonathon Harejsh2@ecs.soton.ac.uk

School of Electronics & Computer ScienceUniversity of Southampton

Why do we want to retrieve images?

http://editorial.gettyimages.com/Editorial/Sports.aspx

http://www.flickr.com/explore/interesting/7days/

Approaches to Image Retrieval

• Text-based Retrieval

• Collateral text in web-pages

• Manual indexing (keywords, captions, etc)

• Content-based Retrieval

• Similar image

• Query by sketch

Metadata Image

circa 1960: A windmill in

Romania. (Photo by Keystone/Getty Images)

Labour politician Ellen Wilkinson (1891-1947) making a speech at a 'Save Peace' demonstration in Trafalgar Square, London, on the Czech-German crisis, 18th September 1938. Photo by A. Hudson/Topical Press Agency/Hulton Archive/Getty Images)

circa 1960: The Slovnaft oil works near Bratislava at the end of one branch of the Druzba/Friendship pipeline which brings oil from the USSR to Czechoslovakia. (Photo by Three Lions/Getty Images)

search statement: romania

vocabulary control

text matcher

Text-based Retrieval

Google Images http://images.google.com

Content-based Retrieval

1569

3408

491

0 0

902

2146

5026

0 0 56

3633

0 0 0

6827

Descriptors

Raw Mediaf(x)

Descriptors

Imagesearch statement:

similarity matcher

descriptor extraction

Similar Image

RETIN: Fournier et al. http://dupont.ensea.fr/~ruven/

Query by Sketch http://labs.systemone.at/retrievr/

Content-based Retrieval Applications

Query by fax

Multi-scale Queries

picturesnapper

mapsnapper

!

:

!

Kirby Moor Country House Hotel AA**

Longtown Road, Brampton, CA8 2AB

Phone: +44 (0)1697 73893

Fax: +44 (0)1697 741847

Check Availability

Into the Future of Image Retrieval

Unified approaches

Mix content and semantic based retrieval http://echase.it-innovation.soton.ac.uk

Automatic annotation

semanticspaceshttp://forge.ecs.soton.ac.uk/projects/semanticspace

Semantic Spaces Searching by Keyword

SUN

TRAIN

Semantic Spaces Searching by Keyword

SUN

TRAIN

Ranked Search Results:

Search for images about “SUN”

Semantic Spaces Searching by Keyword

SUN

TRAIN

Ranked Search Results:

Search for images about “SUN”

SUN

Semantic Spaces Searching by Keyword

SUN

TRAIN

Ranked Search Results:

Search for images about “SUN”

SUN

http://www.echase.org

http://forge.ecs.soton.ac.uk/projects/semanticspace

http://forge.ecs.soton.ac.uk/projects/mediaengine

http://www.multimedia.ecs.soton.ac.uk