View
225
Download
0
Tags:
Embed Size (px)
Citation preview
A. Frank
Multimedia
Multimedia/Video Search
A. Frank2
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
A. Frank3
Multimedia Resources
• ,Pictures, Photos, Maps תמונות – Graphics, Animations
• Audio, Sounds – שמע
• Music, MP3 –מוסיקה
• Radio, Podcasts, Speeches – הקלטות
• Video, Flash, Vlogs –חוזי
• Movies, TV – סרטים
A. Frank4
MM Search & Information Retrieval (MMIR)
• Search & information retrieval of multimedia objects is hard!!• Using SEs to find relevant, quality MM objects can be difficult
and frustrating. • Sample query: find a movie that has a large mouse chasing a
Persian cat with twins watching from above?• Common MM search is text-based; object text can be
derived from:– caption of the MM object – text surrounding the MM object– filenames of MM object and the containing page– manual/automatic annotations
• Recently, tag-based MM search is effective (social computing).• However, long due, content-based MM search on the Web
is forthcoming.
A. Frank5
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
A. Frank6
Multimedia: Google
A. Frank7
Multimedia: Google
A. Frank8
Multimedia: Yahoo
A. Frank9
Yahoo Audio Search
A. Frank10
A. Frank11
AOL Video Search
A. Frank12
Multimedia: MSN Live
A. Frank13
MSN Live
A. Frank14
MSN Live Search
A. Frank15
AltaVista Multimedia Search
A. Frank16
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
A. Frank17
Images: Picsearch
A. Frank18
Picsearch Advanced Interface
A. Frank19
Audio: FindSounds
A. Frank20
Types of Sounds
A. Frank21
Video: SearchVideo
A. Frank22
Video: Clipblast!
• Organizes and makes the video Web relevant, fast and simple to navigate.
• Video Search and Navigation technology makes it easy to search, browse and personalize the video that viewers want, when they want it.
• Crawls the entire Web for all video content available, indexing more video content providers than any other video search engine.
• Largest video distribution platform ever.
A. Frank23
Video: Clipblast
A. Frank24
Multimedia: Flurl
A. Frank25
Flurl list of sites
A. Frank26
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
A. Frank27
Flickr
A. Frank28
Flickr
A. Frank29
Flickr
A. Frank30
Image/Maps: Panoramio
A. Frank31
Image/Maps: Panoramio
A. Frank32
Image/Maps: Panoramio
A. Frank33
Video: YouTube
A. Frank34
Keotag Meta-SE
A. Frank35
Keotag Meta-SE
A. Frank36
Contents
• Multimedia (MM) and search/retrieval
• Text-based MM search in General SEs
• Text-based MM search in Vertical SEs
• Tag-based MM search
• Content-based MM search
A. Frank37
“Content-based“ Image Search/Retrieval
• “Content based” image DB or Web site(s) that contain a large number of images.
• Image analysis techniques used to describe images by their extracted visual features such as color, texture, shape and orientation.
• These objects can be represented by sets of image features and then compared with the sets extracted from other images.
• Feature sets are described by their statistics, and the computer searches for statistically likely matches.
• Queries can be formulated in terms of visual features or by similarity options.
A. Frank38
Image: Squid
• Has about 1100 images of marine creatures in its database.
• Each image shows one distinct species on a uniform background.
• Every image is processed to recover the boundary contour, which is then represented by three global shape parameters and the maxima of the curvature zero-crossing contours in its Curvature Scale Space (CSS) image.
A. Frank39
Images: Squid
A. Frank40
Images: Squid
A. Frank41
Images: Squid
A. Frank42
Images: Squid
A. Frank43
Image: retriever
• Experimental service which lets you search and explore in a selection of Flickr images by drawing a rough sketch.
• Matches the most pronounced shapes and slabs of colors.
• The results are usually fairly good, sometimes even stunning,
• But it doesn't do object/face recognition of any kind, so if drawing an outline sketch of a chair, it almost certainly won't get you one back.
A. Frank44
Images: retriever
A. Frank45
Images: retriever
A. Frank46
Images: retriever
A. Frank47
MelodyHound
A. Frank48
MelodyHound
A. Frank49
MelodyHound
A. Frank50
MelodyHound
A. Frank51
Musipedia
A. Frank52
Audio: MusiPedia
A. Frank53
Music: Musipedia
A. Frank54
Musipedia or Web Search?
A. Frank55
Video: Blinkx
• World's largest and most advanced video search engine.
• Use of visual analysis and speech recognition to better understand rich media content.
• Advanced image and audio analysis techniques are used to automatically extract information about a video in real time and to create a rich index about the video content.
• Automatically and implicitly conducts searches based on the content being viewed by each individual user.
• With the advent of blinkx TV, users can now search and access news, movie trailers, popular multimedia segments and other video formats on demand.
A. Frank56
Video: Blinkx
A. Frank57
Video: Blinkx
A. Frank58
Blinkx TV Architecture
A. Frank59
References
• Kate Greene, Better, More Accurate Image Search, Technology Review, April 9, 2007, http://www.technologyreview.com/Infotech/18501/
• Kate Greene, Surfing TV on the Internet, Technology Review, April 18, 2007, http://www.technologyreview.com/Infotech/18545/
• Amit Agarwal, 10 Interesting Things You Can Do with YouTube or Google Videos, Digital Inspiration (Blog), July 27, 2006, http://labnol.blogspot.com/2006/07/10-interesting-things-you-can-do-with.html