Upload
suchi
View
61
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Video Information Retrieval . Mark Ruzomberka IST 497 11/07/02. Joke. Outline. What is Video Information Retrieval (VIR) ? Reasons VIR is necessary Theoretical Where we are today Examples Problems Future Work Conclusion. What is Video Information Retrieval (VIR) ?. - PowerPoint PPT Presentation
Citation preview
Video Information Retrieval
Mark RuzomberkaIST 49711/07/02
Joke
OutlineWhat is Video Information Retrieval (VIR) ?Reasons VIR is necessaryTheoreticalWhere we are todayExamplesProblemsFuture Work Conclusion
What is Video Information Retrieval (VIR) ?
Recognition technologies Image Voice Text transcripts
Document retrieval technologies Topic segmentation Topic matching Text summarization
Presentation Technologies Combine Recognition and retrieval technologies
Result is an integrated application
VIR-Need, or Why do I care?
Consider the task of trying to find a five minute video clip of interest in a library of 1000 hour long tapes.
Consider the “go to the part where” problem
What do people want from IR
D-Lib Magazine’s asks:
“What do People want from Information Retrieval?”
# 8 Multimedia
Specificly, Reasons for Video IR
Reading is slow compared to your potential for understanding information
Humans think in pictures not words Reading is particularly slow on a computer screen Example: Daydreaming while some one is talking Reading a page in a book and not remembering what it was about
VIR makes for quicker human understanding. Palm/Grafitti 25 Hand Writing 35-40 Typing 50-70 Speaking 135-175 Reading 200 Listening 400 - 500 Thinking 500+
•Video IR allows for faster access to information
Theoretical: Think of the “Jetsons mail system”
You “talk” to the computer, Computer intelligently “talks” back to you
Where we are today
Two of Video Information Retrieval System are currently available:
Type One- keyword/text basedType Two- Content based
Type One- keyword/text based
•DVR- basic expansion of image IR, •not as interesting
Type Two- Content based
Video Mail Informedia MSR Video Skimmer
Example: Video Mail University of Cambridge
1994-1996
AT&T 1999
2000-project ended
Video Mail: Medusa network
Medusa multimedia environment at Olivetti Research Ltd. In Cambridge
It takes a modular approach unlike that of a pc or workstation Unified by a common interface to ATM network Devices plug directly into network and include:
Cameras Audio devices Networked frame buffers Processor farms Disk drives
Video Mail: Medusa Network
“The network is the computer” metaphor is used Solves storage and network speed problems Complicates expense problem
How it works-Overview
The Integrated Application
“narrow” by sender,date, time
Video Mail: Video Browser
Content is now being viewedKeywords are flagged
Video Mail: Video BrowserIn the latest version
“thumb-nailed” pictures of key frames replace color coded line of the search keyword
Informedia
The Informedia Digital Video Library Project automatically combines speech, image and natural language understanding to create a full-content searchable digital video library.
Informedia
Informedia: human factor issues
Interaction MotivationEffective usage modes
Commercial compressionVHS quality playback. Terabyte (1,000 gigabytes) of storage 1000 hours of video.
Problems
1.Human understanding2.Spoken document retrieval3.Poor video browsers4.Expensive5.Slow access to data6.Large amounts of data
Microsoft Research (MSR) Video Skimmer
Microsoft Research (MSR) Video Skimmer
Enhanced Browser Controls: Time Compression Pause Removal Textual Indices:
TOC, Notes Visual Indices
Shot Boundary FramesTimeline Markers
Jump Control (Back/Next)
Problem: Poor Content Based Video Browsers
Current VCR model allows for poor navigation “go the the part where they say” problem
Problem: Expensive
Hard drive space expensive Video adds to problem
High bandwidth needs are also expensive
Year Drive Size Drive Cost Per MB/Cost
1956 5 megabytes 50,000.00 10,000.00
1980 26 megabytes 5,000.00 193.00
1985 10 megabytes 710.00 71.00
1989 40 megabytes 1,199.00 36.00
1995 1.2 gigabytes 680.00 68.60
2000 30.0 gigabytes 249.99 0.96
•http://www.littletechshoppe.com/ns1625/winchest.html
Problem: Slow Access to Data
Broadband still not available everywhereAvailability doesn’t mean acceptanceEspecially after dot com crash 2000
Problem: Large Amounts of Data
Current Systems use MPEG2Newer compression technologies
MPEG 4-DIVX -DVD QualityVideo consumes orders of magnitude
more storage than textMPEG 7 is on horizon
Future Work ?
Sky the limit ?Sci-Fi the limit ?
Hard Drive Space, Bandwidth are current limitations.
ConclusionNot yet ready for prime timeStorage and Network Costs decreasingSuccess is in day to day usageSlowly Becoming Mainstream E.x.TivoProblems of “real world tests”
Idiot proof ATM and Medusa aren’t mainstream
Papers Video Mail Retrieval Using Voice: Report on Keyword.. - Jones, Foote, Jones.. (1994) What do people want from Information Retrieval?. Croft, Bruce W. D-Lib Magazine. (1995) Video Skimming for Quick Browsing based on Audio and Image.. - Smith, Kanade (1995) The VISION digital video library (context) - Gauch, Li et al. – (1997) Informedia: News-on-Demand Multimedia Information.. - Hauptmann, Witbrock (1997) M.G. Christel and D.J. Martin, "Information Visualization within a Digital Video Library", J.
Intelligent Info. Systems 11(3), (1998), pp. 235-257 Browsing Digital Video. Li, Gupta, Sanocki et. Al.
Questions?
Joke?
"There are 10 types of people in the world...
those who understand binary and those who don't."