Advanced Computer Vision Devi Parikh Electrical and Computer Engineering

Preview:

Citation preview

Advanced Computer Vision

Devi Parikh

Electrical and Computer Engineering

Plan for today

• Topic overview

• Introductions

• Course overview: – Logistics– Requirements

• Please interrupt at any time with questions or comments

Computer Vision

• Automatic understanding of images and video

– Computing properties of the 3D world from visual data (measurement)

– Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities. (perception and interpretation)

– Algorithms to mine, search, and interact with visual data (search and organization)

Kristen Grauman

What does recognition involve?

Fei-Fei Li

Detection: are there people?

Activity: What are they doing?

Object categorization

mountain

building

tree

banner

vendorpeople

street lamp

Instance recognition

Potala Palace

A particular sign

Scene and context categorization

• outdoor

• city

• …

Attribute recognition

flat

graymade of fabric

crowded

Why recognition?

• Recognition a fundamental part of perception– e.g., robots, autonomous agents

• Organize and give access to visual content– Connect to information – Detect trends and themes

• Where are we now?

Kristen Grauman

We’ve come a long way…

We’ve come a long way…

We’ve come a long way…

Posing visual queries

Kooaba, Bay & Quack et al.

Yeh et al., MIT

Belhumeur et al.

Kristen Grauman

Exploring community photo collections

Snavely et al.

Simon & SeitzKristen Grauman

http://www.darpa.mil/grandchallenge/gallery.asp

Autonomous agents able to detect objects

Kristen Grauman

We’ve come a long way…

Fischler and Elschlager, 1973

We’ve come a long way…

We’ve come a long way…

Dollar et al., BMVC 2009

Still a long way to go…

Dollar et al., BMVC 2009

Dollar et al., BMVC 2009

Dollar et al., BMVC 2009

Challenges

Challenges: robustness

Illumination

Object pose

ViewpointIntra-class appearance

Occlusions

Clutter

Kristen Grauman

Challenges: context and human experience

Context cues

Kristen Grauman

Challenges:context and human experience

Context cues Function Dynamics

Video credit: J. DavisKristen Grauman

Challenges: scale, efficiency

• Half of the cerebral cortex in primates is devoted to processing visual information

• ~20 hours of video added to YouTube per minute

• ~5,000 new tagged photos added to Flickr per minute

• Thousands to millions of pixels in an image

• 30+ degrees of freedom in the pose of articulated objects (humans)

• 3,000-30,000 human recognizable object categories

Kristen Grauman

Challenges: learning with minimal supervision

MoreLess

Cropped to

object, parts and

classes labeled

Classes labeled,

some clutter

Unlabeled,

multiple

objects

Kristen Grauman

Slide from Pietro Perona, 2004 Object Recognition workshop

Slide from Pietro Perona, 2004 Object Recognition workshop

Recognizing flat, textured objects (like books, CD

covers, posters)

Reading license plates, zip codes, checks

Fingerprint recognition

Frontal face detection

What kinds of things work best today?

Kristen Grauman

Inputs in 1963…

L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.

Kristen Grauman

Personal photo albums

Surveillance and security

Movies, news, sports

Medical and scientific images

Slide credit; L. Lazebnik

… and inputs today

… and inputs today

Images on the Web Movies, news, sports

916,271 titles

10 mil. videos, 65,000 added daily

350 mil. photos, 1 mil. added daily

1.6 bil. images indexed as of summer 2005

Satellite imagery City streets

Slide credit; L. Lazebnik

Understand and organize and index all this data!!

Introductions

• What is your name?• Which program are you in? How far along?• What is your research area and current project about?

– Take a minute to explain it to us– In a way that we can all follow

• Have you taken a computer vision course before? Machine learning or pattern recognition?

• What are you hoping to get out of this class?

This course

• ECE 5984• TR 3:30 pm to 4:45 pm• Hutcheson (HUTCH) 207• Office hours: by appointment (email)

• Course webpage: http://filebox.ece.vt.edu/~S14ECE5984/(Google me My homepage Teaching)

This course

• Focus on current research in computer vision

• High-level recognition problems, innovative applications.

Goals

• Understand state-of-the-art approaches

• Analyze and critique current approaches

• Identify interesting research questions

• Present clearly and methodically

Expectations

• Discussions will center on recent papers in the field [15%]

• Paper reviews each class [25%]– Can have 3 late days over the course of the semester

• Presentations (2-3 times) [25%]

– Papers and background reading

– Experiments

• Project [35%]No “Assignments”,

Exams, etc.

Prerequisites

• Course in computer vision

• Courses in machine learning is a plus

Paper reviews

• For each class – Review one paper in detail– Review one paper at a high-level– (Reduced from last time I offered this course)

• Email me reviews by noon (12:00 pm) the day of the class

• Skip reviews the classes you are presenting.

Paper review guidelines• One page• Detailed review:

– Brief (2-3 sentences) summary – Main contribution– Strengths? Weaknesses? – How convincing are the experiments? Suggestions to improve them?– Extensions? Applications?– Additional comments, unclear points

• High-level review:– Problem being addressed– High-level intuition/idea of approach

• Relationships observed between the papers we are reading• Will pick on students in class during discussions• Write in your own words• Write well, proof read

Paper presentation guidelines

• Papers

• Experiments

Papers• Read selected papers in topic area and look at

background papers as necessary• Well-organized talk, 45 minutes• What to cover?

– Topic overview, motivation– For selected papers:

• Problem overview, motivation• Algorithm explanation, technical details• Experimental set up, results• Strengths, weaknesses, extensions

– Any commonalities, important differences between techniques covered in the papers.

• See class webpage for more details.

Experiments

• Implement/download code for a main idea in the paper and evaluate it:– Experiment with different types of training/testing data sets

– Evaluate sensitivity to important parameter settings

– Show an example to analyze a strength/weakness of the approach

– Show qualitative and quantitative results

Tips

• Look up papers and authors. Their webpage may have data, code, slides, videos, etc.– Make sure talk flows well and makes sense as a whole.– Cite ALL sources.

• Don’t forget the high-level picture.

• Give a very clear and well-organized and thought out talk.

• Will interrupt if something is not clear

Tips• Make sure you are saying everything we need to

know to understand what you are saying.

• Make sure you know what you are talking about.

• Think about your audience.

• Make your talks visual (images, video, not lots of text).

ProjectsPossibilities:

– Extension of a technique studied in class– Analysis and empirical evaluation of an

existing technique– Comparison between two approaches– Design and evaluate a novel approach– Be creative!

Can work with a partner

Talk to me if you need help with ideas

Project timeline• Project proposals (1 page) [10%]

– March 6th

• Mid-semester presentations (10 minutes) [20%]– March 27th and April 1st

• Final presentations (20 minutes) [35%]– April 24th to May 6th

• Project reports (4 pages) [35%]– May 12th

– Could serve as a first draft of a conference submission!

Implementation

• Use any language / platform you like

• No support for code / implementation issues will be provided

Miscellaneous

• Best presentation, best project and best discussion prizes!– We will vote– Dinner

• Feedback welcome and useful

Coming up• Read the class webpage

– Schedule is up– Tour of schedule

• Select 6 dates (topics) you would like to present – Email me by Wednesday (tomorrow)– Webpage shows how many people have already signed

up for a topic– Select those that have fewer selections

• Overview of my research on Thursday– How many of you were at the ECE grad seminar in

November?

Questions?

See you Thursday!

Recommended