Computer Vision: 3D Shape Reconstruction

Computer Vision: 3D Shape Reconstruction

• Use images to build 3D model of object or site

3D site model built from laser range scans collected by CMU

autonomous helicopter

Computer Vision: Guiding Motion

• Visually guided manipulation– Hand-eye

coordination• Visually guided

locomotion– robotic vehicles

CMU NavLab II

Computer Vision: Recognition & Classification

Challenges in Object Recognition245 267 234 142 22 28 38121 156 187 98 73 32 12123 21 21 38 209 237 12199 87 59 197 216 244

Object Recognition Research

Low Image Quality

Large Quantity of Data

Intra-class

Object Variation

Large number of

Object Classes

Automated Learning

Robust Algorithms

Advanced Image Enhancement

Segmentation and Hierarchical Analysis

LipsFace

Text

Building

Hand Gesture

Vehicle

Clock License Plate

Object Detection

Object Detection Issues

Qu

ality/Q

ua

ntity Issu

es

Intra-Class Variation

Lighting Variation

Geometric Variation

Simpler Problem: Classification• Fixed size input • Fixed object size, orientation, and alignment

“Object is present” (at fixed size and alignment)

“Object is NOT present”(at fixed size and alignment)

Decision

Detection: Apply Classifier Exhaustively

Search in position

Search in scale

View-based Classifiers

FaceClassifier #1

FaceClassifier #2

FaceClassifier #3

1) Apply Local Operators

f1(0, 1) = #3214

f1(0, 0) = #5710

fk(n, m) = #723

2) Look Up Probabilities

f1(0, 1) = #3214

f1(0, 0) = #5710

fk(n, m) = #723

P1( #5710, 0, 0 | obj) = 0.53

P1( #5710, 0, 0 | non-obj) = 0.56

P1( #3214, 0, 1 | obj) = 0.57

P1( #3214, 0, 1 | non-obj) = 0.48

Pk( #723, n, m | obj) = 0.83

Pk( #723, n, m | non-obj) = 0.19

3) Make DecisionP1( #5710, 0, 0 | obj) = 0.53

P1( #5710, 0, 0 | non-obj) = 0.56

P1( #3214, 0, 1 | obj) = 0.57

P1( #3214, 0, 1 | non-obj) = 0.48

Pk( #723, n, m | obj) = 0.83

Pk( #723, n, m | non-obj) = 0.19

0.53 * 0.57 * . . . * 0.83

0.56 * 0.48 * . . . * 0.19

> l

Two Classifiers Trained for Faces

Eight Classifiers Trained for Cars

Probabilities Estimated Off-Line

f1(0, 0) = #567 H1(#567, 0, 0) = H1(567, 0, 0) + 1

fk(n, m) = #350 Hk(#350, 0, 0) = Hk(#350, 0, 0) + 1

P1(#567, 0, 0) =S H1(#i, 0, 0)

H1(#567, 0, 0)

Pk(#350, 0, 0) =S Hk(#i, 0, 0)

Hk(#350, 0, 0)

Training Classifiers• Cars: 300-500 images per viewpoint• Faces: 2,000 images per viewpoint• ~1,000 synthetic variations of each original image

– background scenery, orientation, position, frequency• 2000 non-object images

– Samples selected by bootstrapping• Minimization of classification error on training set

– AdaBoost algorithm (Freund & Shapire ‘97, Shapire & Singer ‘99) • Iterative method• Determines weights for samples

Web-based Demo of Face Detectorhttp://www.vasc.ri.cmu.edu/cgi-bin/demos/findface.cgi