Transcript
Page 1: Computer Vision: 3D Shape Reconstruction

Computer Vision: 3D Shape Reconstruction

• Use images to build 3D model of object or site

3D site model built from laser range scans collected by CMU

autonomous helicopter

Page 2: Computer Vision: 3D Shape Reconstruction

Computer Vision: Guiding Motion

• Visually guided manipulation– Hand-eye

coordination• Visually guided

locomotion– robotic vehicles

CMU NavLab II

Page 3: Computer Vision: 3D Shape Reconstruction

Computer Vision: Recognition & Classification

Page 4: Computer Vision: 3D Shape Reconstruction

Challenges in Object Recognition245 267 234 142 22 28 38121 156 187 98 73 32 12123 21 21 38 209 237 12199 87 59 197 216 244

Page 5: Computer Vision: 3D Shape Reconstruction

Object Recognition Research

Low Image Quality

Large Quantity of Data

Intra-class

Object Variation

Large number of

Object Classes

Automated Learning

Robust Algorithms

Advanced Image Enhancement

Segmentation and Hierarchical Analysis

LipsFace

Text

Building

Hand Gesture

Vehicle

Clock License Plate

Object Detection

Object Detection Issues

Qu

ality/Q

ua

ntity Issu

es

Page 6: Computer Vision: 3D Shape Reconstruction

Intra-Class Variation

Page 7: Computer Vision: 3D Shape Reconstruction

Lighting Variation

Page 8: Computer Vision: 3D Shape Reconstruction

Geometric Variation

Page 9: Computer Vision: 3D Shape Reconstruction

Simpler Problem: Classification• Fixed size input • Fixed object size, orientation, and alignment

“Object is present” (at fixed size and alignment)

“Object is NOT present”(at fixed size and alignment)

Decision

Page 10: Computer Vision: 3D Shape Reconstruction

Detection: Apply Classifier Exhaustively

Search in position

Search in scale

Page 11: Computer Vision: 3D Shape Reconstruction

View-based Classifiers

FaceClassifier #1

FaceClassifier #2

FaceClassifier #3

Page 12: Computer Vision: 3D Shape Reconstruction

1) Apply Local Operators

f1(0, 1) = #3214

f1(0, 0) = #5710

fk(n, m) = #723

Page 13: Computer Vision: 3D Shape Reconstruction

2) Look Up Probabilities

f1(0, 1) = #3214

f1(0, 0) = #5710

fk(n, m) = #723

P1( #5710, 0, 0 | obj) = 0.53

P1( #5710, 0, 0 | non-obj) = 0.56

P1( #3214, 0, 1 | obj) = 0.57

P1( #3214, 0, 1 | non-obj) = 0.48

Pk( #723, n, m | obj) = 0.83

Pk( #723, n, m | non-obj) = 0.19

Page 14: Computer Vision: 3D Shape Reconstruction

3) Make DecisionP1( #5710, 0, 0 | obj) = 0.53

P1( #5710, 0, 0 | non-obj) = 0.56

P1( #3214, 0, 1 | obj) = 0.57

P1( #3214, 0, 1 | non-obj) = 0.48

Pk( #723, n, m | obj) = 0.83

Pk( #723, n, m | non-obj) = 0.19

0.53 * 0.57 * . . . * 0.83

0.56 * 0.48 * . . . * 0.19

> l

Page 15: Computer Vision: 3D Shape Reconstruction

Two Classifiers Trained for Faces

Page 16: Computer Vision: 3D Shape Reconstruction

Eight Classifiers Trained for Cars

Page 17: Computer Vision: 3D Shape Reconstruction

Probabilities Estimated Off-Line

f1(0, 0) = #567 H1(#567, 0, 0) = H1(567, 0, 0) + 1

fk(n, m) = #350 Hk(#350, 0, 0) = Hk(#350, 0, 0) + 1

P1(#567, 0, 0) =S H1(#i, 0, 0)

H1(#567, 0, 0)

Pk(#350, 0, 0) =S Hk(#i, 0, 0)

Hk(#350, 0, 0)

Page 18: Computer Vision: 3D Shape Reconstruction

Training Classifiers• Cars: 300-500 images per viewpoint• Faces: 2,000 images per viewpoint• ~1,000 synthetic variations of each original image

– background scenery, orientation, position, frequency• 2000 non-object images

– Samples selected by bootstrapping• Minimization of classification error on training set

– AdaBoost algorithm (Freund & Shapire ‘97, Shapire & Singer ‘99) • Iterative method• Determines weights for samples

Page 19: Computer Vision: 3D Shape Reconstruction
Page 20: Computer Vision: 3D Shape Reconstruction
Page 21: Computer Vision: 3D Shape Reconstruction

Web-based Demo of Face Detectorhttp://www.vasc.ri.cmu.edu/cgi-bin/demos/findface.cgi

Page 22: Computer Vision: 3D Shape Reconstruction

Recommended