29
29 April 2008 Birkbeck College, U. Lond on 1 Geometric Model Acquisition Steve Maybank School of Computer Science and Information Systems Birkbeck College London, WC1E 7HX Edited version of the slides for the VVG Summer School, held at the University of Bath 21 September 2007

Geometric Model Acquisition

  • Upload
    kalb

  • View
    21

  • Download
    0

Embed Size (px)

DESCRIPTION

Geometric Model Acquisition. Steve Maybank School of Computer Science and Information Systems Birkbeck College London, WC1E 7HX Edited version of the slides for the VVG Summer School, held at the University of Bath 21 September 2007. Geometric Model Acquisition. - PowerPoint PPT Presentation

Citation preview

29 April 2008 Birkbeck College, U. London 1

Geometric Model Acquisition

Steve MaybankSchool of Computer Science and Information

SystemsBirkbeck College

London, WC1E 7HX

Edited version of the slides for the VVG Summer School, held at the University of Bath

21 September 2007

29 April 2008 Birkbeck College, U. London 2

Geometric Model Acquisition

Aim: make a 3D model of a scene from two or more images taken from different viewpoints.

Why is it possible: the image differences depend in part on the shapes of the objects in the scene.

29 April 2008 Birkbeck College, U. London 3

Two Images of the Same Scene

http://vasc.ri.cmu.edu/idb/images/stereo/fruitSOURCE "University of Illinois, Bill Hoff“DESCRIPTION "Fruit on table, digitized from 35mm."

29 April 2008 Birkbeck College, U. London 4

Two Images of a Point in R3

• p

c1 •

c2•

q1 •

q2 •

Epipolar plane: <c1,c2,x>

Image 1

Image 2

objectpoint

opticalcentre

opticalcentre

29 April 2008 Birkbeck College, U. London 5

Corresponding Points Points in different images

correspond,qq ~,

qq ~ if they are projections of the same

scene point p. In projective coordinates, projection is a matrix application,

qpM

qMp~~

29 April 2008 Birkbeck College, U. London 6

Method for Finding Corresponding Points

oods.neighbourh levelgrey similar have ,~ i.e. large, is

~

~.~.~.

~.~,

ncorrelatio theif ~ thatassumed isIt .~, of oodsneighbourh levelgrey be ~,Let

2/12/1

qq

uu

uu

uuuu

uuuu

qq

qquu

29 April 2008 Birkbeck College, U. London 7

Example 1 of Correlation Based Matching

Points in lh image =(150,100), (250,150), (350,250), (450,350), (250,450)Correlations (ρ) = 0.750, 0.685, 0.912, 0.644, 0.691Search area = (2d+1)x(2d+1) box, d=20.

29 April 2008 Birkbeck College, U. London 8

What Do We Need for GAM?

Description of image formation in the camera.

Description of the relative positions of the cameras.

Equations involving the measurements, the scene points and the relative positions of the cameras.

Statistical description of the errors in the measurements.

29 April 2008 Birkbeck College, U. London 9

Pinhole Camera

Light tight box

Small hole (optical centre)

Viewingscreen(image)

Object

Light rays

Central perspective projection model for imageformation (Brunelleschi, 15th C.).

29 April 2008 Birkbeck College, U. London 10

Camera Coordinate Frame

(X,Y,Z)(0,0,0)(0,0,-f)

Origin (0,0,0) at the pin hole.Focal length of the camera = f.Axes of image coordinate frame are parallel to X, Y axes

of the CCF.Image point = (-Xf/Z, -Yf/Z)

• Z

YXx

y

29 April 2008 Birkbeck College, U. London 11

Mathematical Version of the Camera Coordinate Frame

(X,Y,Z)

(0,0,0)

(0,0,f)

Origin (0,0,0) at the pin hole.Focal length of the camera = f.The image is in front of the pin hole!Image point = (Xf/Z, Yf/Z). The minus signs have gone.

• Z

yxX

Y Image plane

29 April 2008 Birkbeck College, U. London 12

Relative Position of the Cameras

Y X~

ZX Z~

Y~

The relative position of the cameras is describedby an orthogonal matrix R and a translation vector t.

R, t

29 April 2008 Birkbeck College, U. London 13

Transformation of Coordinates

YX~

ZX

Z~

Y~

If a point p has coordinates (X,Y,Z)T in the first CCF,then in the second CCF the same point p has coordinates

R, t

● p

t

Z

Y

X

R

29 April 2008 Birkbeck College, U. London 14

Properties of Orthogonal Matrices

angle.rotation the

for one and axis for the two:freedom of degrees threehave rotations The

100

0cossin

0sincos

:matrixrotation a of Example

.1det ifonly and ifrotation a represents matrix orthogonal The

.1det that followsIt

.

100

010

001

ifonly and if orthogonal is matrix 3x3 The

R

RR

R

IRRR T

29 April 2008 Birkbeck College, U. London 15

Projection Ray

Tfyx ,,point Image

Y

X

Z

0,,, :ray Projection Tfyx

Any scene point projecting to (x, y, f)T is on the projection ray.

CCF

29 April 2008 Birkbeck College, U. London 16

Projection Rays of Corresponding Points 1

The projection rays of corresponding points intersect ata scene point. Geometric model acquisition is based onthis single constraint.

For an extreme example, see http://www.wisdom.weizmann.ac.il/~vision/VideoAnalysis/Demos/Traj2Traj/hall.htm

29 April 2008 Birkbeck College, U. London 17

Projection Rays of Corresponding Points 2

Tfyx ,, Tfyx ~

,~,~~ ●

The equations of the projectionrays are known, but they hold indifferent coordinate systems.

f

y

x

f

y

x

~~

~

29 April 2008 Birkbeck College, U. London 18

Transformation of Coordinates

by CCF second in thegiven is ,,ray The Tfyx

t

f

y

x

R

such that ~, numbers realexist therethus

.~~

~

~

f

y

x

t

f

y

x

R

,~

,~,~ith equation w theofproduct scalar theby taking eliminated are ~, numbersunknown TheT

fyxt

.0.~~

~

f

y

x

R

f

y

x

t

29 April 2008 Birkbeck College, U. London 19

The Essential Matrix

.0~ thatfollowsIt .by matrix essential theDefine

.~~0

thus

0.~ then~ If

.~~such that matrix 33 theDefine

Eqq

RTEE

RqTqRqTq

RqqT

qq

tqqT

T

T

t

tTT

tT

t

t

t

29 April 2008 Birkbeck College, U. London 20

Model Acquisition

matrix essentialan find ,1,~ points matchingGiven Niiqiq

.1 ,0~such that NiiEqiqE T

such that points find and from ,Recover ixEtR

,1 ,~~NiiqitiRqiix

camera. second theof CCF in the

29 April 2008 Birkbeck College, U. London 21

Naïve Estimates of E

9. parameters ofNumber .1 subject to

,~

minimised, is expression following thefor which an Find

1

2

E

iEqiq

EN

i

T

6. parameters ofNumber .1 and subject to

,~

minimised, is expression following thefor which , Find

1

2

tIRR

iRqTiq

tR

T

N

it

T

29 April 2008 Birkbeck College, U. London 22

Better Way of Estimating E

minimiseThen small. are

~ ,

and tsmeasuremen ueunknown tr theare ~ , where

,1 ,~~~

,1 ,

Write

ii

iaia

Niiiaiq

Niiiaiq

.1 , and ,1 ,0~ subject to

,~

1

22

tIRRNiiRaTia

ii

Tt

T

N

i

29 April 2008 Birkbeck College, U. London 23

Geometric Picture

i i~ iq iq~● ●

First image Second image lEll ,

~~

l l~

29 April 2008 Birkbeck College, U. London 24

Camera Calibration

Ideal pixelcoordinates

Measured pixelcoordinates

Ideal CCF

Camera calibration is a transformation from measured pixelcoordinates to ideal pixel coordinates.

29 April 2008 Birkbeck College, U. London 25

Calibration Matrix

plane. image theofation transformaffinean defines matrix The

1,, scoordinate pixel ideal

1,, scoordinate pixel measured

100

0

21

21

K

Krq

qqq

rrr

v

us

K

T

v

u

29 April 2008 Birkbeck College, U. London 26

Fundamental Matrix

.0~therefore

~~~ ,

0~

~ :points Matching

EKrKr

rKqKrq

Eqq

qq

TT

T

The fundamental matrix F is defined by

EKKF T~

29 April 2008 Birkbeck College, U. London 27

Properties of E and F det(E)=det(Tt)det(R)=0 The matrix E is essential iff

SingularValues(E) = (σ,σ,0)

det(F)=det(K~)det(E)det(K)=0 The matrix F is fundamental iff

det(F)=0.

29 April 2008 Birkbeck College, U. London 28

Minimal Data

.)identified are 0 with , (Solutions

solutions. 10 have ,51 ,0~ equations thegeneral,In

.532 is of freedom of degrees ofNumber

EE

iiEqiq

ET

solutions. 3 have ,70~ equations thegeneral,In

7.2-9 is of freedom of degrees ofNumber

iiFqiq

FT

29 April 2008 Birkbeck College, U. London 29

Books

D.A. Forsyth and J. Ponce. Computer Vision: a modern approach. Prentice Hall, 2003.

R.C. Gonzalez and R.E. Woods. Digital Image Processing. Second edition, Prentice Hall, 2002.