Camera Models CMPUT 498/613 Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2000 Readings: HZ Ch 6, 7

Camera Models

CMPUT 498/613CMPUT 498/613

Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2000

Readings: HZ Ch 6, 7

Hierarchy of cameras

Camera center

Image plane Object plane

X0(origin)xpersp

Perspective:

xparap

Para-perspective:

First order approximation of perspective

z

y

x

persp

t

t

t

f

f

P

k

j

i

1

xorth

Orthographic:

1Ty

x

orth t

t

P

0

j

i

Weak perspective:

xwp

11 Ty

x

wp t

t

k

k

P

0

j

i

Examples of camera projections

perspective Orthographic (parallel)

Animal eye:

a looonnng time ago.

Pinhole perspective projection: Brunelleschi, XVth Century.Camera obscura: XVIth Century.

Photographic camera:Niepce, 1816.

TT ZfYZfXZYX )/,/(),,(

101

0

0

1

Z

Y

X

f

f

Z

fY

fX

Z

Y

X

Pinhole camera model

Pinhole camera model

101

0

0

Z

Y

X

f

f

Z

fY

fX

101

01

01

1Z

Y

X

f

f

Z

fY

fX

PXx

0|I)1,,(diagP ff

Principal point offset

Tyx

T pZfYpZfXZYX )/,/(),,(

principal pointT

yx pp ),(

101

0

0

1

Z

Y

X

pf

pf

Z

ZpfY

ZpfX

Z

Y

X

y

x

x

x

Principal point offset

101

0

0

Z

Y

X

pf

pf

Z

ZpfY

ZpfX

y

x

x

x

camX0|IKx

1y

x

pf

pf

K calibration matrix

Camera rotation and translation

C~

-X~

RX~

cam

X10

RCR

1

10

C~

RRXcam

Z

Y

X

camX0|IKx XC~

|IKRx

t|RKP C~

Rt PXx

CCD camera

1yx

xx

p

p

K

11y

x

x

x

pf

pf

m

m

K

Finite projective camera

1yy

xx

p

ps

K

C~

|IKRP

non-singular

11 dof (5+3+3)

decompose P in K,R,C?

4p|MP 41pMC

~ MRK, RQ{finite cameras}={P4x3 | det M≠0}

If rank P=3, but rank M<3, then cam at infinity

Camera anatomy

Camera centerColumn pointsPrincipal planeAxis planePrincipal pointPrincipal ray

Column vectors

0

0

1

0

ppppp 43212

Image points corresponding to X,Y,Z directions and origin

Row vectors

1p

p

p

0 3

2

1

Z

Y

X

y

x

T

T

T

1p

p

p0

3

2

1

Z

Y

X

w

yT

T

T

note: p1,p2 dependent on image reparametrization

Camera center

0PC null-space camera projection matrix, (3 plane intersection)

The principal point

principal point

0,,,p̂ 3332313 ppp

330 Mmp̂Px

Action of projective camera on point

PXx

MdDp|MPDx 4

Forward projection

Back-projection

xPX 1PPPP

TT IPP

(pseudo-inverse)

0PC

λCxPλX

1

p-μxM

1

pM-

0

xMμμX 4

-14

-1-1

xMd -1

CD

Camera matrix decomposition

Finding the camera center

0PC (use SVD to find null-space)

Finding the camera orientation and internal parameters

KRM (use RQ decomposition ~QR)

Q R=( )-1= -1 -1QR

(if only QR, invert)

When is skew non-zero?

1yy

xx

p

ps

K

1

arctan(1/s)

for CCD/CMOS, almost always s=0

Image from image, s≠0 possible(non coinciding principal axis)

HPresulting camera:

Euclidean vs. projective

homography 44

0100

0010

0001

homography 33P

general projective interpretation

Meaningfull decomposition in K,R,t requires Euclidean image and space

Camera center is still valid in projective space

Principal plane requires affine image and space

Principal ray requires affine image and Euclidean space

Cameras at infinity

00

dP

Camera center at infinity

0Mdet

Affine and non-affine cameras

Definition: affine camera has P3T=(0,0,0,1)

Affine cameras

Affine cameras

C~

rr

C~

rr

C~

rr

KC~

|IKRP3T3T

2T2T

1T1T

0

C~

r3T0 d

t

t

dt

t

t

3T

2T2T

1T1T

33T3T

32T2T

31T1T

r

C~

rr

C~

rr

K

r-C~

rr

r-C~

rr

r-C~

rr

KP

modifying p34 corresponds to moving along principal ray

Affine cameras

003T

2T2T

1T1T

0

3T

2T2T

1T1T

0

0

/r

C~

rr

C~

rr

K

r

C~

rr

C~

rr

1

/

/

KP

dddd

d

d

dd

dd

t

t

t

t

t

t

now adjust zoom to compensate

0

2T2T

1T1T

0

C~

rr

C~

rr

KPlimP

dt

t

Error in employing affine cameras

1

βrαrX

21

1

rβrαrX

321

XPXPXP t0

point on plane parallel with principal plane and through origin, then

general points

Δ

~

~

KXPx

0

0proj

d

y

x

0

affine~

~

KXPx

d

y

x

projxaffinex

0x

Affine imaging conditions

0proj0

projaffine x-xx-xd

Approximation should only cause small error

1. much smaller than d0

2. Points close to principal point (i.e. small field of view)

Decomposition of P∞

0

02x2

0

t~R~

10

x~KP

d

10

t~R~

10

x~K 02x2-1

0d

absorb d0 in K2x2

10

0R~

10

x~t~KK

10

x~Kt~R~

10

0K

10

x~t~KR~

K

02x22x2

0-12x22x202x22x2

10

0R~

10

x~K

10

t~R~

10

0KP 02x22x2

alternatives, because 8dof (3+3+2), not more

Summary parallel projection

1000

0010

0001

P canonical representation

10

0KK 22 calibration matrix

principal point is not defined

A hierarchy of affine cameras

Orthographic projection

Scaled orthographic projection

1000

0010

0001

P

10

tRH

10

r

r

P 22T

11T

t

t

k

t

t

/10

r

r

P 22T

11T

(5dof)

(6dof)


Weak perspective projection (useful infinite appr. of unknown CCD camera)

k

t

t

y

x

/10

r

r

1

α

α

P 22T

11T

(7dof)

1. Affine camera=camera with principal plane coinciding with ∞

2. Affine camera maps parallel lines to parallel lines

3. No center of projection, but direction of projection PAD=0(point on ∞)


Affine camera

k

t

ts

y

x

A

/10

r

r

1

α

α

P 22T

11T

(8dof)

1000P 2232221

1131211

tmmmtmmm

A

affine 44100000100001

affine 33P

A

Hierarchy of cameras

Camera center

Image plane Object plane

X0(origin)xpersp

Perspective:

xparap

Para-perspective:

First order approximation of perspective

z

y

x

persp

t

t

t

f

f

P

k

j

i

1

xorth

Orthographic:

1Ty

x

orth t

t

P

0

j

i

Weak perspective:

xwp

11 Ty

x

wp t

t

k

k

P

0

j

i

Other Cameras

Pushbroom cameras

T)(X X,Y,X,T T),,(PX wyx T)/,( wyx

Straight lines are not mapped to straight lines!

(otherwise it would be a projective camera)

(11dof)

Line cameras

ZYX

pppppp

yx

232221

131211

(5dof)

c~|IRKP 22222232

Null-space PC=0 yields camera centerAlso decomposition

• Projection equationProjection equation

xxii=P=PiiXX

• Resection:Resection:– xxii,X P,X Pii

A 3D Vision Problem: Multi-view geometry - resection

Given image points and 3D points calculate camera projection matrix.

Estimating camera matrix P

•Given a number of correspondences between 3-Given a number of correspondences between 3-

D points and their 2-D image projections D points and their 2-D image projections XXii x xii, we would like to determine the , we would like to determine the camera camera

projection matrixprojection matrix PP such that such that xxi i == PXPXii for for

all all ii

A Calibration Target

courtesy of B. Wilburn

XZ

Y

Xi

xi

Estimating P: The Direct Linear Transformation (DLT) Algorithm

• xxi i == PXPXii is an equation involving homogeneous is an equation involving homogeneous

vectors, so vectors, so PXPXii and and xxi i need only be in the same need only be in the same

direction, not strictly equaldirection, not strictly equal

• We can specify “same directionality” by using a We can specify “same directionality” by using a cross product formulation:cross product formulation:

DLT Camera Matrix Estimation: Preliminaries

•Let the image point Let the image point xxii = ( = (xxii, , yyii, , wwii))TT

(remember that (remember that XXii has 4 elements) has 4 elements)

•Denoting the Denoting the jjth row of th row of PP by by ppjTjT (a 4-element (a 4-element row vector), we have: row vector), we have:

DLT Camera Matrix Estimation: Step 1

•Then by the definition of the cross product, Then by the definition of the cross product,

xxi i PXPXii is: is:


•The dot product commutes, so The dot product commutes, so ppjTjT XXii ==

XXTTii ppjj, and we can rewrite the preceding as:, and we can rewrite the preceding as:


• Collecting terms, this can be rewritten as a Collecting terms, this can be rewritten as a matrix product:matrix product:

where where 00TT = (0, 0, 0, 0) = (0, 0, 0, 0). This is a . This is a 3 x 12 matrix times a 12-element column 3 x 12 matrix times a 12-element column vector vector pp = ( = (pp11TT, , pp22TT, , pp33TT))TT

What We Just Did


• There are only two linearly independent rows here There are only two linearly independent rows here – The third row is obtained by adding The third row is obtained by adding xxii times the first row to times the first row to yyii times the times the

second and scaling the sum by second and scaling the sum by -1/-1/wwii


• So we can eliminate one row to obtain the So we can eliminate one row to obtain the

following linear matrix equation for the following linear matrix equation for the iith pair th pair of corresponding points:of corresponding points:

• Write this as Write this as AAii pp = = 00


• Remember that there are 11 unknowns which Remember that there are 11 unknowns which generate the 3 x 4 homogeneous matrix generate the 3 x 4 homogeneous matrix PP

(represented in vector form by (represented in vector form by pp))• Each point correspondence yields 2 equations Each point correspondence yields 2 equations

(the two row of (the two row of AAii)) We need at least 5 ½ point correspondences to We need at least 5 ½ point correspondences to

solve for solve for pp• Stack Stack AAii to get homogeneous linear system to get homogeneous linear system AA p p

= 0= 0

Experiment:

Documents

Camera Models CMPUT 498/613 Richard Hartley and Andrew Zisserman, Multiple View Geometry, Cambridge University Publishers, 2000 Readings: HZ Ch 6, 7