Projective cameras

• Motivation• Elements of Projective Geometry• Projective structure from motion

Planches :– http://www.di.ens.fr/~ponce/geomvis/lect3.ppt

– http://www.di.ens.fr/~ponce/geomvis/lect3.pdf

Weak-Perspective Projection Model

r(p and P are in homogeneous coordinates)

p = A P + b (neither p nor P is in hom. coordinates)

p = M P (P is in homogeneous coordinates)

Affine Spaces: (Semi-Formal) Definition

Affine projections induce affine transformations from planesonto their images.

Affine Structure from Motion

Reprinted with permission from “Affine Structure from Motion,” by J.J. (Koenderink and A.J.Van Doorn, Journal of the Optical Society of America A,8:377-385 (1990). 1990 Optical Society of America.

Given m pictures of n points, can we recover• the three-dimensional configuration of these points?• the camera configurations?

(structure)(motion)

Geometric affine scene reconstruction from two images(Koenderink and Van Doorn, 1991).

The Projective Structure-from-Motion Problem

Given m perspective images of n fixed points P we can write

Problem: estimate the m 3x4 matrices M andthe n positions P from the mn correspondences p .

i

j ij

2mn equations in 11m+3n unknowns

Overconstrained problem, that can be solvedusing (non-linear) least squares!

j

The Projective Ambiguity of Projective SFM

If M and P are solutions, i j

So are M’ and P’ wherei j

and Q is an arbitrary non-singular 4x4 matrix.

When the intrinsic and extrinsic parameters are unknown

Q is a projective transformation.

Projective Spaces: (Semi-Formal) Definition

A Model of P( R )3

Projective Subspaces and Projective Coordinates

Projective Subspaces and Projective Coordinates

Projectivecoordinates

P

Projective Subspaces

Given a choice of coordinate frame

Line: Plane:

When do m+1 points define a p-dimensional subspace Y of ann-dimensional projective space X equipped with some coordinateframe?

Writing that all minors of size (p+1)x(p+1) of D are equal tozero gives the equations of Y.

Rank ( D ) = p+1, where

mnnn

m

m

xxx

xxx

xxx

D

...

............

...

...

10

11101

01000

Hyperplanes and duality

This can be rewritten as u0x0+u1x1+…+unxn = 0, orT P = 0, where = (u0,u1,…,un)T.

Consider n+1 points P0, … , Pn-1, P in a projective space X ofdimension n. They liein the same hyperplanewhen Det(D)=0.

nnn xxx

xxx

xxx

D

...

............

...

...

10

11101

01000

Hyperplanes form a dual projective space X* of X, and anytheorem that holds for points in X holds for hyperplanes in X*.

What is the dual of a straight line?

Affine and Projective Spaces

Affine and Projective Coordinates

Cross-Ratios

Collinear points

Pencil of coplanar lines Pencil of planes

{A,B;C,D}=sin(+)sin(+)

sin(++)sin

Cross-Ratios and Projective Coordinates

Along a line equipped with the basis

In a plane equipped with the basis

In 3-space equipped with the basis

*

Projective Transformations

Bijective linear map:

Projective transformation:( = homography )

Projective transformations map projective subspaces ontoprojective subspaces and preserve projective coordinates.

Projective transformations map lines onto lines andpreserve cross-ratios.

Perspective Projections induce projective transformationsbetween planes.

Projective Shape

Two point sets S and S’ in some projective space X are projectively equivalent when there exists a projective transformation : X X such that S’ = ( S ).

Projective structure from motion = projective shape recovery.

= recovery of the corresponding motion equivalence classes.

Epipolar Geometry

• Epipolar Plane

• Epipoles

• Epipolar Lines

• Baseline

Geometric SceneReconstruction

Idea: use (A,B,C,D,F) as a projective basis and reconstruct O’ and O’’, assuming that the epipolesare known.

A

B

CD F

GH

IJ

K

E

O’

O’’

Geometric SceneReconstruction II

Idea: use (A,O”,O’,B,C)as a projective basis, assuming again that theepipoles are known.

Multi-View Geometry

• Epipolar Geometry

• The Essential Matrix

• The Fundamental Matrix

Epipolar Geometry

• Epipolar Plane

• Epipoles

• Epipolar Lines

• Baseline

Epipolar Constraint

• Potential matches for p have to lie on the corresponding epipolar line l’.

• Potential matches for p’ have to lie on the corresponding epipolar line l.

Epipolar Constraint: Calibrated Case

Essential Matrix(Longuet-Higgins, 1981)

Properties of the Essential Matrix

• E p’ is the epipolar line associated with p’.

• E p is the epipolar line associated with p.

• E e’=0 and E e=0.

• E is singular.

• E has two equal non-zero singular values (Huang and Faugeras, 1989).

T

T

Epipolar Constraint: Small MotionsTo First-Order:

Pure translation:Focus of Expansion

Epipolar Constraint: Uncalibrated Case

Fundamental Matrix(Faugeras and Luong, 1992)

Properties of the Fundamental Matrix

• F p’ is the epipolar line associated with p’.

• F p is the epipolar line associated with p.

• F e’=0 and F e=0.

• F is singular.

T

T

The Eight-Point Algorithm (Longuet-Higgins, 1981)

|F | =1.

Minimize:

under the constraint2

Non-Linear Least-Squares Approach (Luong et al., 1993)

Minimize

with respect to the coefficients of F , using an appropriate rank-2 parameterization.

The Normalized Eight-Point Algorithm (Hartley, 1995)

• Center the image data at the origin, and scale it so themean squared distance between the origin and the data points is 2 pixels: q = T p , q’ = T’ p’.

• Use the eight-point algorithm to compute F from thepoints q and q’ .

• Enforce the rank-2 constraint.

• Output T F T’.T

i i i i

i i

Data courtesy of R. Mohr and B. Boufama.

Wit

hout

nor

mal

izat

ion

Wit

h no

rmal

izat

ion

Mean errors:10.0pixel9.1pixel

Mean errors:1.0pixel0.9pixel

Trinocular Epipolar Constraints

These constraints are not independent!

Trinocular Epipolar Constraints: Transfer

Given p and p , p can be computed

as the solution of linear equations.

1 2 3

Trifocal Constraints

Trifocal Constraints

All 3x3 minorsmust be zero!

Calibrated Case

Trifocal Tensor

Trifocal Constraints

Uncalibrated Case

Trifocal Tensor

Trifocal Constraints: 3 Points

Pick any two lines l and l through p and p .

Do it again.2 3 2 3

T( p , p , p )=01 2 3

Properties of the Trifocal Tensor

Estimating the Trifocal Tensor

• Ignore the non-linear constraints and use linear least-squaresa posteriori.

• Impose the constraints a posteriori.

• For any matching epipolar lines, l G l = 0.

• The matrices G are singular.

• They satisfy 8 independent constraints in theuncalibrated case (Faugeras and Mourrain, 1995).

2 1 3T i

1i

For any matching epipolar lines, l G l = 0. 2 1 3T i

The backprojections of the two lines do not define a line!

Multiple Views (Faugeras and Mourrain, 1995)

Two Views

Epipolar Constraint

Three Views

Trifocal Constraint

Four Views

Quadrifocal Constraint(Triggs, 1995)

Geometrically, the four rays must intersect in P..

Quadrifocal Tensorand Lines

Scale-Restraint Condition from Photogrammetry