Uncalibrated reconstruction

Calibration with a rig Uncalibrated epipolar geometry Ambiguities in image formation Stratified reconstruction Autocalibration with partial scene knowledge

S. Soatto, J. Kosecka

ICRA 2003 2

Uncalibrated Camera

• Image Plane Coordinates

• Perspective projection

Calibrated camera

• Pixel coordinates

• Camera intrinsic parameters

• Projection Matrix

Uncalibrated camera

xyc

ICRA 2003 3

Taxonomy of uncalibrated reconstruction

K is known, back to calibrated case K is unknown

Calibration (with a rig) – estimate K Uncalibrated reconstruction despite the lack

of knowledge of K Autocalibration (recover K from images)

Use complete scene knowledge (a rig) Use partial knowledge

Parallel lines, vanishing points, planar motion, constant intrinsic

Ambiguities, stratification (multiple views)

ICRA 2003 4

Calibration with a rig

• Given 3-D coordinates on known object

• Eliminate unknown scales

• Recover Projection Matrix•

• Factor the into and using QR decomposition• Solve for translation

ICRA 2003 5

Uncalibrated camera vs. distorted space

Inner product in Euclidean space : compute distances and angles

Calibration K transforming spatial coordinates

Transformation induced on the inner product

S (the metric of the space) and K are equivalent

ICRA 2003 6

Calibrated vs. Uncalibrated Space

ICRA 2003 7

Calibrated vs. Uncalibrated Space

Distances and angles are modified by S

ICRA 2003 8

Motion in the distorted space

• Uncalibrated coordinates are related by

• Conjugate of the Euclidean group

Calibrated space Uncalibrated space

ICRA 2003 9

Uncalibrated Camera or Distorted Space

Uncalibrated camera with Calibration matrix K viewing points in Euclidean space and moving with (R,T) is equivalent to calibrated camera viewing points in distorted space governed by S and moving with motion conjugate to (R,T)

ICRA 2003

Uncalibrated Epipolar Geometry

• Fundamental matrix

• equivalent forms of F

• Epipolar Constraint

ICRA 2003 11

Imagecorrespondences

• Epipolar lines• Epipoles

Properties of the fundamental matrix

ICRA 2003 12

Characterization of the fundamental matrix

A nonzero matrix is a fundamental matrix if has A singular value decomposition (SVD) with

For some .

There is little structure in the matrix except that

ICRA 2003 13

Decomposing the fundamental matrix• Decomposition of the Fundamental matrix into skewsymmetric matrix and nonsingular matrix

• Decomposition of F is not unique

• Unknown parameters - ambiguity

• Corresponding projection matrix

ICRA 2003 14

What does F tell us?

F can be inferred from point matches (8-point algorithm)

Cannot extract motion, structure and calibration from one fundamental matrix (2 views)

F allows reconstruction up to a projective transformation (as we will see soon)

F encodes all the geometric information among two views when no additional information is available

ICRA 2003 15

Ambiguities in the image formation Potential Ambiguities

Ambiguity in K (K can be recovered uniquely – Cholesky or QR)

Structure of the motion parameters

Just an arbitrary choice of reference frame

ICRA 2003 16

Ambiguities in the image formationStructure of the Motion Parameters

ICRA 2003 17

Ambiguities in the image formation

For any invertible 4 x 4 matrix H In the uncalibrated case we cannot

distinguish between camera imaging word

from camera imaging distorted world In general, H is of the following form

In order to preserve the choice of the first reference frame we can restrict some DOF of H

Structure of the projection matrix

ICRA 2003 18

Structure of the projective ambiguity

• H can be further decomposed as

• For i-th frame

• 1st frame as reference

• Choose the projective reference frame

then ambiguity is

ICRA 2003 19

Geometric stratification (contd.)

ICRA 2003 20

Projective reconstruction

From points, extract F; from F extract X

Canonical decomposition

Projection matrices

ICRA 2003 21

Projective reconstruction• Given projection matrices recover projective structure

• Given 2 images and no prior information, the scene canbe recovered up a a 4-parameter family of solutions. This is the best one can do!

ICRA 2003 22

Affine upgrade

Upgrade projective structure to an affine structure

Exploit partial scene knowledge Vanishing points, no skew, known principal point

Special motions Pure rotation, pure translation, planar motion,

rectilinear motion Constant camera parameters (multi-view)

ICRA 2003 23

Affine upgrade using vanishing points

Maps the points

To points with affine coordinates

ICRA 2003 24

Vanishing point estimation

ICRA 2003 25

Euclidean upgrade

Exploit special motions (e.g. pure rotation)

If Euclidean is the goal, perform Euclidean reconstruction directly (no stratification)

Multi-view reconstruction (tomorrow)

Autocalibration (Kruppa)

ICRA 2003 26

Overview of the methods

ICRA 2003 27

Direct stratification for multiple views

Absolute Quadric Constraint

ICRA 2003 28

Direct methods - Summary

ICRA 2003 29

Summary