Uncalibrated Geometry & Stratification Yi Ma …2 Uncalibrated Camera pixel coordinates calibrated coordinates Linear transformation 2 Siggraph’04 August 08 3 Overview Calibration

1

1Siggraph’04, August 08l

UncalibratedUncalibrated Geometry & StratificationGeometry & Stratification

Yi Ma

Electrical & Computer Engineering Dept.University of Illinois at Urbana-Champaign

Siggraph’04 August 082

UncalibratedUncalibrated CameraCamera

pixelcoordinates

calibratedcoordinates

Linear transformation

2


OverviewOverview

Calibration with a rig

Uncalibrated epipolar geometry

Ambiguities in image formation

Stratified reconstruction

Autocalibration with partial scene knowledge


UncalibratedUncalibrated CameraCamera

xyc

• Pixel coordinates

• Projection matrix

Uncalibrated camera

• Image plane coordinates

• Camera extrinsic parameters

• Perspective projection

Calibrated camera

3


Taxonomy on Taxonomy on UncalibratedUncalibrated ReconstructionReconstruction

is known, back to calibrated case

is unknownCalibration with complete scene knowledge (a rig) – estimate Uncalibrated reconstruction despite the lack of knowledge of Autocalibration (recover from uncalibrated images)

Use partial knowledgeParallel lines, vanishing points, planar motion, constant intrinsic

Ambiguities, stratification (multiple views)


Calibration with a RigCalibration with a Rig

Use the fact that both 3-D and 2-D coordinates of feature points on a pre-fabricated object (e.g., a cube) are known.

4


Calibration with a RigCalibration with a Rig

• Eliminate unknown scales

• Factor the into and using QR decomposition

• Solve for translation

• Recover projection matrix

• Given 3-D coordinates on known object


UncalibratedUncalibrated EpipolarEpipolar GeometryGeometry

• Epipolar constraint

• Fundamental matrix

• Equivalent forms of

5


Properties of the Fundamental MatrixProperties of the Fundamental Matrix

Imagecorrespondences

• Epipolar lines

• Epipoles


Properties of the Fundamental MatrixProperties of the Fundamental Matrix

A nonzero matrix is a fundamental matrix if has a singular value decomposition (SVD) with

for some .

There is little structure in the matrix except that

6


• Fundamental matrix can be estimated up to scale

• Find such F that the epipolar error is minimized

Estimating Fundamental MatrixEstimating Fundamental MatrixEstimating Fundamental Matrix

• Denote

• Rewrite

• Collect constraints from all points

Pixel coordinates


• Project onto the essential manifold:

• cannot be unambiguously decomposed into poseand calibration

Two view linear algorithm – 8-point algorithmTwo view linear algorithm Two view linear algorithm –– 88--point algorithmpoint algorithm

• Solve the LLSE problem:

• Compute SVD of F recovered from data

• Solution eigenvector associated with smallest eigenvalue of

7


Calibrated vs. Calibrated vs. UncalibratedUncalibrated SpaceSpace


Calibrated vs. Calibrated vs. UncalibratedUncalibrated SpaceSpace

Distances and angles are modified by S

8


Motion in the distorted spaceMotion in the distorted space

• Uncalibrated coordinates are related by

• Conjugate of the Euclidean group

Calibrated space Uncalibrated space


What Does F Tell Us?What Does F Tell Us?

can be inferred from point matches (eight-point algorithm)

Cannot extract motion, structure and calibration from one fundamental matrix (two views)

allows reconstruction up to a projective transformation (as we will see soon)

encodes all the geometric information among two views when no additional information is available

9


Decomposing the Fundamental MatrixDecomposing the Fundamental Matrix

• Decomposition of is not unique

• Unknown parameters - ambiguity

• Corresponding projection matrix

• Decomposition of the fundamental matrix into a skew symmetric matrix and a nonsingular matrix


Projective ReconstructionProjective Reconstruction

From points, extract , followed by computation of projection matrices and structure Canonical decomposition

Given projection matrices – recover structure

Projective ambiguity – non-singular 4x4 matrix

Both and are consistent with the epipolar geometry – give the same fundamental matrix

10


Projective ReconstructionProjective Reconstruction• Given projection matrices recover projective structure

• This is a linear problem and can be solve using linear least-squares

• Projective reconstruction – projective camera matrices and projective structure

Euclidean Structure Projective Structure


Euclidean Euclidean vsvs Projective reconstructionProjective reconstruction

Euclidean reconstruction – true metric properties of objectslenghts (distances), angles, parallelism are preserved Unchanged under rigid body transformations=> Euclidean Geometry – properties of rigid bodies underrigid body transformations, similarity transformation

Projective reconstruction – lengths, angles, parallelism are NOTpreserved – we get distorted images of objects – their distorted 3D counterparts --> 3D projective reconstruction=> Projective Geometry

11


Homogeneous Coordinates (RBM)Homogeneous Coordinates (RBM)

3-D coordinates are related by:

Homogeneous coordinates:

Homogeneous coordinates are related by:


Homogenous and Projective CoordinatesHomogenous and Projective Coordinates

Homogenous coordinates in 3D before – attach 1 as the last coordinate – render the transformation as linear transformationProjective coordinates – all points are equivalent up to a scale

• Each point on the plane is represented by a ray in projective space

2D- projective plane 3D- projective space

• Ideal points – last coordinate is 0 – ray parallel to the image plane • points at infinity – never intersects the image plane

12


Vanishing points Vanishing points –– points at infinitypoints at infinity

Representation of a 3-D line – in homogeneous coordinates

When λ -> 1 - vanishing points – last coordinate -> 0

Similarly in the image plane


Ambiguities in the image formationAmbiguities in the image formationPotential Ambiguities

Ambiguity in K (K can be recovered uniquely – Cholesky or QR)

Structure of the motion parameters

Just an arbitrary choice of reference frame

13


Ambiguities in Image FormationAmbiguities in Image Formation

For any invertible 4 x 4 matrix

In the uncalibrated case we cannot distinguish between camera imaging word from camera imaging distorted world

In general, is of the following form

In order to preserve the choice of the first reference frame we can restrict some DOF of

Structure of the (uncalibrated) projection matrix


Structure of the Projective AmbiguityStructure of the Projective Ambiguity

• 1st frame as reference

• Choose the projective reference frame

then ambiguity is

• can be further decomposed as

14


Stratified (Euclidean) ReconstructionStratified (Euclidean) Reconstruction

General ambiguity – while preserving choice of first referenceframe

Decomposing the ambiguity into affine and projective one

Note the different effect of the 4-th homogeneous coordinate


Affine upgradeAffine upgrade

Upgrade projective structure to an affine structure

Exploit partial scene knowledgeVanishing points, no skew, known principal point

Special motionsPure rotation, pure translation, planar motion, rectilinear motion

Constant camera parameters (multi-view)

15


Affine upgrade using vanishing pointsAffine upgrade using vanishing points

Maps the points To points with affine coordinates

How to compute

Vanishing points – last homogeneous affine coordinate is 0


Affine Upgrade Affine Upgrade

Need at least three vanishing points

3 equations, 4 unknowns (-1 scale) Solve for

Set up and update the projectivestructure

16


Euclidean upgradeEuclidean upgradeWe need to estimate remaining affine ambiguity

In the case of special motions (e.g. pure rotation) – no projectiveambiguity – cannot do projective reconstruction

Estimate directly (special case of rotating camera – follows) Multi-view case – estimate projective and affine ambiguity togetherUse additional constraints of the scene structure (next)Autocalibration (Kruppa equations)

Alternatives:


Euclidean Upgrade using vanishing pointsEuclidean Upgrade using vanishing points

Vanishing points – intersections of the parallel lines

Vanishing points of three orthogonal directions

Orthogonal directions – inner product is zero

Provide directly constraints on matrix

S – has 5 degrees of freedom, 3 vanishing points – 3 constraints (need additional assumption about K)Assume zero skew and aspect ratio = 1

17


Geometric Stratification Geometric Stratification


Special Motions Special Motions –– Pure Rotation Pure Rotation

Calibrated Two views related by rotation only

Mapping to a reference view – rotation can be estimated

Mapping to a cylindrical surface

18


Pure Rotation Pure Rotation -- UncalibratedUncalibrated CaseCase

Calibrated Two views related by rotation only

Conjugate rotation can be estimated

Given C, how much does it tell us about K (or S) ?

Given three rotations around linearly independentaxes – S, K can be estimated using linear techniquesApplications – image mosaics


Overview of the methodsOverview of the methods

19


SummarySummary


Direct Direct AutocalibrationAutocalibration MethodsMethods

satisfies the Kruppa’s equations

The fundamental matrix

If the fundamental matrix is known up to scale

Under special motions, Kruppa’s equations become linear.

Solution to Kruppa’s equations is sensitive to noises.

20


Direct Stratification from Multiple ViewsDirect Stratification from Multiple Views

we obtain the absolute quadric contraints

From the recovered projective projection matrix

Partial knowledge in (e.g. zero skew, square pixel) rendersthe above constraints linear and easier to solve.

The projection matrices can be recovered from the multiple-view rank method to be introduced later.


Summary of (Auto)calibration MethodsSummary of (Auto)calibration Methods

Documents

Uncalibrated Geometry & Stratification Yi Ma …2 Uncalibrated Camera pixel coordinates calibrated coordinates Linear transformation 2 Siggraph’04 August 08 3 Overview Calibration