International Journal of Computer Vision 71(1), 5–47, 2007c© 2006 Springer Science + Business Media, LLC. Manufactured in the United States.

DOI: 10.1007/s11263-005-4796-1

Critical Configurations for Projective Reconstruction from Multiple Views

RICHARD HARTLEYAustralian National University, and National ICT Australia∗

FREDRIK KAHLAustralian National University, and Centre for Mathematical Sciences, Lund University, Sweden

Received May 11, 2004; Revised July 19, 2005; Accepted July 19, 2005

First online version published in June, 2006

Abstract. This paper investigates a classical problem in computer vision: Given corresponding points in multipleimages, when is there a unique projective reconstruction of the 3D geometry of the scene points and the camerapositions? A set of points and cameras is said to be critical when there is more than one way of realizing the resultingimage points. For two views, it has been known for almost a century that the critical configurations consist of pointsand camera lying on a ruled quadric surface. We give a classification of all possible critical configurations for anynumber of points in three images, and show that in most cases, the ambiguity extends to any number of cameras.

The underlying framework for deriving the critical sets is projective geometry. Using a generalization of Pascal’sTheorem, we prove that any number of cameras and scene points on an elliptic quartic form a critical set. Anotherimportant class of critical configurations consists of cameras and points on rational quartics. The theoretical resultsare accompanied by many examples and illustrations.

Keywords: projective geometry, structure from motion, degeneracy, critical sets, multiple view geometry, geom-etry 3D reconstruction

1. Introduction

The problem of recovering scene points and camerapositions from images has a number of inherentambiguities. It is well-known that from images alone,the cameras and the scene points can generally only berecovered up to an unknown projective transformation(Hartley and Zisserman, 2004). In this paper, wederive under what conditions additional ambiguitieswill occur. Configurations of points and cameras aretermed critical if the derived image projections allowprojectively distinct reconstructions.

∗National ICT Australia is funded by the Australian Govern-ment’s Department of Communications, Information Technology,and the Arts and the Australian Research Council through Back-ing Australia’s Ability and the ICT Research Centre of Excellenceprograms.

It was noted already in 1940 by Krames (1940) thatwhen a set of points and two cameras lie on a specialsurface, there may be additional solutions. In theGerman literature on photogrammetry, such surfacesare known as “gefahrliche Orter” or “critical surfaces”.In the two-view case, the critical surfaces are ruledquadrics; see Maybank (1993) for a more recent treat-ment. Partial results concerning ambiguous configura-tions with more than two views have previously beenreported in the literature. Maybank and Shashua (1998)considered the case of many views of six points, show-ing that criticality occurs if the points and cameras alllie on a ruled quadric. This was shown to be dual to theclassical two-view ambiguity in Hartley and Debunne(1998) in the Carlsson duality sense Carlsson (1995).The critical sets for lines are studied in Navab andFaugeras (1993) and Maybank (1994) for three views.The classification of critical sets for the 1D camera can

6 Hartley and Kahl

be found in Astrom and Kahl (2003) and calibratedcameras are analysed in Kahl and Hartley (2002).

This paper is an attempt to give a thorough under-standing of the necessary and sufficient conditions forthe reconstruction problem for points. A classificationof the possible critical configurations is presentedusing classical projective geometry. The material inthis paper has evolved over some time and partialresults have been reported in the following conferencepapers Hartley (2000), Kahl, et al. (2001) and Hartleyand Kahl (2003) and the thesis Kahl (2001).

Reading Guide. The paper contains many theoreti-cal results in terms of lemmas and theorems. Some ofwhich are easy to understand, others which may beharder to grasp at first sight. In order for the reader toget an overview of the most important results and notto get stuck in details, we give the following advice. InSections 2 and 3 some background and an introductionto the problem are given. Both these sections are es-sential for the remainder of the paper. Section 4 dealswith so-called trivial ambiguities and may be skippedat a first read-through. The basic theory for critical setsin two and three views are presented in Section 5 andSection 6. Before giving some examples in Section 7,the highlight of the paper is reached in Section 8 whereall classes of critical configurations are listed. In par-ticular, the results concerning Pascal’s Theorem arerecommended. The extension to arbitrary number ofviews is done in Section 9. Throughout the paper, var-ious examples of critical sets are given. These are suit-able for verification by the reader, using Mathematica,Maple or some similar software package. Normally, itis not intended that they should be derived, or verifiedeasily by hand.

2. Background

We begin with some background material. More de-tailed background is contained in the appendices.

2.1. The Camera Model

We assume a perspective/pinhole camera. It projectsa point X in space (which can be identified as theprojective space P3) to a point x in the image plane (orP2) as

x = PX.

Here the points are represented by homogeneous coor-dinates and the equality sign should be interpreted as

equality up to scale. In this expression, P is a rank-3matrix of size 3 × 4 called the camera matrix. Thegenerator of the (right) nullspace of P is a vector rep-resenting the position of the camera centre.

2.2. The Fundamental Matrix

Corresponding image points x0 and x1 in two viewssatisfy the so-called epipolar constraint:

x�1 F

10x0 = 0,

where F10 is a rank two matrix—the fundamental ma-trix. Given F10 it is possible to compute the corre-sponding camera matrices P0 and P1 and vice versa.The mapping from F10 to the pair (P0,P1) modulo thechoice of a projective coordinate system is one-to-one.Another way to characterize F10 which will turn outto be useful later is as the unique, non-zero singularmatrix F10 which satisfies the following Eq. (8):

P1�F10P0 + P0�F01P1 = 0, (1)

where the notation F01 is used to denote F10�. The(right) nullspace of F10 corresponds to the image of thecamera centre of P1 and similarly for the left nullspace.Thus, the images of the camera centres, denoted e0 ande1, respectively, satisfy F10e0 = F01e1 = 0 and theyare called epipoles.

Numbering. Camera matrices will be numbered asPi , with numbering usually starting at 0. Similarly,a point in the i-th image may be written as xi. Thefundamental matrix corresponding to two cameras Pi

and P j will be written as Fi j . The ordering of the twoindices i and j is chosen so that points in the i-th imageappear on the left, and points in the j-th image appearon the right of Fi j . This is reflected in the positionof the two indices. Thus, for corresponding points inimages i and j, we write x�

i Fi j x j = 0. Similarly, the

expression 1 is correctly written as

Pi�Fi jP j + P j�F j iPi = 0.

Epipoles are numbered in a similar way. The epipoleeij represents the point in the i-th image correspond-ing to the projection of the centre of the j-th camera.Thus, the first index indicates the image in which theepipole resides. The epipoles are therefore defined bythe equation ei j�Fi j = 0.

Critical Configurations for Projective Reconstruction from Multiple Views 7

2.3. Compatible Fundamental Matrices

Given three camera matrices, it is always possible tocompute three fundamental matrices, one for each pair.However, given three fundamental matrices it is notalways possible to find a consistent triple of cameramatrices. An essential issue here is the compatibilityof the three fundamental matrices.

Definition 2.1. Three fundamental matrices F01, F02

and F12 are compatible if they satisfy the followingcompatibility conditions:

1. The triangulation conditions:

e02�F01e12 = e01�F02e21 = e10�F12e20 = 0. (2)

2. The collinearity conditions

e01 = e02, e10 = e12 and e20 = e21 (3)

or the non-collinearity conditions

e01 �= e02, e10 �= e12 and e20 �= e21. (4)

The importance of the compatibility conditions isgiven by the following theorem. It is proved for the caseof the non-collinearity condition in Hartley and Zisser-man (2004).1 The collinear case is easier to prove, andis left to the reader.

Theorem 2.2. Three fundamental matrices F01, F02

and F12 satisfy the compatibility conditions of Defini-tion 2.1 if and only if there exist three camera matricesP0, P1, P2 such that Fi j is the fundamental matrix cor-responding to the pair (Pi , P j ).

In the non-collinear case (when (4) holds) the cam-era matrices are uniquely determined up to projec-tive equivalence. In the collinear case, there is a 4-parameter family of projectively inequivalent solu-tions.

Note in fact that the collinearity conditions ensurethat the three cameras are collinear, whereas the non-collinearity conditions ensure the opposite.

2.4. Quadrics

A quadric in Pn is defined to be the set of points Xsatisfying the equation

X�SX = 0,

where S is an (n + 1) × (n + 1) matrix. It is cus-tomary to represent a quadric by a symmetric matrix,but it is not essential that the matrix S be symmetricfor this definition to make sense. In the rest of thispaper, quadrics will commonly be represented by non-symmetric matrices. Note that X�SX = 0 if and onlyif X�(S + S�)X = 0. So, S and its symmetric partS+ S� represent the same quadric.

Notation. Sometimes we wish to represent quadricsexplicitly by symmetric matrices. To accomodate this,the notation Ssym = S+S� is used. Thus, for example

(P1�F10

Q P0)

sym = P1�F10Q P

0 + P0�F01Q P

1.

We are interested here only in points in the real(as opposed to the complex) projective space Pn , andhence only real quadrics. We distinguish two classesof quadrics, those that are ruled and those that arenot. In this paper a ruled quadric is any quadric thatcontains a straight line. A straight line on a quadric willbe referred to as a generator. A quadric is said to benon-degenerate if the matrix S+S� defining it is non-singular (therefore having rank 4). The most interestingruled quadric is the non-degenerate one, known as thehyperboloid of one sheet (see Hartley and Zisserman,2004). A basic property of this quadric is given in thefollowing theorem (Semple and Kneebone, 1979).

Theorem 2.3. The generators on a non-degenerateruled quadric may be divided into two classes in sucha way that any two generators of the same class areskew to each other, whereas two generators not in thesame class intersect. Through any point on the quadricpass two distinct generators, one from each class.

The different types of ruled quadrics are shown inFig. 1. A rank-3 quadric is a cone. The generators ona cone all pass through the vertex, and there is onlyone generator through any other point of the cone.Two planes can be considered as a degenerate rank-2quadric.

Pencils of Quadrics. Given two quadrics representedby matrices A and B, the pencil generated by themconsists of all quadrics of the form αA + βB where(α, β) ∈ P1. The intersection of two quadrics is afourth-degree curve called an elliptic quartic. If V isthe intersection of the quadrics A and B, then V lies on

8 Hartley and Kahl

Figure 1. Ruled quadrics, from left to right, top to bottom: hyperboloid of one sheet, hyperbolic paraboloid, cone, two planes and a single(double) plane. The first two quadrics are non-degenerate (and in fact, projectively equivalent) while the others are degenerate. One further ruledquadric (not shown) consists of a single line, with equation X2 + Y2 = 0.

each of the quadrics αA + βB in the pencil. If

det

[α β

γ δ

]�= 0

then the intersection of quadrics αA+βB and γ A+ δBis the same as the intersection of the quadrics A and B.

A pencil of quadrics may be defined by either ruledor unruled quadrics. In non-degenerate cases these cor-respond to hyperboloids of one sheet or ellipsoids. Itis important to note that a pencil of quadrics definedby two ellipsoids may contain hyperboloids and evenimaginary quadrics (those with no real points). Sim-ilarly, a pencil defined by hyperboloids may containellipsoids and imaginary conics. This is easily seen byexample.

2.5. Rational Curves

We give some basic definitions of rational curves here.More details concerning rational curves are to be foundin Semple and Kneebone (1979).

Definition 2.4. A rational curve of degree d inPn is the locus of points of the form X(θ ) =(a0(θ ), a1(θ ), . . . , an(θ ))� where each ai (θ ) is a realpolynomial of degree at most d, at least one ai (θ ) hasdegree d and the ai (θ ) do not have a common factor ofdegree greater than zero.

For brevity we will often use the word curve to meana rational curve. A rational curve in Pn may be writtenin the form A(1, θ, θ2, . . . , θd )� where A is the (n+1)×(d+1) matrix of coefficients of the several polynomialsai (θ ). We denote the vector (1, θ, θ2, . . . , θd )� by �(d).With this notation, the rational curve is the locus ofpoints

X(θ ) = A�(d).

Normally, we will require that the matrix A rep-resenting the curve has full row-rank, that is, rankn + 1. This is the requirement that the curve not liein a subspace (hyperplane) in Pn . For if π representsa hyperplane containing points A�(d) for all θ , thenπ�A�(d) = 0 for all θ , so π lies in the left null spaceof A and conversely. A curve that does not satisfy thisrank condition is said to be degenerate.

3. Critical Sets for Reconstruction

Problem Formulation. We consider the case of n ≥2 cameras and denote the camera matrices by Pi fori = 0, . . . , n −1. Consider also a set of points Pj. Notethat cameras and points are represented by symbolsusing different fonts. The problem considered in thispaper is under what circumstances there exists an al-ternative set of camera matrices Qi and points Qj suchthat the image projections are the same, that is,

Pi P j = Qi Q j for all i, j,

Critical Configurations for Projective Reconstruction from Multiple Views 9

but {P j } and {Q j } are not projectively equivalent pointsets. Here, two such point sets are called projectivelyequivalent, if there exists a 3D projective transforma-tion, H such that Q j = HP j for all j.

Based on this intuitive concept we make the follow-ing definition.

Definition 3.5. A configuration {Pi , P j } of cameramatrices Pi and points Pj is called a critical configura-tion if there exists an alternative configuration {Qi , Q j }such that

1. Pi P j = Qi Q j for all i, j, but2. there does not exist a 3D homography H, such that

HP j = Q j for all j.

In such a case when {Pi , P j } is critical, the two con-figurations {Pi , P j } and {Qi , Q j } are called conjugateconfigurations.

We will also (somewhat less formally) use the term“ambiguity” to describe the situation described in thisdefinition. One form of ambiguity excluded by theabove definition is the ambiguity of camera resection-ing for a single camera. In such an ambiguity, the pointsets are the same, but there are several possible place-ments of the camera. Thus two configurations that dif-fer only by a camera-resectioning ambiguity of one ormore of the cameras are not considered to be criticalaccording to this definition. Camera-resectioning am-biguity occurs for instance, when all points and thecamera centre lie on a twisted cubic curve (Buchanan,1988). Another example of ambiguity for camera re-sectioning is when all points lie in a plane. A completeanalysis of camera resectioning ambiguities is given inHartley and Zisserman (2003).

Standard Camera Configurations. We now show theimportant result that the property of being a criticalconfiguration does not depend on any property of thecamera matrices involved, other than their camera cen-tres. The following remark is well-known.

Proposition 3.6. Let P0 and P1 be two cameramatrices with the same centre. Then there exists a2D projective image transformation represented by anon-singular matrixH such thatP1 = HP0. Conversely,for any such matrix H, two cameras P0 and P1 = HP0

have the same centre.

This proposition may be geometrically interpreted assaying that an image is determined up to projectivity bythe camera centre alone. Here a more algebraic proofis given.

Proof: The camera centre is determined by thenullspace of the camera matrix, and hence we maywrite Pi = Hi [I | −Ci

P] where CiP denotes the

non-homogeneous coordinates of the camera centre.Suppose C0

P = C1P = CP and let H = H1H

−10 then

P1 = H1[I | −CP] = H1H−10︸ ︷︷ ︸

H

H0[I | −CP] = HP0.

Conversely, suppose P1 = HP0 for some non-singularH, then it trivially follows that the (right) nullspaces ofP0 and P1 are equal. �

It has the following consequence.

Proposition 3.7. If {Pi , P j } is a critical configurationand {Pi } is set of cameras with the same centres as {Pi },then {Pi , P j } is a critical configuration as well.

Proof: Since {Pi , P j } is a critical configuration thereexists a conjugate configuration {Qi , Q j } such thatPi P j = Qi Q j for all i and j. However, since Pi andPi have the same camera centre, Pi = HiPi accordingto Proposition 3.6. Therefore

Pi P j = HiPi P j = HiQ

i Q j .

It follows that {HiQi , Q j } is a conjugate configurationto {Pi , P j }, which is therefore critical. �

For convenience, it may therefore be assumed thatthe camera matrices are of the (standard) form Pi =[I | −Ci

P], where CiP represents the camera centre in

non-homogenous coordinates.

4. Trivial Ambiguities

We consider a pair of conjugate configurations{Pi , P j } and {Qi , Q j }. According to Definition 3.5Pi P j = Qi Q j for all i and j, but the point sets P j

and Q j are not projectively equivalent. There are twopossibilities:

1. There exists at least one pair of indices (i1, i2) suchthat Fi1i2

P = Fi1i2Q , or

10 Hartley and Kahl

2. For all pairs (i1, i2) the two fundamental matricesFi1i2P and Fi1i2

Q are different.

The study of the second of the two cases will occupymost of this paper. We therefore modify Definition 3.5to make a new definition, as follows:

Definition 4.8. A configuration {Pi , P j } of cameramatrices Pi and points Pj is called a non-trivial criticalconfiguration if there exists an alternative configura-tion {Qi , Q j } such that

1. Pi P j = Qi Q j for all i, j, but2. there does not exist a 3D homography H, such that

HP j = Q j for all j, and3. for all pairs of indices (i1, i2) the two fundamental

matrices Fi1i2P and Fi1i2

Q are different.

As we said, in this paper we will be mainly con-cerned with non-trivial critical configurations. In thissection, however, we consider the possibility of “trivialambiguities.” As mentioned previously, a configura-tion that satisfies conditions 1 and 3 in the above defi-nition is called a resection ambiguity, and according toDefinition 3.5 it is not a critical configuration. Insteadwe consider critical configurations that satisfy the firsttwo conditions in Definition 4.8 but not the third one.Thus, in such a configuration there exists a pair of cam-eras indexed by i1 and i2 such that Fi1i2

P = Fi1i2Q . By a

suitable renumbering of the cameras, we may assumethat F10

P = F10Q .

If F10P = F10

Q , then the two corresponding camerapairs are projectively equivalent and we may assumefurther that P0 = Q0 and P1 = Q1. Furthermore, thepoints P j = Q j for all points except for those that lieon the line containing the two camera centres. Notethat the position of any point P lying on the line join-ing the two camera centres cannot be determined fromits images in any set of cameras lying on this line.The position of the point may be anywhere along theline. Clearly, this ambiguity is not destroyed by addingany number of extra cameras lying on the same line.Therefore, we have identified a first kind of trivial am-biguity.

• There exist cameras Pi = Qi all lying on a singleline, some points P j = Q j for j in some index setJ0, and other points P j �= Q j for j in a non-emptyindex set J1, points in this second set all lying on theline containing the camera centres.

Note that the index set J1 must be non-empty, forotherwise the point sets are identical and the P and Qconfigurations are not a conjugate pair.

Example: Trivial ambiguities are not so trivial. Thequestion arises whether more interesting cases exist, inwhich all the cameras are not in the same line. We settlethis question at once by giving an example. Define thecamera matrices

Pθ =

1 −θ 0 0

0 1 −θ 0

0 0 1 −θ

Qθ =

1 −sθ 0 0

0 1 −sθ 0

0 0 1 −sθ

,

where s is some non-zero real number. Note that thecamera centre of Pθ is the point (θ3, θ2, θ, 1)�, sincePθ (θ3, θ2, θ, 1)� = 0. Thus, the cameras representedby Pθ all lie on a twisted cubic curve. The cameracentre for Qθ is the point (s3θ3, s2θ2, sθ, 1)�, whichlies on the same curve.

Also, consider a different set of camera matricesindexed by α,

Pα = Qα =

1 0 0 −α

0 1 0 0

0 0 1 0

.

The camera centre for this camera is at the point(α, 0, 0, 1)�, and so all the cameras Pα and Qα lie on astraight line. This straight line meets the twisted cubic(θ3, θ2, θ, 1)� at two points, namely when (θ = 0, α =0), and (θ = ∞, α = ∞). This latter intersection is atthe point (1, 0, 0, 0)�.

We now define points as follows.

Pλ = Qλ = (λ3, λ2, λ, 1)�, Pβ = (β, 0, 0, 1)�and

Qβ = (sβ, 0, 0, 1)�.

Then, it is easily verified that

PθPλ = QθPλ = QθQλ ≈ (λ2, λ, 1)�

Pθ Pβ = Qθ Qβ ≈ (β, 0,−θ )�

PαPλ = QαQλ

PαPβ = QαQβ ≈ (1, 0, 0)�.

Critical Configurations for Projective Reconstruction from Multiple Views 11

Thus, the set of all cameras {Pθ , Pα} and the points{Pλ, Pβ} form a critical set. An alternative reconstruc-tion is given by the cameras {Qθ , Qα} and the points{Qλ, Qβ}. This is an example of an ambiguity consist-ing of an infinite number of cameras and point.

There are several things to notice about this example:

1. There are two classes of cameras,Pθ and Pα indexedby two index sets I0 = {θ} and I1 = {α}. All thecameras indexed by I0 lie on a twisted cubic, andthose indexed by I1 lie on a straight line meetingthe twisted cubic in two points.

2. There are two classes of points, Pλ and Pβ indexedby two index sets J0 = {λ} and J1 = {β}. Thepoints indexed by J0 lie on the same twisted cubicas the cameras indexed by I0. The points indexedby J1 lie on the same straight line as the camerasindexed by I1.

3. In the alternative reconstruction, involving the cam-eras Q and points Q, we have Pα = Qα for cameraindexed by I1, but Pθ �= Qθ for those indexed by I0.Similarly Pβ �= Qβ for points indexed by J1, butPλ = Qλ for those indexed by J0.

4. The points Pλ form a critical set for camera resec-tioning. Furthermore Pθ and Qθ are alternative re-sectioning solutions. In other words, PθPλ = QθPλ

for all λ.

It will be seen next that these conditions are gener-ally true for trivial ambiguities.

General Trivial Ambiguities. We derive properties oftrivial ambiguities in general, and will show that the ex-ample just described is representative. It was remarkedpreviously that if all the cameras Pi lie on a singlestraight line, then a trivial ambiguity exists. Consideran ambiguity of this kind involving two cameras P0

and P1, and suppose that there exists at least one fur-ther camera, which we will denote P2, not lying on thebase line of two cameras P0 and P1. Suppose by wayof hypothesis that Q2 = P2. It will be shown that thisis not possible.

Let Pj be a point such that P j �= Q j ; that is j ∈ J1.Since the configuration is ambiguous, Q2Q j = P2P j

and Q0Q j = P0P j . However, since Q2 = P2, and Q0 =P0, this means that P2P j = P2Q j and P0P j = P0Q j .Since P j �= Q j , it follows that Pj and Qj both lie onthe line joining P0 and P2. Similarly, however, Pj mustlie on the line joining P1 and P2. This means that P2

itself must lie on the line joining P0 and P1, contrary

to hypothesis. This argument shows by contradictionthat for any further camera P2 not lying on the baseline defined by P0 and Q0, the conjugate camera Q2 isdifferent from P2.

Now, consider a point Pj with j ∈ J0 (so P j = Q j ).Then, we have P2P j = Q2Q j = Q2P j , but P2 �= Q2.Therefore, the points Pj with j ∈ J0 must form acritical set for resection, andP2 andQ2 are two differentresection results. This proves the following result.

Theorem 4.9. Let {Pi , P j } and {Qi , Q j } be a criticalconfiguration and its conjugate, such that F01

P = F01Q .

Without loss of ambiguity up to projectivity, we mayassume P0 = Q0 and P1 = Q1. Then, the set of pointsmay be divided into two sets indexed by J0 and J1, andthe cameras into two sets indexed by I0 and I1 with thefollowing properties.

• Pi �= Qi for all i in I0.• Pi = Qi for all i in I1.• P j = Q j for all j in J0.• P j �= Q j for all j in J1.• The cameras Pi ; i ∈ I1 and the points P j ; j ∈ J1 all

lie on a single straight line.• The points P j ; j ∈ J0 constitute a critical set for

camera resection and for all i ∈ I0 the camerasPi and Qi are a conjugate pair for resection withrespect to the points P j ; j ∈ J0.

An illustration of the theorem is given in Fig. 2.

Twisted Cubic Ambiguity. The most interesting am-biguous set for resectioning is where points lie on atwisted cubic and cameras P2 and Q2 lie on the sametwisted cubic. We consider this configuration in moredetail. Under these circumstances, it is well-known that

Figure 2. A trivial ambiguity consisting of points and cameras ona line and a twisted cubic. The points on the cubic can be recovereduniquely, but not the points on the line. In contrast, the cameras onthe cubic cannot be recovered unambiguously while the cameras onthe line may be determined uniquely.

12 Hartley and Kahl

for such a P2 and points P j ; j ∈ J0 lying on a twistedcubic, there exists a cameraQ2 placed at any pre-chosenposition on the twisted cubic such that P2P j = Q2P j

for all j ∈ J0. However, in order for the two configura-tions to be conjugate, it is necessary that P2P j = Q2Q j

for all j ∈ J1 as well. Recall that Pj and Qj are pointslying on the base line of the cameras P0 and P1.

From the point of view of the camera P2, the twistedcubic appears in the image as a conic and the base lineappears as a straight line. From the point of view ofQ2 the twisted cubic and straight line must be imagedas the same conic and line. Now, a conic and a line inan image must meet in two points (possibly imaginaryor perhaps a double intersection point). Let these twopoints in the image be x0 and x1. There are correspond-ing points X0 and X1 lying on the twisted cubic suchthat P2X0 = Q2X0 = x0 and P2X1 = Q2X1 = x1.Furthermore, the image of the line joining X1 and X0

is identical with the image of the base line of P0, P1,since both these straight lines pass through x0 and x1.This applies to the images taken by both cameras P2

and Q2. However, this implies that the two 3D linesX0X1 and the base line defined by P0,P1 are identical,since otherwise the points X0, X1 and the centres ofP0 and P1 must lie on a plane passing through eachof the camera centres of P2 and Q2. This plane wouldthen contain four points X0, X1 and the centres of P2

and Q2 all lying on the twisted cubic. Since a twistedcubic cannot meet a plane in more than three distinctpoints, this is impossible. Briefly, we have shown thatthe twisted cubic must meet the base line of P0 and P1

in the two points X0 and X1.Conversely, we show that any configuration consist-

ing of a twisted cubic and a line (referred to as the baseline) meeting it in two points forms a critical configura-tion in this way. The example given earlier in this sec-tion is a particular case of this. Let X0 and X1 be the twopoints of intersection of the cubic and the line. and letP2 andQ2 be two cameras on the twisted cubic such thatP2X = Q2X for any point X on the twisted cubic. Then,the line through X0 and X1 is the same in both images,since P2X0 = Q2X0 = x0 and P2X1 = Q2X1 = x1.Denote this line by 1. Let P j ; j ∈ J1 be a point onthe base line and let P2P j = x j . Since x j lies on theline 1, there exists a point Qj on the base line suchthat Q2Q j = x j = P2P j . Furthermore, if P0 = Q0

and P1 = Q1 are cameras lying on the base line, thenclearly Pi P j = Qi Q j for j = 0, 1. Thus the pair ofpoints (P j , Q j ) are conjugate, and Pi P j = Qi Q j fori = 0, 1, 2.

• A configuration of points and cameras lying on theunion of a twisted cubic and a straight line intersect-ing in two points is ambiguous provided there existat least one point on the straight line.

The example given earlier shows that this ambiguityremains even with more than one camera (in fact in-finitely many) on the twisted cubic. Other ambiguitiesof resectioning other than a twisted cubic do exist—acomplete list is given in Hartley and Zisserman (2003).The most notable is where all points and cameras lieon the union of a line and a plane. Corresponding tothese there will potentially be other trivial reconstruc-tion ambiguities, but these are left for the reader toinvestigate.

5. Two-View Ambiguity

Henceforth, we will consider only non-trivial criticalconfigurations, and when we speak of critical configu-rations, it is to be understood that we mean non-trivialones. It is well-known that critical configurations in-volving two cameras occur when both cameras and allthe points lie on a ruled quadric (in the non-degeneratecase, a hyperboloid of one sheet). Not quite sowell-known is that not all such configurations are criti-cal, or the exact details of how these critical configura-tions occur. In order to study the critical configurationsinvolving several cameras, we will need to understandtwo-view configurations thoroughly. That will be thegoal of this section.

The first lemma shows that any two non-projectively-equivalent camera pairs (P0,P1) and(Q0,Q1) belong to a pair of conjugate configurations.We denote the centre of a camera with matrix such asP0 by C0

P.

Lemma 5.10. Consider two pairs of cameras(P0,P1) and (Q0,Q1), with corresponding (distinct)fundamental matricesF10

P andF10Q . Define two quadrics

by

SP = P1�F10Q P

0 and SQ = Q1�F10P Q

0. (5)

1. The quadric SP contains the camera centres of P0

and P1. Similarly, SQ contains the camera centresof Q0 and Q1.

Critical Configurations for Projective Reconstruction from Multiple Views 13

2. If P and Q are 3D points such that P0P = Q0Q andP1P = Q1Q, then P lies on the quadric SP and Qlies on SQ.

3. Conversely, if P is a point lying on the quadric SP,then there exists a point Q lying on SQ such thatP0P = Q0Q and P1P = Q1Q.

4. If e01Q is the epipole defined by F10

Q e01Q = 0, then

the ray passing through C0P consisting of points X

such that P0X = e01Q = Q0C1

Q lies on the quadricSP. Thus the quadric is ruled, and this ray is agenerator. Denote this generator by g01

P .5. Let e10

Q be the other epipole defined by e10�Q F10

Q = 0and g10

P be the generator consisting of points Xsuch that P1X = e10

Q . Then g01P and g10

P eitherdo not meet, or meet at a singular point of thequadric SP.

6. Let g01P be another generator passing through C0

P,other than g01

P (that is, from the opposite generatorclass if SP is a hyperboloid), and let P be a point onthis generator. Then the points Q such that Q0Q =P0P form a generator on S01

Q , denoted g01Q , passing

through C0Q.

A singular point X on a quadric Ssym is one forwhich SsymX = 0. For the different possible ruledquadrics (see Fig. 1) the singular points are as follows.A non-degenerate quadric has no singular points; thevertex of a cone is its only singular point; for a pair ofplanes, the singular points are the line of intersectionof the two planes; and for a single plane or line, everypoint is singular.

Now, we prove the lemma.

Proof: The matrix F10P corresponding to a pair of

cameras (P0,P1) is characterized by the fact thatP1�F10

P P0 is skew-symmetric (see (1)). Since F10

P �=F10Q , however, the matrices SP and SQ defined here are

not skew-symmetric, and hence represent well-definedquadrics.

1. The camera centre of P0 satisfies P0C0P =

0. Then C0�P SPC0

P = C0�P (P1�F10

Q P0)C0

P =C0�P (P1�F10

Q )P0C0P = 0. So, C0

P lies on the quadricSP. In a similar manner, C1

P lies on SP.2. Under the given conditions one sees that

P�SPP = P�P1�F10Q P

0P = Q�(Q1�F10

Q Q0)Q = 0

since Q1�F10Q Q

0 is skew-symmetric. Thus, P lies onthe quadric SP. By a similar argument, Q lies onSQ.

3. Let P lie on SP and define x0 = P0P andx1 = P1P. Then, from P�SPP = 0 we deduce0 = P�P1�F10

Q P0P = x1�F10

Q x0 and so x0 ↔ x1

is a corresponding pair of points with respect toF10Q . Therefore, there exists a point Q such thatQ0Q = x0 = P0P, and Q1Q = x1 = P1P. Frompart 2 of this lemma, Q must lie on SQ.

4. For a point X such that e01Q = P0X, one verifies that

SPX = P1�F10Q P

0X = P1�F10Q e01

Q = 0, so X lies onSP.

5. Suppose the two generators meet and that a point Xbelongs to both g01

P and g10P . Then, P0X = e01

Q andP1X = e10

Q . Consequently,

(SP + S�

P

)X = P1�F10

Q P0X + P0�F01

Q P1X

= P1�F10Q e01

Q + P0�F01Q e10

Q = 0

and the quadric defined by SP + S�P is degenerate.

6. All the points P lying on g01P project to the same im-

age point via camera P0. The corresponding pointsQ defined by Q0Q = P0P must also project to thesame point via Q0. Therefore, they lie on a generatorof S10

Q . In fact, this will be the generator g01Q defined

similarly to g01P with the roles of P and Q reversed.

�

This lemma completely describes the sets of 3Dpoints giving rise to ambiguous image correspon-dences. Note that any two arbitrarily chosen camerapairs can give rise to ambiguous image correspon-dences, in which the critical 3D points lie on the givenquadrics.

In most cases there are two conjugates for a givencritical two-view configuration. In the next lemma, weshow how these correspond to the generators passingthrough the camera centres on the critical quadric. Firstwe make a definition.

Definition 5.11. A pair of generators (g0, g1) lying ona ruled quadricS are called permissible in the followingcases:

1. If the quadric is a hyperboloid, then two (distinct)generators from the same class are permissible.

2. If the quadric is a cone, then any two (distinct)generators are permissible.

14 Hartley and Kahl

Figure 3. For a hyperboloid of one sheet, there are two possible pairs of permissible generators for two cameras, provided the camera centresare not on a single generator, but only one pair of the cameras lie on a common generator (rightmost diagram).

Figure 4. Different configurations of two cameras on a cone. Inthe first two cases, there are permissible pairs of generators, but notin the last case.

Figure 5. Different configurations of two cameras on two planeswith possible pairs of permissible generators.

3. If the quadric consists of two planes, then any twogenerators on the same plane and intersecting on theline of intersection of the two planes are permissi-ble. This includes the case where the two generatorsare both identical with the line of intersection of thetwo planes.

4. All other pairs of generators are not permissible.Thus generators on other quadrics (line or singleplane) are not permissible.

Thus, two generators are permissible if they do notmeet, or meet only in singular points of the quadric. Thepossible permissible generators are shown in Figs. 3, 4and 5 for various types of ruled quadrics.

Lemma 5.12. Consider a ruled quadric surface S ofrank 3 or 4 (a hyperboloid or a cone) and two camerasP0 and P1 lying on S. Then, for each pair (g0, g1)of permissible generators on S, passing through C0

P

and C1P, respectively, there exists a unique fundamental

matrix F10Q such that S = (P1�F10

Q P0)sym and such that

g0 is the ray defined by P0X = e01Q , and g1 is the ray

defined by P1X = e10Q .

Proof: We seek a matrix F10Q such that S =

(P1�F10Q P

0)sym. Consider the mapping f from the vec-tor space of all 3 × 3 matrices F to the vector spaceof symmetric 4 × 4 matrices S; the mapping givenby S = (P1�FP0)sym. This is a linear mapping from a9-dimensional space to a 10-dimensional space. How-ever, if Ci

P are the centres of the two camera matrices,then we see that

Ci�P SCi

P = Ci�P (P1�FP0)symCi

P = 0,

so the range of this mapping consists only of those Sthat satisfy Ci�

P SCiP = 0. These are two linear restric-

tions on the space of all possible S, so the range of fis at most an 8-dimensional subspace. Geometrically,we may say that the range of f is contained in the vec-tor space of all S that represent quadrics containingthe two camera centres Ci

P. Thus, f represents a map-ping from a 9-dimensional space into an 8-dimensionalsubspace. The kernel of this mapping is easily found:according to (1), the kernel of f is generated by thefundamental matrix F10

P corresponding to the two cam-era matrices. Since the kernel is one-dimensional, itfollows by counting dimensions that f maps onto the8-dimensional space just described. From this we de-duce the following:

• If P0 and P1 are camera matrices with centres lyingon a quadric S, then there exists a matrix G suchthatS = (P1�GP0)sym. Furthermore, any matrix thatsatisfies this property can be written in the formF = G+ λF10

P .

Consequently, we define F(λ) = G+ λF10P . �

Now, suppose that X is a point on the generator g0 onS passing through C0

P. We would like to choose λ suchthat F(λ)P0X = 0, for then F(λ) is singular and we candefine F10

Q = F(λ) as the required fundamental matrixwith epipole e01

Q = P0X, and the proof is complete.Unfortunately, since F(λ)P0X is a vector (not a scalar),

Critical Configurations for Projective Reconstruction from Multiple Views 15

it is not clear that such a λ exists. To show this, we mustuse the fact that X lies on a generator of the quadric.

If X lies on a generator passing through C0P, then for

any value µ we have (C0P + µX)�S(C0

P + µX) = 0,which leads to two conditions:

X�SX = 0 and C0�P SX = 0.

Writing S = P1�GP0 +P0�GP1, expanding and settingP0C0

P = 0 leads to

X�P1�GP0X = 0 and C0�P P1�GP0X = 0. (6)

Now the first condition of (6) says that consideredas a 3-vector, GP0X is orthogonal to P1X and thesecond condition states that GP0X is orthogonal toP1C0

P = e10P . These two conditions can be written as

GP0X = −λe10P × P1X = −λ[e10

P ]×P1X (providede10P �= P1X) for some λ and define F10

Q = F(λ). How-ever, [e10

P ]×P1X is simply the epipolar line in the sec-ond image, passing through the point P1X. This mayalso be expressed as the epipolar line corresponding tothe point P0X in the other image. Thus, [e10

P ]×P1X =F10P P

0X. Consequently, GP0X = −λF10P P

0X, and soF10Q (λ)P0X = (G+ λF10

P )P0X = 0 as required.This conclusion will hold unless e10

P = P1X, asflagged above. In this case, P1X = e10

P = P1C0P, and so

X lies on the line or generator joining the centres of thetwo cameras, and in particular the camera centre C1

P

lies on the chosen generator g0. It follows that the pairof generators g0 and g1 (which also passes through C1

P)can not be a permissible pair, unless C1

P is a singularpoint of the quadric. This is the situation depicted inthe second example of Fig. 4. In this case, we reversethe roles of the two cameras P0 and P1 in the aboveargument, and deduce that for every generator g1 pass-ing through C1

P, except the one passing thorugh C0P, the

required fundamental matrix F10Q exists.

One point that we have ignored so far is the possibil-ity that the matrix F10

Q so constructed, though singular,may have rank less than 2. From the definition of SP,

SP = P1�F10Q P

0 + P0�F01Q P

1,

it follows that if F10Q has rank one, then SP has nec-

essarily (at most) rank two, which is excluded by as-sumption. �

For completeness, we explain what happens in thecase of quadrics of rank 2 or 1. The various possi-bilities may be computed exactly. Starting with the

quadric S and two camera matrices P0 and P1 ingeneric positions, it is easy to find a matrix G such thatS = (P0�GP1)sym. The possible fundamental matricesF10Q are then found by determining for what values of

λ the matrix F(λ) = G+λF10P has rank 2. The epipoles

of F10Q may then be computed. Results are summarized

in Fig. 5 which shows the possible configurations ofcamera centres and epipoles that occur in critical con-figurations. Note that a single plane is not critical anda single line constitutes a trivial ambiguity.

Let us interpret lemma 5.12. It gives us a way of con-structing critical configurations as follows. Considerpoints lying on a ruled quadric SP, and two cameras P0

and P1 located on SP. We wish to construct a conju-gate configuration. If we can find a pair of permissiblegenerators g0 and g1 passing through the two cameracentres, then according to Lemma 5.12 we can find afundamental matrixF10

Q that satisfies the conclusions ofthe Lemma. The corresponding pair of camera matricesQ0 and Q1 will satisfy the conditions of Lemma 5.10,and ensure that SP is a critical surface, and (Q0,Q1)are the conjugate cameras. Thus, as long as permis-sible generators g0 and g1 exist passing through thecentres of P0 and P1, then the configuration is critical.In fact, (for rank 3 or 4 quadrics) the conjugate con-figurations are in one-to-one correspondence with thedistinct pairs of permissible generators. In the case ofa hyperboloid of one sheet, one can always find twosolutions provided the cameras are not on the samegenerator, see Fig. 3.

Theorem 5.13. If SP is a ruled quadric of rank 3 or4, and (P0,P1) are a pair of cameras with centres onthe quadric, then distinct conjugate configurations for(P0,P1,SP) are in one-to-one correspondence to thepairs of permissible generators (g0, g1) with gi passingthrough the centre of camera Pi .

For quadrics of rank 2, the conjugate configurationsare in correspondence with pairs of generators, as de-tailed in Fig. 5.

5.1. Two-View Summary

The existence of critical surfaces for two views hasbeen known for quite some time now, see for exampleKrames (1940) which dates back to 1940. In May-bank (1993) it was shown that a two-view configura-tion is critical only if all points and camera centres lieon a ruled quadric. Until now, it has mistakenly beenbelieved that all ruled quadrics are critical, that is a

16 Hartley and Kahl

Table 1. Summary of the different critical configurations and their conjugates. As the first row of the table shows, the two indicatedconfigurations of different types are conjugate to each other. In all other cases, the conjugates of a configuration of a given type are all of thesame type.

Configuration No. of conjugates Conjugate configuration

Hyperboloid XY = ZT – centres on a common generator 1 Cone X2 + Y2 = Z2 – cameracentres on different generators

Hyperboloid XY = ZT – centres not on a common generator 2 Same

Cone X2 + Y2 = Z2 – one camera centre at the vertex ∞ Same

Two planes XY = 0 – cameras on one plane, but not on the intersection ∞ Same

Two planes XY = 0 – one camera on the intersection, the other not ∞ Same

Single plane X2 = 0 – both cameras on the plane ∞ Same

Single line X2 + Y2 = 0 – both cameras on the line ∞ Same

Cone X2 + Y2 = Z2 – both cameras on the same generator, but not the vertex 0 –

Two planes XY = 0 – camera centres not on the same plane 0 –

two-view configuration is critical if and only if allpoints and camera centres lie on a ruled quadric.

Theorem 5.13 allows us to enumerate all the possiblecritical configurations for two views, and count thenumber of conjugate configurations except for the casewhere S is a single plane (or a single line). In these lastcases, the conjugate points are equal P j = Q j for all Pj

not on the line between the two cameras, but FP �= FQ.So, according to Definition 4.8, it is actually critical,but perhaps not very interesting.

Table of Two-View Configurations. A summary of allthe different two-view critical configurations is given inTable 1. Any configuration consisting of a ruled quadricand two points (camera centres) on the quadric is pro-jectively equivalent to one of the listed configurations.Furthermore, any two configuration of a quadric andtwo points belonging to the same class are projectivelyequivalent. Hence, we may select a representative pairof cameras on the quadric. The form of the conjugateconfiguration may then be found by direct computa-tion. In most cases, the conjugate configuration belongsto the same class.2 However, there is one surprise, givenin the first line of the table, which shows the configu-ration and its conjugate belonging to different classes.The number of conjugate configurations shown in thetable verifies the theoretical arguments given earlier.

Appearance of Conjugate Configurations. Anotherinteresting question is what the image of a conjugatesurface might look like. Consider the two camera pairs

(P0,P1) and (Q0,Q1) given by

P0 =

1 0 0 0

0 1 0 0

0 0 1 0

, P1 =

1 0 0 −1

0 1 0 −1

0 0 1 −1

and

Q0 =

1 −2 1 0

1 0 3 0

1 0 −1 0

, Q1 =

2 −4 2 0

1 −3 3 −1

3 −3 1 −1

.

The conjugate point to the point P = (x, y, xy, 1)� isgiven by

Q = ((1 + 3x)(1 − x)y, (1 − x)(−2x + xy + y),

(1 − x)2 y, (1 + 3x)(−2x + xy + y))�.

Suppose the quadric surface in the first configurationis textured with a checkerboard pattern. What does thecorresponding surface in the conjugate configurationlook like? The answer is illustrated in Fig. 6. Thisexample shows how completely different the possiblereconstructions of the scene may appear.

6. Three-View Ambiguity—Basic Theory

Now, we turn to the subject of critical configurationsinvolving three cameras. A simple observation is thata critical configuration for three or more views is alsoa critical set for each of the pairs of views as well.

Critical Configurations for Projective Reconstruction from Multiple Views 17

Figure 6. Two conjugate configurations of points on the critical quadric Z = XY viewed in the direction of the z-axis. Camera centres aredrawn with black circles.

This leads to the following characterization of criticalconfigurations involving several views.

Theorem 6.14. If {Pi , Pk}, i = 0, . . . , n−1 is a criti-cal set for reconstruction involving n views then for anypair of distinct indices i and j where i, j = 0, . . . , n−1,and corresponding pairs of camera (Pi ,P j ) there existruled quadrics Si j

P such that all points Pk lie on theintersection of all quadrics:

Pk ∈n−1⋂

i, j=0

Si jP .

Furthermore, denoting the centre of camera Pi by CiP,

CiP ∈

n−1⋂

j=0

Si jP .

This theorem gives a necessary condition for a con-figuration of cameras and points to be critical. The mainstudy of the rest of the paper is aimed at determiningto what extent the conditions is sufficient. We now turnour attention to the three camera case; the case for ncameras will be largely resolved as we obtain moreunderstanding of the conditions for three-view ambi-guity.

Suppose that the camera matrices (P0,P1,P2) and(Q0,Q1,Q2) are fixed and we wish to find the set of allpoints such that Pi P = Qi Q for i = 0, 1, 2. Note thatwe are trying here to copy the two-view case in whichboth sets of camera matrices are chosen up front. Later,we will turn to the less restricted case in which just oneset of cameras is chosen in advance.

Corresponding to the critical configuration{P0,P1, P j }, there exists a conjugate configuration

{Q0,Q1, Q01j } for which Pi P j = Qi Q01

j for i = 0, 1.Similarly, for the critical configuration {P0,P2, P j },there exists a conjugate configuration {Q0,Q2, Q02

j }for which Pi P j = Qi Q02

j for i = 0, 2. However, thepoints Q02

j are not necessarily the same as Q01j , so we

cannot conclude that there exist points Q j such thatPi P j = Qi Q j for i = 0, 1, 2 and all j—at least notimmediately. However, under fairly general conditionsthis will be so, as the following theorem states.

Theorem 6.15. Let (P0,P1,P2) and (Q0,Q1,Q2) betwo triples of camera matrices. For each of the pairs(i, j) = (1, 0), (2, 0) and (2, 1), let Si j

P and Si jQ be

the ruled quadric critical surfaces defined in (5) forcamera pairs (Pi ,P j ) and (Qi ,Q j ), respectively.

(i) If there exist points P and Q such that Pi P =Qi Q for all i = 0, 1, 2 then P must lie on theintersection S10

P ∩ S20P ∩ S21

P and Q must lie onS10Q ∩ S20

Q ∩ S21Q .

(ii) Conversely, if P is a point lying on the intersectionof quadrics S10

P ∩ S20P ∩ S21

P , but not satisfying thecondition

(e01Q × e02

Q

)�P0

(e12Q × e10

Q

)�P1

(e20Q × e21

Q

)�P2

P = 0, (7)

where each ei jQ is an epipole, then there exists a

point Q lying on S10Q ∩ S20

Q ∩ S21Q such that Pi P =

Qi Q for all i = 0, 1, 2.

Proof: The first part of the theorem is simply thestatement of Theorem 6.14. To prove the converse,suppose that P lies on the intersection of the threequadrics. Then from Lemma 5.10, applied to each of

18 Hartley and Kahl

the three quadrics Si jP , there exist points Qi j such that

the following conditions hold:

P0P = Q0Q01; P1P = Q1Q01

P0P = Q0Q02; P2P = Q2Q02

P1P = Q1Q12; P2P = Q2Q12.

It is easy to be confused by the superscripts here, butthe main point is that each line is precisely the resultof Lemma 5.10 applied to one of the three pairs ofcamera matrices at a time. Now, these equations maybe rearranged as

P0P = Q0Q01 = Q0Q02

P1P = Q1Q01 = Q1Q12

P2P = Q2Q02 = Q2Q12.

Now, the condition that Q1Q01 = Q1Q12 means thatthe points Q01 and Q12 are collinear with the cameracentre C1

Q of Q1. Thus, assuming that the points Qi j

are distinct, they must lie in a configuration as shownin Fig. 7. One sees from the diagram that if two of thepoints are the same, then the third one is the same asthe other two. If the three points are distinct, then thethree points Qi j and the three camera centres Ci

Q arecoplanar, since they all lie in the plane defined by Q01

and the line joining Q02 to Q12. Thus the three points alllie in the plane of the camera centres Ci

Q. Howevever,since P0P = Q0Q01 = Q0Q02 and P0 = Q0, it followsthat P must lie along the same line as Q01 and Q02 andhence must lie in the same plane as the camera centresCiQ.An equivalent condition is that the projection of P in

camera P0 lies on the line formed by the two epipoles

Figure 7. Configuration of the three camera centres and the threeambiguous points. If the three points Qi j are distinct, then they alllie in the plane of the camera centres Ci

Q.

e01Q and e02

Q and similarly for cameras P1 and P2 whichis equivalent to condition (7). �

If a point P happens to satisfy the condition (7) thenthere may or may not be a conjugate point Q. In areasonable sense, most points lying on the intersectionS10P ∩ S20

P ∩ S21P are critical. Notice, however that if

the three cameras Qi are collinear, then each of thevector products (ei j

Q × eikQ ) vanishes and so condition

(7) is satisfied for all P. In this case we can make noconclusion regarding the existence of a conjugate pointQ. However, if the three cameras Q are not collinearthen we may say more.

Lemma 6.16. Given the assumptions of Theorem6.15 suppose further that the three cameras Qi are dis-tinct and non-collinear. Then any point satisfying thecondition (7) must lie on the intersection of quadricsS10P ∩ S20

P ∩ S21P .

Proof: Let i, j and k represent the three indices 0, 1and 2 in some permuted order, that is i �= j �= k. If thethree cameras are non-collinear, then the two epipolesin one image corresponding to the other two camerasare different. Thus, for each i, the cross product ei j

Q ×eikQ

is non-vanishing. Let P be a point satisfying (7). Then,for each i,

(ei jQ × eik

Q

)�(Pi P) = 0,

which implies that Pi P lies in the span of ei jQ and eik

Q

and so we write

Pi P = αi j ei jQ + αikeik

Q

for some constants αi j and αik . Now, P lies on Si jP if

and only if P�(Pi�Fi jQ P

j )P = 0. Substituting for Pi Pand P j P gives

P�(Pi�Fi j

Q Pj)P= (

αi j ei jQ + αikeik

Q

)�Fi jQ

(α j i e

j iQ + α jke jk

Q

)

= (αikeik

Q

)�Fi jQ

(α jke jk

Q

).

The last equality holds, because ei j�Q Fi j

Q = 0 andFi jQ e j i

Q = 0. Finally, eik�Q Fi j

Q e jkQ = 0, since eik

Q and e jkQ

are a matching point pair in images i and j, correspond-ing to the camera centre of Qk . Thus,

P�Si jP P = P�(

Pi�Fi jQ P

j)P = 0

and so P lies on Si jP . �

Critical Configurations for Projective Reconstruction from Multiple Views 19

The points P that satisfy (7) must be either a singlepoint, a line or a plane lying in the intersection of thethree quadrics Si j

P . If this quadric intersection does notcontain a complete line or a plane, then the latter twocases are not possible. In addition, it may be shown bycontinuity that if (7) defines a single point, then thispoint must be critical (a conjugate Q exists) unless itis an isolated single point in the intersection of the Si j

P .Based on these observations, we may therefore state ageneral ambiguity result:

Theorem 6.17. Let (P0, P1, P2) and (Q0, Q1, Q2)be two triples of camera matrices, with cameras Qi

non-collinear. Then for any point P in the intersectionS10P ∩ S20

P ∩ S21P there exists a conjugate point Q satis-

fying Pi P = Qi Q for all i, with the possible exceptionof

1. A single isolated point P in S10P ∩ S20

P ∩ S21P , or

2. Points P on a single line or plane contained in theintersection S10

P ∩ S20P ∩ S21

P .

This theorem simplifies the search for critical con-figurations, since it is not necessary to worry about thepoints. It is sufficient to find sets of cameras that definequadric intersections of interest.

The question arises as to whether the exceptionalconditions of Theorem 6.17 really occur (the pointsthat are non-critical). Generically, the three quadricswill intersect in 8 isolated points. It was shown inMaybank and Shashua (1998) that if the three quadricsintersect in 8 points then indeed one of these points (theexceptional point identified in Theorem 6.17) does nothave a conjugate. A specific example of this will begiven in Section 8.5. We shall also encounter exam-ples later, where the second of the exceptional cases inTheorem 6.17 occurs.

6.1. Algorithms for Finding Critical Configurations

Given three quadrics S10P ,S20

P and S21P and three cam-

era matrices Pi , it is shown in Theorem 6.17 that theintersection of the three quadrics is a critical set forthe three cameras, if there exist three camera matricesQi (with corresponding fundamental matrices Fi j

Q ) suchthat Si j

P = (Pi�Fi jQ P

j )sym. More exactly, for this con-clusion to hold, Theorem 6.17 requires that the threecamera matrices Qi be non-collinear, and an isolatedlinear subspace contained in S10

P ∩ S20P ∩ S21

P may notbe critical.

In this section we give an algorithm, suitable forimplementation in a symbolic computation packagesuch as Mathematica or Maple, that determines if sucha set of three camera matrices Qi exists, and if so findsall such sets.

Given: Three camera matrices P0, P1 and P2 and threeruled quadrics S10

P , S20P and S21

P .Required: To find all (if any) triples of camera matrices(Q0, Q1, Q2) such that Si j

P = (Pi�Fi jQ P

j )sym.

Algorithm:

1. Knowing the form of the matrix Si jP , we may

compute the fundamental matrices Fi jQ from the

equations Si jP = (Pi�Fi j

Q Pj )sym and det(Fi j

Q ) = 0.The first of these conditions gives a linear set ofequations in the entries of Fi j

Q and the second givesa cubic equation. These equations are easily solvedto find Fi j

Q . There will be one or two solutions forFi jQ depending on whether the two camera centres

are on the same generator or not,3 or possibly in-finitely many solutions in the case where the quadricis degenerate (see Table 1).

2. The previous step gives solutions for the fundamen-tal matrices Fi j

Q . We must test each combination ofsolutions F10

Q , F20Q and F21

Q to find such a triple that iscompatible, by testing the compatibility conditionsof Definition 2.1.

3. Once a triple of compatible fundamental matricesFi jQ has been found such that Si j

P = (Pi�Fi jQ P

j )sym,we may compute the three camera matrices thatcorrespond to the fundamental matrices Fi j

Q . Thismay be done by the following procedure:

(a) Assume that the first camera matrix is Q0 =[I | 0].

(b) From F10Q , we may compute a matrix Q1 =

[[e10Q ]×F10

Q | e10Q ], where e10

Q is found by solv-ing e10�

Q F10Q = 0. Similarly, we compute Q2 =

[[e20Q ]×F20

Q | e20Q ]. These matrices are chosen

such that F10Q and F20

Q are the fundamental ma-trices corresponding to the given camera matrixpairs.

(c) Generally the pair of matrices (Q2, Q1) willnot correspond to the given fundamental ma-trix F21

Q . Modify Q2 so that this will be true. Todo this, replace Q2 computed in the previous

20 Hartley and Kahl

step by a new camera matrix

Q2 = Q2

1

1

1

a b c d

where (a, b, c, d) is chosen such that Q2�F21Q Q

1

is skew symmetric. According to (1), this en-sures that F21

Q is the fundamental matrix cor-responding to the pair of cameras (Q2,Q1),while also F20

Q will correspond to the cam-eras (Q2,Q0). Finding the correct (a, b, c, d)involves solving a set of linear equations. Ifthe three fundamental matrices Fi j

Q correspondto collinear cameras, then there will be a 4-parameter family of solutions for this problem,according to Theorem 2.2.

(d) The three camera matrices (Q0,Q1, Q2) corre-spond to the three given fundamental matricesFi jQ .

In the case where the three camera matrices Qi cor-respond to collinear cameras, then we may still deter-mine a critical set. Unless the three camera matrices Pi

are collinear, then we may apply Theorem 6.17 to thecamera matrices Qi (reversing the roles of the P andQ camera matrices) to conclude that the intersectionS10Q ∩ S20

Q ∩ S21Q is a critical configuration of points for

the camera matrices Qi and the corresponding conju-gate configuration of points lies on the intersection ofthe three quadrics Si j

P .

7. Examples of Critical Sets

We give examples of critical sets for three or moreviews.

7.1. The Elliptic Quartic Critical Set

The first critical set example consists of the intersectionof two quadrics, known as an elliptic quartic curve.According to Theorem 6.14, a critical set for threeviews must consist of the intersection of three quadrics.In the example to be given, all three quadrics intersectin the same elliptic quartic.

Let S10P be the quadric ZT = XY, represented by the

matrix

A =

0 1 0 0

1 0 0 0

0 0 0 −1

0 0 −1 0

, (8)

Let camera matrices be defined by P0 = [I | 0 ]and P1 = [I | (−1,−1,−1)� ] and P2 = [I |(1, 1,−1)� ].

The three camera centres are at (0, 0, 0)� and(1, 1, 1)� and (−1,−1, 1). Let B be another quadricthat passes through the three camera centres. The fam-ily of all such quadrics are represented by matrices ofthe form form B = B′ + B′�, where

B′ =

p q s − t −s − u0 r s + t −s + u0 0 −p − q − r − v v

0 0 0 0

.

(9)As B varies over all such quadrics, the intersection

of A and B encompasses all elliptic quartics passingthrough the three camera centres. Since we are onlyconsidering the pencil of quadrics defined by A and B,we may assume that v = 0, since this can otherwise beachieved by adding A, without changing the intersec-tion of the two quadrics A and B. Thus, the matrix B isdefined by 6 parameters, {p, q, r, s, t, u}.

Two alternative reconstructions involving camerasQi and points Q are given in Tables 2 and 3. It maybe verified directly that Pi P = Qi Q for all points P =(x, y, xy, 1)� and corresponding points Q, providedthat P lies on the quadric B. (It always lies on quadricA). The easiest way to see this is to verify that (Pi P) ×(Qi Q) = 0 for all such points. In fact for i = 0, 1, thecross-product is always zero, whereas for i = 2 it maybe verified by direct computation that

(P2P) × (Q2Q) = (P�BP)(4,−4x, 4)�

for the first solution, and

(P2P) × (Q2Q) = (P�BP)(−4y, 4, 4)�

for the second solution. Thus P2P = Q2Q if and onlyif P lies on B.

Discussion. In this example, the three quadrics Si jP in-

tersect in a common intersection curve, which is thesame as the intersection of any two of them, hence an

Critical Configurations for Projective Reconstruction from Multiple Views 21

Table 2. First conjugate solution to reconstruction problem for cameras Pi and points on the intersection ofquadrics A and B given by (8) and (9) respectively.

The camera matrices are

P0 = [I|0], P1 = [I|(−1, −1,−1)�] and P2 = [I|(1, 1, −1)�]

and

Q0 =

1 0 0 00 1 0 00 0 1 0

, Q1 =

−4 0 0 00 0 2 10 0 −2 1

and

Q2 =

−4 (2p + q − t + u) 8r 4 (p + q + 2r + s + t) −2 (p + q − s − t)

0 8 (r + s − u) −2 (q − t + u) −q + t − u8p −8r −2 (2p + q − 2s + 3t + 3u) 2p + q − 2s − t − u

The conjugate point to P = (x, y, xy, 1)� is Q = ((x − 1)x, (x − 1)y, (x − 1)xy,−2x (−2 + y + xy))� .

elliptic quartic. The centres of the cameras Pi all lie onthe same elliptic quartic. In the conjugate configura-tion, the three quadrics Si j

Q also intersect in an ellipticquartic curve, and all points Q lie on that curve, as dothe centres of the cameras Qi .

It will be shown later in Section 8.2.2 that forthis critical configuration consisting of points andcameras on an elliptic quartic, a one-parameter familyof conjugate configurations exist, and that further, theambiguity extends to the case where there are any num-ber (even infinitely many) cameras lying on the ellipticquartic.

In this example, no two of the camera centres lie onthe same generator of the rank-4 quadric S10

P . Besidesthis assumption, it is quite generic, since up to pro-jective equivalence, the quadric S10

P and the cameras

may be assumed to be of the given form. The examplethen gives all configurations of this form. Neverthe-less, the example is unsatisfactory in that it is essen-tially pulled out of a hat, and gives little insight into thereasons such examples exist. We return to this topic inSection 8.2, and see that it is tied up with a gener-alization of Pascal’s Theorem. The proof given thereavoids tedious computation, and holds equally well forarbitrary configurations of cameras on the quadric S10

P .

7.2. The Rational Quartic Critical Set

Consider a pair of rational curves in projective3-space, P3 : P(θ ) = (1, θ, θ2, θ3)� and Q(θ ) =(θ, θ2 − θ3, θ4, 1)�. The first of these curves is a cubic

Table 3. Second conjugate solution to reconstruction problem.

The camera matrices are

P0 = [I | 0 ], P1 = [I | (−1, −1,−1)� ] and P2 = [I | (1, 1,−1)� ]

and

Q0 =

1 0 0 00 1 0 00 0 1 0

, Q1 =

0 0 −2 10 4 0 00 0 2 1

and

Q2 =

−8 (p + s + u) 0 2 (q + t − u) −q − t + u

−8p 4 (q + 2r + t − u) −4 (2p + q + r + s − t) −2 (q + r − s + t)8p −8r 2 (q + 2r − 2s − 3t − 3u) q + 2r − 2s + t + u

.

The conjugate point to P = (x, y, xy, 1)� is Q = ((y − 1)x, (y − 1)y, (y − 1)xy, 2y (−2 + x + xy))� .

22 Hartley and Kahl

curve (often called a twisted cubic), and the second isa quartic curve, since it contains a fourth power of theparameter θ .

Let λ be a variable parameter and let

Pλ =

1 0 0 0

λ − λ2 1 − λ −1 0

λ3 λ2 λ 1

and Qλ =

1 0 0 −λ

0 1 0 −λ2 + λ3

0 0 1 −λ4

be a pair of camera matrices. The camera centres ofthese two cameras trace out trajectories parametrizedby λ, as follows

CP(λ) = (0,−1, λ − 1, λ)�

and CQ(λ) = (λ, λ2 − λ3, λ4, 1)�.

We see that the first camera centre moves along astraight line, whereas the second camera centre movesalong the quartic curve Q(θ ).

Projecting the points on the two space curves intothe two images leads to

(θ − λ)PλP(θ ) = QλQ(θ ) = (θ − λ, θ2 − λ2

− (θ3 − λ3), θ4 − λ4)�.

Ignoring the insignificant scale factor θ − λ, this saysthat PλP(θ ) = QλQ(θ ), which means that the images ofthe two curves in the respective cameras are the same.Thus in this example, a camera moving in a straightline viewing a twisted cubic gives the same image as acamera moving along a quartic curve viewing its owntrajectory curve. This is an example of reconstructionambiguity. From the set of image measurements andcorrespondences between the successive images it isimpossible to compute the camera motion and scenegeometry unambiguously.

Remarks on This Example. This example of ambi-guity is quite distinct from the critical configurationsdiscussed in Section 8.2.1 and has many curious prop-erties. At first sight, it may appear that it is just a

special case of the elliptic quartic examples, in whichthe twisted cubic and line in the first (P(θ )) configura-tion are simply the intersection of two ruled quadricsand the quartic curve in the conjugate configuration,containing points Q(θ ), is likewise the intersection oftwo distinct ruled quadrics—an elliptic quartic curve.However, that is not the case as will be seen later. Inparticular it will be shown that the curve Q(θ ) lieson a unique quadric and similarly, the twisted cubiccontaining points P(θ ) and the line of centres of thecameras Pλ are not in the intersection of two quadrics.

The Critical Quadrics. Let us compute the criticalquadrics in the case of the example. We choose twovalues λ0 and λ1 of the parameter λ and compute thecritical quadrics according to (5). The computation isstraight-forward, leading to

Sλ0λ1P =

0 0 λ0 + λ1 − λ0λ1 1 − λ0 − λ1

0 −2(λ0 + λ1 − λ0λ1) −1 + λ0 + λ1 −1λ0 + λ1 − λ0λ1 −1 + λ0 + λ1 2 0

1 − λ0 − λ1 −1 0 0

(10)

and

SQ = Sλ0λ1Q =

2 −1 0 0−1 0 0 −1

0 0 0 −10 −1 −1 0

. (11)

Note the fact that SQ is independent of the choice of λ0

and λ1. In other words for all choices of pairs of cam-eras (Pλ0 ,Pλ1 ) and corresponding conjugate cameras(Qλ0 ,Qλ1 ), the critical quadric Sλ0λ1

Q is the same.It is easily verified that the complete curve Q(θ )

lies on the quadric SQ and the curve P(θ ) lies on eachSλ0λ1P , in agreement with the general theory. The inter-

section of the quadrics Sλ0λ1P is the curve P(θ ), hence a

twisted cubic. In addition, the camera centres CP(λ0)and CP(λ1) lie on the quadric Sλ0λ1

P , as they must. How-ever these are the only two points of intersection of theline of camera centres CP(λ) and the quadric Sλ0λ1

P .Since a quadric is defined uniquely by a twisted cubicand two points we may state:

• Sλ0λ1P is the unique quadric that contains the twisted

cubic P(θ ) and the two camera centres CP(λ0)and CP(λ1). Each different pair of camera centres(CP(λ0), CP(λ1)) on the straight line CP(λ) definesa different quadric.

Critical Configurations for Projective Reconstruction from Multiple Views 23

As the example shows, the whole of the ruled quadricSQ indeed does not form a critical set in the currentexample. However, the curve CQ(λ) which lies on SQdoes form a critical set. Note that with the roles ofthe P and Q configurations reversed, the hypothesesof Theorem 6.17 do not hold in this instance, sincethe cameras Pλ are collinear. Consequently, we can notconclude that the whole of SQ is critical, and indeed itis not.

Examples of this type will be considered againin Section 8.3, where it will be proved that suchquadric/cubic curve pairs always constitute a pair ofconjugate configurations.

7.3. Critical Sets with All Quadrics the Same

We investigate next the possibility that a critical set mayexist for three views, consisting of a complete quarticsurface SP. According to Theorem 6.14 this can onlyoccur when the three critical quadrics Si j

P arethe same,and equal to SP. As long as SP is a non-degeneratequadric, we may assume that it is the quadric given byequation ZT = XY.

The previous section gave an example in which allthe quadrics SQ were the same, but it turned out thatthe set of critical points only consisted of a quarticcurve on this quadric. In this section a further examplewill be given, which indicates that this is the typicalsituation, and that it will not happen that the whole ofthe common quadric is critical.

Various configurations for the three camera centreson the quadric SP are possible. In this example, weassume that no two of them lie on the same generatorof the quadric, and then without loss of generality,we may assume that they lie at centres (0, 0, 0, 1)�,(1, 1, 1, 1)� and (−1,−1, 1, 1)�. We attempt to find

a conjugate configuration, using the algorithm ofSection 6.1. The computation shows that there exista family of conjugate configurations. The calculationis carried out in a way as to find the most generalsolution.

According to Proposition 3.7, only the positions ofthe cameras are relevant to the existence of a criticalconfiguration. Consequently, the three camera matricesmay be taken to be

P0 =

1 0

1 0

1 0

; P1 =

1 −1

1 −1

1 −1

;

P2 =

1 1

1 1

1 −1

.

Using the algorithm of Section 6.1 we find that thecamera matrices Qi that corresponds to this configura-tion are of the following form, where a, b, c and d arevariable parameters, defining a 4-parameter family ofmatrices Qi .

Q0 =

1 0

1 0

1 0

; Q1 =

0 0 −1 1

0 2 0 0

0 0 1 1

;

Q2 =

−a −b 1 − c −d

0 2 0 0

a b 1 + c d

.

These cameras are located at positions (0, 0, 0, 1)�,(1, 0, 0, 0)� and (−d/a, 0, 0, 1)�, which are collinear.Corresponding conjugate points are given by

P(θ ) =

0 2 + b + 2d −2 + 2d −b 0

0 −a −1 − c + d 2 + a + 2d −1 + c + d

0 0 2 + b + 2d −2 + 2d −b

−a −1 − c + d 2 + a + 2d −1 + c + d 0

1

θ

θ2

θ3

θ4

and

Q(θ ) =

2 + b + 2d 2(−1 + d) −b 0

−a −1 − c + d 2 + a + 2d −1 + c + d

0 2 + b + 2d 2(−1 + d) −b

−2a −4 − 2a − b − 2c −2(b + c) −b

1

θ

θ2

θ3

24 Hartley and Kahl

where it may be verified that the curve Q(θ ) is theintersection of the three quadrics Si j

Q . By inspection,the curve P(θ ) lies on the quadric SP.

The reader is left to verify that Pi P(θ ) = Qi Q(θ ) forall i = 0, 1, 2 and each θ .

Second Class of Solutions. There is a second similarclass of solutions given by

Q0 =

1 0

1 0

1 0

; Q1 =

−2 0 0 0

0 0 1 1

0 0 −1 1

;

Q2 =

−2 0 0 0

b a −1 + c −d

−b a −1 − c d

.

with points

P(θ ) =

0 −a −1 − c + d 2 + a + 2d −1 + c + d

0 2 + b + 2d 2(−1 + d) −b 0

0 0 2 + b + 2d 2(−1 + d) −b

−a −1 − c + d 2 + a + 2d −1 + c + d 0

1

θ

θ2

θ3

θ4

and

Q(θ ) =

a 1 + c − d −2 − a − 2d 1 − c − d

−2 − b − 2d 2 − 2d b 00 −2 − b − 2d 2 − 2d b

−2a −4 − 2a − b − 2c −2(b + c) −b

1

θ

θ2

θ3

.

The Bidegree of the Curve. Given the curveP(θ ) = (X(θ ), Y(θ ), Z(θ ), T(θ ))�, consider the equa-tions X(θ ) = constT(θ ) and Y(θ ) = constT(θ ). In thefirst family of solutions we see that the equationX(θ ) = constT(θ ) is a cubic equation in θ , havingtherefore in general three solutions. However, y(θ ) =constT(θ ) is linear in θ , and hence there is only onesolution. This means that the curve P(θ ) meets the gen-erators of one class in three points, and generators ofthe other class in a single point. We say that it is acurve of bidegree (3, 1). The second family of solu-tions has the same property, but the roles of X and Yare reversed.

We note that the points and cameras in theP-configuration lie on a quartic curve, whereas in theQ-configuration, the points lie on a twisted cubic, andthe cameras lie on a straight line. This is exactly the

same type of configuration as the rational quartic con-figuration in Section 7.2.

The computation shows that it is not possible (at leastfor this configuration of camera centres) for the criticalpoint set to comprise the complete quadric SP. The setof critical points is restricted to a quartic curve lyingon the quadric. In this computation we considered onlythe case where no two of the cameras lie on a commongenerator of SP. Other configurations of the cameracentres are possible, but the computation yields similarresults, as discussed later in Section 8.1.

8. Classifying All Critical Configurations

It was seen in Theorem 6.14 that the points in a criticalconfiguration involving three cameras Pi must all lie inthe intersection of three ruled quadrics Si j

P . Therefore,

to determine all the possible critical configurations, weneed only to catalog the possible intersections of threeruled quadrics.

However, we have not shown that all possible suchquadric intersections are indeed critical. To reach thisconclusion, there must exist three camera matrices Qi

such that Si jP = (Pi�Fi j

Q Pj )sym, as stated explicitly in

Theorem 6.15.Finding such Qi is not always possible, or straight-

forward, but most of the critical configurations dis-cussed in this paper have been found this way. Anapproach is to find fundamental matrices Fi j

Q such thatSi jP = Pi�Fi j

Q Pj for all pairs (i, j), and then check the

compatibility conditions of Definition 2.1 to try to finda compatible set of Fi j

Q . This is the approach discussedin the algorithm of Section 6.1.

This method will not always work to determinewhether a given quadricintersection is critical, because

Critical Configurations for Projective Reconstruction from Multiple Views 25

knowing the intersection of the quadrics does not nec-essarily uniquely determine the three quadrics Si j

P . Inaddition, Theorem 6.15 specifies that the critical pointset must be the intersection of three quadrics, but thecamera centres do not always lie in this intersection.Nevertheless, finding triples of compatible fundamen-tal matrices Fi j

Q is the basic method for discoveringsuch critical configurations.

We do not intend exhaustively to enumerate and con-sider all the possible intersections of three quadrics. InSection A.1, it is shown how to derive possible quadricintersections and a complete list is given for possi-ble intersections when (at least) one of the quadrics isnon-degenerate. However, we do consider all types ofintersections, listed next.

1. The three quadrics Si jP are all the same, and hence

their intersection is a quadric.2. The intersection of the three quadrics is the same as

the intersection of just two of them. The intersectionof two quadrics is an elliptic quartic, or some de-generate form of elliptic quartic.

3. The intersection of the three quadrics is a curve,namely a twisted cubic, a conic or a line (possiblywith some isolated points).

4. The quadrics intersect in eight isolated points (pos-sibly some complex).

The first of these cases is interesting, since it is theonly possibility for a critical configuration in which thepoints constitute a complete surface. In the other cases,the point sets are of lower dimension.

The intersections of three quadrics for which explicitexamples are not given in this paper are those intersec-tions consisting of one, two or three lines, as well as aconic plus an intersecting line. A critical set consistingof points on a single line is not very interesting, andthose consisting of two or three intersecting lines maybe considered as degenerate conics or twisted cubics.Similarly, the critical set consisting of a conic plus aline is a degenerate case of the twisted cubic. Thesecurves can be analyzed in the same way as is done inSection 8.4 for twisted cubic and conic curves.

8.1. Three Quadrics the Same

We consider first the case in which all three quadricsSi jP

are the same, and determine whether this quadric formsa critical set for three views. In this investigation wefocus mainly on the case where SP is a non-degenerate

ruled quadric. The case where SP is a cone is alsoconsidered, but not in so great detail. The planar caseis given in the next subsection.

Non-Degenerate Quadric. In Section 7.3 the conju-gate configurations were computed in the case where(without loss of generality) the common quadric SPwas given by the formula XY = ZT, or simply Z = XY

in non-homogeneous coordinates. The generators onthis quadric are simply the lines of constant X or Y

coordinate, and the computation was carried out in thecase where no two of the camera centres were on thesame generator. Assuming that the camera centres aredistinct, there are four possible projectively inequiva-lent configurations for the camera centres, namely:

Configurations of Three Camera centres

1. No two camera centres lie on a common gen-erator. We may assume that the camera centresare at the points (0, 0, 0, 1)�, (1, 1, 1, 1)� and(−1,−1, 1, 1)�.

2. Two camera centres lie on a common generator (butthe third one does not). The camera centres can beassumed to be at points (0, 0, 0, 1)�, (−1, 0, 0, 1)�

and (1, 1, 1, 1)�.3. The three cameras all lie on the same generator. We

may assume that they are at positions (0, 0, 0, 1),(1, 0, 0, 1)� and (−1, 0, 0, 1)�.

4. The second and third cameras lie on the two gener-ators passing through the first camera centre. Thus,the camera centres are at positions (0, 0, 0, 1)�,(1, 0, 0, 1)� and (0, 1, 0, 1)�.

The fact that these configurations are all that need beconsidered follows from the following lemma.

Lemma 8.18. Let f and g be 1-dimensional projec-tive transformations. Then there exists a 3D projectivetransformation T that takes the point (x, y, xy, 1)�

to ( f (x), g(y), f (x)g(y), 1)�, and hence fixes thequadric Z = XY .

In other words, we may apply independent projectivetransformations to the X and Y axes on the quadric Z =XY and extend it to a transformation of the whole ofP3

fixing the quadric. From this lemma it is easy to see thatany two configurations of three points (camera centres)on the quadric Z = XY is equivalent to one of thefour transformations listed above. The lemma is quitesimple to show, but a proof is not considered central tothe argument of this paper, and so it is omitted.

26 Hartley and Kahl

The first step in deriving possible critical sets is todetermine consistent triples of permissible generatorpairs. Recall from Lemma 5.12 that the permissiblegenerator pairs determine the conjugate fundamentalmatrices and, in turn, these must be compatible ac-cording to Definition 2.1. If generator gi j

P = gikP then

it means that the epipoles ei jQ and eik

Q are equal, andhence the conjugate cameras are collinear. On the otherhand, if gi j

P �= gikP then ei j

Q �= eikQ and the conjugate

cameras must be non-collinear. Thus, it follows fromTheorem 2.2 that the two generators passing througha single camera centre are all either pairwise identicalor pairwise distinct. In addition, according to Lemma5.10, permissible generator pairs do not meet, that is,gi jP ∩g j i

P = ∅ or meet at a singular point of the quadric.Consequently, we are required to find generators gi j

P

satisfying the following two conditions:

∀i : gi jP = gik

P or ∀i : gi jP �= gik

P

gi jP and g j i

P are disjoint or intersect in a singular point

of the quadric. (12)

Using these conditions it is easy for the reader toverify in the case of a non-degenerate quadric the fol-lowing:

1. There are two possible conjugate camera tripleswhen no camera centres lie on a common gener-ator.

2. There is one possible conjugate camera triple when(i) two camera centres lie on a common generator(but the third one does not) or (ii) the three camerasall lie on the same generator.

3. There is no conjugate camera triple when the secondand third cameras lie on the two generators passingthrough the first camera centre. See Fig. 8 for anillustration.

4. In all these cases, the camera centres of the conju-gate triple are collinear.

At this point, we may carry out the algorithm ofSection 6.1 explicitly to determine the collinear con-jugate configurations. The result of this calculation isa triple of quadrics Si j

Q and a set of conjugate camerasQ0, Q1 and Q2.

The next step is to compute the intersection of thethree quadrics, S10

Q ∩ S20Q ∩ S21

Q . In all cases, the threequadrics intersect in a twisted cubic curve. The inter-section curve may be parametrized by a parameter θ ,and expressed in the form Q(θ ) = A�, where A is

Figure 8. Three cameras on the same quadric. With camera centresas shown on generators of the quadric, there is no consistent set ofpermissible generators gi j

P satisfying the conditions (12). If g12P and

g21P are disjoint then just one of them (suppose g12

P ) must pass throughC0P. On the other hand, neither g10

P nor g20P can pass through C0

Potherwise they will intersect with g01

P or g02P . However, this means

that g10P �= g12

P , whereas g20P �= g21

P , in contradiction of the firstcondition of (12).

a 4 × 4 matrix, and � = (1, θ, θ2, θ3)�. Accordingto Theorem 6.17 this must be a critical set for recon-struction (except in the case when the three cameras Pi

are collinear, though in reality it is critical in this casealso). To find the corresponding curve P(θ ) we proceedto solve the equations

P0P(θ ) = Q0Q(θ ); P1P(θ ) = Q1Q(θ )

for P(θ ). Since all other quantities in these equationsare known, this is simply a triangulation problem.

The result in the case where the camera centres alllie on different generators of the quadric is given as anexample in Section 7.3. The example gives explicitlyall the possibilities (up to projectivity) in that case.

The computations are carried out in a similar mannerin all the other possible cases. In the case where twocameras are on a common generator, but the third is not,there is one conjugate configuration but otherwise theresult is the same: the three cameras Qi are collinear,and the curves Q(θ ) and P(θ ) are cubic and quadriccurves respectively.

In the case where the three camera centres of thePi lie on a single generator, the three quadrics Si j

Q areall degenerate (cones). However, in this case also theyintersect in a twisted cubic Q(θ ), and the curve P(θ ) isa quartic curve, as in the other cases.

The Cone Case. When SP is equal to a cone, there areonly two inequivalent configurations, namely:

1. The three camera centres are on three different gen-erators, and

2. One camera centre is at the vertex and the other twoon different generators.

Critical Configurations for Projective Reconstruction from Multiple Views 27

Two camera centres on the same generator are in-feasible.

In the cone case, it follows immediately from thefact that there is only one generator class on a cone,that there is only one conjugate camera triple and thatthe camera centres are collinear.

The result of the computations and the above dis-cussion may be summarized as follows.

Proposition 8.19. Let Pi and Qi for i = 0, 1, 2 betwo sets of camera matrices, satisfying the propertythat the three quadrics Si j

P = (Pi�Fi jQ P

j )sym are all thesame non-degenerate quadric or cone, denoted simplySP. Then

1. The camera centres of the three Qi are collinear.2. The three quadrics Si j

Q = (Qi�Fi jP Q

j )sym intersect ina twisted cubic curve.

3. This twisted cubic curve is a curve of critical points.In other words, if Q(θ ) is a point parametrized byθ , lying on this curve, then there exists a conjugatepoint P(θ ) lying on SP such that Pi P(θ ) = Qi Q(θ )for all i = 0, 1, 2.

4. The points P(θ ) lie on a rational quartic (fourth-degree) curve parametrized by θ . This curve lieson the quadric SP, contains the centres of the threecameras Pi and in the case of a non-degeneratequadric, the curve has bidegree (3, 1) on SP.

5. For the nondegenerate quadric case, any configura-tion of the centres of the three cameras Pi can occur,except that in which two of the camera centres lie ongenerators from opposite classes, passing throughthe third camera centre. (Three such camera centrescan not all lie on a quartic curve of bidigree (3, 1)).

6. For the cone case, any configuration of the threecameras Pi can occur, except where two cameraslie on the same generator (but not at the vertex).

Since for all cases, the camera centres in the conju-gate configuration have collinear centres, it is impossi-ble to conclude from Theorem 6.17 that the quadric SPforms a critical set. Nevertheless, although the wholeof SP is not critical, there exists a quartic curve P(θ )lying on SP that is critical, and the corresponding con-jugate curve Q is the intersection of the three quadricsSi jQ .We will see more about this configuration in

Section 8.3 and it will be shown that it may be extendedto any number of cameras as long as their centres alllie on the same quartic curve P(θ ).

Planar Case. In the case where the three quadricsSi jP are all equal to the degenerate quadric XY = 0

consisting of two planes X = 0 and Y = 0, somethinga little different happens.

As seen in Table 1, in order for such a configurationto be critical, all the camera centres must lie on thesame plane (say the plane X = 0). With three cameras,the different cases are that 0, 1, 2 or all 3 camera centresline on the line X = Y = 0 of intersection of the twoplanes, and there are sub-cases according to whetherthe three camera centres Ci

P are collinear or not.In all these cases, it turns out from a simple hand-

calculation that there are conjugate configurations inwhich the three camera centres Ci

Q are non-collinear.For example, the configuration of generators shownin Fig. 9 satisfies the conditions of (12), with gi j

P �=gikP for i �= k. Consequently, the three corresponding

fundamental matrices Fi jQ are compatible, there exist

three camera matrices Qi in a conjugate configuration,and the camera centres of the Qi are not collinear. Thiscontrasts with the case where all the Si j

P are the samecone or non-degenerate quadric, and we have seen thatthe three camera centres Ci

Q must be collinear. Directcomputation shows that the three quadrics Si j

Q are thesame and consist (like the Si j

P ) of a pair of planes.Now, appealing to Theorem 6.17, it follows that the

complete set of points on Si jP belongs to the critical set

except for the points lying on a single line or plane. Inother words, one of the planes (it turns out to be theplane Y = 0 not containing the three cameras centres)is critical, and the other plane X = 0 may not be. Onceagain direct computation shows however that there is afurther conic curve lying on the plane X = 0 consistingof critical points, that is, ones that have conjugates.

In summary: if all Si j − {P} are the same degen-erate two-plane quadric, then the critical set of pointsconsists of the plane not containing the camera centers,plus a conic curve lying on the other plane. This is the

Figure 9. Three cameras on a plane and a consistent set of per-missible generator pairs.

28 Hartley and Kahl

only non-trivial three-view critical configuration con-taining a dimension-2 surface. An example will helpunderstand this critical configuration better.

Example. We define a camera matrix

Pθ =

1 0 0 0

0 −θ 1 0

0 −θ2 0 1

and a point Pλ = (0, 1, λ, λ2)�. The centre of thecamera Pθ is located at the point (0, 1, θ, θ2)�, andhence as θ varies, it traces out a conic curve on theplane X = 0 . The point Pλ traces out the same curveas λ varies.

Now, note that for a point (X, 0, Z, T)� lying on theplane Y = 0 , we have Pθ (X, 0, Z, T)� = (X, Z, T)�,independent of θ . (We can think of the plane Y = 0 asthe focal plane of the image, since points on this planeare mapped to points with the same coordinate in theimage.) We may also compute

PθPλ =

0

λ − θ

λ2 − θ2

=

0

1

λ + θ

.

Now, let α be any fixed value, and define a second setof camera matrices Qθ = Pθ+α , and points Qλ = Pλ−α .

Then, we may compute

Qθ

X

0

Z

T

= Pθ+α

X

0

Z

T

=

X

Z

T

= Pθ

X

0

Z

T

and

QθQλ = Pθ+αPλ−α = (0, 1, (θ + α) + (λ − α))�

= (0, 1, θ + λ)� = PθPλ .

This means that Pθ and Qθ are conjugate sets of cam-eras, and the corresponding points in the critical set arethe points Pλ lying on a conic in the plane X = 0, plusall the points in the plane Y = 0.

Finally, we note that there is no homography thattakes Pλ = (0, 1, λ, λ2)� to Qλ = (0, 1, λ + α, (λ +α)2)� while at the same time fixing the points on theplane Y = 0. Thus, this does not constitute a trivialambiguity.

8.2. Pascal’s Theorem and Elliptic Quartics

In Section 7.1 an example was given of a critical config-uration in which the three quadrics Si j

P all intersectedin a common curve, equal to the intersection of anytwo of them. The three quadrics in this case all belongto a pencil (one-parameter family). This example wasfound by a process of ad-hoc reasoning that will not berepeated here. (See Kahl, Hartley, and Astrom (2001)for a brief explanation of this).

Examples of this kind of critical configuration cannot be found directly using the computation method ofSection 6.1. This is because we do not know the threequadricsSi j

P in the pencil that defines the elliptic quarticcurve — it is not possible to choose arbitrary quadricsin this pencil. Instead, we will give a more general andabstract argument to prove that any number of pointson an elliptic quartic curve and any number of camerasalso lying on the same curve, always form a critical set.A key argument in the proof relies on a generalizationof Pascal’s theorem.

At the age of 16, Pascal discovered a famous theoremconcerning conics.

Theorem 8.20 (Pascal’s Theorem). If A, B, C, A′,B′ and C′ are six points on a conic, then the pointsAB ∧ A′ B ′, BC ∧ B ′C ′ and C A′ ∧ C ′ A are collinear.

In this theorem (for instance) AB represents the linejoining points A and B and AB ∧ A′ B ′ represents theintersection point of the two lines. The theorem is il-lustrated in Fig. 10.

It is clear that Pascal’s Theorem is a theorem ofprojective geometry, that is unchanged by a projectivetransformation. This being so, we may choose the linecontaining the intersections of the chords such as AB∧A′ B ′ to be the line at infinity. This means that the chordsin question are parallel. In particular, the conclusionof Pascal’s Theorem may be stated as: if AB ‖ A′B′

and BC ‖ B′C′ then CA′ ‖ C′A, where the notation

Figure 10. Pascal’s Theorem of six points on a conic.

Critical Configurations for Projective Reconstruction from Multiple Views 29

AB ‖ A′B′ means that the two lines in question areparallel.

The construction in Pascal’s Theorem involves twocurves—a conic and a line. This is just a special caseof a degenerate cubic curve. In fact, Pascal’s Theoremis really about cubics in the plane. First, a convientdefinition is introduced to simplify things. Given fourpoints on a cubic A, B, C and D, we say that two chords,AB and CD, are quasi-parallel if they meet in a commonthird point on the cubic. We use the notation AB ‖ C Dto mean that the lines in question are quasi-parallel.Then, one can state Pascal’s Theorem for cubics asfollows.

Theorem 8.21. Let A, B, C, A′, B ′ and C ′ denote sixpoints on a cubic in the plane P2. If AB ‖ A′ B ′ andBC ‖ B ′C ′ then C A′ ‖ C ′ A.

The theorem is a simple consequence of anotherwell-known theorem, namely the “Nine AssociatedPoints Theorem” which states that any cubic curvethat passes through eight of the nine intersections oftwo given cubic curves automatically passes throughthe ninth (Evelyn et al., 1974).

A higher dimensional analogue of Pascal’s Theoremturns out to be crucial in the analysis of three-viewcritical configurations. The role of the conic is takenby an elliptic quartic curve. The notion of parallelism isnot precisely what is needed here, since a line throughA′ parallel to the line AB will not in general meet thequartic in any other point B ′. The concept that weneed to replace parallelism is that the two chords ABand A′ B ′ are generators on a common quadric lying inthe pencil of quadrics that defines the elliptic quarticcurve and moreover, they belong to the same class ofgenerators on that quadric.

Before we state the theorem, a simple observationconcerning chords of an elliptic quartic is needed.

Lemma 8.22. Let X and B be two points on an el-liptic quartic defined as the intersection of a pencil ofquadrics. Then there exists a quadric S belonging tothe pencil such that the line AB is a generator of S.The quadric S is unique, unless the line AB lies in theintersection of all quadrics in the pencil.

Proof: The pencil is generated by two quadrics S0

and S1 and a general quadric in the pencil is defined byS = λS0 + µS1. Let C be a third point on the line AB.If C lies on the quadric S, then the whole of the line

AB lies on S, since a line not lying on a quadric meetsit in only two points. The condition that C lies on thequadric S leads to a homogeneous linear equation in λ

and µ, which has a unique solution up to scale—thatis, unless C lies on both S0 and S1, and hence on allquadrics in the pencil. �

In order to state Pascal’s theorem for elliptic quarticcurves, we need a definition of quasi-parallelism, justas we did for the case of cubic curves.

Definition 8.23. Two chords of an elliptic quarticcurve are called quasi-parallel if they both lie on acommon quadric in the pencil defining the quarticcurve and constitute a permissible pair of generatorson that quadric (see Definition 5.11).

For points A, B, C, D on an elliptic quartic, we againuse the notation AB ‖ C D to mean that the lines inquestion are quasi-parallel.

Restrictions. When we talk of elliptic quartics, weare really only interested in those curves that ariseas the intersection of two ruled quadrics. In addition,since they do not arise as critical point sets, we are notinterested in the case where any of the quadrics is aplane (such as X2 = 0), or a single line ( X2+ Y2 = 0).This leaves three types of quadrics: hyperboloid, coneor two planes (XY = 0).

If the pencil contains the two-planes quadric, thenthe elliptic quartic consists of two planar conics inter-secting in two points. This case is somewhat special,but the theorems proved below remain valid for thiscase, with certain provisos. To avoid breaking the flowof argument the stated results and proofs will assumethat this case does not occur (elliptic quartics involveddo not consist of two planar conics), but we will pointout special conditions that apply to this case in foot-notes.

Any pencil that contains a hyperboloid may be de-fined as the intersection of two hyperboloids. However,there may still remain isolated rank-3 quadrics (cones)in the pencil. The possibility exists that the singularpoint (vertex) of the cone lies on the elliptic quartic.To avoid complication, we often need to rule out suchpoints, so we make the following definition.

Definition 8.24. A point on an elliptic quartic curveis called a regular point if it is not a singular point ofany quadric in the pencil that defines the curve.

30 Hartley and Kahl

Most elliptic quartics contain only regular points.Their importance derives from the observation thatthrough any regular point A on the curve, there exists aunique chord AB quasi-parallel to any other chord CD4.Now, we state the generalization of Pascal’s Theorem:

Theorem 8.25. Let A, B, C, A′, B ′ and C ′ denotesix regular points on an elliptic quartic curve in P3. IfAB ‖ A′ B ′ and BC ‖ B ′C ′ then C A′ ‖ C ′ A.

It is possible to prove this theorem by selecting anarbitrary point on the quartic, and projecting from thispoint onto P2. The quartic curve projects to a planecubic curve, and the result follows from Pascal’s The-orem for Cubics (Theorem 8.21). However, since wehave not given a full proof of Theorem 8.21, we preferto give a direct proof of Theorem 8.25.

The following remark will be used repeatedly in theproof of Theorem 8.25.

Lemma 8.26. If four points W, X, Y and Z on anelliptic quartic are coplanar, then there exists a quadricin the pencil on which WX and YZ are intersectinggenerators.

Proof: Let SW X be the quadric in the pencil thatcontains the line WX. The intersection of SWX withthe plane is a conic, containing the line WX and theother two points Y and Z. Consequently the conic isdegenerate, consisting of the two intersecting lines WXand YZ. �

Note that the order of the coplanar points is immate-rial, so in the same way there are conics containing theline pairs (WY, XZ) and (WZ, XY). We introduce fur-ther notation: for points on an elliptic quartic, we write[ABCD] to denote that there exists a plane meeting thequartic in just these four points. Note that the order ofthe points in the bracket is immaterial.

A few properties of coplanar points will facilitatethe proof of Theorem 8.25.

Lemma 8.27. Let an elliptic quartic curve be givenand let A, B, C and D denote regular points on thecurve.

1. Given A, B and C, then there exists a further pointD such that [ABCD].5

2. [ABCD] and AB ‖ A′ B ′ implies [A′ B ′C D].3. [ABCD] and [A′ B ′C D] implies AB ‖ A′ B ′.

Proof:

1. Suppose [ABCD] and AB ‖ A′ B ′. On the quadricSAB, line AB is a generator meeting CD, whereasA′ B ′ is a generator from the same class as AB.Consequently, A′ B ′ and CD intersect and so A′, B ′,C and D are coplanar.

2. If [ABCD] and [A′ B ′C D], then on the quadric SCD,both AB and A′ B ′ are generators meeting the gen-erator CD. So AB ‖ A′ B ′.

�

Now, it is simple to prove Theorem 8.25.

Proof: For points on an elliptic quartic, we assumeAB ‖ A′ B ′ and BC ‖ B ′C ′. It is required to prove thatC A′ ‖ C ′ A. All steps of the proof follow directly fromLemma 8.27.

1. Find a point D such that [ABCD]. Then,2. [ABCD] and AB ‖ A′ B ′ implies [A′ B ′C D].3. [ABCD] and BC ‖ B ′C ′ implies [AB ′C ′ D].4. Finally [A′ B ′C D] and [AB ′C ′ D] implies A′C ‖

AC ′ as required.

In the case where the quartic curve consists of twoconics, there are two possibilities. If all of A, B andC lie on the same conic, then so must A′, B′ and C′,by the definition of permissible generators on a pairof planes. Then the theorem reduces directly to theclassical Pascal’s Theorem (Theorem 8.20.) If A, Band C do not all lie on the same conic, then the proofgoes through directly as given. �

8.2.1. The Elliptic Quartic Critical Set. We nowinvestigate critical configurations consisting of ellipticquartics. Some examples of randomly generated ellip-tic quartics are given in Fig. 11. Notice that ellipticcurves do not need to be connected. It will be shownthat any number of points and any number of cameraslying on an elliptic quartic constitutes a critical con-figuration. The proof makes essential use of Pascal’sTheorem for elliptic quartics, Theorem 8.5.

Theorem 8.28. Let P0, P1 and P2 be three cameraslying on an elliptic quartic.6 Then there exists a conju-gate configuration of three cameras Q0, Q1 and Q2 suchthat for any point P lying on the quartic, there exists apoint Q such that Pi P = Qi Q for all i.

Critical Configurations for Projective Reconstruction from Multiple Views 31

Figure 11. Examples of randomly generated elliptic quartics. In all examples, the pencils contain ruled quadrics.

Proof: Denote the centre of camera Pi by CiP and

let A1 be any other point lying on the quartic curve,as shown in Fig. 12. Define A0 such that C1

PA0 isthe chord quasi-parallel to C0

PA1 and define A2 suchthat C1

PA2 is quasi-parallel to C2PA1. According to

Theorem 8.25, the chords C0PA2 and C2

PA0 are alsoquasi-parallel. Now denote the ruled quadric Si j

P tobe the one containing the elliptic quartic and the twochords Ci

PA j and C jPAi as generators (see Lemma

8.22).Consider in particular S10

P . This is a quadric contain-ing the two camera centres C0

P and C1P. The rays C0

PA1

and C1PA0 are a pair of generators from the same class

on S10P . According to Lemma 5.12, there exists a funda-

mental matrix F10Q such that S10

P = (P1�F10Q P

0)sym. Fur-thermore, F10

Q may be chosen such that e01Q = P0A1 and

e10Q = P1A0, where e01

Q and e10Q are the epipoles of F10

Q .In the same way, it is possible to choose fundamental

matrix F20Q such that e02

Q = P0A2 and e20Q = P2A0

and fundamental matrix F21Q such that e12

Q = P1A2

and e21Q = P2A1. After rearranging the order of these

expressions, we obtain

e10Q = P1A0; e20

Q = P2A0

e01Q = P0A1; e21

Q = P2A1 (13)

Figure 12. Elliptic quartic illustration used in proof of Theorem8.28.

e02Q = P0A2; e12

Q = P1A2.

We wish to show that the three fundamental matricesso chosen are compatible, as defined in Definition 2.1.Using the first pair of these equalities, we may write

e20Q F

21Q e10

Q = A�0 P

2�F21Q P

1A0 = A�0 S

21P A0,

which is zero, because A0 lies on all quadrics in thepencil and in particular on S21

P . From the other tworows of (13) we derive similar equations, resulting inthe following conditions:

e02Q F

01Q e12

Q = 0; e01Q F

02Q e21

Q = 0; e10Q F

12Q e20

Q = 0

32 Hartley and Kahl

These are precisely the conditions for the three fun-damental matrices Fi j

Q to be compatible and it resultsfrom Theorem 2.2 and Theorem 6.17 that there ex-ist three camera matrices Q0, Q1 and Q2 that form aconjugate configuration to the camera matrices Pi . �

Notice that it follows from Theorem 6.14 that if thepoints P in a critical configuration all lie on an ellipticquartic, then the camera centres must lie on the samequartic.

8.2.2. A Family of Solutions. Given a quartic curvedefined as the intersection of a pencil of quadrics con-taining three camera centres, a conjugate solution wasexplicitly constructed in the proof of Theorem 8.28.One may ask if there exist more conjugate solutionsfor a given critical configuration.

In the geometrical construction, the point A1 waschosen arbitrarily on the quartic. The critical quadricS10P was then set to the ruled quadric containing the

chord C0PA1 as one if its generators. By picking an-

other point on the quartic A′1 another conjugate solu-

tion is obtained. It is easy to see that the conjugatesolutions arising from two different points A1 and A′

1are projectively inequivalent. For let S10

P and S′10P be the

corresponding ruled quadrics and let F10Q and F′10

Q bethe fundamental matrices for the two different config-urations. Since S10

P = P1�F10Q P

0 and S′10P = P1�F′10

Q P0

are different, it follows that F10Q �= F′10

Q . Since the fun-damental matrices are different, the two configurationsare not projectively equivalent. This shows:

Theorem 8.29. Suppose three cameras and a set ofpoints {Pi , P j } lie on a curve defined as the intersectionof a pencil of quadrics. Then for each ruled quadricS in the pencil there exists a conjugate configurationof the form {Qi , Q j }, such that the critical quadricS10P = P1�F10

Q P0 = S.

Thus, unlike critical configurations in the two-viewcase, which generally allow two conjugate solutions(according to Table 1), critical configurations for threeviews give rise to an infinite family of conjugate con-figurations.

8.3. The Rational Quartic Critical Set

In Sections 7.2 and 7.3 examples were given of criticalconfigurations consisting of a rational quartic curve,conjugate to a twisted cubic. In this section, we will

show that this is representative of all rational quarticcurves, which in fact are always critical. More back-ground information on rational quartics and their rela-tionship to elliptic quartics can be found in Appendix B.

We prove in this section that all rational quartics arecritical, as expressed in the following theorem.

Theorem 8.30. Let A�(3) and A′�(4) be non-degenerate cubic and quartic curves in P3. Let Qbe a camera with centre lying on the quartic curveA′�(4). Then there exists a camera P such thatPA�(3) =QA′�(4), for all θ . Furthermore, as the camera Q movesalong the quartic curve, the locus of the camera cen-tre of P is a straight line, determined only by the twocurves.

If the cubic curve A�(3) is fixed, then for varyingchoices of the quartic curve A′�(4), this straight lineranges over all possible lines in space.

This result may be expressed more simply as thefollowing corollary.

Corollary 8.31.

1. Any configuration of points and cameras all lyingon a rational quartic curve is critical.

2. Any configuration of points lying on a twisted cubicand cameras lying on a straight line is critical.

These two types of configuration are conjugate, in thata configuration of one type always has a conjugateconfiguration of the other type.

We now proceed to prove Theorem 8.30.

Proof: We are free to choose a coordinate frame insuch a way that the cubic curve is equal to �(3) =(1, θ, θ2, θ3)�, that is A is the identity matrix. Denotethe vector (1, λ, . . . , λ4) by (4). Now letQ be a camerawith centre lying on the curve A′�(4). Thus, for somevalue of a parameter λ, the point A′(4) is the cameracentre for Q, which means QA′(4) = 0. Now defineP = QA′G, where G is the 5 × 4 matrix

G =

0 0 0 0

1

λ 1

λ2 λ 1

λ3 λ2 λ 1

.

Critical Configurations for Projective Reconstruction from Multiple Views 33

It will be shown that P�(3) = QA′�(4) for all θ . First,we compute

G�(3) =

0

1

λ + θ

λ2 + λθ + θ2

λ3 + λ2θ + λθ2 + θ3

from which it follows that

(θ − λ)G�(3) =

1

θ

θ2

θ3

θ4

−

1

λ

λ2

λ3

λ4

Consequently,

(θ − λ)P�(3) = (θ − λ)QA′G�(3) =QA′�(4) − QA′(4)

= QA′�(4)

The last step follows from the fact that QA′(4) = 0.Hence, ignoring the insignificant scale factor θ −λ wesee that P�(3) = QA′�(4) as required.

Next, we compute the camera centre of P. Let n =(n0, n1, . . . , n4)� be the generator of the (right) null-space of A′. Then we show that the camera centre of Pis the point

CP = λ(n0, n1, n2, n3)� − (n1, n2, n3, n4)�,

which traces out a straight line as λ varies. Computefirst

GCP =

0 0 0 0

1

λ 1

λ2 λ 1

λ3 λ2 λ 1

λn0 − n1

λn1 − n2

λn2 − n3

λn3 − n4

=

0

λn0 − n1

λ2n0 − n2

λ3n0 − n3

λ4n0 − n4

= n0(4) − n.

Consequently, PCP = QA′(n0(4) − n) = 0, since n is

in the null space of A′ and QA′(4) = 0. Thus, CP is thecamera centre of P as required.

Finally, we wish to show that any line may be writtenin the form

λ(n0, n1, n2, n3)� − (n1, n2, n3, n4)�

for n the null space of some A′ representing the co-efficients of a quartic curve. First, it may easily beverified that any vector n can appear as the gener-ator of the null-space of a suitable matrix A′ repre-senting a quartic curve, except for vectors of the form(1, α, α2, α3, α4)� for some α (including the limit case(0, 0, 0, 0, 1)� when α = ∞). This exception is nec-essary, since if A′(1, α, α2, α3, α4)� = 0 then θ − α

is a common factor of the polynomials ai(θ ) whereA′�(4) = (a1(θ ), a2(θ ), a3(θ ), a4(θ ))� and so A′

does not represent a quartic curve (see Definition2.4). Note however that for these excluded vectors,the expression λ(n0, n1, n2, n3)� − (n1, n2, n3, n4)� =(λ − α)(1, α, α2, α3)� does not represent a line as λ

varies, but only a single point. Thus, we need only showthat any line in space can be represented in the aboveform for some choice of n. This line passes throughthe points (n0, n1, n2, n3)� and (n1, n2, n3, n4)�. Con-sider a line passing through two arbitrary pointsp = (p0, p1, p2, p3)� and q = (q0, q1, q2, q3)�. Weneed to show that there exist values a, b, c, d suchthat ap + bq = (n0, n1, n2, n3)� and cp + dq =(n1, n2, n3, n4)�. This leads to a set of equations ofthe form

p1 q1 −p0 −q0

p2 q2 −p1 −q1

p3 q3 −p2 −q2

a

b

c

d

= 0

which must always have a non-zero solution for a, b,c, d. �

Thus, this theorem has shown that both cubic/lineand rational quartic configurations are critical, for ar-bitrary cubic/line pairs and for arbitrary rational quar-tics. Note that as a corollary of this result, an arbitraryrational quartic will always be contained in a quadricsurface (the critical quadric for this configuration).

34 Hartley and Kahl

8.4. Lower Degree Critical Sets

The possible intersections of three quadrics are dis-cussed in Section A.2. If the intersection of all threequadrics is the same as the intersection of two of them,then the intersections is an elliptic quartic (or someseparable fourth-degree curve such as two intersect-ing conics). These configurations have been consid-ered in Section 8.2 where it was shown that they werecritical.

Now, we consider the case where the intersection ofthree quadrics is a proper subset of the intersection ofany two. We may form this intersection by taking theintersection of two quadrics, and intersecting it with athird one. According to Theorem A.35 each componentintersects the third quadric in a finite set of points, ora complete curve. Thus, the only curves that can beformed by the intersection of three quadrics (a propersubset of the intersection of two quadrics) are: a twistedcubic; a conic; one to three lines; or a conic plus a line.In addition, there may be some isolated intersectionpoints, which we will not be very interested in.

In this section, we will consider the question of iden-tifying the curves that may be conjugate to a curve ofone of the above types lying on a quadric. Such acurve is first of all critical with respect to any pair ofthe views, and its conjugate is the conjugate with re-spect to this pair of views. This leads us to considercurves lying on a two-view critical quadric, denotedSP = S10

P . Our purpose is to give a way of enumeratingall the possibilities of critical curves of this type andtheir conjugates.

As mentioned previously (Section 7.3) algebraiccurves on a non-degenerate ruled quadric may be clas-sified by their bidegree, which is the number of timesthey meet the two different classes of generators. In thecase of a critical configuration consisting of two cameracentres on a quadric, we may extend this classificationby relating the curve to the two camera centres. Givena conjugate pair of two-view critical configurations,the two generators through camera centre C0

P on SP aredenoted by g0

P and g0P respectively, with notation as in

Lemma 5.10. We say that a curve is of type (g, g; c0, c1)on SP if it meets the generator g0

P in g points, generatorg0P in g points, and camera centres C0

P and C1P in c0 and

c1 points respectively. (In counting all these intersec-tions we include also complex or double intersectionpoints.) Note that such a curve must also meet g1

P ing points and g1

P in g points. Note that (g, g) is thebidegree of the curve, and so its degree is g + g.

We now wish to determine the type of the corre-sponding curve onSQ. To this end, we list the followingcorrespondences between points on SP and points onSQ in Table 4, which is derived directly from Lemma5.10.

Table 4. Correspondence of points on SP and SQ. For instancethe first line indicates that the conjugate of a point on g0

P other thanthe camera centre C0

P must be the camera centre C1Q in the conjugate

configuration.

Point or line in S10P

Corresponding point orline in S10

Q

g0P − C0

P C1Q

g1P − C1

P C0Q

g0P − C0

P g0Q − C0

Q

g1P − C1

P g1Q − C1

Q

C0P g1

Q − C1Q

C1P g0

Q − C0Q

Now, referring to this table, we see that a curve oftype (g, g; c0, c1) on SP meets g0

P−C0P in g−c0 points.

The conjugate of each of these points is the cameracentre C1

Q in the conjugate configuration. This meansthat the conjugate curve is of type (∗, ∗; ∗, g − c0).Continuing with this argument leads to the followingresult:

Lemma 8.32. Given a curve of type (g, g; c0, c1) ona non-degenerate quadric SP, the conjugate curve onS10Q (also a non-degenerate quadric) is of type (g, g +

g − c0 − c1; g − c1, g − c0).This result can be extended also to the case of a

cone (degenerate conic of rank 3), in the case wheretwo camera centres are on different generators. In thiscase, we define the type (g, g; c0, c1) of a curve in thesame way, except that g equals the number of times thecurve crosses a generator g0

P less the number of timesit passes through the vertex; and g is the number oftimes the curve crosses the generator g0

P. As before,the degree of the curve is equal to g + g. In this case,the conjugate quadric S10

Q is a non-degenerate quadric,and the two camera centres lie on the same generator(according to Table 1).

Now, this lemma allows us to enumerate the differ-ent critical curves and their conjugates. Consider threequadrics Si j

P intersecting in a twisted cubic. Since thereare three camera centres, either at least two lie off thecubic, or at least two lie on the cubic. We choose twoof the views, where both cameras lie on or both off the

Critical Configurations for Projective Reconstruction from Multiple Views 35

cubic. Denote the critical quadric for that pair by SP.We enumerate the various possible types of curve.

Twisted cubics and their conjugates. First, we list thedifferent classes of twisted cubic curves. Such a curvemust have bidegree (2, 1), or (1, 2), so the followingpossibilities exist.

1. Type (1, 2; 0, 0) → (1, 3; 1, 1). (This notationmeans that the conjugate of a curve of type(1, 2; 0, 0) is of type (1, 3; 1, 1). The conjugate ofa twisted cubic not containing the camera centresis a bidegree (1, 3) quartic curve, containing thetwo camera centres. This is the type of ambiguitydiscussed in Section 8.3.

2. Type (1, 2; 1, 1) → (1, 1; 0, 0). In this case thetwisted cubic containing the two camera centres isconjugate to a conic (bidegree (1, 1) curve) not con-taining the camera centres. This ambiguity exists.From a point on the twisted cubic, the cubic curvelooks the same as a conic viewed from a point offthe conic.

3. Type (2, 1; 0, 0) → (2, 3; 2, 2). The conjugate ofa twisted cubic is a quintic curve passing twicethrough each of the two camera centres. This am-biguity does not extend to three views. This is be-cause the quintic can not be the intersection of twoquadrics, and it does not occur as the curve (quar-tic) when all the quadrics Si j

P are the same (seeSection 8.1). The two camera centres are the dou-ble points of the curve, but in general there are notthree such double points (see Fig. 13), which givesanother way of seeing that the ambiguity can notextend to a third camera.

4. Type (2, 1; 1, 1) → (2, 1; 1, 1). Both the curve andits conjugate are twisted cubics containing the cam-era centres. Since any two twisted cubics are pro-jectively equivalent, this is a trivial critical set. Thedifference in the two configurations is simply dueto the resection ambiguity (ambiguity of the cameracentre) for points on a twisted cubic.

Conics and their conjugates. Now, we consider conics.A conic must have bidegree (1, 1), and we obtain thefollowing list of conic curves and their conjugates.

1. Type (1, 1; 0, 0) → (1, 2; 1, 1). This is the sameconic / twisted cubic ambiguity discussed above.

2. Type (1, 1; 1, 1) → (1, 0; 0, 0). A conic (bidegree(1, 1)) seen from points on the conic looks like aline (bidegree (1, 0)) from point off the conic.

A similar analysis may be applied to other triplequadric intersection curves, such as one to three lines,or a conic plus intersecting line. Finally, an ellipticquartic curve containing the two camera centres (seeSection 8.2) has type (2, 2; 1, 1), and this must corre-spond to a curve of the same type (2, 2; 1, 1) by Lemma8.32.

Some examples of critical curves and their conju-gates are shown in Fig. 13.

8.5. The 7 Point Critical Set

Generically, three quadrics will intersect in 8 isolatedpoints and provided there is a conjugate configuration,at least 7 of these points will be critical according toTheorem 6.17. It was shown in Maybank and Shashua(1998) that indeed the 8th point (that is, the excep-tional point identified in Theorem 6.17 does not havea conjugate.

We will go one step further and ask when is a config-uration of 7 points and 3 cameras critical? In order forsuch a configuration to be critical, the pairwise viewsmust critical. As 9 points (generically) determine aquadric uniquely, it is necessary that the 7 points andeach combination of two camera centres form a ruledquadric. This is however not sufficient. We have the fol-lowing geometrical (but somewhat complicated) char-acterization.

Theorem 8.33. Let P j , j = 0, . . . , 6 denote 7 pointsand let P0, P1, P2 denote three cameras.

• Suppose that the 7 points and each combination oftwo camera centres lie on a ruled quadric (of rank3 or 4),

• Suppose there exist 3 permissible generator pairs(g01, g10), (g02, g20) and (g12, g21) where each gi j

goes through the camera centre of Pi and gi j �= gik .

Then, the configuration is critical if and only if thereexists a point (distinct from all P j ) lying on the inter-section of

(i) the three planes formed by the generator pairs(g01,g02), (g10,g12) and (g20, g21), and

(ii) the three ruled quadrics.

36 Hartley and Kahl

Figure 13. Sample curves on the quadric and their conjugate curves. Each row gives three conjugate curves on a non-degenerate quadric. Thediagram represents the quadric XY = Z T projected onto the XY -plane. The full range (−∞,∞) in each axial direction is compressed to theinterval (−2, 2) so that the whole plane can be seen. Curves continue from top to bottom, or left to right across the plane at infinity.

Critical Configurations for Projective Reconstruction from Multiple Views 37

Proof: Let X be an intersection point of the threeplanes as defined above. Note that three planes al-ways meet in at least one point, so it is clear thatsuch a point exists. The point can be expressed asX = C0

P + α01X01 + α02X02 for some non-zero scalarsα01 and α02, where X01 and X02 are some points on thegenerators g01 and g02, respectively. Similarly, X canbe expressed as linear combinations of points on theother two planes.

Now, suppose the configuration is critical and hencea conjugate configuration exists. Then P0X = P0(C0

P+α01X01 +α02X02) = α01e01

Q +α02e02Q , and similarly for

the other two cameras. This shows that the point Xsatisfies the exceptional condition in (7). Since gi j �=gik it follows that ei j

Q �= eikQ and hence the conjugate

camera centres are non-collinear. Now, Lemma 6.16can be applied which shows that condition (ii) aboveis indeed valid.

Conversely, suppose there exists a point X fulfillingboth (i) and (ii) above. We need to show that thereexists a compatible conjugate configuration, and thencriticality follows from Theorem 6.15. Since X lies onthe intersection of the quadrics, we get (cf. proof ofLemma 6.16)

0 = X�Si jP X = X�(

Pi�Fi jQ P

j)X

= (αi j e

i jQ + αikeik

Q

)�Fi jQ

(α j i e

j iQ + α jke jk

Q

)

= αikα jkeik�Q Fi j

Q e jkQ ,

for suitable combinations of i, j, k, which proves theconjugate configuration is compatible. �

8.5.1. Example of 7-Point, 3-View Ambiguity. Wedefine camera matrices by

P0 =

2 0 0 −1

0 3 0 −1

0 0 6 −1

,

P1 =

−2 0 0 −1

0 4 0 −1

0 0 −8 −1

,

P2 =

3 0 0 −1

0 2 0 −1

0 0 6 −1

and

Q0 =

−2 0 0 5

0 1 0 5

0 0 10 5

,

Q1 =

−4 0 0 3

0 −2 0 3

0 0 −6 3

,

Q2 =

3 0 0 5

0 −1 0 5

0 0 15 5

.

One may verify that the critical quadrics are

S10P =

0 60 1274 91

60 0 −1092 −273

1274 −1092 0 −60

91 −273 −60 0

,

S20P =

0 1 −60 10

1 0 60 −10

−60 60 0 −1

10 −10 −1 0

,

and

S21P =

0 −75 357 63

−75 0 −406 −14

357 −406 0 75

63 −14 75 0

.

The three critical quadrics intersect in eight points, thecolumns of the matrix below:

P j =

0 0 0 1 1 −1 867008 −2427052

0 0 1 0 1 −1 715616 −2003254

0 1 0 0 1 1 177859 2331851

1 0 0 0 1 1 744832 9765248

.

The intersection points Q j of the corresponding S10Q ,

S20Q and S21

Q are:

Q j =

0 0 0 1 1 −1 −69920 −40183934915

0 0 1 0 1 1 434720 52210262660

0 1 0 0 1 1 −33649 53953141148

1 0 0 0 1 −1 113344 −87313676262

38 Hartley and Kahl

The first 7 of the points P j are critical and the corre-sponding first 7 points of Q j are the conjugate points.One may verify that Pi P j = Qi Q j for i = 0, 1, 2 andj = 0, . . . , 6, but the last point P j has no conjugate.

8.6. Summary of 3-View Critical Sets

Theorem 6.14 gave the necessary conditions for a con-figuration of points and 3 cameras to constitute a crit-ical configuration. In the past sections we have shownthat essentially all possible intersections of 3 quadricsindeed do occur as critical configurations. We have notbeen completely exhaustive in our analysis, since enu-merating and giving examples of critical sets consist-ing of intersections of three quadrics becomes tedious.However, most possibilities have been considered ex-plicitly, as enumerated below. The only cases for whichwe have not given explicit examples are the degeneratecases consisting of one to three lines or a conic plusintersecting lines. These may be considered as degen-erate forms of twisted cubics or conics.

• In the case where all the quadrics Si jP are the same

hyperboloid or cone, the whole of the quadric doesnot constitute a critical set. Nevertheless, we showedthat there exists a rational quartic curve lying on thequadric Si j

P , which is critical.• In the case where all the quadrics consist of the same

intersection of two planes, the critical set consists ofone of the planes, plus a conic curve in the otherplane, the camera centres lying on this conic.

• The case where the three quadrics all intersect in anelliptic quartic constitutes a critical set. The ellip-tic quartic must contain all points and the cameracentres; then it is critical. This result also holds fordegenerate forms of intersections of two quadrics,such as a twisted cubic plus a line, two conics or thevarious configurations of lines.

• If the intersection of all three quadrics is smallerthan the intersection of any two of them, then thisintersection consists either of eight isolated points(some maybe complex), or a curve (conic, a twistedcubic, or some lines) plus some possible isolatedpoints.The case of points on a twisted cubic appears asthe conjugate configuration to the rational quarticcritical set. The points lie on a twisted cubic, andthe cameras on an arbitrary straight line. All suchtwisted cubic and line pairs are possible. This istherefore different from the twisted cubic plus line

that arise as the intersection of two quadrics, sincein this case they line and cubic must meet in twopoints.A twisted cubic containing the camera centres hasas a conjugate a conic not containing the cameracentres.A conic containing the camera centres has as a con-jugate a line not containing the camera centres.

• Finally, in the cases where the intersection containsisolated points, it is typical that one of the intersec-tion points is not critical.

The net result of this investigation, stated somewhatinformally is that essentially all possible intersectionsof three quadrics do form a critical point set for 3-view reconstruction. In the next (and final) section ofour paper, it will be shown that in most cases thisconclusion extends also to N-view reconstruction.

9. Multi-View Ambiguity

The main part of this paper has been concerned withambiguities in three views. We conclude the main tech-nical exposition by showing briefly that three-viewcritical configurations can often be extended to anynumber of views. Although we have seen examples ofmulti-view ambiguities, for instance, in the case of ra-tional quartics curve (see Section 8.3), it is not clearwhat happens in general. By adding a fourth camerato a critical three-view configuration, one might sus-pect that the criticality would disappear. However, thisis often not the case. The goal of this section is thefollowing theorem.

Theorem 9.34. Given n ≥ 3 cameras, Pi , i =0, . . . , n − 1 and m points, P j , j = 0, . . . , m − 1,then the configuration is critical if and only if eachsubset of three cameras and all points are critical.

Proof: If the whole configuration is critical, theneach subset of three cameras must also be critical. Nowsuppose each subset of three cameras is critical.

For simplicity of notation, we consider the 4-viewcase. The result for n views follows by induction. Thethree cameras P0,P1,P2 along with the points form acritical configuration and hence a conjugate configura-tion exists. Similarly a conjugate configuration existsfor the points along with the three cameras P0,P1,P3.The goal is to show that these two conjugate configu-rations are compatible. This will be done by showing

Critical Configurations for Projective Reconstruction from Multiple Views 39

that the conjugate cameras Q0 and Q1 may be chosen tobe the same in each of the two configurations derivedfrom the two camera triples.

Consider the way that the cameras Q0 and Q1 areconstructed. Let Pi ; i = 0, . . . , 2 be the first triple ofcameras. The first two cameras and the points P j lie(without loss of generality) on some ruled quadric S.The fundamental matrix F10

Q is chosen so that S =P1�F10

Q P0. From this value of F10

Q two camera matricesQ0 and Q1 may be computed. The choice of the pair(Q0,Q1) is unique up to a projectivity.

We proceed in the same way with the second tripleof cameras P0,P1,P3. Since the camera P2 or P3 doesnot take part in this construction, the resulting funda-mental matrix F10

Q is the same and from it one mayextract two camera matrices, which will be denoted(Q′0,Q′1), which must be projectively equivalent to thepair (Q0,Q1). However (and this is the main point) aconjugate configuration is defined only up to projectiv-ity. Consequently, it is possible to choose the camerapair (Q′0,Q′1) to be identical to (Q0,Q1). In this way,from the two conjugate configurations correspondingto the two triples of cameras with indices (0, 1, 2) and(0, 1, 3), we get four camera matrices Qi ; i = 0, . . . , 3.

So far we have not mentioned the points. Now letQ j be points derived from the (0, 1, 2)-configurationand let Q′

j be the points derived from the (0, 1, 3)-configuration. By the definition of criticality,

Pi P j = Qi Q j for i = 0, 1, 2

and

Pi P j = Qi Q′j for i = 0, 1, 3

Considering only the first two cameras here, we seethat

Qi Q j = Qi Q′j for i = 0, 1.

Since the image of a point in two views is suffi-cient to determine its position (by triangulation), itfollows that Q j = Q′

j . Consequently Pi P j = Qi Q j

for i = 0, . . . , 3 and so {Qi , Q j } forms a conjugateconfiguration to {Pi , P j }, which therefore is a criticalconfiguration. �

The conclusion is that essentially nothing new hap-pens for arbitrary number of views. This means, forexample, that an arbitrary number of points and an ar-bitrary number of cameras on a rational or an elliptic

quartic form a critical configuration. This is becausewe have shown previously that three cameras and anynumber of points in these configurations form a criticalset, and the theorem allows us to extend this result toany number of cameras.Duality. One notable topic that has not been consid-ered in this paper is duality, in the sense of Carlsson,see Carlsson (1995). It has been shown in Hartley andDebunne (1998), Hartley and Zisserman (2003) that then-view, 6-point problem considered in Maybank andShashua (1998) is dual to the 2-view, n +4-point prob-lem, and hence a critical configuration occurs when allpoints and cameras lie on a single ruled quadric. Wehave not considered such configurations with minimalnumbers of points in this paper. The important classesof critical configurations of this paper, such as the el-liptic quartic and rational quartic ambiguities appear tobe self-dual.

Summary. The critical configurations derived in thispaper for a set of at least three cameras and at least7 points are summarized in Table 5. The first resulton elliptic quartics can be found in Section 8.2.1, thesecond on rational quartics is given in Section 8.3,third and fourth results are presented in Section 8.4and Section 8.1.1, respectively, and finally details onthe 7 point ambiguity can be found in Section 8.5.

10. Conclusions

In the study of critical configurations of multiple views,in particular, the extension from two to three views,much has been learnt. At first, it looked like a trivialexercise, since one only needs to consider pairwiseviews except for a special conditions. However, nowthat the full picture has been revealed, it has turnedout to be a problem with rich structure and beatifulgeometrical constructions.

In the two-view case, we have shown exactly whichruled quadrics may be critical, thereby correcting aclassical result. In three views and more, it has beenshown that the main critical classes are the elliptic andthe rational quartic curves, and additionally,lower de-gree quadric intersections. This is true for any numberof points and any number of cameras lying on such acurve.

After having proved all the theorems and lemmas,some words on practical implications. It is clear thatthe multi-view configurations are not likely to occur inpractise. Still, configurations close to critical ones will

40 Hartley and Kahl

Table 5. Summary of multi-view critical sets. The way to read this table is that any one of the point/line configurations in the right twocolumns, or left two columns is critical. The conjugate configuration is of the other type in the same line of the table.

First configuration Second configuration

Points Cameras Points Cameras

Elliptic quartic Same Elliptic quartic Same

Rational quartic Same Twisted cubic Line

Intersection of three quadrics (twisted cubic,conic, conic + line or 1–3 lines) of type(g, g; c0, c1) on critical quadric

Various Curve of type (g, g + g − c0 − c1;g − c1, g − c0)

Various

Plane + conic curve in a different plane In same plane as conic Plane + conic curve In same plane as conic

7 isolated points 3 arbitrary cameras 7 isolated points 3 arbitrary cameras

not be stable in the sense that small noise perturbationsin the data may change the reconstruction drastically.The typical behaviour is that small amounts of noisecan cause a reconstruction algorithm to land in anyof the possible conjugate configurations. In the caseswhere an infinite family of conjugate configurationsexist (the elliptic quartic case for instance) reconstruc-tion will be unstable, with the computed reconstructionvarying among the infinite family of different conju-gate configurations.

Finally, from a practical perspective, it is good toknow that the critical configurations are rare and henceone can easily design camera set-ups avoiding suchconfigurations.

Appendix A: Quadric Intersection

We investigate here the intersections of two and threeruled quadrics, starting with two quadrics. Note thatonly real intersection points are of interest. We alsoexamine pencils of quadrics.

The results derived here are known. Nevertheless,we include a summary and derivation using elemen-tary algebra so as to make the material easily acces-sible. Most books consider the theory of intersectionsin P3(C), whereas we are particularly interested inreal intersections (in P3(R)). The approach to findingintersections of quadrics considered below has the ad-ditional advantage that it gives a method for actuallycomputing the intersection of two given quadrics, orconversely finding examples of two quadrics that in-tersect in a given intersection curve.

A.1. Intersections of Two Quadrics

We compute the possible intersections of two quadrics.Our assumption is that at least one of the quadrics is a

non-degenerate ruled quadric. We may assume there-fore that the first quadric is represented by the equationZT = XY, since all non-degenerate ruled quadrics areequivalent. The two quadric may then assumed to berepresented by the matrices

S0 =

0 1 0 0

1 0 0 0

0 0 0 −1

0 0 −1 0

S1 =

2a b c d

b 2e f g

c f 2h j

d g j 2k

. (14)

In addition, it may be assumed that b = 0, since thismay otherwise be achieved by adding a multiple of S0

without changing the intersection of the two quadrics.We dehomogenize by setting T = 1. Now, let X =(X, Y, Z, 1)� be a 3D point on the intersection of thetwo quadrics, expand out the expression X�S1X = 0and substitute Z = XY to eliminate Z. This results in anequation

aX2 + cX2Y + dX + eY2 + f XY2 + gY +hX2Y2 + jXY + k = 0, (15)

which is an arbitrary fourth-degree equation in X andY with no term involving X or Y to powers greaterthan 2. This equation will be written as p(X, Y) = 0.The intersection of the two quadrics may then be foundby computing the solution set V(p) of this polynomialto get the X and Y coordinates of an intersection point.The Z coordinate is then given by Z = XY.

Critical Configurations for Projective Reconstruction from Multiple Views 41

Figure 14. Intersections in the (x, y)-plane for different factors.See text for details.

Let us consider the form of the polynomial p(X, Y)and its possible irreducible factors. See Fig. 14 forillustrations in the (x, y)-plane.

Factors of degree 1 in X and 0 in Y (or vice versa).A factor of the form X − a or Y − a where a is realrepresents a straight line in (x, y)-space, Fig. 14(a).However, the lines with constant X or Y coordinate arethe generators (straight lines) on the quadric Z = XY.This means that the factors of p(X, Y) of this formrepresent straight-line components of the intersectionof the two quadrics. More specifically, a factor X −a corresponds to the straight line consisting of points(a, Y, aY, 1)�. A factor of the form X = a for a complexmust be matched by a conjugate factor X = a. Neitherof these factors corresponds to any real intersectionpoints in the (x, y)-plane, or in 3D.

Factors of degree 1 in both X and Y. Solving aXY+bX + cY + d = 0 for Y gives Y = −(bX + d)/(aX +c); see Fig. 14(b). In the figure, the asymptotes aredrawn with dotted lines. Consequently, Z = −X(bX +d)/(aX + c). A general point on the curve then may bewritten (in homogeneous coordinates) as

(X,−(bX + d)/(aX + c),−X(bX + d)/(aX + c), 1)�

= (X(aX + c),−(bX + d),−X(bX + d), a(X + c))�.

This is the form of a rational curve, parametrized byX and containing only second powers of the parameterand hence it represents a planar conic.

Factors of degree 2 in X and degree 0 in Y. Anirreducible quadratic factor containing only terms in X

(such as aX2 + bX + c) factors over the complex field

into two complex factors. These do not correspond toany real intersection points.

Factors of degree 2 in X and degree 1 in Y. Solvingfor Y gives Y = a(X)/b(X) where both a(X) and b(X)are quadratic in X, see Fig. 14(c). Then Z = XY =Xa(X)/b(X) and the 3D curve is

(X, a(X)/b(X), Xa(X)/b(X), 1)�

= (Xb(X), a(X), Xa(X), b(X))�

which is a rational curve of degree three—a twistedcubic.

Factors of degree 2 in both X and Y. Such a factoris not a proper factor of p(X, Y), but must be the wholeof p(X, Y) itself. The curve corresponding to such irre-ducible polynomial is called an elliptic quartic. Unlikethe other types of factors given above, an elliptic quar-tic may decompose into two real curves, or else containisolated real points.

It is now possible to enumerate all the possible waysin which two quadrics may meet. The following listgives all possible ways in which two quadrics canintersect.7

1. An elliptic quartic curve, see Fig. 15(a) (or seeFig. 11 for random examples).

2. A twisted cubic and a line, see Fig. 15(b). Thetwisted cubic and the line meet in two points (pos-sibly complex or multiple intersection). The inter-section in the (x, y)-plane is plotted in Fig. 14(d).

3. Two conics, meeting in two points (possibly com-plex or multiple intersection), see Fig. 15(c) for agraph and Fig. 14(e) for the (x, y)-plane intersec-tion. Note that asymptotes are drawn with dottedlines.

4. A conic plus two coincident lines, which also meetthe conic (one from each class of generators on oneof the quadrics) Fig. 15(d). This may be thought ofas the intersection of two conics, one of which isdegenerate.

5. Four lines, two from each class of generators on oneof the quadrics Fig. 15(e).

6. Two lines, both from the same class of generators onone of the quadrics Fig. 15(f). (The other irreduciblefactor contains only complex points).

42 Hartley and Kahl

Figure 15. Possible intersections of two quadrics in 3D-space.

The only one of these configurations that can containisolated real points is the quartic curve, but they are notisolated in the complex projective space.

Finding examples of quadric intersections. It iseasy, given possible intersection of two quadrics toconstruct a specific example of two quadrics that havethat intersection. As usual let one of the quadrics be thestandard quadric Z = XY. Let the intersection be givenby a curve of the form (15). The correspondence of thecoefficients of (15) and the quadric S1 in (14) allowsan immediate construction of the required quadric S1.Alternatively, one can arrive at the equation for thedesired curve by reversing the substitution Z = XY, thatis replacing X 2Y 2 by Z 2, or X 2Y by XZ and so on. Ifthe resulting quadric is singular, then adding a multipleof the standard quadric Z = XY may result in a non-singular quadric, without changing the intersection. Asan example we find the quadric that meets Z = XY

in four lines, given by equation (X − 1)(X + 1)(Y −1)(Y + 1) = 0. Multiplying out gives the form X2Y2 −X2 − Y2 + 1 = 0. Now, replacing X2Y2 by Z2 givesX2 + Y2 = Z2 + 1, the equation of the curve S1. Thetwo quadrics Z = XY and X2 + Y2 = Z2 + 1 meet inthe desired four straight lines.

A.2 Intersection of Three Quadrics

We consider the intersection of three quadrics. Thisintersection may be computed by intersecting an extraquadric with the intersection of two quadrics. Considerthe intersection of each of the components of the inter-section of the first two quadrics with a third quadric.The following result shows how each irreducible com-ponent can intersect with the third quadric.

Theorem A.35. If c is an irreducible componentof the intersection of two quadrics, and S is a third

quadric. Then either c intersects S in a finite numberof points, or the whole of c lies on S.

Proof: Let c be the intersection of the standardquadric Z = XY with a further quadric and let p(X,Y ) be the curve that defines c. By assumption, p isan irreducible polynomial. Let q(X, Y ) be the curveof intersection of the third quadric with Z = XY. Theintersection of the two curves p(X, Y ) and q(X, Y ) isthe intersection of the three quadrics. By Bezout’s the-orem, this consists of a finite number of points unlessp(X, Y) and q(X, Y) have a common factor. Since p(X,Y) is irreducible, this can only occur if p(X, Y) dividesq(X, Y ). In this latter case, the curve p(X, Y ) = 0 is asubset of q(X, Y )= 0 and the three quadrics intersectin c. �

The possible intersections of three quadrics can noweasily be enumerated. One starts with the possible in-tersections of two quadrics, and intersects each irre-ducible component with the third quadric. The intersec-tion of each irreducible component curve must eitherbe the whole of the curve (quartic, twisted cubic, conicor line), or a finite number of points. The reader maycompile a complete list of possibilities if interested.

Appendix B: Rational and Elliptic Quartics

We investigate the relationship between rational andelliptic quartics.

Some quartics formed as the intersection of tworuled quadrics are rational. It was shown above thattwisted cubics and conics may arise from the intersec-tion of two quadrics and such curves are rational. Now,we observe that there are also rational quartic curvesthat arise from the intersection of two quadrics.

Some elliptic quartics are rational. Let the firstquadric be the usual Z = XY and the intersectioncurve be X2 + Y2 = 1, or in homogeneous coordinatesX2 + Y2 − T2 = 0. This curve is a circle, which maybe parametrized as (X, Y, T)� = (θ2 − 1, 2θ, θ2 + 1)�.Now, letting Z = XY /T leads to Z = 2θ (θ2−1)/(θ2+1).The complete 3D curve is given by

(X, Y, Z, T)�

= (θ2 − 1, 2θ, 2θ (θ2 − 1)/(θ2 + 1), θ2 + 1)�

= (θ4 − 1, 2θ (θ2 + 1), 2θ (θ2 − 1), (θ2 + 1)2)�.

Critical Configurations for Projective Reconstruction from Multiple Views 43

This is a rational quartic curve. Evidently the sameargument will work whenever the intersection curvep(X, Y) in (x, y)-space represents a conic.

Not all elliptic quartics are rational. Not all quarticcurves are rational,since in order to be parametrizable,the real part of the curve must be connected. However,not all elliptic quartics are connected – see Fig. 11.

Not all rational quartics are elliptic. An ellipticquartic arises from the intersection of two quadrics, butis it true that any rational quartic lies on two quadrics aswell? We have seen previously that some rational quar-tics have this property. Actually, there are two types ofrational quartics.

Theorem B.36. A rational quartic curve lies on atleast one quadric surface (a supporting quadric). Thereare two classes of rational quartics:

1. Those that meet all generators of a supportingquadric in one class in a single point and all gener-ators of the other class in three points. Quartics inthis class lie on a single quadric surface and henceare not elliptic quartics.

2. Those that meet both classes of generators of a sup-porting quadric in two points. Quartics in this classconstitute the intersection of a pencil of quadricsand consequently are elliptic quartics.

The rational quadrics of the first type is said to havebidegree (3, 1) and the second type bidegree (2, 2).

Proof: We first prove that all rational quartic curveslie on a quadric. Let A�(4) be the curve. The con-dition for the curve to lie on a quadric S is that�(4)�A�SA�(4) = 0 for all θ . The matrix S is a sym-metric matrix with 10 independent entries. The matrixA�SA has (i j)-th entry a�

i Sa j where ai is the i-th col-umn of A. Consequently,

�(4)�A�SA�(4) = 0 =5∑

i, j=1

ai�Sa jθ

i+ j−2

=8∑

k=0

∑

i+ j=k+2

ai�Sa jθ

k .

Since this must be zero for all values of θ , the co-efficient of each power of θ must vanish, so for all

k = 0, . . . , 8, we have

∑

i+ j=k+2

ai�Sa j = 0.

These equations are linear in the entries of the sym-metric matrix S. There are 9 equations in the 10 in-dependent entries of S and hence a non-zero solutionexists.

A rational quartic meets any plane in four points (in-cluding complex values of the parameter and multipleintersections). Now, consider a rational quartic Q lyingon a quadric S and consider a plane tangent to S at apoint not lying on the curve Q. This plane intersects Qin a pair of generators and any intersection of Q withthe plane must lie along the pair of generators. Thereare two possibilities: Q meets both generators in twopoints or Q meets one generator in one point and theother in three points – the curve is defined as havingbidegree (2, 2) or (3, 1) according to which of thesecases holds.

Now, let 1 be one of these generators, meeting Q inn = 1, 2 or 3 points. Let 1′ be any generator from theopposite class to 1, hence meeting 1 in some point X.The plane containing 1 and 1′ meets Q in four pointsand hence Q meets 1′ in 4 − n points. This shows thatthe curve meets all generators of one class in the samenumber of points (n) and the generators of the otherclass in 4 − n points.

The remaining claims follow from Proposition B.37and Proposition B.38 below. �

Proposition B.37. An elliptic quartic meets any gen-erator of a supporting quadric in two points.

Proof: Assume that S and S′ are two quadrics con-taining the quartic curve. Since an arbitray space linemeets a quadric in two points, a generator of S meetsS′ (and hence the quartic curve formed by the inter-section of S and S′) in two points. In other words, anygenerator of S meets the quartic curve in two pointsand the same is true of any generator of S′. �

So, rational quartics lying on two quadrics (andhence elliptic) have bidegree (2, 2). Finally, by show-ing that any bidegree (2, 2) is elliptic, one can concludethat bidegree (3, 1) rational curves must lie on a singlequadric.

Proposition B.38. Any rational quartic of bidegree(2, 2) is elliptic.

44 Hartley and Kahl

Proof: Without loss of generality, one may as-sume that the quartic lies on the quadric ZT = XY.The generators of this quartic are the lines of con-stant X or Y coordinate. A rational quartic curve(X(θ ), Y(θ ), Z(θ ), T(θ ))� lying on the quadric ZT = XY

must be of the form

(x(θ )u(θ ), y(θ )v(θ ), x(θ )y(θ ), u(θ )v(θ ))�

and if this is to have bidegree (2, 2), then each of x, y,u and v is (at most) a quadratic polynomial.

Now, for quadratic polynomials a, b, c and d, define

H (a, b, c, d) = det

a0 a1 a2 0

0 b0 b1 b2

c0 c1 c2 0

0 d0 d1 d2

.

We define the matrix

G =

H (y, v, v, y) H (y, v, x, u) −H (y, v, v, u) −H (y, v, x, y)

H (u, x, v, y) H (u, x, x, u) −H (u, x, v, u) −H (u, x, x, y)

−H (u, v, v, y) −H (u, v, x, u) H (u, v, v, u) H (u, v, x, y)

−H (y, x, v, y) −H (y, x, x, u) H (y, x, v, u) H (y, x, x, y)

Then

(X, Y, Z, T)G(X, Y, Z, T)�

= (xu, yv, xy, uv)G(xu, yv, xy, uv)� = 0.

It is straightforward to check that the quadric G isdifferent from the quadric ZT = XY also containing thecurve. �

For instance, applying this method to the quartic(θ4 − 1, 2θ (θ2 + 1), 2θ (θ2 − 1), (θ2 + 1)2)� givenabove (“Some elliptic quartics are rational”), yieldsthe quadric X2 + Y2 = T2. The curve is the intersectionof this quadric with XY = ZT .

B.1. Group Structure on an Elliptic Quartic

It is well known that a planar cubic curve has a groupstructure, in which the elements of the group are simplythe points on the curve. A commutative group operationis defined for pairs of points on the curve.

In a similar way, it results from Theorem 8.25 that agroup structure may be defined for points on an ellip-tic quartic. The group multiplication may be defined

directly. We assume that the elliptic quartic is definedby a pencil that does not contain any quadrics of rankless than 3, and that all points on the quartic are regularpoints.

We select a random point on the curve to be theidentity of the group, and denote it by 0. Now, forpoints A and B on the quartic curve, we find the quadricin the pencil that contains the line AB. On this quadric,there is a generator of the same class as the line AB,passing through the point 0. This line is the line quasi-parallel to AB, and it meets the curve in a further pointC such that AB ‖ 0C. We define a commutative groupoperation + on points of the curve by letting A + B bethe point C defined in this way.

It is easily seen that this operation is commutative,the product is always defined, and A + 0 = A for allA. Inverses are constructed as follows. Let S0 be thequadric that contains the line tangent to the curve at the

point 0. Choose the line quasi-parallel to this throughA. Its other intersection with the curve is the point −A,as may be verified from the definition of the groupoperation.

Finally, the associativity of the group operation +is nothing other than Pascal’s Theorem for the ellip-tic quartic. To see this, note that it follows from thedefinition of + that A + B = A′ + B ′ if and only ifAB ‖ A′ B ′, where A, B, A′ and B′ are any four pointson the curve. With this observation we see that Pascal’sTheorem may be written as

A + B = A′ + B ′ and B + C = B ′ + C ′

implies A′ + C = A + C ′

(see the statement of Theorem 8.25). Now, specializingto the case where B ′ = 0, and substituting A′ = A+ Band C ′ = B + C in the expression A′ + C = A + C ′

yields (A + B) + C = A + (B + C) as required.

Critical Configurations for Projective Reconstruction from Multiple Views 45

Appendix C: Practical Implications

From a practical point of view, it is not likely thatall points and two cameras centres happen to lie on aruled quadric. Hence, one might draw the conclusionthat critical sets have no effect in real life. On thecontrary, for configurations “close” to critical ones, itturns out that the solution is not stable in the sensethat small perturbations of the data may change thereconstruction drastically.

It is beyond the scope of the present paper to pursuethis topic in depth, but we will give an example of thedescribed situation.

Example of Two-View Ambiguity. Consider the fol-lowing camera matrices:

P0 =

1 0 0 0

0 1 0 0

0 0 1 0

, P1 =

1 0 0 −1

0 1 0 −1

0 0 1 −1

and

Q0 =

1 −2 1 0

1 0 3 0

1 0 −1 0

, Q1 =

2 −4 2 0

1 −3 3 −1

3 −3 1 −1

.

One may verify that the camera centres for both pairsare given by C0

P = (0, 0, 0, 1)� and C1P = (1, 1, 1, 1)�,

and that the critical quadrics SP and SQ are both equalto Z = XY . We also know that there is a third possibleconfiguration.

Given image points, perturbed by random noise, onemay ask which is the most likely solution to be recov-ered from a standard structure and motion algorithm?In order to examine this, simulated image points weregenerated in the following manner:

1. A set of 20 points with X- and Y-coordinates wererandomly sampled from a uniform distribution of[−10, 10]. The Z-coordinate was set according to Z

= XY .2. To each point coordinate, X, Y and Z, independent

and zero-mean Gaussian noise was added with astandard deviation of N3D . This has the effect ofmoving the points slightly off the critical surface.

3. The 3D-points were projected to two images us-ing (P0,P1) and then again, the image coordinateswere perturbed by independent zero-mean Gaus-sian noise with standard deviation N2D = 0.01. Theamount of noise equals approximately 1 percent ofthe value of the image coordinates.

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

N3D

–standard deviation of distance from Z=XY

Rel

ativ

e fr

eque

ncy

True solutionConjugate configuration 1Conjugate configuration 2

Figure 16. The graph shows which solution (the true or one of the conjugate solutions) is recovered over 1000 trials when the configurationis critical, N3D = 0, and when the configuration is further and further away from a critical set, that is, as N3D increases.

46 Hartley and Kahl

Then, for varying values of N3D , using the lineareight-point algorithm followed by bundle adjustment(Hartley and Zisserman, 2004), estimates of the cam-eras and the 3D-points were computed. Bundle adjust-ment optimizes the reprojection errors in the images.In order to check which of the solutions was closest,we used the Frobenius distance between the estimatedmatrix Fe and the three possible fundamental matrices,namely ||Fe ± Fi ||, where each matrix was scaled tohave unit norm. The average result of 1000 trials forvarying N3D is graphed in Fig. 16. In about 10 per-cent of the cases (independent of N3D), the recoveredfundamental matrix was not close to any of the threesolutions and they have been discarded. In these cases,the solution probably represented a local minimum.

As can be seen in Fig. 16, for a true critical config-uration corresponding to N3D = 0, the three possiblesolutions are about equally likely to occur. As N3D in-creases, the 20 point-configuration moves away from acritical set and the conjugate solutions are less likely tobe recovered. Still, for instance with N3D = 1, about 5percent of the solutions end up in one of the conjugateconfigurations. Note that only local minima are recov-ered with bundle adjustment, and the global minimummay be different.

Acknowledgements

We wish to acknowledge Charles Thas (Ghent Univer-sity, Belgium) and Rey Casse (University of Adelaide,Australia) who provided proofs (each different) of The-orem 8.25. The proof given here is modified from thegeneral proof outline given by Rey Casse.

Notes

1. Not contained in the first edition of the book.2. To clarify a possible point of confusion: the two conjugate con-

figurations of a quadric and two camera centres are projectivelyequivalent, but the cameras are not the same, and the correspon-dence between between conjugate points on the two quadrics isnot a projective equivalence; see Fig. 6.

3. It may seem paradoxical that a cubic equation has one or twosolutions. In fact, the solution to the linear set of equations hasone free parameter λ (see the proof of Lemma 5.12), and thedeterminant condition collapses to a polynomial of degree at most2 in λ, therefore having 0, 1, 2 or ∞ solutions. It is interesting thatthe Fi j

Q computed in this way does not involve square (or cube)roots.

4. A chord is a line joining two points on the quartic curve. Thestatement in the text is true, except in the case where the ellipticquartic consists of two planar conics, and the chord CD joins

two points on the same conic. In this case, there exists no quasi-parallel chord, unless A lies on the same conic as the chordCD.

5. In the case where the curve consists of two conics, this will nothold if A, B and C are all on the same conic, since then all pointson the conic lie on the same plane.

6. We exclude from consideration the case where the pencil definingthe quartic contains a single plane (X2 = 0), or line quadric(X2 +Y2 = 0). If the pencil contains the rank-2 two-plane quadric(XY = 0), then the elliptic quartic consists of two conics. In thiscase, we require for this theorem that the three cameras all lie onthe same conic. The points Ai introduced in the proof must lie onthe other conic.

7. Some intersections may occur on the plane at infinity. The sumof the degrees of the intersection curves should always equalfour.

References

Astrom, K., and Kahl, F. 2003. Ambiguous configurations for the lDstructure and motion problem. Journal of Mathematical Imagingand Vision, 18(2):191–203.

Buchanan, T. 1988. The twisted cubic and camera calibration.Computer Vision, Graphics and Image Processing, 42:130–132.

Carlsson, S. 1995. Duality of reconstruction and positioning fromprojective views. In IEEE Workshop on Representation of VisualScenes, Cambridge Ma, USA, pp. 85–92.

Evelyn, C.J.A., Money-Coutts, G.B., and Tyrrell, J.A. 1974. TheSeven Circles Theorem and Other New Theorems. StaceyInternational: London.

Hartley, R. 2000. Ambiguous configurations for 3-view projectivereconstruction. In European Conf. Computer Vision, Dublin,Ireland, Vol. I, pp. 922–935.

Hartley, R., and Debunne, G. 1998. Dualizing scene reconstructionalgorithms. In 3D Structure from Multiple Images of Large-ScaleEnvironments, European Workshop, SMILE, Freiburg, Germany,pp. 14–31.

Hartley, R., and Kahl, F. 2003. A critical configuration forreconstruction from rectilinear motion. In Conf. Computer Visionand Pattern Recognition, Madison, USA, Vol. I, pp. 511–517.

Hartley, R.I. 1994. Projective reconstruction and invariants frommultiple images. IEEE Transactions on Pattern Analysis andMachine Intelligence, 16:1036–1041.

Hartley, R. I., and Zisserman, A. 2003. Multiple View Geometry inComputer Vision (2nd edn.), Cambridge University Press.

Kahl, F. 2001. Geometry and Critical Configurations of MultipleViews. PhD thesis, Sweden, Lund Institute of Technology.

Kahl, F., and Hartley, R. 2002. Critical curves and surfaces forEuclidean reconstruction. In European Conf. Computer Vision,Copenhagen, Denmark, Vol. II, pp. 447–462.

Kahl, F., Hartley, R., and Astrom, K. 2001. Critical configurationsfor N-view projective reconstruction. In Conf. Computer Visionand Pattern Recognition, Hawaii, USA, Vol. II, pp. 158–163.

Krames, J. 1940. Zur Ermittlung eines Objectes aus zwei Perspec-tiven (Ein Beitrag zur Theorie der gefahrlichen Orter). Monatsh.Math. Phys., 49:327–354.

Maybank, S. 1993. Theory of Reconstruction from Image Motion.Heidelberg: Berlin: Springer-Verlag: New York.

Critical Configurations for Projective Reconstruction from Multiple Views 47

Maybank, S. 1995. The critical line congruences for reconstruc-tion from three images. Applicable Algebra in Engineering,Communication and Computing, 6(2), pp. 89–113.

Maybank, S., and Shashua, A. 1998. Ambiguity in reconstructionfrom images of six points. In Int. Conf. Computer Vision,Mumbai, India, pp. 703–708.

Navab, N., and Faugeras, O. D. 1993. The critical sets of lines forcamera displacement estimation: A mixed Euclidean-projectiveand constructive approach. In Int. Conf. Computer Vision, pp.713–723.

Semple, J. G., and Kneebone, G. T. 1979. Algebraic ProjectiveGeometry. Oxford University Press.