Grassmann-Cayley algebra for modelling systems of camerasand the algebraic equations of the manifold of trifocal tensors*

Olivier Faugeras and Théodore Papadopoulo

INRIA2004 Route des Lucioles

B.P. 93, 06902 Sophia-Antipolis CedexFrance

{faugeras,papadop}@sophia.inria.fr

Abstract. After a brief introduction of lhe Grassmann-Cayley OI' double algebra we proceed to demon-strate its use for modelling systems of cameras. In the case of three cameras, we give a new interpretationof the trifocal tensors and study in detail some of the constraints that they satisfy. In particular we provethat simple subsets of those constraints characterize the trifocal tensors, in other words, we give thealgebraic equations of the manifold of trifocal tensors.

Resumo. Depois de uma introdução da álgebra de Grassman-Cayley ou álgebra dupla, nós mostramosa sua utilidade na representação de sistemas de câmeras. no caso de três câmeras, nós damos umanova interpretação dos tensores trifocais e estudamos em detalhe algumas das propriedades que elessatisfazem. Particularmente, nós demonstramos que um subconjunto dessas propriedades caracteriza ostensores trifocais, nos damos as equações algébricas do conjunto dos tensores trifocais.

1 Introduction

This article deals with the problem of representing thegeometry of several (up to three) pinhole cameras. Theidea that we put forward is that this can be done el-egantly and conveniently using the formalism of theGrassmann-Cayley algebra. This formalism has al-ready been presented to the Cornputer Vision comrnu-nity in a numbel' of publications such as, for example,[Carlsson (1994), Faugeras and Mourrain (l995a)] butno effort has yet been made to systematically exploreits use for representing the geometry of systems of carn-eras .

The thread that is followed here is to study therelations between the 3D world and its images ob-tained from one, two OI' three cameras as well as, whenpossible, the relations between those images with theidea of having an algebraic formalism that allows usto compute and estimate things while keeping the ge-ometric intuition which , we think , is important. TheGrassmann-Cayley OI' double algebra with its two op-erat?rs join and meet that correspond to the geometricoperations of summing and intersecting vector spacesOI' projective spaces was precisely invented to fill thisneed .

After a very brief introduction to the double al-gebra (more detailed contemporary discussions can befound for example in [Doubilet et ai. (1974)] and[Barnabei et ai. (1985)]), we apply the algebraic-

"This work was partially supported by lhe EEC under lhe reactiveLTR project 21914-CUMULI.

geometric tools to the description of one pinhole carn-era in order to introduce such notions as the optical cen-ter, the projection planes and the projection rays whichappear Iater. This introduction is particularly denseand only meant to make the papel' more OI' less self-contained.

We then move on lo lhe case of two camerasand give a simple account of the fundamental matrix[Longuet-Higgins (1981), Faugeras (1992), Carlsson(1994), Luong and Faugeras (1995)] which sheds somenew Iight on its structure.

The next case we study is the case of three carn-eras . We present a new way of deriving the trifocal ten-sors which appear in severaI places in lhe literature. lthas been shown originally by Shashua [Shashua (1994)]that the coordinates of three corresponding points inthree views satisfy a set of algebr aic relations of degree3 called the trilinear relations. It was later on pointedout by Hartley [Hartley (1994)] that those trilinear re-lations were in fact arising from a tensor that governedthe correspondences of lines between three views andwhich he called lhe trifocal tensor. Hartley also COf-

rectly pointed out that this tensor had been used, if notformally identified as such, by researchers working onthe problem of the estimation of motion and structurefrom line correspondences [Spetsakis and Aloimonos(l990b)] . Given three views, there exist three such ten-sors and we introduce them through the double algebra.

Each tensor seems to depend upon 26 parameters(27 up to scale), these 26 parameters are not indepen-dent since the number of degrees of freedom of three

I

18

views has been shown to be equal to 18 in the projectiveframework (33 parameters for the 3 perspective projec-tion matrices minus 15 for an unknown projective trans-formation) [Luong and Viéville (1994)] . Therefore thetrifoca l tensor can depend upon at most 18 independentparameters and its 27 components must satisfy a num-ber of algebraic constraint s, some of them have beenelucidated [Shashua and Werman (1995) , Avidan andShashua (1996)] . We have given a slightly more com-plete account of those constrains in [Faugeras and Pa-padopoulo (1998)], used them to parameterize the ten-sor s minimally (i.e. with 18 parameters) and to designan algorithm for their estirnation given line correspon-dences. In this paper we explore those constraints ingreat detail and prove that two particular simple subsetsare sufficient for a tensor to arise from three camera(theorems 4 and 5) .

We denote vectors and matrices with bold letters ,e.g. x and 'P . The determinant of a square matrix Ais noted det (A) . When the matrix is defined by a setof vectors, e.g. A = [aI a2 a 3] we use I aI a 2 a 3 I·The canonicaf basis of is noted e, i = 1,2,3. Whendealin g with projective spaces , such as ]p2 and ]p3, weoccasionally make the distinction between a projectivepoint, e.g. x and one of its coordinate vectors, x . Thedual of ]pn, the set of projective points is the set of pro-jective lines (n = 2) or the set of projective planes(n = 3), it is denoted by ]p*n. Let a , b , c, d be fourvectors of We will use in sec tion 5.5 Cramer 's re-fation (see [Faugeras and Mourrain (1995b)]) :

I bcd I a-I a c d .] b-l- I abd I c-I abc I d =°2 Grassman-Cayleyalgebra

Let E be a vector space of dimension 4 on the field IR.The corresponding three-dimensional projective spaceis noted ]P'3 . We con sider ordered sets of k, k :::; 4 vec-tors of E . Such ordered sets are called k-sequences.We first define an equivalence relation over the setof k-sequences as follows. Given two k-sequencesa j , . . . , ak and b l , . . . , bk, we say that they are equiv-alent when , for every choice of vectors Xk+ l , .. . , X nwe have :

I a I .. . a k Xk+l .. . X4 1=1b l . . . b, Xk+1 . . . X4 I(1)

That this defin es an equi valence relation is immediate.An equivalence class und er this relation is called an ex-tensor of step k and is writtenas

(2)

The product operator \I is called the join for reasons re-lated to its geometric interpretation . Let us denote byG k (E), 1 :::; k :::; 4 the vector set generated by ali ex-tenso rs of step k , i.e. by ali linear combinations oftermsIike (2). It is clear from the definition that G I (E) = E.

To be complete one defines Go(E) to be equal to thefield IR. The dimension ofGk(E) ism.The join op-erator corresponds to the union of projective subspacesof P (E) . The exterior algebra is the direct sum of thevector spaces Gk , k = 0, . . . ,4 with the join operator.For example, a point of ]P'3 is represented by a vectorof E, its coordinate vector, or equivalently by a pointof GI(E) . The join M I \1M2 of two distinct pointsMI and M2 is the line (MI , M2 ) . Similarly, thejoinM I \1M2 \1M3 of three distinct points MI , M2 , M3 isthe plane (M I , M2 , M3 ) . It is an extensor of step 3.The set of extensors of step 3 represents the sets ofplanes of]P'3.

Let us study in more detail the case of the lines of]P'3. Lines are extensors of step 2 and are representedby six-dimensional vectors of G2(E) with coordinates(L i j , 1 :::; i < j :::; 4) which satisfy the well-known thePlücker relation :

This equation allows us to define an inner product be-tween two elements L and L' of G2 ):

[L IL'] =L I 2L;4+ - L13L;4 - +L14L;3+ (4)

We will use this inner product when we describe theimaging of 3D fines by a camera in section 4 . Not ali el-ements of G2 (E ) are extensors of step 2 and it is knownthat:

Proposition 1 An element L 01 G2(E) represents aline if and only if [L IL] is equal to O.To continue our program to define algebraic operationswhich can be interpreted as geometric operations on theprojective subspaces of P(E) we define a second oper-ator, called the mee t, and noted 6. , on the exterior alge-bra G(E ). This operator corresponds to the geometricoperation of intersecti on of projective subspaces. If Ais an extensor of step k and B is an extensor Df step h,k + h 2: 4, the meet A 6. B of A and B is an extensorof step k + h - 4. For exampl e, if III and II2 are nonproportional extensors of step 3, i.e. representing twoplane, the ir meet III 6. II2 is an extensor of step 2,representing the line of intersection of the two planes.Similarly, if II is an extensor of step 3 representing aplane and L an extensor of step 2 representing a line,the meet II 6. L is either Oif L is contained in II or anextensor of step 1 representing the point of intersectionof L and lI. Finally, if II is an extensor of step 3, aplane, and M an extensor of step 1, a point, the meetII 6. M is an extensor of step O, a real number, whichturns out to be equal to IITM, the scalar product of theusual vector representation of the plane II with a coordi-nate vector of the point M which we note (II, M). Theconnection between a plane as a vector in G3(E) andthe usual vector representation II is through the Hodge

19

opera tor and can be found for example in [Barnabei etal. (1985) ].

We also define a special elernent of G4 (E), calledthe integral. Let {aj , . . . , a4} be a basis ofE such thatI ai . . . a4 1= 1 (the 4 x 4 determinant), a unimodalbasis. The extensor I = a , V . .. Va4 is called the inte-gral . In section 4 we will need the following proper tyof the integral:

Proposition 2 Let A and E be two extensors such. thatstep(A) + step( E) = 4. Then

A vB = (A t, B)VI =1 A B 1 I

This matrix is of rank 3. Its nullspace is thereforeof dimension I , corresponding to a unique point of 1P3 ,the optical center C of the camera.

We give a geornetric interpretaíion of the rows ofthe projection matrix . We use the notation :

(9)

where r , A, and 8 are the row vectors of P . Eachof these vectors represent a plane in 3D. These threeplanes are called the projec tion planes of the carnera.The projection equation (8) can be rewritten as:

The inner product (4) has an interesting interpre -tation in terrns of the join LvL', an extensor of step4:

Proposition 3 Wehave the fol lowing relation:

x: y: z = (r ,M) : (A,M) : (8 ,M)

where, for example, (r ,M) is lhe dot product of theplane represented by r with lhe point represented byM . This relation is equivalent to the three scalar equa-tions, of which two are independent:

[L ILI] =1 ABA/BI I

[L ILI] = (PIA/)(QIB /) - (QIA/)(PIB /) (7)

ij the two lines are represented as the meets of twoplanes P and Q and P' and QI, then:

/f one line is represented as the meet of two planes Pand .Q and the other as the join of two points AI andB I

, then:

(10)oOO

x (A ,M ) - y(r, M)y (8,M) - z (A,M)z (A ,M ) - x (8, M )

The optical center is the unique point C which sat-isfies PC = O. Therefore this point is the intersec-tion of the three planes represented by r, A , 8 . In theGrassmann-Cayley forrnalism, it is represented by themeet of those three planes r t, A t, 8 . Th is is illus-trated in Fig. I. Because of the definition of the meetoperator, the projective coordinates of C are the four3 x 3 minors of matrix P :

The planes of equation (r,M) = O, (A,M)O and (8,M) = O are mapped ta the image Iines ofequations x = O, y = O, and z = O, respectively. Wehave the proposition :

Proposition 6 The three projec tion planes of a per-spective camera intersect the retinal plane along thethree lines going througn the first three po ints of thestandard projective basis.

(6)[L ILI ] =1PQP/Q' I

where e l ., . . . , e4 is the canon ical basis of IR4. In sec-tion 3, we will use the following results on lines:

Proposition 4 Let L and LI be two tines. If the twolines are represented as thejoins oftwo points A and Eand AI and E I, respectively, then:

We will also use in section 4 the following result:

Proposition 5 Let L and LI be two tines. The innerproduct [L 1 L'] is equal to Oif and only if the two tinesare coplanar.

3 Geometry of one viewWe consider that a camera can be modeled accuratelyas a pinhole and performs a perspective projection. Ifwe consider two arbitrary systems of projective coor-dinates, for the image and the object space, the rela-tionship between 2-D pixels and 3-D points can be rep-resented as a linear projective operation which mapspoints of 1P3 to points of 1P2 . This operation can bedescribed by a 3 x 4 matrix P , called the perspectiveprojection matrix of the camera:

Proposition 7 The optical center C of the camera isthe meet r t, A t, 8 ofthe three projection planes.

The three projection planes intersect along thethree lines r t, A, A t, 8 and 8 t, r ca lled the pro-jection rays. These three lines meet at the opt ical centerC and intersect the retinal plane at the first three pointseI, e2 and e3 of the standard projective basis. Given apixel m, its optical ray (C, m) can be expressed verysirnply as a linear comb ination of the three projectionrays:

(8)

Proposition 8 The optical ray Lm of the pixel m ofprojective coordinates (x, y, z ) isgiven by:

L m = xA t, 8 + y8 t, r + zr t, A ( I I )

20

Figure I: Geometrical interpretation of the three rowsof the proj ection matrix as planes. The three projec-tion planes r , A and 8 are projected into the axesof the retinal coordinate system. The three projectionrays intersect the retinal plane at the first three points ofthe retina) projective basi s. The three projection planesmeet at the optical center.

Proof: Let con sider the plane :l:A - yr. This planecontains the optical center r 6. A 6. 8 since both Aand 8 do. Moreover, it also contains the point 1\11 . Tosee this, let us take the dot product:

(:rA - yr ,M) = x (A,M) - y(r ,M)

but since .r : y = (r, M) : (A,M), this expression isequal to O. Therefore , the plane xA - yr contains theoptical ray (C , 771.) . Similar1y, the planes zI' - x8 andy8 - .::A also contain the optical ray (C , '171) , which cantherefore be found as the intersection of any of thesetwo planes. Taking for instance the first two planes thatwe considered, we obtain :

(;l:A - yr) 6. (zr - x 8 ) =- :r(:l :A 6. 8 + y8 6. r + zr 6. A)

The scale fact or :"1: is not significant, and if it is zero, an-othcr choice of two planes can be made for the calcula-tion . We conclude that the optical ray Lm = (C ,171) isrepresented by the line x A 6. 8 + y8 6. r + zI' 6. A(see proposition I I) for another interesting interpreta-tion of this formula ). O

We also have an interesting interpretation ofmatrix pTwhich we give in the following proposition:

Proposition 9 The transpose pT of the perspectiveproje ction matrix defines a mappingfrom the ser oflineso] the retinal plane TO the set of planes going throughthe op tica l centei: This mapping asso ciates to the lineI represented by the vecror 1 = [x , y ,z]T the plane

T'P 1= zI' + yA + z8 .

Proof: The fact that pT maps planar lines to planesis a consequence of duality. The plane xr + yA + z8contains the optical center since eaeh projection planecontains it. O

Having discussed the imaging of points, let ustackle the imaging of \ines which plays a central rolein subsequent parts of this paper. Given two 3-D points.M1 and ,M 2 , the line L :::::: M 1 'i7M2 is an element of0 3 (IR4 ) represented by its Plüeker eoordinates. Theimage l of that \ine through a eamera defined by theperspective projeetion matrix P is represented by the3 x 1 veetor:

1=PM 1 X PM 2 =[(A,M1)(8, M 2 ) - (8, M 2)(A, M 1) ,

(8, M 1)(r ,M2 ) - (r,M2)(8, M 1 ) ,

(r,M1)(A, M 2 ) - (A, M 2)(r ,M 1)]T

Equation (7) of proposition 4 allows us to reeognize theinner produets of the projeetion rays of the eamera withthe line L:

1 [[A 6. 8 IL] , [8 6. r I L] , Ir 6. A IL]f (\ 2)

We ean rewrite this in matrix form:

(13)

where P is the following 3 x 6 matrix:

The matrix P plays for 3-D lines the same role that thematrix P plays for 3-D points. Equation (13) is thusequivalent to

li : l2 : l3 = [A 6. 8 IL] : [8 6. r IL] : [r 6. A IL]We have the following proposition :

Proposition 10 The pinhole camera also defines amapping from the set of lines ofjp3 to the set of tines ofjp2 . This mapping is an application from the projectivespace P(02(IR4 ) ) (the set of3D lines) to the projectivespace P(O2 (IR3)) (the set of2D tines). It is representedby a 6 x 4 matrix, notedP whose row vectors are thePlücker coordinates of the projection rays of the cam-era:

(14)

The image l ofa 3D tine L is given by:

LI : L2 : L3 = [A !:::. 8 I L] : [8 !:::. r I L] : [r !:::. A I L]The null spac e of this niappin g contains lhe se t of!ines goingthrough lhe optical cent er of the camera.

21

Proof: We have already provcd lhe lirs t part . Rcgard-ing lhe nullspace, if L is a 3D line such that PL = O,then L intcrsect ali three projection ray s 01' lhe camcraand hcnce goes through lhe optical ccntcr. O

Thc dual intcrpretation is also 01' irucrcst :- '}'

Proposition 11 tt« 3 x 6 matrix P represents amapping .!ilJIII 11"2 lo lhe set o] 3D lines. subset o]

)). which associates lo each pixel 11I its "P:tical ray L", .

Proof: Sincc P rcprcscnts a linear mapping írom. - 'J'

G2( IR') lo G2(IR3 ), P reprc scnts a linear mappinglrom lhe dual G2(IR3) * ofG2(IR:l ) which we can idcn-ti fy lo G 1(IR:3 ) , lo lhe dual G2(IR'I )'" which we canidcntiíy lo (see for examplc [Barnabc i et al.(1985)1 for lhe definition 01' lhe Hodg c opcrator and du-ality). Hencc it corresp onds lo a mapping frorn 11"2 toP(G2( IR'1)). 11' lhe pixel 117 has projcciivc coord ina tcs:/: , 11 and c . we havc:

simil arly F 21 = associatcs lo a pixel '17)'2 01' lhesccond view its epipolar line in lhe first one.

The rn atrix F I2 (rcsp. F 21) is 01' rank 2, lhe po intin its null-spacc is lhe epip ole c1,2 (resp. lhe epipolc('2,1):

FI 2el ,2 = F 21e2,1 = OThere is a very simple and na tural way oI' deri ving

lhe fundam ental matrix in lhe Grassman -Cayley for-malism. We use lhe sim pie idea that two pixels '17).and 11I' are in corrcspo ndcnce il' and only if their op-iical rays (C , 11I.) and (C', m' ) intc rscct . We then writedown this condit ion using propositi ons 5 and obtain lhefundamen tal matri x using lhe properti es 01' lhe doubl ealg cbra .

We will denote lhe rows of P by r , A , e , andlhe rows 01' -p ' hy r' . A', e'. We have lhe Iol lowingpropos ition :

Proposition 13 The exp ression ofthefundamenta! II/a-trix F as a function ot' lhe r(}\1' vectors o( lhe matricesP atul -p ' is:

- TP m :rA 6 e + ye 6 r + .:r 6 A

and we recognize lhe righr hand side lo bc a rcprcsc n-tation 01' lhe optical ray L",. O

[

I AeA'e' 1F = IAee'r' I

1 x o r- A' 1

I erA'e'1I ere'r'lI o r r -A'I

1 r AA ' e' 1 ]I r x er- II r x r :A' 1

( 15)

3.1 Affine digression

In lhe affinc Iramework we can give an inicrcsi ing inter-prct ation 01' lhe th ird projcciion plane of lhe pcrspcctivcprojc ction rnatr ix:

Proposi tion 12 The third projection plane e oI" lheperspectiva projection matrix P is lhe jocal plan« o]lhe comera.

Proof: Thc points of lhe plane 01' cqua tion (e , 1\1) =oare mapped lo lhe poinls in lhe relinal plane such lhal:; = O. Thi s is lhe equal ion 01' lhe line al infinily in lherelinal plane. The plane represenled by e is lhercforelhe sei 01' poinls in 3D space which do nol projecl ai fi-nile dislance in lhe relinal plane. These po inls form lhefoU/1 plalle, which is lhe plane conlaining lhe oplicalcenler, and parai lei to lhe relinal plane. O

When lhe focal plane is lhe pla ne ai infinily , i.e. ee 'l, lhe camera is called an aftine camera and performsa parallel projeclion . Nole lhal Ihis c1ass oI' cameras isimportanl in applicalions. including lhe Orlhographic .weak perspecl ive, and scaled orlhographic projeclions.

4 Geometry of two vicws

In lhe case oI' lwo cameras. il ois well-k nown lhal lhegeomelry oI' cor respondences helween lhe lwo viewscan be descrihed compadly by lhe fundamenlal malri x.noled F 12 which associales lo each pixel 1111 o I'lhe fi rslview ils epipolar line noled I"" in lhe second image:

22

Proof: Lei 111 and 11I' hc two pixels. Thcy are in corre-spondcncc if and only if thc ir opucal rays (C ,nl ) = L ",and (C', 111') = L;", intcrsect . According lo propo-sitiou 5, this is equivalem lo lhe [act that lhe innerproduct [L ,,,IL; ", 1 of lhe two opt ical rays is equ al lOI), Lct us translatc ihis algcbraically. Lei (: 1:. .11.': ) (rcsp .(.r' . .li' , .:' )) bc lhe coordinatcs 01'11I (rcsp. 1/1') . Usingproposi tion I I. wc writc :

- I"L ", P ll1 = .rA 6 e + .IIe 6 r + cI' 6 A

and:- ' I"

:::: P 111 ' = .1" A' 6 e' + .II'e' 6 r' + .: 'r' 6 A'

We now wanl lo compulc [L '" In order lo do lhis .we usc proposilion 3 and compule L", \7 L;II' :

L," 6 e + .IIe,\6 r + .: r /::, A) \7(.r'A' 6 e' + .II'e' 6 r' + .: 'r ' 6 A')

Using lhe linearily oI' lhe jo in operalo!". we ohla in <Inexprcssion which is hiline ar in lhc coordinales 01' 111and11I ' and conlains lerms such as :

(A 6 e)\7(A ' 6 e')

Sinc e A 6 e and A ' 6 e ' are cxlensors oI' slep 2. wecan apply proposilion 2 and wrile:

(A 6 e )\7(A' 6 e ') =1AeA'e' I Iwhere I is lhe inlegral delincd in seclion 2. We h,l\'esimilar exprcssion s ror all lcrms in L ", . . We Ihusohlain:

I ' I"i L ", \7 L ",. = (m Fm)I

Proposition 14 The expression ofthe epipo les e and e'as a function of the row vectors of the matrices P and-p' is:

where the 3 x 3 matrix F is defined by equation (15) .Since L", V'L:n , = [Lm IL:n , ] I , the conclusion 1'01-lows. O

Let us determine the epipo les in this forma lism. Wehave the follow ing simple proposit ion :

Proof: We have see n previously that e (resp. e') isthe image of C ' (res p. C) by the first (resp . the sec-ond) came ra , Accord ing to proposition 7, these opti-ca l ce nters are represent ed by the vectors of G 1 (IR4 )r 6 A 6 e and r' 6 A' 6 e '. Therefore we have,for examp le, that the first coordinate of e is:

which arise naturally from the trifocal plane: for exam-pIe, the epipolar line in view 2 of the epipole e1,3 isrep resented by F 12e1,3 and is the image in view 2 ofthe optical ray (C1, e1:3) which is identical to the line(C1 , C3 ) . This image is the trifocal line t z which goesthrough e2,3, see figure 2, hence the first equation in(17 ).

[I r 'r Ae I ]

e' IA'r Ae Ile'rAel[

I r r' A' e' I ]e IA r' A' e' I

ler'A'e'1

(r ,r ' 6 A' 6 e')

Proposition 15 The fundamental niatrix of two affinecanieras lias thefonn:

whic h is eq ua l to (r ' 6 A ' 6 e ') 6 r =- I r r' A' e ' I. o

4.1 Another affine digression

Let us now ass ume that the two cameras are affine cam-era, i.e. e e' e4 (in fac t it is sufficient thate e'). Because 01' the standard properties of deter-min ant s, it is clear from equation (15) that the funda-mentai matrix takes a spec ial form:

Figure 2: The trifocal geometry.

2. In the spec ial case where the 'three optica l ce n-ters are aligned, the pred iction with the funda-ment aI matrices fails a lways since, fo r example,F l 3rn1 F 23rn2, for ali corresponding pix elsm1 and m2 in views I and 2, i.e. such thatrn§'F j '},ln1 = O.

For those two reasons, as we ll as for the es timation

I . In the general case where the three optical center sare not aligned, when the 3D points lie in the tri-foca l plane (the plane defined by the three op ticalcenters) , the predi ction with the fundamental ma -trices fails because , in the previous example bothepipolar lines are equal to the trifocal line ts .

This has an imp ortant imp act on the way we haveto es timate the fundamental matrices when three viewsare available : very effic ient and robust algorithms arenow available to estimate the fund amental matrix be-tween two views from point corre spondenees [Zhanget al. (1995 ), Torr and Zissermann (1997), Hartl ey( 1995)]. Th e constraint s (17) mean tha t these algo -rithms cannot be used blindly to est ima te th e three fun-.damental matr ices independently because the resultingmatri ces will not satisfy the constrai nts causin g errorsin further proces ses such as prediction.

Ind eed , one of the important use s of the fundamen-taI matrices in trifocal geometry is the fact that they ingeneral allow to predict from two correspo ndences, say(Tnj, Tn '2) where the po int Tn3 should be in the thirdimage: it is simp ly at the intersection of the two ep ipo-lar lines represe nted by F 13rn1 and F 23rn2, when thisinters ect ion is well-defined.

It is not well-defined in two cases :

I r AA'e' I]I r Ae ' r' 1I r Ar ' A' I

( 16)

OO

I err'A'1[

OF = O

I Ae r' A'1

5 Geometry of three views

5.1 Trifocal geometry from binocular geometry

When we add one more view, the geome try becomesmore intricate, see figure 2. Note that we ass ume thatlhe three opt ica l ce nters C j , C'2 , C3 are diffe rent, andcull this conditi on the general viewpoint assumption.When they are not aligned they defin e a plan e, ca lledthe tri focal plane , which intersccts the three imageplanes along the trifocal tines t i , t-z t3 whic h containthe cp ipo les e ; ,j , I, =f:. i i = 1, 00, , 3, j = 1, 00 , , 3 .The three fundamenta l matrices F I '}" F 23 and F 3 1 arenot indepcnd ent since they must satisfy the thre e con-straints:

23

problem mcnti oncd prcviously, it is intcrcsting lo char-actcrizc lhe gco rnc try of thrcc views hy anothcr cntity.lhe tri focal tensor.

The trifocal tensor is rcally mcun t at descri h-ingI inc corrcspondcnccs and , as such , has bccn wcll-known under disguise in lhe pari o f lhe computc r vi-sion community dealing with lhe prob lcm of structurc[rom motio n ISpctsakis and A loimonos ( 1990h ). Spc t-sakis and Aloirnonos ( )990a) , Weng et al. ( 1992) I he-Iorc it was Iormally idcnt iíicd hy Hart lcy anel ShashuaIHartl cy ( 1994) , Shashua ( 1( 95 )I,

5.2 Thc trifocal tcnsors

t.

LeI us considcr thrcc views. with projcction matri ccsP II . "II. = 1,2 , :1 , a 3D line L with images f ll ' Given twoimagcs I i and I., of L, L can he dc lincd as lhe intcrscc-

. T 'I"tion (lhe mcct ) o f lhe two planes P j l i und P ., 1.:

Figure 3: Thc line l i is lhe imagc hy carnc ra i 0 1" lhe 3Dline L intcrs cction of lhe planes dcfi ncd hy lhe opticalccntcrs o I" lhe ca rncrasj and k and lhe lines l i and t i:rcspcctivc ly.

The vcctor L is lhe Gx 1 vcctor of Pl ückcr coordinatcso I" lhe line L.

LeI us writc lhe right -hand sidc of this cqu.uioncxpli citly in tcrms of lhe row vcctors o! lhe matri ccsP j and P ., and lhe coo rdinatcs orl j and h:

By cxpand ing lhe mcct opcrator in lhe prcv ious cqua-tion, it can bc rewritt cn in lhe Iol lowing lcss co rnpac tIorrn with lhe advantagc 01" making lhe dcpcndcncy onlhe project ion plan es 0 1" lhe matriccs P j and P . ex-plicit :

This expression can hc also PUI in a xlightly Icsscompact Iorm with lhe advantagc of making lhe dcpcn-dency on lhe proj ccii on planes o f lhe matriccs P 11 . 11 =1. :2, 3 cxplicit :

(21 )

This is, in lhe projcciivc Irarnework, lhe cx act analogof lhe cquation used in lhe work of Spetsaki s and Aloi-monos ISpctsakis and Aloimonos ( )990h)110 study lhestruc turc trom motion probl cm fro rn line corrc spon-dcnccs,

The thrcc 3 x 3 matriccs G i'.11 = 1. 2.3 are oh-tuin cd Irorn cquations ( I X) and ( 19):

[ r , /'; r . r i /'; A. r i /'; 8 . ]L (I" A i /'; r . A i /'; A. A i /'; 8 . I..I e , /'; A. o, /'; A. e , /'; 8 .( IX)

=11\,6 r r, 11 1\:6 :1\';r ; 111\,6 ,6 ',r " 1

I 1\,6 ,r,1\,.. 111\,6, 1\,1\,.. 111\,6;6',1\,.,1

I 1\,e,r ,6" 1 ]11\,e, 1\',6" I11\,e, 6 ,6 ",1

( 22 )

wh ich is valid 1"01' i. f. j f. k . Nole lhal if we exchangeview .i anel view I.. , we JUSI change lhe sign of l i andlherefore we do nol change I;. A geo l1le lric inler prela-lion of lhis is shown in figure 3. For co nvenience. werewril e equalion (19) in a more compacl form :

Thi s equation should bc intcrprcicd as giving lhePlückcr coordinatcs 01" L as a linear co rnbination of lhelines defin ed by lhe mcets ofthe proj cct ion plane s otthcperspec tive matri cc s P j and P ." lhe cocffic ients bcin glhe product s 0 1" lhe proj ccti vc coo rdinales of lhe lines 11and Ik .

The il1lage l i of L is lherel"ore ohlained hy appl y-ing lhe l1lalrix P; (delin ed in seclion 3) lo lhe Plück ercoard'inales 01" L , hence lhe equat ion :

lo predi<':l f. frol1l lhe images I; and / , o f L. 11" L isalso in an epipolar plane 01" lhe camera pair ( i . .I )il is in lhe lrifocal plane o I" lhe lhree call1eras and

Note that cquaiion ( 19) allows us lo prcdict thc co-ordinatcs of a line I; in image i given two images l i andI., o f an unknown 3D line in images j and k. cxccpt intwo cases where T ;(l j ,h) = o:

JI. When lhe lwo planes dete rmined hy l i and f., areidenlical i.e. when l i and h ai'e correspondingepipo lar lines belween views j and k. Thi s isequivalenl lo say ing lhal lhe 3D Iinc L is in anepipolar plane o r lhe camera pair (j .!.:) . The meellhal appears in equ alion ( 19) is lhen O and lhe lineI ; is undcfined , see ligure 4. Ir L is nol in an epipo-lar plane of lhe camera pair (i , j) lhen we can uselhe equalion :

(20)

( 19 )

24

prediction is not possible by any of the formulassuch as (19).

2. When lj and li: are epipolar tines between views iand j and i and k, respectively. This is equivalentto saying that they are the images of the same op-tical ray in view i and that li is reduced to a point(see figure 5).

C 'J

Figure 4 : When lj and h are corresponding epipolarlines, lhe two plan es P Jl j and P[lk are identical andthere fore T ;(lj , ld = O.

L_----- C:'(: ,

Figure 5: When lj and lk are epipolar lines with re-spect to view i , the line li is reduced to a point, henceT ;(li ,h) = o.

Except in those two cases , we have defined an applica-tion T;. frorn 1F'*2 x 1F'*2 , the Cartesian product of twoduais of the projective plane, into 1F'*2. This applicationis represented by an application T i from IR3 x IR3 intoIR3 . Th is application is bilinear and antisymmetric andis represented by the three matrices Gi\ n = 1,2,3. Itis caJled the trifocal tensor for view i. The propertiesof this application can be summarized in the foJlowingtheorem :

Theorem 1 The application T;. : 1F'*2 x 1F'*2 ----+ 1F'*2is represented by the bilinear application T i sucli that

- T T .T ;(lj ,ld P i(P j r, !:::, P klÚ T i hasthe followingproperties:

/ . /t is equal to O iff

(a) lj and lk are epipolar lines with respect tothe i th view, or

(b) lj and lk are corresponding epipolar lineswith. respect to the pair (j, k) ofcameras.

2. Let lk be an epipolarline with respect to view i andli the corresponding epipolar line in view i, thenfor ali lines lj in view j which are not epipolarlines with respect to view i: T i(lj , lk ) li.

3. Similarly, let lj be an epipolar tine with respectto view i and li the corresponding epipolar line inview i, thenfor all lines lk in view k which are notepipolar tines with respect to view i: Ti (l j, lk)l i

4. /f lj and lk are non corresponding epipolar tineswith respect to the pair (j, k) of views, thenT(l j , lk) = t i , the trifocal tine of the ith view,if the optical centers are not aligned alld O other-wíse.

Proof: We have already proved point I. In orderto prove point 2, we notice that when lk is an epipolarline with respect to view i, the line L is contained inan epipolar plane for the pai r (i, k) of cameras. Twocases can happen. If L goes through Ci , i.e. if lj is anep ipolar line with respect to view i, then li is reduced toa point and this is point 1.a of the theorem. IfL does notgo through Ci, lj is not an epipolar line with respect toview i and the image of L in view i is independent of its-position in the epipolar plane for the pair (i , k) , it is theepipolar line li corresponding to lk. The proof of point3 is identical after exchanging the roles of cameras kand j .

If lj and lk are non corresponding epipolar linesfor the pair (j, k) of views, the two planes (Cj , lj) and(C/c, ld intersect along the line (Cj ,Cd. Thus, if C,is not on that line, its image li in view i is indeed thetrifocalline t i, see figure 2. O

A more pictorial view is shown in figure 6: the ten-sor is represented as a 3 x 3 cube, the three horizontalplanes representing the matrices Gi ,n = 1,2,3. It canbe thought of as a black box which takes as its inputtwo lines, lj and lk and outputs a third one, li. Hartleyhas shown [Hartley (1994), Hartley (1997)] that the tri-focal tensors can be very simply parameterized by theperspective projection matrices 'P« , n = 1,2,3 of thethree cameras. This result is summarized in the follow-ing proposition:

Proposition 16 (Hartley) Let 'P« , n = 1,2,3 be thethree perspective projection matrices of three camerasin general viewing position. After a change of coor-dinates, those matrices can be writien, P 1 = [13 O],P 2 = [X e2,d and P3 = [Ye3,l] and the matrices

25

G í' can lr« cxpressed os:

Figure 6 : A three-di rncnsi onal represcntati on of the tri-local ten sor.

whcre tlie vectors X ( IL ) and y ( " ) ore tlie COIIlJII II vec-tots ofthe niatrices X and Y. respecti vel v.

Tlie correspotuling tines in vie lV i a re represented liye " x e i» and can be obtained as T ;(Ij , n = 1, 2, 3fo r an)' lj not equal to lj' (see proposition J9).

Proof: Th e nu llspace of G ;' is the set o f linessu ch that T i(lj , Ir) ha s a ze ro in lhe n -th co ord ina tefor ali lines lj . The co rre spo nd ing lincs I; such thatli = T i (Ij , Ir ) a li go th rou gh lhe poinl represented bye ,,, n = 1, 2, 3 in lhe -i -Ih retinal plan e . T his is tru c ifand only if is lhe image in the I.: - lh retinal pl ane oftheproj ec tion ray Ai 6. 8 i (n = 1) , 8 i 6. r ; ('/I. = 2) andr i 6. Ai (n = 3): lr' is an epipo lar lin e w ith respecllo view i and theorern I, point 2, shows that ror cachn. lhe corresp onding line in vicw r is ind ependem of l i 'Moreover. it is rcprescntcd hy e" x e i.h" O

A sim ila r rea soning applies lo lhe rnntriccs G i' '!':

Proposition 19 (Ha r t ley) The nullspaccs oft lie nuuri-ces G( I' ore the three epipolar tines. no ted l i' . 11 =1. 2,3. in the j -th retina! plan e o] the threc projectionmys o] cumera i. Thesc threc tines inte rsect (J{ theep ipo le ('j ,i . see figure 7. The corresponcling line s inview i are represented b)' e " x ei ,j and can be obtain edas T i (Ij" Id ,n = 1, 2, 3 fo r auy i, not equa l to I;:'

ti = 11:

We use this propositi on as a dcfinition:

Definition 1 AII,v tensor ot' the [o nn (23) is a trifocalfcnsot:

5.3 A third affine digression

If the th rcc ca me ras ar e a ffinc , i .e . if 8 i :::::: 8 j :::::: 8 /,; ::::::e.l , then wc can read o ff eq uation (22) the Io rrn o f thematriccs G ;' , l1 = 1, 2, 3.

Proposition 17 For affin e canteras, the trifo cal tensortakes thc simp le fo rm:

[ I A ; e ; r', r,.. I I A ; e i r , A ,. I o ]I A i e ; A j r ,. I I A ; e iA,A,· 1 oo o o

[ I e ; L r , r,. I I e ;r;r, A ,. I o ]I e ; r ; A j r ,.. I I e , r ; A iA,. I (I

o o o

J [ I r ; A ; r' , r , I , r ; A; r j A ,. 1 I r , A ; r' , e ,o I ]G ;= I r ;A;Air/.. 1 I r, A ; A j A,.. I I r,A;Ai e ,. II r ; A , e j r,. I I L A ie jA i. I (I

,5.4 AIgebraic and geornetric properties of the tri-foca l tensors

The maU'ice s Gi' , '/I. = 1, 2, 3 have inler esling proper -tie s wh ich are cl os el y rel ated to the epipo lar geol11e-try o f th e view s j and k . We st arL with the foll owingpropos it ion , which wa s pro ved for exal11 p le in [H arll ey( 1997) J. The proof hop efull y g ives some more geo me t-rie insight of what is going o n:

Proposition 18 (HartIey) The IIwfrices Gi' are ()t'ra llk 2 alltl fheir lIu llspaces are lhe fh ree ep ipola r lill es,lIofetl Ir' in view k oI fhe fhree pmjecfioll rays ot' ('(1 //1 -era i. These fhree lill es illferseCf (J{ fhe ep ipo le (' Li .

Figure 7 : The lines lj' (r esp . l r), 11. = 1, 2, 3 in thcnull spaces 01' lhe matrices G;"r (resp. G i' ) are lhe im-ages 01' lhe th ree projecti on rays 01' carnera 'i . Hcncc,they inte rse c t at lhe epipole e j . ; (res p . ( 'h' ,;). T hc cor-res ponding epipo lar lines in ca rncra i are ohtai ncd asT;{l j' . ld (rcsp. ror " cF 'r' (res p . l i cF i ;' ).

This pro vid es a geor netri c in tcrp rc ta tion o f lhe mat ri-ce s G ;': they represem mappings Irorn lhe se t 01' Iine sin view I.: lo lhe sct 01' points in view .i locatcd on theepipo lar line li' dcfincd in prop osition 19. This rnap-ping is geo rnc trically delined hy laking lhe inl er sectiono I'the plan e delined by lhe oplica l ce nle r o I'lh e I.: th ca l11-e ra and a ny line 01' ils relinal plan e w ilh lhe '/I Ih projec-tion ray o I'lhe ilh came ra and ronning lhe image o I' lhi spo inl in lhe j lh camera . Th is llo inl does nol ex ist whe nlhe pl ane conlain s lhe projection ray. Th e correspond-ing lin e in lhe {,:lh retinal plane is lhe ep ipo lar linedelined in propositi on 18. Moreover , the lh ree co lum ns01' G ;' represenl three poinls which ali bel ong to theepipo lar tine li'.

Similarl y, lhe maU'ices G;"I' represe n t l11appingsfro l11 the set oI' lines in view .i to the seI 0 1' poinls in

26

view J.: located on the ep ipo lar line lí.:' .

Remark 1 It is importan t to note that the rank of themat rices G " cannot be less than 2. Consider for exam -ple the case n = 1. We ha ve seen in prop osition 18 thatthe nullspace ofG 1 is the image of the projection rayA i t::, 8 i in view k . Under our general viewpoint as-sumption, this projection ray alld the opt ical center Ckdefine a unique plane unless it goes througli Ck, a situ -at ion that CCI/1 be avoided by a change of coordinates inthe retina ! plane of the ith camera. Theref ore there is aunique line in the righ t nullspace ofG; and its rank isequal to 2. Simi lar reaso nings app ly to and Gr.

A question that will turn out to be important lateris that 01' knowing how many distinct lines lk (resp. lj)can there be. This is described in the following propo-sition:

Proposition 20 Under the general viewpo int assump-tion, tlie rank of the matri ces [lLli and [1; 1; 1]] is2.

Proof: We know from propositions 18 and 19 thatlhe lhe ranks are less than 01' equa l to 2 because eac htripl et 0 1' lines intersect at an ep ipole. In order for theranks to be equal to I, we would need to have only oneline in ei ther retinal plane. But this would mean thatthe three planes defin ed by Ck (resp. Cj ) and the threeprojection rays 01' the i th ca rnera are identi cal which isimpossiblc since C; I- Ci. (resp. C j I- CiJ and thethree projection rays o f the it h camera are not cop lana r.O

Algebraically, this implies that the three determi-nants det( G i' ), n = 1,2 ,3 are equal to O. Another con-strainl impli ed by proposit ion 20 is that the 3 x 3 deter-minants forrned with the three vec tors in the nul!spacesofthe G ;" 'n = 1, 2, 3 (resp . ofthe Gi T ,n = 1,2,3)are equal to O. It turns out that the applications T i , i =1,2 ,3 satisfy other algebraic constra ints which are alsoimportant in practic e.

The quest ion 01' characterizing exactly the con-stra ints sa tisfied by lhe tensors is 01' great practical im-portance for lhe problem 01' est ima ting the tensors fromtripl cts 01' line correspondences (see [Faugeras and Pa-padopoul o ( 1998)]). To be more specific, we know thatthe tensor is equ ivalent to the knowledge 01' the threepers pec tive projection matrices and tha t they dependupon 18 parameters. On the other hand a trifocal tensordepends upon 27 parameters up to sca le, i.e. 26 pa-ramcters . To be more precise , this means that the setof tri focal tensors is a mani fold of dimension 18 in theprojcctivc space of dimensio n 26. The re must thereforecxis t constra ints between lhe coefficients that define thetensor, Our next task is lo discover some 01' those con-stra ints and find subse ts 01' thern which characterize thetrifocal tcnso rs, i.e . that guarantee that they have the1'01'111 (23).

To simp\ify a bit the notations, we will assume inlhe sequeI that i = 1, j = 2, k = 3 and will ignore thei th index everyw here, e.g. denote TI by T .

We have already seen severaI such constraintswhen we studied the matrixes G" . Let us summarizethose constra ints in the fol!owi ng proposition :

P roposition 21 Under the general viewpoint assump-tion, the trifocal tensorT satisfie s the three constraints,called the rank con straints:

rank( G n) = 2 => det(G ") = O n = 1, 2, 3

The trifocal tenso r T satisfies the two constraints,called the epipolar constraints:

rank( [q q]) = rank( [q q q]) = 2 =>Iq I= I q I= O

Those five constraints which are c1early algebraicallyindependent since the rank constraints say nothingabout the way the kernels are related constrain the form01' the mat rices G " .

We now show that the coeffi cients of T sat isfynine more algebrai c constrai nts of degree 6 which aredefined as follows. Let en , n = 1,2 , 3 be the canon-ical basis 01' 1R3 and let us eonsider the four \inesT ( e k 2 ,ek3 )' T ( e I2 , e k3 ) ' 7 (e k2 ,e/3 ) and T ( e /2 ,eI3 )where the indexes k2 and l2 (resp. k3 and l3) are dif-ferent. For example, if k2 = k3 = 1 and lz = l3 = 2,the four lines are the images in camera I of the four 3D\ines r 2 t::, f 3 , A 2 t::, r 3 , r 2 t::, A3 and A2 t::, A3 .

These four lines can be chosen in nine differentways satisfy an algebraic constraint which is detailed inthe followi ng theorem which isproved in [Faugeras andMour rain (1995b)).

Theorem 2 The trifocal tensor T satisfies the 9 alge-braic constraints of degree 6, called the vertical con-strai nts:

I T 'k 2k3T ' k213T ' /2/3 I1 T 'k2k3T '12k J7'/2 /3 I -I T '12k3T ' k 213T .121J 11 T' k2k3T '12k3T ' k 213 1=O,

(24)

where T ,lm represents the vec tor T ( el , em),

The reade r can convince himself that if he takes anygeneral set of lines, then equation (24) is in general notsatisfied. For instance, let li = e . , i = 1, 2,3,4. It isreadily verified that the left hand side of (24) is equal to-2.

Referring to figure 6, what theorem 2 says is thatif we take four ver tical columns of the trifoca l cube(shown as das hed lines in the figure) arra nged in sucha way that they form a prism with a square basis, thenthe expression (24 ) is equal to O. Thi s is the reason whywe cal! these const ra ints the vertical constrai nts in thesequeI.

27

It turns out that lh e samc kin el 01' relaiions hold forlhe o the r two principa l d ircciion s o lthc cube (shown asso lid lines o f dilfcrcnt widths in lhe samc figure ):

Theorem 3 The I rifá {'([ I tenso r T sat isfie s al so lhenin e algchrai« const raints, ca llccl lhe row constrain ts:

I T k,.k"TÁ.,./:,T', h ·11 T k, .Á·"T', .k"T" .I:. I -I T',. Á·"Tk,.,,,T, , .I,, 11 T k, .Á·"T', .k;,T k, .I" 1= 0,

(25)

and lhe nin e algebrai c coustraints. called lhe columnconstraints:

IT Á:, Á'c T k" ".T,, ,". 11 T Á,kc.T" Á·c.T" ,c. l-IT" Á'" T Á·" ., .T" ,c. I1T Á,Á" T " k".T,." , . 1= (J

(26)

Proof: We do lhe proof for lhe Iirst sct of con xtruintswh ich co nccrn lhe co lumns of lhe matri ccs G il. Th cproo f is ana logous for lhe othcr sct concerning lhe rows.

T he th ree co lumns o f G il represenl th rcc poinlsG' h: ' I,: = 1, 2 , 3 of lhe cpi polar line 12 (scc lhe d is-cuss ion a lter proposition 19). To he co ncre te , lct uscon sid cr lhe Iirst two co lumns o f G I and G 2 , lheproof is s imi lar for lhe o the r co rn bina tio ns.We cons idc rlhe two se ts of poi nls dcfi ncd hy {/2G : + b2G i and{/2Gf + These two seis are in proj cct ivc cor-rcsp onden cc , lhe col lin eation hcin g lhe idcruity. lt isknown th at lhe lin e j oining two corrcsponding poinlsen velops a conic. It is eas ily shown that lhe dci crmi-nanl 01' lhe ma tr ix defining this coni c is equa l lo :

I TI 1T l.2T'2.2 I1T IIT2.IT22 I -I T 2 .JTI .2T2. 2 11 TI .IT2.ITI2 I

In order lo show tha t thi s exp ress ion is cq ua l lo 0 ,we show that lhe co n ic is degenerare, co nta in ing twopo ints . T his result is readil y obta ine d íro m a gcome ui cinterpret ation 01' what is go ing on.

The poinl {/2G{ + b2G i is lhe image by G 1 oflhe lin e (/ 2ej + b2e 2 , i.e . lhe firs l se I of poinls is lheimage by G J o f lh e pen c il 01' lin es go ing lh rough lhepoinl e3. Using aga in the geometric inlerprel ali on o fG ' , we re ali ze lhat th ose points are the images in theseeonel im age or the points 0 1' inler seeli on o f lhe flr stpr oi ecti on ray AI 6 8 I of the fl rs t came ra w ilh thepenei I 0 1' p lanes go in.g throu gh the lhird projee lion rayT 3 6 A 3 01' lhe lhird camera. S im ilarly , the second set01' points is the image of the points o f int er seclion of lhesecond proj eetion ray 8 1 6 ri 01' lhe rirst camer a wit hthe penci! 01' p lan es going through the third projecli onray r 3 6 A :1 01' the thi l:d camera .

The lin es j oin ing lwo corresp onding po inls o fthose two se ts are thu s the images o f the lines j oininglhe two points of in le rsection o f a p lane co nta in ing lhethirel projectio n ray r 3 6 A, 01' lhe third eame ra withthe firs l and the second projection rays , AI 6 8 I and

28

8 1 6 ri ,of the first ca rnera. Th is line lies in the thirdproj ecti on plane 8 I of lhe firsl ca rnera anel in the pl an eTI of the pcncil . Th ere fore it gocs throu gh the point o fintcrscction o f lhe third proj ccti on plane e I 01' the firstcarncra anel the thir d pr ojecti on ray 8 2 6 r :1 o f lhethi rd carneru, sce figure 8. In im age two , ali lhe linesgoi ng ihrough two corrc sp on din g poi nts go throu gh lheimagc o f th at po int. A spcc ial case occu rs whe n the

TI

Figure 8: A plan e o f lhe pene il o lax is r 3 I:::. A3 int er-sec ts lhe plane ri al on g a line go ing through lhe po intri 6 r , 6 A :1· T he poinls a({/2 , b2 ) anel b ({/2, b'2 ) ar elhe images by G I ar {[.2 G : + b2Gi and {[.2 G f +rcsp ect ively.

plane TI go es through lhe tirst opiica l conter, lhe twopoirus are identi cal to lhe epipo le ('2 , I and the Iine j oin-ing thcrn is no t de lineei. T hc rclorc lhe co n ic is red uccdlo lhe two poin ts C2 ,1 anel lhe point o f int ersecti on o flhe two lines (G : , Gf) and (Gi , This po int is lheimage in the se co nd carner a 01' th c poinl o f int ersecti onr I 6 r 3 6 A:1 o f the firs t projection plan e , ri , o f lhefirst carncra w ith lhe third projccti on ray, r :1 6 A :1, o flhe third camera, O

T he thco rern elraws our att cntion lo thrce se is o f th rccpo inls, i.e . th rec triangles, which have some very in-leresling properti es . T he lri an gl e lh al ca me up in lheproor is lhe one whose ve rtexes are lhe imag es o r lhepo ints AI = ri 6 AI 6 8 a , B I = ri I:::. 8 a I:::. r :1and D I = r I 6 r 3 6 A a. The o the r two lri an gl es arelh ose whose verlexes are lhe images 01' the poinls A '2 =AI I:::. A 3 6 8 a , B 2 = AI 6 8 :1 I:::. r:1, D '2 = A I I:::.r 3 I:::. A a on one hanel, anel A :1 = 8 1 I:::. A:1 I:::. 8 a ,B a = 8 1 6 8 :1 6 r a , D a = 8 1 6 r :l I:::. A I on theothe r.

Not e lh al the lhree se ts 01' vertexes A I , A 2 , A:1,B I , B 2 , B3 and D I , D 2 , D a, ar e aligneel o n lhe lhreeprojecli on rays o f lhe lhird came ra and lhereforelheir images are also ali gn eel, lhe three lin es / 2 1 =(a i , (J.2, (/:l), 122 = (bJ , b2 , b:l ) anel 12:1 = (d i , ri:!. rl:dco nverg ing to the ep ipole 1'2, :1, see figure 9. The co rre-spo nding epipol ar lines in image 3 are represented byh; = ea.2 x e ; , oi = 1, 2, 3, resp eclivcl y. Note that a li

poi nts (Li , bi , di , i = 1, 2, 3 can be expressed as sim-pIe functio ns 01' the co lumns of the matrixes G " . Forexample:

Figure 9 : Th e three triangles have corr esponding ver-texes aligned on epipolar lines for the pair (2,3) 01' im-ages.

Th e same is true 01' the constraints on the rows 01'lhe matrices G " . More spec ifically the co nstraints (26)int rodu ce nine other po ints (HiJC , lVIi ), i = 1,2,3with H I = r i l::i A 2 l::i O2, K I = r i l::i O 2 l::i r 2,M I = r i l::i r 2 l::i A 2, H 2 = AI l::i A 2 l::i O 2,K 2 = A I l::i O 2 l::i r 2 , M 2 = A I l::i r 2 l::i A 2,and H 3 = 0 1 l::i A 2 l::i O 2, K 3 = 0 1 l::i O 2 l::ir 2, M 3 = 0 1 l::i r 2 l::i A 2 . The three se ts ofpoints H , , H2 , H3 , ](1 , ](2 , ](3andJ11l, lVh , 1113 arealigned on the three projection rays 01' the second ca rn-era and therefore their images are also aligned, thethree (ines = (h l ,h2,h3 ) , = (k l , k2 ,k3 ) and

= ('171'1,777.2, 7Tl.3) converging to the ep ipo le e3,2'

Th e co rresponding epipo lar lines, l; i ' i = 1, 2, 3 in im-age 3 are represented by e2,3 x e. , i = 1, 2, 3, respec-tively.

Note that this yie lds a way 01' recovering the fun-dament aI ma trix F 23 s ince we obtain the two epipo lese2 ,3 and e3 ,2 and three pair s of co rresponding epipo larlines, in fac t six pairs. We wi ll not add ress further herethe problem 01' recoveri ng the epipo lar geome try oI' thethree views, let us simp ly menti on the fact that the fun-damenta i matrices whic h are recovered from lhe trifocalten sor are conipatible in the sense that they satisfy theconstrain ts ( 17).

Th ere is a further set 01' constraints that are sa tis-ti ed by any trifocal tensor and are also 01' interest . Th eyare describe d in the next pro position:

Proposition 22 The trifoca l tensor T satisfies the tenalgebraic constraints, called the extended rank con-straint s.

3

rank( L ÀnGn ) 2 VÀn n = 1, 2, 3n =l

Proof: The proof can be done either algebraically OI'

geometrically. The algebraic pro of simp ly uses the pa-rameterization (23) and verifi es that the constraints de-scribed in proposition 23 are sat isfied. In the geometricproof one notices that for fixed values (no t ali ze ro) ofthe Àn 's, and for a give n line l3 in view 3, the poin twhich is the image in view 2 of line l3 by Àn G nis the image of the point defin ed by :

ÀI1'fll::i (AI l::i 0 r) + À21'fll::i (0 1 l::i r r) +À31'fll::i (r i l::i AI )

Thi s express ion can be rew ritten as :

1'fll::i (ÀIA l l::i 0 1 + À20 l l::i ri + À3 r l l::i A I )(27)

The line ÀIA l l::i 0 1 + À2 0 l l::i r i + À3r l l::i A Iis an opt ical ray 01' the firsl camera (proposition 8), andwhen l varies in view 3, the point defined by (27) is we lldefined except when l is the image of that line in view3. In that case the meet in (27) is zero and the image ofthat line is in the null space of À n G ". O

No te that the proposition 22 is eq uiva lent to the van-ishing of the 10 coefficients of the homogeneous poly-nom ial of degree 3 in the three variables Àn , n =1, 2,3 equ al to ÀnGn ). The coefficients01' the terms n = 1,2 ,3 are the determ inant sdet(G n) , n = 1, 2, 3. Therefore the ex tended rank con-straints co ntai n the rank cons traints.

To be complete, we give the express ions of theseve n extended rank co nstra ints which are differentfrom the three rank constraints:

Proposition 23 The sevell extended rank const raintsare given by:

>.i>'2 1G: G ; 1+ I G: G ; G; 1+ 1G i G; 1= o(28)

),i),3 1G: 1+ 1G: G; 1+ I G; 1= o(29)

),; )" 1Gi G ; G ; 1+ I G i G; 1+ 1G : G ; G ; 1= o(30)

..\;..\ 3 1G i G ; I + I Gi G; 1+ 1 G; G ; 1= o(3 1)

),;),1 1 G; 1+ 1 1+ 1G : 1= o(32)

'\; ..\2 1 G; I + I G ; 1+ I G i 1= o(33 )

>' 1>' 2>'3 1G: G ; 1+ 1G: G; 1+ I G i Gg 1+1Gi G; 1+ 1 G; 1+ 1 G ; G ; 1= o

(34)

29

5.5 Constraints that characterize the tensor

We now show two results which are related to the ques-tion of finding subsets of constraints which are suffi-cient to characterize the trifocal tensors . These subsetsare the implicit equations of the manifold of the tr ifocaltensors. The first result is given in the following theo-rem :

Theorem 4 Let T be a bilinear mapping froni 1P'*2 x1P'*2 to 1P'*2 which sat isfies the fourteen rank, epipolarand vertical constraints. Then this mapping is a trifo -cal tensor, i.e. it satisfies definition 1. Those fourteenalgebraic equations are a set of implicit equations ofthe manifold of trifocal tensors.

The second result is that the ten extended constraintsand the epipolar constraints characterize the trifocaltensors :

With obv ious notations, we have in parti cular :

Let us now interpret this equation geornetrically: theline represented by the vector x i.e. the linegoing through the points xiI) and x àl ) belongs to thepencil of lines defined by the two lines represented bythe vectors Z I and Z2. Th erefore it goes through theirpoint of intersection represented by the c ro ss-productZI x Z2 and we write as a linear combination of(I) -

X I and Z I Z2:

= + (31ZI x Z2We write e2,1 for Z I x Z2 and note that our reasoningis valid for xin ) and x àn) :

Theorem 5 Let 7 be a bilinear mapping from 1P'*2 x1P'*2 to 1P'*2 which satisfie s the twelve extended rank andepipolar constraints. Then this mapping is a trifocaltensor, i.e. it satisfi es definition 1. Tho se twelve alge-braic equations are another set of implicit equations of

of trifocal tensors.

The proof of those theorems will take us sometime. We start wi th a proposition we will use to provethat the three rank constraints and the two epipolar con-straints are not sufficient to characterize the set of trifo-cal tensors :

Prop osit ion 24 lf a tensor T satisfies the three rankconstraints and the two epipolar con straints, then itsmatrices Gn, n = 1,2,3 can be written:

The same exact reasoning ca n be applied to the pairsn = 1, 2,3 yielding the expression :

Y (n) y (n) $;I = "Yn 2 + Un e3,1

We have exchanged the roles and for rea-sons of symmetry in the final expression of G n . Re-placing and by their values in the defin ition(36) of the matrix G n , we obtain :

G n ( + ) X(n)y(n)T + Ó X (n ) T + (3 y (n )T= Cl<n In 1 2 n 1 e 3 ,l n e 2 , l 2

We can absorb the coefficients ón in xin ) , the coefficients(3n in and we obtain the announced relation. O

[X (I) x X(1) X (2) x X(2) X (3) x X(3)]I 2 I 2 1 2

ZIT i + z2TI

where e2,1 (resp. e3,1) is a fixed point of image 2(resp . of image 3), the three vectors x(n) repres entthree points ofimage 2, and the three vectors y (n ) rep-resent thre e points of image 3.

Pro of : The rank constraints allow us to write:

(37)

P roof: In order to show this, we show that the ninevertical constraints imply that 7(121 ,131) = O for alipair of epipolar lines (l 21 , l31), i.e . for ali pairs of linessuch that l21 contains the point e2,1 and l31 containsthe po int e3,1 defined in (24) . Indeed, this implies thata n (1I1x (n)) . (y (n)TI3d = O for alI pairs of epipolarlines (121, 131) which implies an = Ounless either x (n)is identical to e2,1 or y (n ) is identical to e3,1 whichcontradicts the hypothesis that the rank of Gn is two.

In order to show this it is sufficient to show thateach of the nine co nstraints implies thatT (hl i , hlj ) =O, i, j = 1,2,3 where 12li (resp. 131j) is an epipolarline for the pair (1,2) (resp. the pair (1,3)) of cameras,

The next proposition is a proof of theorem 4 tha t thefourteen rank and epipolar con straints characterize theset of trifocal tensors:

P roposit ion 25 Let T be a bilinear mapping from1P'*2 x 1P'*2 to 1P'*2 which satisfies the fourteen rank ,epipolar and vertical constraints. Then its matrices G n

take the form:(36)

where the six vectors n = 1,2, 3 repre-sent six points of the second image and the six vectors

n = 1,2,3 represent six points of the thirdimage.

The right nullspace of Gn is simply the cro ss-product x the left nullspace being x

Those two sets of three nullspaces are of rank 2(proposition 21). Let us consider the first set. We canwrite the corresponding matrix as :

30

going the ith (resp. the jth) point of the canonical ba-sis o This is sufficient because we can assume that, forexample, e2 , ] does not belong to the line represented bye 3 . In that case , any epipolar line l2 1 can be representedas a linear combination of 12 1J and h12 :

Similarly, any epipolar line l 3J can be represented as alinear combination ofl311 and 13J2 , given that e3, 1 doe snot belong to the line represented by e 3 :

We now use the constraint (24) :

to replace the coefficient of the second term byÀ] À 3 À 4 1I À] À 2 À 4 I. The coefficient I À] À 3 À4is a factor and we have:

T( I À 2À3À4 1 ek2 - 1À ]À3À4 1 e12 ,

I À ]À3À4 I ekJ+ I À JÀ2À3 [ el J ) =I À IÀ3).4 I (I ). 2À 3).4 I À I- I ).1).3).4 1).2 +

I ).IÀ2).4 I ).3- I À IÀ2).3 I À 4)

The bilin earity of T allows us to conclude thatT(121 ,l3tl = O.

To simplify the notations we defin e:

To help the reader follow the proof, we encourage himto take the example k 2 = k3 = 1 and l2 = l3 = 2.If the tensor T were a tri focal tensor, the four lines)'1 , À2 , À3 , À4 would be the images of the 3D linesr 2 /'; r 3 , A 2 /'; r 3 , r 2 /'; A 3 , A 2 /'; A 3 , respectiv ely.

We now con sider the two lines di and d2 in im-age I which are defined as follows . dI goes throughthe point of intersection of À I and À2 (the image ofthe point r 2 /'; A 2 /'; r 3 ) and the point of intersec-tion of the lines À3 and À4 (the image of the pointr 2 /'; A 2 /'; A 3 ) . In our example, di is the imageof the projection ray r 2 /'; A 2 . d2 , on the other hand,goes through the point of intersection of À2 and À4 (theimage of the point A 2 /'; r 3 /'; A 3 ) and the pointof intersection of ÀJ and À 3 (the image of the pointr 2 /'; r 3 /'; A 3 ) . In our example, d2 is the image ofthe projection ray r 3 /'; A 3 . Using elementary geome-try, it is easy to find:

À] = T(ek2,ek3)À 3 = T(ek2 ,el3)

À 2 = T(eI2,ekJ

À 4 = T (el2, el 3)

The second factor is seen to be equal to O because ofCramer's relation.

We therefore have two set s of three line s, onein image 2 noted lau , i = 1,2,3, one in image 3noted l 3l j ,j = 1,2 ,3, corresponding to the choices ofk2 , l 2 , k 3 , l3 and such that T(hli , 131j ) = 0 , i , j =1,2 ,3. For example, one of the lines l 21 i is representedby I ).2 À 3 À 4 I ek 2- [ ).] À 3 ).4 . I el 2 and one ofthe lines l 31 j is represented by I À I ).3 ).4 I ekJ + [À] À 2 À 3 I el J •

Let us see what this means in terms of the linearapplications defined by the matrices G " . Consider thefirst line in image 3, l311 its image by G "' , 17, = 1,2 ,3is a point on the line l2' 17, = 1,2,3. According towhat we have j ust proved, those three points are a lsoon the thre e lines l 2l i , i = 1,2 ,3, see figure lO. Thisis only possible if a) the three lines laic, i = 1,2 ,3are identical, which they are not in general, or if b) thethree points are identical and the three lines go throughthat point. The second possibil ity is the correct one andimplies that a) the three points are identical with thepoint of intersection , e2,1, of the three Iines l;\ 17, =1,2,3 and b) that the three lines [2I i , i = 1,2 , 3 gothrough e2 ,1 ' A similar reasoning shows that the threelines l3 ] j , j = 1,2,3 go through the epipo le es.i .

d J =1 À 2 À 3 À 4 I À I - I À J À 3 À 4 I À 2d 2 =1 À J À 3 À 4 I À 2+ I À I À 2 À 3 I À 4

According to the definition of the lines À I , À2 , À3 , À4 ,di is the image by T of the two lines (I À 2 À 3 À 4 Iek2- I À ] À 3 À 4 I e12, ek 3) and d2 the image by T 01'the two lines (eI2' I À] À 3 À 4 I ek 3+ I À I À 2 À 3 1el J.

We now proceed to show that T (I À 2 À 3 À 4 Iek 2- I À I À 3 À 4 I e12 , I À I À 3 À 4 I ek3+I À] À 2 À 3 I e13 ) = O. Using the bilinearity of T , wehave:

T( I À 2À3À4 I ek2 - I À] À 3À4 I e12 ,

I À]À3À4 1ek3+ I À IÀ2À3 I e13) =I À 2À3À4 11 À]À3À4 IT(e k2,ekJ +[ À 2À3À4 1I À I À 2À3 IT(ek 2, e13) -IÀ]À3À4 11 À IÀ3À4 IT(eI2, e k3)-I À]À3À4 11 À]À2À3 1 T(eI 2 ,eI3)

Figure 10: The three lines lau , i = 1, 2, 3 are identical ,see the pro of of proposition 25 .

This completes the proof of the proposition and oftheorem 4. O

An intriguing que st ion is whether there are other sets ofconstraints that imply this parametrization, or in otherwords does there exist simpler implicit parametrizationof the manifold of trifocal tensors? One answer is con -tained in theorem 5. Before we prove it we prove twointeresting results , the first one is unrelated, the secondIS:

31

Proposit ion 26 AIlY bilinear II/(/ppillf; T wliich satis-fi es the 14 rank , epipolar {///(I vertical constraints alsosatisfies the 18 ro H' and CO I Il Il IllS constraint s.

Irwe use lhe paramctrizati on (35) and evalua te lheconstraints (28-33) , we find:

Proof: The proof consists in noticin g that if T satis-fies lhe rank , epipo lar and vertical constraint s, accord-ing lo prop osition 25 , it satisfies definit ion 1 and thcrc-fore, according lo theorern I 3, it satisfies lhe row andcolumn constraints. O

The reader may wonder about lhe tcn cx tcndcdrank const raint s: Are they sufficicnt lo charactcrizc lhelrilinear tensor? lhe following proposiii on unswcrs thisquesti on ncgativc ly.

Proposition 27 The teu extended rank const raints donot imply lhe ep ipo la r const raint s.

-(/"2 I e "2 ,1 X( l ) X ( 2 ) II e 3,1 y (l ) y( 2 ) I-(/3 I e "2 ,l X (I) X( 3) II e :l ,l y(l) y( 3) I- (/ 1 I e "2 ,l X (l) X ( "2 ) II e 3 , l y( l ) y (2 ) I- (/:1 I e"2,l X ( "2 ) X ( 3 ) II e :! ,l y ( 2 ) y P ) I- (f I I e2 ,l X (I ) X ( 3 ) II e :l , l y (l ) y (3 ) I- (/2 I e2 , l X ( "2 ) X (3 ) II e:u y ( :! ) y( :n I

(38)

(39)

(40)

(41 )

(42)

(43 )

Proof: The proof consi sts in cxhibiting a counterex-ample. The reader can verify that thc tensor T definedby:

[O O O] [ - 1 -1 O]G 1 1 -1 O G:!= O O O1 O 1 O 1 O

1 O 1 ]a [ O -1 OG =O O O

satisfies lhe ten extendcd rank constraints and that lhecorresponding three left nullspac es are the canonic linesrepresent ed by en , n. = 1, 2, 3 which do nol satisfy oneof the ep ipolar constra ints . O

Before we prove theorcrn 5 we prove lhe followingpropo sitio n:

Proposition 28 The th ree rank constraints and lhe twoep ipo lar constraints do 1101 cha racteri ze lhe set oftrifo-ca l tensors.

Proof: Indeed , proposition 24 gives us a parametriza-tion of lhe matrices Gn in that case. lt can be verifiedthat for such a parameri zation , the vertica l constraintsare not satisfied. Assume now that the rank and epip o-lar constrain ts imply that rhe tensor is a trifocal tensor,then, according lo propos ition 2, it satisfies lhe verticalconstra ints, a contradiction. O

We are now ready lo prove theorern 5:Proof r The proof consists in showing that any bilinearapplication T that satisfies the fivc rank and epip olarconstraints , i.e. whose matr ices G " can be written as in(35) and lhe remaining seven extended rank constraints(28-34) can be writl en as in (37) , i.e. is such that (/n =O, n = 1, 2,3 .

32

In those formul as, our attention is drawn lo deterrni-nants of lhe Iorm I e2 , 1 X {i ) X U ) I, -i i= j (type 2)and I e 3 , I y( i ) y (j) I. i i= j (type 3). The null ity ofa determinam of lhe first type impl ies that the epipolee2, l (resp. e3 , l ) is on the line defined by lhe twopoint s X( i ) , X (j) (resp. Y{i) , y (j)) , if the correspond-ing points are distinct .

If ali deterrninants are non zero , lhe constraints(38-43) imply that ali (/ n ' S are zero. Thin gs are slightlymore compli cated if some of the deterrninants are equallo O.

We prove that if lhe matrices G " are of rank 2, nomore than one of the three determinants of each of thetwo types can equal O. We consider several cases.

The first case is when ali point s of one type aredifferenl. Suppose first that the three points representedby the three vectors x (n) are not aligned. Then, hav-ing two of the determi nants of type 2 equal to Oimp liesthat lhe point e"2. ! is identic al to one of the points x( n )since it is at the intersection of two of the lines they de-fine. But , according to equation (35) , this implies thatthe corresponding matri x G " is of rank I, co ntrad ictingthe hypothesis that this rank is 2. Similarly, if the threepoints x (n) are al igned , if one determinant is equal to O,the epipole e2, 1 belongs to the line (X(I) , X( "2 ) , X (3 ) )

which means that.the three epipolar lines l1, areidentical contradicting the hypothesis that they form amatri x of rank 2. Therefore, in this case. ali three deter-minants are non null .

The second case is when two of the point s areequa l, e.g . X(I ) X (2). The third point must thenbe diffe rent, otherwise we would only have one epipo-lar line contradictin g the rank 2 assumption on thoseepip olar lines, and, if it is di fferent , the epipo le e 2 , I

must not be on the line defined by the two point s forthe same reason. Therefore in this case also at most oneof the determinants is equal to O.

Having at most one determinan t of type 2 and oneof type 3 equal to O implies th'at at least two of the a"are O. This is seen by inspecting the constraints (38-43).

11' we now express the seventh constraint :

ai 0,2 0,3 I y{l) y(2) y(3) 11 X(l) X(2) X(3) I -

(I e 2,1 X(l) X( 2) 11 e 3 ,1 y(l) y(3) 1+I e 3 ,1 y(l) y( 2) II e 2,1 X(l) X( 3) 1)0.1 +(I e:3,1 y(l) y(2) I1 e 2,1 X( 2) X (3 ) 1+I e 3 ,1 y( 2) y(3) 11 e 2 ,1 XCI ) X(2) 1)0.2 -

(I e 2 , 1 X( 2) X( 3) 11 e 3 ,1 y(l) y(3) 1+I e:l,l y(2)y( 3 ) I1 e 2 ,lX(J) ,X(3) 1)0.3 +(I e 2.1 X(l) X(2) I1 y(l) y(2) y(3) 1+

I e :3 ,1 y(J ) y(2) 11 X(J) X (2) X(3) l)al(L2 +(I e :l,l y (2) y (3) 11 X(J) X(2) X(3) 1+

I e2, 1 X(2) X( 3) 11 y (l) y(2) y (3) 1)0,20, 3 -

(I e2 ,J X(J) X(3) I1y(l) y(2) y(3) 1+IX(J) X(2) X(3) 11 e 3 ,1 y(J) y(3) 1)0.1(L3 ,

we (ind that it is equal to the third CLn multiplied by two01' the non zero deterrninants, implying that the third CLnis null and cornplcting the proof.

Let us g ive a few examples of the variouscases . Let us assume first that I e 2 ,1 X(l) X( 2) 1=I e:l ,l y(l) y( 2) 1= O. We find that the constraints(42), (43) and (41) imply 0'1 = 0,2 = CL:l = O.The sccond situation occurs if we assume for exampleI e 2 ,1 X( I) X( 2) 1=1 e 3 ,1 y(l) y( 3) 1= O. We find thatthe constraints (43) and (41) imply CL2 = 0,3 = O. Theconstraint (34) takes then the forrn:

and implies aI = O. O

Note that Irorn a practical standpoint, theorem 5 pro-vides a simple set 01' sufficient constraints than theorem4: The ten extended constraints are of degree 3 in theclcmcnts 01' T whereas the nine vertical constraints areof degree 6 as are the two epipolar constraints .

This situation is more or less similar to the onewith the E-ll1au'ix [Longuet-Higgins (1981 )]. It hasbcen shown in severaI places, for example in [Faugeras( 1993)] proposition 7 .2 and proposition 7.3, that the setof real E-matrices is characterized either by the twoequations:

or hy the nine equations:

In a somewhat analogous way, the set 01' trifocal tensorsis characterized either by the fourteen, rank, epipolarand vertical constraints (theorem 4) or by the twelveextended rank and epipolar constraints (theorem 5).

6 ConcIusion

We have shown a variety of applications of theGrassrnann-Cayley or double algebra to the problem ofmodeling systems of up to three pinhole cameras. Wehave analyzed in detail the algebraic constraints satis-fied by the trilinear tensors which characterize the ge-ometry of three views. In particular, we have isolatedtwo subsets of those constraints that are sufficient toguarantee that a tensor that satisfies them arises fromthe geometry 01' three cameras. Each of those subsetsis a set of implicit equations for the manifald of trifocaitensors. We have shown elsewhere [Faugeras and Pa-padopoulo (1998)] how to use some of those equationsto parameterize the tensors and estimate them from linecorrespondences in three views.

References[Avidan and Shashua, 1996] S. Avidan and A. Shashua. Ten-sorial transfer: Representation of n > 3 views of 3dscenes. In Proceedings of the ARPA /mage Understand-ing Workshop. darpa, morgan-kaufmann, February 1996.

[Barnabei e/ ai. , 1985] Marilena Barnabei, Andrea Brini ,and Gian-Carlo Rota. On the exterior calcu1us of invari-ant theory. Joumal ofAlgebra, 96:pp 120-160, 1985.

[Carlsson , 1994] Stefan Carlsson. Multip1e image invari-ance using the double algebra. In loseph L. Mundy, An-drew Zissermann, and David Forsyth, editors, Applicationsof Invariance in Computer Yision, volume 825 of Lec-fure Notes in Computer Science , pages 145-164. Springer-Verlag, 1994.

[Doubilet et aI., 1974] Pete r Doubilet, Gian-Carlo Rota, andlo el Stein. On the foundations of cornbinatorial theory :Ix combinatorial methods in invariant theory . Studies inApplied Mathematics, 53:pp 185-216, 1974.

[Eklundh , 1994] 1-0. Eklundh, editor. volume 800-801 ofLecture Not es in Computer Scien ce, Stockholm, Sweden ,May 1994. Springer- Verlag.

[Faugeras and Mourrain, 1995a] Olivie r Faugeras andBernard Mourrain. On the geometry and algebra ofthe point and line correspondences between n .images.In Proceedings of lhe Stli /nternational Conference onComputer Vision [1995], pages 951-956.

[Faugeras and Mourrain , 1995b] Olivier Faugeras andBernard Mourrain. On the geometry and algebra ofthe point and line correspondences between n images .Technica1 Report 2665, INRIA, October 1995.

[Faugeras and Papadopoulo, 1998] Olivier Faugeras andThéodore Papadopoulo. A nonlinear method for estimat-ing the projective geometry of three views. In Proceedingsof lhe ôth lnternational Conference on Compute;' Yision,Bombay, India, lanuary 1998. IEEE Computer SocietyPress.

[Faugeras, 1992] Olivier Faugeras . What can be seen in threedimens ions with an uncalibrated stereo rig. In G. Sandini,editor, Proceedings of the 2nd Europ ean Conference onComputer Vision, volume 588 of Lecture Notes in Com-put er Science, pages 563-578, Santa Margherita Ligure,Italy, May 1992. Springer-Verlag.

33

IFaugcrn s, I')<)31 OI ivicr Faugcr,» . Thrcr -Dinu-nsionul('1I1I/11If11'l" v is ion : a G r om ct ri« v icwpoi nt. M IT Prcss.1<)<)3.

IHartlc y. 1<) <)41 Richard Hartlcy. Lincx and pointx in thrccvicws-un intcgr.ucd appruach. In Pmu 'l'!lill gs IIFtIIl' AUPAIII/age Undrrstand ing Wllrkshll l l . Dclc nsc Advanccd Re-scarch Projccis Agcncy. Morga n Kaulmann Puh lishcrs .l nc.. 1<)<)4.

1Hart lcy. 1<)<)5 / R.1. Hnrtlcy. In dc lcncc 01' thc X-point algo-rithm, In Proccctlings 111' lhe 5 1h lntcrnntional ('1I1I/á" IIC{ '0 11 Cmnputcr vision I1 <)')51. pages 10(,4-1 070.

IHanley. I <)<)71 Richard I. Hart lcy. Lincx and points in thrccvicws und lh e tri tocal tcnsor. Thc 11I/1'l'I/ oliOl /(/1 .lournal 111'('O//(II I/II' r visiou, 21(2): 125- 140. M arch 1<)<)7 .

licc.I <) <)51 Boston, MA. .IUI1l: 1<) <)5. IEEE Computcr Soc ielyPrcss,

1Longuct-H iggi ns. I 'JX11 H .C. Longuct- Higg ins. A com-put cr algorirhm ror rcconstruc ring a sccnc lrom two pro-jcc tions. Nature, 293: 133- 135. I <)XI .

[Luong and Faugcra s, 1<)95/ Quang-Tuan Luong und OlivierFuugcras. T hc fundamental matri x: thcory, algorilhms.nnd stability ana lys is. Thc lnt rrn ational .I11/(l'I/a l 111' ('1111/ -

I I/ /Ii'r vision, 17( 1):..B -7CJ. January 19<)5 .

[Luong and Viévi ll e. 1<)94/ Q .-T Lu ong and T V i évi l!c.Cunonic rcprcscniati ons ror lhe gcomctri cs 01' muhi plcprojccii vc vicws, In Eklundh I 1<)<)41 . pugcs 5X<)- 5'J'J.

[Shashua and Wcrmun . 19<)51 A. Shashua and M. Wcrman.On thc trilinca r tensor 01' ih rcc pcrspcctivc vicws and irsundcrlying gcornctry. In Pro cccdings oI' lhe 51h 11111'1"110 -tional Co njercncc 011 C01l1111f11'l" vis ion 11995/.

[Shashua, 19941 A mnon Shashua. Tr i lineari ty in visualrcco gni ti on hy al ig umcnt. In Eklundh 119941. pugcs 47<)-484.

IShashuu. I <)951 A mnon Shushua. A lgcbrai c tuncuons rorrccogniti on. 11':/:'/:' Tm nsact ions 0 11 Pott crn /vna lvsis atu!Machin« lnt elligcnce , 17(X):77<)- 7X<) . 1<)<)5.

1Spetsakis and Aloi rnonos. I<)90aI Minas E. Spcrsaki s andJohn A lo imonos. Structurc trorn Moiion Using L inc Cor-rcspondcnccs . Tlt e ln ten uuional Journal oI' ('Oll/I I /(I,' r Vi-sion , 4: 171- 183. 1990 .

[Spctsaki s und A loimonos. 1990b l M inas E. Spctsakis undY. A lo imono s. A unifi cd thcory 01' structurc írom motion.In Proc. DARPA lU workshop: pagcs 27 1-283. 19<)().

ITorr und Zisscrmann, 19971 P.H .S. Torr and A . Z isscrrn unn.Perfo rmance charactcrizarion 01' fundamental matrix csii -mation undcr imagc dcgradation. Machinc vision and A I I -plications, 9:32 1- 333. 1997 .

[Wcng et a i.. 19921 J. Weng. TS . Huang. and N. A huja. M o-tion und stru cturc I rorn line co rrcspondcnccs: Closcd-Ionnsolution. uniqucncss and optimization. Transa r-tions 0 11 Pattern Atialvs is and Machine IlIlelligellce . 14(3 ).March 1992.

[Zhang et ai.. 19951 Z . Zhang, R. Dcr ichc , O. Faugc rus. andQ.-T Luong. A robust tcchniquc ror matching two uncali-brated images through lh e rccovcry 01' lh e unknown cpipo-lar geomctry. Arlificial Int el ligen ce Jou rnal, ?ll: 87- 119.Octobcr 1995.

34