Computer Vision and Image Understanding 114 (2010) 245–253

Contents lists available at ScienceDirect

Computer Vision and Image Understanding

journal homepage: www.elsevier .com/ locate/cviu

Estimation of the epipole using optical flow at antipodal points

John Lim *, Nick BarnesResearch School of Information Sciences and Engineering, Australian National University, Canberra ACT 0200, AustraliaNICTA, Canberra Research Laboratory, Australia

a r t i c l e i n f o a b s t r a c t

Article history:Received 10 March 2008Accepted 21 April 2009Available online 22 July 2009

Keywords:Egomotion estimationOmnidirectional camerasMulti-view Geometry

1077-3142/$ - see front matter � 2009 Elsevier Inc. Adoi:10.1016/j.cviu.2009.04.005

* Corresponding author. Address: Research SchoolEngineering, Australian National University, Canberra

E-mail addresses: john.lim@rsise.anu.edu.au (J. Lim(N. Barnes).

We present algorithms for estimating the epipole or direction of translation of a moving camera. We useconstraints arising from two points that are antipodal on the image sphere in order to decouple rotationfrom translation. One pair of antipodal points constrains the epipole to lie on a plane, and two such pairswill correspondingly give two planes. The intersection of these two planes is an estimate of the epipole.This means we require image motion measurements at two pairs of antipodal points to obtain an esti-mate. Two classes of algorithms are possible and we present two simple yet extremely robust algorithmsrepresentative of each class. These are shown to have comparable accuracy with the state of the art whentested in simulation under noise and with real image sequences.

� 2009 Elsevier Inc. All rights reserved.

1. Introduction

A monocular observer moving in a scene of unknown depthundergoes a rigid motion that is a combination of translationaland rotational motions. We focus on estimating these motionsusing image motion measurements, or optical flow. This is a classi-cal problem and the last few decades of research has produced avast number of self-motion estimation methods. These includethe well-known epipolar geometry formulation of [1,2] whichleads to algorithms such as the 8-point [3], 6-point [4] and 5-pointalgorithms [5,6]; methods involving nonlinear optimization [7,8];qualitative search methods [9,10] and the recovery of a ‘flow fun-damental matrix’ from optical flow [11]. Other approaches of noteinclude [12–15] and a great many others, which can be found in re-views such as [16,17].

However, most approaches assume the use of a planar imagewith a limited field-of-view (FOV) such as that found in traditionalcameras. Although omnidirectional cameras have become widelyavailable of late, few self-motion estimation algorithms actuallyexploit the large FOV property explicitly in order to aid or simplifythe task. [9,18] are examples of some such prior work. However,the method proposed here is significantly more efficient comparedto those algorithms, which solve the problem via a search.

This paper expands on our recent work in [19], where we pre-sented a method for estimating the direction of translation or theepipole of the camera based on the geometrical properties ofpoints that are antipodal on the image sphere. The image sphere

ll rights reserved.

of Information Sciences andACT 0200, Australia.), Nick.Barnes@nicta.com.au

is simply a more natural representation for the images of omnidi-rectional cameras. From the optical flow at two antipodal pointswe were able to constrain the epipole to lie on a great circle (thatis, the intersection of a plane through the camera center with theimage sphere). The rotational contribution to the measured opticalflow was geometrically eliminated, leaving an equation dependentonly on the translational component of motion. These constraintswere then used to estimate the location of the epipole.

Whilst our method does not directly estimate rotational mo-tion, we will show that once direction of translation has beenfound, a least squares estimate of rotation that is robust to outlierswould not be difficult to recover. Note that due to the use of antip-odal points, this algorithm is, by its very nature, suited for omnidi-rectional sensors and large FOV cameras.

A somewhat similar antipodal point constraint was observed in[20]. However, significant differences in the theoretical derivationand in the resulting constraint exist between the method of [20]and our work (Section 2.1). Furthermore, as the authors of [20]noted, the lack of widely available omnidirectional sensors at thattime (1994) meant that such a constraint was hardly of any prac-tical use then. As a result, their constraint was merely observedas an equation and was not investigated further. The emergenceof omnidirectional cameras as a popular tool in computer vision to-day warrants an investigation into methods that specifically ex-ploit the large FOV of these sensors.

Our constraint is somewhat simpler to derive and use comparedto [20] and it succeeds in certain scene depth configurations where[20] fails. Both the constraint presented here and that of [20] maybe thought of as special cases of the linear subspace methodsinvestigated by Jepson and Heeger [12].

The method proposed here is also related to the approach of[21] which utilizes antipodal points as well. However, whereas

246 J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253

our approach is based on eliminating the rotational component offlow in order to constrain translation, the method of [21] obtainsconstraints without eliminating rotation. As a result, the methodof [21] obtains constraints on translation and on rotation, whilstthe method proposed here constrains translation only (rotation isfound via a second step). However, this also means that the con-straint on translation in [21] is weaker than the one proposed here.

Section 2 begins with recapitulating the theory presented in[19]. In Section 3, we identify two basic classes of algorithms utiliz-ing the antipodal point constraint – point-based algorithms andline/curve-based algorithms. We then present two novel algorithmsrepresentative of each class. The first is an algorithm using the de-rived constraint within a Random Sample and Consensus (RANSAC)[25,26] framework. The second algorithm performs Hough-remi-niscent voting [22–24] along the great circles that constrain thedirection of translation. It runs faster than the first algorithm butis just as robust to noise and outliers.

In Section 4, we compare the performance of our two algo-rithms with what is often considered the state-of-the-art for self-motion estimation in calibrated cameras – the 5-point algorithm[5,6] within a RANSAC framework. Comparisons were done usingMatlab simulations under noise, and also using noisy real imagesequences that involved independently moving objects in thescene. In Section 5, we show that our method is just as accurateas 5-point with RANSAC, with the added advantages of having im-proved robustness to outliers, constant run-time with increasingnoise and outliers (for the voting algorithm), and a naturally paral-lelizable framework which makes the method potentially viable foregomotion estimation at high speeds.

1.1. Background

For an image sphere, the equation relating the rigid motion ofthe camera with image motion is given in Eq. (1) [27]. At an imagepoint r (refer Fig. 1), the optical flow _r resulting from the transla-tional motion t and the rotational motion w is given by:

_r ¼ 1jRðrÞj t � rð Þr� tð Þ �w� r ð1Þ

The self-motion estimation task attempts to recover the transla-tional and rotational motions, where it is well-known that thetranslation can only be recovered up to a scale [2]. A least squaressolution is not possible since the scene depth, R, is a function of rand the system of equations is under-constrained. Without any

aFig. 1. For some translational motion, t and some rotation about the axis w, at pointr on the image sphere, with scene depth R, the optical flow is _r and the flow vectorlies on the tangent plane to the sphere at that point.

knowledge of depth, recovering the five unknown motion parame-ters is difficult.

In this work, the constraints described will recover the directionof translation or the epipole. The epipole can be defined as theintersection of the line joining the two camera centers with the im-age sphere [2]. The second camera center is related to the firstcamera center via some translation, so the direction of translationand the epipole are equivalent.

Note that the above equation is an approximation that onlyholds for small baselines and small rotations. This assumption isgenerally true in image sequence videos, where the motion be-tween frames is small. Section 6 discusses how our algorithmcan be implemented to give sufficiently real-time, high frame rateestimates of translation for such videos.

In this paper, we use the term ‘optical flow’ to refer to any mea-surement or approximation of image motion. This includes imagevelocities obtained from feature correspondences such as SIFT[28] or Harris corners, as well as image velocity fields obtainedusing methods like Lucas–Kanade [29] or Horn and Schunk [30].For the former, point correspondences are found and matched fortwo images and the velocity vector transforming one point to theother is calculated. The method presented here works with bothclasses of image motion measurements.

2. Removing rotation by summing flow at antipodal points

On the image sphere, the optical flow, _r1 and _r2, at two antipo-dal points, r1 and r2, can be written as:

_r1 ¼1

jRðr1Þjðt � r1Þr1 � tð Þ �w� r1 ð2Þ

_r2 ¼1

jRðr2Þjðt � r2Þr2 � tð Þ �w� r2

¼ 1jRð�r1Þj

ðt � r1Þr1 � tð Þ þw� r1 ð3Þ

since r2 ¼ �r1 if they are antipodal. By summing Eqs. (2) and (3), wehave an expression that arises purely from the translational compo-nent of motion. The rotational components cancel out.

_rs ¼ _r1 þ _r2 ¼1

jRðr1Þjþ 1jRð�r1Þj

� �ðt � r1Þr1 � tð Þ

¼ K ðt � r1Þr1 � tð Þ ð4Þ

From Eq. (4), we see that vectors r1; _rs and t are coplanar. The nor-mal of that plane is given by r1 � _rs, where � denotes the vectorcross product. The epipole or direction of translation, t, lies on thatplane. The intersection of that plane with the image sphere gives agreat circle. See Fig. 2a for an illustration.

By picking another pair of antipodal points (that do not lie onthe first great circle) and repeating, we obtain a second such planeor great circle. The intersection of the two planes or great circlesgives an estimate of the epipole, t. See Fig. 2b for an illustration.

Of course, the intersection of the two great circles actuallyyields two points corresponding to t and �t. To disambiguate be-tween the two, one may pick one of the two points and calculatethe angle between it and the vector _rs. If the angle is larger thanp=2 radians, then that point is t. Otherwise, it is �t.

2.1. Comparison with the constraint of Thomas and Simoncelli [20]

The underlying principle here was first observed in [20]. How-ever, major differences differentiate this work from that of [20].The approach used there defined angular flow, which is obtainedby taking the cross product of optical flow at a point with thedirection of that point. In effect, a dual representation of flow is

a bFig. 2. (a) Summing the flow _r1 and _r2 yields the vector _rs , which, together with r1 (or r2), gives rise to a plane on which t is constrained to lie. The intersection of that planewith the sphere is the great circle C. (b) From two pairs of antipodal points, two great circles C1 and C2 are obtained. t is the intersection of these two circles.

J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253 247

obtained and [20] shows that if the angular flow at two antipodalpoints was subtracted from each other, the rotational componentwould vanish.

Both the constraint developed here and the one in [20] probablystem from a similar geometrical property inherent to antipodalpoints but the methods by which they were derived and the finalresults are quite different. The constraint in this paper does not re-quire the transformation of optical flow into angular flow as in[20], a step which incurs an additional cross product.

Furthermore, the method of [20] requires a subtraction of angu-lar flow at antipodal points and thus, that method fails when theworld points projecting onto the antipodal points on the imagingdevice are at equal distances from the camera center (subtractingthe angular flow in that case yields zero) – a problem observedby the authors of that paper. An example of this would be the dis-tances to the two opposite walls when the camera is exactly in themiddle of a corridor. The method presented in this paper sums theoptical flow at antipodal points and therefore, does not encountersuch instabilities when antipodal scene points are at equal, or closeto being at equal distances.

Finally, because omnidirectional and panoramic sensors werenot widely available then, the method of [20] was not fully devel-oped into an algorithm and no implementations or experimentswere ever conducted. Here two complete algorithms are presented,tested on simulations under increasing noise and outliers, anddemonstrated to work on fairly difficult real image sequences.

3. Obtaining robust estimates of t

We now have a method for obtaining some estimate of thedirection of t given the flow at any pair of antipodal points. In thissection, we outline methods for doing this robustly. Two classes ofapproaches are possible and we present an algorithm representa-tive of each class in Sections 3.1 and 3.2.

Class 1: Point-based – Firstly, the flow at two pairs of antipodalpoints gives a unique (up to a scale) solution for the location ofthe epipole from the intersection of two great circles. Repeatedlypicking two pairs from a set of N antipodal point pairs can giveup to NC2 ¼ N!

2!ðN�2Þ! possible solutions, where each candidate solu-tion is a point on the image sphere. We loosely term the class ofmethods that estimates the best solution from a set of point solu-tions, point-based algorithms.

Section 3.1 presents an algorithm representative of this classthat finds the solution point best supported by the optical flowdata using the RANSAC framework. The maximum-inlier methodused in [19] is also a member of this class. Moreover, since the

points all lie in a Lie-group (which a sphere is), the work of [31]shows that the mean shift algorithm [32] may also be used for find-ing the mode of the points. K-means is another possibility for clus-tering points on a sphere [33]. Other algorithms in this classinclude robust estimators such as [34] and variants of RANSACsuch as [35–38].

Class 2: Line/Curve-based – Alternatively, since the flow at everypair of antipodes yields a great circle which must intersect t, theepipole could be estimated by simultaneously finding the bestintersection of many of these great circles. We will later show thiscan be reduced to the problem of intersecting straight lines. Onceagain, many algorithms exist, including linear and convex pro-gramming. We observe that the problem is quite similar to thatof intersecting many vanishing lines to find the vanishing point[42–44], and inspired by a popular solution in this area, we use aHough-reminiscent voting approach. There are of course myriadvariations to Hough voting (such as [22–24,39–41]) and we pres-ent an algorithm representative of the class which is not necessar-ily the optimal solution, but which nevertheless works very well inthe experiments.

Algorithms that are a combination of the two classes are alsopossible, such as finding the best supported solution with RANSAC,and then doing linear programming using the inlier flow measure-ments that have been found.

3.1. Antipodal constraint + RANSAC

Algorithm 1. (Estimating the epipole – antipodal + RANSACalgorithm)

1:

Set M ¼ 1; count ¼ 0 2: while M > count AND count < MAX do 3: Select a pair of antipodes r1; r2 with optical flow _r1, _r2. 4: _rs ¼ _r1 þ _r2 and n1 ¼ _rs � r1

5:

Repeat for a different pair of antipodes to obtain n2 such thatn1 – n2

6:

Take cross product t̂ ¼ n1 � n2

7:

for i ¼ 1 to N do 8: For the ith antipodal pair, find the plane normal ni

9:

If p2 � cos�1ðni � t̂Þ < hthres , increment support for t̂

10:

end for 11: Update M, increment count 12: end while 13: The best supported t̂ is the solution.

Fig. 3. Projection of a point rsph which lies on the image sphere, to a point rpln whichlies on the plane that is tangent to the sphere at point n.

248 J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253

In our previous paper [19], we presented a consensus style algo-rithm which calculated a measure of the density of the ‘cloud’ ofcandidate solution points at each point, and used points with thehighest density to obtain the best solution. The idea is that the moresolutions there are that are ‘close’ to a particular candidate solution,the more likely it was that the candidate was a good solution.

Here, we use a different approach, where the quality of a solu-tion is measured not by the number of nearby candidate solutions,but by the proportion of data points (i.e. optical flow measure-ments) that fit the solution. Algorithm 1 summarizes our use ofthe constraint within the RANSAC sampling framework.

Each pair of antipodal flow vectors yields a plane (or great cir-cle) on which the true solution must lie. Step 9 in Algorithm 1 findsthe angle between the candidate solution and such a plane. If thatangle is less than some threshold, hthres, we consider that pair ofantipodal flow vectors as data which supports the candidate.

M is the number of iterations of the algorithm and is calculatedas:

M ¼ logð1� pÞlogð1�wsÞ ð5Þ

where p is the probability of choosing at least one sample of datapoints free of outliers, w the inlier probability, and s is the numberof data points required to obtain a candidate solution.

N is the number of antipodal flow measurements available. MAXis the upper limit on number of iterations to guarantee terminationof the algorithm. If several solutions have the same maximum sup-port, then some mean of their directions can be used instead. De-tails of the RANSAC framework may be found in [2,25].

The method is quite simple but very robust, as the results inSection 5 will show. However, some tuning of the parameters is re-quired for best results. Furthermore, the processing time increasesquite quickly as the probability of outliers and noise in the data in-creases1. If processing time is critical, then a more efficient methodis necessary, which leads us to the voting approach in the nextsection.

3.2. Antipodal constraint + voting

Algorithm 2. (Estimating the epipole – antipodal + votingalgorithm)

1:

1 Foincreaprobabof outlcan bepaper

for i ¼ 1 to Mcoarse do

2: Select a pair of antipodes r1; r2 with optical flow _r1; _r2. 3: _rs ¼ _r1 þ _r2

4:

The sphere center, vectors _rs and r1 (or its antipode, r2) define agreat circle, Ci on the image sphere.

5:

The spherical coordinates ðh;/Þ of points on great circle Ci arecalculated and coarse voting is performed.

6:

end for 7: Find the bin with maximum votes. This is the coarse estimate. 8: Choose a projection plane at or close to coarse estimate. 9: for j ¼ 1 to Mfine do 10: Repeat steps 2–4 to obtain a great circle Cj. 11: Project Cj onto the projection plane to obtain a straight line Lj. 12 Perform fine voting by voting along the straight line Lj. 13: end for 14: Find the bin with maximum votes. This is the fine estimate.

r any estimation technique, the difficulty of finding a high quality solutionses with the percentage of outliers. If an accurate estimate is sought with highility, RANSAC and its variants will show increasing time with large proportionsiers, as exemplified in this paper by the standard approach (see Section 5). Thismitigated by trading accuracy for processing time to some extent, but in this

we seek to compare the ability to find a high quality solution.

Another approach to finding a robust estimate of translationdirection, t, was to simultaneously find the best intersection ofall the great circles arising from our method of summing flow atantipodal points. An efficient way of doing this is via a Hough-rem-iniscent voting method as summarized in Algorithm 2.

Our implementation uses coarse-to-fine voting. First, we pickthe flow at Mcoarse pairs of antipodes and obtain Mcoarse great circleswith the steps detailed in Section 2. Having divided the imagesphere into a two-dimensional voting table according to the azi-muth-elevation convention, we proceed to vote along each greatcircle. This is the coarse voting stage and the area on the sphererepresented by a voting bin can be fairly large. The area with max-imum votes is then found, giving us a coarse estimate of transla-tion, tcoarse.

For the fine voting stage, a region of the image sphere in theneighborhood of the coarse estimate is projected onto a plane thatis tangent to a point on the sphere (refer Fig. 3). The point can be atthe location of the coarse estimate, tcoarse, found earlier or any-where near it. The center of projection is the sphere center andthe mapping will project points on the surface of the sphere ontothe tangent plane. Let the sphere center be the origin and rsph bea point that lies on the unit image sphere. If the plane is tangentto the sphere at a point n, then n is also a unit normal of that plane.In that case, the projection of rsph onto the plane is a point, rpln, thatlies on the tangent plane and is given by rpln ¼ rsph=ðrsph � nÞ.

This is called a gnomonic projection [45]. This projection willmap great circles on the sphere onto straight lines on the plane.This is an advantageous mapping since very efficient algorithmsfor voting along straight lines exist, such as the Bresenham linealgorithm [46]. This is the motivation for performing the fine vot-ing stage on a projection plane rather than on the image sphere.We vote along Mfine straight lines that have been obtained by pro-jecting Mfine great circles onto the plane, and the point with maxi-mum votes represents the best estimate of the intersection of allthe great circles. If necessary, this can be repeated for progressivelyfiner scales such as with the schemes proposed by [39,41].

One possible variation to this algorithm involves ‘blurring’ outthe votes for the straight lines in a Gaussian fashion. This meansthe highest votes are cast along the center of the line, with thevotes falling off in a Gaussian manner in the direction normal tothe line. Furthermore, many optical flow calculation methods re-turn a covariance matrix which gives an uncertainty measure ofthe accuracy of the calculated flow. Hence, straight lines resultingfrom flow with high accuracy can be given higher weights duringvoting compared to lines arising from flow of uncertain accuracy.In our experiments however, we found that the unmodified versionof the algorithm was sufficiently robust and accurate and we didnot require any of these extra measures.

J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253 249

4. Experiments-simulations and real videos

Both the algorithms presented here were tested in Matlab sim-ulations and with real image sequences. We compared the perfor-mance of our method against the 5-point algorithm within aRANSAC framework [5,6].

We chose 5-point with RANSAC as a comparison since it is well-known, well-studied, widely used and code is widely available. Weused the five-point algorithm of [5] (code available from author)and it was implemented within a RANSAC framework using codefrom [47]. Sampling was adaptive with probability p ¼ 0:99 andSampson distance threshold of 0.01 (see [2] for details).

The error in an estimate of the translation direction was mea-sured as the angle between the recovered motion direction andthe true motion (known in simulations and measured in the realvideos).

4.1. Matlab simulations

We simulated the camera to simultaneously rotate and trans-late such that an optical flow field was generated. Since the imag-ing surface is a sphere and flow is available in every direction, wecan fix the direction of translation without any loss of generality asthe result would be the same in any direction. The axis of rotationhowever, was varied randomly from trial to trial since the anglemade between the direction of translation and axis of rotation doesin fact influence the outcome. Baseline was two units and the rota-tion angle 0.2 radians.2 500 pairs of random antipodal scene pointswere uniformly distributed in all directions with depth ranging from10 to 15 units. Results were averaged over 100 trials.

In the simulations, 5-point used the point matches as inputwhilst our method took the flow vectors as input. Experimentswere conducted for increasing probability of outliers (with noGaussian noise) and also for an increasing level of Gaussian noise(with no added outliers). Outliers were simulated by randomlyreplacing matches or flow with errors. To simulate Gaussian noise,the position of a matched point was perturbed from its true valueby a noise vector modeled as a 2D Gaussian distribution with stan-dard deviation r and zero mean.3

4.2. Real videos

Real sequences were captured with a Ladybug camera [49]which returns five images positioned in a ring. These are mappedonto an image sphere according to a calibration method suppliedby Point-Grey Research, the makers of the camera system. Thecamera translated along the ground with baseline 2 cm per framein the direction of the x-axis, which is parallel to the ground plane.It simultaneously rotated about the z-axis, which is perpendicularto the ground plane. The results in Section 5 are for rotation anglesof 5� per frame and 2� per frame. The former case involves the cam-era moving in a static scene whilst the latter involves the camera

2 If the motion is too small, 5-point may become less stable. By trial and error, wepicked a motion size for which the operating range of both methods overlap. Theerrors observed were small (Fig. 6) so both methods appear to be working stably andproperly for the motions used. Also, in the real image experiments for 5-point, weskipped every alternate frame so that the motion was twice as large compared to thatused for our method.

3 Note that this noise model is different from that used previously in [19].Previously, following [48], we modeled noise as a perturbation in only the angle of theflow vector. In this paper the noise model perturbs both angle and magnitude of theflow vector. Thus, for the same r, the level of noise is roughly higher in this papercompared to in [19].

moving in a scene that contains multiple independently movingobjects.

For our algorithms, optic flow was calculated using the iterativeLucas–Kanade method [29] in pyramids using code available fromOpenCV [50]. The iterative pyramidal scheme is a multi-scalerefinement process that calculated flow quickly and accurately gi-ven a list of antipodal points. An example of the typically noisyflow found is shown in Fig. 4. Meanwhile, the 5-point with RANSACmethod obtained point matches using Scale Invariant FeatureTransform (SIFT) feature matching with code from [51].

Let us label the ring of five cameras on the Ladybug camera rig ascam0, cam1, . . . , cam4, thus completing the ring. Fig. 4 shows twoimages taken from cameras on the Ladybug multi-camera rig thatwere facing different directions. Blue lines pair up antipodes whereflow (red vectors) was found stably at both points. The blue linesshow the pairing of points in cam2 with their antipodes in cam4.

Note that Fig. 4 only shows the antipodal pairs for cam2 andcam4. More antipodes for the cam2 image exist in cam0, and moreantipodes for cam4 exist in cam1. Fig. 5a shows the flow at pointsin cam2 that had antipodes in cam0 and cam4. Fig. 5b shows theflow at points in cam4 that had antipodes in cam1 and cam2.

Flow vectors with large error (the flow algorithm returns anuncertainty measure) were thrown away, but a few obvious mis-matches remain. However, most of the flow appears reasonablystable. Over the entire view-sphere, for these experiments, flowwas found for about 500–700 pairs of antipodes, which is generallymore than our algorithms need for a good estimate.

Previously, in [19], we used SIFT matching as an input for themaximum-inlier algorithm presented there. However, SIFT can beslower than Lucas–Kanade flow and is less suitable for real-time,parallel implementations. Therefore, although SIFT tends to bemore accurate compared to the noisier, pyramidal Lucas–Kanade,we chose to use the latter in this paper. In spite of this, Section 5will show little degradation of the results compared to that re-ported in [19], which reinforces the point about the robustnessof the methods.

In terms of a comparison on the accuracy of 5-point with RAN-SAC (using SIFT) versus the antipodal point methods (using Lucas–Kanade), this gives a slight disadvantage to the antipodal methodssince their inputs are noisier, but as we shall see in the results ofSection 5, the performance of both approaches on real images isroughly equivalent.

5. Results

The results summarized in Fig. 6 demonstrate that the antipo-dal + RANSAC and antipodal + voting algorithms, which are basedon the sum of flow at antipodal points constraint, work accuratelyand robustly in simulation and with real image sequences under avariety of circumstances.

5.1. Outliers

Outliers are large errors in flow data due to causes such as pointmismatches and the presence of objects moving independently inthe scene. The RANSAC framework for 5-point is known to be aneffective means of separating the inliers and outliers in a dataset, so that the estimation is performed only with inliers. However,the simulation result of Fig. 6a shows that both our algorithms per-form better than 5-point with RANSAC when faced with an increas-ing proportion of random outliers in the data. This is an interestingresult and there are several reasons for it, as discussed in thefollowing.

The Hough-like voting algorithm is extremely resistant to outli-ers. Fig. 6c shows the votes cast in a voting table for 60% random

Fig. 4. Blue lines indicate corresponding antipodes in two cameras (cam2 and cam4) facing different directions. Flow vectors shown in red. (For interpretation of thereferences to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 5. (a) All the antipodal flow found for the cam2 image using views from cam0 and cam4. (b) All antipodal flow found for the cam4 image using views from cam1 andcam2.

250 J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253

outliers in the data. The figure shows the straight lines obtainedfrom projecting great circles onto a plane during the fine-votingstage of the antipodal + voting algorithm detailed in Section 3. In-lier straight lines intersect at a common point whilst the outlierstraight lines do not. Most of them fall outside the region of theplane shown in the figure but quite a few outliers can still be seen.From the figure, it is clear that the intersection can easily be foundeven for this high proportion of outliers.

The antipodal + RANSAC framework worked better because itneeded only two points for a constraint as opposed to five pointsfor the 5-point-RANSAC algorithm. If q is the probability of an in-lier, then the probability of obtaining a good constraint is q2 forthe antipodal method and q5 for 5-point. Therefore, both antipodalmethods are always more likely to obtain more good constraintswhen flow at antipodal points is available. Here, errors in RANSACarise because some outliers fall within the model distance thresh-old used (0.01 for 5-point-RANSAC). Reducing distance threshold

improves performance; but in general, the antipodal methods tendto outperform 5-point-RANSAC for very large outlier proportions.

5.2. Gaussian noise

Fig. 6b shows the error in estimated translation direction underincreasing Gaussian noise. The trend is that all three methods de-grade gracefully with increasing noise. For large Gaussian noise,the antipodal point methods tended to perform better than 5-point-RANSAC since outliers would begin to creep into the dataat that stage.

5.3. Processing time

An interesting property of the antipodal + voting algorithm is itsconstant processing time. The number of antipodal points consid-ered is predetermined, and the processing time is quite fast and

0 0.1 0.2 0.3 0.4 0.50

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Outlier Proportion

Erro

r (de

g)Translation Error vs Outliers

5pt RANSACantipodal+votingantipodal+RANSAC

0 1 2 3 4 5 6 70

2

4

6

8

10

12

14

Std Dev (deg)

Erro

r (de

g)

Translation Error vs Gaussian Noise

5pt RANSACantipodal+votingantipodal+RANSAC

0 20 40 60 80 100 1200

20

40

60

80

100

120Voting table (50 points, 60% outliers, no Gaussian noise)

0 0.1 0.2 0.3 0.4 0.50

20

40

60

80

100

120

140

160

180

200

Outlier Proportion

Tim

e (s

ec)

Run time vs Outliers

5pt RANSACvoting

0 2 4 6 8 10 12 14 160

1

2

3

4

5

6

7

8

9

10

Frame No.

Erro

r (de

g)

Translation Error for Real Seqeunce

antipodal+votingantipodal+RANSAC5pt RANSAC

0 2 4 6 8 10 12 14 160

1

2

3

4

5

6

7Real Sequence (independently moving objects)

Frame No.

Erro

r (de

g)

antipodal+votingantipodal+RANSAC5pt RANSAC

a b

c d

e f

Fig. 6. (a) Error in estimated epipole as proportion of outliers in data increases. (b) Error as Gaussian noise increases. (c) The voted lines for the voting algorithm. Data with60% outlier probability. (d) Runtime for voting algorithm and 5-point with RANSAC as outlier proportion increases. (e) Error in estimated epipole for real sequence-baseline2 cm/frame, rotation 5�/frame, static scene (f) Error for real sequence – 2 cm/frame, 2�/frame, multiple independently moving objects in the scene.

J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253 251

always constant, regardless of the probability of outliers or amountof noise in the data.

This is in contrast to 5-point-RANSAC which runs inapproximately cubic time for our experiments as outliers increase.Theoretically, increasing outliers means that RANSAC will draw

more samples from the data according to the formulaM ¼ logð1� pÞ=logð1�wsÞ (Eq. (5)), where s ¼ 5 in this case (refer[2] for details). This is illustrated in Fig. 6d. The numbers in Fig. 6dare obviously implementation dependent but the trend will remainthe same.

252 J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253

The antipodal + RANSAC algorithm has similar limitations andsince the results show only a small difference in accuracy com-pared to the voting + antipodal algorithm, we recommend usingvoting for applications which are time critical.

5.4. Real videos

Fig. 6e shows the error in the estimated translation directionwith respect to measured ground truth. Our algorithms performwith accuracy comparable to 5-point-RANSAC. Fig. 6f shows the er-ror for a different scene; one containing multiple independentlymoving objects. The flow arising from these would be outlierswhilst the flow arising purely from the egomotion of the cameraare the inliers. Once again, our methods show excellent results.The differences in the average errors of different methods werequite small-around 1–2� – which is within the bounds on the accu-racy of the ground truth measurements.

In the supplementary material, refer to ‘‘(A)frontal-staticscene.mpg” for the sequence of Fig. 6e and ‘‘(B)frontal-indepen-dently moving objects.mpg” for the sequence of Fig. 6f. Those vid-eos show the view from the frontal camera of the Ladybug camerasystem, where antipodal estimates, 5-point + RANSAC estimatesand the ground truth are marked as colored crosses.

6. Discussion

6.1. Improving the accuracy of the voting method

A consequence of the voting step was the segmentation ofantipodal flow data into inliers and outliers. Inliers are the antipo-dal flows that gave rise to great circles or straight lines (after pro-jection) that intersected at the estimated translation direction.

If a more accurate translation estimate is desired (one not lim-ited by voting bin resolution), a maximum likelihood estimate(assuming Gaussian noise) can be found by finding the intersectionof the inlier straight lines by linear least squares. This additionalstep incurs little extra effort and we note that it is standard prac-tice in motion estimation algorithms to add a linear or non-linear(e.g.: bundle adjustment) optimization step at the end.

6.2. Finding rotation robustly

We remind the reader that at this point, we have only recoveredthe epipole or direction of translation. The rotation was not esti-mated directly from our method. However, knowing translation,it becomes much easier to then estimate rotation. For instance, tak-ing dot products on both sides of Eq. (1) with t� r eliminatesdependence on depth:

ðt� rÞ � _r ¼ ðt� rÞ � 1jRðrÞj ðt � rÞr� tð Þ� �

� ðt� rÞ � ðw� rÞ

ðt� rÞ � _r ¼ ðt� rÞ � ðw� rÞ ð6Þ

The estimated translation direction is substituted into Eq. (6), givingus an equation linear in rotation, w. Taking flow at several points,we can build a system of over-constrained linear equations whichis solvable by the method of linear least squares. This would bedone using only the inlier flow vectors obtained from the antipo-dal + voting or antipodal + RANSAC algorithms, so the rotation esti-mate would also be independent of outliers. However, whilstalmost all outlier flow vectors were discarded, it is still possiblefor a few outliers to slip in (called ‘leverage points’ in statistics). Thishappens if, by chance, the noise in two outlier antipodal flow vec-tors cancel when added, giving a great circle that passes throughthe epipole. Therefore, another layer of outlier rejection may beneeded in practice. Fortunately, since previous steps would have re-

duced the number of unknowns and weeded out almost all outliers,this should converge very quickly.

6.3. Speed

The voting approach is potentially a method for estimating mo-tion at high speeds. The flow calculation is typically performed foronly a few hundred antipodal points (�200–500) and the mathe-matical operations used in the voting algorithm are simple – afew dot products, addition and the casting of votes along circles(the coarse-voting stage) or lines (fine-voting). Furthermore, sinceboth the pyramidal Lucas–Kanade flow calculation and the votingstep are naturally parallelizable, it is possible to obtain fast imple-mentations using parallel computing architectures such as FPGAsand GPUs. The final step to robustly recover rotation via leastsquares should incur only a small, additional processing time.

6.4. Complex environments

The voting algorithm’s resistance to outliers (Fig. 6a) and itsconstant processing time with respect to increasing outliers(Fig. 6d) would make this method well suited for certain applica-tions and scene environments – for instance, a moving camera inthe midst of a crowd of hundreds of pedestrians. Other examplesinclude estimating vehicle self-motion in busy traffic scenes, orin a windy forest where constantly waving leaves and branchesmay occupy large sections of the image. Due to the excessivelylarge proportion of outliers from the independently moving objectsin the scenes, 5-point-RANSAC would incur a large processing timeand may potentially be less accurate, making the antipodal + vot-ing algorithm preferable in such situations.

7. Conclusion

In summary, we have presented a constraint on the epipole aris-ing from the optical flow at two antipodal points. We demon-strated the validity of the constraint and presented two classes ofalgorithms utilizing it to estimate the location of the epipole. Wehave shown that this method works robustly and accurately bothin simulations and with noisy, real images, giving results compara-ble to the current state-of-the-art.

Acknowledgments

NICTA is funded by the Australian Government as representedby the Department of Broadband, Communications and the DigitalEconomy and the Australian Research Council through the ICT Cen-ter of Excellence program.

Appendix A. Supplementary material

For the real image sequence of Fig. 6e, see the video ‘‘(A)frontal-static scene.mpg” for the frontal camera view of the moving Lady-bug camera system. Estimates and ground truth are marked withcrosses. For the sequence of Fig. 6f which is a scene involving mul-tiple independently moving objects, see ‘‘(B)frontal- independentlymoving objects.mpg”. The views from the other cameras on theLadybug that were also used to estimate egomotion are includedfor the reader’s reference.Supplementary data associated with thisarticle can be found, in the online version, at doi:10.1016/j.cviu.2009.04.005.

References

[1] H. Longuet-Higgins, A computer algorithm for reconstruction of a scene fromtwo projections, Nature 293 (1981) 133–135.

J. Lim, N. Barnes / Computer Vision and Image Understanding 114 (2010) 245–253 253

[2] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision,Cambridge University Press, 2000.

[3] R. Hartley, In defense of the 8-point algorithm, in: Proceedings of the 5thInternational Conference on Computer Vision, 1995, pp. 1064–1075.

[4] H. Li, A simple solution to the six-point two-view focal-length problem, in:Proceedings of ECCV’06, 2006, pp. 200–213.

[5] H. Li, R. Hartley, Five-point motion estimation made easy, in: Proceedings of18th International Conference of Pattern Recognition, 2006, pp. 630–633,<http://users.rsise.anu.edu.au/~hongdong/lhd_5ptEssentialfordistribution.zip>.

[6] D. Nister, An efficient solution to the five-point relative pose problem, in: IEEETransactions on Pattern Analysis and Machine Intelligence (PAMI’04), 2004,pp. 756–770.

[7] A.R. Bruss, B.K. Horn, Passive navigation, in: Computer Vision, Graphics andImage Processing, vol. 21, 1983, pp. 3–20.

[8] J.W. Roach, J.K. Aggarwal, Determining the movement of objects from asequence of images, IEEE Transactions on Pattern Analysis and MachineIntelligence 2 (6) (1980) 55–62.

[9] C. Fermuller, Y. Aloimonos, Qualitative Egomotion, International Journal ofComputer Vision 15 (1995) 7–29.

[10] C. Silva, J. Santos-Victor, Direct egomotion estimation, in: Proc. 13thInternational Conference on Pattern Recognition, 1996.

[11] K. Kanatani, Y. Shimizu, N. Ohta, M.J. Brooks, W. Chojnacki, A. van denHengel, Fundamental matrix from optical flow: optimal computation andreliability evaluation, Journal of Electronic Imaging 9 (2) (2000) 194–202.

[12] A. Jepson, D. Heeger, Subspace methods for recovering rigid motion I:algorithm and implementation, International Journal of Computer Vision 7(2) (1992) 95–117.

[13] B. Horn, E. Weldon, Direct methods for recovering motion, InternationalJournal on Computer Vision (1988) 51–76.

[14] S. Negahdaripour, B. Horn, Direct passive navigation, IEEE Transactions onPattern Analysis and Machine Intelligence 9 (1) (1987) 168–176.

[15] J.J. Koenderink, A.J. van Doorn, Invariant properties of the motion parallax fielddue to the movement of rigid bodies relative to an observer, Optica Acta 22 (9)(1975) 773–791.

[16] T.Y. Tian, C. Tomasi and D.J. Heeger, Comparison of approaches to egomotioncomputation, in: Proc. IEEE Conference on Computer Vision and PatternRecognition, 1996, pp. 315–320.

[17] T. Huang, A. Netravali, Motion and structure from feature correspondences: areview, Proceedings of the IEEE 82 (2) (1994) 253–268.

[18] R. Nelson, J. Aloimonos, Finding motion parameters from spherical flow fields(or the advantages of having eyes in the back of your head), BiologicalCybernetics 58 (1988) 261–273.

[19] J. Lim, N. Barnes, Estimation of the epipole using optical flow at antipodalpoints, in: OMNIVIS’07, 2007.

[20] I. Thomas, E. Simoncelli, Linear Structure from Motion, Technical Report: IRCS,University of Pennsylvania, 1994.

[21] J. Lim, N. Barnes, Directions of egomotion from antipodal points, in:Proceedings of Conference on Computer Vision and Pattern Recognition(CVPR’08), Anchorage, 2008.

[22] P.V.C. Hough, Method and Means for Recognizing Complex Patterns, US Patent3,069,654, December 18, 1962.

[23] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley, NewYork, 1973.

[24] J. Illingworth, J.V. Kittler, A survey of the Hough transform, CVGIP 44 (1) (1988)87–116.

[25] M.A. Fischler, R.C. Bolles, Random sample consensus: a paradigm for modelfitting with applications to image analysis and automated cartography, CACM24 (6) (1981) 381–395.

[26] R. Raguram, J.M. Frahm, M. Pollefeys, A comparative analysis of RANSACtechniques leading to adaptive real-time random sample consensus, in:Proceedings of ECCV, 2008, pp. 500–513.

[27] T. Brodsky, C. Fermuller, Y. Aloimonos, Directions of motion fields are hardlyever ambiguous, International Journal of Computer Vision 26 (1998) 5–24.

[28] David G. Lowe, Object recognition from local scale-invariant features, in:International Conference on Computer Vision, September, 1999, pp. 1150–1157.

[29] B.D. Lucas, T. Kanade, An iterative image registration technique with anapplication to stereo vision, in: Proceedings of Imaging UnderstandingWorkshop, 1981, pp. 121–130.

[30] B.K.P. Horn, B.G. Schunck, Determining optical flows, Artificial Intelligence 17(1981) 185–203.

[31] O. Tuzel, R. Subbarao, P. Meer, Simultaneous multiple 3d motion estimationvia mode finding on lie groups, in: Proceedings of 10th IEEE InternationalConference on Computer Vision, 2005, pp. 18–25.

[32] D. Comaniciu, P. Meer, Mean shift: a robust approach toward feature spaceanalysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (5)(2002) 603–619.

[33] A. Torii, A. Imiya, The randomized-Hough-transform-based method for great-circledetection on a sphere, Pattern Recognition Letters 28 (10) (2007) 1186–1192.

[34] H. Chen, P. Meer, Robust regression with projection based M-estimators, in:International Conference on Computer Vision, October 2003, pp. 878–885.

[35] P.H.S. Torr, A. Zisserman, MLESAC: a new robust estimator with application toestimating image geometry, Journal of Computer Vision and ImageUnderstanding 78 (1) (2000) 138–156.

[36] J. Matas, O. Chum, Randomized ransac with Td;d test, Image and VisionComputing 22 (10) (2004) 837–842.

[37] O. Chum, J. Matas, Optimal randomized RANSAC, IEEE Transactions on PatternAnalysis and Machine Intelligence 30 (8) (2008) 1472–1482.

[38] D. Nister, Preemptive RANSAC for live structure and motion estimation,Machine Vision Applications 16 (5) (2005) 321–329.

[39] M. Atiquzzaman, Multiresolution Hough transform – an efficient method ofdetecting patterns in images, IEEE Transactions on Pattern Analysis andMachine Intelligence 14 (11) (1992) 1090–1095.

[40] L. Xu, E. Oja, P. Kultanen, A new curve detection method: randomized Houghtransform, Pattern Recognition Letters 11 (5) (1990) 331–338.

[41] J. Illingworth, J.V. Kittler, The adaptive Hough transform, IEEE Transactions onPattern Analysis and Machine Intelligence 9 (5) (1987) 690–698.

[42] L. Quan, R. Mohr, Determining perspective structures using hierarchical Houghtransform, Pattern Recognition Letters 9 (4) (1989) 279–286.

[43] M.J. Magee, J.K. Aggarwal, Determining vanishing points from perspectiveimages, Computer Vision Graphics and Image Processing 26 (1984) 256–267.

[44] J.A. Shufelt, Performance evaluation and analysis of vanishing point detectiontechniques, IEEE Transactions on Pattern Analysis and Machine Intelligence 21(3) (1999) 282–288.

[45] H.S.M. Coxeter, Introduction to Geometry, second ed., Wiley, New York, 1969.pp. 93, 289–290.

[46] J.E. Bresenham, Algorithm for computer control of a digital plotter, IBMSystems Journal 4 (1) (1965) 25–30.

[47] P.D. Kovesi, MATLAB and Octave Functions for Computer Vision and ImageProcessing, School of Computer Science & Software Engineering, TheUniversity of Western Australia, <http://www.csse.uwa.edu.au/~pk/research/matlabfns/>.

[48] J.L. Barron, D.J. Fleet, S.S. Beauchemin, Performance of optical flow techniques,International Journal of Computer Vision 12 (1) (1994) 43–77.

[49] Point Grey Research, 2007, <http://www.ptgrey.com>.[50] Intel Open Source Computer Vision Library, 2007, <http://sourceforge.net/

projects/opencv>.[51] D.G. Lowe, <http://www.cs.ubc.ca/~lowe/keypoints/>.