Multiaperture telecentric lens for 3D reconstructionJun-Sik Kim* and Takeo Kanade

The Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, Pennsylvania 15213, USA*Corresponding author: kimjs@cs.cmu.edu

Received November 22, 2010; revised February 4, 2011; accepted February 15, 2011;posted February 23, 2011 (Doc. ID 138517); published March 17, 2011

We present a telecentric lens that is able to gain 3D information. The proposed lens system has multiple aperturestops, which enable it to capture multidirectional parallel light rays, while a conventional telecentric lens has onlyone aperture stop and can capture only light rays that are perpendicular to the lens. We explain the geometry of themultiaperture telecentric system and show that correspondences fall on a line like those in a conventional stereo. Asit is a single-lens sensor, we also introduce the principles of 3D reconstruction. Unlike a conventional stereo camera,the disparity of a scene point measured by the proposed lens system is linearly proportional to the depth of a scenepoint. © 2011 Optical Society of AmericaOCIS codes: 110.6880, 120.3620, 150.6910.

Telecentric lenses have been widely used in manymachine vision applications such as noncontact mea-surement and inspection systems [1,2]. This is mainlybecause these lenses provide purely orthographic projec-tions of scene points by locating an aperture stop at thefocal point of a lens. This property makes it easier to mea-sure or compare physical lengths of objects indepen-dently from the depths of the objects from the camera.However, this desirable property has one drawback: it isnot possible to obtain depth information from the image,as there is no foreshortening effect in telecentric images.We present a system for obtaining depth information

while still preserving the advantages of a telecentric lens.The proposed system has multiple aperture stops ratherthan one, as in a conventional telecentric lens. Aconventional telecentric lens selectively passes light raysthat are parallel to the optical axis, because the aperturestop at the focal point of the object lens passes only therays, as shown in Fig. 1(a). Light rays that are parallel toeach other but not parallel to the optical axis converge atanother point on the focal plane, which is perpendicularto the optical axis and passes through the focal point.Figure 1(b) shows the ray diagram of a set of parallel lightrays that are not parallel to the optical axis in a bitele-centric lens. Assuming we have an infinitely small aper-ture stop, the light rays selected by the aperture stop areparallel to the vector from the lens center to the aper-ture stop.Assume that there are two aperture stops on the focal

plane, as in Fig. 2. One of the stops is located at the focalpoint of the lens, and the other is at a distance b from thefocal point. For a scene point located at ð−z1 − f ; y1Þ, thelight rays selected by the first and the second stops passthrough points ðf ;−y1Þ and ðf ;−y1 þ ðz1f − 2ÞbÞ at the im-age plane, which is located at a distance f from the focalplane. The disparity d (displacement between twoimages) is

d ¼ −

�z1f− 2

�b: ð1Þ

When z1 ¼ 2f , d becomes zero. Note that this disparity dcan also be negative. In (1), both the baseline length bbetween two aperture stops and the focal length f ofthe lens are fixed. Thus, the disparity d is linearly propor-

tional to the point depth z1 from the lens. This is the cri-tical difference from conventional stereo systems, inwhich the disparity is proportional to the inverse depthof a scene point. The absolute location of a point otherthan the depth can be easily obtained using the imagethat was captured by the aperture stop at the focal point,just as in the conventional telecentric lens. Thus, the pro-posed lens will maintain the advantages provided by aconventional telecentric lens.

Though the idea to use multiple apertures is not new[3–7], to the best of our knowledge, it has not beenadopted in a telecentric lens. A telecentric lens with mul-tiple aperture stops has a major advantage: the disparitygiven by the proposed multiaperture telecentric system islinearly proportional to the depth, while that given by aconventional stereo system is proportional to the inversedepth, typically resulting in an unstable depth recovery[8]. Figure 3 shows a comparison between the proposedsystem and a conventional stereo system. As Fig. 3(b)shows, when a point goes farther, the angle between rayscollected from the conventional stereo system becomesnarrower, i.e., there is less disparity in the image space.The angle between the light rays collected by the pro-posed imaging system remains constant, as shown inFig. 3(a). As shown in Figs. 3(c) and 3(d), the disparityis linearly proportional to the scene depth, and thus, thesensitivity of depth estimation remains constant, an ad-vantage of the proposed system. In addition, unlike theconventional stereo system, the proposed system doesnot require any photometric calibration or intensity ad-justment between cameras because it is a single-lensimaging sensor. When the scene point is a Lambertianor a point light source, i.e., a source that emits the sameamount of light in every direction, the brightness of the

Fig. 1. Location of an aperture stop in a telecentric lens.(a) Aperture stop on the optical axis selects light rays parallelto the optical axis. (b) Aperture stop off the optical axis selectsdifferent directional rays.

1050 OPTICS LETTERS / Vol. 36, No. 7 / April 1, 2011

0146-9592/11/071050-03$15.00/0 © 2011 Optical Society of America

image captured by the proposed imaging system wouldremain constant.The epipolar geometry in this multiaperture telecentric

imaging system is similar to a conventional stereo sys-tem. Figure 4 shows the path of the light rays travelingbetween the lens and the image plane for two scene

points. The rays should pass through one of the two aper-ture stops and build a pencil of planes containing thebaseline. Because the image plane cuts the pencil ofplanes, all the correspondences should be on a line. Foran image plane parallel to the focal plane, all epipolarlines are parallel to the baseline and, of course, parallel

Fig. 2. (Color online) Proposed telecentric imaging systemwith two aperture stops.

Fig. 3. (Color online) Comparison between (a) the proposedmultiaperture telecentric system and (b) the conventionalstereo system for objects in different depths from the sensor.Theoretical disparity and its first derivatives when f ¼ 20and b ¼ 5 are presented in (c) and (d), respectively.

Fig. 4. (Color online) Epipolar geometry on the image plane ofthe multiaperture telecentric imaging system. Set of light raysbuilding a pencil of planes.

Fig. 5. Ray diagram of the multiaperture telecentric imagingsystem. These stops are drawn much larger than the actual size.

Fig. 6. (Color online) Multiaperture telecentric imagingsystem using a Fresnel lens with a focal length of 215mm.

Fig. 7. (Color online) Images on the screen for (a) closer and(b) farther objects.

April 1, 2011 / Vol. 36, No. 7 / OPTICS LETTERS 1051

to each other. Thus, the proposed telecentric lens, whichhas multiple apertures, generates a rectified image [8].As a telecentric lens, each aperture stop selects light

rays within a small range of angles and also has smallconfusion circles. In addition, the confusion circle ofeach stop does not interfere with another, thus the depthof field of the proposed system remains the same as thatof the conventional telecentric lens. This is shown inFig. 5, which gives a ray diagram of the telecentric imag-ing system with two apertures.Figure 6 shows our implementation of the proposed

system using a Fresnel lens and an aperture stop plane.A lens, an aperture stop plane, and an image plane lie ona linear rail guide. The focal length of the Fresnel lensmeasures 215mm. It has three aperture stops with base-line lengths of 30 and 15mm. The diameter of each aper-ture stop is less than 1mm. On the image plane, weplaced a white square screen (190:5mm × 190:5mm).Figure 7 shows the images of a clear light bulb that ap-

pear on the screen when the distance of the light bulbbecomes farther from the lens. The filament in the clearlight bulb can be modeled as a set of point light sources.One can easily see that the disparity of a closer object is

larger than that of a farther one. The input images in Fig. 7were rectified using the four corners of the white screen.Once the rectified image was obtained, we searched forcorrespondences in the vertical direction because theaperture stops were aligned vertically.

The 3D recovery of the filament was successfullyachieved, as shown in Fig. 8. In this case, we used onlythe disparity measured between the center and the lowerimages, not the upper images. The depths of points werecomputed using Eq. (1), and the other coordinates weredirectly obtained from the center image, which is thesame image captured by a conventional telecentric lens.

In summary, we propose a method to extract 3D infor-mation using a telecentric lens, by placing an additionalaperture stop on the focal plane along with the stop at thefocal point. We investigate the structure of the proposedsystem and its epipolar geometry that all the correspon-dences are placed on lines parallel to the direction of thetwo aperture stops. In addition, we show that, unlike aconventional stereo system, the disparity obtained bythe proposed system is linearly proportional to the depthof a scene point. Three-dimensional reconstruction of ascene is straightforward and intuitive because the pro-posed system inherits the benefits of the conventionaltelecentric lens. Finally, we designed a simple prototypeof the proposed system using a Fresnel lens, providingpromising 3D reconstruction results.

References

1. M. Bass, C. DeCusatis, J. Enoch, V. Lakshminarayanan,G. Li, C. Macdonald, V. Mahajan, and E. Van Stryland,Handbook of Optics (McGraw-Hill, 2010).

2. Opto Engineering, “Telecentric lenses: basic informationand working principles,” http://www.opto‑engineering.com/telecentric‑lenses‑tutorial.html.

3. E. H. Adelson and J. R. Bergen, in Computational Models ofVisual Processing (MIT, 1991), pp. 3–20.

4. E. H. Adelson and J. Y. A. Wang, IEEE Trans. Pattern Anal.Machine Intell. 14, 99 (1992).

5. B. Wilburn, N. Joshi, V. Vaish, M. Levoy, and M. Horowitz, inIEEE Society Conference on Pattern Recognition (IEEE,2004), pp. 294–301.

6. B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez,A. Barth, A. Adams, M. Horowitz, and M. Levoy, ACM Trans.Graph. 24, 765 (2005).

7. A. Zomet and S. Nayar, in IEEE Conference on ComputerVision and Pattern Recognition (CVPR) (IEEE, 2006),pp. 339–346.

8. D. Forsyth and J. Ponce, Computer Vision: A ModernApproach (Prentice-Hall, 2003).

Fig. 8. (Color online) Reconstruction of the filament using theproposed system. In these figures, the locations of a lens (blue),an aperture plane (red), and a screen (black) are shown, as wellas the reconstructed filament in closer (blue) and farther (red)locations.

1052 OPTICS LETTERS / Vol. 36, No. 7 / April 1, 2011