470 IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 11, NO. 3, JUNE 1995

needed to properly manufacture a part are not relaxed, similar to our previously published paper [2]. In fact, our current algorithm is designed to address exactly this problem, and thus generate high-quality schedules efficiently when products and parts have complex manufacturing requirements. This decomposition approach is currently being utilized for the daily scheduling of F’ratt & Whitney.

REFERENCES

H. D. Sherali, S. C. Sarin, and R. Desai, “Models and algorithms for job selection, routing, and scheduling in a flexible manufacturing system,” Annals Operat. Res., vol. 26, pp. 433453, 1990 D. J. Hoitomt, P. B. Luh, E. Max, K. R. Pattipati, “Scheduling jobs with simple precedence constraints on parallel machines,” IEEE Cont. Syst. Mug., vol. 10, no. 2, pp. 34-40, also in Dynamics of Discrete Event Systems, Y . C. Ho, Ed. New York: IEEE Press, 1992, pp. 271-277. P. B. Luh, D. J. Hoitomt, E. Max, and K. R. Pattipati, “Schedule gen- eration and reconfiguration for parallel machines,”IEEE Trans. Robot. and Automat., vol. 6, no. 6, pp. 687-696, Dec. 1990. N. D. Sherali, S. C. Sarin, and R. Desai, “Models and algorithms for job selection, routing and scheduling in a flexible manufacturing system,”Annals Operut. Res., vol. 26, pp. 433-453, 1990. K. R. Baker, Introduction to Sequencing and Scheduling. New York: Wiley, 1974. N. T. Bruvold and J. R. Evans, “Flexible mixed-integer programming formulations for production scheduling problems,”IIE Tmns., vol. 17, no. 1, pp. 2-7, Mar. 1985.

A Note on “On Single-Scanline Camera Calibration”’

Hanqi Zhuang

Abstruct- In this correspondence, the single-scanline camera model proposed in the above paper’ is modified to include a lens distortion coefficient. By this modification, the calibration approach presented in the above paper will become more practical.

I. INTRODUCTION Single-scanline cameras may be used for dimensional measurement

and inspection. In such cases, the accuracy performance of such cameras is very important. To be able to perform dimensional measurement, parameters of a camera model that relates image measurements to 3-D measurements have to be estimated through calibration.

Horaud et al. in the above paper’ presented a linear method for the calibration of single-scanline cameras. Their approach is very easy to implement due to its simplicity. However, the camera model they used is basically a pin-hole model that does not account for any geometric distortions of the camera lens. Lens distortion is a dominant factor that influences the accuracy performance of the camera system.

Lens distortion may be corrected by a separate procedure before applying the method proposed in the above paper. It can also be com- pensated by a more elaborate camera model. In this correspondence, the camera model given in the above paper is modified to include a

Manuscript received February 11, 1994. The author is with the Robotics Center and Department of Electrical

IEEE Log Number 9404605. ’R. Horaud, R. Mohr, and B. Lorecki, IEEE Trans. Robot. Automat., vol.

9, no. 1, pp. 71-75, Feb. 1993.

Engineering, Florida Atlantic University, Boca Raton, FL 33431.

I / I I - 1

- 2 0.02 -

0.01 -

0 20 4 0 80 8 0 100

Lma distortion eooffldmt

Fig. 1. Relative errors in some of the camera parameters versus lens distor- tion coefficient. Note: 1) 0 denotes relative errors of nl , and x denotes those of n2. 2) Lens distortion coefficient is scaled by lo-’’.

lens distortion coefficient that represents geometric distortion of the camera lens.

II. A SINGLE-SCANLINE CAMERA MODEL The model presented here modifies the one given in the above

paper by adopting some techniques for modeling array cameras [2]. Let X, Y, Z be world coordinates of a 3-D point, and U, U be the undistorted image coordinates of its projection. Let z,y be its distorted image coordinates. Then from the above paper ,

(1) n1Y + n2Z + 7L3

U = 7LrY + n5Z + 1

where 7 1 1 , . . . 7 ~ 5 are camera parameters.

coordinate U by The distorted image coordinate 2 is related to the undistorted image

U = rc(1 + k z 2 ) (2)

where the high order terms have been dropped. In (2), k is the distortion coefficient. Combining (1) and (2) yields

(3)

Moreover, the world coordinates of the object point are constrained by the following viewing-plane from the above paper,

X = p Y + q Z + r . (4)

Equations (3) and (4) relate world coordinates of an object point to its image coordinates. The parameters to be calibrated in this model a r e n l , - . . , n S , k , p , q , a n d r .

The model given in (4) is nonlinear, therefore the linear algorithm given in the above paper cannot be used to compute n1, . . . , 7 ~ 5 and I;. The calibration problem can however be solved by using a nonlinear optimization procedure. Let g = [nl, . . . , n5, kIT, and let

f ( X , Y , Z , g ) = 2 ( 1 + k z 2 ) ( n 4 Y + 7 L s Z + 1 ) - (n1Y + n2Z + 71.3). (5 )

1042-296X/95$04.00 0 1995 IEEE

IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. I I , NO. 3, JUNE 1995 47 1

The problem is to minimize the following cost function

by choosing g, where i denotes the ith measurement and m is the total number of measurements. A good initial condition is normally needed for a proper convergence of nonlinear optimization algorithms. The procedure given in Section IV-A of the above paper provides an initial condition that is close to the optimal solution because the distortion coefficient is usually small.

Finally, it should be noted that the method presented in the above paper for the calibration of p , q, and r is still applicable here after slight modification. The measured image coordinate z has to be mapped to the undistorted image coordinate U by (2) before it is used to compute the cross-ratio given in the above paper. It should be noted that because the distortion coefficient must be computed a priori for the calibration of p , q, and r, the two steps given in the above paper are no longer independent.

III. A NUMERICAL EXAMPLE In this example, the “true” camera parameters 7Ll-R.5 are assumed

to be the same as those given in Table I of the above paper. To simulate actual cameras, the lens distortion coefficient k is given in the horizontal axis of Fig. 1. In the simulation, 150 measurements from X E [-25 mm, 25 mm], Y E [-25 mm, 25 mm] and 2 E [0 mm, 25 mm] (constrained by a viewing plane) are taken to compute 7i 1-n5 using the procedure given in Section IV-A of the above paper, in which k is assumed to be zero. Relative errors for some of the camera parameters are shown in Fig. 1. Errors of other parameters are of the same order of magnitudes. It can be seen that even for a very small distortion coefficient, errors in the camera parameters can be significant.

Author’s Reply by Radu Horaud

The improvement proposed by Hanqi Zhuang to include geometric distortions due to camera lens is a sensible one. The distortion model is the one classically used in photogrammetry and introduced in Computer Vision by Tsai [l]. More recently, Horst Beyer proposed a similar nonlinear model that takes into account both geometric (lens) and electronic distortions [2].

In his note Hanqi Zhuang assumes that the distortion is limited to one dimension, i.e., along the camera’s scanline. Since the distortion is produced by the lens, there is no reason to think that the distortion occurs only along that direction. A more rigorous approach would have been to substitute the viewing plane with a second order “viewing surface.” But then one cannot use the cross-ratio anymore’ and we are left with the problem of estimating the parameters of such a surface.

The author casts the problem of estimating R I through n5 and k into a non linear optimization problem. I think that there is a better choice in terms of the error function to be minimized, namely the sum of squares of the true Euclidean distances between the actual

Manuscript received October 5, 1994. The author is with LIFIA-IMAG, 46 avenue Felix Viallet, 3803 1 Grenoble,

IEEE Log Number 9404606. France.

pixel positions and the positions estimated using the camera model

n 1 Y + n 2 Z + n 3 * n 4 Y + n 5 2 + 1 1 . z (1 - k z 2 ) - (

My last comment concerns the experimental results. Rather than simulating lens distortions it would have been preferable to compare the error between the linear and nonlinear camera models using a real lens. This can be easily done with a standard camera by considering only one image row or column.

REFERENCES

[l] R. Y. Tsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lens,” IEEE Trans. Robot. Automat., vol. 3, no. 4, pp. 323-344.

[2] H. A. Beyer, “Geometric and radiometric analysis of a CCD-camera based photogrammetric closed-range system,” Ph.D. dissertation, Institut fur Geodasie und Photogrammetrie, Zurich, Switzerland, 1992.

Comments on “A Sliding Mode Controller with Bound Estimation for Robot Manipulators”’

Han Ho Choi, Hyung Kyi Yi, and Myung Jin Chung

In the above paper’ an adaptive sliding mode control scheme for the trajectory control of robot manipulators is developed. The proof of Theorem 1 in the paper is accomplished by resorting to the Lasalle theorem. It is well known that the Lasalle theorem can be successfully applicable to either autonomous systems or periodic nonautonomous systems but it cannot be applicable to general nonautonomous systems [ 11-[4]. The Lasalle theorem cannot be applicable to the manipulator tracking control system given in the paper because the tracking control system considered in the paper is neither an autonomous system nor a periodic nonautonomous system but it can be classified into general nonautonomous systems. Therefore, the proof of Theorem 1 is not correct. It was already pointed out in [4] that the Lasalle theorem cannot be applicable to manipulator tracking control systems with time-varying trajectories.

Fortunately, nothing is lost. If we use more advanced stability results we can show that the control and adaptation laws of (&(lo) given in $e paper guarantee the asymptotic stability of the tracking error (i, 4) = 0 and the boundedness of estimation vector r l L .

Proposition 1: Consider robotic system (1) with sliding surface (4) and control laws (8t( lO), which are given in the.paper. The position tracking error i as well as the velocity error 4 converges asymptotically to zero and the parameter estimation vector i jL is

Manuscript received September 3, 1993. The author is with the Department of Electrical Engineering, Korea Ad-

vanced Institute of Science and Technology, 373-1 Kusong-dong, Yusong-gu, Taejon 305-701 Korea.

IEEE Log Number 9412204. ‘C. Y. Su and T. P. Leung, IEEE Trans. Robot. Automat., vol. 9, no. 2, pp.

208-214, Apr. 1993.

1042-296X/95$04.00 0 1995 IEEE