Improvement of least-squares integration method with .... Improvement of lea… · This integration process, commonly with a two-dimensional (2D) data set, does affect the profile

This document is downloaded from DR‑NTU (https://dr.ntu.edu.sg)Nanyang Technological University, Singapore.

Improvement of least‑squares integration methodwith iterative compensations in fringereflectometry

Huang, Lei; Asundi, Anand Krishna

2012

Huang, L., & Asundi, A. K. (2012). Improvement of least‑squares integration method withiterative compensations in fringe reflectometry. Applied optics, 51(31), 7459‑7465.

https://hdl.handle.net/10356/95887

https://doi.org/10.1364/AO.51.007459

© 2012 Optical Society of America. This paper was published in Applied Optics and is madeavailable as an electronic reprint (preprint) with permission of Optical Society of America.The paper can be found at the following official DOI:[tp://dx.doi.org/10.1364/AO.51.007459]. One print or electronic copy may be made forpersonal use only. Systematic or multiple reproduction, distribution to multiple locationsvia electronic or other means, duplication of any material in this paper for a fee or forcommercial purposes, or modification of the content of the paper is prohibited and issubject to penalties under law.

Downloaded on 25 Mar 2021 22:52:30 SGT

Improvement of least-squares integration method withiterative compensations in fringe reflectometry

Lei Huang* and Anand AsundiSchool of Mechanical and Aerospace Engineering, Nanyang Technological University,

Singapore 639798, Singapore

*Corresponding author: [email protected]

Received 30 July 2012; accepted 31 August 2012;posted 28 September 2012 (Doc. ID 173476); published 22 October 2012

Least-squares integration is one of the most effective and widely used methods for shape reconstructionfrom gradient data, which result from gradient measurement techniques. However, its reconstructionaccuracy is limited due to the imperfection of the Southwell grid model, which is commonly appliedin the least-squares integration method. An operation with iterative compensations is therefore pro-posed, especially for the traditional least-squares integration method, to improve its integration accu-racy. Simulation and experiment are carried out to verify the feasibility and superiority of the proposedoperation. This compensatory operation with iterations is suggested, and its good performance onintegration accuracy improvement is shown. © 2012 Optical Society of AmericaOCIS codes: 150.6910, 120.3940, 110.3010.

1. Introduction

There are a variety of optical techniques in three-dimensional (3D) shape metrology for industrial in-spection applications [1]. Among these measurementtechniques, some are referred to as direct methods,as they directly get 3D results from the received op-tical information, e.g., the locations of laser spots orthe distortion of pattern images. Other techniquesdeduce the surface shape by integration of the gradi-ent information. The fringe reflection technique [2,3]and Shack–Hartmann methods [4] are two represen-tatives of these indirect 3D shape measurementmethods. A process to integrate the gradient datais consequently required to reconstruct the surfaceshape or the wavefront.

This integration process, commonly with a two-dimensional (2D) data set, does affect the profileresults, and hence research efforts are being put toexplore how to make this process more accurateand faster. Fried [5], and Hudgin [6,7] studied thewavefront reconstruction from phase difference or

slope using least-squares fitting and evaluated theerror propagation in reconstructions. Southwell [8]provided the wavefront estimation from slope withthe least-squares method in more detail includingits grid models, solutions, and errors. Li et al. [9]reviewed several evaluation methods, includingthe path-guided integration method, the Fourier-transform-based integration method, and theleast-squares integration method, for gradient mea-surement techniques and made a comparison ofthese integration methods showing the superiorityof the least-squares integration method in terms ofaccuracy, robustness, and flexibility. Legarda-Saenzand Espinosa-Romero [10] presented a Fourier-transform-based method to reconstruct the wave-front by integrating the multiple directional deriva-tives for deflectometric measurement. Koskulicset al. [11] further developed the least-squares inte-gration methods with Southwell grid model by simul-taneously considering the surface elevation, slope,and curvature for surface retrieval when the surfaceelevation, slope, and curvature are available fromthe measurement.

The least-squares integration method with South-well grid model [8] is basically good at handling large

1559-128X/12/317459-07$15.00/0© 2012 Optical Society of America

1 November 2012 / Vol. 51, No. 31 / APPLIED OPTICS 7459

gradient data sets due to its low memory cost byusing sparse matrices. However, the Southwell gridmodel assumes the surface could be expressed asbiquadratic functions. Obviously, this assumption isnotalwaysbesatisfied inpractical situations, inwhichthe surface cannot bedescribedwithbiquadratic func-tions, and finally it results in integration errors.

This work consequently aims to reduce theintegration errors from the imperfection of theSouthwell grid model in the traditional least-squaresintegration method. Section 2 first recalls the least-squares integration with Southwell model and thendescribes the operation of iterative compensationsstep by step. Simulations are demonstrated inSection 3 to show the feasibility and effectivenessof the proposed iterative compensations in improvingthe integration accuracy. Section 4 shows theexperimental results by using the proposed methodwith the comparison of traditional method. Section 5discusses the merits and limitations of theproposed method, and Section 6 concludes thework.

2. Method

In the gradient-measuring optical metrology meth-ods,numerical integrationconvertsgradientdata intosurface shape or profile by using least-squares fitting.At first, it is very necessary to recall the traditionalleast-squares integration with Southwell grid model[8]. Following the Southwell grid model with a grid

size of M ×N as shown in Fig. 1, the relationshipbetween slope and shape can be expressed as( pi;j�1�pi;j

2 � zi;j�1−zi;jxi;j�1−xi;j

; i� 1;…;M; j� 1;…;N −1qi�1;j�qi;j

2 � zi�1;j−zi;jyi�1;j−yi;j

; i� 1;…;M −1; j� 1;…;N;

�1�where x, y, z are the world coordinates. p and q denotethe slopes in x and y directions, respectively, i.e., p �dz∕dx and q � dz∕dy.

According to Eq. (1), the height information can beacquired as8>>><>>>:zi;j�1 − zi;j � 1

2 �pi;j�1 � pi;j��xi;j�1 − xi;j�;i � 1;…;M; j � 1;…; N − 1

zi�1;j − zi;j � 12 �qi�1;j � qi;j��yi�1;j − yi;j�;

i � 1;…;M − 1; j � 1;…; N

; �2�

Furthermore, Eq. (2) can be rewritten in terms ofmatrices as

DZ � G; (3)where

D �

2666666666664

−1 0 � � � 0 1 0 � � � � � � 00 −1 0 � � � 0 1 0 � � � 0... ..

. ... ..

. ... ..

. ... ..

. ...

0 � � � � � � 0 −1 0 � � � 0 1−1 1 0 � � � � � � � � � � � � � � � 00 −1 1 0 � � � � � � � � � � � � 0... ..

. ... ..

. ... ..

. ... ..

. ...

0 � � � � � � � � � � � � � � � 0 −1 1

3777777777775

��M−1�N�M�N−1��×MN

; (4)

Z �

26664

z1;1z2;1...

zM;N

37775

MN×1

; (5)

G�12

266666666666664

�p1;2�p1;1��x1;2−x1;1��p1;3�p1;2��x1;3−x1;2�

..

.

�pM;N�pM;N−1��xM;N −xM;N−1��q2;1�q1;1��y2;1−y1;1��q3;1�q2;1��y3;1−y2;1�

..

.

�qM;N�qM−1;N��yM;N −yM−1;N�

377777777777775

��M−1�N�M�N−1��×1

:

(6)

In a practical situation (MN > M �N), Eq. (3)is commonly overdetermined, i.e., �M − 1�N�

M×N

jN1

i

M

1

Position for height estimation

Point with measured gradient

Fig. 1. (Color online) In Southwell grid model, the points forheight estimation are at the same locations of those whose gradi-ent data have been measured out.

7460 APPLIED OPTICS / Vol. 51, No. 31 / 1 November 2012

M�N − 1� > MN. Hence, a least-squares estimationcan be carried out to provide the height distributionas an optimized solution. By recalling the traditionalleast-squares integration, it can be found that thereis an assumption implied in Eq. (1) that the slope dis-tribution is bilinear within each tiny quadrilateral,i.e., the variation of surface height in a single quad-rilateral is biquadratic. As mentioned above, thisassumption is not always satisfied in actual mea-surements, and hence it is one of the major errorsources in integration.

With noticing the existing imperfection of thisassumption, an iterative compensation procedureis thus suggested to improve the integration accu-racy. Basically, higher-order terms are contained inthese residual gradient data, which cannot be fitwith the biquadratic surface. It is possible to compen-sate the errors due to higher-order terms by integrat-ing these residual gradient data. With severaliterative compensations, the final shape can be accu-rately reconstructed free from the influence of theimperfect assumption. The specific steps of the algo-rithm are described as follows.

Step 1. Integrate the gradient data with the tradi-tional least-squares integration method recalledabove to get initial height z0�x; y�, and set the currentheight z�x; y� � z0�x; y�.

Step 2. Calculate the residual slopes in x and y di-rections dp�x; y� and dq�x; y� with current height dis-tribution z�x; y�.

Step 3. Integrate the residual slopes dp�x; y� anddq�x; y� with the traditional least-squares integra-tion method as well to get zc�x; y� for compensation.

Step 4. Compensate the height z�x; y� � z�x; y��zc�x; y�∕nk, where the parameter for the kth compen-sation nk is an empirical constant, which can bedetermined through simulation with the ideal sur-face containing a certain higher-order polynomialcomponent, and k stands for the number of compen-sation times.

Step 5. Repeat the compensation as a loop fromStep 2 to Step 4 until the compensating termzc�x; y�∕nk is less than a preset threshold zthr.

Step 6. Record the current z�x; y� as the finalheight.

The empirical parameters nk are determined by si-mulation with surfaces containing different higher-order components. Through our calculation, theseempirical parameters nk are n1 ≈ 3.0000, n2 ≈

4.0909, n1 ≈ 3.0000, n3 ≈ 4.9476, and n4 ≈ 5.6768when k � 1, 2, 3, and 4, respectively. From the simu-lation study, the traditional integration is intrinsi-cally able to perfectly integrate the surface withpolynomial components up to the second order as

analyzed above. Furthermore, the encounterederror when reconstructing a surface with polynomialcomponents up to the 2�n� 1�th order could be nicelycompensated by taking the nth compensation.

3. Simulation

It is the objective of the proposed method to makeimprovement on integration accuracy. Since bench-marks are easy to find in simulation for assessmentof the algorithm performance, a series of simulationsare conducted to investigate the feasibility and effi-ciency of the iterative compensation.

The first example in simulation is to show theiterative compensation works efficiently to improvethe integration accuracy when dealing with a casethat the assumption of biquadratic form is not satis-fied. Both x and y coordinates range from −5 to 5 mmwith an interval of 0.02 mm; i.e., the size of matricesis 500 by 500, which yields 500,000 x- and y-slopevalues in total. As shown in Fig. 2(a), the trueout-of-plane dimension is predefined as

z � 0.3 cos�0.4x2 � 2x� cos�0.4y2 � 2y�� 0.7 cos��x3 � y2�∕4π�; (7)

and its corresponding slopes in x and y directions pand q are shown in Figs. 2(b) and 2(c).

The surface shape of the tested sample is speciallychosen on purpose to investigate the iterative perfor-mance of the proposed method. In Fig. 2(b), the xslope varies relatively large when the absolute valueof x is relatively big, and the y slope varies slightly atthe lower left corner and acutely at the upper rightcorner in Fig. 2(c).

As a comparison, the result of the traditionalmethod is shown at first to provide a rough idea ofthe conventional integration work. The result fromthe traditional least-squares integration methodwith no compensation is demonstrated in Fig. 3(a),which shows large integration errors as the true sur-face does not satisfy the biquadratic surface assump-tion in the traditional method described in Section 2.With a strategy of compensating these errors inFig. 3(a), the proposed method with iterations couldaccurately reconstruct the out-of-plane heightdistribution.

As shown in Figs. 3(b)–3(d), with the proposedcompensation process after the first compensation,the peak-to-valley value of height error significantlyreduces to around 0.4% [from about 2000 nm inFig. 3(a) down to about 8 nm in Fig. 3(b)], and evendown to around 0.15% after the third compensationwith iterations [from about 2000 nm in Fig. 3(a) downto about 3 nm in Fig. 3(d)].

The majority of compensated amount is completedafter one or two compensations, which is also shownin Fig. 3(e). The standard deviation (Std.) of theheight errors varies from about 250 nm by usingthe traditional method with no error compensationto less than 0.7 nm after the first compensation,


and even to about 0.15 nm after the third compensa-tion with iterations. A one-line profile of the error isshown in Fig. 3(f) as well to see the capability of theproposed method to enhance the integration accu-racy. As shown in Fig. 3(g), after the third iterativecompensation, the out-of-plane height is effectivelyand accurately reconstructed. Since there is no noiseexisting on the gradient data in this simulation, theresidual errors after iteration can be considered theso-called algorithm error. The next step is thereforeto investigate the performance of the proposed meth-od under noisy condition.

The second simulation demonstrates the ability ofthe proposed method to operate in the presenceof noise. The true height shown in Fig. 4(a) is definedby

z � 3�1 − x�2 · e−x2−�y�1�2− 10

�x5− x3 − y5

�

· e−x2−y2 −13e−�x�1�2−y2 ; (8)

where the in-plane dimensions x and y are limitedwithin a range from −1 to 1 with sampling pointsof 400 by 400 and the corresponding out-of-planeheight varies by more than 5 mm.

In a practical reflectometric measurement withfringe phase, noise exists on retrieved fringe phase,and further it can be approximately considered asnormally distributed on surface-normal directions.Normally distributed angular noise with a standarddeviation of 8 arc sec is added on the normal direc-tions, which is a typical noise level in practical mea-surement with fringe reflection technique. Theassociated noisy slopes in x and y directions areshown in Figs. 4(b) and 4(c).

The integration error from the traditional least-squares method is shown in Fig. 5(a) without anycompensation. The height error is not only resultedfrom the influence from gradient noise but also fromthe mismatch between the assumed biquadratic sur-faces and the actual surface to be reconstructed.Most of the errors due to this assumption can be com-pensated through the proposed iterative process withshowing the height error in Fig. 5(b). In addition, aone-line profile (y � 0) of errors before and after com-pensation is compared as well in Fig. 5(c), fromwhichit can be easily found that the iterative process im-proves the integration accuracy with successful elim-ination of the waviness, and the residual error ismainly from the gradient noise. The result showsthe iterative compensation is still effective in thepresence of noise.

Fig. 2. (Color online) True shape is designed as the ground truth, and its corresponding gradient data are analytically calculated insimulation to test the performance of the iterative compensation. (a) The true shape, and the corresponding true slopes in x-directionp � dz∕dx (b) and in y-direction q � dz∕dy (c).


Fig. 3. (Color online) Integration results with the traditional method and the proposed iterative compensation are compared, showing thevalidity of the iterations. (a) Height errors with no compensation with the traditional method, (b) after the first compensation, (c) after thesecond compensation, (d) after the third compensation, (e) standard deviations of height error before and after compensations, (f) one-lineprofile of the errors with the traditional method and proposed compensation approach, and (g) the reconstructed result after the thirdcompensation.


4. Experiment

To verify the validity of the proposed iterative com-pensation process, a practical measurement is car-ried out with fringe reflection technique [2,3]. Thetarget to measure in the experiment is a concave mir-ror. The fringe patterns with phase shifting are dis-played on an LCD, screen and the typical imagescaptured by a CCD camera are shown in Fig. 6(a).The slope data in x and y directions determined bythe phase measuring reflectometric method areshown in Figs. 6(b)–6(c).

By applying the proposed integration method withiterative compensations on the measured gradientdata, the surface shape of the concave mirror can bereconstructed as shown in Fig. 6(d). It indicates theproposed method is effective and feasible in a prac-tical measurement. In addition, the shape differencebetween the proposed and the traditional methods isalso investigated and shown in Fig. 6(e). Improve-ment in integration accuracy is achieved by usingthe iterative operation.

5. Discussion

Generally speaking, a fast, accurate, and easilyimplemented iterative compensation process is pre-sented to improve the integration accuracy by

compensating the errors due to imperfection ofbiquadratic surface assumption. The iterations withhigh efficiency do not sacrifice much computationaltime. The merits of the revised integration methodwith the proposed iterative compensations can besummarized as follows.

First, the proposed compensation process im-proves the accuracy of the traditional least-squaresintegration method. It is able to compensate the er-ror from the imperfection of assumption implied inthe traditional least-squares integration method.This is the major improvement.

Second, as improving the accuracy by iterations,the algorithm is still fast since the iterative compen-sation is very effective. Commonly, only one or twocompensations are enough to get a satisfactoryresult.

Third, the same as the traditional least-squaresmethod, the proposed method is able to cope withboth a complete and incomplete data set. A dataset with a size of 500,000 can be handled by theproposed method without stitching.

The limitation for this proposed method is also in-vestigated. As the traditional least-squares integra-tion method, the proposed method can only handle agradient data set with rectangular mesh grid, and it

Fig. 4. (Color online) Integrate gradient data with noise. (a) True shape to be reconstructed, (b) slope p � dz∕dx, and (c) slope p � dz∕dy.

Fig. 5. (Color online) A comparison is carried out between the traditional method and the proposed iterative compensation with theexistence of noise on gradient data. (a) Height error before compensation, (b) height error after iterative compensation, and (c) profilesof height error in one line (y � 0).


is not able to integrate an arbitrarily distributeddata set, but generally the data are in a rectangularmesh grid in many practical optical measurements.

6. Conclusion

This work aims to improve the least-squares integra-tion method with iterative compensation to solve theissue of the incorrect biquadratic shape assumption.The proposed method is investigated by both simula-tion and experiment. Improvement in integration ac-curacy is verified by comparing with the traditionalleast-squares integration method. The merits of theproposed method are accurate, fast, and able to han-dle large data sets. In summary, this least-squaresintegration with iterative compensation method isan effective and accurate 2D integration tool to han-dle shape from slope problems in some gradient-measuring-based optical inspection applications.

References1. F. Chen, G. M. Brown, and M. Song, “Overview of three-

dimensional shape measurement using optical methods,”Opt. Eng. 39, 10–22 (2000).

2. M. C. Knauer, J. Kaminski, and G. Häusler, “Phase measuringdeflectometry: a new approach to measure specular free-formsurfaces,” Proc. SPIE 5457, 366–376 (2004).

3. T. Bothe, W. Li, C. von Kopylow, and W. P. O. Jüptner, “High-resolution 3D shape measurement on specular surfaces byfringe reflection,” Proc. SPIE 5457, 411–422 (2004).

4. B. C. Platt and R. Shack, “History and principles of Shack–Hartmann wavefront sensing,” J. Refractive Surg. 17,S573–S577 (2001).

5. D. L. Fried, “Least-square fitting a wave-front distortionestimate to an array of phase-difference measurements,” J.Opt. Soc. Am. 67, 370–375 (1977).

6. R. H. Hudgin, “Wave-front reconstruction for compensatedimaging,” J. Opt. Soc. Am. 67, 375–378 (1977).

7. R. H. Hudgin, “Optimal wave-front estimation,” J. Opt. Soc.Am. 67, 378–382 (1977).

8. W. H. Southwell, “Wave-front estimation from wave-frontslope measurements,” J. Opt. Soc. Am. 70, 998–1006 (1980).

9. W. Li, T. Bothe, C. von Kopylow, and W. P. O. Jüptner,“Evaluation methods for gradient measurement techniques,”Proc. SPIE 5457, 300–311 (2004).

10. R. Legarda-Saenz and A. Espinosa-Romero, “Wavefront re-construction using multiple directional derivatives andFourier transform,” Opt. Eng. 50, 040501 (2011).

11. J. Koskulics, S. Englehardt, S. Long, Y. Hu, and K. Stamnes,“Method of surface topography retrieval by direct solution ofsparse weighted seminormal equations,” Opt. Express 20,1714–1726 (2012).

Fig. 6. (Color online) A comparison of the traditional and proposed methods is also carried out with experimental data from the fringereflection technique. (a) Fringe patterns from fringe reflection technique, (b) slope in x-direction, (c) slope in y-direction, (d) integratedmirror surface by using proposed method, and (e) the shape difference between proposed method and the traditional least squares in-tegration method.


Documents

Improvement of least-squares integration method with .... Improvement of lea… · This integration process, commonly with a two-dimensional (2D) data set, does affect the profile