Fringe pattern analysis using hybrid image processing

Kunihiko Mori, Yoshiaki Nakano, and Kazumi Murata

Hybrid image processing using optical defocusing and digital computing techniques is investigated for use inthe analysis of interferograms or moire fringes. The optical thickness distribution of a glass plate is measuredfrom the interferogram, and the power distribution of an eyeglass lens is tested from the moire pattern. Theoptical defocusing technique is also used for preprocessing of the input image to suppress high frequencynoises. The optimum defocus amount relating to the processed result is discussed.

1. IntroductionTo measure the surface profile of a reflecting object

or the phase distribution of a transparent object, opti-cal interferometry and the moire method are widelyused. In optical interferometry, information aboutthe object is contained in the phase term of the inter-ference fringe. For exact investigation of this phaseterm, heterodyne interferometry1 which introducestime frequency and a Fourier transform method2' 3

which introduces spatial frequency have been used.In the conventional peak detection method for inter-

ferogram analysis, the phase in formation is examinedby measuing the fringe positions as well as the fringeorders. If the peak positions can be confirmed visual-ly, we can investigate the phase information by thismethod, even when the bias or the contrast of theinterferogram varies spatially. As is well known, if thephase variation is smaller than the wavelength unit,the phase information cannot be investigated by thismethod. However, the conventional method is sim-pler and easier than the other investigative methodsmentioned above. Furthermore, with the convention-al method, phase information of less-than-wavelengthunits can be investigated by introducing a spatial carri-er frequency with a tilt of the reference wave.

Using the moire method, the profile or the amount ofdeformation of a test object can be investigated by

Kunihiko Mori is with Hokkaido Polytechnic College, Depart-ment of Image Processing, 3-190 Zenibako, Otaru 047-02, Japan;Yoshiaki Nakano is with Hokkaido Institute of PharmaceuticalSciences, Physics Department, 7-1 Katsuraoka, Otaru 047-02, Ja-pan; Kazumi Murata is with Hokkaido University, Department ofApplied Physics, N13 W8, Sapporo 060, Japan.

Received 27 February 1989.0003-6935/90/111646.06$02.00/0.© 1990 Optical Society of America.

measuring the peak positions of the moire fringes ob-tained.4 6 For example, the focal length of a lens canbe tested by measuring the inclination angle of themoire fringe.7 8

For the analysis of interferograms or moire fringeshaving sinusoidal intensity distributions, the normalmethod, in which the distribution of the peaks or maxi-mum gradient positions is measured, is widely used.With a view to analyzing the fringes, high speed peakdetection methods have been studied.9 Adaptive bin-arization of the interferogram by means of the localaverage of the image using an optical defocusing tech-nique has also been studied.10

Recently, in interferometry and the moire method,fringes have been detected using a TV camera.Fringes have also been easily recorded photographical-ly. With a high speed camera, a fast moving object orthe momentary situation of an object can be recordedand analyzed. One of the advantages of photographicrecording is that the processing can be done at anotherplace. Therefore, off-line processing is possiblethrough the medium of the photograph.

We studied hybrid image processing using the defo-cusing technique for fringe pattern analysis, and at-tempted to combine the simplicity of the peak detec-tion method and the generalization of fringe patternanalysis by photographic recording. A hybrid imageprocessing system is proposed and some experimentalresults are shown.

11. Hybrid Image Processing Using the DefocusingTechnique

In an interferometer, for example, a Mach-Zehnderor a Twyman-Green interferometer, the intensity dis-tribution of the interferogram is given as

I(x,y) = a(xy) + b(x,y) cos[k(xy)], (1)

where a(x,y) is the bias component and b(x,y) is the

1646 APPLIED OPTICS / Vol. 29, No. 11 / 10 April 1990

contrast component. They are varying functions ofspatial coordinates x and y due to the irregularity ofphotoprocessing or the nonuniformity of incident lightand the nonhomogeneity of optical elements. Here,e(x,y) is the spatial phase distribution depending onthe optical path difference of two light waves. Ingeneral, it is important to measure the phase distribu-tion, for (xy) includes information about the testobject.

On the other hand, moire fringes are formed bysuperimposing two gratings. In moire topography,the profile of an object is investigated by measuring themoire fringe distribution generated by the object andreference gratings. In the moire method, there is onefrequency component that includes phase informationabout the object and a higher frequency componentfrom the two gratings. From this point of view, themoire fringe is different from the interferogram.However, if only the frequency component that in-cludes phase information is considered, the moirefringe can be represented by Eq. (1). To eliminate thehigher frequency component, the moire fringe is im-aged by defocusing which has a low pass filter effect asis well known.

As mentioned above, interferograms and moirefringes are represented by Eq. (1). Therefore, phasedistribution q(x,y) is particularly important in bothinterferometry and the moire method. The simplestway to investigate the phase distribution is by the peakdetection method. There are two approaches: hu-man visual recognition and digital image processing.In digital image processing, it is difficult to detect thepeaks if the variation of bias component a(x,y) in Eq.(1) cannot be ignored in comparison with the variationof the phase term. Therefore, this bias componentmust be eliminated. For this purpose, image binariza-tion using the local average method has been investi-gated. In this method, the image is divided into smallareas, and binarized as a unit for each area by means ofaverage intensity. Let the fringe image be f, the out-put image g, and the local area P X P, then the localaverage operation is given by

P P

g(m,n) = -2 E Zf(m - k,n - 1). (2)k

If the moving average filter is h', Eq. (2) can be writtenas

P P

g(mn) = , , f(k,l)h(m - k,n - 1), (3)h 1

where h' is equal to 1/p 2 in the area of P X P and is zeroin other areas. If we let the point spread function(PSF) of the defocused optical system be h(xy), theoutput image is represented by

g(x,y) = J f(x',y')h(x - x',y - y')dx'dy'. (4)

Equations (3) and (4) are both convolution operations;the former is a digital and the latter is an analogousdescription. It is known that the PSF of the defocusedoptical system has a form like that shown in Fig. 1(a) or

(a)h(xY)A$ x

y

( b)Fig. 1. PSF of the defocused optical system in the case of (a) slightand (b) large defocus amounts. These have approximately square-

of-Bessel and so-called top-hat distribution, respectively.

t-

zLi

z

focused defocused

xFig. 2. Intensity distribution in a cross section of the fringe image.Solid and dashed lines plot the focused and the defocused intensity,

respectively.

Fig. 1(b) for a slight or a large defocus amount, respec-tively. The PSF shown in Fig. 1(b), having a distribu-tion predicted by geometrical optics," is similar to themoving average filter h'. Therefore, the defocusingoperation is expected to give the same effect as thelocal average by adjusting the defocus amount.

In the hybrid fringe image processing proposed here,the local average operation is executed by optical defo-cusing, and the positions of fringes are measured bydigital processing. In Fig. 2, the intensity distributionin a cross section of the fringe image is plotted by asolid line, and that of its defocused image is plotted bya dashed line. If phase (x,y) of the fringes is to be

10April 1990 / Vol. 29, No. 11 / APPLIED OPTICS 1647

evaluated at its peak position, we should localize thepoints of maximum intensity value. To separate onefringe from another, we use the intersect points of thesolid and dashed lines as shown in Fig. 2. Then themaximum value among the intersect points is desig-nated the peak position.

The input fringe image recorded as a reflective ortransparent picture is reimaged by a TV camera. Theimage plane is moved along its optical axis to obtainthe defocused image. By this operation, the magnifi-cation is varied in a conventional lens system. There-fore, fringes cannot be separated from each other bythe proposed method and the exact fringe positionscannot be found. To avoid this difficulty, an aperturestop is put in the front focal plane of the lens system inthis experiment so that the magnification does notchange when the image plane moves. This system issaid to be telecentric on the image side.12

In hybrid image processing, focused and defocusedimages are stored sequentially in a computer disk sys-tem. The defocus amount is optimized by monitoringthe fringes. Then the intersect points of focused anddefocused images are detected and the peak positionsof the fringe are found using a digital computer. Theresult is displayed on the monitor.

Obviously, from the algorithm of this hybrid imageprocessing, image data are processed by line scanning.If the scanning direction in digital processing is close tothe direction of the fringe as shown in Fig. 3, the fringeimage is not accurately investigated by scanning alongonly one direction. However, better results can beobtained by synthesis of the results obtained by scan-ning both horizontally and vertically. In such a way,any distribution of the fringe image can be processedby the use of two scanning directions with a 90° differ-ence. This can be done by changing the row andcolumn data in the computer.

Hybrid image processing, as mentioned above, hasthe advantages of both optical and digital processing.To obtain the optical local average by defocusing is notonly an easy but also a reliable and rapid technique,because the processing is spatially analogous. More-over, the first image may also be slightly defocused.Hence, we may store two images in series, one that isslightly defocused and another that is considerablydefocused. The processing is flexible enough to beused for noisy images, because the noise erasure effectcaused by defocusing is well known. Especially in themoire method, the erasing effect of the high frequencycomponents is expected, so that we can treat only thelow frequency component as including phase informa-tion to be analyzed.

Ill. Use in the Analysis of InterferogramsWe used hybrid image processing for the analysis of

the interferograms generated by a Mach-Zehnder in-terferometer.

In the interferometer, if the reference wave has tiltangle 0 for axis x perpendicular to the optical axis, theintensity distribution of the interferogram is ex-pressed by

- Tst. SCANr __ _ _ _ _ __ _ _ _ _ _ _ __ __ -

ZI

<I(=IUg

Fig. 3. Peak detection of a fringe image by use of scanning inhorizontal and vertical directions.

IFig. 4. Interferogram generated by a Mach-Zehnder interferome-ter with a tilted reference wave. The image in the square area, 64 X

64 pixel, is processed.

I(x,y) = a(xy) + b(xy) cos[2'ruox + ip(xy)], (5)

where uo is a carrier frequency that has the relation u0= sin6/X. In this experiment, an optical slide glass wasused as a sample. If phase distribution 4i(xy) is deter-mined and the refractive index of the sample is uni-formly equal to n, thickness distribution d(x,y) of thesample is given by

d(x,y) = X O(xy)/27r(n - 1). (6)

Therefore, if the peak position in the distribution ofthe interferogram intensity is located at x',y' and thefringe order is m, the thickness distribution is given by

d(x',y') = (mX - x' sinO)/(n - 1). (7)

From this equation, a thickness distribution smallerthan that of wavelength unit X can be accurately ob-tained.

The interferogram of the sample is shown in Fig. 4.The image in the square area having 64 X 64 pixel inFig. 4 is processed. Figure 5 represents the result ofhybrid image processing. The thickness distributionof the sample is calculated using this result and Eq. (7).A 3-D display of the thickness distribution is shown inFig. 6. The thickness values between the above-deter-mined peak positions are lineally interpolated. Forconvenience, the order of the fringe at the upper right-

1648 APPLIED OPTICS / Vol. 29, No. 1 / 10April 1990

I

I

Fig. 5. Resultant peak intensity distribution obtained by hybridimage processing.

Fig. 6. Three-dimensional plot of the thickness distribution of thesample.

hand corner of Fig. 4 is set to 1. Since the referencewave is tilted, the fringe order increases monotonicallyas shown in Fig. 4. Therefore, the fringe orders areeasily decided by computer.

IV. Use in the Analysis of Moire FringesIn this section, we describe the characteristics of

moire fringes, and then use hybrid image processing inthe analysis of moire fringes obtained experimentally.

The moire fringe is generated by superimposing twocosinusoidal gratings having the same period, p, andbeing relatively inclined by small angle 0. Let intensi-ty transmittance I, of the first grating be (1/2) [1 +cos(27rx/p)], and the intensity transmittance I2 of thesecond grating be (1/2)11 + cos[27r(x cosO + y sinO)/p]J.The intensity distribution just behind the second grat-ing is given by

I = I, X I2 = (1/4) + (1/4) cos(2irx/p) + (1/4)

x cos[27r(x cosO + y sinO)/p]

+ (1/8) cosf22r[x(1 + cosO) + y sinO]/p}

+ (1/8) cosf2ir[x(1 - cosO) - y sinO]/p}. (8)

The intensity consists of one bias component and fourcosinusoidal components. The second and thirdterms correspond to the original gratings, the fourthterm is the higher frequency cross term, and the fifthterm is the lower frequency cross term which is themoire fringe. If one of the gratings is deformed, theobserved moire fringes are also deformed with magni-

Fig. 7. Moire fringe pattern generated through a power-distributedlens. The image in the rectangular area, 221 X 249 pixel, is pro-

cessed.

fication. In the moire technique, the informationfrom the sample gives deformation of the grating.Hence, the index distribution or the surface profile ofan object can be investigated by measuring the moirefringes. In the next experiment, the moire fringes on afrosted glass are imaged by defocusing to eliminate theunnecessary higher frequency components.

There are many uses for moire techniques. Themoire fringes in deflection mapping7 are analyzed byhybrid image processing; for example, a power distrib-uted lens can be measured. 8 The refractive power D isgiven by

D(x,y) = [sinO- tana(xy) + cosO - 1] - (9)

where X is the wavelength of the incident light, anda(x,y) the inclination distribution of the curved moirefringes observed.

Figure 7 shows the moire fringe obtained from frost-ed glass. Hybrid image processing is executed by stor-ing slightly defocused and considerably defocused im-ages in series. The moire fringe image having 221 X249 pixel, corresponding to the rectangular area shownin Fig. 7, was processed in this manner. Figure 8 showsthe maximum gradient points and the peak positionsin the intensity distribution of the moire fringe. Thepeak positions corresponding to each moire fringe arenot plotted by smooth curves because the inherenthigher frequency components of the moire and thenoise component caused by the frosted glass remain inspite of preprocessing. Each moire fringe in Fig. 8 isapproximated by a 7th order polynomial as shown inFig. 9. The peak points are directly plotted by thesolid lines in Fig. 8, and the dashed line is the result ofthe polynomial approximation. Figure 10 shows a 3-Ddisplay of the power distribution of the sample. Thepower obtained in this experiment is equal to 14.5diopters in the upper left-hand corner of Fig. 10, and0.087 diopters in the lower right-hand corner.

V. DiscussionWe described hybrid image processing by the use of

optical defocusing and digital computing techniques.

10 April 1990 / Vol. 29, No. 11 / APPLIED OPTICS 1649

\\ \C ..

~~~~~~~~~~~~~~~

,! ) o y2 g$J| (a ) H(UV)

./' ('." '' 1.' ,1 1 .......... 1 ,H,' v)

Fig. 8. Contours of maximum gradient points and distribution ofpeak positions of moire fringes shown in Fig. 7.

(b)Fig. 11. OTFs corresponding to each PSF shown in Fig. 1.

In the experiment, the defocus amount was adjustedby monitoring the fringe image. The relation of thedefocus amount and the processed result is discussedin this section.

Figure 11 shows the optical transfer function [OTF;H(u,v)] corresponding to each PSF [h(x,y)] shown inFig. 1. If the defocus amount is large, the PSF has anapproximately top-hat distribution (1 where (x2 + y2)1/2 < L, 0 otherwise). The OTF of the top-hat functionis equal to Ji(27rLp)/Lp, where J1 is a Bessel function of

*/ g ;> t the first kind, order one, and p (U2 + 2)1/2. If thedefocus amount is increased, the bandwidth of its OTFnarrows.

Fig. 9. The solid lines directly plot the peak positions shown in Fig. The spatial frequency of the bias component a(xy)8, and the dashed lines are the results from the polynomial approxi- in Eq. (1) is assumed to be exceedingly low and an

mation. adequate distance from the spatial frequencies of thephase term. Therefore, defocused image intensity I'of the fringe image is written as

I'(x,y) = a(xy) + FT-'[H(u,v)FTb(xy) cos[0(xy)]J], (10)

where FT denotes the Fourier transform operation.Hence, subtraction of I' from focused fringe image I isrepresented by

I - ' = b(x,y) cos[(xy)]

As shown in Fig. 12, if the spatial frequencies ofcos[ck(x,y)] are in the area where the OTF is nearlyequal to zero, the second term in Eq. (11) can benegligible. Therefore, only the first term is obtainedwithout an unwanted bias component a(x,y).

Fig. 10. Three-dimensional plot of the power distribution of the The intersect points in the intensity distribution oflens. the fringe image and the defocused image shown in Fig.

1650 APPLIED OPTICS / Vol. 29, No. 11 / 10 April 1990

H(u)

\\ \ large defocus

small defocus

_ \ U

Fig. 12. OTFs for different defocus amounts.

2 are used in hybrid image processing. If bias compo-nent a(xy) disappears in the subtraction of the twoimages as shown in Eq. (11), the intersect points ap-proximately coincide with the maximum gradientpoints in the intensity distribution. It is assumed thatbias component a(xy) is nearly constant. If the biasvaries too much, the intersect points do not correspondto the maximum gradient points and some processingerrors occur.

From this discussion, it is clear that the defocusamount must be adjusted so that the OTF of the defo-cus is nearly equal to zero in the spatial frequencyregion of the fringes.

VI. ConclusionFringe images can be easily analyzed by digital im-

age processing combined with the optical defocusingtechnique. We measured the relative thickness distri-bution of the sample from the interferogram, and test-ed a power distributed lens on the moire fringe. Manykinds of fringe image may be analyzed by the proposedmethod. This processing is easy, flexible, accurate,and speedy. A fast moving object and the momentarysituation of an object may be also investigated becausephotographs of their fringe images can be used withinterferometry or the moire method.

References1. R. Crane, "Interference Phase Measurement," Appl. Opt., 8,

538-542 (1969).2. M. Takeda, H. Ina, and S. Kobayashi, "Fourier-Transform

Method of Fringe-Pattern Analysis for Computer-Based To-pography and Interferometry," J. Opt. Soc. Am., 72, 156-160(1978).

3. L. Mertz, "Real-Time Fringe-Pattern Analysis," Appl. Opt., 22,1535-1539 (1983).

4. H. Takasaki, "Moire Topography," Appl. Opt., 9, 1467-1472(1970).

5. D. M. Meadows, W. 0. Johnson, and J. B. Allen, "Generation ofSurface Contours by Moire Patterns," Appl. Opt., 9, 942-947(1970).

6. D. T. Moore and B. E. Truax, "Phase-Locked Moire FringeAnalysis for Automated Contouring of Diffuse Surfaces," Appl.Opt., 18, 91-96 (1979).

7. Y. Nakano and K. Murata, "Measurements of Phase ObjectsUsing the Talbot Effect and Moire Techniques," Appl. Opt., 23,2296-2299 (1984).

8. Y. Nakano and K. Murata, "Talbot Interferometry for Measur-ing the Focal Length of a Lens," Appl. Opt., 24, 3162-3166(1985).

9. T. Yatagai, S. Inaba, H. Nakano, and M. Suzuki, "AutomaticFlatness Tester for Large Scale Integrated Circuit Wafers," Opt.Eng., 23, 401-405 (1984).

10. J. Tokumitsu, S. Kawata, Y. Ichioka, and T. Suzuki, "AdaptiveBinarization Using a Hybrid Image Processing System," Appl.Opt., 17, 2655-2657 (1978).

11. J. W. Goodman, Introduction to Fourier Optics, 1st Edition,(McGraw-Hill, New York, 1968), pp. 124-125.

12. M. Born and E. Wolf, Principles of Optics, 3rd Edition, (Perga-mon, New York, 1965), pp. 186-187.

10 April 1990 / Vol. 29, No. 11 / APPLIED OPTICS 1651