Journal of the

O PTICALof

VOLUME 57, NUMBER 3

SOCIETYAMERICA

MARCH 1967

Restoration of Turbulence-Degraded Images*BENJAMIN L. MCGLAMERY

Visibility Laboratory, Scripps Institution of Oceanography, La Jolla, California 92037(Received 22 August 1966)

Turbulence-degraded images have been processed to obtain an improvement of their visual image quality.The initial objects were photographed through laboratory-generated turbulence. The resulting transparen-cies of the degraded images were digitized by a photoelectric scanner and processed on a digital computer.The processing consisted of applying corrections to the amplitude and phase coefficients of the two-dimen-sional Fourier series representing the degraded images. The correction factors were obtained from the opticaltransfer function of the turbulence measured at the time the images were photographed. The experimentwas done for 5-msec and 1-min exposure times. The processed data were used to generate photographs. Theprocessed images were found to have significantly more visual detail than the original degraded images; the5-msec-exposure restorations were superior to the 1-min-exposure restorations.

Index HEADINGS: Photography; Image structure; Atmospheric optics; Computers.

THE transmission of an image through a turbulentTmedium degrades the quality of the final imageformed by an optical system. This degradation causesloss of some of the information contained in the image(in the context of information theory) and increaseddifficulty of extraction of the remaining information bythe human observer. The lost information is irretriev-able. However, the image can be processed so that theremaining information is more easily interpreted by ahuman viewer. If an optical transfer function can beassociated with the turbulence and if this function isknown, then one method of processing is to correct thespatial frequency spectrum of the degraded image bythis function. This method of processing has been dis-cussed by Harris' who showed examples of the methodapplied to time-averaged images degraded by turbu-lence. The transfer function used for correction was ananalytical function based on a simple model of theturbulence and empirically obtained constants. Theprocessed images showed a definite improvement inquality.

In some cases, the effect of the turbulence may resultin an optical transfer function which is not easilyrepresented by an analytical function. An example is

* Contribution from the Scripps Institution of Oceanography,University of California, San Diego, 92152. The research reportedin this paper was supported by the Advanced Research ProjectsAgency.

I J. L. Harris, J. Opt. Soc. Am. 56, 569 (1966).

the case in which the turbulence causes rapid non-symmetrical changes in the structure of the image, andthe image is recorded in a short period of time. Toapply image-processing techniques to a single imageunder these conditions requires that the optical transferfunction be recorded in detail at the same time as theimage. The purpose of this paper is to present experi-mental results demonstrating this type of processing ofimages degraded by laboratory-generated turbulence.The results are shown for both 1-min and 5-msecexposures of the image, to demonstrate the effect ofexposure time on the quality of the processed image.The 1-min exposures are referred to as time-invariantimages because consecutive exposures appear identical;the 5-msec exposures are referred to as time-variantimages because consecutive exposures appear different.

THEORY

For clarity, the basic theory will be briefly reviewed.'Consider an incoherently illuminated object imaged byan optical system. In the absence of turbulence theimage-plane irradiance distribution is the ideal image,H(x,y); in the presence of turbulence, the distributionis the input image HI (x,y). The turbulence is character-ized by an input point spread function SI (x,y) which isassumed to be independent of its location in the imageplane. Since the Fourier transform of Si (x,y) is identicalto the optical transfer function, the relationship be-

293

Copyright © 1967, by the Optical Society of America.

BENJAMIN L. McGLAMERY

Equation (2) is one of the most direct methods ofimage processing; it applies not only to turbulence-degraded images but to many other types as well. Themain problem in applying it to time-variant turbulence-degraded images is in determining the optical transferfunction, F[Sr(X,Y)]. It may be found by including inthe field of view of the optical system an object whoseideal image is known. The input image of this knownobject is recorded simultaneously with the input imageof the unknown object. Let the known and unknownobjects be denoted respectively by the subscripts Kand U. For example, the input image of the knownobject is [HI(X,Y)]K. From Eq. (1) the optical transferfunction can be found by use of the known object:

FESr(x,y)J]= F[Hi(Xy)]K/FEH(x,y)]K. (3)

The optical transfer function found from the knownobject can now be applied to the unknown object.Equation (2) then becomes

[H(x,y)]u =Th{FEHI(x,y)]uXF[H(x,y)]K/F[HI(x,y)]K}. (4)

This is the equation which describes the processing

200 sec ~ 10-3 rad

AERI

.05 cm

- -- - Y- ---.AL IMAGE FORMED /1 50 cm

BY REDUCING LENS

/ 1Ocm

[H (X, Y)]

FIG. 1. Block diagram of the system usedin the restoration experiment.

tween the transforms of H(x,y), HI(x,y), and SI(xy) is

F[H1 (x,y)] = FE1 (xy)]FEsi (x,y)], (1)

where F is the Fourier-transform operator. The idealimage may be found by solving Eq. (1) for FEH(x,y)]and taking the inverse transform:

H (x,y) = F-1 '(FH (x,y)]}

= F-1'F[Hr (xy)]/F[Si (x,y)]). (2)

FiG. 2. Optical system used to obtain theturbulence-degraded images.

COLLIMATOR OBJEC7IVE

FiG. 3. Paths of flux through the turbulence area.

used on the turbulence-degraded images to be shown inthis paper. In the actual processing, the Fourier seriesrather than the transform was used. Equation (4) isan operation applied separately to each frequency termof the Fourier series.

The method of processing as described by Eq. (2)or Eq. (4) is valid only when the point spread functionis independent of location in the image plane. The con-ditions under which this requirement is satisfied dependon the nature of the turbulence and its location withrespect to the object and optical system, the diameterof the entrance pupil of the system, and the exposuretime for the recorded image. More data on naturalturbulence are needed before we can specify how oftenand under what conditions this method of imageprocessing can be applied to images recorded throughnatural turbulence. In this work, the experimentalconditions were chosen so that the point spread func-tion was independent of position.

EXPERIMENTAL PROCEDURE

Figure 1 is a block diagram of the entire processfrom generation of the turbulence-degraded images tothe processed images.

294 Vol. 57

March 1967 RESTORATION OF TURBULENCE-DEGRADED IMAGES

The optical system is shown in Fig. 2. A reduced imageof the objects was formed and the image was effectivelyput at infinity by a collimating lens. The unknownobject was a numeral 5 and the known object was apinhole. In the collimated area, the angular height ofthe numeral 5 was 40 sec of arc, the diameter of thepinhole was 4 sec of arc, and the separation of thenumeral and pinhole was 200 sec of arc. The collimatorand objective lenses were of 230-cm focal length; theobjective was stopped to a diameter of 10 cm, and thedistance between the two lenses was 50 cm. The turbu-lence was introduced in the area between the two lensesby a heater about 40 cm below the optical axis. Figure 3shows the paths of flux through the turbulent area.Since the flux from both objects passed through thesame tube of turbulence within 0.05 cm out of a totalof 10 cm, the requirement that the point spread func-tion be invariant over the images was fulfilled for allpractical purposes.

The objective lens formed the degraded images inits focal plane, from which they were reimaged byanother lens into the film plane of a 35 mm camera.

(a) (b) (c)

FIG. 4. Image-plane photographs. Top row: the known image;bottom row: the unknown image. (a) no turbulence; (b) withturbulence, 1-min exposure time (time-invariant image); (c)with turbulence, 5-msec exposure time (time-variant image).

The time-invariant images were recorded on Plus-Xfilm at one-minute exposure times. The time-variantimages were recorded on Tri-X film at 5-msec. Fluxlevels were adjusted for optimum exposures in bothcases. In addition, gray scales were printed onto thefilm for determination of the film H and D curve.

Following photographic processing, the negativetransparencies were scanned by a system which re-corded the two-dimensional transmittance character-istics on punched cards. The transmittance valueswere measured at intervals 0.1 mm apart on the film.The recorded image of the unknown object in theabsence of degradation was about 2.0-mm high on thetransparency. Thus there were about 20 scan elementsacross the undegraded image. The H and D curve wasobtained by scanning the gray scale printed on thesame transparency.

(a) (b) (c)

FIG. 5. The processed images. Top row: the time-invariant imageof Fig. 4(b) after processing. Bottom row: the time-variant imageof Fig. 4(c) after processing. The processing was done at spatial-frequency cutoffs of 2, 3, and 5 cycles/mm [(a), (b), and (c)].

The card decks were processed on a CDC 3600 com-puter. The first step was to transform the data via theH and D curve into data representing the three positiveimages: [H(x,y)jK, [Hr(x,y)]K, and [Hr(x,y)]u. Theprocessing described by Eq. (4) was then made and adeck of cards representing the irradiance values of theunknown ideal image, [H(xy)]u, was punched out.These cards were read by a system which produced aphotographic print of the processed image.

In reconstructing the processed images, the numberof terms of the Fourier series was varied to producepictures whose maximum spatial frequencies were 2,3, and 5 cycles/mm. This was done in order to choosesubjectively the best compromise between imagesharpness and noise, both of which increase as a func-tion of frequency.

EXPERIMENTAL RESULTS

Figure 4 shows the undistorted objects and the time-invariant and time-variant turbulence-degraded images.The processed images are shown in Fig. 5 for the variousmaximum spatial frequencies. Figure 6 shows the time-variant restoration at 3 cycles/mm after all valuesbelow a constant value were set equal to zero. Sincethe image of interest was essentially superimposed on anearly uniform background, this step resulted in ahigh-contrast picture.

DISCUSSION OF RESULTS

Comparison of Figs. 4 and 5 shows that the processingresulted in either an improvement or further degrada-

FIG. 6. Processed time-variant image ofFig. 5 (b) with the background removed byzeroing all values below a selected con-stant to increase contrast. -

295

BENJAMIN L. McGLAMERY

10,000

1, 000

0 .4 .8 1.2 1.6 2.0 2.4 2.8 3.2 3.6 4.0 4.4 4.8

SPATIAL FREQUENCY, (cycles per millimeter)

FIG. 7. Restoration factors applied to the amplitudes of thehorizontal frequencies of the degraded images.

tion of the quality of the original degraded images,depending upon the number of spatial frequencies usedin the reconstruction. For the reconstructions having amaximum spatial frequency of 2 cycles/mm the resultsfor the time-variant and time-invariant cases areapproximately equivalent. At 3 cycles/mm the time-variant restoration is obviously superior. At 5cycles/mm, which is the maximum reproducible fre-quency of the scanning process, the time-invariantrestoration is completely obscured while the time-variant restoration is quite clear. The reason for thesedifferences can be understood by considering the ampli-tude and phase corrections applied to the spectra ofthe distorted images. These are shown in Figs. 7 and 8.

Figure 7 shows the amplitude restoration factorsapplied to the horizontal frequencies of the degradedunknown images. These factors are the reciprocals ofthe moduli of the optical transfer function.2 Comparisonof the two curves shows that at the higher spatialfrequencies the time-invariant case required muchhigher restoration factors than did the time-variantcase; for the restorations at 5 cycles/mm the time-

2 A nonlinearity, probably in the correction for the film II andD characteristics, caused the fundamental frequency of theoptical transfer function for the time-variant case to have a valuegreater than unity. A partial correction was made by normalizingthis frequency to unity. Therefore, the restoration factors for thetime-variant case shown in Fig. 7 should be considered to beapproximate values.

invariant factors ranged up to 2600, while the time-variant factors ranged up to only 110. Thus the time-invariant images were much more susceptible to theeffects of perturbing factors such as film noise, systemnoise, errors in determining and mathematical repre-sentation of the H and D curve, and truncation errorscaused by recording the initial data with four significantfigures. The effects of these various factors are notsufficiently determined to enable us to say which wasthe major limiting influence.

Figure 8 shows the corrections applied to the phasesof the horizontal frequencies. These are the negativesof the phases of the optical transfer function. In general,the time-invariant case had phase shifts much smallerthan those of the time-variant case.

Figures 7 and 8 reveal an important change of thetype of image degradation due to turbulence whenexposure time is varied. Specifically, the effect ofcertain types of turbulence for short exposures is tocause moderate amplitude attenuations and large phaseshifts while the effect for long exposures is to cause greatamplitude attenuations and relatively small phase shifts.In terms of signal-to-noise ratio a small amplitude atten-uation and a large phase shift is more favorable than theconverse. Thus, from the standpoint of image processingand for a given optical system with a fixed entrance-pupil diameter, the exposure time should be as shortas possible, consistent with flux requirements for goodrecording. If the exposure time and entrance-pupildiameter are both variable and only a limited amountof flux is available, then the choice of the optimumpupil-diameter-time combination becomes more com-plex, since there is a spatial integration effect over theentrance pupil which also affects the optical transferfunction.

2 ok

TIME -VARIANT

.0

tX ~TIME -INVARIANT

0 .4 .8 1.2 1.6 2.0 2.4 2.8 3.2 3.6 4.0 4.4 4.8

SPATIAL FREQUENCY, (cycles per millimeter)

FIG. 8. Phase corrections applied to the phases of the horizontalfrequencies of the degraded images.

i IS I I I I I I I I I I I

296 Vol. 57

March 1967 RESTORATION OF TURBtVLENCE-DVEGRADEr) IMAGES

Although some of the processed images (Fig. 5) showa definite improvement over the unprocessed images(Fig. 4) there is a suggestion of the ideal image in bothof the unprocessed images. We might be tempted tosay that an experienced observer could extract as muchinformation from the unprocessed images as from theprocessed images and conclude that the processing wasnot necessary. Figure 9 shows a case in which this con-clusion would not be valid. The degraded image ofFig. 9 was obtained under nearly the same experi-mental conditions as those previously described exceptthat the image size in the collimated area was 20 secof arc, one-half that of the previous experiment.Visually there is no remnant of the ideal image in thedegraded image; however, the processed image is quiteclear. This experiment shows that a great deal of infor-mation can be obtained within a turbulence-degradedimage which cannot be extracted by the unaidedobserver.

CONCLUSIONS

The results of these experiments show that a signifi-cant improvement of the visual quality of a turbulence-degraded image can be obtained by processing theimage after its initial recording if the detailed opticaltransfer function of the turbulence is known. The ex-periment also shows that greater improvements are

(a) (b)

FIG. 9. Time-variant degraded image before and after processing.The experimental conditions were the same as for the time-variantimage of Fig. 4(c) except that the angular size of the image in theturbulence area was one-half of that of Fig. 4(c).

possible when, for a fixed optical system, the exposuretime used in the initial recording is kept as short aspossible consistent with flux requirements.

ACKNOWLEDGMENTS

Several individuals have contributed significantly tothis experiment. Appreciation goes to J. L. Harris,Sr., whose contributions to image processing were basicto the experiment; to W. H. Richardson, who wrote thecomputer programs; and to N. Richards for her assis-tance in running the programs.

297