Part 4. Processing 3 - Fourier methods 4.A. FFT requirementsprism.mit.edu/iptk/tutorial/4_fourie.pdf · Tutorial Part 4 Page # 6 Fourier methods. 4.C. Periodic noise removal is commonly

Part 4. Processing 3 - Fourier methods

4.A. FFT requirements4.B. Applying high and low pass filters

4.B.1. Inverting filters4.B.2. Avoiding ringing

4.C. Periodic noise removal4.C.1. In-place operation4.C.2. Periodic signals

4.D. Measurements on the power spectrum4.E. Averaging periodic signals

4.E.1. Processing for peak finding4.E.2. Filters for regular lattices

4.F. Correlation4.F.1. Cross-correlation4.F.2. Autocorrelation

4.G. Convolution and Deconvolution4.H. Halftone and moiré removal

4.H.1. Color images4.I. JPEG compression4.J. The linear Hough transform

4.J.1. Measurements in Hough space

Frequency space offers some attractive advantages for image processing. It makes large linear filtering operations(equivalent to the application of large neighborhood kernels) much faster, and it collects information together indifferent ways that can sometimes separate signal from noise or allow measurements that would be very difficultin the spatial domain.

4.A. FFT Requirements: The common denominator in all of these operations is the FFT or fast Fouriertransform. The software requires that the image be square with dimension equal to an exact power of two (e.g.,128, 256, 512, ...). There are several easy ways to select a suitable region from a larger image. One is to copy andpaste the image into a new window of the appropriate size. The step-by-step process is Select | All; Edit |Copy; File | New (enter the dimensions in pixels); Edit | Paste . If the original image is larger than the newwindow you can shift the image within the window to select the region, before de-selecting the region. Note thatyou must deselect the pasted region before performing any filter operation, or else the program will attempt toapply the process to the entire clipboard rather than just to the image area (usually with unexpected and incorrectresults). This can be done in many ways, for instance by choosing Select | All , by clicking in the image withthe ROI/Marquee tool, or by flattening the image (Layer | Flatten Image). If the original image is smallerthan the new window, the surrounding area should ideally be filled with an intermediate grey value (this is called"padding") to minimize the amplitude of the Fourier terms needed to accommodate the transition to white. Thisfilling can be done using the paintbucket.

Tutorial Part 4 Page # 1 Fourier methods

Another way is to select a region within the current image, either bycarefully dragging the ROI to the appropriate dimensions as shownin the Info window (be sure the dimensions are shown in pixels) orby setting the ROI to a fixed size in the Marquee Options window(double click the Marquee or ROI tool to display this window). Witheither method, it is important to have the image displayed at 100%magnification. If it is reduced in size it will not in general be possibleto actually draw the region of interest with the desired exactdimensions and the FT routines will not be able to run.

Once a grey scale image of an appropriate dimension isready, select Filter | IP•FFT | Forward FT toperform the conversion from the spatial domain to thefrequency domain. This will replace the original image withthe power spectrum, which in most cases will not bedirectly interpretable. However, a variety of processing andmeasurement procedures can be applied, some of whichmodify the power spectrum and reconstruct a processedspatial domain image.

4.B. Applying high and low pass filters . Traditional low-pass filters used for smoothing and high-passfilters for edge sharpening can be applied by selecting a circular aperture in frequency space and keeping eitherthe low frequency data inside or the high frequency data outside the circle. Smoothing the edge of the aperturewill reduce ringing, and of course the cutoff frequency itself depends on what you know about the contents of theimage and what you want to keep or eliminate.

To apply a basic circular aperture to the data, create a new, blank image that is the same size as the original (forexample, select Image | Duplicate , Select | All and press Delete to erase the contents). Then choose theelliptical marquee tool and depress the shift key (which constrains the shape to a circle) and option (Mac) or alt(Windows) which causes the shape to be drawn from its center, and draw a circle starting from the center of thepower spectrum display. The result is a circular aperture that can be filled with black by selecting Edit | Fill .The surroundings can be erased by Select | Inverse and pressing Delete.

If the central circular aperture is black and the surroundings white, the filter will keep low frequencies and erasehigh frequencies. To apply this filter to the stored Fourier transform and reconstruct a filtered spatial domain


image, select Filter | IP•FFT | Filter+Inv . Note: the terms "mask" and "filter" are used interchangeablyhere. In some cases the term mask is reserved for operations that remove or keep portions of the power spectrum,while the term filter is used for the more general case in which some terms may be reduced in magnitude withoutbeing eliminated. Since the filters used in the FFT plug-ins are proportional in value, and a black and white filtercan be used as an ideal mask, the distinction is ignored here. Note that circular masks or filters can also beproduced without manual drawing using the Generate Filters function below (choose annular and set the widthequal to twice the radius).

Original Girl image Power spectrum from FT

Low-pass cutoff filter Inverse FT with low-pass cutoff


4.B.1. Inverting the filter or mask image before performing the inverse transforms thelow pass filter to a high pass filter (the black values mark frequencies that are included inthe inverse transform, white values are eliminated). Select Image | Adjust | Invert toreverse the grey scale of the mask or filter image. Note in the example shown that a singlepoint has been added to the mask at the very center to keep the constant term (this wasdone manually with the pencil tool using the Info window to locate the center - for a256x256 pixel image the center is at 128,128). The central point defines the averagebrightness value of the image.

Hi-pass cutoff filter Inverse FT with hi-pass cutoffSelecting the proper size for the circular filter is sometimes easier if you can see the power spectrum while youare constructing the filter. To do this, do not use the Delete key to erase the newly created duplicate image asdiscussed above. Instead, just draw the circular region right on the power spectrum display or a duplicate of it,and then press Delete to erase the area inside the circle. Invert the selection (Select | Inverse) interchange theforeground/ background colors (click on the tool palette) and press Delete to fill the area outside the circle withblack, and you will have created a hi-pass filter (white in the center and black outside the circle). You can ofcourse invert this (Image | Adjust | Invert) to make a low-pass filter.


4.B.2. The similarity between the low-pass frequency filteredresult and smoothing in the spatial domain, and between thehi-pass frequency filtered result and a Laplacian in the spatialdomain, reveals that these operations are mathematicallyequivalent. In both the high pass and low pass examples thereis a very evident "ringing" phenomenon that arises from thesharp edge of the cutoff filter. There are many ways toovercome this, including simply blurring the mask or filterusing the standard Photoshop blur tools (e.g., Filter | Blur |Gaussian Blur). There are also facilities to generate otherfilter masks, including 1/f, Hamming and Butterworth filters(select Filter | IP•FFT | Gen. Filters). The latter isillustrated below; notice the reduction in ringing. As above,inverting the mask image (and if necessary adding a point atthe center to keep the average brightness value of the image)converts a low-pass to a high-pass filter.

Butterworth low-pass filter Result from low-pass Butterworth


Butterworth hi-pass filter Inverse with hi-pass Butterworth

Instead of using the built-in filter shapes (ideal inverse, Hamming and Butterworth), the user can of course alsoconstruct any desired bandpass filter. The filter is simply a grey scale image, with values that vary between black(unity) and white (zero), typically with radial symmetry. Tools such as the Photoshop gradient tool can be used toconstruct such patterns. There is also a plug-in to assist with this (Filter | IP•FFT | Bandpass). The slidersare used much as a hi-fi equalizer to set amplitude as a function of frequency (shown in terms of pixel spacing);the initial default is an ideal inverse filter. The filter can then applied using Filter | IP•FFT | Filter &Inverse , or by Filter | IP•FFT | Apply Filter followed by Filter | IP•FFT | Inverse .

Bandpass filter dialog and the resulting filter.


4.C. Periodic noise removal is commonly performed in frequency space. In images with periodic noise,which can result from the image sensor, from electronic interference, etc., the frequency transform collects all ofthe noise into a few spots which can be eliminated. There are automatic ways to locate peaks in the FFT image,for instance a top hat filter, but in the first examples we will select them manually. The same forward FT isperformed, and then the paintbrush is used to mark each spot corresponding to the halftone pattern present in theoriginal image. Only points in one-half of the power spectrum need to be marked as the space is symmetric andredundant. Select Filter | IP•FFT | Symmetry to duplicate the marks in any half of the power spectrum inthe other half as well.

Thresholding (Image | Adjust | Threshold) the marked image will select just the darkest pixelscorresponding to the marked spots, after which the mask filter can be inverted (Image | Adjust | Invert)because we want to remove the spots, and smoothed (Filter | Blur | Blur More). Performing the inversetransform (Filter | IP•FFT | Filter+Inv.) produces the spatial domain image with the periodic noiseremoved.

Note that the marking of points using thepaintbrush may be simplified if you set thepreferences to show the brush as a circle of theappropriate size (select File | Preferences |Display & Cursors) and set the PaintingTool Cursor to Brush Size and Other Tools toPrecise).

Original Clock image FT Power spectrum of Clock image


Mask filter to remove periodic noise Inverse FT using mask filter

4.C.1. An example of doing this “in place” in a regionwithin an image is shown using the Clown image. Theregion of interest was drawn using the shift key torestrict the marquee to a perfect square while watchingthe Info window to set it to 128 pixels in dimension.Then Filter | IP•FFT | Forward FT was used totransform just the selected region, manual marking wasused to mark the two dominant noise frequencies, thiswas thresholded and inverted to produce the mask, andthen Filter | IP•FFT | Filter+Inverse used toobtain the image with noise removed.


Manual marking of the spikes in the power spectrum is not the only way to create a filter or mask to removethem. As shown in the example, selecting a region of the image that contains just the noise by drawing a region ofinterest asks the user to identify the noise pattern without having to interpret the power spectrum image. Eraseeverything else in the image (Select | Inverse followed by Edit | Cut and Selection | Deselect) and thenperform the Fourier transform (Filter | Plug-ins | IP•FFT | Forward FT). The power spectrum showsthe spikes that arise from the noise pattern. Treating this like any other grey scale image can produce a filter ormask. In this example, the image was smoothed, leveled, thresholded, inverted, and the central spot filled.

Selected region of periodic noise from Clock image Power spectrum


Filter mask produced from the power spectrum Application of the filter mask to the entire image

4.C.2. In some cases it is the periodic signal that is important. The procedure is similar. Perform the forward FTand mark the spots representing periodic information with a black paint circle. If this is done on one-half of thepower spectrum, select Filter | IP•FFT | Symmetry to duplicate the spots for the entire area. Threshold thespots (Image | Adjust | Threshold) and optionally smooth them (Filter | Blur | Gaussian Blur).Then select Filter | IP•FFT | Filter+Inv. to produce the filtered spatial domain result.

In the next example, another way to create the filter mask is shown. A circular region of interest is drawn in thesame way described above (set the marquee style to Elliptical, depress the shift and option keys to draw a circlefrom the center, and draw the circle so that it fits the circular array of spots that correspond to the periodic spacingof atom planes in the image). Then convert the circular region of interest outline to a line on the image byselecting Edit | Stroke and choosing an appropriate width. Then perform the inverse transform using this mask(Filter | IP•FFT | Filter+Inv.) to produce an image with the periodic atom planes enhanced.


Original Carbon image FT Power spectrum

Annular filter mask Enhanced visibility of atomsTutorial Part 4 Page # 11 Fourier methods

Another way to create an annular filter is to use the circle drawing procedure to mark a circle region (as describedjust above) inside the ring to be preserved and then press Delete to erase it. Then draw another circle just outsidethe ring of frequencies to be kept, choose Select | Inverse to change the circle selection to one that includesthe pixels outside the circle, and press Delete to erase that. This will leave just the ring of spots. Then use Image| Adjust | Threshold to turn all non-white pixels to black, which will produce an annular ring that can besmoothed and used as a filter mask.

Finally, the Filter | IP•FFT | Gen Filters function will directly generate an annular filter by allowing youto enter numbers (the fraction of the radius from center to edge of the power spectrum) for the radius and width ofthe annular ring. There is likewise a sector mask selection that will generate a mask to select a range of directions,by angle. In both cases, inverting the mask can be used to remove rather than keep the range of frequencies ordirections, and applying a Gaussian smooth to blur the edges of the mask will reduce ringing.

4.D Measurements on the power spectrum . In order to generate an annular filter, it is useful to be ableto measure the radius of the ring of spots in the Carbon image. This can be done using the Photoshop cursor andthe Info window, by reading the pixel addresses of the spots and calculating the distance to the center.

With Photoshop 5 it is more convenient to use the rulertool, place one end on the center of the power spectrumand drag the other to the ring, and read the distance inpixels directly in the Info window. The angle is alsodisplayed. (Digital Darkroom offers a similar rulercapability.)


The Carbon image is a goodexample of an image in which theperiodic spacing can be measuredin the FT power spectrum muchmore easily than in the spatialdomain image. A circularlyaveraged radial profile plot of thepower spectrum shows a peak thatcorresponds to the reciprocal of theatom plane spacing.

This plot is generated as one option by the Filter | IP•FFT| Profiles.. . function, which can also be used to plot theintensity in any selected direction (along a radial line from thecenter or DC point to a maximum radius equal to the size ofthe inscribed circle in the power spectrum).

Another example of measuring spacings using the FT power spectrum is shown next. In this case, the spacing isnot perfectly regular but the average and variation in spacing and direction can still be measured efficiently fromthe power spectrum.


Original Hyphilic image. FT Power spectrum.

Circularly averaged radial plot ofpower spectrum. The values andthe minimum and maximum limits(loge) are taken from the actualstored power spectrum, not fromthe 8 bit grey scale display.

Regularly spaced features are much more easily and accurately measured in frequency space than in the spatialdomain image. In the example (Muscle2) the striations are very faint but produce a clear point in the FFT powerspectrum (select Filter | IP•FFT | Forward FT). The location of this point averages all of the repetitivespacing in the image and can be read from the info window by using the cursor.


Muscle2 image (contrast enhanced) FFT Power spectrum

Use the cursor to locate the darkspike in the power spectrumcorresponding to the regular linespacing, and read the location inthe Info window. Since theoriginal image is 512x512, thecentral point in the powerspectrum is at x=256, y=256.Hence the distance from thecenter to the spike is 27 pixels,corresponding to a spacing of512/27 = 18.96 pixels in theoriginal image. With Photoshop5, it is easier to use the ruler toolto measure the distance from thecenter to the spot directly.


4.E Averaging periodic signals can be done in another way as well. Rather than manually selecting thespots in the FT power spectrum, any image processing technique can be applied (such as a top hat filter to selectspots, or background removal to eliminate the gradually varying background) to enhance the spots so that theycan be thresholded to become the filter mask. In the example shown, direct thresholding with (Image | Adjust| Threshold) is adequate to select the peaks, after which the Filter | IP•FFT | Filter+Inv. selection willproduce a result that averages together the information in all of the periodic repetitions of the pattern in the image.

Original Mullite image FT Power spectrum

Thresholded spots (filter mask). Inverted (averaged) image.


4.E.1. Since the power spectrum image can be processedlike any other grey scale image, additional tools can be usedto select just the periodic spots in cases like this. Onepowerful tool for finding small dark spots (the dark spikesin the power spectrum) is the top hat filter. Applying a tophat (Filter | IP•Rank | TopHat ) with an inner radiusof 1 pixel and an outer radius of 3 to thepower spectrumsets all pixels that are within the range (darkest to lightest)of the surrounding neighborhood to medium grey, butleaves pixels that are darker or lighter than theirsurroundings with the difference. Thresholding this imagefor (in this case) the dark spots produces a mask thatspecifically selects the spots, and then applying Filter |IP•FFT | Filter+Inv results in a very clean image ofthe periodic structure.

Filter | IP•Rank | TopHat applied to power spectrum

Thresholded top hat result Inverse FT using the mask


4.E.2. Regular lattices (such as high resolution TEM images of atomic structure) typically generate Fouriertransforms whose power spectra show a regular 2-D array of harmonics. An efficient way to produce a mask orfilter for these spots is to manually mark two fundamental (lowest frequency) spots. All of the other spots willthen be positioned at the locations formed by the vector sums of these point locations (from the center of thepower spectrum), times small integers. The Filter | IP•FFT | Harmonics routine performs this operation. Itrequires that you mark one or two points using some unique color or grey scale value. It finds these marks whichmatch the current pen or foreground color, and generates the harmonic spots which are shown superimposed inblack on the power spectrum. They can be thresholded using Filter | IP•Threshold | BiLevel or Filter |IP•Threshold | Pen Color (with the pen set to black) and then used as any other filter. The examples showthe application of this procedure to the HRTEM image and the Muscle3 image (the latter requires only a singlepoint since the pattern has only one direction).

Original HRTEM image Fourier power spectrum with two points marked.


Automatically generated lattice mask Inverse transform showing averaged atomic image

Original Muscle3 image (fragment) Power spectrum with one point marked


Row of harmonic frequencies selected Filter and Inverse result

4.F. Correlation: Fourier space is also used for locating patterns within images. Cross-correlation can beunderstood most straightforwardly as an operation in the spatial domain. If the shape of a target is made into akernel whose values are multiplied by the pixel values at every location in the image and normalized for theabsolute values of the pixels, the largest response will result when the image contains the same pattern of greyscale values. However, the operation is more efficiently performed in the Fourier domain by multiplying(remembering that these are complex numbers) the transforms of the target and the image together andperforming an inverse transformation on the result. With either implementation, the locations in the image thatmatch the targer are found efficiently even in the presence of noise, shading, or partial obscuration of the target.

4.F.1. Cross-correlation requires two images, which need not be the same size (the program automaticallypads them to the nearest power of two). First copy the target image into memory using Setup 2nd Image .Then select the image to be matched, and choose Filter | IP•FFT | CrossCorrelate . As shown in theexamples, the program returns the magnitude of the cross-correlation value which marks the location of matchingregions. The first example is an artificial image containing text and the second one a real image of latex spheres.

TextSam1 image TextSam2 image Cross-correlation result


The cross-correlation image is shown first using indexedcolor (Image | Mode | Indexed Color) to assist inlocating the maximum point, at the location of the scriptm in the image. The image can also be rendered as asurface (Filter | IP•Render | Surface) to show thehighest point. Notice that the rotated m at the bottom ofthe image is not matched. Cross correlation matching oftext requires the letters to be the same size, font andorientation.

Image of latex spheres (XC_Many) Target image (XC_Targt)


Cross-correlation image Rendered cross-correlation image showing matches

The cross-correlation technique can also be applied to color images. All of the Fourier processing routines extractthe grey scale brightness from color images to use in calculating the transforms. The example (Trees) alsoillustrates the use of a feature within the image as the target. Manually draw a rectangular region of interestaround a single, well defined treetop. Select Filter | IP•2nd Image | Setup to put this region into thesecond image buffer. Then remove the selection marquee (Select | Deselect) and perform the cross-correlation(Filter | IP•FFT | CrossCorrelate). The result is a grey scale image that shows bright spots correspondingto the tree tops, which can be processed (e.g. with a top-hat) thresholded and counted.

Original Trees image with selected region of interest.


Cross-correlation result.

4.F.2. Autocorrelation is similar mathematically to cross-correlation except that the image itself is used as thetarget image. This is used to find degrees of self-similarity or repetition within an image. The procedure is thesame as the cross correlation except that the same image is used as the target. Select Filter | IP•2nd Image |Setup , then Filter | IP•FFT | Cross Correlate .

Original Shotblasted image Autocorrelation image


In the example, the autocorrelation image shows thespatial correlation in the shotblasted surface (a rangeimage) and reveals a slight anisotropy, shown bydrawing several contours (Filter | IP•Threshold |IsoLines). This is one of the standard methods forquantifying anisotropy in surfaces.

When applied to an image with periodic structure, theautocorrelation function identifies and isolates thatstructure. For example, when applied to the Muscle2image shown above the result is the regular spacing ofthe striations.

4.G. Convolution and Deconvolution: Convolution in the Fourier domain is identical in result to that inthe spatial domain, in which a kernel of values (a blur function) is applied to every pixel in the image (Filter |IP•Process | Convolve). In Fourier space, this simply involves multiplying the transform of the blur functionwith the transform of the image, pixel by pixel (select Filter | IP•FFT | Convolution). In many cases it willbe easier to construct the kernel as an image rather than as a set of numerical values.This is particularly true whenthe kernel is not symmetrical. Sometimes the blur function can be measured directly by acquiring an image of apoint (e.g., a star in astronomical images). Some examples of such convolutions are shown applied to the Bugimage below.


This also implies that it is possible to remove a blur from an image by dividing by the transform of the blur, andthis process is called Deconvolution. The Filter | IP•FFT | Deconvolution plug-in carries out thisprocedure. It deconvolves the main image using the contents of the image previously designated with the Setup2nd Image function. The two images do not need to be the same size, nor are they restricted to powers of 2 insize. Deconvolution is typically used to remove blur due to out-of-focus optics or motion from images; thesecond image must contain the point spread function of the blur. In some cases it is practical to acquire such animage by imaging a point (astronomers use the image of a star for this purpose). The deconvolution is actuallyperformed in the frequency domain by dividing the complex transform of the main image by that of the secondimage, but apodizing the values so that points in the second image transform with small amplitudes are not used.The transform of the reconstructed image is stored and can be displayed using the Filter | IP•FFT | PowerSpectrum function.

In the examples shown, the Bug image has first been convolved with three different types of blur or “point spreadfunctions” (using Filter | IP•FFT | Convolution) , and then Filter | IP•FFT | Deconvolution used torecover the original image. The first example is a symmetrical blur as might be produced by out-of-focus optics,the second is motion blur which corresponds to a straight line (whose direction and distance must be known), andthe third example is a “positive-negative” point spread that corresponds to oblique illumination or DICmicroscopy.

Bug image blurred with symmetrical point spread function (center) and deconvolved

Bug image blurred with line (motion) function (center) and deconvolved


Bug image convolved with DIC point spread (center) and deconvolved

4.H. Half tone and moiré removal: Removal of half-tone patterns from photographs is often an importantproblem to address, and processing in frequency space is a much better solution than processing such as blurringthat can hide the dots but also reduces edge sharpness. The resolution of the scanner used to input the image alsoaffects the high frequency noise pattern in the acquired image and can produce additional moiré patterns. Thisartefact is also characteristic of grey scale images obtained from single-chip video cameras. The best method forremoving such regular patterns utilizes Fourier transforms and works in frequency space instead of the originalarray of pixels in the image.

The example shows an example of an image scanned in from a newspaper; the halftone dots are very evident.This is actually a small portion of the original image, enlarged to show the dots. In the power spectrum from aFourier transform, produced using Filter | IP•FFT | Forward FT , the magnitude of low frequencyinformation (gradually varying grey scale with position in the image) is plotted in the center of the array and highfrequency information (rapidly varying grey scale with position) is plotted at increasing radii outwards. The darkspots ("spikes") in the power spectrum far away from the center are the indication of periodic noise in the image.They align with the repetitive pattern of printed dots, and their darkness indicates how much of variousfrequencies are present. Removing them is equivalent to removing the periodic noise. A mask was produced toremove these spikes. This could have been done manually with the paint brush, but instead the top hat filterdiscussed above was used. A small Gaussian blur was applied to the mask so that the pixels on the edges of theholes in the mask are set to grey, which improves the results as shown before.


Original Halftone image FT power spectrum

Mask to remove the periodic noise spikes Image with periodic noise removed

The result of applying this mask as a filter to remove the spikes (select Filter | IP•FFT | Filter+Inverse)shows that the periodic noise has been removed without affecting any of the other information present. There isstill some pixel-to-pixel variation because the image has been scanned at a higher magnification than it wasprinted. A second filter was employed to remove the remaining pixel noise. This is a Butterworth second-orderhigh-frequency cut-off filter (Filter | IP•FFT | Gen Filters), which keeps low frequencies (gradualvariations in grey scale) while progressively cutting off higher ones (the more rapid variations associated withpixel noise). In this case the mid-point of the cutoff was set to the spacing of the half-tone dots in the originalimage.


Butterworth filter Power spectrum with both filters applied

The final version of the power spectrum,with the periodicspots removed and the high frequencies attentuated can beshown (Filter | IP•FFT | Power Spectrum) forinformation purposes. Selecting Filter |IP•FFT |Inverse produces the final image. Note that even thecat's whiskers which are barely discernible in the originalimage can be clearly seen. The use of a smoothing or blurfunction would have erased these fine lines long beforethe halftone dots were smoothed out.

Final processed image

4.H.1. The desirability of processing in frequency space is especially true for color images, in which the half-tone patterns used for the different color planes have different orientations in order to avoid visible patterns in theprinted image. In this case, the image can be separated into RGB planes and each one subjected to an FFT,removal of the noise peaks, and then recombining the resulting planes to form the final image. In the exampleshown, the three FFT power spectra are shown as color planes as well to emphasize the different locations of thenoise peaks in each. Depending on the origin of theimage, it may be necessary to separate the image into CMYplanes.


Original Membrane color image Recombined RGB image after filtering

RGB combination of separate FFT power spectra of theR G B planes of the original image, showing the distincthigh frequency noise peaks from each plane. For manyprinted images, separation into CMYK planes is a bettermethod because those are the color inks used to print theinterleaved halftones that make up the image.

4.I. JPEG Compression , one of the major compression techniques used in multimedia imaging, is based ona frequency transform - the discrete cosine transform is closely related to the FFT. The power spectrum orfrequency transform is scanned and small terms are eliminated, which results in compression. For visualreproduction, the amount of information in the few terms kept may be enough, and image storage requirementsmay be reduced by a factor of 10 or more, but note that edge contrast is distorted, small details are lost, and otherartefacts appear that would seriously compromise image measurements. Photoshop supports opening and savingJPEG images, so you can experiment with compressing images by various amounts and then re-opening them forvisual comparison. The FT power spectrum of the compressed image will show that some information has beenremoved but not where. The JPEG compression method transforms separately each 8x8 pixel square in the


original image and eliminates terms that have small magnitudes, but the specific terms are not the same for allregions depending on the presence of detail there.

Girl image (JPEG compressed 10:1) Power spectrum of compressed image.

Difference between original and compressed powerspectra.

The loss of information in JPEG compression can also be seen by subtracting the original image from one aftercompression. In the example shown the difference has been contrast expanded to emphasize these effects. Notethat edges have been displaced, features altered in texture and dimension, colors shifted, and that the brightnessoffsets between adjacent 8x8 pixel blocks can be seen. In this case the compression reduced the original file size


from about 550K to about 24K bytes, or a 20:1 compression. Larger degrees of compression make the effectsmore noticeable.

Original Flowers image Difference between original and JPEG version

The images shown in this tutorial have all been JPEG compressed from the original versions. This is done byAdobe Acrobat® in order to compress the size of the “pdf” files. Many of them are also shown reduced in size.But the original versions are on the CD-ROM and may be examined before and after processing.

4.J. The Linear Hough Transform. There are many transforms besides the familiar Fourier transformthat map the image from the spatial domain of pixels into some other space where particular kinds of informationcan be extracted or processing performed. One that is quite powerful is the Hough transform. The implementationprovided as in this set of plug-ins is the Linear Hough Transform, which is useful for finding linear alignments ofpoints in an image. The transform uses a space with polar coordinates of angle (the slope of the line, representedon the horizontal axis from 0 to 180 degrees) and distance (how far the line passes from the origin of the image,which is located at the upper left hand corner of the original image, and represented on the vertical axis with arange of ± the diagonal dimension of the image.

To execute the forward Hough transform, which maps every black (pixel value zero) point in an image into theHough space, first store the image of interest as the "second" image (either the Filter | IP•2nd Image |Setup). Then create a new image (File | New) to use as the Hough space. This image does not need to be the


same size as the original spatial domain, in fact it will often be much larger to provide better resolution of the ρ,θvalues for the line(s). Then select File | IP•Lines+Points | Hough(Fwd) to calculate the Houghtransform.

Single marked point in a spatial image. The forward Hough transform represents it as a sine curve.

Each pixel in the original spatial image that is black (non-black pixels are ignored) will generate a portion of asine curve in Hough space, corresponding to the slope and intercept values of all of the possible lines that wouldpass through the original pixel location in the spatial domain image. You may experiment by marking singlepoints or a few points using the pencil or paintbrush tool and calculating the transforms to understand this better.Likewise, each point in Hough space corresponds to a line in the spatial domain. The inverse Hough transformuses the ρ,θ values of any black pixel (non-black pixels are ignored) to draw the corresponding straight lines in

the spatial domain image. To use the inverse transform, store the Hough transform image as the second image(Filter | IP•2nd Image | Setup), select the original spatial domain image as the foreground image, andchoose Filter | IP•Lines+Points | Hough(Inv) . The program will draw the lines in the currentforeground or pen color on top of the original image.


Single point in Hough space. The inverse Hough transform makes it a line.

The key to the usefulness of Hough space is that when two (or more) points are present in the spatial image, theirsine curves cross each other and the locations of the crossing points identify the slope and intercept of the line thatwould pass through the original points. In the example shown, there are several points (actually small circlesmarked with the paint brush) that more-or-less lie on a straight line. In the Hough transform this produces acrossover point, and the Hough space adds together all of the sine waves so the maximum value is easily found.This is the set of ρ,θ values that identifies the line that is the "best fit" to the points.

Many points in the spatial domain. The forward Hough transform (note crossover at left).


The specific grey scale coding used in the transform assigns a pixel value of 1 to each sine wave (nearly black, sothe sine waves show up well), and increments the value for each overlapping sine wave. Hence the brightest pointin the crossover region identifies the desired point in Hough space (and its value reports the number of points inthe spatial image that "voted" for it). Converting this point to black (pixel value 0) can be done either manuallywith the pencil tool or using automatic image processing tools, such as a top hat filter. In the example, thisproduces a line that passes through or near all of the points and identifies the alignment which they represent.This provides a tool to find such alignments of disconnected and irregularly spaced features.

The line drawn on the original spatial domainimage by the inverse Hough transform of thecrossover point.

Such alignments are sometimes easy to recognize visually, because human vision is sensitive to such things (it isone reason that we see constellations in the night sky). But the Hough transform provides a tool to fit lines to thedata and determine the best-fit angle and position. In the Sat-1 image, for example (only a portion of the image isshown), there is a straight feature that might be a road or fault. Processing this with a Sobel operator highlightsthe disconnected pixels along the edges of the feature. Selecting a region around this feature (the Lasso tool wasused to select the region, the Select | Inverse was used to reverse the selection and the Delete key used toerase the surrounding region) and thresholding (Image | Adjust | Threshold) to make just the pixels with alarge gradient value produced a suitable set of points for the transform. Finding the maximum value in thetransform image and using the inverse Hough transform to draw the image back onto the original Sat-1 imageshows the alignment found.


Sat-1 image (fragment) Selected portion after Sobel processing

Line reconstructed from maximum point in theHough transform

4.J.1. Hough space is also useful for measurements. The Linewidt image was processed using a horizontalderivative, thresholded, and the Hough transform applied. In ths resulting image the four crossover points thatcorrespond to the vertical lines along the sides of the vertical structures in the SEM picture are clearly marked.Even though the original image is noisy so that a continuous delineation of the edges would be difficult, the datafrom many points are combined in the Hough transform space so that the separation distance between the linescan be easily and accurately measured (the vertical spacing between the crossover maximum points). The Houghtransform image can be made as large as needed to provide the desired measurement resolution.


Original Linewidth image

Processed (horizontal derivative)

Thresholded to delineate edge pixels


Hough transform (vertical separation of the bright spots gives spacing of edges).

The Hough transform can also be used to measure orientation. In the example shown (IC image), the edges of thedevice are oriented at a slight angle. Applying an edge-enhancement operator (Filter | IP•Process |SobelMag) followed by thresholding produces an image of pixels along the edges. Making this the secondimage (Filter | IP•2nd Image | Setup) and then creating a new image (File | New) whose width is 720pixels (providing enough precision to measure the orientation accurately), followed by calculating the Houghtransform (Filter | IP•Lines+Points | Hough(Fwd) , produces an image in which the bright spots thatmark the lines are found at two angles, as shown on the image. These are about 3.25 degrees less than 90 and 180degrees, and so a rotation of the image by that amount produces good alignment. The rotation was accomplishedby duplicating the image in a layer (Layer | Duplicate Layer) and then selecting numerical rotation (Layer| Transform | Numeric) and entering in the desired rotation value.


Original IC image Sobel Edge-enhancement and Thresholding

Hough transform with bright spots marked

Image rotation dialog Rotated resultTutorial Part 4 Page # 38 Fourier methods


Documents

Part 4. Processing 3 - Fourier methods 4.A. FFT requirementsprism.mit.edu/iptk/tutorial/4_fourie.pdf · Tutorial Part 4 Page # 6 Fourier methods. 4.C. Periodic noise removal is commonly