8
PREPROCESSING OF BIOMEDICAL IMAGES FOR NEUROCOMPUTER ANALYSIS* Dwight D. Egbert, Ph.D. (member IEEE), Edward E. Rhodes, MS. (member IEEE) Department of Electrical Engineering / computer Science and Philip H. Goodman, M.D., F.A.C.P. Department of Internal Medicine, School of Medicine University of Nevada Reno ABSTRACT Thermal infrared images were analyzed to evaluate the growth of tumors in laboratory mice. Special segmentation algorithms were used to extract the areas of interest. The data were then normalized and used to train a three level backpropagation neural network which was able to learn to distinguish between seven stages in a tumor growth. Following this, the data were level sliced and feature attributes were extracted in the spatial frequency domain. After all preprocessing, a much smaller and faster backpropagation network was able to accurately classify the seven different tumor stages from less than 1 percent of the original image data. Classification accuracy was maintained, even when up to 30 percent noise was added to the images. INTRODUCTION The long-term objectives of the research reported here are to develop biomedical image evaluation techniques for use as part of decision support systems. These techniques would be useful for a broad range of image generating technologies such as thermography, computed tomography, magnetic resonance imaging, positron emission tomography, neurometric electroencephalography, and laser based microscopy. The results presented here are derived from incorporating neurocomputing analysis as part of the overall image characterization process. Neurocomputer techniques are being used to support and extend previous research efforts using conventional and specially developed algorithmic pattern analysis approaches.[ll This paper reports on the application of neurocomputing techniques to a specific thermography program: the analysis of the growth pattern of muscular tumors in laboratory mice. In order to effectively utilize the neurocomputer, several preprocessing procedures were evaluated. Special segmentation algorithms were used to extract the areas of interest, the data were level sliced, and feature attributes were extracted in the spatial frequency domain. The data were then normalized and used to train a three level backpropagation neural network. * Supported in part by Grants from Computerized Thermography Centers / IFEX, inc. New York, NY and Research Advisory Board, University of Nevada Reno, Reno, NV. 1-561

[IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

Embed Size (px)

Citation preview

Page 1: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

PREPROCESSING OF BIOMEDICAL IMAGES FOR NEUROCOMPUTER ANALYSIS*

Dwight D. Egbert, Ph.D. (member IEEE), Edward E. Rhodes, MS. (member IEEE) Department of Electrical Engineering / computer Science

and Philip H. Goodman, M.D., F.A.C.P. Department of Internal Medicine, School of Medicine

University of Nevada Reno

ABSTRACT

Thermal infrared images were analyzed to evaluate the growth of tumors in laboratory mice. Special segmentation algorithms were used to extract the areas of interest. The data were then normalized and used to train a three level backpropagation neural network which was able to learn to distinguish between seven stages in a tumor growth. Following this, the data were level sliced and feature attributes were extracted in the spatial frequency domain. After all preprocessing, a much smaller and faster backpropagation network was able to accurately classify the seven different tumor stages from less than 1 percent of the original image data. Classification accuracy was maintained, even when up to 30 percent noise was added to the images.

INTRODUCTION

The long-term objectives of the research reported here are to develop biomedical image evaluation techniques for use as part of decision support systems. These techniques would be useful for a broad range of image generating technologies such as thermography, computed tomography, magnetic resonance imaging, positron emission tomography, neurometric electroencephalography, and laser based microscopy.

The results presented here are derived from incorporating neurocomputing analysis as part of the overall image characterization process. Neurocomputer techniques are being used to support and extend previous research efforts using conventional and specially developed algorithmic pattern analysis approaches.[ll This paper reports on the application of neurocomputing techniques to a specific thermography program: the analysis of the growth pattern of muscular tumors in laboratory mice.

In order to effectively utilize the neurocomputer, several preprocessing procedures were evaluated. Special segmentation algorithms were used to extract the areas of interest, the data were level sliced, and feature attributes were extracted in the spatial frequency domain. The data were then normalized and used to train a three level backpropagation neural network.

* Supported in part by Grants from Computerized Thermography Centers / IFEX, inc. New York, NY and Research Advisory Board, University of Nevada Reno, Reno, NV.

1-561

Page 2: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

After preprocessing, the backpropagation network was able to accurately classify seven different tumor stages from less than 1 percent of the original image data. Classification accuracy was maintained, even when up to 30 percent noise was added to the images.

Thermography research at UNR and elsewhere has shown that several statistical parameters can be reliably used in patient diagnosis support.[21 software developed at UNR has resulted in a complete system for processing biomedical images using these pattern analysis techniques. Clinical application of this system has proven that it provides an effective analysis environment.

Even though this system has proven successful, research to improve it is continuing through the development of new and more powerful analysis techniques. The most promising new approach is the incorporation of a neurocomputer coprocessor as an adjunct to the algorithmic procedures.

IMAGE ACQUISITION AND SEGMENTATION

The original thermal images were obtained at the University of Nevada Infrared Research Laboratory. This laboratory is environmentally isolated with a laminar air flow system which replaces the entire room's air each 15 seconds and maintains a constant temperature to within 0.2 degrees centigrade.

Within this laboratory a BSI 7000 Infrared Imaging System generates a digitized thermal image, of up to 6 4 0 by 5 1 2 pixels, of the subject positioned in front of the scanner field-of-view. The image data are recorded as 1 6 bit values and represent the subject's temperature with a variable resolution, which at one meter distance is 0.5 millimeter per pixel, at a precision of 0.05 degrees centigrade.

Data from the BSI 7000 were transferred to an IBM AT computer via an IEEE 4 8 8 interface, and analysis research was accomplished independently of the BSI 7000. During this transfer the data were restricted to a range of 15.0 to 4 0 . 0 degrees with a precision of 0.1 degree centigrade so they could be stored as one pixel per byte. All neurocomputing was performed on a Hecht-Nielson ANZA Neurocomputing Coprocessor (AZ500) capable of implementing neurocomputing models with 30,000 processing elements and 480 ,000 interconnects, and processing up to 45,000 interconnects per second.

All analysis was performed on portions of each thermal image which had been extracted using one of two segmentation algorithms developed at UNR. The first of these algorithms uses a travel axis with variable length lines perpendicular to the travel axis which are used to interactively define an image segment which is then re-rasterized and saved as a new image file. This technique was developed to extract arbitrarily shaped subsections from raster scan images for which the segment shape has physiological significance.

The second algorithm generates a focal segment image file with circularly symmetrical re-rasterization. The area to be segmented is defined by a center point and a radius (R). The center point becomes the first row of a new rectangular segment, while each column is calculated by moving out along the radius in unit steps and averaging all pixels on that radius over an arc of 5.6 degrees. This results in an image segment file with 6 4 co1,umns and R rows.

1-562

Page 3: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

These segmentation algorithms are the first of the image preprosessing steps. For clinical applications they have proven to be valuable. Because these techniques create data arrays which are symmetrical about a polygon travel axis or focal center, pattern analysis calculations relative to the segment orientation and shape are straight forward and easily implemented.

PRELIMINARY IMAGE ANALYSIS

The Department of Biochemistry in the UNR School of Medicine is currently involved in the development of a new family of chemotherapy drugs. While current drugs act on the basis of inhibiting the reproduction of cells at the DNA level, the new family of drugs operate on a different mechanism. This new mechanism is related to the supply of blood to the cancer.

A special breed of laboratory mice has been developed which will not reject human cancer cells. Thus, the evaluation of the drug's efficacy can be performed using actual human cancer cells as opposed to animal cancer cells. Further, another characteristic of these mice is that they are hairless. Thus , the techniques of thermography can be i lssd to evalilate t h e growth of the tumor as well as the distribution of blood flow in the area of the tumor.

For the neurocomputing analysis described here seven different thermal images were used, as shown in Figure 1. Because the tumors have a circular symmetry the focal segmentation algorithm was used for the majority of the analysis. As a control, the travel axis algorithm was used to extract a square segment containing the tumor in its original orientation. Figure 2 shows a four gray level image for both the travel axis and focal segment number four. All seven images were obtained from a single mouse over a period of two weeks following the injection of the cancer to follow the progression of the tumor growth.

Each travel axis image segment which covered the entire tumor was 79 by 79 pixels. The corresponding focal segments were 64 by 40 pixels. Thus, the focal segmentation reduced the image data content from 6,241 pixels to 2,560 pixels. backpropagation network with any reasonable number of hidden layer processing elements on version 1.2 of ANZA. The primary restriction of this version is the limitation of 16,384 interconnects between any two network layers imposed by the use of the large memory model of the Microsoft C compiler.

However, even these moderate sizes were too large to implement a

In order to construct a reasonable network it was necessary to reduce the image segment sizes. The travel axis segments were reduced to 19 by 19 pixels, and the focal segments to 21 by 13 pixels. With this reduction a network for the travel axis segments was constructed with 361 input elements, 45 hidden layer elements, and 7 output elements. The focal segment network consisted of 273 input elements, 59 hidden layer elements, and 7 output elements.

This was accomplished through simple pixel averaging.

Each network was trained with repeated applications of the seven image segments. During training the desired output for one of the 7 output elements corresponding to the correct segment was specified as 0.95 and for the other six outputs as 0.05. A threshold test was performed after each iteration of the network to evaluate the learning status.

1-563

Page 4: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

Figure 1. Seven thermal infrared image segments showing the progress of tumor growth over a two week period (above). Thermal image of laboratory mouse with tumor at stage four (below).

1-564

Page 5: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

Figure 2. Example 4 gray level images for image segment number four, (a) travel axis, (b) focal segment.

The network was considered to have learned to distinguish between all seven tumor stages as soon as the one correct output was greater than 0.92, while each of the other six outputs were less than 0.08, for all of the seven image segments. The networks were able to learn from either set of image segments. The travel axis segments were learned in 965 iterations, and the focal segments in 575 iterations.

Next, each of the original image segments was level sliced into a reduced number of intensity levels, The original temperature range was 5.8 degrees centigrade producing a maximum of 58 gray levels. A sequence of tests were performed with 32, 16, 8, and 4 gray levels. The backpropagation networks were able to learn the tumor stages even with 4 gray levels. In fact, because of the low spatial frequency nature of the tumor characteristics the 4 gray level image segments were learned faster than the originals. The travel axis segments were learned in 590 iterations, and the focal segments in 323.

As a test of the recognition capabilities of the networks random noise at levels between 10 and 30 percent was added to each of the original image segments. Figure 3 shows examples of the 4 gray level images with noise for both travel axis and focal segment number four. In all cases the networks were able to accurately classify the images with 30 percent noise. Figure 4 shows the output states of the networks for all seven focal images at 58 and 4 gray levels with 30 percent noise. The 4 gray level classification has started to degrade at this level of noise.

FREQUENCY DOMAIN ANALYSIS

Even though the backpropagation networks were able to classify the tumor stages and recognize them in the presence of noise the two dimensional nature of the spatial patterns has been lost at the one dimensional network input. In order to characterize the two dimensional patterns a discrete two dimensional fast Fourier transform (FFT) was calculated for each image segment using a 128 by 128 sample space.

1-565

Page 6: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

Figure 3. Example 4 gray level images with 30 percent noise number four, (a) travel axis, (b) focal segment.

SEGMENT OUTl OUT2 OUT3 OUT4 OUT5 OUT6

1AN30 4 0.90 0.06 0.07 0.05 0.01 0.07 2AN30-4 0.06 - 0.95 0.05 0.04 0.07 0.03 3AN30-4 0.10 0.00 - 0.93 0.07 0.05 0.04 4AN30-4 0.06 0.02 0.07 0.72 0.07 0.06 5AN30-4 0.05 0.09 0.05 0.03 - 0.79 0.05 6AN30-4 0.03 0.01 0.01 0.06 0.04 - 0.80 7AN30-4 0.01 0.03 0.03 0.08 0.03 0.22

INPUT -

SEGMENT OUTl OUT2 OUT3 OUT4 OUT5 OUT6 INPUT 1 ~ ~ 3 0 4 0.83 0.03 0.07 0.05 0.04 0.02

4~~30-4 0.02 0.03 0.01 - 0.87 0.05 0.06

2FN30-4 0.05 - 0.74 0.01 0.14 0.04 0.00 3FN30-4 0.06 0.00 - 0.67 0.15 0.01 0.03

5FN30-4 0.13 0.06 0.01 0.37 - 0.45 0.08 6FN30-4 0.16 0.01 0.02 0.17 0.06 0.66 7FN30-4 - 0.03 0.01 0.06 0.07 0.02 0.02

Figure 4. Backpropagation network output states for 4 gray

for image segment

OUT7

0.02 0.05 0.05 0.06 (a) 0.04 0.20 0.93 - OUT7

0.00 0.00 0.02 0.01 (b) 0.01 0.03 0.25

level input images with 30 percent noise added, (a) travel axis, (b) focal segments,

It is well documented that the FFT phase component contains the bulk of the pattern information in images.[3] However, the quantity of information in the FFT is even larger than in the original image. Therefore, if this information is to be of value in neurocomputing some rational method of reduction is required. Optimum sampling procedures for the FFT still require large amounts of data.141 An examination of the features of interest for distinguishing the tumor stages shows that the low frequency components contain the bulk of the information.

1-566

Page 7: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

Unfortunately, conventional graphical displays of the FFT phase and magnitude are more difficult to distinguish and classify than the original images. The most visually distinguishable displays were found to be two dimensional plots of the real vs. imaginary components of the FFT drawn in such a way that as the spatial frequency array is scanned a line is drawn from one r,i point to the next. Further, it was found that by using only those real and imaginary components representing the largest 50 to 100 FFT magnitudes distinctive patterns were generated for each of the seven image segments.

Figure 5 shows the real vs. imaginary plots for image segments four and five using only those 50 points which have the largest magnitude. Even though these two segments look very much alike in figure 1, their plots in figure 5 have quite distinctive shapes. This was found to be the case for all of the image segments.

(a)

Figure 5. Real vs. imaginary ordered plots for the 50 FFT components with the largest magnitude for image segments, (a) four and (b) five.

A backpropagation network was defined with 98 input elements, 50 hidden layer elements, and 7 output elements and trained with 49 real and imaginary pairs. This network learned to distinguish between the seven tumor stages in 995 iterations for the travel axis segments and 1,333 iterations for the focal segments. Likewise, networks were able to learn to distinguish between the seven segments using any combination of real, imaginary, magnitude, and phase.

The most efficient and accurate in the presence of noise was the use of the phase alone. A network with 49 input elements, 50 hidden layer elements, and 7 output elements learned from the phase in 769 iterations for the travel axis segments and 654 for the focal. Further, this network required approximately one forth the time per iteration than the networks training on original data.

However, when noise was added to the image segments a few points in the set of 49 with the largest magnitude were changed and this caused a shift in the network input data. Even though the shape of the data remained consistent, the shift caused the classification accuracy to degrade rapidly with increasing noise.

1-567

Page 8: [IEEE Proceedings of 1993 IEEE International Conference on Neural Networks (ICNN '93) - San Diego, CA, USA (1993.03.28-1993.04.1)] IEEE International Conference on Neural Networks

The use of the 49 FFT components with the largest magnitude included redundant information because of the frequency domain symmetry. Also, as expected these components were found to consistently be in an area near the origin of the spatial frequency axes. Therefore, a systematic procedure to extract 49 non-redundant components near the zero frequency component was adopted. This technique eliminated the shift in data created by noise, and produced classification results comparable with those from the original data as shown in the output state table of Figure 6 for the focal image segments.

SEGMENT OUT1 OUT2 OUT3 OUT4 OUT5 OUT6 OUT7

1FN30 4 0.92 0.01 0.06 0.08 0.03 0.06 0.01 2FN30-4 0.00 - 0.84 0.03 0.18 0.09 0.03 0.04 3FN30-4 0.28 0.03 0.61 0.02 0.10 0.01 0.06

INPUT

- 4FN30-4 0.09 0.09 0.02 - 0.86 0.06 0.06 0.00 5FN30-4 0.05 0.03 0.04 0.08 - 0.93 0.03 0.04 6FN30-4 0.20 0.01 0.03 0.13 0.03 0.83 0.01 7FN30-4 - 0.04 0.06 0.06 0.01 0.10 0.07 0.90

Figure 6. Backpropagation network output states for 49 FFT phase inputs from 4 gray level focal images with 30 percent noise added.

CONCLUSIONS

It is clear that a three layer backpropagation neural network can distinguish between very subtle changes in image patterns characteristic of many biomedical images. Further, through the use of preprocessing and spatial frequency techniques a small network can produce equivalent results using less than 1 percent of the original data. The additional time required for the frequency domain processing can easily be reduced using currently available digital signal processing integrated circuits which can calculate a 128 by 128 FFT in less than 2 seconds.[5] Based on the results achieved with the backpropagation network, additional work is planned using more complex network architectures such as the adaptive resonance and neocognitron networks. Also, additional biomedical images will be used to evaluate the extent of the image characterization capabilities of the neural networks.

REFERENCES

111 Goodman, Philip H., and Dwight D. Egbert, "Decision Support Technology: Quantitative Analysis Of Infrared Images", AMERICAN ACADEMY OF THERMOLOGY, 15th Annual Meeting, Johns Hopkins University, June 12-15, 1986.

121 Goodman, Philip H., M. Murphy, G. Siltanen, M. Kelley, and L. Rucker, "Normal Temperature Asymmetry of of the Back and Extremities by Computer- Assisted Infrared Imaging", THERMOLOGY, vol. 1, No. 4, pp. 195-202, 1986.

Proceedings of the IEEE, vol. 69, pp. 529-541, May 1981.

Transform", IEEE Trans. Inform. Theory, vol. 24, pp. 683-692, Nov. 1978.

131 Oppenheim, Alan V., and J. S. Lim, "The Importance of Phase in Signals",

[41 Pearh", William A . , and R. Gray, "Source Coding of the Discrete Fourier

151 Wiegand, Jim, "Digital Signal Processing Enters the Mainstream", ELECTRONIC DESIGN NEWS, pp. 111-117, August 6, 1987.

1-568