An

Independent Study Report

on

Enhancement of CVIP-FEPC and Color Normalization Program

Independent Study

(ECE 591)

SUBMITTED BY

Kumari Heema Poudel(800517296)

December 12, 2013

I

Abstract

Image analysis is an important part of Digital Image Processing which comprises of computer

vision and human vision applications. Usually, feature analysis and pattern classification are the

final and significant steps in image analysis. The Computer Vision and Image Processing Feature

Extraction and Pattern Classification Tool, CVIP-FEPC, was developed for exploring both

human and computer vision applications. The main function of this tool is to analyze feature

extraction and pattern classification process. This enables the user to perform batch processing

with large sets of image files which is much more efficient than processing one image at a time.

This tool allows the user to load multiple images and run experiments based on different

parameters by specifying the classes, selecting the features, selecting the test set, choosing the

pattern classification parameters and then letting the program to process on the entire image set.

Result files are stored in the output folder. One of the main objectives of this independent study

is to develop the next stable version of CVIP-FEPC that includes fixing the bugs present in

current CVIP-FEPC and to enhance the tool by adding more features.

Another main focus of this study is on the Color Normalization Program, which is used to do a

remapping to the color values of thermographic images so that all of the images are mapped to a

common temperature scale. Graphical User Interface (GUI) has been added to the Color

Normalization program to make it more user friendly and a mechanism has been developed to

organize the generated output files.

I

Acknowledgement

I would like to express my deepest appreciation to Dr. Scott E. Umbaugh for his continuous guidance and

encouragement throughout this independent study. Without his great supervision, constant help and

valuable suggestions this study would not have been fruitful.

Additionally I wish to thank Samrat Subedi and Jiyuan Fu; members of Computer Vision and Image

Processing Lab, Southern Illinois University Edwardsville for their help and support in testing the

application with the experiment on real images.  I would like to thank Long Island Veterinary Specialists

Dr. Dominic J. Marino and Dr. Catherine A. Loughin for providing images which have been used in

testing the application. Last but not the least I want to thank Ravneet Kaur and Krishna Regmi for their

help.

 Kumari Heema Poudel

II

Table of Contents

Abstract...................................................................................................................................................I

Acknowledgement.................................................................................................................................II

Table of Contents.................................................................................................................................III

List of Figures......................................................................................................................................IV

1. Introduction............................................................................................................................................1

2. Materials and Methods...........................................................................................................................2

2.1 Development Environment............................................................................................................2

2.2 Color Normalization......................................................................................................................2

2.3 CVIP-FEPC....................................................................................................................................4

3. Overall Flow of FEPC...........................................................................................................................9

4. Known Bugs.........................................................................................................................................10

5. Conclusion...........................................................................................................................................11

6. Future Work.........................................................................................................................................12

7. Appendices...........................................................................................................................................12

7.1 Color Normalization....................................................................................................................12

7.2 Screenshots..................................................................................................................................14

8. References............................................................................................................................................16

III

List of Figures

Figure 2.1: Color Normalization GUI and help file………………………………………………………...4

Figure 2.2: Exception occurred while using Leave One Out with Combinatoric test……………………...5

Figure 2.3: Help File of FEPC……………………………………………………………………………...6

Figure 2.4: Feature Selection screen with Variable Texture Distance……………………………………...7

Figure 4.1: Exception occurred while using the different feature category in second experiment……….11

Figure 7.1: Original Thermographic image and their normalized equivalents…………………………....14

Figure 7.2: Main window of CVIP-FEPC………………………………………………………………..14

Figure 7.3: Pattern Classification window of CVIP-FEPC ……………………………………………...15

Figure 7.4: Class List window of CVIP-FEPC …………………………………………………………15

IV

1. Introduction

This study is about the further development on Color Normalization program and CVIP–FEPC.

The CVIP–FEPC tool is extensively being used for the project ‘Veterinary Thermographic Image

Analysis’ from Long Island Veterinary Specialists, in which the feature analysis and pattern

classification is done for cats’ and dogs’ thermographic images to classify the cats and dogs for

normal and abnormal conditions of bone cancer, hyperthyroid and the gate analysis.

CVIP-FEPC is not limited for the analysis of thermographic images only. This can be used for

the analysis of any image types. One area of concern may be to run the experiments with

multiple texture distances at the same time. In previous version of CVIP-FEPC, this feature of

variable texture distance was not available. So, this is the one feature that has been added in the

CVIP-FEPC. A help file is always very useful for any application to make the user familiar with

the tool and this has been added in the current CVIP-FEPC.

Other than FEPC, work has been done on Color Normalization program which was developed

for the ‘Veterinary Thermographic Image Analysis’ to normalize the temperature range

according to the color of the thermographic image. A graphical user interface has been

developed to make the program user friendly and some other features have been added to

mitigate the manual work to manage the output files in folders. The new version of Color

Normalization program automatically creates the folders and keeps all the output files to their

respective folders.

This report will describe the changes made in FEPC to the program detail and their reflection in

GUI together with the changes in Color Normalization program.

1

2. Materials and Methods

2.1 Development Environment

CVIP-FEPC was developed with C#.NET programming language using Visual Studio 2008.The further

enhancements are done using the same programming language but with the Visual Studio 2010 and the

CVIP-FEPC is converted to C#.NET 2010. The following FEPC files are modified to achieve the variable

Texture Distance feature: MainForm.cs, FeatureHeader.cs, FeatureChooser.cs, FeatureVector.cs, and

ImageObject.cs.

Color Normalization program has been tested with the thermographic images provided by Long Island

Veterinary Specialists taken with a Meditherm Med2000 IRIS. The unit testing is done by the developer

whereas the functional testing is done by the researchers of the Veterinary Thermographic Image

Analysis.

Similarly for the FEPC features, unit and functional testing have been done with small set of images by

the developer, whereas researcher performed the functional testing on the real set of thermographic

images.

2.2 Color Normalization

The research project undergoing in SIUE, ‘Veterinary Thermographic Image Analysis’, uses

thermographic images of cats and dogs to classify them into different categories like normal and

abnormal. Within any thermographic image, each color represents a specific temperature. But in different

images, the same color may represent different temperatures, which can introduce noise in the pattern

classification process. To overcome this, a remapping is done to ensure that all the images are mapped to

a common temperature scale which is done in Color Normalization program. More about Color

Normalization methods can be found in the appendix [1].

2

2.2.1 Output Folder Structuring

In the previous version of Color Normalization, all the output normalized files were stored at the same

folder as input. It was a tedious task to manually manage the folders for placing normalized output files

and their masks for each category together. Now this version of the program makes this task easier by

automatically managing the organization of file.

Following folders are created automatically at the location where input files are located:

orig: All the input files are kept here together with the Mask folder.

lum: All lum normalized files are placed here concatenating ‘–lum’ in the original file name.

Similarly, mask file names are also changed by concatenating ‘-lum’ in mask file name and

these mask files are also copied together with the Mask folder within the Lum folder.

normGrey: All normGrey normalized files are kept here with concatenating ‘–normGrey’ in

the original file name. Similarly mask files name are also changed by concatenating ‘-

normGrey’ in mask file name and these mask files are also copied together with the Mask

folder within the normGrey folder.

normRGB: All normRGB normalized files are kept here with concatenating ‘–normRGB’ in

the original file name. Similarly mask files name are also changed by concatenating ‘-

normRGB’ in mask file name and these mask files are also copied together with the Mask

folder within the LnormRGB-lum folder.

normRGB-lum: All normRGB-lum normalized files will be kept here with concatenating ‘–

normRGB’ in the original file name. Similarly mask files name are also changed by

concatenating ‘-normRGB-lum’ in mask file name and these mask files are also copied

together with the Mask folder within the normRGB-lum folder.

2.2.2 GUI based Color Normalization

A simple GUI has been designed to make it user friendly. This GUI will load a *.txt file which contains

the path of the images followed by maximum and minimum temperatures of that image. The exact file

3

information is available in Help file of the application as shown in Figure 2.1.

Figure 2.1: Color Normalization GUI and help file

2.3 CVIP-FEPC

CVIP - FEPC is composed of four graphical interfaces: the Main window, the Feature Selection window,

the Pattern Classification Selection window and the Class List window. The main interface is the Main

window which is seen when the application is executed. The Feature Selection window appears when the

user clicks on Features button of Main window and Pattern Classification window appears when the

Classification button of the Main window is clicked. There is a setting menu in the main window which

directs to the Class List window.

The following works have been done in CVIP-FEPC:

2.3.1 Bug Fixing

The previous version of CVIP-FEPC was not working for Leave One Out method with Combinatoric test.

The following error was encountered when the user tried to execute CVIP-FEPC with the Leave one out

method and Combinatoric test, which is shown in Figure 2.2.

4

Figure 2.2: Exception occurred while using Leave One Out with Combinatoric test

After debugging, it was found that the training and test classes were being used without initializing them,

which was causing the error of “Object reference not set to an instance of an object”. For using any

method of a class, the class needs to be initialized and this was the missing part. After writing the codes

for initializing the class and calling its function in an appropriate manner, issue was fixed.

2.3.2 Help File

A help file has been added in the FEPC to make it easier to use and to have a better understanding of the

software. The help file is basically a Compiled HTML file (CHM file) which is designed and compiled

using ‘HTML Help Workshop’ software. Snapshot for the help file is shown in Figure 2.3 below.

5

Figure 2.3: Help File of FEPC

2.3.3 Variable Texture Distance:

In the current version of FEPC, we have the option of selecting one texture distance at a time. This means

any experiment even if it is combinatoric, it will run with only one texture distance. What if a user wants

to analyze the output of experiment with several texture distances with the same feature set? For this, the

user has to run the experiment several times by passing the texture distance of choice. A better option is

to incorporate the choice of selecting multiple texture distances in FEPC itself.

With the option of selecting variable texture distance, the feature selection window looks like as shown in

Figure 2.4.

6

Figure 2.4: Feature Selection screen with Variable Texture Distance

This variable texture distance feature is available only if the experiment is to be run for ‘Combinatoric’

option and not for the ‘Single Test’. There is a button Switch to Single Test Mode, which can be selected

if the user wants to run Single Test experiment only.

With the addition of this new feature the first file that gets affected is MainForm.cs. In this file the

program checks whether the user has selected the variable texture option or not. If the multiple texture

distance is selected, the program calculates the different texture distances (d1, d2,…dn) and the

experiment runs with all the texture distances.

7

The second file that gets affected is FeatureDlg.cs. This is the interface for feature selection and three

boxes have been added here which are the parameters for variable texture distance:

i. Lower: This is the first texture distance value with which the experiment will start its

execution.

ii. Increment: This is the increment on the lower texture distance.

iii. Upper: This is the final limit of the texture distance.

To illustrate this process, let us take an example where lower is set as 2, increment is set as 2 and upper is

set as 8.

Then formula used to find the iteration is

N= ((Upper –Lower)/Increment) + 1

D[1]=lower;

D [ n ]=D [ n−1 ]+increment

Where n varies from 2 to N

So in our example N= ((8-2)/2)+1=4

So d1=lower=2

D2=d1+increment=2+2=4

D3=d2+increment =4+2=6

D4=d3+increment=6+2=8

So the experiment will run with these 4 texture distances along with all other features and classification

algorithms selected at the beginning.

The output files of the experiment will be saved into four folders, with four different texture distances.

There can be cases where the user wants to run the combinatoric experiments but want to select just one

texture distance. For this, the desired texture distance need to be typed in lower box and upper box should

have the same value. Value entered in the increment box does not matter here.

FeatureChooser.cs and FeatureHeader.cs are the files where output files are being written. So,

8

appropriate texture distance is being passed to be written in the output files in case of multiple texture

distances.

FeatureVector.cs and ImageObject.cs are the files which calculate the feature vectors, so proper distance

is being passed while calculating the texture features for the variable texture distance case.

3. Overall Flow of FEPC

The Main Window (MainForm.cs) appears when a user runs the application. In this window, the user can

add multiple images at the same time. Now the user can select the features by clicking on Features button

and this is the point where program control goes over to FeatureDlg.cs which is the interface for feature

selection window. All the binary, histogram, RST invariant, texture and spectral features are listed here

which can be selected.

After selecting the features user can set the parameters: data normalization methods, distance and

similarity measures and classification algorithm for pattern classification. If combinatoric test has been

selected, the user can choose multiple methods for every category otherwise the user can select only one

method for each category. After selecting the classification methods, the user will classify the images to

any classes like normal and abnormal and hit the button Run Test.

MainForm.cs is the file where all the set up for running the experiment gets done. This file handles the

deleting of images (response of Delete button in the same window) and changing the output location

(action of clicking on Browse button). FeatureVector.cs is the file which calculates all the feature vectors.

Normalizer.cs is the file where all the calculations regarding data normalizations are done.

FeatureHeader.cs is the file which generates all the text output feature files. FeatureFile.cs is used for

getting the feature files which are to be written in the output directory. This file checks if the user has

selected any of the normalization method, if yes this file directs the program control to do the

normalization and then get the feature file otherwise will get the feature file without any normalization

9

being done. FeatureChooser.cs file checks for the features which are selected by the users and calculates

the number of combinations with the number of features selected. For any combinatoric test, the total

number of combinations will be: 2n, where n is the total number of the features and pattern classification

methods selected for each texture distance. The Classifier.cs file checks for the classification methods

being selected by the user and recalculates the number of combinations since the number of features is

increased with the selection of classification methods and thus prepares the content for the output file.

4. Known Bugs

A bug was encountered in the previous version of CVIP-FEPC. If the user runs the application for the

first time and has selected a few categories of the features, then the application will work perfectly, but if

the user tries to run another experiment without restarting the application and selects the features of the

categories not selected previously then the error will pop up.

For example, if the user has selected features from binary features, histogram and texture features

categories in the first experiment and runs the experiment on these feature set, the experiment runs

successfully. But now with the addition of any feature from the RST-invariant category into the feature

set and running the experiment will produce an error. The error message is shown in the snapshot in

Figure 4.1 below. This was noticed at the later stage of implementing the variable texture distance, so this

still exists in the current application.

10

Figure 4.1: Exception occurred while using the different feature category in second experiment

5. Conclusion

All the bugs except for the one mentioned in the Known Bugs section are successfully fixed. The major

concern was the bug which was preventing the use of Leave One Out method with Combinatoric Test,

which is very important functionality of CVIP-FEPC, is now fixed. The newly added help file includes

information about overview of CVIP-FEPC and how to use it. The variable texture distance feature is also

working successfully for both Training and Test Set and Leave One Out, except of the bug mentioned

above.

Enhancements of Color Normalization program are also giving the expected results. Output files are

structured and stored in a consistent way and the help file includes the necessary information regarding

11

how to create the *.txt file to feed as input in the same program.

6. Future Work

In the near future, the first task would be to fix all the known bugs so that the application will be bug free.

The other work areas are in the ring and the sector option of feature selection page where the user will be

able to select multiple values of rings and sectors at the same time. The main purpose of all these is to

make all the features able to have dynamic values which are static for now so that the user will have the

ability to run the Combinatoric test with all the choices of features in a single experiment.

7. Appendices

7.1 Color Normalization

Four different Color Normalization methods are used:

Luminance: For each pixel in the original image, the color components are weighted as follows

to generate a gray level value: Color components are calculated by the following formula to

generate a grey level value for each pixel of image:

GrayLevel=0 .3∗Red+0 .6∗Green+0 .1∗Blue

Norm-Gray: In this method, the temperature values between 15° C and 45° C are mapped to gray

levels from 0 to 255. For this, the temperature represented by the color of the corresponding pixel

in the original image is used to map each pixel in the normalized image by coloring with the gray

level for that temperature. Then, a histogram stretch is done as the final step of this normalization

method that is obtained by saving the image as a TIFF file.

Norm-RGB: This process is similar to the Norm-Gray but temperatures are mapped to a

continuous version of the original color palette.

12

Norm-RGB-Lum: This method is done in two step process: first Norm-RGB is applied to

generate gray scale image and then luminance normalization is applied to the resultant image [1].

The original thermographic images for the research have been provided by the Long Island Veterinary

Specialists.

Figure 7.1(a): Original Thermographic Image Figure 7.1(b): Luminance Normalized Image

Figure 7.1(c): Norm Grey Normalized Image Figure 7.1(d): Norm-RGB Normalized Image

13

Figure 7.1(e): Norm-RGB-Lum Normalized Image

Figure 7.1: Original Thermographic image and their normalized equivalents

7.2 Screenshots

The following are the screenshots of different windows seen during execution of CVIP-FEPC.

Figure 7.2: Main window of CVIP-FEPC

14

Figure 7.3: Pattern Classification window of CVIP-FEPC

Figure 7.4: Class List window of CVIP-FEPC

15

8. References

[1] Umbaugh, E S.,Solt P. (2008), “Veterinary Thermographic Image Analysis Data and

Temperature Normalization”.

[2] CVIPtools(2010), “Computer Vision and Image processing Tools”.

http://cviptools.ece.siue.edu

[3] Umbaugh, E S. (2010), “Digital Image Processing and Analysis: Human and Computer

Vision Applications with CVIPtools, Second Edition”. The CRC Press, Boca Raton, FL, 2010.

16