A Methodology of Computer AidedDiagnostic System on Breast Cancer

Hee-Jun Song*, Seon-Gu Lee, Gwi-Tae Park

Intelligent System Research Laboratory,Department of Electrical Engineering, Korea University,

1, 5-Ka Anam-Dong, Seoungbuk-Gu, Seoul 136-701, Korea Email: {nyaong7, cr981027, qtpark}@korea.ac.kr

Abstract— In this paper, a new approach using ANFIS(Adaptive Neuro-Fuzzy Inference System) as a diagnosis systemon Wisconsin Breast Cancer Diagnosis (WBCD) problem isproposed. The automatic diagnosis of breast cancer is animportant, real-world medical problem. It is occasionally difficult to attain the ultimate diagnosis even for medical expertsdue to the complexity and non-linearity of the relationshipsbetween the large measured factors. It is possibly resolved witha human like decision-making process using artificial intelligence (AI) algorithms. ANFIS is an AI algorithm whichhas the advantages of both fuzzy inference system and neuralnetworks. Therefore, it can deal with ambiguous data and learnfrom the past data by itself. Considering these features,applying ANFIS as a diagnostic system was considered in ourexperiment. In addition, in real implementations, theperformance of diagnosis system in computation is an important issue as well as the correctness of the output from the inferencesystem. A couple of methods using recommended inputs generated by genetic algorithm, decision tree and correlationcoefficient computation with ANFIS are proposed to reduce thecomputational overhead and they possibly enhance theperformance by eliminating less-relevant input features.

I. INTRODUCTION

MAJOR problem in medial science is attaining thecorrect diagnosis of disease in precedence of its

treatment. For the ultimate diagnosis, many tests are generally involved. All of these test procedures are said to be necessary in order to reach the ultimate diagnosis. Howeveron the other hand, too many tests could complicate the maindiagnosis process so that even the medical experts might have difficulty obtaining end results from those tests. Particularlyin the case of where there are many tests performed for the ultimate diagnosis, wherein the results are not so relevant the diagnosis process could be even more complex even formedical experts. This kind of difficulty could be resolvedwith the aid of a computerized diagnosis system. A well-designed computerized diagnosis system could be usedto directly attain the ultimate diagnosis with the aid ofartificial intelligent algorithms which perform roles as classifiers.

There have been a substantial previous research works with WBCD database to achieve an automatic ultimate

diagnostic system. Genetic Algorithm (GA) [13], FuzzyInference System (FIS) [9,15], Neural Networks [12],Adaptive Boosting (AdaBoost) [8] and Neuro-Fuzzy HybridModels [12,14] have been applied to this problem. Theperformances of each inference system were evaluated with calculating the degree of correctness in predicted resultsagainst diagnosed results represented as PPV (Positive Predicted Value) in each work. Each system shows the PPV within the range from less than 60% (AdaBoost) up to over 95% (Neuro-Fuzzy Hybrid Models). Among thosealgorithms, Neuro-Fuzzy Hybrid models provide relativelyremarkable performances in diagnosis. Those models are thecombination of Neural Networks and Fuzzy Inference Systems encouraging the advantages and resolving thedrawbacks of both NNs and FIS models. For our experiments,a modified method of using Adaptive Neuro-Fuzzy Inference System (ANFIS) [1,10] was mainly applied to attaining theultimate diagnosis and Adaptive Boosting (AdaBoost) [2,7] was also tested for the comparison. These 2 algorithms are available as functioning classifiers so that they are used forthe intelligent diagnosis system which attains the ultimatediagnosis as being either benign or malignant.

Although the ultimate diagnosis is possibly attained by thecomputerized diagnosis system, important consideration as tohow reliable the ultimate diagnosis is and the availability of the inference system could be used in the real medical field. Therefore a good model of breast cancer diagnosis system isconcluded to have high accuracy on the diagnosis and low cost in computational and storage devices.

In this point of view, we focused our experiments ondesigning a diagnosis system which has a lowercomputational cost with highest possible accuracy on providing the diagnosis. In applying ANFIS and otherartificial intelligent algorithm, the required system size is inproportion to the size of the inference system such as thenumber of inputs, the number of internal nodes and thenumber of learning iteration. Among those critical factors of the inference system, the number of internal nodes and learning iteration are changeable only in the process of designing the system. Therefore, methods for reducing the

* Corresponding author

A

Proceedings of the2005 IEEE Conference on Control ApplicationsToronto, Canada, August 28-31, 2005

TB3.2

0-7803-9354-6/05/$20.00 ©2005 IEEE 831

number of input factors within range of not losing theaccuracy of diagnosis were considered. In addition, thosemethods possibly increase the performance of the inferencesystem since they are able to eliminate less-relevant features from whole input set. For the purpose, we used these 3methods as the pre-processes / input-recommenders, whichare Genetic Algorithm [3], Decision Tree [4] and the correlation coefficient between the individual inputs and thetest diagnosis results. ANFIS algorithms are used for establishing the main part of the diagnosis system. AndAdaBoost algorithm was also considered to be compared.

The rest of this paper is organized as follows: in Section 2,WBCD database is introduced and Section 3 briefly describesthe ANFIS structure. In Section 4, ANFIS and itscombinational way of using with input recommenders isproposed. The experiment result is given and compared withprevious research works in Section 5.

II. WISCONSIN BREAST CANCER DIAGNOSIS DATABASE

Breast cancer is the most common tumor-related disease among women in Korea and throughout the world, and themortality rate caused by breast cancer is dramaticallyincreasing. It is considered to be the major cause to death to women, seriously threatening women health. In the diagnosisprocess, fine needle aspiration of breast masses is a mostlynon-invasive diagnostic test that obtains information needed to evaluate malignancy [5,8,9].

The Wisconsin breast cancer diagnosis (WBCD) databaseis the result of efforts provided by the University of Wisconsin Hospital based on microscopic examination ofbreast masses with fine needle aspirate tests. The WBCDdatabase consists of nine measures represented as an 1-10integer value as follows: (1) Clump Thickness(X1); (2)Uniformity of Cell Size (X2) (3) Uniformity of Cell Shape (X3) (4) Marginal Adhesion (X4) (5) Single Epithelial Cell Size (X5) (6) Bare Nuclei (X6) (7) Bland Chromatin (X7) (8) Normal Nucleoli(X8) (9) Mitosis (X9). The database itself contains 683 cases, with each entry representing the classification for a certainensemble of measured values:

TABLE 1. WBCD DATABASEcase X1 X2 X3 … X9 Diagnostics

1 5 1 1 … 1 Benign2 3 2 2 … 1 Malignant: : : : : :

Training data

341 4 8 8 … 1 Malignant342 6 6 6 … 2 Benign

: : : : : :Test data 683 4 8 8 … 1 Malignant

Note that the diagnostics do not provide any informationabout the degree of benignity or malignancy. In consideringof the relationship between the measured values and thediagnostics, there are almost no relationships which stand out.Therefore, there is no convenient and effective method to

attain the ultimate diagnostics with this original data even for the specialists. In addition to that, there may be the possibilitythat one or more of the measured pieces do not affect the diagnosis result. These are the reasons that artificialintelligent system can be used as an expert to assist thespecialist in diagnosing correctly.

In our experiments, the data in the WBCD database was divided into 2 sets: training and test datasets. The datasets arenormalized to the range [0,1]. The output is classified withthe following classification rules:

, 0.5, 0.5

Benign if yClass

Malignant if y (1)

The training dataset was used to figure out what were themost effective and dominant inputs of the inference system.They result in: genetic algorithm, decision tree andcorrelation coefficient computation, as well training thesystem with the recommended inputs using ANFIS and AdaBoost algorithms. The test dataset was used forcorrectness verification of the output.

III. DIAGNOSTIC SYSTEM USING ANFISA human-like learning and decision making system has

been mostly modeled with fuzzy system. [6] The existing fuzzy inference system block diagram is shown in (Fig. 4).However, the fuzzy inference system has several weak pointsin its learning algorithm. They are:

No standard methods exist for transforming humanknowledge or experience into the rule base and database of a fuzzy inference system.There is a need for effective methods for tuning themembership functions so as to minimize the output errormeasure or maximize performance index.

The Adaptive Neuro-Fuzzy Inference System (ANFIS), proposed by J. S. R. Jang [1] is an alternate method whichcombines the advantages of both fuzzy rule based system and neural networks. It is a hybrid learning algorithm which is a fuzzy inference system implemented in the framework ofadaptive neural networks. (Fig. 4) shows the block diagramof ANFIS structure. As shown in the diagram, ANFIS uses ahybrid learning rules to optimize the fuzzy system parameters.In the forward pass, functional signals go forward till layer 4and the consequent parameters are identified by the leastsquares estimate (LSE). In the backward pass, the error rates

Fig. 1. ANFIS structure diagram

832

propagate backward and the premise parameters are updatedby the gradient descent.

For simplicity, the fuzzy inference system having only 2inputs and 1 output was depicted in (Fig. 1). In our experiment, an ANFIS structure containing 4 to 9 inputs and 1 output was used. The ANFIS system has the inferencefunction of fuzzy system and the learning function of neuralnetworks. Supposing that the rule base contains a fuzzyif-then rule, the function in each layer is described below.

Rule1: IF x is and is1A y 1B , then 1 1 1f p x q y r1

Rule2: IF x is 2A and isy 2B , then 2 2 2 2f p x q y r

Layer 1: Membership functions (MF) are formed at eachnode in layer 1. Usually bell-shaped MFs are chosen withmaximum equal to 1 and minimum equal to 0, such as,

12

2

1( ), ( )1

( ) exp 1

i

i

Ai Aii

ii

iA

i

O x xx c

ba

x cor x

a

(2)

where, { a b } is parameter set(premised parameters ), ,i i icLayer 2: The firing strengths are generated in layer 2.

( ) ( ) 1,2i Ai Biw x y i (3) Layer 3: Normalizing the firing strengths

1 21, 2i

iw

w iw w

(4)

Layer 4: Every node in layer 4 has parameter referred toconsequence parameters

4 (i i i i i iiO w f w p x q y r ) (5)

where { , , }i i ip q r is the parameter setLayer 5: The overall output is computed as the summationof all signals

51

ii

i iii i

w fO z w f

w (6)

In this procedure, an ANFIS system can learn from theexisting data with correct output and possibly predict a resultwith new input set by tuning its internal node values. In ourexperiments, ANFIS is used with 9 inputs (full database) orless inputs from the WBCD database and generates 1 output(benign or malignant). The experiment result data withANFIS is shown in section 5.

IV. ANFIS SYSTEM WITH DOMINANT INPUT SELECTORS

A. ANFIS with GA Input Recommender Genetic Algorithm (GA) was used as a dominant input

selector. In out experiment, a GA input recommender systemwas used to have the optimized input value set with ANFIS

test error estimation as its object function. From the result ofpreceding cost evaluation experiment, the maximum numberof recommended inputs by GA was restricted up to 6 measured data inputs. (Fig. 2) describes how the GA is usedfor dominant input feature selecting process. Each individualin the population consists of 9 strings which represent thecandidate inputs for the output classification algorithm suchas ANFIS. Each string in an individual is represented as either0 or 1: 0 for not use the input data, 1 for use the input data.Since we limited the maximum number of recommendedinputs up to 6, the number of all cases to be considered iscalculated as

6

9, 1

! 456 ( )! !

r

n r

ncases

n r r (7)

With the assumption that there is no re-appearance ofindividual which is already considered, the number ofgenerations will be calculated as

. 456 46

. 1 10num of all cases

number of generationsnum of cases in st generation

(8)

Therefore the relevant parameter settings are: TABLE 2. PARAMETER SETTING FOR GA INPUT RECOMMENDER

Population size 10Number of generations 46

Probability of crossover 0.9Probability of mutation 0.1

(a) GA input selector

(b) GA input recommenderFig. 2. Block diagram of GA input recommender

833

The brief block diagram of the GA input recommender isshown in (Fig. 2). Once the GA input recommender generates10 individuals in a generation, the inputs are applied to theoutput classification algorithm and the 10 output errors between the computed output values by the output generatorand the real diagnosis results. The output generators such as ANFIS have restricted values of their parameters – iterationnumber in ANFIS and the number of hypotheses in AdaBoostfor the fast computation time. Then the GA ranks the candidates in the current generation and remains the bestfitted result as the parent of the next generation. Then, the 9other candidates are selected with another random inputgeneration process, crossover and mutation process. Finally,the GA input recommender attains the best input combinationfor the diagnosis process through this procedure. The resultderived by this experiment with GA is shown in section 5.

B. Decision Tree The second input recommender used in our experiment

was a decision tree learning algorithm using SAS9™ package. A decision tree (DT) is known as a good classifier of hugedata. It classifies input data by partitioning example spaces with entropy calculation. DT is especially useful for these cases: examples are represented by attribute-value pairs and the target function has discrete output value. The WBCDconsists of 9 measured data which are represented as aninteger between 1 and 10. Therefore DT can be a goodclassifier for WBCD dataset. In our experiment, a binarydecision tree was constructed to select dominant inputs. The diagram of decision tree is presented in (Fig. 2). In each node of the tree, the most useful attribute for classifying whole datais selected by calculating the information gain with followingformula.

( )

| |( , ) ( ) ( )

| |v Values X

DvGain D X Entropy D Entropy Dv

D (9)

1, ( ) log

c

ii

where Entropy D p pi (10)

The DT is constructed in a way to reduce the entropy withthe attribute which has the highest gain value at each node. Through this way, the final DT model has the most usefulmeasured data on the top node, next useful one on the right

node of the top node and so on. The input selection process derived by DT is presented in (Fig. 3).

C. ANFIS with Correlation Coefficient ComputationThe third method used for dominant input selecting

process is calculating correlation coefficients between eachmeasured input data and the diagnosis results. The Correlation Coefficient is a numerical measure of the degree of the linear relationship between two variables. The value of the correlation coefficient always falls between –1 and 1. A positive correlation coefficient indicates a direct relationship,and a negative correlation coefficient indicates an inverserelationship between two variables. The closer to 1 thecorrelation coefficient is, the stronger relationship the twodata have and a value near 0 indicates no relationship. In thecalculation, first it is possible to assume that all thecorrelation coefficients by calculating with data in WBCDshould be positive. Then we selected 4 measured input data from the one which has the highest correlation coefficient fordiagnosis system. The correlation coefficient can becalculated by this following formula.

( , )( ) ( )( )

ij i jij

i jii jj

C cov X XX XC C

(11)

2 2, ,ii jj ijC Xi X C Xj X C Xi X Xj X (12)

Each result by the correlation coefficient calculation between each measured input and the correct output indicates the degree of linear relationship between them. In thisprocedure, the selected input features are possibly said tohave more linear relationships so that they affect more to theresults. Therefore, the inputs highly correlated with theoutput were selected as dominant inputs in our experiment.The result of dominant input selection by the correlationcoefficient calculation is given in (Fig. 4).

Fig. 4 Result of calculating correlation coefficient: correlation coefficientsare computed between each measured input and diagnosis output; the 4

Fig. 3. DT dominant input selection for ANFIS diagnostic system

834

V. EXPERIMENT RESULTS

In our experiment, we focused on the cost down incomputation time and data storage with accuracy. Therefore, the experiment evaluating the computation time with thenumber of inputs from 2 to 9 and the number of nodes(required memory) using ANFIS are conducted in advance. (Table. 3) shows the result of computational cost evaluationtests using ANFIS with various numbers of inputs. As shownin the table, after 7 inputs applied, the computation time andthe needed data storage increase significantly so the inputselection method could be appropriate for faster result andcosting down as long as the system is not losing or enhancingthe diagnostic accuracy.

TABLE. 3. COMPUTATIONAL COST EVALUATION TESTS RESULT USING ANFISnumber of inputs 2 3 4 5 6 7 8 9

computation Timewith ANFIS (minutes) 0.1 0.2 0.4 1.1 1.5 4.4 13.1 37.3

number of nodes 21 34 55 92 161 294 555 1072number of parameters 24 50 104 222 484 1066 2352 5174

Number of fuzzy rules 4 8 16 32 64 128 256 512

(Tests were conducted with Pentium4™ 2.0Ghz, 256 Mb memory system using MATLAB™ r13)

The recommended inputs are presented in the followingtable in each case.

TABLE. 4. RECOMMENDED INPUTS BY EACH INPUT RECOMMENDER

Selected inputWith ANFIS X1, X4, X5, X6, X7, X8Genetic Algorithm

With AdaBoost X1, X5, X6, X7, X8Decision tree X6, X3, X7, X8

Correlation coefficient X2, X7, X3, X6

These selected input dataset were applied to ANFIS and AdaBoost inference system to train and test the proposedsystem. The structure of ANFIS used in the experiment isshown in (Fig. 1) in Section 3 with expanded inputs between4 and 6.

To evaluate the correctness of the proposed system, PPV(positive predicted value) was computed in each case. (Table. 5) shows the experiment result with ANFIS and the followingtable presents PPVs in each case of experiment. (Table. 6) is given for the comparison between our experimental resultand previous works. The results conducted with AdaBoostare presented to compare with the ANFIS results in ourexperiment. PPV is computed as:

100 ( )Correct results

PPV percentAll results

(12)

TABLE. 5. EXPERIMENTAL RESULTS WITH RECOMMENDED INPUTS

Experimented case Number of correct results PPV

Full data 9 327 / 341 95.89 ANFISGenetic algorithm 6 333 / 341 97.65

Decision tree 4 334 / 341 97.95Correlation coefficient 4 332 / 341 97.36

Full data 9 211 / 341 61.87Genetic algorithm 5 212 / 341 62.17

Decision tree 4 208 / 341 61.01Ada-

BoostCorrelation coefficient 4 215 / 341 63.05

TABLE. 6. EXPERIMENTAL RESULTS OF PREVIOUS WORKS

Experiment dataset PPV (percent) ReferenceANFIS USF 52.9 [11]

AdaBoost USF 57.6 [11]Fuzzy-Genetic WBCD 97.07 [13]

ILFN WBCD 97.23 [14]Fuzzy WBCD 96.71 [14]

ILFN &Fuzzy WBCD 98.13 [14]SANFIS WBCD 96.07~96.3 [15]

NNs WBCD 97.95 [16]

In our experiment, ANFIS shows even better performancethan AdaBoost as an inference system on breast cancer diagnosis problem. And the reduced input dataset showsalmost the same performances or better performances with the same learning iteration number and shows better/similarperformance against the results of previous works. Since theresult derived by the reduced input dataset shows betterperformance and it has significantly higher advantage incomputation, it would be a better method to be implementedin real situations. Therefore, the proposed methods –combined algorithm with ANFIS and dominant inputrecommenders, can be appropriate methods of inferencesystem for the problem of breast cancer diagnosis.

VI. CONCLUSION

In this paper, methods of automatic breast cancer diagnosissystem with ANFIS were proposed with experiments focused on computational performance improving by input dataselection. Artificial intelligence algorithms such as fuzzylogic, neural networks and several modified algorithms based on them are highly available for automatic diagnosis systemsin real medical field. The problems of slow performancecaused by huge computations and required storage device arepossibly solved by input reduction methods. Our experimentindicates a way to have higher computational performanceand enhanced accuracy with these powerful inference systemalgorithms and it provides reliable results. Thus thosebenefits which mentioned in this paper, the method usingANFIS with input reduction can be used for many otherproblems which have high complexity and strongnon-linearity with huge data to be analyzed.

REFERENCES

[1] Jyh-Shing Roger Jang. “ANFIS: Adaptive-Network Based FuzzyInference System.”, IEEE Trans. on System, Man and Cybernetics . vol. 23, no. 3. 1993

835

[2] Y. Freund and R. E Schapire. “Experiments with a New Boosting Algorithm” Machine Learning: Proceedings of the Thirteenth International Conference, 1996

[3] Goldberg. D, “Genetic Algorithm in Search, Optimization, and Machine Learning”, Addison-Wesley, 1989.

[4] George H. John, Ron Kohavi, Karl Pfleger, “Irrelevant Features and the Subset Selection Problem”, Machine Learning: Proceedings of the Eleventh International Conference, 1994.

[5] O.L. Mangasarian, W.N. Street, and W.H. Wolberg, “Breast cancer diagnosis and prognosis via linear programming,” Mathematical Programming Technical Report 9410, University of Wisconsin, 1994.

[6] T. Takagi and I. Hayashi. “NN-driven fuzzy reasoning.” Int. J. Approximate Reasoning, vol. 5, no. 3. 1985.

[7] Robert E. Schapire. “Theoretical Views of Boosting and Applications.”, Proceedings of Algorithmic Learning Theory. 1999.

[8] W. H. Land, Jr., T. Masters, and J. Y. Lo. “Application of a New Evolutionary Programming/Adaptive Boosting Hybrid to Breast Cancer Diagnosis.” IEEE Congress on Evolutionary Computation Proceedings, 2000.

[9] C. A. Pena-Reyes and M. Sipper. “Evolving fuzzy rules for breast cancer diagnosis.” In Proceedings of 1998 International Symposium on Nonlinear Theory and Applications, 1998.

[10] J.-S. R. Jang and C. T.Sun. "Neuro-fuzzy modeling and control," Proceedings of the IEEE. 1995.

[11] Walker H. Land. Jr. and Elizabeth A. Veheggen, “Experiments Using an Evolutionary Programmed Neural Network with Adaptive Boosting for Computer Aided Diagnosis of Breast Cancer”, IEEE International Workshop on Soft Computing in Industrial Application, 2003

[12] G. Arulampalam and A. Bouzerdoum, “Application of Shunting Inhibitory Artificial Neural Networks to Medical Diagnosis”, Seventh Australian and New Zealand Intelligent Information Systems Conference, 2001.

[13] C. A. Pena-Reyes and M. Sipper. “Designing Breast Cancer Diagnostic System via a Hybrid Fuzzy-Genetic Methodology”, IEEE International Fuzzy Systems Conference Proceeding, 1999.

[14] Phayung Meesad and Gary G. Yen, “Combined Numerical and Linguistic Knowledge Representation and Its Application to Medical Diagnosis”, IEEE Transactions on Systems, Man, and Cybernatics, 2003.

[15] Jeen-Shing Wang and C. S. George Lee, “Self-Adaptive Neuro-Fuzzy Inference Systems for Classification Applications”, IEEE Transactions on Fuzzy Systems, 2002.

[16] Rudy Setiono, “Generating Concise and Accurate Classification Rules for Breast Cancer Diagnosis”, Artificial Intelligence in Medicine, 2000.

836