Predicting Patients with Heart Disease by Using an Improved Back-propagation Algorithm
7
JOURNAL OF COMPUTING, VOLUME 3, ISSUE 2, FEBRUARY 2011, ISSN 2151-9617 HTTPS://SI TES.GOOGLE.COM/SITE/JOURNALOFCOMPUTING/WWW.JOURNALOFCOMPUTING.ORG 53 1 INTRODUCTION Predicting Patients with Heart Disease by Using an Improved Back-propagation Algorithm Nazri Mohd Nawi , Rozaida Ghazali and Mohd Najib Mohd Salleh Abstract A study on improving training efficiency of Artificial Neural Networks algorithm was carried out throughout many previous works. This paper presents a new approach to improve the training efficiency of back propagation neural network algorithms. The proposed algorithm (GDM/AG) adaptively modifies the gradient based search direction by introducing the value of gain parameter in the activation function. It has been shown that this modification significantly enhance the computational efficiency of training process. The proposed algorithm is generic and can be implemented in almost all gradient based optimization processes. The robustness of the proposed algorithm is shown by comparing convergence rates and the effectiveness of gradient descent methods using the pr oposed method on heart disease data. Index TermsBack propagation, Search direction, adaptive g ain, effectiveness, computational efficiency. he back-propagation algorithm has been the most popular and most widely implemented algorithm for training these types of neural network. When us- ing the back-propagation algorithm to train a multi- layer neural network, the designer is required to arbi- trarily select parameter such as the network topology, initial weights and biases, a learning rate value, the activation function, and a value for the gain in the activation function. Improper selection of any of these parameters can result in slow convergence or even network paralysis where the training process comes to a virtual standstill. Another problem is the ten- dency of the steepest descent technique, which is used in the training process, can easily get stuck at local minima. Recently, improving training efficiency of back- propagation neural network based algorithm is an active area of research and numerous papers have been proposed in the literature. Early research on back propagation algorithms saw improvements on: (i) selection of better error functions [1-8]; (ii) different choices for activation functions [3, 9] and, (iii) selec- tion of dynamic learning rate and momentum [10-12]. Later, as summarized by Bishop [13], various op- timization techniques were suggested for improving efficiency of the error minimization process or in oth- er words the training efficiency. Among these are me- thods of Fletcher and Powel [14] and the Fletcher- Reeves [15] that improve the conjugate gradient me- thod of Hestenes and Stiefel [16] and the family of Quasi-Newton algorithms proposed by Huang [17]. This research suggests that a simple modification to the gradient based search direction used by almost all optimization method that has been summarized by Bishop [13] can substantially improve the training efficienc y . The gradient based search direction is lo- cally modified by a gain value used in the activation function of the corresponding node to improve the convergence rates respective of the optimization algo- rithm used. The remaining of the paper is organized as fol- lows: Section two states the research objectives. Sec- tion three illustrates the proposed method and the implementation of the proposed method in gradient descent optimization process. In Section four, the ro- bustness of proposed algorithm is shown by compar- ing convergence rates for gradient descent methods on Cleveland Heart Disease data. The paper i s con- cluded in the final section along with short discussion on further research. 2. RESEARCH OBJ ECTIVES This research will demonstrate the robustness of the proposed algorithm by comparing its convergence rates in predicting patients diagnosed with heart dis- ease. The data used in this research is based on Cleve- land Heart Disease data. The proposed algorithm sig- nificantly enhances the computational efficiency of the training process. The proposed algorithm is gener- ic and can be implemented in almost all gradient T•N.Mohd Nawi is with Universiti tun Hussein Onn Malaysia, 86400, Parit Raja, Batu Pahat, MALAYSIA.•R. Ghazali is with Universiti tun Hussein Onn Malaysia, 86400, Parit Raja, Batu Pahat, MALAYSIA. •M.N. Mohd Salleh is with U niversiti tun Hussein Onn Malaysia, 86400, Parit Raja, Batu Pahat, MALAYSIA.
Predicting Patients with Heart Disease by Using an Improved Back-propagation Algorithm
![Forecasting Earth Quake Using Back Propagation Algorithm ...serialsjournals.com/serialjournalmanager/pdf/1483683448.pdf · successful implementation of predicting earthquakes. [1]](https://img.pdfslide.us/doc/110x75/5aaa47487f8b9a95188de25c/forecasting-earth-quake-using-back-propagation-algorithm-implementation-of-predicting.jpg)
