Regression, theil’s and mlp forecasting models of stock index

International Journal of Computer Engineering and Technology (IJCET), ISSN 0976 – 6367(Print),

ISSN 0976 – 6375(Online) Volume 1, Number 1, May - June (2010), © IAEME

82

REGRESSION, THEIL’S AND MLP FORECASTING

MODELS OF STOCK INDEX

K. V. Sujatha

Research Scholar

Sathyabama University, Chennai

E-mail: [email protected]

S. Meenakshi Sundaram

Department of Mathematics

Sathyabama University, Chennai

E-mail: [email protected]

ABSTRACT

Financial Forecasting or specifically Stock Market prediction is one of the hottest

fields of research lately due to its commercial applications owing to the high stakes and

the kinds of attractive benefits that it has to offer. Financial time-series is one of the

‘noisiest’ and ‘non-stationary’ signals present and hence very difficult to forecast. In this

paper we have made an attempt to forecast the daily prices of stock index using a

Regression, Theil’s and MLP models and the predictive ability of these models are

compared using standard error measures.

Keywords: Forecasting, Regression, Principal Component, Perceptron, MAPE.

1. INTRODUCTION

Trading in stock market indices has gained unprecedented popularity in major

financial markets around the world. However, the prediction of stock price index is a very

difficult problem because of the complexity of the stock market data, and is affected by

many factors including political events, general economic conditions, and investors’

expectations. Modeling the behavior of a market index is a challenging task for several

reasons. There are two major approaches (fundamental and technical) for analyzing stock

price prediction [1].

International Journal of Computer Engineering

and Technology (IJCET), ISSN 0976 – 6367(Print)

ISSN 0976 – 6375(Online) Volume 1

Number 1, May - June (2010), pp. 82-91

© IAEME, http://www.iaeme.com/ijcet.html

IJCET

© I A E M E



83

Due to the lack of profound knowledge about interior running rules in nonlinear

systems like stock system, we have no idea about the variables which are more influential

and important and which are not. Input variables are selected only depending on opening

and objective historical data in a stock market. To avoid missing important data

influencing prediction from the historical data, Principal Component Analysis (PCA), is

usually used. A functional principal component technique for the Statistical analysis of a

set of financial time series highlights some relevant statistical features of such related

datasets [3]. This method is to replace original variables with new ones, which are less in

number and not mutually correlative, and contain most of the information of original

variables [6]. Xiaoping Yang [4] used PCA to find the principal components that are

taken as inputs for predicting stock prices using neural network. Variables high, low,

open, volume and adjusted closing were considered for prediction of closing prices using

Hybrid Kohonen Self Organizing Map [5]. Liu et al [7] used the back propagation neural

networks using moving average, deviation from moving average, turnover moving

average, and relative index for prediction. In Versace et al’s work[8], values used are

open, high, low, close and volume of a specific stock while Baba [9] used change of

index, PBR, changes of the turnover by foreign traders, changes of current rates, and

turnover in local stock market. MLP outperformed RBF in predicting weekly closing

prices using the variables open, high, low and volume [10]. In the recent years, Artificial

Neural Networks (ANNs) have been applied to many areas of statistics. One of these

areas is time series forecasting [11-19]. The variables considered in this article for

predicting the daily closing prices are the historic prices, daily opening, low and high

prices of BSE Sensex from 1st January 2009 till 31

st March 2010. Principal component

analysis resulted in a single set of variable.

The closing prices are predicted by fitting a parametric model Simple Linear

Regression and also by classical Non parametric model Theil’s Incomplete Method.

Multilayer Perceptron is another non parametric model that is used to forecast the daily

closing prices taking the principal component as the predictor variable. The forecast error

values are measured which is the difference between the actual value and the forecast

value for the corresponding period all three models. Error values MAPE, SMAPE and



84

MAE are related with how close the forecasted values are to the target ones. Lower the

error values, better is the forecaster.

2 MODEL DESCRIPTION

2.1 PRINCIPAL COMPONENT ANALYSIS

Principal component analysis is appropriate when there are number of observed

variables and wishes to develop a smaller number of artificial variables (called principal

components) that will account for most of the variance in the observed variables. The

principal components may then be used as predictor or criterion variables in subsequent

analyses. Principal component analysis is a variable reduction procedure. It is useful

when there is redundancy in the data obtained on the number of variables. Here

redundancy means that some of the variables are correlated with one another, possibly

because they are measuring the same construct. Because of this redundancy it is possible

to reduce the observed variables into smaller number of principal components that will

account for most of the variance in the observed variables.

Technically, a principal component can be defined as a linear combination of

optimally weighted observed variables. Below is the general form for the formula to

compute the first component extracted in a principal component analysis:

C1 = b11(X1)+ b12(X2)+….. b1p(Xp)

Where, C1= the first component extracted

b1p= the regression coefficient for the observed variable p,

Xp = the value of the observed variable.

2.2 SIMPLE LINEAR REGRESSION

Simple linear regression fits a straight line through the set of n points in such a

way that makes the sum of squared residuals of the model as small as possible.

Regression has the following assumptions

� The dependent variable is linearly related to the independent variable.

� Residuals follow normal distribution.

� Residuals have uniform variance.



85

Regression parameters for a straight line model y = a + bx are calculated by the

least squares method (minimization of the sum of squares of deviations from a straight

line). This differentiates to the following formulae for the slope (b) and the y intercept

(a) of the line

2.3 THEIL’S INCOMPLETE METHOD

A simple, non-parametric approach to fit a straight line to a set of (x,y)-points is

the Theil's incomplete method which assumes that points (x1, y1), (x2, y2) . . . (xN, yN) are

described by the equation y = a + bx

The calculation of a and b takes place as follows:

� All N data points are ranked in ascending order of x-values.

� The data are separated into two equal size (m) groups, the low (L) and the high (H)

group. If N is odd the middle data point is not included to either group

� The slope bi is calculated for all points of each group,

i.e. bi = (yH,I – yL,i)/ (xH,I – xL,i) for i=1,2,…,m

� The median of the m slope values b1, b2, . . ,bm is calculated and it is taken as the

best estimate of the slope (b) of the line, i.e. b = median(b1, b2, . . bm).

� For each data point (xi,yi) the value of intercept ai is calculated using the

previously calculated slope b, i.e. ai=yi- bxi for i=1,2,…N

The median of the N intercept values a1, a2 , . . . aN is calculated and it is taken as

the best estimate of the intercept (a) of the line, i.e. a = median (a1, a2, . .. aN).



86

2.4 MULTILAYER PERCEPTRON.

A multilayer perceptron is a feed forward network model that maps sets of input

data onto a set of appropriate output. It is a modification of the standard linear perceptron

in that it uses three or more layers of neurons (nodes) with nonlinear activation functions,

and is more powerful than the perceptron in that it can distinguish data that is not linearly

separable. The MLP divides the data set in to three parts Training, Testing and Holdout.

� Training - This segment of data is used only to train the network.

� Testing - This segment of data is a part of the training data to prevent over training

� Hold out - This set of data used to assess the final neural network. Hold out data set

gives an honest estimate of the predictive ability of the model.

Multilayer Layer Perceptron has rescaling option which is done to improve

the network training. There are three rescaling options: standardization, normalization,

and adjusted normalization. All rescaling is performed based on the training data, even if

a testing or holdout sample is defined. The activation function of the hidden layer can be

hyperbolic tangent or sigmoid. The units in the output layer can use any one of the

following activation function - Identity, Sigmoid, Softmax or Hyperbolic Tangent.

2.5 ERROR MEASURES

Error Functions that are used are sum of square error and relative error.

Sum of square error is defined as the sum of the squared deviation between observed

and the model predicted value. Relative Error is the ratio of an absolute error to the true,

specified, or theoretically correct value of the quantity that is in error

||1

,1

∑=

−=

n

t

tt PAn

MAEeErrorMeanAverag

∑=

−=

n

t t

tt

A

PA

nMAPErorePercentErMeanAverag

1

||1,

∑=

+

−

=

n

t tt

tt

PA

PA

nSMAPErorePercentErMeanAveragSymmentric

1

||1, ,

Where tA is the actual value and tP

is the predicted value.



87

3 FINDINGS AND RESULTS

Principal Component Analysis of the variable daily high, low and opening prices

of BSE Sensex data resulted in the single principal component which is further used in

predicting the closing prices by the methods discussed above. The factor determining the

number of principal component, the eigen value and the factor loading of the principal

components are given in Table 1.

Table 1 Principal Component Analysis

1 2 3 4

Eigenvalues 3.2787 0.7194 0.0011 0.0008

Difference 2.5593 0.7183 0.0003

Proportion 81.97% 17.98% 0.03% 0.02%

Cumulative 81.97% 99.95% 99.98% 100.00%

Criteria: Kaiser Weights

Factors F1 PCA PCA1

V1 0.9855 V1 0.5442

V2 0.9855 V2 0.5442

V3 0.9883 V3 0.5458

V4 -0.5998 V4 -0.3312

Exp. Var. 3.2787



88

Initial Descriptive analysis of the daily closing prices and the predictor variable

(principal component variable) is given in Table 2. The assumptions of simple linear

regression are checked and then with this set of observation the line of regression is

fitted.

Table 2 Descriptive Statistics

Variable Mean Standard Deviation Skewness Kurtosis

Daily Closing 14337.1182 3041.62375 -.788 -.909

Principal Component 23419.2108 4974.61634 -.785 -.918

Table 3 Tests of Normality

Kolmogorov-Smirnov Shapiro-Wilk

Statistic df Sig. Statistic df Significance

Closing .170 300 .000 .838 300 .000

PCA .171 300 .000 .838 300 .000

Durbin Watson value is 2.11 clearly states the absence of autocorrelation.

Normality tests Kolmogorov-Smirnov and Shapiro-Wilk were performed and the outcome

were displayed in Table 3. From the Table 3 it is clear that both the tests imply that the

condition of normality is not met.

Using method of Least Squares, the Simple Linear Regression Model for the data is

given by

Y = 34.312 +0.611X, where X is the principal component variable and Y

represents the daily closing price of BSE. By the classical Nonparametric model Theil’s

method, the model is given by

Y = 42.15384+0.610456X, where X is the principal component and Y represents

the daily closing price.



89

For modeling the data with Multilayer Perceptron, the Principal component

variable is taken as covariate and the daily closing prices of BSE is considered to be the

target variable. Smoothing (standardized, normalized and adjusted normalized) of both

the dependent variable and covariates are done successively. All possible combination,

changing the activation function of the hidden layer (hyperbolic tangent and sigmoid) and

that of the output layer (Identity, hyperbolic tangent and sigmoid) the sum of square error

and relative error values are measured with different scaling options.

The different combinations of the activation function of the output and the hidden

layer with the three rescaling options of the input and target variables resulted in 30

models. The architecture for which the sum of square and relative error was minimum is

the one in which the smoothing of both the dependent and covariates are normal with

hyperbolic tangent as the activation function of the hidden layer and Identity for the

output layer. Table 5 gives the MAE, MAPE, SMAPE and R square values for the above

models discussed above. Figure 1 shows how the models predict the closing prices for the

last 50 data point.

Table 6 MAE, MAPE and SMAPE values

Model MAE MAPE SMAPE R2 Value

Linear Regression 110.695401 0.0081926 0.0040934 0.9977142

Theil’s Incomplete

Method

110.6996 0.008198 0.004095 0.9977138

Multilayer

Perceptron

118.5105 0.008839 0.004424 0.9974605



90

Figure 1 shows how the models predict the closing prices for the last 50 data point.

4 CONCLUSION

The best model for forecasting the daily closing prices was found to be linear

regression. The model yielded the least error, only 0.0081926 on average measured by

the MAPE, 0.0040934 on average measured by SMAPE and 110.695401 as the MAE

value. The R square value is 0.997714272 which indicates that the model is appropriate

in predicting the daily closing prices when the daily opening, high and low prices are

considered for predicting. This model out performed the nonparametric Theil’s method

and MLP models. It will be interesting to conduct further studies to compare the results

with addition variables.

5. REFERENCES

1. Kai Keng Ang and Chai Quek, (2006), “Stock Trading Using RSPOP: A Novel

Rough Set-Based Neuro-Fuzzy Approach”, IEEE Transactions of Neural

Networks, 17(5):1301–1315.

2. Brabazon. T., (2000) “A connectivist approach to index modelling in financing

markets”, In Proceedings, Coil / EvoNet Summer School. University of Limerick.

3. Salvatore Ingrassia and G. Damiana Costanzo. (2005), “Functional principal

component analysis of financial time series”, Vichi M., Monari P., Mignani S.,

Montanari A. (Eds.) New Developments in Classification and Data Analysis, Pages

351-358, Springer-Verlag, Berlin.



91

4. Xiaoping Yang (2005), “The Prediction of Stock Prices Based on PCA and BP Neural

Networks Chinese Business Review, ISSN 1537-1506, USA Volume 4, No.5 (Serial

No.23), Page 64 – 68.

5. Mark O. Afolabi, Olatoyosi Olude (2007), “Predicting Stock Prices Using a Hybrid

Kohonen Self Organizing Map (SOM)”, Proceedings of the 40th

Hawaii International

Conference on System Sciences, IEEE.

6. Huixin Ke, Jinghua Huang, Hao Shen (2007), “Statistic Analysis in Investigation and

Research”, Beijing: Beijing Broadcast University Press, 465-484.

7. Qiong Liu, Xin Lu, Fuji Ren and Shingo Kuroiwa.( 2004), “Automatic Estimation of

Stock Market Forecasting and Generating the Corresponding Natural language

Expression”, IEEE Proceedings of the International Conference on Information

Technology: Coding and Computing.

8. Versace M., Bhatt R., Hinds O. and Shiffer M. (2004), “Predicting the exchange

traded fund DIA with a combination of genetic algorithms and neural networks.”

Expert Systems with applications, Elsevier.

9. Baba N., Naoyuki I. and Hiroyuki A. (2000), “Utilization of Neural Networks &

GAs for Constructing Reliable Decision Support Systems to Deal Stocks.”

Proceedings of IEEE-INNS-ENNS International Joint Conference on Neural

Networks.

10. Sujatha K. V. and S. Meenakshi Sundaram, (2010), “A MLP, RBF Neural Network

Model for Prediction in BSE SENSEX Data Set”, Proceedings of National

Conference on Applied Mathematics.

11. Katijani, Y., W.K. Hipel and A.I. McLeod, (2005), “Forecasting Nonlinear Time

Series with Feedforward Neural Networks: A Case Study of Canadian Lynx Data”.

Journal of Forecasting, 24: 105-117.

12. Yao, J., Y. Li and C.L. Tan, (2000), “ Option Price Forecasting Using Neural

Networks”. Omega, 28: 455-466.

13. Chakraborty, K., Merotra K., Mohan C.K. and Ranka S, (1992), “Forecasting the

Behavior of Multivariate Time-Series Using Neural Network”, Neural Networks, 5:

461-470.

Documents

Regression, theil’s and mlp forecasting models of stock index