Section C Group 9

7/31/2019 Section C Group 9

1/17

Analysis of presence of

Multicollinearity QAM-II

Submitted to Prof. Abhijit Bhattacharya

SUBMITTED BY GROUP 5

December 13, 2011

AmeyRambole ABM08009 NitinRawat PGP26104

AbhijitTalukdar PGP27134 Akash Joshi PGP27136

Manish Pushkar PGP27161 Suraj Somashekhar PGP27187

SumeetChoudhary PGP27185


2/17

1

Problem StatementThe owner of Pizza Corner, Bangalore would like to build a regression model consisting of six well defined

explanatory variables to predict the sales of pizzas. The six variables are:

X1 : Number of delivery boys

X2 : Cost (in Rupees) of advertisements (000s)X3 : Number of outlets

X4 : Varieties of pizzas

X5 : Competitors activities indexX6 : Number of existing customers (000s)

Sales data of past fifteen months and the above listed variables is given below:

SALES DATA FOR PIZZA

Month Sales X1 X2 X3 X4 X5 X6

1 81 15 20 35 17 4 70

2 23 10 12 10 13 4 43

3 18 7 11 14 14 3 31

4 8 2 6 9 13 3 10

5 16 4 10 11 12 4 17

6 4 1 5 6 12 5 8

7 29 4 14 15 15 2 39

8 22 7 12 16 16 3 40

9 15 5 10 18 15 4 30

10 6 3 5 8 13 2 16

11 45 13 17 20 14 2 30

12 11 2 9 10 12 3 2013 20 5 12 15 12 3 25

14 60 12 18 30 15 4 50

15 5 1 5 6 12 5 20


3/17

2

Initial Regression Model

Y(Sales) was regressed over X1,X2,X3,X4,X5 and X6 in SPSS

The following model was developed:-

^Ysales = 6.372 + .919X1 + .699X2 + 1.620X3 - 1.978X4 + .067X5 + .242X6

SPSS Regression Output for initial model

Descriptive Statistics

Mean Std. Deviation N

Sales 24.20 21.913 15

X1 6.07 4.511 15

X2 11.07 4.758 15

X3 14.87 8.340 15

X4 13.67 1.633 15

X5 3.40 .986 15

X6 29.93 16.529 15

This table depicts mean, standard deviation and size of sample provided across 15 months for variousdependent and independent variables.

Variables Entered/Removedb

Model

Variables

Entered

Variables

Removed Method

1 X6, X5, X4, X1,

X3, X2a. Enter

a. All requested variables entered.

b. Dependent Variable: Sales

This table tells you the method that SPSS used to run the regression. "Enter" means that eachindependent variable was entered in usual fashion. It says all variables were entered and no variable

was removed


4/17

3

Correlations

Sales X1 X2 X3 X4 X5 X6

Pearson Correlation Sales 1.000 .902 .934 .953 .725 -.040 .880

X1 .902 1.000 .905 .845 .672 -.103 .841

X2 .934 .905 1.000 .904 .702 -.189 .867

X3 .953 .845 .904 1.000 .794 -.036 .856

X4 .725 .672 .702 .794 1.000 -.178 .819

X5 -.040 -.103 -.189 -.036 -.178 1.000 .006

X6 .880 .841 .867 .856 .819 .006 1.000

This table provides individual correlation coefficients between dependent and independent variablesModel Summary

b

Model R R Square

Adjusted R

Square

Std. Error of the

Estimate Durbin-Watson

1 .976a

.953 .918 6.260 1.745

a. Predictors: (Constant), X6, X5, X4, X1, X3, X2


R is the correlation between the observed and predicted values of dependent variable.


5/17

4

R-Square95.3 % of variation is explained by model. This is the proportion of variance in thedependent variable (Sales) which can be explained by the independent variables (X6, X5, X4, X1, X3,

and X2). This is an overall measure of the strength of association and does not reflect the extent to

which any particular independent variable is associated with the dependent variable.

Adjusted R-square91.8 % of variation in sales is explained by model- adjusted for number ofindependent variables and sample size.

Std. Error of the Estimate - This is also referred to as the root mean squared error. It is the standarddeviation of the error term and the square root of the Mean Square for the Residuals in the ANOVA

table

ANOVAb

Model Sum of Squares Df Mean Square F Sig.

1 Regression 6408.864 6 1068.144 27.254 .000a

Residual 313.536 8 39.192

Total 6722.400 14

a. Predictors: (Constant), X6, X5, X4, X1, X3, X2


Sum of Squares - These are the Sum of Squares associated with the three sources of variance, Total,Regression and Residual. The Total variance is partitioned into the variance which can be explained by

the independent variables (Regression) and the variance which is not explained by the independent

variables (Residual). df- These are the degrees of freedom associated with the sources of variance. The total variance has 14

(N-1) degrees of freedom. The Regression degrees of freedom correspond to the number of coefficients

estimated minus 1. Including the intercept, there are 7 coefficients, so the model has 7-1=6 degrees of

freedom. The Error degrees of freedom are the DF total minus the DF model, 14 - 6 =8.

Mean Square - These are the Mean Squares, the Sum of Squares divided by their respective DF.


6/17

5

F and Sig. - This is the F-statistic the p-value associated with it. The F-statistic is the Mean Square(Regression) divided by the Mean Square (Residual): 1068.144/39.192 = 27.254. The p-value is

compared to some alpha level in testing the null hypothesis that all of the model coefficients are 0.

H0: 1 = 2= 3=4= 5= 6=0

H1: Not all j are 0

In this model we reject H0 and can conclude that not all j are 0 as p-value is 0

Sales depends significantly upon some predictors.

Coefficientsa

Model

UnstandardizedCoefficients

StandardizedCoefficients

t Sig.

95%

ConfidenceInterval for B Collinearity Statistics

B

Std.

Error Beta

Lower

Bound

Upper

Bound Tolerance VIF

1 (Const

ant)6.372 32.586 .196 .850

-

68.77381.516

X1 .919 .910 .189 1.010 .342 -1.179 3.017 .166 6.018

X2 .699 1.303 .152 .537 .606 -2.306 3.704 .073 13.733

X3 1.620 .618 .617 2.621 .031 .195 3.046 .105 9.500

X4 -1.978 2.310 -.147 -.856 .417 -7.305 3.349 .197 5.083

X5 .067 2.211 .003 .030 .977 -5.032 5.165 .589 1.696

X6 .242 .299 .182 .808 .442 -.448 .931 .115 8.719

a. Dependent

Variable: Sales

B - These are the values for the regression equation for predicting the dependent variable from theindependent variable. Interpretations made are


7/17

6

o Controlling other variables constant, if Number of delivery boys is increased by 1 then Saleswill increase by 0.919

o Controlling other variables constant, if cost (in rupees) of ads (000s) is increased by 1 thenSales will increase by 0.699

o Controlling other variables constant, if Number of outlets is increased by 1 then Sales willincrease by 1.620

o Controlling other variables constant, if varieties of pizza is increased by 1 then Sales willdecrease by 1.978

o Controlling other variables constant, if Competitors activities index is increased by 1 thenSales will increase by 0.067

o Controlling other variables constant, if Number of existing customers (000s) is increased by 1then Sales will increase by 0.242

Std. Error: These are the standard errors associated with the coefficients. Beta - These are the standardized coefficients. By standardizing the variables before running the

regression, you have put all of the variables on the same scale, and you can compare the magnitude of

the coefficients to see which one has more of an effect. We can interpret these coefficients in same way

as we did B values.

Here B for constant is 0.Look for the regression coefficient having the highest magnitude.

Corresponding regressor contributes the most.

T and Sig. - These are the t-statistics and their associated 2-tailed p-values used in testing whether agiven coefficient is significantly different from zero. Using an alpha of 0.05:

o The coefficient for X1 (0.919) is not significantly related to dependent variable because its p-value is 0.342, which is greater than 0.05.


o The coefficient for X3 (1.620) is significantly related to dependent variable because its p-valueis 0.031, which is less than 0.05.

1 2

1 1 2 2

1 2

Standardized ,

Standardized , Standardized

i

Y

i i

i i

X X

Y YY

s

X X X XX X

s s


8/17

7

o The coefficient for X4 (-1.978) is not significantly related to dependent variable because its p-value is 0.417, which is greater than 0.05.



Tolerance - The tolerance of a variable is defined as 1 minus the squared multiple correlation of thisvariable with all other independent variables in the regression equation. Therefore, the smaller the

tolerance of a variable, the more redundant is its contribution to the regression (i.e., it is redundant with

the contribution of other independent variables). Small value of Tolerance indicates multicollinearity.

VIFit is equal to 1/Tolerance. High value (more than 10) of VIF indicates multicollinearity.Collinearity Diagnostics

a

Mode

l

Dimens

ion Eigenvalue

Condition

Index

Variance Proportions

(Constant) X1 X2 X3 X4 X5 X6

1 1 6.470 1.000 .00 .00 .00 .00 .00 .00 .00

2 .388 4.082 .00 .04 .00 .01 .00 .03 .00

3 .052 11.145 .01 .04 .02 .00 .01 .42 .03

4 .044 12.101 .00 .64 .00 .12 .00 .00 .13

5 .032 14.166 .00 .00 .00 .34 .00 .03 .38

6 .012 23.582 .00 .27 .63 .14 .04 .09 .00

7 .001 74.026 .99 .00 .35 .39 .95 .42 .46

a. Dependent Variable: Sales

Smaller value of Eigen value () indicates presence of multicollinearity Condition Index = (Maximum value of)/ (Minimum value of). High value of CI indicates

multicollinearity.


9/17

8

1. Check for MulticollinearityMulticollinearity is a statistical phenomenon in which two or more predictor variables in a multiple regression

model are highly correlated. In this situation the coefficient estimates may change erratically in response to

small changes in the model or the data. Multicollinearity does not reduce the predictive power or reliability of

the model as a whole, at least within the sample data themselves; it only affects calculations regarding

individual predictors. That is, a multiple regression model with correlated predictors can indicate how well theentire bundle of predictors predicts the outcome variable, but it may not give valid results about any individual

predictor, or about which predictors are redundant with respect to others.

The primary concern is that as the degree of multicollinearity increases, the regression model estimates of the

coefficients become unstable and the standard errors for the coefficients can get wildly inflated.

VIF Test: Since VIF value of Variable X2 is greater than 10 (13.733) in Coefficients table there ispresence of collinearity

Eigen Value Test: Very small Eigen value (.001) of 7th dimension and very high value of itscorresponding Condition index (74.026) in Collinearity Diagnostics table indicates presence of

collinearity.

Pearson Correlation:Correlations


Pearson

Correlation

Sales 1.000

X1 .902 1.000

X2 .934 .905 1.000

X3 .953 .845 .904 1.000

X4 .725 .672 .702 .794 1.000

X5 -.040 -.103 -.189 -.036 -.178 1.000

X6 .880 .841 .867 .856 .819 .006 1.000


10/17

9

If the absolute value of Pearson correlation is greater than 0.9, collinearity is very likely to exist. Values

in Bold indicate multicollinearity.

Statistical TestFrom the above correlations table we can see that Pearson correlation between (X1, Sales) is .902, (X2,

Sales) is .934

Corresponding P-value of X1 and X2 are .342 and .606 resp.

Considering 10 % level of significance these high p values does not suggest rejection of H0: j =0. So

via T-test we conclude that X1 and X2 are not significantly related to dependent variable.

So there is presence of collinearity as Pearson correlations and T-test present contradictory information.

Variable X1 X2 X3 X4 X5 X6

T statistic 1.010 .537 2.621 -.856 .030 .808

Sig. (p value) .342 .606 .031 .417 .977 .442

2. SPSS Stepwise Regression equation^Ysales = -11.817 + 1.640X1 + 1.753X3


11/17

10

3. SPSS Stepwise Regression Output

Descriptive Statistics

Mean Std. Deviation N

Sales 24.2000 21.91281 15

X1 6.0667 4.51136 15

X2 11.0667 4.75795 15

X3 14.8667 8.33981 15

X4 13.6667 1.63299 15

X5 3.4000 .98561 15

X6 29.9333 16.52905 15

This table depicts mean, standard deviation and size of sample provided across 15 months for variousdependent and independent variables.

Variables Entered/Removeda

Model

Variables

Entered

Variables

Removed Method

1X3 .

Stepwise (Criteria: Probability-of-F-to-enter = .100).

2X1 .

Stepwise (Criteria: Probability-of-F-to-enter = .100).


Here we can see in first iteration X3 is entered because it has highest r value correlation with Sales(Check from correlation table)


12/17

11

Variables are only added or removed if the Sig F Change value is significant.For this SPSS performs regression between Y and (X3,X1), Y and (X3,X2) and so on till Y and

(X3,X6)

It calculates then

F (1,n-3)= (SSR(X3,XI)-SSR(X3))/MSSE(X3,XI) for all (X3,XI) combinations

It then includes XI which has maximum F ratio

o This value can be obtained from the table ofExcluded Variables. It shows the PartialCorrelation between each candidate for entry and the dependent variable.

o Partial correlation is a measure of the relationship of the dependent variable to an independentvariable, where the variance explained by previously entered independent variables has been

removed from both.

o From theExcluded Variables table we can see X1 has maximum partial correlation .594 andminimum p-value .025. So X1 is next variable to be entered subjected to whether resulting

model with Sales as dependent variable and (X1, X3) as independent variable improve R

The process of adding more variables stops when all of the available variables have been included orwhen it is not possible to make a statistically significant improvement in R using any of the variables

not yet included.


13/17

12

Correlations


Pearson Correlation Sales 1.000 .902 .934 .953 .725 -.040 .880

X1 .902 1.000 .905 .845 .672 -.103 .841

X2 .934 .905 1.000 .904 .702 -.189 .867

X3 .953 .845 .904 1.000 .794 -.036 .856

X4 .725 .672 .702 .794 1.000 -.178 .819

X5 -.040 -.103 -.189 -.036 -.178 1.000 .006

X6 .880 .841 .867 .856 .819 .006 1.000

This table provides individual correlation coefficients between dependent and independent variables In step wise regression this table is used to find out first variable to be entered which has maximum

correlation coefficient with Sales. In this case it is X3.

Model Summaryc

Model R R Square

Adjusted R

Square

Std. Error of the

Estimate Durbin-Watson

1 .953a .908 .900 6.91277

2 .970b

.940 .930 5.78925 1.477

a. Predictors: (Constant), X3

b. Predictors: (Constant), X3, X1

c. Dependent Variable: Sales


14/17

13

As we can see with inclusion of X1 in model in iteration 2 has increased R square value so model isable to explain 94 % of variability in Sales

Other parameters are explained earlier in section

ANOVAc

Model Sum of Squares Df Mean Square F Sig.

1 Regression 6101.176 1 6101.176 127.676 .000a

Residual 621.224 13 47.786

Total 6722.400 14

2 Regression 6320.215 2 3160.108 94.288 .000b

Residual 402.185 12 33.515

Total 6722.400 14

a. Predictors: (Constant), X3

b. Predictors: (Constant), X3, X1 c. Dependent Variable: Sales

F and Sig. - This is the F-statistic the p-value associated with it. The p-value is compared to some alphalevel in testing the null hypothesis that all of the model coefficients are 0.

H0: 1 = 2= 3=j=0 H1: Not all j are 0

In both models we reject H0 and can conclude that not all j are 0 as p-value is 0 in both cases

Sales depend significantly upon X3 in model 1. Sales depend significantly upon X3 or X1 in model 2.

To find which one look in coefficients table output

Other parameters are explained in section 3


15/17

14

Coefficientsa

Model

Unstandardized

Coefficients

Standardized

Coefficients

t Sig.

95% Confidence

Interval for B

Collinearity

Statistics

B

Std.

Error Beta

Lower

Bound

Upper

Bound Tolerance VIF

1 (Constant) -13.013 3.746 -3.474 .004 -21.106 -4.921

X3 2.503 .222 .953 11.299 .000 2.025 2.982 1.000 1.000

2 (Constant) -11.817 3.172 -3.726 .003 -18.728 -4.906

X3 1.753 .347 .667 5.053 .000 .997 2.510 .286 3.498

X1 1.640 .641 .338 2.556 .025 .242 3.038 .286 3.498


T stat and Sig for Model 2o The coefficient for X1 (1.640) is significantly related to dependent variable because its p-value is

0.025, which is less than 0.05.

o The coefficient for X3 (1.753) is significantly related to dependent variable because its p-value is0.000, which is less than 0.05.

B Values for Model 2o Controlling other variables constant, if Number of delivery boys is increased by 1 then Sales will

increase by 0.1.640

o Controlling other variables constant, if Number of outlets is increased by 1 then Sales will increaseby 1.753

Since VIF value of Variable X1 and X3 is less than 10 (3.498) in Coefficients table we can safely saythere is no multicollinearity.

Other parameters are explained in section 3


16/17

15

Excluded Variablesc

Model Beta In t Sig.

Partial

Correlation

Collinearity Statistics

Tolerance VIF

Minimum

Tolerance

1 X1 .338a

2.556 .025 .594 .286 3.498 .286

X2 .400a 2.361 .036 .563 .183 5.465 .183

X4 -.085a -.600 .560 -.171 .370 2.703 .370

X5 -.006a -.064 .950 -.018 .999 1.001 .999

X6 .241a

1.560 .145 .411 .267 3.740 .267

2 X2 .226b

1.085 .301 .311 .113 8.820 .113

X4 -.087b

-.732 .480 -.215 .370 2.703 .193

X5 .019b

.257 .802 .077 .981 1.020 .281

X6 .113b .736 .477 .217 .219 4.569 .214

a. Predictors in the Model: (Constant), X3

b. Predictors in the Model: (Constant), X3, X1

T test and Sigo Model 1

X1 and X2 are eligible to enter after iteration 1 as both have p-value less than .05. ButX1 has higher partial correlation so it enters in second iteration.

X4, X5, and X6 have high p-value > .05 so H0: j =0 is not rejected. So Sales is notsignificantly dependent on X4, X5, and X6.

o Model 2


17/17

16

X2, X4, X5, and X6 have high p-value > .05 so H0: j =0 is not rejected. So Sales is notsignificantly dependent on X2, X4, X5, and X6.

So final model has only significant dependence on X3 and X1

Collinearity Diagnosticsa

Model

Dimens

ion Eigenvalue Condition Index

Variance Proportions

(Constant) X3 X1

1 1 1.879 1.000 .06 .06

2 .121 3.944 .94 .94

2 1 2.761 1.000 .03 .01 .01

2 .198 3.732 .71 .01 .16

3 .040 8.273 .26 .98 .83


In model 2 Eigenvalues are not close to 0 and Condition index is less than 10 so there is less evidence of

multicollinearity.

Documents

Section C Group 9