Upload
hilda-rusher
View
18
Download
0
Embed Size (px)
DESCRIPTION
Uno de los supuestos de violacion de supuestos en la teoria de uno de los metodos en econometria: MCO (Minimo Cuadrados Ordinarios).
Citation preview
X2 X3 Y
10 19 51
11 21 56
12 23 61
13 25 66
14 27 71
15 29 76
MULTICOLLINEARITY
3232 XXY
12 23 XX
1
Suppose that Y = 2 + 3X2 + X3 and that X3 = 2X2 – 1. There is no disturbance term in the equation for Y, but that is not important. Suppose that we have the six observations shown.
MULTICOLLINEARITY
2
The three variables are plotted as line graphs above. Looking at the data, it is impossible to tell whether the changes in Y are caused by changes in X2, by changes in X3, or jointly by changes in both X2 and X3.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Change from previous observation
X2 X3 Y X2 X3 Y
10 19 51 1 2 5
11 21 56 1 2 5
12 23 61 1 2 5
13 25 66 1 2 5
14 27 71 1 2 5
15 29 76 1 2 5
MULTICOLLINEARITY
3
3232 XXY
12 23 XX
Numerically, Y increases by 5 in each observation. X2 changes by 1.
MULTICOLLINEARITY
4
Hence the true relationship could have been Y = 1 + 5X2.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 1 + 5X2 ?
MULTICOLLINEARITY
3232 XXY
12 23 XX
5
However, it can also be seen that X3 increases by 2 in each observation.
Change from previous observation
X2 X3 Y X2 X3 Y
10 19 51 1 2 5
11 21 56 1 2 5
12 23 61 1 2 5
13 25 66 1 2 5
14 27 71 1 2 5
15 29 76 1 2 5
MULTICOLLINEARITY
6
Hence the true relationship could have been Y = 3.5 +2.5X3.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 3.5 + 2.5X3 ?
MULTICOLLINEARITY
7
These two possibilities are special cases of Y = 3.5 – 2.5p + 5pX2 + 2.5(1 – p)X3, which would fit the relationship for any value of p.
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 3.5 – 2.5p + 5pX2 + 2.5(1 – p)X3
MULTICOLLINEARITY
8
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
Y
X3
X2
Y = 3.5 – 2.5p + 5pX2 + 2.5(1 – p)X3
There is no way that regression analysis, or any other technique, could determine the true relationship from this infinite set of possibilities, given the sample data.
MULTICOLLINEARITY
uXXY 33221 23 XX
9
What would happen if you tried to run a regression when there is an exact linear relationship among the explanatory variables?
MULTICOLLINEARITY
uXXY 33221 23 XX
10
We will investigate, using the model with two explanatory variables shown above. [Note: A disturbance term has now been included in the true model, but it makes no difference to the analysis.]
MULTICOLLINEARITY
uXXY 33221 23 XX
11
23322 XXYYXX iii
2
33222
332
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
The expression for the multiple regression coefficient b2 is shown above. We will substitute for X3 using its relationship with X2.
MULTICOLLINEARITY
uXXY 33221 23 XX
12
23322 XXYYXX iii
2
33222
332
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
222
2
222
2222
222
233 ][][
XX
XXXX
XXXX
i
ii
ii
First, we will replace the terms highlighted.
MULTICOLLINEARITY
uXXY 33221 23 XX
13
We have made the replacement.
222
222 XXYYXX iii
2
33222
2222
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
222
2
222
2222
222
233 ][][
XX
XXXX
XXXX
i
ii
ii
MULTICOLLINEARITY
uXXY 33221 23 XX
14
222
222 XXYYXX iii
2
33222
2222
22
3322332
XXXXXXXX
XXXXYYXXb
iiii
iiii
222
2222
22223322 ][][
XX
XXXX
XXXXXXXX
i
ii
iiii
Next, the terms highlighted now.
MULTICOLLINEARITY
uXXY 33221 23 XX
15
222
222 XXYYXX iii
00
2222
222
2222
22233
2
XXXXXX
XXYYXXb
iii
iii
222
2222
22223322 ][][
XX
XXXX
XXXXXXXX
i
ii
iiii
We have made the replacement.
MULTICOLLINEARITY
uXXY 33221 23 XX
16
222
222 XXYYXX iii
00
2222
222
2222
22233
2
XXXXXX
XXYYXXb
iii
iii
YYXX
YYXX
YYXXYYXX
ii
ii
iiii
22
22
2233 ][][
Finally this term.
00
2222
222
2222
22222
2
XXXXXX
XXYYXXb
iii
iii
MULTICOLLINEARITY
uXXY 33221 23 XX
17
222
222 XXYYXX iii
YYXX
YYXX
YYXXYYXX
ii
ii
iiii
22
22
2233 ][][
Again, we have made the replacement.
MULTICOLLINEARITY
uXXY 33221 23 XX
18
222
222 XXYYXX iii
00
2222
222
2222
22222
2
XXXXXX
XXYYXXb
iii
iii
It turns out that the numerator and the denominator are both equal to zero. The regression coefficient is not defined.
MULTICOLLINEARITY
uXXY 33221 23 XX
19
222
222 XXYYXX iii
00
2222
222
2222
22222
2
XXXXXX
XXYYXXb
iii
iii
It is unusual for there to be an exact relationship among the explanatory variables in a regression. When this occurs, it s typically because there is a logical error in the specification.
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
20
However, it often happens that there is an approximate relationship. For example, when relating earnings to schooling and work experience, it if often reasonable to suppose that the effect of work experience is subject to diminishing returns.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
21
A standard way of allowing for this is to include EXPSQ, the square of EXP, in the specification. According to the hypothesis of diminishing returns, 4 should be negative.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
22
We fit this specification using Data Set 21. The schooling component of the regression results is not much affected by the inclusion of the EXPSQ term. The coefficient of S indicates that an extra year of schooling increases hourly earnings by $2.75.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 2, 537) = 67.54 Model | 22513.6473 2 11256.8237 Prob > F = 0.0000 Residual | 89496.5838 537 166.660305 R-squared = 0.2010-------------+------------------------------ Adj R-squared = 0.1980 Total | 112010.231 539 207.811189 Root MSE = 12.91
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
23
In the specification without EXPSQ it was 2.68, not much different.
uEXPSQEXPSEARNINGS 4321
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
24
uEXPSQEXPSEARNINGS 4321
The standard error, 0.23 in the specification without EXPSQ, is also little changed and the coefficient remains highly significant.
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
25
uEXPSQEXPSEARNINGS 4321
By contrast, the inclusion of the new term has had a dramatic effect on the coefficient of EXP. Now it is negative, which makes little sense, and insignificant.
MULTICOLLINEARITY
26
uEXPSQEXPSEARNINGS 4321
Previously it had been positive and highly significant.
. reg EARNINGS S EXP
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 2, 537) = 67.54 Model | 22513.6473 2 11256.8237 Prob > F = 0.0000 Residual | 89496.5838 537 166.660305 R-squared = 0.2010-------------+------------------------------ Adj R-squared = 0.1980 Total | 112010.231 539 207.811189 Root MSE = 12.91
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
27
uEXPSQEXPSEARNINGS 4321
The coefficient of EXPSQ is also strange. It is positive, suggesting increasing returns to experience. However, it is not significant.
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
MULTICOLLINEARITY
28
uEXPSQEXPSEARNINGS 4321
The reason for these problems is that EXPSQ is highly correlated with EXP. This makes it difficult to discriminate between the individual effects of EXP and EXPSQ, and the regression estimates tend to be erratic.
. cor EXP EXPSQ(obs=540)
| EXP EXPSQ------+------------------ EXP | 1.0000EXPSQ | 0.9812 1.0000
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
29
The high correlation causes the standard error of EXP to be larger than it would have been if EXP and EXPSQ had been less highly correlated, warning us that the point estimate is unreliable.
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
30
When high correlations among the explanatory variables lead to erratic point estimates of the coefficients, large standard errors and unsatisfactorily low t statistics, the regression is said to said to be suffering from multicollinearity.
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
31
Note that the coefficients remain unbiased and the standard errors remain valid.
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.678125 .2336497 11.46 0.000 2.219146 3.137105 EXP | .5624326 .1285136 4.38 0.000 .3099816 .8148837 _cons | -26.48501 4.27251 -6.20 0.000 -34.87789 -18.09213------------------------------------------------------------------------------
MULTICOLLINEARITY
32
Multicollinearity may also be caused by an approximate linear relationship among the explanatory variables. When there are only 2, an approximate linear relationship means there will be a high correlation, but this is not always the case when there are more than 2.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
1
What can you do about multicollinearity if you encounter it? We will discuss some possible measures, looking at the model with two explanatory variables.
2
Before doing this, two important points should be emphasized. First, multicollinearity does not cause the regression coefficients to be biased.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
3
The problem is that they have unsatisfactorily large variances.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
4
Second, the standard errors and t tests remain valid. The standard errors are larger than they would have been in the absence of multicollinearity, warning us that the regression estimates are erratic.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
5
Since the problem of multicollinearity is caused by the variances of the coefficients being unsatisfactorily large, we will seek ways of reducing them.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(1) Reduce by including further relevant variables in the model.2u
6
We will focus on the slope coefficient and look at the various components of its variance. We might be able to reduce it by bringing more variables into the model and reducing u
2, the variance of the disturbance term.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
7
The estimator of the variance of the disturbance term is the residual sum of squares divided by n – k, where n is the number of observations (540) and k is the number of parameters (4). Here it is 166.5.
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
8
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585-------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
We now add two new variables that are often found to be determinants of earnings: MALE, sex of respondent, and ASVABC, the composite score on the cognitive tests in the Armed Services Vocational Aptitude Battery.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
9
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585-------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
MALE is a qualitative variable and the treatment of such variables will be explained in Chapter 5.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
10
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585-------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
Both MALE and ASVABC have coefficients significant at the 0.1% level.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 45.57 Model | 22762.4472 3 7587.48241 Prob > F = 0.0000 Residual | 89247.7839 536 166.507059 R-squared = 0.2032-------------+------------------------------ Adj R-squared = 0.1988 Total | 112010.231 539 207.811189 Root MSE = 12.904
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585-------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471
11
However they account for only a small proportion of the variance in earnings and the reduction in the estimate of the variance of the disturbance term is likewise small.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP EXPSQ MALE ASVABC
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
12
As a consequence the impact on the standard errors of EXP and EXPSQ is negligible.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP EXPSQ MALE ASVABC
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
13
Note how unstable the coefficients are. This is often a sign of multicollinearity.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP EXPSQ MALE ASVABC
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
14
Note also that the standard error of the coefficient of S has actually increased. This is attributable to the correlation of 0.58 between S and ASVABC.
. cor S ASVABC(obs=540) | S ASVABC--------+------------------ S | 1.0000 ASVABC | 0.5810 1.0000
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.754372 .2417286 11.39 0.000 2.279521 3.229224 EXP | -.2353907 .665197 -0.35 0.724 -1.542103 1.071322 EXPSQ | .0267843 .0219115 1.22 0.222 -.0162586 .0698272 _cons | -22.21964 5.514827 -4.03 0.000 -33.05297 -11.38632------------------------------------------------------------------------------
. reg EARNINGS S EXP EXPSQ MALE ASVABC
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
15
This is a common problem with this approach to attempting to reduce the problem of multicollinearity. If the new variables are linearly related to one or more of the variables already in the equation, their inclusion may make the problem of multicollinearity worse.
. cor S ASVABC(obs=540) | S ASVABC--------+------------------ S | 1.0000 ASVABC | 0.5810 1.0000
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
16
The next factor to look at is n, the number of observations. If you are working with cross-section data (individuals, households, enterprises, etc) and you are undertaking a survey, you could increase the size of the sample by negotiating a bigger budget.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
(2) Increase the number of observations.
Surveys: increase the budget, use clustering
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
17
Alternatively, you could make a fixed budget go further by using a technique known as clustering. You divide the country geographically by zip code or postal area.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
(2) Increase the number of observations.
Surveys: increase the budget, use clustering
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
18
You select a number of these randomly, perhaps using stratified random sampling to make sure that metropolitan, other urban, and rural areas are properly represented.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
(2) Increase the number of observations.
Surveys: increase the budget, use clustering
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
19
You then confine the survey to the areas selected. This reduces the travel time and cost of the fieldworkers, allowing them to interview a greater number of respondents.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
(2) Increase the number of observations.
Surveys: increase the budget, use clustering
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(2) Increase the number of observations.
Surveys: increase the budget, use clustering
Time series: use quarterly instead of annual data
20
If you are working with time series data, you may be able to increase the sample by working with shorter time intervals for the data, for example quarterly or even monthly data instead of annual data.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source | SS df MS Number of obs = 2714-------------+------------------------------ F( 5, 2708) = 183.99 Model | 161795.573 5 32359.1147 Prob > F = 0.0000 Residual | 476277.268 2708 175.877869 R-squared = 0.2536-------------+------------------------------ Adj R-squared = 0.2522 Total | 638072.841 2713 235.190874 Root MSE = 13.262
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.312461 .135428 17.08 0.000 2.046909 2.578014 EXP | -.3270651 .308231 -1.06 0.289 -.9314569 .2773268 EXPSQ | .023743 .0101558 2.34 0.019 .0038291 .0436569 MALE | 5.947206 .5221755 11.39 0.000 4.923303 6.971108 ASVABC | .2086846 .0336869 6.19 0.000 .1426301 .2747392 _cons | -27.40462 2.579435 -10.62 0.000 -32.46248 -22.34676------------------------------------------------------------------------------
21
Here is the result of running the regression with all 2,714 observations in the EAEF data set.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 540------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 2714------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.312461 .135428 17.08 0.000 2.046909 2.578014 EXP | -.3270651 .308231 -1.06 0.289 -.9314569 .2773268 EXPSQ | .023743 .0101558 2.34 0.019 .0038291 .0436569 MALE | 5.947206 .5221755 11.39 0.000 4.923303 6.971108 ASVABC | .2086846 .0336869 6.19 0.000 .1426301 .2747392 _cons | -27.40462 2.579435 -10.62 0.000 -32.46248 -22.34676------------------------------------------------------------------------------
22
Comparing this result with that using Data Set 21, we see that the standard errors are much smaller, as expected.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 540------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 2714------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.312461 .135428 17.08 0.000 2.046909 2.578014 EXP | -.3270651 .308231 -1.06 0.289 -.9314569 .2773268 EXPSQ | .023743 .0101558 2.34 0.019 .0038291 .0436569 MALE | 5.947206 .5221755 11.39 0.000 4.923303 6.971108 ASVABC | .2086846 .0336869 6.19 0.000 .1426301 .2747392 _cons | -27.40462 2.579435 -10.62 0.000 -32.46248 -22.34676------------------------------------------------------------------------------
23
As a consequence, the t statistics of the variables are higher. However the correlation between EXP and EXPSQ is as high as in the smaller sample and the increase in the sample size has not been large enough to have much impact on the problem of multicollinearity.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 540------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 2714------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.312461 .135428 17.08 0.000 2.046909 2.578014 EXP | -.3270651 .308231 -1.06 0.289 -.9314569 .2773268 EXPSQ | .023743 .0101558 2.34 0.019 .0038291 .0436569 MALE | 5.947206 .5221755 11.39 0.000 4.923303 6.971108 ASVABC | .2086846 .0336869 6.19 0.000 .1426301 .2747392 _cons | -27.40462 2.579435 -10.62 0.000 -32.46248 -22.34676------------------------------------------------------------------------------
24
The coefficients of EXP and EXPSQ both still have unexpected signs since we expect the coefficient of EXP to be positive and that of EXPSQ to be negative, reflecting diminishing returns.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 540------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 2714------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.312461 .135428 17.08 0.000 2.046909 2.578014 EXP | -.3270651 .308231 -1.06 0.289 -.9314569 .2773268 EXPSQ | .023743 .0101558 2.34 0.019 .0038291 .0436569 MALE | 5.947206 .5221755 11.39 0.000 4.923303 6.971108 ASVABC | .2086846 .0336869 6.19 0.000 .1426301 .2747392 _cons | -27.40462 2.579435 -10.62 0.000 -32.46248 -22.34676------------------------------------------------------------------------------
25
The EXPSQ coefficient has a rather large t statistic, which is a matter of concern. We could assume that this has occurred as a matter of chance. Alternatively, it might be an indication that the model is misspecified.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 540------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC Number of obs = 2714------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.312461 .135428 17.08 0.000 2.046909 2.578014 EXP | -.3270651 .308231 -1.06 0.289 -.9314569 .2773268 EXPSQ | .023743 .0101558 2.34 0.019 .0038291 .0436569 MALE | 5.947206 .5221755 11.39 0.000 4.923303 6.971108 ASVABC | .2086846 .0336869 6.19 0.000 .1426301 .2747392 _cons | -27.40462 2.579435 -10.62 0.000 -32.46248 -22.34676------------------------------------------------------------------------------
26
As we will see in the next and subsequent chapters, there are good reasons for supposing that the dependent variable in an earnings function should be the logarithm of earnings, rather than earnings in linear form.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
27
A third possible way of reducing the problem of multicollinearity might be to increase the variation in the explanatory variables. This is possible only at the design stage of a survey.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(3) Increase MSD(X2).
28
For example, if you were planning a household survey with the aim of investigating how expenditure patterns vary with income, you should make sure that the sample included relatively rich and relatively poor households as well as middle-income households.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(3) Increase MSD(X2).
(4) Reduce .32 ,XXr
29
Another possibility might be to reduce the correlation between the explanatory variables. This is possible only at the design stage of a survey and even then it is not easy.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(5) Combine the correlated variables.
30
If the correlated variables are similar conceptually, it may be reasonable to combine them into some overall index.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
31
That is precisely what has been done with the three cognitive ASVAB variables. ASVABC has been calculated as a weighted average of ASVAB02 (arithmetic reasoning), ASVAB03 (word knowledge), and ASVAB04 (paragraph comprehension).
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585-------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
32
The three components are highly correlated and by combining them as a weighted average, rather than using them individually, one avoids a potential problem of multicollinearity.
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 5, 534) = 37.24 Model | 28957.3532 5 5791.47063 Prob > F = 0.0000 Residual | 83052.8779 534 155.529734 R-squared = 0.2585-------------+------------------------------ Adj R-squared = 0.2516 Total | 112010.231 539 207.811189 Root MSE = 12.471
------------------------------------------------------------------------------ EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- S | 2.031419 .296218 6.86 0.000 1.449524 2.613315 EXP | -.0816828 .6441767 -0.13 0.899 -1.347114 1.183748 EXPSQ | .0130223 .021334 0.61 0.542 -.0288866 .0549311 MALE | 5.762358 1.104734 5.22 0.000 3.592201 7.932515 ASVABC | .2447687 .0714294 3.43 0.001 .1044516 .3850858 _cons | -26.18541 5.452032 -4.80 0.000 -36.89547 -15.47535------------------------------------------------------------------------------
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
(6) Drop some of the correlated variables.
33
Dropping some of the correlated variables, if they have insignificant coefficients, may alleviate multicollinearity.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
34
However, this approach to multicollinearity is dangerous. It is possible that some of the variables with insignificant coefficients really do belong in the model and that the only reason their coefficients are insignificant is because there is a problem of multicollinearity.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(6) Drop some of the correlated variables.
35
If that is the case, their omission may cause omitted variable bias, to be discussed in Chapter 6.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(6) Drop some of the correlated variables.
(7) Empirical restriction
36
A further way of dealing with the problem of multicollinearity is to use extraneous information, if available, concerning the coefficient of one of the variables.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
uPXY 321
37
For example, suppose that Y in the equation above is the demand for a category of consumer expenditure, X is aggregate disposable personal income, and P is a price index for the category.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
(7) Empirical restriction
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
uPXY 321
38
To fit a model of this type you would use time series data. If X and P are highly correlated, which is often the case with time series variables, the problem of multicollinearity might be eliminated in the following way.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(7) Empirical restriction
uPXY 321
39
Obtain data on income and expenditure on the category from a household survey and regress Y' on X'. (The ' marks are to indicate that the data are household data, not aggregate data.)
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(7) Empirical restriction
uPXY 321 uXY ''2
'1
' ''
2'1
'ˆ XbbY
40
This is a simple regression because there will be relatively little variation in the price paid by the households.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(7) Empirical restriction
uPXY 321 uXY ''2
'1
' ''
2'1
'ˆ XbbY
41
Now substitute b' for 2 in the time series model. Subtract b' X from both sides, and regress Z = Y – b' X on price. This is a simple regression, so multicollinearity has been eliminated.
22
2
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(7) Empirical restriction
uPXY 321 uXY ''2
'1
'
uPXbY 3'21
uPXbYZ 21'2
''2
'1
'ˆ XbbY
42
There are some problems with this technique. First, the 2 coefficients may be conceptually different in time series and cross-section contexts.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(7) Empirical restriction
uPXY 321 uXY ''2
'1
'
uPXbY 3'21
uPXbYZ 21'2
''2
'1
'ˆ XbbY
uPXY 321 uXY ''2
'1
'
uPXbY 3'21
uPXbYZ 21'2
''2
'1
'ˆ XbbY
43
Second, since we subtract the estimated income component b' X, not the true income component 2X, from Y when constructing Z, we have introduced an element of measurement error in the dependent variable.
2
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(7) Empirical restriction
44
Last, but by no means least, is the use of a theoretical restriction, which is defined as a hypothetical relationship among the parameters of a regression model.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(8) Theoretical restriction
45
It will be explained using an educational attainment model as an example. Suppose that we hypothesize that highest grade completed, S, depends on ASVABC, and highest grade completed by the respondent's mother and father, SM and SF, respectively.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(8) Theoretical restriction
uSFSMASVABCS 4321
. reg S ASVABC SM SF
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686-------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
46
A one-point increase in ASVABC increases S by 0.13 years.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686-------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
47
S increases by 0.05 years for every extra year of schooling of the mother and 0.11 years for every extra year of schooling of the father.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686-------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
48
Mother's education is generally held to be at least, if not more, important than father's education for educational attainment, so this outcome is unexpected.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686-------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
49
It is also surprising that the coefficient of SM is not significant, even at the 5% level, using a one-sided test.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 3, 536) = 104.30 Model | 1181.36981 3 393.789935 Prob > F = 0.0000 Residual | 2023.61353 536 3.77539837 R-squared = 0.3686-------------+------------------------------ Adj R-squared = 0.3651 Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
50
However assortive mating leads to correlation between SM and SF and the regression appears to be suffering from multicollinearity.
. cor SM SF(obs=540) | SM SF--------+------------------ SM | 1.0000 SF | 0.6241 1.0000
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
51
Suppose that we hypothesize that mother's and father's education are equally important. We can then impose the restriction 3 = 4.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(8) Theoretical restriction
uSFSMASVABCS 4321
43
52
This allows us to rewrite the equation as shown.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
(8) Theoretical restriction
uSFSMASVABCS 4321
43
uSPASVABC
uSFSMASVABCS
321
321 )(
(8) Theoretical restriction
uSFSMASVABCS 4321
43
uSPASVABC
uSFSMASVABCS
321
321 )(
53
Defining SP to be the sum of SM and SF, the equation may be rewritten as shown. The problem caused by the correlation between SM and SF has been eliminated.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
2,2
2
2,
222
22
3232
2 11
)(MSD11
XX
u
XXi
ub rXnrXX
. g SP=SM+SF
. reg S ASVABC SP
Source | SS df MS Number of obs = 540-------------+------------------------------ F( 2, 537) = 156.04 Model | 1177.98338 2 588.991689 Prob > F = 0.0000 Residual | 2026.99996 537 3.77467403 R-squared = 0.3675-------------+------------------------------ Adj R-squared = 0.3652 Total | 3204.98333 539 5.94616574 Root MSE = 1.9429
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469 SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014 _cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608------------------------------------------------------------------------------
54
The estimate of 3 is now 0.083.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. g SP=SM+SF
. reg S ASVABC SP
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469 SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014 _cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608------------------------------------------------------------------------------
. reg S ASVABC SM SF
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
55
Not surprisingly, this is a compromise between the coefficients of SM and SF in the previous specification.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. g SP=SM+SF
. reg S ASVABC SP
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469 SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014 _cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608------------------------------------------------------------------------------
. reg S ASVABC SM SF
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
56
The standard error of SP is much smaller than those of SM and SF. The use of the restriction has led to a large gain in efficiency and the problem of multicollinearity has been eliminated.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. g SP=SM+SF
. reg S ASVABC SP
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469 SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014 _cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608------------------------------------------------------------------------------
. reg S ASVABC SM SF
------------------------------------------------------------------------------ S | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646 SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309 SF | .1076825 .0309522 3.48 0.001 .04688 .1684851 _cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681------------------------------------------------------------------------------
57
The t statistic is very high. Thus it would appear that imposing the restriction has improved the regression results. However, the restriction may not be valid. We should test it. Testing theoretical restrictions is one of the topics in Chapter 6.
POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY