MULTIPLE REGRESSION WITH P > 2 PREDICTORSion.uwinnipeg.ca/~clark/teach/4100/_Fall/30-Ch11-15.pdf · 15 59 96 17 143 16 41 75 19 166 17 69 112 20 146 18 63 108 13 115 19 67 120 14

MR and Multiple Predictors 11.1

CHAPTER 11:

MULTIPLE REGRESSION WITH P > 2 PREDICTORS

The MR Equation with Multiple Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

The Overall Relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Significance of Individual Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Strength of Unique Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

A Four Predictor Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Overall Regression Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Unique Contribution of Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Using SPSS Menus for Regression with Multiple Predictors . . . . . . . . . . . . . . . . . . . . . 14

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15


DATA LIST FREE / subj grde abil stdy wght.BEGIN DATA 1 83 112 22 133 2 75 113 16 128 3 70 103 21 154 4 71 106 26 189 5 48 67 27 136 6 81 103 18 157 7 48 95 20 138 8 62 73 27 153 9 61 121 14 137 10 58 85 22 12911 80 123 12 120 12 66 89 18 12113 81 114 25 167 14 61 71 29 16815 59 96 17 143 16 41 75 19 16617 69 112 20 146 18 63 108 13 11519 67 120 14 146 20 71 100 16 14421 54 88 20 154 22 72 107 18 15423 69 86 25 115 24 71 92 22 140END DATA.

Box 11.1. Study Data with Three Predictors.

Previous examples of multiple regression have involved designs with only two predictors,

so the essential question has been whether one, neither, or both predictors contribute to the

prediction of y, and if so, how to apportion the variability in y to the two predictors. The basic

methods that we have used also apply to studies with more than two variables, but the analytic

and conceptual problems increase. This chapter begins to examine these issues. First, let us see

how the methods developed in preceding chapters can be extended to multiple predictor studies.

Earlier we discussed a hypothetical study of the relation between grades, ability, and

study time. Equally-hypothetical critics of the study questioned the conclusion that study time

had a positive effect on grades

controlling for ability. The critics

argued that statistics can prove

anything and that the complicated

methods used by the researchers

would probably show that weight of

students predicts grades. To answer

this charge, the researchers obtained

data for 24 high school students on

grades, ability, study time, and

weight. The data are shown in Box 11.1 (two cases per row).

Once data is entered into SPSS, regression and supplementary analyses are informative

about the relationship between grades and all three predictors, and about the unique contribution

of each individual predictor. We previously saw that ability and study time contributed

significantly to grades (see also later analyses in this chapter), so our focus will be primarily on

whether weight adds anything new to the prediction. At the same time, we want to appreciate

how information about the overall effect of predictors can be generalized to three or more

predictors.


THE MR EQUATION WITH MULTIPLE PREDICTORS

Determining a number of statistics becomes much more complicated when we have more

than two predictors. The slopes, for example, must take into consideration the correlation

between grades and all three predictors, as well as the three correlations among the three

predictors (a total of six rs). We therefore obtain these slopes from the SPSS output rather than

compute them directly. Box 11.2 shows the SPSS commands to produce and analyze the three

predictor equation. The only difference from earlier SPSS commands is that we now list four

variables (i.e., one dependent variable and three predictors), rather than three (i.e., one dependent

variable and two predictors). The /ENTER command instructs SPSS to use all but the dependent

variable as predictors (i.e., the remaining three variables). We examine in turn the overall

relationship and then the unique contribution of each predictor (focussing on Weight, our new

variable). Box 11.2 also shows the first three cases with the derived scores, as well as the

REGRESS /VARI = grde abil stdy wght /DEP = grde /ENTER /SAVE PRED(prdg.asw) RESI(resg.asw).

Model R R Square Adjusted R Square Std. Error of the Estimate 1 .762(a) .580 .517 7.6198780

Model Sum of Squares df Mean Square F Sig. 1 Regression 1605.374 3 535.125 9.216 .000(a) Residual 1161.251 20 58.063 Total 2766.625 23

Coefficients(a) Unstandardized Coefficients Standardized Coefficients t Sig. Model B Std. Error Beta 1 (Constant) -12.456 20.550 -.606 .551 ABIL .677 .131 1.013 5.169 .000 STDY 1.389 .501 .601 2.773 .012 WGHT -.111 .099 -.186 -1.120 .276

Residuals Statistics(a) Minimum Maximum Mean Std. Deviation N Predicted Value 46.203117 80.817650 65.875000 8.3545723 24 Residual -16.244148 16.237690 .000000 7.1055728 24

LIST. SUBJ GRDE ABIL STDY WGHT PRDG.ASW RESG.ASW 1.0000 83.0000 112.000 22.0000 133.000 79.08295 3.91705 2.0000 75.0000 113.000 16.0000 128.000 71.97994 3.02006 3.0000 70.0000 103.000 21.0000 154.000 69.26472 .73528

...

Box 11.2. SPSS Commands and Output for 3-Predictor Regression Equation.


�g.asw = -12.455881 + .6767a + 1.3894s -.1114w

Mean Std Dev SSGrades 65.8750 10.968 2766.831 � SSTot

�g.asw 65.8750 8.3546 1605.385 � SSReg

y-�g.asw .0000 7.1056 1161.260 � SSRes

Source df SS MS FRegression 3 1605.37420 535.12473 9.21635 p = .0005Residual 20 1161.25080 58.06254Total 23 2766.625

Rg.asw = .76175 = ry.� R2g.asw = .58026 = 1605.3742/2766.625

FR = (.761752/3)/((1-.761752)/(24-3-1)) = 9.216 = FReg

Box 11.3. MR Analysis of Expanded Study Time Results.

original variables.

The predicted score for subject (case) one is y’ = -12.456 + .677x112 + 1.389x22 -

.111x133 = 79.163 �79.08295, and the residual score is y - y’ = 83.00 - 79.08295 = 3.91705.

Predicted scores for all subjects could be calculated by substituting each subject’s scores on the

predictor variables into the best-fit prediction equation. These scores contain a certain amount of

the variability in grades, namely that variability predictable from ability, study time, and weight.

The remaining variability in grades is contained in the residual scores.

The Overall Relationship

The Unstandardized Coefficients column shows the slopes for each of the three

predictors, controlling for the other two predictors included in the equation, as well as the

intercept (i.e., the Constant in SPSS terms). This equation can be used to generate predicted and

residual scores, as just shown, which are saved by SPSS. The SAVE also generates descriptive

statistics for the predicted and residual scores, which can be used to calculate SSRegression and

SSResidual.

The statistical results from the multiple regression of grades on the three predictors are

expanded in Box 11.3. The analysis produces a regression equation, now with three predictors.

The regression coefficients indicate that grades are positively related to ability and study time,

but somewhat negatively related to weight. These regression coefficients take into consideration

the relations among the threee predictors as well as the relations between predictors and the

criterion

variable,

grades.

The

equation

partitions

the total

variability in

grades

(SSGrades =


2766.625) into variability that can be predicted (SSRegression = SS� = 1605.374) and variability that

cannot be predicted (SSResidual = SSy-� = 1161.251) on the basis of Ability, Study Time, and Weight.

These quantities are calculated in Box 11.3 from the standard deviations of the Grade, Predicted,

and Residual scores. The three variables predict 58.026% of the variation in grades, R2 =

SSRegression/SSTotal.= 1605.3742/2766.625 = .5803. This gives Rg.aws= �.58026 = .76175, also shown

in the regression output in Box 11.2.

Box 11.4 shows descriptive statistics and correlations for the original and derived scores.

Note first in Box 11.4 that the descriptive statistics provide again the necessary information for

computing SSTotal, SSRegression, and SSResidual. Note also that the correlations between the residual

scores and all three predictors are 0. All the variability in Grades related to the predictors has

been captured by the regression equation and put in the Predicted Grade scores. As shown

previously, the predicted and residual scores also share zero variability. With respect to the

partitioning of the variability in Grades into predicted and residual components, the correlation

between Grades and PRDG.ASW is .762, our multiple R as calculated above.

The correlation between Grades and RESG.ASW represents the remaining (unpredicted)

variability in Grades, SQRT(1 - R2) = SQRT(1 - .7622) = .648. That is, 1 - R2 = 1 - .5803 = .4197

� .64792 = r2 between GRDE and RESG.ASW. Another way of considering what has happened

to the variability in Grades is to note

that .7622 + .6482 = 1.00 (i.e., all the

variability in Grades can be

accounted for by what can be

predicted and what cannot be

predicted). Correlational analysis of

original and derived scores has

shown that residual scores account

for the variability in y not predicted

by the 3-equation model, that

predicted and residual scores are

independent, and that residual scores are independent of the three predictors. These observations

CORR grde abil stdy wght prdg.asw resg.asw /STAT = DESCR. Mean Std. Deviation N GRDE 65.875000 10.9675906 24 ABIL 98.291667 16.4117645 24 STDY 20.041667 4.7409334 24 WGHT 143.875000 18.3286720 24 PRDG.ASW 65.8750000 8.35457229 24 RESG.ASW .0000000 7.10557284 24

Correlations GRDE ABIL STDY WGHT PRDG.ASW ABIL .646 STDY -.139 -.649 WGHT -.049 -.127 .442 PRDG.ASW .762 .849 -.182 -.064 RESG.ASW .648 .000 .000 .000 .000

Box 11.4. Descriptive Statistics and Correlations for

Original and Derived Scores.


about the overall regression are analogous to those for a two-predictor model (or for a 10-

predictor model).

The significance of the overall relationship between grades and the three predictors is

determined using analysis of variance with either SSs or R2. Given MSRegression = SSRegression / p =

1605.3742 / 3 = 535.1247 and MSResidual = SSResidual / (n - p - 1) = 1161.2508 / (24 - 3 -1) = 58.0625,

F = MSRegression / MSResidual = 535.1247 / 58.06254 = 9.21635 = (R2 / p) / ((1 - R2) / (n - p - 1)) =

(.761752 / 3) / ((1 - .761752) / (24 - 3 - 1)), df = 3, 20. The results confirm that this strong relation

is significant, F = 9.216, p = .0005, and we can reject H0: �g.asw = 0.

Significance of Individual Predictors

As for two-predictor regression, the unique contributions of individual predictors can be

assessed in various ways. We focus on the statistics for weight, the new variable, but the general

points also apply to ability and study time. The Coefficients section of Box 11.2 shows several

REGRESS /VARI = grde abil stdy wght /STAT = DEFAULT ZPP CHANGE /DEP = grde /ENTER abil stdy /ENTER.

Model Summary R R Adjusted Std. Error of Change Statistics Square R Square the Estimate Model R Square Change F Change df1 df2 Sig. F Change 1 .744(a) .554 .511 7.6660706 .554 13.038 2 21 .000 2 .762(b) .580 .517 7.6198780 .026 1.255 1 20 .276


2 Regression 1605.374 3 535.125 9.216 .000(b) Residual 1161.251 20 58.063 Total 2766.625 23

Coefficients(a) Unstand Coeff Standard Coeff t Sig. Correlations Model B Std. Error Beta Zero-order Partial Part 1 (Constant) -19.737 19.613 -1.006 .326 ABIL .642 .128 .961 5.017 .000 .646 .738 .731 STDY 1.122 .443 .485 2.532 .019 -.139 .484 .369 WGHT 2 (Constant) -12.456 20.550 -.606 .551 ABIL .677 .131 1.013 5.169 .000 .646 .756 .749 STDY 1.389 .501 .601 2.773 .012 -.139 .527 .402 WGHT -.111 .099 -.186 -1.120 .276 -.049 -.243 -.162

Excluded Variables(b) Beta In t Sig. Partial Collinearity Statistics Correlation Model Tolerance 1 WGHT -.186(a) -1.120 .276 -.243 .760

Box 11.5. Unique Contribution of Weight to Three-Predictor Equation.


measures of the contribution of weight to the prediction equation; these also appear in the output

for Model 2 in Box 11.5. The REGRESSION in Box 11.5 has added Weight separately to the

regression, and includes the CHANGE and ZPP options.

The significance of the slope for weight can be determined with the SE of the slope, which

is calculated using the following formula: SEw.as3 = SQRT(MSE / ((1 - R2w.as) x SSw)). MSE =

58.063 from the ANOVA for Model 2 in Box 11.5. Using sWeight from Box 11.4, SSw = (24 -

1)18.3286722 = 7726.625. A regression with Weight as the dependent variable and Ability and

Study Time as predictors determined that R2w.as = .2395. This gives a value of .099 as SE for

weight with all predictors in the equation (i.e., Model 2). The t-test for weight when ability and

study time are also in the equation is not significant, tgw.as = -.111/.099 = -1.12, p = .2758,

indicating that weight makes no additional contribution to the prediction of grades over and above

what Ability and Study Time can predict. Box 11.5 shows this same statistic, as well as a more

complete regression analysis that focusses on the unique contribution of Weight.

Conceptually, the t-test on the slope for Weight is testing the significance of the unique

contribution of Weight over and above the variability in Grades already predicted by Ability and

Study Time. This is calculated as SSChange; specifically, SSg’w.as = SSg’.was - SSg’.as = 1605.374 -

1532.484 = 72.89. The significance of this unique contribution is F = (72.89/1)/58.063 = 1.255,

which appears as FChange for Model 2 in the regression output. As well, F = 1.1202 = t2,

demonstrating the equivalence between the F test for the change when Weight was added to the

other predictors, and the t test for the slope for Weight when all predictors were in the equation.

The F would be tested for significance with dfNumerator = 1 and dfDenominator = 20, and the t with df =

20. If critical values for F and t were obtained, FAlpha = t2Alpha (assuming a non-directional or two-

tailed test was conducted for t). Both FChange and the t-test for the slope have a significance of

.276, indicating that the contribution of Weight is not significant when Aptitude and Study time

are controlled statistically.

Strength of Unique Contribution

Researchers are interested not only in the significance of the unique contribution, but also

in the strength. The part r2 is the proportion of total variability in Grades predicted uniquely by

Weight; hence, r2g(w.as) = 72.89 / 2766.625 = .026, or equivalently, = R2

g.was - R2

g.as = .580 - .554 =


.026. This value appears as R2Change for Model 2 in the Change section of the printout. In addition,

rg(w.as) = SQRT(.026) = .162, which appears in the Part column for Weight in the Coefficients

section of the output. The part r has a negative sign because the slope for Weight is negative in

the multiple regression equation with all three predictors.

The part r can also be computed using residual scores, as shown in Box 11.6. Two

regression are shown (in part). In the first regression, Weight is the dependent variable and

Ability and Weight are the predictors. Residual Weight scores are saved as resw.as and then

REGRESS /VARI = wght abil stdy /DEP = wght /ENTER /SAVE RESI(resw.as).



...


CORR resw.as WITH grde abil stdy. GRDE ABIL STDY RESW.AS Pearson Correlation -.162 .000 .000

REGRESS /VARI = grde abil stdy /DEP = grde /ENTER /SAVE RESI(resg.as).



...Residuals Statistics(a) Minimum Maximum Mean Std. Deviation N Predicted Value 49.747364 81.526260 65.875000 8.1627031 24 Residual -15.714190 14.391576 .000000 7.3251841 24

CORR resg.as WITH resw.as abil stdy. RESW.AS ABIL STDY RESG.AS Pearson Correlation -.243 .000 .000

Box 11.6. Part and Partial rs from Residual Scores.


correlated with GRDE, ABIL, and STDY. As expected, resw.as correlates 0 with both Ability

and Study time; that is, the variability in resw.as represents the variability in Weight that is

completely independent of the other two predictors. The correlation between resw.as and Grades

is our part correlation. The unique variation in weight (i.e., the variation in weight independent of

the other two predictors) predicts -.1622 = .026 of the total variability in Grades, as previously

shown.

The partial correlation is another statistic that describes the unique contribution of each

predictor. The partial correlation represents how much is uniquely predicted by Weight of the

variability in Grades not already predicted by Ability and Study Time. Ability and Study Time

account for 1532.484 units of variability in Grades, leaving 1234.141 units as the residual. This

becomes the denominator for the partial r2; that is, r2gw.as = 72.89 / 1234.141 = .059, and rgw.as =

SQRT(.059) = .243, which appears as -.243 in the Partial column of the three-predictor equation.

The partial correlation can also be computed using residual scores, and the relevant

analyses are shown in the bottom half of Box 11.6. The dependent variable Grades is regressed

on Ability and Study Time, and a residual Grade score saved as resg.as. This residual score has

1234.141 units of variability, the same as the denominator in the preceding calculation of the

partial. The correlation between resw.as and resg.as is the partial correlation. That is, Weight

can uniquely predict -.2432 (or 5.9%) of the 1234.141 units of variability in Grades not already

explained by Ability and Study Time. Although still modest, 5.9% is considerably larger than

2.6% because Ability and Study Time together account for over half of the variability in Grades.

That is, the denominator for the partial r2 is much smaller than the denominator for the part r2.

A FOUR PREDICTOR EXAMPLE

Regression analyses with numerous predictors operate on the same principles. The

significance and part correlations for one variable in a 6-predictor model represent the

contributions of that variable over the effects of the other 5 variables. That is, the tests reflect the

differences in R2 and SSRegression between a 6-predictor regression with that variable and a 5-

predictor regression without that variable.


WORD RT AGE CON FRQ LEN WORD RT AGE CON FRQ LEN

1 800 5 1 56 8 21 1020 12 5 37 11 2 1213 10 7 67 10 22 982 7 4 60 5 3 1052 7 7 47 10 23 1150 14 6 58 13 4 1103 11 7 37 10 24 889 8 4 55 6 5 1016 5 6 52 4 25 1071 7 4 59 8 6 1020 5 5 21 6 26 896 7 6 27 9 7 874 6 7 47 5 27 1041 8 6 53 8 8 891 6 4 44 9 28 1104 9 5 35 8 9 944 9 6 37 9 29 910 7 3 36 810 1060 6 3 28 11 30 906 7 4 47 1011 1163 12 3 81 8 31 1098 7 4 63 1012 874 8 5 61 8 32 1001 7 4 43 913 1148 11 7 68 9 33 889 9 7 60 614 951 8 4 36 5 34 1060 11 4 68 1015 889 5 2 47 9 35 1119 12 3 110 1016 1064 8 4 48 9 36 979 10 6 68 917 1119 12 6 75 7 37 989 7 6 64 518 770 8 2 62 8 38 1113 9 6 50 819 1034 6 2 59 8 39 708 4 3 25 820 1054 12 3 53 9 40 987 12 9 79 5

Box 11.7. Raw Data for Naming Study.

We illustrate these points for a 4-predictor regression problem that involves picture

naming reaction times (RT) obtained for 40 words, along with measures of Concreteness (CON),

Length (LEN), Frequency in print (FRQ), and rated Age of acquisition (AGE). The data are

shown in Box 11.7. Note that

for the CON variable low

scores indicate concrete words

and high scores abstract

words, and that for the FRQ

variable, low scores indicate

more frequent words and high

scores less frequently

occurring words. These scales

were reversed so that all

variables would be expected to

have positive relationships with RT. That is, naming would be slower for words that were

abstract, were long, appeared infrequently in print, and were learned at a later age. The following

analyses focus on the Age predictor.

The SPSS regression commands and

preliminary output appear in Box 11.8. The

dependent variable and all four predictors

appear on the VARIABLES line and age is

entered after the other three predictors because

that will be our primary focus in the following

discussion. We have also requested ZPP and

CHANGE, as well as the DEFAULT statistics.

All four predictors correlate positively with

RT, as expected (recall that CON and FRQ are

reversed). The correlation matrix also shows,

however, that the predictors are correlated with

REGRESS /VARI = rt age con frq len /DESCR /STAT = DEFAU ZPP CHANGE /DEP = rt /ENTER con frq len /ENTER age /SAVE PRED(pr.acfl) RESI(rr.acfl).

Descriptive Statistics Mean Std. Deviation N RT 998.775000 114.0576749 40 AGE 8.350000 2.4966644 40 CON 4.750000 1.7795130 40 FRQ 53.075000 17.3772434 40 LEN 8.200000 1.9767884 40

RT AGE CON FRQ AGE .585 CON .318 .355 FRQ .349 .524 .055 LEN .323 .344 -.160 -.030

Box 11.8. Regression Command and

Preliminary Output for Four Predictors.


one another to a considerable degree. In particular, age correlates with all three of the other

predictors.

Box 11.9 shows the remaining printout from the regression in Box 11.8. In the final

regression equation, the overall contribution of all four predictors is both highly significant and a

strong effect, F = 6.011, p = .001, R2 = .407. With respect to the individual predictors, however,

the unique contribution is not significant for any of the predictors, ps > .10, and the part

correlations are all modest, part rs < .22. This seemingly anomalous outcome is explained later.

Overall Regression Results

The best-fit regression equation with all four predictors is: y = 622.601 + 15.073C +

1.147F + 14.700L + 14.751A. The predicted scores based on this equation (not shown) account

Model Summary(c) R R Adjusted Std. Error of Change Statistics Square R Square the Estimate Model R Square Change F Change df1 df2 Sig. F Change 1 .601(a) .361 .308 94.9137441 .361 6.773 3 36 .001 2 .638(b) .407 .339 92.6989896 .046 2.741 1 35 .107



Coefficients(a) Unstand Coeff Stand Coeff t Sig. Correlations Model B Std. Error Beta Zero-order Partial Part 1 (Constant) 584.562 94.505 6.186 .000 CON 23.240 8.664 .363 2.682 .011 .318 .408 .357 FRQ 2.237 .876 .341 2.554 .015 .349 .392 .340 LEN 22.570 7.791 .391 2.897 .006 .323 .435 .386 AGE

2 (Constant) 622.601 95.116 6.546 .000 CON 15.073 9.795 .235 1.539 .133 .318 .252 .200 FRQ 1.147 1.080 .175 1.062 .296 .349 .177 .138 LEN 14.700 8.972 .255 1.638 .110 .323 .267 .213 AGE 14.751 8.910 .323 1.656 .107 .585 .269 .215

Excluded Variables(b) Beta In t Sig. Partial Collinearity Statistics Correlation Model Tolerance 1 AGE .323(a) 1.656 .107 .269 .445


Box 11.9. Regression Results for Four-Predictor Regression from Box 11.8.


for (40 - 1)72.78322 = 206598.382 units of variability in RT and fail to account for (40 -

1)87.816622 = 300758.593 units of variability in RT. SSRegression + SSResidual = 507356.975 = SSRT.

The predicted variability represents 40.72% of the total variability in RT; that is, R2r.cfla =

206598.382 / 507356.975 = .4072, and Rr.cfla = SQRT(.4072) = .638. These values appear in the

model summary section. We cannot predict 59.28% of the variability in RT; that is, 1 - R2 = 1 -

.4072 = .5928, and SQRT(.5928) = .770.

Box 11.10 shows the

descriptive statistics and correlations

for the original and derived scores.

SSRegression, SSResidual, and SSTotal could

be obtained from these statistics. The

correlation between RT and the

predicted scores (pr.acfl) = .638 =

Rr.acfl. The correlation between RT

and the residual scores (rr.acfl ) =

.770 = SQRT(1 - R2r.acfl). The

correlation matrix also shows that the residual scores correlate 0 with all four predictors and with

the predicted scores themselves. These relationships have been seen many times before.

The significance of the overall regression is tested using Analysis of Variance, and the

relevant results are shown in Box 11.9. SSRegression is divided by 4, its df = p, to give MSRegression and

SSResidual is divided by 35, its df = n - p - 1, to give MSResidual. F = MSRegression / MSResidual = 6.011,

which is highly significant. F could also be calculated using our formula based on R2, and the

results would be identical, except perhaps for rounding. We can clearly reject the null hypothesis

of no relationship in the population between RT and the collective effects of all four predictors.

Adjusted R2 and Standard Error of the Estimate. Two entries in the Box 11.9 regression

output have not previously been explained. The Adjusted R2 adjusts for chance relations between

the predictors and y; this is particularly important when multiple predictors are used. There are

several ways to think about why this adjustment is necessary. First, remember that multiple

regression minimizes SSResidual (i.e., maximizes SSRegression) for this sample of data. It is expected

CORR rt pr.acfl rr.acfl age con frq len /STAT = DESCR.

Mean Std. Deviation N RT 998.775000 114.0576749 40 PR.ACFL 998.7750000 72.78320138 40 RR.ACFL .0000000 87.81662030 40 AGE 8.350000 2.4966644 40 CON 4.750000 1.7795130 40 FRQ 53.075000 17.3772434 40 LEN 8.200000 1.9767884 40

RT PR.ACFL RR.ACFL AGE CON FRQ LEN PR.ACFL .638 1 .000 .917 .499 .547 .506 RR.ACFL .770 .000 1 .000 .000 .000 .000 AGE .585 .917 .000 1 .355 .524 .344 CON .318 .499 .000 .355 1 .055 -.160 FRQ .349 .547 .000 .524 .055 1 -.030 LEN .323 .506 .000 .344 -.160 -.030 1

Box 11.10. Correlation of Original and Derived Scores.


that the relationship in the population will not benefit from this minimization to a particular set of

sample data, and hence R2 in the population is expected to be somewhat smaller than that

computed for the sample. Second, multiple regression is so powerful that it can capitalize on

patterns in the data that essentially are due to random covariation, and multiple regression will be

particularly sensitive to this chance variation when n is small or p is large. The formula for

R2Adjusted = 1 - (1 - R2) × ((n - 1)/(n - p - 1)) = 1 - (1 - .407) x ((40 - 1) / (40 - 4 - 1) = 1 - .593 x

1.114 = .339, as shown in Box 11.9. R2 adjusted is always smaller than R2, and the most dramatic

reductions occur when n is small and p is large because the magnitude of the adjustment

(reduction) depends on the sample size n and the number of predictors p.

The other statistic that we have not discussed is the Standard Error for the overall

regression results. This value is simply the square root of MSResidual (i.e., �8593.103 = 92.699 �

Standard Error). If one wanted to report a standard deviation associated with the error or residual

variation, this would be the appropriate statistic to use.

Unique Contribution of Predictors

The story with respect to individual predictors appears to be complex. As noted

previously, none of the predictors is significant in the four-predictor equation, despite the high

level of overall significance. Briefly (more extensively in the next chapter), this occurs because

what the predictors can explain is due to their shared variability with one another, and not from

anything unique to any of the predictors. Let us examine more closely the significance of Age,

although any of the predictors could be examined in this manner.

The significance of the unique contribution of Age can be calculated from a t-test on the

slope, t = 14.751 / 8.910 = 1.656, p = .107, as shown on the Age line of the Coefficient section of

the printout. With df = 35, this difference would be marginally significant using a one-tailed test,

that is, p = .107/2 = .0535. We could also perform this test as an F test. SSPredicted increased by

SSr’.acfl - SSr’.clf = 206598.382 - 183046.697 = 23551.685, seemingly a large amount except that

SSTotal is a very large number. F = (23551.685/1) / 8593.103 = 2.741 = 1.6562. This F and its

corresponding p = .107 appear as FChange and Significance of F Change for Model 2 in the Change

section of the printout. Despite the overall relationship being highly significant, Age (and the

other predictors) do not uniquely predict significant amounts of variability in RT.


Figure 11.1. Using Menu to Obtain Unique Contribution

with Multiple Predictors.

There are several measures of the

relative strength of the unique contribution

of Age, including the part correlation,

r2r(a.cfl) = 23551.685 / 507356.975 = .046 =

R2r’.acfl - R

2r’.cfl. This part r2 appears in the

R2 Change section of the printout. The part

r is the square root of this value; that is,

rr(a.cfl) = .215, which appears in the Part

column for the Age predictor. Note that

the part r is considerably smaller than the simple r between RT and Age, rr.a = .585. The part r for

Age can also be obtained by computing a residual Age score that is uncorrelated with the other

predictors. Box 11.11 shows the procedure. Other measures of strength shown in Box 11.9

include the partial r of .269 and the standardized slope of .323.

Using SPSS Menus for Regression with Multiple Predictors

Regression analysis with

multiple predictors can be done by

SPSS either using the syntax

approach, as emphasized in this

chapter, or using the menu system.

To obtain the unique contribution

of Age, for example, first enter a

block of predictors that includes

Concreteness, Frequency, and

Length (see Figure 11.1 for this

first step). Then click on Next to

initiate a new block of predictors.

This will bring up the screen

shown in Figure 11.2. Age can

then be selected as an additional predictor to add to those already in the equation.

REGRE /VARI = age con frq len /DEP = age /ENTER /SAVE RESI(ra.cfl)....CORR ra.cfl WITH rt age con frq len /STAT = DESCR.

Mean Std. Deviation N RA.CFL .0000000 1.66588503 40 AGE 8.350000 2.4966644 40 ... RT AGE CON FRQ LEN RA.CFL .215 .667 .000 .000 .000

Box 11.11. Part r for Age by Residual Scores.


Figure 11.2. Block 2 and Statistics Options.

Once Age as been selected,

as shown in bottom menu screen in

Figure 11.2, select Statistics to

request Change and Part and Partial

rs, as shown in top menu screen in

Figure 11.2. Then select Continue

and OK to run the analysis.

The syntax generated by

SPSS is shown in Box 11.12.

Although somewhat more detailed

than syntax commands we have

entered because defaults are

specified, Box 11.12 shows essential

commonalities with

commands used earlier. If you

do use the menu system for

regression analyses, remember

to save the syntax commands

produced by SPSS in a syntax

file in case the analysis needs

to be redone.

CONCLUSIONS

This chapter has shown that our approach to the overall regression results with two

predictors generalizes directly to multiple regression with three or more predictors. Moreover,

tthe unique contribution of predictors is also quite general and can be extended to three, four, or

more predictors. SSChange for the last predictor entered into the equation is central to one approach.

This allows us to calculate FChange and R2Change statistics relevant to the significance and strength of

the unique contribution of each predictor. A second approach to part and partial correlation

REGRESSION /MISSING LISTWISE /STATISTICS COEFF OUTS R ANOVA CHANGE ZPP /CRITERIA=PIN(.05) POUT(.10) /NOORIGIN /DEPENDENT RT /METHOD=ENTER CON FRQ LEN /METHOD=ENTER AGE .

Box 11.12. Syntax Generated by Menu Commands.


R2Adj'1&(1&R2)× n&1

n&p&1

Equation 11.1. Formula to Calculate R2 Adjusted.

SE1.23...p

'

MSResidual

SS1(1 &R2

1.23...p)

Equation 11.3. Formula for SE Slope with More than TwoPredictors.

SSy1.23 ÿp

'SSy.123 ÿp

&SSy.23 ÿp

R2y1.23 ÿp

'

SSy1.23 ÿp

SSTotal

'R2y.123 ÿp

&R2y.23 ÿp

Equation 11.2. Equations for SSChange and R2Change for Multiple Predictor Equations.

strength of the unique contribution of each predictor. A second approach to part and partial

correlation computes a residual predictor score that is independent of the other predictors in the

equation and, for the partial correlation, a residual dependent score that reflects variability in the

criterion variable that is independent of other predictors in the equation.

Equation 11.1 shows the formula for R2 Adjusted, which provides a more accurate

estimate of R2 in the population than does the unadjusted R2. The unadjusted R2 capitalizes on

chance relationships in the data and over-estimates the magnitude of the population R2.

Equation 11.2 summarizes some of the extensions for the unique contribution of a

predictor, although a conceptual understanding is ultimately more important than formula at this

stage of our understanding of multiple regression.

Equation 11.3 shows the

slightly modified formula for the

standard error of the regression

coefficients. Now the standard

error is based on the variability in

each predictor that is independent of all of the other predictors in the equation. To get R21.234..., we

would need to run the appropriate multiple regression analysis (e.g., regress x1 on predictors x2 to

xp). Note that, other things being equal, SE becomes smaller as the unique variation in SS

increases, and SE becomes larger as the unique variation in SS decreases. This means that for the

purpose of testing the unique contribution of predictors, it is desirable to have more unique

variability in the predictors, rather than less.

SPSS and Multiple Predictors 12.1

CHAPTER 12

MORE ON REGRESSION ANALYSES WITH MULTIPLE PREDICTORS

Automated Strategies for Variable Selection in MR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Forward Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Backward Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Stepwise Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Examples of Automated Regression-Building Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

MR for the Extended Study Time Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Automated Procedures using SPSS Menus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Word Attributes and Naming Latency Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Interpretation of Unique Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

SPSS Regression Options and Menus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Complexities in Regression Model Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Redundant Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Coincident Outcomes as Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Suppressor Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Testing All Possible Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25


This chapter examines additional aspects of multiple regression for designs with more

than two predictors. Following a brief introduction, we examine several automated ways to enter

predictors into a multiple regression analysis, including the SPSS commands to perform the

various selection methods.

The preceding analyses and those in earlier chapters have demonstrated that MR assesses

the independent contribution of predictors beyond what other predictors already explain; does x1

add anything to what x2 to xp can predict? Equivalently, does the residual of x1, after being

regressed on x2 to xp, contribute anything unique to the prediction of y? One implication of this

approach is that the predictors in a multiple predictor design compete with one another for

significance. Two correlated variables that account only for the same variation in y could both be

not significant because of their redundancy with the other predictor. Alternatively, a predictor

might only be significant when some other predictor or set of predictors is also in the equation

because its contribution is masked by off-setting influences.

Because of these subtleties, multiple predictors often forces researchers to decide which

variables, if any, should be included in the regression equation. Depending on the choice of

predictors, it is possible that some variables will not contribute uniquely to the overall regression,

either because they are not related to the criterion variable, because they are redundant with other

variables in the set, or because a critical covariate has not been entered as an additional predictor.

Because of the complex ways in which combinations of variables influence one another,

selection of the "best" set of MR predictors is a challenging problem.

Selection of predictors for an MR model is further complicated by the fact that the

number of different combinations of predictors increases dramatically as the number of

predictors increases. With four predictors (1 to 4), there are 16 (24 = 2 × 2 × 2 × 2) possible

combinations of the four predictors: 1 set of no predictors, 4 singles (1, 2, 3, 4), 6 pairs (12, 13,

14, 23, 24, 34), 4 triplets (123, 124, 134, 234), and 1 quartet (1234). With 6 predictors, there are

26 = 64 combinations: 1 empty set, 6 singles, 15 pairs, 20 triplets, 15 quartets, 6 pentads, and 1

hexad. In general, the number of combinations of predictors is 2p, where p is the number of

predictors. Choosing the "best" of these 2p possibilities is not easy when more than a few

predictors are involved, as is the case for the increasingly complex MR studies used in


contemporary research.

AUTOMATED STRATEGIES FOR VARIABLE SELECTION IN MR

A variety of strategies have been developed to assist with the selection problem, and

some strategies have been implemented in computer packages such as SPSS. Several approaches

are introduced here and illustrated for a few designs in later parts of the chapter. Readers should

be forewarned, however, that blind statistical approaches to predictor selection in MR have many

limitations and dangers. To avoid these potential pitfalls, researchers need to understand the

fundamental nature of multiple regression, one objective of this book. Even more importantly,

researchers always require sound theoretical and empirical knowledge of the domain being

studied, and should use that knowledge when developing regression models for complex

phenomena. This more sophisticated and reasoned type of model building is presently beyond

the capabilities of basic multiple regression, as we have been doing with SPSS. With these

caveats in ming, the three automated methods that we consider are Forward, Backward, and

Stepwise selection.

Forward Selection

The Forward selection method selects the predictor at each stage that (a) adds the most to

the prediction of the criterion, and (b) reaches some specified probability level (called PIN in

SPSS to enter the equation (by default, PIN = .05). Selection proceeds at each step of the

regression until no more variables reach the probability level specified for entry into the equation.

At step 1, the predictor most strongly related to y would be entered in the equation if its

probability was less than the entry p value; let us say this is predictor x1. After x1 is entered,

each of the remaining predictors is re-analyzed for its contribution to the equation over and above

x1 (i.e., ty2.1, ty3.1, ..., typ.1). The strongest of these predictors is entered as long as its probability is

less than the critical p value to enter the equation; suppose this predictor is x2. The contribution

of each remaining variable when included with x1 and x2 is evaluated (i.e., ty3.12, ty4.12, ..., typ.12)

and the process continues until none of the variables have a low enough probability to enter the

equation or until all predictors are in the regression equation. Note in particular about this brief

description that the probability associated with the unique contribution of a predictor must be


recalculated anew at each stage. This is because the probability being considered is the

probability associated with each predictor if it alone was added to predictors aleady in the

equation.

The preceding paragraph referred deliberately to "probability to enter" rather than

significance level. Significance refers to the probability of a Type I error and MR selection

procedures complicate determining the "true" probability of a Type I error (i.e., �). The

probability normally specified as � assumes that researchers are performing one test of

significance. But MR calculates probabilities for multiple predictors and then selects the variable

with the lowest probability. The probability that one of p predictors falls in the rejection region

is greater than the nominal probability used for each of the p statistical tests. The actual

probability of rejecting a true H0 can therefore be much larger than the nominal probability level

when multiple predictors are involved, as in the Forward procedure and in other statistical

procedures for selecting predictors. This is one of a number of problems with automatic

regression-building procedures.

Backward Selection

An alternative to Forward selection is Backward selection, which removes variables from

the equation rather than adding them. Backward selection begins with all predictors in the

equation (i.e., R2y.12...p). The significance level for each predictor is determined with all of the

other predictors included (e.g., ty1.2...p, ty2.1...p, typ.1...p-1). A predictor is then removed if it (a) adds

least to the prediction of the criterion, and (b) fails to reach some specified probability level

(called POUT in SPSS) to remain in the equation (by default, POUT = .10, unless some user-

specified value is provided). This variable (e.g., x1) is removed from the equation and a

probability is calculated for the contribution of each predictor without x1 in the equation (e.g.,

ty2.3...p, ...). If the probability level for the weakest predictor is greater than the probability to

remain, then that predictor is removed. The procedure continues until the remaining variables all

have probabilities lower than the probability to remain or all variables have been removed. As in

the Forward procedure, probabilities must be recalculated anew at each stage of the analysis.

Forward and Backward selection procedures do NOT necessarily result in the same

equation, although they might, depending on the relations among the predictors and between the


predictors and the criterion variable. One factor leading to differences is that SPSS uses different

default values for Entering and Removing predictors; more on this shortly. A more fundamental

factor is that the significance of a predictor at each stage depends on the other predictors in the

equation (i.e., the context), which can differ for Forward and Backward analyses. The

reservations stated earlier about interpreting probabilities as significance levels apply to the

Backward procedure as well as to the Forward procedure.

Stepwise Selection

A third approach to variable selection is essentially a combination of the Forward and

Backward procedures and is called Stepwise selection. At each step, Stepwise selection involves

a Forward selection of the predictor that has the lowest probability at that point and that

surpasses some specified probability level (PIN) to enter the equation. The procedure then

examines the variables currently in the equation to determine if one or more should be removed

because their probability levels have fallen below the value specified for remaining in the

equation (POUT). Variables that no longer meet the criterion to remain in the equation are

successively removed. The next variable for entry is then selected if its probability is less than

the probability value to enter the equation. All predictors in the equation are again reevaluated

for removal, and so on until no more predictors can be entered or removed.

In Stepwise selection, the probability value to enter the equation (e.g., default PIN = .05)

must be less than the probability to remain in the equation (e.g., default POUT = .10); that is, the

probability to remove a predictor will be greater than the probability to enter a predictor. If the

probability to remain were equal to or lower than the probability to enter, then predictors could

be entered and removed from the equation in an endless, repeating cycle. The Stepwise method

is prone to the same misuse of probabilities to represent significance levels as are the Forward

and Backward selection procedures. In fact the significance problem may be even more serious

with stepwise regression.

Forward, Backward, and Stepwise methods illustrate the general statistical considerations

involved in selection of predictors in an MR study with multiple predictors. Similar statistical

considerations underlie other selection procedures. With a modest number of predictors, for

example, it is practical to perform regression for all-possible prediction equations, but some


criteria (e.g., R2Change) must then be used by the researcher to choose the best of the resulting

equations. Because the various selection methods are all calculation-intensive, the procedures

are illustrated only using SPSS, although in principle simple examples could be done

“manually.”

EXAMPLES OF AUTOMATED REGRESSION-BUILDING PROCEDURES

This section illustrates the automated procedures using SPSS and sample problems that

we have discussed in previous chapters. Refer to those earlier chapters for details about the

studies and for fuller explanation of the basic analyses. Here we concentrate on the regression-

building procedures discussed in the preceding section.

Note that automated procedures require that all predictors be specified for the regression

analysis at some point, and not simply introduced as each stage of the equation is constructed.

That is, researchers must use the /VARIABLES = option or list multiple predictors when

specifying the method being used (e.g., /METHOD = FORWARD x1 x2 x3 ...). The automated

procedures select from among some set of predictors and need to know what variables are to be

included in the candidate set.

MR for the Extended Study Time Study

We earlier examined a study in which grades (GRDE) for 24 students were regressed on

study time (STDY), ability (ABIL), and weight (WGHT). The following analyses do not provide

much additional information beyond what has already reported for this study. The difference is

that instead of researchers specifying what variables are entered (with the ENTER keyword),

SPSS automatically enters or removes predictors according to criteria specified by the user.

Box 12.1 shows a FORWARD regression for the extended Study Time data. The

researcher specifies /METHOD = FORWARD (instead of /ENTER varnames; here the user could

have simply entered /FORWARD) and SPSS proceeds to enter and remove variables according

to default probability values for entering (PIN = .05) and removing (POUT = .10) the predictors.

These default values for entering and removing predictors are reported early in the output (the

first line in our edited version).

SPSS selects the predictor with the lowest probability that is less than PIN (i.e., the most


REGRESS /VARI = grde abil stdy wght /DESCR /STAT = DEFAU CHANGE ZPP /DEP = grde /METHOD = FORWARD.

grde abil stdy wght abil .646 1.000 -.649 -.127 stdy -.139 -.649 1.000 .442 wght -.049 -.127 .442 1.000

Variables Entered/Removed(a) Model Variables Variables Method Entered Removed 1 abil . Forward (Criterion: Probability-of- F-to-enter <= .050) 2 stdy . Forward (Criterion: Probability-of- F-to-enter <= .050)

Model R R Adjusted Std. Error of Change Statistics Square R Square the Estimate R Square Change F Change df1 df2 Sig. F Change 1 .646(a) .418 .391 8.5567541 .418 15.786 1 22 .001 2 .744(b) .554 .511 7.6660706 .136 6.409 1 21 .019

a Predictors: (Constant), abilb Predictors: (Constant), abil, stdy



a Predictors: (Constant), abilb Predictors: (Constant), abil, stdy

Coefficients(a) Model Unstandardized Standardized t Sig. Correlations Coefficients Coefficients B Std. Error Beta Zero-order Partial Part 1 (Constant) 23.419 10.828 2.163 .042 abil .432 .109 .646 3.973 .001 .646 .646 .646

2 (Constant) -19.737 19.613 -1.006 .326 abil .642 .128 .961 5.017 .000 .646 .738 .731 stdy 1.122 .443 .485 2.532 .019 -.139 .484 .369

Excluded Variables(c) Model Beta In t Sig. Partial Collinearity Correlation Statistics Tolerance 1 stdy .485(a) 2.532 .019 .484 .579 wght .033(a) .199 .844 .043 .984

2 wght -.186(b) -1.120 .276 -.243 .760

Box 12.1. Forward Regression of Study Time Study.

"significant" predictor) to enter first. As shown by the correlation matrix, the ability variable has

the strongest simple relationship with grades. It is therefore selected to enter at Step 1; its

probability is .0006, which is less than .05 (PIN) and is also lower than the probabilities for

Study Time and Weight, the other predictors being considered. Statistics are also computed for

both STDY and WGHT as second predictors in addition to Ability, and these statistics are

printed in the "Excluded Variables" section for Model 1. In essence, SPSS is reporting here the p

value for study time if added to ability (.019), and for weight if added to ability (.844) .


REGRESS /VARI = grde abil stdy wght /CRITERIA PIN (.30) POUT (.40) /DEP = grde /METHOD = STEP.

Model Variables Variables Method Entered Removed 1 abil . Stepwise (Criteria: Probability-of-F-to-enter <=.300, Probability-of-F-to-remove >=.400). 2 stdy 3 wght

Model Summary Model R R Square Adjusted R Std. Error of Square the Estimate 1 .646(a) .418 .391 8.5567541 2 .744(b) .554 .511 7.6660706 3 .762(c) .580 .517 7.6198780a Predictors: (Constant), abilb Predictors: (Constant), abil, stdyc Predictors: (Constant), abil, stdy, wght

ANOVA(d) Model Sum of Squares df Mean Square F Sig. 1 Regression 1155.828 1 1155.828 15.786 .001(a) Residual 1610.797 22 73.218 Total 2766.625 23


3 Regression 1605.374 3 535.125 9.216 .000(c) Residual 1161.251 20 58.063 Total 2766.625 23

Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 23.419 10.828 2.163 .042 abil .432 .109 .646 3.973 .001

2 (Constant) -19.737 19.613 -1.006 .326 abil .642 .128 .961 5.017 .000 stdy 1.122 .443 .485 2.532 .019

3 (Constant) -12.456 20.550 -.606 .551 abil .677 .131 1.013 5.169 .000 stdy 1.389 .501 .601 2.773 .012 wght -.111 .099 -.186 -1.120 .276

Excluded Variables(c) Model Beta In t Sig. Partial Collinearity Correlation Statistics Tolerance 1 stdy .485(a) 2.532 .019 .484 .579 wght .033(a) .199 .844 .043 .984

2 wght -.186(b) -1.120 .276 -.243 .760

Box 12.2. Stepwise Regression with PIN=.3 and POUT=.4.

The probability for STDY, when added to Ability, is .019, which is less than PIN and less

than .844, the probability for Weight when added to Ability. Therefore, SPSS enters STDY into

the prediction equation at step 2. After entering Study Time, SPSS computes statistics for the

remaining variable (or variables if p > 3); the statistics are reported in the Excluded Variables

section for Model 2. Although Weight now has a much lower p value than before, .276 vs. .844,

it is still not less

than .05, the

criterion that

SPSS uses to

admit new

predictors. SPSS

stops at the two

predictor

equation.

SPSS has

selected the two-

predictor

equation as the

most suitable

model for this

particular study.

Almost any

selection method

and researchers

themselves are

likely to agree

that Weight is not

a valid predictor

of Grades and


that the model including only Ability and Study Time is the most theoretically useful. It is

critical to remember, however, that judgments about theoretical meaningfulness were not part of

the statistical selection procedure, and that there is no necessary correspondence between

statistical and theoretical results. The ease of Stepwise and other selection procedures sometimes

blinds researchers to the fact that such methods ignore conceptual meaningfulness, an essential

component of model and theory building.

Box 12.2 shows a Stepwise analysis of the data, but now an additional command is

included to modify the default PIN and POUT values. The CRITERIA subcommand permits

researchers to control the criteria used to enter and remove variables from the equation. There are

several criteria available (e.g., F to enter or remove), but we consider only the probabilities

associated with entering and removing predictors (i.e., PIN and POUT). The desired probabilities

are entered in parentheses after the PIN and POUT keywords as shown in Box 12.2. POUT must

be greater than PIN, as shown here, or SPSS will increase its value (to 1.1 times PIN).

The results of the new analysis are identical to those in Box 12.1 for the first two steps.

Ability and then Study Time are successively selected for entry into the equation. After Step 2,

however, the PIN check now finds that the probability for Weight (.276) is less than the new PIN

(.30); indeed, the PIN of .30 was chosen so as to permit Weight to enter the equation. Weight is

entered, and because .276 is also less than POUT (.40), Weight remains in the equation once

entered. The final equation is equivalent to that resulting from the /ENTER option, which forces

all predictors into the equation.

Researchers sometimes want to see the equation with all p predictors, even when Stepwise

or other selection procedures are used. To force the equation with all predictors, follow

/STEPWISE with /ENTER. When Stepwise selection is finished, /ENTER then adds any variables

omitted from the Stepwise equation and reports the results with all variables in the final equation.

If the entry of excluded variables does not modify the probabilities from the Stepwise regression

(e.g., including Weight did not increase or decrease the probabilities for Ability and Study Time),

researchers might feel more certain that excluded variables were unrelated to the criterion. If

significant predictors become more or less significant when the final variables are added,

however, then excluded variables could have some relation with the criterion that is


Figure 12.1. Stepwise Procedure via Menus.

overshadowed by other predictors, or excluded variables could somehow weaken or enhance the

contribution of the other predictors, or there may be some complex inter-relationship among the

included and excluded predictors that is not readily picked up by automated procedures that

consider only one predictor at a time.

In the

preceding

example, the

various

statistical

procedures led

to the same

final equation.

But this is by

no means

always the case. Box 12.3 shows

FORWARD and BACKWARD analyses

for the first Grades study, with just

Ability and Study Time as predictors.

Recall that earlier analyses revealed that

neither predictor alone was significant,

although both were significant when the

two predictors were together in the

equation. Box 12.3 shows the

implications of this for the automated

procedures. /FORWARD and

/STEPWISE would stop without entering

any predictors because neither predictor

is significant at the .05 level, whereas

/BACKWARDS stops after both predictors

REGRE /VARI = grde abil stdy /DEP = grde /FORWARD.

Variables Entered/Removed(a) a Dependent Variable: grde

REGRE /VARI = grde abil stdy /DEP = grde /BACKWARD.

Model Variables Variables Method Entered Removed 1 stdy, abil(a) . Enter

Model R R Square Adjusted R Std. Error of Square the Estimate 1 .649(a) .421 .332 8.386014928... Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) -8.535 25.161 -.339 .740 abil .447 .178 .662 2.516 .026 stdy 1.358 .465 .769 2.922 .012

Box 12.3. Analysis of First Grades Study.


are entered because both predictors remain significant at the .10 level (indeed at a much higher

level of significance).

Automated Procedures using SPSS Menus

Figure 12.1 shows several screens relevant to specifying automated procedures using the

SPSS Menu system. Selecting Analyze | Regression | Linear activates the Linear Regression box

shown in Figure 12.1. The user specifies the Dependent variable (grde in this example) and all of

the predictors to be processed automatically by SPSS (abil, stdy, and wght here). Clicking

Method shows a series of options, including Stepwise, Forward, and Backwards. Stepwise has

been selected in Figure 12.1. Selecting Options opens the Linear Regression: Options dialogue

box also shown in Figure 12.1. Here users can control the p values for entering and removing

predictors (i.e., PIN and POUT). Clicking Continue and Ok produces the requested analysis,

which would agree with earlier printouts.

SPSS would also produce syntax for this menu command, which illustrates another feature

of the Regression syntax. Specifically, it is possible to omit the /VARIABLES= portion of the

regression command if the dependent variable is identified by the /DEPENDENT= command and

the independent variables are explicitly listed on the /METHOD= line. That is, users can write

commands such as: REGRESSION /DEP = grde /STEP abil stdy wght.

Word Attributes and Naming Latency Study

An earlier chapter described a study in which researchers measured naming reaction time

(RT) and various word characteristics related to ease of naming: concreteness (CON), frequency

(FRQ), length (LEN), and rated age of word acquisition (AGE). The results appear in Box 12.3.

Before considering the underlying structure of these data, which are known for this artificial case,

let us see what MR reveals about the data.


REGR /VAR = rt TO len /DESC /DEP = rt /ENTER

Correlations: RT AGE CON FRQAGE .585CON .318 .355FRQ .349 .524 .055LEN .323 .344 -.160 -.030

Multiple R .63813 Analysis of VarianceR Square .40721 DF Sum Squares MeanSquareAdj. R Square .33946 Regression 4 206598.3814 51649.5954Stand. Error 92.69899 Residual 35 300758.5933 8593.1027

F = 6.01059 Signif F = .0009

Variable B SE B Beta T Sig TLEN 14.699981 8.972093 .254772 1.638 .1103FRQ 1.146724 1.079876 .174709 1.062 .2956CON 15.073202 9.794688 .235170 1.539 .1328AGE 14.751414 8.910410 .322901 1.656 .1068(Constant) 622.600736 95.116180 6.546 .0000

Box 12.4. MR Analyses of Naming Data and ENTER Results.

REGRESS /DEP = RT /FORWARD age con frq len.

Model R R Square Adjusted R Std. Error of Square the Estimate 1 .585(a) .343 .325 93.6742945


Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 775.439 52.307 14.825 .000 AGE 26.747 6.008 .585 4.452 .000

Excluded Variables(b) Model Beta In t Sig. Partial Collinearity Correlation Statistics Tolerance 1 CON .127(a) .898 .375 .146 .874 FRQ .059(a) .375 .710 .062 .726 LEN .138(a) .984 .331 .160 .882

Box 12.5. Forward Selection Procedure.

Box 12.4 shows the MR results when all predictors are forced into the equation with

ENTER. Several aspects of the analysis warn us to interpret the data carefully. Although the

overall regression is highly

significant and the predictors

account for over 40% of the

variation in naming RT, no

predictor is individually

significant. This outcome

indicates redundancy among

the predictors. The

correlation matrix shows that

the age of acquisition

variable (AGE) may be the

problem. AGE correlates with concreteness, frequency, and length, which are relatively

independent, and all predictors are correlated with the criterion variable RT. AGE has a higher

correlation with RT than do the other predictors.


REGRE /VARI = rt age con frq len /DEP = rt /STEPWISE.

Model Variables Variables Method Entered Removed 1 AGE . Stepwise(Criteria: Probability-of-F-to-enter <= .050, Probability-of- F-to-remove >= .100).



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 775.439 52.307 14.825 .000 AGE 26.747 6.008 .585 4.452 .000

Excluded Variables(b) Model Beta In t Sig. Partial Collinearity Correlation Statistics Tolerance 1 CON .127(a) .898 .375 .146 .874 FRQ .059(a) .375 .710 .062 .726 LEN .138(a) .984 .331 .160 .882

Box 12.6. Stepwise Selection Method.

Box 12.5 shows the results of the Forward selection procedure, which is requested in

SPSS by using the /FORWARD keyword in place of /ENTER or /STEP. The results of this

analysis are straightforward. AGE enters as a predictor at Step 1, because AGE has the strongest

relation with RT. Once AGE is in the equation, none of the other predictors meet the PIN

probability level, the closest is Length (LEN) with a p of only .331. SPSS settles for an equation

with just one predictor. Note in Box 12.5 that the predictors were listed on the /FORWARD

subcommand rather than being entered on a separate /VARIABLES subcommand. This approach

is available for the various automated methods available in SPSS.

Box 12.6 reports the results for the Stepwise selection procedure. Since Stepwise is a

combination of Forward and Backward methods, we should not be surprised that the resulting

equation has a single predictor, age of acquisition. The output from the Stepwise and Forward


REGRE /VARI = rt age con frq len /DEP = rt /BACKWARD.

Model Variables Variables Method Entered Removed 1 LEN, FRQ, CON, . Enter AGE(a) 2 . FRQ Backward (criterion: Probability of F-to-remove >= .100). 3 . CON Backward (criterion: Probability of F-to-remove >= .100). 4 . LEN Backward (criterion: Probability of F-to-remove >= .100).

Model R R Square Adjusted R Std. Error of Square the Estimate 1 .638(a) .407 .339 92.6989896 2 .623(b) .388 .337 92.8631792 3 .600(c) .360 .325 93.7129335 4 .585(d) .343 .325 93.6742945



3 Regression 182418.761 2 91209.380 10.386 .000(c) Residual 324938.214 37 8782.114 Total 507356.975 39

4 Regression 173911.784 1 173911.784 19.819 .000(d) Residual 333445.191 38 8774.873 Total 507356.975 39

Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 622.601 95.116 6.546 .000 AGE 14.751 8.910 .323 1.656 .107 CON 15.073 9.795 .235 1.539 .133 FRQ 1.147 1.080 .175 1.062 .296 LEN 14.700 8.972 .255 1.638 .110

2 (Constant) 675.100 81.402 8.293 .000 AGE 20.524 7.073 .449 2.902 .006 CON 12.237 9.440 .191 1.296 .203 LEN 11.485 8.461 .199 1.357 .183

3 (Constant) 728.287 70.948 10.265 .000 AGE 24.580 6.401 .538 3.840 .000 LEN 7.957 8.084 .138 .984 .331

4 (Constant) 775.439 52.307 14.825 .000 AGE 26.747 6.008 .585 4.452 .000

Excluded Variables(d) Model Beta In t Sig. Partial Collinearity Correlation Statistics Tolerance 2 FRQ .175(a) 1.062 .296 .177 .626 3 FRQ .106(b) .655 .517 .109 .676 CON .191(b) 1.296 .203 .211 .784 4 FRQ .059(c) .375 .710 .062 .726 CON .127(c) .898 .375 .146 .874 LEN .138(c) .984 .331 .160 .882

Box 12.7. Backward Selection Analysis.

selection methods are identical for this study. Age of acquisition is the only predictor that gets

entered into the equation. Perhaps the Backward procedure will produce different results.


Box 12.7 shows the Backward selection results, requested in SPSS using the

/BACKWARD keyword. Backward selection begins with all variables in the equation, and this is

shown as steps 1 to 4 at the first block. The initial regression is identical to that in Box 12.4,

when all predictors were forced into the equation with /ENTER. Now, however, SPSS has been

requested to remove variables whose probabilities are less than .10, the default value for POUT.

FRQ is removed first because it has the highest probability level when all predictors are in

the equation, p = .2956, and its probability is greater than POUT = .10. SPSS proceeds to remove

in turn CON, p = .2031, and finally LEN, p = .3314. Note that p values are recomputed after each

predictor is removed from the equation. All predictors but AGE end up being removed, and AGE

becomes increasingly significant as more of the competing predictors are excluded. The net result

is an equation with just one predictor, age of acquisition, the same outcome as occurred with the

Forward selection procedure. To the extent that these methods are adequate for building

meaningful regression models, the one-predictor equation looks increasingly plausible.

All three selection methods resulted in a single equation with age of acquisition as the

predictor. Researchers who put excessive faith in blind selection methods might believe that this

is in fact the most sensible model for the data. But in the present case, we have the advantage of

knowing how the data were generated, which means that the results of the automated regression

analyses can be compared to the "true" underlying structure.

The data were actually produced as follows. Random scores were produced for the item

attributes of Concreteness, Frequency, and Length. These scores (plus some random variation)

were then used to generate both the Reaction Time and the Age of Acquisition scores. Age of

Acquisition scores were NOT used directly in the generation of RT scores; that is, there is no

direct causal relation between AGE and the RT data. Rather, AGE is highly correlated with RT

because both AGE and RT were determined by Concreteness, Frequency, and Length. In a sense,

the high correlation between AGE and RT fooled the regression program and its statistical criteria

into choosing AGE as the primary and indeed the only predictor of RT.

Box 12.8 shows the results of a regression analysis that the automated procedures never

got to; namely, the regression of RT on CON, FRQ, and LEN without AGE in the equation. The

prediction is very good, slightly stronger than AGE alone comparing R2s, and each of the


REGRE /STAT = DEFAU ZPP /DEP = rt /ENTER con frq len.



Model Unstandardized Standardized t Sig. Correlations Coefficients Coefficients B Std. Error Beta Zero-order Partial Part 1 (Constant) 584.562 94.505 6.186 .000 CON 23.240 8.664 .363 2.682 .011 .318 .408 .357 FRQ 2.237 .876 .341 2.554 .015 .349 .392 .340 LEN 22.570 7.791 .391 2.897 .006 .323 .435 .386

Box 12.8. MR Analysis for ENTER without AGE.

predictors makes a highly significant contribution to the regression equation. These significant

effects were overshadowed by the domineering AGE variable, and its spurious correlation with

RT.

Perhaps the most appropriate conclusions from this hypothetical study are that

Concreteness, Length, and Frequency contribute significantly to both RT and Age of Acquisition,

but that RT and AGE are not themselves directly linked. The relations between AGE and the

other predictors makes sense if subjects in the Age of Acquisition rating task did not actually

remember at what age they learned the words (a plausible assumption?), but rather made

inferences about Age of Acquisition from other word attributes. That is, subjects know that

concrete, short, and familiar words are learned earlier than abstract, long, and less familiar words.

This example involved only four predictors, yet still presented analytic challenges for the

standard MR selection procedures. The theoretical and empirical difficulties become even more

subtle and complex as the number of predictors increases, so users of MR must prepare for careful

theoretical and empirical analysis of the relations among variables in MR studies. Only

occasionally will the relations be sufficiently simple to trust automated analysis using purely

statistical selection criteria, and unfortunately, researchers never know which occasions are the

simple ones. Some of these complexities are described shortly.


Interpretation of Unique Contribution

Irrespective of how the final model is arrived at, the interpretation of the unique

contribution of each predictor follows the basic principles described previously. Boxes 12.9 and

12.10 demonstrate the application of our principles to understanding the unique contribution of

CON in the regression analysis of Box 12.8. Box 12.9 shows the change analysis with CON

added after FRQ and LEN are already in the equation.

The difference between SSRegression for Models 1 and 2 (or equivalently between SSResidual)

would give SSR’C.FL, which divided by 9008.619 would give FChange = 7.196 = t2 = 2.6822. Dividing

SSR’C.FL by SSTotal would give the part r2, r2R(C.FL) = .128 = .361 - .233 = .3572. Dividing SSR’C.FL by

389133.844 would give the partial r2, r2RC.FL = .4082.

REGRE /STAT = DEFAU CHANGE ZPP /DEP = rt /ENTER frq len /ENTER con.




Model Unstandardized Standardized t Sig. Correlations Coefficients Coefficients B Std. Error Beta Zero-order Partial Part 1 (Constant) 715.919 87.334 8.197 .000 FRQ 2.356 .945 .359 2.492 .017 .349 .379 .359 LEN 19.246 8.311 .334 2.316 .026 .323 .356 .333

2 (Constant) 584.562 94.505 6.186 .000 FRQ 2.237 .876 .341 2.554 .015 .349 .392 .340 LEN 22.570 7.791 .391 2.897 .006 .323 .435 .386 CON 23.240 8.664 .363 2.682 .011 .318 .408 .357

Box 12.9. Change Statistics for Unique Contribution of CON.


Box

12.10 shows the

creation of

residual

predictor and

criterion

variables that

also produce

the part and partial rs (shown

in italics and bold in the

correlation matrix).

It will always be the

case that our methods for

understanding the unique

contribution of predictors in

multiple regression can be

applied to regression

analyses of any sort (i.e.,

using ENTER, FORWARD,

BACKWARD, or

STEPWISE).

SPSS Regression Options

and Menus

Figure 12.2 shows

again how Stepwise

regression would be accessed

by the menu system. The

various independent

variables are selected and

REGRESS /DEP = con /ENTER frq len /SAVE RESI(resc.fl).REGRESS /DEP = rt /ENTER frq len /SAVE RESI(resr.fl).VARIABLE LABELS resc.fl '' resr.fl ''.CORR rt frq len resc.fl resr.fl.

RT FRQ LEN resc.fl resr.fl RT Pearson 1 .349 .323 .357 .876 FRQ Pearson .349 1 -.030 .000 .000 LEN Pearson .323 -.030 1 .000 .000 resc.fl Pearson .357 .000 .000 1 .408

resr.fl Pearson .876 .000 .000 .408 1

Box 12.10. Part and Partial Using Residual Variables.

Figure 12.2. Stepwise Regression Using Menus (and

Corresponding Syntax).


moved into the Independent(s) field and the Stepwise Method is selected. The resulting syntax is

shown in the output screen above the Regression screen. Rather than being listed on a

/VARIABLES = subcommand, the predictors are included on the /METHOD = STEPWISE

subcommand. The actual analysis would be identical to that shown in Box 12.6.

COMPLEXITIES IN REGRESSION MODEL BUILDING

Although computers make possible the calculation-intensive but automatic selection

procedures described above, no purely statistical method is completely satisfactory for the

selection of predictors in MR designs. Researchers must interpret findings with an understanding

of the theoretical meaning of the constructs and relations being examined, something automated

procedures do not do, and with an adequate understanding of the significance tests and other

statistics reported by MR for the individual predictors and used by the various selection

procedures. In short, care with both theory and data analysis is essential for reducing the

likelihood of inappropriate interpretations from MR designs. We consider three simple examples

of problems that can arise: redundant predictors, coincident criterion variables as predictors, and

suppressor effects.

Redundant Predictors

Consider first a study in which two highly overlapping predictors are included in a

regression analysis; perhaps the variables are alternative measures of the same underlying

construct. To illustrate, educational researchers in the Study Time project could have measured

both Study Time and self-reported Effort for each student. It is likely that Study Time and Effort

would be highly correlated, since amount of studying would be a major contributor to or

manifestation of effort.

Regressing grades on both Study Time and Effort and examining their individual

contribution would be equivalent to producing residual scores for Study Time partialling out

Effort, and residual scores for Effort partialling out Study Time. The residual scores for Study

Time would be independent of Effort, and the residual scores for Effort would be independent of

Study Time. Such residuals could be conceptually meaningless. It seems possible, perhaps even

probable, that we would remove the variability in one or both of the scores that is responsible for


its relation with grades. That is, Study Time independent of Effort and Effort independent of

Study Time might be uncorrelated with grades, and might also represent unusual and perhaps

uninteresting aspects of the data.

What could happen with blind use of an automated selection procedure is that one of the

variables would be selected for entry into the equation and the other variable excluded.

Whichever variable happened to have the highest simple or partial correlation with the criterion

grades would be entered, no matter how slight its superiority. The other predictor would not be

selected because it no longer met the probability for entry, once its correlated partner was already

in the equation. Naive researchers might conclude that the omitted variable (e.g., Study Time)

was not important for grades, when in fact a more correct interpretation would be that the critical

variability in Study Time has already been credited to the Effort variable.

A more adequate solution to this problem would be to combine the Study Time and Effort

measures together to obtain a single predictor that reflected some underlying composite or general

construct, perhaps a revised construct of Effort that included study effort. An alternative approach

would be to identify aspects of Effort that are potentially independent of Study Time (e.g., talking

to teacher or other students about course) and using both Study Time and this "purer" and perhaps

less correlated measure of non-studying Effort as predictors.

Coincident Outcomes as Predictors

A related problem arises when one predictor is determined by a collection of the other

predictors, all of which have relations to the criterion. Consider the previously-mentioned study

of picture naming latency with mean naming reaction time (RT) for a large set of words as the

dependent variable. Each variable has scores for length in letters, frequency in print,

concreteness, and rated age of word acquisition, and these are used to predict naming RT.

Stepwise or some other selection procedure might result in an equation with age of acquisition as

the sole predictor, because its correlation with RT tends to be higher than each of the other

variables. Given age of acquisition is in the equation, the other predictors make no additional

contribution to the prediction of RT.


Figure 12.3. Relation of Predictors to RT and

Age of Acquisition.

Figure 12.4. Negative Relation Between

Predictors Positively Related to Criterion.

Any conclusion that gives undue

weight to this finding may be incorrect if age

of acquisition is itself dependent on length,

frequency, and concreteness. That is, perhaps

subjects do not rate age of acquisition directly,

but rather give early ratings to words that are

short, frequent, and concrete, and late ratings

to words that are long, infrequent, and abstract. The hypothesized model is diagrammed in Figure

12.3. Age of acquisition is the best predictor of RT not because age of acquisition contributes

directly to naming RT, but because age is a proxy for all of the other variables that do contribute

to RT. That is, age includes predictive variability associated with length, concreteness, and

frequency, not leaving anything to be accounted for by those variables, the "true" predictors in this

hypothetical situation.

Suppressor Effects

Other problems, such as suppressor effects, may be specific to particular selection

procedures. For example, two negatively correlated predictors can have hidden positive effects on

a criterion variable. Their positive effects could be hidden or masked because of their negative

relation with the other predictor that is also positively related to the criterion. One example of

such a situation, which we analyzed in earlier chapters, is illustrated in Box 12.3. Grades are

positively determined by ability and study time, which are negatively related to one another.

Such underlying structures can pose

problems for some automated selection

methods. If the simple relations with grades

not controlling for the other variables are

nonsignificant, then the two variables might

never get in the equation together using the

Forward or Stepwise selection procedures.

The simple correlations (or part correlations

with all but the critical remaining predictor) never achieve the requisite probability level to be


entered because of the masking effect of the confounded predictor. The Backward selection

procedure would not have this difficulty as long as the probabilities for the effects both remained

low enough to remain in the equation. If the probability for one of the variables increased to

above the critical value, however, then it and subsequently the other predictor would be ejected.

Redundant predictors, coincident outcomes, and suppressor effects illustrate just a few of

the complexities of multiple regression. Overcoming these and other difficulties requires an

adequate conceptual understanding of the domain being investigated, knowledge about the

statistical methods that underlie MR, experience, and a healthy dose of skepticism about the

adequacy of automated selection procedures.

Testing All Possible Equations

Another approach to regression modelling that is also flawed but sometimes used by

researchers is to examine all-possible equations to determine the best combination of predictors.

Although this approach has the

potential to identify some of the

complex relationships just

discussed, it is also extremely

susceptible to the problem of

inflated Type I errors. That is,

all-possible regressions is prone

to identifying predictors as

significant simply because of the

inflated probability of a Type I

error. Although SPSS does not

automatically examine all-

possible equations, it is relatively

simple to obtain relevant basic

statistics except for exceptionally

large numbers of predictors.

Box 12.11 shows one way to efficiently get basic statistics (in particular R and R2) for all

*All Possible Regressions: # equations = 2^4 - 1 = 15. Model R R Square Adjusted R Std. Error of Square the EstimateREGR /STAT = R /DEP = rt /ENTER age. 1 .585(a) .343 .325 93.6742945REGR /STAT = R /DEP = rt /ENTER con. 1 .318(a) .101 .078 109.5332346REGR /STAT = R /DEP = rt /ENTER frq. 1 .349(a) .122 .099 108.2802034REGR /STAT = R /DEP = rt /ENTER len. 1 .323(a) .104 .081 109.3570377REGR /STAT = R /DEP = rt /ENTER age con. 1 .597(a) .357 .322 93.9146992REGR /STAT = R /DEP = rt /ENTER age frq. 1 .588(a) .345 .310 94.7517013REGR /STAT = R /DEP = rt /ENTER age len. 1 .600(a) .360 .325 93.7129335REGR /STAT = R /DEP = rt /ENTER con frq . 1 .460(a) .212 .169 103.9639084REGR /STAT = R /DEP = rt /ENTER con len. 1 .495(a) .245 .204 101.7493731REGR /STAT = R /DEP = rt /ENTER frq len. 1 .483(a) .233 .192 102.5530639REGR /STAT = R /DEP = rt /ENTER age con frq. 1 .601(a) .362 .309 94.8428311REGR /STAT = R /DEP = rt /ENTER age con len. 1 .623(a) .388 .337 92.8631792REGR /STAT = R /DEP = rt /ENTER age frq len. 1 .606(a) .367 .314 94.4441804REGR /STAT = R /DEP = rt /ENTER con frq len. 1 .601(a) .361 .308 94.9137441REGR /STAT = R /DEP = rt /ENTER age con frq len. 1 .638(a) .407 .339 92.6989896

Box 12.11. All Possible Regression Statistics.


possible regression equations for the naming study. Although 15 separate regressions must be

done, including /STATISTICS = R greatly reduces the output. Examining the R2s in Box 12.11

reveals that the amount of variation accounted for is roughly equivalently for the following 7

equations: any equation involving Age, either alone or with one, two, or all three other predictors,

and the equation with just the three other predictors of Con, Frq, and Len. This pattern of results

could lead to an interpretation such as that suggested earlier, namely that Age overlaps with all

three other predictors and is therefore the best single predictor of the four. Although in this case

the simple correlation matrix helps to support this same interpretation, other more complex

patterns of relationships among predictors might not be discovered without examining all-possible

combinations (or subsets) of predictors, as in Box 12.9.

SPSS has another regression option that can help with deciphering complex regressions.

The /TEST option in Regression allows users to specify sets of predictors for which basic change

statistics will be computed (in particular, R2). But it is important to understand the way in which

/TEST works. Specifically, it tests whether removal of the specified set of predictors results in a

REGRE /DEP = rt /TEST (age) (con frq len).

Model Sum of Squares df Mean Square F Sig. R Square Change

1 Subset AGE 23551.684 1 23551.684 2.741 .107(a) .046 Tests CON, FRQ, LEN 32686.598 3 10895.533 1.268 .300(a) .064

Regression 206598.382 4 51649.595 6.011 .001(b) Residual 300758.593 35 8593.103 Total 507356.975 39

REGR /STAT = DEFAU CHANGE /DEP = rt /ENTER con frq len /ENTER age.

Model R R Adjusted Std. Error of Change Statistics Square R Square the Estimate R Square Change F Change df1 df2 Sig. 1 .601(a) .361 .308 94.913744124 .361 6.773 3 36 .001 2 .638(b) .407 .339 92.698989556 .046 2.741 1 35 .107

REGR /STAT = DEFAU CHANGE /DEP = rt /ENTER age con frq len /REMOVE age.

Model R R Adjusted Std. Error of Change Statistics Square R Square the Estimate R Square Change F Change df1 df2 Sig. 1 .638(a) .407 .339 92.698989556 .407 6.011 4 35 .001 2 .601(b) .361 .308 94.913744124 -.046 2.741 1 35 .107

REGR /STAT = DEFAU CHANGE /DEP = rt /ENTER age /ENTER con frq len.

Model R R Adjusted Std. Error of Change Statistics Square R Square the Estimate R Square Change F Change df1 df2 Sig. 1 .585(a) .343 .325 93.674294525 .343 19.819 1 38 .000 2 .638(b) .407 .339 92.698989556 .064 1.268 3 35 .300

Box 12.12. Using /TEST Option to Test Contribution of Single Predictor.


substantive change in R2 or F. Box 12.12 demonstrates the procedure for the word naming study,

as well as the more familiar regression commands that would produce equivalent statistics. The

first regression uses the /TEST option to determine the contribution of the two subsets listed to the

overall regression. Here we see that Age does not contribute significantly over and above Con,

Frq, and Len, F = 2.741, p = .107, R2Change = .046.

Simultaneously, we see that Con, Frq, and Len together (df = 3) do not contribute

significantly over and above Age alone, F = 1.268, p = .300, R2Change = .064. Note that these

statistics are equivalent to those shown for Model 2 in the Change Statistics section of the last

regression analysis. Model 2 was when Con, Frq, and Len were added to Age, hence dfNumerator =

3. Although this is one of the few times that we have examined change statistics for more than

one additional predictor, the procedures would be similar to those we have done before; namely,

SSChange = SSy.ACFL - SSy.A, and dfNumerator for SSChange = number of additional predictors = 3.

Box 12.13 illustrates the use of /TEST to examine the removal of all-possible

combinations of predictors. This is the converse of the analyses in Box 12.9, where all possible

combinations of predictors were entered into the equation. Two analyses were used in Box 12.13

REGRE /DEP = rt /TEST (age) (con) (frq) (len) (age con) (age frq) (age len) (con frq) (con len) (frq len).

Model Sum of Squares df Mean Square F Sig. R Square Change 1 Subset AGE 23551.684 1 23551.684 2.741 .107(a) .046 CON 20350.722 1 20350.722 2.368 .133(a) .040 FRQ 9689.929 1 9689.929 1.128 .296(a) .019 LEN 23067.261 1 23067.261 2.684 .110(a) .045 AGE, CON 88375.251 2 44187.625 5.142 .011(a) .174 AGE, FRQ 82299.999 2 41149.999 4.789 .015(a) .162 AGE, LEN 99155.694 2 49577.847 5.769 .007(a) .195 CON, FRQ 24179.621 2 12089.811 1.407 .258(a) .048 CON, LEN 31423.148 2 15711.574 1.828 .176(a) .062 FRQ, LEN 25580.324 2 12790.162 1.488 .240(a) .050


REGRE /DEP = rt /TEST (age con frq) (age con len) (con frq len) (age con frq len).

Model Sum of Squares df Mean Square F Sig. R Square Change 1 Subset AGE, CON, FRQ 153681.951 3 51227.317 5.961 .002(a) .303 AGE, CON, LEN 144776.300 3 48258.767 5.616 .003(a) .285 CON, FRQ, LEN 32686.598 3 10895.533 1.268 .300(a) .064 AGE, CON, FRQ, 206598.382 4 51649.595 6.011 .001(a) .407 LEN


Box 12.13. Using /TEST to Examine Removal of Combinations of Predictors.


because of memory limitations (i.e., SPSS would not examine all possible combinations without

increasing the amount of computer memory available for the analysis). Note that the final R2s in

Boxes 12.11 and 12.13 equal .407, whether it is the R2 for entering all four predictors (Box 12.11)

or for removing all four predictors (Box 12.13). Box 12.13 reveals that the effect of removing

each predictor individually, including Age, is modest, as is the effect of removing various

combinations of Con, Frq, and Len. But removing Age plus any combination of the other

predictors results in a more substantial decrease. It is perhaps more challenging, but one can

through thoughtful examination of the R2s in Box 12.13 arrive at the combination of Con, Frq, and

Len as arguably the most intelligible combination of predictors.

Although all-possible subsets of predictors and SPSS’s /TEST procedure are susceptible to

many of the same flaws as the automated procedures (i.e., STEPWISE, ...), there is one important

difference — researchers can use their judgment and knowledge of theory and previous findings

to inform their selection of the “most appropriate” model. Selection is not simply a function of

some uninformed statistical criterion.

CONCLUSIONS

Chapter 11 demonstrated that the interpretation of individual predictors in multiple

predictor studies involves a straightforward extension of the approaches developed for the two-

predictor MR design. Each predictor's contribution controls statistically for all other predictors in

the equation. Unfortunately, not all aspects of multiple regression with many predictors are as

straightforward, and Chapter 12 has described some special tools that have been constructed to

help with deciphering the most meaningful combination(s) of predictors. However, the number

and subtle influences on one another of multiple predictors present additional complexities for the

selection of appropriate models using Forward, Backward, Stepwise, or other approaches to

predictor selection. Purely statistical methods must be balanced by thoughtful consideration of the

theory that underlies the variables and relations of interest.


SET SEED = 27395137INPUT PROGRAMLOOP SUBJ = 1 TO 24COMP #z1 = NORMAL(1)COMP #z2 = NORMAL(1)COMP grde = RND(65+10*(#z1*.4+#z2*.4+NORMAL(1)*SQRT(1-.68**2)))COMP abil = RND(100 + 15*#z1)COMP stdy = RND( 20 + 5*(#z1*-.5 + #z2*SQRT(1-.5**2)))COMP wght = RND(140 + NORMAL(20))END CASEEND LOOPEND FILEEND INPUT PROGRAM

SET SEED = 39513773INPUT PROGRAMLOOP WORD = 1 TO 40COMP #c = NORM(1) /* concretenessCOMP #f = NORM(1) /* frequencyCOMP #l = NORM(1) /* lengthCOMP rt = RND(1000 +100*(#c*.5+#f*.5+#l*.5+NORM(1)*SQRT(1-.75**2)))COMP age = RND( 8 + 2*(#c*.5+#f*.5+#l*.5+NORM(1)*SQRT(1-.75**2)))COMP con = RND( 5 + 1.5*(#c*.8 + NORM(1)*SQRT(1-.8**2)))COMP frq = RND( 50 + 15 *(#f*.8 + NORM(1)*SQRT(1-.8**2)))COMP len = RND( 8 + 2 *(#l*.8 + NORM(1)*SQRT(1-.8**2)))END CASEEND LOOPEND FILEEND INPUT PROGRAM

Notes

Note 12.1. The following SPSS program was used to generate the new study time data,

including weight as an additional predictor. The program assumes that grades are dependent

positively on both ability and study time, that study time is negatively related to ability, and that

weight is independent of ability, study time, and grades. Twenty-four observations are randomly

selected from a population of scores with these characteristics.

Note 12.2. The following SPSS program was used to generate the data for the naming

study. See the text for discussion of the underlying structure, which in essence creates both

reaction time and age of acquisition scores as a function of concreteness, word length, and

frequency.

MR and Non-Linear Relationships 13.1

CHAPTER 13 -

MULTIPLE REGRESSION AND NON-LINEAR RELATIONSHIPS

Using Plots to Identify Nonlinear Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Plot of Dependent Variable Against Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Plot of Residuals Against Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Polynomial Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Plotting Nonlinear (Quadratic) Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Value of Changing Slope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Transforming Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Using COMPUTE to Transform Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Using SPSS’s Curve Estimation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Discussion of Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Imitation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20


Standard correlation and regression methods are designed for linear relationships between

the criterion and predictor variables. Linear relations are those in which changes on the predictor

are associated with constant changes on the criterion (i.e., y increases or decreases by the same

amount when x changes from 0 to 1 as when x changes from 10 to 11). Changes in y that vary

across different levels of a single predictor indicate a nonlinear relation. Unless prepared for

nonlinear relationships, researchers might wrongly conclude that there is a linear relation

between the variables or, in extreme cases of curvilinearity, that there is no relationship between

the criterion and the predictor. Chapter 13 examines several ways of determining whether a

nonlinear relationship exists, and if so, of analyzing such data.

USING PLOTS TO IDENTIFY NONLINEAR RELATIONSHIPS

Consider a study of the

effect on reaction time (RT) of

number of alternatives

(uncertainty or U). In each of

8 groups, 4 subjects pressed

different keys as rapidly as

possible when numbers appeared on a computer screen (e.g., key 1 for the number 1, key 2 for 2,

and so on). Subjects in the lowest uncertainty group (U = 1) only saw one number and pressed

one key. Subjects in U = 2, had two stimuli and responses, and so on up to U = 8, where subjects

saw one of 8 numbers and pressed one of 8 keys. RT to press the key was determined in ms

(milliseconds or 1000ths of a second). The results are shown in Box 13.1, along with the SPSS

command to plot RT as a function of uncertainty (u).

DATA LIST FREE / u rt.BEGIN DATA1 303 1 297 1 296 1 312 2 318 2 320 2 323 2 3363 342 3 340 3 342 3 347 4 363 4 364 4 353 4 3495 363 5 366 5 379 5 349 6 373 6 367 6 389 6 3707 374 7 367 7 377 7 396 8 392 8 390 8 387 8 386END DATA.GRAPH /SCATTERPLOT(BIVAR) = u WITH rt.

Box 13.1. RTs as a Function of Stimulus Uncertainty.


Uncertainty

1086420

Reaction T

ime

400

380

360

340

320

300

280

R^2 = .88

Figure 13.1. Observed and Predicted (Linear)

RTs.

Plot of Dependent Variable Against Predictor

Figure 13.1 shows the plot of the

observed RTs at each uncertainty level, as

well as the predicted RTs from a linear

regression of RT on Uncertainty. Actual

regression analyses are presented later.

Figure 13.1 shows that RT increased (people

got slower) as the number of alternative

responses increased. This result agrees with

several theories of choice RT because, as

uncertainty increases, the mind must make

more decisions to identify which stimulus

occurred and to select the appropriate

response. Simple regression indicates a strong linear component to the relation between

Uncertainty and RT, as indicated by R2 = .88. The fit to the observed data is quite good, as

would be expected in a predictor that explains 88% of the variability in RT.

Despite the robust linear effect, however, Figure 13.1 reveals that the effect on RT of

increasing Uncertainty is not uniform across all levels of Uncertainty. Changes in U at the low

end (1 to 2 to 3) produce larger increases in RT than changes in U at the upper end (6 to 7 to 8).

The rate of increase in RT levels off or decreases as Uncertainty increases, leading to a flattening

of the relationship at the upper levels of uncertainty. These systematic deviations from linearity

can be seen by comparing the observed RTs to the RTs predicted by the best-fit linear equation.

As shown in Figure 13.1, the linear equation predicts RTs that are too high (slow) at the low and

high ends of U and RTs that are too low (fast) at the middle levels of U. The observed values

start out lower than the predicted values, rise above the predicted values in the middle range of

U, and then curve below again. Such systematic deviations from predicted values indicate a

nonlinear relation. If the relation between RT and U were linear, deviations from the predicted

values (i.e., residual scores) would be scattered randomly about the best-fit linear equation over

the entire range of Uncertainty. That is, the best-fit line would pass through the center of the


Uncertainty

1086420

Re

sid

ua

l S

co

re (

Lin

ea

r)

20

10

0

-10

-20

Figure 13.2. Plot of Linear Residuals Against U.

observations at all levels of uncertainty.

Plot of Residuals Against Predictor

This non-linear effect can be seen even more clearly by plotting residuals from the linear

regression against the predictor, U.

Figure 13.2 shows the plot. The

horizontal dashed line at 0 indicates the

mean of the residual scores; recall that

residual scores sum to 0 and hence have

a mean of 0. If there were no nonlinear

relationship between RT and U, then the

residuals in Figure 13.2 would be

scattered randomly about the mean of 0

over the full range of the predictor U.

But clearly, the residual scores are not

random. In fact, low and high values of

U are associated with predictions that are too high (i.e., the residuals fall below the overall mean

of 0), and middle values of U are associated with predictions that are too low (i.e., these residuals

fall above their overall mean of 0).

Figures 13.1 and 13.2 show visual ways to determine whether a relationship is linear or

non-linear; that is, plot the original y scores or the residual scores from a linear regression against

the original predictor scores. These procedures are particularly robust when nonlinear relations

are quite striking, although they still involve some degree of subjective judgment. The plots of

residuals are generally somewhat more sensitive than plots of the original y scores. But better yet

is to perform statistical procedures to determine whether deviations from linearity are significant

and how significant they are. Such procedures are described in the following sections.


POLYNOMIAL REGRESSION

To determine whether the apparent deviation from linearity is significant or should be

attributed to chance, researchers can use multiple regression with polynomial predictors.

Polynomial predictors are similar to interaction terms (discussed in a later chapter) that involve

multiplying predictors together to generate additional predictors sensitive to differences in the

slope for one variable at different levels of another variable. In the case of non-linear regression,

a single predictor is multiplied by itself one or more times. That is, in addition to x in the

equation, we include x2 (x × x) and occasionally higher powers (i.e., x3, x4, ...). The variable x is

the linear term, x2 the quadratic term, x3 the cubic term, and so on. Including these powers of x

in the prediction equation provides one way to accommodate nonlinear relationships.

Box 13.2 shows the polynomial regression for the uncertainty study. A new predictor, U2,

COMPUTE u2 = u**2.REGRESS /VARI = rt u u2 /STAT = DEFAU CHANGE ZPP /DEP = rt /ENTER u /ENTER u2 /SAVE PRED(prdr.uu2) RESI(resr.uu2).

R R Adjusted Std. Error of Change Statistics Square R Square the Estimate Model R Square Change F Change df1 df2 Sig. 1 .938(a) .880 .876 10.1966576 .880 220.605 1 30 .000 2 .962(b) .925 .920 8.1910931 .045 17.489 1 29 .000



Unstand Coeff Stand Coeff t Sig. CorrelationsModel B Std. Error Beta Zero Partial Part1 (Constant) 301.482 3.973 75.891 .000 U 11.685 .787 .938 14.853 .000 .938 .938 .938 U2

2 (Constant) 281.661 5.714 49.294 .000 U 23.577 2.913 1.893 8.093 .000 .938 .833 .411 U2 -1.321 .316 -.978 -4.182 .000 .870 -.613 -.212


Box 13.2. Polynomial Regression with U and U2 as Predictors.


Uncertainty

1086420

Re

actio

n T

ime

400

380

360

340

320

300

280

Figure 13.3. Polynomial Prediction Equation.

is first computed and then a regression analysis includes both U and U2 as predictors of RT. U

and U2 have been added sequentially to demonstrate more fully the additional contribution of the

non-linear component. Adding U2 increases R2 by .045 (part r = -.212), which is a highly

significant change, FChange = 17.489, p = .000, or equivalently, tU2 = -4.182, p = .000.

Note that the coefficient for U is much steeper in Model 2, BU = 23.577 than in Model 1

where BU = 11.685, and that the slope for U2 is negative, BU2 = -1.321. Essentially what this

equation does is start out with a much steeper increase in RT as a function of initial increases in

U, but the slope becomes increasingly shallow as U increases. Specifically, the multiple

regression of RT on U and U2 produces � = 281.661 + 23.577 × U -1.321 × U2. The coefficient

for U is now almost twice as large as it was when RT was regressed on U alone. Moreover, R2

has increased significantly, from .876 to .920, and the negative coefficient for U2 is significant, as

shown by tbu2 (tU2 = -4.182, p = .0002) or the equivalent FChange. We later discuss these statistics

and others for U2 in more detail.

Plotting Nonlinear (Quadratic) Equations

This effect, and the improvement in the prediction can be seen in Figure 13.3, which

shows the predicted scores given the non-linear equation including both U and U2. The

polynomial prediction equation goes more clearly through the center of all the data points across

the entire range of the Uncertainty predictor.

There is no obvious tendency for the equation

to under-predict or over-predict at different

locations.

Figure 13.3 was produced in two steps.

First a scattergram was requested using either

the menu system, Graph | Scatter | Simple, or

the following syntax: GRAPH

/SCATTERPLOT(BIVAR) = u WITH RT. Then

the Chart Editor was activated by double-

clicking on the original graph. One option

available in the Chart Editor is to fit either


linear or quadratic equations to the data. Figure 13.3 shows the Quadratic fit (Figure 13.1 shows

the Linear fit). The curve in Figure 13.3 generates predicted values closer to the actual data than

does the straight line in Figure 13.1. In particular, there is no longer any systematic under- or

over-predicting across the range of uncertainty values. Predictions fall in the “middle”

throughout, with residuals both above and below the predicted values.

Value of Changing Slope

Because U2 = U × U, the best-fit regression equation can be rewritten as: � = 281.661 +

(23.577 - 1.321 × U) × U. The value in parentheses (23.577 - 1.321 × U) becomes smaller as U

increases; for example, it will be 23.588 - 1.321 × 1 = 22.256 for U = 1 and 23.577 - 1.321 × 8 =

13.01 for U = 8. Because the adjusted value in parentheses is then multiplied by U, the predicted

change in RT as U increases becomes smaller as U gets larger. For U = 1, � = 281.661 +

22.256×1 = 303.917. For U = 8, � = 281.661 + 13.01×8 = 385.74. Thus the polynomial term

reflects the change in the slope as the predictor increases, and the predicted values level off at the

upper end of U.

Although the preceding description provides an intuitive sense that the slope will change

as U increases, it does not in fact provide the proper way to calculate what the actual slope will

be for any particular value of x. The formula to determine the slope for a quadratic regression

equation is: b1 + 2b2X, where X is a particular point on the predictor variable (U in the present

example). Here are some values for the slope at different values of U in the present example: bu=1

= 23.577 + 2 x -1.321 x 1 = 20.935, bu = 4 = 23.577 + 2 x -1.321 x 4 = 13.009, bu = 8 = 23.577 + 2 x

-1.321 x 8 = 2.441. Note how the slope decreases systematically from U = 1 to 4 to 8.

The interaction term, U2, in this study was negative, indicating that the slope became

shallower as the predictor increased. If the effect of U on RT increased as U got larger (i.e., the

slope got steeper), then the coefficient for U2 would be positive. If the effect of U on RT was

constant across levels of U, then the coefficient for U2 would be approximately zero, and not

significant. This would indicate a linear relationship or a more complex nonlinear function that

was not fit well by x2. Polynomial regression is one powerful method for the analysis of

nonlinear relations.

Although polynomial regression provides a way to accommodate nonlinear relationships,


the approach does have some limitations. A close look at Figure 13.3, for example, shows that

predicted values are actually starting to decrease at the highest levels of uncertainty. That is, the

slope has become negative, rather than positive. Extrapolating beyond the top value in Figure

13.3 would make this effect even stronger. But it is highly unlikely from a theoretical

perspective that actual reaction times would start to decrease. Rather RTs are likely to continue

increasing at a decreasing rate. But polynomial regression does not allow for such levelling off,

rather than actual changes in the sign of the slope. Alternative approaches are less vulnerable to

this problem, including more sophisticated nonlinear regression approaches not discussed in this

book and transforming predictors.

TRANSFORMING VARIABLES

A second approach to nonlinear relationships is to transform the criterion variable or the

predictors to make the relation more linear. In Figure 13.1, for example, the relation would be

more linear if the U scale were compressed so that the higher values of U were pushed closer to

the lower values, with little or less compression at the lower levels of U. There are various

operations that compress variables, including a square root transformation. To illustrate, original

values of 1, 4, and 9 would become 1, 2, and 3. Note that the original differences of 3 between 1

and 4, and of 5 between 9 and 4 have been equalized; that is, both differences are now 1 (2 vs 1

and 3 vs 2).

Other transformations produce even more compression. Logarithms to the base 10, for

example, would make the difference between 1 and 10 exactly the same as the difference

between 10 and 100; that is 1, 10, and 100 would become 0, 1, and 2. Reciprocals (one divided

by the values of the original predictor) also produce compression; for example, .3333, .5, and 1

would become 1/.333 = 3, 1/.5 = 2, and 1/1 = 1 (note the reversal in ordering using the reciprocal

transformation).

These are common transformations to compress predictors. Transformations that

compress predictors involve powers less than 1 (square root is power of .5, logarithm � power of

0, and reciprocals are powers of -1). Transformations using powers greater than 1 spread

predictors out; for example, squaring 1, 2, and 3 produces 1, 4, and 9, so that now the distance


Figure 13.4. Plot of RT Against LOGU.

between the 2nd and 3rd levels of the predictor is greater than the distance between the 1st and 2nd

levels.

When predictors are transformed in the preceding ways, they are used by themselves or

with other predictors in the regression analysis. This is different than the polynomial regression

approach, which includes both the original predictor (e.g., U) and the newly created predictor

(e.g., U2). In the approach being discussed here, transformed predictors replace the original

predictor, rather than complementing it in the regression analysis.

Figure 13.4 shows a plot of the observed data using log uncertainty, instead of the

original uncertainty levels; that is,

log(1) = 0, log(2) = .301, log(3) = .477,

..., log(8) = .903. The relation between

mean RT and log U is clearly more

linear than the relation between RT and

the original Uncertainty scores.

Predicted RTs from the best-fit

equation using log U are also plotted in

Figure 13.4. The observed and

predicted values are much closer, and

there are now few systematic

deviations about the regression line. R2

has also increased from .88 (see Figure

13.1 or Model 1 in Box 13.2) to .925

(see later analyses for origin of this

new value).


COMP lgu = LG10(u)REGR /VAR = rt lgu u /DESC /DEP = rt /ENTER lgu

Correlation: RT UU .938LGU .962 .959

Multiple R .96187R Square .92519Adjusted R Square .92270Standard Error 8.06043

DF Sum of Squares Mean SquareRegression 1 24106.75793 24106.75793Residual 30 1949.11707 64.97057

F = 371.04120 Signif F = .0000

Variable B SE B Beta T SigTLGU 96.052116 4.9865 .9619 19.262 .000(Const) 298.766251 3.2049 93.223 .000

Box 13.3. Regression of RT on Log U.

Figure 13.5. SPSS’s Curve Estimation Dialogue

Box.

Using COMPUTE to Transform Predictors

Box 13.3 shows the regression of RT on log Uncertainty scores that were created by the

COMPUTE statement on the

first line. Various measures

from the regression analysis

indicate a stronger relation

between RT and log U than

between RT and U. For

example, R2 = .9252, as seen

previously in Figure 13.4,

instead of .8803, indicating

that an additional 4.5% of the

variability is predicted by log

U. This increase is due to the

compression of the high end

of U. Note in Figure 13.4 that the points, which represent levels 1 to 8 on U, become closer

along the horizontal axis to the immediately preceding point as log U increases. For example,

log(8) - log(7) = .903 - .845 = .058, whereas log(2) - log(1) = .301 - 0 = .301. The distance

between 7 and 8 on the log scale is about 1/5th the distance between 1 and 2 on the log scale. On

the original scale, the distance between 7

and 8 equalled the distance between 1 and

2.

Using SPSS’s Curve Estimation

Procedure

In addition to its standard Linear

Regression option, SPSS provides a Curve

Estimation procedure that can

automatically conduct analyses for certain

kinds of transformations. The Curve


Estimation Dialogue box shown in Figure 13.5 can be accessed by the following menu

commands: Analyze | Regression | Curve Estimation. In Box 13.5, RT has been identified as the

Dependent variable, U as the Independent or predictor variable, and the Logarithmic

transformation has been selected. Other selected options are Display ANOVA table, Include

constant in equation, and Plot models. Clicking Ok runs the analysis.

The output for

the Curve Estimation

analysis is shown in

Box 13.4, along with

the syntax commands

equivalent to the

options in Figure 13.5.

Note the many

similarities to our

earlier analysis using computed logarithm scores, including: R2 = .92519, F = 371.04120, t =

19.262, intercept = 298.766251.

There is one notable difference between the results in Box 13.3 and 13.4, the value of the

slope for U. Specifically, B = 41.7149 in Box 13.4 versus B = 96.0521 in Box 13.3. This

difference in slopes arises from the different bases used to calculate the logarithms in the two

analyses. We computed logu = lg10(u), which uses 10 as the base for the logarithm. To

illustrate, lg10(2) = .30103 because 10.301 = 2.00. The Curve Estimation procedure, however,

computes what are called natural logarithms, using e = 2.71828 as the base. The logarithm of 2 to

the base e is .69315; that is, 2.71828.693 = 2.00. Because the natural logarithms are .69315/.30103

= 2.3026 times larger, the slope for the base 10 logarithms is 96.0521/41.7149 = 2.3026 times

larger. To make the two analyses equivalent, we would just need to change the compute in Box

13.3 to: COMPUTE logu = ln(u); ln is SPSS’s way of specifying natural logarithms. Under

some circumstances, it could be meaningful to specify logarithms using other bases (e.g., base 2 logarithms).

CURVEFIT /VARIABLES=RT WITH U /CONSTANT /MODEL=LOGARITHMIC /PRINT ANOVA /PLOT FIT.


DF Sum of Squares Mean SquareRegression 1 24106.758 24106.758Residuals 30 1949.117 64.971F = 371.04120 Signif F = .0000

-------------------- Variables in the Equation --------------------Variable B SE B Beta T Sig TU 41.714904 2.165610 .961870 19.262 .0000(Constant) 298.766251 3.204860 93.223 .0000

Box 13.4. Curve Estimation Analysis.


280

300

320

340

360

380

400

1 2 3 4 5 6 7 8

U

Observed

Logarithmic

RT

Figure 13.6. Plot of Logarithmic Function.

Figure 13.6 shows the plot of the logarithmic function to the data. The fit using the

logarithm of u generates

predicted values (the points on

the line) that are much closer to

the observed values than does a

straight linear equation. Like the

polynomial regression, the use of

logarithms for the predictor

scores allows the slope to be

steeper when U is small and

become gradually shallower as U

increases. Unlike the polynomial

fit, however, the predicted value

will not start to decrease if U is

extrapolated to higher values.

Finally, note in Figure

13.5 that one of the options on the Curve Estimation procedure is to do Linear and Quadratic fits

to the data. These options allow users to conduct polynomial regressions equivalent to that

discussed earlier.

Discussion of Transformations

People sometimes object to transformations of data because numbers are thought of as

having an absolute meaning that should not be violated. For example, how can a log

transformation change 1, 10, and 100 into 1, 2, and 3 without violating some aspect of the

original data? Such questions are important and researchers should consider them when

transforming scores. But carried to extreme, these concerns fail to appreciate that numbers can

be somewhat arbitrary with respect to the particular psychological or physical dimension being

measured. People expressing these concerns also fail to appreciate that some commonly used

scales are in fact transformations from the original scale; the decibel scale of loudness, for

example, is a logarithmic scale rather than a scale of equal intervals of loudness.


And often numerical scores depend upon the composition of the measure. Consider a test

of arithmetic ability with 30 questions. If the test included 10 easy questions, 10 medium

questions, and 10 difficult questions each worth one mark, then students would fall into four

clusters: those with scores around 0, those with scores around 10, those with scores around 20,

and those with scores around 30. People would be spaced out fairly evenly and the differences

between the groups would be about the same (and meaningfully so). But imagine a second test

with 14 easy questions, 14 medium questions, and only 2 difficult questions. Now people would

fall at 0, 14, 28, or 30 instead of 0, 10, 20, and 30. But the difference between 28 and 30 on the

second test represents the same difference in ability as the difference between 20 and 30 on the

first test. Such complications mean that numbers should not be treated absolutely and that they

can often be transformed without violating the data. In this example, a transformation that

expanded the difference between 28 and 30 relative to differences at the lower end of the scale

might be appropriate.

The malleable nature of numbers is shown clearly in cases where equally acceptable

alternative measures lead to somewhat different conclusions. Consider 3 subjects who are given

as long as necessary to complete 20 arithmetic problems. Subjects 1, 2, and 3 take 10, 20, and 30

minutes, respectively. Using time as the dependent measure, it took subjects an average of 20

minutes to complete the problems, subject 2 is exactly at average, and subjects 1 and 2 are above

and below subject 2 and the average by the same absolute amount (10 minutes).

Rather than using time as the dependent variable, the researchers could report the data in

terms of the rate or speed with which the problems were solved (i.e., #problems / time). This

new measure produces scores for subjects 1, 2, and 3 of 2.000 (20/10), 1.000 (20/20), and .667

(20/30), respectively. That is, subjects 1, 2, and 3 solved 2, 1, and .667 problems per minute.

The mean of these scores is 1.222, approximately 1 and 1/5th items per minute. But notice that

subject 2 is no longer at the average; subject 2 only solved 1 item per minute, whereas the mean

was 1.222 items per minute. Moreover, subjects 1 and 3 are no longer equal distance from

subject 2 (2 - 1 = 1 > 1 - .667 = .333) or from the mean (2 - 1.222 = .778 > 1.222 - .667 = .555),

which was the case for the time scores.


0

1

2

10 20 30

Time to Solve 20 Problems

Figure 13.7. Time and Rate Measures of

Performance.

NUM STOP RES.NUM NUM2 NUMROOT 1 5 -15.36962 1.00 1.00 1 7 -13.36962 1.00 1.00 2 18 -4.51008 4.00 1.41 2 23 .48992 4.00 1.41 4 46 19.20901 16.00 2.00 4 40 13.20901 16.00 2.00 8 38 2.64718 64.00 2.83 8 45 9.64718 64.00 2.83 16 50 -2.47648 256.00 4.00 16 43 -9.47648 256.00 4.00

Box 13.5. Data from Imitation Study.

These differences between the two

measures occur because the transformation to

rate is a nonlinear transformation, as shown in

Figure 13.7. Note that the rate measure on

the vertical axis decreases more as time

increases from 10 to 20 than as time increases

from 20 to 30. Similar nonlinear

transformations are legitimate in many other

circumstances, although the acceptability of

alternative scales may not be as obvious as for

measures of time and rate.

Neither the time nor the rate scale is more correct than the other, since both are sensible

measures for these data. It is equally reasonable to ask how long it took to solve the 20 problems

(i.e., the time measure) and how many problems were solved in each minute (i.e., the rate

measure). The two measures just don't provide perfectly equivalent results because of their

nonlinear relationship to one another.

IMITATION STUDY

To examine the relation between

number of people performing an action

and the percent of passers-by imitating

the action, social psychologists had

varying numbers of confederates (NUM

= 1, 2, 4, 8, or 16) stand on a street

looking up. The experiment was

performed twice on different days and at

different locations. The percent of passers-by who stopped (STOP) was determined for each

level of NUM. The 10 observations are shown in Box 13.5, along with several variables created

during the analyses.


Figure 13.8. Scattergram of Stopping as a

Function of Number of Observers.

Figure 13.9. Plot of Residuals from Linear

Regression with Predictor.

The researchers hypothesized that

the effect of increased numbers of

bystanders would be larger at lower levels

than at higher levels. That is, increasing the

number of observers from 2 to 3 was

expected to have a larger effect than the

increase from 12 to 13 observers.

Consistent with prediction, the plot in

Figure 13.8 shows that the relation between

STOP and NUM is definitely nonlinear.

The effect of number of observers appears

to level off at 4 observers, with few

additional stoppers as the observers

increased from 4 to 16. Figure 13.8 shows

that the linear regression (dashed line) overpredicts Stopping at low and high values of the

predictor, and underpredicts stopping in the middle range of the predictor.

The nonlinear nature of the relationship is

shown even more clearly in Figure 13.9, which

plots residuals from a simple regression of STOP

on NUM (see Box 13.6 for the analysis). The

mean for residuals is zero, since the sum of

residuals above and below the best-fit straight line

cancel out. The mean of 0 is shown in Figure

13.9 by the dashed line. For a linear relation, the

residuals should scatter randomly about this

average value. Instead, the residuals in this study

vary systematically, curving from below to above

the mean and then back below as NUM increases.

This curvilinear pattern indicates a nonlinear


REGR /DEP=stop /ENTER num /SAVE=RESID(res.num)

Multiple R .73288R Square .53712

DF Sum of Squares Mean SquareRegression 1 1363.47110 1363.47110Residual 8 1175.02890 146.87861F = 9.28298 Signif F = .0159

Variable B SE B Beta T Sig TNUM 2.14046 .70253 .73288 3.047 .0159(Constant) 18.22917 5.80170 3.142 .0138

COMP num2 = num*numREGR /DEP = stop /ENTER num num2

Multiple R .89777R Square .80599 > R²S.N = .53712


Variable B SE B Beta T Sig TNUM2 -.37935 .12179 -2.29977 -3.115 .0170NUM 8.68422 2.15645 2.97344 4.027 .0050(Constant) 3.52941 6.19649 .570 .5868

Box 13.6. Simple and Polynomial Regression for Imitation Study.

relation between NUM and STOP, which is consistent with the impression from the original plot

in Figure 13.8. The solid line in Figure 13.9 demonstrates that there is a considerable curvilinear

relationship with Number of Observers after the linear effect is removed. This line was added in

SPSS’s chart editor by requesting a polynomial fit to the residuals.

Box 13.6 shows the simple and polynomial regression of STOP on NUM and NUM2.

The prediction equation with both NUM and NUM2 is clearly an improvement over the simple

regression with NUM alone. R2 has increased dramatically, as reflected in the significant t for

NUM2. Other statistics could have been requested that would also have confirmed the significant

nonlinear effect (e.g., FChange, part r).

The

best-fit

nonlinear

equation is: �

= 3.53 +

8.68N -

.38N2, which

can be

rewritten as �

= 3.53 +

(8.68 -

.38N)N. This

second

version

shows how

the

coefficient

for the final N varies across different values of N. For NUM = 1, � = 3.53 + (8.68 -.38×1)N =

3.53+8.3N = 11.83, whereas, for NUM = 16, � = 3.53 + (8.68 -.38×16)N = 3.53+2.6N = 45.13.

Number of bystanders is multiplied by 8.3 for NUM=1 and by only 2.6 for NUM=16. As noted


Figure 13.10. Quadratic Fit to Stopping Data.

earlier, the actual slope at any given point on x could be obtained by calculating b1 + 2 x b2 x X.

The nonlinear nature of the

polynomial equation permits a better fit

to the data than the linear equation, as

shown in Figure 13.10. The linear

equation in Figure 13.8 predicts too

many passers-by will stop at the low and

high ends of NUM, and that too few will

stop at moderate levels of NUM. It is

these systematic deviations from the

predicted values that were plotted in

Figure 13.9. The polynomial predictions

bend with the data and come much closer

to the observed values, as expected given

the larger R2.

Although the nonlinear equation is an improvement for the fitted values, Figure 13.8

suggests some possible difficulties for the model with N and N2, as noted earlier. Specifically,

the predicted value from the nonlinear equation actually declines somewhat from NUM = 8 to

NUM = 16, and this decline would be even more marked if higher levels of NUM were

examined. It seems unlikely, however, that the percent of people stopping would actually

decrease as crowd size increased beyond 16. The nonlinear equation using just NUM and NUM2

would therefore provide an increasingly poor fit for higher numbers of onlookers.

Instead of polynomial regression, the researchers could transform the predictor variable

NUM, perhaps using a square root transformation (power of .5 or 1/2) to compress the upper

values of NUM (i.e., 1, 2, 4, 8, and 16 become 1, 1.414, 2, 2.828, and 4). Or the logarithm to the

base 2 might also be a good choice: 20 = 1, 21 = 2, 22 = 4, 23 = 8, 24 = 16.


COMP numroot = sqrt(num) /* Compresses number scaleREGR /DEP = stop /ENTER numroot

Multiple R .82417R Square .67925 > R²S.N = .53712

Adjusted R Square .63916Standard Error 10.08849


Variable B SE B Beta T Sig TNUMROOT 12.27631 2.98257 .82417 4.116 .0034(Constant) 3.89638 7.42653 .525 .6140

Box 13.7. Square Root Transformation of NUM.

Box 13.7 shows the regression analysis for the square root transformation. Although R2

for the �NUM is an improvement over the R2 for NUM alone, the transformed variable does not

achieve as good a fit to the data as the polynomial regression. The researchers might try other

meaningful transformations, such as logarithmic, to see if any provides an adequate depiction of

the data.

Box 13.8

shows that in fact the

logarithmic

transformation with

base = 2 does a much

better job than the

square root

transformation. The

first line computes the transformed scores. To compute logarithms to any desired base, simply

divide lg10(x) by lg10(base). Now our R2 becomes .820, even better than the value we obtained

using polynomial regression, although the transformation only uses a single (albeit transformed) predictor.

COMP log2num = lg10(num)/lg10(2).REGRE /DEP = stop /ENTER log2num.



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 11.100 4.143 2.679 .028 log2num 10.200 1.691 .905 6.031 .000

Box 13.8. Logarithm (Base 2) Transformation.


0

10

20

30

40

50

0 5 10 15

NUM

Observed

Linear

Logarithmic

STOP

Figure 13.11. Linear and Logarithmic

Models.

Box 13.9

shows the results of

the linear and

logarithmic fits to the

stopping study using

the Curve Estimation

(CURVEFIT) in

SPSS. The

logarithmic

regression clearly

provides the better fit

as indicated by R2, F,

or the observed p

value for the fit. This

analysis could also

have been requested

via menus: Analyze | Regression | Curve Estimation.

A plot of both equations appears in Figure

13.11 and very clearly shows the marked difference

between the fit of the linear and logarithmic models.

The logarithmic model allows for a change in the

slope as number of viewers increases, and this change

in slope more closely corresponds to the pattern

demonstrated by participants in this study. That is, the

likelihood of a passerby stopping increases quite

markedly as number of viewers initially increases, but

eventually number of viewers reaches some level

above which there is a much diminished influence on

passersby. Note as well that predictions from the logarithmic fit flatten out at upper levels of

CURVEFIT /VARIABLES=STOP WITH NUM /CONSTANT /MODEL=LINEAR LOGARITHMIC /PRINT ANOVA /PLOT FIT.

Dependent variable.. STOP Method.. LINEAR



-------------------- Variables in the Equation --------------------Variable B SE B Beta T Sig T

NUM 2.140457 .702527 .732883 3.047 .0159(Constant) 18.229167 5.801696 3.142 .0138

Dependent variable.. STOP Method.. LOGARITH

Listwise Deletion of Missing DataMultiple R .90537R Square .81970Adjusted R Square .79716Standard Error 7.56389


-------------------- Variables in the Equation --------------------Variable B SE B Beta T Sig TNUM 14.715489 2.440085 .905371 6.031 .0003(Constant) 11.100000 4.142916 2.679 .0280

Box 13.9. Linear and Quadratic Fit for Stopping Data.


NUM, but do not decrease, as was the case for the quadratic regression.

CONCLUSIONS

Chapter 13 has demonstrated how MR methods can be used to analyze nonlinear relations

between a single predictor and a criterion. The presence of nonlinear relations can be determined

by plotting original or residual scores against the predictor variable. These plots will reveal any

systematic deviations from a linear relation, although regression analyses with polynomial or

transformed predictors may be necessary to determine whether the deviations are significant.

The procedures discussed here all made use of basic linear regression, but with accommodations

(i.e., additional polynomial terms, transformations) that allowed for nonlinear relationships. In

addition to the techniques discussed here, there are more advanced ways to analyze nonlinear

relationships that actually allow researchers to fit truly nonlinear equations to data.

Categorical Predictors and Regression 14.1

CHAPTER 14

CATEGORICAL PREDICTORS AND REGRESSION

Multiple Regression and Categorical Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Categorical Predictor with k = 2 (p = 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Categorial Predictors with k > 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

ANOVA Example for k = 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Indicator Variables as Patterns Correlated with Y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Two Groups: k = 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Three Groups: k = 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Age Effects on Memory Independent of Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Age Effects on Memory Accounted for by Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Note 14.1:

Alternative Indicator Variables for Treatment Study . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Note 14.2:

Table of Indicator Variables (IVs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25


Although multiple regression is generally presented as a way to analyze non-experimental

datasets involving numerical predictors, the techniques are equally well-suited to the analysis of

categorical predictors, either from experimental or non-experimental studies. Categorical

predictors have levels that differ qualitatively from one another; hence, numbers assigned to the

different levels of the predictor are entirely or partly arbitrary (e.g., Religion: 1 = Roman Catholic,

2 = Protestant, 3 = Other; Gender: 1 = Male, 2 = Female; Treatment Group: 1 = Control, 2 =

Treatment A, 3 = Treatment B). This chapter demonstrates how multiple regression

accommodates categorical predictors, a topic touched on briefly in an earlier chapter and

elaborated further in the second half of the course covering analysis of variance.

MULTIPLE REGRESSION AND CATEGORICAL PREDICTORS

The standard statistical approach to categorical predictors is to determine whether the

means on the criterion (or dependent) variable differ significantly from one another using either a

t-test or analysis of variance (ANOVA). ANOVA may be followed by additional tests to make

comparisons between specific groups. ANOVA was discussed briefly in an earlier chapter, and

full descriptions are available elsewhere, including later sections of this text. This chapter

considers only the case of a single categorical predictor, beginning with categorical variables with

only two levels (i.e., two conditions) and then more than two levels of a single categorical

variable.

Categorical Predictor with k = 2 (p = 1)

ANOVA terminology uses k to indicate the number

of levels of a predictor variable (also known as a factor in

ANOVA). When there are two groups being compared, k =

2. Such datasets can also be analyzed using t-tests that are

equivalent to the corresponding ANOVA. Box 14.1 shows

data from a comparison of verbal ability scores for males

and females. There were 8 subjects in each group. Previous

research generally shows a difference favouring females on

verbal ability tests.

Males Females31 3137 4530 3631 3124 2827 4133 3632 34

Box 14.1. Verbal Ability Scores

for Males and Females.


Box 14.2 shows the independent groups t-test and one-way ANOVA results for these data

(the data for these analyses were input as pairs of scores, with the gend score being 1 for males

and 2 for females, and the verb score being the verbal ability scores). Using non-directional (i.e.,

two-tailed) tests, the difference is only marginally significant (note that p = .076 for the

independent groups t and for the ANOVA F statistics). If a one-tailed test were deemed

appropriate, then p = .076/2 = .038, and the difference would be judged to be significant. The

equivalence of t and F can also be demonstrated by squaring t to obtain F, or taking the square

root of F to obtain t; that is, t2 = F. The critical values of t and F (assuming a non-directional test

or appropriate selection of F) would also be equivalent using the same transformation.

The df for the numerator of the F test (i.e., df between groups) is k - 1 = 2 - 1 = 1, which is

why the F and t are equivalent. No such equivalence is possible when there are more than two

groups (i.e., when dfNumerator > 1 for F). The fact that df = 1 for the F test also provides some

indication of the kind of regression analysis that would be necessary to duplicate these results;

specifically, a single predictor would suffice, since df for FRegression is p, the number of predictors.

The formula for calculating t and F for this design were presented in earlier chapters. For

the t-test, t = (y�1 - y�2) / SQRT(sp2(1/n1 + 1/n2)), where sp

2 = (SS1 + SS2)/(n1 + n2 - 2). For the F

test, F = MSBetween / MSWithin, where MSWithin = sp2 as computed for the t-test, and MSBetween = njs y�

2,

where s y�2 = �(y�j - y�G)2 / (k - 1). With more than two groups, sp

2 = �SSj / (N - k). Although not

TTEST /GROUP = gend /VARI = verb.

GEND N Mean Std. Deviation Std. Error Mean VERB 1.0000 8 30.625000 3.8890873 1.3750000 2.0000 8 35.250000 5.5997449 1.9798088

Levene's Test t-test for Equality of Means Equal Variances

F Sig. t df Sig. Mean Std. Error (2-tailed) Difference Difference VERB Equal variances 1.079 .317 -1.919 14 .076 -4.625000 2.4104497

ONEWAY verb BY gend.

Sum of Squares df Mean Square F Sig. Between Groups 85.563 1 85.563 3.682 .076 Within Groups 325.375 14 23.241 Total 410.938 15

Box 14.2. T-test and ANOVA Results for Verbal Ability Study.


emphasized earlier, SSWithin in the ANOVA summary table is SS1 + SS2 = �SSj. SSBetween is nj�(y�j

- y�G)2, with df = k - 1.

Box 14.3 shows the comparable regression analysis. It is actually very straightforward

because we simply regress the verbal ability scores (verb) on the gender scores (gend). Note first

that the values for the F and t tests for the regression analysis duplicate those from the

independent groups t-test and the ANOVA. Moreover, p = .076 for both F and t in Box 14.3.

The reason for the equivalence between the regression analyses and t-tests or ANOVA on

the differences between means is also apparent. The regression coefficient for Gender is 4.625,

REGRESS /VARI = verb gend /DEP = verb /ENTER /SAVE PRED(prdv.g) RESI(resv.g).



Coefficients(a) Unstandardized Coefficients Standardized Coefficients t Sig. Model B Std. Error Beta 1 (Constant) 26.000 3.811 6.822 .000 GEND 4.625 2.410 .456 1.919 .076


LIST. SUBJ GEND VERB PRDV.G RESV.G 1.0000 1.0000 31.0000 30.62500 .37500 2.0000 1.0000 37.0000 30.62500 6.37500 3.0000 1.0000 30.0000 30.62500 -.62500 4.0000 1.0000 31.0000 30.62500 .37500 5.0000 1.0000 24.0000 30.62500 -6.62500 6.0000 1.0000 27.0000 30.62500 -3.62500 7.0000 1.0000 33.0000 30.62500 2.37500 8.0000 1.0000 32.0000 30.62500 1.37500 9.0000 2.0000 31.0000 35.25000 -4.2500010.0000 2.0000 45.0000 35.25000 9.7500011.0000 2.0000 36.0000 35.25000 .7500012.0000 2.0000 31.0000 35.25000 -4.2500013.0000 2.0000 28.0000 35.25000 -7.2500014.0000 2.0000 41.0000 35.25000 5.7500015.0000 2.0000 36.0000 35.25000 .7500016.0000 2.0000 34.0000 35.25000 -1.25000

Box 14.3. Regression Analysis for Verbal Ability Study.


the difference between the means for the two groups. This occurs because there is exactly a one-

unit difference between the numerical codes for the two groups; that is, Males = 1 on gend and

Females = 2. The PRDV.G scores listed below the regression analysis show this basis for the

correspondence; specifically, the predicted score for the 8 males is 30.625, M for Males, and the

predicted score for the 8 females is 35.250, M for Females, a difference of 4.625 units. Since the

predicted scores are the group means, SSRegression represents variation due to the differences

between the group means, which is equal to SSBetween in the ANOVA and has the same df. The

variability in the RESV.G scores represents deviations from the predicted scores (i.e., the group

means), and hence is equivalent to SSWithin in ANOVA or SS1 + SS2 in the t-test.

Although we used the original Gender scores as our predictor (i.e., 1 and 2), we could

actually have recoded these scores to any number of alternatives and obtained the equivalent

statistical results (see Box 14.4) as long as members of group 1 were coded as one value and

members of group 2 as a second value. If we had used 0 for Males and 1 for Females, for

RECODE gend (1 = 0) (2 = 1) INTO genddum.REGRE /DEP = verb /ENTER genddum.



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 30.625 1.704 17.968 .000 genddum 4.625 2.410 .456 1.919 .076

RECODE gend (1 = -1) (2 = +1) INTO gendeff.REGRE /DEP = verb /ENTER gendeff.



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 32.938 1.205 27.329 .000 gendeff 2.313 1.205 .456 1.919 .076

Box 14.4. Alternative Indicator Variables (Predictors) for Gender.


example, then the intercept would have been MMales and the slope would have been MFemales - MMales

(the same slope as in Box 14.3). If we had used -1 for Males and +1 for Females, then the

intercept would have been the overall Mean for the two groups (called the grand Mean in

ANOVA terminology), and the slope would have been the deviation of the group Means from the

grand Mean. Because the denominator values would also change proportional to the change in the

numerators, the tests of significance are equivalent to those in Box 14.3 (see Box 14.4). These

tests would be the same using any of a number of alternative ways to code the gender predictor.

This example illustrates that regression can easily be used to analyze categorical variables

with only two groups (i.e., k = 2) because a single predictor variable encodes the difference

between the two groups. The situation becomes slightly more complex with more than two

groups, although still relatively simple given our understanding of multiple regression. The main

difference for k > 2 is that more predictors are necessary; specifically, k - 1 predictors are needed

to perform regression analyses equivalent to analysis of variance on k group means. Predictors

that encode differences between categorical groupings are often called indicator variables.

Categorial Predictors with k > 2

The added complexity with k > 2 can be derived from our earlier observation that the

dfRegression (i.e., p) must equal dfNumerator for ANOVA, which is k - 1. This means that researchers

need k - 1 predictors to duplicate ANOVA on k groups (i.e., 2 predictors for k = 3, 3 predictors for

k = 4, and so on). In short, there will be p = k - 1 indicator variables for analysis of the differences

between k groups, and hence the dfs for the numerators of regression and anova will be

equivalent.

Box 14.6 shows data for a comparison of verbal ability scores of 15 children in three grade

levels (grades 1, 2, and 3), along with some scores generated for the purpose of and resulting from

the multiple regression analysis. We would expect that verbal ability would differ across grade

levels, presumably with scores getting higher as children advance in grade.


Box 14.5 shows the standard ANOVA results for these data. The means for Grades 1 (M

= 22.333), 2 (M = 28.500), and 3 (M = 43.333) differ significantly, F(2, 15) = 37.162, p = .000.

Verbal ability

scores clearly

increase with

Grade level,

although the

increase from

grade 1 to 2 is

considerably

smaller than the

increase from grade 2 to 3 (i.e., the relationship between verbal ability and grade is not linear, as

revealed in a subsequent analysis).

To duplicate these ANOVA results, we will need to use k - 1 = 2 predictors or indicator

variables. There are many possible ways to generate two predictors that will produce the overall F

and related statistics. There are several advantages to using indicator variables that have a mean

of 0 (i.e., the sum of the predictor values is 0), and there are also advantages to using predictors

that are uncorrelated with one another (i.e., they are orthogonal in ANOVA terminology). For

example, we could compare Grade 1 children with Grade 2 children using one predictor (p1 = -1

+1 0 for Grades 1, 2, and 3, respectively) and compare Grade 1 children with Grade 3 children

using the other predictor (p2 = -1 0 +1 for Grades 1, 2, and 3, respectively). These predictors

clearly sum to 0; however, they are not uncorrelated. Alternatively, we could use predictor one to

compare Grade 1 children with Grades 2 and 3 combined (i.e., p1 = -2 +1 +1 for Grades 1, 2, and

3, respectively) and use predictor two to compare Grade 2 children Grade 3 (i.e., p2 = 0 -1 +1, for

Grades 1, 2, and 3, respectively). Here the two predictors are uncorrelated. A third type of

categorical predictor for three groups reflects linear (-1, 0, +1) and quadratic (-1, +2, -1) patterns

in the data. These are called polynomial coefficients and are also independent of one another.

ONEWAY verb BY grade /STAT = DESCR.

N Mean Std. Std. 95% Confidence Interval for Mean Deviation Error Lower Bound Upper Bound 1.0000 6 22.333333 3.8297084 1.5634719 18.314301 26.352366 2.0000 6 28.500000 4.9295030 2.0124612 23.326804 33.673196 3.0000 6 43.333333 4.1793141 1.7061979 38.947412 47.719255

Total 18 31.388889 9.9418242 2.3433104 26.444936 36.332842


Box 14.5. Verbal Ability by Grades ANOVA Results.


Box 14.6 shows a regression analysis using polynomial predictors. Polynomial predictors

examine the linear and non-linear components of the categorical predictor and hence are only

appropriate when there is a meaningful ordering of the levels (as is the case here for grade level).

The linear predictor for three groups is -1, 0, +1 for Grades 1, 2, and 3, respectively. The non-

RECODE grade (1 = -1) (2 = 0) (3 = +1) INTO lin.RECODE grade (1 = -1) (2 = +2) (3 = -1) INTO qua.REGRESS /VARI = verb lin qua /DEP = verb /ENTER /SAVE PRED(prdv.lq) RESI(resv.lq).



Coefficients(a) Unstandardized Coefficients Standardized Coefficients t Sig. Model B Std. Error Beta 1 (Constant) 31.389 1.022 30.705 .000 LIN 10.500 1.252 .887 8.386 .000 QUA -1.444 .723 -.211 -1.998 .064


LIST. SUBJ GRADE VERB LIN QUA PRDV.LQ RESV.LQ 1.0000 1.0000 24.0000 -1.0000 -1.0000 22.33333 1.66667 2.0000 1.0000 26.0000 -1.0000 -1.0000 22.33333 3.66667 3.0000 1.0000 25.0000 -1.0000 -1.0000 22.33333 2.66667 4.0000 1.0000 18.0000 -1.0000 -1.0000 22.33333 -4.33333 5.0000 1.0000 24.0000 -1.0000 -1.0000 22.33333 1.66667 6.0000 1.0000 17.0000 -1.0000 -1.0000 22.33333 -5.33333 7.0000 2.0000 30.0000 .0000 2.0000 28.50000 1.50000 8.0000 2.0000 27.0000 .0000 2.0000 28.50000 -1.50000 9.0000 2.0000 21.0000 .0000 2.0000 28.50000 -7.5000010.0000 2.0000 27.0000 .0000 2.0000 28.50000 -1.5000011.0000 2.0000 30.0000 .0000 2.0000 28.50000 1.5000012.0000 2.0000 36.0000 .0000 2.0000 28.50000 7.5000013.0000 3.0000 49.0000 1.0000 -1.0000 43.33333 5.6666714.0000 3.0000 42.0000 1.0000 -1.0000 43.33333 -1.3333315.0000 3.0000 39.0000 1.0000 -1.0000 43.33333 -4.3333316.0000 3.0000 48.0000 1.0000 -1.0000 43.33333 4.6666717.0000 3.0000 40.0000 1.0000 -1.0000 43.33333 -3.3333318.0000 3.0000 42.0000 1.0000 -1.0000 43.33333 -1.33333

Box 14.6. Regression Analysis Using Polynomial Predictors.


linear (or quadratic) predictor for three groups is -1, +2, -1 for Grades 1, 2, and 3, respectively.

These contrasts sum to 0 and are uncorrelated. The RECODE commands in Box 14.5 generate the

linear (lin) and quadratic (qua) predictors.

The REGRESSION command regresses the dependent variable, verb, on the lin and qua

predictors, and saves the predicted and residual scores. Note first the many equivalencies between

the statistics for the overall regression and the previous ANOVA: FRegression = FANOVA, SSRegression =

SSBetween, SSResidual = SSWithin, and between the corresponding dfs and MSs. These correspondences

occur because the best-fit regression equation produces the group means as the predicted scores

(compare the PRDV.LQ column with the group means from the ANOVA), and the deviations

from the group mean as the residual scores.

Briefly, the tests of significance for the two predictors are also informative. Note that the

linear effect is highly significant (p = .000) and the quadratic (i.e., non-linear) effect is marginally

significant by a two-tailed test (p = .064). This suggests that there may be a significant non-linear

component to the effect of Grade on verbal ability (keep in mind the small n for this study,

something that works against finding a significant effect). In fact, using categorical predictors is a

third way to analyze non-linear predictors (in addition to the two methods discussed in the

previous chapter, namely, polynomial regression and transformations of predictors). Table T5 in

the Appendix shows coefficients that would be used to test linear and various non-linear

components for categorical variables with varying numbers of groups or levels.

ANOVA Example for k = 4

As a final

example of how

regression handles

categorical predictors,

consider a study in

which a Control Group

(Treat = 1) is being

compared with three

different treatment

LIST. SUBJ TREAT ASSERT P1 P2 P3 PRDA.123 RESA.123 1.0000 1.0000 29.0000 -3.0000 .0000 .0000 28.25000 .75000 2.0000 1.0000 31.0000 -3.0000 .0000 .0000 28.25000 2.75000 3.0000 1.0000 30.0000 -3.0000 .0000 .0000 28.25000 1.75000 4.0000 1.0000 23.0000 -3.0000 .0000 .0000 28.25000 -5.25000 5.0000 2.0000 34.0000 1.0000 -1.0000 -1.0000 32.00000 2.00000 6.0000 2.0000 27.0000 1.0000 -1.0000 -1.0000 32.00000 -5.00000 7.0000 2.0000 35.0000 1.0000 -1.0000 -1.0000 32.00000 3.00000 8.0000 2.0000 32.0000 1.0000 -1.0000 -1.0000 32.00000 .00000 9.0000 3.0000 26.0000 1.0000 -1.0000 1.0000 33.50000 -7.5000010.0000 3.0000 32.0000 1.0000 -1.0000 1.0000 33.50000 -1.5000011.0000 3.0000 35.0000 1.0000 -1.0000 1.0000 33.50000 1.5000012.0000 3.0000 41.0000 1.0000 -1.0000 1.0000 33.50000 7.5000013.0000 4.0000 44.0000 1.0000 2.0000 .0000 39.50000 4.5000014.0000 4.0000 37.0000 1.0000 2.0000 .0000 39.50000 -2.5000015.0000 4.0000 34.0000 1.0000 2.0000 .0000 39.50000 -5.5000016.0000 4.0000 43.0000 1.0000 2.0000 .0000 39.50000 3.50000

Box 14.7. Original and Derived Scores for Assertiveness Study.


conditions (Treat = 2, 3, and 4) for Assertiveness (ASSERT in the datafile), with the expectation

that all of the conditions will produce higher Assertiveness scores than the control group and that

treatment 4 will be better than the other two treatment conditions. The data for the four subjects

in each of the 4 groups are shown in Box 14.7, along with some scores generated for and by the

regression analysis in Box 14.9.

The ANOVA results are shown in Box 14.8. The overall difference among the four means

is significant, F(3, 12) = 3.999, p = .035, and the pattern of differences among the means (see the

Mean column in Box 14.8) is consistent with the predictions.

Box 14.9 shows the corresponding regression analysis, using predictors that map onto the

ONEWAY assert BY treat /STAT = DESCR.

N Mean Std. Std. 95% Confidence Interval for Mean Deviation Error Lower Bound Upper Bound 1.0000 4 28.250000 3.5939764 1.7969882 22.531181 33.968819 2.0000 4 32.000000 3.5590261 1.7795130 26.336795 37.663205 3.0000 4 33.500000 6.2449980 3.1224990 23.562815 43.437185 4.0000 4 39.500000 4.7958315 2.3979158 31.868762 47.131238 Total 16 33.312500 5.9185443 1.4796361 30.158730 36.466270


Box 14.8. ANOVA Results for Assertiveness Study.

RECODE treat (1 = -3) (2 = +1) (3 = +1) (4 = +1) INTO p1.RECODE treat (1 = 0) (2 = -1) (3 = -1) (4 = +2) INTO p2.RECODE treat (1 = 0) (2 = -1) (3 = +1) (4 = 0) INTO p3.REGRESS /VARI = assert p1 p2 p3 /DEP = anx /ENTER /SAVE PRED(prda.123) RESI(resa.123).



Unstandardized Coefficients Standardized Coefficients t Sig. Model B Std. Error Beta 1 (Constant) 33.313 1.170 28.476 .000 P1 1.687 .675 .510 2.499 .028 P2 2.250 .955 .481 2.356 .036 P3 .750 1.654 .093 .453 .658

Box 14.9. Regression Analysis for Assertiveness Study.


expected pattern of results. Specifically, predictor one (p1) compares the control group (treat =1)

to the three treatment groups; that is, p1 = -3 +1 +1 +1 for groups 1 to 4, respectively. Predictor

two (p2) compares the last treatment group (which was expected to do better) to the other two

treatment groups; that is, p2 = 0 -1 -1 +2 for groups 1 to 4, respectively. The third and final

predictor compares groups 2 and 3 to one another; that is, p3 = 0 -1 +1 0 for groups 1 to 4,

respectively.

The overall analysis in Box 14.9 agrees exactly with the ANOVA in Box 14.8, just as we

saw for the earlier analyses with k = 2 and 3. In addition, the significance tests for the individual

predictors confirm the predictions. The comparison between the Control group and the three

Treatment groups (i.e., p1) is significant, p = .028, as is the comparison between the last treatment

group and groups 2 and 3 (i.e., p2), p = .036. Alternative analyses for this study using different

indicator variables are shown in the Note 14.1 at the end of this chapter, and Note 14.2 shows

various alternative indicators for incorporating categorical predictors in multiple regression.

INDICATOR VARIABLES AS PATTERNS CORRELATED WITH Y

One can think about indicator variables, such as those generated for the preceding

examples, as patterns that researchers are looking for in the data. The unique effect of each

predictor represents the strength and significance of the relationship between the data and that pre-

defined pattern. Here we develop briefly this idea that indicator variables encode patterns as

numerical codes that can be correlated with our dependent variable, y.

Two Groups: k = 2

With only two groups, the idea of a “pattern” is limited to one group being greater or less

than another; that is, the two group means

differing significantly. Even so it is still

informative to see what happens when we

correlate y with our simple pattern scores (1

vs. 2, 0 vs. 1, -1 vs. +1). Box 14.10 shows the

correlation between verbal scores and the

gender indicator variable. Note first that the p

CORR verb gend /STAT = DESCR.

Mean Std. Deviation N VERB 32.93750000 5.234102916 16 GEND 1.50000000 .516397779 16

VERB GEND GEND Pearson .456 1 Correlation Sig. (2-tailed) .076 .

Box 14.10. Correlation with Gender ‘Scores”.


value for the correlation coefficient, p = .076, is exactly the same as the p reported by the

independent groups t-test and the equivalent analysis of variance. Those tests were in essence

tests of the correlation between verbal scores and the dichotomous gender variable. Second, a t or

F statistic for this correlation coefficient would equal the t and F for the difference between

means: tr = (.456 - 0) / SQRT((1 - .4562)/(16 - 2)) = 1.917, and Fr = (.4562/1) / ((1 - .4562)/(16 - 2)

= 3.675 = tr2. Third, SSRegression = .4562 x SSTotal = .4562 x (16 - 1) 5.23412 = 85.449 = SSBetween. In

short, the tests of the difference between the two means is essentially testing the strength and

significance of the correlation between y, the verbal ability scores, and the gender indicator

variable.

Three Groups: k = 3

Our second example involved verbal

ability scores across three grade levels. We

tested the results for linear and non-linear (or

quadratic) patterns in the data. The

coefficients for the linear indicator variable

were: -1 0 +1; that is, the linear indicator

variable was ordered and its values were equal

distances apart (defining a linear relationship). The quadratic coefficients were: -1 +2 -1, which

defines a curvilinear or quadratic effect (inverted u or an opposite u-shaped pattern).

Box 14.11 shows the correlations between our two indicator variables and the verbal

scores. The significance tests are not shown in Box 14.11 because the computed tests are not

appropriate. We want to know the significance of lin and qua in the multiple regression equation

with both predictors in the equation, not their separate effects when alone. Nonetheless, the

correlations in Box 14.11 do represent the unique contribution of each predictor in the earlier

regression analysis. The simple rs represent the unique contribution because the two indicator

variables are completely independent of each other; note that r = 0 between lin and qua in Box 14.11.

CORR verb lin qua /STAT = DESCR.

Mean Std. Deviation N VERB 31.38888889 9.941824243 18 LIN .00000000 .840168050 18 QUA .00000000 1.455213750 18

VERB LIN QUA LIN Pearson .887 1 .000 QUA Pearson -.211 .000 1

Box 14.11. Correlation of Scores with Indicator

Variables for the Three Grade Levels.


Figure 14.1. Fit of Data to Linear Coefficients.

Let us demonstrate the equivalence for lin. Lin accounts for .8872 × 1680.278 (i.e., SSTotal)

= 1321.991 units of variability in verbal

scores. Using MSError from the regression

analysis, this produces F = (1321.991/1)

/ 18.811 = 70.278, the square root of

which is 8.383, which is approximately

equal to tLin from the regression analysis.

Figure 14.1 shows the plot relating

verbal scores to the linear component of

the grade effect. Clearly the linear

pattern specified by the linear

coefficients accounts for considerable

variability in the verbal scores.

Figure 14.1 also reveals that there

is some systematic, albeit slight,

deviation from a strictly linear pattern.

Specifically, the increase in verbal scores from -1 to 0 (i.e., from grade 1 to grade 2) is not quite as

large as the increase from 0 to +1 (i.e., from grade 2 to grade 3). This deviation from the linear

pattern is captured by the quadratic coefficients and is almost significant, p = .064 in Box 14.5.


In a somewhat different fashion, Box 14.12 shows for the four-group study that each

indicator variable is sensitive to the degree of relationship between itself and the dependent

variable, assertiveness scores in this example. The analysis focusses on p1, which has values of -

3 1 1 1, and looks for a pattern of results in which the Control group (group 1) differs from the

three treatment groups. Note in particular that the simple r of .510 is identical to the part r

because the three predictors are independent, and that .5102 = .260 = R2Change, which is tested for

significance by either FChange for Model 2 or tP1. And because the indicator variables are

uncorrelated, R2 = .5102 + .0932 + .4812 = .500.

COMBINING CATEGORICAL AND NUMERICAL PREDICTORS

In the preceding examples, regression analysis was used to replicate results that could have

been produced using ANOVA. This nicely demonstrates the equivalence of regression (the

General Linear Model) and ANOVA. The primary benefit of accommodating categorical

REGRE /STAT = DEFA ZPP CHANGE /DESCR /DEP = assert /ENTER p2 p3 /ENTER p1.

Mean Std. Deviation N ASSERT 33.31250000 5.918544303 16 P2 .00000000 1.264911064 16 P3 .00000000 .730296743 16 P1 .00000000 1.788854382 16

ASSERT P2 P3 P1 P2 .481 1.000 .000 .000 P3 .093 .000 1.000 .000 P1 .510 .000 .000 1.000




Model Unstandardized Standardized t Sig. Correlations Coefficients Coefficients B Std. Error Beta Zero-order Partial Part ... 2 (Constant) 33.313 1.170 28.476 .000 P2 2.250 .955 .481 2.356 .036 .481 .562 .481 P3 .750 1.654 .093 .453 .658 .093 .130 .093 P1 1.687 .675 .510 2.499 .028 .510 .585 .510

Box 14.12. Simple and Part Correlations for k = 4 Study.


variables in regression analysis, however, is to allow researchers to include both categorical and

numerical predictors in their analysis. Here we illustrate using a study of the relationship between

age, exercise, and memory performance.

Age Effects on Memory Independent of Exercise

The first version of the study involved 24 participants from 3 age

levels (1 = Young, 2 = Middle Age, and 3 = Old). Box 14.13 shows

scores on age, exercise, and memory for the 24 participants. The

researchers were interested in whether the categorical (although ordered)

variable age was related to memory performance, and whether any

relationship between age and memory performance might be mediated by

the amount of exercise people engaged in. If exercise mediated the

relationship between age and memory, then using multiple regression to

control for differences in exercise should eliminate any differences in

memory performance across the three age groups. If exercise does not

mediate the relationship between age and memory, then controlling

statistically for exercise should have no effect on differences across ages

in memory performance.

Preliminary analyses were conducted to determine whether there

were indeed any differences across age in memory or exercise. These analyses were done using

both ANOVA and regression approaches, although the two analyses are equivalent as we saw

earlier in the chapter. Because age is an ordered variable, polynomial indicator variables (linear

and quadratic) were used for the regression analyses.

s age exe mem 1 1 22 33 2 1 16 41 3 1 24 47 4 1 19 44 5 1 22 42 6 1 14 37 7 1 24 48 8 1 25 43 9 2 24 4310 2 21 3511 2 20 4112 2 20 4513 2 17 3014 2 18 4915 2 17 3516 2 19 3617 3 18 2918 3 14 2919 3 23 3020 3 24 3521 3 20 3522 3 20 3223 3 21 4124 3 20 28

Box 14.13. Results for

Age and Memory

Study.


Box 14.14 shows the results of the analyses to determine whether memory performance

differs significantly across ages. The ANOVA demonstrates clearly that the three group means

differ significantly, F = 6.897, p = .005. The regression analysis using linear and quadratic

indicator variables replicates this analysis exactly. The reason is that using these (or other)

indicator variables generates the group means as the predicted values, as shown for the first two

subjects in each group at the bottom of Box 14.14.

ONEWAY mem BY age /STAT = DESCR.

N Mean Std. Std. 95% Confidence Interval for Minimum Maximum Deviation Error Mean Lower Bound Upper Bound 1 8 41.88 4.970 1.757 37.72 46.03 33 48 2 8 39.25 6.296 2.226 33.99 44.51 30 49 3 8 32.38 4.406 1.558 28.69 36.06 28 41

Total 24 37.83 6.499 1.327 35.09 40.58 28 49


RECODE age (1 = -1) (2 = 0) (3 = 1) INTO agelin.RECODE age (1 3 = -1) (2 = 2) INTO agequa.REGRESS /DEP = mem /ENTER agelin agequa /SAVE PRED(prdm.lq) RESI(resm.lq).



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 37.833 1.079 35.079 .000 agelin -4.750 1.321 -.610 -3.596 .002 agequa .708 .763 .157 .929 .364


LIST. s age exe mem agelin agequa prdm.lq resm.lq 1 1 22 33 -1.0000 -1.0000 41.87500 -8.87500 2 1 16 41 -1.0000 -1.0000 41.87500 -.87500... 9 2 24 43 .0000 2.0000 39.25000 3.7500010 2 21 35 .0000 2.0000 39.25000 -4.25000...17 3 18 29 1.0000 -1.0000 32.37500 -3.3750018 3 14 29 1.0000 -1.0000 32.37500 -3.37500...

Box 14.14. ANOVA for Memory and Equivalent Regression Analysis.


Box 14.15 shows the corresponding analyses for the relationship between age and

exercise. Here we see that exercise does not in fact differ significantly across the three age levels,

F = .306, p = .740 for both the ANOVA and regression analyses. Again, predicted scores are

equal to the group means, which explains the correspondence between the two analyses.

Together these analyses suggest that any differences across age in memory performance

probably are not due to differences in exercise. The rational is that there are no significant

ONEWAY exe BY age /STAT = DESCR.

N Mean Std. Std. 95% Confidence Interval for Minimum Maximum Deviation Error Mean Lower Bound Upper Bound 1 8 20.75 4.027 1.424 17.38 24.12 14 25 2 8 19.50 2.330 .824 17.55 21.45 17 24 3 8 20.00 3.071 1.086 17.43 22.57 14 24

Total 24 20.08 3.120 .637 18.77 21.40 14 25

Sum of Squares df Mean Square F Sig. Between Groups 6.333 2 3.167 .306 .740 Within Groups 217.500 21 10.357 Total 223.833 23

RECODE age (1 = -1) (2 = 0) (3 = 1) INTO agelin.RECODE age (1 3 = -1) (2 = 2) INTO agequa.REGRESS /DEP = exe /ENTER agelin agequa /SAVE PRED(prde.lq) RESI(rese.lq).

Model R R Square Adjusted R Std. Error of Square the Estimate 1 .168(a) .028 -.064 3.218

Model Sum of Squares df Mean Square F Sig. 1 Regression 6.333 2 3.167 .306 .740(a) Residual 217.500 21 10.357

Total 223.833 23

Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 20.083 .657 30.572 .000 agelin -.375 .805 -.100 -.466 .646 agequa -.292 .465 -.135 -.628 .537

Residuals Statistics(a) Minimum Maximum Mean Std. Deviation N Predicted Value 19.50 20.75 20.08 .525 24 Residual -6.750 4.500 .000 3.075 24

LIST s, age, exe agelin agequa prde.lq rese.lq. s age exe agelin agequa prde.lq rese.lq 1 1 22 -1.0000 -1.0000 20.75000 1.25000 2 1 16 -1.0000 -1.0000 20.75000 -4.75000... 9 2 24 .0000 2.0000 19.50000 4.5000010 2 21 .0000 2.0000 19.50000 1.50000...17 3 18 1.0000 -1.0000 20.00000 -2.0000018 3 14 1.0000 -1.0000 20.00000 -6.00000

Box 14.15. ANOVA and Equivalent Regression Analysis for Relation between Age and

Exercise.


differences across age in exercise that could potentially account for the memory differences. To

formally test the hypothesis, we conduct a multiple regression in which age and exercise are

included together as predictors of memory.

Box 14.16 shows the relevant analysis, with agelin and agequa entered together after

exercise, so that we can isolate the overall effect of age. FChange = 7.277, pChange = .004, indicating

that the differences in memory performance among the three groups remain significant even when

exercise is controlled. Note in the change statistics that the df for FChange = 2 because we entered

two predictors into the equation at once (agelin and agequa). This is one illustration of a situation

in which it makes sense to test the significance of the change when more than one predictor is

added to the equation.

The conclusion from this analysis is that there are indeed significant differences among the

ages in memory performance and these differences cannot be accounted for by differences in

exercise. Let us now change the data set “slightly,” and see how the analyses would differ.

REGRE /STAT = DEFA CHANGE /DEP = mem /ENTER exe /ENTER agelin agequa.




Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 23.806 8.500 2.801 .010 exe .698 .418 .335 1.669 .109

2 (Constant) 25.114 6.878 3.651 .002 exe .633 .339 .304 1.870 .076 agelin -4.512 1.255 -.579 -3.595 .002 agequa .893 .728 .199 1.227 .234

Box 14.16. Effect of Age Controlling for Exercise.


Age Effects on Memory Accounted for by Exercise

Box 14.17 shows the modified data set. In essence, as we will

see shortly, the change that has been introduced is to insert differences in

exercise across age levels. This change raises the possibility that

differences in memory performance across age may now become non-

significant when exercise is controlled in a multiple regression analysis.

First, we repeat the ANOVA and regression analyses for the relationship

between age and exercise to demonstrate that there are indeed significant

differences in exercise across the ages. We do not present here the

corresponding analyses for memory, which would be identical to those

presented in Box 14.14 (note that the memory data is exactly the same in

the two data sets; only the data for exercise has been modified for this

second variant of the study.

s age exe mem 1 1 22 33 2 1 16 41 3 1 24 47 4 1 19 44 5 1 22 42 6 1 14 37 7 1 24 48 8 1 25 43 9 2 21 4310 2 18 3511 2 17 4112 2 17 4513 2 14 3014 2 15 4915 2 14 3516 2 16 3617 3 12 2918 3 8 2919 3 17 3020 3 18 3521 3 14 3522 3 14 3223 3 15 4124 3 14 28

Box 14.17. Modified

Memory Data Set.

ONEWAY exe BY age /STAT = DESCR.

N Mean Std. Std. 95% Confidence Interval for Minimum Maximum Deviation Error Mean Lower Bound Upper Bound 1 8 20.75 4.027 1.424 17.38 24.12 14 25 2 8 16.50 2.330 .824 14.55 18.45 14 21 3 8 14.00 3.071 1.086 11.43 16.57 8 18

Total 24 17.08 4.190 .855 15.31 18.85 8 25


REGRESS /DEP = exe /ENTER agelin agequa /SAVE PRED(prde2.lq) RESI(rese2.lq).



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 17.083 .657 26.005 .000 agelin -3.375 .805 -.672 -4.195 .000 agequa -.292 .465 -.101 -.628 .537

LIST s, age, exe, mem, agelin, agequa, prde2.lq, rese2.lq.

s age exe mem agelin agequa prde2.lq rese2.lq 1 1 22 33 -1.0000 -1.0000 20.75000 1.25000 2 1 16 41 -1.0000 -1.0000 20.75000 -4.75000... 9 2 21 43 .0000 2.0000 16.50000 4.5000010 2 18 35 .0000 2.0000 16.50000 1.50000...17 3 12 29 1.0000 -1.0000 14.00000 -2.0000018 3 8 29 1.0000 -1.0000 14.00000 -6.00000

Box 14.18. Analyses of Relationship between Age and Exercise in Revised Data Set.


Box 14.18 shows the results of the (equivalent) ANOVA and regression analyses of the

relationship between age and exercise. Amount of exercise now decreases significantly from

20.75 to 16.50 to 14.00 as age level increases from young to middle-age to old. This difference is

highly significant. The critical question now is whether the difference across age in memory

performance will remain significant when we control for differences in exercise.

Box 14.19 shows the relevant results. Exercise is entered first and now there is no

significant change in prediction of memory performance when the two age predictors are added to

the equation (nor is either individual predictor significant in the Coefficient section of the

analysis).

The conclusion from this analysis is markedly different from that with the original data set.

Now we would conclude that exercise might indeed be the mediating factor in deteriorating

memory performance across ages. When we control statistically for exercise (i.e., examine the

effect of age independent of exercise), we no longer observe a relationship between age and

memory performance. Our conclusion must, of course, be tentative because this is a non-

experimental study and does not allow strong conclusions about the presence or absence of causal

relationships.

CONCLUSIONS

This chapter has briefly demonstrated how it is possible to accommodate categorical

REGRE /STAT = DEFA CHANGE /DEP = mem /ENTER exe /ENTER agelin agequa.




Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 21.575 4.587 4.704 .000 exe .952 .261 .614 3.645 .001

2 (Constant) 27.014 5.875 4.598 .000 exe .633 .339 .408 1.870 .076 agelin -2.612 1.693 -.335 -1.543 .138 agequa .893 .728 .199 1.227 .234

Box 14.19. Nonsignificant Age effect in Revised Memory Study.


predictors in multiple regression. Essentially, researchers define k - 1 predictors to represent,

ideally, meaningful comparisons among the k conditions. The capacity to include a mix of both

numerical and categorical predictors makes multiple regression an extremely general and

powerful data analytic technique, including the possibility of examining categorical predictors

controlling for relationships with other predictors that are numerical in nature.


NOTE 14.1:

ALTERNATIVE INDICATOR VARIABLES FOR TREATMENT STUDY

RECODE treat (1 3 4 = 0) (2 = 1) INTO dum12.RECODE treat (1 2 4 = 0) (3 = 1) INTO dum13.RECODE treat (1 2 3 = 0) (4 = 1) INTO dum14.REGRE /DESCR /DEP = assert /ENTER dum12 dum13 dum14 /SAVE PRED(prddum) RESI(resdum).

Mean Std. Deviation N assert 33.312500 5.9185443 16 dum12 .250000 .4472136 16 dum13 .250000 .4472136 16 dum14 .250000 .4472136 16

assert dum12 dum13 dum14 dum12 -.132 1.000 -.333 -.333 dum13 .019 -.333 1.000 -.333 dum14 .623 -.333 -.333 1.000



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 28.250 2.340 12.074 .000 dum12 3.750 3.309 .283 1.133 .279 dum13 5.250 3.309 .397 1.587 .139 dum14 11.250 3.309 .850 3.400 .005


LIST assert treat dum12 dum13 dum14 prddum resdum.

assert treat dum12 dum13 dum14 prddum resdum 29.0000 1.0000 .0000 .0000 .0000 28.25000 .75000 31.0000 1.0000 .0000 .0000 .0000 28.25000 2.75000 30.0000 1.0000 .0000 .0000 .0000 28.25000 1.75000 23.0000 1.0000 .0000 .0000 .0000 28.25000 -5.25000 34.0000 2.0000 1.0000 .0000 .0000 32.00000 2.00000 27.0000 2.0000 1.0000 .0000 .0000 32.00000 -5.00000 35.0000 2.0000 1.0000 .0000 .0000 32.00000 3.00000 32.0000 2.0000 1.0000 .0000 .0000 32.00000 .00000 26.0000 3.0000 .0000 1.0000 .0000 33.50000 -7.50000 32.0000 3.0000 .0000 1.0000 .0000 33.50000 -1.50000 35.0000 3.0000 .0000 1.0000 .0000 33.50000 1.50000 41.0000 3.0000 .0000 1.0000 .0000 33.50000 7.50000 44.0000 4.0000 .0000 .0000 1.0000 39.50000 4.50000 37.0000 4.0000 .0000 .0000 1.0000 39.50000 -2.50000 34.0000 4.0000 .0000 .0000 1.0000 39.50000 -5.50000 43.0000 4.0000 .0000 .0000 1.0000 39.50000 3.50000


RECODE treat (1 = -1) (2 = 1) (3 4 = 0) INTO eff12.RECODE treat (1 = -1) (3 = 1) (2 4 = 0) INTO eff13.RECODE treat (1 = -1) (4 = 1) (2 3 = 0) INTO eff14.REGRE /DESCR /DEP = assert /ENTER eff12 eff13 eff14 /SAVE PRED(prdeff) RESI(reseff).

Mean Std. Deviation N assert 33.312500 5.9185443 16 eff12 .000000 .7302967 16 eff13 .000000 .7302967 16 eff14 .000000 .7302967 16

assert eff12 eff13 eff14 eff12 .231 1.000 .500 .500 eff13 .324 .500 1.000 .500 eff14 .694 .500 .500 1.000



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 33.313 1.170 28.476 .000 eff12 -1.313 2.026 -.162 -.648 .529 eff13 .187 2.026 .023 .093 .928 eff14 6.188 2.026 .763 3.054 .010


LIST assert treat eff12 eff13 eff14 prdeff reseff.

assert treat eff12 eff13 eff14 prdeff reseff 29.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 .75000 31.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 2.75000 30.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 1.75000 23.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 -5.25000 34.0000 2.0000 1.0000 .0000 .0000 32.00000 2.00000 27.0000 2.0000 1.0000 .0000 .0000 32.00000 -5.00000 35.0000 2.0000 1.0000 .0000 .0000 32.00000 3.00000 32.0000 2.0000 1.0000 .0000 .0000 32.00000 .00000 26.0000 3.0000 .0000 1.0000 .0000 33.50000 -7.50000 32.0000 3.0000 .0000 1.0000 .0000 33.50000 -1.50000 35.0000 3.0000 .0000 1.0000 .0000 33.50000 1.50000 41.0000 3.0000 .0000 1.0000 .0000 33.50000 7.50000 44.0000 4.0000 .0000 .0000 1.0000 39.50000 4.50000 37.0000 4.0000 .0000 .0000 1.0000 39.50000 -2.50000 34.0000 4.0000 .0000 .0000 1.0000 39.50000 -5.50000 43.0000 4.0000 .0000 .0000 1.0000 39.50000 3.50000


RECODE treat (1 = -1) (2 = 1) (3 4 = 0) INTO or1v2.RECODE treat (1 2 = -1) (3 = 2) (4 = 0) INTO or12v3.RECODE treat (1 2 3 = -1) (4 = 3) INTO or123v4.REGRE /DESCR /DEP = assert /ENTER or1v2 or12v3 or123v4 /SAVE PRED(prdorth) RESI(resorth).

Mean Std. Deviation N assert 33.312500 5.9185443 16 or1v2 .000000 .7302967 16 or12v3 .000000 1.2649111 16 or123v4 .000000 1.7888544 16

assert or1v2 or12v3 or123v4 or1v2 .231 1.000 .000 .000 or12v3 .240 .000 1.000 .000 or123v4 .623 .000 .000 1.000

Model Summary(b) Model R R Square Adjusted R Std. Error of Square the Estimate 1 .707(a) .500 .375 4.6792984


Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 33.313 1.170 28.476 .000 or1v2 1.875 1.654 .231 1.133 .279 or12v3 1.125 .955 .240 1.178 .262 or123v4 2.062 .675 .623 3.054 .010


LIST assert treat or1v2 or12v3 or123v4 prdorth resorth.

assert treat or1v2 or12v3 or123v4 prdorth resorth 29.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 .75000 31.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 2.75000 30.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 1.75000 23.0000 1.0000 -1.0000 -1.0000 -1.0000 28.25000 -5.25000 34.0000 2.0000 1.0000 -1.0000 -1.0000 32.00000 2.00000 27.0000 2.0000 1.0000 -1.0000 -1.0000 32.00000 -5.00000 35.0000 2.0000 1.0000 -1.0000 -1.0000 32.00000 3.00000 32.0000 2.0000 1.0000 -1.0000 -1.0000 32.00000 .00000 26.0000 3.0000 .0000 2.0000 -1.0000 33.50000 -7.50000 32.0000 3.0000 .0000 2.0000 -1.0000 33.50000 -1.50000 35.0000 3.0000 .0000 2.0000 -1.0000 33.50000 1.50000 41.0000 3.0000 .0000 2.0000 -1.0000 33.50000 7.50000 44.0000 4.0000 .0000 .0000 3.0000 39.50000 4.50000 37.0000 4.0000 .0000 .0000 3.0000 39.50000 -2.50000 34.0000 4.0000 .0000 .0000 3.0000 39.50000 -5.50000 43.0000 4.0000 .0000 .0000 3.0000 39.50000 3.50000


NOTE 14.2:

TABLE OF INDICATOR VARIABLES (IVS)

Number of Groups (k) 2 3 4

Dummy Coding D1 D1 D2 D1 D2 D3 1 0 0 0 0 0 0

Group 2 1 1 0 1 0 0 3 0 1 0 1 0 4 0 0 1

Effect Coding E1 E1 E2 E1 E2 E3 1 -1 -1 -1 -1 -1 -1

Group 2 1 1 0 1 0 0 3 0 1 0 1 0 4 0 0 1

Orthogonal Coding O1 O2 O1 O2 O3 1 -1 -1 -1 -1 -1

Group 2 1 -1 1 -1 -1 3 0 2 0 2 -1 4 0 0 3

Polynomial Coding L Q L Q C 1 -1 1 -3 1 -1

Group 2 0 -2 -1 -1 3 3 1 1 1 -1 -3 4 3 1 1

Dummy Coding

One group coded 0 on all IVs (Group 1 above) and each of k - 1 other groups coded 1 on different IVs

Not a contrast (i.e., do not sum to 0) and not orthogonal (i.e., rs not equal to 0)

Compares each of k - 1 groups to group coded 0 on all indicator variables

Effect Coding

One group coded -1 on all IVs (Group 1 above) and each of k - 1 other groups coded 1 on different IVs

A contrast (i.e., sum to 0) but not orthogonal (i.e., rs not equal to 0)

Compares each of k - 1 groups coded 1 to grand mean (equivalently to average of other groups)

Orthogonal Coding

Numerous alternative kinds of Orthogonal IVs (e.g., see Polynomial codes)

Above, Group 1 compared to Group 2, Groups 1 & 2 to 3, and so on to Groups 1 to k - 1 compared to Group k

A contrast (i.e., sum to 0) and orthogonal (i.e., rs equal 0)

Equivalent to test of difference between means for group(s) coded -1 and group coded non-zero

Polynomial Coding (Orthogonal)

Contrasts corresponding to Linear (equal steps), Quadratic (u-shaped or inverted u), and higher patterns

A contrast (i.e., sum to 0) and orthogonal (i.e., rs equal 0)

Testing for Linear, Quadratic, and higher patterns in ordered categorical predictor

For higher values of k, see Table T.5 in Appendix of Tables


Interactions and Regression 15.1

CHAPTER 15

INTERACTION AND REGRESSION

MR and Statistical Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Separate Regression Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Combined Regression Analysis with Interaction Term . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

SPSS and Interaction Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Interaction Between Numerical Predictors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Graphing Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Interactions and Nonlinear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18


The previous chapter demonstrated how multiple regression can accommodate categorical

predictors. Chapter 15 examines the related question of interactions and how multiple regression

can incorporate such effects. Interaction has a specific meaning in statistics; specifically, an

interaction occurs when the effect of one variable varies across the levels of another variable.

For example, a non-smoking program might benefit people who smoke relatively few cigarettes a

day but not help people who smoke a lot. Because the Program effect varies across levels of the

Amount Smoked variable (i.e., the program has little effect for heavy smokers and a robust effect

for light smokers), the Program and Amount Smoked variables demonstrate a statistical

interaction. Or a treatment might be effective for Men but not for Women (or vice versa).

Multiple regression can be used to examine such interactions, but researchers must create special

predictor variables that are sensitive to the interactions. This approach also provides yet another

way to incorporate non-linear relationships into regression analyses, and it can be extended to

interactions between numerical predictors.

MR AND STATISTICAL INTERACTIONS

It is often the case in Psychology and other disciplines that the effect of one predictor

depends on or interacts with another predictor. One common type of interaction in educational

research, for example, is the Aptitude-Treatment interaction, which occurs when certain methods

of teaching (the Treatment) work better with certain types of students (the Aptitude). To

illustrate, stronger students might learn better from unstructured, participatory instruction than

from traditional lectures, whereas weaker students might learn better from structured lectures.

This is an interaction between Type of Instruction and Student Aptitude. A second example of

an interaction in educational research would be attitudes toward mathematics having more of an

effect on performance in Mathematics courses for one gender than the other.


6

8

10

12

14

2 4 6 8

Social Class

Closed Society

Open Society

Figure 15.1. Interaction Between Social Class

and Societal Value on Education.

In regression analyses, interactions

appear as differences in regression

coefficients between two or more groups.

Consider the relation between social class and

educational achievement in different

societies, as shown in Figure 15.1. In a

closed society that does not emphasize

education and includes monetary barriers to

schooling, social class might be a strong

predictor of educational achievement.

Members of disadvantaged groups would lack the financial resources and perhaps the

encouragement to pursue higher education and would therefore be more likely to terminate their

education earlier than advantaged members of that society. In an open society that values

education for all its members and has few financial barriers, the relation between social class and

educational achievement would be weaker and perhaps even eliminated.

Such an interaction would be revealed by differences between the two societies in the

regression coefficients for predicting educational achievement from some measure of social

class. In Figure 15.1, the slope for the closed society is steeper than for the open society. The

interaction could also be described in terms of the differing effect of society type (Open or

Closed) for lower and upper social classes. Society type has little or no effect on the upper

classes; they achieve high levels of education irrespective of whether the society is open or

closed. On the other hand, the effect of society type is quite dramatic for the lower classes,

whose members achieve more schooling in the open society than in the closed society. The

effect of society type for intermediate social classes falls between these extremes.

Our hypothetical example of an interaction involves the effect on romantic intimacy of

the duration of the relationship and external stress. Researchers theorized that many couples

become less intimate as they spend longer in a relationship. The researchers also proposed,

however, that this negative relation could be partly overridden by stressful events that reduce

intimacy and have a larger absolute effect on new couples.


Statistics (S) Yes (S=0) No (S=1)SB Yrs Int SB Yrs Int 1 17 19 11 13 26 2 13 26 12 20 22 3 9 15 13 10 26 4 11 23 14 11 38 5 12 16 15 16 29 6 12 20 16 13 29 7 16 22 17 15 27 8 11 15 18 10 34 9 12 18 19 17 2510 20 10 20 18 30

Box 15.1. Data for Intimacy Study.

No Statistics Students

� = 39.55 - .766 × Yrs

R = .574 R2 = .33

DF SS MS F pReg 1 63.421 63.421 3.934 .08Res 8 128.979 16.122Tot 9 192.400

tb1 = -1.983 = �3.934

Statistics Students

� = 22.81 - .33 × Yrs

R = .238 R2 = .057

DF SS MS F pReg 1 11.011 11.011 .480 .51Res 8 183.389 22.92358Tot 9 194.400

tb1 = - .693 = �.480

Box 15.2. Relation of Intimacy to Years for Two Groups.

To examine this theory, the researchers

obtained scores for length in the relationship (Yrs) and

amount of intimacy (Int) for 20 university students, 10

of whom were experiencing the stress of taking

statistics and 10 of whom were not taking statistics.

The results are shown in Box 15.1 (SB is simply a

subject code). The basic question is whether the

relationship between Years in the relationship (YRS)

and Intimacy (INT) is the same for the Statistics (S = 0)

and No Statistics (S = 1) groups. That is, do subjects 1 to 10 (S = 0) show the same relationship

between YRS and INT as subjects 11 to 20 (S = 1)?

Separate Regression Analyses

Box 15.2 shows the

results from simple

regressions of Intimacy on

Years, separately for the

Statistics and No Statistics

groups. The No Statistics

results confirm the research

hypothesis about the relation

between Intimacy and Years

in the relationship: the

negative relationship with

Years accounts for 32.96%

of the variability in

Intimacy. The (equivalent)

F and t tests demonstrate

that the effect is significant by a directional test, p = .08/2 = .04. The best-fit regression equation

is � = 39.553 - .766 × Y. The No Statistics group starts out quite high on the Intimacy scale


Figure 15.2. Regression of Intimacy on Length

of Relationship for Two Groups.

(Intercept = 39.553), but decreases by .766 units with each additional year in the relationship.

The Statistics group shows less evidence for a negative relation between Years and

Intimacy. The weak negative relation accounts for only 5.7% of the variation in intimacy and

does not approach significance, p = .51. Intimacy declines less with years for the Statistics group

than for the No Statistics group, primarily because the Statistics group has a lower level of

intimacy than the No Statistics group for relations in their early Years (i.e., difference in b0 =

39.553 - 22.811 = 16.742 units).

The two regression equations are

plotted in Figure 15.2, and show the

somewhat steeper slope for the No Statistics

group than for the Statistics group. When

plotted in this way, interactions appear as

non-parallel regression lines. Sometimes the

interaction is so robust that the regression

lines actually intersect or cross. Other times,

as here, the regression lines do not cross, at

least for the range of Years plotted. The lines

would, however, converge and eventually

cross at higher levels for the duration of

relationship variable if the trends of each line

continued unchanged.

The slope for the No Statistics group is significantly different from 0 and the slope for the

Statistics group is not significantly different from 0, but these separate significance tests do not

directly determine whether the difference between the two coefficients (i.e., between -.766 and

-.330) is significant. It is possible that -.766 differs significantly from 0, but does not differ

significantly from -.330. In terms of the non-parallel regression lines in Figure 15.2, the

differences between the slopes may have occurred by chance. To determine the significance of

the difference between the regression coefficients, the results for both groups must be analyzed

together, with the interaction between the two groups (i.e., the difference between their


regression coefficients) coded somehow by a predictor variable.

Combined Regression Analysis with Interaction Term

The previous chapter demonstrated how codes can be used to indicate group membership

for categorical predictors, such as Statistics vs. No Statistics in the present study. A multiple

regression analysis for all 20 scores could include Years as a numerical predictor and another

variable S as a categorical predictor for the Statistics (S = 0) vs. No Statistics (S = 1) Groups.

The two-predictor equation would indicate whether Years (Y) and Statistics (S) predict Intimacy

jointly and independently.

This two-predictor equation would not, however, provide any measure of the difference

between the regression coefficients for Years in the Statistics and No Statistics groups. Testing

the interaction directly requires a third predictor that is the product of Years times the Statistics

code (i.e., SY = S × Y). The regression coefficient for SY reflects the difference in the

regression coefficients of the separate Statistics and No Statistics equations.

As shown shortly, the best-fit regression equation for our three-predictor equation is: � =

22.811 + 16.742 × S - .332 × Y - .434 × SY. Because SY = S × Y, this equation can be rewritten

as: � = (22.811 + 16.742 × S) - (.332 + .434 × S)×Y. The revised equation can be used to

determine two separate equations, by replacing S with 0 for the Statistics group (i.e., � = 22.811 -

.332 × Y) and by replacing S with 1 for the no statistics group (i.e., � = 39.553 - .766 × Y).

These two equations correspond to the simple equations for the Statistics and No Statistics

groups, as reported in Box 15.2. The single multiple regression equation therefore contains the

simple equations for both of the groups.

The coefficients in the 3-predictor equation and their statistical tests are directly related to

the simple equations or to differences between the simple equations. The intercept in the MR

equation is b0 = 22.811, which is b0 for the Statistics group (S = 0). The coefficient for Years is

bY = -.332, which is bY for the Statistics group (S = 0). Thus, b0 and bY in the MR equation

define the simple equation for the Statistics group, the group coded zero by the dummy variable.

The remaining regression coefficients represent differences between the Statistics and No

Statistics groups. The coefficient for the S indicator variable is bS = 16.742, which is the

difference in the b0s from the separate regressions (i.e., 39.553 = 22.811 + 16.742). This term


essentially represents differences between groups at a value of zero on the predictor variable (i.e.,

Years = 0 in our example). Interpretation of this coefficient and its significance depends on a

number of factors, such as whether the interaction term is also significant, and whether the

intercept value of the predictor is meaningful. Here the difference in intercepts would be for

years in relationship equal to 0, probably not that meaningful in considering the impact of

statistics on intimacy for the two groups.

Of primary importance here is the coefficient for the SY interaction term, bSY = -.434.

This coefficient is the difference in the bYs from the simple regressions (i.e., -.766 = -.332 +

-.434). The SY regression coefficient reflects the differences between the slopes for the two

groups, and the t-test for the contribution of SY to the MR equation is therefore a test of the

significance of the difference in the separate slopes. In the present study, tSY = -.709, p = .49, so

we do not reject H0: ßIYwithinStats = ßIYwithinNoStats (withinStats indicates the regression coefficient for

the Statistics group and withinNoStats indicates the regression coefficient for the No Statistics

group). Concretely, -.766 and -.332 do not differ significantly from one another because

differences in slopes this large could occur too often (> .05) by chance.

The unique feature of these analyses for interaction is simply that one of the predictors is

the product of other predictors. This method can also be used to study interactions between two

numerical variables; the numerical variables are multiplied together and the regression

coefficient for the resulting product term reflects the interaction between the two numerical

variables that were multiplied together. If the interaction coefficient is nonsignificant, then the

slope for one predictor does not vary across levels of the other predictor. If the interaction is

significant, the slope for one predictor either increases or decreases across levels of the other

predictor.


LIST

SUBJ STAT YEARS INTIM STXYR PRD.ALL RES.ALL 1.00 .00 17.00 19.00 .00 17.17283 1.82717 2.00 .00 13.00 26.00 .00 18.49950 7.50050 3.00 .00 9.00 15.00 .00 19.82617 -4.82617 4.00 .00 11.00 23.00 .00 19.16284 3.83716 5.00 .00 12.00 16.00 .00 18.83117 -2.83117 6.00 .00 12.00 20.00 .00 18.83117 1.16883 7.00 .00 16.00 22.00 .00 17.50450 4.49550 8.00 .00 11.00 15.00 .00 19.16284 -4.16284 9.00 .00 12.00 18.00 .00 18.83117 -.8311710.00 .00 20.00 10.00 .00 16.17782 -6.1778211.00 1.00 13.00 26.00 13.00 29.59574 -3.5957412.00 1.00 20.00 22.00 20.00 24.23404 -2.2340413.00 1.00 10.00 26.00 10.00 31.89362 -5.8936214.00 1.00 11.00 38.00 11.00 31.12766 6.8723415.00 1.00 16.00 29.00 16.00 27.29787 1.7021316.00 1.00 13.00 29.00 13.00 29.59574 -.5957417.00 1.00 15.00 27.00 15.00 28.06383 -1.0638318.00 1.00 10.00 34.00 10.00 31.89362 2.1063819.00 1.00 17.00 25.00 17.00 26.53191 -1.5319120.00 1.00 18.00 30.00 18.00 25.76596 4.23404

Box 15.3. Original and Derived Scores for Intimacy

Study.

SPSS and Interaction Terms

The hypothetical study just

discussed examined the relationship

between scores on a measure of

intimacy (INTIM) and the duration of

interpersonal relationships (YEARS),

as qualified by whether or not a person

is taking statistics (STAT = 0 for

Statistics group and STAT = 1 for No

Statistics group). The original data are

presented in Box 15.3, along with

variables created during the analysis.

The basic questions to be addressed

are: (a) does intimacy decrease or

increase or remain constant with years in a relationship, and (b) is the relation between intimacy

and years the same for STAT and NOSTAT subjects?

Box 15.4 reports separate regressions of INTIM on YEARS for STAT and NOSTAT

participants; TEMPORARY and SELECT IF statements were used to analyze the two groups

separately (alternatively, SPSS’s SPLIT FILE command could have been used to obtain these

separate analyses). When the STAT group alone is studied (STAT = 0), there is little evidence

for any relation between YEARS and INTIM: R = .238, R² = .057, and the equivalent F and t

tests do not approach significance, p = .5079.


TEMPORARY.SELECT IF STAT=0. /* Statistics GroupREGR /VAR = intim years /DEP = intim /ENTER.

Multiple R .23800R Square .05664Adjusted R Square -.06128Standard Error 4.78786


F = .48035 Signif F = .5079

Variable B SE B Beta T Sig TYEARS -.331668 .478547 -.237998 -.693 .5079(Constant) 22.811189 6.542275 3.487 .0082

TEMPORARY.SELECT IF STAT=1. /* No Statistics GroupREGR /VAR = intim years /DEP = intim /ENTER.



F = 3.93375 Signif F = .0826

Variable B SE B Beta T Sig TYEARS -.765957 .386190 -.574136 -1.983 .0826(Constant) 39.553191 5.666609 6.980 .0001

Box 15.4. Separate Regressions for Statistics and No Statistics Groups.

The

results for the

NOSTAT

group reveal

a stronger

relation

between YRS

and INTIM:

R = .574, R2

= .3296. The

F- and t-tests

here

approach

significance,

p = .0826,

and would be

significant by

a one-tailed

test. A one-

tailed or

directional test is appropriate given the predictions of the researchers. With respect to intercepts,

the NOSTAT group is initially higher on Intimacy than the STATS group (39.553 - 22.811 =

16.742), but the scores decrease more with years for the NOSTAT group (-.766 vs. -.332).


COMP stxyr = stat*years.REGR /VAR = intim stat years stxyr /DESC /STAT = DEFA ZPP /DEP = intim /ENTER /SAVE PRED(prd.all) RESID(res.all).

Mean Std Dev LabelINTIM 23.500 6.909STAT .500 .513YEARS 13.800 3.350STXYR 7.150 7.714

Correlation: INTIM STAT YEARSSTAT .757YEARS -.148 .153STXYR .638 .951 .366

Multiple R .80969 R Square .65560


F = 10.15271 Signif F = .0006

Variable B SE B Beta Corr Part Partl T SigTSTXYR -.4343 .6129 -.4849 .6384 -.1040 -.1744 -.709 .489YEARS -.3317 .4416 -.1608 -.1478 -.1102 -.1845 -.751 .464STAT 16.7420 8.6796 1.2431 .7573 .2833 .4344 1.929 .072Const 22.8112 6.0375 3.778 .002

Box 15.5. MR Analysis with Interaction Term.

To test whether the slopes for STAT and NOSTAT subjects differ significantly, the group

variable (i.e., STAT) is multiplied times the YEARS variable to generate a new predictor

variable called STXYR. Intimacy scores are then regressed on three predictors: Years, the

Statistics indicator variable, and the interaction or product term STXYR. The SPSS commands

and results for this analysis are shown in Box 15.5. The initial COMPUTE statement creates the

interaction

variable. The

best-fit

regression

equation

agrees with

that presented

earlier. Of

particular

interest is the

STXYR

term, which

represents the

difference

between the

slopes from

the simple

regressions for STAT and YEARS (see Box 15.3 for these scores).

The three predictors account for much of the variability in Intimacy scores, R2 = .656 and

FRegression = 10.15, p = .0006. Despite this substantial prediction, none of the individual predictors

are significant in the 3-predictor model, although p = .07 for STAT by a two-tailed test.

Redundant predictors is a serious problem with interaction variables, however, because product

terms tend to correlate highly with one or more of the predictors from which they were formed.

The correlation matrix in Box 15.5 shows an r of .951 between STAT and the interaction term.


Multiple R .80299 R Square .64480Adjusted R Square .60301 Standard Error 4.35329


F = 15.42992 Signif F = .0002

Variable B SE B Beta Corr Part Partl T SigTYEARS -.5572 .3017 -.2701 -.1478 -.2669 -.4088 -1.847 .082STAT 10.7572 1.9701 .7987 .7573 .7893 .7980 5.460 .000Const 25.8102 4.2422 6.084 .000

Box 15.6. MR Without Interaction Term.

There are ways to reduce the problem of correlated predictors (e.g., center the predictors by

subtracting their Ms before computing the cross-product or interaction term).

The significance tests associated with bS and bSY test the differences between the separate

regression equations. The marginally significant t for bS = 16.742, t = 1.929, p = .072, suggests

that the difference between intercepts for the STATS and NOSTATS groups almost reaches

conventional levels of significance. The nonsignificant t for bSY = -.434 (t = -.709, p = .489)

indicates that the difference between the regression coefficients for the STATS and NOSTATS

groups is not significant.

Given the nonsignificant interaction between STAT and YEARS, the interaction term

could be omitted and intimacy regressed on STAT and YEARS alone. The two-predictor

regression results are presented in Box 15.6.

STAT now makes a highly significant contribution to prediction, t = 5.46, p = .000, with

students not taking statistics (STAT = 1) showing more intimacy than those taking statistics

(STAT = 0). The highly correlated interaction term masked this significant effect in the three-

predictor equation. Intimacy declines somewhat with years (bI = -.557, t = -1.847, p = .082,

nondirectional test), and the decline is presumed to be the same for both groups in this analysis

(i.e., there is no interaction term to accommodate different slopes). Together the two variables

account for 64.48% of the variation in INTIM scores, only slightly less than the 65.56% when the

interaction was included.

When the emphasis is on the effect of the categorical variable (i.e., the STAT dummy

variable), the analysis in Box 15.6 is an example of Analysis of Covariance. Partialling out the


effect of the numerical predictor (the covariate) reduces the error term for the test of differences

between groups, and can statistically control for differences on the covariate. This analysis is

identical to earlier analyses in Chapter 14 involving categorical and numerical predictors without

interaction terms.

INTERACTION BETWEEN NUMERICAL PREDICTORS

The preceding example involved an interaction between a categorical predictor (STAT =

0 or 1) and a numerical predictor (YEARS). But it is also possible to test interactions between

two numerical

variables. To

illustrate, the

relation of

negative

affect

(NEGAFF) to

social support

(SUPP) and

external

stressors

(STRESS)

was studied to

determine whether social support ameliorated (i.e., reduced) the harmful effect of external

stressors on negative affect. That is, more stress is generally associated with more negative

affect, but the researchers hypothesized that this relation would be stronger for people with little

social support and weaker for people with good social supports. The hypothetical data are

presented in Box 15.7 along with some additional variables produced for and by the regression

analysis described below.

To test the predicted interaction (i.e., that the effect of stress would vary as a function of

level of social support), the researchers first centered the two predictor variables by subtracting

LIST. SUBJ STRESS SUPP NEGAFF STR2 SUP2 STRXSUP PRDN.SSX RESN.SSX 1.0000 1.0000 1.0000 52.0000 -2.0000 -2.0000 4.0000 52.36000 -.36000 6.0000 2.0000 1.0000 51.0000 -1.0000 -2.0000 2.0000 57.32000 -6.3200011.0000 3.0000 1.0000 63.0000 .0000 -2.0000 .0000 62.28000 .7200016.0000 4.0000 1.0000 70.0000 1.0000 -2.0000 -2.0000 67.24000 2.7600021.0000 5.0000 1.0000 71.0000 2.0000 -2.0000 -4.0000 72.20000 -1.20000 2.0000 1.0000 2.0000 55.0000 -2.0000 -1.0000 2.0000 52.52000 2.48000 7.0000 2.0000 2.0000 58.0000 -1.0000 -1.0000 1.0000 56.89000 1.1100012.0000 3.0000 2.0000 67.0000 .0000 -1.0000 .0000 61.26000 5.7400017.0000 4.0000 2.0000 62.0000 1.0000 -1.0000 -1.0000 65.63000 -3.6300022.0000 5.0000 2.0000 71.0000 2.0000 -1.0000 -2.0000 70.00000 1.00000 3.0000 1.0000 3.0000 55.0000 -2.0000 .0000 .0000 52.68000 2.32000 8.0000 2.0000 3.0000 56.0000 -1.0000 .0000 .0000 56.46000 -.4600013.0000 3.0000 3.0000 64.0000 .0000 .0000 .0000 60.24000 3.7600018.0000 4.0000 3.0000 61.0000 1.0000 .0000 .0000 64.02000 -3.0200023.0000 5.0000 3.0000 64.0000 2.0000 .0000 .0000 67.80000 -3.80000 4.0000 1.0000 4.0000 50.0000 -2.0000 1.0000 -2.0000 52.84000 -2.84000 9.0000 2.0000 4.0000 51.0000 -1.0000 1.0000 -1.0000 56.03000 -5.0300014.0000 3.0000 4.0000 58.0000 .0000 1.0000 .0000 59.22000 -1.2200019.0000 4.0000 4.0000 68.0000 1.0000 1.0000 1.0000 62.41000 5.5900024.0000 5.0000 4.0000 69.0000 2.0000 1.0000 2.0000 65.60000 3.40000 5.0000 1.0000 5.0000 56.0000 -2.0000 2.0000 -4.0000 53.00000 3.0000010.0000 2.0000 5.0000 56.0000 -1.0000 2.0000 -2.0000 55.60000 .4000015.0000 3.0000 5.0000 55.0000 .0000 2.0000 .0000 58.20000 -3.2000020.0000 4.0000 5.0000 60.0000 1.0000 2.0000 2.0000 60.80000 -.8000025.0000 5.0000 5.0000 63.0000 2.0000 2.0000 4.0000 63.40000 -.40000

Box 15.7. Original and Derived Scores for Negative Affect Study.


their Ms, which in the present case were both 2.00, and then creating the cross-product of the two

centered variables. These operations produced the STR2, SUP2, and STRXSUP variables shown

in Box 15.7.

The commands to produce these new predictors are shown at the beginning of Box 15.8,

which also shows the resulting regression analysis. In the analysis, the interaction variable was

entered separately (last) to allow for a clearer impression of the benefits of including an

interaction component in the regression equation.

COMP str2 = stress-3.COMP sup2 = supp -3.COMP strxsup = str2*sup2.REGRE /VARI = negaff str2 sup2 strxsup /DESCR /STAT = DEFAU CHANGE ZPP /DEP = negaff /ENTER str2 sup2 /ENTER /SAVE PRED(prdn.ssx) RESI(resn.ssx).

Mean Std. N Deviation NEGAFF 60.240000 6.6035344 25 STR2 .000000 1.4433757 25 SUP2 .000000 1.4433757 25 STRXSUP .000000 2.0412415 25

NEGAFF STR2 SUP2 STRXSUP STR2 .826 1.000 .000 .000 SUP2 -.223 .000 1.000 .000 STRXSUP -.182 .000 .000 1.000

R R Adjusted Std. Error Change Statistics Square R Square of the Model Estimate R Square F Change df1 df2 Sig. F Change Change 1 .856(a) .732 .708 3.5682947 .732 30.097 2 22 .000 2 .875(b) .766 .732 3.4178105 .033 2.980 1 21 .099

Model Sum of df Mean Square F Sig. Squares 1 Regression 766.440 2 383.220 30.097 .000(a) Residual 280.120 22 12.733 Total 1046.560 24


Unstandardized Standardized t Sig. Correlations Coefficients Coefficients Model B Std. Error Beta Zero-order Partial Part 1 (Constant) 60.240 .714 84.410 .000 STR2 3.780 .505 .826 7.491 .000 .826 .848 .826 SUP2 -1.020 .505 -.223 -2.021 .056 -.223 -.396 -.223

2 (Constant) 60.240 .684 88.127 .000 STR2 3.780 .483 .826 7.820 .000 .826 .863 .826 SUP2 -1.020 .483 -.223 -2.110 .047 -.223 -.418 -.223 STRXSUP -.590 .342 -.182 -1.726 .099 -.182 -.353 -.182

Box 15.8. Regression Analysis for Negative Affect Study.


65

50

41

60

70

2Stress3

80

Affect

Social Support3 24 15 6

Figure 15.3. 3-D Graph of Negative Affect

Data.

The Model 2 regression analysis in Box 15.8 demonstrates a highly significant overall

effect, F = 22.864, p = .000, R2 = .766, with Stress being a particularly strong predictor, t =

7.820, p = .000, pr2 = .8262 = .682, Beta = .826. The contribution of Social Support was more

modest, t = -2.11-, p = .047, pr2 = -.2232 = .050, and the interaction between Stress and Social

Support was marginally significant, t = -1.726, p = .099 (two-tailed), pr2 = -.1822 = .033, and in

the direction expected. The negative slope for the interaction means that the effect of Stress

became smaller as level of Social Support increased.

The nature of the interaction can be seen if one carefully examines the raw scores and the

predicted scores in Box 15.7. The scores are listed sorted first by level of Social Support (i.e.,

the 5 people with the lowest level of social support are presented first), and then sorted by Stress

within levels of Social Support (i.e., cases increase in stress levels from 1 to 5 within each level

of Social Support). Looking now at the NEGAFF column, note that the effect of Stress (i.e., the

increase in NEGAFF scores) is greater for lower levels of Social Support than for higher levels.

This is true both for the observed and predicted scores.

Graphing Interactions

Figure 15.3 shows a 3-dimensional

graph of the data for this hypothetical study.

One can see the interactive effect of Social

Support (left-right axis) and Stress (front-back

axis) on Negative Affect (the vertical axis) by

comparing the effects of each predictor across

levels of the other predictor. At high levels of

Social Support, for example, the effect of

Stress is less than at low levels of social

support. At Social Support = 1, Negative

Affect increases from just over 50 units to over

70 units. At Social Support = 5, however, Negative Affect scores increase more modestly from

about 56 to about 63. Equivalently, note that the effect of social support varies across levels of

stress. For low stress levels, there is little effect of social support as negative affect scores


Figure 15.4. Predicted Negative Affect

Including Interaction Term.

Figure 15.5. Another Visual Representation

of the Interaction.

remain about 55 or so. At high levels of stress, however, negative affect decreases with social

support, from about 75 units for level 1 of social support to about 60 units at level 5. In general,

then, the effect of each predictor varies with levels of the other predictor. This is the defining

characteristic of an interaction.

Figure 15.4 shows a three-dimensional plot

of the prediction equation including the interaction

between stress and social support. The interaction

is shown even more clearly than the plot of the

observed data in Figure 15.3. The slope for social

support begins flat at low stress levels and becomes

increasingly negative with increases in stress.

Conversely, the slope for stress is steeper for low

levels of social support than for high levels of

social support.

Figure 15.5 shows a second way to

represent the interaction visually. Here a two-

dimensional scattergram is presented, with Social

Support on the horizontal axis and Negative Affect

on the vertical axis. The levels of the stress

variable are represented by separate symbols and

line styles for the five levels of stress. The graph

clearly shows that the effect of social support is

helpful for the highest three stress levels;

specifically, negative affect decreases with

increasing social support. But for the lowest two

levels of stress, there is no marked effect of Social

Support, and if anything increases in social support

are associated with slight increases in levels of

negative affect, rather than decreases.


There are several reasons why the interaction term may have reached only marginal levels

of significance, including factors that we have discussed before (e.g., small sample size). To

illustrate the importance of sample size, Box 15.9 shows a regression analysis of the same data,

but with scores

now doubled

simply by

duplicating the

scores shown in

Box 15.7. Note

now that the

interaction effect

is highly

significant, although the values for the intercept and slope are identical to those in Box 15.8. The

extra 25 subjects is sufficient to greatly strengthen the significance of the effect.

In addition to ensuring that sample sizes are adequate, researchers must appreciate that

obtaining significance for interactions can be more challenging than for the overall effect of a

predictor. This topic is discussed more fully in texts on ANOVA for factorial studies, and in

books devoted to interaction analysis in regression.

INTERACTIONS AND NONLINEAR REGRESSION

Chapter 13 showed various ways for researchers to accommodate nonlinear relationships

within a regression analysis: polynomial regression, transformations of predictors, and multiple

indicator variables for linear and nonlinear components of the relationship. Interaction analysis

provides another way of examining nonlinear effects. We illustrate with the reaction time study

analyzed in Chapter 13.

REGRE /DEP = negaff /ENTER strdev supdev strxsup.



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 60.240 .462 130.430 .000 strdev 3.780 .327 .826 11.574 .000 supdev -1.020 .327 -.223 -3.123 .003 strxsup -.590 .231 -.182 -2.555 .014

Box 15.9. Analysis of Interaction with Double the Sample Size.


Figure 15.6. Predicted Values Using Interaction Terms.

Box

15.10 shows a

simple linear

regression

analysis for

this study, R2 =

.880, a strong

and highly

significant

effect.

Although

highly

significant,

plots of the original data or the residual scores from a linear regression reveal a marked bend in

the data. In essence, the line tends to flatten out as uncertainty (U) increases.

Box 15.10 illustrates how to

accommodate this bend using

interaction terms. First, the levels of u

are divided into low (lohi = -1) and

high (lohi = +1) levels. Second, the

original u score is centered by

subtracting out its mean. Finally, we

compute the interaction between these

two predictors. A regression with

these three predictors increases the

strength of the relationship, R2 = .928.

The plot of the predicted scores in

Figure 15.5 shows that the equation

produced in Box 15.10 is actually two

REGRE /DEP = rt /ENTER u. Model R R Square Adjusted R Std. Error of Square the Estimate 1 .938(a) .880 .876 10.196657560...

RECODE u (1 2 3 4 = -1) (5 6 7 8 = +1) INTO lohi.COMP udev = u - 4.5.COMP uxlohi = udev * lohi.REGRE /DEP = rt /ENTER udev lohi uxlohi /SAVE PRED(prdint).



Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) 364.763 2.963 123.112 .000 udev 13.075 1.293 1.050 10.111 .000 lohi -3.650 2.963 -.128 -1.232 .228 uxlohi -5.350 1.293 -.210 -4.137 .000

Box 15.10. Interaction Analysis for Nonlinear Relationships.


straight lines, with somewhat different slopes. The slope is steeper for the low values of u and

shallower for the high values of u. It is this bend that allows additional variation to be captured.

And that additional variation is significant, as shown by the significance of the interaction term

in Box 15.10, t = -4.137, p = .000. In principle, it would be possible to fit a series of straight

lines with different slopes to the data, but much beyond two or three segments are likely to be

difficult to explain theoretically and difficult to justify statistically.

CONCLUSIONS

This chapter has briefly demonstrated how it is possible to accommodate interaction

effects in multiple regression (i.e., that the effect of one variable depends on the level of other

variables in the analysis). We illustrated the use of cross-products of predictors to measure and

test interaction effects. The inclusion of categorical predictors (Chapter 14) and interactions

(Chapter 15) makes multiple regression an extremely general and powerful data analytic

technique.

Documents

MULTIPLE REGRESSION WITH P > 2 PREDICTORSion.uwinnipeg.ca/~clark/teach/4100/_Fall/30-Ch11-15.pdf · 15 59 96 17 143 16 41 75 19 166 17 69 112 20 146 18 63 108 13 115 19 67 120 14