Upload
adela-parker
View
214
Download
0
Embed Size (px)
Citation preview
Curvilinear 2
Modeling Departures from the Straight Line
(Curves and Interactions)
Skill Set
• How does polynomial regression test for quadratic and cubic trends?
• What are orthogonal polynomials? When can they be used? Describe an advantage of using orthogonal polynomials over simple polynomial regression.
• Suppose we have one IV and we analyze this IV twice, once thru linear regression and once as a categorical variable. What does the test for the difference in R-square between the two tell us? What doesn’t it tell us, that is, if the result is significant, what is left to do?
More skills
• Why is collinearity likely to be a problem in using polynomial regression?
• Describe the sequence of tests used to model curves in polynomial regression.
• How do you model interactions of continuous variables with regression?
• What is the difference between a moderator and a mediator? How do you test for the presence of each?
Nonlinear Trends in Experimental Research• Suppose we go to Bush Gardens and
Measure reactions to a roller coaster as a function of time.
• We ask for excitement ratings (1 to 10 scale) either immediately after the ride or at 5, 10 or 15 minutes after.
Roller Coaster Ratings
Rating (DV) Time
(Contin IV) V1 V2 V3 Stats
10 0 1 0 0 9 0 1 0 0 10 0 1 0 0 8 0 1 0 0 M=9.2 9 0 1 0 0 SD=.84 8 5 0 1 0 7 5 0 1 0 7 5 0 1 0 8 5 0 1 0 M=7.8 9 5 0 1 0 SD=.84 7 10 0 0 1 6 10 0 0 1 8 10 0 0 1 5 10 0 0 1 M=6.6 7 10 0 0 1 SD=1.14 5 15 0 0 0 6 15 0 0 0 7 15 0 0 0 7 15 0 0 0 M=6.6 8 15 0 0 0 SD=1.14
Note that IV is represented in 2 ways (a) as a continuous IV, and (b) as a dummy coded (3 vector) categorical IV.
Excitement as a function of time.
SAS boxplotsof roller coaster data.
The GLM Procedure Dependent Variable: rating Sum of Source DF Squares Mean Square F Value Pr > F Model 1 20.25000000 20.25000000 19.49 0.0003 Error 18 18.70000000 1.03888889 Corrected Total 19 38.95000000 R-Square Coeff Var Root MSE rating Mean 0.519897 13.50012 1.019259 7.550000 Source DF Type I SS Mean Square F Value Pr > F time 1 20.25000000 20.25000000 19.49 0.0003 Source DF Type III SS Mean Square F Value Pr > F time 1 20.25000000 20.25000000 19.49 0.0003 Standard Parameter Estimate Error t Value Pr > |t| Intercept 8.900000000 0.38137179 23.34 <.0001 time -0.180000000 0.04077036 -4.41 0.0003
R-square
Roller coaster data analysis with time as continuous IV
The GLM Procedure Dependent Variable: rating Sum of Source DF Squares Mean Square F Value Pr > F Model 3 22.95000000 7.65000000 7.65 0.0022 Error 16 16.00000000 1.00000000 Corrected Total 19 38.95000000 R-Square Coeff Var Root MSE rating Mean 0.589217 13.24503 1.00000 7.550000 Source DF Type I SS Mean Square F Value Pr > F v1 1 18.15000000 18.15000000 18.15 0.0006 v2 1 4.80000000 4.80000000 4.80 0.0436 v3 1 0.00000000 0.00000000 0.00 1.0000 Source DF Type III SS Mean Square F Value Pr > F v1 1 16.90000000 16.90000000 16.90 0.0008 v2 1 3.60000000 3.60000000 3.60 0.0760 v3 1 0.00000000 0.00000000 0.00 1.0000 Standard Parameter Estimate Error t Value Pr > |t| Intercept 6.600000000 0.44721360 14.76 <.0001 v1 2.600000000 0.63245553 4.11 0.0008 v2 1.200000000 0.63245553 1.90 0.0760 v3 0.000000000 0.63245553 0.00 1.0000
R-square
Analysis with time as a categorical IV
Testing for curvesCompare the R-square values.
Linear: .51987.Categorical: .5892. )1/()1(
/)(
12
123.
212.
2123.
kNR
kkRRF
y
xyy
35.1025675.
0346515.
)1320/()5892.1(
)13/()51987.5892(.
F
Critical value (alpha = .05) of F(2,16) = 3.63, n.s.
If significant, the F test indicates departure from linearity, but not where or how. If you have M levels, can have up to (M-1) bends.
Review
Suppose we have one IV and we analyze this IV twice, once thru linear regression and once as a categorical variable. What does the test for the difference in R-square between the two tell us? What doesn’t it tell us, that is, if the result is significant, what is left to do?
Orthogonal Polynomials
Sometimes orthogonal polynomials can be used to analyze experimental data to test for curves. Two restrictive assumptions must be met to use orthogonal polynomials: (1) equal ‘spacings’ of the IV, and (2) equal numbers of observations (people) at each cell (e.g., coaster data).Orthogonal polynomials are special sets of coefficients that test for bends but manage to remain uncorrelated with one another. This gives them an advantage in statistical power and in simplicity of understanding.
Orthogonal Poly Rating (DV)
Time (Contin. IV)
Lin 1
Quad2
Cub 3
10 0 -3 1 -1 9 0 -3 1 -1 10 0 -3 1 -1 8 0 -3 1 -1 M=9.2 9 0 -3 1 -1 S=.84 8 5 -1 -1 3 7 5 -1 -1 3 7 5 -1 -1 3 8 5 -1 -1 3 M=7.8 9 5 -1 -1 3 S=.84 7 10 1 -1 -3 6 10 1 -1 -3 8 10 1 -1 -3 5 10 1 -1 -3 M=6.6 7 10 1 -1 -3 S=1.14 5 15 3 1 1 6 15 3 1 1 7 15 3 1 1 7 15 3 1 1 M=6.6 8 15 3 1 1 S=1.14
Coaster data with orthogonal polynomial vectors. Note the pattern in the vectors. Switches indicate bends. Find orthogonal polynomials in a table.
Orthogonal Poly TablePolynomial X=1 2 3 4Linear -1 0 1
Quadratic 1 -2 1
Linear -3 -1 1 3Quadratic 1 -1 -1 1Cubic -1 3 -3 1
Note. Rows in table will be columns in data. Cols in table represent levels of IV.
Correlations Among VectorsR Time Excite
(Rating)L Q C
Time 1
Excite -.72 1
Linear 1.00 -.72 1
Quad .00 .25 .00 1
Cubic .00 .08 .00 .00 1
These are correlations among the vectors for the coaster data. Time in minutes since leaving the coaster correlated -.72 with excitement ratings. Time correlates 1.0 with the linear trend. Note that the Linear, Quadratic and Cubic vectors are uncorrelated (orthogonal).
Regression with Orthogonal Polynomials
Source df Estimate Type I & Type III SS
F P
Intercept 7.55
Linear 1 -.45 20.25 20.25 .0004
Quad 1 .35 2.45 2.45 .1371
Cubic 1 .05 0.25 0.25 .6239
Note that R-square for the model using orthogonal polynomials is the same as that using the dummy vectors. The F for the linear component is larger using orthogonal polynomials than it was for the linear regression because the error term is smaller due to the quadratic and cubic terms. Orthogonal polynomials provide a powerful test of effects. Also can be used to graph results to show bends.
Review
•How does polynomial regression test for quadratic and cubic trends?
•What are orthogonal polynomials? When can they be used?
Nonlinear Relations in Nonexperimental Research• Create power terms (IV taken to
successive powers)
• Test for increasing numbers of bends by adding terms
• Quit when adding a term does not increase variance accounted for.
Rating(DV)
Time Time**2 Time**3
10 0 0 09 0 0 010 0 0 08 0 0 09 0 0 08 5 25 1257 5 25 1257 5 25 1258 5 25 1259 5 25 1257 10 100 1000
6 10 100 1000
8 10 100 1000
5 10 100 1000
7 10 100 1000
5 15 225 3375
6 15 225 3375
7 15 225 3375
7 15 225 3375
8 15 225 3375
Polynomials to model bends in nonexperimental research
Correlations among terms
Excite Time Time**2 Time**3
Excite 1
Time -.72 1
Time**2 -.62 .96 1
Time**3 -.55 .91 .99 1
Note that terms with higher exponents are VERY highly correlated. There WILL be problems with collinearity.
Sequence of tests. Start with time, add time squared. If significant, add time cubed. Stop when adding a term doesn’t help. Each power adds a bend. Quadratic is one bend, cubic is two, and so forth.
Results of Polynomial Regression
Excite Time Time**2 Time**3
Excite 1
Time -.72 1
Time**2 -.62 .96 1
Time**3 -.55 .91 .99 1
Model Intercept b1 b2 b3 R2 R2 Ch
1 Time 8.90 -.18 .52 .52
2 Time, Time2
9.25 -.39 .014 .58 .06
3 Time,Time2,Time3
9.20 -.23 -.02 .001 .59 .01
Note that polynomial is a special case of hierarchical reg.
Polynomial Results (2)
Suppose it had happened that the term for time-squared had been significant. The regression equation is Y' = 9.25 -.39X + .014X2. The results graphed:
201612840-4
Time
12
9
6
3
0
Exc
item
ent
Roller Coaster Evaluations
Interpreting Weights in Polynomial Regression• All power terms for an IV work together to
define the curve relating Y to X.• Do not interpret b weights for polynomials.
They change if you subtract the mean from the raw data.
• To estimate ‘importance’ look to the change in R-square for the block of variables that represent the IV.
• Never use polynomials in a variable selection algorithm (e.g., stepwise regression).
• Specialized literature on nonlinear terms in path analysis and SEM (hard to do).
ReviewDescribe an advantage of using orthogonal polynomials over simple polynomial regression.
•Why is collinearity likely to be a problem in using polynomial regression?
•Describe the sequence of tests used to model curves in polynomial regression.
Interactions
• An interaction means that the ‘importance’ of one variable depends upon the value of another.
• An interaction is also sometimes called a moderator, as in “Z moderates the relations between X and Y.”
• In regression, we look to see if the slope relating the DV to the IV changes depending on the value of a second IV.
Example Interaction
1062-2-6-10
Creativity
10
5
0
-5
-10
Pro
duct
ivity
Low Cognitive Ability
1062-2-6-10
Creativity
10
5
0
-5
-10
Pro
duct
ivity
Medium Cognitive Ability
1062-2-6-10
Creativity
10
5
0
-5
-10
Pro
duct
ivity
High Cognitive Ability
For those with low cog ability, there is a small correlation between creativity and productivity.
As cognitive ability increases, the relations between creativity and productivity become stronger. The slope of productivity on creativity depends on cog ability.
Interaction Response SurfaceInteraction
108
64
20
108
64
20
80
60
40
20
0
X1X2
Y
The slope of X1 depends on the value of X2 and vice versa. Regression is looking to fit this response surface and no other when we do the customary analysis for interactions with continuous IVs. More restrictive than ANOVA.
Interaction
108
642
0
10
8
6
4
2
0
80
60
40
20
0
X1
X2
Y
Significance Tests for Interactions• Subtract means from each IV (optional).• Compute product of IVs.• Compute significance of change in R-square
using interaction(s).• If R-square change is n.s., no interaction(s)
present.• If R-square change is significant, find the
significant interaction(s).• Graph the interaction(s)
data d1; input person product create cog; inter=create*cog;
cards;
1 50 40 100
2 35 45 80
3 40 50 90
4 50 55 105
5 55 60 110
6 35 40 95
7 45 45 100
8 55 50 105
9 50 55 95
10 40 60 90
11 45 40 110
12 50 45 115
13 60 50 120
14 65 55 125
15 55 60 105
16 50 40 110
17 55 45 95
18 55 50 115
19 60 60 120
20 65 65 140
proc print; proc corr; proc glm; model product = create cog;
run;
Data to test for interaction between cognitive ability and creativity on performance.
Pearson Correlation Coefficients, N = 20
Prob > |r| under H0: Rho=0
person product create cog inter
person 1.00000 0.65629 0.32531 0.66705 0.57538
0.0017 0.1616 0.0013 0.0079
product 0.65629 1.00000 0.50470 0.83568 0.78465
0.0017 0.0232 <.0001 <.0001
create 0.32531 0.50470 1.00000 0.38414 0.84954
0.1616 0.0232 0.0945 <.0001
cog 0.66705 0.83568 0.38414 1.00000 0.80732
0.0013 <.0001 0.0945 <.0001
inter 0.57538 0.78465 0.84954 0.80732 1.00000
0.0079 <.0001 <.0001 <.0001
Correlation Matrix
The GLM Procedure Dependent Variable: product Sum of Source DF Squares Mean Square F Value Pr > F Model 2 1080.151718 540.075859 23.93 <.0001 Error 17 383.598282 22.564605 Corrected Total 19 1463.750000 R-Square Coeff Var Root MSE product Mean 0.737935 9.360042 4.750222 50.75000 Source DF Type I SS Mean Square F Value Pr > F create 1 372.8504184 372.8504184 16.52 0.0008 cog 1 707.3012991 707.3012991 31.35 <.0001 Source DF Type III SS Mean Square F Value Pr > F create 1 57.9376107 57.9376107 2.57 0.1275 cog 1 707.3012991 707.3012991 31.35 <.0001 Standard Parameter Estimate Error t Value Pr > |t| Intercept -11.31387459 9.26553773 -1.22 0.2387 create 0.23848700 0.14883268 1.60 0.1275 cog 0.47077911 0.08408699 5.60 <.0001
Results for 2 IVs (Main Effects)
Dependent Variable: product Sum of Source DF Squares Mean Square F Value Pr > F Model 3 1088.423539 362.807846 15.47 <.0001 Error 16 375.326461 23.457904 Corrected Total 19 1463.750000 R-Square Coeff Var Root MSE product Mean 0.743586 9.543519 4.843336 50.75000 Source DF Type I SS Mean Square F Value Pr > F create 1 372.8504184 372.8504184 15.89 0.0011 cog 1 707.3012991 707.3012991 30.15 <.0001 inter 1 8.2718212 8.2718212 0.35 0.5609 Source DF Type III SS Mean Square F Value Pr > F create 1 15.35701934 15.35701934 0.65 0.4303 cog 1 49.38823497 49.38823497 2.11 0.1661 inter 1 8.27182120 8.27182120 0.35 0.5609 Standard Parameter Estimate Error t Value Pr > |t| Intercept -45.48374780 58.31267387 -0.78 0.4468 create 0.87233592 1.07813934 0.81 0.4303 cog 0.79009054 0.54451482 1.45 0.1661 inter -0.00587585 0.00989498 -0.59 0.5609
Result for interaction
Moderator and Mediator
• Moderator Means Interaction. Slope of one depends on the value of the other. Use moderated regression (test for an interaction) to test.
• Mediator means there is a causal chain of events. The mediating variable is the proximal cause of the DV. A more distal cause changes the mediator. Use path analysis to test. In this graph, X2 is the mediator.
Moderator Mediator
X1 X2
Y
X1 X2 Y
Review
• How do you model interactions of continuous variables with regression?
• What is the difference between a moderator and a mediator? How do you test for the presence of each?