1G89.2229 Lect 6M

• Comparing two coefficients within a regression equation

• Analysis of sets of variables: partitioning the sums of squares

• Polynomial curve fitting.

G89.2229 Multiple Regression

Week 6 (Monday)

2G89.2229 Lect 6M

Comparing Two Regression Coefficients

• Suppose we have» Y=b0+b1X1+b2X2+e

• Suppose one theory states that b1=b2, but another theory says that they should differ.

• How do we carry out the test?

• Create a Wald statistic» In the numerator,» In the denominator, the standard

error of the numerator.

• Recall for two vars W and Z:V(k1W+ k2Z)

= k12 w

2 + k22 z

2 + 2 k1k2 wz

21ˆˆ bb

3G89.2229 Lect 6M

Decomposition of Regression and Residual Variance

• Step 1: Estimate regression coefficients using OLS and compute predicted (fitted) values of Y (Y).

• Step 2: Estimate Regression Sums of Squares as (Y-Y)2, MSR=SSR/df

• Step 3: Estimate Residual Sums of Squares as e2, MSE=SSE/df

• Under H0, MSR/MSE is distributed as central F on (q,n-q-1) df

Source df SS MS

Regression q (Y-Y)2 SSR/qResidual n-q-1 e2 SSE/(n-q-

1)

^

^

^

^

^

¯

¯

4G89.2229 Lect 6M

Test of Incremental R2 Due to Xq

• Hierarchical Regression» Fit reference model with X1, X2, ...,Xq-1

• Determine Regression Sums of Squares• This determines R2 of reference model

» Fit expanded model with Xq added to reference model

• Determine increase in Regression Sums of Squares (SSq)

» on 1 df for single predictor Xq

• Determines R2 increment» “semipartial squared correlation”

• Determine Sums of Squares & Mean Squares for residual from expanded model

» MSE is mean square for residual» on (n-q-1) degrees of freedom

» Under null hypothesis, H0:Bq=0

• MSq is simply fitted random variation

• MSq/MSE ~ F[1, (n-q-1)]

5G89.2229 Lect 6M

Example: Predicting Anger on Day 29 with Day 28 Measures

• Does Anger on day 28 improve the fit of Anger on day 29 after four other moods have been included in the model?

• Do two emotional support variables on day 28 improve the fit of Anger 29 after five moods have been included?

6G89.2229 Lect 6M

Numerical Results

SourceCum R

dfCum R

SSIncrm

dfIncrm

SSMean

Sq Cum F Incrm F

4 Moods (ignoring anger & support) 4 9.90 4 9.90 8.1 8.1

4 Moods +Anger (ignoring support) 5 25.14 1 15.24 16.5 50.15 Moods + Support 7 25.29 2 0.15 11.9 0.2Residual 60 18.27 0.3044Total 67 43.56

7G89.2229 Lect 6M

What to do when relation of Y to X is not linear?

• E.g. Confidence regarding opinion as a function of expressed attitude level.» The stronger the attitude, the more

confident respondents tend to be» Relation is not monotonic:

• Alliance between therapist and patient as a function of time in treatment» Alliance is initially high but decreases

as the hard work of therapy begins.» Alliance recovers for successful therapy» Relationship is not monotonic

X

Y

8G89.2229 Lect 6M

Other Examples of Nonlinear Relations

• Developmental growth as a function of exposure to environmental toxins such as lead.» Low levels of exposure may not reveal

deficits.

• Relation of productivity to impact

9G89.2229 Lect 6M

Modeling Nonlinear Patterns

• It is sometimes possible to transform the outcome so that it can be fit by a truly linear function» Typically works for monotonic

nonlinear relationships» Often nonlinear shape is

accompanied by heteroscedascity» Box-Cox transformation provide a

large class of alternatives• Call possible transformations, h(Y).

» E.g. h(Y)=Yh(Y) = ln(Y)h(Y) = (Ya - 1)/a

10G89.2229 Lect 6M

Polynomial models provide fitting flexibility

• Consider the polynomial equation:» Y = b0+ b1X+b2X2+ ...+bpXp+e

• b1 is the (partial) linear effect)• b2 is the (partial) quadratic effect• b3 is the (partial) cubic effect and so

on» Each term adds the ability to fit certain

patterns of variation.

Polynomial Building Blocks

-12-10-8-6-4-202468

1012

-10 -5 0 5 10

X

Y

(1)X

(1/10)X2

(1/100)X3

11G89.2229 Lect 6M

Application of polynomials

• Polynomial equations are very flexible in fitting a variety of curves» Often we need only consider

quadratic or cubic level equations» Sometimes theoretical

considerations tell us how many terms we need

» Many researchers use polynomial equations simply to be descriptive

• When reviewing research that uses polynomials, ask whether they are being used as structural or descriptive models.

12G89.2229 Lect 6M

Constructing polynomial fits

• Two approaches for constructing polynomial fits» Simply create squared, cubed

versions of X» Center first: Create squared,

cubed versions of (X-C)

• Xc=(X-X)

• Xc and Xc2 will have little or no

correlation

• Both approach yield identical fits

• Centered polynomials are easier to interpret.

13G89.2229 Lect 6M

Interpreting polynomial regression

• Suppose we have the model» Y=b0+b1X1+b2X2+e

» b1 is interpreted as the effect of X1 when X2 is adjusted

• Suppose X1=W, X2=W2

• What does it mean to "hold constant" X2 in this context?

• When the zero point is interpretable» Linear term is slope at point 0» Quadratic is acceleration

at point 0» Cubic is change in acceleration at

point 0