1G89.2229 Lect 6W

• Polynomial example

• Orthogonal polynomials

• Statistical power for regression

G89.2229 Multiple Regression Week 6 (Wednesday)

2G89.2229 Lect 6W

Constructing polynomial fits

• Two approaches for constructing polynomial fits» Simply create squared, cubed

versions of X» Center first: Create squared,

cubed versions of (X-C)

• Xc=(X-X)

• Xc and Xc2 will have little or no

correlation

• Both approach yield identical fits

• Centered polynomials are easier to interpret.

3G89.2229 Lect 6W

Example from Cohen

• Interest in minor subject as a function of credits in minor

0 5 10 15 20

x1

5.0000

10.0000

15.0000

20.0000

25.0000

ybr

Y: Interest in further coursework in minor subject.

X: Number of credits in minor subject

Cohen Ch 6 Ex 1

R Sq Quadratic =0.673

4G89.2229 Lect 6W

Interpreting polynomial regression

• Suppose we have the model» Y=b0+b1X1+b2X2+e

» b1 is interpreted as the effect of X1 when X2 is adjusted

• Suppose X1=W, X2=W2

• What does it mean to "hold constant" X2 in this context?

• When the zero point is interpretable» Linear term is slope at point 0» Quadratic is acceleration

at point 0» Cubic is change in acceleration at

point 0

5G89.2229 Lect 6W

Orthogonal Polynomials

• In experiments, one might have three or four levels of treatment with equal spacing.» 0, 1, 2» 0, 1, 2, 3

• These levels can be used with polynomial models to fit» Linear, quadratic or cubic trends» We would simply construct

squared and cubic forms.

27931

8421

1111

0001

6G89.2229 Lect 6W

Making polynomials orthogonal

• The linear, quadratic and cubic trends are all going up in the same way.

• The curve for the quadratic is like the one for cubic.

• Orthogonal polynomials eliminate this redundancy hierarchically.» The constant is removed from the linear

trend» The const and linear are removed from

quadratic» The const, lin and quad are removed

from the cubic.

0.50 -0.67 0.50 -0.220.50 -0.22 -0.50 0.670.50 0.22 -0.50 -0.670.50 0.67 0.50 0.22

7G89.2229 Lect 6W

Analysis with Orthogonal polynomials

• If we substitute orthogonal polynomials for the usual linear, squared, and cubic terms, we» Recover the same polynomial fit» Obtain effects that are useful in

determining the polynomial order

• Even when cubic effects are included, with orthogonal effects» The linear is the average effect» The quadratic is adjusted for the

linear, but not adjusted for cubic» The cubic is adjusted for all

before

• The regression coefficients, however are difficult to interpret.

8G89.2229 Lect 6W

Computing Orthogonal Polynomials

• Can copy values from Cohen et al or other tables» Substitute original polynomial values for

orthogonal version

• Can use the matrix routine of SPSS to implement a special Transformation.» Read the polynomial data into Matrix.» Use program provided that essentially

does four things• Computes the polynomial sums or

squares/cross products• Finds a Cholesky factor• Inverts the Cholesky factor• Transforms the polynomial values to

be orthogonal

9G89.2229 Lect 6W

The Matrix programMATRIX.

GET X

/VARIABLES = X, XSQ, XCUB.

COMPUTE XFULL={MAKE(100,1,1),X}.

COMPUTE XX=T(XFULL)*XFULL.

COMPUTE XCHOL=CHOL(XX).

PRINT XCHOL.

COMPUTE ICHOL=INV(XCHOL).

PRINT ICHOL.

COMPUTE XORTH=XFULL*ICHOL.

SAVE XORTH

/OUTFILE=*

/VARIABLES= OTH0 ORTH1 ORTH2 ORTH3.

END MATRIX.

MATCH FILES /FILE=*

/FILE='C:\My Documents\Pat\Courses\G89.2229 Regression\Examples\Reg06W.sav'.

EXECUTE.

10G89.2229 Lect 6W

The transformed variables

0.00 5.00 10.00 15.00 20.00

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60ORTH1X

ORTH2X

ORTH3X