Upload
sanjeev-kumar
View
160
Download
4
Tags:
Embed Size (px)
DESCRIPTION
multiple regression
Citation preview
Multiple Regression
PSYC 6130A, PROF. J. ELDER 2
Multiple Regression
• Multiple regression extends linear regression to allow for 2 or more independent variables.
• There is still only one dependent (criterion) variable.
• We can think of the independent variables as ‘predictors’ of the dependent variable.
• The main complication in multiple regression arises when the predictors are not statistically independent.
PSYC 6130A, PROF. J. ELDER 3
Example 1: Predicting Income
Age
Hours Worked
MultipleRegression
Income
PSYC 6130A, PROF. J. ELDER 4
Example 2: Predicting Final Exam Grades
Assignments
Midterm
MultipleRegression
Final
PSYC 6130A, PROF. J. ELDER 5
Coefficient of Multiple Determination
• The proportion of variance explained by all of the independent variables together is called the coefficient of multiple determination (R2).
• R is called the multiple correlation coefficient.
• R measures the correlation between the predictions and the actual values of the dependent variable.
• The correlation riY of predictor i with the criterion (dependent variable) Y is called the validity of predictor i.
PSYC 6130A, PROF. J. ELDER 6
Uncorrelated Predictors
21 Yr 2
2Yr
Total variance
Variance explained by assignments Variance explained by midterm
2 2 2 2 21 2=Total proportion of variance explained = Y Y Y YR r r
PSYC 6130A, PROF. J. ELDER 7
Uncorrelated Predictors
• Standardized regression equation for uncorrelated predictors:
1 1 2 2Y Y Yz r z r z
PSYC 6130A, PROF. J. ELDER 8
Example 1. Predicting Income
Correlations
1 .040* .229**
.012 .000
3975 3975 3975
.040* 1 .187**
.012 .000
3975 3975 3975
.229** .187** 1
.000 .000
3975 3975 3975
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
AGE
HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week
TOTAL INCOME
AGE
HOURSWORKEDFOR PAY
OR INSELF-
EMPLOYMENT - inReference Week
TOTALINCOME
Correlation is significant at the 0.05 level (2-tailed).*.
Correlation is significant at the 0.01 level (2-tailed).**.
PSYC 6130A, PROF. J. ELDER 9
Correlated Predictors
21 Yr 2
2Yr
Total variance
Variance explained by assignments Variance explained by midterm
2 2 21 2=Total proportion of variance explained < Y YR r r
PSYC 6130A, PROF. J. ELDER 10
Correlated Predictors
• Due to the correlation in the predictors, the optimal regression weights must be reduced:
1 1 2 2Yz B z B z
1 2 12 2 1 121 22 2
12 12
where
and 1 1
Y Y Y Yr r r r r rB B
r r
1 1 beta weights
(standardized partial re
and
gres
are calle
sion coeffi
d th
c
s)
e
ient
B B
2 22 1 2 1 2 12
1 1 2 2 212
2
1Y Y Y Y
Y Y
r r r r rR B r B r
r
PSYC 6130A, PROF. J. ELDER 11
Raw-Score Formulas
0 1 1 2 2Y b b X b X
1 2
1 1 2 2
0 1 1 2 2
where
and
and
Y Y
X X
s sb B b B
s s
b Y b X b X
PSYC 6130A, PROF. J. ELDER 12
Example 1. Predicting Income
Correlations
1 .040* .229**
.012 .000
3975 3975 3975
.040* 1 .187**
.012 .000
3975 3975 3975
.229** .187** 1
.000 .000
3975 3975 3975
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
AGE
HOURS WORKEDFOR PAY OR INSELF-EMPLOYMENT- in Reference Week
TOTAL INCOME
AGE
HOURSWORKEDFOR PAY
OR INSELF-
EMPLOYMENT - inReference Week
TOTALINCOME
Correlation is significant at the 0.05 level (2-tailed).*.
Correlation is significant at the 0.01 level (2-tailed).**.
1 1 2 2Yz B z B z
1 2 12 2 1 121 22 2
12 12
where
and 1 1
Y Y Y Yr r r r r rB B
r r
2 22 1 2 1 2 12
1 1 2 2 212
2
1Y Y Y Y
Y Y
r r r r rR B r B r
r
PSYC 6130A, PROF. J. ELDER 13
Example 1. Predicting Income
020
4060
80
0
20
40
60
800
1
2
3
4
5
6
7
x 104
Age (years)Hours worked per week (hours)
An
nu
al I
nco
me
(C
AD
)
PSYC 6130A, PROF. J. ELDER 14
Degrees of freedom
1 where
sample size
number of predictors
df n k
n
k
PSYC 6130A, PROF. J. ELDER 15
Semipartial (Part) Correlations
• The semipartial correlations measure the correlation between each predictor and the criterion when all other predictors are held fixed.
• In this way, the effects of correlations between predictors are eliminated.
• In general, the semipartial correlations are smaller than the validities.
PSYC 6130A, PROF. J. ELDER 16
Semipartial Correlations
21 Yr 2
2Yr
Total variance
Variance explained by assignments Variance explained by midterm
Semipartial correlation of Predictor 2 with Y
PSYC 6130A, PROF. J. ELDER 17
Calculating Semipartial Correlations
• One way to calculate the semipartial correlation for a predictor (say Predictor 1) is to partial out the effects of all other predictors on Predictor 1and then calculate the correlation between the residual of Predictor 1 and the criterion.
• For example, we could partial out the effects of age on hours worked, and then measure the correlation between income and the residual hours worked.
• A more straightforward method:
1 2 12(1.2) 2
121Y Y
Y
r r rr
r
PSYC 6130A, PROF. J. ELDER 18
Calculating Semipartial Correlations
• A more straightforward method:
1 2 12(1.2) 2
121Y Y
Y
r r rr
r
(1.2)where is the semipartial correlation between Predictor 1 and Yr Y
i.e., the correlation between and Predictor 1
after partialling out the effects of Predictor 2 on Predictor 1.
Y
PSYC 6130A, PROF. J. ELDER 19
PSYC 6130A, PROF. J. ELDER 20
Example 2: Predicting Final Exam Grades
Assignments
Midterm
MultipleRegression
Final
PSYC 6130A, PROF. J. ELDER 21
Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)
Correlations
1 .356 .127
.233 .680
13 13 13
.356 1 .615*
.233 .025
13 13 13
.127 .615* 1
.680 .025
13 13 13
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Assignments
Midterm
Final
Assignments Midterm Final
Correlation is significant at the 0.05 level (2-tailed).*.
212 120.356 0.127r r 2
1 120.127 0.016Yr r 22 20.615 0.378Y Yr r
PSYC 6130A, PROF. J. ELDER 22
Example 2. Predicting Final Exam Grades (PSYC 6130A, 2005-2006)
212 120.356 0.127r r 2
1 120.127 0.016Yr r 22 20.615 0.378Y Yr r
1 1 2 2Yz B z B z
1 2 12 2 1 121 22 2
12 12
where
and 1 1
Y Y Y Yr r r r r rB B
r r
2 2
2 1 2 1 2 121 1 2 2 2
12
2
1Y Y Y Y
Y Y
r r r r rR B r B r
r
PSYC 6130A, PROF. J. ELDER 23
Example 2. Predicting Final Exam Grades
7080
90100
20
40
60
800
50
100
150
Assignment grade (%)Midterm grade (%)
Fin
al g
rad
e (
%)