Statistical Analysis
Regression & Correlation
Psyc 250
Winter, 2008
Review:
Types of Variables&
Steps in Analysis
Variables & Statistical TestsVariable Type Example Common Stat
MethodNominal by nominal
Blood type by gender
Chi-square
Scale by nominal GPA by gender
GPA by major
T-test
Analysis of Variance
Scale by scale Weight by height
GPA by SAT
Regression
Correlation
Evaluating an hypothesis
• Step 1: What is the relationship in the sample?
• Step 2: How confidently can one generalize from the sample to the universe from which it comes?
p < .05
Evaluating an hypothesisRelationship in
SampleStatistical
Significance
2 nom. vars. Cross-tab / contingency table
“p value” from Chi Square
Scale dep. & 2-cat indep.
Means for each category
“p value” from t-test
Scale dep. & 3+ cat indep.
Means for each category
“p value” from ANOVA
2 scale vars. Regression line
Correlation r & r2
“p value” from reg or correlation
Evaluating an hypothesisRelationship in
SampleStatistical
Significance
2 nom. vars. Cross-tab / contingency table
“p value” from Chi Square
Scale dep. & 2-cat indep.
Means for each category
“p value” from t-test
Scale dep. & 3+ cat indep.
Means for each category
“p value” from ANOVA
2 scale vars. Regression line
Correlation r & r2
“p value” from reg or correlation
Relationships betweenScale Variables
Regression
Correlation
Regression• Amount that a dependent variable
increases (or decreases) for each unit increase in an independent variable.
• Expressed as equation for a line –
y = m(x) + b – the “regression line”
• Interpret by slope of the line: m
(Or: interpret by “odds ratio” in “logistic regression”)
Correlation• Strength of association of scale measures
• r = -1 to 0 to +1
+1 perfect positive correlation
-1 perfect negative correlation
0 no correlation
• Interpret r in terms of variance
Mean&
Variance
Survey of Classn = 42
• Height• Mother’s height• Mother’s education• SAT• Estimate IQ• Well-being
(7 pt. Likert)
• Weight• Father’s education• Family income• G.P.A.• Health (7 pt. Likert)
Frequency Table for: HEIGHT Valid CumValue Label Value Frequency Percent Percent Percent 59.00 1 2.4 2.4 2.4 61.00 2 4.8 4.8 7.1 62.00 3 7.1 7.1 14.3 63.00 3 7.1 7.1 21.4 65.00 5 11.9 11.9 33.3 66.00 3 7.1 7.1 40.5 67.00 4 9.5 9.5 50.0 68.00 5 11.9 11.9 61.9 69.00 1 2.4 2.4 64.3 70.00 6 14.3 14.3 78.6 71.00 1 2.4 2.4 81.0 72.00 4 9.5 9.5 90.5 73.00 3 7.1 7.1 97.6 74.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0
Frequency Table for: HEIGHT Valid CumValue Label Value Frequency Percent Percent Percent 59.00 1 2.4 2.4 2.4 61.00 2 4.8 4.8 7.1 62.00 3 7.1 7.1 14.3 63.00 3 7.1 7.1 21.4 65.00 5 11.9 11.9 33.3 66.00 3 7.1 7.1 40.5 67.00 4 9.5 9.5 50.0 68.00 5 11.9 11.9 61.9 69.00 1 2.4 2.4 64.3 70.00 6 14.3 14.3 78.6 71.00 1 2.4 2.4 81.0 72.00 4 9.5 9.5 90.5 73.00 3 7.1 7.1 97.6 74.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0 Descriptive Statistics for: HEIGHT ValidVariable Mean Std Dev Variance Range Minimum Maximum N HEIGHT 67.33 3.87 14.96 15.00 59.00 74.00 42
mean
Variance
x i - Mean )2
Variance = s2 = ----------------------- N
Standard Deviation = s = variance
Frequency Table for: WEIGHT Valid CumValue Label Value Frequency Percent Percent Percent 115.00 1 2.4 2.4 2.4 120.00 1 2.4 2.4 4.8 124.00 1 2.4 2.4 7.1 125.00 4 9.5 9.5 16.7 128.00 1 2.4 2.4 19.0 130.00 6 14.3 14.3 33.3 135.00 4 9.5 9.5 42.9 136.00 1 2.4 2.4 45.2 140.00 3 7.1 7.1 52.4 145.00 2 4.8 4.8 57.1 150.00 3 7.1 7.1 64.3 155.00 2 4.8 4.8 69.0 160.00 6 14.3 14.3 83.3 165.00 2 4.8 4.8 88.1 170.00 1 2.4 2.4 90.5 185.00 1 2.4 2.4 92.9 190.00 2 4.8 4.8 97.6 210.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0 Descriptive Statistics for: WEIGHT ValidVariable Mean Std Dev Variance Range Minimum Maximum N WEIGHT 146.38 21.30 453.80 95.00 115.00 210.00 42
mean
Relationship of weight & height:
Regression Analysis
“Least Squares” Regression Line
Dependent = ( B ) (Independent) + constant
weight = ( B ) ( height ) + constant
Regression line
Regression: WEIGHT on HEIGHT Multiple R .59254R Square .35110Adjusted R Square .33488Standard Error 17.37332 Analysis of Variance DF Sum of Squares Mean SquareRegression 1 6532.61322 6532.61322Residual 40 12073.29154 301.83229 F = 21.64319 Signif F = .0000 ------------------ Variables in the Equation ------------------ Variable B SE B Beta T Sig T HEIGHT 3.263587 .701511 .592541 4.652 .0000(Constant) -73.367236 47.311093 -1.551
[ Equation: Weight = 3.3 ( height ) - 73 ]
Regression line
W = 3.3 H - 73
Strength of Relationship
“Goodness of Fit”: Correlation
How well does the regression line “fit” the data?
Frequency Table for: WEIGHT Valid CumValue Label Value Frequency Percent Percent Percent 115.00 1 2.4 2.4 2.4 120.00 1 2.4 2.4 4.8 124.00 1 2.4 2.4 7.1 125.00 4 9.5 9.5 16.7 128.00 1 2.4 2.4 19.0 130.00 6 14.3 14.3 33.3 135.00 4 9.5 9.5 42.9 136.00 1 2.4 2.4 45.2 140.00 3 7.1 7.1 52.4 145.00 2 4.8 4.8 57.1 150.00 3 7.1 7.1 64.3 155.00 2 4.8 4.8 69.0 160.00 6 14.3 14.3 83.3 165.00 2 4.8 4.8 88.1 170.00 1 2.4 2.4 90.5 185.00 1 2.4 2.4 92.9 190.00 2 4.8 4.8 97.6 210.00 1 2.4 2.4 100.0 ------- ------- ------- Total 42 100.0 100.0 Valid cases 42 Missing cases 0 Descriptive Statistics for: WEIGHT ValidVariable Mean Std Dev Variance Range Minimum Maximum N
WEIGHT 146.38 21.30 453.80 95.00 115.00 210.00 42
mean
mean
Variance = 454
Regression line
mean
Correlation: “Goodness of Fit”• Variance (average sum of squared
distances from mean) = 454
• “Least squares” (average sum of squared distances from regression line) = 295
• 454 – 295 = 159 159 / 454 = .35
• Variance is reduced 35% by calculating from regression line
r2 = % of variance in WEIGHT
“explained” by HEIGHT
Correlation coefficient = r
Correlation: HEIGHT with WEIGHT HEIGHT WEIGHT HEIGHT 1.0000 .5925 ( 42) ( 42) P= . P= .000 WEIGHT .5925 1.0000 ( 42) ( 42) P= .000 P= .
r = .59
r2 = .35
HEIGHT “explains” 35% of variance in WEIGHT
Sentence & G.P.A.
• Regression: form of relationship
• Correlation: strength of relationship
• p value: statistical significance
grade point average
grade point average
4.003.903.803.753.703.403.333.203.00
Fre
qu
en
cy
7
6
5
4
3
2
1
0
Statistics
grade point average23
1
3.5752
.35057
.12290
Valid
Missing
N
Mean
Std. Deviation
Variance
G. P. A.
jail sentence
jail sentence
18.0012.0011.009.006.004.003.002.00.00
Fre
qu
en
cy
7
6
5
4
3
2
1
0
Statistics
jail sentence24
0
5.1250
5.44788
29.67935
Valid
Missing
N
Mean
Std. Deviation
Variance
Length of Sentence (simulated data)
grade point average
4.24.03.83.63.43.23.02.8
jail
sen
ten
ce
20
10
0
-10
Scatterplot: Sentence on G.P.A.
Regression Coefficients
Coefficientsa
17.853 12.097 1.476 .155
-3.534 3.368 -.223 -1.049 .306
(Constant)
grade point average
Model1
B Std. Error
UnstandardizedCoefficients
Beta
StandardizedCoefficients
t Sig.
Dependent Variable: jail sentencea.
Sentence = -3.5 G.P.A. + 18
grade point average
4.24.03.83.63.43.23.02.8
jail
sen
ten
ce20
10
0
-10
Sent = -3.5 GPA + 18
“Least Squares” Regression Line
Correlation: Sentence & G.P.A.
Correlations
1 -.223
. .306
23 23
-.223 1
.306 .
23 24
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
grade point average
jail sentence
grade pointaverage jail sentence
Interpreting Correlations
• r = -22 p = .31
• r2 = .05
G.P.A. “explains” 5% of the variance in length of sentence
Write Results
“A regression analysis finds that each higher unit of GPA is associated with a 3.5 month decrease in sentence length, but this correlation was low (r = -.22) and not statistically significant (p = .31).”
Multiple Regression
• Problem: relationship of weight and calorie consumption
• Both weight and calorie consumption related to height
• Need to “control for” height
Regression line
mean
Multiple Regression
Multiple Regression
• Regress weight residuals (dependent variable) on height (independent variable)
• Statistically “controls” for height: removes effect or “confound” of height .