1 of 47 Department of Cognitive Science Inferential Statistics II: Introduction to ANOVA Michael J....

1 of 47

Department of Cognitive Science

Inferential Statistics II: Introduction to ANOVA

Michael J. Kalsher

MGMT 6970 PSYCHOMETRICS © 2012, Michael Kalsher

2 of 47

ANOVA: A Framework• Understand the basic principles of ANOVA

– Why it is done?– What it tells us?

• Theory of one-way independent ANOVA

• Following up an ANOVA:– Planned Contrasts/Comparisons

• Choosing Contrasts• Coding Contrasts

– Post Hoc Tests

• Writing up Results

3 of 47

Why ANOVA?

• t tests are limited to situations in which there are only two levels of a single independent variable or two associated groups.

• There are many instances in which we’d like to compare more than two levels. But … performing multiple t tests can inflate the Type I error rate.

• The concept of “Omnibus test.”

• Familywise error: 1 - (.95)n

4 of 47

What is ANOVATests the null hypothesis that three or more means are roughly equal.

Produces a test statistic termed the F-ratio.

Systematic Variance (SSM)

Unsystematic Variance (SSR)

The F-ratio tells us only that the experimental manipulation has had an effect—not where the effect has occurred.

-- Planned comparisons (before you collect the data)

-- Post-hoc tests (after-the-fact “snooping”).

-- Both conducted to control the overall Type I error rate at 5%.

-- Guards against familywise error [1 - (.95)n ]

-- For a study with 3 conditions, Type 1 error rate = 14.3%!

5 of 47

ANOVA as RegressionSuppose we are interested in determining whether the drug Viagra is effective at making someone a better lover. A group of men are then randomly assigned to one of three groups: (1) Placebo (sugar pills); (2) Low-dose Viagra; or (3) High-dose Viagra. The dependent measure is an “objective” measure of libido. The data are below:

6 of 47

ANOVA as RegressionTo predict levels of libido from the different levels of

Viagra, we can use the general equation:

Outcomei = (model) + errori

7 of 47

Model for the Placebo group (base category)

We are predicting the level of libido when both doses of Viagra are ignored. Thus, the predicted value will be the mean of the placebo group.

The intercept, b0, is always equal to the mean of the base category

8 of 47

Model for the High-dose group

b2 represents the difference between the means of the High-dose group and the Placebo (base) group.

9 of 47

Model for the Low-dose group

b1 represents the difference between the means of the Low-dose group and the Placebo (base) group.

10 of 47

Regression Output

b1 = Low Dose

b2 = High Dose

Note:

b2 = the difference between the means of the high-dose group and the placebo group (5.0 - 2.2 = 2.8).

b1 = the difference between the means of the low-dose group and the placebo group (3.2 - 2.2 = 1)

Conclusion: Using group means to predict the outcome is significantly better than using the overall mean.

11 of 47

Theory of ANOVA: Partitioning Variance

• We compare the amount of variability explained by the Model (MSM), to the error in the model [individual differences] (MSR)

– This ratio is called the F-ratio.

• If the model explains a lot more variability than it can’t explain, then the experimental manipulation has had a significant effect on the outcome (DV).

12 of 47

Partitioning the Variance

Total sum of squares (SST)

Model sum of squares (SSM)

Residual sum of squares (SSR)

13 of 47

SST = based on

differences between each data point and

the grand mean

SSM = differences

between predicted values (group means) and the

grand mean

Grand Mean = 3.467

XPlacebo = 2.20

XLowDose = 3.20

XHighDose = 5.00

SSR = based on

differences between person’s score and their group mean.

Basis for SST

Basis for SSR

Basis for SSM

Par

titi

on

ing

th

e V

aria

nce

Base Category

14 of 47

Total sum of squares (SST)Compute the difference between each data point and the grand mean, square these differences, then add them together.

1

2

3

4

Grand Mean = 3.467

Grand SD = 1.767

Grand Variance = 3.124

15 of 47

Model sum of squares (SSM)1. Calculate the difference between the mean of each group and the

grand mean.

2. Square each of these differences

3. Multiply each result by the number of participants within that group.

4. Add the values for each group together.

16 of 47

Residual sum of squares (SSR)

SSR = SST - SSM … But, don’t rely on this relationship (e.g., might not work out if either SST or SSM is miscalculated)

SSR is the difference between what the model predicts and what was actually observed. Therefore it is calculated by by looking at the difference between the score obtained by a person and the mean of the group to which the person belongs.

17 of 47

Total sum of squares (SSR)

18 of 47

Mean Squares (MSM and MSR)

SSM = amount of variation explained by the model (exp. manipulation).

SSR = amount of variation due to extraneous factors.

These are “summed” scores and will therefore be influenced by the number of scores. To eliminate this bias we calculate the average sum of squares (mean squares) by dividing by the appropriate degrees of freedom.

Calculating Degrees of Freedom (for one-way independent groups ANOVA)

dftotal = N - 1 (number of all scores minus 1)

dfM / between = k - 1 (number of groups minus 1)

df R / within = N - k (number of all scores minus number of groups)

19 of 47

The F-ratio

F = MSM/MSR and is a measure of the ratio of variation due to the experimental effect to the variation due to individual differences (unexplained variance).

Therefore, we again return to the basic conceptual model:

Test Statistic = systematic variance unsystematic variance

20 of 47

Assumptions

The one-way independent groups ANOVA test requires the following statistical assumptions:

1. Random and independent sampling.

2. Data are from normally distributed populations. Note: This test is robust against violation of this assumption if n > 30 for both groups.

3. Variances in these populations are roughly equal (Homogeneity of variance). Note: This test is robust against violation of this assumption if all group sizes are equal.

21 of 47

1-Way Independent-Groups ANOVA

Characteristics:• Used when testing 3 or more experimental groups.• Each participant contributes only one score to the data.• Used to test the (null) hypothesis that several means are equal.

Example: Testing the idea that exposure to mobile phones will lead to brain tumors.

IV: Hrs./day exposure

DV: Size of brain tumor

Mobile Phone Use

(hours per day)

22 of 47

-- Calculate the Model (or between-groups) sum of squares (SSM) and the Residual (or within-groups) sum of squares (SSR).

-- The number of scores used to calculate SSM and SSR are different, so to make them comparable, we convert them to the average sums of square or “mean squares” (MS) by dividing by their appropriate degrees of freedom.

-- F = MSM/MSR and is a measure of the ratio of variation due to the experimental effect to the variation due to individual differences (unexplained variance).

Steps in the Analysis

23 of 47

Degrees of Freedom (df): One-way Independent-Groups ANOVA

Formulas for the number of degrees of freedom in one-way independent-groups ANOVA differ for each SS term. The degrees of freedom (df) reflect the number of values free to vary in any sum of squares term.

Calculating Degrees of Freedom

dftotal = N - 1 (number of all scores minus 1)

dfbetween = k - 1 (number of groups minus 1)

df within = N - k (number of all scores minus number of groups)

24 of 47

SPSS Summary Tables

N Mean

Std.

Deviation

Std. Error

Lower

Bound

Upper

Bound Min. Max.

0

1

2

3

4

5

Total

20

20

20

20

20

20

120

.0175

.5149

1.2614

3.0216

4.8878

4.7306

2.4056

.01213

.28419

.49218

.76556

.69625

.78163

2.02662

.00271

.06355

.11005

.17118

.15569

.17478

.18500

.0119

.3819

1.0310

2.6633

4.5619

4.3648

2.0393

.0232

.6479

1.4917

3.3799

5.2137

5.0964

2.7720

.00

.00

.48

1.77

3.04

2.70

.00

.04

.94

2.34

4.31

6.05

6.14

6.14

Sum of

Squares df

Mean

Square F Sig.

Between Groups

Within Groups

Total

450.664

38.094

488.758

5

114

119

90.133

.334

269.733 .000

95% Confidence IntervalHrs. of Exposure

Size of Tumor

25 of 47

Levene’s Test: Testing the Homogeneity of Variance Assumption

If Levene’s test is significant (p<.05), then the variances are significantly different.

What to do if Levene’s test is significant?1. Transform the data

2. Report the F value and the Levene’s test results and allow your audience to assess the accuracy of your results for themselves.

Levene

Statistic df1 df2 Sig.

10.245 5 114 .000

26 of 47

Computing Effect Size (2 - eta squared and 2 - omega squared)Analogous to r2, eta-squared reflects the proportion of DV variance explained by each IV in the sample data. It tends to be upwardly biased (i.e., an overestimate).

2 = SSModel / SStotal

• One 2 for each effect (i.e., 1 per IV plus 1 per interaction) and the 2s sum to 1.• SPSS reports partial eta squared (p

2 = SSEffect / SSEffect+ SSError)

• In 1-way ANOVA, eta-squared and partial eta-squared are the same.

An alternative, omega-squared (2), estimates the proportion of DV variance explained by the IVs in the population (see Field, 2005, p. 357-359).

27 of 47

2 = SSM = 450.66 = .922

SST 488.76

2 = SSM - (dfM)(MSR) = 450.664 - (5 x .334) SST + MSR 488.758 + .334

= 449.01 = .918 489.09

Computing Effect Size (2 vs. 2 omega squared)

(called eta squared, 2 )

(called omega squared, 2 )

28 of 47

Reporting the Results

Levene’s test indicated that the assumption of homogeneity of variance had been violated, F(5, 114) = 10.25, p<.001. Transforming the data did not rectify this problem and so F tests are reported nonetheless. The results show that using a mobile phone significantly affected the size of brain tumor found in participants, F(5, 114) = 269.73, p<.001, 2 = .92. Tumor size (measured in mm) increased with the duration of exposure. Specifically, tumor size for 0, 1, 2, 3, 4 or 5 hours/day of exposure was .02 mm, .51 mm, 1.26 mm, 3.02 mm, 4.89 mm, and 4.73 mm, respectively. The effect size indicated that the effect of phone use on tumor size was substantial. Games-Howell post hoc tests revealed significant differences between all groups (p<.001 for all tests) except between 4- and 5-hours (ns).

29 of 47

Computing a One-way Independent-groups

ANOVA by hand

30 of 47

31 of 47

32 of 47

Critical Values for F

33 of 47

2 = SSM = 93.17 = .69

SST 134.67

2 = SSM - (dfM x MSR) = 93.17 - (2 x 4.61) MSR + SST 4.61 + 134.67

= 83.95 = .60 139.28

Computing Effect Size (2 vs 2 Omega squared)

34 of 47

Computing the One-way Independent-groups ANOVA using SPSS

35 of 47

SPSS Variable View: Assign Value Labels

36 of 47

SPSS Data View: Type in the Data

37 of 47

SPSS Data Editor: Select Procedure

38 of 47

38

Specify IV and DV, then select “options”

Select “Descriptives” and “Homogeneity test”. If the variances are unequal, choose “Brown-Forsythe” or “Welch” tests.

39 of 47

39

Choosing Follow-up tests: Contrasts vs. Post-Hoc

Select “Contrasts” if you have apriori hypotheses.

Select “Post-Hoc” if you do not.

40 of 47

40

Planned Contrasts (if we have specific hypotheses)

The first contrast compares the combined “experimental drug groups” against the control (Saltwater) group.

The second contrast compares the the two experimental groups, Fenfluramine vs. Amphetamine.

41 of 47

41

Post-Hoc Tests (if we do NOT have specific hypotheses)

LSD (Least Significant Difference) does not control Type I error rate. Bonferroni tightly controls Type I error rate. Other post-hoc procedures have specific applications (see Field text).

42 of 47

SPSS ANOVA Output: Descriptives and ANOVA Summary Table

43 of 47

SPSS ANOVA Output: Planned Contrasts

Levene’s test was not significant (p>.05) and so we can assume equal variances.In the Contrast Tests summary, use the “Assume equal variances” row values.

Contrast 1: weight gain among rats receiving the experimental drugs is significantly different from rats receiving saltwater.

Contrast 2: weight gain among rats taking fenfluramine and amphetamine are not significantly different.

44 of 47

44

Post-hoc testsAfter the Fact “Snooping”: Some Considerations

LSD does not control Type I error rate, whereas the Bonferroni test tightly controls the Type I error rate.

Given equal sample sizes and equal variances, use REGWQ or Tukey HSD, as both have good power and tight control over Type I error rate.

If sample sizes across groups are slightly different, use Gabriel’s procedure because it has greater power, but if sample sizes are very different use Hochberg’s GT2.

If there is any doubt that the population variances are equal, use the Games-Howell procedure, in place of, or in addition to other tests.

45 of 47

Post-Hoc tests: LSD vs Bonferroni

46 of 47

Reporting the ResultsThe results show that the amount of weight gained by the rats was significantly affected by the type of drug they received, F(2,9) = 10.10, p < .01, 2 = .60. Specifically, weight gain was greatest for rats receiving Saltwater (M=10.0, SE=.82) and least for rats receiving Fenfluramine (M=3.25, SE=1.11). Weight gain for rats receiving Amphetamine was intermediate (M=5.75, SE=1.25).

If you used planned comparisons …Planned comparisons revealed a significant difference in weight gain between the combined experimental groups and the saltwater control group, t(9)=-4.18, p<.05 (1-tailed). The difference in weight gain between the two experimental groups receiving fenfluramine and amphetamine, respectively, was not significant, t(9)= -1.65, p>.05 (1-tailed).

If you used post-hoc tests …Bonferroni post hoc tests revealed a significant difference in weight gain between the Fenfluramine and Saltwater groups (p<.05). No other comparisons were significant (ps>.05).

47 of 47

Sample Problem: Your turn…