Statistics for Everyone, Student Handout

Part 2: Statistics as a Tool in Scientific Research: One-Way Analysis of Variance: Comparing 2 or More Levels of a Variable

A. Terminology and Uses of the One-Way ANOVA

Independent variable (IV) (manipulated): Has different levels or conditions; e.g., Presence vs. absence (Drug, Placebo); Amount (5mg, 10 mg, 20 mg); Type (Drug A, Drug B, Drug C);

Quasi-Independent variable (not experimentally controlled; e.g., Gender)

Dependent variable (DV) (measured variable): e.g., Number of white blood cells, temperature, heart rate

Used for: Comparing differences between average scores in different conditions to find out if overall the IV influenced the DV

Use when: The IV is categorical (with 2 or more levels) and the DV is numerical (interval or ratio scale), e.g., weight as a function of race/ethnicity; # of white blood cells as a function of type of cancer treatment; mpg as a function of type of fuel

Note: Can use either t test or F test if two conditions in experiment because F test with two levels yields identical conclusions to t test with two levels. You must use the F test if there are three or more levels

# factors = # IVs = # “ways” (e.g., one-way ANOVA; two-way ANOVA; three-way ANOVA)

B. Types of One-Way F Tests

Independent Samples ANOVA: Use when you have a between-subjects design -- comparing if there is a difference between two or more separate (independent) groups

• Some people get Drug A, others get Drug B, and others get the placebo. Do the groups differ, on average, in their pain?

• Some plants are exposed to 0 hrs of artificial light, some are exposed to 3 hours, and some are exposed to 6 hours. Does the number of blooms differ, on average, as a function of amount of light?

• Some cars use Fuel A, some use Fuel B, some use Fuel C, and some use Fuel D. Do different types of fuel result in better fuel mileage, on average, than others?

Repeated Measures ANOVA: Use when you have a within-subjects design – each subject experiences all levels (all conditions) of the IV; observations are paired/dependent/matched

• Each person gets Drug A, Drug B, and the placebo at different times. Are there differences in pain relief across the different conditions?

• Each plant is exposed to 0 hrs of artificial light one week, 3 hours another week, and 6 hours another week. Do the different exposure times cause more or less blooms on average?

• Cars are filled with Fuel A at one time, Fuel B at another time, Fuel C at another time, and Fuel D at yet another time. Is there a difference in average mpg based on type of fuel?

Materials developed by L. McSweeney and L. Henkel for the Quantitative Reasoning Pathway of the Core Integration Initiative

C. Hypothesis Testing Using F Test

The F test allows a researcher to determine whether their research hypothesis is supported by determining if overall there is a real effect of the IV on the DV, Is the MAIN EFFECT of the IV on the DV significant?

An ANOVA will just tell you yes or no – the main effect is significant or not significant. It does NOT tell you which conditions are really different from each other

So all hypotheses are stated as “there are real differences between the conditions” vs. “there are not real differences between the conditions”

Null hypothesis H0: • The IV does not influence the DV• Any differences in average scores between the different conditions are probably just due to chance

(measurement error, random sampling error)

Research hypothesis HA: • The IV does influence the DV• The differences in average scores between the different conditions are probably not due to chance

but show a real effect of the IV on the DV

Null hypothesis: Average pain relief is the same whether people have Drug A, Drug B, or a placebo Research hypothesis: Average pain relief differs whether people have Drug A, Drug B, or a placebo

Null hypothesis: Plants exposed to 0, 3, or 6 hours of artificial light have the same number of blooms, on average Research hypothesis: Plants exposed to 0, 3, or 6 hours of artificial light have a different number of blooms, on average

Null hypothesis: For all fuel types (A, B, C and D), cars get the same average mpgResearch hypothesis: There is a difference in average mpg based on fuel type (A, B, C and D)

The question at hand is: Do the data/evidence support the research hypothesis or not? Did the IV really influence the DV, or are the obtained differences in averages between conditions just due to chance?

D. Understanding Probability: What Do We Mean by “Just Due to Chance”?

p value = probability of results being due to chance

When the p value is high (p > .05), the obtained difference is probably due to chance; .99 .75 .55 .25 .15 .10 .07

When the p value is low (p < .05), the obtained difference is probably NOT due to chance and more likely reflects a real influence of the IV on DV; .04 .03 .02 .01 .001

In science, a p value of .05 is a conventionally accepted cutoff point for saying when a result is more likely due to chance or more likely due to a real effect

Not significant = the obtained difference is probably due to chance; the IV does not appear to have a real influence on the DV; p > .05

Statistically significant = the obtained difference is probably NOT due to chance and is likely due to a real influence of the IV on DV; p < .05

2

E. The Essence of an F Test for a Main Effect

An F test for a main effect answers the question: Is the research hypothesis supported? In other words, did the IV really influence the DV, or are the obtained differences in the averages between conditions just due to chance?

It answers this question by calculating an F value. The F value basically examines how large the difference between the average score in each condition is, relative to how far spread out you would expect scores to be just based on chance (i.e., if there really was no effect of the IV on the DV)

The key to understanding ANOVA is understanding that it literally analyzes the variance in scores. The question really is why are all the scores not exactly the same? Why is there any variability at all, and what accounts for it?

Suppose 5 patients took Drug A, 5 took Drug B, and 5 took a placebo, and then they had to rate how energetic they felt on a 10-pt scale where 1=not at all energetic and 10=very energetic. You want to know whether their energy rating differed as a function of what drug they took.

Drug A Drug B Placebo

10 5 47 1 6

5 3 9

10 7 3

8 4 3

M1=8 M2 = 4 M3 = 5

Why didn’t every single patient give the same exact rating? The average of these 15 ratings is 5.67 (SD=2.77). So why didn’t everybody give about a 5? One reason is that the drug is what did it, that the drug made people feel more energetic (this is variability due to the influence of the treatment)Another reason different people feel things differently and some people are naturally upbeat and optimistic and tend to give higher ratings than people who are somber and morose and give low ratings. The ratings differ because people differ (this is variability due to uncontrolled factors – called error variance -- unique individual differences of people in your sample, random sampling error, and even measurement error [the subjectivity or bias inherent in measuring something])

So scores vary due to the treatment effect (between groups variance) and due to uncontrolled error variance (within groups variance)

3

For example, why didn’t everybody who took Drug A give the same rating? Within a group, scores vary because of:

• Inherent variability in individuals in sample (people are different)• Random sampling error (maybe this group of 5 people is different than some other group

of 5)• Measurement error (maybe some people use a rating of 5 differently than other people do)

This is called within-treatments variability (also called error variance): It is the average spread of ratings around the mean of a given condition (spread around A + spread around B + spread around C)

Drug A Drug B Placebo

10 5 4

7 1 6

5 3 9

10 7 3

8 4 3

M1 = 8M2 = 4 M3 = 5

Why didn’t all 15 people give the same exact rating, say about a 5? Between the different groups, ratings vary because of the influence of the IV (e.g., one of the drugs increases how energetic someone feels). This is called between-treatments variability, and it equals the spread of means around grand mean, where GM = (M1+M2+M3)/# conditions

Drug A Drug B Placebo

10 5 4

7 1 6

5 3 9

10 7 3

8 4 3

M1 = 8M2 = 4 M3 = 5

4

Understanding the source of variance – where the variance comes from – is the heart of what an ANOVA does because it literally is an Analysis Of the Variance

 

Sources of Variance in ANOVATotal variance in scores

Variance within groups Variance between groupsAverage variability of scores within each condition around the mean of that condition (X-M)2

Doesn’t matter if H0 is true or not

Represents error variance (inherent variability from individual differences, random sampling error, measurement error)

Average variability of the means of each group around the grand mean (M-GM)2

Represents error variance PLUS variance due to differences between conditions

If H0 is true, then the only variance is error variance

If H0 is not true, then there is both error variance and treatment variance

Each F test has certain values for degrees of freedom (df), which is based on the sample size (N) and number of conditions, and the F value will be associated with a particular p value

SPSS calculates these numbers. Their calculation differs depending on the design (independent-samples or a repeated-measures design)

F. Summary Table for One-Way Independent-Samples (Between-subjects) Design

Source Sum of Squares Df Mean square F

5

Between (MGM)2 k 1 k = # levels of IV

SSbetweendfbetween

MSbetweenMSwithin

Within(error)

(XM)2 (n1) n = # subjects in a given condition

SSwithindfwithin

 

Total (XGM)2 N1 N= total # subjects

   

To report, use this format: F(dfbetween, dfwithin) = x.xx, p _____.

6

G. Understanding the F Test

A test for a main effect gives you an F ratio: The bigger the F value, the less likely the difference between conditions is just due to chance

The bigger the F value, the more likely the difference between conditions is due to a real effect of the IV on the DV

So big values of F will be associated with small p values that indicate the differences are significant (p < .05)

Little values of F (i.e., close to 1) will be associated with larger p values that indicate the differences are not significant (p > .05)

H. Understanding the Repeated-Samples (Within Subjects) ANOVA

The basic idea is the same when using a repeated-measures (within-subjects) ANOVA as when using an independent-samples (between-subjects) ANOVA: An F value and corresponding p value indicate whether the main effect is significant or not

However, the repeated-measures ANOVA is calculated differently because it removes the variability that is due to individual differences (each subject is tested in each condition so variability due to unique individual differences is no longer uncontrolled error variance). Hence the formulas differ:

Source SS df MS F Between treatments SSbet treat k1 MSbet MSbet treat/MSerrorWithin treatments SSwith (nx1)

Between subjects SSbet subjs n1 (# per cond)Error SSerror dfwith-dfbet subjs MSerror

Total SStot N1

To report, use this format: F(dfbet treat, dferror) = x.xx, p _____

I. Interpreting One-Way ANOVA Results

Cardinal rule: Scientists do not say “prove”! Conclusions are based on probability (likely due to chance, likely a real effect…).

Based on p value, determine whether you have evidence to conclude the difference was probably real or was probably due to chance: Is the research hypothesis supported?

p < .05: Significant• Reject null hypothesis and support research hypothesis (the difference was probably real; the IV likely influences the DV)

p > .05: Not significant• Retain null hypothesis and do not accept the research hypothesis (any difference was probably due to chance; the IV did not influence the DV)

7

If the F value is associated with a p value < .05, then your main effect is significant. The answer to the question: Did the IV really influence the DV?” is “yes.”

If the F value is associated with a p value > .05, then the main effect is NOT significant. The answer to the question: Did the IV really influence the DV?” is “no.”

If the main effect is significant, all you know is that at least one of the conditions is different from the others. You need to run additional comparisons to determine which specific conditions really differ from the other conditions: Is A different from B? Is A different from C? Is B different from C?

Each of these different patterns would show a significant main effect – additional comparisons are needed to understand which conditions are really different from each other

J. Use of Follow Up Comparisons When Main Effect is Significant

There are different statistical procedures one can use to “tease apart” a significant main effect Bonferroni procedure: use a = .01 (lose power though) Planned comparisons (a priori comparisons, contrasts, t tests) Post hoc comparisons (e.g., Scheffe, Tukeys HSD, Newman-Keuls, Fishers protected t)

Recommendations:If you have clear cut hypotheses about expected differences and only 3 or 4 levels of the IV, you can run pairwise t tests (compare A to B, A to C, A to D, B to C, B to D, C to D). Be sure use the t test appropriate to the design: between-subjects (independent) or within-subjects (paired)

If not, use the Scheffe post hoc test to examine pairwise differences

8

0123456789

10

Drug A Drug B Placebo

Drug Condition

Ener

getic

ness

Rat

ing

0123456789

10

Drug A Drug B Placebo

Drug Condition

Ener

getic

ness

Rat

ing

Note: SPSS will allow you to run post hoc tests only on between subjects factors, not on within subjects factors

Pairwise follow up comparisons look at the difference in average scores in two conditions, relative to how much variability there is within each condition

An F test can answer the question “Overall, did the IV really influence the DV?” by looking at differences between 2 or more conditions

Pairwise comparisons also answer that same question (“Did the IV really influence the DV?”) but because they involve only two conditions, when the answer is “yes” (i.e., when the test is significant; p < .05), that means that the difference between the conditions is probably real (e.g., scores in Condition A really are lower than scores in Condition B)

Planned t tests to look at average differences between two levels of an IV: Each t test gives you a t score, which can be positive or negative; It’s the absolute value that matters

The bigger the |t| score, the less likely the difference between conditions is just due to chance

The bigger the |t| score, the more likely the difference between conditions is due to a real effect of the IV on the DV

So big values of |t| will be associated with small p values that indicate the differences are significant (p < .05)

Little values of |t| (i.e., close to 0) will be associated with larger p values that indicate the differences are not significant (p > .05)

Post Hoc comparisons (e.g., Scheffe test) to look at average differences between two levels of an IV: The Scheffe test is interpreted much the same way a t test is: Each test will have a p value, indicating the probability that the mean difference between conditions is likely due to chance

As always, small p values indicate the difference is significant, i.e., is probably real (p < .05)

Larger p values indicate the difference is not significant, i.e., is probably due to chance (p > .05)

Note: Post hoc tests, such as the Scheffe test, protect against the risk of an increased experiment-wise Type I error rate, and hence are preferable to t tests when there are more than 4 conditions

9

K. Running the One-Way Independent Samples ANOVA for Between Subjects Design

Setting up SPSS Data FileTwo columns, one for the IV (use value labels, e.g., 1=Drug A, 2=Drug B, 3=placebo), one for the DV

Drug Type (IV)Energy Rating (DV)1 91 81 82 52 62 33 23 43 4

1. Analyze ® Compare means ® One-way ANOVA 2. Send your DV to the box labeled “Dependent list” 3. Send your IV to the box labeled “Factor”4. Click on “Options,” check the box that say “Descriptive statistics” and then “Continue”5. Hit “Ok” and the analysis will run

The output file will contain several parts. Look for the following important elements.

10

Running Follow Up T Test Comparisons for Between Subjects DesignIf the main effect was significant, you could run pairwise t tests comparing conditions to each other:Analyze ® Compare means ® Independent samples t test You have to run each t test separatelyTest variable = DV; Grouping variable = IV For the first one, define groups as 1 and 2 as the codes; run the t test; then define groups as 1 and 3, run; then 2 and 3, etc. You have to do this separately for each t test

Here is an example of what the output looks like for one of the t tests:

11

Group Sta tistics

11 8.7273 1.10371 .3327815 8.3333 1.44749 .37374

DrugDrug ADrug B

Energy rat ingN Mean Std. Deviation

Std. ErrorMean

Independent Samples Test

1.981 .172 .755 24 .458 .39394 .52209 -.68359 1.47147

.787 23.936 .439 .39394 .50043 -.63904 1.42692

Equal variancesassumedEqual variancesnot assumed

Energy rat ingF Sig.

Levene's Test forEquality of Variances

t df Sig. (2-tailed)Mean

DifferenceStd. ErrorDifference Lower Upper

95% ConfidenceInterval of the

Difference

t-test for Equality of Means

Report as: t(24) = 0.75, p = .46 (not significant)

Running Follow Up Post Hoc Scheffe Tests for Between Subjects DesignIf the main effect was significant, you could instead run post hoc Scheffe tests comparing conditions to each other

1. Analyze ® Compare means ® One-way ANOVA 2. Send your DV to the box labeled “Dependent list” 3. Send your IV to the box labeled “Factor”4. Click on “Options,” check the box that say “Descriptive statistics” and then “Continue”5. Click on box that says “Post hoc” and then choose the appropriate post hoc test (Scheffe is a good one for many purposes) and then “Continue”6. Hit “Ok” and the analysis will run

Here is an example of the output for the posthoc Scheffe tests:

12

Multiple Comparisons

Dependent Variable: Energy ratingScheffe

.39394 .61306 .814 -1.1632 1.95115.11616* .59106 .000 3.6149 6.6174-.39394 .61306 .814 -1.9511 1.16324.72222* .53993 .000 3.3508 6.0936

-5.11616* .59106 .000 -6.6174 -3.6149-4.72222* .53993 .000 -6.0936 -3.3508

(J) DrugDrug BPlaceboDrug APlaceboDrug ADrug B

(I) DrugDrug A

Drug B

Placebo

MeanDifference

(I-J) Std. Error Sig. Lower Bound Upper Bound95% Confidence Interval

The mean difference is s ignificant at the .05 level.*.

L. Running the One-Way Repeated Measures ANOVA for Within Subjects Design

Setting up SPSS Data FileOne column for each level of the IV

Drug A Drug B Drug C Placebo10 5 10 29 8 10 39 6 7 410 5 9 48 6 8 27 7 9 19 5 9 2

1. Analyze ® General Linear Model ® Repeated measures2. Type in name of your IV where is says “Within-subjects factor name” 3. Type in number of levels of your IV where it says “Number of Levels”4. Click on the “Add” button and then click on “Define”5. Send your variables in order (each column) to the “Within subjects variable box”6. Click on “Options,” check the box that say “Descriptive statistics” and then “Continue”7. Hit “Ok” and the analysis will run

The output file will contain several parts. Look for the following important elements.

Report as F(3, 33) = 38.95, p < .001. [Remember, when “sig” says .000, report as p < .001]

13

Descriptive Sta tistics

8.6667 1.07309 124.9167 1.72986 128.5833 1.08362 124.0833 1.16450 12

DrugADrugBDrugCPlacebo

Mean Std. Deviation N

Tests of Within-Subjects Effe cts

Measure: MEASURE_1

208.396 3 69.465 38.950 .000208.396 1.923 108.359 38.950 .000208.396 2.322 89.739 38.950 .000208.396 1.000 208.396 38.950 .00058.854 33 1.78358.854 21.155 2.78258.854 25.545 2.30458.854 11.000 5.350

Sphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-bound

Sourcedrug

Error(drug)

Type III Sumof Squares df Mean Square F Sig.

Running Follow Up Comparisons for a One-Way Repeated Measures (Within subjects) ANOVA If the main effect was significant, you could run pairwise t tests comparing conditions to each other

1. Analyze ® Compare means ® Paired samples t test 2. You can run all 3 t tests simultaneously; send each pair of variables (1 vs. 2, 1 vs. 3, 2 vs. 3) over to the “Paired variables box”3. Click OK and the analysis will run

There are 6 t tests listed here; the MS and SDs for each condition appear in the top table, and the t test itself is in the bottom table. The t test comparing energy ratings for those who had Drug A to those who had Drug B is significant and would be reported as t(11) = 5.85, p < .001. We see from the top table that energy ratings were higher for those given Drug A (M=8.67) than for those who had Drug B (M=4.92).

14

Paired Samples Statistics

8.6667 12 1.07309 .309774.9167 12 1.72986 .499378.6667 12 1.07309 .309778.5833 12 1.08362 .312828.6667 12 1.07309 .309774.0833 12 1.16450 .336164.9167 12 1.72986 .499378.5833 12 1.08362 .312824.9167 12 1.72986 .499374.0833 12 1.16450 .336168.5833 12 1.08362 .312824.0833 12 1.16450 .33616

DrugADrugB

Pair1

DrugADrugC

Pair2

DrugAPlacebo

Pair3

DrugBDrugC

Pair4

DrugBPlacebo

Pair5

DrugCPlacebo

Pair6

Mean N Std. DeviationStd. Error

Mean

Paired Sa mples Test

3.75000 2.22077 .64108 2.33899 5.16101 5.849 11 .000.08333 .99620 .28758 -.54963 .71629 .290 11 .777

4.58333 2.02073 .58333 3.29943 5.86724 7.857 11 .000-3.66667 2.14617 .61955 -5.03028 -2.30305 -5.918 11 .000

.83333 1.58592 .45782 -.17431 1.84098 1.820 11 .0964.50000 2.06706 .59671 3.18665 5.81335 7.541 11 .000

DrugA - DrugBPair 1DrugA - DrugCPair 2DrugA - PlaceboPair 3DrugB - DrugCPair 4DrugB - PlaceboPair 5DrugC - PlaceboPair 6

Mean Std. DeviationStd. Error

Mean Lower Upper

95% ConfidenceInterval of the

Difference

Paired Differences

t df Sig. (2-tailed)

M. Reporting ANOVA Results and Follow Up Comparisons

State key findings in understandable sentences, and use descriptive and inferential statistics to supplement verbal description by putting them in parentheses and at the end of the sentence. Use a table and/or figure to illustrate findings.

Step 1: Write a sentence that clearly indicates what statistical analysis you usedA one-way ANOVA of [fill in name of IV] on [fill in name of DV] was conducted.

A [type of ANOVA and design] ANOVA was conducted to determine whether [name of DV] varied as a function of [name of IV or name of conditions]

A one-way independent samples ANOVA was conducted to determine whether people’s pulse rates varied as a function of their weight classification (obese, normal, underweight).

A repeated-measures ANOVA was conducted to determine whether calories consumed by rats varied as a function of group size (rats tested alone, tested in small groups, rats tested in large groups).

A one-way between-subjects ANOVA of drug type (Type A, B, placebo) on patient’s energy ratings was conducted.

Step 2: Report whether the main effect was significant or not; is there a real difference in the averages of the different treatment levels/conditions The main effect of [fill in name of IV] on [fill in name of DV] was significant [or not significant], F(dfbet, dferror) = X.XX [fill in F], p = xxxx.

There was [not] a significant main effect of [fill in name of IV] on [fill in name of DV], F(dfbet, dferror) = X.XX [fill in F], p = xxxx.

The main effect of weight classification on pulse rates was not significant, F(2, 134) = 1.09, p > .05. (This means that there is not a significant difference in the average pulse rate based on weight classification.)

There was a significant main effect of group size on number of calorie consumption, F(2, 45) = 12.36, p = .002.

The main effect of drug type on energy ratings was significant, F(3, 98) = 100.36, p < .001

Step 3: Report follow up comparisons if main effect was significant

Fictional example using t tests: Additional analyses revealed that patients who took Drug A gave significantly higher energy ratings on average (M = 8.00, SD = 2.12) than patients who took either Drug B (M = 4.00, SD = 2.24), t(8) = 2.90, p < .05, or the placebo (M = 5.00, SD = 2.55), t(8) = 2.02, p < .05. However, no significant difference was found in average energy ratings for patients who took Drug B or the placebo, t(8) = 0.66, p = .53. [see section below for a more in depth overview of this]

Fictional example using post hoc Scheffe tests: Post hoc Scheffe tests were conducted using an alpha level of .05. Results revealed that patients who took Drug A gave significantly higher energy ratings on average (M = 8.00, SD = 2.12) than patients who took either Drug B (M = 4.00, SD = 2.24), p = .03, or the placebo (M = 5.00, SD = 2.55), p = .02. However, no significant difference was found in average energy ratings for patients who took Drug B or the placebo, p = .55.

15

These should always be nice, easy-to-understand grammatical sentences that do not sound like “Me Tarzan, you Jane!” Your professor may want you to explicitly note whether the research hypothesis was supported or not. “Results supported the hypothesis that increased dosages of the drug would reduce the average number of white blood cells…”

When there are only 3 levels of the IV, it is best to report all 3 t tests, including nonsignificant ones. When there are 4 or more levels, you may opt to state that only significant comparisons are reported and then just report those ones

Be sure that you note the unit of measure for the DV (miles per gallon, volts, seconds, #, %). Be very specific

If using a Table or Figure showing M & SDs or SEs, you do not necessarily have to include those descriptive statistics in your sentences You can only use the word “significant” only when you mean it (i.e., the probability the results are due to chance is less than 5%). Do not use the word “significant” with adjectives (i.e., it is a mistake to think one test can be “more significant” or “less significant” than another). “Significant” is a cutoff that is either met or not met -- Just like you are either found guilty or not guilty, pregnant or not pregnant. There are no gradients. Lower p values = less likelihood the result is due to chance, not “more significant”.

A Closer Look at Reporting Significant Comparisons

Step 3a: Write a sentence that clearly indicates what pattern you saw in your data analysis – Did the conditions differ, and if so, how (i.e., which condition scored higher or lower?)

Average [Name of DV] was significantly higher/lower for [name of Condition 1] than the average for [name of Condition 2]

[Name of DV] was significantly higher/lower on average for [name of Condition 1] than for [name of Condition 2]

[Name of DV] did not significantly differ between [name of Condition 1] and [name of Condition 2] on average

The pollen count was significantly lower on average on college campuses than in the state of CT in general. (Lower tailed test)Pulse rates did not significantly differ on average whether people were obese or underweight. (Two tailed test)The number of calories on average consumed by rats was significantly higher when the rats ate alone than when they ate in groups. (Upper tailed test)

Step 3b: Tack the descriptive and inferential statistics onto the sentence• Put Ms and SDs in parentheses after the name of the condition. (Possibly include Ns for two

sample tests.)• Put the t test results at the end of the sentence using this format: t(df) = x.xx, p = .xx

The pollen count was significantly lower on average on college campuses (M=8.35, SD=2.25) than in the state of CT in general (m=9.85), t(12) = -2.40, p = .02.Pulse rates did not significantly differ on average whether people were obese (M=95 beats per minute, SD=21, N = 24) or underweight (M=91 beats per minute, SD=18, N = 23), t(45)= 0.70, p = .49.The number of calories on average consumed by rats was significantly higher when the rats ate alone (M=100.34, SD = 12.64) than when they ate in groups (M=87.65, SD = 13.43), t(22) = 2.38, p = .01.

16

A Closer Look at Reporting Nonsignificant ComparisonsIf the difference was not significant, do not write your sentences to imply that the difference was real. Not significant = no difference. One mean will no doubt be higher than the other, but if it’s not a significant difference, then the difference is probably not real, so do not interpret a direction (Saying “this is higher but no it’s really not” is silly)

Option 1: Simply say there was no significant differencePulse rates did not significantly differ on average whether people were obese (M=95 beats per minute, SD=21, N = 24) or underweight (M=91 beats per minute, SD=18, N = 23), t(45)= 0.70, p = .49.

Option 2: Word the sentence so that it is clear that the research hypothesis was not supportedCounter to the research hypothesis, pulse rates were not significantly higher on average for people who were obese (M=95 beats per minute, SD=21, N = 24) than for people who were underweight (M=91 beats per minute, SD=18, N = 23), t(45)= 0.70, p = .49.

M. Effect Size for One-Way ANOVA

When a result is found to be significant (p < .05), many researchers report the effect size as well

Significant = Was there a real difference or not? Effect size = How large the difference in scores was; Effect size: How much did the IV influence the DV? How strong was the treatment effect?

This is measured by eta squared:

h2 = (M-GM) 2 = SSbet (X-GM) 2 = SStotal

Small 0 - .20 Medium .21 - .40 Large > .40

SPSS will calculate effect size for ANOVAs. Follow the same steps to run an ANOVA as outlined above, although for a between-subjects design you will need to do this under Analyze → General Linear Model → Univariate rather than under the “one-way ANOVA” option under “Compare Means.” Under the “options” button, click on the box for “estimates of effect size” and eta-squared will appear on your output.

Step 4: Report the effect size if the main effect was significant ****THIS STEP IS OPTIONAL***After the ANOVA results are reported, say whether it was a small, medium, or large effect size, and report eta squared

There was a significant main effect of group size on number of calorie consumption, F(2, 45) = 12.36, p = .002, and the effect size was medium, h2 = .33. The main effect of drug type on energy ratings was significant, F(3, 98) = 100.36, p < .001, and the effect size was large, h2 = .57. The main effect of weight classification on pulse rates was not significant, F(2, 134) = 1.09, p > .05 No reason to report effect size because t test was not significant.

N. Check Assumptions for ANOVA17

• Numerical scale (interval or ratio) for DV

• The distribution of scores for each condition is approximately symmetric (normal) thus the mean is an appropriate measure of central tendency

• If a distribution is somewhat skewed (not symmetric) it is still acceptable to run the test as long as the sample size per condition is not too small (say, N > 30)

• Variances of populations are homogeneous (i.e., variance for Condition A is similar to variance for Condition B) [SPSS will run homogeneity of variance tests]

• Sample size per condition doesn’t have to be equal, but violations of assumptions are less serious when equal

18