7/25/2019 A Nova Sumner 2016

1/23

ANOVA

Jeremy SumnerMaths and Physics, University of Tasmania

KMA711, June 2016

7/25/2019 A Nova Sumner 2016

2/23

Useful resources

http://rtutorialseries.blogspot.com.au/

http://www.r-tutor.com/elementary-statistics/analysis-variance

http://www.statmethods.net/stats/anova.html

http://www.r-bloggers.com/one-way-analysis-of-variance-anova/

http://www.stat.columbia.edu/martin/W2024/R3.pdf

7/25/2019 A Nova Sumner 2016

3/23

Time to coagulation vs. diet

7/25/2019 A Nova Sumner 2016

4/23

When is ANOVA a useful tool?

Categorical explanatory variable(s)DATA:

Continuousresponse variable

e.g. Time to blood coagulation (response variable) under different diets(explanatory variable). . .

Analysis goals: Is the response variable significantly affected bythe different levels (A,B,C,D) of the explanatory variable?

i.e. Does diet affect time to coagulation?

7/25/2019 A Nova Sumner 2016

5/23

ANOVA basics and assumptions

ANOVA is pretty much the same idea as regression exceptexplanatory variables are categorical/factors.

Coagulate TimeDiet exampleDATAis Time =yij, for Dieti=A, B, C, Dand observationj

NULL: different groups/levels for the categorical/factorvariables make no difference to the response.

ASSUMPTION:ANOVA assumes the response variable is normally dis-tributed with identical variance about the group means.

The idea is to compare the withingroups to the betweengroup variation.

Simple ANOVA model: yij=i+ ij, where errors areindependent and N(0, 2).

7/25/2019 A Nova Sumner 2016

6/23

ANOVA in a nutshell

Question: How can we statistically quantify what we see in

the box plots?Answer:

1 Use data to compute summary statistic F2 Assuming NULL (no effect), useF to compute a p-value

7/25/2019 A Nova Sumner 2016

7/23

What is a p-value again?

A p-value is the probability of observing the data (or

more extreme) if the NULL hypothesis is true

i.e. What is the probability of seeing box plots like this if dietmakes no difference?

7/25/2019 A Nova Sumner 2016

8/23

ANOVA basics

Time to coagulation by Diet

NULL: A = B= C= D

Groups normal about their mean

ALT: i= jBoxes: withingroup variation

Averages: betweengroup variation

Model: yij=i+ random

7/25/2019 A Nova Sumner 2016

9/23

ANOVA in a nutshell PART 2

withinvariation less than between variation means the

diet choice probably matters

this implies small p-value, so NULL hypothesis is probablyfalse

7/25/2019 A Nova Sumner 2016

10/23

Summary statistics

DATA assumptions

yij=i+ ij= + i+ ij

yij isjth sample fromith group

is the grand meani= +iare group means

ij N(0, 2)

Group Data Dist1 y11, y12, . . . , y1N1 N(1, 2)

2 y21, y22, . . . , y2N2 N(2, 2)

3 y31, y32, . . . , y3N3 N(3, 2)

......

...

k yk1, yk2, . . . , ykNk N(k, 2)

ANOVA summary statistics

Sample means: yi= 1Ni

Nij=1yij

Sample variances: s2i = 1Ni1

Nij=1(yij yi)2

Size Mean Variance

N1 y1 s2

1N2 y2 s

22

N3 y3 s23

......

...Nk yk s

2k

7/25/2019 A Nova Sumner 2016

11/23

ANOVA in a nutshell PART 2

withinvariation MSEshould roughly equal 2

between variation MS Groupsshould roughly equal 2

F statistic: F = MSGroupsMSE

= 1?

p-value is probability of observing F >1

7/25/2019 A Nova Sumner 2016

12/23

Some equations and the F-test

Uses2i to obtain a pooled estimate of2:

MSE = SSE

k 1= s

2p =

(N1 1)s21 + (N2 1)s22 +. . .

+ (Nk 1)s2ki(Ni 1)

=

ki=1

Nij=1

yij yi

2

N k=

total variation around group means

# data points # of means computed

Under NULL yi

N(, 2/Ni), so independentestimate of2:

MS Groups =SS Groups

k 1=

k

i=1

(yi y)2

k 1=

variation of group means about grand mean

# groups 1

Total variation: SST =

i,j(yij y)2

MIRACLE OF ANOVA: SST= SS Groups + SSE

Under NULL, the ratio F = MSGroupsMSE

should be F-distributed about 1with numerator DF (k 1) and denominator DF (N k).

SS Groups is what is explained by the separate group means and

SSE is what is left over.

7/25/2019 A Nova Sumner 2016

13/23

Simple ANOVA outputs

Generic tableSource df SS MS F

Between k 1 SS Groups = iNi(yi y)2 SS Groupsk1 F = MSGroupsMSEWithin N

k SSE = i(Ni

1) s2iSSENk

Total n 1 SST =i,j(yij y)2

R output for the Diet data

Df Sum Sq Mean Sq F value Pr(>F)Diet 3 228 76.0 13.57 4.66e-05 ***Residuals 20 112 5.6

Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1

7/25/2019 A Nova Sumner 2016

14/23

ANOVA as regression

Consider regression model on 4 diets:

time =0(diet A)+1(diet B)+2(diet C)+3(diet D)(diet A) is the indicator function: = 0 or 1

In R: 0 =A, 1 = (B A), 2 = (C A), 3 = (D A)

Regression style outputs

Coefficients:Estimate Std. Error t value Pr(>|t|)

(Intercept) 6.100e+01 1.183e+00 51.554 < 2e-16 ***DietB 5.000e+00 1.528e+00 3.273 0.003803 **DietC 7.000e+00 1.528e+00 4.583 0.000181 ***DietD -3.333e-15 1.449e+00 0.000 1.000000

Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1

7/25/2019 A Nova Sumner 2016

15/23

Plot checks for ANOVA assumptions

Visual checks for heteroscedascity, non-linearity, normality, and leverage:

7/25/2019 A Nova Sumner 2016

16/23

Test for ANOVA assumptions

Bartlett Test for equal variance across groups:

bartlett.test(Time

Diet,data=d)

Essentially simultaneously compares ratio of pooled varianceto variance for each group

This test is sensitive to departures from normality.

7/25/2019 A Nova Sumner 2016

17/23

Pairwise comparisions: A vs. B? A vs. C? B vs. D? . . .

Which diet is best A,B,C,or D?Imagine doing multiple pairwise comparisons i=jusing at-test at 95% = 100 (1 )% confidence levelType I error: Chance of rejecting null hypo when we shouldnthave, ie. = 0.05

We are guaranteed to stuff up 5% of the time!

Under multiple tests, we will eventually make a Type I error

Bonferroni: multiply p-value by # of tests k

Conservative. Chance of observing at least one ofkevents isless than the sum of probs for each event, ie.

7/25/2019 A Nova Sumner 2016

18/23

Contrast choice and data dredging

Standard ANOVA null hypothesis: 1=2=. . .= k.Contrasts: 1 2= 0 and 1 12 (2+ 3) = 0.Often the treatment structurewill suggest useful contrasts.

Snail tissue.

{LL,LH,HL,HH

}: humidity L/H, and temp L/H.

Contrast Comparison12 (LH+ HH) 12 (HL+ LL) Temperature matters12 (HL+ HH) 12 (LH+ LL) Humidity matters12 (LL+ HH) 12 (LH+ HL) Same matters

Be VERY careful if using data to suggest a contrast!In the lab we look at planned and un-plannedcomparisons.

7/25/2019 A Nova Sumner 2016

19/23

What to do when data is not normal?

Is it possible to apply a transformation to the responsevariable?

yij ory2ij or log(yij)?

Box-Cox :

y()ij =

(yij 1)/ = 0

log(yij) = 0

And worry a bit more to find best sensible using ML

Use when Bartlett test says unequal variances.

Welch s method: essentially a series of t-tests but doesntpool variances across groups

In R: pairwise.t.test(d$Time ,d$Diet,pool.sd=F),adjusted p-values

7/25/2019 A Nova Sumner 2016

20/23

What to do when data is not normal?

To the rescue! Non-parametric testsCareful! Valid for a wider range of distributions but lose power

Skewed data or extreme outliers?

Analysis of Medians:

Kruskal-Wallis : rank based, average rank for each group,variation in these rank-averages in analysed

Moods Median test: Contingency table, greater than grandmedian? less than? 2 test

Both assume groups have same shaped distribution, andKruskal-Wallis is more powerful than Mood

pairwise Wilcoxon: assumes roughly symmetric distribution,rank based, Holm adjusted p-values

7/25/2019 A Nova Sumner 2016

21/23

Why not always use a non-parametric test?

Its easier to reject NULL using tests with strong assumptions.

Type I error, : rejected NULL and shouldnt haveType II error, : kept NULL and should have rejected

power = (1 ), ie. power to reject.

Null hypothesis Weak assumptions Strong assumptions

(or model) (eg. non-parametric) (eg. Simple ANOVA)# parameters many (!?) few

p-values large small

power low high

fit good bad

Type I rate low high

Type II rate high low

Sample size large small

Bias low high

Variance high low

7/25/2019 A Nova Sumner 2016

22/23

Factorial ANOVA

Effect of sleeping tablets AND alcohol.

More than the sum of the parts= interaction

Model: ResponseSleep Tab + Alcohol + Sleep Tab:AlcoholFactorial design: All treatments in all combinations.

Eg. 5 people given nothing, 5 given sleep tabs, 5 givenalcohol, and 5 given sleep tabs & alcohol (ST:A).

In factorial ANOVA, the main effect is the effect of eachvariable separately, but now also have an interaction effect.

General model: yijk= + i+ j+ ij+ ijk with ijkindependent and N(0, 2).

ij is the interaction term.

7/25/2019 A Nova Sumner 2016

23/23

Take home messages

You should always double check that your data satisfies theassumptions of the method you are applying.

The more you can assume the better, as you can use a morepowerful test and hence reduce Type II error

For ANOVA there is a sequence of assumptions across groups:

normal with identical variance. . . normal without identicalvariance . . . not normal but same shape. . . completely nuts

The equal means null hypothesis is a good start, but if itsfalse we always want to know more this is where contrasts

come in.Multiple tests lead to increased chance of Type I error

Contrasts are great, but p-values must be corrected formultiple testing, AND dont use the data to suggest contrasts.