Chapter 13

Chapter 13

Analysis of Variance (ANOVA)


ANOVA can be used to test for differences between three or more means. The hypotheses for an ANOVA are always:

H0: m1 = m2 = . . . = mk

(where k is the number of groups)

Ha: Not all of the population means are equal

Analysis of Variance: Assumptions

•For each population the response variable is normally distributed•The variance of the response variable, s2, is the same for all of the populations•The observations must be independent


The ANOVA hypothesis test is based on a comparison of the variation between groups (treatments) and within groups (treatments).

Between vs. Within VariationFamily Cars Attribute 1 Attribute 2

1 1 A X2 1 A Y3 1 A Z4 2 B X5 2 B Y6 2 B Z7 3 C X8 3 C Y9 3 C Z

Between vs. Within Variation

Group A Group B Group C1 2 31 2 31 2 3

Case 1: All variation is due to differences between groups

Between vs. Within Variation

Group X Group Y Group Z1 1 12 2 23 3 3

Case 2: All variation is due to differences within groups


If the variation is primarily due to differences between groups then we would conclude the means are different and reject H0.

If the variation is primarily due to differences within the groups then we would conclude the means are the same and accept H0.

Analysis of Variance (ANOVA)The relative sizes of between and within group variation are measured by comparing two estimates of the variance.

nsx̄�2 is used to estimate nsx̄�

2 and s2. If the means are equal nsx̄�

2 will be an unbiased estimator of s2. If the means are not equal nsx̄�

2 will overestimate s2.

22

xn

n

n

x

x

Sampling Distribution of Given H0 is True

1x 3x2x

Sample means are close together because they are drawn from the same sampling distribution when H0 is true.

22x n

ANOVAx

Sampling Distribution of Given H0 is False

33 1x 2x3x 11 22

Sample means come from different sampling distributionsand are not as close together when H0 is false.

ANOVAx


The second way of estimating the population variance is to find the average of the variances of the different groups.

This approach provides an unbiased estimate regardless whether or not the null hypothesis is true.

k

k

jjs

1

2


ks

nsk

jj

x

1

2

2

If we take the ratio of the two approaches we have a measure that has an expected value of 1 if the null hypothesis is true. It will be larger than 1 if the null hypothesis is false.


If the null hypothesis is true and the conditions for conducting the ANOVA test are met then the sampling distribution of the ratio is an F distribution with k-1 degrees of freedom in the numerator and nT – k degrees of freedom in the denominator.

F Distribution

a

F Distribution

As before a is the probability of rejecting H0 when it is true (the probability of making a Type I error).

Fa is the critical value such that an area equal to a lies in the upper-most tail.

For example, with 5 degrees of freedom in the denominator and 10 degrees of freedom in the numerator, an F value of 4.74 would capture an area equal to 0.05 in the tail.

ANOVA Hypothesis Test

The steps for conducting an ANOVA hypothesis test are the same as for conducting a hypothesis test of the mean:

1. State the hypotheses2. State the rejection rule3. Calculate the test statistic4. State the result of the test and its implications

ANOVA Table

Typically when we do the test we organize the calculations in table with a specific format:

Source of variation

Sum of Squares

Degrees of freedom

Mean Square

F

Treatments SSTR k-1 MSTR FError SSE nT-k MSE

Total SST nT-1

Between-Treatments Estimateof Population Variance

Denominator is thedegrees of freedomassociated with SSTR

Numerator is calledthe sum of squares dueto treatments (SSTR)

The estimate of 2 based on the variation of the sample means is called the mean square due to treatments and is denoted by MSTR.

1

1

2

k

xxn

MSTR

k

jjj

Between-Treatments Estimateof Population Variance

21

2

1

2

11ns

k

xxn

k

xxn

MSTR

k

jj

k

jjj

Assume there are the same number of observations in each group so that nj = n.

The estimate of 2 based on the variation of the sample observations within each sample is called the mean square error and is denoted by MSE.

Within-Treatments Estimateof Population Variance s 2

Denominator is thedegrees of freedomassociated with SSE

Numerator is calledthe sum of squaresdue to error (SSE)

kn

sn

MSET

k

jjj

1

21

Assume there are the same number of observations in each group so that nj = n and nT = nk.

Within-Treatments Estimateof Population Variance

k

s

nk

sn

knk

sn

kn

sn

MSE

k

jj

k

jj

k

jj

T

k

jjj

1

2

1

2

1

2

1

2

1

1

11

ANOVA Table

With the entire data set as one sample, the formula for computing the total sum of squares, SST, is:

SST divided by its degrees of freedom nT-1 is the overall sample variance that would be obtained if we treated the entire set of observations as one data set.

SSESSTRxxSSTk

j

n

iij

j

2

1 1

MSTRSSTR

-

k 1MSTR

SSTR-

k 1

MSESSE

-n kT

MSESSE

-n kT

MSTRMSE

MSTRMSE

Source ofVariation

Sum ofSquares

Degrees ofFreedom

MeanSquare F

Treatments

Error

Total

k - 1

nT - 1

SSTR

SSE

SST

nT - k

SST is partitionedinto SSTR and SSE.

SST’s degrees of freedom(d.f.) are partitioned intoSSTR’s d.f. and SSE’s d.f.

ANOVA Table

A B C0 2 121 3 06 4 22 2 61 0 4

Assume we are interested in finding if the average number of cars owned is different for three different towns. Five people are interviewed in each town. Assume a = .05

ANOVA Example


H0: m1 = m2 = m3

Ha: Not all of the population means are equal

Reject H0 if: F > Fa F > 3.89

Given k - 1 = 3 - 1 = 2 df in the numeratorand nT - k = 15 - 3 = 12 df in the denominator

A B C0 2 121 3 06 4 22 2 61 0 4

Mean= 2 2.2 4.8

ANOVA Example

= (2+2.2+4.8)/3 = 9/3 = 3x


SSTR = 5(2-3)2 + 5(2.2-3)2 + 5(4.8-3)2

= 24.4

SSE = (0-2)2 + (1-2)2 + (6-2)2 + (2-2)2 + (1-2)2 +(2-2.2)2 + (3-2.2)2 + (4-2.2)2 + (2-2.2)2 + (0-2.2)2 +(12-4.8)2 + (0-4.8)2 + (2-4.8)2 + (6-4.8)2 + (4-4.8)2 =115.6

ANOVA Table

Source of variation

Sum of Squares

Degrees of freedom

Mean Square

F

Treatments 24.4 2 12.2 1.27Error 115.6 12 9.6Total 140 14

Anova: Single Factor

SUMMARYGroups Count Sum Average Variance

Column 1 5 10 2 5.5Column 2 5 11 2.2 2.2Column 3 5 24 4.8 21.2

ANOVASource of Variation SS df MS F P-value F crit

Between Groups 24.4 2 12.2 1.266 0.3169 3.885Within Groups 115.6 12 9.633

Total 140 14

A B C1 4 53 1 32 3 22 4 6

Assume we are interested in finding if the average number of bedrooms per home is different for three different towns. Data on four houses were collected in each town. Assume a = .05

Graded Homework

P. 401, #7 (just do a hypothesis test)

Documents

Chapter 13