22.0 Two-Way ANOVA

• Answer Questions

• Fixed, Random, and Mixed Effects

• Fractional Factorial Designs

• Filtration Example

1

22.1 Fixed, Random, and Mixed Effects

In the previous two-way ANOVA examples, we assumed (without comment)

that the two treatments had fixed effects. A factor has fixed effects if the

treatment levels are not chosen at random. In contrast, a factor has random

effects if the treatment levels are chosen at random.

In the example on the compressive strength of concrete cylinders, recall that

we had two factors: drying type, and batch. The three drying types were

specifically chosen, and thus fixed. If we had had more than one observation

for each combination of drying type and batch, then we could have done a

two-way ANOVA with interaction analysis. In that case, the batch factor

would be random, since we drew five batches from the company’s history of all

batches.

2

In a two-way ANOVA with interaction, if both factors have non-random

levels, then it is called a fixed effects design. If both factors have levels that

are chosen at random, then it is called a random effects design. And if one

factor has random levels and the other has non-random levels, then it is a

mixed effects design.

In the concrete drying example, if analyzed as a two-way ANOVA with

interaction, we would have a mixed effects model. This is because the

methods of drying are three non-randomly chosen industrial processes, but the

five batches are random.

These distinctions are important because the test statistics used to test

hypotheses in the random and mixed effects models are different from the test

statistics used in the fixed effects model.

3

Recall that for the fixed effects model, the test statistics are:

tsA =SSA/(I − 1)

SSerr/(n... − IJ)= MSA/MSerr

tsB =SSB/(J − 1)

SSerr/(n... − IJ)= MSB/MSerr

tsAB =SSAB/(I − 1)(J − 1)

SSerr/(n... − IJ)= MSAB/MSerr

In contrast, for the random effects model, the test statistics are

tsA =SSA/(I − 1)

SSAB/(I − 1)(J − 1)= MSA/MSAB

tsB =SSB/(J − 1)

SSAB/(I − 1)(J − 1)= MSB/MSAB

tsAB =SSAB/(I − 1)(J − 1)

SSerr/(n... − IJ)= MSAB/MSerr

4

For the mixed effects model, assume that factor A is fixed and factor B is

random. Then the test statistics are

tsA =SSA/(I − 1)

SSAB/(I − 1)(J − 1)= MSA/MSAB

tsB =SSB/(J − 1)

SSerr/(n... − IJ= MSB/MSerr

tsAB =SSAB/(I − 1)(J − 1)

SSerr/(n... − IJ)= MSAB/MSerr

The interaction test is always the same. For random effect tests, use the mean

squared error as the denominator; for fixed effect tests, use the mean square

for interaction as the denominator.

The situation is complex when there are more than two factors, or when

factors have both random and fixed levels. Satterthwaite’s approximation is

needed to create synthetic mean squared terms to make the tests.

5

The reason for these different test statistics is the expected value of the Mean

Squared term. For simplicity, assume that nij. = n. Then, using distribution

theory we don’t have in this class:

IE[MSerr] = σ2

and if A and B are fixed-effect factors

IE[MSA] = σ2 +Jn

I − 1

I∑

i=1

α2

i

IE[MSB] = σ2 +In

J − 1

J∑

j=1

β2

j

IE[MSAB] = σ2 +n

(J − 1)(I − 1)

I∑

i=1

J∑

j=1

(αβ)2ij

where σ2 is an estimate of the variance of ǫ.

6

But if both A and B are random-effect factors,

IE[MSA] = σ2

err + nJσ2

A + nσ2

AB

IE[MSB] = σ2

err + nIσ2

B + nσ2

AB

IE[MSAB] = σ2

err + nσ2

AB

where σ2

A, σ2

B, σ2

AB are the variances of the effect sizes for factors A, B, and

the interactions, respectively.

Thus the F-statistics are always being formed as the ratio of two Mean Squares

which, when the null hypothesis is true, are estimating the same thing.

7

22.2 Mixed-Effects Example

Consider an experiment to test four different lesson plans for the same topic,

as taught by five instructors. Each instructor taught each set of lesson four

times, to different classes. The response is the average score of the class on a

quiz at the end of the session.

The lesson plans are fixed effects. There are four, and they were not developed

at random.

The instructors are random effects. Conceptually, they are a random draw

from the pool of all possible instructors.

8

The numbers reported are from real data, but we know how to calculate them

from previous formulae.

Mixed Effects Two-Way ANOVA

Source df SS MS EMS F

Lesson 3 42 14.0 σ2 + nσ2

LI + nJI−1

∑

α2

i 14.0/3.9= 3.59

Inst. 4 54 13.5 σ2 + nIσ2

I 13.5/2.1=6.43

L*I 12 47 3.9 σ2 + nσ2

LI 3.9/2.1=1.86

error 60 126 2.1 σ2

total 79 269

At the .05 level, the F12,60 cv is 1.92, the F4,60 cv is 2.53, and the F3,12 cv is

3.49. What do we conclude?

9

22.3 Factorial Designs

A factorial design is one in which every possible combination of treatment

levels for different factors appears.

The two-way ANOVA with interaction we considered was a factorial design.

We had n observations on each of the IJ combinations of treatment levels.

If there are, say, a levels of factor A, b levels of factor B, c levels of factors

C, then a factorial design requires at least abc observations, and more if one

wants to estimate the three way interaction among the factors. This can get

expensive when experiments have many different factors.

10

To keep experimental costs under control, one approach is to use fractional

factorial designs. In these, one does not take measurements upon every

possible combination of factor levels, but only upon a very carefully chosen

few.

These few are selected to ensure that the main effects and low-order

interactions can be estimated and tested, at the expense of high-order

interactions.

The scientific intuition is that it is unlikely for there to be complex interactions

among many different factors; instead, there are probably only main effects

and a few low-order interactions.

Thus one might design the collection in a fractional factorial so that all main

effects and two-way interactions can be tested, but not three-way or higher

interactions.

11

Consider three-way ANOVA (with factors A, B, C having n observations at all

combinations of levels). The general model is

Yijkℓ = µ + αi + βj + γk + (αβ)ij + (αγ)ik + (βγ)jk + (αβγ)ijk + ǫijkℓ

where everything is a usual and (αβγ)ijk is the three-way interaction.

If the three-way interaction is significant, then we say that all the lower-order

interactions are significant. (Sometimes the calculation finds estimates of

these as zero, but fundamentally high-order interactions imply lower-order

interactions are just masked.)

Obviously, this model can be used with fixed, random, or mixed effect designs.

The model is the same for all, but the ratios taken in the ANOVA table

depend upon which factors are fixed and which are random.

12

You can always estimate the highest order interaction as:

ˆ(αβγ)ijk = Yijk. − [µ + αi + βj + γk + ˆ(αβ)ij + ˆ(αγ)ik + ˆ(βγ)jk]

where the estimates are found in the usual way:

µ = Y....

αi = Yi... − Y....

ˆ(αβ)ij = Yij.. − Yi... − Y.j.. + Y....

and so forth.

Note Bene: The SS formula get a bit more complicated when there are

unequal numbers of observations for each combination of factor levels (i.e.,

there is no fixed n). Use a statistical package.

13

A special kind of factorial design are the 2k factorials. In these, each of the

k factors have exactly 2 levels, so there are 2k different combinations of

treatment levels.

For a full factorial, one would need a minimum of 2k observations, and even

that would not allow enough degrees of freedom for the error term.

2k− 1 = k(2− 1)+

(

k

2

)

(2− 1)(2− 1)+ · · ·+

(

k

k

)

(2− 1)(2− 1) . . . (2− 1).

This is based on a standard combinatorial identity due to Pascal:

2k =

k∑

i=0

(

k

i

)

.

Typically, one uses the highest order interaction as if it were an error term, or

uses a normal probability plot.

14

Pascal corresponded with Fermat to develop the laws of probability, and

developed Pascal’s Wager. He was also a great prose stylist, and pioneered the

moder understanding of atmospheric pressure.

15

The following table shows the data for a full 23 factorial design. Note that

the signs in each interaction column can be found by multiplying the signs in

corresponding main-effect columns.

run A B C AB AC BC ABC obs

1 - - - + + + - Y111 (Y1)

2 + - - - - + + Y211 (Y2)

3 - + - - + - + Y121 (Y3)

4 + + - + - - - Y221 (Y4)

5 - - + + - - + Y112 (Y5)

6 + - + - + - - Y212 (Y6)

7 - + + - - + - Y122 (Y7)

8 + + + + + + + Y222 (Y8)

16

The estimated main effect due to factor A is the average difference between

the high and low levels of factor A, or:

α =1

2

(

Y2 + Y4 + Y6 + Y8

4−

Y1 + Y3 + Y5 + Y7

4

)

and similarly for the main effects of B and C. So, as per the signs in the table,

α =1

8(−Y1 + Y2 − Y3 + Y4 − Y5 + Y6 − Y7 + Y8).

The AB interaction is half the difference between the main effect of factor A

at the high level of factor B and that at the low level of factor B, or:

ˆ(αβ) =1

2

[

1

2

(

Y4 + Y8

2−

Y3 + Y7

2

)

−

1

2

(

Y2 + Y6

2−

Y1 + Y5

2

)]

As per the signs in the table,

ˆ(αβ) =1

8(Y1 − Y2 − Y3 + Y4 + Y5 − Y6 − Y7 + Y8).

17

The ABC interaction is half the difference between the two-factor interaction

AB at the high and low levels of factor C, or

ˆ(αβγ) =1

2

[

1

2

(

Y8 − Y7

2−

Y6 − Y5

2

)

−

1

2

(

Y4 − Y3

2−

Y2 − Y1

2

)]

Note that

ˆ(αβγ) =1

8(−Y1 + Y2 + Y3 − Y4 + Y5 − Y6 − Y7 + Y8)

as per the signs in the previous table.

These relationships give us an automatic way to estimate the main effects

(and, implicitly, the sums of squares) for inference.

18

But taking 2k observations can quickly get expensive. Instead, one can

carefully select half that number so as to still permit estimation of main effects

and low-order interactions.

19

Consider a 23−1 fractional factorial design (The previous figure gave two

illustrations.)

run A B C AB AC BC ABC obs

1 - - + + - - + Y112

2 + - - - - + + Y211

3 - + - - + - + Y121

4 + + + + + + + Y222

Note that because we have taken only half of the 8 observations needed for a

full factorial, some of the columns have identical entries.

Columns that have identical entries correspond to effects that are confounded

or aliased.

20

In order to estimate the effects, note that:

1

4(Y1 + Y2 + Y3 + Y4) = µ + ˆ(αβγ)

1

4(−Y1 + Y2 − Y3 + Y4) = α + ˆ(βγ)

1

4(−Y1 − Y2 + Y3 + Y4) = β + ˆ(αγ)

1

4(Y1 − Y2 − Y3 + Y4) = γ + ˆ(αβ)

Thus the estimate of the mean is confounded with the three-way interaction,

the estimate of the A effect is confounded with the BC interaction, the

estimate of the B effect is confounded with the AC interaction, and the

estimate of the C effect is confounded with the AB interaction.

If one assumes that there are no interactions, then one can make tests about

the main effects, or use a normal probability plot.

21

Note that we write 2k−p to denote a fractional factorial design in which each

factor has 2 levels, there are k factors, and we are taking a 1/2p fraction of the

number of possible factor level combinations.

In order to construct a fractional factorial that deliberately confounds

pre-selected factors, one needs to use a generator.

The generator uses the fact that squaring the entries in any given column gives

a column of ones, which can be thought of as an identity element I. If we

want to confound the A effect with the BC interaction, then that is equivalent

to declaring A ∗ BC = ABC = I. It follows that B = BI = B ∗ ABC = AC,

so B is confounded with the AC interaction. Similarly, C is confounded with

AB, and the overall mean (I) is confounded with ABC.

22

Consider a 24−1 fractional factorial design.

run A B C D AB AC BC ABC obs

1 - - - - + + + - Y1111

2 + - - + - - + + Y2112

3 - + - + - + - + Y1212

4 + + - - + - - - Y2211

5 - - + + + - - + Y1122

6 + - + - - + - - Y2121

7 - + + - - - + - Y1221

8 + + + + + + + + Y2222

23

What factors and interactions are confounded?

First, note that the ABC interaction has the same signs in its column as D.

Thus D and ABC are confounded, or D=ABC. This implies that I=ABCD is

the generator. (Why?)

From this, the confounding pattern is as follows:

A is confounded with BCD AB is confounded with CD

B is confounded with ACD AC is confounded with BD

C is confounded with ABD AD is confounded with BC

D is confounded with ABC ABCD is confounded with µ

This design has Resolution IV. This is because all the main effects (order I)

are confounded with three-way interactions (order III) and the resolution is

I+III = IV. Such a design is sometimes written as 24−1

IV .

24

One can go the other way, picking the generator and then deriving the

confounding pattern (and thus the design). For example, suppose we had

decided to confound D with AB.

In that case, the generator is I=ABD. Thus:

A is confounded with BD AC is confounded with BCD

B is confounded with AD ACD is confounded with BC

C is confounded with ABCD ABC is confounded with CD

D is confounded with AB ABD is confounded with µ

This has Resolution III, since some of the main effects (order I) are

confounded with two-way interactions (order II), giving resolution I+II = III.

This design is written as 24−1

III .

In general, we prefer designs that have higher resolution. This ensures that

one can make relatively clean tests of main effects (and, for larger numbers of

factors, the tests of two-way interactions).

25

Consider a 24−1 fractional factorial design. The data are from a slump test of

concrete; the four factors are water (A), mixing (B), gravel proportion (C),

and cement proportion (D).

run A B C D AB AC BC ABC obs

1 - - - - + + + - 21

2 + - - + - - + + 23

3 - + - + - + - + 27

4 + + - - + - - - 24

5 - - + + + - - + 23

6 + - + - - + - - 26

7 - + + - - - + - 33

8 + + + + + + + + 37

26

26.75 = (y1 + · · · + y8)/8 = µ + ˆABCD

0.75 = (−y1 + y2 − y3 + y4 − y5 + y6 − y7 + y8)/8 = A + ˆBCD

3.5 = B + ˆACD

3.0 = C + ˆABD

−.5 = (y1 − y2 − y3 + y4 + y5 − y6 − y7 + y8)/8 = AB + CD

1.0 = AC + BD

1.75 = BC + AD

.75 = (−y1 + y2 + y3 − y4 + y5 − y6 − y7 + y8) = D + ˆABC

If the null hypothesis of no effects is true, then the 3.5 and 3.0 look relatively

large. These deserve more study.

27

22.4 Example

Consider an experiment designed to study seven factors in only eight runs.

This means we need a 27−4 fractional factorial design.

To do this, one needs more than one generator (in fact, one needs four

generators, since each halves the number of observations). One strategy is to

write out a full 23 factorial design, and then associate (confound or alias) the

interactions with each of the four additional factors.

28

D E F G

run A B C AB AC BC ABC obs

1 - - - + + + - Y111

2 + - - - - + + Y211

3 - + - - + - + Y121

4 + + - + - - - Y221

5 - - + + - - + Y112

6 + - + - + - - Y212

7 - + + - - + - Y122

8 + + + + + + + Y222

This implies D=AB, E=AC, F=BC, and G=ABC, from which we get the

generating relations I=ABD, I=ACE, I= BCF, and I = ABCG.

29

Since the product of I with itself is also I, then one can multiply any pair or

triple or quadruple of the generating relations and still get I. Thus:

I = ABD = ACE = BCF = ABCG

= BCDE = ACDF = CDG = ABEF

= BEG = AFG = DEF = ADEG

= BDFG = CEFG = ABCDEFG

This is cumbersome and tedious, but it enables one to calculate all the aliased

main effects and interactions.

This is called the defining relation.

30

If we multiply the defining relation by A, we find that

A = BD = CE = ABCF = BCG

= ABCDE = CDF = ACDG = BEF

= ABEG = FG = ADEF = DEG

= ABDFG = ACEFG = BCDEFG

Thus A is confounded with two-way, three-way, and higher interactions.

Similar calculations can be done for the other main effects. Each main effect is

confounded with a two-way interaction (e.g., G is confounded with CD, BE,

and AF. This means we have a 27−4

III design.

31

Box and Hunter (Technometics, 1961, 311-351) describe a 27−4

III design for a

company that experienced difficulty (long delays) with a filtration system that

was part of their manufacturing process. The factors were:

A: water supply (town, well)

B: raw mateial (two suppliers)

C: temperature at filtration (high and low)

D: recycling (included or omitted)

E: rate of addition of caustic soda (slow, fast)

F: type of filter cloth (old, new)

G: prior hold-up time (short, long)

The raw data from the experiment are shown in the following table.

32

D E F G

run A B C AB AC BC ABC obs

1 - - - + + + - 68.4

2 + - - - - + + 77.7

3 - + - - + - + 66.4

4 + + - + - - - 81.0

5 - - + + - - + 78.6

6 + - + - + - - 41.2

7 - + + - - + - 68.7

8 + + + + + + + 38.7

If one estimates the seven main effects (each of which is aliased with

interactions), one finds A is -5.4, B is -1.4, C is -8.3, D is 1.6, E is -11.4, F is

-1.7, and G is 0.3. It seems that E, C, and A (or corresponding interactions)

merit more thought.

33

These extreme fractions are difficult—one cannot get much information from,

say, eight observations, but these designs cleverly ensure that one gets the

most one can.

The designs are especially useful in stagewise experimentation. Often one

starts with a large set of factors, and then develops a series of fractional

factorial experiments that home in on the most important effects.

In the filtration experiment, the firm can now design follow-up experiments

that focus on the key factors. Or they can try to run their system at the high

levels of A, B, C, D, E, F, and G, which gave a filtration time of 38.7.

34