SJS SDI_9 1

Design of Statistical Investigations

Stephen Senn

9 Unbalanced Designs

SJS SDI_9 2

Lack of Orthogonality

• So far we have been considering “balanced designs”– for example every treatment appears equally

frequently in every block

• Sometimes we do not have such balance– by accident

• missing observations

– by design

SJS SDI_9 3

Consequences

• Some loss of efficiency– compared to some theoretical optimum

• CAUTION: this may not be obtainable in practice and may be why an unbalanced design has been chosen

• Complications in analysis– Sums of squares may depend on what other terms

have been fitted• so far only residual sum of squares has had this property

SJS SDI_9 4

Exp_11Senn 2002 Example 5.1

• Cross-over trial in asthma

• Comparison of salbutamol, formoterol, placebo

• Trial run in six sequences

• Unequal numbers of patients per sequence

SJS SDI_9 5

Exp_11Sequences and Periods:Number of Observations I II III

FSP 5 5 5SPF 3 3 3PFS 6 6 6FPS 6 6 6SFP 5 5 5PSF 5 5 5

Patients by Sequence FSP SPF PFS FPS SFP PSF

5 3 6 6 5 5

Note that although there are no missing data due to patients not having completed a sequence, the numbers of patients are unbalanced by sequence

SJS SDI_9 6

F

S

P

1000 2000 3000

1FSP

2FSP

1000 2000 3000

3FSP

4FSP

1000 2000 3000

5FSP

6FSP

F

S

P

1SPF

2SPF

3SPF

4SPF

5SPF

6SPF

F

S

P

1PFS

2PFS

3PFS

4PFS

5PFS

6PFS

F

S

P

1FPS

2FPS

3FPS

4FPS

5FPS

6FPS

F

S

P

1SFP

2SFP

3SFP

4SFP

5SFP

6SFP

F

S

P

1PSF

1000 2000 3000

2PSF

3PSF

1000 2000 3000

4PSF

5PSF

1000 2000 3000

6PSF

forced expiratory volume in one second

tre

atm

en

t

SJS SDI_9 7

Exp_11Data

1 FSP 3500 3200 290010 FSP 3400 2800 220017 FSP 2300 2200 170021 FSP 2300 1300 140023 FSP 3000 2400 1800 4 SPF 2200 1100 2600 8 SPF 2800 2000 280016 SPF 2400 1700 3400 6 PFS 2200 2500 2400 9 PFS 2200 3200 330013 PFS 800 1400 100020 PFS 950 1320 148026 PFS 1700 2600 240031 PFS 1400 2500 2200

2 FPS 3100 1800 240011 FPS 2800 1600 220014 FPS 3100 1600 140019 FPS 2300 1500 220025 FPS 3000 1700 260028 FPS 3100 2100 2800 3 SFP 2100 3200 100012 SFP 1600 2300 160018 SFP 1600 1400 80024 SFP 3100 3200 100027 SFP 2800 3100 2000 5 PSF 900 1900 2900 7 PSF 1500 2600 200015 PSF 1200 2200 270022 PSF 2400 2600 380030 PSF 1900 2700 2800

SJS SDI_9 8

Exp_11Not Fitting Period

> fit1 <- lm(fev1 ~ patient + treat)> summary(fit1, corr = F)

Coefficients: Value Std. Error t value Pr(>|t|)…... treatS -424.6667 87.3127 -4.8637 0.0000treatP -1099.0000 87.3127 -12.5869 0.0000

Residual standard error: 338.2 on 58 degrees of freedom

Multiple R-Squared: 0.8569

SJS SDI_9 9

Exp_11 Fitting Period

> fit2 <- update(fit1, . ~ . + period)summary(fit2, corr = F)Call: lm(formula = fev1 ~ patient + treat + period)

Coefficients: Value Std. Error t value Pr(>|t|)…... treatS -422.6220 88.2647 -4.7881 0.0000 treatP -1103.4638 87.8208 -12.5649 0.0000 periodII -109.7228 87.8208 -1.2494 0.2167periodIII -42.7659 88.2647 -0.4845 0.6299

Residual standard error: 339.4 on 56 degrees of freedomMultiple R-Squared: 0.8608

SJS SDI_9 10

Exp_11ANOVA

> aov.1 <- aov(fev1 ~ patient + treat)> summary(aov.1) Df Sum of Sq Mean Sq F Value Pr(F) patient 29 21279472 733775 6.41677 1.065573e-009 treat 2 18428682 9214341 80.57832 0.000000e+000Residuals 58 6632451 114353

> aov.2 <- aov(fev1 ~ patient + period) > summary(aov.2) Df Sum of Sq Mean Sq F Value Pr(F) patient 29 21279472 733774.9 1.703663 0.0424500 period 2 80282 40141.1 0.093199 0.9111486Residuals 58 24980851 430704.3 >

SJS SDI_9 11

Exp_11ANOVA

> aov.3 <- aov(fev1 ~ patient + period + treat)>summary(aov.3) Df Sum of Sq Mean Sq F Value Pr(F) patient 29 21279472 733775 6.37115 0.0000000 period 2 80282 40141 0.34853 0.7072422 treat 2 18531248 9265624 80.45067 0.0000000Residuals 56 6449603 115171> aov.4 <- aov(fev1 ~ patient + treat + period) > summary(aov.4) Df Sum of Sq Mean Sq F Value Pr(F) patient 29 21279472 733775 6.37115 0.0000000 treat 2 18428682 9214341 80.00540 0.0000000 period 2 182848 91424 0.79381 0.4571415Residuals 56 6449603 115171

SJS SDI_9 12

Exp_11ANOVA

> ssType3(aov.3)Type III Sum of Squares Df Sum of Sq Mean Sq F Value Pr(F) patient 29 21279472 733775 6.37115 0.0000000 period 2 182848 91424 0.79381 0.4571415 treat 2 18531248 9265624 80.45067 0.0000000Residuals 56 6449603 115171 > ssType3(aov.4)Type III Sum of Squares Df Sum of Sq Mean Sq F Value Pr(F) patient 29 21279472 733775 6.37115 0.0000000 treat 2 18531248 9265624 80.45067 0.0000000 period 2 182848 91424 0.79381 0.4571415Residuals 56 6449603 115171

SJS SDI_9 13

Exp_11Standard Errors

Period effect not fitted

1 1114353 87.3127

30 30SE

Period effect fitted

1 187.8208, SE=88.2647 115171

30 30SE

SJS SDI_9 14

Incomplete Blocks

• These designs arise when the number of treatments exceeds the number of units in a typical block

• Not possible to have every treatment in every block

• Each block receives a subset of the units

• These to be chosen in a sensible manner

SJS SDI_9 15

Exp_12Senn 2002 Example 7.2

• Placebo (P) controlled cross-over design to compare two doses of formoterol – F12 : 12 mg in a single puff– F24: 24 mg in a single puff

• Patients could only be treated in two periods

• Incomplete blocks design

• 24 Patients to be allocated in equal numbers to each of six sequences

SJS SDI_9 16

EXP_12Sequences used

P F12

F12 P

P F24

F24 P

F12 F24

F24 F12

The basic design is said to be that of balanced incomplete blocks.

In this context balance has a special meaning: each pair of possible treatments appears equally often in every block

Because this is a cross-over design and we are worried about period effects the design is also balanced by period (order) but that is another matter

SJS SDI_9 17

EXP_12The sad reality

• Two incorrect packs were picked up.– One was for correct sequence– One was not

Numbers of Observations PeriodSequence 1 2 F12F24 3 3 F12P 5 5 F24F12 4 4 F24P 4 4 PF12 4 4 PF24 4 4

F12 F24 has one fewer patient

F12 P has one more

SJS SDI_9 18

EXP_12The Data

6 F12F24 2.500 2.45010 F12F24 1.750 1.72515 F12F24 1.370 1.120 4 F12P 3.400 2.50011 F12P 2.250 1.92514 F12P 1.460 1.26021 F12P 1.480 0.88023 F12P 2.050 2.100 2 F24F12 2.700 2.25012 F24F12 0.900 0.92513 F24F12 1.270 1.01024 F24F12 2.150 2.100

3 F24P 1.750 1.350 7 F24P 2.525 2.15018 F24P 1.080 0.84022 F24P 3.120 2.310 5 PF12 2.500 3.500 9 PF12 1.600 2.65016 PF12 1.750 2.19019 PF12 0.640 0.840 1 PF24 2.100 3.100 8 PF24 2.300 2.70017 PF24 1.030 1.87020 PF24 0.810 0.940

SJS SDI_9 19

F12

F24

P

1.0 1.5 2.0 2.5 3.0 3.5

1F12F24

2F12F24

1.0 1.5 2.0 2.5 3.0 3.5

3F12F24

4F12F24

1.0 1.5 2.0 2.5 3.0 3.5

5F12F24

F12

F24

P

1F12P

2F12P

3F12P

4F12P

5F12P

F12

F24

P

1F24F12

2F24F12

3F24F12

4F24F12

5F24F12

F12

F24

P

1F24P

2F24P

3F24P

4F24P

5F24P

F12

F24

P

1PF12

2PF12

3PF12

4PF12

5PF12

F12

F24

P

1PF24

1.0 1.5 2.0 2.5 3.0 3.5

2PF24

3PF24

1.0 1.5 2.0 2.5 3.0 3.5

4PF24

5PF24

FEV1

SJS SDI_9 20

Exp_12Analysis 1

> fit1 <- lm(FEV1 ~ patient + period + treat)> summary(fit1, corr = F)

Call: lm(formula = FEV1 ~ patient + period + treat)...Coefficients: Value Std. Error t value Pr(>|t|) (Intercept) 2.8164 0.1854 15.1874 0.0000 patient2 -0.3770 0.2350 -1.6042 0.1236 ... patient24 -0.7270 0.2350 -3.0933 0.0055 period 0.0310 0.0667 0.4652 0.6466 treatF24 0.0402 0.0973 0.4134 0.6835 treatP -0.5041 0.0914 -5.5148 0.0000

SJS SDI_9 21

Exp_12Analysis 2

> aov1 <- aov(FEV1 ~ patient + period + treat)> summary(aov1) Df Sum of Sq Mean Sq F Value Pr(F) patient 23 22.46280 0.976643 18.37451 0.0000000 period 1 0.00083 0.000833 0.01568 0.9015459 treat 2 2.32792 1.163962 21.89871 0.0000073Residuals 21 1.11619 0.053152 > ssType3(aov1)Type III Sum of Squares Df Sum of Sq Mean Sq F Value Pr(F) patient 23 23.64324 1.027967 19.34011 0.0000000 period 1 0.01150 0.011501 0.21638 0.6466003 treat 2 2.32792 1.163962 21.89871 0.0000073Residuals 21 1.11619 0.053152

SJS SDI_9 22

Exp_12Analysis 3

> aov2 <- aov(FEV1 ~ patient + treat + period)> summary(aov2) Df Sum of Sq Mean Sq F Value Pr(F) patient 23 22.46280 0.976643 18.37451 0.0000000 treat 2 2.31726 1.158628 21.79836 0.0000075 period 1 0.01150 0.011501 0.21638 0.6466003Residuals 21 1.11619 0.053152 > ssType3(aov2)Type III Sum of Squares Df Sum of Sq Mean Sq F Value Pr(F) patient 23 23.64324 1.027967 19.34011 0.0000000 treat 2 2.32792 1.163962 21.89871 0.0000073 period 1 0.01150 0.011501 0.21638 0.6466003Residuals 21 1.11619 0.053152

SJS SDI_9 23

Standard Errors• Consider the standard error of the contrast

F24 versus F12

• This is given as 0.0973

• How could this be calculated?

• There are two sequences in which these drugs could be compared– F12F24 with 3 patients– F24F12 with 4 patients

SJS SDI_9 24

However

1 10.053152 =0.1761 0.0973

3 4

Thus the standard error we have from fitting the regression model is actually lower than that produced by a naïve argument.

SJS SDI_9 25

QuestionsExp_12

• Why is the SE produced by the regression analysis lower than that produced by using the pooled MSE and the direct comparison of the means?

• What would the treatment estimate be if this naïve approach was used?

• How does it compare to that produced?

• What further information is the regression approach taking into account?

SJS SDI_9 26

Block Size and ComparisonsSuppose that the block size is k (there are k units per block) and that there are b blocks in total and bk units in total

Suppose that we have v treatments and r replicates. There must also be rv units in total

Hence rv = bk = N .

Each block permits k(k-1)/2 comparisons. There are bk(k-1)/2 in total.

However, there are v(v-1)/2 possible pair-wise comparisons.

SJS SDI_9 27

Block Size and Comparisons

Let be the average number of repetitions of the pair-wise comparisons in the design. Hence

( -1) 2 ( 1) ( 1)

( -1) 2 ( 1) ( 1)

bk k rv k r k

v v v v v

Obviously unless this is an integer, it will not be possible to “balance” the blocks.

If v-1 is a multiple of k-1 then it becomes particularly easy to balance the blocks

SJS SDI_9 28

Exp_13

• It was desired to compare three doses each of two formulations of formoterol to placeo– ISF 6, ISF12, ISF24– MTA6, MTA12,MTA24– Placebo

• There are thus seven treatments• Maximum number of acceptable periods was

deemed to be five

SJS SDI_9 29

Exp_13Possible solution

• Since 7-1 = 6 is twice 4-1 = 3 use design in 4 periods• If seven sequences are used it will also be possible to

make the treatments “uniform” on the periods• There are (7 6)/2 = 21 possible pair-wise comparisons

of treatments• Each patient provides (4 3)/2 = 6 possible comparison• There are 7 6 = 42 = 2 21 such comparisons per set

of seven sequences

SJS SDI_9 30

I A B C D

II E A F B

III G C A E

IV D F G A

V F G B C

VI B D E G

VII C E D F

A Balanced Design Uniform on the Periods for 7 treatments in 4 periods

SJS SDI_9 31

QuestionsExp_13

Exp_13 was in fact run using five periods and 21 sequences

• Check that such a design can be “balanced”

An alternative considered was to use five periods and seven sequences

• Show that such a design cannot be balanced• Why might it be preferable to the design in four

periods and seven sequences?