8/8/2019 16 Ex Design

1/53

lab exam

when: Nov 27 - Dec 1

length = 1 hour each lab section divided in two

register for the exam in your section so there is acomputer reserved for you

If you write in the 1st hour, you cant leave early!If you write in the second hour, you cant arrivelate!

8/8/2019 16 Ex Design

2/53

lab exam

format:

open book!

similar to questions in lab manual

last section in the lab manual has review

questions

show all your work: hypotheses, tests of

assumptions, test statistics, p-values andconclusions

8/8/2019 16 Ex Design

3/53

Experimental Design

8/8/2019 16 Ex Design

4/53

Experimental Design

Experimental design is the part of

statistics that happens before you carry

out an experiment

Proper planning can save many

headaches

You should design your experiments with

a particular statistical test in mind

8/8/2019 16 Ex Design

5/53

Why do experiments?

Contrast: observational study vs.

experiments

Example:

Observational studies show a positive

association between ice cream sales and

levels of violent crime

What does this mean?

8/8/2019 16 Ex Design

6/53

Why do experiments?

Contrast: observational study vs.

experiments

Example:

Observational studies show a positive

association between ice cream sales and

levels of violent crime

What does this mean?

8/8/2019 16 Ex Design

7/53

Alternative explanation

Hot weather

Ice cream

Violent

crime

8/8/2019 16 Ex Design

8/53

Alternative explanation

Hot weather

Ice cream

Violent

crime

Correlation isnot causation

8/8/2019 16 Ex Design

9/53

Why do experiments?

Observational studies are prone to

confounding variables: Variables that

mask or distort the association between

measured variables in a study

Example: hot weather

In an experiment, you can use random

assignments of treatments to individuals toavoid confounding variables

8/8/2019 16 Ex Design

10/53

Goals of Experimental Design

Avoid experimental artifacts

Eliminate bias

1. Use a simultaneous control group

2. Randomization3. Blinding

Reduce sampling error

1. Replication

2. Balance

3. Blocking

8/8/2019 16 Ex Design

11/53

Goals of Experimental Design

Avoid experimental artifacts

Eliminate bias

1. Use a simultaneous control group

2. Randomization3. Blinding

Reduce sampling error

1. Replication

2. Balance

3. Blocking

8/8/2019 16 Ex Design

12/53

Experimental Artifacts

Experimental artifacts: a bias in a

measurement produced by unintended

consequences of experimental procedures

Conduct your experiments under as

natural of conditions as possible to avoid

artifacts

8/8/2019 16 Ex Design

13/53

Experimental Artifacts

Example: diving birds

8/8/2019 16 Ex Design

14/53

Goals of Experimental Design

Avoid experimental artifacts

Eliminate bias

1. Use a simultaneous control group

2. Randomization3. Blinding

Reduce sampling error

1. Replication

2. Balance

3. Blocking

8/8/2019 16 Ex Design

15/53

Control Group

A control group is a group of subjects left

untreated for the treatment of interest but

otherwise experiencing the same

conditions as the treated subjects

Example: one group of patients is given an

inert placebo

8/8/2019 16 Ex Design

16/53

The Placebo Effect

Patients treated with placebos, including

sugar pills, often report improvement

Example: up to 40% of patients with

chronic back pain report improvement

when treated with a placebo

Even sham surgeries can have a

positive effect

This is why you need a control group!

8/8/2019 16 Ex Design

17/53

Randomization

Randomization is the random assignment

of treatments to units in an experimental

study

Breaks the association between potential

confounding variables and the explanatory

variables

8/8/2019 16 Ex Design

18/53

Experimental units

Confou

ndingv a

riable

8/8/2019 16 Ex Design

19/53

Experimental units

Confou

ndingv a

riable

Treatments

8/8/2019 16 Ex Design

20/53

Experimental units

Confoundingv a

riable

Treatments

Without

randomization,

the confounding

variable differs

among

treatments

8/8/2019 16 Ex Design

21/53

Experimental units

Confou

ndingv a

riable

Treatments

8/8/2019 16 Ex Design

22/53

Experimental units

Confou

ndingva

riable

Treatments

With

randomization,

the confounding

variable does

not differ

among

treatments

8/8/2019 16 Ex Design

23/53

Blinding

Blinding is the concealment of information

from the participants and/or researchers

about which subjects are receiving which

treatments

Single blind: subjects are unaware of

treatments

Double blind: subjects and researchers

are unaware of treatments

8/8/2019 16 Ex Design

24/53

Blinding

Example: testing heart medication

Two treatments: drug and placebo

Single blind: the patients dont know whichgroup they are in, but the doctors do

Double blind: neither the patients nor the

doctors administering the drug know whichgroup the patients are in

8/8/2019 16 Ex Design

25/53

Goals of Experimental Design

Avoid experimental artifacts

Eliminate bias

1. Use a simultaneous control group

2. Randomization3. Blinding

Reduce sampling error

1. Replication

2. Balance3. Blocking

8/8/2019 16 Ex Design

26/53

Replication

Experimental unit: the individual unit to

which treatments are assigned

Experiment 1

Experiment 2

Experiment 3

Tank 1 Tank 2

All separate tanks

8/8/2019 16 Ex Design

27/53

Replication

Experimental unit: the individual unit to

which treatments are assigned

Experiment 1

Experiment 2

Experiment 3

Tank 1 Tank 2

All separate tanks

2 ExperimentalUnits

2 Experimental

Units

8 Experimental

Units

8/8/2019 16 Ex Design

28/53

Replication

Experimental unit: the individual unit to

which treatments are assigned

Experiment 1

Experiment 2

Experiment 3

Tank 1 Tank 2

All separate tanks

2 ExperimentalUnits

2 Experimental

Units

8 Experimental

Units

Pseudoreplication

8/8/2019 16 Ex Design

29/53

Why is pseudoreplication bad?

problem with confounding and replication!

Imagine that something strange happened, by chance, to tank 2 but not to tank 1

Example: light burns out

All four lizards in tank 2 would be smaller

You might then think that the difference was due to the treatment, but its actuallyjust random chance

Experiment 2

Tank 1 Tank 2

8/8/2019 16 Ex Design

30/53

Why is replication good?

Consider the formula for standard error of

the mean:

SEY= s

n

Larger n Smaller SE

8/8/2019 16 Ex Design

31/53

Balance

In a balanced experimental design, all

treatments have equal sample size

Better than

Balanced Unbalanced

8/8/2019 16 Ex Design

32/53

Balance

In a balanced experimental design, all

treatments have equal sample size

This maximizes power

Also makes tests more robust to violating

assumptions

8/8/2019 16 Ex Design

33/53

Blocking

Blocking is the grouping of experimental

units that have similar properties

Within each block, treatments are

randomly assigned to experimental

treatments

Randomized block design

8/8/2019 16 Ex Design

34/53

Randomized Block Design

8/8/2019 16 Ex Design

35/53

Randomized Block Design

Example: cattle tanks in a field

8/8/2019 16 Ex Design

36/53

8/8/2019 16 Ex Design

37/53

Very sunny

Not So Sunny

8/8/2019 16 Ex Design

38/53

Block 1

Block 4

Block 2

Block 3

8/8/2019 16 Ex Design

39/53

What good is blocking?

Blocking allows you to remove extraneous

variation from the data

Like replicating the whole experiment

multiple times, once in each block

Paired design is an example of blocking

8/8/2019 16 Ex Design

40/53

Experiments with 2 Factors

Factorial design investigates all

treatment combinations of two or more

variables

Factorial design allows us to test for

interactions between treatment variables

8/8/2019 16 Ex Design

41/53

Factorial Design

5.5 6.5 7.5

25 n=2 n=2 n=2

30 n=2 n=2 n=2

35 n=2 n=2 n=2

40 n=2 n=2 n=2

Te

mperatu

re

pH

8/8/2019 16 Ex Design

42/53

Interaction Effects

An interaction between two (or more)

explanatory variables means that the

effect of one variable depends upon the

state of the other variable

I t t ti f 2 ANOVA

8/8/2019 16 Ex Design

43/53

Interpretations of 2-way ANOVA

Terms

0

10

20

30

40

50

60

70

25 30 35 40

Temperature

pH 5.5

pH 6.5

pH 7.5

Effect of pH and Temperature,

No interaction

I t t ti f 2 ANOVA

8/8/2019 16 Ex Design

44/53

0

5

10

15

20

25

30

35

40

45

25 30 35 40

Temperature

pH 5.5

pH 6.5

pH 7.5

Interpretations of 2-way ANOVA

Terms

Effect of pH and Temperature,

with interaction

8/8/2019 16 Ex Design

45/53

Goals of Experimental Design

Avoid experimental artifacts

Eliminate bias

1. Use a simultaneous control group

2. Randomization3. Blinding

Reduce sampling error

1. Replication

2. Balance3. Blocking

8/8/2019 16 Ex Design

46/53

What if you cant do experiments?

Sometimes you cant do experiments

One strategy:

Matching

Every individual in the treatment group is

matched to a control individual having the

same or closely similar values for known

confounding variables

8/8/2019 16 Ex Design

47/53

What if you cant do experiments?

Example: Do species on islands change

their body size compared to species in

mainland habitats?

For each island species, identify a closely

related species living on a nearby

mainland area

8/8/2019 16 Ex Design

48/53

Power Analysis

Before carrying out an experiment you

must choose a sample size

Too small: no chance to detect treatment

effect

Too large: too expensive

We can use power analysis to choose our

sample size

8/8/2019 16 Ex Design

49/53

Power Analysis

Example: confidence interval

For a two-sample t-test, the approximatewidth of a 95% confidence interval for the

difference in means is:

(assuming that the data are a randomsample from a normal distribution)

precision = 4 2n

8/8/2019 16 Ex Design

50/53

Power Analysis

Example: confidence interval

The sample size needed for a particular

level of precision is:

n = 32

Precision

2

8/8/2019 16 Ex Design

51/53

Power Analysis

Assume that the standard deviation of exam scores for a class is 10.

I want to compare scores between two lab sections. A. How many

exams do I need to mark to obtain a confidence limit for the

difference in mean exam scores between two classes that has a

width (precision) of 5?

n = 32

Precision

2

n = 3210

5

2

=128

8/8/2019 16 Ex Design

52/53

Power Analysis

Example: power

Remember, power = 1 - = Pr[Type II error] Typical goal is power = 0.80

For a two-sample t-test, the sample size

needed for a power of 80% to detect adifference of D is:

n = 16

D

2

8/8/2019 16 Ex Design

53/53

Power Analysis

Assume that the standard deviation of exam scores for a class is 10.

I want to compare scores between two lab sections. B. How many

exams do I need to mark to have sufficient power (80%) to detect a

mean difference of 10 points between the sections?

n = 16 D

2

n = 16 1010

2= 16