CS2008 WK 5 Experiments I Lecture

8/12/2019 CS2008 WK 5 Experiments I Lecture

1/26

CS2008 WK5:

Experimental Design IDr. Christopher L. Cummings

[email protected]

Telephone: 6316 2916Office: Room #03-43

Office hours: after class & by appointment

1CS 2008: Experimental Design I
mailto:[email protected]:[email protected]


2/26

First organized procedures as early as thethirteenth century in Greece.

More formally, the experimental methodreceived its greatest impetus from the scientificadvances of the 16thand 17thcenturies.

Allowed Sir Isaac Newton (1643-1727) toproclaim the qualities of bodies are only

known to us by experiments

History of experimentation


3/26

Vitamin C

Scurvy


4/26


5/26

Sir James Lind (1716-1799)

Scottish physician and thepioneer of naval hygiene.

Conducted one of the firstdocumented clinical trial(experiment) in 1747.

Sought a cure for scurvy,but he didnt know whatwould cure it.


6/26

ConditionsEach received the same typical

naval diet (bread, butter, beans, & beer)Group 1: Quart of cider

Group 2: 25 drops of vitriol (sulfuric acid)

Group 3: 6 spoonfuls of vinegar

Group 4: litre of seawater

Group 5: 2 oranges and 1 lemon

Group 6: Spicy pepper paste and litre of barley

water


7/26

James Lind(17161799)

Condition 5 stopped after 6 days of treatment whenthey ran out of fruit, but by then

the most sudden and visible good effects were

perceived from the use of the oranges andlemons; one of those who had taken them, being

at the end of six days fit for duty.

A treatise of the scurvy, 1753


8/26

Which textbook improves student

learning the greatest?


9/26

Simple Design

Dependent variable: Degree of student learning.

Independent variable: Textbook given tostudents

Condition A: Textbook A

Condition B: Textbook B

Condition C: Textbook C


10/26


11/26

19thand 20thcentury classical experimentation Practice of holding everything constant except the

one variable under consideration.

Move to the modern: 1935: Sir Ronald A. Fishers The Design of Experiments

Experience carefully planned in advance

Not controlling all potential variables, but rather

manipulating levels or amounts of selected independentvariables in order to examine their influence on dependentvariables.

Classicvs Modern Experiments


12/26

Why Do We Need Experimental Design?

VariabilityWewouldnt need experimental design if:

If all units (students, teachers, & schools) were

identical, and If all units responded identically to treatments.

We need experimental design to control

variability so that the effects of our independentvariable of interest can be isolated and observed

on the dependent variable.


13/26

Non-experimental methods have greaterdifficulty in assessing nomothetic causation.

How do you know that the independent variable

caused some change in the dependent?

Experiments provide much greater control overvariables.

Typically has better external validity than non-experimental research.

Advantages of Experimental-methods


14/26

Vital concepts for experimental approaches:

Manipulation

Control

Basics


15/26

Manipulation of the Independent Variable is keyto experimentation.

manipulate IV and then observe changes in the

DV for conditions and compare results.

What variable are we manipulating when

we conduct experiments?

Condition

1

Independ

ent

Variable

Condition

2

Dependent

Variable

Dependent

Variable

CompareResults


16/26

Influence of quantity of water

on grass growth.

Water

Hi

Quantityof

water

Water

Low

Grass growth

in cm

Grass growth

in cm

CompareResults


17/26

In the scientific method, control allows forcomparison between concepts.

In this case, we often look at concepts as

varying dimensions/levels of IVs.

Controlling variables and attributes in theexperimental setting and data so that adequatecomparative conclusions can be reached.

Control


18/26

Control groups In designed experiments, we separate research subjects into experimental

groups (conditions).

Experimental groups/Conditions: groups that receive some form ofcontrolled stimulus.

Control group Group of participants to whom no experimental stimulus I

administered but who should resemble the experimental group in allother respects. Comparison of the control group to the experimentalgroup(s) demonstrates the effect of the stimulus.


19/26

3 types of control

Experimental design controls backgroundvariability so that systematic effects oftreatments can be observed.

1. Control by matching

2. Control by randomization

3. Control by statistical adjustment


20/26

Control by Matching

Some sources of variation may be eliminated by matching

Eliminating genetic variationCompare animals from the same litter of mice

Eliminating district or school effectsCompare students within districts or schools

However matching is limited

matching is only possible on observablecharacteristics perfect matching is never possible matching inherently limits generalizability by removing (possibly

desired) variation


21/26

Control by Matching

Matching ensures that groups compared are alikeon specific known and observablecharacteristics (inprinciple, everything we have thought of)

Wouldnt it be great if there were

a method of making groups alike

on not only everything we have

thought of, but everything wedidnt think of too?


22/26

Assigning of experimentalsubjects to experimentalconditions and control groupsat random.

Randomization is like lifeinsurance. It is a precaution

against disturbances that mayor may not occur and that

may or may not be serious ifthey do occur (Cochran and

Cox, 1957, p. 8).

RandomizationWhat randomization succeeds in

accomplishing, at least in theory, isthe conversion of all irrelevant

sources of possible systematic

variability into unsystematic

variability, that is, into random error.

The only thing that persons in group A

should have in commonthat is, the

only source of systematic variability

is exposure to the same [stimulus]

(Brown and Melamed, 1990, p. 3).


23/26

Control by Randomization

Matching controls forthe effects of variationdue to specificobservablecharacteristics.

Randomization controlsfor the effects all(observable or non-observable, known orunknown) characteristics.

Randomization makesgroups equivalent (onaverage) on all variables(known and unknown,observable or not)

Randomization also givesus a way to assess

whether differences aftertreatment are larger than

would be expected dueto chance.

23


24/26

Control by Randomization

Random assignment is notassignment with no particularrule. It is a purposeful process.

Assignment is made at random. This does

not mean that the experimenter writesdown the names of the varieties in any

order that occurs to him, but that he

carries out a physical experimental process

of randomization, using means which shallensure that each variety will have an equal

chance of being tested on any particular

plot of ground(Fisher, 1935, p. 51)


25/26

Control by Statistical Adjustment

Control by statistical adjustmentis a form of pseudo-matching.

It uses statistical relations to

simulate matching. Statistical control is important

for increasing precision but

should not be relied upon tocontrol biases that may existprior to assignment.


26/26

Using Principles of Experimental Design

You have to have a great deal of knowledge andinformation to effectively use matching and statisticalcontrol.

You do not have to be smart to use randomization

effectively.

Documents

CS2008 WK 5 Experiments I Lecture