Aim: What goes behind designing experiments?

What is a study?

• A study is an experiment when we actually do something to people, animals, or objects to observe the response.

What is the basic vocabulary of experiments?

• The individuals on which the experiment is done are the experimental units. When the units are human beings, they are called subjects. A specific experimental condition applied to the units is called a treatment.

Experiments: Good Evidence for Causation

• Because the purpose of an experiment is to reveal the response of one variable to changes in the other, the distinction between explanatory and response variables is important.– The explanatory variable in an experiment are

often called factors

Why are experiments good evidence of causation?

• We study the specific factors we are interested in while controlling the effects of lurking variables.

Example• Do smaller classes in elementary school really benefit students in areas such as

scores on standard tests, staying in school, and going on to college? We might do an observational study that compares students who happened to be in smaller and larger classes in their early school years. Small classes are expensive, so they are more common in schools that serve richer communities. Students in small classes tend to also have other advantages: their schools have more resources, their parents are better educated, and so on. Confounding makes it impossible to isolate the effects of small classes.

• The Tennessee STAR program was an experiment on the effects of class size. It has been called “one of the most important educational investigations ever carried out.” The subjects were 6385 students who were beginning kindergarten. Each student was assigned to one of three treatments: regular class (22 to 25 students) with one teacher, regular class with a teacher and a full-time teacher’s aide, and small class (13 to 17 students). These treatments are levels of a single factor, the type of class. The students stayed in the same type of class for four years, then all returned to regular classes. In later years, students from the small classes had higher scores on standard tests, were less likely to fail a grade, had better high school grades, and so on. The benefits of small classes were greatest for minority students.8

What is a simple design study?

• Use of only one treatment– TREATMENT OBSERVE RESPONSE– For example, we may subject a beam to a load

(treatment) and measure its deflection (observation). • We rely on the controlled environment of the

laboratory to protect us from lurking variables. • When experiments are conducted in the field or with

living subjects, such simple designs often yield invalid data. That is, we cannot tell whether the response was due to the treatment or to lurking variables.

Important Terms

• Control Group: the group not receiving treatments

• Placebo: participants think they are receiving a treatment but they are not (dummy treatment)

• Example giving them a vitamin instead of the treatment drug

• Placebo Effect: participants receiving placebo think that the treatment changed them because they believe in their head that they received the treatment – A response to the placebo

Bias

• The design of a study is biased if it systematically favors certain outcomes.

• Example: An uncontrolled study of a new medical therapy is biased in favor of finding the treatment effective because of the placebo effect. It should not surprise you to learn that uncontrolled studies in medicine give new therapies a much higher success rate than proper comparative experiments. Well-designed experiments usually compare several treatments.

Class Work 1. Storing food for long periods of time is a major challenge for those

planning for human space travel beyond the moon. One problem is that exposure to radiation decreases the length of time that food can be stored. One experiment examined the effects of nine different levels of radiation on a particular type of fat or lipid. The amount of oxidation of the lipid is the measure of the extent of the damage due to radiation. Three samples are exposed to each radiation level. Give the experimental units, the treatments, and the response variable. Describe the factor and its levels. There are many different types of lipids. To what extent do you think the results of this experiment can be generalized to other lipids?

2. An instructor in an elementary statistics course wants to know if using a new statistical software package will improve students’ final-exam scores. He asks for volunteers and about half of the class agrees to work with the new software. He compares the final-exam scores of the students who used the new software with the scores of those who did not. Discuss possible sources of bias in this study.