Introduction to Research Design

Threats to Internal ValidityOne Group Pretest-Posttest Design

  O   X   O

• Campbell & Stanley “pre-experimental.”• Wuensch “experimental.”• OK design if can achieve “experimental

isolation,” as in the chemistry lab.• Correlated samples t test or nonparametric

equivalent.

History

Events other than X between pretest and posttest

• Pre = subjects’ energy consumption.• X = Education on importance of

conserving resources.• Post = subjects’ energy consumption.• History = price of energy increases 50%

between pre and post.

Maturation

Processes that normally cause subjects to change across time.

• Subjects = newly hired employees• Pre = Test of morale• X = Six month program to elevate morale• Post = Test of morale• Maturation = end of honeymoon effect

Testing

Pretesting subjects can change them.• Pre = frequency of conservation behaviors

– have you installed a low-flow shower head?– et cetera

• X = Education on importance of conserving resources

• Post = frequency of conservation behaviors• Testing = just (pre) asking them about

certain behaviors might cause them to try them.

Instrumentation

The measuring instrument changes across time.

• The $1.99 scale for our fishing experiment.– AM versus PM weight of our catch– Spring stretched

• The human observer as instrument– effect of treatment on number of problems in

computer lab– changes in observers from pre to post

Statistical Regression

When scores have an error component, both high and low scores regress towards the mean upon retesting.

• My ESP demo in PSYC 2101• Educational research at Miami Univ.

– mean IQ in school district = 102– mean of selected students = 80– how much regression is expected?

Percent of regression towards the mean, PRM = 100(1 – ρ).

• If pretest-posttest corr = .8,• PRM = 100(1 - .8) = 20%.• Expect regression (up) of .2(102 - 80) =

4.4 points.• That may be enough to get significant pre-

post change.

Mortality

Subjects who drop out of the experiment may differ from those who stay in.

• Subjects = patients with wasting disease• Pre, Post = body weight, X = New drug• 20 patients at pre, 10 at post• Pre mean weight = 97 lb, post = 125.• Was the drug effective?

• Perhaps the sickest (and lightest) 10 dropped out.

• Maybe positive effect from some, who stay in, negative for others, who drop out.

• Correct by computing means only for those who complete study.– creating a problem of external validity.