CHAPTER THREE

Research Design: The Experimental Model and Its Variations

Research Design

Research design is the plan, the blueprint, or a schematic for a study – it flows from

the problem formulation stage.

The who, what, where, when, why, and how

of an investigation.

The goal is how best to address the hypothesis.

Causality

The purpose of scientific investigation is:

to isolate, define, and explain the relationship between key variables

in order to predict and understand

the underlying nature of reality.

(who, what, where, when, and how)

Causation lies at the basis of reality.

Questions

How would causation apply to the variables:

foot patrol and crime?

Does showing a relationship (correlation) between

two variables imply or demonstrate causation?

Steps for Resolvingthe Causality Problem

• Demonstration of a relationship between variables (covariance). In other words, is there a predictable relationship in the value of one variable to the value of another variable.

• Specifying or indicating the time sequence or time order of the relationship. Which variable will be “X” (predictor) and which variable will be “Y” (outcome).

• Eliminate rival causal factors to avoid a spurious relationship (a false relationship) or, avoid the elimination of other variables that could conceivably explain away the original relationship.

Classic Experimental Design

E O1 X O2E O1 O2

E = equivalenceO = observationX = treatment1,2 = time

Examples: Candid Camera, Scared Straight, Community Policing (Text, pages: 91-93).

Experimental DesignTerms:

X = treatment (independent variable)

Y = outcome (dependent variable)

Z = any rival causal factors

O = observation (some measurement or assessment of the dependent variable)

E = equivalence (randomization or matching)

1,2 = number of times

Rival Causal Factors

Factors other than “X” (the treatment) that may be responsible for the relationship.

Validity refers to accuracy or correctness in research, i.e., internal and external.

Internal Validity is concerned with a variable other than X that may have produced the change in Y.

External Validity is concerned with what other variables may limit one’s ability to generalize the findings in a study to larger groups or populations.

Internal Validity

History: Refers to other specific events that may have occurred over the time of the study that may have produced the results. Example: a new program introduced such as “Crime Watch” or urban renewal.

Maturation: Biological or psychological changes in respondents during the course of study that are not due to the treatment variable. Example: age.

Testing: Pretest bias, bias and foreknowledge introduced to respondents as a result of having been pretested.

Internal Validity (cont’d)

Instrumentation: Changing the measurement instrument from the beginning or the first period of evaluation to the second or final evaluation. Ex.: the method of recording citizen complaints changes.

Statistical Regression: the tendency of groups selected for study on the basis of high or low scores to regress or move toward the mean or the average on second testing. Scores become more normal upon retest. Ex.: First test may have been atypical.

Internal Validity (cont’d)

Selection Bias: Choosing nonequivalent groups for testing. Ex: Selecting all volunteer prisoners or all model prisoners.

Experimental Mortality: Expected loss of subjects in the sample group over a period of time. Ex: Following recidivism cohorts over long periods of time.

Selection-Maturation Interaction: Selection bias coupled with issues that emphasize biological or psychological changes during the course of the study or over time. Ex.: Age and Recidivism

External Validity

Testing Effects: Exposure to pretests by respondents negates the generalizability of the results to larger populations that have not been pretested. Example: Pretest of community attitudes toward the police prior to a foot patrol experiment followed up by a posttest regarding community attitudes toward the police after the experiment.

Selection Bias: Specific studies that are based on a specific group may not be comparable to a larger group that does not have the same specific characteristics. Ex.: Local drug use studies.

External Validity (Cont’d)

Reactivity: Awareness of being studied tends to produce atypical or unnatural behavior on the parts of subjects. Example: Hawthorne Effect

Multiple-Treatment Interferences: Occurs when more than one treatment or predictor variable is used on the same subjects. Example: addition of foot patrol plus foot patrol officers were also unarmed and wore different uniforms.

Question

What might be a rival causal factor which may affect the relationship between our

predictor variable foot patrol (X),

and our outcome variable crime (Y)?

Related Rival Causes(be able discuss the following):

Hawthorne Effect

Halo Effect

Post Hoc Error

Placebo Effect

Diffusion of Treatment

Compensatory Equalization of Treatment

Local History

Experimental DesignTerms:

X = treatment (independent variable)

Y = outcome (dependent variable)

Z = any rival causal factors

O = observation (some measurement or assessment of the dependent variable)

E = equivalence (randomization or matching)

1,2 = number of times

Classic Experimental Design

E O1 X O2E O1 O2

E = equivalenceO = observationX = treatment1,2 = time

Examples: Candid Camera, Scared Straight, Community Policing (Text, pages: 91-93).

THREE ELEMENTS OFTHE CLASSIC EXPERIMENTAL DESIGN

1) Equivalence is the assignment to comparison groups in a manner in which the subjects are alike in all major respects, i.e., randomization and matching. (E)

Randomization is the random assignment of subjects where all individuals have an equal probability to being assigned to a particular group.

Matching is the selecting of subjects for comparison groups based on key characteristics so that the group is similar in respect to these characteristics.

Three Elements(Cont’d)

2) The classic experiment consists of pretests and posttests.

A pretest is an observation prior to exposure to treatment (O1).

A posttest is an observation and measurement after treatment (O2).

Three Elements(Cont’d)

3) The classic experiment consists of experimental and control groups.

The experimental group is exposed to treatment (X).

The control group is not exposed to the treatment.

Solomon Four-Group Design

E O1 X O2

E O1 O2

E X O2

E 02

Eliminates testing effects and reactivity (awareness of being studied)

Cross-sectional andLongitudinal Design

Often referred to as cross-sectional (one group at one time); longitudinal (one group over time), i.e., time-series, cohort studies, panel studies, and trend studies.

Time series: measuring a single variable at successive points in time (E = matching)

Interrupted time-series: measurement before and after treatment for an equivalent period of time

Trend studies: analyze different sample of the same population longitudinally

Cohort studies: Analyze subgroups over time.

Panel studies: Analyze the same group over time.

Time Series Design

Interrupted Time-Series DesignsO O O X O O O

Multiple Time-Series DesignsO O O X O O OO O O O O O

Examples: Problem Oriented Policing (Spelman, et al.), Monahan and Walker’s Study of mental health centers, and Shneider and Smykla’s “War and Capital Punishment” study (Text, pages: 102-103).

Advantages andDisadvantage of Experiments

Advantages: 1) Control of rival factors (internal validity); 2) Quick and inexpensive; 3) Manageability; and, 4) Can be applied to natural settings.

Disadvantages: 1) Artificiality (hinders generalizability); and, 2) Difficult to apply experiments to human subjects and situations in criminal justice (i.e., ethical issues and experimenter effects). Disadvantages in criminal justice subject matter often outweigh the advantages.

Questions

What type of design was the

Kansas City Gun Experiment?

What were the major findings of the project?

What is a rival factor of concern

when evaluating shock incarceration programs?

Why are time-series designs particularly useful in criminal justice studies?