FDN 5000 Research in Education

Dr. George H. Olson, Instructor

Research DesignsDescriptive Research DesignsCorrelation Research DesignsCausal-comparative DesignsExperimental DesignsNon-experimental Designs

Single-subject designs

Purposes of Research Designs

Provide guidance in conducting researchProvide guidance in interpreting research

Guidance in recognizing types of research Guidance in recognizing potential threats to

the validity of research conclusionsHelp guard against threats to internal and

external validity

Descriptive research designs

Survey research Longitudinal research designs

Cohort designsQuasi-cohort designs

Cross-sectional researchCase studiesEthnographic studies

Correlational research designs

Objective: study the relationship between variables Examine scatter plots (e.g., pages 223

and 225 in our text) Compute correlation coefficients

range from -1 through 0 to +1do not imply causationlack of correlation does NOT imply no

causation

Interpreting correlations

Rules of thumb:Correlation coefficient Strength of relationship

.00 to .20 negligible .24 to .40 low .40 to .60 moderate .60 to .80 substantial .80 to 1.00 high, very high

Causal comparative research designs

A.k.a. ex-post-facto designsAimed at discovering cause and effect

relationshipsDefined groups are compared after

they have been formedTheory plays an important role

In arguing for the cause-effect relationship In eliminating rival explanations

Causal comparative designs: An example

Of students receiving the This proportion is known to

following grades…. drive a car to school

A 8%B 23%C 43%D 77%F 96%

What should the superintendent conclude?

Diagramming Experimental Research Designs

Symbols used: T = treatment intervention C = control or comparison condition

(often simply no treatment) O = observation (often some test score) R = designates random assignment M = designates matching

Examples of Research Design Diagrams

(1) T O

(2) O1 T O2

O1 T O2

(3) ----------------------- (No Random Assignment)

O1 C O2

Three pre-experimental designs

Three designs frequently used in education research that… are not sufficient for permitting strong

tests of causal hypotheses often due suggest new ideas

One-group posttest-only designOne-group pretest-posttest designComparison-group posttest-only design

One-group, posttest-only research design

T O

A treatment followed by an observation Should not be confused with the one-shot case

study Threats to internal validity:

ALL (except regression and mortality) Threats to external validity:

ALL

One-group pretest-posttest research design

Opre T Opost

One of the most frequently used research

designs in education Threats to internal validity:

extraneous events (history and maturation) statistical regression testing experimenter and subject effects

Threats to external validity selection and settings interactions with treatment

Comparison-group, posttest-only design

T O--------------C O--------------C2 O--------------C3 O

Comparison-group, posttest-only design

Threats to internal validity Since the comparison groups are non-

equivalent, the major threat is selection Other threats include mortality, and

subject and experimenter reactive effectsThreats to external validity

Selection and settings by treatment interaction

True experiment research designs

Randomized experimentsResult in probabilistic equivalenceNot a panacea that rules out all

threats to internal validity Does not control for experimenter and

subject reactive effects. Does not guarantee group equivalency

(especially in small samples).

Randomized posttest comparison group design

T OR: ---------------

C O

Note: R: means RANDOMIZATION

Randomized pretest-posttest control group design

Opre T Opost

R: ---------------------------Opre C Opost

R: ---------------------------Opre C2 Opost

Randomized matched-group design

T OM: R: ---------------

C O

Randomized factorial designs

TA1,B1 O---------------TA1,B2 O

R: ---------------TA2,B1 O---------------TA2,B2 O

Factorial Design: Example

Method (B)____ Word Type (A) Computer Handwriting

B1 B2

Easy A1 20 26Hard A2 16 20

____________________________

Quasi-experiments: Time series designs

O1 O2 O3 O4 T O5 O6 O7 O8

Pre-observations to establish a baseline

A treatment intervention Post-observations to establish new

baseline

Quasi-experiments: Non-equivalent control groups

In these designs, randomization is either not possible or not feasible.

Characterized by ... using intact groups for treatment and

comparison manipulated independent variable

Often, the best we can expect from education research

Non-equivalent, control group, pretest-posttest design

Opre T Opost

------------------------- Opre C Opost

Except for reactive effects, most threats to internal validity are controlled

Again settings and selection by treatment interactions pose threats to external validity

Matched comparison group, posttest design

T OM: ---------

C OValidity depends upon how well

matching is achievedPotential threats to internal validity

are same as those for posttest-only designs

Single-subject designs

Similar to time-series designs, only with a single individual

Repeated measurements over time (baselines)

Subjects serve as their own controlsInvolve a manipulated independent

variable (the intervention)

Basic single-subject designs

Reversal: A - B - ADouble reversal: A - B - A - BMultiple baseline:

A - B - A------------------------ A - B - A------------------------

A - B - A------------------------

A - B - AA is a period of no treatmentB is a period of treatment

Example of a stable baseline

0 10 20 30 40 50 60 70 80 90

100

Stable Baseline Pattern

Example of an increasing baseline

0

20

40

60

80

100

Increasing Baseline Pattern

Non-experimental research designs

Characterized by the lack of manipulation of an independent variable

Three types of non-experimental research designs: Causal comparative research designs Correlational research designs Descriptive research designs