EXPERIMENTAL DESIGNEXPERIMENTAL DESIGN

Experimental and Pre (Quasi) Experimental and Pre (Quasi) Experimental DesignsExperimental Designs

Basic Issues in Experimental Basic Issues in Experimental DesignDesign

Manipulation of the Independent VariableManipulation of the Independent Variable

Selection of the Dependent VariableSelection of the Dependent Variable

Assignment of subjects (or other test units)Assignment of subjects (or other test units)

Control over extraneous variablesControl over extraneous variables

Manipulation of the Independent Manipulation of the Independent VariableVariable

In a classical experiment, the researcher In a classical experiment, the researcher measures the condition of the subjects – measures the condition of the subjects – both treatment and control groups - on the both treatment and control groups - on the dependent variable and then dependent variable and then “manipulates” the independent variable for “manipulates” the independent variable for the “treatment group” while not the “treatment group” while not “manipulating” that variable for the “manipulating” that variable for the “control” group, and then “re-measures the “control” group, and then “re-measures the dependent variable for both groups while dependent variable for both groups while controlling “all other variables”.controlling “all other variables”.

Laboratory vs Field ExperimentsLaboratory vs Field Experiments

Artificial – Low Artificial – Low RealismRealism

Few Extraneous Few Extraneous VariablesVariables

High ControlHigh Control

Low CostLow Cost

Short DurationShort Duration

Subjects Aware of Subjects Aware of ParticipationParticipation

Natural – High Natural – High RealismRealism

Many Extraneous Many Extraneous VariablesVariables

Low ControlLow Control

High CostHigh Cost

Long DurationLong Duration

Subjects Aware of Subjects Aware of ParticipationParticipation

““ONE-SHOT” CASE STUDYONE-SHOT” CASE STUDY

T1T1 T2T2 T3T3

XX OO

Where: Where: X = TreatmentX = Treatment

O = ObservationO = Observation

How good is this research design?How good is this research design?

Scientifically it is not good at allScientifically it is not good at all– However, that the classical case studies in all of the social However, that the classical case studies in all of the social

sciences have contributed more to those discipline’s knowledge sciences have contributed more to those discipline’s knowledge than any other designthan any other design

First – much of the pioneer work that shaped the social First – much of the pioneer work that shaped the social sciences was accomplished before the advent of sciences was accomplished before the advent of sophisticated research designssophisticated research designsSecond –research is always the enterprise of individuals Second –research is always the enterprise of individuals and individual skill can “substitute” for sophisticated and individual skill can “substitute” for sophisticated design controlsdesign controlsThird – research designs which require sophisticated Third – research designs which require sophisticated controls are limited to different kinds of inquirescontrols are limited to different kinds of inquires

““ONE-GROUP”ONE-GROUP” PRETEST - POSTTEST PRETEST - POSTTEST

T1T1 T2T2 T3T3

OO XX OO

““STATIC GROUP” COMPARISONSTATIC GROUP” COMPARISON

T1T1 T2T2 T3T3

XX OO

OO

““PRETEST - POSTTEST”PRETEST - POSTTEST”CONTROL GROUP DESIGNCONTROL GROUP DESIGN

T1T1 T2T2T3T3

TrtTrt OO XX OO

CtrlCtrl OO OO

Posttest-Only Control GroupPosttest-Only Control Group

T1T1 T2T2 T3T3

OO XX OO

OO

““SOLOMON” FOUR - GROUP SOLOMON” FOUR - GROUP DESIGNDESIGN

T1T1 T2T2 T3T3

OO XX OO

OO OO

XX OO

OO

More Complex Experimental More Complex Experimental DesignsDesigns

Completely randomized designsCompletely randomized designs

Randomized block designRandomized block design

Factorial designFactorial design

Latin square designLatin square design

Experimental ConditionsExperimental Conditions

Double Blind experiment – neither Double Blind experiment – neither “observer” nor the “subjects” know “observer” nor the “subjects” know whether the subject are in the treatment or whether the subject are in the treatment or control groups or receiving a placebo.control groups or receiving a placebo.Placebo effect – One experimental group Placebo effect – One experimental group is given the “treatment” that is being tested is given the “treatment” that is being tested while another “treatment” group receives a while another “treatment” group receives a “treatment” but one that is, by design, “treatment” but one that is, by design, “benign” in its effect.“benign” in its effect.

Extraneous VariablesExtraneous Variables

HistoryHistory

MaturationMaturation

TestingTesting

InstrumentationInstrumentation

SelectionSelection

MortalityMortality

Extraneous VariablesExtraneous Variables

History – specific events in the environment History – specific events in the environment between the before and after measurement that between the before and after measurement that are beyond the experimenter’s control are beyond the experimenter’s control

e.g. a major employer closes its plant in a e.g. a major employer closes its plant in a test market areatest market area

Maturation – subjects change during the course Maturation – subjects change during the course of the experimentof the experiment

e.g. subjects become tirede.g. subjects become tired

Extraneous variablesExtraneous variables (cont -2)(cont -2)

Testing – the before measurement alerts Testing – the before measurement alerts subject to the nature of the experimentsubject to the nature of the experiment– e.g. questionnaire about traditional role of e.g. questionnaire about traditional role of

women triggers enhanced awareness of women triggers enhanced awareness of women in the experimentwomen in the experiment

Instrumentation – changes in measuring Instrumentation – changes in measuring instrument result in response biasinstrument result in response bias– e.g. new questions about women are e.g. new questions about women are

interpreted differently than earlier questionsinterpreted differently than earlier questions

Extraneous variablesExtraneous variables (cont -3)(cont -3)

Selection – sample selection error occurs Selection – sample selection error occurs because of differential selection of comparison because of differential selection of comparison groupsgroups– e.g. control and experimental groups are assigned e.g. control and experimental groups are assigned

self-selecting groups based on preference for soft self-selecting groups based on preference for soft drinksdrinks

Mortality – sample attrition, some subjects Mortality – sample attrition, some subjects withdraw from experimentwithdraw from experiment– e.g. subjects in one group of students withdraw from e.g. subjects in one group of students withdraw from

schoolschool

Control for Extraneous VariationControl for Extraneous Variation

Eliminate Extraneous VariablesEliminate Extraneous Variables

Hold Conditions ConstantHold Conditions Constant

RandomizationRandomization

Matching SubjectsMatching Subjects

Demand CharacteristicsDemand Characteristics

Design procedures that “hint” to subjects Design procedures that “hint” to subjects about the experimenter’s hypothesisabout the experimenter’s hypothesis– Guinea Pig effect – when subjects exhibit Guinea Pig effect – when subjects exhibit

behavior that they feel is expected of thembehavior that they feel is expected of them– Hawthorne effect – causes people to perform Hawthorne effect – causes people to perform

differently just because they know they are differently just because they know they are experimental subjectsexperimental subjects

Problems of Internal ValidityProblems of Internal ValidityDesignDesign HistoryHistory MaturationMaturation TestingTesting InstrumentationInstrumentation SelectionSelection MortalityMortality

One shot designOne shot design 00 00 NRNR NRNR 00 00

One group One group pre/post test pre/post test designdesign

00 00 00 00 00 00

Static group Static group designdesign XX ?? XX XX 00 00

PosttestPosttestControl Group Control Group onlyonly XX XX XX XX XX XX

Pre/PostPre/PostControl Group Control Group designdesign

XX XX XX XX XX XX

Solomon Four Solomon Four Group designGroup design

XX XX XX XX XX XX

0 = weakness X = factor controlled ? = possible concern NR = not relevant0 = weakness X = factor controlled ? = possible concern NR = not relevant

Ethical IssuesEthical Issues

HarmHarm

““Unfair benefit”Unfair benefit”

PrivacyPrivacy

ConfidentialityConfidentiality

DeceptionDeception

Not using debriefingNot using debriefing