Business Research Methods

William G. Zikmund

Chapter 12: Experimental Research

Experiment

• A research investigation in which conditions are controlled

• One independent variable is manipulated (sometimes more than one)

• Its effect on a dependent variable is measured• To test a hypothesis

Basic Issues of Experimental Design

• Manipulation of the Independent Variable• Selection of Dependent Variable• Assignment of Subjects (or other Test

Units)• Control Over Extraneous Variables

The experimenter has some degree of control over the independent variable. The variable is independent because its value can be manipulated by the experimenter to whatever he or she wishes it to be.

Experiment Treatment

Alternative manipulations of the independent variable being

investigated

Independent Variable

• The experimenter controls independent variable.

• The variable’s value can be manipulated by the experimenters to whatever they wish it to be.

Manipulation of Independent Variable

• Classificatory Vs. continuous variables• Experimental and control groups• Treatment levels• More than one independent variable

Experimental Treatments

• The alternative manipulations of the independent variable being investigated

Dependent Variable

• Its value is expected to be dependent on the experimenter’s manipulation

• Criterion or standard by which the results are judged

Dependent Variable

• Selection– e.g... sales volume, awareness, recall,

• Measurement

Test Units

• Subjects or entities whose response to the experimental treatment are measured or observed.

Two Types of Experimental Error

• Constant errors• Random errors

Field versus Laboratory Experiments

Controlling Extraneous Variables

• Elimination of extraneous variables• Constancy of conditions• Order of presentation• Blinding• Random assignment

How May an Experimenter control forExtraneous Variation?

• Eliminate Extraneous Variables• Hold Conditions Constant• Randomization• Matching Subjects

Establishing Control

Demand Characteristics

• Experimental procedures that intentionally hint to subjects something about the experimenter’s hypothesis

Demand Characteristics

• Guinea pig effect

• Hawthorne effect

Field Vs. Laboratory Experiment

Laboratory Experiment Field Experiment

Artificial-Low Realism

Few ExtraneousVariables

High control

Low Cost

Short Duration

Subjects Aware ofParticipation

Natural-High Realism

Many ExtraneousVariables

Low control

High Cost

Long Duration

Subjects Unaware ofParticipation

Control Groups

Isolate extraneous variation

When does an Experiment have Internal Validity?

Internal Validity - The ability of an experiment to answer the question whether the experimental treatment was the sole cause of changes in a dependent variable

Did the manipulation do what it was supposed to do?

Factors Influencing Internal Validity

• History• Maturation• Testing• Instrumentation• Selection• Mortality

Isolating Extraneous Variationwith a Control Group

• History Effects• Maturation Effects• Mortality Effects

Type of Extraneous Variable Example

History - Specific events in theenvironment between the Beforeand After measurement that are beyond the experimenter’s control

Maturation - Subjects changeduring the course of the experiment

Testing - The Before measure alertsor sensitizes subject to nature of experiment or second measure.

A major employercloses its plant intest market area

Subjects become tired

Questionnaireabout the traditionalrole of women triggers enhanced awareness of womenin an experiment.

Instrument - Changes ininstrument result in response bias

Selection - Sample selectionerror because of differentialselection comparison groups

Mortality - Sample attrition; some subjects withdraw from experiment

New questions aboutwomen are interpreteddifferently from earlierquestions.

Control group and experimental group isself-selected groupbased on preference forsoft drinks

Subjects in one groupof a hair dying study marry rich widows and move to Florida

How can Internal Validity Increase?

Increasing Internal Validity

• Control group

• Random assignment

• Pretesting and posttesting

• Posttest only

What are the Different Basic Experimental Designs?

Quasi-Experimental Designs

• One Shot Design (After Only)• One Group Pretest-Posttest• Static Group Design

One Shot Design (After Only)

X O1

One Group Pretest-Posttest

O1 X O2

Static Group Design

Experimental Group X O1 Control Group O2

Three Good Experimental Designs

• Pretest - Posttest Control Group Design

• Posttest Only Control Group

• Solomon Four Group Design

Pretest-Posttest Control Group Design

Experimental Group R O1 X O2

Control Group R O3 X O4

Posttest Only Control Group

Experimental Group R X O1

Control Group R O2

One-Shot DesignInternal Validity Problems• History

– weak• Maturation

– weak• Testing

– not relevant

• Instrumentation– not relevant

• Selection– weak

• Mortality– weak

One-Group Pretest-PosttestInternal Validity Problems

• History– weak

• Maturation– weak

• Testing– weak

• Instrumentation– weak

• Selection– controlled

• Mortality– controlled

Static-Group DesignInternal Validity Problems

• History– controlled

• Maturation– possible source of

concern• Testing

– controlled

• Instrumentation– controlled

• Selection– weak

• Mortality– weak

Pretest-Posttest ControlInternal Validity Problems• History

– controlled• Maturation

– controlled• Testing

– controlled

• Instrumentation– controlled

• Selection– controlled

• Mortality– controlled

Solomon Four-Group DesignInternal Validity Problems• History

– controlled• Maturation

– controlled• Testing

– controlled

• Instrumentation– controlled

• Selection– controlled

• Mortality– controlled

Posttest-Only ControlInternal Validity Problems

• History– controlled

• Maturation– controlled

• Testing– controlled

• Instrumentation– controlled

• Selection– controlled

• Mortality– controlled

Solomon Four Group Design

Experimental Group 1: R O1 X O2

Control Group 1: R O3 O4

Experimental Group 2: R X O5

Control Group 2: R X O6

Advanced Experimental Designs are More Complex

• Completely randomized• Randomized block design• Latin square• Factorial

Completely Randomized Design

• An experimental design that uses a random process to assign subjects (test units) and treatments to investigate the effects of only one independent variable.

Completely Randomized Designs

Average minutesshopper spendsin store

Control:no music

Experimentaltreatment:slow music

Experimentaltreatment:fast music

16 18 12

Independent Variable A

Group A Group B Group C

Level 1 Level 2 Level 3

Completely Randomized Design

With a pretest posttest

Group A R O1 X1 O2

Group A R O3 X2 O4

Group A R O5 X3 O6

With a posttest

Group A R X1 O1

Group B R X2 O2

Group C R X3 O3

Completely Randomized Design

Randomized Block Design

• An extension of the completely randomized design in which a single extraneous variable that might affect test units’ response to the treatment has been identified and the effects of this variable are isolated by blocking out its effects.

Independent Variables

Control:no music

Experimentaltreatment

slow music

Experimentaltreatment: fast music

Mornings andafternoons

Evening hours

Blo

ckin

g va

riabl

e

Randomized Block Design

Factorial Design

• An experiment that investigates the interaction of two or more variables on a single dependent variable.

Independent Variable 1

No Musiccart signs

Slow Music Fast MusicNo Music

Grocerycart signs

Inde

pend

ent V

aria

ble

2

Price Red Gold

$25 Cell 1 Cell 4$30 Cell 2 Cell 5$35 Cell 3 Cell 6

Package Design

Factorial Design -- Roller Skates

Effects

• Main effect• The influence of a

single independent variable on a dependent variable.

• Interaction effect• The influence on a

dependent variable by combinations of two or more independent variables.

Men

Women

Ad A Ad B

65

65

70 60

Main Effectsof Gender

Main Effects of Ad

>

>

2 x 2 Factorial Design

100

90

80

70

60

50

40

30

20

10Ad A Ad B

Women

Men

Bel

ieva

bili t

yInteraction Between Gender and

Advertising Copy

Level 1 Level 2

Level 1

Level 2

Group A

Group DGroup C

Group B

Inde

pend

ent

Var

iabl

e 2

Independent Variable 1

Group A R O1 X11 O2

Group B R O3 X21 O4

Group C R O5 X12 O6

Group D R O7 X22 O8

2 x 2 Factorial with a Pretest Posttest

Group A R X11 O1

Group B R X21 O2

Group C R X12 O3

Group D R X22 O4

2 x 2 Factorial Design with a Posttest Measure

A Test Market Experiment on Pricing

Sales in Units (thousands)

Regular Price$.99

1301188784

X1=104.75X=119.58

Reduced Price$.89

145143120131

X2=134.75

Cents-Off CouponRegular Price

1531299699

X1=119.25

Test Market A, B, or CTest Market D, E, or FTest Market G, H, or ITest Market J, K, or L

MeanGrand Mean

Latin Square Design

• A balanced, two-way classification scheme that attempts to control or block out the effect of two or more extraneous factors by restricting randomization with respect to the row and column effects.

1 2 3

1 A B C2 B C A3 C A B

Order of UsageS

UB

JEC

T

TEST MARKETING

Controlled experimentationControlled experimentation

Not just tryingNot just tryingsomethingsomethingoutout

But scientificBut scientifictestingtesting

Controlled experimentationControlled experimentation

Not just tryingNot just tryingsomethingsomethingoutout

But scientificBut scientifictestingtesting

Test Marketing

Test Marketing

• An experimental procedure that provides an opportunity to test a new product or a new marketing plan under realistic market conditions to measure sales or profit potential.

ESTIMATEESTIMATEOUTCOMESOUTCOMES

IDENTIFY ANDIDENTIFY ANDCORRECTCORRECT

WEAKNESSESWEAKNESSESIN PLANSIN PLANS

Functions of Test Marketing

A Lengthy and Costly Procedure

$$$$$$$$$$

Loss of Loss of SecrecySecrecy

When notWhen notto Test?to Test?

How LongHow LongShould aShould a

Test Last?Test Last?

Popular Test Markets

• Pittsfield, Massachusetts

• Charlotte, North Carolina

• Columbus, Ohio• Little Rock, Arkansas• Evansville, Indiana • Cedar Rapids, Iowa

• Eau Claire,Wisconsin• Wichita, Kansas• Tulsa, Oklahoma • Omaha, Nebraska• Grand Junction.

Colorado• Wichita Falls, Texas• Odessa-Midland, Texas

Selecting a Test Market

• Population size• Demographic composition• Lifestyle considerations• Competitive situation• Media• Self-contained trading area• Overused markets - secrecy

Control Method of Test Marketing

• Small city• Low chance of being detected• Distribution is forced (guaranteed)

The Advantages of Using the Control Method of Test Marketing

• Reduced costs • Shorter time period needed for reading test

market results• Increased secrecy from competitors• No distraction of company salespeople from

regular product lines

Some Problems Estimating Sales Volume

• Over-attention• Unrealistic store conditions• Reading competitive environment

incorrectly• Incorrect volume forecasts

– Adjusted data– Penetration and repeat purchase rate

• Time lapse

High Tech Test Markets

ElectricElectricTestTest

MarketsMarketsSimulatedSimulated

TestTestMarketsMarkets Virtual-realityVirtual-reality

SimulatedSimulatedTest MarketsTest Markets