Experimental Design (1)

Experimental Research

© 1–1

"Influence of brand name on consumers' taste perceptions" Explain through experiment

“ Audible range influenced by rhythmicity of sound” Explain through experiment.

“use of product often influenced by the ads presentation” Explain through experiments.

12–2

Creating an Experiment

• Subjects• The sampling units for an experiment, usually human

respondents who provide measures based on the experimental manipulation.

• Independent Variables• Experimental conditions

• One of the possible levels of an experimental (independent) variable manipulation.

• Blocking variables• Variables included in the statistical analysis as a way of

controlling or accounting for variance due to that variable:• Categorical variables

12–3

An experiment investigating “how self-efficacy might influence employee’s attitude toward their job” (Self-efficacy is a person’s confidence and belief in their own abilities to accomplish tasks at hand).

12–4

EXHIBIT 12.1 Experimental Conditions in Self-Efficacy Experiment

12–5

Creating an Experiment (cont’d)

• Main Effect• The experimental difference in dependent variable

means between the different levels of any single experimental variable.

• Interaction Effect• Differences in dependant variable means due to a

specific combination of independent variables.

12–6

EXHIBIT 12.2 Job Satisfaction Means in Self-Efficacy Experiment

12–7

EXHIBIT 12.3 Experimental Graph Showing Results within Each Condition

12–8

Designing an Experiment to Minimize Experimental Error

• Manipulation of the Independent Variable• Experimental treatment: the way an experimental

variable is manipulated.

• Categorical variables: described by class or quality

• Continuous variables: described by quantity (level)

• Experimental Group

• A group of subjects to whom an experimental treatment is administered.

• Control Group

• A group of subjects to whom no experimental treatment is administered.

12–9

Designing an Experiment (cont’d)

• More than One Independent Variable• Cell: a specific treatment combination associated with

an experimental group.

• Computation of the number of cells in an experiment:

K = (T1)(T2)..(Tm)

• Repeated Measures• Experiments in which an individual subject is exposed

to more than one level of an experimental treatment.

12–10


• Selection and Measurement of the Dependent Variable• Selecting dependent variables that are relevant and

truly represent an outcome of interest is crucial.

• Choosing the right dependent variable is part of the problem definition process.

• Thorough problem definition will help the researcher select the most important dependent variable(s).

12–11


• Selection and Assignment of Test Units• Test units: the subjects or entities whose responses to

treatment are measured or observed.

• Sample Selection And Random Sampling Errors• Systematic or nonsampling error

• Subject selection, experimental design, and unrecognized extraneous variables

• Overcoming sampling errors• Randomization• Matching• Control over extraneous variables

12–12


• Sample Selection And Random Sampling Errors• Experimental Confound

• When there is an alternative explanation beyond the experimental variables for any observed differences in the dependent variable.

• Once a potential confound is identified, the validity of the experiment is severely questioned.

• Extraneous variables• Variables that naturally exist in the environment

that may have some systematic effect on the dependent variable.

12–13

Demand Characteristics

• Demand Characteristic• An experimental design element or procedure that

unintentionally provides subjects with hints about the research hypothesis.

• Demand Effect• Occurs when demand characteristics actually affect

the dependent variable.

• Hawthorne Effect• People will perform differently from normal when they

know they are experimental subjects.

12–14

Reducing Demand Effects

12–15

Establishing Control

• Constancy of Conditions• Subjects in all experimental groups are exposed to

identical conditions except for the differing experimental treatments.

• Counterbalancing• Attempts to eliminate the confounding effects of order

of presentation by varying the order of presentation (exposure) of treatments to subject groups.

12–16

Ethical Issues in Experimentation

• Debriefing experimental subjects• Communicating the purpose of the experiment

• Explaining the researcher’s hypotheses

• Attempts to interfere with a competitor’s test-marketing efforts• Such acts as changing prices or increasing

advertising to influence (confound) competitors’ test-marketing results are ethically questionable.

12–17

Practical Experimental Design Issues

• Basic versus Factorial Experimental Designs• Basic experimental designs – a single independent variable and

a single dependent variable.• Factorial experimental design – allows for an investigation of the

interaction to two or more independent variables.

• Laboratory Experiment• A situation in which the researcher has more complete control

over the research setting and extraneous variables.

• Field Experiments• Research projects involving experimental manipulations that are

implemented in a natural environment.

12–18

EXHIBIT 12.5 The Artificiality of Laboratory versus Field Experiments

12–19

Within-Subjects and Between-Subjects Designs

• Within-Subjects Design• Involves repeated measures because with each

treatment the same subject is measured.

• Between-Subjects Design• Each subject receives only one treatment

combination.

• Usually advantageous although they are usually more costly.

• Validity is usually higher.

12–20

EXHIBIT 12.6

Within- and Between-Subjects Designs

12–21

Issues of Experimental Validity

• Internal Validity• The extent that an experimental variable is truly

responsible for any variance in the dependent variable.

• Does the experimental manipulation truly cause changes in the specific outcome of interest?

• Manipulation Checks• A validity test of an experimental manipulation to

make sure that the manipulation does produce differences in the independent variable.

12–22

Extraneous Variables Affecting Internal Validity

12–23

Internal Validity

Internal Validity

MaturationMaturation

TestingTesting

InstrumentationInstrumentationSelectionSelection

MortalityMortality

HistoryHistory

Effects of Extraneous Variables on Validity

• History Effect• Occurs when some change other than the

experimental treatment occurs during the course of an experiment that affects the dependent variable.

• Cohort Effect• A change in the dependent variable that occurs

because members of one experimental group experienced different historical situations than members of other experimental groups.

12–24

Effects of Extraneous Variables… (cont’d)

• Maturation Effects• Effects that are a function of time and the naturally

occurring events that coincide with growth and experience.

• Testing effects• A nuisance effect occurring when the initial

measurement or test alerts or primes subjects in a way that affects their response to the experimental treatments.

12–25

Effects of Extraneous Variables… (cont’d)

• Instrumentation Effect• A change in the wording of questions, a change in

interviewers, or a change in other procedures causes a change in the dependent variable.

• Selection• The selection effect is a sample bias that results from

differential selection of respondents for the comparison groups, or a sample selection error.

• Mortality Effect (Sample Attrition)• Occurs when some subjects withdraw from the

experiment before it is completed.

12–26

Extraneous Variables

12–27

HistoryUncontrollable events occurring in the environment between before and after measurements

MaturationChanges in subjects during the course of the experiment

TestingA before measure that alerts or sensitizes subject to the nature of experiment or second measure.

A major employer closes its plant in test market area.

Subjects become tired during the experiment.

A questionnaire about the traditional role of women triggers enhanced awareness of females in an experiment.

Extraneous Variable Example

Extraneous Variables (cont’d)

12–28

Instrument –Changes in instrument result in response bias

SelectionSample selection error because of differential selection comparison groups

MortalitySample attrition; some subjects withdraw from experiment

New questions about women are interpreted differently from earlier questions.

Control group and experimental group is self-selected group based on preference for soft drinks

Subjects in one group of a hair dying study marry rich widows and move to Florida

Extraneous Variable Example

Issues of Experimental Validity (cont’d)

• External Validity• The accuracy with which experimental results can be

generalized beyond the experimental subjects.• Student surrogates: Atypical?

• Trade-Offs Between Internal and External Validity• Artificial laboratory experiments usually are high in

internal validity, while naturalistic field experiments generally have less internal validity, but greater external validity.

12–29

Laboratory versus Field Experiments

12–30

LaboratoryExperiment

FieldExperiment

Artificial: Low RealismArtificial: Low Realism

Few ExtraneousVariables

Few ExtraneousVariables

High controlHigh control

Low CostLow Cost

Short DurationShort Duration

Subjects Aware ofParticipation

Subjects Aware ofParticipation

Natural: High RealismNatural: High Realism

Many ExtraneousVariables

Many ExtraneousVariables

Low controlLow control

High CostHigh Cost

Long DurationLong Duration

Subjects Unaware ofParticipation

Subjects Unaware ofParticipation

Classification of Experimental Designs

• Basic Experimental Design• An experimental design in which only one variable is

manipulated.

• Diagramming Experimental Designs: Symbols

12–31

Examples of Quasi-Experimental Designs

• Quasi-experimental Designs• Experimental designs that do not involve random

allocation of subjects to treatment combinations.

• One Shot Design (After Only): X O1

• One Group Pretest–Posttest: O1 X O2

• Static Group Design: Experimental X O1

Control O2

12–32

Alternative Experimental Designs

• Pretest–Posttest Control Group Design(Before–After with Control)• Experimental R O1 X O2

• Control R O3 X O4

• Posttest Only Control Group(After-Only with Control)• Experimental R X O1

• Control R O2

12–33

EXHIBIT 12.7 Product Preference Measure in an Experiment

12–34

EXHIBIT 12.8

Selected Time Series Outcomes

12–35

Complex Experimental Designs

12–36

Completely Randomized

Design

Randomized Block Design

Factorial

Complex Experimental Designs (cont’d)

• Completely Randomized Design• An experimental design that uses a random process

to assign subjects (test units) to treatment levels to investigate the effects of an experimental variable.

12–37


• Randomized Block Design• An extension of the completely randomized design in

which a single categorical extraneous variable that might affect test units’ responses to the treatment is identified and the effects of this variable are isolated by being blocked out.

• Blocking Variable• A categorical variable that is expected to be

associated with different values of a dependent variable for each group. It effectively controls for an extraneous cause in experimental analysis.

12–38

EXHIBIT 12.9 Randomized Block Design

12–39


• Factorial Design• An experiment that investigates the interaction of two or

more independent variables on a single dependent variable.

12–40

EXHIBIT 12.10 Factorial Design—Salary and Vacation

12–41

Effects in Factorial Design

• Main effect• The influence of a single independent variable on a

dependent variable.

• Interaction effect• The influence on a dependent variable by

combinations of two or more independent variables.

• Interaction occurs if the effect of one treatment differs at various levels of the other treatment.

12–42

EXHIBIT 12.11 A 2 × 2 Factorial Design That Illustrates the Effects of Sex and Ad Content on Believability

12–43

EXHIBIT 12.12 Graphic Illustration of Interaction between Gender and Advertising Copy

12–44

Documents

Experimental Design (1)