Upload
fei-chris-shen
View
216
Download
0
Embed Size (px)
Citation preview
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 1/47
Experimentation and Consumer Research
Fei Shen
COM 5405 Consumer Behavioral
Insight
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 2/47
Experiment
WHAT comes to your
mind?
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 3/47
These?
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 4/47
1. Let·s take a look at two experiments
y Stanely Milgram Experiment -1963
y Obedience Experiment at Yale University
y Learner,Teacher, Experimenter
y Word pairs learning: cat ² redy Increasing in 15-volt increments
for each wrong answer
y Learner ² actor/confederate
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 5/47
Milg ram Experiment Result
y All of the poll respondents believed that only a very small
fraction of teachers (the range was from zero to 3 out of 100,
with an average of 1.2) would be prepared to inflict the
maximum voltage.
y In Milgram's first set of experiments, 65 percent (26 of 40)
of experiment participants administered the experiment's
final massive 450-volt shock, though many were very
uncomfortable doing so.
y When authority is not present, less than 3 percent of the
participants administered the massive volt shock.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 6/47
Not only for basic research
y Wide business application as well
y Almquist andWyner (2001). Boost your marketing ROI with
experimental design.Harvard Business Review.
y
Guess work -> accurate campaign and marketingy Maximize ROI (return on investment)
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 7/47
The economist subscription study
y DanAriely ² Duke Univ.
y Trying to subscribe the Economist online
y Curiosity arose
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 8/47
Pricing experiment
Product Price
Web subscription $ 59
Print subscription $ 125
Print & web subscription $ 125
Product PriceWeb subscription $ 59
Print & web subscription $ 125
FirstDay
SecondDay
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 9/47
Pricing experiment
Condition 1
Product Price Subscriber distribution
Web subscription $ 59 68%
Print & web subscription $ 125 32%
Condition 2
Product Price Subscriber distribution
Web subscription $ 59 16%
Print subscription $ 125 0%
Print & web subscription $ 125 84%
Let·s suppose 100 people subscribe everyday.Then adopting the second pricing strategy
will help you make $3,432 more everyday.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 10/47
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 11/47
2. The essence of experimentation
y To manipulate some stimuli, applying them to your
research participants
y To establish causal relations:
y Cause and effect covary (suppose we have it)y Cause proceeds effects (absolutely so in an experiment)
y Exclude alternative explanation (that·s the main task)
y Make sure the whole process won·t be ´contaminated by
other factorsµ, meaning that the effect is really ´causedµby your stimuli.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 12/47
X Y X Y
Z
X Y
Correlation
Spurious
correlation
(e.g., ice-
creamconsumption
and drowning)
Causality
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 13/47
ovariation � Causation
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 14/47
Graphic explanation
Water with water Water with milk
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 15/47
Key terms def ined
y anipulation: assigning different stimuli to research
participants
y Control: a researcher tries systematically to rule out
variables that are possible ¶causes· of the effects he/she isstudying other than the variables that he/she has
hypothesized to be the ¶cause·. (Kerlinger, 1974)
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 16/47
Key terms def ined
y Manipulated variable: when researchers control
participants· exposure to a variable, that variable is called
manipulated (controlled) variable ² always the independent
variables.
y Ways of manipulation:
y Presence vs. absence (Watch advertisement vs. watch nothing)
y gradation on a scale (Watch 10 advertisements, watch 9, 8,
7«.1,0 advertisements)
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 17/47
Experimental Desig ns
y A category of designs focused on the testing of cause-effect
hypotheses through the direct manipulation of one variable (the
independent variable) and random assignment of people to levels
of that manipulation prior to the measurement of another variable
(the dependent variable).
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 18/47
Single g roup post-test only desig n
XX OO
Intervention Time Observation
The design has little scientific merit. There is no way of comparing
measurement to any relevant baseline, so it is impossible to know
what, if anything, affected the measurements.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 19/47
Single g roup pre-test post-test desig n
OO11 XX OO22
This design is better, but it has problems. Threats to
validity include testing, history.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 20/47
Post-test only nonequivalent control
g roup desig n
XX OO11
==============OO22
This design still has problems
Testing and history are no longer a threat.
The major threat to internal validity is selection.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 21/47
Extraneous influences/Noises
y Sensitization:The effects of one measurement on a
subsequent measurement.
y Testing effect is a sub-category.
y H
istory: changes in the external environment that impingeover the course of an experiment
y Selection effect
y Attrition
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 22/47
RANDOMASSIGNMENTRANDOMASSIGNMENT
True Experimental
Research Desig ns
The distinguishing feature of a
true experimental design:
RANDOMASSIGNMENTRANDOMASSIGNMENT
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 23/47
Random Assig nment
y Random assignment to experiment is like random sampling
to survey research
y Random assignment
y
Recruiting research participantsy Randomly assignment them into multiple groups, depending on
how many treatment/control groups you have
y Ways of random assignment: simple random sampling, by
birthday,
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 24/47
Simple random sampling
y Number all your participants: 1 2 3 4 5 6 7 8 9 10
y Generate random numbers within the range from 1-10
y Half being assigned to groupA, the other half being assigned
to group B
y The consequences of R.A.
y Ruling out initial differences.
y Guarantee that people in different groups are about the same inevery aspects that are related to the study
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 25/47
Post-test only control g roup desig n
RR XX OO11
==============RR OO22
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 26/47
Pre-test Post-test control g roup desig n
RR OO11 XX OO22
==================RR OO
33 OO
44
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 27/47
To really learn, quit studying and take a
test
y http://www.nytimes.com/pages/science/index.html
y New YorkTimes, Jan 21, 2011
y
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 28/47
Desig n with multiple f actors
y AdvertisementA vs. B
y PricingA vs. B
y Factorial design:An experiment using factorial design allows
one to examine simultaneously the effects of multipleindependent variables and their degree of interaction.
Ad A Ad B
PricingA Group 1 Group 2
Pricing B Group 3 Group 4
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 29/47
Can random assig nment take care of
all the errors?
y Random assignment allows us to rule out pre-existing differences
between people as the explanation for differences between
groups
yBut just as important is minimizing other differences betweenthe levels of the manipulation other than the intended
manipulation
3. External Validity
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 30/47
Internal Validity
y The extent to which competing explanations for
experimental results can be avoided
y Variation in the dependent variable was caused by exposure
to the treatment conditions and not by other causal factors
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 31/47
External Validity
y Population Validity (Representativeness)
Are the participants in the study representative of a broader population
of interest?y Ecological Validity (Realness)
Is the experimental context representative of the ´real-worldµ context
of interest?
External validity pertains to the ´generalizabilityµ of the research
findings. In the case of laboratory-based experimentation, concern is on
generalizability to the ´real worldµ of interest
Twobroad dimensions of external validity
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 32/47
Population Validity
y Study how health information campaign influence the elderly
making decision about medical product
y You need to make sure you are studying the right people
y
Young people might be influenced by a campaign, but thisdoesn·t lead to a conclusion about the elderly
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 33/47
Ecolog ical Validity
y You want to study which advertisements people likemost
y You invite students to a lab, showing them differentversions of advertisements
y Even if you find difference, you cannot make a solidconclusion. Because people usually do not watchadvertisements with high attention in real life.
y Failed to fulfill mundane reality and experimental reality.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 34/47
The Asch Paradig m
A B C
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 35/47
Evaluating Ecolog ical Validity
y Mundane RealismThe similarity of the experimental stimuliand/or situation to ´real worldµstimuli/situations of interest.
y Experimental RealismThe degree to which the stimuli orsituations in the study are impactful,
interesting, and involving, to theparticipants.And therefore theparticipants might forget they are in anexperiment.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 36/47
The classic case: Pepsi vs. Coke
y Misuse of experiment: Low ecological validity
y ´When we put the new formula in a Coke can and compared
it side by side with the old formula Coke, the new product
was chosen by consumers 61% to 39%.µ
y "We failed to tell the tasters graphically enough that their
preference for the new would mean that they would never be
able to taste the original Coke again," explained Ira Herbert,
vice president for worldwide marketing.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 37/47
Did you notice this?
y Viewpoints from your textbook:
y Why experiment is not conducted often?
y High cost ² the costs of testing three alternative advertisingcampaigns in three different geographic areas
y Security issues ² exposing a marketing plan to competitorsy Implementation ² no one wants to run the risk
y Viewpoint from the HBR article
y ´Now, however, marketers have easier access, at relatively lowcost, to experimental design techniques long applied in otherfields«µ (paragraph 2, sentence 1)
y WHY?
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 38/47
Lab experiment and f ield experiment
y Lab ² experiment conducted in a controlled setting
y Focus on individual consumers, and then make inference and
prediction about collective consumer
y Higher internal validity
y Lower external validity
y Field ² experiment conducted outside the lab in an actual
market environment
y Focus on collective consumer
y Lower internal validity
y Higher external validity
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 39/47
Alternative approach ² low cost
y Simulated test market (STM)
y No actual testing in the market place
y Expose a sample of individuals, representative of the target
group, to various stimuli and ask them to make purchase
choices between them.
y The results are used to input to mathematical models and make
estimations.
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 40/47
Data Extrapolation for f actorial
desig n ² low cost
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 41/47
Pause for a while
THINK about your own experiment
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 42/47
Our little g ame f rom last week y I want to know whether ´anchor effectµ is true or not
y In other words, whether exposure to your own phone number willinfluence the max amount you are willing to pay for a product
y Now, activity time, let·s first discuss in groups, answer the followingquestionsy
1. what is the manipulation in the experiment?y 2. is there any control group in the experiment?y 3. what is the dependent variable?y 4. what kind of design this experiment use?y 5. is there any history effect?y 6. do you think the manipulation is random?y 7. comment on the quality of this experiment in terms of its external
validity including ecological validity and population validity, and internalvalidity.
y 8.How would you improve the experiment?
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 43/47
A f irst look at some results
y Data cleaning ² deleted extreme cases (3 cases) who will pay
more than 10,000 for a product
y 81 cases (afternoon class + evening class) ² to enhance
statistical power
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 44/47
Rational consumer?
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 45/47
Who? - Just for f un ² NO scientif ic
base
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 46/47
Rational consumer? ² percentag e of
willing to pay at the price of your cell
phone number
8/7/2019 COM 5405 LT2 - experimentation v
http://slidepdf.com/reader/full/com-5405-lt2-experimentation-v 47/47
Anchoring Eff ect? ² averag e amount of
money willing to pay by diff erent g roups