Embed Size (px)
Citation preview
21112005
Kyrre Rickertsen
Based on Book Chapter with Frode Alfnes
Workshop in National Chung Cheng University in Chia-Yi, Taiwan
March 8, 2011
Experimental Methods in Nonmarket Valuation
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Sch
ool o
f Eco
nom
ics and B
usin
ess
Outline Presentation
Introduction
Incentive compatible (IC) valuation mechanisms
Validity of bid and choices
– Laboratory versus field experiments
– Internal validity (design and other issues)
– External validity – validity of results outside the laboratory
Some empirical results
Recommendations
2
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Introduction 1
When (high-quality) market data are available, they are usually preferred to experimental data
Market data are frequently unavailable
– New product or product with new characteristics
• In peak year 1995, 16,900 food and beverage products introduced in the US (Nestle 2002)
• Controversial technologies used in production
– Growth hormones, GMO, irradiation, and cloning
– New labeling or information
• Nutritional, organic, and sustainable labeling
– Maybe little variability in market data (e.g., prices)
– Effects of socioeconomic variables?
Sch
ool o
f Eco
nom
ics and B
usin
ess
3
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Introduction 2
Alternatives are:
– Stated preference data
• Can ask about nonexistent products BUT
– Lack economic incentives
– Hypothetical bias (e.g., List and Gallet 2001)
– Experimental data (hypothetical market data)
Experiments also used to develop economic theory, for example:
– WTP – WTA disparity (e.g., Shogren et al. 1994 - AER)
– Preference reversals (e.g., List 2002 - AER)
– Coherent arbitrariness (Ariely et al. 2003 – QJE)
Sch
ool o
f Eco
nom
ics and B
usin
ess
4
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Introduction 3
In experimental valuation studies, participants make bids or choices with real products and real money
Experiments used for a long time in marketing:
– As a method of research, controlled experimentation is universally used among scientists. They conduct experiments on a small scale to discover facts from which overall conclusions may be reached. In the gradual evolution of marketing research from an art to a science, some practitioners have endeavored to employ experimental methods where possible. William Applebaum and Richard F. Spears (1950: 505)
– The first scientific article describing a test marketing experiment is Ginzberg (1936) in AER.
– Test marketing was the preferred method
• New products or marketing strategies tested in a few stores
Sch
ool o
f Eco
nom
ics and B
usin
ess
5
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Introduction 4
Applied economists started to use experimental methods around 1990 to study the value of quality attributes of foods (e.g., Menkhaus et al. 1992 – JARE; Buhr et al. 1993 – JARE)
– Little reference to previous marketing literature
– Focus on WTP while previous marketing literature focused on market shares
– This line of research is in the focus of this presentation
Sch
ool o
f Eco
nom
ics and B
usin
ess
6
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 1
A mechanism is incentive compatible (IC) given that the utility of the participant cannot be increased by submitting a different bid or making a different choice
IC mechanisms include (complete listing see table 1):
– Sealed bid auctions
• 2nd price, nth price, random nth price
– Becker-DeGroot-Marschak (BDM) mechanism
– Price list experiments
– Real choice (RC) experiments
Non-IC mechanisms include:
– Dutch and first-price auction
– The price paid is not independent of the winner’s bid
Sch
ool o
f Eco
nom
ics and B
usin
ess
7
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 2Sealed-Bid Auctions with Endogenous Market Prices
2nd price (Vickrey 1961)
– Participants submit bids and one unit of the good is sold to the highest bidder for a price equal to second-highest bid
– Two problems
• May not engage low-value bidders (especially in a multiple trial setting with posted prices)
• Some may see auction as competition with one winner
A random nth price auction where n-1 units of the good is sold for the nth highest bid
– Engage high- and low-value bidders
– n -1 winners reduce any auction winning utilities
– Lottery effect?
Sch
ool o
f Eco
nom
ics and B
usin
ess
8
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 3Sealed-Bid Auctions with Endogenous Market Prices
In the endowment approach, the participants are endowed with a product and bid for an upgrade (Hayes et al. 1995)
– The bid reflects the difference in value between the two alternatives
– The bid (and thereby the value difference) may be affected by all sorts of anchors
In full bidding approach, the participants bid on two (or more) products and one is randomly chosen for sale (Hoffmann et al. 1993)
– The difference in bids reflects the difference in value between the two alternatives
– Bid differences less affected by anchors
See Lusk and Shogren (2007: 95-112) and Alfnes (2009) for detailed discussions
Sch
ool o
f Eco
nom
ics and B
usin
ess
9
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 4Becker-DeGroot-Marschak (BDM) Mechanism
No auction but strategically equivalent to IC auctions
Each participant submits a sealed bid
Sales price is randomly drawn from a distribution from zero to a price that is higher than the anticipated max bid
If bid higher than price, then the participant purchases one unit for the drawn price
Can be conducted with only one participant
– Easy to do at a grocery store
Lottery element?
Sch
ool o
f Eco
nom
ics and B
usin
ess
10
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 5Price List Experiments
a) Real dichotomous choice
– Offer the participant to buy a commodity for a price. If yes, sale is completed
b) Multiple price list (payment card method)
– A list of questions on the form
• “At a price of $8.75 I will buy _____ I will not buy ____”
• One row is implemented as the final sale
c) Open-ended choice experiment
– How many units do you want to purchase at different prices?
– Can calculate demand curves
– But people tend to stock up on storable products at low prices and the interpretation of the demand curve becomes difficult
Sch
ool o
f Eco
nom
ics and B
usin
ess
11
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 6Real Choice Experiments
Departm
ent o
f Eco
nom
ics and R
eso
urce
M
anagem
ent
12
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 7Real Choice Experiments
Extension of stated choice experiment
– But your choices have real economic consequences
– Need to have the product
Participants make choices in a series of scenarios
Product attributes (including prices) vary between scenarios
One scenario randomly drawn as binding
Advantage: The choice task is similar to choices made in a food store
Weakness: Individual WTP is not observable
– WTP has to be estimated based on choices of all participants
– Estimated WTP for each participants affected by the choices of other participants
– Estimated WTP for each participants affected by model specification
Sch
ool o
f Eco
nom
ics and B
usin
ess
13
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 8Comparisons of Mechanisms
In theory, sealed bid auctions and BDM should result in identical bids and bid differences
– In practice, not always the case
Induced value experiments
– Sealed-bid auctions produce more accurate bids than BDM (Noussair et al. 2004; Lusk and Rousu 2006)
Homegrown value experiments
– Impossible to test whether bids equal homegrown values (private and unknown to experimenter)
– Many comparisons with mixed results (see paper)
Sch
ool o
f Eco
nom
ics and B
usin
ess
14
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
IC Valuation Mechanisms 9Choice of Mechanism
Choice of mechanism a trade-off
– Auctions perform better than BDM in induced value experiments
– 2nd price auction perform well for high-value bidders while random nth price performs better for low-value bidders
• Recently most studies use n > 2
– BDM easy to implement on individual basis and in the field
– RC experiments resembles choices in grocery stores but do not give individual WTP values
Sch
ool o
f Eco
nom
ics and B
usin
ess
15
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesLaboratory versus Field Experiments
In marketing literature
– Laboratory experiments are conducted in a laboratory setting with a high degree of control of external factors
– Field experiments are conducted in an actual market place with less control but a familiar context for the specific choice
Harrison and List (2004) have a refined categorization depending on
– Students versus general population
– Field goods versus induced values
– Field context versus laboratory
– Subjects know or do not know that they participate in an experiment
Sch
ool o
f Eco
nom
ics and B
usin
ess
16
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesInternal Validity 1
Internal validity:
– The ability to demonstrate that observed correlations are causal (Roe and Just 2009)
– Does the experiment control for alternative explanations?
Some factors affecting the internal validity
– Training is important (Plott and Zeiler 2005)
• Most IC mechanisms are unfamiliar and small differences may affect the bids as discussed above (2nd vs. nth price)
– Presence of field substitutes (Harrison et al. 2004)
• Bids are truncated by prices of field substitutes
• But is cod the only field substitute to cod?
– Or is fish in general or maybe meat the field substitute?
• Bid differences may be more robust than bid levels
Sch
ool o
f Eco
nom
ics and B
usin
ess
17
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesInternal Validity 2
Bid affiliation and posted prices in multiple-round auctions
In induced value experiments, it has been shown that it takes several rounds to reach the theoretical WTP
What about homegrown value experiments? Increasing prices typically observed
– We have bid affiliation when bids in later rounds are affected by posted prices in earlier rounds
• Some participants may try to buy at low prices and discover that it does not work, i.e., “market discovery”
• Some participants may take high posted prices as a signal of quality and revise their bids, i.e., “preference learning”
• Some participants may consider the auction as one competition, i.e., “auction winning utilities”
Sch
ool o
f Eco
nom
ics and B
usin
ess
18
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesInternal Validity 3
No easy solution to bid affiliation
Problem is the interpretation of the bids (and the internal validity)
One-shot auctions with training with a different product is recommended by Harrison et al. (2004)
– Do we find the “correct” WTP for unfamiliar food products in the first shot?
– Does it solve the problem to use a different training product?
• Anchors
• Incidental prices
• Coherent arbitrariness
Recent papers seem to use few but more than one round
Sch
ool o
f Eco
nom
ics and B
usin
ess
19
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesInternal Validity 4
Participants use numbers that are presented to them as anchors
Examples on anchors
– Reference price of field substitute
– Distribution of prices in price list experiments
– The limits of the price distribution in BDM
– Bids in training sessions
– Posted prices
Sch
ool o
f Eco
nom
ics and B
usin
ess
20
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesInternal Validity 5
Arbitrary numbers can also serve as anchors
– Nunes and Boatwright (2004) found that exposing the participants to prices that were completely unrelated to the goods in an auction (incidental prices) had substantial effects on the bids (e.g., sweatshirt price affected the bids for CDs)
– Ariely et al. (2003) found that absolute bids on ordinary products (like computer keyboards) were affected by random numbers such as the participants’ social security numbers
• Participants were asked to write down their social security number before bidding. High numbers resulted in high bids in the following auction (arbitrariness).
• The relative valuations were not affected (coherent)
Given random anchors, we recommended to use the full bidding approach and relative WTP (or alternatively bid differences when many zero bids)
Sch
ool o
f Eco
nom
ics and B
usin
ess
21
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesExternal Validity 1
External validity
– The ability to generalize the relationships found in a study to other contexts (Roe and Just 2009). For example, from lab to market
– Mixed results; see paper
Several factors may reduce the external validity
Unfamiliar environment:
– People may behave different in a laboratory than a retail setting (Levitt and List 2007)
– People are not in laboratory because they need to buy food
– The time and day of the experiment are decided by researcher
• Corrigan and Rousu (2008) found that people that intended to buy bananas on the day of the experiment bid more, i.e., regular buyers may behave as non-buyers in an experiment
– Experiments in a store setting may be preferred but lack of control and limited possibilities to train participants
Sch
ool o
f Eco
nom
ics and B
usin
ess
22
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesExternal Validity 2
Nature of decision task different in store and experiment
– Only a small number of products
– A lot of focus on the (few) products
– Can typically only buy one unit of one product
– May have to bid for products instead of choosing among them
– Random draws may resemble lotteries
Representative samples
– Self-selected students may behave differently than representative consumers
• Low age, low income, more education, strong beliefs
– Nonstudent samples may be unrepresentative outside sample area
– Users may have different preferences than the general population
• For many food products, 10% of users buy almost everything
Sch
ool o
f Eco
nom
ics and B
usin
ess
23
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesExternal Validity 3
Information
– In experiments there is usually a focus on a few products and, consequently, few prices. Information is typically provided (e.g., PowerPoint etc).
• Participants encouraged to make “rational” decisions using the information
• Small samples encourage the use of within- rather than between-sample tests
– Within-sample tests draw attention to the treatment and gives larger effects than between-sample tests (Johansson-Stenman and Svedsäter 2008)
– In food store many products, less emphasis on information about the product and many prices
• Many purchases based on habits
Sch
ool o
f Eco
nom
ics and B
usin
ess
24
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Validity of Bid and ChoicesExternal Validity 4
Presence of researchers
– People may change behavior when they are scrutinized
• Behave like they believe the experimenter want
• Give a socially desirable impression
– Especially for product attributes with dimensions such as animal friendly, “fair trade”, GMO, etc
Repeated purchase
– People may bid high because they want to try the product (preference learning)
– A high premium in the laboratory does not necessarily imply repeated purchase of the good
Sch
ool o
f Eco
nom
ics and B
usin
ess
25
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Examples on Empirical Results
Food marketing issues: Packing technologies, use of insecticides, use of different types of feed, grading for tenderness, labels etc
– Segments of consumers with different preferences
– Typically some increased mean WTP for new product
• Do not investigate the profitability of the new product
Controversial technologies: GMO, hormone treatment, irradiation
– Segments of consumers with different preferences
– US participants more positive than European
Effects of information
– Moderate effects of scientifically balanced information
– Stronger effects of information from environmental groups, etc
– Effects of negative information dominate effects of positive information
Sch
ool o
f Eco
nom
ics and B
usin
ess
26
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Our Recommendations 1No Universal Agreement
Do not deceive participants or lie to them
– You do not need to tell participants everything but what you tell them should be true
Use representative consumers
– Students differ from other consumers
• Likely to have other preferences concerning sensory quality of foods, attitudes towards animal welfare or controversial technologies
• Different sensitivity for prices
– Student sample are OK for testing designs and theoretical studies
– No purpose in including vegetarians in a study of beef attributes
• Consumption constraint
Sch
ool o
f Eco
nom
ics and B
usin
ess
27
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Our Recommendations 2
Make sure the participants understand the mechanism
– Many procedures are unfamiliar and have lottery elements
• Inform and train participants by using the same mechanism and number of goods as in the experiment
Calculate relative WTP values
– According to micro theory relative prices are important
– Systematic anchoring effects will cancel out
– WTP differences can be used with many zero bids
Use a context as free of scrutiny as possible
– Let participants feel relaxed and not scrutinized. Fill in any forms in a room allowing for some privacy and let the participant identify themselves only through numbers that are unknown for the other participants
Sch
ool o
f Eco
nom
ics and B
usin
ess
28
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Our Recommendations 3
Use mechanisms with strong real economic incentives
– If you have 15 participants there are stronger incentives in a median price than a second price auction
Collect background information
– Socioeconomic, attitudinal, and knowledge variables can be used to define segments
– Segments of interest for policy makers and industry
Let participants taste unfamiliar products
– Tasting reduces sensory uncertainty, which potentially either can reduce bids (due to risk aversion) or increase bids (participant wants to taste a new product)
– Tasting results in increased external validity
Sch
ool o
f Eco
nom
ics and B
usin
ess
29
NO
RW
EG
IAN
UN
IVER
SIT
Y O
F LIF
E S
CIE
NC
ES
www.umb.no
Our Recommendations 4
Treat all products equally
– Rotate order of presentation between sessions
– Taste all or none of the products
– Use full bidding format in experimental auctions
Delete participants with a nonresponse to all alternatives, i.e., always bid zero or choose “none of the alternatives”
– May reflect that participant does not like any of products
– May reflect inconvenience, lack of trust in the experiment, or pure laziness
Let participants evaluate multiple items of heterogeneous products
– To avoid deception by letting the participants bid on as many items as there are participants and, next, let each of the participants draw one item as their binding product
Sch
ool o
f Eco
nom
ics and B
usin
ess
30
21112005
Thank You for Your Attention!
