View
220
Download
0
Category
Preview:
Citation preview
8/12/2019 Experimental Tests of Rationality
1/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
..............................................................................................................
EXPERIMENTALT E S T S O F
RATIONALITY..............................................................................................................
. I..........................................................................................................................................
R choice theory is an instrumental theory. It assumes that agents have a
set of basic preferences and values which they undertake to satisfy, and it then
specifies the optimal way to achieve those values. Rational choice theory is some-
times proposed as a purely normative theory, with no bearing on the descriptive
question of what people actually do. It is also widely used as a working hypoth-
esis about real human behavior, with many taking the view that behavior, when
properly understood, follows the tenets of the theory, and an equally large number
taking the opposite view. The majority of experimental studies in the fields of
behavioral economics and judgment and decision-making are either explicit tests
of rational choice theory or developments of theories that started out that way.The focus of this chapter is on the success of rational choice theory as a behavioral
hypothesis.
The chapter is organized as follows. Section . is an informal account of the
conditions which preferences must meet if they are to be judged rational. This
I am grateful to Paul Anand, Ken Binmore, and Jon Leland for useful discussion and comments, and
especially to Stephen Humphrey for a very useful review. Special gratitude is also due to John Conliskfor providing me with his unpublished data.
8/12/2019 Experimental Tests of Rationality
2/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
Table 8.1.Decision table showing states, options, and consequences
Option State of the world
S1 S2 . . . Sm
O1 C11 C21 . . . Cm1O2 C12 C22 . . . Cm2. . . . . . . . . . . . . . .
On C1n C2n . . . Cmn
section draws primarily on the influential work of Savage (), who will through-
out act as our guide to what is rational. Sections .. consider some empiri-cal challenges to the view that behavior conforms to these rationality conditions.
The challenges involve showing that the preference ordering over options depends
significantly on differences between choice circumstances that, according to the
rationality conditions, are normatively irrelevant. In Section . I will return to
Savages own discussion of the descriptive applicability of his conditions.
. I C
RC ..........................................................................................................................................
In this section I provide a sketch of rational choice theory that will form the
backdrop to the subsequent discussion. I adopt a highly simplified version of the
framework used by Savage () in his Foundations of Statistics. Savages views
are of particular interest to empirical researchers because he was concerned both
with providing a normative system, and with the problem of whether this system
was descriptively accurate. On the latter point he was more optimistic than many
subsequent researchers.
In Savages framework the decision-maker is seen as choosing between a range
of options, which have consequences that are uncertain because they depend onunknown states of the world. The general situation can be described with the aid
of Table.. We can illustrate this with the decision of whether to take an umbrella
to the office on a day that threatens rain (Table .). Stated this way the options
are {Take umbrella; Dont take umbrella}, the states are {Rain; No rain}, and the
consequences are the anticipated outcome from choosing each option, conditional
1 Anormative theory concerns what a rational agent shoulddo, and a descriptive theory what heactuallydoes.
8/12/2019 Experimental Tests of Rationality
3/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
Table 8.2.Decision of whether to take an umbrella
Option State of the world
Rain No rain
Take umbrella Dry DryDont take umbrella Wet Dry
on whether or not it rains. In the table the consequences are simply Dry or Wet.
Both options and states are mutually exclusive and exhaustive, meaning that no
other options and states are possible, although it is always possible to describe them
at different levels of detail. The option Dont take umbrella, for example, can bebroken down into Take a raincoat but no umbrella, and Take neither a raincoat
nor an umbrella.
Rational choice theory specifies certain minimal conditions whichshouldbe met
for the choosing agent to be rational. There are two kinds of justification for main-
taining that these are conditions of rationality. First, the rationality conditions are
such that people will, on reflection,wantto conform to them. Although someone
might occasionally express preferences contrary to the conditions, when the contra-
diction is pointed out to her, she will want to change the expressed preferences and
bring them in line. Or, as Robert Strotz put it, it would be a strange man indeed
who would persist in violating these precepts once he understood clearly in what
way he was violating them (, p.).Second, they are necessary for preferences to be consistent. Someone who does
not comply with the conditions, it is argued, runs the risk of having a Dutch
Book taken against them, or being turned into a money pump. To illustrate a
money pump, imagine someone who prefers an orange to a lemon, a lemon to
an apple, and an apple to an orange. This is, as will be indicated in the next
paragraph, an intransitive preference. Someone with this preference (and, it must
be acknowledged, no memory or common sense) might be induced to give a lemon
plus a small amount of money for an orange, to give an orange plus a small
amount for an apple, to give an apple plus a small amount for a lemon, and so on,
forever.
The rationality conditions are often presented as axioms or postulates. In thischapter they are presented informally, with many technical details being dropped.
Hence I use the term condition. The first two conditions are:
2 It should be emphasized that Savages view (and the closely related one of von Neumann andMorgenstern ) are mainstream accounts of rational choice, and alternatives are available which
drop or replace the rationality conditions described here. Whether these alternatives are indeed theo-ries of rational choice is a matter for debate (see Sugden).
8/12/2019 Experimental Tests of Rationality
4/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
Completeness: For any pair of options O1and O2, either O1is preferred to O2, O2is preferred to O1, or the decision-maker is indifferent between them.
Transitivity: For any triple of options, O1, O2, and O3, if O1is preferred to O2and
O2is preferred to O3, then O1is preferred to O3.
Combined, these produce apreference orderingover options, so that all options
can be rank-ordered and equivalence classes can be formed, which are sets of options
between which the decision-maker is indifferent. Consequently, these condi-
tions are all that is necessary for what are called riskless choices, meaning choices
between options when only one state can actually occur (Marschak). Such a
riskless choice is made when you must decide whether to take an umbrella given
that it is already raining.
When risk and uncertainty enter the picturemeaning that the decision prob-lem requires at least two states to be fully specifiedfurther rationality conditions
are required. I will describe two, which are enough to get our discussion started.
The first is:
Independence: Given a pair of options O1and O2that have the same consequences
under some states of the world, then these common consequences will not influence
preference between the options. (This is also called the sure-thing principle.)
This principle appears innocuous. To illustrate with the umbrella example above,
it merely says that because the consequences for you are the same regardless of what
you choose if it doesnt rain tomorrow, these consequences should not influence
your ultimate decision. Some readers may be tempted to object that the conse-quences described in the umbrella example are incompleteit is annoying to carry
an umbrella, so we should fill in the cells with consequences like Dry, but carrying
umbrellabut for the moment we will assume that the consequences given are
correct. The readers hypothetical objection will soon be given its voice.
Closely related to the independence condition is dominance:
Dominance: If under at least one state of the world the consequence of O1 is
preferred to that of O2, and if under no state of the world is the consequence of
O2preferred to that of O1, then O1is preferred to O2.
One option dominates the other if, no matter what occurs, it leads to a better
outcome. In the umbrella example, taking an umbrella dominates not taking one,
3 Bothconditions have been challenged as necessary foundations for rationality. Briefly, why should
an agent have a preference between all options, even those they have never encountered before orwill never have to choose between? (e.g. Sugden; Binmore). Likewise, cannot an agent havechoice-set-dependent values that make the consequences of (say) getting a lemon from the set {lemon,orange} be different from those of getting a lemon from {lemon, apple}? (e.g. Anand, Ch. above;
Sugden). The discussion of regret theory below hints at this issue, which goes well beyond thescope of this chapter.
8/12/2019 Experimental Tests of Rationality
5/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
because if it rains you stay dry if you take an umbrella, but if it doesnt rain it makes
no difference.
This dominance condition is sometimes called state dominance, which can be
distinguished from stochastic dominance. We can illustrate the difference with a
simple gamble based on the toss of a coin. Imagine I toss a singlefair coin and give
you a prize based on the outcome. You have two options, which pay offas follows:
Heads Tails
O1 $ 0
O2 $ 0
If you choose O1 I will pay $if heads comes up and nothing if tails comes up,
while if you choose O2I will pay $if heads comes up and nothing if tails comes
up.Statedominance dictates that you should choose O1because it might make youbetter offthan O2 and will certainly not make you worse off. Now imagine that I
offer you a similar gamble, but this time I toss two fair coins (A and B), and O1and
O2represent the choice of coin. Now the decision table is as follows:
A: Heads A: Heads A: Tails A: Tails
B: Heads B: Tails B: Heads B: Tails
O1(Coin A) 0 0
O2(Coin B) 0 0
In probability terms, O1 dominates O2 (a percent chance of $, versus a
percent chance of $), but it is nonetheless possible for the O2coin to come up
heads while the O1coin comes up tails.Stochasticdominance refers to dominancein these probability terms. More formally, if O1 can be transformed from O2 by
iteratively increasing the probability of a preferred consequence over a less preferred
one, or by improving a consequence while holding its probability constant, then O1stochastically dominates O2.
In the next four sections I describe experimental findings that appear to demon-
strate how changing the circumstances of choice can make preferences vary in ways
that systematically depart from the rationality conditions. The effect of changing
circumstances can be considered a kind of spotlight effect, analogous to the con-
cept of attentional spotlight from cognitive psychology (e.g. Cave and Bichot ).
The fundamental idea is one that is more or less the default among psychologists
who study choice behavior. It starts with the notion that people, and indeed allorganisms, have a severe limitation in their capacity to process information. More-
over, and indeed as a consequence of this limitation, they do not have well-defined
preferences over all options. Rather, when making a choice, the valuation is done at
the moment, based on information recruited by the specific choice circumstances.
These circumstances direct attention to certain problem featuresfeatures that
can be described using the terminology of states, options, and consequences
and deflect attention from others. The circumstances also act as memory cues and
8/12/2019 Experimental Tests of Rationality
6/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
triggers for the construction of scenarios. Our preferences are a function, always
partly and sometimes decisively, of what the spotlight reveals.
This spotlight effect, under a variety of names and descriptions, is the central
principle underlying the experimental investigation of preference. We can even
summarize the core research strategy of behavioral economics and behavioral
decision-making as having three steps. The researcher
. Infers, based on introspection or other evidence, that certain option charac-
teristics will influence preference in a specific way.
. Demonstrates logically how the influence from () can appear contrary to one
or more of the rationality conditions.
. Develops a method for shining a spotlight on those characteristics from () by
varying some element of the decision context.To give an example that will reappear below, a researcher might notice that
he is willing to pay $ for a high-priced car stereo when it is an option on a
new car but not when it is an after-market purchase. From this he hypothesizes a
general tendency to evaluate the magnitude of costs relative to irrelevant back-
ground costs (step ); he then shows that this tendency can lead to preferences
inconsistent with the incompleteness axiom (step ), and then develops a method
for showing this in the laboratory by asking people to decide how much e ffort
they would exert to save $X when their attention is drawn to a large or small
background cost (step ). The next sections discuss many results from applying
this research strategy, with each sections having a heading indicating how step is
accomplished.
. C C , W I
RC ..........................................................................................................................................
In many demonstrations of apparently irrational preference reversals, agents areplaced in two situations in which options appear to differ objectively, but the
rationality conditions do not admit the relevance of these differences. Among
these are two of the most famous, and historically the earliest, examples of
4 Variations of this view are found in, among many other places, Payne, Bettman, and Johnson
(); Read, Loewenstein, and Rabin (); Simon (); Trope and Liberman (); and Tverskyand Kahneman ().
5 The reason for using the term appear becomes evident in Sect. ..
8/12/2019 Experimental Tests of Rationality
7/26
8/12/2019 Experimental Tests of Rationality
8/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
Table 8.4.Allais questions reformulated as a lotteryover lotteries
Option State of the world
1% 10% 89%
O1 Lottery I Lottery I 1 millionO2 Lottery II Lottery II 1 million
O1 Lottery I Lottery I 0O2 Lottery II Lottery II 0
This effect of problem description on the Allais task is profound. Conlisk ()showed that the paradox practically goes away if the problem is described as a
lottery over lotteries. He first informed respondents of two available lotteries:
Lottery I: Certainty ofmillion
Lottery II: /chance of nothing;/chance ofmillion
He then asked them to make the standard Allais choices, now described as a lottery
over these lotteries. These choices are depicted, in state form, in Table .. There
were no longer any systematicpreference reversals, and the same proportion of
people chose O1over O2as O
1over O
2 .
If the Allais paradox can be eliminated this way, it might seem that the prob-
lem has disappeared. Perhaps it arises only when we choose the wrong way toformulate the problem, and when we find the right wayone that does not bam-
boozle the respondent with irrelevant detail, or induce them to misrepresent the
problem in their mindtheir true preferences are revealed. Some variant on this
viewpoint has been proposed by many researchers, and the debate continues (e.g.
Binmore ). I will point out two objections to this defense of the rationality
conditions.
First, merely because preferences can change, and even become consistent
through choice redescription, does not mean that these now consistent preferences
are the agents true preferences, and that the inconsistent preferences were counter-
feit. Using the spotlight metaphor helps. Each form of the Allais question is a spot-
light that highlights some aspects of the question and produces corresponding pref-erences. Two spotlights can produce the same preferences because they highlight the
same aspects of the question, or because, while they highlight different aspects, they
both produce the same preference ordering. From a theoretical perspective, there
7 A third issue, deliberately avoided in this chapter, is that merely because choices are consistentdoes not mean that they are optimal from an instrumental point of view. An agent can consistently
choose options that are worse for him over those that are better (some discussion: Sugden ; Read).
8/12/2019 Experimental Tests of Rationality
9/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
is an inferential asymmetrywhile inconsistency in one context demonstrates the
operation of the spotlight, consistency in another context does not demonstrate
that it is not operating.
The second objection to the defense invokes a theme that will be present through-
out the remainder of this chapter. The fact that preferences vary as a function of
choice description might be a greater challenge to the rationality conditions than
any local violation of those conditions. If I simultaneously prefer O 1to O2, and O
2
to O1 it is bad enough, but if I also sometimes prefer O
1 to O
2 and/or O2 to O1I am doing more than just violating the independence condition, and I am prime
material for money pumps and Dutch Books. This point has led many researchers
to propose a further rationality condition that is perhaps more fundamental than
the original conditions themselves:
Invariance: Preferences do not depend upon irrelevantaspects of the context, of
the option descriptions, or of the procedures used to elicit those preferences.
An alternative way of putting this is that the rationality conditions are defined
over the consequences of options and the states under which they occur, and not
over how those options are described or how preferences are obtained. The key
word in the definition is irrelevant, and this word could bring an uncomfortably
subjective element to the rationality conditions. Since every preference reversal is
a consequence ofsome variation in the options, we must then decide whether a
difference is relevant or not.
To illustrate the problem, while most might agree that Conlisks two versions of
the Allais paradox should not have yielded different preferences, it is not obviousthat the differences between the two original choice pairs are irrelevant in this way.
Loomes and Sugden (; see also Bell) proposed that one way the two pairs
might differ is in the degree of regret the respondent anticipates if he does badly. In
the first pair, choosing O2 and ending up with nothing could produce great regret
because the agent will know with certainty that she would have had $ million if
she had chosen differently. To avoid this regret, she might choose O1. In the second
pair, however, choosing O2 and ending up with nothing does not lead to much
regret because you doubt that you would have got anything anyway. Perhaps this
is a good explanation of the Allais paradox, and it does rescue the rationality
conditions. But similar justifications may not be available in every case. It is a
major challenge to determine whether a given difference between options does ordoes not justify the resultant variation in preference. If invariance is not obviously
followed, and if we permit some at-first-glance irrelevant variations in the task to
be rationalized using a method like the one just described, then deciding whether
the rationality conditions are being met becomes a matter for judgement. We will
8 Anands discussion of intransitivity in Ch. above and elsewhere (Anand ) involves a closeexamination of what differences are relevant in the context of intransitive choice.
8/12/2019 Experimental Tests of Rationality
10/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
return, inevitably, to this question once we have seen more apparent violations of
the principle of invariance.
The Allais paradox is noteworthy because, for many people at least, the rational-
ity of their paradoxical choices is compelling even after reflection. It is difficult
to persuade people that the differences between the two choice tasks are in fact
irrelevant (Slovic and Tversky), a fact that will be well known to anyone who
uses them in teaching. The preference reversals discussed next are more like those
observed in comparisons between Conlisks different ways of describing the same
problem. The fact that there is a problem is obvious, and most people will feel that
something about the preference pattern needs to be explained.
. C M O E
..........................................................................................................................................
If preferences for options that we agree are exactly the samechange systematically in
response to apparently irrelevant changes in circumstances, then we have a prima
facie case for violations of invariance and decidability. Of the many ways to elicit
such preference changes, perhaps the most venerable and reliable is through chang-
ing the valuation procedure. For instance, we can ask people to choose between
pairs of options, to reject one option from a pair, to rank several options, to assign avalue to them, or to otherwise specify an option that is equivalent to another option.
It turns out that preferences are systematically influenced by all these procedures,
and others as well. Simply stating your preferences in one way versus another seems
to direct the spotlight on different reasons for having those preferences, and thus
change the preferences altogether.
The term preference reversal was coined by Lichtenstein and Slovic ()
to describe a systematic discrepancy between choice and pricing. They offered
respondents a choice between gambles, and later asked them to price each gamble
separately. One gamble in each pair offered a high probability of winning a small
prize (the so-called P-bet, e.g. apercent chance of winning $.and apercent
chance of losing $.), the other offered a moderate probability of winning a largerprize (the$-bet, e.g. a percent chance of winning $., a percent chance of
losing $.). There were many preference reversals, with the P-bet being chosen
more often than the$-bet, but the$-bet being priced higher. This was a very strong
effect: in their first experiment, percent of respondents always put a lower price
on a P-bet they had chosen than on a $-bet they had not chosen. Lichtenstein
and Slovic () replicated this with gamblers in Las Vegas, and, in a later pa-
per, Grether and Plott () imposed all the controls available to experimental
8/12/2019 Experimental Tests of Rationality
11/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
economics (such as greater task transparency and the use of real monetary incen-
tives) and replicated Lichtenstein and Slovics results almost exactly.
An appealing explanation for preference reversals is that they arise from what
Tversky, Sattath, and Slovic () called compatibility effects. When values are
expressed on a quantitative dimension (such as price), they are disproportionately
influenced by information in the same or similar currency to that dimension.
When pricing a gamble, for instance, the price put on it will be highly influenced
by the potential payoffs. Hence, the $-bet gets a higher price than the P-bet.
Compatibility effects can be viewed as one instantiation of the anchoring-and-
adjustment heuristic, by which quantities are estimated by starting with an available
estimate (even an arbitrary one), and then incompletely adjusting that estimate in
the appropriate direction based on other information. When pricing a gamble, the
logical starting point is the prize to be won, which is then adjusted downward basedon the probability of that prize. The insufficient adjustment leads the$-gamble to
be overpriced.
These classic preference reversals are a special case of a systematic difference
in preferences elicited under choiceand matching. Matching is the procedure, like
pricing, in which one option is given (e.g. a gamble) and the respondent specifies
another option equivalent in value to the first (e.g. a price). Tversky, Sattath,
and Slovic () showed this using a series of questions such as the following
(p.):
About people are killed each year in Israel in traffic accidents. The ministry of trans-
portation investigates various programs to reduce the number of casualties. Consider thefollowing two programs, described in terms of yearly costs (in millions of dollars) and
the number of casualties per year that is expected following the implementation of each
program:
Expected casualties Cost
Program X $million
Program Y $million
(Notice that in this problem the future states are not given explicitly, as in the
Savage table, but rather given as a summary of consequences multiplied by their
probability and summed over all possible states.) One group chose between the
two programs, while another group (theImplicit choicegroup) saw the matrix with
one value missing, which they then completed to make the two programs equal in
9 Grether and Plott () also provided evidence about whether people will spontaneously recog-nize inconsistencies. Their respondents first priced several items, then immediately made a set of
choices, and then priced the remaining items. The likelihood of a preference reversal was not in-fluenced by whether choice preceded pricing or pricing preceded choice, suggesting that the choiceinconsistency was not noticed. Lichtenstein and Slovic had deliberately separated their two tasks bysome time and with a lot of filler tasks,presumably because they were concerned that their respondents
would strive to be consistent. Grether and Plotts research suggests that inconsistencies must be veryobvious for them to be noticed.
8/12/2019 Experimental Tests of Rationality
12/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
value. To see how the responses were compared, imagine someone who has chosen
Program X, implying they would be willing to pay $million to save seventy lives.
Now, suppose she is asked to complete the matrix below:
Option Expected casualties Cost
Program X ??
Program Y $million
For choice and implicit choice to be consistent, the value specified should be no
less than $million, otherwise she would prefer Program Y. For each blank, there
is a similar range of values consistent with a choice of X over Y (e.g. casualties in
Program X can be no more than , and so on), and values outside of this range
indicate an inconsistency. In fact, for this particular question Tverskyet al. ()
found that percent of the direct choices were for Program X, as compared withonlypercent of the implicit choices.
In general, Tverskyet al. () observed many more direct choices of the option
superior on the primary dimension, meaning the dimension that people view as
most important. In the Israeli traffic example, the primary dimension is the number
of lives that can be saved. Although choice can be made using lexicographic proce-
dures, such as choosing the option superior on the most important dimension,
matching cannot. Matching demands an explicit tradeoffbetween option features.
For instance, in the above example it is possible to choose Program X using the
lexicographic rule that life has no value. But in matching, this rule would imply
an infinite cost for Program X.
A further example of preference reversals resulting from differing valuation pro-cedures is the comparison between choice and ranking, reported by Bateman, Day,
Loomes, and Sugden (). They conducted a study testing Allais-type prefer-
ences, like those above but with some modifications. Respondents either chose
between several pairs of gambles, or else rank-ordered all the gambles contained in
the pairs. They found that Allais-type preferences were common in choice (repli-
cating earlier studies), but not in ranking. Ranking did not eliminate all apparent
irrationalities, however, and actually increased the rate at which another rationality
condition, betweenness (a derivation of those already given), was violated. Bateman
et al. give a theoretical explanation for their finding, but for our purposes the most
important result is that two theoretically interchangeable measurement procedures
yield strikingly different results.One important difference between binary choice and ranking is in the number
of options on the table. In binary choice there are two, while in ranking there are
three or more, so that even if the agent focuses exclusively on the preference order
10 They tested thecommon ratio effect, an extension of the original Allais task. The finding is thatif a person is indifferent between two lotteries, one riskier than the other, then they will prefer the
riskier of the two if all the probabilities in the two lotteries are scaled down through multiplication bya common positive constant< 1.
8/12/2019 Experimental Tests of Rationality
13/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
between O1 and O2, there is at least an O3 to which both options can be com-
pared. It turns out that another source of violations of the rationality conditions is
that preference order depends even on options that are never chosen.
. IM O CS
..........................................................................................................................................
A seemingly trivial implication of the rationality conditions is that if an additional
option is taken into consideration, it will not influence the relative value of those
already there. That is, if O1 is preferred to O2, and O3 is then made available,
the only permissible rankings of the three options are those that maintain the
preference of O2over O3. For example, to take the example from Table., suppose
you are planning not to take your umbrella today, but then somebody reminds you
that you could take a raincoat instead; this should not make you decide to take your
umbrella. This follows both from the Decidability and Transitivity conditions, and
is sometimes called theindependence of irrelevant alternatives.
Batemanet al.s () comparison between ranking and choice, however, sug-
gests that this might not always be true. For ranking and choice to di ffer, it must be
that sometimes O1 is preferred to O2in binary choice, while O2is preferred to O1when additional options are on the table. It will not surprise the reader, therefore,
to learn that it is easy to construct scenarios in which many people choose O1from
the set {O1, O2} and then switch to O2when given the set {O1, O2, O3}.
One robust set of such context effects will be explained with the aid of Figure..
It depicts factors that might influence the choice between O1and O2, two options
that vary on two dimensions, with higher values on the dimensions corresponding
to improvement. The dimensions could be payoffand probability, price and quality,
or anything. In addition to O1 and O2, two further options are depicted, labeled
D and E for dominatedand extreme. D is dominated by O1 but not by O2; E is
an extreme item that is better than B on one dimension, and worse than B on
the other. Examples of such items are gambles like the following (e.g. Wedell ;
Herne):
O1: percent chance of $
O2: percent chance of $
D(O3): percent chance of $
E(O4): percent chance of $
There are two effects of adding the third item, either D or E. The first is the usual
substitution effect, in which some people choose the added option rather than
one of the original pair. Of course, this does not often happen with D, since it is
8/12/2019 Experimental Tests of Rationality
14/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
E: O4
D: O3
O1
O2
Dimension 1
Dimension2
Fig. 8.1. Adding additional items to a choice set.
dominated by O2. The second effect is that peoples preferences among the original
choice pair changes, with choices of O1 switching to choices of O2. The effect of
adding D is called theasymmetric dominance effect (and sometimes the attraction
effect), and the effect of adding E is called the compromise effect (sometimes ex-
tremeness aversion).The labels D and E come from the proposed locus of the spotlight effect. The
idea is that one way people determine what they prefer is by thinking of reasons
for having the preference. The reasons are justifications elicited by the context,
and different contexts elicit different justifications (e.g. Simonson ; Shafir,
Simonson, and Tversky). For these options, adding a new item gives an addi-
tional reason for choosing O2over O1which can overturn the preference expressed
without the new item. The new reason is that O2is clearly better than D, or that it
is a compromise between the two extremes O1 and E. The role of reasons can be
imagined by thinking of what someone would have to do to justify their choice to
someonethey could say, for instance, that Gamble O2is a compromise between
the boring O1 and the too-risky E. This, however, is a contingent argument. Ifthe added option had been different, one that instead made O1 the compromise
candidate, matters would be quite different.
The number of options has also been shown to matter in investigations ofjoint
versusseparate evaluation (Hsee ). In joint evaluation, the menu consists of
11 The original reports of these effects were in studies of consumer behavior. Huber, Payne, and
Puto () first described the asymmetric dominance effect, and Simonson () the compromiseeffect.
8/12/2019 Experimental Tests of Rationality
15/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
several items, so that people are able to make direct comparisons during evaluation.
In separate evaluation, however, they are evaluated one at a time. The original
preference reversal studies of Lichtenstein and Slovic are one example of such a
situation, but in those studies the evaluation procedure (choice versus pricing)
was confounded with whether there were one or two gambles (corresponding to
separate and joint evaluation) being evaluated. It turns out, however, that even
when the valuation procedure is held constant, joint and separate evaluation can
produce different preferences. A compelling example, because it involved real-
world transactions and real money, was described by List (), who replicated
the method and manipulations of Hsee () with dealers and collectors at a sports
card show. Lists participants were invited to bid on one or both of two sets of
sports cards. The inferiorset contained ten high-quality (near-mint) cards, while
thesuperiorset contained thirteen cards, including the ten high-quality cards plusthree low-quality cards. The estimated true market price of the superior set was
about $ higher than that of the inferior set. There were three groups: one group
bid separately on the inferior set, another bid on the superior set, and a third bid
on both. The result was that during separate evaluation, both dealers and collectors
(but especially collectors) bid more for the inferior set of cards, but during joint
evaluation they both bid more for the superior set.
Hsees () argument, reflecting a recurring theme, is that option features vary
in the degree to which they can be easily evaluated in isolation, and those features
that are difficult to evaluate will show a disproportionate increase in importance
during joint evaluation. The traders put too little weight on the numberof cards
when looking at only one set, while the poor quality of some of those cards is verysalient when looking at the superior set. Lists study shows just how easy it is to find
features that are difficult to evaluate in isolation.
. D O D W
..........................................................................................................................................
In the previous sections, there were some differences between situations that pro-
duced preference reversals. Either the evaluation method or the context was altered.
12 The fact the asymmetry was smaller among dealers points to the importance of learning andexperience in the application of rational choice theory. Binmore (e.g. , pp. ) argues thatrational choice theory is applicable only to situations in which the conditions are favorable, and
specifically that there is a good deal of evidence that adequately motivated people sometimes canand do learn to choose rationally if they are put in similar situations repeatedly and provided withmeaningful feedback on the outcome of their previous decisions.
13 A robust finding is that people are quite insensitive to scopeof a good when placing a value on it(Frederick and Fischhoff ).
8/12/2019 Experimental Tests of Rationality
16/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
No such differences are actually required to produce preference reversals. Even
seemingly modest redescriptions of options can accomplish the task, as demon-
strated by the widely cited Asian disease problem of Tversky and Kahneman
(). Their respondents chose between two medical responses to an epidemic that,
in the absence of some response, would certainly kill people. In one version
(they called this the loss frame) respondents chose between deaths for sure
and a one third chance ofdeaths, and a two thirds chance of no deaths. In
the second version (the gain frame) they chose between saving lives for sure
and a one third chance of saving everybody, and a two thirds chance of saving
nobody. A majority prefer to save lives for sure in the gain frame, but to take
a chance of saving no one in the hope of saving all in the loss frame. Although
when presented together, the two choices are transparently the same, the different
descriptions highlight different features of the options.Kahneman and Tversky attributed this result to a general pattern of risk seeking
for losses and risk avoidance for gains. A further demonstration of this produced
one of the most striking examples of how relatively modest variations in problem
description can lead to changes in preference. Kahneman and Tversky () asked
separate groups to make the following choices:
Group The following two gambles will be enacted simultaneously. Choose one
from (a), and one from (b)
(a) Choose between:
A) Win $
B) % chance to win $;% chance to win nothing
(b) Choose between:
C) Lose $
D) % chance to lose $;
% chance to lose nothing
Group Choose between:
A) % chance to win $;
% chance to lose $
B) % chance to win $
% chance to lose $.The pair most frequently chosen by Group was {A, D}about percent of
people chose this combinationwhile for Group everyone chose B. The latter
is not surprising, sinceA stochastically dominates B. However, ifA andD are
combined, it can be seen that theyareoptionA, while combiningB andC is the
14 It doesnt matter whether the same group or different groups do the taskthe results are
identical. I have used this problem in classes for many years, and these within-respondent results areinvariably indistinguishable from those reported by Kahneman and Tversky ().
8/12/2019 Experimental Tests of Rationality
17/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
dominantB. Kahneman and Tversky explain this using the same analysis as used
for the Asian disease problem. The Group(a) question is in the domain of gains
and leads to risk avoidance, while the Group (b) question is in the domain of losses
and leads to risk seeking.
There are three important features of responses to these questions. First, they
demonstrate how we evaluate outcomes from the perspective of a reference point
(i.e. as losses or gains) and that the reference point is highly malleable. Second, we
see a violation of (stochastic) dominance, so that unlike most systematic variations
in preference, including those described earlier, not only do people have different
preferences under different circumstances, but it is obvious when they are making
a mistake.
But the most important lesson from these questions is that it demonstrates
the central feature of the attentional spotlight. The choice of { A, D} shows thatrespondents failed to integrate their chosen options into a portfolio, but rather
treated them as separate options, as if each choice they were making was the only
one they would ever have to make. And they did this despite the fact that the
two gambles were explicitly described as being enacted simultaneously and were
given one after the other. Very similar results, often involving quite astonishing
degrees of isolation, have been described by Redelmeier and Tversky ( ) and
others (e.g. McCafferey and Baron; for a review and more examples see Read,
Loewenstein, and Rabin).
Although I earlier described the Allais paradox as a case in which the conse-
quences were changed, but in a way that is irrelevant to the rationality conditions,
the above result might lead us to reconsider this conclusion. Perhaps the differencein consequences between the two Allais questions is itself an illusion due to the
operation of the spotlight. Consider the following variant of the Allais problem,
itself adapted from one of Conlisks () variations. This shows the second pair of
Allais options (O1and O
2) transformed into a two-stage game.
The decision-maker has a ticket numbered between 1 andbut does not know
which. If the number is between and he gets nothing, but if it is between
and he has a chance of a prize, with the chance depending on whether he chooses
O1or O
2. Consider two occasions on which he can make this choice, either at CN1(choice node ) or at CN2. If he chooses at CN1 he will probably choose O
1 , as
/people did in Conlisks study. On the other hand, if he chooses at CN 2, he
is now much more likely to choose O2, as/did in Conlisks study. The rateof systematic preference change will be very high.
It is difficult to rationalize this. Anyone faced with a decision like that at CN2will have come to that point by way of a series of choices and accidents, like those
at CN1, that were not guaranteed to achieve that choice point. This uncertainty
is represented by a single chance node in Figure ., but in reality it would be a
15 John Conlisk kindly provided these data which were not included in hispaper.
8/12/2019 Experimental Tests of Rationality
18/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
CN 1
CN
11--100
1--10
1--10
1--9
10 0
0
1 million
5 million
A2
A1
CN1
CN2
Fig. 8.2. Sequential Allais decisions.
multiplicity of chance occurrencesfor instance, after years of struggle an entre-
preneur has the opportunity to sell out for a big killing, or to take a chance by
continuing with the chance of an even bigger killing or failing altogether. If the
entrepreneur was required to decide before he knew he would get to that stage (at
the equivalent of CN1) he would very likely make different and more risk-seeking
decisions than if he decided afterwards (CN2).
This is a very general pattern. Whenever we are faced with a choice, the fact that
we have that choice at all is contingent on events that might not have occurred.
Consequently, if we had decided in advance, based on the compounded expecta-tions in which many consequences along with their probabilities or uncertainties
were combined, we would have made different decisions than if we decide based on
the local returns.
The framing effects just described work because people do not consider infor-
mation or consequences that are not explicitly included in the frame they have
been given. They either put too little weight on unspoken consequences (as in the
lives saved or lost in the Asian disease problem), or do not combine sequences of
outcomes and uncertainties into composite or portfolio prospects. The effects of
framing can result from even more subtle manipulations. One such manipulation is
simply to divide states into increasing numbers of categories (e.g. Fischhoff, Slovic,
and Lichtenstein ; Fox and Clemen ). I will describe one example of this,which Starmer and Sugden () call event-splitting.
Starmer and Sugden offered people a choice between lotteries. The respondent
could draw a lottery ticket from a pool, and would win a prize contingent on the
number that came up (the state of the world) and the option they had chosen.
The choices were described using a strip display, a way of indicating separately,
for each ticket number, what the outcomes will be for each option. This allowed
the researchers either to combine or to divide ranges of states that yielded the
8/12/2019 Experimental Tests of Rationality
19/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
1--15 16--30 31--100
1--30 31--100
State (Ticket)
State (Ticket)
O1
O1 $100
$100$100 0
0
Fig. 8.3. Two strip displays like those used in the event-splitting study of Starmerand Sugden (1993).
same outcome. An example of two strip displays for the same option is given in
Figure .. Observe that O1and O2are identical, except that in O1the numbers
are divided into two states.
Starmer and Sugden obtained choices between lotteries like O1 and a standard
lottery, and lotteries like O1 and the same standard lottery. The standard lotteries
were always described in the same way. Since O1and O
1are formally identical, the
same proportion of people should have preferred them to the standard, otherwise
this would imply that one of two identical options is preferred to the other. Infact, the option, like O1, for which the state space for the most attractive out-
come was subdivided, was typically preferred to the one in which that space was
undivided.
Starmer and Sugdens result, along with an abundance of related findings, show
that the weight we put on information is highly dependent on how closely we
scrutinize it. We can return to our example of the decision about whether to
take an umbrella when it rains. What this result suggests is that if we divided
the state Rain into the two states Some rain but no more than centime-
ters, and More than centimeters we would then be more likely to take that
umbrella.
16 This point is made briefly, but the central problem is related to one raised by Luce and Raiffa
(). Suppose we have complete ignorance about which state of the world obtains. The principleof insufficient reason then dictates that we should treat each state as equally likely. But given that thestates about which we areignorant can be divided and combined in variousways, each equally sensible,it follows that the principle of insufficient reason will dictate different probability distributions over
the same states. It turns out that people are prone to the bias that might be expected from such a naveapplication of this principle (Fox and Clemen ).
8/12/2019 Experimental Tests of Rationality
20/26
8/12/2019 Experimental Tests of Rationality
21/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
The collapsing of grand worlds into small worlds has obvious parallels with the
process of shining a spotlight on certain aspects of the world and ignoring others.
When deciding whether to take an umbrella to work, for instance, it is natural
to focus on states relevant to the decision (e.g. Rain versus No rain), and on
consequences relevant to the use of umbrellas (Will I get wet or not?), but to
ignore everything else. This can be described as the formulation of a small world,
or as the deployment of the attentional spotlight.
But if the grand world can be collapsed into different (and not necessarily even
nested) small worlds, this raises the question ofwhichsmall worlds the rationality
conditions should apply to. One possible view is that as long as the conditions hold
over any small world, they are satisfied. For instance, if the small world of Table .
accurately describes how a decision-maker thinks of the problem at the moment of
choice, then so long as he takes his umbrella, the conditions are satisfied. If changesin the evaluation circumstances change the small world inanyway, the rationality
conditions should then be reapplied to this new situation. From this perspective,
the rationality conditions are not definitely violated by any of the studies described
above, and perhaps they never can be. It is unlikely that many knowingly and delib-
erately choose to violate these conditions when their applicability is obvious, and
if their applicability is not obvious, then no violation is possible. We can illustrate
this point by referring back to Kahneman and Tverskys two-stage question from
Section .. If the small world to which the rationality conditions are intended to
apply is at the level of the question (i.e. the Group (a) and Group questions
taken separately), then no individual choice can violate those conditions, and no
choicepatternis even relevant, since it is a pattern formed by combining separateand independent small worlds. Likewise, the apparent inconsistency between the
choices by Groups and is not relevant to the rationality conditions because it also
involves comparisons between different small worlds. Finally, when, as in Group ,
the small world involves a clear opportunity to violate a rationality condition (in
this case stochastic dominance), nobody does so.
Many will feel that so stripping the rationality conditions of their content is
an unsatisfactory way to shield them from refutation. Savage did not propose
this, and in fact provided a more restricted description of the small worlds to
which the rationality conditions should apply. He focused on what he called real
microcosms, which are small worlds that locally satisfy the rationality conditions (as
in, the decision-maker takes the umbrella given Table.) but, in addition: (a) theprobabilities of states are the same in the small world as in the grand world, and (b)
the utility of options is the same in the small world and the grand world (see Savage
, p.). A small world that meets the rationality conditions internally but does
not satisfy these additional requirements is, in Savages own words, a pseudo-
microcosm that is not a real microcosm. He was, as the following passage shows,
optimistic that the rationality conditions were applicable to circumstances beyond
that of the small world: I feel, if I may be allowed to say so, that the possibility of
8/12/2019 Experimental Tests of Rationality
22/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
being taken in by a pseudo-microcosm that is not a real microcosm is remote, but
the difficulty I find in defining an operationally applicable criterion is, to say the
least, ground for caution (Savage, p.).
The reason for Savages optimism is the psychologicalpremise, which he shares
with many researchers who come to behavioral studies from the discipline of
economics, that while someone might initially be misled by a small world, they
will eventually settle on a real microcosm when they have time to reflect on their
preferences. Savage illustrates this with the following example: A man buying a car
for $.is tempted to order it with a radio installed, which will bring the total
price to $., feeling that the difference is trifling. But when he reflects that, if
he already had the car, he certainly would not spend $. for a radio for it, he
realizes that he has made an error (p. ).
Savages intuition that an expensive radio might be chosen when its price is addedto the sticker price of a car, but not when it is seen in isolation, is undoubtedly
correct. But Savage presumes that the man changes his mind because he has
made an error which is now corrected. But there is nothing in the rationality
conditions themselves that shows that either option is the correct one (cf. Shafer
, for a related discussion of the same passage from Savage).
The difference between the small world and the spotlight viewpoints revolves
around the question of whether or not reframing a decision moves the decision-
maker towards a real microcosm. According to the small-world view, this is what
happens. We can see this reflected not only in the above passage, but more forcefully
yet in Savages reevaluation of his own preferences following his mistake when
answering Allaiss question (see n. ). According to the spotlight view, however,while rethinking highlights different aspects of choice, the changed preferences
are not necessarily correct or even better than those they have replaced. Rather,
they are determined by the path taken by the process of thinking and rethinking.
Moreover, since most people are like Savage, in that they do wish to maintain
consistency in their preferences, even local conformity to the rationality conditions
does not indicate that we have located a real microcosm, since a di fferent path of
thinking could have led to different preferences that conformed equally well to those
conditions.
To illustrate this, consider a case of intransitive preference. It is rare to find
situations in which people openly state that they simultaneously prefer A to B, B to
C, and C to A. Indeed, experimentalists traditionally separate questions with filleritems so that respondents will not realize they are making a mistake. According
to the small-world view, if we give an agent the three pairwise choices, he or
she will eventually work out the correctpreference ordering for the three options.
According to the spotlight view, on the other hand, the final preferences of this
18 This passage, like many of Savages examples, inspired an extensive and ongoing research pro-gram, starting with Tversky and Kahneman ().
8/12/2019 Experimental Tests of Rationality
23/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
agent will depend on the order in which the choices are presented. If we start with
the choice between C and A, then we will end up with the preference order CAB,
while if we start with the choice between A and B, we will end up with the order
ABC.
This phenomenon was characterized ascoherent arbitrarinessby Ariely, Loewen-
stein, and Prelec (, ), who investigated it with a series of clever exper-
iments. In one study, people first stated how much they were willing to pay for
an item, such as an average bottle of wine. The situation was engineered so that
some would give higher prices than others, by first asking respondents whether
they would pay a dollar amount equal to the last two digits of their social security
number, and then to state their maximum willingness to pay. This naturally resulted
in a strong anchoring effect, with people willing to paymuch more when their social
security number ended with a high number than a low one. The prices they gaveto other goods were also consistent with these initial prices. Everyone, for example,
was willing to pay more for a fancy bottle of wine than for an average one. But
because the average bottle was so overpriced in the high number condition, the
expensive bottle was correspondingly overpriced. The preferences were consistent,
but based on an arbitrary starting point.
The same authors, indeed, later showed that even whether an activity was posi-
tive, so that one should pay to do it, or negative, so that one should pay not to do it,
depends on apparently trivial features of the decision context, and that once this is
decided, people will be consistent with that judgment. In their study, people asked
how much they would pay to hear Ariely read from Walt WhitmansLeaves of Grass
would pay more to enjoy a longer reading, while a matched group asked how muchthey would have to be paid to hear him required more if they had toendurea longer
reading (Ariely, Loewenstein, and Prelec ).
Research like Arielyet al.s provides a good place to conclude this survey, because
it both demonstrates the ultimate implications of the spotlight view and raises
other issues about rationality to which I will only allude. In this chapter I have
deliberately restricted my focus to the question of whether preferences do, in any
meaningful way, conform to rationality conditions like Savages. The experimental
evidence reviewed gave scant support to the view that the rationality conditions are
adhered to. Instead, preferences shift dramatically as we change the circumstances
in which they are to be expressed. I likened this to the operation of a spotlight,
where each new circumstance highlights a restricted subset of task features andleads to the creation of a correspondingly restricted (and distorted) representation
19 This is the primary reason why we do not expect individuals to be transformed into moneypumps by repeating the same choices. Even if a person can be found to have a cycle of preferences,they will refuse to engage in a transparent cycle of transactions. But as Rabin and Thaler () haveobserved, this does not mean that we cannot create money pumps across people. If we can identify
situations in which the majority of people have a cyclical preference and different endowments, we canbe sure to earn money by offering a single trade to each person.
8/12/2019 Experimental Tests of Rationality
24/26
-Anand-c OUP-Anand (Typeset by SPi, Delhi) of December, :
of the world. This representation is not, except accidentally, coordinated with other
representations that we would have adopted if the spotlight had been di fferent.
In Savages words, the spotlight directs us to a pseudo-microcosm that is not a
real microcosm. I have not unconsidered the problem of whether rationality can
and should be identified with some degree oflocalconformity to the rationality
conditions. That is, given that in the broad sense we are inevitably inconsistent, is
it nonetheless better to be consistent as far as we can be? No evidence is available to
answer this question. Arielyet al. show some ways in which local consistency can
lead to error, but they have not shown (or tried to show) that inconsistency would
lead tolesserror. To answer this question, we need different criteria for rationality
than the conditions of Savage or von Neumann and Morgenstern, criteria which
can rank choices as good or bad, so that we can know whether we are more or
less likely to choose the good options when we are consistent or when we areinconsistent.
R
A, M. (). Le comportement de lhomme rationnel devant le risque: critique des
postulats et axioms de lcole Amricaine. Econometrica,,.
AP. (). The Philosophy of Intransitive Preference. Economic Journal,,.
A, D., L, G., and P, D. (). Coherent Arbitrariness: Stable De-
mand Curves without Stable Preferences. Quarterly Journal of Economics,,.
(). Tom Sawyer and the Construction of Value. Journal of Economic
Behavior and Organization,,.
B, A., D, B., L, G., and S, R. (). Ranking versus Choice in the
Elicitation of Preferences. Working Paper, University of East Anglia.
B, D. (). Regret in Decision Making under Uncertainty.Operations Research, ,
.
B, K. ().Rational Decisions. Princeton: Princeton University Press.
C, K R., and B, N P. (). Visuo-spatial Attention: Beyond a Spotlight
Model.Psychonomic Bulletin and Review,/,.
C, J. (). Three Variants on the Allais Example.American Economic Review, ,
.
E, D. (). Risk, Ambiguity, and the Savage Axioms. Quarterly Journal of
Economics,,.F, B., S, P., and L, S. (). Fault Trees: Sensitivity of Estimated
Failure Probabilities to Problem Representation.Journal of Experimental Psychology: Hu-
man Perception and Performance,,.
F, C., and C, R. T. (). Subjective Probability Assessment in Decision Analysis:
Partition Dependence and Bias Toward the Ignorance Prior. Management Science, ,
.
F, S., and F, B. (). Scope (In)Sensitivity in Elicited Valuations. Risk,
Decision, and Policy,,.
8/12/2019 Experimental Tests of Rationality
25/26
8/12/2019 Experimental Tests of Rationality
26/26
Recommended