The Gambler’s Fallacy• A coin flip comes up heads three times in a row. What are the odds that

it will be heads on the next toss?

• A rational decision-maker knows that they are 50-50. But it’s easy tosuccumb to the belief that streaks don’t occur by chance. This commonmisperception is known as the gambler’s fallacy. In Decision-Makingunder the Gambler’s Fallacy: Evidence from Asylum Judges, LoanOfficers, and Baseball Umpires (NBER Working Paper No. 22026),Daniel Chen, Tobias J. Moskowitz, and Kelly Shue find that in anumber of different settings, individuals have a slight bias againstdeciding the same way in successive cases. The researchers find, forexample, that the odds that a judge rejects an asylum seeker are 3.3percentage points higher if the judge has approved the previous case,all else being equal. They also note that the likelihood that judges areinfluenced by a prior decision increases with the length of the sequence

1

of positive or negative rulings and the similarity of the previous cases.

• The researchers also study loan officers in India. The officers wereasked to review loan application files that had already been processed,and to make recommendations about whether to approve the loan. Theyfaced different incentive schemes, which placed different degrees ofemphasis on an accurate assessment. Because the files had beenreviewed previously, the authors could evaluate the quality of theofficers’ decisions by examining the actual performance of the loan andexploring whether recommended loans on average were performingbetter.

• Under a flat incentive scheme which rewarded approvals regardless ofloan quality, officers—who rejected many loans despite the incentive,perhaps out of intrinsic or reputational motivation—were eightpercentage points less likely to approve the loan currently under reviewif they had approved the previous loan. This bias became much less

2

significant under plans with stronger incentives for accuracy.

• Turning to baseball, the researchers analyze major league umpires.They examine 1.5 million called pitches, when the batter did not swing,between 2008 and 2012. They control for a wide array of factors, suchas pitch count; pitch spin and acceleration; the relative importance ofthe at-bat to the outcome of the game; and whether the batter was onthe home team. They rely on data compiled by the PITCHf/x system,which tracks the speed and trajectories of pitches in every major leaguestadium.

• They find that umpires were 1.5 percentage points less likely to call astrike if the previous pitch was a called strike. The bias toward anopposite call was also significantly more pronounced when theprevious two calls were the same.

• Do umpires use subsequent calls to make up for calls they regret? The

3

researchers find that, if anything, umpires were less likely to make anopposite call following an incorrect call than after a correct one. Theyconclude that ”[F]airness concerns and a desire to be equally nice totwo opposing teams are unlikely to explain our results.”

• The researchers discover that the gambler’s fallacy tends to be moreevident following longer streaks of decisions in the same direction andwhen the previous cases have similar characteristics and occur closer intime. It is less evident among more experienced decision-makers.

4

Chen et al. (2016)• Abstract: We find consistent evidence of negative autocorrelation in

decision making that is unrelated to the merits of the cases consideredin three separate high-stakes field settings: refugee asylum courtdecisions, loan application reviews, and Major League Baseball umpirepitch calls. The evidence is most consistent with the law of smallnumbers and the gambler’s fallacy—people underestimating thelikelihood of sequential streaks occurring by chance—leading tonegatively autocorrelated decisions that result in errors. The negativeautocorrelation is stronger among more moderate and less experienceddecision makers, following longer streaks of decisions in one direction,when the current and previous cases share similar characteristics oroccur close in time, and when decision makers face weaker incentivesfor accuracy. Other explanations for negatively autocorrelated decisionssuch as quotas, learning, or preferences to treat all parties fairly are less

5

consistent with the evidence, though we cannot completely rule outsequential contrast effects as an alternative explanation.

6

Introduction• The “law of small numbers” and the “gambler’s fallacy” are both

names for the well-documented tendency of people to overestimate thelikelihood that a short sequence will resemble the general population orunderestimate the likelihood of streaks occurring by chance.

– For example, people often believe that a sequence of coin flips suchas “HTHTH” is more likely to occur than “HHHHT” even thougheach sequence occurs with equal probability.

– Similarly, people may expect flips of a fair coin to generate highrates of alternation between heads and tails even though streaks ofheads or tails often occur by chance.

• Most of the existing empirical literature examines behavior in gamblingor laboratory settings (e.g., Suetens et al. (2016)) and does not testwhether the gambler’s fallacy can bias high-stakes decision making in

7

real-world or field settings.

• Sequential contrast effects (SCE): the decision maker’s perception ofthe quality of the current case is negatively biased by that of theprevious one (Simonsohn and Loewenstein (2006); Simonsohn (2006)).

– Bhargava and Fisman (2014) find that speed dating subjects aremore likely to reject the next candidate if the previous candidatewas very attractive.

– Hartzmark and Shue (2018) find that investors perceive today’searnings news as less impressive if unrelated firms released goodearnings news in the previous day.

• The sizable body of psychology literature using vignette studies ofsmall samples of judges suggests unconscious heuristics (e.g.,anchoring, status quo bias, availability) play a role in judicial decisionmaking.

8

Asylum Judges: Data Description and Institutional Context

• We use administrative data on U.S. refugee asylum cases considered inimmigration courts from 1985 to 2013.

• Cases considered within each court are randomly assigned to the judgesassociated with the court (on average, there are eight judges per court).

– The judges then review the queue of cases following a“first-in-first-out” rule.

• Within the same four-year time period in the court of New York, twojudges granted asylum to fewer than 10% of the cases considered, whilethree other judges granted asylum to over 80% of cases considered.

9

Empirical Specification Details

• A set of dummies for the number of affirmative decisions over the pastfive decisions (excluding the current decision) of the judge controls forrecent trends in grants, case quality, or judge’s mood.

– A set of dummies for the number of grant decisions over the pastfive decisions across other judges (excluding the current judge) inthe same court controls for recent trends at the court level.

– The judge’s leave-out-mean grant rate for the relevant nationality ×defensive category, calculated excluding the current observation,and the court’s average grant rate for the relevant nationality ×defensive category, calculated by excluding the judge associatedwith the current observation, control for longer-term trends injudge- and court-specific grant rates.

• We control for the characteristics of the current case:

10

presence-of-lawyer indicator, family size, nationality × defensivestatus fixed effects, and time-of-day fixed effects(morning/lunchtime/afternoon).

– Including time-of-day fixed effects is designed to control for otherfactors such as hunger or fatigue, which may influence judicialdecision making (as shown in the setting of parole judges byDanziger et al. (2011)).

11

Results• We further restrict the sample to cases that follow another case on the

same day (rather than the previous day).

– The stronger negative autocorrelation when two consecutive casesoccur more closely in time is broadly consistent with saliency andthe gambler’s fallacy decision making model because more recentcases may be more salient and lead to stronger expectations ofreversals.

– These results are also consistent with experimental results in astudy, which finds that laboratory subjects who are asked to predictcoin flips exhibit less gambler’s fallacy after an interruption whenthe coin “rests.”

• We restrict the sample further to cases in which the current andprevious case have the same defensive status.

12

– Individuals seeking asylum affirmatively, where the applicantinitiates, can be very different from those seeking asylumdefensively, where the government initiates.∗ In affirmative cases, applicants typically enter the country legally

and are applying to extend their stay. In defensive cases,applicants often have entered the country illegally and have beendetained at the border or caught subsequently.

∗ Judges may view these scenarios to be qualitatively different.

• While there is significant negative autocorrelation when sequentialcases correspond to different applicant nationalities, the negativeautocorrelation is three times larger when the two cases correspond tothe same nationality.

– The negative autocorrelation in decisions may be tied to saliencyand coarse thinking: more likely when the previous case consideredis similar in terms of characteristics, in this case nationality.

13

• Judges who have less than the median experience in the sample (eightyears) display stronger negative autocorrelation.

– Reduced negative autocorrelation does not necessarily imply thatexperienced judges are more sophisticated in terms ofunderstanding random processes.

∗ Experienced judges may draw, or believe they draw, moreinformative signals regarding the quality of the current case. Ifso, experienced judges will rely more on the current signal andless on their prior beliefs, leading to reduced negativeautocorrelation in decisions.

• Time variation in case quality (e.g., a surge in refugees from a hotconflict zone) should originate at the court level and is likely to bepositively autocorrelated on a case-by-case level.

14

Loan Officers: Data Description and Institutional Context• Real loan officers were recruited for the experiment from the active

staff of several commercial banks.

– They had an average of 10 years of experience in the banking sector.

– They were sent to the experiment by their home bank and theexperiment was conducted at a loan officer training college. At theend of the experiment, they received a completion certificate and adocument summarizing their overall accuracy rate.∗ They were told that it would only report that without reporting

the ordering of their specific decisions and associated accuracy.

• Performing loan files were approved and did not default during theactual life of the loan. Nonperforming loans were either rejected by thebank in the loan application process or were approved but defaultedduring the actual life of the loan.

15

– The percentage of performing loans they approved is 78%, whilethe percentage of nonperforming loans they approved is 62%, whichshows they exhibited some ability to sort loans.

• Participants in each session were randomly assigned to one of threeincentive schemes, which offered payouts of the form [wP,wD, w̄],where wP is the payout in rupees for approving a performing loan, wD

is the payout for approving a nonperforming loan, and w̄ is the payoutfor rejecting a loan (regardless of actual loan performance).

– [20,20,0], [20,0,10], and [50,−100,0] in the “flat,” “stronger,” and“strongest” incentive schemes, respectively.

• The loan officers were told that they were drawn from a large pool ofloans, of which approximately two-thirds were performing loans.Because the loan officers reviewed loans in an electronic system, theycould not review the loans in any order other than the order presented.They faced no time limits or quotas.

16

Empirical Specification Details

• We control for heterogeneity in mean approval rates at the loan officer× incentive scheme level using the mean loan officer approval ratewithin each incentive treatment (calculated excluding the sixobservations corresponding to the current session).

– Because the loan officer field experiment data is limited in size andeach session consists of only six loan decisions, we do not controlfor a moving average of each loan officer’s average decision rateover the past five decisions within the session (as we do in theasylum judge setting).

17

Results

• In (2), we control for the true quality. Therefore, the coefficientsrepresent mistakes on the part of the loan officer. After including thiscontrol variable, we find quantitatively similar results, indicating thatthe negatively autocorrelated decision making results in decision errors.

18

19

• The original experimenters implemented a balanced session design.Each session consisted of four performing loans and twononperforming loans.

– If the loan officers had realized that sessions were balanced, arational response would have been to reject loans with a greaterprobability after approving loans within the same session (and viceversa).

∗ If they had “figured out” that sessions were balanced, we wouldexpect that they would be more likely to use this informationwhen subject to stronger pay for performance. In other words,there should have been greater negative autocorrelation withinthe incentive treatments with stronger pay-for-performance.

∗ The better educated may be more likely to deduce that sessionsare balanced, so they should display stronger negativeautocorrelation.

20

Baseball Umpires: Data Description

• PITCHf/x tracks the trajectory and location of each pitch with respectto each batter’s strike zone as the pitch crosses in front of home plate.The location measures are accurate to within a square centimeter.

21

Empirical Specification• We include detailed controls for the characteristics of the current pitch,

such as the pitch location relative to an absolute point on home plateusing indicators for each 3×3 inch square, whether the current pitchwas within the strike zone based on its location, and also the speed,acceleration, curvature, and spin in the x, y, and z directions of thepitch, which may affect an umpire’s perception.

– Our control variables address the concern that pitch characteristicsare not randomly ordered.∗ We show that pitchers are more likely to throw another strike

after the previous pitch was called a strike, resulting in positive,rather than negative, coefficients on the previous call.

– The fact that we control for whether the current pitch is actuallywithin the true strike zone implies that any nonzero coefficients onother variables represent mistakes on the part of the umpire.

22

• Parsons et al. (2011) show evidence of discrimination in calls: umpiresare less likely to call strikes if the umpire and pitcher differ in race andethnicity.

– While biases against teams or specific types of players could affectthe base rate of called pitches within innings or against certainpitchers, they should not generate high-frequency negativeautocorrelation in calls.

– We include umpire, batter, and pitcher fixed effects, which shouldaccount for these sorts of biases, and find similar effects.

• Moskowitz and Wertheim (2014) show that umpires prefer to avoidmaking calls that result in terminal outcomes or that may determinegame outcomes.

• To differentiate our finding from such other types of biases that mayaffect the probability of the umpire calling strike versus ball at different

23

points in the game, we control for indicator variables for every possiblecount combination (number of balls and strikes called so far on thebatter), the leverage index (a measure developed by Tom Tango of howimportant a particular situation is in a baseball game depending on theinning, score, outs, and number of players on base), indicators for thescore of the team at bat, indicators for the score of the team in the field,and an indicator for whether the batter belongs to the home team.

• We present our baseline regression results without including controlsfor individual heterogeneity (the lack of controls should be a biasagainst findings of negative autocorrelation).

– Among umpires who have made more than 500 calls, the standarddeviation in the mean rate of calling strikes is 0.01, potentiallybecause extreme umpires would be much less accurate and umpireaccuracy can be judged relative to the PITCHf/x system.

24

Results

• We find that the magnitude of negative autocorrelation coefficients are10 to 15 times larger when the current pitch is ambiguous relative towhen the current pitch is obvious.

– We can confidently reject equality of the estimates for ambiguousand obvious pitches, which is consistent with the gambler’s fallacymodel that the decision maker’s prior beliefs about case quality willhave less impact on the decision when the signal about current casequality is more informative.

• An increase in leverage (the importance of a particular game situationfor determining the game outcome) leads to significantly strongernegative autocorrelation in decisions. However, the magnitude of theeffect is small: a one standard deviation increase in game leverage leadsto less than a 10% increase in the extent of negative autocorrelation.

25

• A one standard deviation increase in umpire accuracy (calculated as thefraction of pitches correctly called by the umpire in other gamesexcluding the current game) reduces negative autocorrelation by 25%.

• We divide game attendance into quintiles and compare the highest andlowest quintiles to the middle three quintiles.

– In the highest quintile, the negative autocorrelation increases by18%, while the difference is only marginally significant.

• The marginally stronger negative autocorrelation effects for highleverage situations and high attendance games may be consistent withumpires worrying about appearing biased in more heavily scrutinizedenvironments, where fans, analysts, and the media may suffer from thegambler’s fallacy.

26

Addressing Alternative Explanations: Quotas and Learning• One may be concerned that decision makers self-impose quotas.

– Even without a self-imposed quota, decision makers may believethat the correct fraction of affirmative decisions should be somelevel of θ .

– Under a learning model, the decision maker may be unsure ofwhere to set the quality bar to achieve an affirmative target rate ofθ , and learn over time.

• We control for the fraction of the previous N decisions that were madein the affirmative, where N equals 2 or 5, and testing whether theprevious single decision still matters.

– We find that, holding constant the fraction of the previous two orfive decisions decided in the affirmative, the previous singledecision negatively predicts the next decision.

27

∗ Decision makers in our three settings are highly experienced andshould have a standard of quality calibrated from many years ofexperience. These decision makers are probably not learningmuch from their most recent decision.

∗ Baseball umpires should make decisions according to anobjective quality standard (the officially defined strike zone)rather than according to a target affirmative decision rate.

28

External Perceptions and Preferences for Alternation andFairness

• Concerns abount external perceptions: the decision maker fullyunderstands random processes, but cares about the opinions of others,for example, promotion committees or voters, who are fooled byrandomness. These rational decision makers will choose to makenegatively autocorrelated decisions, even if they know they are wrong,to avoid the appearance of being too lenient or too harsh.

– They are unlikely to drive the results in the context of loan approval,which is an experimental setting where payouts depend only onaccuracy and the ordering of decisions, and their associatedaccuracy are never reported to participants or their home banks.

• Umpires may have a preference to be equally nice or “fair” to twoopposing teams.

29

– The desire to be fair to two opposing teams is unlikely to driveresults in the asylum judges and loan officers settings because thedecision maker reviews a sequence of independent cases, and thecases are not part of any teams. However, in baseball, the umpiremakes sequential calls on the same team at bat. Fairness motivesmay lead umpires to undo a previous marginal or mistaken call,which could result in negative autocorrelation.

– The negative autocorrelation is stronger following a previous correctcall than following a previous incorrect call, which is inconsistentwith a fairness motive because umpires concerned with fairnessshould be more likely to reverse the previous call if it was incorrect.

– The negative autocorrelation remains equally strong or strongerwhen the previous call was obvious.∗ In these cases, the umpire is less likely to feel guilt about making

a particular call because the umpire could not have called it any

30

other way (e.g., he, and everyone else, knew it was the right callto make).

– We restrict the sample to called pitches following previous calls thatwere either obvious or ambiguous. We further divide previousambiguous calls into those that were called correctly (60%) andthose that were called incorrectly (40%).

∗ If fairness concerns drive the negative autocorrelation in calls, thenegative autocorrelation should be strongest following previousambiguous and incorrect calls.

31

Conclusion

• Beyond the three settings we study, negatively autocorrelated decisionmaking could have broader implications. For example, financialauditors, human resource interviewers, medical doctors, and policymakers all make sequences of decisions under substantial uncertainty.Our results suggest that misperceptions of what constitutes a fairprocess can perversely lead to unfair or incorrect decisions in manysituations.

32

Danziger et al. (2011)• Abstract: Are judicial rulings based solely on laws and facts? Legal

formalism holds that judges apply legal reasons to the facts of a case ina rational, mechanical, and deliberative manner. In contrast, legalrealists argue that the rational application of legal reasons does notsufficiently explain the decisions of judges and that psychological,political, and social factors influence judicial rulings. We test thecommon caricature of realism that justice is “what the judge ate forbreakfast” in sequential parole decisions made by experienced judges.We record the judges’ two daily food breaks, which result insegmenting the deliberations of the day into three distinct “decisionsessions.” We find that the percentage of favorable rulings dropsgradually from 5%; to nearly zero within each decision session andreturns abruptly to 5%; after a break. Our findings suggest that judicialrulings can be swayed by extraneous variables that should have no

33

bearing on legal decisions.

34

Hartzmark and Shue (2018)

• Abstract: A contrast effect occurs when the value of a previouslyobserved signal inversely biases perception of the next signal. Wepresent the first evidence that contrast effects can distort prices insophisticated and liquid markets. Investors mistakenly perceiveearnings news today as more impressive if yesterday’s earnings surprisewas bad and less impressive if yesterday’s surprise was good. A uniqueadvantage of our financial setting is that we can identify contrast effectsas an error in perceptions rather than expectations. Finally, we showthat our results cannot be explained by an alternative explanationinvolving information transmission from previous earningsannouncements.

35

Parsons et al. (2011)

• Abstract: Major League Baseball umpires express their racial/ethnicpreferences when they evaluate pitchers. Strikes are called less often ifthe umpire and pitcher do not match race/ethnicity, but mainly wherethere is little scrutiny of umpires. Pitchers understand the incentivesand throw pitches that allow umpires less subjective judgment (e.g.,fastballs over home plate) when they anticipate bias. These direct andindirect effects bias performance measures of minorities downward.The results suggest how discrimination alters discriminated groups’behavior generally. They imply that biases in measured productivitymust be accounted for in generating measures of wage discrimination.

36

Simonsohn (2006)

• Abstract: Previous experimental research has shown that people’sdecisions can be influenced by options they have encountered in thepast. This paper uses PSID data to study this phenomenon in the field,by observing how long people commute after moving between cities. Itis found, as predicted, that (i) people choose longer commutes in a citythey have just moved to, the longer the average commute was in thecity they came from, and (ii) when they move again within the new city,they revise their commute length, countering the effect their origin cityhad on their initial decision.

37

Simonsohn and Loewenstein (2006)

• Abstract: Based on contrast effects studies from psychology, wepredicted that movers arriving from more expensive cities would rentpricier apartments than those arriving from cheaper cities. We alsopredicted that as people stayed in their new city they would get used tothe new prices and would readjust their housing expenditurescountering the initial impact of previous prices. We found support forboth predictions in a sample of 928 movers from the PSID. Alternativeexplanations based on unobserved wealth and taste, and on imperfectinformation are ruled out.

38

Suetens et al. (2016)

• Abstract: We investigate the “law of small numbers” using a data seton lotto gambling that allows us to measure players’ reactions to draws.While most players pick the same set of numbers week after week, wefind that those who do change, react on average as predicted by the lawof small numbers as formalized in recent behavioral theory. Inparticular, players tend to bet less on numbers that have been drawn inthe preceding week, as suggested by the “gambler’s fallacy,” and betmore on a number if it was frequently drawn in the recent past,consistent with the “hot-hand fallacy.”

39

References

Bhargava, Saurabh and Ray Fisman (2014) “Contrast Effects in SequentialDecisions: Evidence from Speed Dating,” Review of Economics andStatistics, Vol. 96, No. 3, pp. 444–457, July.

Chen, Daniel L., Tobias J. Moskowitz, and Kelly Shue (2016) “DecisionMaking Under the Gambler’s Fallacy: Evidence from Asylum Judges,Loan Officers, and Baseball Umpires,” Quarterly Journal of Economics,Vol. 131, No. 3, pp. 1181–1242.

Danziger, Shai, Jonathan Levav, and Liora Avnaim-Pesso (2011)“Extraneous factors in judicial decisions,” Proceedings of the NationalAcademy of Sciences, Vol. 108, No. 17, pp. 6889–6892.

Hartzmark, Samuel M. and Kelly Shue (2018) “A Tough Act to Follow:

40

Contrast Effects in Financial Markets,” Journal of Finance, Vol. 73, No.4, pp. 1567–1613.

Moskowitz, Tobias and L. Jon Wertheim (2014) Scorecasting: The HiddenInfluences Behind How Sports Are Played and Games Are Won, NewYork: Crown.

Parsons, Christopher A., Johan Sulaeman, Michael C. Yates, and Daniel S.Hamermesh (2011) “Strike Three: Discrimination, Incentives, andEvaluation,” American Economic Review, Vol. 101, No. 4, pp. 1410-35,June.

Simonsohn, Uri (2006) “New Yorkers Commute More Everywhere:Contrast Effects in the Field,” Review of Economics and Statistics, Vol.88, No. 1, pp. 1–9.

Simonsohn, Uri and George Loewenstein (2006) “Mistake #37: The Effect

41

of Previously Encountered Prices on Current Housing Demand,”Economic Journal, Vol. 116, No. 508, pp. 175–199.

Suetens, Sigrid, Claus B. Galbo-Jørgensen, and Jean-Robert Tyran (2016)“Predicting Lotto Numbers: A Natural Experiment on the Gambler’sFallacy and the Hot-Hand Fallacy,” Journal of the European EconomicAssociation, Vol. 14, No. 3, pp. 584–607.

42