Testing Transitivity with Individual Data Michael H. Birnbaum and Jeffrey P. Bahra California State...

Testing Transitivity with Individual Data

Michael H. Birnbaum and Jeffrey P. Bahra

California State University, Fullerton

Transitivity of Preference

• If A > B and B > C then A > C.• Weak Stochastic Transitivity: • If P(A, B) > 1/2 and P(B, C) > 1/2

then P(A, C) > 1/2.

Tversky (1969)

• Tversky (1969) reported that selected subjects showed a pattern of intransitive data consistent with a lexicographic semi-order.

• Tversky tested Weak Stochastic Transitivity.

Issues

• Iverson & Falmagne (1985) argued that Tversky’s statistical analysis was incorrect of WST.

• Tversky went on to publish transitive theories of preference (e.g., CPT).

Renewed Interest in Intransitive Preference

• New analytical methods for analysis of transitivity (Iverson, Myung, & Karabatsos; Regenwetter & Stober, et al); Error models (Sopher & Gigliotti, ‘93; Birnbaum, ‘04; others).

• Priority Heuristic (Brandstaetter, et al., 2006); stochastic difference model (González-Vallejo,

2002; similarity judgments, Leland, 1994; majority rule, Zhang, Hsee, Xiao, 2006). Renewed interest in Fishburn, as well as in Regret Theory.

Lexicographic Semi-order• G = (x, p; y, 1 - p). F = (x’, q; y’, 1 -

q).

• If y - y’ ≥ L choose G

• If y’ - y ≥ L choose F

• If p - q ≥ P choose G

• If q - p ≥ P choose F

• If x > x’ choose G; if x’ > x choose F;• Otherwise, choose randomly.

Priority Heuristic• 10% of largest prize, rounded to nearest

prominent number(In this study, L = $10).

• Compare gambles by lowest consequences. If difference exceeds the aspiration level, choose by lowest consequence.

• If not, compare probabilities; choose by probability if difference ≥ 0.1 (P = 0.1).

• Compare largest consequences; choose by largest consequences.

Birnbaum & Gutierez (OBHDP, in press)

• Four studies used Tversky’s 5 gambles, formatted with tickets or with pie charts.

• Failed to find evidence that more than a very small percentage of participants (~ 6%) were intransitive.

• Other tests refuted lexicographic semiorder and priority heuristic.

Pie Chart Format

Response to Birnbaum-Gutierrez

• Perhaps the intransitivity only develops in longer studies. Tversky used 20 replications of each choice.

• Perhaps consequences of Tversky’s gambles diminished since 1969 due to inflation. Perhaps those prizes are now too small.

Birnbaum & Bahra• Collected up to 40 choices/pair per

person. (20 reps). 2 Sessions, 1.5 hrs, 1 week apart.

• Cash prizes up to $100. • 51 participants, of whom 10 to win

the prize of one of their chosen gambles.

• 3 5 x 5 Designs to test transitivity vs. Priority heuristic predictions

Choice Format

Notation-Two-branch Gambles

• G = (x, p; y, 1 - p); x > y ≥ 0• L = Lower Consequence• P = Probability to win higher prize• H = Higher consequence

LH (Lower & Higher Consequences) Design

• A = ($84, .50; $24)• B = ($88, .50; $20)• C = ($92, .50; $16)• D = ($96, .50; $12)• E = ($100, .50; $8)

LP (Lower & Probability) Design

• F = ($100, .50; $24)• G = ($100, .54; $20)• H = ($100, .58; $16)• I = ($100, .62; $12)• J = ($100, .66; $8)

PH (Probability & Higher) Design

• K = ($100, .50; $0)• L = ($96, .54; $0)• M = ($92, .58; $0)• N = ($88, .62; $0)• O = ($84, .66; $0)

Priority Heuristic implies: Intransitive and Consistent

LH design Second GambleFirst A B C D E

A=(84, 24) 2 2 1 1B= (88, 20) 1 2 2 1C= (92, 16) 1 1 2 2D= (96, 12) 2 1 1 2E= (100, 8) 2 2 1 1

Transitive & Consistent1 = Chose First; 2 = Chose

SecondSecond Gamble

First A B C D EA 1 1 1 1B 2 1 1 1C 2 2 1 1D 2 2 2 1E 2 2 2 2

Within-Rep Consistency

• Count the number of consistent choices in a replicate of 20 choices (10 x 2).

• If a person always chose the same button, consistency = 0.

• If a person were perfectly consistent within replicate, consistency = 10.

• Randomly choosing between responses produces expected consistency of 5.

Within-Replicate Consistency

• The average rate of agreement was 8.63 (86% self-agreement).

• 46.4% of all replicates were scored 10.

• An additional 19.9% were scored 9.

LH Design: Overall Proportions Choosing Second Gamble

Second GambleFirst A B C D E

A= (84, 24) 0.41 0.38 0.34 0.27

B= (88, 20) 0.58 0.40 0.36 0.30

C= (92, 16) 0.61 0.59 0.44 0.32

D= (96, 12) 0.64 0.61 0.55 0.33

E= (100, 8) 0.70 0.69 0.66 0.66

LP Design: Overall Proportions Choosing Second Gamble

LP Second GambleFirst F G H I J

F 0.44 0.43 0.42 0.36G 0.54 0.42 0.42 0.38H 0.54 0.55 0.45 0.40I 0.56 0.56 0.53 0.41J 0.60 0.59 0.57 0.56

PH Design: Overall Proportions Choosing Second Gamble

PH Second GambleFirst K L M N O

K 0.61 0.64 0.64 0.64L 0.37 0.61 0.63 0.65M 0.34 0.37 0.64 0.64N 0.34 0.35 0.33 0.63O 0.34 0.33 0.35 0.34

Averaged Data Fit WST

• LH Design A > B > C > D > E• LP Design F > G > H > I > J• PH Design O > N > M > L > K• Consistent with special TAX with its

“prior” parameters.• This analysis obscures individual

diffs

Individual Data

• Choice proportions calculated for each individual in each design.

• These were further examined within each person replication by replication.

S# 8328 C = 9.6 Reps = 20

S# 8328 C = 9.8 Reps = 20

LP Second GambleF G H I J

F 0.05 0.02 0.00 0.00G 0.00 0.00 0.00H 0.05 0.02I 0.00J

S# 8328 C = 9.9 Reps = 20

PH Second GambleK L M N O

K 1.00 1.00 1.00 0.98L 1.00 1.00 1.00M 0.95 1.00N 0.95O

S# 6176 C = 9.8 Rep = 20; started with this pattern, then switched to perfectly consistent

with the opposite pattern for 4 replicates at the end of the first day; back to this pattern for 10

reps on day 2.

PH Second GambleK L M N O

K 0.28 0.20 0.23 0.20L 0.25 0.20 0.20M 0.20 0.20N 0.20O

Data Summary

• With n = 51, there are 153 matrices. Of these, 87% were perfectly consistent with WST: P(A,B) ≥ 1/2 & P(B,C) ≥ 1/2 then P(A,C) ≥ 1/2.

• 32 people (63%) had all three arrays fitting WST; no one fit priority heuristic nor did anyone have all three intransitive arrays.

• Those arrays that were not perfect fits to WST were either close to perfect, from “noisy” participants, or from people who changed orders.

Within-Person Changes in Preference Pattern

• Criterion: Person show perfect consistency (10 out of 10) to one pattern in one replication, and perfect consistency to different pattern on another replication.

• 16 Such arrays were found (~10% of 153) involving 12 participants.

• This result is troubling to the assumption that errors arise independently, but consistent with idea that people have changing parameters that drift rather than random changes.

Summary

• Recent studies fail to confirm systematic violations of transitivity predicted by lexicographic semiorders, including priority heuristic.

• No individual’s data agreed with predictions of priority heuristic. These data add to growing case against this model as a description.

• Individual data are mostly transitive.