On the Heterogeneity, Stability and Validity of Risk Preference Measures
Chetan
Dave
Catherine
Eckel
Cathleen
Johnson
Christian
RojasUniversity of Texas
at DallasUniversity of Texas
at DallasUniversity of
ArizonaUniversity of
Massachusetts (Amherst)
ESA, July, 2007
Measuring Preferences: Key Questions
Are risk preferences stable across elicitation measures? Since Slovic (1964), considerable evidence they are not
Do risk preferences vary systematically by subject characteristics in the same way across measures? (Gender, age, income)
Does math ability affect elicited preferences? Ability: Low/Medium/High Math Literacy
Are risk preferences stable across time? Two Datasets: Test Data vs. Re-Test Data
Motivation
Lots of folks are working on heterogeneity in risk preferences (many of them are here)
There are lots of elicitation tasks, depending on the purpose of the study
Our purpose: to develop a simple, easy to administer measure of individual risk preferences that can be used to “predict” or better understand individual choices. Household decisions, human capital choices, technology
adoption
Motivation, cont’d
We compare two lottery choice methods (more on these soon)
Non-student data set (collected for another purpose)
Subjects complete both risk elicitation measures
Survey, math literacy measure Subset of pre-selected subjects participated
in a retest
The CSL Study
Purpose: study barriers to post-secondary acquisition of human capital
Data: lab experiments with non-traditional subject pool ~900 adult participants in Canada (recruited, volunteers) 102 sessions Ages 18-54 years Urban and Non-Urban samples 156 participate in retest 6 months later
Average earnings: CAD$165 Subjects paid for one decision chosen at random 100 decisions: 40 time pref; 32 risk and ambiguity; 28
financing for post-secondary education, max = $1000 grant, $5000 guaranteed loan
The CLS Study (cont’d.)
Two experiments to elicit risk preferences: Holt-Laury (2002)
10 binary choices between more and less risky gambles
Eckel-Grossman (2002) Choice of 1 from set of 6 50/50 gambles
Discrete binary response data 0 = less risky choice 1 = more risky choice
Holt-Laury Task
OPTION A Your Choice A or B
OPTION B
Decision 42 1/10 of $40 9/10 of $32 A 1/10 of $77 9/10 of $2
Decision 43 2/10 of $40 8/10 of $32 A 2/10 of $77 8/10 of $2
Decision 44 3/10 of $40 7/10 of $32 A 3/10 of $77 7/10 of $2
Decision 45 4/10 of $40 6/10 of $32 A 4/10 of $77 6/10 of $2
Decision 46 5/10 of $40 5/10 of $32 B 5/10 of $77 5/10 of $2
Decision 47 6/10 of $40 4/10 of $32 B 6/10 of $77 4/10 of $2
Decision 48 7/10 of $40 3/10 of $32 B 7/10 of $77 3/10 of $2
Decision 49 8/10 of $40 2/10 of $32 B 8/10 of $77 2/10 of $2
Decision 50 9/10 of $40 1/10 of $32 B 9/10 of $77 1/10 of $2
Decision 51 10/10 of $40 0/10 of $32 B 10/10 of $77 0/10 of $2
Eckel-Grossman Task
x
Short Detour
FAQ: How do you code Eckel-Grossman as a set of binary choices?
Notice the gambles are ordered Increase linearly in risk and return as you
go down the table One choice implies binary ordering
Figure 2: Risk and Return of Gamble Choices
25
27
29
31
33
35
37
0 10 20 30 40
Risk (Standard Deviation)
Exp
ecte
d R
etu
rn
1
65
4
3
2This person chooses Gamble 3This implies 5 binary choices:
2 is preferred to 1
3 is preferred to 2
3 is preferred to 4
4 is preferred to 5
5 is preferred to 6
Comparison of Eckel-Grossman and Holt-Laury
H-L: varying probabilities, constant payoffs
E-G: constant probability, varying payoffs, all 50-50 gambles
Cumulative Distribution of r values
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
-2 -1 0 1 2 3 4
r value
Per
cen
t
Holt/Laury
Eckel/Grossman
The Numeracy Measure
Subcomponent of the Educational Testing Service’s Adult Literacy and Lifeskills Survey (ALLS)
31 problems involving the use of mathematics in real-life situations.
Low numeracy ≤ 1 standard dev. below average
High numeracy ≥ 1 standard dev. above average
Model and Estimation
Given a CRRA utility function for money (M) parameterized by r
U(M|r) = M1-r/(1-r) Can construct a likelihood function
L(r,,|Y) where measures ‘noise’ in binary risk
choices (Y) r and can also be made a function of
observed characteristics of subjects
Results 1: Does the Elicitation Instrument Affect Estimates?
Estimates of CRRA Utility Function
Estimate Std. Err. p-value Estimate Std. Err. p-value
r 0.449 0.024 0.000 0.626 0.020 0.000 0.044 0.002 0.000 0.122 0.007 0.000
Eckel-Grossman Task Holt-Laury Task
r = Coefficient of relative risk aversion
= Logit error parameter
Differences significant at p = 0.000, 2(1) = 52.62
Estimates of CRRA Utility (cont’d.)
Eckel-Grossman and Holt-Laury give significantly different estimates of r
“Ruler” affects measure! H-L has higher error parameter than E-G
Why?
Results II: How do demographics and ability affect estimates ?
IIA: Effect on preference parameters
IIB: Effect on parameters and errors
IIC: Are parameters stable across instruments?
Effect on preference parametersEstimateStd. Err.p-value
r Instrument (H/ L) 0.219 0.038 0.000 Consistent (44% are) 0.026 0.023 0.265 Female 0.007 0.025 0.782 Low Income (<$CA30K) 0.037 0.029 0.202 Medium Income ($CA30-65K) 0.045 0.027 0.098 Young (<25) -0.100 0.031 0.001 Middle (25-44) -0.055 0.025 0.025 Low Math Score (-1 std dev) 0.199 0.041 0.000 Medium Math Score 0.041 0.040 0.305 Constant 0.461 0.043 0.000 0.080 0.007 0.000
Effect on Preference Parameters (cont’d.)
Income, Age and Math Literacy are significant determinants (correlates) of preferences
Low Income: more risk averse Young: less risk averse Low math score: appear more risk averse
Could this be due to greater errors?
Effect on Parameters and Errors
EstimateStd. Err.p-valuer Instrument 0.107 0.053 0.041 Consistent 0.093 0.038 0.015 Female 0.128 0.034 0.000 Low Income 0.011 0.039 0.767 Medium Income 0.046 0.052 0.380 Young -0.030 0.039 0.445 Middle -0.047 0.045 0.298 Low Math Score 0.133 0.053 0.012 Medium Math Score 0.035 0.037 0.336 Constant 0.388 0.055 0.000
Effect on Parameters and Errors (cont’d)
Estimate Std. Err. p-value
Instrument 0.058 0.008 0.000 Consistent -0.017 0.006 0.008 Female -0.028 0.009 0.002 Low Income 0.006 0.007 0.377 Medium Income -0.001 0.013 0.937 Young -0.034 0.011 0.002 Middle -0.024 0.015 0.123 Low Math Score 0.039 0.022 0.076 Medium Math Score -0.005 0.008 0.520 Constant 0.090 0.015 0.000
IIB. Parameters and Errors (cont’d.)
People who are consistent are more risk averse and have less noise (randomness)
Young have less noise Lower numerical literacy increases
randomness in risk parameters; appear more risk averse
Females are more risk averse and have lower noise
IIIC: Parameter stability across measures
Is the impact of gender, age, income similar across measures?
Is the impact of math literacy similar across measures?
Separate regressions for EG and HL instruments E/ G H/ L Estimate p-value Estimate p-value r Consistent -0.061 0.543 0.175 0.000 Female 0.364 0.000 0.099 0.009 Low Income -0.015 0.864 0.018 0.670 Medium Income -0.014 0.870 0.049 0.187 Young (<25) 0.072 0.367 -0.017 0.752 Middle (25-44) 0.054 0.655 -0.066 0.233 Low Math Score -0.065 0.668 -0.049 0.500 Medium Math Score -0.188 0.190 -0.019 0.642 Constant 0.481 0.000 0.529 0.000 Consistent -0.009 0.327 -0.031 0.003 Female -0.025 0.000 -0.033 0.008 Low Income 0.008 0.380 0.005 0.690 Medium Income 0.011 0.157 -0.006 0.614 Young (<25) -0.026 0.015 -0.045 0.031 Middle (25-44) -0.027 0.001 -0.015 0.489 Low Math Score 0.022 0.271 0.126 0.001 Medium Math Score -0.006 0.594 0.020 0.084 Constant 0.072 0.000 0.136 0.000
Distribution of noise parameter0
.2.4
.6.8
1
0 .1 .2 .3Linear prediction
cum_mu_HL cum_mu_HL_lowcum_mu_HL_med cum_mu_HL_highcum_mu_EG
How good are predictions?
Calculate index:
n is the number of decisions for the task (5 for Eckel-Grossman and 10 for Holt-Laury)
‘fraction’ variables represent the true and the predicted fraction of safe choices for decision i across all individuals.
Holt-Laury task has indices of 0.79, 0.88 and 0.92 for low, medium and high math literacy subjects
Eckel-Grossman instrument has a precision index of 0.81.
1
Precision 1 1/ Actual Fraction Predicted Fractionn
i iin
IIC: Comparison
Women are substantially more risk averse in E/G; lower noise in both
Young have lower noise in both Low math literacy affects noise in H/L only Noise parameter is similar to E/G for H/L
high literacy, but higher for med and especially low literacy
“Precision” is equivalent for high literacy, not for Medium or Low.
Results III: Are Preferences Stable Across Time?
IIIA. Eckel-Grossman
IIIB. Holt-Laury
IIIA. Stability of Eckel-Grossman
EG InstrumentEstimat
eStd. Err.
p-value
r Re-test 0.080 0.090 0.377Constant 0.432 0.061 0.000 Re-test -0.032 0.008 0.687Constant 0.052 0.005 0.000
LogL: 920.631# obs 1560
IIIB. Stability of Holt-Laury
HL Instrument
EstimateStd. Err.
p-valu
er Re-test 0.118 0.080 0.141Consta
nt 0.707 0.055 0.000 Re-test -0.044 0.028 0.122Consta
nt 0.112 0.020 0.000
LogL: 1182.848
# obs 3,120
Concluding Remarks
Question # 1: Are preferences stable across measures used to elicit them? Answer: A qualified No. Elicitation
instruments do matter to an extent Question # 2: Do preference
estimates by subject characteristics vary across measures? Answer for Demographics: Yes
Concluding Remarks (cont’d.)
Question #3: Are preference measures sensitive to math literacy: Answer: Yes, H/L only
Question # 4: Are preference measures stable across time? Answer: Yes. But E-G does “better”
than H-L Question #A1: Internal v. external
validity?
Validity: Correlations of risk measures with “real” decisions.
Hs : 80 obs Eckel/ Grossman Holt/Laury
Preference for $1000 grant - full time 0.21 0.14
Preference for $1000 grant - Part time 0.18 0.14
At least one loan preferred over cash 0.21 0.12At least one grant preferred over cash 0.08 0.15Not HS : 801 obs
Preference for $1000 grant - full time 0.04 0.01
Preference for $1000 grant - Part time 0.10 0.00
At least one loan preferred over cash 0.05 0.06
At least one grant preferred over cash 0.08 -0.02
Measuring preferences
Need to develop instruments that are appropriate for population, purpose
Finer screen v. accessibility tradeoff For literate population, no tradeoff For medium and low literate population, there
may be a serious tradeoff! Considerable precision is lost with more
complex instrument Important to take care with design for low-
literacy populations
Ex: Next gen of instruments (risk)