Oliver Serfling, Department for Statistics & Econometrics, WWZ, Uni-Basel Rounding Behavior of Respondents in Household Surveys Dr. des. Oliver Serfling

Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel Rounding Behavior

of Respondents in Household Surveys

Dr. des. Oliver SerflingUniversity of BaselPresentation November 11, 2005Swiss Statistical Meeting, Zürich

November 11, 2005 Swiss Statistical Meeting, Zürich 2

Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Agenda

Types of Survey Measurement Errors

The Rounding Phenomenon Theoretical Issues & Literature Research Goals Literature on rounding behavior Our Data: SHP Empirical Strategy Rounding Patterns Conclusion

Survey

Rounding

Response

Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Introduction & Motivation


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Types of Survey Measurement Errors

INRItem

Nonresponse

MRE:Misreporting

Error

MME:Measurement

Error

MCE:Misclassification

error

Generally, measurement error occur if the reported value (Z) is not identical with the „true“ value (X):

True value X is not reported, Z=?

Continuous X is reported with error as continous Z:Z=X+

Continuous X is reported as a discrete interval with midpoint Z where X lies in Rounding

Discrete X is reported as wrong but discrete Z

XZ


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

The Rounding Phenomenon

Rounding as a data coarsening: Loss of information and data quality Small changes in the variable become unobservable

Problem for sensitivtiy analysis Variance is upward biased

Rounding as a response phenomenon: Rounding may indicate motivation of respondent.

Therefore, it may be a precursor of item or unit nonresponse

Rounding may be a strategy of the respondent to avoid/reduce disclosure of privacy


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Literature: Rounding as coarsening

Sheppard (1898): Examines grouping effects on normal distribution

Effect on mean is negligible Variance is upward biased by 1/12w with w=rounding interval

Sheppards correction: calculate unbiased estimator of variance

Eisenhart (1947): analyzes the effects of rounding with different sample sizes

Tricker (1984): analyzes rounding on non-symmetrical dist.: gamma, log-

normal Rounding error in mean and variance is positively related to

skewness of distribution and rounding degree


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Three types of rounding

Presented literature deals only with same rounding behavior on every observed value

... but in survey interviews every respondent may have its own degree of rounding, which can be: at random or systematic

Under the assumption that respondents round correctly:(A1)

And the rounding error is uniformly distributed in the rounding interval:(A2) e ~ U[-w/2 ; w/2]

3 types of rounded data can be distinguished:(R1) every value is rounded to same degree of rounding (w):

Z = X + e with e ~ U[-w/2 ; w/2](R2) degree of rounding (w) differs over individuals (i):

Z = X + e with e ~ U[-wi/2 ; wi/2](R3) degree of rounding (w) is a function of X:

Z = X + e with e ~ U[-w(X)/2 ; w(X)/2]

1)( 22 ww ZXZP


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

R1 effects on distribution

Simulated right-skewed distribution of „money“ amounts


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel


Simulated distribution of „money“ amounts rounded to 10s


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel




Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel




Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel


Simulated distribution, individual rounding intensity at random


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel


Simulated distribution, rounding intensity dependent on absolute value


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

R1-R3 effects on moments

Deviance (%) of rounded moments from their population counterpart

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

-2 -1 0 1 2 3

d, Rounded to 1E(+d) units

dev

ian

ce [

%]

mean (R1)

variance (R1)

skewness (R1)

kurtosis (R1)

mean (R2)

variance (R2)

mean (R3)

variance (R3)


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Research goals

Q1.) Find an appropriate rounding intensity measure

Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior

Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? Characteristics of respondent (Respondent Effects) Person of the interviewer (Interviewer Effects) Interview type and interview situation (Situation Effects)

Q4.) Is the degree of rounding driven by the value of concerned variable?

Q5.) Is there a panel duration effect?

Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Results from literature

Rounding as respondent behavior


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Literature: Rounding as resp. behav.

Schweitzer, Severance-Lossin (1996): 71% of all reported earnings in CPS (Current Population

Survey) March 1994 are multiples of $1,000

Rounding behavior is highly systematic and correlated with respondents‘ earnings level

Systematic nature substantially affects some common used measures on earnings data: Inequaltity summary measures (Gini-coefficient) Earnings quantiles Kernel density estimates

In particular, statistics are sometimes altered at levels of annual change and/or standard errors.


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel


Schräpler (1999): Data: Gross income question of waves 1-12 of GSOEP Roundings to 100, 500, 1000 in 67-77% of income statements Method: Multinomial Logit estimation

categories of dependent var: exact, 10, 100, 500/1000 Results:

Sex: Men have higher rounding propensity (5-7% higher probability of choosing 500/1000; Female interviewers provoque extreme rounding intensities (exactness and 500/1000 rounding). Male I‘s provoque middle rounding intensity.

Age of respondent and precision of statement seem to be correlated

Interview duration: positively correlated with presicion – it takes time to provide exact values

Interview mode: in self administered quest. low rounding, higher in face-to-face interviews

Experience: of respondents with interview provoques rounding Income: low roundings in first quartile, high in fourth quartile


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel


Hanisch (2003): Data: Finish sample of ECHP Roundings after 1 or 2 significant digits:

80% of gross wage statement 95% of net disposable income question

Method: ordered probit on number of significant digits Results:

Sex: males provide higher precision (scandinavian artifact) Foreigners have lower roundings Interview mode: CAPI leads to highest precision, longer

interview duration produced more precision Job effects: some professions are more precise than others Panel participation does not have a monotone effect on

rounding behavior.


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel


Kroh (2004): analyses interview effects on rounding with self-reported

body weight Data: body weight of GSOEP 2002 Method: Binary Probit on the event of rounded weight

statement Results:

Sex: Women provide rounded weights more often Lower educated interviewees and singles provide

rounded weights more frequently Overweighted people tend to stronger roundings!

Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Our Data

The Swiss Household Panel


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

The Swiss Houeshold Panel (SHP)

SHP is an annually collected comprehensive survey Comprises information on:

housing, living standard, income and ist components socio-demographics, education, employment, politics, values, and leisure.

Three separate questionnaires: grid personal household

Personal questionnaire has to be answered by every household-member who reached the age of 14

SHP is completely surveyed by CATI (Computer Assisted Telephone Interviews)

Sample size: 7,799 persons (1999) to 5,220 (2003), (refresh: 2004)


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

SHP Interviewer Survey

Additionally, in second wave (2000): survey of the interviewers with 24 questions on: Socio-demographics Interviewer experience and occupation Opinions towards the survey

From 53 interviewers worked for SHP in 2000: 45 participated 41 filled in questionnaire completely

No information on interviewers in 1999, and 2001-2003 Therefore, missing interviewer information on

1,211 out of 7,799 cases in 1999 approx. 700 cases in 2001, 2002 and 2003

Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Own analysis


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Research goals revisited

Q1.) Find an appropriate rounding intensity measure

Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior

Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? Characteristics of respondent (Respondent Effects) Person of the interviewer (Interviewer Effects) Interview type and interview situation (Situation Effects)

Q4.) Is the degree of rounding driven by the value of concerned variable?

Q5.) Is there a panel duration effect?


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Rounding Decision Model

Hypothesis: The respondent is free to decide about his rounding intensity

(RI) … which is determined by the costs and benefits of precision:

i.e. cognitive burden, disclosure of privacy The respondent chooses the RI which maximizes his utility:

If the cost and benefit components are attributed to the characteristics of the respondent, his interviewer and the interactions thereof, the latent rounding intensity (RI*) is:

With: αt =baseline cost-surplus in answering the question at time t, R it are the characteristics of the respondent i, Ij are the characteristics of the interviewer j, (R*I) are the interaction of both and εit is white noise

0)()( !

RI

RICosts

RI

RIBenefit

itjtitjtittit IRIRRI 321 )*(*


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Rounding measures

Which measure reflects the latent rounding intensity? NRD: Number of rounded digits

(discrete absolute measure)

NSD: Number of significant digits (discrete absolute measure)

RQ: Rounding–Quotient = rounding digit / number of digits (discrete relative measure)

RSM: Rounding strain measure = NRD-(NSD-1)

Relative rounding error (%)(continous relative measure)


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Empirical strategy

Regression of rounding measure on possible determinants: Respondent characteristics: sex, age, education, employment

status, satisfaction, health status, language, experience, nationality

Interviewer characteristics and interview experience Interviewer-Respondent interactions Interview situation effects: panel duration The value of rounded variable, log amount-splines, higher

polynomials of variables value

Using: Ordered Probit model

with a set of fully interacted covariates (RHS Var * NoD-dummies)

Dependent variable: Number of Rounded Digits for the first income statement in

the SHP questionnaire


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Correlation Rounding <-> Nonresponse

Nonresponse Rounding Measures UNRt INRt DKt ESTt dt RSMt NRDt NSDt INRt-1 0.06 0.43 0.05 0.03 0.02 0.03 0.03 DKt-1 0.07 0.08 EST t-1 0.04 0.05 0.07 0.04 0.03 -0.04 d t-1 0.04 0.21 0.16 0.19 -0.05 RSM t-1 0.04 0.02 0.05 0.13 0.31 0.32 -0.21 NRD t-1 0.04 0.03 0.16 0.30 0.43 -0.16 NSD t-1 -0.01 -0.02 -0.03 -0.04 -0.22 -0.17 0.29 INR t-2 0.02 0.42 0.07 0.02 0.02 DK t-2 0.02 0.12 EST t-2 0.06 0.04 0.03 -0.03 d t-2 0.03 0.03 0.02 0.14 0.13 0.16 -0.04 RSM t-2 -0.03 0.04 0.04 0.04 0.11 0.27 0.28 -0.16 NRD t-2 -0.03 0.04 0.03 0.03 0.15 0.26 0.36 -0.12 NSD t-2 -0.02 -0.03 -0.03 -0.18 -0.14 0.23

large autocorrelations of rounding measuressmall positive correlation of rounding with Item-Nonresponse


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Respondent Effects

… on Rounding Intensity (NRD):

0 immigrants

+males (mean number of rounded digits by +0.02 digits)

+ tertiary education

+good / very good health status

- french speaking resp.

+/-Age/Age2: concave with max. at 38 yrs


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Interviewer Effects

Weak but significant effects, since SHP is conducted via CATI (telephone interviews)

+/-

Experience: Convex effect with min. at 2.3 yrs.

0Int.: no income provision

0Int. would not participate

0 Age

0 Mother tongue

No significant Interviewer-Respondent Interaction / Social Distance effects!


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

NoD or Income Effect?

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0 10 100 1'000 10'000 100'000 1'000'000

amount (logarithmic scale)

mea

n(NR

D)

Model is augmented with log-income splines for 2,3,5, and 6 digits (4 digits as reference)(robustness check: estimation of 5th order income polynomial)

We find different slopes of the income effect by NoDwith a negative effect for 6-digit incomes

no log-linear income effector additional NoD-Effect


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Conclusion

Rounding in income data of the SHP is a rule, rather than an exception

Rounding intensity differs over respondents

There are robust patterns of influences on rounding behavior by respondents characteristics, interviewers characteristics, but non for interviewer-respondents interactions

Rounding intensity is also driven by the amount of

considered variable, but its magnitude seems to be relatively decreasing


Oli

ver

Ser

flin

g,

Dep

artm

ent

for

Sta

tist

ics

& E

con

om

etri

cs,

WW

Z,

Un

i-B

asel

Thank you for your attention !

The End

Paper will soon be available at:http://www.wwz.unibas.ch/stat/team/serfling

Documents

Oliver Serfling, Department for Statistics & Econometrics, WWZ, Uni-Basel Rounding Behavior of Respondents in Household Surveys Dr. des. Oliver Serfling