Creating Honesty: The effect of requesting telephone numbers and reporting on self or others to identify honest responders

Please tweet! @ESRAsurvey @LoveStats

Please tweet! @LoveStats #ESRAsurvey

Call Me Maybe: Effects of asking for telephone numbers on survey honesty

Presented by Annie PettitChief Research Officer at Peanut Labs


Research Questions

Should you ask at the beginning or the end of the survey?

When To Ask

Will people endorse more socially desirable, charitable behaviours?

Will people endorse fewer illegal, unethical, and embarrassing behaviours?

Social Desirability Illegal BehavioursCBA


MethodGroup 1•Beginning•Phone number request was first question•N=500

Group 2•End•Phone number request was last question•N=500

Non-probability, incentivized research panelSampled and weighted to US census targets


Question Categories

Compliment employee

Bought someone

coffee

Kind Charitable

Pee in shower

Embarrassing

Called in sick

Unethical

Listen to music

Neutral

Impaired driving

Illegal


RESULTSBEGINNING VS END


Neutral% of People Endorsing Neutral Behaviours


NeutralIt doesn’t matter if you were asked at the beginning or the end, endorsement rates for neutral behaviours were the same.

Average Endorsement

Standard Deviation


CharitableEndorsement of charitable behaviours didn’t change if you were asked at the beginning or the end.


Embarrassing Endorsement of embarrassing behaviours didn’t change if you were asked at the beginning or the end.


IllegalThough there was more variability, endorsement of illegal behaviours didn’t change if you were asked at the beginning or the end.


ormakes no Beginning

differenceend

CONC

LUSI

ON


RESULTSAGREE VS DECLINE


Method

Group 1•Yes, here’s my phone number•555-5555

Group 2•No! You can’t have my phone number•BUG-OFFF


IllegalPeople who agree endorse more illegal behaviours


EmbarrassingPeople who agree endorse more embarrassing behaviours


CharitablePeople who agree endorse more charitable behaviours


NeutralPeople who agree endorse more neutral behaviours


ormakes aAgreeing

differencedisagreeing

CONC

LUSI

ON


RESULTSINTERACTIONS


MethodGroup 1

BeginningDecline

Group 2EndDecline

Group 3Beginning

Agree

Group 4End Agree


Neutral QuestionsEndorsement Rates Not Affected


Agreers Are Confessors

Illegal Unethical Embarrass Kind Charitable Neutral Average


theandlisten

If

agreersbeginning

youwanthonestyaskattoCO

NCLU

SION


Many new questions• Type of PII: What if we asked for email or physical address instead?

• Position of Ask: What if we ask half-way through? After an embarrassing question?

• Survey Style: What if the entire survey is embarrassing?

• Screen Outs: What if everyone was told there would be no screen-outs?


Thank you!Annie Pettit

Chief Research [email protected]

ca.linkedin.com/in/AnniePettit/facebook.com/AnniePettit

twitter.com/LoveStats

Jonathan CheriffDirector of Sales & [email protected]

Find PeanutLabs on LinkedIn Facebook Twitter YouTube

DIY SURVEY PROGRAMMING DIY POLLINGDIY SAMPLE

Data & Analytics

Creating Honesty: The effect of requesting telephone numbers and reporting on self or others to identify honest responders