Want to be an Early Bird? Can encouraging respondents to contact interviewers to make appointments

increase co-operation and save costs?

Matt Brown and Lisa CalderwoodCentre for Longitudinal Studies

Sub-brand to go here

CLS is an ESRC Resource Centre based at the Institute of Education

GSS Methodology Symposium – 27th June

Context

• Survey costs increasing (Stoop, 2005)

• Increased focus on cost-effectiveness

• Main element of survey costs is fieldwork

• Experiment conducted to attempt to reduce fieldwork costs by encouraging respondents to initiate contact with interviewers to arrange appointments

Background

• “Early Bird” innovation pioneered by National Longitudinal Studies – 1979 cohort

• Respondents sent letter 2 weeks prior to fieldwork inviting them to call free telephone number to arrange appointment for interview.

• W22 (2004): $60/$80 incentive paid if telephoned within 4 weeks of receiving letter (+ standard incentive ($40) for completing interview).

• 49% took up offer• Some impact on response rates:

• 80% overall• 83% amongst those offered Early Bird

• Big impact on fieldwork effort:• 3 hours to complete interviewing for Early Birds• 5 hours to complete interviewing for ‘non’ Early Birds

Research questions

• Can this approach be successful on longitudinal studies in the UK context?– Incentives typical in household panel surveys but usually much lower

value than US.

• Can sample members be motivated to be ‘early-birds’ without a financial incentive?– Incentives unusual in cohort studies in the UK– Appeal to ‘helping’ tendencies (Groves, Cialdini and Couper, 1992)– Increasingly consumer-drive, time-poor society

UKHLS

• Understanding Society: UK Household Longitudinal Study (UKHLS)• 40,000 households

• Experiment conducted on the ‘Innovation Panel’ – 1500 households • Develop and evaluate methodologies for longitudinal data collection• Open call for proposals to carry out experimental designs in a longitudinal

context • A unique resource for methodological research.

Experiment design

GroupTreatment N

EB offer with incentive Additional £5 incentive paid to all household members who

completed an interview (standard incentive £5 or £10)347

EB offer – no incentive Appeal for help 366

Control – no EB offer No request to contact interviewer in advance 375

Implementation

• Two treatment groups sent letter three weeks before fieldwork– Next wave of study about to begin– “Opportunity to request an Early Bird Appointment by contacting

interviewer on their mobile phone to arrange your interview at a time that suits you”.

• Also sent a leaflet “Want to be an Early Bird?” which explained the offer (identical other than mention of incentive or appeal)

• Given two weeks to contact interviewer to book an appointment for any date within first 4 weeks of data collection. (Interviewers all issued with mobile phones).

• Control group just sent letter

Results

• Take-up of offer

• Impact on response rate

• Impact on fieldwork efficiency

Results – take-up of offer

Contacted interviewer

Made and kept appointment

N % N % Base

EB Offer – Incentive 40 11.5 36 10.4 347

EB Offer – No incentive 24 6.6 21 5.7 366

Any Early Bird Offer 64 9.0 57 8.0 713

Results – Response Rates

Productive interviews

N % Base

EB Offer

EB Offer – Incentive 266 76.7 347

EB Offer – No incentive 285 77.9 366

Any Early Bird Offer 551 77.2 713

Control group – No EB 274 73.1 375

Results – Impact on fieldwork effort

Mean Max St. Dev N

EB Offer taken up

EB Offer – incentive 1.37 4 0.73 36

EB Offer – no incentive 1.45 5 1.0 21

All taking up offer 1.40 5 0.83 57

EB Offer not taken up

EB offered –incentive 3.49 14 2.29 231

EB offered – no incentive 3.41 21 2.04 265

EB not offered 3.44 13 2.04 274

All not taking up offer 3.45 21 2.12 770

All 3.31 21 2.12 825

Number of interviewer visits to complete all interviewing

Results – Impact on fieldwork effort

Mean St. Dev N

EB Offer

EB Offer – incentive 3.10 2.29 347

EB Offer – no incentive 3.21 2.15 366

Any EB offer 3.16 2.22 713

Control group – no EB offer 3.37 2.22 375

Total Number of Interviewer Visits – All issued households

Early Bird Characteristics

Early Birds Non-Early Birds

Sig.

Individual

Average age 58.0 48.8 ***

Sex (% Female) 74.1 52.0 ***

Economic activity status (% Retired) 41.4 25.0 **

Household

Average size 2.2 2.5 -

Children in home (<15) (%) 18.2 23.6 -

13

• Low take-up rates:– Small incentive – in absolute terms?– Small increase in incentive relative to standard incentive?– Poor marketing? Emphasis on the term ‘Early-Bird’?– Materials not read– Mode effects – Face to face vs telephone?– Panel loyalty?

• Take-up rate significantly higher if incentive offered– Appeal to ‘helping tendencies’ unsuccessful?– More emphasis on how beneficial to the respondent?

14

Summary and Conclusions

15

Summary and Conclusions

• When taken up EB leads to big reduction in interviewer visits needed to fully complete a case (as per NLSY)

• Low take-up means little impact on overall fieldwork effort

• Need to boost take-up rates• Higher incentive rates?• Better marketing of the EB offer?

Groves, R.M., Cialdini, R.B. and Couper, M.P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475-495

Stoop, I. A. L. (2005). The Hunt for the Last Respondent: Nonresponse in sample surveys. The Hague: Social and Cultural Planning Office.

 

16

References