Improving Response Rates

Improving Response RatesThom Allen

SESRC at Washington State University

Response Rate▪ Ratio of completed surveys to the eligible starting list

Completed and Partially Completed Surveys

Entire Eligible List (sample)

▪ Not the Cooperation Rate

▪ Further Reading:▪ AAPOR (American Association for Public Opinion Research) aapor.org▪ CASRO (Council of American Survey Resource Organizations) casro.org

How To Improve Response Rates

Three Theories▪ Social Exchange Theory (Homans)

▪ Benefits must outweigh the costs.

▪ Leverage Saliency Theory (Groves)

▪ People respond to what is important to them.

▪ Total Design Method (Dillman)

▪ It is not one factor, but the whole design that affects response.

▪ Synergetic effect.

Four Critical Points▪ Delivery

▪ Opening

▪ Completing

▪ Returning

Seven Techniques▪ Prior Letters

▪ Pre-Incentive

▪ Mixed Appeals

▪ Number of Contacts

▪ Mixed Contact Modes

▪ Sequential Mixed Mode Instruments

▪ Credible Survey Sponsor

Prior Letters

November 2, 2007 «city» Area Resident«address» «unit»«city», «state» «zip»«dash»«zip4» Dear Joe Smith, I am writing to ask for your help with an important study being conducted by Washington State University to understand the quality of life of residents in the Lewiston-Clarkston area. In the next few days you will receive a request to participate in this project by answering questions about your experiences living in the area and about important issues that currently face residents here. We would like to do everything we can to make it easy and enjoyable for you to participate in the study. I am writing in advance because many people like to know ahead of time that they will be asked to fill out a questionnaire. This research can only be successful with the generous help of people like you. To say thanks, you will receive a small token of appreciation with the request to participate. I hope you will take 10-15 minutes of your time to help us. Most of all, I hope that you enjoy the questionnaire and the opportunity to voice your thoughts and opinions about the Lewiston and Clarkston area. Best Wishes,  

Don A. DillmanRegents Professor and Deputy Director

▪ People appreciate being alerted in advance.

▪ Reinforces importance of the survey task.

▪ Chance to deliver pre-incentive.

▪ Opportunity to present appeals.

Prior Letters▪ These are true prior letters;

advanced warning w/o giving ability to actually respond.

▪ Trend: Studies using true prior letters tended to have better response rates.

* Analysis of 94 mail surveys at SESRC from 2009-2011

▪ A pre-incentive is one given to everybody in the survey list.

▪ It is presented as the “task” is presented.

▪ Cash works

▪ $1, $2, $5

▪ Promised rewards, not as much

Pre-Incentive

* Analysis of 94 mail surveys at SESRC from 2009-2011

Amount of Pre-Incentive

Mixed Appeals▪ The request to complete the survey must utilize multiple appeals

▪ Various appeals might include:

▪ Benefit to respondent

▪ Participation helps society/environment

▪ Respondent has unique knowledge

▪ We are asking for help

▪ Respondent represents many others

Number of Contacts▪ Trend: Much agreement in the

research that added contacts raises response rates.

▪ Maximizing response rate on FIRST contact(s) decreases overall number (and the subsequent higher cost) of additional contacts to non-respondents.

* Analysis of 94 mail surveys at SESRC from 2009-2011

Mixed Contact Modes▪ When collecting surveys over the

web, email is not the only means of communication.

▪ Email is a poor contact mode for surveys.

▪ Mail, phone and face-t0-face are often better choices

▪ Email is highly effective as an augmentation mode.

▪ Change Subject line

▪ Email from credible sponsor

Sequential Mixed Modes▪ Choice can sometimes be bad.

▪ Sequencing modes has proven effective

▪ Web

▪ Paper

▪ Phone, In-Person, etc.

▪ Important not to mention other modes.

Putting It All Together

Thank You!Thom Allen

Study Director

Social and Economic Sciences Research Center

Washington State University

ted@wsu.edu