Stat-472Survey design

Questionnaire Design IssuesNonresponse, Refusals and Response Rate

Non-response, Refusals and Response Rate

• Response rates are a big concern in survey research, if a high percentage of non-response the researchers become cautious about generalizing from the result

• Low Response rates can create bias and weak validity

• Failure to get a response from a sampled respondent can take several forms:

(1)The respondent could not be contact(2)The respondent was unable to

complete the survey (e.g., spoke another language, had no time, was ill)

(3)Refused to complete the questionnaire or refused to be interviewed, or to answer some questions

That due to many factors:Privacy concerns: A fear of strangers and crime Social isolation An overload of surveysUnfavorable past experience:Unfavorable past experience with surveys is a major cause of non response. Legitimate survey research is impeded by misused survey techniques

Insensitive interviewers Poorly designed or written questionnaires Inadequate explanations of surveys to

respondents.Researchers disagree about what constitutes an adequate

response rate

Adequate :Is a judgment call that depends on the population, practical limitations, the topic, and the response with which specific researchers feel comfortable

adequate response rate < 50% it’s poor

adequate response rate > 90% it’s excellent

50% < adequate response rate < 75% The survey results can differ significantly from what they would be if everyone responded, because who do not respond have different views from those who do

Calculation of adequate response rate:

Telephone or F/F interviews:Calculated based on the percentage responding of the number who were located and contacted, not of the number who were sampled

Telephone: Adequate response rate is about 80%

To increase the adequate response rate:

(1) Make five callbacks before dropping a RS at different times (even the percentage doesn’t changed)(2) Send a letter 3-5 days in advance telling the RS to expect the call (the letter contains: Interviewer name, organization, general topic of the survey)

Self-administered interviews:Who were sampled and were completed questionnairePresent little problem. Adequate response rate is closed to 100%

Mail questionnaire:Adequate response rate of 10% - 50% (major problem is here)

F/F: Adequate response rate is high about 90%To increase the adequate response rate:

In advance letter or telephone call to arrange an appointment (repeated visits may be necessary, RSs may hesitate/refuse)

Survey responses from some populations, such asLow-income, inner-city minorities

Poses a special challenge

Journalistic style letterPottick and Lerman (1991) used journalistic style letter introducing the survey and a personal telephone call reminding of an interview. They compared this approach with standard method (academic-style letter and a follow-up letter), journalistic style letter approach produced a more rapid response and more RSs65% participation compared to 39% for the standard method