1

Non-Response

Non-response is the failure to obtain survey measures on a sample unitNon-response increases the potential for non-response bias.It occurs after the sampling step of a surveyNon-response rates can help us assess the ACCURACY of our survey statistics

2

Non-Response Measurement

The survey community has not developed a standard approach in measuring non-response

Statisticians are working together to develop a core set of unit non-response measures• AAPOR - American Association for Public

Opinion Research

• IHSNG - Interagency Household Survey Non-Response Group

3

Types of Non-Response in Household Censuses or Surveys

No one contacted

Refusal

No one home

Temporarily Absent

Language Problem

Other Reason

4

Core Set of Non-Response Measures (IHSNG)

Initial non-response ratesThe initial non-interview rateThe initial non-interview language problem rateThe initial non-interview temporary absence rateThe initial non-interview not at home rateThe initial non-interview refusal rateThe initial non-interview other reasons rate

5

Variables Needed to Compute Core Non-Response Rates

NEU: Number of Eligible units

NINT: Number of Interviewed Units

NENIU: Number of Eligible Non-Interviewed Units

NEUE: Number of Eligible non-interviewed Units Excluding refusals, language problems, no one at home, temporarily absent

NREF: Number of Eligible Units Refusing to be Interviewed.

NEUNH: Number of Eligible Units not interviewed due to No One Home

NEUTA: Number of eligible Units never interviewed because occupants are away during the field interviewing period.

NEUL: Number of eligible Units not interviewed due to language problems

6

Non-Response Rates

All rates include the NEU (Number of Eligible Units) as the denominator

INR: Initial Non-interview Rate• Definition: Combination of eligible interviewing

units that were not interviewed due to language problems, refusal, no one home, temporarily absent or other reasons divided by the total number of eligible interviewing units.

• Computation: (NENIU/ NEU)*100

7

Field Data Requirements

Master address list, field updatesAlso good to measure characteristics of interview and non-interview by looking at enumerator field notes, enumerator checklistEnd goal would be to have a coded list of characteristics for non-interview and maybe interviewed unitsHaving the enumerator track his/ her experience can be a good way to find out ways to make contact and conduct an interview

8

Post Enumeration Survey (PES)

Post enumeration surveys (PES) are an accepted census self-evaluation tool. The PES should be a independent survey that replicates a census, and occurs three weeks after census enumeration. The PES and census results are then compared during the matching phase of the PES.The PES Evaluates:• Coverage of persons, that is the extent of the undercount of individuals• Coverage of dwellings, that is where a separate sample of dwellings

was used to compare dwellings listed by PES interviewers with those compiled by census collectors, and

• Accuracy of responses to particular census questions.PES results can be used to adjust Census results • Under or over-counts can be converted into adjustment factors and the

census population increased or decreased accordingly by these factors.PES does not allow for intervention during field operations.• An advantage of taking good field records, updating master address

database is that you can calculate a unit non-response measure and assess areas that need intervention.

9

Importance of the Enumerator

The enumerator is the link between the survey organization and the householder

Enumerators contact the respondent, communicate the purpose of the survey, the importance of participation, the confidentiality policy

Enumerator communicates the observations to the survey organization

10

Enumerator Characteristics

Socio-demographic characteristics:• In areas where linguistic

or cultural barriers might limit interaction between enumerators and respondents, select enumerators that are

“like” the respondents.

Confidence:• Enumerators confidence

can have a significant effect on survey participation

Enumerator Experience:• Studies indicate that

enumerator experience can have a significant effect on survey participation

11

Improving Contact and Cooperation

Develop a contact and cooperation strategy:• Field workers should study at-home patterns to determine

times of day when sample households are occupied

• What are the social environment predictors to contact and cooperation?

• Arm enumerator with communication skills to remain flexible and positive during introduction and in refusal cases

• Provide enumerator with a short checklist to complete after each interview

• Determine best way to leave message with potential respondent

• Written notice, visit to neighbor?

12

Enumerator Messages

Survey or Census Institution• There have been several studies

to determine if the organization sponsor affects the respondents decision

• In the US, the overwhelming majority of these studies have clearly demonstrated that government or university sponsorship is the most effective in capturing respondents.

Standardization• Studies indicate that

enumerator behavior should be standardized during the questionnaire

• HOWEVER, standardization of interviewer behavior during the introductory interaction with sample householders is ineffective

13

Enumerator Messages

Topics of Study

• Differential response effects are found for different topics studied, regardless of the conditions of the interview.

• KNOW THE SURVEY

Anonymity and Confidentiality

• Some studies have shown that response rate is affected by the anonymity/ confidentiality policy of a study. Others have reported that responses became more distorted when subjects felt threatened that their identities would become known. Others have found that anonymity and confidentiality issues do not affect response rates or responses.

14

Publicity and Community Liaison Strategy

Increasing public understanding by advance notice of survey or census will help improve response rates, and higher quality data will be achieved.

Develop content of key Census/ Survey information explaining the dates, general process, confidentiality, topic, importance, sponsorship

The NSO (field operations or media office) will coordinate with partner organizations to deliver the message in the most appropriate way for the community they are targeting

Media, publicity, community organizations contact data base should be created to facilitate communication with potential publicity partners.

Regular, scheduled information kits should be delivered to contacts so they are aware of operational changes

15

Hard to Reach Populations

Partnerships with community groups• Field Operations should have a data base/

list of community groups that work with hard to reach populations (nomads, homeless, orphans).

• Recruitment of enumerators from these hard to reach populations

• Enumeration in a single night

16

Practical implications for Census and Survey Implementation

For National Statistical Offices:• Non-response measurement is possible with minimal data and can help

improve the methodologies used

• Focus on developing training curriculum that emphasizes contact, communication and cooperation training. What are the budget implications?

• Importance of budgeting for high quality, pre-tested training instruments

For Field Operations• Enumerator training should develop communications skills, strategies

for tailoring and maintaining interaction

• Use of a checklist for enumerators

• Importance of a Master Address Database, and the listing operation so you can have the number of eligible units

• Importance of recording case characteristics, interview and non-interview alike