Understanding the Respondent – a Key to Improve our Data Collection

Strategies?

Seminar on Statistical Data Collection

26th September 2013

Anton Johansson, Statistics Sweden

2

Data collection and total survey experience

All three steps are a part of the overall survey experience for the respondent.

This presentation focus on the first step – to make contact with the respondent.

1.Contact

2.Cooperation

3.Measurement

3

How to establish contact

Groves & Couper (1998). Nonresponse in Household Surveys. John Wiley & Sons Inc, Hoboken.

Social environmental

attributes

Socio-demographic

attributes

Physical impediments

Accessible at-home patterns

Likelihood of contact

Interviewer attributes

Number of calls

Timing of callsX

XWe want to establish contact with the respondent.

If we succeed or not depends both on the respondent…

…and our data collection strategies.

4

The Respondent

• Accesible-to-contact• How old is the sample person? • Occupation? • Family situation?• Where does he or she live? Urban or rural areas?• Available telephone numbers?• Impediments to contact

• automatically rejecting calls from unknown numbers

• At what times is the individual respondent willing to answer to an incoming contact?

5

Our own contact attempts• Contact strategies

• When we call• How we select the most appropriate sample case for contact at a given

moment (randomly or by some other algorithm)

• Staffing and scheduling• When do interviewers work? • How many interviewers work at different times of the day?

• ”Technology”. • What ways of contact are available for outgoing contacts?• Landline/Mobile phone• SMS• E-mail• ”Social media”• Etc

• Do our procedures and contact strategies keep up with changing ways of communication?

6

Do we need to change our perspective?

Sample UnitIncoming

Contacts from Unknown Sources

Incoming Contacts from

Known Sources

- Telemarketing

- Survey operations

- Other

- Family

- Friends

- Work

- Other

From our perspective where we want to contact as many sample units as possible…

…to a perspective where we acknowledge that we are just one of many incoming contacts to the individual respondent?

7

What do the nonrespondents say?

A qualitative study was conducted among nonrespondents in the Labour Force Survey (LFS) and the Survey of Living Conditions (SLC). Statistics Sweden, Fjelkegård & Wallenborg, 2013

The Cognitive Unit at Statistics Sweden conducted interviews with 11 nonrespondent and asked them about their reasons not to participate.

Quotes what the nonrespondents said about the contacts that were made in the LFS and SLC:

“ I get tired of all the people calling that I don’t want to talk to. So that might affect your call as well – although it was the ten telemarketers calling before you that I was actually tired off”.

“I think it very much depends on what your situation in life is or what you’re doing on the particular day you are calling. For me there is a lot of things going on all the time”

8

EXPERIMENT IN THE LABOUR FORCE SURVEY

How to make contact?

9

About the Labour Force Survey (LFS)• Rotating Panel design

• Each survey unit is contacted eight times over two years time (every third month)

• 29 500 sample units each month• About 3700 sample new sample units each month

10

Noncontacts in the Labour Force Survey

Noncontacts per month in LFS (%). January 2001-July 2013.

11

A new strategy to contact new sample units?• Noncontact among new sample units in December

2012 • 17 % noncontacts (618 of 3 717 new sample units)• How could we reach those cases?

• Experiment with additional contact attempts between the two panel waves.

2012-12-01 2013-05-01

2012-12-03 - 2013-01-21LFS-December

First time in panel

2013-03-04 - 2013-04-17LFS-March

Second time in panel

2013-01-28 - 2013-03-04Extended contact phase for noncontacts

in LFS-December (experiment group)

Ordinary field work (wave 1) Ordinary field work (wave 2)Extended field work (wave 1)

12

About the experiment• Could we make better contact with more time and better contact attempts?

• Collect better contact information in advance for the second wave in March 2013.

• A subsample was taken from the noncontacts in LFS-December (n=309).

• Would it lead to a higher response rate in LFS-March in the experiment group?

2012-12-01 2013-05-01

2012-12-03 - 2013-01-21LFS-December

First time in panel

2013-03-04 - 2013-04-17LFS-March

Second time in panel

2013-01-28 - 2013-03-04Extended contact phase for noncontacts

in LFS-December (experiment group)

Ordinary field work (wave 1) Ordinary field work (wave 2)Extended field work (wave 1)

13

How we worked with the experiment group• Tailoring contact attempts• Interviewers used their own experience in judging

suitable times for contact for the individual sample unit• Suitable time = when the interviewer suspected that

the likelihood of contact was high• The interviewers used available information about the

sample unit • Administrative registers• Records of previous contacts

• When calls had been made earlier• What numbers had been used in those contact attempts

14

Results of the extended field work

• 256 cases were given contact attempts during the extended field work • (53 of the 309 cases were judged not worth contacting)

• Results for the contacted cases.• Contact with respondent 44%• Contact with respondent (refusal) 3 %• Contact with somebody else 27%• No contact at all 26 %

15

Results in the second panel wave (March 2013)• Significantly higher response rate in the

experiment group in LFS-March• Response rate

• Control 25%• Experiment 36%

• Explanations?• Tailoring of contact attempts – more ”care” about

each individual respondent in the experiment group• Experienced interviewers dedicated in the

experiment

16

Use the same approach in monthly production?• A big difference in managing an experiment with

256 sample units and six interviewers…• …compared to a whole sample – 29 500 sample

units per month and 250 interviewers

• How to tailor contact strategies to respondents while also considering practical implementation?

• Require strong collaboration between• IT & technology (CATI-system functionality)• Management (training of interviewers, staffing etc.)• Methods (well designed contact strategies)

17

Thinking about contact strategies as part of the survey experience

• Each sample unit represents an individual• We have to find good ways of contacting this

individual…• … or at least avoid bad contact attempts that might

cause annoyance…• …which might affect the overall survey experience.

Sample UnitIncoming

Contacts from Unknown Sources

Incoming Contacts from

Known Sources

- Telemarketing

- Survey operations

- Other

- Family

- Friends

- Work

- Other

18

TACK!