Statistics Canada Statistique Canada Cost-Efficient Framework for Data Collection for CATI Surveys Social Surveys Collection Research Steering Committee

Statistics Canada • Statistique Canada

Cost-Efficient Framework for Data Collection for CATI Surveys

Social Surveys Collection Research Steering Committee (SSCRSC)

François LaflammeDecember 2010

23-04-20Statistics Canada • Statistique Canada2

Past and current research

Paradata research has mainly directed its effort in improving the current data collection process and practices

Paradata research has also suggested that the interviewer staffing levels are not always optimally allocated with respect to the workload sample and the expected productivity


Challenges and objective

One of the main challenges is to collect cost-effective data while maintaining a high level of quality According to the forecasted budget reductions, the

pressure has increased and will continue to grow

The objective of the project is to propose a cost-efficient framework for data collection for CATI surveys


Cost-efficient framework

Five 5 dimensions:

1. Metrics used for costing and budgeting surveys

2. Resources allocation within surveys

3. Resources allocation between surveys

4. Collection process and practices

5. Operational constraints


1. Metrics used for costing and budgeting surveys

Should be based on objective, measurable and comprehensible assumptions

Should be monitored throughout the survey collectionBudget too high - low productivityBudget too low - more difficult to reach targets

Budget often based on previous survey cycles or comparable surveysBut it is exactly what we try to improve (e.g. low TPU)


2 & 3. Resources allocation within and between surveys

Within surveyInterviewer staffing levels should consider sample

workload and collection progress in taking advantage of the entire collection period

● Survey productivity decreases for all CATI surveys during collection

Between surveysMany concurrent surveys compete for data collection

resources


Resources allocation within and between surveys What is the best strategy within survey?

Examples of Resources Allocation by Data Collection Days

Data Collection Days

Ho

urs

Uniform

Proportional

Front Loading


Resources allocation within and between surveys

The main idea is to try to maintain the survey productivity for all surveys for a given RO (red line) between reasonable boundaries around the average productivity

How?

Daily Productivity, Some Social Surveys, January to March 2010Winnipeg

0%

20%

40%

60%

80%

100%

02-J

an

05-J

an

08-J

an

11-J

an

14-J

an

17-J

an

20-J

an

23-J

an

26-J

an

29-J

an

01-F

eb

04-F

eb

07-F

eb

10-F

eb

13-F

eb

16-F

eb

19-F

eb

22-F

eb

25-F

eb

28-F

eb

03-M

ar

06-M

ar

09-M

ar

12-M

ar

15-M

ar

18-M

ar

21-M

ar

24-M

ar

27-M

ar

30-M

ar

Data Collection Day

LFS LFSTFC

CCHS SLID

All Surveys Average All Surveys



The strategy is to distribute system time for a given period (day or week) in function of the inverse of the proportion of the budget spent (inversely correlated to productivity) to maintain productivity at RO level

RO Budget (hrs)Survey 1 2 3 4 5 6 7 8 9 10 11 12 13 14 … … 30

CCHS Budget CCHSLFS Budget LFSHES Budget HESCTUMS Budget CTUMS….. …..Survery X Budget survey_X

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C30

Active surveys on that dayPast collection day

Next data collection day is the 3rd

Data Collection Day

Ro Capacity - Daily Hours



In practiceNeed to get consistent budget figures for all surveysNeed to get daily or weekly capacity system timeNeed to consider the active collection period of each

surveyNeed to take into account operational constraintsIs it feasible? Can it be operationalized and automated?

The objective is to produce guidelines for RO to help them allocate collection resources only at the aggregate level.


4. Collection process and practices

Focus of the past researchSome examples

Best time to callCall scheduler features

● Cap on calls, customized time slices, Z-groups etc…Increased the proportion of evening shifts versus day

shifts to improve contact rate and productivityActive managementResponsive collection design

No significant efficiency if budget and resources allocation remain the same.


5. Operational constraints

RO capacityVaries by day (interviewer availability)LFS weekHolidays and special event

Average productivity Varies by RO and survey

Interviewer schedule in advanceInterviewer training across surveysEtc…


Comments

3 Separate projectsCosting metricsResources allocation within and between surveysCollection process and practicesAll need to take into account operational constraints

All dimensions have to be considered simultaneously to succeed in improving the cost-efficiency of data collection.


For more information, please contact

Pour plus d’information, veuillez contacter

François [email protected]

Documents

Statistics Canada Statistique Canada Cost-Efficient Framework for Data Collection for CATI Surveys Social Surveys Collection Research Steering Committee