Maintaining high quality surveys with optimized interviewers replacements : the new French sample monitoring strategy

Sébastien Faivre, INSEE, Head of the Sampling Unit

Q 2014

1. INSEE Household Surveys Sampling

2. The issue of unexpectedly missing interviewers

3. INSEE new sample monitoring strategy to deal with unexpectedly missing interviewers

4. Conclusion

3

INSEE Household Surveys Sampling

INSEE surveys are mainly face to face surveys drawn in the French Master Sample and carried out by INSEE interviewers network

Sampled dwellings are concentrated in a limited number of areas (primary units) to:

- Reduce collection costs (journey time,tracking…) - Hire a fixed network of investigators who gain experience of field operations over time

 

4

The sample frame: the French rolling census

Small municipalities (less than 10 000 inhabitants) :

building 5 random samples of municipalities (« rotation groups »), with equal probabilities

whole Census each year of all municipalities belonging to one of the rotation groups.

Big municipalities (over 10 000 inhabitants) : Building in each of them 5 samples of addresses (« rotation

groups ») from a file updated each year (RIL, register of located blocks).

Drawing each year a sample of dwellings (clusters of addresses) ; the average sample rate is about 40 % of all dwellings belonging to the current rotation group.

Census of this sample of dwellings.

5

The French New Master Sample 2009

Takes profit from the « freshness » brought by the rolling Census :

Using as a frame of a given year n + 1 all the dwellings covered by the Census at year n.

The yearly sampling frame covers only a part of the territory.

 

6

The French New Master Sample 2009

=> Advantages :

Limit the wastes due to the changes in buildings (demolition, main homes becoming secondary homes and V.V….), which create unwished costs.

Draw in a more efficient way samples on particular sub-populations (whose recent characteristics are known).

Get rid of a specific system to cover new dwellings.

 

The French New Master Sample 2009

It implies to build specific primary units called IAA (Interviewers Action Area).

An IAA may be either:

- a big municipality

- a collection of small municipality that includes at least one municipality in each of the five rotation groupes and 300 dwellings in each of the five rotation groups

Example of an IAA formed of small municipalities

9

The French Master Sample 2009

Among the 3743 IAA built in the French Metropolitan Territory:

- 37 have been incorporated in the take all stratum (big municipalties with more than 40 000 dwellings in 1999)

- 488 additional have been selected proportionnaly to size with a regional stratification and a balanced sample design

Each year, the samples are drawn among the dwellings censused the year before in the drawn IAA.

 

10

The French Master Sample 2009

The number of dwellings drawn in each IAA is calculated:

- in order to minimise the variance of the final weights of dwellings

- under constraints of minimum and maximum number of dwellings drawn in each IAA, to provide a reasonable amount of work to the interviewer in charge of the IAA

 

11

The issue of unexpectedly missing interviewers

Hired interviewers are expected to devote a fixed amount of time (set beforehand by contract) to data collection.

It comprises:

-The total travel time to reach the dwellings

-The total time surveying sampled households

 

The issue of unexpectedly missing interviewers

Generally not possible to increase interviewer’s time devoted to data collection

Usually costly and complicated to recruit another interviewer in a specific area and train him/her in a very short time

Therefore, missing interviewers may not always be replaced

INSEE new sample monitoring strategy

Aim : minimising the sample weights dispersion

Under the constraints of not exceeding available interviewers’ time devoted to data collection

INSEE new sample monitoring strategy

The method is based on two hypothesis:

- The number of visits per household is the same in the usual IAA and in the vacant IAA

- The total time surveying sampled households for the interviewer remains the same after the redesign of the sample

INSEE new sample monitoring strategy

Under those two assumptions, the number of dwellings finally surveyed by the interviewer in the usual IAA

and in the vacant IAA shall fullfill following constraint:

where denotes the number of dwellings the interviewer was exepected to survey in its usual IAA, the journey time for the interviewer to reach its usual IAA and its journey time to reach the vacant IAA.

Notice that asking the interviewer to carry out one survey in the vacant IAA leads to give up dwellings in its usual IAA

finalUn

finalVn

UinitialUV

finalVU

finalU TnTnTn

initialUn

UTVT

U

V

T

T

INSEE new sample monitoring strategy

Let denote the final sample weight of dwelling d

The method finds the allocations ( ) that minimize the sample weights dispersion under the constraints of the interviewers’ time devoted to data collection.

An additional constraint of a minimum number of dwellings to be surveyed may also be introduced

A step by step approach to find the solution

d

IAA

IAAIAAd n

N

IAAn

Example of use of the method

Taken from the Housing Survey in the region Rhône-Alpes

2591 dwellings have been drawn in the region. The initial weights dispersion is 1954

The red IAA (Vizille) is vacant, with 32 dwellings drawn in this IAA. Interviewers affected to the blue IAA (La Mure) and the yellow IAA (Saint-Marcellin) can be asked to carry out surveys in the vacant IAA.

Example of use the method

IAA Number of dwellings surveyed

Journey time to the usual IAA

Journey time to the vacant

IAA

Blue (La Mure) 52 10 25

Yellow (Saint-Marcellin)

43 29 55

Example of use of the method

For one dwelling surveyed in the red IAA by the interviewer affected to the blue IAA, it has to give up 25/10=2,5 dwellings in the blue IAA

For one dwelling surveyed in the red IAA by the interviewer affected to the yellow IAA, it has to give up 55/29=1,9 dwellings in the yellow IAA

=> it is less « expensive » to ask the interviewer of the yellow IAA to carry out surveys in the red IAA, though the blue IAA is nearer from the red IAA.

Results

IAA whose interviewers carry out replacements

Total regional sample

Weights dispersion

Blue and Yellow 2544 1955

Blue 2544 2233

Yellow 2550 2192

Resultats

14 dwellings on 32 will be surveyed in the red IAA

35 dwellings on 52 will be surveyed in the blue IAA

30 dwellings on 43 will be surveyed in the yellow IAA

Surveyed dwellings in the three IAA are randomly selected among the initiallly sampled dwellings

Results

The interviewer affected to the yellow IAA surveys 7 dwellings in the red IAA and has to give up 13 dwellings in its usual IAA

The interviewer affected to the blue IAA surveys 7 dwellings in the red IAA and has to give up 17 dwellings in its usual IAA

Conclusion

This sample redesign methodology has been implemented for two surveys so far.

It has proved efficient to reorganise field operations in order to avoid bias due to a vacant IAA

When the results of those surveys will be available, accuracy estimations will be carried out to account for the impact on quality of this new sample monitoring strategy