Lost but not forgotten : attrition in the Étude longitudinale du développement des Lost but not forgotten : attrition in the Étude longitudinale du développement des enfants du Québec (ÉLDEQ), 1998-2004enfants du Québec (ÉLDEQ), 1998-2004

Julien BÉRARD-CHAGNON and Simona BIGNAMI-VAN ASSCHEJulien BÉRARD-CHAGNON and Simona BIGNAMI-VAN ASSCHE

Département de démographieDépartement de démographie

Project financed by the Fonds québécois de recherche sur la société et la culture (FQRSC)Project financed by the Fonds québécois de recherche sur la société et la culture (FQRSC)

BackgroundBackground

Longitudinal surveys are increasingly used in the social sciences to describe behaviour dynamics, to identify the influence of past on current behaviours, and to make Longitudinal surveys are increasingly used in the social sciences to describe behaviour dynamics, to identify the influence of past on current behaviours, and to make stronger causal inferences than it is possible with cross-sectional surveys. stronger causal inferences than it is possible with cross-sectional surveys.

In demography, longitudinal data are particularly relevant for the study of family transitions and life course analysis.In demography, longitudinal data are particularly relevant for the study of family transitions and life course analysis.

One of the main weaknesses of longitudinal surveys is that they are prone to attrition, that is, the loss of study subjects over time that is due to respondents leaving the One of the main weaknesses of longitudinal surveys is that they are prone to attrition, that is, the loss of study subjects over time that is due to respondents leaving the study prematurely and permanently.study prematurely and permanently.

Why study survey attrition ?Why study survey attrition ?

Reduces the sample size (thus reducing the Reduces the sample size (thus reducing the power of statistical estimations).power of statistical estimations).

Makes the study of small sub-samples harder.Makes the study of small sub-samples harder.

Affects sample representativeness.Affects sample representativeness.

Might lead to selection bias.Might lead to selection bias.

DataData

ÉLDEQ: Ongoing longitudinal survey aimed at studying ÉLDEQ: Ongoing longitudinal survey aimed at studying the factors influencing child development in Québec for a the factors influencing child development in Québec for a cohort of 2120 children born in 1997-98.cohort of 2120 children born in 1997-98.

The ÉLDEQ survey team has made lots of efforts to The ÉLDEQ survey team has made lots of efforts to minimize attrition and track down attritors.minimize attrition and track down attritors.

Selected sample for the analysis: waves 1-8.Selected sample for the analysis: waves 1-8.

Attrition in the ÉLDEQAttrition in the ÉLDEQCumulative attrition rate (%), waves 1-8,

ƒLDEQ

0

5

10

15

20

25

30

1 2 3 4 5 6 7 8

Attrition is measured by the participation Attrition is measured by the participation to the main survey instrument (QIRI).to the main survey instrument (QIRI).

Attrition was very low in the first phase of Attrition was very low in the first phase of the survey (waves 1-5, before the sampled the survey (waves 1-5, before the sampled children entered primary school), but high children entered primary school), but high in the second phase (waves 6-8).in the second phase (waves 6-8).

Many factors (uncertainty about the Many factors (uncertainty about the future of the survey, increasing length of future of the survey, increasing length of interviews, etc.) explain this finding.interviews, etc.) explain this finding.

Attritors’ characteristicsAttritors’ characteristics

Attritors have different characteristics than the other respondents in the sample.Attritors have different characteristics than the other respondents in the sample.

Overall, attritors distinguish themselves from non-attritors by their characteristics associated Overall, attritors distinguish themselves from non-attritors by their characteristics associated with instability, poverty, immigration and social exclusion.with instability, poverty, immigration and social exclusion.

Type of family arrangement at the time of the survey for three attrition groups (p<0,05)

71,4%

82,8% 82,8%

12,7%

7,4%10,7%

15,9%

9,8%6,5%

A B C

Intact family Step-parents family Single-parent family

Mother's first spoken language for three attrition groups (p<0,01)

50,8%

81,0%84,9%

7,9%

7,4%

7,4%

41,3%

11,6%7,7%

A B C

French English Other

A : Wave 1 attritors B : Wave 5 attritors C : Non-attritors

Mother's highest diploma for three attrition groups (p<0,01)

32,3%

16,5% 15,1%

25,8%

27,3%25,4%

21,0%

35,5%

29,6%

21,0% 20,7%

29,9%

A B C

No Highschool Highschool PS (except univ.) University

Study objectivesStudy objectives

1.1. Compare the Compare the characteristics of attritorscharacteristics of attritors and non- and non-attritors using chi-square tests and one-way ANOVA.attritors using chi-square tests and one-way ANOVA.

2.2. Identify the Identify the factors influencing the probability of attritionfactors influencing the probability of attrition using multivariate probit regressions.using multivariate probit regressions.

3.3. Measure Measure attrition biasattrition bias using BGLW tests for selected using BGLW tests for selected variables.variables.

ConclusionsConclusions

Probability of attrition by mother's highest diploma, wave 1

0,238

0,3060,323

0,398

University PS (except univ.) Highschool No highschool

Probability of attritionProbability of attrition

The probability of attrition is modeled using a probit model with a set of The probability of attrition is modeled using a probit model with a set of background and other individual characteristics as independent variables.background and other individual characteristics as independent variables.

Most variables do not predict significatively the probability of attrition.Most variables do not predict significatively the probability of attrition.

Attrition biasAttrition bias

Attrition bias is evaluated by means of BGLW tests (Becketti, Gould, Lillard and Welch, Attrition bias is evaluated by means of BGLW tests (Becketti, Gould, Lillard and Welch, 1988) by regressing a selected variable of interest on a set of control variables plus a 1988) by regressing a selected variable of interest on a set of control variables plus a dichotomous variable representing attrition in the following waves. The presence and dichotomous variable representing attrition in the following waves. The presence and magnitude of attrition bias is inferred from the significance of the estimated coefficient for magnitude of attrition bias is inferred from the significance of the estimated coefficient for attrition in this equation.attrition in this equation.

Attrition does not exert a signficant bias on most variables of interest (e.g. delay in child’ Attrition does not exert a signficant bias on most variables of interest (e.g. delay in child’ growth) with the exception of growth) with the exception of mothers’ immigrant statusmothers’ immigrant status and and abortionabortion..

******

******

Probability of attrition by immigration status of the mother, wave 1

0,214

0,343

Non-immigrant Immigrant

******

Probability of attrition by whether the mother is overprotective of her child (on a continuous scale

between 0 and 10), wave 1

0

0,1

0,2

0,3

0,4

0,5

0 1 2 3 4 5 6 7 8 9 10

Legend: * Legend: * pp<0,10; ** <0,10; ** pp<0,05; *** <0,05; *** pp<0,01. <0,01.

Notes: The household’s characteristics included in the models are: household income, household income squared, number of Notes: The household’s characteristics included in the models are: household income, household income squared, number of siblings, whether home is owned). The individual characteristics of the mother included in the models are: age, highest siblings, whether home is owned). The individual characteristics of the mother included in the models are: age, highest diploma, occupation. All probabilities are calculated using the mean score for continuous variables and the mode for discrete diploma, occupation. All probabilities are calculated using the mean score for continuous variables and the mode for discrete variables.variables.

Probability of attrition by whether the mother had an abortion, wave 1

0,257

0,209

Mother had an abortion Mother didn't have an abortion

****

Probability that the mother had an abortion by attrition status (attrition bias), wave 1

0,322

0,385

Attritors Non-attritors

****

Legend: * Legend: * pp<0,10; ** <0,10; ** pp<0,05; *** <0,05; *** pp<0,01. <0,01.

Note: The background characteristics considered for the BGLW tests are the same used for the probit models.Note: The background characteristics considered for the BGLW tests are the same used for the probit models.

Mother's overprotection mean score by attrition status (attrition bias), wave 1

4,58

4,38

Attritors Non-attritors

Respondents’ attitude towards surveys (i.e. level of education) and geographic Respondents’ attitude towards surveys (i.e. level of education) and geographic mobility are the two most important factors associated with attrition. mobility are the two most important factors associated with attrition.

Although attrition exerts important biases for univariate analyses, it does not Although attrition exerts important biases for univariate analyses, it does not generally bias multivariate analyses.generally bias multivariate analyses.

The main effect of attrition for analyses of the ÉLDEQ data is to decrease the The main effect of attrition for analyses of the ÉLDEQ data is to decrease the sample size and thus reduce the power of statistical inferences.sample size and thus reduce the power of statistical inferences.

Continuing efforts are made by the ÉLDEQ survey team to track down Continuing efforts are made by the ÉLDEQ survey team to track down respondents and thus limit attrition in future waves. respondents and thus limit attrition in future waves.

Future research should focus on the consequences of attrition for longitudinal Future research should focus on the consequences of attrition for longitudinal analyses of the ÉLDEQ data (survival analysis and multi-level analysis).analyses of the ÉLDEQ data (survival analysis and multi-level analysis).

Researchers using longitudinal survey data should always check for attrition bias Researchers using longitudinal survey data should always check for attrition bias in their analyses.in their analyses.

Probability that the mother is immigrant by attrition status , wave 1

0,304

0,187

Attritors Non-attritors

******