Attrition and Non-response in the Survey of Labour and

Preliminary and incomplete, please do not cite!

Attrition and Non-response in the Survey of Labour and Income Dynamics

Brahim Boudarbat (*) (Université de Montréal and IZA)

Lee Grenon (**) (Statistics Canada)

This version: April 2005

Abstract: The longitudinal Survey of Labour and Income Dynamics (SLID) is one of the most important data sources for analyzing incomes in Canada. However, like any other panel survey, the SLID experiences a problem of attrition (or exit behaviour from the sample) and non-response. Among longitudinal respondents who provided labour information in the first year of the second panel (1996-2001), about 22% of this sample became either out of scope or non-respondent by the end of the panel’s reference period. As respondents are lost from the survey’s sample, the data set becomes less representative of the population from which the longitudinal sample was selected. Estimates using available data are then subject to selection bias if the attrition and non-response behaviours were not random. To understand the selection biases, we consider a structural model composed of three equations for attrition/non-response, employment and earnings. The three equations are freely correlated. The model is estimated using microdata from 23,317 individuals who provided sufficient information in the first wave of the second panel of the SLID. Results provide evidence for non randomness of attrition behaviour. Attritors and non-respondents likely are less attached to employment and come from low-income population. We find small, though significant, correlation between attrition and employment. In addition, the earnings gap between attritors/non-respondents and a random sample with identical characteristics is mild. These findings are consistent with results from similar studies using data from other countries.

The analysis in this manuscript use microdata from Statistics Canada. The interpretation and views expressed in this manuscript do not represent the position or views of Statistics Canada.

(*) Brahim Boudarbat, School of Industrial Relations, Université de Montréal, C.P. 6128, Succursale Centre-ville, Montreal, QC, H3C 3J7. Email: [email protected]

(**) Lee Grenon, Statistics Canada, British Columbia Inter-university Research Data Centre, Walter C. Koerner Library, room 202, 1958 Main Mall, Vancouver, BC, V6T 1Z2. Email: [email protected]

Attrition and non-response in the Survey of Labour and Income Dynamics

1

I- Introduction

The increased availability of longitudinal data surveys has significantly boosted the empirical

studies aimed at analyzing the dynamic of individual behaviours. Indeed, combining both cross-

section and cross-time provides useful leverage for identifying the parameters that drive those

behaviours, and allows controlling for individual-specific unobservable effects. In Canada, the

longitudinal Survey of Labour and Income Dynamics (SLID) is increasingly used by

researchers for the study of education, family, labour and income. The survey has the advantage

of surveying very large samples of persons and providing detailed panel data (six waves) on a

large variety of variables that are of interest for many research topics (see Section II). However,

the value of the SLID for empirical analyses relies upon the conjecture that data are

representative of the population of concern. While the initial sample is constructed to meet this

assumption, there should be a concern about the quality of the panel data owing to the non-

random attrition and non-response. Among longitudinal respondents who report labour force

status in the first year of each completed panel, a significant proportion is out of scope or non-

respondents by the end of the panel. In panels 1 (1993-1998) and 2 (1996-2001) of the SLID,

respectively 15% and 13% of these working-age longitudinal respondents became out of scope

during the panel (see Table 1). Respondents become out-of-scope when they migrate away

from the Canadian provinces, are institutionalized, or are deceased. The out-of-scope

respondents are outside of the target population for SLID, and are not eligible to participate for

the reference year.

The other component of attrition is non-respondents who include those who can not be located

or contacted, and those who completely refuse to participate. The non-respondents are

potentially still in the target population of the survey, but are no longer participating. In panel 2

of the SLID, complete non-response to the labour interview was at 9.13% at the end of the

panel (2001) of working-age longitudinal respondents who respondent in the first wave (1996).

The non-response rate for this sample peaked at 11.42% in 2000. Thus, attrition and non-

response behaviours concern over one-fifth (22%) of the working-age longitudinal sample who

were in-scope and responded to the first wave of the labour interview (see Table 1).

As sampled individuals drop from the survey, the data set becomes less representative of the

population from which the first sample was drawn, which may affect the quality of the


2

available panel data if attrition and non-response were non-random. Studies that ignore this fact

may be subject to selection biases. Several studies analyze the attrition issue for some American

and European panel surveys (see for instance the Special Issue “Attrition in Longitudinal

Surveys,” of The Journal of Human Resources, Spring, 1998, Vol. 33, No. 2). But, to our

knowledge, this issue received little attention in Canadian surveys,1 which may raise concerns

about data quality.2 Our study aims at filling up this gap.

In the literature, most of available studies lead to negligible selection biases from attrition. Van

Den Berg, Lindeboom and Ridder (1994) use panel data from The Netherlands and find that

unobserved explanatory variables for the duration of panel survey participation of an individual

are not related to unobserved explanatory variables for the duration of unemployment of that

individual. In a subsequent study, Van Den Berg and Lindeboom (1998) find significant

dependence between labour market durations and attrition, but there is little bias from ignoring

this dependence. Lillard and Panis (1998) find, on the basis of data from the Panel Study of

Income Dynamics (PSID), that despite the evidence of selectivity in attrition, the biases that are

introduced by ignoring selective attrition are very mild. In the same vein, Zabel (1998)’s study

which uses data from the PSID and the Survey of Income and Program Participation (SIPP),

shows little indication of bias due to attrition in a model of labour market behaviour, though

there is evidence that the labour market behaviour of attritors and participants is different.

In this study, we aim at verifying whether the above consistent result regarding attrition applies

to the SLID. Since the latter focuses on the labour and income dynamics, we think that it is

important to evaluate the selection bias from attrition in models that estimate the labour force

participation and earnings. For this purpose, we consider a structural model composed of three

freely related equations for non-attrition/response, labour force status and earnings. The

relationship between these equations arises from the fact that the labour force status (whether

employed in our case) is observed only for the participants (i.e., those who do not attrite and

respond to the labour interview), and that available earnings observations are for the

participants who are employed. Thus, the model allows testing for the selectivity from attrition

in both employment and earnings equations. Our model is developed in Section III. Then, we

1 An early study by Statistics Canada (1995) discusses some data quality measures for the SLID. 2 For instance, because of the uncertainty about the attrition effects, Boudarbat, Lemieux and Riddell (2003) favour data from the census in their study on the evolution of the earnings structure in Canada.


3

evaluate the model using microdata from 23,317 working-age respondents who participated in

the first wave of the second panel of the SLID. The latter is described in Section II and

structural parameter estimates are presented in Section IV. Our results provide evidence for non

randomness of attrition/non-response behaviours. We find a positive and significant, though

small, correlation between participating in the survey and being employed. Attritors and non-

respondents are likely less attached to the labour force, more “mobile” and have lower

education levels. They also likely come from large cities and the lower end of the income

distribution. With regards to the earnings equation, we find that the earnings gap between the

available sample and a sample selected randomly with the same characteristics is less than 7%

due to attrition selectivity as compared to more than 34% due to employment selectivity.

Hence, our findings appear to agree with those of previous studies concerning the limited

selectivity in attrition. Section V offers a short summary with concluding remarks.

II- Data

This study uses microdata from Statistics Canada’s Survey of Labour and Income Dynamics.

SLID is the principal household survey for information on income and one of the major data

sources for labour dynamics of Canadian individuals and families. Historically, information on

individual and family annual income was collected through the Survey of Consumer Finances,

which was an annual cross-sectional survey of households in the ten provinces. Starting with

the 1996 reference year, SLID replaced the SCF as the principal source of data on individual

and family annual income, and the SCF was discontinued. SLID provides several important

advantages over the SCF. First, SLID is a longitudinal survey that facilitates analysis of family,

income and labour dynamics over time. Second, SLID collects the same income information as

the SCF, however SLID ads a diverse range of information on transitions in jobs, income and

family events. The essential goal of the survey is to improve our understanding of the economic

well-being of Canadians.

SLID is a panel of longitudinal respondents selected at the beginning of the reference period

and then interviewed annually for six years. The longitudinal sample is selected from

households in the Labour Force Survey. The LFS is a multi-stage probability area sample. The

longitudinal sample for SLID comprises all persons living in the selected households at the time

of the LFS reference period. After the panel reference period begins, any cohabitant who lives


4

with a longitudinal respondent for any period of the reference year is also interviewed.

However, information for cohabitants is only collected for reference years in which they live

with a longitudinal respondent. SLID permits proxy interviews and uses computer assisted

telephone interviewing technology which enables feedback of information from previous

interviews to reduce inconsistencies or “seam” problems between waves.

The longitudinal dimension and the collection of information on cohabitants enables analysis of

transitions of families and households over time. An important topic of this research has been

the transitions into and out of low-income status. SLID’s longitudinal dimension also facilitates

extensive research on human and social capital development and dynamics. Other research

topics include the dynamics of human and social capital formation as related to changes in

education, work experience, and family relationships.

Each panel of the SLID has a relatively large sample size of approximately 15,000 households

and 31,000 persons of 16 years of age or older, which is representative of the population of the

ten provinces of Canada at the time of selection. Residents of the Yukon, Northwest Territories,

and Nunavut, institutional residents, and persons living on Indian reserves are not eligible for

selection into the longitudinal sample.

Analyses with SLID will typically use all selected respondents who are in scope at the end of

the selected reference period. Respondents who are out of scope at the end of the selected

reference period are typically excluded from analyses and have a final longitudinal weight

equal to zero. Longitudinal respondents may become out of scope due to institutionalization,

migration out of the ten provinces, or death. This study initially selected the 25,033 longitudinal

working-age (16 to 69 years-old) respondents from panel 2 who were in scope and responded to

the labour interview in the first reference year. The first reference year for panel 2 was 1996

and the last reference year was 2001. For each reference year, information on labour is

collected for respondents who are from 16 to 69 years of age on December 31 of the reference

year.

The 25,033 respondents initially selected for this analysis represent 16.8 million persons age 16

to 69 years old on December 31, 1996. The survey weight used here is for all persons selected

for the longitudinal sample at the time of the initial sample selection in December 1995

(Levesque and Franklin, 2000). This initial longitudinal sample weight assigns a weight greater


5

than zero for all persons selected in December, 1995 for the longitudinal sample. Use of the

initial longitudinal sample weight allows us to show the impact of attrition on the weighted

sample. In contrast most longitudinal analyses will use the final longitudinal weight which is

adjusted for non-response and inter-provincial migration since the start of the panel. The final

longitudinal weight is not useful for the analyses here since out of scope cases are assigned a

weight equal to zero.

Data quality for SLID can be assessed through a variety of measures of non-response and

estimates of variance (Michaud and Webber, 1995). This study specifically analyses the impact

on labour force and earnings estimates from attrition of respondents from the sample and non-

response to the annual labour interview. Data collection for each reference year occurs in two

phases. The first phase is a labour and household interview conducted shortly after the end of

the reference year. The second data collection phase involves an income interview conducted

later in the spring for the reference year. A substantial proportion of respondents agree to share

their income tax information with Statistics Canada instead of responding to the income

interview (Michaud and Latouche, 1996). SLID imputes values for selected income variables

for non-response when sufficient information is available for other characteristics of the

respondent (Webber and Cotton, 1998).3

The analysis here excludes longitudinal respondents who were out of scope at the end of the

first reference year or who did not respond to the labour interview for the first reference year.

Including only longitudinal respondents who provided information in the labour interview

ensures that a minimum of information is available for comparing the characteristics of

respondents who exit the sample in subsequent years to those of respondents who are still in

scope and providing information at the end of the panel period. Attrition from the longitudinal

sample can occur for several different reasons including: migration out of the ten Canadian

provinces; institutionalization for more than six months; deceased; inability to contact; refusal;

and found not to be a person. These different types of attrition are expected to be non-random

characteristics within the sample. This study is concerned with the overall impact of attrition

and non-response on selected estimates, rather than distinguishing the characteristics of each of

the attrition and non-response groups. 3 For further information on the SLID design and data processing refer to the SLID Microdata User’s Guide (Statistics Canada, 1997).


6

The labour interview collects an extensive retrospective history of spell durations of jobs and

job search during the preceding reference year. This information is used for deriving weekly,

monthly and annual labour force status variables. The annual labour force status variable

indicates if the respondent was employed, unemployed and/or not in the labour force for any

part of the reference year. Other derived variables are also used here for unemployment

duration, annual weeks worked and earnings.

The percentage of our study sample that became out of scope (attrition) reached 12.7% by the

end of the panel (see Table 1). This attrition represents 14.2% of the population estimate at the

time of the initial longitudinal sample selection.

Table 1: Annual Attrition and Non-Response Rates

Attritors (out of scope) Non-Respondents to the labour interview (among

in scope) Total

Wave Number (1)

Sample percentage

(2)

Number (1’)

Sample percentage

(2’) (1)+(1’) (2)+(2’)

1997 692 2.76 1,353 5.40 2,045 8.17 1998 1,084 4.33 1,636 6.54 2,720 10.87 1999 1,762 7.04 2,103 8.40 3,865 15.44 2000 2,634 10.52 2,858 11.42 5,492 21.94 2001 3,169 12.66 2,286 9.13 5,455 21.79

Note: Includes the 25,033 longitudinal working-age (16 to 69 years-old) respondents who were in-scope and responded to the first wave of the labour interview in the 1996 reference year.

The other issue of concern in this analysis is complete non-response to the labour interview

among respondents who were in-scope. Table 1 shows that complete non-response to the labour

interview was at 9.13% of our study sample at the end of the panel. This percentage was the

highest in 2000 (at 11.42% of our initial sample). In total, over one-fifth of the longitudinal

sample who were in-scope and responded to the first wave of the labour interview were out of

scope or non-respondents to the labour interview by the last reference year of the panel. These

attritors and non-respondents represent nearly one-quarter (24.60%) of the initial population

estimate at the time of sample selection.

Only a minority, about one-fifth, of respondents who are out of scope or a non-respondent to

the labour interview are converted to labour interview respondents in the following wave. This


7

conversion dropped considerably in the latter half of the panel. In our sample, all cases were

respondents to the labour interview for 1996; therefore conversions from out of scope and non-

response begin for 1998.4

The characteristics of attritors/non-respondents and non-attritors differ, and suggest that

attrition may not be random within the SLID longitudinal sample. Averages of the

characteristics of the selected observations are presented in Table 2. Attritors include

longitudinal respondents who were out-of-scope or labour interview non-respondents for at

least one reference year from 1997 through 2001. Non-attritors include respondents who were

in-scope and labour interview respondents for all reference years from 1996 through 2001. Both

attritors and non-attritors were in-scope and labour interview respondents for the 1996

reference year.

In 1996, respondents who would later become attritors were on average younger and more

likely to be male, to have never married, and to be immigrants than non-attritors. Respondents

who were students or who moved during 1996 were more likely to become attritors in

subsequent years of the panel. Contacting respondents in subsequent years for data collection is

more difficult when respondents move to a different residence and/or live without a spouse or

common-law partner.

It follows that on average in 1996, respondents who would later become attritors had lower

wages and salaries and lower total household income. Attritors were more likely to have lived

in urban areas with larger populations in their residential area in 1996 than non-attritors.

Our final sample used in the estimation of the model developed in Section III, is of 23,317

longitudinal respondents who were in scope and responded to the labour interview in 1996. We

retain individuals who were aged 65 or younger in the last reference year (i.e., 2001) and

provided a minimum of information on variables of interest. By the end of 2001 (last reference

year), 7,381 individuals dropped out of the sample or were in scope but did not respond to the

labour force interview, which corresponds to an attrition rate of about 32%. This percentage is

higher than the one shown in Table 1 because we consider attrition as an absorbing state (see

4 In Section III, we assume that attrition and non-response are absorbing states in order to ease the estimation of our model.


8

Section III); so those who were out of scope or did not respond to the labour force interview

during a wave, are considered as attritors for the subsequent waves. We ignore the fact some

attritors were converted to labour interview respondents in the following wave.

Table 2: Characteristics of attritors/non-respondents and non-attritors

Mean Variable Attritors/Non-

respondents Non-attritors

Age 37.9 (0.20)

41.1 (0.14)

Female 49.0 51.0 Married 54.9 68.9 Single 34.6 21.9 Immigrant 23.6 17.0 Moved during year 16.4 12.8 Student 23.1 16.0 Highest level of education

High school diploma 34.0 31.2 Non-university diploma 23.8 27.7 University degree 14.2 16.4

Labour characteristics

Annual wages and salaries 16,705 (328)

20,304 (256)

Annual number of weeks worked 34.1 (0.33)

36.1 (0.23)

Employed during year 73.3 75.1 Labour interview proxy response 41.1 37.9 Household characteristics

Total household income 56,127 (686)

59,429 (447)

Household size 3.19 (0.02)

3.18 (0.01)

Residence area population 1,407,387 (24,598)

1,055,529 (17,618)

Urban area household 85.5 80.5 Province of residence

Atlantic provinces 6.7 9.7 Quebec 26.4 26.0 Ontario 41.6 35.0 Prairie provinces 13.7 16.4 British Columbia 11.6 12.9

Note: All estimates are weighted. In parentheses are standard-deviations.


9

III- Econometric Specification This study focuses on the possible correlation between three variables: earnings, employment

and the sample attrition/interview non-response. Earnings are only observed for employed

workers. In addition, the labour force status (i.e. employed or not) is only observed for

individuals who participate in the survey and provide information on their status. Therefore,

data on labour status is censored (missing) for non-participants, and earnings are censored for

non-participants and for participants who are not employed. If this two-level censorship is not

random, results based on observed data are plagued by a selection problem. In order to evaluate

this selection problem in the SLID data, we propose the following model.

Consider a longitudinal survey that includes T waves. At time period t = 1 a probabilistic

sample is obtained from the target population. At time t 2≥ , some individuals drop out of the

sample. These individuals are called “attritors.” If attritors do not return to the sample in any

subsequent period, attrition is an absorbing state. To ease the estimation of our model, we will

consider attrition to be an absorbing state.5 Even if an individual is located (i.e. contacted by an

interviewer), she may refuse to respond to the labour questionnaire. In this case, she is

considered to be a non-respondent. Thus, at each wave an individual may be an attritor, a non-

respondent or a respondent. Only a minority of respondents who are attritors (out of scope) or

non-respondents to the labour interview are converted to labour interview respondents in a

subsequent wave. In our sample, all cases were respondents to the labour interview for 1996;

therefore conversions from non-response begin in 1998.

A respondent may be either employed or not employed. If she is employed, her earnings are

observed. Therefore, there are two selection sources depicted by the reduced-form equations (1)

and (2) below:

Non-attrition and response criterion:

( )*it 1iti t 1a Z θ ε−= + , i=1,…,n; t=2,…,Ti (1)

where i indexes for individuals and t indexes for time periods (i.e. waves of the survey).

5 The return to the sample may also be non-random, which may counterbalance or intensify the effect of attrition.


10

Individual i remains and participates in the survey in period t ( ita 1= ) if *ita 0≥ , and leaves or

does not cooperate in period t ( ita 0= ) if *ita 0< . Because of the assumption that attrition is an

absorbing state, if ita 0= , then 0' =ita for any t’>t. Since information for the current period is

not available for individuals who drop from the sample or do not respond, we use lagged

variables. The initial period of analysis of attrition is the second wave since all individuals

participate in the first one.

Employment criterion:

*it it 2ite X α ε= + , i 1,...,n= , it 1,...,T= (2)

Respondent i is employed ( ite 1= ) if *ite 0≥ , and is not employed ( ite 0= ) if *

ite 0< . A person

is not employed if she is either unemployed or out of the labour force.

itZ and itX are vectors of exogenous regressors, and it1ε and it2ε are random components

capturing unobserved variables. itZ and itX are observed whenever 1ita = , and Zi1 is observed

for all individuals in the sample. Like in Zabel (1998), we include a 1 T× vector of wave

dummies in itZ and itX to account for duration dependence. A monotonic change in the

coefficients on the wave dummies indicates the presence of such dependence. In Equation (1),

negative dependence signifies that the probability of dropping from the survey or not

cooperating when located is increasing over time, ceteris paribus. In other words, individuals in

the sample are less willing to continue in the survey from one wave to another. Such negative

dependence arises when the coefficients on wave dummies are monotonically decreasing. On

the other hand, positive duration implies that participants get more familiar with the survey and

are more willing to participate in the subsequent waves. Such dependence duration occurs when

the coefficients on wave-dummy variables are monotonically increasing.

Next, earnings are given by the following equation:

it it 3ity W β ε= + , i = 1,…, n, t = 1,…, Ti (3)

where ity is log annual earnings, itW is a vector of exogenous regressors, and it3ε is a random

component.


11

The structural model is given by Equations (1), (2) and (3). This model is sequential since

dummy variable ite is observed only if 1ita = (the individual is in the scope and cooperates),

and ity is observed only if 1ita = and ite 1= (the individual is in the scope, cooperates and is

employed) (see Maddala, 1983, pp. 278-283, for further examples on multiple criteria for

selectivity).

Ideally, one would like to estimate the model by considering jitε , j=1,2,3, it 1,...,T= , are freely

correlated for a same individual. However, doing so will involve computing joint probabilities

from a 3xTi variate distribution, which is practically problematic. In order to ease the estimation

of the model, we will adopt the random effect model approach (see below). We also estimate

the model consistently in two stages following the approach suggested by Ham (1982).6 The

latter approach is an extension of the two-stage estimator for the one selection rule proposed by

Heckman (1979), and is computationally more attractive than the maximum likelihood method

and produces consistent parameter estimates. The first stage involves a joint estimation of the

selection equations (1) and (2). Then, correction terms using obtained parameter estimates are

calculated and inserted in the earnings equation (3) to account for selection bias.

Stage 1: Selection Equations

In order to simplify the computation of joint probabilities, we adopt the random effects model,

which specifies:

1it 1i 1itε u v= +

2it 2i 2itε u v= + (4)

where 1iu and 2iu are individual specific effects assumed to be freely correlated, but

independent of itZ and itX , and of jitv for j=1,2 and it 1,...,T= . We also assume that error

terms jitv are independently distributed over individuals and time. In addition, jitv are mutually

independent, and independent of it3ε . However, 1iu and 2iu may be correlated with it3ε .

6 Ham(1982) uses cross-sectional data, whereas we use longitudinal data.


12

The correlation between Equation (1) and Equation (2) is given by the correlation between

individual specific effects 1iu and 2iu . Let ( )i 1i 2iu u ,u '= . Conditional on iu , it1ε and it2ε are

independent. The vector iu is assumed to follow a bivariate normal distribution:

( )iu ~ N 0,Σ (5)

iu can be specified as linear combinations of two independent ( )N 0,1 : i iu Γη= , where Γ is a

2×2 lower triangular matrix of unknown parameters to be estimated. It follows that the

covariance matrix of iu is Σ = ΓΓ ' = 11 12

22

σ σσ

⎛ ⎞⎜ ⎟⎝ ⎠

.

Attrition and non-response are random and there is no selectivity bias in employment equation

estimates (Equation 2) if unobserved determinants of employment are uncorrelated with

unobserved determinants of attrition/non-response, (i.e. if 12σ 0= ). If this is not the case,

individuals who drop from the survey or do not respond likely do it because of their labour

status. Positive values of 12σ mean that attritors and non-respondents likely originate from the

employed population, which ultimately leads to underestimating the employment rate. In

general, non nil covariance implies that results based on employment criterion and ignoring

attrition and non-response are biased.

The contribution of an individual to the likelihood function conditional on iu is:

( )i iL u = ( )iT

it it 1

L u=∏ (6)

where

( )it iL u = ( ){ } it1 ait iPr a 0 |u −= × ( ){ }( ) ( ){ }

itit it

a1 e eit it i it it iPr a 1, e 0 |u Pr a 1, e 1|u−⎡ ⎤= = × = =⎢ ⎥⎣ ⎦

(6.1) for t 2≥ and

( )it iL u = i1L = ( ){ }( ) ( ){ }it it1 e eit i it iPr e 0 |u Pr e 1|u−= × = for t 0= (6.2).

Since all individuals participate in the survey at time period t =1, the contribution of an

individual to the likelihood function depends only on her labour force status at this period.

Since it1ε and it2ε are independent conditional on iu , Equation (6.1) simplifies to:


13

( )it iL u = ( ){ } it1 ait iPr a 0 |u −= × ( ) ( )( ) ( )( )[ ] ititit

aeiit

eiitiit ueueua |1Pr|0Pr|1Pr 1 === − (6.3)

For identification issue, we shall assume that v1it and v2it follow ( )N 0,1 .

Thereafter, the unconditional contribution of an individual to the likelihood function is:

iL = ( ) ( )i i i 1i 2iL u g u du du∞ ∞−∞ −∞∫ ∫ (7)

where ( )ig u is the joint density function of iu .

Finally, full maximum likelihood estimates of the parameters in (1) and (2) with the

homoskedasticity assumption are obtained by maximizing the log likelihood function:

( ) ( )n

ii 1

log L log L=

= ∑ (8)

Since the function in (8) involves two-dimensional integration, direct optimization is generally

not feasible. We will use maximum simulated likelihood instead (see Green, 2002 for some

useful applications). Notice that the function in (7) is an expectation ( ( )ii u i iL E L u⎡ ⎤= ⎣ ⎦ ), which

can be approximated by a simulated mean:

( )R

is i irr 1

1L L uR =

= ∑ (9)

where iru , r =1,…,R, are R draws from the bivariate distribution of i iu Γη= . Parameters in (1)

and (2) including Γ are obtained by maximizing the simulated log likelihood:7

( ) ( )n

s isi 1

log L log L=

= ∑ (10)

Sample from iu is constructed using draws on ( )i 1i 2iη η ,η '= where 1iη and 2iη are two

independent ( )N 0,1 . Gourieroux and Monfort (1996) show that if n / R 0→ and R and

7 See Gourieroux and Monfort (1996) and Train (2002) for discussion and statistical background. See also Green (2002) for some applications.


14

n →∞ , then the maximum simulated likelihood estimator and the true maximum likelihood

estimator are asymptotically equivalent. In the empirical application, we use R = 50.8

Stage 2: Selection-Adjusted Earnings Equation

In the second stage, we estimate the selection-corrected earnings equation. As shown by Ham

(1982), the expectation of ity conditional on participating and being employed is (and ignoring

correlation across observations):9

( )* *it it it it 13 1t 23 2tE y | a 0, e 0 W β σ λ σ λ≥ ≥ = − − (4)

where

( ) ( )*1t it iti t 1

11Z Z / P

1θλ φ Φσ−

⎛ ⎞= ⎜ ⎟⎜ ⎟+⎝ ⎠

( )*2t it it it

22X X / P

1αλ φ Φσ

⎛ ⎞= ⎜ ⎟⎜ ⎟+⎝ ⎠

( )* 2it it i t 1

22 11Z X Z / 1

1 1α θρ ρσ σ−

⎛ ⎞= − −⎜ ⎟⎜ ⎟+ +⎝ ⎠

( )* 2it iti t 1

11 22X Z X / 1

1 1θ αρ ρσ σ−

⎛ ⎞= − −⎜ ⎟⎜ ⎟+ +⎝ ⎠

1it 2it

11 22corr ,

1 1ε ερσ σ

⎛ ⎞= ⎜ ⎟⎜ ⎟+ +⎝ ⎠

= ( )( )2211

12

11 σσσ

++

( )it iti t 111 22

P F Z ,X ,1 1θ α ρσ σ−

⎛ ⎞= ⎜ ⎟⎜ ⎟+ +⎝ ⎠

= ( )1,1Pr == itit ea

σ111+ and σ221+ are respectively the standard deviations of ε1it and ε2it , and ( ).φ and

( ).Φ are respectively the density and distribution functions of the standard normal and F is the

bivariate standard normal distribution function. In Ham (1982), 1itε and 2itε are ( )N 0,1 . To

produce the same situation, random terms in Equations (1) and (2) are transformed into

( )N 0,1 by dividing each equation by the standard deviation of its error term.

8 We initially estimated the model using R = 30. There is little change in the results when increasing the number of draws from 30 to 50. 9 We couldn’t find any study in the literature that corrects for two selection sources using panel data. Wooldrige (2001, Chapter 17), presents a case with one selection criterion.


15

For time period t = 1, there is only one source of selection which is employment status (same as

the case described by Heckman, 1979). In this case, the correction term is simply the inverse

Mills ratio α αφ Φσ σit it

22 22X / X

1 1

⎛ ⎞ ⎛ ⎞⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟+ +⎝ ⎠ ⎝ ⎠

. Finally, notice that coefficients on the

correction terms (- 13σ and - 23σ ) are time invariant,10 and that 1λ and 2λ are simply inverse

Mill's Ratios if the attrition and employment are independent (i.e. 12σ is nil).

Parameter estimates from the first stage are used to form consistent estimates 1tλ̂ and 2tλ̂ of 1tλ

and 2tλ . Then we estimate β , 13σ and 23σ by running random-effects GLS using the observed

sample (i.e. the sample satisfying ita 1= and ite 1= ):

*it it 13 1it 23 2it 3it

ˆ ˆy Z β σ λ σ λ ε= − − +

where ( ) ( )*3it 3it 13 1it 1it 23 2it 2it

ˆ ˆε ε σ λ λ σ λ λ= + − + −

In order to correct for general heteroskedasticity and first stage estimation, standard errors of

the slopes are bootstrapped based on 1000 replications.

IV- Empirical Results

Structural parameters estimates are obtained following the procedure described in Section III

and are presented in Tables 3 and 4.

Non-attrition and Employment Equations

At the outset, we notice that the estimated correlation between the unobserved determinants of

non-attrition and employment, it1ε and it2ε , is statistically significant, though it is small

(0.0227). We remind that the correlation between these terms is made only through the random

individual effect terms, itu1 and itu2 (cf. Equation 4). The estimated correlation between the

latter is 0.5809 and is highly significant. The positive and significant correlation between

participating in the survey and being employed indicates that the attrition behaviour is more

likely among non-employed compared to employed individuals. The latter are naturally less

10 See Wooldrige (2001, Chapter 17) for a general case where the coefficients on the error terms are time variant using one selection criterion.


16

mobile because of their work attachment and, consequently, are relatively easier to locate

especially in the subsequent waves of the survey.

Table 3: Estimated Censored Bivariate Selection Model

Non-attrition Equation Employment Equation

Variable Coefficients Standard errors Coefficients Standard

errors Constant 1.0353* 0.0461 4.4152* 0.0920 Age / 10 0.0684* 0.0063 -0.7191* 0.0139 Female 0.0559* 0.0124 -1.3516* 0.0363 Married 0.1700* 0.0208 0.1794* 0.0388 Single 0.0082 0.0244 -0.7583* 0.0521 Student - - -0.9279* 0.0270 Immigrant -0.1605* 0.0193 -0.4062* 0.0549 Education:

High school 0.0820* 0.0168 1.1596* 0.0321 Non-university degree 0.1114* 0.0171 1.9437* 0.0383 University degree 0.1503* 0.0208 2.3627* 0.0499

Household size 0.0149* 0.0049 - - Urban areas -0.0411* 0.0154 0.0046 0.0303 Urban population size /100,000 -0.0065* 0.0007 0.0054* 0.0016

Household income /100,000 0.0357* 0.0127 - - Moved during the reference year -0.1788* 0.0171 - -

Proxy for labour force status 0.0156 0.0132 - - Region of residence:

Quebec -0.0663* 0.0202 -0.0939 0.0524 Ontario -0.1328* 0.0189 0.4070* 0.0471 Prairies -0.0385 0.0201 0.8811* 0.0510 B.C. -0.0506* 0.0256 0.4614* 0.0678

Wave dummies: 1997 - - 0.1015* 0.0274 1998 0.1867* 0.0186 0.1134* 0.0253 1999 0.0584* 0.0182 -0.0376 0.0248 2000 -0.1827* 0.0174 -0.0232 0.0269 2001 0.1644* 0.0204 0.0659* 0.0269

σj (Standard deviation of the individual random effect term) 0.0425* 0.0137 2.3804* 0.0269

ρ12 (1) 0.0227* 0.0073 ρ (2) 0.5809* 0.0109 Mean Log-Likelihood -2.5026 Sample size 23,317

Notes: (1) Correlation coefficient between error terms (including individual effects) in attrition and employment equations. (2) Correlation coefficient between individual random effect in attrition equation and individual random effect in employment equation. * Significant at the 5% level


17

This negative relationship between attrition and work attachment is also supported by other

structural estimates in Table 3. Indeed, the variables that increase work attachment and/or

reduce mobility (for instance, age, education, family size, being married) also increase the

likelihood of remaining in the scope of the survey and cooperating. In the same vein, a person

who moved during a year (a signal of the geographical mobility) is more likely to drop from the

sample in the subsequent year. Results also indicate that being female and increased family

income also lowers this likelihood. The latter result is in agreement with MaCurdy, Mroz, and

Gritz (1998) who find that individuals who drop from the National Longitudinal Survey of

Youth (NLSY) come disproportionately from the low income population. In addition, being

immigrant or living in urban areas increases the probability of attrition or non-response.

Moreover, within urban areas, increased population size increases this probability, resulting in

rural respondents and respondents from small areas over-represented in the sample.

Another important point that is made obvious by Table 3 is that the probability of attrition/non-

response is not affected when the interview is given to a proxy instead of the sampled person.

By province, residents of Ontario are the most likely to attrite or not cooperate compared to

residents in other provinces. Living in Quebec or British Columbia (B.C.) also tends to increase

the likelihood of attrition and non-response. This result agrees with the one from the population

size since Ontario, Quebec and B.C. count the largest urban areas (Toronto, Montreal and

Vancouver for instance).

The study by Zabel (1998)11 using U.S. data leads to comparable results, with attritors having

lower labour force attachment compared to individuals who continue in the survey. The study

also finds the same direction of the effects of some variables on attrition especially with regard

to the effects of education, moving during the previous year and interviewing by proxy.

Coefficient estimates on wave dummies - all significant at the level 1% - support neither

positive duration dependence (increased degree of cooperation from surveyed individuals) or

negative duration dependence (decreased degree of cooperation), since these estimates do not

change monotonically. Consequently, survey participants do not seem to get more familiar with

the survey as their behaviour may vary from one year to another. The likelihood of attrition was

11 Zabel (1998) analyzes attrition behaviour in the Panel Study of Income (PSID) and the Survey of Income and Program Participation (SIPP) in the U.S.A.


18

the highest in 1997 and 2000 and the lowest in 1998 and 2001, a fact that agrees with statistics

in Table 1.

With regard to employment determinants, first we remind that non-employed population

includes unemployed as well as persons who are out of the labour force. Hence, a variable that

decreases the probability of employment does not necessarily increases the probability of

unemployment. Increased schooling or population size increases the probability of

employment. Residents of large urban areas enjoy more employment opportunities and are

more likely to enter the labour force compared to residents of small urban or rural areas. On the

other hand, increased age, being female, non-married, student or immigrant reduces the

probability of employment. The latter also varies across regions of residence. Individuals from

Western Canada and Ontario are likely to be employed. Once again, coefficient estimates on

wave dummies do not show a monotonic trend in the probability of being employed.

Coefficient estimates on 1997, 1998 and 2001 dummies are positive and significant, but those

on 1999 and 2000 are negative and non-significant.

In conclusion, results from Table 3 indicate that individuals who leave the sample or do not

cooperate, likely are young, not married, from low-income and small families, and have lower

education levels. These same characteristics point to a lower degree of attachment to

employment and a higher level of geographical mobility.

With regard to unobservable factors, results suggest that individual specific effect is more

important on employment than on attrition and non-response behaviour. The estimated variance

of the individual random effect term is much larger in the employment equation. Thus, after

controlling for observed characteristics, there is still more remaining heterogeneity between

individuals concerning employment than concerning attrition.

Earnings Equations

We focus on analyzing the biases that arise from ignoring attrition and employment selections

on the estimation of the earnings equation rather than analyzing the effects of covariates on the

earnings level. Selection adjusted earnings equation estimates along with unadjusted estimates

are presented in Table 4. Potential years of experience are calculated as the difference between

age at the time of the survey and estimated number of years of schooling corresponding to the

education level of the respondent. We eliminate workers aged 16 with a high school diploma or


19

more, aged 17 with more than high school, aged 18 to 21 with a bachelors’ degree or more, and

aged 22 with a post-graduate university degree.

Table 4: Estimated Earnings Equations

Selection adjusted Unadjusted Variable Coef. Std. Err. Coef. Std. Err.

Constant 7.602 0.0271 7.1804 0.0219 Potential experience 0.0511 0.0007 0.0441 0.0007 Potential experience squared -0.0004 0.0000 -0.0004 0.0000

Female -0.3092 0.0095 -0.523 0.0118 Education:

High school graduate 0.2495 0.0184 0.4671 0.0131 Post-secondary (non-university) 0.3643 0.0201 0.7073 0.0140

University degree 0.6430 0.0223 1.0364 0.0169 Urban areas 0.0700 0.0097 0.0726 0.0095 Region of residence:

Quebec 0.1566 0.0116 0.1528 0.0181 Ontario 0.1986 0.0133 0.2825 0.0160 Prairies 0.0035(ns) 0.0138 0.1416 0.0170 B.C. 0.1548 0.0181 0.2366 0.0221

Immigrant -0.0185(ns) 0.0086 -0.0812 0.0192 # Worked weeks 0.0289 0.0004 0.0298 0.0002 Correction term 1 (non-attrition) 0.5700 0.0293 -- --

Correction term 2 (employment) -1.0849 0.0426 -- --

Notes: - Standard-Errors in the selection adjusted equation are bootstraped using 1000 replications - ns: Non significant at the level 5 percent. All other coefficients are significant at the level 5 percent.

The most interesting result is that coefficient estimates on correction terms are significant, a

fact that provides an additional evidence for non-randomness of attrition and employment

behaviours. However, the data indicates that the extent of selection from employment is more

significant than the extent of selection from attrition. The earnings gap between the selected

(available) sample and a sample drawn randomly with identical observed characteristics is

estimated at -6.75% due to non-attrition selection as opposed to +34.4% due to employment


20

selection.12 In other words, there is limited bias, though significant, from attrition compared to

the bias from the censure due to non-employment.

For comparison, Table 4 also shows unadjusted earnings estimates obtained by ignoring the two

correction terms. By doing so, the effects of several covariates on annual earnings are

overestimated. For instance, selection adjustment yields to a male-female gap of 30.9% versus

52.3% without such adjustment. Similarly, the gap between a non-immigrant and an immigrant

decreases significantly, dropping from 8.1% to 1.9%. On the other hand, returns to education

are underestimated, since they decrease significantly after the selection correction. With regard

to the province of residence, the coefficient on dummy variable “Prairies” is the most affected

by the correction.

V- Concluding Remarks

The SLID is increasingly used by a diverse range of policy and academic researchers to

investigate education, family, income and labour dynamics in Canada. Given the substantial

impact that analyses using SLID have on policy and academic knowledge, it is important to

evaluate the extent of selection bias from attrition and non-response and its impact on estimates.

This study examines the specific effect of attrition and non-response among working-age

longitudinal respondents on employment and earnings estimates. Our structural model is

computationally demanding, but yields efficient parameter estimates. We find evidence for the

non-randomness of the attrition behaviour in the SLID. The correlation coefficient between

random components in non-attrition and employment equations is positive and significant,

though small. Generally, the factors that increase the probability of being employed also

increase the probability of being in the scope of the survey and responding to the labour

interview. Also, the coefficient on the attrition correction term in the earnings equation is

significant. However, the estimated earnings gap between non-attritors and a random sample

with the same characteristics is mild. These findings agree with those from different studies on

the same subject, but using American and Dutch data.

12 The gap is calculated by multiplying minus the selection coefficient times the mean value of the correction term.


21

Further research into the impact of attrition and non-response on estimates from longitudinal

surveys in Canada is warranted. In particular, evaluations of data quality for those estimates

that are central to public policy development is necessary.


22

References

Burkam, D. T.; V. E. Lee (1998). “Effects of Monotone and Nonmonotone Attrition on Parameter Estimates in Regression Models with Educational Data: Demographic Effects on Achievement, Aspirations, and Attitudes,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 555-574.

Becketti, S.; W. Gould; L. Lillard; F. Welch (1988). “The Panel Study of Income Dynamics after Fourteen Years: An Evaluation,” Journal of Labor Economics, Vol. 6, No. 4 (Oct., 1988), pp. 472-492.

Boudarbat, B.; T. Lemieux and C. Riddell (2005). “Recent Trends in Wage Inequality and the Wage Structure in Canada,” forthcoming in “Inequality In Canada,” D. Green and J. Kesselman (eds.) UBC Press.

Evangelos M. F.; H. Elizabeth Peters (1998). “Survey Attrition and Schooling Choices,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 531-554.

Falaris, E. M.; H. E. Peters (1998). “Survey Attrition and Schooling Choices,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 531-554.

Fitzgerald, J.; P. Gottschalk and R. Moffitt (1998). “An Analysis of Sample Attrition in Panel Data: The Michigan Panel Study of Income Dynamics,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 251-299.

Fitzgerald, J.; Gottschalk, P. and Moffitt, R. (1998). “An Analysis of the Impact of Sample Attrition on the Second Generation of Respondents in the Michigan Panel Study of Income Dynamics,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 300-344.

Green, W. H. (2003). “Econometrics Analysis,” 5th edition, Prentice Hall, 2003.

Green, W. H. (2002). “Convenient Estimators for the Panel Probit Model: Further Results,” Department of Economics, Stern School of Business, New York University,Revised, October, 2002

Griliches, Z.; B.H. Hall, and J.A. Hausman (1978). “Missing Data and Self Selection in Large Panels,” Annales de l'INSEE (Avril-Sept. 1978).

Gourieroux, C. and A. Monfort (1996). “Simulation Based Econometric Methods,” Oxford University Press, New York, 1996.

Hausman. J.A.; D.A. Wise (1979). “Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment,” Econometrica, Vol. 47, No. 2. (Mar., 1979), pp. 455-474.

Heckman, J. (1979). “Sample selection bias as a specification error,” Econometrica 47 : 153-161

Heckman, J., and Singer, B. (1984). “A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data,” Econometrica 52 (Mar. 1984), 271-320.

Lancaster, T. (1990). “The Econometric Analysis of Transition Data,” Cambridge University Press.


23

Lee A. Lillard; Constantijn W. A. Panis (1998). “Panel Attrition from the Panel Study of Income Dynamics: Household Income, Marital Status, and Mortality,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 437-457.

Levesque, Isabelle; Sarah Franklin (2000). “Longitudinal and Cross-Sectional Weighting of the Survey of Labour and Income Dynamics”. Catalogue no. 75F0002MIE2000004. Ottawa: Statistics Canada.

MaCurdy, T.; Mroz, T.; Gritz, R.M. (1998). “An Evaluation of the National Longitudinal Survey of Youth,” Journal of Human Resources, Vol. 33(2), pp. 345-436.

Maddala, G.S. (1983). “Limited Dependent and Qualitative variables in Econometrics,” Cambridge University Press, 1983.

Michaud, Sylvie; Michel Latouche (1996). “Some Data Quality Impacts When Merging Survey Data on Income with Tax Data”. Catalogue no. 75F0002MIE1996012. Ottawa: Statistics Canada.

Michaud, Sylvie; Maryanne Webber (1994). “Measuring Non-Response in a Longitudinal Survey: The Experience of the Survey of Labour and Income Dynamics”. Catalogue no. 75F0002MIE1994016. Ottawa: Statistics Canada.

Nijman, T. and Verbeek, M (1992). “Nonresponse in Panel Data: The Impact on Estimates of a Life Cycle Consumption Function,” Journal of Applied Econometrics, Vol. 7, No. 3. (Jul. - Sep., 1992), pp. 243-257.

Ridder, Geert (1990). "Attrition in Multi-Wave Panel Data," in Hartog, et al. (eds.), Panel Data and Labor Market Studies. Amsterdam: North-Holland, 45-67.

Train, K., Discrete Choice Methods with Simulation, Cambridge University Press, Cambridge, 2002.

Statistics Canada (1995). Measuring Non-Response in a Longitudinal Survey: The Experience of the Survey of Labour and Income Dynamics. Catalogue no. 75F0002MIE1994016. Ottawa.

Statistics Canada (1996). Some Data Quality Impacts When Merging Survey Data on Income with Tax Data. Catalogue no. 75F0002MIE1996012. Ottawa.

Statistics Canada (1997). Survey of Labour and Income Dynamics Microdata User’s Guide. Catalogue no. 75M0001GIE. Ottawa.

Statistics Canada (1998). “Impact of Edit and Imputation on Income Estimates: A Case Study,” Catalogue no. 75F0002MIE1998012. Ottawa.

Statistics Canada (2000). “Longitudinal and Cross-Sectional Weighting of the Survey of Labour and Income Dynamics.” Catalogue no. 75F0002MIE2000004. Ottawa.

Van Den Berg, G. J.; M. Lindeboom; G. Ridder (1994). “Attrition in Longitudinal Panel Data and the Empirical Analysis of Dynamic Labour Market Behaviour,” Journal of Applied Econometrics, Vol. 9, No. 4. (Oct. - Dec., 1994), pp. 421-435.

Van Den Berg, G.J.; Maarten Lindeboom (1998). “Attrition in Panel Survey Data and the Estimation of Multi-State Labor Market Models,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 458-478.


24

Webber, Maryanne; Cathy Cotton (1998). “Impact of Edit and Imputation on Income Estimates: A Case Study”. Catalogue no. 75F0002MIE1998012. Ottawa: Statistics Canada.

Wooldridge, J. M. (2001). “Econometric Analysis of Cross Section and Panel Data,” M. Cambridge, Mass: The MIT Press, 2001.

Zabel, J. E. (1998). “An Analysis of Attrition in the Panel Study of Income Dynamics and the Survey of Income and Program Participation with an Application to a Model of Labor Market Behavior” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 479-506.

Ziliak, J.P.; T.J. Kniesner (1998). “The Importance of Sample Attrition in Life Cycle Labor Supply Estimation,” The Journal of Human Resources, Vol. 33, No. 2 (Spring, 1998), pp. 507-530.

Documents

Attrition and Non-response in the Survey of Labour and