Errors in Factual Questions and their Consequences

Annette Scherpenzeel

QMSS Seminar

12-08-2004 Lugano

Swiss Household Panel

AimStudy of social change, in particular, the dynamics of living conditions in the population of Switzerland

MethodThe survey "Living in Switzerland" is carried out among a sample of households representative of the Swiss resident population of 1999.

ThemesComputer-assisted telephone interviews are conducted with every person older than 14 in the households.

PeriodBoth objective data (resources, living conditions, life events, social position, participation, etc.) and subjective data (attitudes, perceptions, satisfaction, values, lifestyles, etc.) are collected

ParticipationSince 1999, five waves have been carried out. The sixth will start in September 2004

FieldworkIn the first wave, 7799 persons in 5074 households have been interviewed. In the fifth wave, 5226 persons in 3296 households have been interviewed.

Finance and organization

The telephone interviews are conducted every year by the Institute M.I.S. Trend in Lausanne and Bern in German, French and Italian. The household-level interview takes on average 12 minutes, the individual interviews on average 55 minutes.

International Standard Classification of Occupations (ISCO)

1. Legislators, senior officials, and managers

2. Professionals

3. Technicians and associate professionals

4. Clerks

5. Service workers and shop and market sales workers

6. Skilled agricultural and fishery workers

7. Craft and related trades workers

8. Plant and machine operators and assemblers

9. Elementary occupations

10. Armed forces

ISCO Example1

[1] Example taken from Bergman and Joye, "Comparing Social Stratification Schemas".

Nuclear physicist

unit group 2111 (physicists and astronomers)

minor group 211 (physicists, chemists and related professions)

sub-major group 21 (physical, mathematical and

engineering science professionals)

group 2 (professionals)

Elementary occupations

Plant and machine operator assemblers

Craft and related trades workers

Skilled agriculturaland fishery workers

Service workers, market sales workers

Clerks

Technicians and ass-ociate professionals

Professionals

Legislators, senior officials, managers

n = 4993 in 1999, n = 3783 in 2002

Year of panel data collection

2002200120001999

Per

cen

tag

e

100

80

60

40

20

0

11111211

12111213

14141413

25262625

19191817

8878

International Standard Classification of Occupations Distributions of first four waves

International Standard Classification of Occupations Change between first and second wave

4 %

n = 4993 in 1999, n = 4673 in 2000

Year of panel data collection20001999

Per

cen

tag

e

100

80

60

40

20

0

1211

1213

1413

2625

1817

78

64 %

3 %

1 %

1 %

6 %

15 %

4 %

Elementary occupations

Plant and machine operator assemblers

Craft and related trades workers

Skilled agriculturaland fishery workers

Service workers, market sales workers

Clerks

Technicians and ass-ociate professionals

Professionals

Legislators, senior officials, managers

VariableALL working

persons

Persons who have NOT

changed job or employer

Persons who have changed

job or employer

  % % %

Type of employment: Self-employed, partner, private, employee 80 81 72

Company: Number of employees (9 response categories) 59 63 24

Percentage of part-time work (recoded into categories) 67 68 40

Type of working hours: fixed, flexible, rotating 57 58 47

Qualifications for job: Not sufficient, correspond, superior, not related 76 77 64

Position: Management, training, production 68 68 68

Job tasks: Design, consulting, analysis 51 51 49

Job with supervisory tasks: Yes, no 76 77 64

Job with participation in decision making: Decision, opinion, no 73 73 67

International standard classification of occupation (ISCO) 68 70 51

Wright 64 66 49

Goldthorpe 59 60 46

Swiss Socio-Professional Categories (CSP-CH) 70 72 49

Camsis 54 56 33

 n*  3828 3439 383 * Total number of working persons with completed individual interviews in both 1999 and 2000. For some variables in the table the n was smaller because of filters to the questions (e.g. "Percentage of part-time work" was only asked to people working part-time).

Stability of various other demographic variables

Percentage of people with the same score in 1999 and 2000

* Total number of persons with completed individual interviews in both 1999 and 2000. For some variables in the table the n was smaller because of filters to the questions ** Total number of working persons with completed individual interviews in both 1999 and 2000. For some variables in the table the n was smaller because of filters to the questions.

Variable All persons

%

Highest level of education achieved: 10 categories 95

Actual occupation: Paid work, school, family, retired, unemployed, etc., 10 categories

94

Member of which political party: 16 categories 90

From work module:

Job limitation in time, Yes, no 92

Part-time or Full-time 92

Private company or government organization 91

n 10112* / 3828**

Four-wave Simplex model of ISCOCategory 5 (Service workers, market sales workers)

e3 e4

T2ISCOcatg 5

isc25

e2

0.82

T3ISCOcatg 5

isc35

0.82

T4ISCOcatg 5

isc45

0.82

T1ISCOcatg 5

isc15

e1

0.83

0.93 0.94 0.99

d1 d2 d3

Four-wave Simplex model using one indicator of hierarchy of work-position.

T2manage

p00w34

e2

0.81

T3manage

p01w34

e3

0.79

T4manage

p02w34

e4

0.77

T1manage

p99w34

e1

0.78

0.99 0.93 0.99

d1 d2 d3

Coefficients estimated with the four-wave Simplex model for all ISCO categories.

n in 1999* Reliability coefficient

Stability coefficient

ISCO (dummy variables) T2T1 T3T2 T4T3

1 Legislators, senior officials, managers 210 0.71 0.88 0.83 0.92

2 Professionals 467 0.85 0.95 0.96 0.99

3 Technicians and assoc. professionals 637 0.77 0.97 0.94 0.99

4 Clerks 312 0.81 0.95 0.94 0.97

5 Service workers, market sales workers

247 0.82 0.93 0.94 0.99

6 Skilled agriculture and fishery workers

71 0.95 0.91 0.98 0.98

7 Craft and related trades workers 249 0.88 0.96 0.96 0.96

8 Plant and machine operator assemblers

69 0.78 - 0.83 0.89 0.84 1

9 Elementary occupations 104 0.78 0.82 0.95 1

n total** 2370

* Frequency distribution of ISCO-1999 for persons having a valid ISCO score in all four waves.** Total number of persons who have a valid ISCO score in all four waves.

General two-wave causal model of satisfaction

condition 1 in wave 1

condition 1 in wave 2

condition 2 in wave 1

condition 2 in wave 2

condition 3 in wave 1

condition 3 in wave 2

satisfaction in wave 1

satisfaction in wave 2

relative situation

event between waves

StableComponent

Contribution of the different factors to the explanation of satisfaction

DOMAIN

relationsworkhealthfinances

Per

cen

tag

e o

f ex

pla

ined

var

ian

ce80

60

40

20

0

FACTOR

Stable component

Relative situation

Events

Living conditions

28

48

11

23

11 24

12

3841

Conclusions

• Repeated cross-sections give a false impression of stability in "objective" variables. In reality, these variables can contain considerable random variation

• This unreliability can only be known by repeating the same question in each wave. Because of the nature of the "objective" variables, we can distinguish unreliability from real change

• The unreliability in the "objective" variables affects their explicative power in longitudinal models. It impairs the opportunities to analyze labor market mobility by panel studies

Discussion: What is the cause

• Swiss Household Survey data collection• Not likely: many other variables are stable

• ISCO coding database• SOEP-EG: 20% no change in occupational title but change in

occupational class.

• But: other working variables also unstable?

• Interviewer: interpretation, training, search strategy, etc • Explains only about 2% of the variation over waves

• Time-interval between measurements: missing events? • Selection bias: attrition nor occupational mobility are randomly sampled

• But: this would select for stability overestimation?

• Respondent: interpretation, frame of reference, memory, satisfice, etc • Question formulation

Next:

• ISCO: Compare with retrospective data from the same respondents

• ISCO: Compare ISCO occupational title with occupational class

• Other variables: methodological experiment?