This project has received funding from the European Union’s Seventh Programme for Research, Technological Development and Demonstration under Grant Agreement No 312691 www.inclusivegrowth.be

SEMINAR AT HIVA ■ RESEARCH INSTITUTE FOR WORK AND SOCIETY

The use of auxiliary and event data in tracking an inhomogeneity of substantive data in longitudinal studies.

The cases of ESS Round 7th and EWCS Wave 5th

Teresa Żmijewska-Jędrzejczyk

Institute of Philosophy and Sociology at the Polish Academy of Sciences

PhD Student: University of Warsaw

Questions: teresa.zmijewska@ifispan.waw.pl

Friday, 18 November 2016

2

Agenda

Homogeneity of a data in longitudinal crossnational studies –

why important?

The necessity to track transitory factors in trend analysis

Why EWCS and ESS data?

EWCS and ESS data – results of the study

Summary and problems to be solved

Disscussion

3

Why important?

Standardization as a key premise of survey

The challenges of measurement validity in broad sense: non-response bias, measurement equivalence

»Ref.: Jowell 2004, Billiet 2015

»Ref.: Commission de sondages

Statistical analysis of hierarchical models

4

Surveys are prone to the different effects

Among long list of the different surveys’ effects, three types are relevant to the length of the survey period

Context Effects Survey climate (Ref.: Loosveldt & Joye 2016)

Sudden and long-lasting events’ effects

Interviewer Effects Fatigue Effects (interviewer loses focus, become tired)

Learning Effects (interviewer knows by heart a questionnaire)

Effects connted with the (in)stability of the attitudes

Mode effects

Many others (Ref.: Weisberg H. F. 2005)

5

Patterns of the an inhomogeneity of substantive data - examples

6

The necessity to track transitory factors

In trend analysis from longitudinal studies the postulates of homogeneity of the data

As a result of the different conditions under which cross-country and longitudinal studies are carried out, it is necessary to collect contextual data about the economic situation, the level of democratization, political regime, political events like the imminence of general elections

That impose also the necessity to control the impact of transitory factors - events

Such factors might biased the long-term trends in attitudes!

7

Rationale for selecting data

Cross-country with long tradition

Longitudinal and long fieldwork period

High quality data and different topics

8

Procedure

1 step

• All variables

• All countries

2 step

• Select the countries for further analysis

• Variable prone to be an inhomogeneous

3 step

• Marked variables and countries

• Additional controlled variables

European Working Conditions Survey

5th wave (2010)

Representative sample based on the whole country population 15+ or 16+ (Spain, UK, Norway) Coverage: EU27, Norway, Croatia, FYROM, Turkey, Albania, Montenegro and Kosovo Main subject: provide an overview of the everyday reality of men and women at work Frequency: since 1990 every 5 years

Aim: derive those variables and those countries prone to be an inhomogeneous (red flags)

1st step

All variables and all countries

12

Method and assumptions

DV: list of 390 variables - all for which the scale level was know were used (ISCO occupation classification excluded)

IV: date of the fieldwork (month)

No weights applied

=.001

The estimation method (and test stat) for the variables Metrics: OLS (F-test),

Ordinal: ordered probit (chi2)

Nominal: multinomial logit regression (chi2)

No control variables

Sem Vandekerckhove

13

Results: red flags Country red flags

Turkey N=2100 64

France N=3037 50

Albania N=1000 48

Belgium N=3995 40

Portugal N=1000 30

Luxembourg N=999 27

Latvia N=1001 22

Croatia N=1100 21

Germany N=2133 17

Slovenia N=1404 17

Greece N=1037 15

Country red flags

UK N=1575 14

FYROM N=1100 14

Malta N=1000 13

Ireland N=1003 12

Norway N=1085 11

Montenegro N=1041 11

Kosovo N=1018 10

Bulgaria N=1014 8

Estonia N=1000 8

Netherlands N=1017 8

Austria N=1003 7

Poland N=1497 7

Country red flags

Sweden N=1004 5

Spain N=1007 4

Italy N=1500 4

Hungary N=1006 4

Romania N=1017 4

Czech Rep. N=1000 3

Cyprus N=1000 3

Lithuania N=1004 3

Denmark N=1069 1

Slovakia N=1002 1

Finland N=1028 1

14

Results: red flags by scale and topic

Country Total Nominal Ordinal Metrics Topics

Turkey N=2100 64 31 30 3 All

France N=3037 50 29 18 3 All

Albania N=1000 48 21 24 3 All

Belgium N=3995 40 23 14 3 All

Portugal N=1000 30 11 14 5 All

Different sample size: Belgium, France vs. Albania, Portugal Differnt suvey climate across Europe Variable prone to be an inhomogeneous are from different topics

European Social Survey

7th round of ESS (2015/2016)

an academically driven

Sample based on the whole country population 15+ Coverage: crosseuropean – in 2015/16: Austria, Belgium, Czech Republic, Denmark, Estonia, Finland, France, Germany, Hungary, Ireland, Israel, Latvia, Lithuania, Netherlands, Norway, Poland, Portugal, Slovenia, Spain, Sweden, Switzerland, United Kingdom Main subject: since 2001 every 2 years measures the attitudes, beliefs and behavior patterns of diverse populations across Europe

16

Method and assumptions

DV: Substantive questions selected from the core questionnaire, e.g.: trust in public institutions, assessment of the public institutions (incl. the European Parliament and the attitude to European integration), political activity, subcetive well-being, happiness

IV: date of the fieldwork (days)

No weights applied

The estimation method (and test stat) for the variables

– 11 point numeric scales: linear regression

– ordinal variables: ordinal logistic regression (PLUM)

– dichotomous variables: binary logistic regression

No control variables

17

Results for France

***p=<0,001 **p<0,05 *p<0,1

b

Trust: parliment 0,013***

Trust: legal system 0,007**

Trust: police 0,011***

Trust: politicians 0,009***

Trust: political parties 0,010***

Trust in the European Parliament 0,010***

How satisfied with the national government 0,013***

How satisfied with the way democracy works in country 0,016***

State of health services in country nowadays 0,005*

18

Summary

Variable prone to be an inhomogeneous are from different topics and they have more country specific patterns

Thus fieldwork period is an important element in the proces of assessment the data homogeneity, both in terms of measurement equivalence and statistical analysis (e.g.: hierarchical models) in cross-country comparisons

Tracing the homogeneity of the data is still a challenge for researchers. Why? In cross-countries data sets it may be difficult to formulate even preliminary hypotheses what kind of transitory factors, effects or events are relevant and what variables might be affected

The impact of transitory and non-transitory factors

Where’s variable quality hiding?

20

Other effects – example from the ESS

Demographics

Gender

Age

Education

Rural vs. urban area

Event data to control the impact of transitory factors

21

Charlie Hebdo terrorist attack

France, 7.-9. 01.2015 r.

Charlie Hebdo satirical weekly magazine.

The magazine has been the target of two terrorist attacks, in 2011 and 2015.

Both were presumed to be in response to a number of controversial Muhammad

cartoons it published. In the second of these attacks, 12 people were killed, including

publishing director Charb and several other prominent cartoonists.

22

London

Paris

Charlie Hebdo solidarity acts

The Je suis Charlie ("I am Charlie") slogan became an endorsement of freedom of speech and press.

23

Charlie Hebdo - > influence on the fieldwork The front cover of edition of 14 January 2015,

with a cartoon in the same style as 3 November 2011 cover,

uses the phrase "Je Suis Charlie".

(Headline translation: "All is forgiven.")

ESS 7 in France

Fieldwork: 31. 10. 14 – 03. 03. 2015

Sample: N=1 917

RR=50.9%

24

Method and assumptions

DV: same substantive questions

IV: gender, age, education (5 category),

place of residence: 1 - big city or its suburbs (26.4%)

roots outside France: 1 - respondent, mother/father born outside

(24.6%)

interview after the attack on Charlie Hebdo:

1 - yes: N = 275 interviews started on January 7 by 14 or later (14.3%)

0 - no: N = 1642

Design weight applied

25

Results for France

***p=<0,001 **p<0,05 *p<0,1

b

Trust: parliment 0,586***

Trust: police 0,268

Trust: politicians 0,475***

Trust: political parties 0,563***

Trust in the European Parliament 0,396**

European unification go further or gone too far 0,007*

How satisfied with the national government 0,801***

How satisfied with the way democracy works in country 0,913***

State of health services in country nowadays 0,176

26

Comparison: results with and without CV

***p=<0,001 **p<0,05 *p<0,1

b b without CV

Trust: parliment 0,586*** 0,013***

Trust: police 0,268 0,009**

Trust: politicians 0,475*** 0,011***

Trust: legal system - 0,007***

Trust: political parties 0,563*** 0,010***

Trust in the European Parliament 0,396** 0,010***

European unification go further or gone too far 0,007* 0,007*

How satisfied with the national government 0,801*** 0,013***

How satisfied with the way democracy works in country 0,913*** 0,016***

State of health services in country nowadays 0,176 0,005*

27

Summary

In many cases differences are significant

Analyses indicate that the event not related thematically, influence the process of formulating opinion on various topics

Again, tracking the effects of contextual current events is an important element of the assessment of their value in terms of homogeneity

DV: MIP QUESTION for EWCS EVENTS/CLAIMS for ESS CV: WEEK OF THE FIELDWORK and other control variables for a group of countries

Further analysis

This project has received funding from the European Union’s Seventh Programme for Research, Technological Development and Demonstration under Grant Agreement No 312691 www.inclusivegrowth.be

SEMINAR AT HIVA ■ RESEARCH INSTITUTE FOR WORK AND SOCIETY

Dank je!