View
216
Download
2
Tags:
Embed Size (px)
Citation preview
Cultural differences as statistical artefacts?
Reanalysing cross-national data with more advanced techniques
QMSS Conference Prague 21/06/2007
Dr Michael HoelscherDepartment of Education
University of Oxford
Overview
Context of the study Introduction to data Cultural differences within Europe – a first
approach Reanalysing the data with CFA Applying a correction for measurement errors Conclusions
1. Context of the study
European integration and enlargement often discussed in economic terms only
However: Cultural influences might play a crucial role Comparison of values in different spheres for all
countries in the EU (Religion, Family and Gender, Economy, Welfare State, Democracy)
“Normative” starting point: Position of the EU institutions, as found in its body of law and the treaties
Three year project, financed by VolkswagenStiftung
2. Introduction to data
European Values Study– 1999/2000– Wide variety of topics– Including all member and applicant countries of the EU
(except Cyprus)
28 countries are compared in our study Today the focus is on “Democracy and Civic Society” Secondary analysis
– Indicators are not always “perfect”
2. Introduction to data
Democracy: 4 Indicators
– “Having a strong leader” (v216)– “Having the army ruling” (v218)– “Having a democratic political system” (v219)– “Democracy may have problems, but best form of
government” (v220)
(all measured on a scale with 4 categories)
2. Introduction to data
Civic Society:
2 Indicators
– “People can be trusted” (v66)– “Membership in voluntary organisations”
(Index of membership in 14 groups; trade union membership is ignored)
3. Cultural differences within Europe – a first approach
Methods Comparisons of raw country means for each
indicator Integration of single indicators by using a
discriminant analysis
(see Fuchs/Klingemann 2002 in “West European Politics”)
Explanation of differences on the individual level by multiple regressions
3. Cultural differences within Europe – a first approach
Results Large differences between the countries, but
also within the countries Old member countries support position of EU
most, followed by new members Bulgaria, and especially Romania and Turkey
showed much lower support
3. Cultural differences – a first approach
Sweden 1Netherlands 2Denmark 3Finland 4Austria 5Belgium 6Germany_West 7Greece 8Luxembourg 9Ireland 10Czech Republik 11Germany_East 12Italy 13Malta 14France 15Slovenia 16Spain 17Slovakia 18Great Britain 19Estonia 20Hungary 21Bulgaria 22Portugal 23Poland 24Lithuania 25Latvia 26Romania 27Turkey 28
Overall support for the EU’s position in the field of Democracy/Civic Society
(RANK)
4. Re-analysing the data with CFA
Aim – To compare two different methods– Not: Building the best model!
Balance of model fit and equivalence of approaches is needed
4. Re-analysing the data with CFA
Advantages of CFA
Generally– CFA is the more appropriate technique– More flexible– Can easily be extended to an explanatory SEM
4. Re-analysing the data with CFA
Advantages
Measurement model– Test if measurement is the same in different
countries and therefore a comparison is appropriate
– Correction for measurement error possible (Saris/Gallhofer 2007)
4. Re-analysing the data with CFA
Advantages
Structural model– Relationship between “democracy” and “civic
society” can be estimated
4. Re-analysing the data with CFA
Great Britain, N = 728
Democracy
Zv216 e1.50
Zv218 e2.39
Zv219r e3.73
Zv220r e4.59
Civic SocietyZv66r e5
Zlmember e6
.53
.34
.39
.23
Standardized estimateschi-square=9.505 df=7 p-value=.218
gfi=.996 agfi=.987 rmsea=.022
4. Re-analysing the data with CFA
Running the model for all 25 countries without constraints
Chi-square = 333.29, df = 175, p-value=.000 CFI = .988 RMSEA = .006 (adjusted: 0.032)
All modification indices within the countries are well below 20, in most cases below 5
One can assume configural invariance
Introducing constraints: Model comparison
4. Re-analysing the data with CFA
2. Model “Equal Measurement Weights”Chi-square= 725,5 df = 271CFI = .965RMSEA = .008 (adjusted .04)
3. Model “Equal Measurement Weights and Intercepts”Chi-square= 4752.256 df = 367CFI = .666RMSEA = .023 (adjusted .115)
1. Model “Unconstrained”Chi-square= 333,29 df = 175CFI = .988RMSEA = .006 (adjusted .032)
Introducing constraints: Model comparison
4. Re-analysing the data with CFA
2. Model “Equal Measurement Weights”
=> Metric invariance can be assumed
3. Model “Equal Measurement Weights and Intercepts”
=> Scalar invariance can not be assumed!=> Mean comparison is (in priniciple) not appropriate with this model=> Adjustments (freeing some parameters)
1. Model “Unconstrained”
=> Configural invariance can be assumed
4. Re-analysing the data with CFA
SwedenCFA-Rank
1RANK CFA-Book
0Denmark 2 -1Netherlands 3 1Austria 4 -1Luxembourg 5 -3Germany_West 6 -1Belgium 7 1Italy 8 -4Ireland 9 0Finland 10 6Czech Republic 11 1Malta 12 -1France 13 -1Romania 14 -11Spain 15 -1Slovakia 16 -1Slovenia 17 2Germany_East 18 7Great Britain 19 1Bulgaria 20 -1Hungary 21 1Estonia 22 3Lithuania 23 0Poland 24 2Latvia 25 1
Comparison of ranks
5. Correction for measurement errors
SEM allows to correct for measurement errors Saris, Gallhofer et al. (2007) have introduced a tool
to estimate the quality (reliability and validity) of an instrument
From a huge amount of MTMM experiments they estimated the influence of certain characteristics on the quality
By coding one’s own questions one can predict their quality
=> http://www.sqp.nl/
5. Correction for measurement errors
Idea:– What has to be equal for cross-country-comparisons
is the factor structure– The quality of the instrument might influence this
factor structure, if one does not correct for measurement error if the quality is different in different countries
“We suggest that equivalence should (…) be tested by the equality of loadings based on the observed covariance matrix corrected for measurement error”
5. Correction for measurement errors
y1 T1
F
e1
q1
λ1
y2e2
q2 T2
y3e3
q3 T3
λ2
λ3
Indicators True scores Latent concept (by definition)
5. Correction for measurement errors
Applying the correction to a subsample of 9 countries:
- “Democracy”-indicators - The validity was nearly 1 for all countries- Reliability is different in countries, but reasonably
good- Problems with the “Civic Society”-indicators
- Unable to code the quality of the index straightforward
- Low quality of the “Trust” variable
5. Correction for measurement errors
Results:
- Factor loadings increase- Model fit decreases very slightly- At least for this specific subsample the ranks
do not change - Check for whole sample, especially the
“difficult” cases, is still missing
6. Conclusions
Advantages of the SEM approach
More appropriate More flexible (integration of additional indicators) Can detect problems with measurement model Easily extendable to an explanatory model Relationship between the latent constructs can
be estimated
6. Conclusions
“Problems” of the SEM approach
More demanding (data quality) Is it realistic to assume equal means and factor
loadings over so many countries? Partial invariance?
Taking requirements very seriously wouldn’t allow a comparison of all countries
6. Conclusions
Comparing the “outcome” of the three methods:
Small differences for the overall ranking
The methods seem to come to pretty similar results
However: Some extreme cases (Turkey), couldn’t be included or shifted quite a lot (Romania)
Thank you!
Dr Michael HoelscherDepartment of Education
University of [email protected]
Quantitative Methods in the Social Sciences Conference, Prague, 20-23 June 2007
Literature
Michael Hoelscher (2006): Wirtschaftskulturen in der erweiterten EU. Die Einstellungen der Buergerinnen und Buerger im europaeischen Vergleich. Wiesbaden: VS Verlag
Juergen Gerhards (unter Mitarbeit von Michael Hoelscher) (2005, second edition 2006): Kulturelle Unterschiede in der Europaeischen Union. Wiesbaden: VS Verlag
Dieter Fuchs/Hans-Dieter Klingemann (2002): Eastward Enlargement of the European Union and the Identity of Europe. West European Politics, 25, 2: 19-54.
Willem E. Saris/Irmtraud Gallhofer (2007): Design, Evaluation, and Analysis of Questionnaires for Survey Research. Wiley.