Embed Size (px)
Citation preview
Measurement Equivalence with Multilevel Constructs
Paul J. Hanges & Michele J. GelfandUniversity of Maryland
Oct 1, 2004
Measurement Equivalence
Comparison between cultures
Measurement EquivalenceAssessment of equality of scale properties in multiple groups/countries
Structural Equation Modeling approach• Multigroup confirmatory factor analysis
– Spini (2003)– Steenkamp & Baumgartner (1998)
Multigroup CFACountry 1 Country 2
Construct
Item 3Item 1 Item 2
Construct
Item 3Item 1 Item 2
Problems applying technique to group-level constructs
Assumes items meaningfully covary at the individual level of analysis
Group level constructsTeam potencyTeam leadership Organizational climateSocietal culture
Group level Constructs
Kozlowski & Klein (2000)Composition versus Configural constructs
• Convergent– Responses tend to center about a single value
usually represented by the group mean
• Emergent– Even though the origin of these constructs are a
function of the cognition, affect, and personality of the survey respondents, the properties of these constructs are manifested at the group level.
Convergent-Emergent Constructs
TeamLeadership
Between Groups
Mean 1 Mean 2 Mean 3
HabitualFormal Procedural
TeamLeadershipBetween Groups
Within Groups Within GroupConstruct
Mean 1 Mean 2 Mean 3
HabitualFormal Procedural
Convergent-Emergent Constructs
Convergent-Emergent ConstructsTeam
Leadership
Within GroupConstruct 1
Mean 1 Mean 3 Mean 4
HabitualFormal ProceduralConventional
Mean 2
Within GroupConstruct 2
Multi-level CFABengt Muthen (1990, 1994)
Ignoring the nested structure of the data and conducting a CFA on the total variance/covariance matrix causes problems
• House, Hanges, Javidan, Dorfman, & Gupta (2004) – Dyer, Hanges, & Hall (in press)
Multi-level CFASpecification of the multilevel CFA was initially very difficult
M-Plus, LISREL, EQS now have options to conduct multilevel analyses
M-Plus multilevel CFA syntaxVARIABLE:
NAMES ARE COUNTRY ORG x1 x2 x3 x4 x5 x6;USEVARIABLES ARE COUNTRY ORG x1 x2 x3 x4 x5 x6;CLUSTER is org;
ANALYSIS:TYPE IS TWOLEVEL;
MODEL:%BETWEEN%
Latentb by x1-x6;
%WITHIN%Latent1 by x1 x3 x5;Latent2 by x2 x4 x6;
Cross-Cultural Measurement Equivalence with Group Level Constructs
Case 1Multiple groups sampled within each country
• Phase 3 of GLOBE
Case 2Construct is at the country level of analysis
• Societal culture
Measurement Equivalence of Multilevel Constructs: Case 1
Country 2Country 1
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
Comparison of the two CFA approaches
Simulation2 countries
• 100 organizations– 50 people within each organization
6 itemsOne organizational-level culture constructTwo within organization-level constructs
True Model Group Level
Within Level 1
Mean 5Mean 2 Mean 3
Item 3 Item 5Item 2Item 1 Item 4 Item 6
Mean 4 Mean 6Mean 1
Within Level 2
0.80.8
0.8 0.8 0.8
0.8
0.6 0.6 0.6 0.6 0.60.6
Comparison of analysesMultigroup CFA
CFI = 0.59RMSEA = .18χ2(23) = 3852.35**Loadings
• X1 .26• X2 .65• X3 .29• X4 .63• X5 .28• X6 .65
Multigroup MultilevelCFI = 0.99RMSEA = .01χ2(48) = 74.72*Loadings
• X1 .80• X2 .89• X3 .81• X4 .76• X5 .87• X6 .81
Measurement Equivalence of Multilevel Constructs: Case 2
Societal CultureConvergent-emergent construct
• Convergence is at cultural level• Scale emergence is at cultural level
Multi-group CFA approach will not provide accurate resultsMulti-group multi-level CFA has to be modified
Multigroup-Multilevel CFACountries
N=40
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
Group1 Group 2
Between
Within
Mean 1 Mean 2 Mean 3
Item 3Item 1 Item 2
ConclusionTraditional (multi-group CFA) approach can yield misleading results for group level constructs.
Need to use analytic procedure that incorporates the multilevel structure of the data
ConclusionMulti-group multi-level CFA is a useful approach for assessing measurement equivalence of group level constructs
