Upload
others
View
16
Download
0
Embed Size (px)
Citation preview
Culture & Survey Measurement
Timothy JohnsonSurvey Research LaboratoryUniversity of Illinois at Chicago
What is culture?
It is “the collective programming of the mind which distinguishes the members of one group from another” (Hofstede, Culture’s Consequences, 1980, p. 21)
What is culture? cont’d
The “shared elements that provide the standards for perceiving, believing, evaluating, communicating, and acting among those who share a language, a historic period, and a geographic location”(Triandis, American Psychologist, 1996).
Some dimensions of cultureIndividualism/collectivismUncertainty avoidancePower distanceLong vs. short term time orientationCultural tightness & complexityOpenness-to-change vs. ConservatismSelf-enhancement vs. Self-transcendence
Some dimensions of culture cont’d
Achievement vs. ascriptionNeutral vs. emotional expression of feelingsDiffuse vs. specific involvement in affairs of othersVertical vs. horizontal relationships
Types of equivalence1. Calibration 17. Factor 33. Measurement 49. Stimulus2. Complete 18. Factorial unit 50. Structural3. Conceptual 19. Formal 34. Metaphorical 51. Substantive4. Construct 20. Full 35. Metric 52. Syntactic5. Construct 21. Functional 36. Motivational 53. Technical
operationalization 22. Grammatical- 37. Normative 54. Text6. Content syntactical 38. Operational 55. Theoretical7. Contextual 23. Indicator 39. Pseudo 56. Translation8. Credible 24. Idiomatic 40. Psychological 57. True-score 9. Criterion 25. Instrument 41. Psychometric 58. Verbal10. Cross-cultural 26. Item 42. Relational 59. Vignette11. Cross-national 27. Language 43. Relative equivalence12. Cultural 28. Lexical 44. Response 60. Vocabulary13. Definitional 29. Linguistic 45. Scalar equivalence14. Direct 30. Literal 46. Scale15. Exact 31. Meaning 47. Semantic16. Experiential 32. Measurement 48. Situational
Two general forms of equivalence
• Procedural EquivalenceEmphasis on equivalent methods
• Interpretive EquivalenceEmphasis on equivalent meaning
Cognitive survey response model
Question interpretation
Memory retrieval
Judgment formation
Response editing
Category fallacy
Assuming a question or concept is universally understood when in fact understanding is culturally conditioned
“In this question, what does the word ‘stress’ mean to you?”
0
5
10
15
20
25
30
AfricanAmerican
MexicanAmerican
Puerto Rican White
Health ProblemsSocial Problems
Comprehension difficulties by race/ethnicity
12.0% 12.6%13.5%
14.5%
0
2
4
6
8
10
12
14
16
Per
cen
t
AfricanAmerican
MexicanAmerican
Puerto Rican White
Comprehension difficulties by reading difficulty & race/ethnicity
0
5
10
15
20
Less than 6th grade 6th-8th grade Grade 9+
Reading difficulty level (U.S. grade level)
Pro
port
ion
wit
h c
ompr
ehen
sion
di
ffic
ult
ies
White
African American
Mexican American
Puerto Rican
0
5
10
15
20
Least Somewhat Most
Abstraction level
Pro
port
ion
wit
h c
ompr
ehen
sion
dif
ficu
ltie
s
White
African American
Mexican American
Puerto Rican
Comprehension difficulties by abstraction level & race/ethnicity
Comprehension difficulties for vague quantifier format by race/ethnicity
11.0% 12.5% 12.6%
6.6%
0
5
10
15
20
25
Per
cent
African American MexicanAmerican
Puerto Rican White
Comprehension difficulties for numeric question format by race/ethnicity
18.9% 20.0%21.2%
15.6%
0
5
10
15
20
25
Per
cent
AfricanAmerican
MexicanAmerican
Puerto Rican White
Comprehension difficulties for yes/no format by race/ethnicity
7.2% 6.7%7.7%
4.3%
0
5
10
15
20
25
Per
cent
AfricanAmerican
MexicanAmerican
Puerto Rican White
Measurement artifacts in survey research
Extreme Response Styles
Acquiescent Response Style
May be misinterpreted as substantive differences across groups
Acquiescent response style
The tendency to agree with survey questions, regardless of question format (also known as “yea-saying.”)
1. Strongly disagree2. Somewhat disagree3. Neither disagree nor agree4. Somewhat agree5. Strongly agree
Cross-cultural findings: Extreme response styles (ERS)*
More ERS in countries high in uncertainty avoidanceLess ERS in countries high in individualism
*van de Vijver (2004). Symposium on Cross-Cultural Survey Research, SRL, Urbana, IL.
Cross-cultural findings: Acquiescence (ACQ)*
More ACQ in countries high in uncertainty avoidance & power distance
Less ACQ in countries high in individualism
*van de Vijver (2004). Symposium on Cross-Cultural Survey Research, SRL, Urbana, IL.
% uncomfortable discussing alcohol use with interviewers from same/different cultural groups
0
5
10
15
20
25
30
35
40
AfricanAmerican
MexicanAmerican
PuertoRican
White
Same CultureDifferent Culture
A. Question Development Phase
B. Questionnaire Pretesting Phase
C. Data Collection Phase
D. Data Analysis Phase
Available methods for addressing cross-cultural equivalence
Question development stage1. Expert consultation/collaboration
2. Ethnographic & other qualitative approaches
3. “Good” question-wording practices
4. “Good” translation practices
5. Facet analysis
Questionnaire pretesting phase
1. Cognitive interviews/structured probes
2. Comparative response scale calibration
3. Comparative behavior coding
4. Compare alternative data collection modes
Data collection phase
1. Use multiple indicators
2. Use both emic & etic questions
3. Respondent/interviewer matching
Data analysis phase1. Item analysis
2. Item response theory
3. Anchoring vignettes
4. Confirmatory factor analysis
5. Multidimensional scaling
6. Applying statistical controls
7. Identity-equivalence method