Culturally-Related Response Styles for Attitude Questions · • These results indicate that there are cultural differences in the response styles of American respondents as compared

Survey Research and Methodology

UNL-GALLUP Research Center

Culturally-Related Response Styles

for Attitude Questions:

A Comparative Analysis of Chinese and

American Respondents

1

Presenter: Mia (Mengyang) Wang

Mia (Mengyang) Wang

Allan L. McCutcheon

GALLUP Research Center

Survey Research and Methodology Program

University of Nebraska-Lincoln



OBJECTIVE

• Under the Cross-National Context:

– To detect the biasing effect of stylistic responding

– To examine whether variable response styles for attitude

questions can be explained by the presence of an

underlying construct related to cultural difference

– Specifically, to study how differences between American

and Chinese cultures influence stylistic responding

2



CULTURALLY-RELATED RESPONSE STYLES

• Cross-cultural researchers often rely on surveys for collecting

data because of ease, convenience, and cost considerations.

• Unfortunately, survey responses can be influenced by individual

response styles (T Hamamura et al., 2008).

• Research also indicates that particular response styles may differ

across cultural groups (Chen et al., 1995).

• Researchers conducting cross-cultural surveys are particularly

challenged in drawing valid conclusions from survey data when

respondents demonstrate culturally-based response patterns that

are systematically different, though unrelated to content

(Dolnicar and Bettina, 2007).

3



CULTRUAL EFFECTS

• Chinese Culture vs. American Culture

In China, the ‘Doctrine of the Mean’ is a central aspect

of Chinese culture. Unlike American culture, Chinese

principles encourage people to express neutral, instead

of polarized, attitudes.

4



STYLISTIC RESPONSES

5

STYLISTIC RESPONSE THEORETICAL DEFINITION

Extreme Response StyleThe tendency to report either strongly disagree or strongly agree to

scalar questions.

Midpoint Response StyleThe tendency to use the neutral middle category of a scale regardless of

content.

Mild Response StyleThe tendency to moderately agree or moderately disagree.

No Opinion Response StyleThe tendency to endorse the “don’t know” or “don’t apply” categories.

Acquiescence Response StyleThe tendency to report agreement with items regardless of content.

Disacquiesence Response StyleThe tendency to report disagreement with items regardless of content.

*Reference: Hans Baumgartner and Jan-benedict E.M. Steenkamp 2001



THE BIG IDEA

• Cultural differences related to response style have

the potential to impact implicit survey response

patterns, which may manifest itself in questionnaire

reporting behaviors.

• This study compares Chinese and American survey

responses on Likert scales in order to investigate

cultural effects in underlying response patterns.

6



RESEARCH QUESTIONS

• Do American respondents demonstrate significantly different

response patterns compared to Chinese respondents?

– Is the occurrence of extreme response styles lower among Chinese

respondents?

– Are Chinese respondents more likely to endorse the midpoints of

scales?

– Are Chinese respondents more likely to report moderate agreement or

moderate disagreement?

7



METHOD

• To examine whether variable response styles for attitude

questions can be explained by the presence of an

underlying construct related to cultural difference,

Latent Class Analyses are employed in this research.

• Specifically, exploratory latent class models are used:

1. To investigate the existence of the underlying stylistic responses

for each country

2. To examine the frequency of different types of respondents in

terms of their underlying stylistic responses within each country.

3. To detect the differences in response patterns of Chinese and

American respondents.

8



DATA

• The Gallup World Poll

Probability-based

Multinational

• Data Collected (2013)

CHINA – Sample Size = 4,244

USA – Sample Size = 1,005

9



CONCEPTUAL ILLUSTRATION

Cultural

Difference

Health

Life

Emotion

Energy

Stylistic Responses on

Attitude Questions

10



CROSS-CULTURAL SURVEY QUESTIONS

11

EXTREME

MILD

NEUTRAL

1 - Strongly

disagree

5 - Strongly

agree

2 - Disagree

4 - Agree

3 - Midpoint


UNL-GALLUP Research Center12

COMPARATIVE TENDENCY OF STYLISTIC RESPONSES

38%

55%

51%

44%

21%

33%29%

18%

Life Health Energy Emotion

EXTREME RESPONSES

USA CHINA




40%

31% 31%

36%

46% 45%43%

44%


MILD RESPONSES

USA CHINA




22%

14%

17%

20%

32%

22%

26%

37%


MIDPOINT RESPONSES

USA CHINA




0.0%

0.3%0.4%

0.3%

0.9%

0.5%

2.0%

0.9%


“DON’T KNOW” RESPONSES

USA CHINA



EXPLORATORY LATENT CLASS MODEL – USA

Exploratory latent class analyses are used to investigate the

existence and the frequency of different types of respondents in

terms of their underlying stylistic responses for American

respondents.

Model X2 P Value AIC BIC DF

One-Class 1062.2625 0.000 587.3202 234.3951 72

Two-Class 176.1059 0.000 23.8267 -284.9827 63

Three-Class 112.2962 0.000 -5.5802 -270.2740 54

Four-Class 69.5729 0.1067 -25.9386 -241.6980 45

Five-Class 46.8363 0.0108 -21.0745 -202.4374 36

Six-Class 39.5284 0.0567 -11.6503 -143.9972 27

16



MODEL SELECTION – USA

• The response styles of American respondents for scalar

attitude questions can be classified by four classes.

• Each respondent from the entire USA sample was assigned to

one of four latent classes:

1. Extreme Responder

2. Mixed-Style Responder

3. Midpoint Adverse Responder

4. Mild Responder

17



CONDITIONAL PROBABILITY AND RELATIVE FREQUENCY– USA

RESPONDENT TYPE – USA

Manifest Questions Response TypeMixed-Style

Responder

Extreme

Responder

Midpoint

Adverse

Responder

Mild

Responder

Learn or Do

Something

Interesting

EXTREME RESPONSE 0.245 0.807 0.283 0.007

MILD RESPONSE 0.437 0.159 0.453 0.682

MIDPOINT RESPONSE 0.318 0.035 0.263 0.310

Encourages to be

Healthy


MILD RESPONSE 0.354 0.000 0.255 0.843


Positive Energy


MILD RESPONSE 0.443 0.081 0.215 0.802


Felt Active and

Productive


MILD RESPONSE 0.332 0.124 0.421 0.734


Relative Class

Frequency 30% 29% 27% 14%

18



RELATIVE DISTRIBUTION – USA

Extreme

Responder

29%

Mixed-Style

Responder

30%

Midpoint Adverse

Responder

27%

Mild Responder

14%

Relative Distribution of American Respondents

Exhibiting Each Response Style

19



MODEL RESULTS – USA

• Mixed-Style Responder30% of respondents were in the largest class, which consistently

showed a tendency not to exhibit a particular response style.

• Extreme Responder29% of respondents tended to select the most extreme response

options.

• Midpoint Adverse Responder27% of respondents avoided endorsing the midpoint, or neutral,

response option.

• Mild Responder14% of respondents exhibited a tendency to select the mild

response options.

20



EXPLORATORY LATENT CLASS MODEL – CHINA

Exploratory latent class analyses are used to investigate the

existence and the frequency of different types of respondents in

terms of their underlying stylistic responses for Chinese

respondents.

Model X2 P Value AIC BIC DF

One-Class 17587.156 0.000 5210.238 4757.102 72

Two-Class 1657.722 0.000 1144.741 30067.991 63

Three-Class 462.558 0.000 344.234 4.382 54

Four-Class 228.514 0.000 130.513 -152.697 45

Five-Class 93.426 0.006 22.786 -203.782 36

Six-Class 59.333 0.004 3.537 -166.389 27

21



MODEL SELECTION - CHINA

• The response styles of Chinese respondents for scalar attitude

questions can be classified by five classes.

• Each respondent from the entire Chinese sample was assigned

to one of five latent classes:

1. Extreme Adverse Responder

2. Mixed-Style Responder

3. Neutral Responder

4. Mild Responder

5. Extreme Responder

22



CONDITIONAL PROBABILITY AND RELATIVE FREQUENCY - CHINA

RESPONDENT TYPE – CHINA

Manifest

Questions

Extreme Adverse

Responder

Mixed-Style

Responder

Mild

Responder

Extreme

Responder

Neutral

Responder

Learn or Do

Something

Interesting

EXTREME RESPONSE 0.124 0.368 0.005 0.946 0.000

MILD RESPONSE 0.587 0.362 0.765 0.045 0.125

MIDPOINT RESPONSE 0.289 0.270 0.230 0.009 0.875

Encourages to

be Healthy


MILD RESPONSE 0.486 0.304 0.987 0.019 0.265


Positive Energy


MILD RESPONSE 0.465 0.322 0.958 0.024 0.146


Felt Active and

Productive


MILD RESPONSE 0.524 0.414 0.827 0.104 0.000


Relative Class

Frequency33% 22% 20% 13% 12%

23



RELATIVE DISTRIBUTION- CHINA

Extreme Adverse

Responder

33%

Mixed-Style

Responder

22%

Mild Responder

20%

Extreme

Responder 13%

Neutral

Responder

12%

Relative Distribution of Chinese Respondents

Exhibiting Each Response Style

24



• Extreme Adverse Responder33% of respondents avoided endorsing the extreme response options, instead

tending to endorse the mild or midpoint options.

• Mixed-Style Responder22% of respondents tended not to exhibit any particular response style.

• Mild Responder

20% of respondents exhibited a tendency to select mild response options.

• Extreme Responder13% of respondents consistently showed a tendency to select the most extreme

response options.

• Neutral Responder12% of respondents exhibited a tendency to select the midpoint, or neutral,

response option.

MODEL RESULTS – CHINA

25



RESULTS

• There were four different types of respondents identified for

Americans and five different types of respondents identified for

Chinese.

• American and Chinese respondents exhibited some overlap in

response styles.

• However, these styles were exhibited in different frequencies in the

US as compared to China:30% 29%

14%

22%

13%

20%

Mixed-Style Responder Extreme Responder Mild Responder

USA CHINA

26



RESULTS

• In addition, American and Chinese respondents

exhibited some unique response styles

• In the US: Midpoint Adverse Responder (27%)

• In China: Extreme Adverse Responder (33%) and Neutral

Responder (12%)

27



CONCLUSIONS

• These results indicate that there are cultural differences in

the response styles of American respondents as compared to

Chinese respondents. The findings of this study suggest that

Chinese respondents do tend to exhibit less extreme

response styles than Americans, which has the potential to

impact survey findings.

• The existence of culturally-related response patterns

represents a serious threat to the correct conclusions and

interpretations drawn from research comparing the United

States and China.

28



FUTURE RESEARCH

• Compare frequencies of acquiescence response style

and disacquiesence response style across cultures.

• Explore methods to control for cultural effects

between Chinese and American respondents when

performing or analyzing cross-national studies.

29

Thank You!Contact Author: [email protected]



CROSS-CULTURAL SURVEY QUESTIONS

• Question Stem:

“Thinking about your life in general, please rate your level of agreement with each of the

following using a five-point scale, where 5 means you STRONGLY AGREE and 1 means you

STRONGLY DISAGREE. You may choose any of the numbers 1, 2, 3, 4, or 5.” (Gallup World Poll,

2013)

31

Topic Questions

Learn or Do Something Interesting You learn or do something interesting every

day

Encouragement to be HealthySomeone in your life always encourages you to

be healthy.

Positive Energy Your friends and family give you positive energy

every day.

Felt Active and ProductiveIn the last seven days, you have felt active and

productive every day.

Documents

Culturally-Related Response Styles for Attitude Questions · • These results indicate that there are cultural differences in the response styles of American respondents as compared