Measuring Happiness in Surveys: A Test of the Subtraction Hypothesis

American Association for Public Opinion Research

Measuring Happiness in Surveys: A Test of the Subtraction HypothesisAuthor(s): Roger Tourangeau, Kenneth A. Rasinski and Norman BradburnSource: The Public Opinion Quarterly, Vol. 55, No. 2 (Summer, 1991), pp. 255-266Published by: Oxford University Press on behalf of the American Association for Public OpinionResearchStable URL: http://www.jstor.org/stable/2749293 .

Accessed: 19/12/2014 00:23

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

American Association for Public Opinion Research and Oxford University Press are collaborating with JSTORto digitize, preserve and extend access to The Public Opinion Quarterly.

http://www.jstor.org

This content downloaded from 128.235.251.160 on Fri, 19 Dec 2014 00:23:22 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=oup

http://www.jstor.org/action/showPublisher?publisherCode=aapor

http://www.jstor.org/action/showPublisher?publisherCode=aapor

http://www.jstor.org/stable/2749293?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


MEASURING HAPPINESS IN SURVEYS: A TEST OF THE SUBTRACTION HYPOTHESIS

ROGER TOURANGEAU, KENNETH A. RASINSKI, and NORMAN BRADBURN

Abstract Responses to an item on general happiness can change when that item follows one on marital happiness. Asking about marital happiness first sometimes reduces reported levels of overall happiness. This reduction may result from a change in the interpretation of the general happiness item. According to this hypothesis, when the general item follows the item on marital happiness, respondents take the item to ask about aspects of their lives other than their marriages-in effect, the respondents subtract their (mostly happy) marriages in answering the general item. The study reported here tests this "subtraction" hypothesis by asking versions of the general happiness item that correspond to the different interpretations. A version of the general item that asked about general happiness "aside from your marriage" yielded responses that were quite similar to those given to the standard item when it followed the item on marital happiness. Another version that asked about general happiness "including your marriage" elicited responses quite similar to those elicited by the standard item when it preceded the marital happiness item. However, reanalysis of the studies that originally demonstrated the impact of the order of the two happiness items casts doubt on the subtraction hypothesis and related models as explanations of the earlier findings.

The literature on survey context effects suggests that earlier questions can affect answers to later ones through a number of distinct mecha- nisms (Schuman and Presser 1981; Smith 1982; Strack and Martin

ROGER TOURANGEAU iS Vice President at CODA, Inc. KENNETH RASINSKI is a Survey Director at the National Opinion Research Center (NORC) and NORMAN BRADBURN iS

NORC's Director. The authors thank Tom Smith and Howard Schuman, who carried out the reanalyses reported here. Their cooperation-and promptness-are greatly ap- preciated. The data collection described here was supported by a grant from the NORC Director's Fund.

Public Opinion Quarterly 55 255-266 ? 1991 by the Amencan Association for Public Opinion Research Published by The University of Chicago Press / 0033-362X/91/0055-02/$2.50



256 Tourangeau, Rasinski, and Bradburn

1987). In their recent review, Tourangeau and Rasinski (1988) argued that earlier questions can change how respondents interpret later items, what information they retrieve in formulating their answers, and how they make the judgments required by later items.

One survey context effect is widely believed to reflect processes operating as the item is interpreted. It involves a question asking about general happiness ("Taken altogether, how would you say things are these days? Would you say that you are very happy, pretty happy, or not too happy?"). This item has figured in some landmark studies of happiness and psychological well-being (e.g., Bradburn 1969). How- ever, answers to the item can vary depending on whether it precedes or follows a similar item on marital happiness ("Taking things all together, how would you describe your marriage? Would you say that your marriage is very happy, pretty happy, or not too happy?"). Schu- man and Presser (1981) reported a contrast effect in which respondents were significantly less likely to describe themselves as very happy when they had first answered the question on marital happiness. Com- plicating matters, Smith (1982) found the opposite-respondents were significantly more likely to say they were very happy when they had first answered the marriage question.

Of the two findings, Smith's is the less puzzling. Most respondents (typically, almost two out of three) report that they are very happily married. It is easy to see why reminding respondents of their mostly happy marriages just before asking them about their overall happiness will produce higher levels of reported happiness. The item on marital happiness should increase the chance that respondents will recall their marriages and take them into account in evaluating their overall happiness (McClendon and O'Brien 1988; Smith 1985; Strack, Martin, and Schwarz 1988). Such retrieval-based assimilation effects have been commonly observed, both with items on happiness and with other attitude items (e.g., Tourangeau et al. 1989a, 1989b). Though readily explained, the assimilation effect is likely to be weak. Even without the reminder provided by the preceding item, marriage is likely to be a salient consideration for most respondents as they assess their overall happiness; because the prior item on marital happiness will only affect those respondents who would not have taken their marriages into account anyway, the impact is likely to be small. Consistent with this analysis, efforts to replicate Smith's findings, such as the one reported by Turner (1984), have generally found nonsignificant effects in the direction of assimilation.

Schuman and Presser's finding demonstrates that, though assimilation may be the rule, there are definite exceptions. Their results raise two questions: Why does the contrast effect occur? What determines which effect occurs? In answer to the first question, several investiga-



Measuring Happiness in Surveys 257

tors have suggested that the contrast effect reflects differences in how the general happiness item is interpreted under the two question- orders. Bradburn (1982), for example, argued that when the general item comes first, respondents may see it as referring to all aspects of their lives including their marriages, but that when it comes second, they see it as referring to aspects of their lives other than their marriages. Because they exclude or "subtract" their marriages in evaluating their overall happiness, respondents who answer the general happiness item after the item on marriage may give a different answer than they would have given had the general item come first. Others have proposed similar accounts of this or related context effects (e.g., Kal- ton, Collins, and Brook 1978; Schuman and Presser 1981).

Researchers have sometimes linked the subtraction hypothesis to more fundamental processes. Bradburn argued that respondents may consider it redundant to base their answers to both happiness items on evaluations of their marriages. Tourangeau (1984; sqe also Strack, Martin, and Schwarz 1988; Tourangeau and Rasinski 1988) proposed that this hypothesized tendency to avoid redundancy may reflect an even more basic principle that governs behavior in conversations- namely, the principle that each contribution should be informative (Grice 1975; Haviland and Clark 1974). If this principle is at work in survey interviews, respondents may interpret the general item to be asking for new information-that is, about areas of their lives not already covered by the preceding item on marital happiness.

The subtraction account suggests some variables that may affect whether assimilation or contrast will occur. For example, summaries are not always redundant, particularly when many or diverse specifics have been touched on. As Tourangeau and Rasinski point out, a long list of prior specifics "may encourage respondents to interpret the general item as a summary of the particulars rather than as a residual category" (1988, 303). Thus, when the marriage and general happiness items are part of a longer series of related items, contrast effects may be less likely. The number of prior specific items may provide a clue about why Smith found an assimilation effect but Schuman and Presser found contrast (Schwarz, Strack, and Mai 1991). In the 1980 General Social Survey (GSS) on which Smith's results are based, the two happiness items followed five questions about satisfaction in other life domains; Schuman and Presser's study did not include the satisfaction items.

Despite its popularity, the evidence in favor of the subtraction account for the contrast effect has mostly been quite indirect. Smith (1982) noted that the subtraction hypothesis implied that the effect of the preceding marital happiness item should depend on how respondents answered that question-that is, the effect of excluding marriage




from the judgment of overall happiness should depend on whether the marriages being left out are, on the average, happy or unhappy ones. Strack, Martin, and Schwarz (1988) went further, arguing that the difference in interpretation should affect the correlation between responses to the two happiness items; the correlation should be reduced when marriage is excluded from the evaluation of general happiness.

But the central claim of the subtraction hypothesis-that the difference in question order can produce a difference in the interpretation of the general happiness item-has only recently been tested. The subtraction hypothesis states that, when the general happiness item follows the item on marital happiness, respondents interpret it to mean something like "Aside from your marriage, how happy are you?" but that when it comes first, respondents interpret it to mean something like "Including your marriage and other aspects of your life, how happy are you?" If this is true, then it should be possible to duplicate the results obtained under the two question orders merely by rephras- ing the general happiness item to make the intended interpretation clear and explicit. This is what a recent study by Schwarz and his colleages attempted to do (Schwarz, Strack, and Mai 1991).

It is possible that this study does not fully capture the essential features of the original studies. First of all, Schwarz and his co-workers made several changes in the wording and format of the happiness items. Their items asked about satisfaction instead of happiness and provided an 11-point response scale instead of the usual 3-point scale. The biggest wording change involved replacing the item on marriage with an item that covered dating relationships as well; this allowed unmarried respondents to take part in the study. However, in a similar study that examined order effects on items asking about satisfaction in several life domains, McClendon and O'Brien (1988) reported differences between married and unmarried respondents-only the married respondents showed assimilation effects. Schwarz and his colleagues do not report the results separately for married and unmarried respondents or say what proportion of their sample was married, so we can- not assess whether including unmarried respondents affected their results.

Second, the respondents in the new study were a convenience sample of volunteers who filled out written questionnaires. The change from personal interviews in the earlier studies to self-administered questionnaires in the new one may have altered the results; elsewhere, Schwarz and Strack have argued that self-administered questionnaires reduce or eliminate question order effects (Bishop et al. 1988; Schwarz et al., in press). Third, although the new study shows clear evidence of assimilation effects, the evidence for contrast is much weaker. Re-




spondents who received the version of the general item that explicitly instructed them to leave out their marital or dating relationship did not show a significantly reduced correlation between the two happiness items relative to the control group, although the difference was in that direction.

We carried out a study that avoided some of these problems; our study reproduces the main features of the original studies more exactly than the study by Schwarz, Strack, and Mai (1991). It uses telephone interviews carried out by experienced interviewers with a randomly selected sample of married respondents. Our study also includes conditions that retain the exact wordings of the original items. Finally, we carried out a detailed comparison of our results with those of the original studies and a recent replication on the 1987 GSS. This comparison proved to be quite revealing.

Method

DESIGN

The study was a split-ballot experiment with four groups. Respondents in two groups got the standard wording of the general happiness item ("Taken altogether, how would you say things are these days? Would you say that you are very happy, pretty happy, or not too happy?"). In one of these standard wording groups, the general happiness item preceded the item on marital happiness; in the other, it followed the item on marital happiness. The two standard wording groups thus replicate the conditions reported by Schuman and Presser (1981) and Smith (1982). The final two groups received variant wordings for the general happiness item. For one group, the item read: "Taking things all together, including your marriage and other important aspects of your life, how would you say things are these days? Would you say that you are very happy, pretty happy, or not too happy?" For the other group, the general item read: "Aside from your marriage, how would you say things are these days? Would you say that you are very happy, pretty happy, or not too happy?" In both variant wording groups, the general happiness item followed the item on marital happiness.

All four groups got the standard wording of the marital happiness item ("Taking things all together, how would you describe your marriage? Would you say that your marriage is very happy, pretty happy, or not too happy?"). In all four, the two happiness items were the last questions in the interview and immediately followed an item that asked about the respondent's marital status; twelve items on welfare and abortion preceded the items on marital status and happiness.




SAMPLE

The experiment reported here was conducted as part of a larger study in which a sample of households in Chicago was contacted by telephone and individuals were selected for interview; those who completed an initial interview were recontacted and interviewed over the telephone a second time. The happiness items were administered dur- ing the second interview.

We drew the sample for the initial interview by randomly selecting telephone numbers from the Chicago directory and replacing the final digit of each number with a random digit. Interviewers called this initial pool of telephone numbers and contacted a total of 1,481 households; calls to an additional 570 numbers were never answered. The selection of a respondent within the household was nonrandom, with interviewers asking for any available adult male or, when no male was available, any adult female. The initial interview, which concerned attitudes on abortion and welfare, was completed by 599 respondents.

Respondents to the initial interview were recontacted by telephone three weeks later. Four hundred ninety-nine of the original respondents also completed the reinterview; 228 of them were married. Of the 228, 48.7% were men and 51.3% were women; 55.3% said they were white, 27.4% Black, 5.8% Hispanic, and 11.4% classified themselves in other race or ethnic categories. Their ages ranged from 18 to 87 years, with a mean of 47 years.

PROCEDURE

Ten experienced telephone interviewers carried out both the initial interviews and the reinterviews. Each interviewer received a batch of reinterview questionnaires, with the different versions in a random order. By working their way through the questionnaires, the interviewers randomly assigned reinterview respondents to one of the four experimental groups.

Results

The hypotheses can be summarized quite simply. The correlation between responses to the two happiness items should be relatively high when the general item preceded the item on marital happiness or when the wording of the general item made it clear that respondents were to take their marriages into consideration. The correlation should be relatively low when the general item followed the item on marital happiness or when the wording of the general item made it clear that respondents were to exclude their marriages from consideration.




Table 1 presents the correlations for the four groups. As is apparent, they conform quite closely to the hypothesized pattern. The two correlations that are supposed to be high are considerably higher than the two that are supposed to be low (.545 and .524 vs. .277 and .267). A contrast that compares the first two of these correlations with the last two reveals a significant difference (z = 2.32; p < .05). Table 1 also shows the significance of the differences between pairs of these correlations; both pairwise comparisons are marginally significant.

Although the correlations between the items vary across the four groups, the marginals on the general happiness item do not. The proportion of respondents describing themselves as "very happy" was 34.0% among those who got the standard item after the marital happiness item, 40.0% among those who got the standard item before the marital happiness item, 33.3% among those who got the "aside from your marriage" revised wording, and 33.9% among those who got the "including your marriage" revised wording. None of the differences in the marginals for the general happiness item are significant.

Discussion

In terms of the correlations between the two happiness items, respondents who got the revised item asking them about their lives aside from their marriages show results very similar to those who received the standard general happiness item after the item on marital happiness. Those who got the revised item asking about their lives including their marriages were quite similar to those who got the standard item first. These results are certainly consistent with the view that the revised items reflect the different interpretations that respondents give to the standard item under the different question orders. What is particularly striking about the findings is how closely the correlations for the two revised groups parallel those for the two standard wording groups.

Our findings differ somewhat from the results in the comparable conditions of the study by Schwarz, Strack, and Mai (1991). First, in our study, when the standard general item followed the item on marital happiness, there was a low correlation between the two (r = .28); in their study, the corresponding correlation was quite high (r = .67). Schwarz and his colleagues included another condition that yielded results more like ours. In that condition, the general item followed the specific item and the two items were explicitly linked by a joint introduction; within this group, the correlation (r = .18) was comparable to the value of .28 that we found. As it happens, our study included




Table 1. Responses to the General Happiness Item and Correlation with Marital Happiness

Correlation "Very with

Happy" Marital Group N Percentage z p Happiness z p

Standard wording Marital happiness

first 53 34.0 0.66 n.s. .277 -1.69 <.10 General happiness

first 60 40.0 .545 Revised wording

"Aside from your marriage" 54 33.3 - 0.07 n. s. .267 - 1.59 <. 12

"Including your marriage" 59 33.9 .524

NOTE: All significance tests are two-tailed.

such a lead-in to the happiness items; just before these items, there was a short introduction that read "Now for a couple of items on an unrelated topic." Even such a vague introduction apparently tended to heighten the relationship between the two items, affecting interpretations of the general item.

A second difference between the results of the two studies involved the correlation between the two happiness items when the general item came first. In our study, this correlation was relatively high (r = .55) when the standard version of the general happiness item was used; in the study by Schwarz and his colleagues, the corresponding correlation was relatively low (r = .32). This difference may reflect the reduced importance or accessibility of dating relationships as compared to marriages. The specific item in the study by Schwarz and his colleagues covered both marriage and dating; ours asked only about marriages. The difference in the correlations in the two studies may also reflect the fact that our study included an item to determine whether the respondent was married; this question came right before the two happiness items. The question on marital status may have temporarily heightened the accessibility of the respondent's marriage, increasing its impact on answers to the general happiness item. Data from the earlier studies suggest that both explanations have some merit. For the conditions in which the general item came first, Smith and Schu- man and Presser report correlations between the two happiness items that are somewhat lower than ours but somewhat higher than the one




reported by Schwarz and his co-workers (r = .39 in Smith 1982, and r = .49 in Schuman and Presser 1981).

Despite these superficial differences, the results of our study are basically consistent with those of Schwarz and his colleagues. The key question that remains is whether either study, both of which found differences in correlations, sheds any light on the original studies, both of which found differences in the overall direction of responses to the general item. If the contrast effect reported by Schuman and Presser results from the reinterpretation of the general happiness item when it follows the marital happiness item, there should be a correlational effect as well as a directional one; as in table 1, the correlation between the two happiness items should be reduced when the general item comes second. Similarly, if the assimilation effect reported by Smith results from the increased accessibility of marital happiness when the item about marriage comes first, this should heighten the correlation between the two items. Were there such correlational effects in the earlier studies?

Apparently not. Table 2 presents the results of a reanalysis of the data from the earlier studies and from a recent replication conducted as part of the 1987 GSS. None of the studies showed a significant difference between the correlations obtained under the two question orders. The results shown in table 2 do not depend on the measure of association used; the pattern remains the same whether correlation coefficients, simple or multiple regression coefficients, or gammas are used to measure the association between the two happiness items.

There is one situation in which both the subtraction and accessibility hypotheses predict directional rather than correlational effects. That situation is when everyone answers the marital happiness item in more or less the same way. The absence of variability in responses to the specific item will guarantee a low correlation with the general item, regardless of question order. Further, because all respondents will be subtracting (or including) material with similar implications for the general happiness question, the specific item should affect answers to the general item in the same way for all respondents, producing a directional effect. The data in table 2, however, are not consistent with this hypothesis either. The correlations there are not particularly low, ranging from the high .30s to the low .50s. These values are closer to the high correlations in table 1 (and in the study by Schwarz and his colleagues) than to the low ones. To put it another way, the correlation between responses to the two happiness items remains relatively high when the respondents are supposed to be discounting their marriages (as in Schuman and Presser 1981) and it gets no higher when the respondents are supposed to have been reminded of their marriages by the preceding item on marital happiness (as in Smith 1982).




Table 2. Responses to the General Happiness Item and Correlation with Marital Happiness

Correlation "Very with

Happy" Marital Study and Group N Percentage z p Happiness z p

Schuman and Presser 1981

Marital happiness first 172 37.2 3.14 <.01 .533 0.56 n.s. General happiness first 201 53.2 .490

Smith 1982 Marital happiness first 613 42.1 -2.22 <.05 .417 0.44 n.s. General happiness first 268 34.3 .390

General Social Survey 1987

Marital happiness first (total) 519 40.5 -1.37a n.s. .373 0.56a n.s.

Satisfaction items before happiness items 261 39.5 0.46b n.s. .418 1.25b n. s.

Satisfaction items after happiness items 258 41.5 .323

General happiness first 265 35.5 .409

NOTE: The five satisfaction items preceded the happiness items in both conditions of Smith 1982; they were omitted from Schuman and Presser 1981.

a Significance tests measure the marital happiness first condition vs. the general happiness first condition.

b Significance tests compare the two satisfaction conditions.

Thus, our study and the one by Schwarz and his colleagues do not appear to capture whatever processes produced directional effects in the original studies. Like Schwarz and his colleagues, we found correlational, but not directional, effects; the earlier studies found directional, but not correlational, effects. Although the results of the newer studies support the idea that respondents may reinterpret apparently redundant items (though perhaps only when the items are explicitly linked by a joint introduction), such reinterpretation does not seem to have produced the contrast effect observed by Schuman and Presser. Similarly, although there is considerable evidence that earlier items can increase the accessibility of material that is then more likely to be used to answer later questions, this does not appear to explain the assimilation effect found by Smith. Moreover, the results of the 1987 GSS experiment indicate that the placement of the satisfaction items had little impact on responses to the two happiness items. This variable




does not seem to determine whether assimilation or contrast effects are observed (cf. Schwarz, Strack, and Mai 1991; Tourangeau and Rasinski 1988).

In short, our conclusions must remain negative. The subtraction hypothesis does not appear to explain the contrast effect involving the happiness items. Similarly, the more complicated model offered by Schwarz and his colleagues, which incorporates the subtraction hypothesis as a special case, also offers no explanation of the original findings; further, that model's explanation of the assimilation effects found in the 1980 GSS appears to be contradicted by the data shown in table 2. Unfortunately, although we may now be clearer about what is not causing the original effects, we are still unsure about what is causing them.

References

Bishop, G., H. Hippler, N. Schwarz, and F. Strack. 1988. "A Comparison of Response Effects in Self-Administered and Telephone Surveys." In Telephone Survey Methodology, ed. R. Groves, P. Biemer, L. Lyberg, J. Massey, W. Nicholls, and J. Waksberg, 321-40. New York: Wiley.

Bradburn, N. 1969. The Structure of Psychological Well-Being. Chicago: Aldine. Bradburn, N. 1982. "Question-Wording Effects in Surveys." In Question Framing

and Response Consistency, ed. R. Hogarth, 65-76. San Francisco: Jossey-Bass. Grice, H. 1975. "Logic and Conversation." In Syntax and Semantics, vol. 3: Speech

Acts, ed. P. Cole and T. Morgan, 41-58. New York: Seminar Press. Haviland, S., and H. Clark. 1974. "What's New? Acquiring New Information as a

Process in Comprehension." Journal of Verbal Learning and Verbal Behavior 13:512-21.

Kalton, G., M. Collins, and L. Brook. 1978. "Experiments in Wording Opinion Questions." Journal of the Royal Statistical Society, ser. C, 27:149-61.

Jabine, T., M. Straf, J. Tanur, and R. Tourangeau. 1984. Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines. Washington, DC: National Academy Press.

McClendon, M., and D. O'Brien. 1988. "Question-Order Effects on Subjective Well-Being." Public Opinion Quarterly 52:351-64.

Schuman, H., and S. Presser. 1981. Questions and Answers in Attitude Surveys: Experiments in Question Form, Wording, and Context. New York: Academic Press.

Schwarz, N., F. Strack, H. Hippler, and G. Bishop. In press. "The Impact of Administration Mode on Response Effects in Survey Measurement." Applied Cognitive Psychology.

Schwarz, N., Strack, F., and H. Mai. 1991. "Assimilation and Contrast Effects in Part-Whole Question Sequences: A Conversational Logic Analysis." Public Opinion Quarterly 55:3-23.

Smith, T. 1982. "Conditional Order Effects." GSS Technical Report no. 33. Chicago: NORC.

Smith, T. 1985. "Unhappiness on the 1985 GSS: Confounding Change and Context." GSS Technical Report no. 34. Chicago: NORC.

Strack, F., and L. Martin. 1987. "Thinking, Judging, and Communicating: A Process Account of Context Effects in Attitude Surveys." In Social Information Processing and Survey Methodology, ed. H. Hippler, N. Schwarz, and S. Sudman, 123-48. New York: Springer-Verlag.




Strack, F., L. Martin, and N. Schwarz. 1988. "Priming and Communication: The Social Determinants of Information Use in Judgments of Life Satisfaction." European Journal of Social Psychology 18:429-42.

Tourangeau, R. 1984. "Cognitive Science and Survey Methods." In Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines, ed. T. Jabine, M. Straf, J. Tanur, and R. Tourangeau, 73-100. Washington, DC: National Academy Press.

Tourangeau, R., and K. Rasinski. 1988. "Cognitive Processes Underlying Context Effects in Attitude Measurement." Psychological Bulletin 103:299-314.

Tourangeau, R., K. Rasinski, N. Bradburn, and R. D'Andrade. 1989a. "Belief Accessibility and Context Effects in Attitude Measurement." Journal of Experimental Social Psychology 25:401-21.

Tourangeau, R., K. Rasinski, N. Bradburn, and R. D'Andrade. 1989b. "Carryover Effects in Attitude Surveys." Public Opinion Quarterly 53:495-524.

Turner, C. 1984. "Why Do Surveys Disagree? Some Preliminary Hypotheses and Some Disagreeable Examples." In Surveying the Subjective Phenomena, vol. 2, ed. C. Turner and E. Martin, 159-214. New York: Russell Sage.



Documents

Measuring Happiness in Surveys: A Test of the Subtraction Hypothesis