Vital andHealth StatisticsCognitive Processesin Long-TermDietary Recall

Series 6:Cognition and SurveyNo, 4

Measurement

This report describes a program of research designed to explore the cognitiveprocesses and representations that subserve dietary reporting in surveys.Specific experiments investigated recall of which foods were eaten, estimationof the frequency with which foods are eaten, and judgments of portion sizes.Implications for cognition, surveys, and nutritional epidemiology are discussed.

US, DEPARTMENT OF HEALTH AND HUMAN SERVICESPubllc Health ServiceCenters for Disease ControlNational Center for Health Statistics

Hyattsvllle, Maryland

October 1991

DHHS Publication No. (PHS) 92-1079

I

I

Copyright Information

All material appearing in this report is in the public domain and may bereproduced or copied without permission; citation as to source, however, is

appreciated.

Suggested citation

Smith AF. Cognitive processes in long-term dietary recall. National Center forHealth Statistics. Vital Health Stat 6(4). 1991.

Librsry of Congress Cataloging-in-Publication Data

Smith, Alberl F.Cognitive processes in long-term dietary recall.p. cm. – (Vital and health statistics. Series 6, Co9ni~on and su~ey

measurement ; no. 4) (DHHS publication ; no. (PHS) 92–1079)By Alberi F. Smith.Includes bibliographical references.ISBN 0-6406-044S71. Reducing diets– Evaluation. 2. Food habits – Evaluation. 3.

Recollection (Psychology) 4. Health surveys– United States. 1. National Centerfor Health Statistics (U.S.) Il. Title. Ill. Series: Vital & health statistics. Series 6.Cognition and suwey measurement: no. 4. IV. Sefies: DHHS publication : no(PHS) 92-1079[DNLM: 1. Cognition. 2. Diet Surveys. 3. Eating. 4. Food Habits. 5.Memory. W2 A N146vf no. 4]RM222.2.S614 1991613.2’5’0723–dc20DNLM/DLCfor Library of Congress 91-13676

CIP

National Center for Health Statistics

Manning Feinleib, M.D., Dr. P.H., Director

Robert A. Israel, Deputy Director

Jacob J. Feldman, Ph.D., Associate Director for Ana$visand Epidemiology

Gail F. Fisher, Ph.D., Associate Director for Planning andExtramural Programs

Peter L. Hurley, Associate Director for Vital and HealthStatistics Systems

Robert A. Israel, Acting Associate Director forInternational Statistics

Stephen E. Nieberding, Associate Director forManagement

Charles J. Rothwell, Associate Director for DataProcessing and Services

Monroe G, Sirken, Ph.D., Associate Director for Researchand Methodology

David L. Larson, Assistant DirectoG Atlanta

Office of Research and Methodology

Monroe G. Sirken, Ph.D., Associate Director

This research was part of a larger project, entitled “National Laboratory forCollaborative Research in Cognition and Survey Methodology:’ being conductedby the National Center for Health Statistics under grant SES-86123320 from theNational Science Foundation (NSF). Monroe G. Sirken, Ph.D., was the principalinvestigator for this grant, and Murray Abom, Ph. D., was the NSF programdirector.

This research was performed under Department of Health and HumanServices contract 282-87-0039 from the National Center for Health Statistics tothe Research Foundation of the State UniversiY of New York. Albert F. Smith,Ph.D., was the principal investigator. Jared B. Jobe, Ph.D., was the NCHSproject officer and is the technical editor and reviewer for series 6. David J.Mingay was the alternate project officer.

Although my name is the only name on the byline ofthis report, many people have contributed to the work thatis described. It is a pleasure to acknowledge them.

In my laboratory, I was assisted in the collection andanalysis of data by a group of talented people thatincluded Kathryn Murphy, Joseph Belluck, David Manzer,Adrian Clark, and Elsa Issa. Important contributions tospecific experiments were made by Jody Layer and Deb-orah Prentice. Several colleagues in the Department ofPsychology, particularly Patricia DiLorenzo and RalphMiller, made constructive suggestions at many pointsduring this project.

Without the extraordinary support of people in theOffices of Sponsored Program Development and Spon-sored Funds Administration at the State University ofNew York at Binghamton – Heather Tomashek, PaulParker, Dennis Saunders, and Joe Walker–it would havebeen impossible to carry out this work.

Much of this report was drafted at the Division ofBiostatistics, Columbia University School of PublicHealth, where Patrick Shrout, Michael Parides, AlanWeinberg, and Carl Pieper provided an encouragingenvironment.

The project was vastly enriched by the enthusiasm ofthe National Center for Health Statistics personnel whohave been involved in this research since its inception.Jared Jobe, David Mingay, Gordon Willis, and MonroeSirken all made contributions that have vastly improvedthe research and the report.

Thanks to Deborah Prentice for the companionshipand help that she has provided since the inception of thisproject and to Snack Smith for the companionship that hehas provided for as long as he has been around.

Albert F. SmithBinghamton, New York

,.,Ill

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ““”m

Summary ,, . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ., . . ...! . . . . ...! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Methods of dietary assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Validation of dietary assessment methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Psychological processes in survey responding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Dietw-yrecall, . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Experiment l: Effects ofrecordkeeping and retention interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Experiment 2: Effects ofretention interval and instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Experiment 3: Free recaIlof an unambiguous target list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12General discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Frequencyjudgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Experiment 4: Effects ofretention interval and reference period length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Experiment 5: Effects ofrecall using closed-ended diaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Experiment 6: Question-induced biasing of frequencyjudgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Portion size estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,,, .,... . . . ...!. ! . . . . . . . . . . . . . . . . . . . . . . . . . 20Experiment 7: Estimation of sizes of food portions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Experiment&Comprehension and utilizationof defined portion sizes.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25overview, . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Implications, . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ! . . . . . . . . . . . . 26Fimdcomment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Appendixes

I. Indirect scoringof a simulated memory experiment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31IL Indexes of reporting performance. . . . . . , . . . . . . . . . . , . . . . . . . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32III. Detailed results of experiment 2.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

List of text figures

1s Mean magnitude estimate, bysizeof displayed stimulus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21‘-)-, Coefficient ofvariation ofresponse ratios, by scaled size difference of displayed and reference stimuli. . . . . . . 22

List oftext tables

A, Mean number of items reported by subjects in experiment 1, by retention interval and diary type . . . . . . . . . . . 8B, Number of subjects in experiment 2, by demographic category and experimental condition . . . . . . . . . . . . . . . . . 9C, Number of items recorded bysubjects in experiment 2, by reference period length and retention interval,,.. IIID, Match and intrusion rates for subjects in experiment 2, by reference period length and retention interval . . . . 10E. Numbers ofitems recorded and reported by subjects in experiment 2, by instruction condition and reference

period length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11F, Match and intrusion rates for subjects in experiment 2, by instruction condition and retention interval.. . . . . . 11

v

G. Match and intrusion rates for subjects in experiment 3, group 1, by retention interval and diary type . . . . . . . . 12H. Match and intrusion rates for subjects in experiment 3, group 2, by reference period length and retention

interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13J. Mean proportion ofunique items reported by subjects in experiment I, by retention interval and diary type.. 14K. Correlations of Iog(frequency estimates) with log(number of recorded occurrences) for subjects in experiment

4, by reference period length and retention interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17L, Geometric means of frequency estimates by subjects in experiment 6, by cognitive context, food item, and

reference period length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19M. Response distributions ofreported portion size in experiment 8,by food item and definition ofmedium . . . . . 23

Symbols

--- Data not available

. . . Category not applicable

Quantity zero

0.0 Quantity more than zero but less than0,05

z Quantity more than zero but less than500 where numbers are rounded tothousands

* Figure does not meet standard ofreliability or precision

# Figure suppressed to comply withconfidentiality requirements

Cognitive Processes inLong-Term DietaryRecallby Albert F, Smith, Ph. D., State University ofNew York at Binghamton

Summary

To quantify dietary intake, three types of informationme needed – information about what individuals eat, howoften they eat those items, and the sizes of the portionsthey eat. This report describes eight experiments that wereconducted to investigate dietary reporting performance.

Dietary recall was studied under various combinationsof acquisition, retention, and retrieval conditions. Whenasked to report their dietary intake for a specified refer-ence period, subjects appear to rely largely on genericknowledge of their diets –they tend to report items thatthey are likely to have eaten or items that they routinelyeat rather than items that they specifically rememberhaving eaten during the designated period. This tendencyincreases as the time of the recall test becomes moreremote from the reference period.

The accuracy of estimates of the frequency with which

foods are eaten during a specified period also deterioratesas the amount of time between the end of the referenceperiod and the frequency test increases. Individualsappear to remember, with reasonable accuracy, how oftenthey ate various foods. However, there is sufficient dis-parity among individuals in their frequency calibrationthat the results of analyses of the frequency with whichdifferent individuals eat particular items –the sorts ofanalyses that are of most interest to epidemiologists —areless satisfacto~.

Two experiments suggest that current methodologiesare not satisfactory for collecting high-quality data onportion sizes, Individuals appear to have fragile memoriesabout the sizes of food portions they view, and they tendto be insensitive to the definitions of portion size providedon food frequency questionnaires.

1

Introduction

Because diet is believed to play a significant role indetermining health status, health scientists from variousdisciplines are interested in evaluating relationshipsbetween nutritional inputs and indexes of human health.Nutritionists, for example, are concerned especially withthe influence of dietary intake on growth and maturation;epidemiologists are concerned with relationships betweenthe intake of various nutrients and the incidence ofspecific diseases (1,2). Efficient evacuation of researchhypotheses about the health consequences of diet requiresaccurate information about the dietary intake ofindividuals.

Collecting information about the dietary intake ofindividuals has long been recognized as a challengingproblem in nutrition and epidemiology (3). Becausepeople tend to eat frequently and to eat a variety of foodsat any given meal and over time, tracking the dietaryintake of any individual is difficult. To observe the diets ofnoncaptive individuals is intrusive; for individuals to keeprecords of their intake is burdensome. Hence, most infor-mation about dieta~ intake is obtained by conductingsurveys, that is, by asking individuals to report what theyhave eaten during some period of time.

Surveys rely on respondents’ memories, and so collec-tors and users of retrospective self-report data must beconcerned about the accuracy of memory-based reports(4). Epidemiologists almost routinely express concernabout the likely fallibility of memory and the possibleconsequences of memory error, not only for dietary surveydata but for all survey data concerning health-relatedevents and behaviors (5–9). Thus, researchers who collectdietary information invest substantial effort in assessingthe accuracy of their data-collection techniques.

Despite much speculation about how memory andother cognitive processes influence dietary reporting,researchers have not explicitly investigated these proc-esses. This report describes a program of researchdesigned to explore the cognitive processes and represen-tations that subserve dieta~ reporting. An improvedunderstanding of these processes could enhance under-standing of the nature of dietary survey responses, suggestthe sorts of dietary data that might reasonably be col-lected, and guide the development of procedures toimprove reporting accuracy. The general strategy of theresearch program was to approach divergently the proc-esses hypothesized to be involved in dietary reporting,

that is, to identi~ the components of the various tasks inwhich respondents to dietary surveys engage and toexplore these in parallel.

In the remainder of this section, some of the tech-niques used to collect reports about dietary intake aredescribed, the psychological questions that are motivatedby considering these methods are discussed, and hypoth-eses about some of the cognitive processes involved indietary reporting are presented. In subsequent sections,experiments that were conducted to address variousaspects of the dieta~ reporting problem are described. Inthe final section, the implications of the experiments forthe collection of dietary data are discussed.

Methods of dietary assessment

Three principal classes of survey methods are used tocollect dietary information —the diet history, dietaryrecall, and food frequency procedures (2,3,10,1 1). All ofthese ask respondents to retrieve and report memories oftheir dietary experiences.

To obtain a diet history, a trained nutritionist con-ducts a detailed interview with a respondent, who is askedto describe the routine features of his or her diet duringsome period of time. The goal is not to obtain specificinformation about what the respondent has eaten, butrather to develop a general characterization of the respond-ent’s “typical” diet. -

Recall procedures ask respondents to report every-thing they ate or drank during a designated period; cus-tomary reference periods in epidemiology are 24 hours,3 days, and 1 week. In some recall procedures, therespondent is asked simply to list the foods and drinks thatcome to mind; in others, he or she is “guided” mentallythrough the reference period by the interviewer, Whenparticipating in recall procedures, respondents may beasked to supplement their statements with descriptions ofthe portions of the items they ate. In any case, in contrastto the diet history, the recall procedure is intended tocollect specific information about dietary intake. In prin-ciple, respondents’ answers could be scored as correct orincorrect according to whether they matched some recordof what the respondent actually ate and drank during thereference period.

Food frequency procedures require respondents toprovide either count or rate information about their intake

2

of each of a set of food and beverage items during somereference period. For example, a respondent. might beasked, “How many artichokes did you eat during the lastyear?” or “How often did you eat artichokes during thelast year?” The first of these questions asks for a coun~the second asks for a rate. Appropriate (and equivalent)answers to these two questions would be 52 times andonce per week, respectively. The customa~ referenceperiods of food frequency questionnaires range from1 month to 1 year.

The usefulness of the data collected with any of theseprocedures in identifying associations between dietaryintake and health outcomes depends on two relatedaspects of the data’s validity. The first is the extent towhich the respondents’ reports are accurate; the second isthe extent to which the respondents’ reports are represent-ative of their dietary intake. Intuitions about memorydecay and dieta~ variability suggest that there is probablya tradeoff between these properties of dietary reports.The most accurate data may be the least representative,whereas the potentially least accurate data may be themost representative. An individual will presumablyremember and report best the bite of food that he or shehas just swallowed, but unless that individual eats only onefoodstuff, the report of that just-swallowed item will notbe representative of his or her diet nor reflect its vari-ability. As the period of time about which an individual isasked to report is extended –bite by bite and meal bymeal - into the past, dietary variability should be repre-sented in reports. However, those reports about pastintake would be increasingly likely to contain errors.

Validation of dietary assessmentmethods

Nutritionists and epidemiologists have invested sub-stantial effort in evaluating the validity of reports aboutdietary intake. True validation of a survey methodrequires that dietary reports be compared to some recordof the respondent’s intake.1 Typically, the record is pre-sumed to be accurate, although it too may contain errors(3).

Consider two examples of the strategies used byepidemiologists to evaluate the validity of dietary reports:Hankin, Rhoads, and Glober (12)asked 50 individuals tokeep a checklist diary for 7 days, indicating each occasionon which they ate any of 33 target food items andrecording the quantity that they ate. On the day after theend of the recording period, subjects were interviewedabout their intake of the 33 foods on the preceding dayand during the preceding 7 days. For each food item andfor each time period, Hankin et al. reported the propor-tion of subjects who reported having eaten the item,

10ftcn, in nutritional epidemiology, comparing data obtained by onereporting technique to that obtained by another, more favored techniqueis called validation (3).

conditional on their having recorded it. Hankin et al. alsoreported the correlation, over subjects, between estimatedquantity of intake from the record and estimated quantityfrom the report. Willett et al. (13) asked 173 nurses tokeep diet records for 4 weeks and, subsequently, tocomplete a food frequency questionnaire. The validi~ ofthe questionnaire was assessed by estimating each individ-ual’s mean daily intake of 16 nutrients from both therecords and the frequency questionnaire and by calcu-lating the correlation, over subjects, of these estimates.

Two aspects of these studies are of special concern.First, studies of the validity of dietary reports are usefulonly insofar as their conclusions can be generalized to thereports of individuals for whom criterion information isnot available. In general, respondents to dietary surveyshave not kept records of their dietary intake, but it is thereporting accuracy of such individuals that is of generalinterest. If keeping dietary records modifies subsequentreports, the generalizability of validity estimates to indi-viduals who have not kept records would be severelylimited. Recordkeeping might improve memory directly,resulting in more accurate reports of dietary intake. Theclose attention to dietary intake required to keep a recordmight result in the encoding of information that is betterthan normal, Alternatively, recordkeeping might affectreports indirectly: Compared to individuals who have notkept records, respondents who have kept a record mightexert greater retrieval effort when tested; alternatively,they might respond more conservatively, given that theyknow that the investigator possesses the true answers (inthe records), Because typical survey respondents have notkept diaries, the performance of participants in validitystudies may differ from the performance of subsequentlysurveyed respondents. To use diaries to assess the accu-racy of subsequent reports requires first establishing thatdiarykeeping does not affect subsequent reports (14),Neither Hankin et al, (12) nor Willett et al. (13) checkedthe effect of recordkeeping on reports, although bothgroups of investigators were aware that recordkeepingmight affect reports.

Second, there appears to be no general consensusamong nutritionists and epidemiologists on standards ofvalidity for dietary reports or even on how validity shouldbe measured (2,3). Some investigators directly comparethe foods reported by a respondent to the foods that he orshe recorded, sometimes taking quantity into account (12).This procedure is most tractable for a closed set offoods –for example, the 33 foods studied by Hankin et al.(12). Recently, however, indirect scoring methods of thesort used by Willett et al. (13) have been more commonlyused. Food composition tables are used to translate therecorded items and the reported items into average dailyvalues of nutrients of interest. Then the associationbetween records and reports is calculated for eachnutrient, over respondents. Reports are declared valid ifsome criterion level of association is met. For example,Willett et al, (13) obtained 16 correlation coefficients, onefor each nutrient, and made separate decisions about thevalidity of reports for each nutrient.

3

Such indirect approaches may fail to detect certaintypes of reporting errors that maybe informative aboutwhat dietary information is available in respondents’memories and about the processes they use whilereporting. Researchers who use indirect methods tend totreat target nutrients individually rather than as an inter-dependent set. Therefore, if respondents erroneouslyreport having eaten items that are similar to consumeditems in some nutrients but not in others, such analysesmay falsely suggest that reports were accurate. This pointis discussed in more detail in appendix 1.

In sum, the decision about how validity is measured issignificant. One might reasonably argue that the measureused to assess validity need not be the measure that isultimately the focus of epidemiological analyses. Forexample, it may be appropriate to assess response validityusing direct scoring methods, and then to transformresponses to nutrient values for subsequent analyses of therelation of nutrient intake to health status. This reportuses direct scoring methods to study the reporting perform-ance of individual respondents.

Psychological processes in surveyresponding

Respondents to dietary surveys are asked what theyhave eaten or how often they have eaten various foods andin what quantities, and, if the survey instrument is consid-ered valid, the responses are used to make scientificinferences about diet-disease relationships and to formu-late health policy. During the last 10 years, survey meth-odologists have become increasingly sympathetic to thenotion that analysis of the cognitive processes that sub-serve survey question responding can contribute toimproved survey design and to the reduction of responseerror (15–18). An enhanced understanding of people’sknowledge of their dietary intake might improve dietarysurvey data by guiding the development of answerablesurvey questions. Initial steps toward such an under-standing of people’s knowledge might be based on anexamination of the types and frequency of errors exhibitedby respondents to dietary surveys.

In this section, the cognitive demands of dietarysurvey procedures are considered, and relevant theory anddata from cognitive psychology are summarized. Theintent of this review is to illustrate what might be reason-ably expected of a dietary survey respondent, to speculateabout what sorts of errors might occur, and to suggestsome hypotheses about and explanations of performancein dietary survey procedures.

Survey responding can be decomposed into questioncomprehension, information retrieval, and response for-mulation components; each of these aspects of answeringsurvey questions may be addressed empirically (15–17,19,20). In dietary surveys, question comprehensionincludes understanding what sort of information is soughtby the interviewer, understanding to what foods the

interviewer is referring, and understanding references toand making judgments about groups of foods (forexample, collard, turnip, and mustard greens), Informa-tion retrieval refers to searching memory for informationrelevant to a question. Response formulation refers to allof the mentaI processes involved in producing a response(for example, using specific information retrieved frommemory, using norms believed to be relevant to thequestion, or using some combination of these).

Completely accurate dietary recall would require thatthe respondent have a cognitive representation of diet thatincluded information about what he or she had eaten,some kind of temporal information, and quantity informa-tion. In addition, an exhaustive search process would berequired to operate on this memory representation toensure the retrieval of all items eaten during the desig-nated period. The target information would have to besufficiently discriminable for retrieval to be limited to theitems eaten during the designated period.

To fulfill all of these requirements would be implau-sible. In general, long-term memory for any set of items isfar from perfect (21). In contrast, memory for certainattributes of experience has proved to be quite accurate.For example, studies of memory for the frequency ofoccurrence of events have shown that people can makereasonably accurate judgments of relative frequency(22,23). To correctly report portion sizes would requireeither that the respondent have precise information aboutportion sizes in memory or that he or she be able totransform the contents of some representation into stand-ard units of size. Little is known about long-term memoryfor size.

What sort of internal representation of dietary infor-mation might serve as the basis of responding in both therecall and food frequency procedures? Consider as aworking hypothesis the following model, based loosely onAnderson and Bower’s (24) two-process model of freerecall: Suppose that an individual has general knowledgeabout his or her diet and that this knowledge can berepresented as an associative network among the variousfoods that the individual eats. (Foods eaten occasionallymay, of course, be added to the network; no specificposition is taken on the relation of the individual’s knowl-edge about his or her diet and that individual’s generalknowledge about food.) Suppose further that, for eachoccasion on which an individual eats a food item, a markeris associated to the mental representation of that fooditem and that these markers decay over time. In essence,then, there are two types of information in this represen-tation. The network of representations of food itemsconstitutes an individual’s generic dietary knowledgeabout his or her own diet. The markers, which may beassociated not only to the mental representation of fooditems but also to other contextual information, instantiatespecific dieta~ memories. When asked to recall dieta~intake, a respondent might use a memo~ search processthat accesses an appropriately marked representation of afood item and that follows associative paths to other items,

4

To report frequency information, the respondent mightcarry out some kind of counting operation on the markersassociated to the representation of the named food item(23).

A survey respondent asked to report dietary intake fora designated period is asked for specific dietary informa-tion and presumably attempts to report appropriateitems – items whose cognitive representations are taggedwith markers that reflect the appropriate time period.However, memory for dietary intake may be imperfect,and imperfect memory might result in two types ofreporting errors: The respondent may fail to report itemsthat he or she ate during the reference period (omissions)or may report items that he or she did not eat during thereference period (intrusions). Intrusions might beobserved if the respondent’s report combined reports ofspecific dietary memories with plausible inferences fromgeneric dieta~ knowledge. Because it is implicit in healthsurvey situations that respondents should be able toanswer the questions, they will be inclined to answer,regardless of whether their knowledge is sufficient, Thus,respondents will likely report having eaten items duringthe specified period that they did not eat.

These expectations are consistent with the results oflaboratory experiments on memory. Generally, in suchexperiments, subjects do not exhibit perfect performancewhen asked to recall a previously presented set of targetitems (for example, a list of words). Typically, when theto-be-recalled items are unrelated, errors are primarilyomissions and intrusions are rare (25), As the retentioninterval (the time between the presentation of the targetitems and the test) is lengthened, the reported proportionof target items decreases, but there is only a modestincrease in the intrusion rate (26). A quite differentpattern is observed when the target items are related (forexample, when they are exemplars of a single semanticcategory): Intrusions are observed, and the intrusion rateincreases as the retention interval is lengthened (27). Theitems that an individual eats during any given period are

related both by being members of a single semanticcategory and by being members of the more specializedcategory, “things that this individual eats.” Thus, it isreasonable to expect that the recall of dietary intakewould follow the recall pattern of related items.

For judgments of frequency, laboratory studies showthat subjects are highly sensitive to variation in the fre-quency with which items are presented and estimatefrequency quite accurately (22,23). Most systematic knowl-edge about the acquisition of frequency informationcomes from experiments in which subjects are exposed toa relatively small set of events, each of which occurs notmore than a few times, and are tested immediately.Respondents to food frequency questionnaires are askedto make judgments about a stimulus list that did not havea well-marked beginning, that occurred over an extendedperiod, and that contained events that occurred with awide range of frequencies. Controlled experiments thatevaluate performance under these conditions have notbeen reported in the psychological literature. Neverthe-less, data concerning memory for frequency suggest thatpeople may reasonably be expected to estimate frequen-cies accurately.

The next sections of this report describe experimentsthat addressed these issues. Recall of foods eaten, judg-ments of frequency, and judgments of portion size areaddressed in separate experiments. The goal of theseexperiments was to examine some of the cognitive proc-esses suspected to underlie dietary survey responding intasks that approximate those used in nutritional surveys.The reader is advised from the outset that these arepsychological experiments, designed to address psycholog-ical questions that appear relevant to some dietary surveymeasurement problems. Therefore, the specific informat-ion collected in the various experiments and the focusesof the analyses are not necessarily identical to the infor-mation that would be collected by nutritionists or epide-miologists or to the analyses that would be carried out byinvestigators in those disciplines.

5

Dietary recall

In the dietary recall procedure, the respondent isasked to think back over some period of time and toreport everything that he or she ate or drank during thatperiod. Although there is general consensus that respond-ents err in the dietary recall procedure, neither a com-plete description nor a full explanation of these errors hasbeen advanced. To develop such an account requiresidenti@ing the nature of the information that is availablein the memories of respondents. An understanding ofwhat information is available could help clarify what sortsof questions might reasonably be asked of respondents todieta~ surveys.

Two hypotheses about performance in the dietaryrecall procedure stem directly from the model of therepresentation and retrieval of dietary information pro-posed in the introduction. First, because of the decay ofthe markers that represent the particular occasions onwhich food items are consumed, the contribution toreports of specific memories should decrease as the lengthof the retention interval increases. Second, as the lengthof the retention interval increases, respondents’ reportswill consist increasingly of reports of generic dieta~ infor-mation. The experiments reported in this section wereconducted to collect data that would help evaluate thesepredictions.

The general strategy of these experiments was to askpeople, after the end of a period during which they hadrecorded their dietary intake, to report what they hadeaten during that period. Reporting accuracy was assessedby comparing the reports to the diary records. To testspecific issues of theoretical concern, various aspects ofthe conditions under which subjects recorded andreported their dietary intake were manipulated. Thesevariables included the duration of the diarykeepingperiod, the amount of time between the end of thediarykeeping period and the memory test, and thereporting instructions given to subjects at the time of thememory test. These experiments also addressed severalmethodological issues, the resolution of which was neces-sary for meaningful interpretation of the accuracy data.

In this section, three experiments that involved twogroups of subjects are described. Experiment 1, conductedwith a group of university undergraduates, evaluatedwhether diarykeeping is a suitable method for validatingsubsequent reports, whether participating in a recall pro-cedure prior to keeping the diary would influence

subsequent performance, and whether reporting perform-ance depends on retention interval, Experiment 2exploited experiment 1‘s findings concerning the suit-ability of diarykeeping to explore further, with a heteroge-neous community sample, the effect of the duration of theretention interval on reporting performance. In addition,experiment 2 examined reporting performance for refer-ence periods of different lengths and, indirectly, the orga-nization of dietary memory. In each of these experiments,analyses of the data were concerned only with recall ofitems that were eaten during the diarykeeping period, notwith the frequency with which those items were eaten norwith the quantities in which they were eaten. Experiment3, conducted with subsets of the participants in experi-ments 1 and 2, assessed the ability of individuals to recalla set of food items that they had listed prior todiarykeeping.

Experiment 1: Effects ofrecordkeeping and retentioninterval

To develop a model of dietary recall performancerequires an understanding of the factors that influencereporting accuracy. However, to evaluate accuracy ofdietary reports requires an acceptable technique for vali-dating responses. The most obvious possibility is a writtenrecord, but to use this technique requires establishing thatrecordkeeping does not affect subsequent reports aboutthe recorded events (14).

Similar considerations apply to the administration of adietary recall procedure prior to any recordkeeping, Datacollected in an initial recall trial have a variety of potentialuses in research of this type. However, if participation insuch a procedure affects a subject’s subsequent perform-ance, it would be inappropriate to conduct the procedure.

This experiment was conducted to evaluate diary-keeping as a method for validating responses, to deter-mine whether a preliminary recall task influencedsubsequent reports, and to examine the effects of variationin the length of the retention interval on performance.

Method

Subjects –Ninety-six university undergraduates wererecruited for a two-session study of health-related

6

behavior. Subjects were given credit toward the researchparticipation requirement of their introductory psychologycourse and were paid $12.50 to keep a diary for 1 week.

Design – To determine whether diarykeeping influ-enced reporting performance, half of the subjects wereassigned to record their food and drink intake for 1 week(food diary condition); the remaining subjects recordedeither their mass media consumption (books, newspapers,magazines, radio, television, etc.), their social interactions,m their telephone activity (nonfood diary condition). Toevaluate whether engaging in the dietary recall procedureprior to recording would influence subsequent perform-ance, half of the subjects were asked at the beginning ofthe first laboratory session to report their dieta~ intakefor the preceding week (initial recall); the remainder didnot complete this task (no initial recall). Finally, to studythe nature of reporting performance as a function of timesince the diarykeeping period, half of the subjects wereassigned to return on the day after the last day of thediarykeeping week (immediate test); the remainder werescheduled to return 3 weeks later (delayed test).

Procedure – On the subjects’ first visits to the labora-tory, those assigned to the initial recall conditionattempted to recall their food intake for the precedingweek, All subjects were instructed on how to keep theirdiaries, Food diary subjects were asked to record everyitem that they ate or drank during the next 7 days;nonfood diary subjects were asked to record one of thealternate sets of events described above, Each subject wasgiven a set of diary forms and envelopes, was asked tostart a new form on each day of the diarykeeping period,and was instructed to return the preceding day’s formeach day to one of three secure campus locations. Subjectsengaged in several additional tasks that are irrelevant tothe present discussion, were scheduled to return for theirsecond sessions, and were dismissed.

In their second sessions, all subjects were asked torecall their food intake for the week during which theyhad kept the diary, Subjects were asked to report whateverthey could remember eating and drinking during thatweek, Subjects reported orally, and reports were tape-recorded, Then, those subjects who had engaged in theinitial recall task were asked also to attempt to recall theset of items they had listed during the first session.(Analyses of these reports are described as experiment 3.)

Data analysis

Two types of data are reported. For some of theresearch questions, the appropriate measures are countsof items recorded in the diaries or reported during therecall test. When simple counts are reported, the numberof distinct items (that is, the number of items after the listof recorded or reported items has been edited to removeduplicates) is of usual interest. For other questions, theappropriate measures are indexes of the correspondencebetween the items reported during the recall test andthose recorded in the diary.

To score the dietary reports, each set of items (thereported items and the recorded items) was edited toeliminate duplicate entries and to consolidate differentlabels for single items (for example, “green salad” and“tossed salad” would have been combined). After editing,each distinguishable item that a subject recorded orreported was treated as having occurred only once in eachset. The edited sets of items were then classified into threemutually exclusive groups: Reported items that had beenrecorded in the diary were classified as matches; recordeditems that were not reported were labeled omissions;reported items that were not recorded were called intru-sions.z These counts were used to calculate the match andintrusion rates of each subject. The match rate is theproportion of recorded items that were reported (the ratioof matches to the sum of matches and omissions), and theintrusion rate is the proportion of reported items thatwere intrusions (the ratio of intrusions to the sum ofmatches and intrusions). (These measures and the ration-ale for their use are discussed more fully in appendix H.)

To assess the reliability of these scoring procedures,two individuals scored the reports of 24 subjects. Thecorrelations between the resulting scores were 0.95 formatch rates and 0.83 for intrusion rates. Thus, the judg-ments of individual scorers were deemed acceptable forthe protocols of the remaining subjects.

Results and discussion

Two subjects were dropped from the study becausethey were unable to return for the second session at theirscheduled times. Each of the other subjects submitted theseven required daily diaries and returned for the secondsession according to the prescribed schedule. This experi-ment was run as two replications of 48 subjects; oneanalysis is reported only for data collected in the firstreplication.

Effect of diarykeeping– Logically, the first issue ofconcern is whether recordkeeping influenced perform-ance. If the subjects’ reports depended on whether theyhad kept a food diary, then the usefulness of diaries as avalidation method would be suspect. To answer this ques-tion requires comparing the reports of subjects who keptfood diaries to those of subjects who did not. Thus, somecharacteristic of the subjects’ reports other than accuracy(for example, number of items reported) must be

2For purposes of this report, intrusions are defined strictly as reporteditems that had not been recorded. It is always possible that a reporteditem was eaten,but not recorded, in which case it would be scored as anintrusion despite its having been eaten during the reference period. Toconduct research of the type decribed in this report, some standard hasto be accepted as representative of the truth; in these experiments, thediaries were taken as this standard. Smith et al. (28), in experiment 2,showed that subjects were highly consistent, over recording periods, inhow many distinct items they recorded. This suggests that the subjects inthese experiments took seriously the diarykeeping task and that thequality of subjects’ diaries did not deteriorate over time.

7

examined, because there is no way to evaluate the accu-racy of the reports of subjects who did not keep diaries.

Table A shows the numbers of items reported bysubjects in the two diary conditions for each retentioninterval. The type of diary kept by the subjects did notinfluence systematically how many items they reported(F(1,90) < 1), and the interaction of the length of theretention interval and diary type was not significant(F(1,90) = 2.49, p > 0.10). Not surprisingly, the numberof reported items depended on the length of the retentioninterval. Subjects who were tested immediately after thereference period reported significantly more items thandid those who reported 3 weeks later (F(1,90) = 5.81,p < 0.02). Taken together, these results indicate thatdiarykeeping is a suitable validation procedure for thesestudies. (For a fuller discussion, see Smith et al. (14).)

Effect of initial recall– Data from subjects’ reportsabout their dietary intake for a period that precedes thediarykeeping period have several potential uses inresearch of this type. However, to collect such data wouldbe appropriate only if the procedure did not affect laterperformance. If, when recalling for the diarykeepingperiod, subjects who had engaged previously in such aprocedure reported more items or reported more accu-rately than subjects who did not, the procedure should notbe carried out. Again, this is because the typical respond-ent to a dietary survey has not engaged in such a previousrecall attempt, and one of the research objectives was tocollect data that are at least somewhat generalizable tosuch respondents.

The initial recall procedure affected neither the countnor the accuracy measure of performance. On average,subjects who had engaged in the initial recall procedurereported 38 different items, whereas those who had notreported 36.2. This difference was not significant (F(1,90)= 1.33), and the initial recall variable did not interact withany other experimental variable. Among subjects who hadkept food diaries, subjects who had participated in theinitial recall procedure did not differ on the accuracyindexes from those who had not (F(2,42) < 1). The matchand intrusion rates of subjects who participated in theinitial recall procedure were 0.48 and 0.30, respectivelyfor subjects with no initial recall, these rates were 0.46 and0.26, respectively.

Retention interval– The major substantive question ofthis experiment was whether reporting performancedepends on the length of the retention interval and, if so,

Table A, Number of items reported by subjects in experiment 1,by retention interval and diary type

Retention interva/

Diary type Immediate test Delayed test Mean

Number

Food . . . . . . . . . . . . . . . . 39.4 36.2 37.9Nonfood . . . . . . . . . . . . . 43.7 28.3 36.2Mean . . . . . . . . . . . . . . . 41.5 32.3 37.0

NOTE 24 subjects participated in each disry group that received an immediate tes~ 23

subjecte participated in each diery group that recaivad a delayed test,

how. Two analyses are reported – an overall analysis ofaccuracy for the entire diarykeeping period (as describedabove), in which a reported item was scored as a match ifit had been recorded at any time during the diarykeepingperiod, and an analysis by days, in which a match rate wascalculated separately for each day of the diarykeepingperiod.

Consistent with extensive data from other memoryexperiments, reporting performance deteriorated as theretention interval was lengthened – the average matchrate decreased, and the average intrusion rate increased,For subjects who were tested immediately after the end ofthe diarykeeping period, match and intrusion rates were0.55 and 0.22, respectively for subjects who were testedafter a 3-week delay, the match and intrusion rates were0.38 and 0.34, respectively. The subjects in the two reten-tion interval groups differed significantly on each meas-ure: For match rate, F(1,43) = 17.36, p < 0.0001; forintrusion rate, F(1,43) = 7.82, p <0.01.

If the probability of reporting a consumed itemdecreases with the amount of time that has elapsed sinceeating that item, one would expect, at least for subjectstested immediately after the end of the diarykeepingperiod, that memory for intake on the later days of thereference period would be superior to memory for intakeon the earlier days. This hypothesis was evaluated bycalculating, for each subject in the first replication of theexperiment, a match rate for each day of the referenceperiod: The number of matches for each day was dividedby the number of items recorded on that day. Then, foreach subject, these daily match rates were regressed onday of the reference period; the slope of the estimatedregression line (change in match rate per day) was takenas an index of memory stability, For example, a slope of Owould indicate the absence of any systematic change inmatch rate over the days of the reference period, whereasa positive slope would indicate higher match rates for thelater days of the reference period than for the earlierones.

The daily match rates of subjects tested immediatelyafter the diarykeeping period increased from the earlydays of the reference period to the later ones, whereasthose of subjects tested after a 3-week delay showed nosystematic relationship to day of the reference period, Themean slopes for subjects in the immediate- and delayed-test conditions were 0,028 and –0.001, respectively; thesediffered significantly, t(45) = 3.28, p <0.005. In terms ofthe model of memory proposed in the introduction, theseresults suggest that specific memories deteriorate day byday, so that the reports of subjects tested after a 3-weekdelay are likely to consist less of specific clietary memoriesthan of generic dietary information. This hypothesis ispursued further in experiment 2.

Experiment 2: Effects of retentioninterval and instructions

Experiment 1 showed that diaries were a suitablestandard against which reports of dietary intake could be

8

scored, that subsequent performance was not affected byengaging in a recall trial prior to recording, and thatdietary reporting performance decreased as the retentioninterval increased. The dependence of reporting perform-ance on the retention interval was shown in both ageneral and a more refined way: Overall performance wasbetter for subjects tested immediately after the retentioninterval than for those tested after a 3-week delay; morespecifically, the match rates of the former subjects were,on average, best for the last day of the reference periodand deteriorated back toward the first day at a rate ofabout 3 percent per day.

Subjects in the delayed-test condition of experiment 1were tested 29 days after the beginning of the referenceperiod, Their match rates were considerably above zero,but their daily match rates did not depend on the day ofthe reference period. Obviously, then, the reports of thesesubjects were based, at least in part, on memories otherthan those whose deterioration was revealed by the sys-tematic decline in daily match rate for subjects in theimmediate-test condition. Experiment 2 was designed topursue the hypothesis that individuals’ reports of dietaryintake are based, in large part, on generic knowledgeabout their diets.

To study the dietary reporting performance of individ-uals who were more similar to national health surveyrespondents than are university undergraduates, a hetero-geneous sample of community volunteers was recruited.Three variables were manipulated to evaluate specifichypotheses about how dietary intake is represented inmemory and, in particular, to examine the proposeddistinction between generic knowledge and specificdietary memories. The length of the retention interval wasvaried to replicate and extend, over a greater span of time,the manipulation of experiment 1. The observed effect ofthe length of the retention interval in that experiment wasinterpreted as evidence that specific dietary memoriesbecome unavailable over time. The length of the referenceperiod was manipulated to evaluate the notion that thereis a tradeoff, reflected in performance measures, betweendietary representativeness and reporting accuracyBecause people have the opportunity to eat a greatervariety of items during a long period than during a shortone, reporting performance for longer reference periodsshould be less sensitive to the length of the retentioninterval than that for shorter reference periods. Reportinginstructions were manipulated to test the hypotheses thatdifferent sets of instructions –in the form of cues intendedto help respondents retrieve memories –might providedifferent entry points into memory and that differentpatterns of responses in different instructional conditionsmight be informative concerning the organization ofdietary memory.

Method

Subjects- Subjects were recruited by advertising in theBinghamton, New York, area for participants for a studyof health-related everyday behavior. Subjects were

assigned randomly to conditions, but under the constraintthat the ages and genders of the subjects be roughlybalanced over the eight experimental conditions definedby the lengths of the reference period and of the retentioninterval. Data from 170 subjects are reported; table Bclassifies these subjects by demographic categories andexperimental conditions and shows also how many sub-jects failed to complete the experiment. Data from sevensubjects who completed the procedure were lost beforethey were analyzed. Subjects were paid $16 for the labo-ratory sessions and $12,50 per week for diarykeeping.

Design – Eight experimental conditions were definedby the orthogonal combination of two reference periodlengths and four retention intervals. Within each of theseconditions, subjects were assigned at random to threeinstruction conditions. Approximately half of the subjectswere assigned to keep a diary for 2 weeks; the remainderwere asked to keep a diary for 4 weeks. Approximatelyone-quarter of the subjects were assigned to each of thefour retention interval conditions– immediately after thereference period (O weeks) or 2, 4, or 6 weeks after theend of the diarykeeping period. When asked to recall theirdietary intake for the reference period, subjects wereassigned randomly to receive one of three different sets ofrecall instructions – to report in reverse chronologicalorder, to report by food groups, or to report by meal.These instructions involved presentation of prompts, orcues (e.g., “meats” and “fruits” for the food group instruc-tions; “breakfast” and “lunch” for meals; a calendarmarked with dates for reverse chronological).

Procedure – Subjects attended two laboratory sessionsand kept a food diary. In the first session, approximatelyhalf of the subjects attempted to recall what they hadeaten and drunk during the preceding 2 or 4 weeks,depending on the length of time for which they would beasked to keep a diary. Then all subjects were instructed onhow to keep the food diary. They were asked to recordeach item that they ate or drank during their assigned

Table B. Number of subjects in experiment 2, by demographiccategory and experimental condition

Demographic categoty

Condition Total F–1 F–2 F-3 F-4 M-1 M-2 M-3 M-4 Drop

Target . . . . . . . . 24 4 5

Total . . . . . . . . . 170 27 372-O . . . . . . . . . . 24 2 72–2 . . . . . . . . . . 23 4 32-4 . . . . . . . . . . 19 3 62-6 . . . . . . . . . . 22 4 54-o . . . . . . . . . . 21 3 54-2 . . . . . . . . . . 22 4 44-4 . . . . . . . . . . 19 4 34-6 . . . . . . . . . . 20 3 4

Drop . . . . . . . . . 14 3 2

Number

5223210

36 15 12 18 20 5 14611241053224003212204412321 14222211522221242132025212214

303210 . . .

NOTE Conditions are dsalgnated by duration of the reference period, In weeks, and theretantion interval, in weeks (for example, 2-o refera to a 2-week reference period and a O-weekretention interval). Demographic groups are designated by gender (M = male! F = female)and age catagory (1 = 21-2S4 2 = 30-44 3 = 45+ 4 = 65 and over). Drop refers toindividuals who failed to complete the experiment. Data from 7 participants who sompleted theexperiment were Ioat prior to data analysis and ara therefore not included In this table.

9

reference periods. Subjects were given a supply of formsand envelopes, told to start a new form each day, andasked to submit their records by mail twice per week.Finally, subjects were told that a followup session wasrequired, and second sessions were scheduled according tothe requirements of the retention interval manipulation.There was no indication that the followup session wouldinvolve memory tests.

In the second session, subjects were asked to reportall items that they had eaten or drunk during the diary-keeping period. Each subject was given one of three setsof instructions for reporting: Reverse chronological order,food groups, and meals. Subjects were told that these cuesmight help them organize their reports. They were toldthat they should not feel constrained by the cues, butrather that they should report whatever came to mind.Subjects engaged in several additional tasks, some ofwhich will be described in subsequent sections of thisreport, and were debriefed.

Results

Although the experimental participants were gener-ally very cooperative in submitting their diaries, some werenot received, For the 2-week and 4-week recordingperiods, diaries were missing for 6 subject-days and 33subject-days, respectively. These constituted 0.5 percentand 1.4 percent, respectively, of all the subject-days forthe two conditions. It was assumed that these losses wouldhave negligible effects on the results.

The data were edited and scored using the proceduresdescribed for experiment 1. Ninety-six report protocolswere scored by two judges: The correlations between thematch and intrusion rates calculated by the two judgeswere 0,91 and 0.90, respectively.

To enhance the clarity of exposition, the principalresults are described here, but presentation of detailedresults, including values of test statistics, is deferred toappendix III. Unless otherwise noted, statements con-cerning the effects of experimental manipulations aremade with a p of 0.01 or less, and statements concerningthe lack of effects are made with ap of 0,25 or greater. Tofurther simplify the presentation of the results, the effectsof the length of the retention interval and the referenceperiod length are discussed first, and then the effect of thereporting instructions is examined.

Effect of retention interval and reference periodlength – Table C shows the mean number of itemsrecorded by subjects in each of the conditions defined bythe lengths of the reference period and of the retentioninterval. The length of the reference period affected thenumber of items recorded in an orderly and sensible way:Subjects who kept diaries for 4 weeks recorded moredifferent items (but only about 1V2 times as many) thandid subjects who kept diaries for 2 weeks. As some itemsare eaten on multiple occasions during the referenceperiod, the rate of growth of the to-be-reported set ofitems is negatively accelerated. Because the retentioninterval followed the recording period, it should have had

Table C. Number of items recorded by subjects in experiment 2,by reference period iength and retention intervai

Reference period length

Retention interval 2 weeks 4 weeks Mean

Number

Oweek . . . . . . . . . . . . . . . 62.9 107.1 83.52weeks . . . . . . . . . . . . . . 63.4 105.5 64.04 weeks . . . . . . . . . . . . . . 69.7 108.7 69.26weeks . . . . . . . . . . . . . . 71.8 104.3 87.3Mean . . . . . . . . . . . . . . . . 66.7 106.4 65.9

NOTE For number of subjects per condition, see table B.

no effect on the number of items recorded. Table C showsthat, indeed, the retention interval did not affect the meannumber of items recorded by subjects during either refer-ence period. There was no interaction of the lengths of thereference period and of the retention interval on thenumber of-items recorded,

Table D shows the mean match and intrusion ratesfor subjects in the eight experimental conditions definedby reference period length and retention interval. Twofeatures of these results are particularly salient. First, theaverage match rate over experimental conditions was only0.38; among the conditions, the highest mean match ratewas only 0.49. Second, on average, approximately one-third of the reported items were intrusions; in no condi-tion was the mean intrusion rate less than 0.27. In general,then, the subjects’ free reports of their dietary intake didnot effectively describe their intake during the specificperiod about which they were asked.

The pattern of match and intrusion rates shown intable D indicates, consistent with the results of experi-ment 1, that dietary reporting performance deterioratedas the length of the retention interval increased. Consist-ent with other data on memory decay, the decrease inthese rates was negatively accelerated. (All tests of theeffect of retention interval length reported in this sectionused a contrast with a negatively accelerated “shape”: Theweights applied to the 4- and 6-week levels of retentioninterval were equal and, in sum, opposite to the weightapplied to the O-week level.)

Although the average match rates of subjects assignedto the txvo reference periods did not differ, subjects whoreported for a 2-week period exhibited higher intrusion

Table D. Match and intrusion rates for subjects in experiment 2,by reference period iength and retention intervai

Reference period length

2 weeks 4 weeks Mean

Retention interval p(mtc) pfint) p(mtc) p(int) p(mtc) p(lnf)

Rate

Oweek . . . . . . . . 0.49 0.27 0.46 0.29 0.48 0.262weeks . . . . . . . 0.42 0.39 0.37 0.28 0.39 0.344weeks . . . . . . . 0.26 0.40 0.34 0.34 0.31 0.376weeks . . . . . . . 0.30 0.42 0.32 0.33 0.31 0.38Mean. .,...... 0.36 0.37 0.37 0.31 0.38 0.34

NOTE P(mtc) is the match rats p(int) Is the intrusion rate. For number of subjects percondtion, eee table B.

10

rates than did subjects who reported for 4 weeks. Further,as shown in table D, the mean intrusion rate increasedquite steeply over retention intervals for subjects in the2-week reference period condition but increased only veryslightlyfor those in the 4-week reference period condition.(The interaction of reference period length and retentioninterval was significant at p e 0.05.)

Effect of reporting instructions – At the time of thememory test, the subjects were given one of three sets ofretrieval cues. They were told that these were intended toaid rather than to constrain their reports and that theyshould feel free at all times to say whatever came to mind.Because subjects could use any preferred reportingstrate~~, the instruction manipulation was not expected toaffect performance markedly. This prediction was incor-rect; the observed results are potentially informative con-cerning the organization of memory for foods and the wayin which personal dietary experiences are retrieved fromthis memory.

Table E shows how many items, on average, wererecorded and reported by subjects who received each ofthe three sets of instructions for each reference periodlength, Appropriately, the reporting instructions did notinfluence the number of items recorded during the diary-keeping period. However, subjects who received foodgroup cues reported significantly more items than didthose who received reverse chronological or meal cues; themean numbers of items reported by subjects in the lattertwo conditions did not differ.

Table F shows the mean match and intrusion ratesfor subjects in the three instruction conditions. Matchrates did not differ significantly over the three conditions,and none of the interactions of instructions with otherexperimental conditions was significant. However, intru-sion rates did differ among the instruction conditions.Subjects who received food group cues exhibited signifi-cantly higher intrusion rates than did subjects whoreceived the reverse chronological or meal cues; the intru-sion rates of subjects in these latter two conditions did notdiffer.

Table E. Numbers of items recorded and reported by subjects inexperiment 2, by instruction condition and reference periodlength

InstructionsReference

period Backward Food group Meal Mean

Number of items recorded

2weeks . . . . . . . . . . . . 67.4 67.0 65.9 66.74weeks, . . . . . . . . . . . 100.2 110.1 106.2 106.4Mean . . . . . . . . . . . . . 82.3 68.9 85.1 65.9

Number of items reported

2weeks . . . . . . . . . . . . 35.5 46.9 36.6 42.64weeks . . . . . . . . . . . . 51.2 66.0 56,2 59.0Mean . . . . . . . . . . . . . 42.6 56.6 45.9 49.2

NOTE Numberof subjects per condition was as follows (from left to right and top to bottom):23, 32, 33, 19, 33, and 30.

Table F. Match and intrusion rates for subjects in experiment 2,by instruction condition and retention interval

hatruction

Backward Food group Meal Mean

Retention interval p(mtc) p(int) p(mtc) p(int) p(mtc) p(int) p(mtc) p(int)

RateOweek . . . . . . . . 0.51 0.17 0.46 0.34 0.46 0.26 0.48 0.282 weeks . . . . . . . 0.46 0.28 0.39 0.38 0.35 0.33 0.39 0.344 weeks . . . . . . . 0.26 0.34 0.33 0.42 0.34 0.37 0.31 0.376 weeks . . . . . . . 0.31 0.40 0.33 0.40 0.28 0.33 0.31 0.38Mean . . . . . . . . . 0.37 0.31 0.39 0.38 0.36 0.32 0.38 0.34

NOTE: P(mtc)la the match rate; p(int) is the intrusion rate. For number of subjects per cell, seetable E.

Discussion

These results build on those of experiment 1 insuggesting the important contribution of generic dietaryknowledge to dietary reporting. Three features of theresults particularly encourage this conclusion.

The first is the retention interval effect: As the reten-tion interval was lengthened, reporting performancedeclined to the point that only 30 percent of recordeditems were reported and 40 percent of reported itemswere intrusions. Even for the shortest retention intervals(which, in this experiment, admittedly followed longacquisition periods), subjects reported only half of theitems they had recorded and exhibited intrusion rates ofabout 30 percent. People, when reporting their dietaryintake for extended periods, clearly do not have access tomemory representations that include accurate temporalinformation. Specific memories surely contribute todietary reports: After all, match rates are highest forsubjects tested immediately after the end of the diary-keeping period, and, as was seen in experiment 1, subjectstested immediately after the end of a l-week diarykeepingperiod report more items from the last days than from theearlier days of the reference period. Nevertheless, dietaryreports appear to consist, in large part, of individuals’guesses about what they probably ate. Intrusions musthave some cognitive origin, and generic knowledge aboutdiet is a likely source, If subjects have general knowledgeabout their diets but imperfect representation of whenthey ate various foods, then, when they are asked to recallwhat they have eaten for a period of 2 or 4 weeks, theymay list foods that they routinely eat without regard towhen they ate them.

The second result that encourages the generic knowl-edge hypothesis is the interaction of the retention intervaland the reference period length on the intrusion rate. Theabsolute intrusion rates of subjects who reported for a4-week period did not depend on the length of theretention interval, whereas those of subjects who reportedfor a 2-week period increased as a function of the lengthof the retention interval. Because subjects who reportedfor a 4-week period had more opportunity, during theirdiarykeeping periods, to eat the routine elements of theirdiets, they had considerably more latitude than did the

11

2-week subjects to misremember and misdate their dietaryexperiences. When accuracy is scored on the basis ofwhether an item was ever eaten during the referenceperiod, the longer the reference period is, the moreaccurate subjects can be. To the extent that they can thinkof items to report, the greater will be the convergencebetween what they are supposed to report and what theydo report. If, as the retention interval is increased, indi-viduals’ reports degenerate to descriptions of their genericdiets, reports and records would become increasinglydiscrepant for subjects reporting for the 2-week interval,and this is precisely what was observed. These results areconsistent with those of laboratory experiments involvingthe recall of lists of categorically related words, whichshow that intrusion rates increase as retention intervalsincrease and, necessarily, decrease as the number ofto-be-remembered words from a category increases(25,29,30).

The third resuIt that supports the conjecture con-cerning the contribution of generic dieta~ knowledge toreports is the observed effect of reporting instructions onintrusion rates. The subjects’ general knowledge aboutfood may be organized by food groups –food groupinstructions increased the number of items that subjectsreported. It is of particular interest that these instructionselevated only the intrusion rate, not the match rate. Inother words, food group cues appear to have given sub-jects mental access to more responses but not to thespecific responses that would have improved performance.

Experiment 3: Free recall of anunambiguous target list

Individuals’ dietary reports appear to be based ongeneric knowledge of their diets. One might be concernedthat the patterns of results suggesting this conclusion arean artifact of the unusual conditions that prevailed duringthe acquisition of the to-be-reported items. Although theresults of experiments 1 and 2 accord reasonably well withrelevant published findings on free recall of related items,a distinctive feature of experiments 1 and 2 was thatacquisition occurred over very long periods of time.Because irrelevant events were interspersed, during theacquisition phase, with the to-be-reported events, subjectsmay have been confused about exactly what items were tobe reported. Experiment 3 was designed to examine recallof an unambiguously defined list of dietary items, Eachsubject was asked to report a set of food items that he orshe had listed during a single brief period, Specifically,each subject who had been assigned to the initial recallcondition of experiment 1 or 2 was asked to recall the setof items that he or she had recorded during the firstlaboratory session. Because that list was created during adiscrete period, there should have been no ambiguity atthe time of recall about what constituted the target set,

Method

Subjects– The two groups of participants in this exper-iment were those subjects who completed the initial recalltask in experiments 1 and 2. For this experiment, thesegroups will be labeled groups 1 and 2, respectively.

Design and procedure– Subjects assigned to the initialrecall conditions of experiments 1 and 2 were asked duringtheir first laboratory sessions to write down everythingthat they had eaten and drunk during the preceding week(experiment 1) or during the preceding 2 or 4 weeks(experiment 2), depending on the amount of time forwhich they would subsequently be asked to keep a diary.In the second session, following the dieta~ recall proce-dure, subjects were asked to recall the items that they hadrecorded during the first laboratory session. For group 1,two retention intervals (1 week and 4 weeks) were crossedwith two types of diary that had been kept during thereference period (food and nonfood). For group 2, fourretention intervals were crossed with two reference periodlengths.

Results and discussion

The recall protocols were scored in the fashiondescribed for experiment 1, with the initially recordeditems serving as the standard set. Table G shows meanmatch and intrusion rates for subjects in group 1 by type ofdiary and length of retention interval. Although the meanmatch rate was higher and the mean intrusion rate waslower for subjects who recalled their list after 1 week thanfor those who recalled their list after 4 weeks, the differ-ences were quite small – 0.06 for match rate and 0.07 forintrusion rate. Numerically, the effect of the retentionintervaI on these data was substantially smaller than thatobserved for reports of intake during the reference period,A multivariate analysis of variance (MANOVA) on thesedata showed a significant effect of retention interval@(2,42) = 3.28, p < 0.05) but neither a significant effectof diary type (F(2,42) = 1.94, p > 0.10) nor a significantinteraction (F(2,42) c 1).

Table H shows the match and intrusion rates forsubjects in group 2 by retention interval and referenceperiod length. Neither of these variables affected system-atically either of the measures of correspondence between

Table G. Match and intrusion rates for subjects in experiment 3,group 1, by retention interval and diary type

Retention interval

/immediate test Delayed test Mean

Diary type p(mtc) p(lnt) p(mtc) p(int) p(mtc) p(ht)

Rate

Food . . . . . . . . . 0.49 0.26 0.40 0.29 0.44 0.28Nonfood . . . . . . 0.52 0.26 0.48 0.37 0.50 0.32Mean . . . . . . . . 0.50 0.26 0.44 0.33 0.47 0.30

NOTE: P(mtc) is the match ratq p(int) Is the intrusion rate.

12

Table H. Match and intrusion rates for subjects in experiment 3,group 2, by reference period length and retention interval

Reference period length

2 weeks 4 weeks Mean

Refetrtlorr Interval p(mtc) p(int) p(mtc) p(int) p(mtc) p(lnt)

Rata

O week . . . . . . . . 0.41 0.45 0.43 0.48 0.42 0.472weeks . . . . . . 0.46 0.40 o.4a 0.44 0.47 0.424weeks . . . . . . . o.3a 0.43 0.43 0.48 0.41 0.466weeks . . . . . . . 0.41 0,48 0.44 0.45 0.42 0.47Mean, .,...,, 0.42 0.44 0.45 0.46 0.44 0.45

NOTE P(mtc) is the match rate; p(int) is the intrusion rate.

the two sets of responses: For reference period,F(2,73) < 1; forretention interval, F(6,146) < 1; for theinteraction of reference period and retention interval,F(6,146) <1. This is not to say that the length of thereference period had no effect on performance. Onaverage, both during the preliminary recall and during therecall of that list, subjects whose reference period was4 weeks recorded and recalled more items than did sub-jects whose reference period was 2 weeks. The averagenumbers of items recorded during the preliminary recalltask were 45.2 and 55.5 by subjects whose referenceperiods were 2 and 4 weeks, respectively; the averagenumbers recalled by these two groups of subjects were32.6 and 47.8. Subjects working with a 4-week referenceperiod record and report more items than do subjectswhose reference period is 2 weeks. However, for neitherthese counts nor for the match and intrusion rates wasthere any dependence, for subjects in group 2, on reten-tion interval.

Subjects appear to have been relatively indifferent towhat they recorded at the initial laboratory session, Thehigh intrusion rates suggest that at the time of recall, andquite possibly at the time of initial recording as well,subjects simply produce a list of items that they are likelyto have eaten.

General discussion

The results of experiments 1–3 converge to suggestthat individuals who are asked to recall their dietaryintake for a specific exlended period rely increasingly ongeneric knowledge about their diets as the time delay fromthe reference period to the recall test increases. Althoughspecific memories clearly contribute to dieta~ reports,those memories are rapidly lost.

Several types of evidence lead to this conclusion. Thedeterioration in reporting performance over time from theend of the reference period, reflected both in decreasingmatch rates and in increasing intrusion rates, suggests thatthe representation of temporal information associatedwith routine dietary intake decays. However, match ratesdo not drop to zero. Even 6 weeks after the end of a2-week reference period, the mean match rate was 0.30. If

subjects use generic dietary knowledge, they would beexpected to report from that knowledge even if they hadno specific memories of the reference period (31). Theobserved interaction in experiment 2 of reference periodlength and retention interval on intrusion rates and theobserved effect of reporting instructions are consistentwith this view.

An additional analysis of the data from experiment 1provides further support for the hypothesis that as reten-tion intervals are lengthened, subjects rely increasingly ongeneric knowledge to report their intake for a designatedperiod. If, as retention intervals are lengthened, reportsdegenerate to descriptions of generic dietary knowledge,then, if reports were scored against a list of items thatconstituted the salient components of generic dietaryknowledge, the reports of subjects tested immediatelywould contain more unique items —items that are not inthat list of elements of generic knowledge – than wouldthe reports of subjects who are tested 3 weeks after theend of the diarykeeping period.

To test this prediction requires some estimate of thecontents of generic dietary knowledge, Because the partic-ipants in experiment 1 were university undergraduates,many of whom lived in university residence halls andparticipated in a campus meal plan, it was assumed thatthe generic dietary knowledge of the subjects was suffi-ciently similar that a common estimate could be used forall of them. That estimate was constructed as follows:Each of 23 undergraduate participants in an unrelatedpsychological experiment was asked to list the items that atypical person of his or her age and gender would eat in atypical week, From these lists, a list was created thatcontained items listed by at least 10 of the 23 subjects;these items are considered stereotypic of the diets of theseindividuals. The l-week dietary report of each experi-ment 1 subject was scored against this list to find theproportion of reported items not on the stereotype list –this was called the proportion of unique items (that is, theproportion of the reported items unique to the subject).For a subject who reported only items from the list ofgeneric items, the proportion of unique items would havebeen O;for a subject who reported no items from the list ofgeneric items, the proportion of unique items would havebeen 1.

Table J shows the mean proportion of unique itemsfor subjects in the four conditions defined by diary typeand retention interval. Consistent with the prediction, the

Table J. Mean proportion of unique items reported by subjects inexperiment 1, by retention interval and diary type

Retention interva/

D;ary type hrrmediate test Delayed test Mean

Food . . . . . . . . . . . . . . . . 0.38 0.28 0.33

Nonfood . . . . . . . . . . . . . . 0.41 0.32 0.37Mean . . . . . . . . . . . . . . . . 0.39 0.30 0.35

NOTE Higher valuea indicate a greater propoflion of Iiated items not on the stereotype 1!s!.

13

proportion of unique items in the subjects’ reports was reports of subjects who were tested after a 3-week delaysignificantly higher for subjects who reported immediately (14).than for subjects who received a delayed test, F(1,89) = Although people have access to and report some14,43,p c 0.001,The dieta~ reports of subjects who were specific memories about their dietary experiences, thesetested immediately following the reference period corre- are incomplete, and dietary reports degenerate rapidly tosponded more poorly to the generic list than did the reports of general knowledge about dietary intake.

.

14

Frequency judgments

To characterize completely the nutritional intake ofindividuals, it is important to know not only what foodsthey eat but also how often they eat those foods: Experi-ments 4-6 investigated several properties of frequencyreports.

The most popular type of instrument for collectingdietary information in large-scale epidemiological studiesis the food frequency questionnaire. Respondents to suchquestionnaires are asked to indicate either the rate atwhich or how many times they ate each of several itemsduring some period – for example, 1 month or 1 year. Is itreasonable to ask people questions of this sort? Howaccurate are the responses? What is the nature of therepresentation in memory on which those responses arebased?

Several distinct lines of research in experimental psy-chology converge to show that people are quite sensitive tovariations among items in frequency of occurrence(26,32,33), Numerous experiments show that subjects canreport experienced frequency quite accurately when theyare tested shortly after experiencing the target set ofevents, Thus, insofar as dietary experiences during someperiod can be construed as a list of stimulus items,subjects might be expected to report, with reasonableaccuracy, how many times they ate various foods duringthat period. However, the conditions under which subjectsexperience the “list” of food items differ in several signif-icant ways from those of typical laboratory experiments onmemory for frequency “List items” are experienced overperiods that range from 1 month to 1 year, the number oflist items exceeds by far the number found in typicallaboratory experiments, and the range of frequencies withwhich list items occur is much larger. The experimentsdescribed in this section examined acquisition, retention,and reporting of frequency information under conditionsthat approximate more closely those under which foodfrequency information is acquired and remembered thando the conditions of most laboratory experiments onmemory for frequency.

Two measurement issues require some prefato~remarks, The first concerns the measurement of memoryfor frequency, Several measures of memory for frequencyare used in this section, One of these is the correlationbetween the actual frequency of occurrence of food items(as recorded in diaries) and subjects’ estimates of thosefrequencies. The correlation coefficient measures the

linear relationship between the two counts. Its advantageover other measures is that it standardizes the variabilitiesof the actual and estimated frequencies, but its disadvan-tage is that a correlation between actual and estimatedfrequencies could be quite high even if the estimates failedto reflect the absolute variation in actual frequencies. Forexample, if reported frequencies were consistently one-tenth of the actual frequencies, the correlation betweenthese sets of counts would be 1. The absolute variation infrequency is reflected by the slope of the function esti-mated by regressing reported frequencies on actualcounts. When feasible, this measure is reported. Variousmeasures of memory for frequency are discussed in detailby Naveh-Benjamin and Jonides (34).

A second measurement issue that requires clarifica-tion concerns the distinction between analysis of memoryfor frequency within subjects and analysis of memory forfrequency for items, across subjects, Most psychologicalexperiments on memory for frequency assess performancewithin subjects. Such experiments ask whether, for a givensubject, there is a functional relationship between theactual and the reported frequencies, and, if so, about thenature of that relationship. These questions are addressedby analyzing the relationship between the frequencies ofthe items presented to the subject and his or her fre-quency estimates. Typically, epidemiologists are not inter-ested in the association between estimated and actualfrequency for an individual over items. Instead, they areinterested in the association between estimates and actualfrequency for a given item over respondents. In otherwords, epidemiologists want to know not whether anindividual who eats chicken more often than hamburgerreports this, but whether an individual who eats morechicken than does a second individual reports a higherestimate for frequency of eating chicken than does thesecond, Such a between-subjects analysis is reported,where appropriate.

The three experiments described in this section inves-tigated various aspects of the ability of people to makefood frequency judgments. Experiments 4 and 5 investi-gated the accuracy of frequency reports for previouslyrecorded dietary events and the conditions that influencethe accuracy of those judgments. Experiment 4, carriedout with a subset of the participants in experiment 2,examined the effects of reference period length and reten-tion interval on frequency judgments. Subjects’ frequency

15

estimates were regressed on counts of the number ofoccasions on which they had recorded target items in theirdiaries. To prepare materials for these subjects, recordeditems were collapsed into more general classes. Becausesubjects and investigators might have disagreed about thiscategorization, experiment 5 asked subjects to report howoften they had eaten each of a closed set of items whichwere described identically both on a checklist diary formand on the frequency questionnaire. Experiment 5 alsoevaluated whether engaging in the dietary recall proce-dure prior to making frequency judgments affected thosefrequency judgments.

The approach of experiment 6 to the study of fre-quency judgments was somewhat different. Experiment 6investigated the sensitivity of frequency judgments toretrieval cues. Prior to making a judgment about howoften they had eaten some food during a specified timeperiod, subjects in experimental conditions were in-structed to engage in thoughts that were designed eitherto promote or inhibit the retrieval from memory ofinstances of target foods.

Experiment 4: Effects of retentioninterval and reference periodlength

Numerous experiments have found a systematic rela-tionship between the remembered frequency of occur-rence of events and the actual frequency with which thoseevents occurre’d. Underwood et al. (26) showed that as thetime between the presentation of the target set of eventsand a frequency test increases, the ability of subjects todiscriminate betxveen different frequencies deteriorates,so that the slopes estimated by regressing estimated fre-quencies on actual frequencies decrease over retentionintervals, Experiment 4 evaluated whether such a system-atic effect of the length of the retention interval on theassociation between actual and estimated counts would beobserved for judgments of food frequency,

A subset of the participants in experiment 2, afterreporting their dietary intake for their diary period, wereasked to indicate how often, during the reference period,they had eaten each of a set of food items. A special testset of foods was constructed for each participant, based onthe contents of his or her diary.

Judgments of frequency might be based on counts ofthe markers that represent specific dietary memories, orthey might be reports of normative frequencies that theindividual supposes are typical of his or her diet (23,35).When subjects report frequencies for some referenceperiod after a delay, estimation may involve countingmarkers that are decayed sufficiently that they may plau-sibly be records of events that occurred during the refer-ence period. Such markers would likely be lessdiscriminable from markers established during adjacentperiods than would the stronger markers establishedduring an immediately past reference period. Given any

such process or mechanism, one would expect, for reasonssimilar to those discussed in experiment 2, that the perform-ance of subjects who report about 2-week referenceperiods would deteriorate as the lengths of their retentionintervals increase. One would further expect that amongsubjects who report about 4-week reference periods, theperformance of those tested immediately would be best,and the performance of those tested after longer retentionintervals would be worse but roughly constant over thoseretention intervals.

Method

Subjects– Of the participants in experiment 2, 128served in this experiment.

Materials– For each subject, the frequency of occur-rence of each item in the diary was counted. If severaldiary entries could be construed as referring to the samefood item or to closely related food items, they werecombined into a single category and their counted fre-quencies were summed (for example, two occurrences of“blueberry pie” and one occurrence of “apple pie” wouldhave been counted as three occurrences of “pie”). Thefrequency questionnaire for each subject was constructedto include items with a broad range of frequencies ofrecorded occurrences. The recorded items were sorted byfrequency and up to 25 items were selected, subject to theconstraints that the 5 most frequent items be tested andthat approximately equal numbers of items be sampledfrom each of four quartiles of the remainder of thedistribution. The selected items were randomized, and aquestionnaire with those items was given to the subject.

Design and procedure- The design paralleled that ofexperiment 2. Subjects had been assigned to keep a fooddiary for 2 weeks or 4 weeks, and returned for memorytests at the end of the diarykeeping period or 2, 4, or 6weeks later. After they had completed the dieta~reporting task described in experiment 2, subjects wereasked to indicate how many times during the diarykeepingperiod they had eaten each of the items that appeared ontheir questionnaires.

Results and discussion

Because the set of food items differed for each sub-ject, only within-subjects analyses were conducted onthese data. The question addressed by this experiment waswhether, as retention interval was lengthened, there wasdeterioration in the ability of subjects to discriminate, intheir frequency estimates, among items that occurred withdifferent frequencies.

Index of peijormance– Subjects in the 4-week refer-ence period ate their most frequently eaten items substan-tially more often than did those in the 2-week referenceperiod condition. In an attempt to equate the variances ofrecorded and reported frequencies over the experimentalconditions, both sets of counts were transformed logarith-mically, and the use of the slopes of regressions of trans-formed estimates on transformed recorded frequencies

16

was considered as a measure of memory, Even afterapplying the logarithmic transformation, significant diffm-ences remained among the experimental conditions in thevariances of both estimates and counts of recorded occur-rences. However, the ratio of the standard deviations ofthese counts depended neither on the reference periodlength nor on the retention interval: The slope found byregressing Iog(estimates) on Iog(recorded occurrences)normalized by the ratio of the standard deviations of thesevariables is the Pearson correlation. The comparison ofperformance in different conditions was carried out onz-transformed correlations,

Effects of retention interval and reference periodlength –Table K shows the mean correlation, over sub-jects, for each of the conditions defined by referenceperiod length and retention interval. The means are allpositive and reasonably high, indicating that the relation-ships between the logarithms of the estimates and thecounts of recorded occurrences are reasonably linear. Aspredicted, for subjects who reported about frequencies ofintake during a 2-week period, the correlation decreasedas the length of the retention interval increased. Forsubjects who reported about 4-week periods, however, thecorrelation was highest for those who reported immedi-ately after the end of the reference period and was lower,but roughly constant, for subjects who reported afterlonger retention intervals, A contrast m-ithe eight meansthat tested this pattern was significant (F(1,120) = 28.16,p c 0.0001); the residuaI was not significant (F(1,120) =1.04,J? > 0,25),

This pattern is consistent with the results of experi-ment 2, Consider first the performance of subjects whosereference period was 4 weeks. When the test was admin-istered immediately after the end of the reference period,available memories about the specific dietary experiencesof the reference period evidently enhanced the accuracyof frequency estimates, but when the test was delayed,performance did not depend on the length of the reten-tion interval, Dietary intake may be sufficiently similarfrom one 4-week period to another that estimates for any4-week period, based on general kn~wledge of one’s owndiet, are reasonably accurate, Performance of subjectswhose reference period was 2 weeks decreased systemati-cally as the retention interval was lengthened. Suchperiods are sufficiently short that specific experiences of

Table K. Correlations of Iog(frequenoy estimates) withIog(number of recorded occurrences) for subjects in experiment4, by reference period length and retention interval

Reference period length

Retent/on /nterva/ 2 weeks 4 weeks Mean

Oweek . . . . . . . . . . . . . . . 0.86 0.84 0.652weeks, . ., . .,,..,... 0.79 0.78 0.774weeks, . . . . . . . . . . . . . 0.70 0.77 0.746weeks .,.........,.. 0.68 0.77 0.73Mean. . . . . . . . . . . . . . . . 0.75 0.78 0.77

NOTE Number of subJectsper condillon was as follows (from left to right and top to bottom):11,9, 17, 17, 19, 17, la, snd 20,

the target 2-week period must be retrieved for responsesto be reasonably accurate.

Among subjects who were tested immediately afterthe ends of their reference periods, the mean correlationof subjects who reported about 2-week periods was higherthan that of subjects who reported about 4-week periods.Available specific information certainly contributes toaccuracy. However, after that information is lost, genericmemory appears to be more effective for estimating fre-quencies for relatively longer, hence more representative,periods than arbitrary shorter ones.

Experiment 5: Effects of recallusing closed-ended diaries

Experiment 4 showed that people’s food frequencyestimates depend, in interpretable ways, on the lengths ofthe reference period and the retention interval. Consistentwith previous experimental research on memory for fre-quency, the accuracy of food frequency estimates deterio-rated as the length of the retention interval increased, andthe decline was more pronounced for judgments about a2-week period than for judgments about a 4-week period(13,26),

Experiment 4 might be criticized on several method-ological grounds. First, the standard against which eachsubject’s frequency estimates were scored was compiled byan experimenter with whom the subject might have dis-agreed about the classification of recorded items. Discrep-ancies between the subject’s estimates and the countsfrom the diaries may have been due, at least in part, tosuch disagreements. Second, all of the subjects in experi-ment 4 completed the frequency questionnaires afterattempting to recall their dietary intake for their referenceperiod (see experiment 2), Engaging in the recall proce-dure may have influenced the frequency judgments sys-tematically. Third, correlation coefficients are not themost desirable index of memory for frequency. A prefer-able measure of relative frequency is the slope of the lineestimated by regressing estimates on counts of recordedoccurrences. The unequal variances over conditions inrecorded occurrences and estimates precluded the anal-ysis of slopes in experiment 4. Fourth, because eachsubject in experiment 4 estimated the frequency of aunique set of items, the data could not be subjected to abetween-subjects analysis for individual items.

Experiment 5 addressed all of these problems. Allsubjects kept a checklist diary about a closed set of dietaryitems for 1 month and then returned for a frequency test.Because the items on the diary form and the items on thequestionnaire were the same, the compilation of thestandard against which estimates were scored did notdepend on the intervening judgment of another person.To assess whether a prior recall experience influencesfrequency estimates, half of the subjects engaged in therecall procedure prior to making frequency judgments.Because all subjects kept diaries for the same amount oftime, the variances of the record counts and the estimates

17

were approximately equal for the subjects in differentconditions, so slopes could be analyzed. In addition,because judgments were about a closed set of items, abetween-subjects analysis of the estimates could beconducted.

Method

Subjects – Thirty-one subjects participated. They wererecruited through newspaper and television advertise-ments without regard to their demographic characteristics.

Design and procedure- Each subject was given asupply of postcards, on each of which a list of 58 fooditems or groups of items had been printed. Subjects wereasked to use a new postcard each day to record eachoccasion on which they ate any of the food items on thelist. Each day, the previous day’s postcard was to bemailed back to the investigators.

Each subject was scheduled to return to the labora-tory for a followup interview 29 days after the originalvisit. Prior to the second laborato~ session, half of thesubjects were assigned randomly to the recall condition.When these subjects arrived for their followup sessions,they were asked to report each item that they had eaten ordrunk during the preceding 4 weeks. All subjects com-pleted a questionnaire that asked them to indicate howoften during the diarykeeping period they had eaten eachof the items listed on the postcards. The 58 items orgroups of items appeared in a different random order oneach questionnaire.

Results and discussion

Effect of recall– For each subject, estimated foodfrequencies were regressed on the sum of recorded occur-rences for each item, and the slope was obtained. Becauseall subjects had kept diaries for the same period of time,no transformation was applied to the counts, Whethersubjects attempted to recall their intake during the diary-keeping period prior to making the frequency judgmentsdid not affect the frequency estimates. The mean slopes ofthe regression lines for the recall and no-recall conditionswere 1.01 and 0.90, respectively these did not differsignificantly (t(29) = 1.22, p > 0.25).

WWzin-wbjects mudysis– On average, the relationshipbetween estimated and recorded frequencies was fit wellby a linear function, and estimates increased as a functionof recorded occurrences by about one occasion perrecorded occasion. Over all subjects, the mean slope ofthe regression functions was 0.96 and the mean correla-tion was 0.86, which is quite similar to the value observedin experiment 4. These data suggest that relative fre-quency estimation is excellent.

Reasonably accurate estimates of frequency atextremes of recorded frequency might have inflated theseestimates. Subjects tended to indicate correctly the itemsthat they had not eaten at all during the reference period;this, together with high estimates for a few frequentlyconsumed items, could improve measured performance,

18

Therefore, additional regression analyses were conductedin which the range of recorded frequencies that wasanalyzed was iteratively narrowed. When attention wasrestricted to items with recorded frequencies of 1 through10, the mean correlation was 0.62 and the mean slope was1.01. Although the average fit of these functions was worsethan was the fit over the entire set of data for each subject,the slopes remained stable at around 1. This suggests thatthe frequency estimates of individual subjects are reason-ably accurate, Individuals give larger estimates for theitems that they have eaten more often than they do foritems that they have eaten less often, and their estimatesreflect a sensitivity to the range of frequencies with whichthey ate those items.

Between-subjects anaZysis– Epidemiologists tend to bemore concerned with whether the estimates of people whoate a particular item with different frequencies reflect, atleast ordinally, those different frequencies. For example, ifone individual eats potatoes more frequently than does asecond, will the former’s frequency estimate be greaterthan that of the latter? If frequency judgments wereperfectly accurate, then, of course, both within-subjectsanalyses (over items), reported in the last section, andbetsveen-subjects analyses (for each item), reported in thissection, would yield the same results.3

For each item, the estimates of the 31 subjects wereregressed on their counts. Over items, the average corre-lation was 0.43 and the average slope of the regressionfunctions was 0.56. These between-subjects analysespresent a bleaker view of performance than do the within-subjects analyses. The discrepancy between the within-subjects analysis and the between-subjects analysisindicates that although the judgments of individuals tendto be internally consistent, people are not sufficientlycalibrated with each other for their frequency estimates toadequately serve the purposes for which epidemiologistscollect them. The development of techniques to stand-ardize respondents with each other could improve thisstate of affairs.

Experiment 6: Question-inducedbiasing of frequency judgments

Experiments 4 and 5 indicate that respondents canprovide orderly numerical estimates of food frequency.These experiments also demonstrate that such judgmentsare sufficiently inaccurate that they may not be adequatefor their intended purposes. Experiment 4 showed that thequality of estimates for a specified period deterioratesover time. The results were generally consistent with theinformal model proposed in the introduction, but theyrevealed little else about the processes that subservefrequency judgments.

31fa subject were perfectly accurate in this task, regression of estimatedfrequencies on actual frequencies would yield a correlation of 1, a slopeof 1, and an estimated-frequency intercept of O.

If dietary experiences are represented in a memorynetwork such that the different episodes of eating anyfood item are interconnected, then a search of that net-work for representations of instances of eating an itemshould lead to the retrieval of relevant instances. A carefulsearch of the memory representation for instances ofeating a food should result in higher frequency estimatesthan judgments made without having engaged in such asearch. If subjects fail to engage in a thorough search,perhaps by engaging in some mental activity that competeswith a search, frequency estimates should be lower thanwhen such a search is completed.

In experiment 6, each subject was asked to estimatehow many times he or she had eaten just one food itemduring a specified reference period, Prior to answering thefrequency question, some subjects were instructed toengage in mental activity that was hypothesized to pro-mote or prevent a search of memory for instances relevantto the frequency question.

Method

Subjects- Responses were collected from 417 intro-ductory psychology students who received course creditfor participating in a session in which they completed avariety of research instruments for several investigators,

Desi~ and procedure – Six frequency questions weregenerated by crossing three different food items with tworeference periods (see table L). Each subject answeredone such question. Certain subjects were instructed toengage in a specific mental activity prior to answering thefrequency question. One-third of the subjects were askedto think and make notes about all the occasions on whichthey ate their target food item; another third of thesubjects were asked to think and make notes about themost recent occasion on which they ate the target food.The all-occasions instruction was intended to promoteretrieval from memory of instances relevant to the fre-quency question; the recent-occasion instruction wasexpected to prevent such retrieval by exaggerating thecognitive salience of that occasion. The remaining third ofthe subjects were given no special instructions.

Results and discussion

Because the three food items were eaten with vastlydifferent frequencies and because people reported havingeaten the foods with different frequencies during the two

Table L. Geometric means of frequency estimates by subjects inexperiment 6, by cognitive context, food item, and referenceperiod iength

Cognitive contextFood item and

reference period Control Recent occasion All occasions

Apples Number

l month . . . . . . . . . . . . . . 5.5 4.8 8.0I year . . . . . . . . . . . . . . . 33.9 27.5 28.8

Chicken

I month . . . . . . . . . . . . . . 14.8 8.1 14.5I year . . . . . . . . . . . . . . . 70.8 57.5 61.7

Pizza

I month . . . . . . . . . . . . . . 9.8 7.1 11.7I year . . . . . . . . . . . . . . . 61.7 52.5 67.6

NOTE: Number of observations per condition was as follows (reading from left to right and topto bottom): 24, 19, 15; 21, 20, 21; 22, 22, 23; 21, 19, 19:23, 23, 20; and 22, 23, 24.

reference periods, subjects’ responses were logarithmicallytransformed. The geometric mean responses for threefood items in the two reference periods are shown intable L.

For each food item for both reference periods, themean frequency estimate by subjects who had been givenall-occasions instructions exceeded that of subjects whohad been given recent-occasion instructions. The meansfor these instructional conditions were 22,1 and 15.4,respectively; these differed significantly, F(1,401) = 3.80,J1 <0.052.

These results show that frequency estimates are sen-sitive to the cognitive activity in which individuals engageprior to making the estimate. Moreover, they support thenotion that memories of the various episodes of eating aparticular food item are linked sufficiently that, withretrieval effort, episodes that are not immediately acces-sible can be retrieved, In this study, responses were notvalidated against any sort of external record, so theaccuracy of the responses is indeterminate. If additionalresearch were to show that subjects in the all-occasionscondition responded more accurately than did those in therecent-occasion condition, it would suggest that such amanipulation of cognitive activity could be used toimprove the accuracy of survey responses. (For a fullerdiscussion of this experiment, including more detailedanalyses of subsets of the data, see Smith, Jobe, andMingay (36).)

19

Portion size estimates

To quantify the dietary intake of individuals, the thirdtype of information about dietary intake that isrequired —after the identity of the foods that are eatenand the frequency with which they are eaten —is portionsize.

Both food frequency questionnaires and dietary recallprocedures sometimes involve asking respondents forinformation about their typical portion sizes. Obviously,an individual’s amount of intake depends on both fre-quency and portion size: A person who eats % cup ofspinach daily eats the same amount of spinach as a personwho eats 1 cup every other day. If these individualsreported accurately the frequency with which they eatspinach, an analysis that took only frequency informationinto account would characterize the former individual aseating twice as much spinach as the latter. If people canreport portion size information accurately, estimates ofintake will clearly be improved.

Two methods are used to collect most data aboutportion sizes. Commonly, when dietary data are collectedduring an interview by a nutritionist, respondents areasked to indicate which of a set of food models mostclosely resembles their portion of each item. On foodfrequency questionnaires, or when data collection time isscarce, respondents are asked to describe their typicalportion of each food item as “large,” “medium,” or“small,” given some quantitative definition of “medium.”For example, a respondent might be asked whether his orher typical serving of coffee is “large,” “medium,” or“small,” given that “medium” is a 6-ounce cup. Respond-ents are clearly expected to know what quantities aresignified by these definitions, It is not unusual for foodfrequency questionnaires to provide different standardsfor seemingly similar foods. For example, on the CancerSupplement Questionnaire to the 1987 National HealthInterview Survey (37), a medium serving of beans wasdefined as V2 cup, whereas a medium serving of carrotswas defined as 3/4 cup. The experiments in this sectionwere designed to investigate what people know aboutportion size and some of the determinants of the state-ments that they make about portion sizes.

The two experiments described in this section investi-gated different aspects of portion size reports, The first(experiment 7) was concerned with the types of memoriesthat are presumably required to accurately report portionsizes using food models: Subjects were asked to provide a

numerical description of displayed food portions eitherrelative to portions that had just been presented or rela-tive to portions that had been presented sometime earlier.The dependent measure of interest was the variability ofthe ratios of the responses to the displayed and referenceportions, The second (experiment 8) investigated themeaningfulness to respondents of the definitions of por-tion sizes that are presented on food frequency question-naires. A medium portion was defined differently fordifferent subjects, and distributions of portion sizeresponses were analyzed to assess whether subjects weresensitive to this manipulation.

Experiment 7: Estimation of sizesof food portions

Subjects in this experiment were shown photographedportions of 11 different food items and were to assignnumbers to describe the sizes of the portions. Eachdisplayed serving was to be judged relative to the previ-ously displayed portion of that food. Subjects wereinstructed to respond so that the ratio of the responses tosuccessive exemplars of each food item would equal theratio of the two portion sizes. For example, if a displayedserving of scrambled eggs appeared twice as large as thepreviously seen serving of scrambled eggs, the numericalresponse was to be twice as large.

Two experimental conditions differed in the locationof the reference stimulus (the previously displayed servingof the food item displayed on a particular trial) in thestimulus sequence, In the blocked condition, sequences ofslides of a single type of food item were presented, so thateach judgment of subjects in the blocked condition was tobe relative to the stimulus and response of the precedingtrial. In the mixed condition, the same slides were pre-sented, but the slides of the different types of food itemswere intermixed. Therefore, each judgment of subjects inthe mixed condition was to be relative to a stimulus andresponse on a trial that had occurred, on average, 11 trialsearlier,

Because subjects were instructed to respond ac-cording to the ratio of the size of a displayed stimulus tothe size of a specific reference stimulus, the ratios ofresponses to these two stimuli can be calculated, and theirconsistency can be evaluated. If intervening trials interferewith memory for size, the response ratios should be more

20

variable for subjects in the mixed condition than forsubjects in the blocked condition, because only subjects inthe mixed condition experienced intervening trialsbetween the displayed and reference stimuli.

Method

Subjects– Twenty-two university undergraduates, par-ticipating to partially fulfill a requirement of their intro-ductory psychology course, sewed in the experiment.Eight subjects were assigned to the blocked condition, andthe remaining 14 were assigned to the mixed condition.Subjects participated in small groups; all subjects in anygroup served in the same experimental condition.

Stimuli-The stimuli were color slides of servings offood, Eleven different-sized samples of each of 11 dif-ferent food items were used, for a total of 121 differentstimuli. For each food item, portion sizes increased inequal arithmetic steps from the smallest to the largest, Forexample, for orange juice, the smallest portion was2 ounces, and the serving sizes increased in l-ounce stepsto the largest portion, which was 12 ounces. For each fooditem, the median portion was designated as the standard(SJ

Design– Each of the 121 stimuli was presented fourtimes, so each subject responded to 484 stimulus presen-tations. In the stimulus sequence for the blocked condi-tion, the 44 exemplars ‘of each food item were arrangedconsecutively, so that each stimulus followed anotherstimulus of its own type, The appropriate standard wasshown before the first trial of each food-item trial block.The stimulus sequence for the mixed condition was con-structed by interleaving the 11 item sequences for theblocked condition, subject to the constraint that oneexemplar of each food item be included in each group of11 slides, All 11 standards were shown prior to the firstblock of trials,

In each condition, the 11 blocks of 44 slides weretreated as three sets of three blocks and one set of twoblocks, and these four units were presented in differentorders to different groups of subjects,

Procedure– The subjects in each experimental condi-tion were told that they would view slides that showedportions of various food items and that the portions wouldvary in size, The subjects were instructed to respond, oneach trial, with a number that reflected the ratio of thesize of the stimulus to the size of the preceding stimulus ofthe same item, The role of the standards was explained.Subjects were told that their first judgments for each fooditem should be relative to the standard portion for thatitem and that the value 100 was to be assigned to thestandards.

Results and discussion

Data editingand aggregation– Each subject’s data wereedited to exclude highly deviant responses. First, thestandard deviations of the logarithms of responses were

calculated for each size level of each food item, and thesestandard deviations were averaged. Then, at each sizelevel of each food item, if the most deviant responsediffered from the mean of the remaining responses bymore than four average standard deviations, it wasremoved from the data. Approximately 2 percent of theresponses were removed in this way. The mean numbersof responses removed for subjects in the blocked andmixed conditions (11.63 and 10.64, respectively) did notdiffer significantly, t(20) c 1.

Judgments of portion size– Figure 1 shows, for eachstimulus-sequence condition, geometric mean responsesas a function of the size of the displayed portions(expressed as log(S/SO)). The response functions from thetwo experimental conditions are essentially superimposed.In each condition, numerical responses increased linearlyand monotonically with the sizes of the displayed portions;the linear component of the stimulus size effect was theonly significant effect in these data, F(1,20) = 1683,04,p <0.0001,

Response ratios– To calculate the coefficient of varia-tion of response ratios, the trials were first sorted by theratio of the size of the displayed stimulus to the size of thereference stimulus for that trial. (In the blocked condition,the reference stimulus was always the stimulus that hadbeen shown on the preceding trial; in the mixed condition,the reference stimulus was the previously presented exem-plar of the displayed food item.) For each trial, the

20

t

,, ~-8 -8 -4 -2 0 2 4

Size of displayed stimulus (10 logS/S.)

NOTE Geometric mean responses are shown as a function of the logarithm of the ratioC4the SIZe of the displayed SUMUIUS to the size of the standard stimulus. Oata areaverages across feed items and acre= subjects In the blocked and mixed condtlons.

Figure 1. Mean magnitude estimate, by size of displayed stimulus

21

ratio of responses to the displayed and reference stimuliwas calculated. Then, foreach class of trials defined byaratio of stimulus sizes, the mean and standard deviation ofthe response ratios were calculated; the ratio of thestandard deviation to the mean is the coefficient of varia-tion of response ratios! For example, for every trial onwhich the stimuIus was twice as large as that trial’sreference stimulus, the ratio of the response on that trialto the response to the reference stimulus was found. Thestandard deviation of these ratios, divided by their mean,is the coefficient of variation for this value of stimulus sizeratio.

Figure 2 shows the coefficient of variation of theresponse ratios of the subjects in each stimulus-sequencecondition as a function of the difference in sizes betweenthe displayed and reference stimuli (expressed as10g(S/S..f), where the subscript denotes the referencestimulus). For example, zero on the abscissa indicates thatthe sizes of the target and reference stimuli were thesame, positive values indicate that the displayed stimuluswas larger than the reference stimulus, and negativevalues indicate that the displayed stimulus was smallerthan the reference stimulus. For each subject, trials fordifferent food items were pooled to calculate the coeffi-cient of variation function, and these were averaged acrosssubjects in each condition,

For subjects in the blocked condition,, the shape of thecoefficient of variation function is a pronounced V, with aminimum value for stimuli that were of the same size asthe reference stimulus. For subjects in the mixed condi-tion, the minimum coefficient of variation of responseratios was also at stimulus-size ratios of around 1, but thedip is far less pronounced than for the blocked condition.The difference between the conditions in the depth of theV was confirmed by an analysis of variance that showedthat the interaction of sequential condition (blockedversus mixed) and a quadratic contrast on log stimulusratios was significant, F(1,20) = 5.10, p C 0.05.

When the size difference between the displayed stim-ulus and the reference stimulus was small, the variabilityof the response ratios of subjects who had to remember

4The coefficient of variation is a normalized measure of variability, the

standard deviation divided by the mean, that is studied when thevariability of a measure is proportional to its mean. Consider, forexample, the weights, in kilograms, of elephants and cats: The standarddeviation of the weights of a sample of elephants will be larger than thatfor a sample of cats simply because the weight of an average elephant isthousands of kilograms, whereas the weight of the average cat is around5 kilograms. The weights of elephants can deviate from their mean muchmore than the weights of cats can deviate from theirs. For the responseratios described in the main text, when a large stimulus is judged relativeto a small one, the ratio will be large, and the variability of severalreplications of this judgment will be large because, in absolute values,there is greater potential for variation than when values are small. Whena small stimulus is judged relative to a large one, the ratios are small andtheir variability is small. The coefficient of variation makes the variabil-ities of these ratios comparable.

U- Blocked

-10.0 0.0 10.0Scaled size difference of displayed and reference stimuli

NOTE: The coefficient of varlstlon of the ratio of responsas to the d!splayad andrefereflUI 5tlIWll Is shown as a function of the logarithm of the fatro d fjI@ SIZ=Se( the displayed and reference stimuli. Dafa are averages across feed items and&xoss subJects In the blocked and mixed conditions.

Figure 2. Coefficient of variation of response ratios, by scaledsize difference of displayed and reference stimuli

back only one trial was considerably less than was thevariability of the response ratios of subjects who had toremember back further. When the size difference ofsuccessive exemplars of a particular food was large, thenormalized variability of response ratios did not dependon whether the reference stimulus for the trial had justbeen presented or had been presented some number oftrials earlier, For extreme size ratios of target to referencestimuli, subjects may be unable to use the ratio responserule and may judge the size of the target stimulus abso-lutely (38,39).

Although estimates of size averaged over many obser-vations did not depend on the sequential arrangement ofthe judged stimuli (see figure 1), memory-dependent rel-ative judgments of size are least variable when the refer-ence stimulus has just been presented and is ofapproximately the correct size. This suggests that the bestestimates of size, using food models, would be obtainedimmediately after the judged food portion has beenviewed, and this does not occur in practice. Most inter-views take place hours or days after the portions aboutwhich the respondents report have been viewed. Theresults of this experiment suggest that food-model-basedportion size estimates for individuals are likely to beinaccurate, although averages of estimates for groups ofindividuals may be quite reasonable.

22

Experiment 8: Comprehension andutilization of defined portion sizes

Many dieta~ questionnaires ask respondents to quan-tify their typical portion sizes for various foods by indi-cating whether their portions are “small,” “medium,” or“large,” with “medium” given a particular quantitativedefinition, For example, respondents might be askedabout their typical portions of coffee, with “medium”defined as 6 ounces, or about their typical portions ofspinach, with “medium” defined as V2 cup, This method ofportion size assessment tends to be used on self-completed questionnaires and on fast-paced generalhealth questionnaires that might be administered as partof a household survey.

To respond accurately to such questions, a respondentmust have precise internal representations of the sizes ofhis or her typical portions (which, of course, might bequite variable) and of the quantity that is defined as“medium.” A respondent who has specific quantitativeknowledge of his or her portion sizes may respond accu-rately to such questions, Lacking such knowledge, orkicking knowledge of the sizes of standard portions, thistask might be quite difficult, Nevertheless, respondentswill likely respond to such questions anyway.

This experiment explored the ability of people toprovide meaningful reports about portion sizes with thisassessment method. The specific question addressed waswhether people are sensitive to the defined portion sizes.Subjects were asked to indicate whether their typicalportion sizes for selected foods were “small,” “medium,”or “large,” Over subjects, the quantitative definition of theresponse “medium” was varied. For some subjects,“medium” was defined as a relatively small portion; forothers, as a relatively large portion; and for the remainder,as an intermediate-sized portion. If subjects have stableknowledge of their portion sizes and stable representa-tions of the quantities in terms of which “medium” isdefined, then the distribution of responses over the threealternatives should depend systematically on the defini-tions. Subjects for whom “medium” was defined as rela-tively small should tend to respond “large” more oftenthan subjects who received larger definitions of medium;subjects for whom “medium” was defined as relativelylarge should tend tosubjects who received

Method

respond “small” more often thansmaller definitions of medium.

Each of 414 introductory psychology students com-pleted a food frequency and portion size questionnaire;each subject responded to four items sampled from amongeight used in the experiment. For each food item, mediumwas defined, over questionnaires, by three different quan-tities; for every item, the largest definition of medium wasat least twice that of the smallest. Table M lists the fooditems and shows the ratio of the largest to the smallestdefinition of medium for each item.

For each of four questions, the subject was asked tomake a frequency judgment and to indicate whether his orher typical portion was “small,” “medium,” or “large.”The quantitative definition of “medium” appeared inparentheses adjacent to that alternative. Among the fouritems presented to each subject was at least one with a lowdefinition of “medium,” one with an intermediate defini-tion of “medium,” and one with a high definition of“medium.” Over 200 responses were collected for eachfood item, with approximately 70 subjects receiving eachdefinition of “medium.”

Results and discussion

Table M shows the distribution of responses over thethree response alternatives for each food item. To eval-uate whether there was an orderly shift over the defini-tions of “medium” in the distributions of responses, riditanalysis was used (40). This technique permits comparisonamong different samples of distributions of an ordered setof responses, In this application, each sample consisted ofthe subjects who read a particular definition of “medium”for a given item. For each condition of each food item, amean ridit was calculated to summarize the distribution ofresponses in a single condition relative to the distributionof responses pooled over conditions, The condition meanridit is an estimate of the probability that a randomresponse from that condition is larger than a responsesampled at random from the distribution of responses

Table M. Response distributions of reported portion size inexperiment 8, by food item and definition of medium

Number of responsesDefinition Mean

Food (large: small) of medium Small Medium Large ridit 2

Sliced cheese . . .(2:1)

Cream cheese . . .(4:1)

French fries. . . . .(2:1)

Fruit juice. . . . . .(2:1)

Green salad . . . .(2:1)

Ice cream . . . . . .(2:1)

Pasta . . . . . . . . .(2:1)

Salty snacks . . .(4:1)

LowMiddle

High

LowMiddle

High

LowMiddle

High

LowMiddle

High

LowMiddle

High

LowMiddle

High

LowMiddle

High

LowMiddle

High

242335

202418

138

18

5

1:

1078

181019

977

6

915

353626

352828

374738

283942

394140

314241

334045

39

3731

132

16

2023

6

151612

3323

20

202421

1719

8

27

24

18

27

2317

0.5333 2.010.46940.4933

0.5236 3.320.51710.4416

0.5046 3.460.53690.4571

0.5690 110.560.51310.4289

0.4824 0.690.51800.4988

0.4995 18.47

0.55510.4429

0.5209 1.40

0.50790.4714

0.5413 24.890.50770.4444

I p < 0.05.

20.05< fJ<0.10.

23

pooled across conditions. Table M shows the conditionmean ridits for each food item.

If people are sensitive to the presented definitions ofportion sizes, then, assuming the average portion sizes ofthe subjects assigned to each condition are equal, as thedefinition of “medium” is increased, more “small”responses should be observed and condition mean riditsshould decrease, Table M shows that for only four of theitems (cream cheese, fruit juice, pasta, and salty snacks)did condition mean ridits decrease in this fashion, and foronly juice were the differences among the condition meanridits significant. Although the condition mean ridits forice cream differed overall, pairwise comparisons showedthat the mean ridits of the low- and high-definition condi-tions did not differ significantly.

These results suggest that, in general, either peoplelack stable representations of the quantities of food thatthey eat, or the defined portion sizes are not meaningfulto them, or both. For half of the items used in thisexperiment, response distributions over the conditions didnot conform to the normatively expected pattern –that asthe defined size of a medium portion is increased, thedistribution of responses shifts toward “small.” For theother four items, condition mean ridits decreased as thedefined size of medium increased, but for three of these,the shift in the response distributions was insufficient toyield a statistically significant result. Inspection of

table M shows that this is due, at least in part, to a strongtendency for these subjects to respond “medium,” regard-less of the quantitative definition ascribed to “medium.”

The one food item for which condition mean riditsdecreased systematically and significantly was fruit juice.Ounces, particularly in the numbers used–4, 6, and8–may correspond to natural cognitive units of juice.People often drink containers of juice whose size, inounces, they know. In contrast, people do not typically eatgreen salad from containers whose size, in cups, they know(although this may change if the 1990’sbecome the age ofthe fast-food salad).

Response alternatives on surveys in general and ondietary surveys in particular must give respondents anappropriate means to communicate their knowledge aboutwhatever the data collector wants to know. This will likelybe best achieved if the format of response alternativescorresponds to that of the respondents’ knowledge.Dietary surveys ask respondents to describe food fre-quency and portion sizes rather than to describe averagedaily intake of specific micrormtrients because it isassumed, probably correctly, that most individuals have noidea what their average daily intakes of those micronutri-ents are. Respondents may not know what a cup of greensalad is, either. The meaningfulness of response optionsmust be evaluated before they are used to collect datafrom respondents.

24

Conclusions

Overview

Dietary data are collected from individuals in order toidentify and investigate relationships between nutritionalintake and health status. Nutritional epidemiologists areaware that’ dietary reports contain some degree ofresponse error, but, nevertheless, they use data collectedwith a variety of methods, including recall and frequencyprocedures (2,1 1). Any response error in survey datareduces the power of analyses of those data, Such loss ofpower may result in the failure to detect relationships thatare present in the studied population or necessitate incur-ring additional expenses to collect data on larger samplesizes (41),

The research program described in this report wasmotivated by the supposition that an understanding ofthe mental representations and processes that underliedietary reporting can improve the design of dietary surveymethods. Such knowledge could inform the developmentof methods to obtain better data than can be collectedwith current methods and could improve the under-standing of the data collected with current methods.Toward this end, several aspects of dietary reportingperformance were studied under conditions that weremanipulated to reveal the functioning of the mental rep-resentations and processes assumed to be responsible forthose reports.

Three types of information are essential to quantifyingdietary intake – what people eat, how often they eat whatthey eat, and the quantities in which they eat what theyeat, Each of these aspects of dieta~ intake maybe studiedsomewhat independently. The experiments described inthis report examined dietary recall performance, foodfrequency estimation, and judgments of portion size.

Experiments 1–3 examined dietary recall under var-ious acquisition, retention, and retrieval conditions. Dif-fcrtmces in reporting performance among experimentalconditions were used to make inferences about the memo-rial representation of dietary information. Unconstrainedreports of dietary intake for reference periods of 1 week to1 month deviated markedly from records of intake thatwere maintained during those periods: Experimental sub-jects both failed to report items that they recorded, andreported substantial numbers of items that they did notrecord, As the amount of time between the end of thereference period and the recall test was increased,

reporting accuracy decreased, with performance ap-pearing to level off after a retention interval of 4 weeks,

Taken together, the results of experiments 1–3encourage the notion that reports of dietary intake are, toa considerable extent, reports of generic knowledge of dietrather than reports of memories of specific experiences(28). Generic knowledge of diet is hypothesized to beorganized information about the routine elements of anindividual’s diet. Recall of dietary intake for extendedperiods appears to be a task that people carry out quiteinefficiently. It is possible that, with appropriate cues, theaccuracy of dietary recall might be improved, although thiswould be at a considerable cost in time. On the otherhand, the information provided by subjects, although inac-curate relative to the intake recorded for a specific period,may characterize what respondents typically eat.

Experiments 4-6 addressed the determinants of accu-racy of people’s reports about the frequency with whichthey eat various foods. In general, subjects provided rea-sonable estimates of the relative frequency with whichthey ate various foods. The judgments of any subject weresuch that estimates for more frequently eaten items werehigher than estimates for less frequently eaten items. Infact, the data of experiment 5 showed that, on average,estimates increased by about one occurrence per recordedoccurrence, indicating that subjects were highly sensitiveto variations in experienced frequency. However, of pri-mary interest in epidemiological research are between-subjects analyses of frequency estimates —that is, analysesof whether the responses of different individuals who eatsome food with different frequencies are at least ordinallyconsistent with the true frequencies. Despite the excellentwithin-subjects frequency estimates observed in experi-ment 5, there is sufficient variability over people in theaccuracy of responses that analyses of frequency judg-ments for items over subjects were less indicative ofaccurate judgments. Nevertheless, the within-subjects con-sistency of the judgments encourages the idea that, ifrespondents could be calibrated to a common standard,between-subjects consistency might be improved,

Several other qualifications concerning frequency esti-mates must be made. Experiment 4 showed that theaccuracy of frequency judgments deteriorates over timeand that relative judgments for longer, presumably morerepresentative, periods of time are more stable than arethose for shorter periods of time. Experiment 6 showed

25

that frequency judgments were sensitive to such external Frequency questionnairesmanipulations as instructions to engage in mental activitythat would either promote or preclude the retrieval ofinformation relevant to a frequency judgment.

The results of the experiments on portion size judg-ments suggest that people do not have high-resolutionlong-term memories for size information, nor do they codein memory the sizes in standard units of their typicalportions of various foods. In experiment 7, in whichsubjects were asked to respond numerically to the sizes ofdisplayed portions, the relative variability of responseratios was low only when the reference stimulus for thejudgment had just been presented and was close in size tothe target stimulus. In the absence of these two condi-tions, as would typicallybe the case in nutritional surveys,the relative variabili~ of response ratios was high, Inexperiment 8, for only one of eight studied food itemswere response distributions sensitive to changes in thequantitative definition of a standard portion.

Implications

What do these results imply for the collection ofdietary survey data for epidemiological purposes? Thispenultimate section is organized around several of thetechniques used in nutritional epidemiology for the collec-tion of such data.

Dietary recall

The results of experiments 1 and 2 suggest that thereports collected in dietary recall procedures for extendedreference periods do not accurately characterize dietaryintake during those periods. Many items are omitted, andmany of the reported items are intrusions. The results ofexperiment 1 suggest that the data collected using 24-hourrecall procedures may be reasonably accurate reportsabout intake on the preceding day, but, of course, oneday’s intake is unlikely to be representative of the diet ofan individual. (Because the procedure used in experiment1 did not permit the calculation of a daily intrusion rate,only this weak statement about the accuracy of datacollected using 24-hour recall can be made.)

The results of experiments 1 and 2 suggest that takingdiet histories– reports about the routine elements of anindividual’s diet —may be a reasonable way to collectinformation. If what is needed for research purposes is aset of items that are typical of the individual’s diet ratherthan a specific list of items eaten during a specific periodof time, the diet history maybe adequate to collect such alist. In such a case, the respondent would be instructed toreport the contents of his or her generic dietary knowl-edge, rather than to attempt to search memory for specificdietary memories. People may be quite able to report whatthey typically eat, even if they cannot report exactly whatthey ate during some specific period of time.

Food frequency questionnaires are used to collectinformation of two types —frequency information andcategorical portion size information. The results of exper-iments 4 and 5 show that people report relative frequencyinformation with reasonable accuracy, although the accu-racy of their estimates —especially for short referenceperiods–declines as the amount of time since the end ofthe reference period increases. Unfortunately, the within-subjects consistency in frequency estimates observed inexperiment 5 does not carry over to the analyses of items(over subjects) that are used commonly in epidemiology.This discrepancy is probably due to intersubject discrep-ancies in calibration for frequency judgments. It would bedesirable to develop ways to align the frequency estimatesof different respondents to food frequency questionnaires.

The results of experiment 6 showed that people’sfrequency estimates may be influenced by the thoughts inwhich they engage prior to making those estimates.Although it would be unusual for a respondent to a foodfrequency questionnaire to be asked to think about themost recent time that he or she had eaten an item prior tojudging the frequency with which he or she eats that item,the results of experiment 6 raise the general concern thatresponses to food frequency questionnaires might be influ-enced by the sequential arrangement of questions. Forexample, a question about one food might lead subjects,on a subsequent question about some other food, toconsider only a restricted range of occasions on which thatfood is eaten. This problem, if it is a problem, could becircumvented by randomizing questions over forms of thequestionnaire.

The results of experiment 8 raise concerns about thecategorical portion size information collected on question-naires, Subjects in experiment 8 displayed a general insen-sitivity to variations in the defined sizes of food portions.This indicates that portion size questions of the type usedin the 1987 National Health Interview Survey (37) maybemeaningless to respondents.

Food models

In dietary interviews, food models are used to collectportion size information. After naming a food item that heor she has eaten, the respondent is asked to indicate whichof a set of models matches the size of the consumedportion. The results of experiment 7 suggest that precisememories for portion sizes do not survive exposure tointervening items. Food models can likely be used tocollect only very crude portion size information.

Final comment

Perhaps the most surprising result of the experimentson recall and frequency estimation was that, from apsychological perspective, the results were not surprising.

26

Despite the differences between the conditions under of these experiments on memory for dietary informationwhich individuals acquire information about their dietary are generally consistent with the results of relevant labo-experiences and those under which subjects acquire infor- ratory research on memory, By virtue of this consistency,mation in laborato~ experiments, and despite differences these results strengthen those principles.in the nature of the to-be-remembered events, the results

27

References

1.

2,

3.

4.

5.

6.

7.

8,

9.

10.

11.

12,

13.

14.

15.

16.

17.

28

Surgeon General. The Surgeon General’s report on nutri-tion and health. DHHS No. 88-50210. Washington: U.S.Government Printing Office, 1988.

Willett WC. Nutritional epidemiology. New York: Wiley.1990.

Block G, A review of validations of dietary assessmentmethods. Am J Epidemiol 115:492-505.1982,

Krall EA, Dwyer J, Coleman KA. Factors influencing accu-racy of dietary recall. Nutri Res 8:829-41.1988.

Coughlin SS. Recall bias in epidemiologic studies. J ClinEpidemiol 43:87-91.1990.

Harlow SD, Linet MS. Agreement between questionnairedata and medical records. Am J Epidemiol 129:233-48.1989.

Lippman A, Mackenzie SG. What is “recall bias” and doesit exist? In: Marois M, ed. Prevention of physical and mentalcongenital defects: Part C: Basic and medical science, edu-cation, and future strategies, New York: Alan R, Liss,205-9.1985.

Mackenzie SG, Lippman A, An investigation of report biasin a case-control study of pregnancy outcome. Am J Epide-miol 129:65-75.1989,

Raphael K. Recall bias: A proposal for assessment andcontrol, Int J Epidemiol 16:167-70, 1987,

Marr JW. Individual dieta~ surveys: Purposes and methods.World Rev Nutr Diet 13:105-64.1971.

Young CM. Dietary methodology. In: Committee on FoodConsumption Patterns, National Research Council,Assessing changing food consumption patterns. Wash-ington: National Academy Press, 89-118.1981.

Hankin JH, Rhoads GG, Glober GA. A dietary method foran epidemiologic study of gastrointestinal cancer. Am J ClinNutr 28:1055-61.1975.

Willett WC, Sampson L, Stampfer MI, et al. Reproducibilityand validity of a semiquantitative food frequency question-naire. Am J Epidemiol 122:51-65.1985.

Smith AF, Jobe JB, Mingay DJ. Concerning the suitabilityof diarykeeping to validate recall of dietary information.Manuscript submitted for publication. 1990.

Bradburn NM, Rips LJ, Shevell SK. Answering autobio-graphical questions: The impact of memory and inferenceon surveys. Science 236:157-61, 1987.

Fienberg SE, Loftus EF, Tanur JM. Cognitive aspects ofhealth survey methodology: An overview. Milbank MemFund Q 63:547-64.1985. ‘-

Fienberg SE, Loftus EF, Tanur JM.health surveys for public informationMem Fund Q 63:598-614.1985.

Cognitive aspects ofand policy. Milbank

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32,

33.

34.

Jabine TB, Straf ML, Tanur JM, Tourangeau R, eds.Cognitive aspects of survey methodology: Building a bridgebetween the disciplines, Washington: National AcademyPress, 1984.Jobe JB, Mingay DJ. Cognition and survey measurement:History and overview. Appl Cognit Psychol 5:175-92.1991,Tourangeau R. Cognitive science and survey methods, In:Jabine TB, Straf ML, Tanur JM, Tourangeau R, eds. Cogni-tive aspects of survey methodology Building a bridgebehveen disciplines. Washington: National Academy Press,73-100.1984.

Crowder RG. Principles of learning and memory, Hillsdale,New Jersey: Lawrence Erlbaum. 1976.Hasher L, Zacks RT. Automatic processing of fundamentalinformation: The case of frequency of occurrence, AmPsychol 39:1372-88.1984.Hintzman DL. Judgments of frequency and recognitionmemory in a multiple-trace memory model, Psychol Rev95:528-51.1988.

Anderson JR, Bower GH. Recognition and retrieval proc-esses in free recall. Psychol Rev 79:97-123.1972,Cofer CN. Does conceptual organization influence theamount retained in immediate free recall? In: KleinmuntzB, ed. Concepts and the structure of memo~. New York:Wiley, 181-214.1967.Underwood BJ, Zimmerman J, Freund JS, Retention offrequency information with observations on recognition andrecall. J Exp Psychol 87:149-62.1971.Rabinowitz JC, Mandler G, Patterson KE. Determinants ofrecognition and recall: Accessibility and generation. J ExpPsychol Gen 106:302-29.1977.

Smith AF, Jobe JB, Mingay DJ. Retrieval from memory ofdietary information. Appl Cognit Psychol 5:269-96.1991.Cohen BH. Recall of categorized word lists. J ExpPsychol 66:227-34.1963.

Deese J. Influence of inter-item associative strength uponimmediate free recall. Psychol Rep 5:305-12.1959.Graesser AC. Prose comprehension beyond the word. NewYork: Springer-Verlag. 1981.

Jonides J, Naveh-Benjamin M, Estimating frequency ofoccurrence. J Exp Psychol (Learn Mem Cogn) 13:230-40.1987.

Liechtenstein S, Slovic P, Fischoff B, et al. Judged frequencyof lethal events. J Exp Psychol (Hum Learn Mere) 4:551-78.1978,

Naveh-Benjamin M, Jonides J. On the automaticity offrequency coding: Effects of competing task load, encodingstrategy, intention. J Exp Psychol (Learn Mem Cogn)12:378-86.1986,

35.

36,

37,

3s,

Blair E, Burton S. Cognitive processes used by surveyrespondents to answer behavioral frequency questions. JConsumer Res 14:280-8.1987.

Smith AF, Jobe JB, Mingay DJ. Question-induced cognitivebiases in reports of dietary intake by college men andwomen. Health Psychol 10:244-51.1991.

Schoenborn CA, Marano M. Current estimates from theNational Health Interview Survey. National Center forHealth Statistics. Vital Health Stat 10(166). 1988.

Green DM, Lute RD, Smith AF. Individual magnitudeestimates for various distributions of signal intensity. Per-cept Psychophys 27:483-8. 1980.

39. Lute RD, Baird JC, Green DM, Smith AF. Two classes ofmodels for magnitude estimation. J Math Psychol22:121-48.1980.

40. Fleiss JL. The design and analysis of clinical experiments.New York: Wiley, 1986.

41. Freudenheim JL, Johnson NE, Wardrop RL. Nutrientmisclassification: Bias in the odds ratio and loss of power inthe Mantel test for trend. Int J Epidemiol 18:232-8.1989.

42. Green DM, Swets JA. Signal detection theory and psycho-physics. Huntington, New York Robert E. Krieger. 1974.

43. Hilgard ER. Methods and procedure in the study oflearning. In: Stevens SS, ed. Handbook of experimentalpsychology. New York: Wiley, 517-67.1951.

29

Appendixes

Contents

I. Indirect scoring of a simulated memory experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31IL Indexes ofreporting performance . . . . . . . , . . . . . . . . . . . . . . . . . . . , . ! . . , . . . . . . . . , . . , . . . . . . . . . . . . . . . . . ., ., , . 32

The indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,. .,,,,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Choice of the indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

III. Detailed results of experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Effect of retention interval and reference period length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Effect ofreporting instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Appendix tables

I. Results ofindirect scoring ofsimulated memory experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31II. Summa~of counts required tocalculate perfomance indexes ... ,,... . . . . . . . . ,, ..,,.. ... .,.,, ,,, ..,,, ., 32

30

Appendix IIndirect scoring of asimulated memoryexperiment

This appendix illustrates the argument that indirectresponse ~ajidity assessment methods, which transformboth recorded and reported food items to mean dailynutrient intakes, might lead to the false conclusion thatreports are highly accurate.

An analog to indirect assessment of response accu-racy, in a psychological experiment on memory for words,might involve decomposing both the to-be-rememberedand the reported words into their component letters and,for each letter, calculating over subjects the correlationbetween the counts of to-be-remembered and reportedletters. In this example, the 26 letters, the components ofwords, are considered to be analogous to the nutrientscontained in foods. If memory performance were evalu-ated with such a set of correlations, a subject’s erroneousreport of Mzkissippi for stresses would not affect thecorrelation calculated for the s’s, but would affect thecorrelations for the other letters. The erroneous report ofan anagram of a to-be-remembered word (for example,clea/z for kmce) would not be reflected as an error. Thenotion of nutritional anagrams, or near-anagrams, couldeasily be developed.

To evaluate the conjecture that the correlations mightsuggest good performance even in cases in which othermeasures would suggest otherwise, a set of memory exper-iments was simulated and scored as follows: Each simula-tion assumed 40 subjects, to each of whom a uniquememory set of 50 items, sampled from a 596-item pool,was presented. Simulated subjects reported a specifiednumber of responses from this pool, with specified matchand intrusion rates, The match rate is the reported pro-portion of to-be-reported items; the intrusion rate is theproportion of reported items that were not in the memory

set. (These measures are discussed in detail in appen-dix II.) Memory and response sets were transformed tocounts of letters, and, for each of the 26 letters, countsfrom the reports were regressed on counts from thememory items and the correlation between the countsover subjects was calculated.

Table I shows, for values of the three performanceparameters (number of items reported, match rate, andintrusion rate), the mean of the slopes and the interceptsof the 26 regression functions and how many of the 26correlations were significant at the 5-percent level. Thetop line shows the results for perfect performance; thebottom line shows the results for subjects who report 25items, all of which are intrusions. The key results are onthe intermediate lines, which show that many significantcorrelations between the counts of components of thememory and response sets may be observed even ifmemory performance is quite poor. For example, the thirdline of the table shows that when match rates are only 0.5and intrusion rates are 0.5, 23 significant correlations wereobserved,

Table L Results of indirect scoring of simulated memoryexperiments

PJ(report) P(mtc) P(int) Slope Intercept Correlations

50 1.0 0.0 1.00 0.00 2650 0.8 0.2 0.79 4.28 2650 0.5 0.5 0.47 9.91 2325 0.5 0.0 0.52 0.16 2625 0,0 1.0 -0.09 11.32 0

NOTE P(mtc) Is match rata; p(int) is intrusion ratq “Correlations” lathe number of correlations(out of 26) significant at the 5-percent level.

31

Appendix IIIndexes of reportingperformance

This appendix clarifies the relation between the twoindexes of performance that are described in this reportand explains why they were chosen over certain possiblealternatives. This discussion is adapted from the appendixof Smith et al. (28).

The indexes

For experiments 1 and 2, subjects recorded items indiaries and, subsequently, were asked to report as many ofthose items as they could. Subjects reported items thathad not been recorded as well as items that had beenrecorded. After preliminary editing of both the diaries andthe report protocols, the reported items were divided intotwo sets according to whether they matched items in thediary, and the diary items were divided into two setsaccording to whether they were matched by items in thereport,

Let D be the number of items in the diary, R be thenumber of items in the report, and m be the number ofitems that are matches between the diary and the report.Also, for convenience, let the number of intrusions be i =R – m, and let the number of omissions be o = D -m. Thetwo indexes of performance are defined as

p(match) = m/D (Al)and

p(intrusions) = 1- (m/R). (A2)

Note that the first of these measures is a fraction of thenumber of items recorded, and the second is a fraction ofthe number of items reported. By definition, the relation-ship between the two indexes is

p(match) = ~1 -p(intrusions) D (A3)

Restrictions on values

In principle, the only restriction on the possible valuesof p(intrusions) is that if p(match) is greater than O,p(intrusions) must be less than 1, because there will be atleast one match. Otherwise, for anyp(match) greater thanO, p(intrusions) can range from O, if all of the reporteditems are matches, to arbitrarily close to 1, if i (thenumber of intrusions) is sufficiently close to R for i/R to belarge. Note particularly that for any D, p(match) does notdepend on i, the number of intrusions, (If p(match) is

equal to Oand R is greater than or equal to 1—that is, if atleast one item is reported – then p(intrusions) must beequal to 1.)

By way of illustration, let the set of items reported byan individual be labeled x(l), x(2), . . . x(R). If these arearranged as if the individual reported first his or hermatches and then his or her intrusions, so that x(l), . . .x(m) are the matches and x(m + 1), . . . x(R) are theintrusions, then, because D is fixed, p(match) = m/D, andthis value would have been determined once the subjecthad reported the mth item. Then, as the subject reportedhis or her intrusions, items x(m + 1) through x(R), onlyp(intrusions) would have changed, growing from O beforethe subject reported the (m + l)st item to (R - m)/R whenhe or she reported the Rth item. Although p(match) andp(intrusions) are related, as described by equation A3, therelationship is through both D and R.

Empirical relationship

P(match) and p(intrusions) are generally unrelated.For example, over all of the subjects of experiment 2, thecorrelation between these indexes was –0.16, which,although statistically significant at p <0.05, is quite small.The four extreme points in the p(match) xp(intrusions)scatterplot were (0.167,0.125), (0.186,0.797), (0.673,0.132),and (0.683,0.423), which shows that at the extreme levelsof p(match), the range of p(intrusions) was reasonablylarge. In the data of experiment 2, the correlation of thesemeasures in the eight experimental conditions defined bycombinations of reference period length and retentioninterval ranged from -0.37 to 0.48, with a mean of 0.02. Inonly one condition was the correlation statisticallysignificant.

Choice of the indexes

These indexes are not ideal. It would be preferable tohave measures corresponding to the hit and false alarmrates of signal detection theory that would permit thecalculation of analogs of the sensitivity and response biasparameters of signal detection theory (42).

The counts that contribute to the calculation of theseindexes are summarized in table IL Examination of thattable clarifies why measures analogous to those of signaldetection theory cannot be computed. Although p(match)corresponds to signal detection theory’s hit rate, to

32

Table Il. Summary of counts required to calculate performanceindexes

/terns reported

items recorded Yes No Sum

Yes . . . . . . . . . . . . . . . . . mNo . . . . . . . . . . . . . . . . . (: (:

Sum . . . . . . . . . . . . . . . . ; (1) (1)

1Count Is neither known nor estimable.NOTE m represents matches, tha count of reported items that match recorded itams; /

represent intrusions, the count of reporled items which were not recorded: o represents

omissions, the count of recorded items which were not raported; Rrepreeents the count of

reported Itemq D represents the count of recorded items.

calculate a false alarm rate requires that i be divided bythe sum of entries in the second row of the table, the valueof which is neither known nor estimable. False alarm ratescan be computed in recognition memory experimentsbecause test items either were or were not presented tothe subject during the acquisition phase of the experiment,but in free recall situations, although the items presentedduring the acquisition phase are known, there is noreasonable way to define the number of items that wereneither recorded nor reported,

The weakness of the measures used in this report isthat a match between a reported item and a recorded itemcannot, by itself, be taken as evidence that the subjectdeliberately retrieved and reported a target. In general, inany situation in which an individual makes a choiceresponse, a correct response is not necessarily evidencethat the subject knew which response to make, and sosome sort of correction for guessing is frequently applied.In other words, in such situations, the observed set ofcorrect responses is assumed to be a mixture of genuinelycorrect responses and guesses that turned out to becorrect. Thus, the meaningfulness of p(match) as a mea-sure of memory performance is not clear: p(match) issurely an overestimate of the remembered fraction of thetarget list, but the extent to which it is an overestimate isnot known.

Each of the indexes used in this report describes aseemingly important aspect of performance. Gordon B.Willis, in personal communication during June 1988, sug-gested that p(match) could be considered a measure. ofsufficiency– the extent to which the reported itemsachieve the goal of completely matching the recordeditems —and that p(intrusions) could be considered ameasure of inefficiency – the extent to which the reportprotocol was cluttered with items that do not contributetoward the goal of matching the recorded items, Clearly,performance is considered superior and is presumed to bebased to a greater extent on retrieval from memory of thetarget items when a report protocol is both highly suffi-cient and efficient than when it is highly inefficient,regardless of its sufficiency.

The question, then, is whether there is a reasonableway to consolidate these aspects of performance into asingle measure. Graesser (31), following Hilgard (43),suggested calculating a memory score defined as

Memory score =p(match) –p(intrusions)

1 – p(intrusions)

Willis independently suggested that another

Memory score .Z?@_m+o

(A4)

measure,

= [D p (match)] - [R p (intrusions)]

D=(2. WZ)-R

(A5)

Dbe used to characterize the memory performance of thesubjects in the experiments discussed in this report. Eachof these measures is rooted in the same underlying philos-ophy— to offset the count of matches of reported items torecorded items by subtracting information about thenumber of items reported, which is assumed to reflect theprobability of matching by chance.

The problem with these measures, and the reason thatthey are not used, is that they make very strong assump-tions about how intrusions should offset nominally correctresponses. For example, the measure described inequation A4 is one of a family of measures that can bewritten as

Memory score =p(match) -[k .p(intrusions)] (A6)

1- [k. p(intrusions)]

with k having some value between O and 1 that describeshow the intrusion rate is charged against the match rate inorder to produce a compromise score. In equation A4,k = 1.In some situations (for example, choice response),the appropriate value of k is obvious, and in others,reasonable arguments can be made for a particular valueof k. For example, Graesser (31) reported experiments inwhich two versions of stories were presented to differentgroups of subjects, and intrusion rates were calculated bycounting as intrusions only items reported by one groupthat had been presented to the other; items that werereported but that had not been presented to the othergroup were ignored. Thus, Graesser based his calculationson a closed set of items; because he knew how many itemshad not been presented, he could actually have calculatedsignal detection measures rather than the memory scorethat he reported. In contrast, the sets of items analyzed inthe free recall experiments described in this report wereopen sets of items. In the absence of any knowledge abouthow to charge intrusions against nominally correct reports,that is, about how to speci~ a value fork in equation A6,it seems preferable to present and interpret the twomeasures separately. The measure defined in equation A4was calculated for the data of experiment 2, and both thepattern of that measure and inferences based upon it wereconsistent with the reported interpretation of the twoindexes that were used,

33

Appendix IllDetailed results ofexperiment 2

This appendix presents, in more detail and with moreformality than the main text, the results of experiment 2.

Effect of retention interval andreference period length

Table C shows the mean number of items recorded bysubjects in each of the experimental conditions defined byretention interval and reference period length. Subjectswho kept diaries for 4 weeks recorded significantly moredifferent items than did those who kept diaries for 2weeks, F(1,162) = 112.52, p c 0.0001. The length of theretention interval did not affect the number of itemsrecorded, F(3,162) <1. Table C shows that the meannumber of different items recorded during a period offixed length was quite stable over the groups of subjectsassigned to each reference period length condition. Therewas no interaction of reference period length and reten-tion interval on the number of items reported,F(3,162) c 1.

Table D shows the mean match and intrusion ratesfor subjects in the eight experimental conditions definedby retention interval and reference period length. Consist-ent with the results of experiment 1, dietary reportingperformance declined as the length of the retentioninterval increased, Consistent with other data on memorydecay, the change over time in the match and intrusionrates was negatively accelerated, so all tests of the effect ofretention interval length used a contrast with a negativelyaccelerated “shape” (for example, the weights applied tothe 4- and 6-week levels of retention interval were equaland, in sum, opposite to that applied to the O-week level).

A multivariate analysis of variance (MANOVA) ofthe match and intrusion rates showed that the contrast onthe levels of retention interval was significant, F(2,161) =46,82, p < 0.0001; the residual was not significant. Thepattern of means of each of the performance indexes waswell described by this contrast: For match rate,F(1,162) = 78,40, and for intrusion rate, F(1,162) =15.19, p ‘c0.0001.

The MANOVA showed a significant main effect ofreference period length (F(2,161) = 4.28, p < 0.02) and asignificant interaction of the reference period length andthe contrast on retention intervals (F(2,161) = 4.00, p c0.05). Univariate analyses showed these effects to beattributable to variations in intrusion rates. Although the

average match rates of subjects assigned to the tworeference periods did not differ (F(l ,162) < 1), the intru-sion rates of subjects who reported for a 2-week periodwere significantly higher than were those of subjects whoreported for 4 weeks, (F(1,162) = 8.58, p c 0,01),and theinteraction of the reference period length and the contraston retention intervals was significant (F(1,162) = 4.50,p c 0.05). Table D shows that the absolute mean intru-sion rate of subjects who kept diaries for 2 weeksincreased steeply over retention intervals, whereas that ofsubjects who kept diaries for 4 weeks was roughlyconstant.

Effect of reporting instructions

Table E shows the number of items recorded andreported by subjects who received each of the three sets ofinstructions for each reference period length. Instructionsdid not influence the number of items recorded,F(2,146) <1. However, subjects who received food groupcues reported significantly more items than did those whoreceived reverse chronological or meal cues, F(1,146) =10.97,p e O.000~ the mean numbers of items reported bysubjects in the latter two conditions were not statisticallydifferent, F(1,146) < 1.

Table F shows the mean match and intrusion ratesfor subjects in the three instructional conditions. AMANOVA conducted on these measures showed anoverall performance difference between subjects whoreceived food group cues and those who received the twoother types of instructions (F(2,145) = 7.61, p c 0.001),but showed no difference in the performance between themeal and reverse chronological conditions (F(2,145) =1.40, p > 0.25). Univariate analyses showed that the dif-ferences among instruction conditions were due to differ-ences in intrusion rates. Subjects who received food groupcues exhibited significantly higher intrusion rates thansubjects who received the two other types of cues,F(1,145) = 15.05, p c 0,001. The intrusion rates of sub-jects in the two other conditions did not differ signifi-cantly, F(1,145) c 1. Match rates did not differsignificantly over the three conditions: For food groupversus the other two conditions, F(1,145) < 1; for mealversus reverse chronological, F(1,145) = 2.30, p >0,1.None of the interactions of instructions with other exper-imental conditions was significant.

34‘i? U.S. GOVERNMENTPRINTINGOFFICE 1991– 281- 79 8/ 400 1 1

- --- —.- ....— —. A. — .-. —.—

Vital and Health Statisticsseries descriptions

SERIES 1,

SERIES 2,

SERIES 3,

SERIES 4.

SERIES 5.

SERIES 6.

SERIES 10.

SERIES 11,

SERIES

SERIES

2.

3,

SERIES 14,

Programs and Collection Procedures–Reports

describing the general programs of the National Center for

Health Statistics and its offices and divisions and the data

collection methods used. They also include definitions and

other matetial necessary for understanding the data.

Data Evaluation and Methods Research – Studies of new

statistical methodology including experimental tests of new

survey methods, studies of vital statmtics collection

methods, new analytical techniques, objective evaluations

of reliability of collected data, and contributions to statistical

theory, Studies also include comparison of U.S.

methodology with those of other countries.

Analytical and Epidemiological Studies – Reports

presenting analytical or interpretive studies based on vitai

and health statistics, carrying the analysis further than the

expository types of reports in the other series.

Documents and Committee Reporls – Finai reports of

major committees concerned with vital and health statistics

and documents such as recommended model vital

registration laws and revised birth and death certificates.

Comparative International Vital and Health Statistics

Reports –Analytical and descriptive reports comparing

US. vital and health statistics with those of other countries.

Cognition and Survey Measurement – Reports from the

National Laboratory for Collaborative Research in Cognition

and Survey Measurement using methods of cognitive

science to design, evaluate, and test survey instruments,

Data From the National Heaith Interview Survey–

Statistics on illness, accidental injuries, disability, use of

hospitai, medical, dental, and other services, and other

health-related topics, all based on data collected in the

continuing national household interview survey,

Data From the National Health Examination Survey and

the National Heaith and Nutrition Examination Survey–

Data from direct examination, testing, and measurement of

national samples of the civilian noninstitutionalized

population provide the basis for (1) estimates of the

medically defined prevalence of spec!fic diseases in the

United States and the distributions of the population with

respect to physical, physiological, and psychological

characteristics and (2) analysis of relationships among the

various measurements without reference to an explicit finite

universe of persons.

Data From the Institutionalized Population Surveys –

Discontinued in 1975. Reports from these surveys are

included in Series 13,

Data on Health Resources Utilization– Statistics on the

utilization of health manpower and facilities providing long-

term care, ambulatory care, hospital care, and family

planning sewices.

Data on Health Resources: Manpower and Facilities –

Statistics on the numbers, geographic distribution, and

characteristics of health resources including physicians,

dentists, nurses, other health occupations, hospitals,

nursing homes, and outpatient facilities.

SERiES 15.

SERiES 16.

SERIES 20.

SERIES 21.

SERiES 22.

SERIES 23.

SERIES 24.

Data From Speciai Surveys– Statistics on health and

health-related topics collected in special surveys that are

not a pari of the continuing data systems of the National

Center for Health Statistics.

Compilations of Advance Data From Vital and Heaith

Statistics – These reporls provide early reiease of data

from the National Center for Health Statistics’ heaith and

demographic surveys, Many of these releases are followed

by detailed reports in the Vital and E@alth Statistics Series.

Data on Mortality –Various statistics on morlality other

than as included in regular annual or monthly reports.

Special analyses by cause of death, age, and other

demographic variables; geographic and time series

analyses; and statistics on characteristics of deaths not

available from the vital records based on sample surveys of

those records.

Data on Nataiity, Marriage, and Divorce –Various

statistics on natality, marriage, and divorce other than

those inciuded in regular annual or monthiy reports.

Special analyses by demographic variables; geographic

and time series analyses; studies of fertility; and statistics

on characteristics of births not avaiiabie from the vital

records based on sample surveys of those records.

Data From the Nationai Mortality and Nataiity Surveys –

Discontinued in 1975, Reports from these sample surveys

based on vital records are included in Series 20 and 21,

respectively.

Data From the National Survey of Family Growth –

Statistics on fertility, family formation and dissolution, famiiy

planning, and related maternai and infant health topics

derived from a periodic survey of a nationwide probability

sample of women 15–44 years of age.

Compilations of Data on Natality, Mortaiity, Marriage,

Divorce, and Induced Terminations of Pregnancy–

Advance reports of births, deaths, marriages, and divorces

are based on finai data from the National Vital Statistics

System and are published annually as supplements to the

Monthly Wal Statistics Report (MVSR). These reports are

followed by the publication of detailed data m Vital

Statistics of the United States annual volumes. Other

reports including induced terminations of pregnancy issued

periodically as supplements to the MVSR provide selected

findings based on data from the National Vital Statistics

System and may be followed by detailed reports in the Vital

and Health Statistics Series.

For answers to questions about this report or for a list of tities of reports

published in these series, contact:

Scientific and Technical Information Branch

National Center for Health Statistics

Centers for Disease Control

Public Health Service

6525 Belcrest Road, Room 1064

Hyattsville, Md. 20782

301-436-8500

DEPARTMENT OFHEALTH & HUMAN SERVICES

Public Health E&viceCenters for Dmase ControlNational Center for Health Statistics6525 Be[crest RoadHyattsville, Mayland 20782

OFFICIAL BUSINESSPENALTY FOR PRIVATE USE, $300

BULK RATEPOSTAGE & FEES PAID

PHWJCI-ISPERMIT’ FJO.G-2E11

DHHS Publication No. (PHS) 92-1079, Series 6, No. 4