Copyright 2005 Psychonomic Society Inc 676

Perception amp Psychophysics2005 67 (4) 676-690

The ability to represent identity and location informa-tion about objects in scenes requires the visual system tosolve a sort of perceptual puzzle Object information isextracted during fixations brief periods in which gaze isheld stationary over a scene but about three times eachsecond our high-resolution fovea are repositioned over ascene by rapid movements of gaze called saccades De-spite their obvious importance in accumulating visual in-formation these saccades also serve to fracture our vi-sual world and make impossible any simple binding ofobject identity to location An object viewed during onefixation will change its retinal location and many aspectsof its visual appearance following a saccade to anotherlocation in a scene The task of representing an object ina scene therefore requires mapping these changing reti-nal descriptions onto a more stable representation thatmaintains its continuity across movements of gaze (for areview see Bridgeman Van der Heijden amp Velichkovsky

1994) The fact that we experience a visual world in whichobjects appear not to jump and change appearance witheach eye movement attests to the skill of our visual systemin solving this puzzle

How can object representations in a scene remain sta-ble in the face of a constantly changing visual input Anearly theory attempting to answer this question took thepuzzle metaphor literally and hypothesized the existenceof an integrative visual buffer in which foveal snapshotsof individuated objects could be accumulated and arrangedin a scene-based reference frame (McConkie amp Rayner1976) Subsequent work however questioned this bufferconception of object representation by documenting clearcases of information not being integrated across saccades(Bridgeman amp Mayer 1983 Irwin Brown amp Sun 1988McConkie amp Zola 1979 Rayner amp Pollatsek 1983) Alsodifficult to reconcile with an integrative visual buffer is Ir-winrsquos (1991 1992) work on transsaccadic memory show-ing that the act of making an eye movement effectivelyerases the sensory representation of a previously fixateditem More recent extensions of this work have clarified theobject properties that can survive an eye movement andhave found these properties to be highly abstracted with re-gard to their visual detail (eg general color and shapeapproximate location categorical definition) not unlikethe properties often discussed in the context of visual short-term memory (STM Carlson- Radvansky 1999 Carlson-Radvansky amp Irwin 1995 Henderson 1997 Hendersonamp Siefert 2001 Irwin amp Gordon 1998)

To account for the survival of object properties acrosschanges in gaze Irwin and colleagues proposed transsac-cadic object file theory (OFT) a modification to tradi-

We thank David Irwin George McConkie Dan Levin and Christo-pher Dickinson for their insightful comments during various stages ofthis project and Gary Wolverton for his help in programming the gaze-contingent display paradigm GJZ was supported by NSF Grant ITR0082602 NIMH Grant R01-MH63748 and Army Research OfficeGrant DAAD19-03-1-0039 LCL and GW were supported by theArmy Federated Laboratory under Cooperative Agreement DAAL01-96-2-0003 Some of the equipment used in this study was provided by NSFInfrastructure Grant CDA 96-24396 A preliminary version of this workwas presented at the 1998 meeting of the Association for Research in Vi-sion and Ophthalmology Correspondence should be addressed to G JZelinsky Department of Psychology Psychology B 240 State Universityof New York Stony Brook NY 11794-2500 (e-mail gregoryzelinskysunysbedu)

Eye movements serialize memory forobjects in scenes

GREGORY J ZELINSKYState University of New York Stony Brook New York

and

LESTER C LOSCHKYKansas State University Manhattan Kansas

A gaze-contingent short-term memory paradigm was used to obtain forgetting functions for realisticobjects in scenes Experiment 1 had observers freely view nine-item scenes After observersrsquo gaze lefta predetermined target they could fixate from 1ndash7 intervening nontargets before the scene was re-placed by a spatial probe at the target location The task was then to select the target from four alter-natives A steep recency benefit was found over the 1ndash2 intervening object range that declined into anabove-chance prerecency asymptote over the remainder of the forgetting function In Experiment 2we used sequential presentation and variable delays to explore the contributions of decay and ex-trafoveal processes to these behaviors We conclude that memory for objects in scenes when serial-ized by fixation sequence shows recency and prerecency effects that are similar to isolated objects pre-sented sequentially over time We discuss these patterns in the context of the serial order memoryliterature and object file theory

MEMORY FOR OBJECTS IN SCENES 677

tional OFT that makes explicit the relationship betweenobject memory and eye movements (Irwin 1996 Irwinamp Andrews 1996) Like traditional OFT (Kahneman ampTreisman 1984 Kahneman Treisman amp Burkell 1983Kahneman Treisman amp Gibbs 1992 Treisman 1988)transsaccadic OFT states that object identityndashlocationbindings exist in the form of a limited number of spa-tially indexed property lists known as object files How-ever transsaccadic OFT adds to this framework the as-sumption that visual STM capacity can accommodateonly 3ndash4 object files across changes in eye position Ifan object in a scene corresponds to one of these 3ndash 4available object files the object properties associatedwith its file can be retrieved and used in a working mem-ory task

This assumption of transsaccadic OFT has importantimplications for the relationship between eye movementsand object representation in scenes Suppose a person isfaced with the task of remembering the objects in an un-familiar room A reasonable plan might be to move gazefrom object to object pausing long enough on each to en-code its identity and location into memory Assumingthat fixations on new objects are likely to be accompa-nied by the opening of an object file transsaccadic OFTpredicts good memory for the 3ndash4 objects visited mostrecently by gaze Of course objects viewed earlier in thesequence of fixations might also be remembered if someof their properties were encoded using a more enduringrepresentation but the existence of object file represen-tations should still result in a memory benefit for themost recently fixated objects

The recency benefit predicted by transsaccadic OFT isreminiscent of the recency effects reported for over acentury in the memory literature on serial position effects(see Baddeley 1986 Greene 1986 and Neath 1998 forhistorical treatments of this topic) Moreover there is atleast a surface similarity between the sequence of gazeshifts made while encoding the objects of a scene and theserial presentation conditions that are central to the ex-pression of serial memory phenomena Given these sim-ilarities one might expect the serial memory literature toinform the question of scene-based object representa-tion but the exchange of ideas between these two re-search communities has been minimal Slowing the flowof information are significant methodological differ-ences between these two classes of studies On the onehand scene-based object memory studies typically pres-ent objects simultaneously as part of a scene and then askobservers to make some judgment about one of the sceneobjects (for a review see Henderson amp Hollingworth1999a) On the other hand studies designed to investi-gate serial position effects on visual memory typicallypresent a series of individual patterns (eg histoformskaleidoscope patterns snowflakes) at a central displaylocation and then ask observers whether a test patternappeared in this study series (Broadbent amp Broadbent1981 Neath 1993 Phillips 1983 Wright 1998)1 Twomethodological concerns therefore prevent the gener-

alization of findings from the serial memory literature tothe question of object representation in scenes First noequivalency has been demonstrated between the sequen-tial presentation paradigm and the order in which objectsare viewed in a scene Are the objects fixated in a sceneserialized in memory in the same way as they would be ifthey were presented individually one after the other Sec-ond because serial memory studies typically present stim-uli at the same location in space these studies provide noinformation about the representation of objects scatteredover many locations in a scene In short many serial mem-ory studies lack a spatial memory component and there-fore cannot be used to study the identityndashlocation bindingthat is central to OFT and more generally the representa-tion of objects in scenes

The extensive literature on spatial working memorysuffers from the opposite problem with regard to inform-ing research on scene-based object memory In a typicalspatial memory study stimuli are presented serially at dif-ferent display locations and observers are tested in one ofthree ways (1) they are either asked to recall the loca-tions of all the items from the memory set without regard for order (eg Hale Myerson Rhee Weiss amp Abrams1996) (2) they are presented with a spatial probe andasked whether the probe location corresponds to one of theitem locations from the memory set (eg Awh Jonides ampReuter-Lorenz 1998 Courtney Petit Maisog Ungerleideramp Haxby 1998) or (3) they are asked to reproduce the en-tire spatiotemporal sequence in which the study itemswere presented (eg Jones Farrand Stuart amp Morris1995 Smyth amp Scholey 1996) Given this taxonomy ofspatial memory studies several methodological differ-ences again prevent the generalization of findings to thequestion of objectndashlocation binding in scenes First manyspatial memory studies are unconcerned with how thestimulus presentation order might affect onersquos spatialmemory ability and therefore do not include order ofstimulus presentation in their analyses (eg Awh et al1998 Courtney et al 1998 Hale et al 1996) Secondthe studies that do take presentation order into accountare primarily concerned with memory for temporal orderper se not the effect of presentation order on memoryfor object representations (eg Jones et al 1995 Smythamp Scholey 1996) Third in many spatial memory stud-ies object identity is irrelevant to the task Popular spa-tial memory paradigms such as the Corsi blocks task orthe 724 spatial recall test do not even manipulate objectidentity instead using simple dots or checker shapes asstimuli (eg Rao Hammeke McQuillen Kharti amp Lloyd1984 Shimozaki et al 2003) Thus the majority of exist-ing spatial memory studies while important shed littlelight on real-world scene representation

To our knowledge only Walker and colleagues (WalkerHitch Doyle amp Porter 1994 Walker Hitch amp Duroe1993) manipulated both the identity and location of ob-jects for the purpose of studying how serial presentationorder affects memory Walker et al (1993) presented asequence of four geometric patterns at four horizontally

678 ZELINSKY AND LOSCHKY

arranged display positions followed by the presentationof a test stimulus at a neutral display location The ob-serverrsquos task was to indicate the previous location of thistest pattern in the study array When plotted as a serialposition function the data from this study revealed apronounced recency effect that was largely limited to thelast item presented in the study sequence This finding isimportant because it demonstrates serial position effectson the basis of the conjunction of spatial and identity in-formation about an object thereby potentially informingthe scene representation literature However it is alsoimportant not to generalize too quickly from this findingto object memory in scenes The patterns used by Walkeret al (1993) were highly confusable nonsense shapeswhereas those appearing in scenes are typically real-world objects The serial position effects reported bythese authors may therefore be influenced by this choiceof stimuli Moreover the placement of patterns in theWalker et al (1993) study was highly predictable andlimited to only four locations in a horizontal array thespatial distribution of objects in scenes is far less con-strained Finally and most important to the present dis-cussion is the fact that Walker et al (1993) still used asequential presentation paradigm In the real world ob-servers typically shift their gaze from object to object asthey encode patterns into memory The relationship be-tween order of stimulus presentation and the eye move-ments to objects appearing in scenes remains a largelyunaddressed question in the serial memory community

Many studies in the visual cognition literature haveused eye movement measures to assess memory in mul-tiobject displays (Hayhoe Bensinger amp Ballard 1998Henderson amp Hollingworth 1999b Henderson Mc-Clure Pierce amp Schrock 1997 Henderson Pollatsek ampRayner 1987 1989 Henderson amp Siefert 1999 2001Hollingworth Schrock amp Henderson 2001 Holling-worth Williams amp Henderson 2001 Zelinsky 2001)but here too there have been few attempts to specifyhow memory is affected by the order of fixations on ob-jects Two recent studies have addressed this relationshipdirectly Irwin and Zelinsky (2002) allowed observers toview a seven-object display for either 1 3 5 9 or 15 fix-ations Following this gaze-dependent viewing period aspatial probe cued one of the object locations in the studydisplay and the observer was given a seven alternativeforced choice (AFC) recognition test for the cued targetWhen Irwin and Zelinsky analyzed their gaze data in termsof recency of object fixation (Figure 8 p 889) a clear re-lationship emerged between recognition accuracy andwhen the target was fixated during study When the targetwas fixated immediately prior to probe onset recognitionaccuracy was over 90 However when the target was fix-ated 1 or 2 fixations earlier accuracy dropped to about80 Accuracy dropped further to 65 for targets fixatedeven farther back in the eye movement sequence (3 4 or5 fixations prior to the spatial probe) but then remainedconstant at this above-chance level Irwin and Zelinskyinterpreted this recency benefit for objects viewed within

the last 3 fixations as support for transsaccadic OFT and itsassumption of a 3ndash4 object memory capacity

In another recent study Hollingworth and Henderson(2002) performed a similar post hoc analysis of their eyeposition data and reached a very different conclusionThese authors made type or token changes to objects inrealistic scenes as observers inspected these scenes inanticipation of a memory test Observers were also askedto indicate whether a scene change was noticed duringviewing Among their many interesting results Holling-worth and Henderson failed to find a reliable recency ef-fect in their data Unlike Irwin and Zelinsky (2002) de-tection accuracy in their task did not vary systematicallywith the number of fixations since the target was lastviewed However consistent with Irwin and Zelinsky(2002 cf Zelinsky amp Loschky 1998) Hollingworth andHenderson did find an above-chance prerecency level ofaccuracy extending several items back in the viewing se-quence which they interpreted as evidence for an LTMrepresentation of object properties

As evidenced by the discrepancy between Irwin andZelinsky (2002) and Hollingworth and Henderson (2002)the literature does not yet have a clear picture of how ser-ial fixation order affects memory for objects in scenesnor is it even clear what theoretical framework will bestdescribe these data patterns once they have stabilizedAlthough transsaccadic OFTrsquos prediction of a 3ndash4 objectrecency effect is well supported by Irwin and Zelinskyand much of the previous transsaccadic memory litera-ture (Irwin 1992 Irwin amp Andrews 1996 Irwin amp Gor-don 1998) this prediction is greater than the one-objectrecency effect often observed in studies using a sequen-tial presentation paradigm (Phillips amp Christie 1977a1977b Wright Santiago Sands Kendrick amp Cook1985 for reviews see Phillips 1983 and Wright 1998)and the zero-object recency effect reported by Holling-worth and Henderson The present study was conductedto address these discrepant estimates of recency in thescene-based object memory literature as well as to ex-plore factors potentially contributing to above-chancelevels of prerecency accuracy reported by Irwin andZelinsky and elsewhere (Hollingworth amp Henderson2002 Kerr Avons amp Ward 1999 Phillips 1983 Phillipsamp Christie 1977a Walker et al 1994 Walker et al1993 Wright 1998 Wright et al 1985) By outlining amethod of serializing the encoding of scene objects usingeye movements it was also our hope to build a bridge be-tween the scene-based object memory literature and themany studies using a sequential presentation paradigm todescribe serial order effects on object memory

EXPERIMENT 1

One possible reason for the discrepancy between thefindings of Irwin and Zelinsky (2002) and Hollingworthand Henderson (2002) may be that different methods ofquantifying serial position were used in the two studiesHollingworth and Henderson plotted percent accuracy

MEMORY FOR OBJECTS IN SCENES 679

for a target object as a function of the number of fixa-tions made after gaze left the target The forgetting func-tion derived by Irwin and Zelinsky (Figure 8 p 889) wassomewhat different plotting target accuracy as a func-tion of the number of objects inspected following fixa-tion on the target This distinction is important becausethe number of fixations made during scene viewing doesnot typically map perfectly onto the number of fixatedobjects Often multiple fixations are devoted to a singleobject and sometimes two neighboring objects are in-spected with only a single fixation (Zelinsky Rao Hay-hoe amp Ballard 1997) Although Irwin and Zelinsky(2002) and Hollingworth and Henderson (2002) bothplotted memory accuracy as a function of viewing orderit is possible that their forgetting functions were not cap-turing the same serial position information and conse-quently could not be directly compared

Experiment 1 describes a purely object-based forget-ting function for items viewed serially in simple scenesAs in the Irwin and Zelinsky (2002) study we monitorthe eye movements of observers as they freely inspect a

multiobject scene and we make the duration of this studyperiod contingent on their oculomotor behavior How-ever rather than terminating the study display after a setnumber of fixations we make testing contingent on thenumber of different objects fixated by observers aftertheir gaze leaves a predesignated target This online mon-itoring of the number of posttarget objects fixated by ob-servers allows us to parametrically manipulate the num-ber of objects viewed between study and test Rather thanrelying on post hoc fixation analyses to derive recencyand prerecency serial memory functions our manipula-tion of intervening objects as an independent variablemeans that our experimental design will have sufficientpower to discern the memory patterns in question We be-lieve that this more powerful design when combinedwith an object-based gaze-contingent methodology willprovide a clearer measure of object memory in scenes

MethodStimuli The stimuli consisted of nine common real-world objects

(toy tool or food items) arranged on an appropriate background sur-

Figure 1 Events comprising a typical trial in Experiment 1 (A) A nine-object scene was presented to observers Theirtask was to study these objects in anticipation of a memory test (B) Observers freely viewed the objects with the inten-tion of remembering their identities and locations eventually fixating the target object (in this case the baby bottle)(C) Unknown to the observer the display program would then count the number of objects fixated after gaze left thetarget (D) When the criterion number of different posttarget objects (in this case three) was fixated the multiobjectdisplay was replaced by a spatial probe at the targetrsquos location (E) The observer then had to select the target object fromamong four alternatives

A B

C

D E

680 ZELINSKY AND LOSCHKY

face (a crib workbench or dining table) Each simple scene sub-tended 18ordm 116ordm of visual angle and was of near-photographicquality (16-bit color)2 Individual objects were scaled to fit insidea 24ordm bounding box and their locations in the scene were con-strained to 18 positions creating what appeared to be a haphazardarrangement of items lying on a surface As a result of these place-ment constraints scene objects had a minimum and maximumcenter-to-center separation of 24ordm and 145ordm respectively Multipletrials for a given scene type were created by randomly pairing theobjects to locations (ie the same nine objects would occupy dif-ferent locations in each scene) with the constraints that no two dis-play configurations were repeated and that the memory target wouldappear equally often in each of the 18 allowable positions

Procedure The experimental procedure is shown in Figure 1 Ascene appeared at the start of a trial (Figure 1A) and the observersrsquotask was to remember the identity and location of every objectmdashinthis case toys in a babyrsquos crib (there were also tools on a workbenchand food items on a dining table) Given this formidable memorytask observers invariably made eye movements to the individualobjects a response that was anticipated and crucial to the currentparadigm but not explicitly mentioned in the experiment instruc-tions Eye position was monitored every millisecond and analyzedonline to determine the object in the scene being fixated by the ob-server Unknown to the observer one object in the display was pre-designated as the memory target for that particular trial (eg thebaby bottle in Figure 1) As the observer freely scanned the scenegaze would eventually be directed to this target object (Figure 1B)This fixation event was detected by the program controlling the ex-periment which then started to count the number of different ob-jects fixated after gaze left the target This ldquocountrdquo constituted thetermination criterion for the display meaning that if the count waspreset to three the observer would be allowed to fixate exactly threeobjects after the target (Figure 1C) As gaze moved away from thethird posttarget object the study scene was replaced by a 1-sec du-ration probe (a colored noise mask) appearing at the targetrsquos loca-tion on an ldquoemptiedrdquo background surface (Figure 1D) Followingthe probe a display showing four objects was presented and theobserver had to indicate which of them appeared at the probed lo-cation (Figure 1E)3 One of these objects was always the target theother three were randomly selected from the study scene Becauseobservers were on a bite-bar and could not easily speak they re-sponded by looking at the desired object causing a white box to bedrawn around the item then pressing a hand-held button when sat-isfied with their selection Observers were asked to respond as ac-curately as possible without regard for time

The independent variable of interest in this study was interven-ing objects the number of different objects looked at after the tar-get There were seven intervening object conditions (1ndash7) For ex-ample in the one-intervening-object condition the study displayterminated during the saccade away from the first posttarget object(ie gaze was not allowed to land on a second nontarget item afterleaving the target) Likewise in the seven-intervening-object con-dition the observer fixated exactly seven different nontarget ob-jects following target fixation The amount of time that observersviewed the study scene therefore varied it was determined by whenthey first fixated the target and the duration of their gaze on eachobject as well as the intervening object criterion set for that partic-ular trial The intervening object conditions were randomly inter-leaved throughout the experiment resulting in an unpredictablestudy scene duration Postexperiment questioning revealed that theparticipants were uniformly unaware that scene presentation timedepended on their own pattern of eye movements attributing thevariable interval instead to some random schedule of presentationdurations

Participants Six experimentally naive observers from the Uni-versity of Illinois at Urbana-Champaign each participated in 378trials 54 in each of the seven intervening object conditions Eye

position was monitored throughout each trial using a Generation Vdual-Purkinje-image (DPI) eyetracker sampling at 1000 Hz Thespatial precision of this tracker was better than 3 min of visual angleat a viewing distance of 62 cm Saccadic eye movements were de-termined online using a velocity-based algorithm implementing aroughly 125ordmsec detection threshold The experiment required two15-h sessions conducted on separate days and the participantswere paid $24 upon completion

Results and DiscussionOur gaze-contingent memory paradigm allows us to

plot response accuracy as a function of intervening ob-jects for trials in which the target was fixated only onceduring viewing (ie no target refixations) This analysisshown in Figure 2A by the solid markers reveals twoclear patterns of behavior First memory declined pre-cipitously between the 1 and 3 intervening object condi-tions dropping linearly from an 87 level of accuracy toonly 65 [F(630) 653 MSe 0044 p 001] Thispattern suggests that the act of fixating the first three ob-jects after the target dramatically interfered with mem-ory for the target objectmdashmuch like the interference ob-served when objects are presented one after another in asequential memory paradigm Second this drop in accu-racy reached an abrupt asymptote at three interveningobjects meaning that item fixations over the 4ndash7 interven-ing object range resulted in no additional object-relatedinterference Moreover because chance response in this4AFC task should produce a 25 level of accuracy the65 level described by this asymptote indicates a clearlyabove-chance and relatively good memory for an objectirrespective of the number of items fixated after it duringstudy4 We refer to the region of declining accuracy in thisintervening object function as the recency memory com-ponent and the above-chance asymptotic region of accu-racy as the prerecency memory component

Our clear evidence for distinct recency and prerecencybehavior although consistent with the data from Irwinand Zelinsky (2002) is inconsistent with Hollingworthand Hendersonrsquos (2002) failure to observe a recency ef-fect To determine whether this discrepancy is due to ourquantification of serial order in terms of intervening ob-jects rather than intervening fixations we reanalyzed ourdata using Hollingworth and Hendersonrsquos interveningfixation method (the open markers in Figure 2A) Thiscomparison revealed considerable agreement betweenthe two functions with mean differences in accuracyamounting to only 255 This high degree of similaritywould be expected if observers tended to devote only asingle fixation to each object during study More impor-tant accuracy again declined with the number of fixa-tions occurring after gaze left the target [F(630) 493MSe 7738 p 001] Note however that this recencyeffect is less pronounced than that in the intervening ob-ject data We attribute this pattern to a special role thatobjects play in producing recency behavior Differencesbetween the intervening object and intervening fixationfunctions are caused by the occasional multiple fixationof individual objects during study Given that these fix-

MEMORY FOR OBJECTS IN SCENES 681

ations on the same object were less detrimental to mem-ory than were fixations on different objects (as indicatedby the slightly better accuracy in the intervening fixationdata) it appears that objects not fixations are primarilyresponsible for the accuracy decline that defines recencyin our task Moreover if one were to replot such an object-based recency effect as a function of intervening fixa-tions the result should be a stretching of the object-basedrecency effect over more intervening events (ie fixa-tions) which is the exact pattern appearing in Figure 2AThis dilution of recency in the intervening fixation datasuggests one factor contributing to Hollingworth andHendersonrsquos (2002) failure to find a recency effect in theirstudy As recency is stretched over more of the interven-ing event function the slope becomes shallower and theeffect more difficult to observe For this reason we be-

lieve that the preferred method of quantifying recencyduring scene viewing is to serialize the data by objectsrather than by fixations

As a working interpretation of our data we hypothe-size that retroactive interference (RI) is a likely cause ofthe observed recency effect Consistent with a classic RIexplanation for STM forgetting (Waugh amp Norman 1965see also Intraub 1984 for RI applied to scenes) each ob-ject fixated after the target in our task might have inter-fered with memory for the target eventually driving recog-nition performance into the prerecency baseline Howeverinterference might be proactive as well as retroactive (Kep-pel amp Underwood 1962 Wickens Born amp Allen 1963)and given that a relatively small set of objects was beingreused in each trial of this experiment the potential forproactive interference (PI) influencing the data wouldseem high Previous research using a serial presentationparadigm has found that between-trial PI effects on pic-ture recognition memory are greatest early in an experi-ment rather than late (Jitsumori Wright amp Shyan 1989Wright 1998) particularly under conditions of immedi-ate testing (eg a 1-sec retention interval) It might thenbe the case that observers were able to accurately re-member the target over a large intervening object rangeearly in the experiment but as a result of PI accumulat-ing over trials this high level of accuracy gradually de-clined in the later trials to chance performance If theforgetting function was changing during the course ofthe experiment the recency and prerecency componentsin Figure 2A might be mere artifacts of averaging overtrials To explore this possibility we evenly divided the378 trials into ldquoearlyrdquo and ldquolaterdquo conditions If PI wasbuilding up throughout the experiment prerecency accu-racy in the early-trial condition should be higher than inthe late-trial condition However as can be seen from Fig-ure 3 this was clearly not the case If anything accuracyactually improved somewhat over the 3ndash7 intervening ob-ject range in the late trials Rather than suffering from abuildup of PI observers benefited slightly from theirgrowing familiarity with the set of stimulus objects

Although between-trial PI appears not to have playeda role in producing the Figure 2A intervening objectfunction there still exists the possibility that anotherform of PI might have influenced our data Given ourfree-viewing paradigm the number of nontarget objectsfixated before gaze first shifted to the target necessarilyvaried with the observersrsquo own idiosyncratic scanningpreferences If observers happened to inspect more pre-target objects in the 3ndash7 intervening object conditionsand if the fixation of these objects interfered with targetmemory the buildup of PI within a trial might result in thelower accuracy observed in the prerecency component

To evaluate this potential for within-trial PI we firsttested the premise that pretarget fixations were less fre-quent in the 1ndash2 intervening object conditions comparedwith the 3ndash7 intervening object conditions The resultsfrom this analysis are shown in Table 1 As expected onthe basis of our random selection of targets the number

Figure 2 Memory forgetting functions relating accuracy to thenumber of different events following viewing of the target Linesextending from the markers indicate one SEM (A) Data from Ex-periment 1 in which a single nine-object study scene was pre-sented on each trial and memory accuracy was defined in termsof either the number of intervening objects fixated between tar-get and test or the number of intervening fixations (B) Data fromExperiment 2 in which each trial consisted of nine one-objectstudy displays presented sequentially over time Square markersindicate data from a 1000ndash0 condition in which individual objectswere presented for 1 sec each without an interstimulus interval(ISI) Data from the 500ndash500 (500-msec presentation 500-msecISI) and 500ndash0 (500-msec presentation 0-msec ISI) conditions areshown by the triangle and circle markers respectively

682 ZELINSKY AND LOSCHKY

of distractor objects fixated before the target in a trialdid not meaningfully differ between the 1ndash2 and the 3ndash7intervening object conditions [Pearson χ 2 (8) 325p 918 N 1457] Our failure to find an overrepre-sentation of pretarget distractor fixations in the 3ndash7 in-tervening object trials therefore weakens the argumentthat within-trial PI could account for our recency dataWe also directly looked for a within-trial PI effect bycomparing trials in which the target was fixated veryearly in the sequence of fixations (with 0 1 or 2 objectsfixated before the target) with trials in which the targetwas first fixated later (three or more objects fixated be-fore the target) If within-trial PI played a role in deter-mining the recency effect accuracy should be higherwhen there were fewer pretarget fixations Table 2 indi-cates that this too was not the case Trials in which thetarget was fixated very early (ie few pretarget fixations)were no more accurate than trials in which the target wasfixated much later (ie several pretarget fixations) Wecan therefore conclude that pretarget fixations and what-ever within-trial PI they may have exerted cannot explainthe recency effects reported in Figure 2A

Unresolved QuestionsSince we dispatched PI as a factor shaping the Fig-

ure 2A forgetting functions an explanation appealing toRI becomes more attractive However two remaining al-

ternative explanations cannot be ruled out based on theExperiment 1 data First it is possible that the recency be-havior reported in Figure 2A reflects a decay processrather than RI from objects fixated after the target Decay-based explanations for STM forgetting enjoyed consider-able popularity in the early days of modern memory re-search (eg Peterson amp Peterson 1959 see also Altmannamp Gray 2002 for a recent formulation of this view) butwere largely replaced by interference-based explanationsafter several studies demonstrated little or no forgettingover time when item interference was controlled (Loess ampWaugh 1967 Waugh amp Norman 1965 Wickens et al1963) Although the latter studies certainly reduce theplausibility that decay was responsible for the currentlyobserved recency behavior this factor cannot be ruledout for the specific presentation and testing conditionsused in Experiment 1 A second factor potentially af-fecting the Figure 2A forgetting data is specific to the si-multaneous presentation paradigm Our use of eyemovements to serialize object processing assumes thatprocessing is restricted to only the currently fixated ob-ject during free viewing Several studies however havealso demonstrated processing benefits for extrafoveallyviewed objects (Henderson amp Anes 1994 PollatsekRayner amp Henderson 1990) If multiple objects werebeing processed during each display fixation this oppor-tunity for extrafoveal processing of the target might ex-plain the above-chance level of accuracy observed in theFigure 2A prerecency data Although such extrafovealprocessing benefits have been reported only for the in-tended target of a saccade and not every object in a dis-play (Henderson Pollatsek amp Rayner 1989) the poten-tial for a more generalized form of extrafoveal processingbenefit affecting our data should be considered

EXPERIMENT 2

Experiment 2 directly addressed the possibilities thatdecay or extrafoveal processes may have influenced theExperiment 1 data The question of whether decay or in-terference underlies the Figure 2A recency effect was re-solved by holding constant the number of interveningobjects appearing between study and test but varying theretention interval If decay was responsible for the ob-served decline in accuracy lengthening the retention in-terval should accelerate the accuracy decline and con-strict the expression of recency on the forgetting function

Table 1Percentage of Trials With a Given Number of Pretarget Object Fixations

(0ndash8) as a Function of Two Intervening Object Groups (1ndash2 vs 3ndash7)

Number of Pretarget Objects Fixated

Number of Intervening Objects 0 1 2 3 4 5 6 7 8

1ndash2 112 134 112 125 90 112 103 106 1063ndash7 103 113 124 120 102 104 109 106 120

NotemdashData exclude trials in which the target object was refixated n = 554 and 903cases for the 1ndash2 and 3ndash7 intervening object groupings respectively

Figure 3 Intervening object functions derived from an early-versus-late segregation of the Experiment 1 data Open markersshow data from the first 189 trials of the experiment closedmarkers show data from the final 189 trials

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

MEMORY FOR OBJECTS IN SCENES 677

tional OFT that makes explicit the relationship betweenobject memory and eye movements (Irwin 1996 Irwinamp Andrews 1996) Like traditional OFT (Kahneman ampTreisman 1984 Kahneman Treisman amp Burkell 1983Kahneman Treisman amp Gibbs 1992 Treisman 1988)transsaccadic OFT states that object identityndashlocationbindings exist in the form of a limited number of spa-tially indexed property lists known as object files How-ever transsaccadic OFT adds to this framework the as-sumption that visual STM capacity can accommodateonly 3ndash4 object files across changes in eye position Ifan object in a scene corresponds to one of these 3ndash 4available object files the object properties associatedwith its file can be retrieved and used in a working mem-ory task

This assumption of transsaccadic OFT has importantimplications for the relationship between eye movementsand object representation in scenes Suppose a person isfaced with the task of remembering the objects in an un-familiar room A reasonable plan might be to move gazefrom object to object pausing long enough on each to en-code its identity and location into memory Assumingthat fixations on new objects are likely to be accompa-nied by the opening of an object file transsaccadic OFTpredicts good memory for the 3ndash4 objects visited mostrecently by gaze Of course objects viewed earlier in thesequence of fixations might also be remembered if someof their properties were encoded using a more enduringrepresentation but the existence of object file represen-tations should still result in a memory benefit for themost recently fixated objects

The recency benefit predicted by transsaccadic OFT isreminiscent of the recency effects reported for over acentury in the memory literature on serial position effects(see Baddeley 1986 Greene 1986 and Neath 1998 forhistorical treatments of this topic) Moreover there is atleast a surface similarity between the sequence of gazeshifts made while encoding the objects of a scene and theserial presentation conditions that are central to the ex-pression of serial memory phenomena Given these sim-ilarities one might expect the serial memory literature toinform the question of scene-based object representa-tion but the exchange of ideas between these two re-search communities has been minimal Slowing the flowof information are significant methodological differ-ences between these two classes of studies On the onehand scene-based object memory studies typically pres-ent objects simultaneously as part of a scene and then askobservers to make some judgment about one of the sceneobjects (for a review see Henderson amp Hollingworth1999a) On the other hand studies designed to investi-gate serial position effects on visual memory typicallypresent a series of individual patterns (eg histoformskaleidoscope patterns snowflakes) at a central displaylocation and then ask observers whether a test patternappeared in this study series (Broadbent amp Broadbent1981 Neath 1993 Phillips 1983 Wright 1998)1 Twomethodological concerns therefore prevent the gener-

alization of findings from the serial memory literature tothe question of object representation in scenes First noequivalency has been demonstrated between the sequen-tial presentation paradigm and the order in which objectsare viewed in a scene Are the objects fixated in a sceneserialized in memory in the same way as they would be ifthey were presented individually one after the other Sec-ond because serial memory studies typically present stim-uli at the same location in space these studies provide noinformation about the representation of objects scatteredover many locations in a scene In short many serial mem-ory studies lack a spatial memory component and there-fore cannot be used to study the identityndashlocation bindingthat is central to OFT and more generally the representa-tion of objects in scenes

The extensive literature on spatial working memorysuffers from the opposite problem with regard to inform-ing research on scene-based object memory In a typicalspatial memory study stimuli are presented serially at dif-ferent display locations and observers are tested in one ofthree ways (1) they are either asked to recall the loca-tions of all the items from the memory set without regard for order (eg Hale Myerson Rhee Weiss amp Abrams1996) (2) they are presented with a spatial probe andasked whether the probe location corresponds to one of theitem locations from the memory set (eg Awh Jonides ampReuter-Lorenz 1998 Courtney Petit Maisog Ungerleideramp Haxby 1998) or (3) they are asked to reproduce the en-tire spatiotemporal sequence in which the study itemswere presented (eg Jones Farrand Stuart amp Morris1995 Smyth amp Scholey 1996) Given this taxonomy ofspatial memory studies several methodological differ-ences again prevent the generalization of findings to thequestion of objectndashlocation binding in scenes First manyspatial memory studies are unconcerned with how thestimulus presentation order might affect onersquos spatialmemory ability and therefore do not include order ofstimulus presentation in their analyses (eg Awh et al1998 Courtney et al 1998 Hale et al 1996) Secondthe studies that do take presentation order into accountare primarily concerned with memory for temporal orderper se not the effect of presentation order on memoryfor object representations (eg Jones et al 1995 Smythamp Scholey 1996) Third in many spatial memory stud-ies object identity is irrelevant to the task Popular spa-tial memory paradigms such as the Corsi blocks task orthe 724 spatial recall test do not even manipulate objectidentity instead using simple dots or checker shapes asstimuli (eg Rao Hammeke McQuillen Kharti amp Lloyd1984 Shimozaki et al 2003) Thus the majority of exist-ing spatial memory studies while important shed littlelight on real-world scene representation

To our knowledge only Walker and colleagues (WalkerHitch Doyle amp Porter 1994 Walker Hitch amp Duroe1993) manipulated both the identity and location of ob-jects for the purpose of studying how serial presentationorder affects memory Walker et al (1993) presented asequence of four geometric patterns at four horizontally

678 ZELINSKY AND LOSCHKY

arranged display positions followed by the presentationof a test stimulus at a neutral display location The ob-serverrsquos task was to indicate the previous location of thistest pattern in the study array When plotted as a serialposition function the data from this study revealed apronounced recency effect that was largely limited to thelast item presented in the study sequence This finding isimportant because it demonstrates serial position effectson the basis of the conjunction of spatial and identity in-formation about an object thereby potentially informingthe scene representation literature However it is alsoimportant not to generalize too quickly from this findingto object memory in scenes The patterns used by Walkeret al (1993) were highly confusable nonsense shapeswhereas those appearing in scenes are typically real-world objects The serial position effects reported bythese authors may therefore be influenced by this choiceof stimuli Moreover the placement of patterns in theWalker et al (1993) study was highly predictable andlimited to only four locations in a horizontal array thespatial distribution of objects in scenes is far less con-strained Finally and most important to the present dis-cussion is the fact that Walker et al (1993) still used asequential presentation paradigm In the real world ob-servers typically shift their gaze from object to object asthey encode patterns into memory The relationship be-tween order of stimulus presentation and the eye move-ments to objects appearing in scenes remains a largelyunaddressed question in the serial memory community

Many studies in the visual cognition literature haveused eye movement measures to assess memory in mul-tiobject displays (Hayhoe Bensinger amp Ballard 1998Henderson amp Hollingworth 1999b Henderson Mc-Clure Pierce amp Schrock 1997 Henderson Pollatsek ampRayner 1987 1989 Henderson amp Siefert 1999 2001Hollingworth Schrock amp Henderson 2001 Holling-worth Williams amp Henderson 2001 Zelinsky 2001)but here too there have been few attempts to specifyhow memory is affected by the order of fixations on ob-jects Two recent studies have addressed this relationshipdirectly Irwin and Zelinsky (2002) allowed observers toview a seven-object display for either 1 3 5 9 or 15 fix-ations Following this gaze-dependent viewing period aspatial probe cued one of the object locations in the studydisplay and the observer was given a seven alternativeforced choice (AFC) recognition test for the cued targetWhen Irwin and Zelinsky analyzed their gaze data in termsof recency of object fixation (Figure 8 p 889) a clear re-lationship emerged between recognition accuracy andwhen the target was fixated during study When the targetwas fixated immediately prior to probe onset recognitionaccuracy was over 90 However when the target was fix-ated 1 or 2 fixations earlier accuracy dropped to about80 Accuracy dropped further to 65 for targets fixatedeven farther back in the eye movement sequence (3 4 or5 fixations prior to the spatial probe) but then remainedconstant at this above-chance level Irwin and Zelinskyinterpreted this recency benefit for objects viewed within

the last 3 fixations as support for transsaccadic OFT and itsassumption of a 3ndash4 object memory capacity

In another recent study Hollingworth and Henderson(2002) performed a similar post hoc analysis of their eyeposition data and reached a very different conclusionThese authors made type or token changes to objects inrealistic scenes as observers inspected these scenes inanticipation of a memory test Observers were also askedto indicate whether a scene change was noticed duringviewing Among their many interesting results Holling-worth and Henderson failed to find a reliable recency ef-fect in their data Unlike Irwin and Zelinsky (2002) de-tection accuracy in their task did not vary systematicallywith the number of fixations since the target was lastviewed However consistent with Irwin and Zelinsky(2002 cf Zelinsky amp Loschky 1998) Hollingworth andHenderson did find an above-chance prerecency level ofaccuracy extending several items back in the viewing se-quence which they interpreted as evidence for an LTMrepresentation of object properties

As evidenced by the discrepancy between Irwin andZelinsky (2002) and Hollingworth and Henderson (2002)the literature does not yet have a clear picture of how ser-ial fixation order affects memory for objects in scenesnor is it even clear what theoretical framework will bestdescribe these data patterns once they have stabilizedAlthough transsaccadic OFTrsquos prediction of a 3ndash4 objectrecency effect is well supported by Irwin and Zelinskyand much of the previous transsaccadic memory litera-ture (Irwin 1992 Irwin amp Andrews 1996 Irwin amp Gor-don 1998) this prediction is greater than the one-objectrecency effect often observed in studies using a sequen-tial presentation paradigm (Phillips amp Christie 1977a1977b Wright Santiago Sands Kendrick amp Cook1985 for reviews see Phillips 1983 and Wright 1998)and the zero-object recency effect reported by Holling-worth and Henderson The present study was conductedto address these discrepant estimates of recency in thescene-based object memory literature as well as to ex-plore factors potentially contributing to above-chancelevels of prerecency accuracy reported by Irwin andZelinsky and elsewhere (Hollingworth amp Henderson2002 Kerr Avons amp Ward 1999 Phillips 1983 Phillipsamp Christie 1977a Walker et al 1994 Walker et al1993 Wright 1998 Wright et al 1985) By outlining amethod of serializing the encoding of scene objects usingeye movements it was also our hope to build a bridge be-tween the scene-based object memory literature and themany studies using a sequential presentation paradigm todescribe serial order effects on object memory

EXPERIMENT 1

One possible reason for the discrepancy between thefindings of Irwin and Zelinsky (2002) and Hollingworthand Henderson (2002) may be that different methods ofquantifying serial position were used in the two studiesHollingworth and Henderson plotted percent accuracy

MEMORY FOR OBJECTS IN SCENES 679

for a target object as a function of the number of fixa-tions made after gaze left the target The forgetting func-tion derived by Irwin and Zelinsky (Figure 8 p 889) wassomewhat different plotting target accuracy as a func-tion of the number of objects inspected following fixa-tion on the target This distinction is important becausethe number of fixations made during scene viewing doesnot typically map perfectly onto the number of fixatedobjects Often multiple fixations are devoted to a singleobject and sometimes two neighboring objects are in-spected with only a single fixation (Zelinsky Rao Hay-hoe amp Ballard 1997) Although Irwin and Zelinsky(2002) and Hollingworth and Henderson (2002) bothplotted memory accuracy as a function of viewing orderit is possible that their forgetting functions were not cap-turing the same serial position information and conse-quently could not be directly compared

Experiment 1 describes a purely object-based forget-ting function for items viewed serially in simple scenesAs in the Irwin and Zelinsky (2002) study we monitorthe eye movements of observers as they freely inspect a

multiobject scene and we make the duration of this studyperiod contingent on their oculomotor behavior How-ever rather than terminating the study display after a setnumber of fixations we make testing contingent on thenumber of different objects fixated by observers aftertheir gaze leaves a predesignated target This online mon-itoring of the number of posttarget objects fixated by ob-servers allows us to parametrically manipulate the num-ber of objects viewed between study and test Rather thanrelying on post hoc fixation analyses to derive recencyand prerecency serial memory functions our manipula-tion of intervening objects as an independent variablemeans that our experimental design will have sufficientpower to discern the memory patterns in question We be-lieve that this more powerful design when combinedwith an object-based gaze-contingent methodology willprovide a clearer measure of object memory in scenes

MethodStimuli The stimuli consisted of nine common real-world objects

(toy tool or food items) arranged on an appropriate background sur-

Figure 1 Events comprising a typical trial in Experiment 1 (A) A nine-object scene was presented to observers Theirtask was to study these objects in anticipation of a memory test (B) Observers freely viewed the objects with the inten-tion of remembering their identities and locations eventually fixating the target object (in this case the baby bottle)(C) Unknown to the observer the display program would then count the number of objects fixated after gaze left thetarget (D) When the criterion number of different posttarget objects (in this case three) was fixated the multiobjectdisplay was replaced by a spatial probe at the targetrsquos location (E) The observer then had to select the target object fromamong four alternatives

A B

C

D E

680 ZELINSKY AND LOSCHKY

face (a crib workbench or dining table) Each simple scene sub-tended 18ordm 116ordm of visual angle and was of near-photographicquality (16-bit color)2 Individual objects were scaled to fit insidea 24ordm bounding box and their locations in the scene were con-strained to 18 positions creating what appeared to be a haphazardarrangement of items lying on a surface As a result of these place-ment constraints scene objects had a minimum and maximumcenter-to-center separation of 24ordm and 145ordm respectively Multipletrials for a given scene type were created by randomly pairing theobjects to locations (ie the same nine objects would occupy dif-ferent locations in each scene) with the constraints that no two dis-play configurations were repeated and that the memory target wouldappear equally often in each of the 18 allowable positions

Procedure The experimental procedure is shown in Figure 1 Ascene appeared at the start of a trial (Figure 1A) and the observersrsquotask was to remember the identity and location of every objectmdashinthis case toys in a babyrsquos crib (there were also tools on a workbenchand food items on a dining table) Given this formidable memorytask observers invariably made eye movements to the individualobjects a response that was anticipated and crucial to the currentparadigm but not explicitly mentioned in the experiment instruc-tions Eye position was monitored every millisecond and analyzedonline to determine the object in the scene being fixated by the ob-server Unknown to the observer one object in the display was pre-designated as the memory target for that particular trial (eg thebaby bottle in Figure 1) As the observer freely scanned the scenegaze would eventually be directed to this target object (Figure 1B)This fixation event was detected by the program controlling the ex-periment which then started to count the number of different ob-jects fixated after gaze left the target This ldquocountrdquo constituted thetermination criterion for the display meaning that if the count waspreset to three the observer would be allowed to fixate exactly threeobjects after the target (Figure 1C) As gaze moved away from thethird posttarget object the study scene was replaced by a 1-sec du-ration probe (a colored noise mask) appearing at the targetrsquos loca-tion on an ldquoemptiedrdquo background surface (Figure 1D) Followingthe probe a display showing four objects was presented and theobserver had to indicate which of them appeared at the probed lo-cation (Figure 1E)3 One of these objects was always the target theother three were randomly selected from the study scene Becauseobservers were on a bite-bar and could not easily speak they re-sponded by looking at the desired object causing a white box to bedrawn around the item then pressing a hand-held button when sat-isfied with their selection Observers were asked to respond as ac-curately as possible without regard for time

The independent variable of interest in this study was interven-ing objects the number of different objects looked at after the tar-get There were seven intervening object conditions (1ndash7) For ex-ample in the one-intervening-object condition the study displayterminated during the saccade away from the first posttarget object(ie gaze was not allowed to land on a second nontarget item afterleaving the target) Likewise in the seven-intervening-object con-dition the observer fixated exactly seven different nontarget ob-jects following target fixation The amount of time that observersviewed the study scene therefore varied it was determined by whenthey first fixated the target and the duration of their gaze on eachobject as well as the intervening object criterion set for that partic-ular trial The intervening object conditions were randomly inter-leaved throughout the experiment resulting in an unpredictablestudy scene duration Postexperiment questioning revealed that theparticipants were uniformly unaware that scene presentation timedepended on their own pattern of eye movements attributing thevariable interval instead to some random schedule of presentationdurations

Participants Six experimentally naive observers from the Uni-versity of Illinois at Urbana-Champaign each participated in 378trials 54 in each of the seven intervening object conditions Eye

position was monitored throughout each trial using a Generation Vdual-Purkinje-image (DPI) eyetracker sampling at 1000 Hz Thespatial precision of this tracker was better than 3 min of visual angleat a viewing distance of 62 cm Saccadic eye movements were de-termined online using a velocity-based algorithm implementing aroughly 125ordmsec detection threshold The experiment required two15-h sessions conducted on separate days and the participantswere paid $24 upon completion

Results and DiscussionOur gaze-contingent memory paradigm allows us to

plot response accuracy as a function of intervening ob-jects for trials in which the target was fixated only onceduring viewing (ie no target refixations) This analysisshown in Figure 2A by the solid markers reveals twoclear patterns of behavior First memory declined pre-cipitously between the 1 and 3 intervening object condi-tions dropping linearly from an 87 level of accuracy toonly 65 [F(630) 653 MSe 0044 p 001] Thispattern suggests that the act of fixating the first three ob-jects after the target dramatically interfered with mem-ory for the target objectmdashmuch like the interference ob-served when objects are presented one after another in asequential memory paradigm Second this drop in accu-racy reached an abrupt asymptote at three interveningobjects meaning that item fixations over the 4ndash7 interven-ing object range resulted in no additional object-relatedinterference Moreover because chance response in this4AFC task should produce a 25 level of accuracy the65 level described by this asymptote indicates a clearlyabove-chance and relatively good memory for an objectirrespective of the number of items fixated after it duringstudy4 We refer to the region of declining accuracy in thisintervening object function as the recency memory com-ponent and the above-chance asymptotic region of accu-racy as the prerecency memory component

Our clear evidence for distinct recency and prerecencybehavior although consistent with the data from Irwinand Zelinsky (2002) is inconsistent with Hollingworthand Hendersonrsquos (2002) failure to observe a recency ef-fect To determine whether this discrepancy is due to ourquantification of serial order in terms of intervening ob-jects rather than intervening fixations we reanalyzed ourdata using Hollingworth and Hendersonrsquos interveningfixation method (the open markers in Figure 2A) Thiscomparison revealed considerable agreement betweenthe two functions with mean differences in accuracyamounting to only 255 This high degree of similaritywould be expected if observers tended to devote only asingle fixation to each object during study More impor-tant accuracy again declined with the number of fixa-tions occurring after gaze left the target [F(630) 493MSe 7738 p 001] Note however that this recencyeffect is less pronounced than that in the intervening ob-ject data We attribute this pattern to a special role thatobjects play in producing recency behavior Differencesbetween the intervening object and intervening fixationfunctions are caused by the occasional multiple fixationof individual objects during study Given that these fix-

MEMORY FOR OBJECTS IN SCENES 681

ations on the same object were less detrimental to mem-ory than were fixations on different objects (as indicatedby the slightly better accuracy in the intervening fixationdata) it appears that objects not fixations are primarilyresponsible for the accuracy decline that defines recencyin our task Moreover if one were to replot such an object-based recency effect as a function of intervening fixa-tions the result should be a stretching of the object-basedrecency effect over more intervening events (ie fixa-tions) which is the exact pattern appearing in Figure 2AThis dilution of recency in the intervening fixation datasuggests one factor contributing to Hollingworth andHendersonrsquos (2002) failure to find a recency effect in theirstudy As recency is stretched over more of the interven-ing event function the slope becomes shallower and theeffect more difficult to observe For this reason we be-

lieve that the preferred method of quantifying recencyduring scene viewing is to serialize the data by objectsrather than by fixations

As a working interpretation of our data we hypothe-size that retroactive interference (RI) is a likely cause ofthe observed recency effect Consistent with a classic RIexplanation for STM forgetting (Waugh amp Norman 1965see also Intraub 1984 for RI applied to scenes) each ob-ject fixated after the target in our task might have inter-fered with memory for the target eventually driving recog-nition performance into the prerecency baseline Howeverinterference might be proactive as well as retroactive (Kep-pel amp Underwood 1962 Wickens Born amp Allen 1963)and given that a relatively small set of objects was beingreused in each trial of this experiment the potential forproactive interference (PI) influencing the data wouldseem high Previous research using a serial presentationparadigm has found that between-trial PI effects on pic-ture recognition memory are greatest early in an experi-ment rather than late (Jitsumori Wright amp Shyan 1989Wright 1998) particularly under conditions of immedi-ate testing (eg a 1-sec retention interval) It might thenbe the case that observers were able to accurately re-member the target over a large intervening object rangeearly in the experiment but as a result of PI accumulat-ing over trials this high level of accuracy gradually de-clined in the later trials to chance performance If theforgetting function was changing during the course ofthe experiment the recency and prerecency componentsin Figure 2A might be mere artifacts of averaging overtrials To explore this possibility we evenly divided the378 trials into ldquoearlyrdquo and ldquolaterdquo conditions If PI wasbuilding up throughout the experiment prerecency accu-racy in the early-trial condition should be higher than inthe late-trial condition However as can be seen from Fig-ure 3 this was clearly not the case If anything accuracyactually improved somewhat over the 3ndash7 intervening ob-ject range in the late trials Rather than suffering from abuildup of PI observers benefited slightly from theirgrowing familiarity with the set of stimulus objects

Although between-trial PI appears not to have playeda role in producing the Figure 2A intervening objectfunction there still exists the possibility that anotherform of PI might have influenced our data Given ourfree-viewing paradigm the number of nontarget objectsfixated before gaze first shifted to the target necessarilyvaried with the observersrsquo own idiosyncratic scanningpreferences If observers happened to inspect more pre-target objects in the 3ndash7 intervening object conditionsand if the fixation of these objects interfered with targetmemory the buildup of PI within a trial might result in thelower accuracy observed in the prerecency component

To evaluate this potential for within-trial PI we firsttested the premise that pretarget fixations were less fre-quent in the 1ndash2 intervening object conditions comparedwith the 3ndash7 intervening object conditions The resultsfrom this analysis are shown in Table 1 As expected onthe basis of our random selection of targets the number

Figure 2 Memory forgetting functions relating accuracy to thenumber of different events following viewing of the target Linesextending from the markers indicate one SEM (A) Data from Ex-periment 1 in which a single nine-object study scene was pre-sented on each trial and memory accuracy was defined in termsof either the number of intervening objects fixated between tar-get and test or the number of intervening fixations (B) Data fromExperiment 2 in which each trial consisted of nine one-objectstudy displays presented sequentially over time Square markersindicate data from a 1000ndash0 condition in which individual objectswere presented for 1 sec each without an interstimulus interval(ISI) Data from the 500ndash500 (500-msec presentation 500-msecISI) and 500ndash0 (500-msec presentation 0-msec ISI) conditions areshown by the triangle and circle markers respectively

682 ZELINSKY AND LOSCHKY

of distractor objects fixated before the target in a trialdid not meaningfully differ between the 1ndash2 and the 3ndash7intervening object conditions [Pearson χ 2 (8) 325p 918 N 1457] Our failure to find an overrepre-sentation of pretarget distractor fixations in the 3ndash7 in-tervening object trials therefore weakens the argumentthat within-trial PI could account for our recency dataWe also directly looked for a within-trial PI effect bycomparing trials in which the target was fixated veryearly in the sequence of fixations (with 0 1 or 2 objectsfixated before the target) with trials in which the targetwas first fixated later (three or more objects fixated be-fore the target) If within-trial PI played a role in deter-mining the recency effect accuracy should be higherwhen there were fewer pretarget fixations Table 2 indi-cates that this too was not the case Trials in which thetarget was fixated very early (ie few pretarget fixations)were no more accurate than trials in which the target wasfixated much later (ie several pretarget fixations) Wecan therefore conclude that pretarget fixations and what-ever within-trial PI they may have exerted cannot explainthe recency effects reported in Figure 2A

Unresolved QuestionsSince we dispatched PI as a factor shaping the Fig-

ure 2A forgetting functions an explanation appealing toRI becomes more attractive However two remaining al-

ternative explanations cannot be ruled out based on theExperiment 1 data First it is possible that the recency be-havior reported in Figure 2A reflects a decay processrather than RI from objects fixated after the target Decay-based explanations for STM forgetting enjoyed consider-able popularity in the early days of modern memory re-search (eg Peterson amp Peterson 1959 see also Altmannamp Gray 2002 for a recent formulation of this view) butwere largely replaced by interference-based explanationsafter several studies demonstrated little or no forgettingover time when item interference was controlled (Loess ampWaugh 1967 Waugh amp Norman 1965 Wickens et al1963) Although the latter studies certainly reduce theplausibility that decay was responsible for the currentlyobserved recency behavior this factor cannot be ruledout for the specific presentation and testing conditionsused in Experiment 1 A second factor potentially af-fecting the Figure 2A forgetting data is specific to the si-multaneous presentation paradigm Our use of eyemovements to serialize object processing assumes thatprocessing is restricted to only the currently fixated ob-ject during free viewing Several studies however havealso demonstrated processing benefits for extrafoveallyviewed objects (Henderson amp Anes 1994 PollatsekRayner amp Henderson 1990) If multiple objects werebeing processed during each display fixation this oppor-tunity for extrafoveal processing of the target might ex-plain the above-chance level of accuracy observed in theFigure 2A prerecency data Although such extrafovealprocessing benefits have been reported only for the in-tended target of a saccade and not every object in a dis-play (Henderson Pollatsek amp Rayner 1989) the poten-tial for a more generalized form of extrafoveal processingbenefit affecting our data should be considered

EXPERIMENT 2

Experiment 2 directly addressed the possibilities thatdecay or extrafoveal processes may have influenced theExperiment 1 data The question of whether decay or in-terference underlies the Figure 2A recency effect was re-solved by holding constant the number of interveningobjects appearing between study and test but varying theretention interval If decay was responsible for the ob-served decline in accuracy lengthening the retention in-terval should accelerate the accuracy decline and con-strict the expression of recency on the forgetting function

Table 1Percentage of Trials With a Given Number of Pretarget Object Fixations

(0ndash8) as a Function of Two Intervening Object Groups (1ndash2 vs 3ndash7)

Number of Pretarget Objects Fixated

Number of Intervening Objects 0 1 2 3 4 5 6 7 8

1ndash2 112 134 112 125 90 112 103 106 1063ndash7 103 113 124 120 102 104 109 106 120

NotemdashData exclude trials in which the target object was refixated n = 554 and 903cases for the 1ndash2 and 3ndash7 intervening object groupings respectively

Figure 3 Intervening object functions derived from an early-versus-late segregation of the Experiment 1 data Open markersshow data from the first 189 trials of the experiment closedmarkers show data from the final 189 trials

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

678 ZELINSKY AND LOSCHKY

arranged display positions followed by the presentationof a test stimulus at a neutral display location The ob-serverrsquos task was to indicate the previous location of thistest pattern in the study array When plotted as a serialposition function the data from this study revealed apronounced recency effect that was largely limited to thelast item presented in the study sequence This finding isimportant because it demonstrates serial position effectson the basis of the conjunction of spatial and identity in-formation about an object thereby potentially informingthe scene representation literature However it is alsoimportant not to generalize too quickly from this findingto object memory in scenes The patterns used by Walkeret al (1993) were highly confusable nonsense shapeswhereas those appearing in scenes are typically real-world objects The serial position effects reported bythese authors may therefore be influenced by this choiceof stimuli Moreover the placement of patterns in theWalker et al (1993) study was highly predictable andlimited to only four locations in a horizontal array thespatial distribution of objects in scenes is far less con-strained Finally and most important to the present dis-cussion is the fact that Walker et al (1993) still used asequential presentation paradigm In the real world ob-servers typically shift their gaze from object to object asthey encode patterns into memory The relationship be-tween order of stimulus presentation and the eye move-ments to objects appearing in scenes remains a largelyunaddressed question in the serial memory community

Many studies in the visual cognition literature haveused eye movement measures to assess memory in mul-tiobject displays (Hayhoe Bensinger amp Ballard 1998Henderson amp Hollingworth 1999b Henderson Mc-Clure Pierce amp Schrock 1997 Henderson Pollatsek ampRayner 1987 1989 Henderson amp Siefert 1999 2001Hollingworth Schrock amp Henderson 2001 Holling-worth Williams amp Henderson 2001 Zelinsky 2001)but here too there have been few attempts to specifyhow memory is affected by the order of fixations on ob-jects Two recent studies have addressed this relationshipdirectly Irwin and Zelinsky (2002) allowed observers toview a seven-object display for either 1 3 5 9 or 15 fix-ations Following this gaze-dependent viewing period aspatial probe cued one of the object locations in the studydisplay and the observer was given a seven alternativeforced choice (AFC) recognition test for the cued targetWhen Irwin and Zelinsky analyzed their gaze data in termsof recency of object fixation (Figure 8 p 889) a clear re-lationship emerged between recognition accuracy andwhen the target was fixated during study When the targetwas fixated immediately prior to probe onset recognitionaccuracy was over 90 However when the target was fix-ated 1 or 2 fixations earlier accuracy dropped to about80 Accuracy dropped further to 65 for targets fixatedeven farther back in the eye movement sequence (3 4 or5 fixations prior to the spatial probe) but then remainedconstant at this above-chance level Irwin and Zelinskyinterpreted this recency benefit for objects viewed within

the last 3 fixations as support for transsaccadic OFT and itsassumption of a 3ndash4 object memory capacity

In another recent study Hollingworth and Henderson(2002) performed a similar post hoc analysis of their eyeposition data and reached a very different conclusionThese authors made type or token changes to objects inrealistic scenes as observers inspected these scenes inanticipation of a memory test Observers were also askedto indicate whether a scene change was noticed duringviewing Among their many interesting results Holling-worth and Henderson failed to find a reliable recency ef-fect in their data Unlike Irwin and Zelinsky (2002) de-tection accuracy in their task did not vary systematicallywith the number of fixations since the target was lastviewed However consistent with Irwin and Zelinsky(2002 cf Zelinsky amp Loschky 1998) Hollingworth andHenderson did find an above-chance prerecency level ofaccuracy extending several items back in the viewing se-quence which they interpreted as evidence for an LTMrepresentation of object properties

As evidenced by the discrepancy between Irwin andZelinsky (2002) and Hollingworth and Henderson (2002)the literature does not yet have a clear picture of how ser-ial fixation order affects memory for objects in scenesnor is it even clear what theoretical framework will bestdescribe these data patterns once they have stabilizedAlthough transsaccadic OFTrsquos prediction of a 3ndash4 objectrecency effect is well supported by Irwin and Zelinskyand much of the previous transsaccadic memory litera-ture (Irwin 1992 Irwin amp Andrews 1996 Irwin amp Gor-don 1998) this prediction is greater than the one-objectrecency effect often observed in studies using a sequen-tial presentation paradigm (Phillips amp Christie 1977a1977b Wright Santiago Sands Kendrick amp Cook1985 for reviews see Phillips 1983 and Wright 1998)and the zero-object recency effect reported by Holling-worth and Henderson The present study was conductedto address these discrepant estimates of recency in thescene-based object memory literature as well as to ex-plore factors potentially contributing to above-chancelevels of prerecency accuracy reported by Irwin andZelinsky and elsewhere (Hollingworth amp Henderson2002 Kerr Avons amp Ward 1999 Phillips 1983 Phillipsamp Christie 1977a Walker et al 1994 Walker et al1993 Wright 1998 Wright et al 1985) By outlining amethod of serializing the encoding of scene objects usingeye movements it was also our hope to build a bridge be-tween the scene-based object memory literature and themany studies using a sequential presentation paradigm todescribe serial order effects on object memory

EXPERIMENT 1

One possible reason for the discrepancy between thefindings of Irwin and Zelinsky (2002) and Hollingworthand Henderson (2002) may be that different methods ofquantifying serial position were used in the two studiesHollingworth and Henderson plotted percent accuracy

MEMORY FOR OBJECTS IN SCENES 679

for a target object as a function of the number of fixa-tions made after gaze left the target The forgetting func-tion derived by Irwin and Zelinsky (Figure 8 p 889) wassomewhat different plotting target accuracy as a func-tion of the number of objects inspected following fixa-tion on the target This distinction is important becausethe number of fixations made during scene viewing doesnot typically map perfectly onto the number of fixatedobjects Often multiple fixations are devoted to a singleobject and sometimes two neighboring objects are in-spected with only a single fixation (Zelinsky Rao Hay-hoe amp Ballard 1997) Although Irwin and Zelinsky(2002) and Hollingworth and Henderson (2002) bothplotted memory accuracy as a function of viewing orderit is possible that their forgetting functions were not cap-turing the same serial position information and conse-quently could not be directly compared

Experiment 1 describes a purely object-based forget-ting function for items viewed serially in simple scenesAs in the Irwin and Zelinsky (2002) study we monitorthe eye movements of observers as they freely inspect a

multiobject scene and we make the duration of this studyperiod contingent on their oculomotor behavior How-ever rather than terminating the study display after a setnumber of fixations we make testing contingent on thenumber of different objects fixated by observers aftertheir gaze leaves a predesignated target This online mon-itoring of the number of posttarget objects fixated by ob-servers allows us to parametrically manipulate the num-ber of objects viewed between study and test Rather thanrelying on post hoc fixation analyses to derive recencyand prerecency serial memory functions our manipula-tion of intervening objects as an independent variablemeans that our experimental design will have sufficientpower to discern the memory patterns in question We be-lieve that this more powerful design when combinedwith an object-based gaze-contingent methodology willprovide a clearer measure of object memory in scenes

MethodStimuli The stimuli consisted of nine common real-world objects

(toy tool or food items) arranged on an appropriate background sur-

Figure 1 Events comprising a typical trial in Experiment 1 (A) A nine-object scene was presented to observers Theirtask was to study these objects in anticipation of a memory test (B) Observers freely viewed the objects with the inten-tion of remembering their identities and locations eventually fixating the target object (in this case the baby bottle)(C) Unknown to the observer the display program would then count the number of objects fixated after gaze left thetarget (D) When the criterion number of different posttarget objects (in this case three) was fixated the multiobjectdisplay was replaced by a spatial probe at the targetrsquos location (E) The observer then had to select the target object fromamong four alternatives

A B

C

D E

680 ZELINSKY AND LOSCHKY

face (a crib workbench or dining table) Each simple scene sub-tended 18ordm 116ordm of visual angle and was of near-photographicquality (16-bit color)2 Individual objects were scaled to fit insidea 24ordm bounding box and their locations in the scene were con-strained to 18 positions creating what appeared to be a haphazardarrangement of items lying on a surface As a result of these place-ment constraints scene objects had a minimum and maximumcenter-to-center separation of 24ordm and 145ordm respectively Multipletrials for a given scene type were created by randomly pairing theobjects to locations (ie the same nine objects would occupy dif-ferent locations in each scene) with the constraints that no two dis-play configurations were repeated and that the memory target wouldappear equally often in each of the 18 allowable positions

Procedure The experimental procedure is shown in Figure 1 Ascene appeared at the start of a trial (Figure 1A) and the observersrsquotask was to remember the identity and location of every objectmdashinthis case toys in a babyrsquos crib (there were also tools on a workbenchand food items on a dining table) Given this formidable memorytask observers invariably made eye movements to the individualobjects a response that was anticipated and crucial to the currentparadigm but not explicitly mentioned in the experiment instruc-tions Eye position was monitored every millisecond and analyzedonline to determine the object in the scene being fixated by the ob-server Unknown to the observer one object in the display was pre-designated as the memory target for that particular trial (eg thebaby bottle in Figure 1) As the observer freely scanned the scenegaze would eventually be directed to this target object (Figure 1B)This fixation event was detected by the program controlling the ex-periment which then started to count the number of different ob-jects fixated after gaze left the target This ldquocountrdquo constituted thetermination criterion for the display meaning that if the count waspreset to three the observer would be allowed to fixate exactly threeobjects after the target (Figure 1C) As gaze moved away from thethird posttarget object the study scene was replaced by a 1-sec du-ration probe (a colored noise mask) appearing at the targetrsquos loca-tion on an ldquoemptiedrdquo background surface (Figure 1D) Followingthe probe a display showing four objects was presented and theobserver had to indicate which of them appeared at the probed lo-cation (Figure 1E)3 One of these objects was always the target theother three were randomly selected from the study scene Becauseobservers were on a bite-bar and could not easily speak they re-sponded by looking at the desired object causing a white box to bedrawn around the item then pressing a hand-held button when sat-isfied with their selection Observers were asked to respond as ac-curately as possible without regard for time

The independent variable of interest in this study was interven-ing objects the number of different objects looked at after the tar-get There were seven intervening object conditions (1ndash7) For ex-ample in the one-intervening-object condition the study displayterminated during the saccade away from the first posttarget object(ie gaze was not allowed to land on a second nontarget item afterleaving the target) Likewise in the seven-intervening-object con-dition the observer fixated exactly seven different nontarget ob-jects following target fixation The amount of time that observersviewed the study scene therefore varied it was determined by whenthey first fixated the target and the duration of their gaze on eachobject as well as the intervening object criterion set for that partic-ular trial The intervening object conditions were randomly inter-leaved throughout the experiment resulting in an unpredictablestudy scene duration Postexperiment questioning revealed that theparticipants were uniformly unaware that scene presentation timedepended on their own pattern of eye movements attributing thevariable interval instead to some random schedule of presentationdurations

Participants Six experimentally naive observers from the Uni-versity of Illinois at Urbana-Champaign each participated in 378trials 54 in each of the seven intervening object conditions Eye

position was monitored throughout each trial using a Generation Vdual-Purkinje-image (DPI) eyetracker sampling at 1000 Hz Thespatial precision of this tracker was better than 3 min of visual angleat a viewing distance of 62 cm Saccadic eye movements were de-termined online using a velocity-based algorithm implementing aroughly 125ordmsec detection threshold The experiment required two15-h sessions conducted on separate days and the participantswere paid $24 upon completion

Results and DiscussionOur gaze-contingent memory paradigm allows us to

plot response accuracy as a function of intervening ob-jects for trials in which the target was fixated only onceduring viewing (ie no target refixations) This analysisshown in Figure 2A by the solid markers reveals twoclear patterns of behavior First memory declined pre-cipitously between the 1 and 3 intervening object condi-tions dropping linearly from an 87 level of accuracy toonly 65 [F(630) 653 MSe 0044 p 001] Thispattern suggests that the act of fixating the first three ob-jects after the target dramatically interfered with mem-ory for the target objectmdashmuch like the interference ob-served when objects are presented one after another in asequential memory paradigm Second this drop in accu-racy reached an abrupt asymptote at three interveningobjects meaning that item fixations over the 4ndash7 interven-ing object range resulted in no additional object-relatedinterference Moreover because chance response in this4AFC task should produce a 25 level of accuracy the65 level described by this asymptote indicates a clearlyabove-chance and relatively good memory for an objectirrespective of the number of items fixated after it duringstudy4 We refer to the region of declining accuracy in thisintervening object function as the recency memory com-ponent and the above-chance asymptotic region of accu-racy as the prerecency memory component

Our clear evidence for distinct recency and prerecencybehavior although consistent with the data from Irwinand Zelinsky (2002) is inconsistent with Hollingworthand Hendersonrsquos (2002) failure to observe a recency ef-fect To determine whether this discrepancy is due to ourquantification of serial order in terms of intervening ob-jects rather than intervening fixations we reanalyzed ourdata using Hollingworth and Hendersonrsquos interveningfixation method (the open markers in Figure 2A) Thiscomparison revealed considerable agreement betweenthe two functions with mean differences in accuracyamounting to only 255 This high degree of similaritywould be expected if observers tended to devote only asingle fixation to each object during study More impor-tant accuracy again declined with the number of fixa-tions occurring after gaze left the target [F(630) 493MSe 7738 p 001] Note however that this recencyeffect is less pronounced than that in the intervening ob-ject data We attribute this pattern to a special role thatobjects play in producing recency behavior Differencesbetween the intervening object and intervening fixationfunctions are caused by the occasional multiple fixationof individual objects during study Given that these fix-

MEMORY FOR OBJECTS IN SCENES 681

ations on the same object were less detrimental to mem-ory than were fixations on different objects (as indicatedby the slightly better accuracy in the intervening fixationdata) it appears that objects not fixations are primarilyresponsible for the accuracy decline that defines recencyin our task Moreover if one were to replot such an object-based recency effect as a function of intervening fixa-tions the result should be a stretching of the object-basedrecency effect over more intervening events (ie fixa-tions) which is the exact pattern appearing in Figure 2AThis dilution of recency in the intervening fixation datasuggests one factor contributing to Hollingworth andHendersonrsquos (2002) failure to find a recency effect in theirstudy As recency is stretched over more of the interven-ing event function the slope becomes shallower and theeffect more difficult to observe For this reason we be-

lieve that the preferred method of quantifying recencyduring scene viewing is to serialize the data by objectsrather than by fixations

As a working interpretation of our data we hypothe-size that retroactive interference (RI) is a likely cause ofthe observed recency effect Consistent with a classic RIexplanation for STM forgetting (Waugh amp Norman 1965see also Intraub 1984 for RI applied to scenes) each ob-ject fixated after the target in our task might have inter-fered with memory for the target eventually driving recog-nition performance into the prerecency baseline Howeverinterference might be proactive as well as retroactive (Kep-pel amp Underwood 1962 Wickens Born amp Allen 1963)and given that a relatively small set of objects was beingreused in each trial of this experiment the potential forproactive interference (PI) influencing the data wouldseem high Previous research using a serial presentationparadigm has found that between-trial PI effects on pic-ture recognition memory are greatest early in an experi-ment rather than late (Jitsumori Wright amp Shyan 1989Wright 1998) particularly under conditions of immedi-ate testing (eg a 1-sec retention interval) It might thenbe the case that observers were able to accurately re-member the target over a large intervening object rangeearly in the experiment but as a result of PI accumulat-ing over trials this high level of accuracy gradually de-clined in the later trials to chance performance If theforgetting function was changing during the course ofthe experiment the recency and prerecency componentsin Figure 2A might be mere artifacts of averaging overtrials To explore this possibility we evenly divided the378 trials into ldquoearlyrdquo and ldquolaterdquo conditions If PI wasbuilding up throughout the experiment prerecency accu-racy in the early-trial condition should be higher than inthe late-trial condition However as can be seen from Fig-ure 3 this was clearly not the case If anything accuracyactually improved somewhat over the 3ndash7 intervening ob-ject range in the late trials Rather than suffering from abuildup of PI observers benefited slightly from theirgrowing familiarity with the set of stimulus objects

Although between-trial PI appears not to have playeda role in producing the Figure 2A intervening objectfunction there still exists the possibility that anotherform of PI might have influenced our data Given ourfree-viewing paradigm the number of nontarget objectsfixated before gaze first shifted to the target necessarilyvaried with the observersrsquo own idiosyncratic scanningpreferences If observers happened to inspect more pre-target objects in the 3ndash7 intervening object conditionsand if the fixation of these objects interfered with targetmemory the buildup of PI within a trial might result in thelower accuracy observed in the prerecency component

To evaluate this potential for within-trial PI we firsttested the premise that pretarget fixations were less fre-quent in the 1ndash2 intervening object conditions comparedwith the 3ndash7 intervening object conditions The resultsfrom this analysis are shown in Table 1 As expected onthe basis of our random selection of targets the number

Figure 2 Memory forgetting functions relating accuracy to thenumber of different events following viewing of the target Linesextending from the markers indicate one SEM (A) Data from Ex-periment 1 in which a single nine-object study scene was pre-sented on each trial and memory accuracy was defined in termsof either the number of intervening objects fixated between tar-get and test or the number of intervening fixations (B) Data fromExperiment 2 in which each trial consisted of nine one-objectstudy displays presented sequentially over time Square markersindicate data from a 1000ndash0 condition in which individual objectswere presented for 1 sec each without an interstimulus interval(ISI) Data from the 500ndash500 (500-msec presentation 500-msecISI) and 500ndash0 (500-msec presentation 0-msec ISI) conditions areshown by the triangle and circle markers respectively

682 ZELINSKY AND LOSCHKY

of distractor objects fixated before the target in a trialdid not meaningfully differ between the 1ndash2 and the 3ndash7intervening object conditions [Pearson χ 2 (8) 325p 918 N 1457] Our failure to find an overrepre-sentation of pretarget distractor fixations in the 3ndash7 in-tervening object trials therefore weakens the argumentthat within-trial PI could account for our recency dataWe also directly looked for a within-trial PI effect bycomparing trials in which the target was fixated veryearly in the sequence of fixations (with 0 1 or 2 objectsfixated before the target) with trials in which the targetwas first fixated later (three or more objects fixated be-fore the target) If within-trial PI played a role in deter-mining the recency effect accuracy should be higherwhen there were fewer pretarget fixations Table 2 indi-cates that this too was not the case Trials in which thetarget was fixated very early (ie few pretarget fixations)were no more accurate than trials in which the target wasfixated much later (ie several pretarget fixations) Wecan therefore conclude that pretarget fixations and what-ever within-trial PI they may have exerted cannot explainthe recency effects reported in Figure 2A

Unresolved QuestionsSince we dispatched PI as a factor shaping the Fig-

ure 2A forgetting functions an explanation appealing toRI becomes more attractive However two remaining al-

ternative explanations cannot be ruled out based on theExperiment 1 data First it is possible that the recency be-havior reported in Figure 2A reflects a decay processrather than RI from objects fixated after the target Decay-based explanations for STM forgetting enjoyed consider-able popularity in the early days of modern memory re-search (eg Peterson amp Peterson 1959 see also Altmannamp Gray 2002 for a recent formulation of this view) butwere largely replaced by interference-based explanationsafter several studies demonstrated little or no forgettingover time when item interference was controlled (Loess ampWaugh 1967 Waugh amp Norman 1965 Wickens et al1963) Although the latter studies certainly reduce theplausibility that decay was responsible for the currentlyobserved recency behavior this factor cannot be ruledout for the specific presentation and testing conditionsused in Experiment 1 A second factor potentially af-fecting the Figure 2A forgetting data is specific to the si-multaneous presentation paradigm Our use of eyemovements to serialize object processing assumes thatprocessing is restricted to only the currently fixated ob-ject during free viewing Several studies however havealso demonstrated processing benefits for extrafoveallyviewed objects (Henderson amp Anes 1994 PollatsekRayner amp Henderson 1990) If multiple objects werebeing processed during each display fixation this oppor-tunity for extrafoveal processing of the target might ex-plain the above-chance level of accuracy observed in theFigure 2A prerecency data Although such extrafovealprocessing benefits have been reported only for the in-tended target of a saccade and not every object in a dis-play (Henderson Pollatsek amp Rayner 1989) the poten-tial for a more generalized form of extrafoveal processingbenefit affecting our data should be considered

EXPERIMENT 2

Experiment 2 directly addressed the possibilities thatdecay or extrafoveal processes may have influenced theExperiment 1 data The question of whether decay or in-terference underlies the Figure 2A recency effect was re-solved by holding constant the number of interveningobjects appearing between study and test but varying theretention interval If decay was responsible for the ob-served decline in accuracy lengthening the retention in-terval should accelerate the accuracy decline and con-strict the expression of recency on the forgetting function

Table 1Percentage of Trials With a Given Number of Pretarget Object Fixations

(0ndash8) as a Function of Two Intervening Object Groups (1ndash2 vs 3ndash7)

Number of Pretarget Objects Fixated

Number of Intervening Objects 0 1 2 3 4 5 6 7 8

1ndash2 112 134 112 125 90 112 103 106 1063ndash7 103 113 124 120 102 104 109 106 120

NotemdashData exclude trials in which the target object was refixated n = 554 and 903cases for the 1ndash2 and 3ndash7 intervening object groupings respectively

Figure 3 Intervening object functions derived from an early-versus-late segregation of the Experiment 1 data Open markersshow data from the first 189 trials of the experiment closedmarkers show data from the final 189 trials

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

MEMORY FOR OBJECTS IN SCENES 679

for a target object as a function of the number of fixa-tions made after gaze left the target The forgetting func-tion derived by Irwin and Zelinsky (Figure 8 p 889) wassomewhat different plotting target accuracy as a func-tion of the number of objects inspected following fixa-tion on the target This distinction is important becausethe number of fixations made during scene viewing doesnot typically map perfectly onto the number of fixatedobjects Often multiple fixations are devoted to a singleobject and sometimes two neighboring objects are in-spected with only a single fixation (Zelinsky Rao Hay-hoe amp Ballard 1997) Although Irwin and Zelinsky(2002) and Hollingworth and Henderson (2002) bothplotted memory accuracy as a function of viewing orderit is possible that their forgetting functions were not cap-turing the same serial position information and conse-quently could not be directly compared

Experiment 1 describes a purely object-based forget-ting function for items viewed serially in simple scenesAs in the Irwin and Zelinsky (2002) study we monitorthe eye movements of observers as they freely inspect a

multiobject scene and we make the duration of this studyperiod contingent on their oculomotor behavior How-ever rather than terminating the study display after a setnumber of fixations we make testing contingent on thenumber of different objects fixated by observers aftertheir gaze leaves a predesignated target This online mon-itoring of the number of posttarget objects fixated by ob-servers allows us to parametrically manipulate the num-ber of objects viewed between study and test Rather thanrelying on post hoc fixation analyses to derive recencyand prerecency serial memory functions our manipula-tion of intervening objects as an independent variablemeans that our experimental design will have sufficientpower to discern the memory patterns in question We be-lieve that this more powerful design when combinedwith an object-based gaze-contingent methodology willprovide a clearer measure of object memory in scenes

MethodStimuli The stimuli consisted of nine common real-world objects

(toy tool or food items) arranged on an appropriate background sur-

Figure 1 Events comprising a typical trial in Experiment 1 (A) A nine-object scene was presented to observers Theirtask was to study these objects in anticipation of a memory test (B) Observers freely viewed the objects with the inten-tion of remembering their identities and locations eventually fixating the target object (in this case the baby bottle)(C) Unknown to the observer the display program would then count the number of objects fixated after gaze left thetarget (D) When the criterion number of different posttarget objects (in this case three) was fixated the multiobjectdisplay was replaced by a spatial probe at the targetrsquos location (E) The observer then had to select the target object fromamong four alternatives

A B

C

D E

680 ZELINSKY AND LOSCHKY

face (a crib workbench or dining table) Each simple scene sub-tended 18ordm 116ordm of visual angle and was of near-photographicquality (16-bit color)2 Individual objects were scaled to fit insidea 24ordm bounding box and their locations in the scene were con-strained to 18 positions creating what appeared to be a haphazardarrangement of items lying on a surface As a result of these place-ment constraints scene objects had a minimum and maximumcenter-to-center separation of 24ordm and 145ordm respectively Multipletrials for a given scene type were created by randomly pairing theobjects to locations (ie the same nine objects would occupy dif-ferent locations in each scene) with the constraints that no two dis-play configurations were repeated and that the memory target wouldappear equally often in each of the 18 allowable positions

Procedure The experimental procedure is shown in Figure 1 Ascene appeared at the start of a trial (Figure 1A) and the observersrsquotask was to remember the identity and location of every objectmdashinthis case toys in a babyrsquos crib (there were also tools on a workbenchand food items on a dining table) Given this formidable memorytask observers invariably made eye movements to the individualobjects a response that was anticipated and crucial to the currentparadigm but not explicitly mentioned in the experiment instruc-tions Eye position was monitored every millisecond and analyzedonline to determine the object in the scene being fixated by the ob-server Unknown to the observer one object in the display was pre-designated as the memory target for that particular trial (eg thebaby bottle in Figure 1) As the observer freely scanned the scenegaze would eventually be directed to this target object (Figure 1B)This fixation event was detected by the program controlling the ex-periment which then started to count the number of different ob-jects fixated after gaze left the target This ldquocountrdquo constituted thetermination criterion for the display meaning that if the count waspreset to three the observer would be allowed to fixate exactly threeobjects after the target (Figure 1C) As gaze moved away from thethird posttarget object the study scene was replaced by a 1-sec du-ration probe (a colored noise mask) appearing at the targetrsquos loca-tion on an ldquoemptiedrdquo background surface (Figure 1D) Followingthe probe a display showing four objects was presented and theobserver had to indicate which of them appeared at the probed lo-cation (Figure 1E)3 One of these objects was always the target theother three were randomly selected from the study scene Becauseobservers were on a bite-bar and could not easily speak they re-sponded by looking at the desired object causing a white box to bedrawn around the item then pressing a hand-held button when sat-isfied with their selection Observers were asked to respond as ac-curately as possible without regard for time

The independent variable of interest in this study was interven-ing objects the number of different objects looked at after the tar-get There were seven intervening object conditions (1ndash7) For ex-ample in the one-intervening-object condition the study displayterminated during the saccade away from the first posttarget object(ie gaze was not allowed to land on a second nontarget item afterleaving the target) Likewise in the seven-intervening-object con-dition the observer fixated exactly seven different nontarget ob-jects following target fixation The amount of time that observersviewed the study scene therefore varied it was determined by whenthey first fixated the target and the duration of their gaze on eachobject as well as the intervening object criterion set for that partic-ular trial The intervening object conditions were randomly inter-leaved throughout the experiment resulting in an unpredictablestudy scene duration Postexperiment questioning revealed that theparticipants were uniformly unaware that scene presentation timedepended on their own pattern of eye movements attributing thevariable interval instead to some random schedule of presentationdurations

Participants Six experimentally naive observers from the Uni-versity of Illinois at Urbana-Champaign each participated in 378trials 54 in each of the seven intervening object conditions Eye

position was monitored throughout each trial using a Generation Vdual-Purkinje-image (DPI) eyetracker sampling at 1000 Hz Thespatial precision of this tracker was better than 3 min of visual angleat a viewing distance of 62 cm Saccadic eye movements were de-termined online using a velocity-based algorithm implementing aroughly 125ordmsec detection threshold The experiment required two15-h sessions conducted on separate days and the participantswere paid $24 upon completion

Results and DiscussionOur gaze-contingent memory paradigm allows us to

plot response accuracy as a function of intervening ob-jects for trials in which the target was fixated only onceduring viewing (ie no target refixations) This analysisshown in Figure 2A by the solid markers reveals twoclear patterns of behavior First memory declined pre-cipitously between the 1 and 3 intervening object condi-tions dropping linearly from an 87 level of accuracy toonly 65 [F(630) 653 MSe 0044 p 001] Thispattern suggests that the act of fixating the first three ob-jects after the target dramatically interfered with mem-ory for the target objectmdashmuch like the interference ob-served when objects are presented one after another in asequential memory paradigm Second this drop in accu-racy reached an abrupt asymptote at three interveningobjects meaning that item fixations over the 4ndash7 interven-ing object range resulted in no additional object-relatedinterference Moreover because chance response in this4AFC task should produce a 25 level of accuracy the65 level described by this asymptote indicates a clearlyabove-chance and relatively good memory for an objectirrespective of the number of items fixated after it duringstudy4 We refer to the region of declining accuracy in thisintervening object function as the recency memory com-ponent and the above-chance asymptotic region of accu-racy as the prerecency memory component

Our clear evidence for distinct recency and prerecencybehavior although consistent with the data from Irwinand Zelinsky (2002) is inconsistent with Hollingworthand Hendersonrsquos (2002) failure to observe a recency ef-fect To determine whether this discrepancy is due to ourquantification of serial order in terms of intervening ob-jects rather than intervening fixations we reanalyzed ourdata using Hollingworth and Hendersonrsquos interveningfixation method (the open markers in Figure 2A) Thiscomparison revealed considerable agreement betweenthe two functions with mean differences in accuracyamounting to only 255 This high degree of similaritywould be expected if observers tended to devote only asingle fixation to each object during study More impor-tant accuracy again declined with the number of fixa-tions occurring after gaze left the target [F(630) 493MSe 7738 p 001] Note however that this recencyeffect is less pronounced than that in the intervening ob-ject data We attribute this pattern to a special role thatobjects play in producing recency behavior Differencesbetween the intervening object and intervening fixationfunctions are caused by the occasional multiple fixationof individual objects during study Given that these fix-

MEMORY FOR OBJECTS IN SCENES 681

ations on the same object were less detrimental to mem-ory than were fixations on different objects (as indicatedby the slightly better accuracy in the intervening fixationdata) it appears that objects not fixations are primarilyresponsible for the accuracy decline that defines recencyin our task Moreover if one were to replot such an object-based recency effect as a function of intervening fixa-tions the result should be a stretching of the object-basedrecency effect over more intervening events (ie fixa-tions) which is the exact pattern appearing in Figure 2AThis dilution of recency in the intervening fixation datasuggests one factor contributing to Hollingworth andHendersonrsquos (2002) failure to find a recency effect in theirstudy As recency is stretched over more of the interven-ing event function the slope becomes shallower and theeffect more difficult to observe For this reason we be-

lieve that the preferred method of quantifying recencyduring scene viewing is to serialize the data by objectsrather than by fixations

As a working interpretation of our data we hypothe-size that retroactive interference (RI) is a likely cause ofthe observed recency effect Consistent with a classic RIexplanation for STM forgetting (Waugh amp Norman 1965see also Intraub 1984 for RI applied to scenes) each ob-ject fixated after the target in our task might have inter-fered with memory for the target eventually driving recog-nition performance into the prerecency baseline Howeverinterference might be proactive as well as retroactive (Kep-pel amp Underwood 1962 Wickens Born amp Allen 1963)and given that a relatively small set of objects was beingreused in each trial of this experiment the potential forproactive interference (PI) influencing the data wouldseem high Previous research using a serial presentationparadigm has found that between-trial PI effects on pic-ture recognition memory are greatest early in an experi-ment rather than late (Jitsumori Wright amp Shyan 1989Wright 1998) particularly under conditions of immedi-ate testing (eg a 1-sec retention interval) It might thenbe the case that observers were able to accurately re-member the target over a large intervening object rangeearly in the experiment but as a result of PI accumulat-ing over trials this high level of accuracy gradually de-clined in the later trials to chance performance If theforgetting function was changing during the course ofthe experiment the recency and prerecency componentsin Figure 2A might be mere artifacts of averaging overtrials To explore this possibility we evenly divided the378 trials into ldquoearlyrdquo and ldquolaterdquo conditions If PI wasbuilding up throughout the experiment prerecency accu-racy in the early-trial condition should be higher than inthe late-trial condition However as can be seen from Fig-ure 3 this was clearly not the case If anything accuracyactually improved somewhat over the 3ndash7 intervening ob-ject range in the late trials Rather than suffering from abuildup of PI observers benefited slightly from theirgrowing familiarity with the set of stimulus objects

Although between-trial PI appears not to have playeda role in producing the Figure 2A intervening objectfunction there still exists the possibility that anotherform of PI might have influenced our data Given ourfree-viewing paradigm the number of nontarget objectsfixated before gaze first shifted to the target necessarilyvaried with the observersrsquo own idiosyncratic scanningpreferences If observers happened to inspect more pre-target objects in the 3ndash7 intervening object conditionsand if the fixation of these objects interfered with targetmemory the buildup of PI within a trial might result in thelower accuracy observed in the prerecency component

To evaluate this potential for within-trial PI we firsttested the premise that pretarget fixations were less fre-quent in the 1ndash2 intervening object conditions comparedwith the 3ndash7 intervening object conditions The resultsfrom this analysis are shown in Table 1 As expected onthe basis of our random selection of targets the number

Figure 2 Memory forgetting functions relating accuracy to thenumber of different events following viewing of the target Linesextending from the markers indicate one SEM (A) Data from Ex-periment 1 in which a single nine-object study scene was pre-sented on each trial and memory accuracy was defined in termsof either the number of intervening objects fixated between tar-get and test or the number of intervening fixations (B) Data fromExperiment 2 in which each trial consisted of nine one-objectstudy displays presented sequentially over time Square markersindicate data from a 1000ndash0 condition in which individual objectswere presented for 1 sec each without an interstimulus interval(ISI) Data from the 500ndash500 (500-msec presentation 500-msecISI) and 500ndash0 (500-msec presentation 0-msec ISI) conditions areshown by the triangle and circle markers respectively

682 ZELINSKY AND LOSCHKY

of distractor objects fixated before the target in a trialdid not meaningfully differ between the 1ndash2 and the 3ndash7intervening object conditions [Pearson χ 2 (8) 325p 918 N 1457] Our failure to find an overrepre-sentation of pretarget distractor fixations in the 3ndash7 in-tervening object trials therefore weakens the argumentthat within-trial PI could account for our recency dataWe also directly looked for a within-trial PI effect bycomparing trials in which the target was fixated veryearly in the sequence of fixations (with 0 1 or 2 objectsfixated before the target) with trials in which the targetwas first fixated later (three or more objects fixated be-fore the target) If within-trial PI played a role in deter-mining the recency effect accuracy should be higherwhen there were fewer pretarget fixations Table 2 indi-cates that this too was not the case Trials in which thetarget was fixated very early (ie few pretarget fixations)were no more accurate than trials in which the target wasfixated much later (ie several pretarget fixations) Wecan therefore conclude that pretarget fixations and what-ever within-trial PI they may have exerted cannot explainthe recency effects reported in Figure 2A

Unresolved QuestionsSince we dispatched PI as a factor shaping the Fig-

ure 2A forgetting functions an explanation appealing toRI becomes more attractive However two remaining al-

ternative explanations cannot be ruled out based on theExperiment 1 data First it is possible that the recency be-havior reported in Figure 2A reflects a decay processrather than RI from objects fixated after the target Decay-based explanations for STM forgetting enjoyed consider-able popularity in the early days of modern memory re-search (eg Peterson amp Peterson 1959 see also Altmannamp Gray 2002 for a recent formulation of this view) butwere largely replaced by interference-based explanationsafter several studies demonstrated little or no forgettingover time when item interference was controlled (Loess ampWaugh 1967 Waugh amp Norman 1965 Wickens et al1963) Although the latter studies certainly reduce theplausibility that decay was responsible for the currentlyobserved recency behavior this factor cannot be ruledout for the specific presentation and testing conditionsused in Experiment 1 A second factor potentially af-fecting the Figure 2A forgetting data is specific to the si-multaneous presentation paradigm Our use of eyemovements to serialize object processing assumes thatprocessing is restricted to only the currently fixated ob-ject during free viewing Several studies however havealso demonstrated processing benefits for extrafoveallyviewed objects (Henderson amp Anes 1994 PollatsekRayner amp Henderson 1990) If multiple objects werebeing processed during each display fixation this oppor-tunity for extrafoveal processing of the target might ex-plain the above-chance level of accuracy observed in theFigure 2A prerecency data Although such extrafovealprocessing benefits have been reported only for the in-tended target of a saccade and not every object in a dis-play (Henderson Pollatsek amp Rayner 1989) the poten-tial for a more generalized form of extrafoveal processingbenefit affecting our data should be considered

EXPERIMENT 2

Experiment 2 directly addressed the possibilities thatdecay or extrafoveal processes may have influenced theExperiment 1 data The question of whether decay or in-terference underlies the Figure 2A recency effect was re-solved by holding constant the number of interveningobjects appearing between study and test but varying theretention interval If decay was responsible for the ob-served decline in accuracy lengthening the retention in-terval should accelerate the accuracy decline and con-strict the expression of recency on the forgetting function

Table 1Percentage of Trials With a Given Number of Pretarget Object Fixations

(0ndash8) as a Function of Two Intervening Object Groups (1ndash2 vs 3ndash7)

Number of Pretarget Objects Fixated

Number of Intervening Objects 0 1 2 3 4 5 6 7 8

1ndash2 112 134 112 125 90 112 103 106 1063ndash7 103 113 124 120 102 104 109 106 120

NotemdashData exclude trials in which the target object was refixated n = 554 and 903cases for the 1ndash2 and 3ndash7 intervening object groupings respectively

Figure 3 Intervening object functions derived from an early-versus-late segregation of the Experiment 1 data Open markersshow data from the first 189 trials of the experiment closedmarkers show data from the final 189 trials

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

680 ZELINSKY AND LOSCHKY

face (a crib workbench or dining table) Each simple scene sub-tended 18ordm 116ordm of visual angle and was of near-photographicquality (16-bit color)2 Individual objects were scaled to fit insidea 24ordm bounding box and their locations in the scene were con-strained to 18 positions creating what appeared to be a haphazardarrangement of items lying on a surface As a result of these place-ment constraints scene objects had a minimum and maximumcenter-to-center separation of 24ordm and 145ordm respectively Multipletrials for a given scene type were created by randomly pairing theobjects to locations (ie the same nine objects would occupy dif-ferent locations in each scene) with the constraints that no two dis-play configurations were repeated and that the memory target wouldappear equally often in each of the 18 allowable positions

Procedure The experimental procedure is shown in Figure 1 Ascene appeared at the start of a trial (Figure 1A) and the observersrsquotask was to remember the identity and location of every objectmdashinthis case toys in a babyrsquos crib (there were also tools on a workbenchand food items on a dining table) Given this formidable memorytask observers invariably made eye movements to the individualobjects a response that was anticipated and crucial to the currentparadigm but not explicitly mentioned in the experiment instruc-tions Eye position was monitored every millisecond and analyzedonline to determine the object in the scene being fixated by the ob-server Unknown to the observer one object in the display was pre-designated as the memory target for that particular trial (eg thebaby bottle in Figure 1) As the observer freely scanned the scenegaze would eventually be directed to this target object (Figure 1B)This fixation event was detected by the program controlling the ex-periment which then started to count the number of different ob-jects fixated after gaze left the target This ldquocountrdquo constituted thetermination criterion for the display meaning that if the count waspreset to three the observer would be allowed to fixate exactly threeobjects after the target (Figure 1C) As gaze moved away from thethird posttarget object the study scene was replaced by a 1-sec du-ration probe (a colored noise mask) appearing at the targetrsquos loca-tion on an ldquoemptiedrdquo background surface (Figure 1D) Followingthe probe a display showing four objects was presented and theobserver had to indicate which of them appeared at the probed lo-cation (Figure 1E)3 One of these objects was always the target theother three were randomly selected from the study scene Becauseobservers were on a bite-bar and could not easily speak they re-sponded by looking at the desired object causing a white box to bedrawn around the item then pressing a hand-held button when sat-isfied with their selection Observers were asked to respond as ac-curately as possible without regard for time

The independent variable of interest in this study was interven-ing objects the number of different objects looked at after the tar-get There were seven intervening object conditions (1ndash7) For ex-ample in the one-intervening-object condition the study displayterminated during the saccade away from the first posttarget object(ie gaze was not allowed to land on a second nontarget item afterleaving the target) Likewise in the seven-intervening-object con-dition the observer fixated exactly seven different nontarget ob-jects following target fixation The amount of time that observersviewed the study scene therefore varied it was determined by whenthey first fixated the target and the duration of their gaze on eachobject as well as the intervening object criterion set for that partic-ular trial The intervening object conditions were randomly inter-leaved throughout the experiment resulting in an unpredictablestudy scene duration Postexperiment questioning revealed that theparticipants were uniformly unaware that scene presentation timedepended on their own pattern of eye movements attributing thevariable interval instead to some random schedule of presentationdurations

Participants Six experimentally naive observers from the Uni-versity of Illinois at Urbana-Champaign each participated in 378trials 54 in each of the seven intervening object conditions Eye

position was monitored throughout each trial using a Generation Vdual-Purkinje-image (DPI) eyetracker sampling at 1000 Hz Thespatial precision of this tracker was better than 3 min of visual angleat a viewing distance of 62 cm Saccadic eye movements were de-termined online using a velocity-based algorithm implementing aroughly 125ordmsec detection threshold The experiment required two15-h sessions conducted on separate days and the participantswere paid $24 upon completion

Results and DiscussionOur gaze-contingent memory paradigm allows us to

plot response accuracy as a function of intervening ob-jects for trials in which the target was fixated only onceduring viewing (ie no target refixations) This analysisshown in Figure 2A by the solid markers reveals twoclear patterns of behavior First memory declined pre-cipitously between the 1 and 3 intervening object condi-tions dropping linearly from an 87 level of accuracy toonly 65 [F(630) 653 MSe 0044 p 001] Thispattern suggests that the act of fixating the first three ob-jects after the target dramatically interfered with mem-ory for the target objectmdashmuch like the interference ob-served when objects are presented one after another in asequential memory paradigm Second this drop in accu-racy reached an abrupt asymptote at three interveningobjects meaning that item fixations over the 4ndash7 interven-ing object range resulted in no additional object-relatedinterference Moreover because chance response in this4AFC task should produce a 25 level of accuracy the65 level described by this asymptote indicates a clearlyabove-chance and relatively good memory for an objectirrespective of the number of items fixated after it duringstudy4 We refer to the region of declining accuracy in thisintervening object function as the recency memory com-ponent and the above-chance asymptotic region of accu-racy as the prerecency memory component

Our clear evidence for distinct recency and prerecencybehavior although consistent with the data from Irwinand Zelinsky (2002) is inconsistent with Hollingworthand Hendersonrsquos (2002) failure to observe a recency ef-fect To determine whether this discrepancy is due to ourquantification of serial order in terms of intervening ob-jects rather than intervening fixations we reanalyzed ourdata using Hollingworth and Hendersonrsquos interveningfixation method (the open markers in Figure 2A) Thiscomparison revealed considerable agreement betweenthe two functions with mean differences in accuracyamounting to only 255 This high degree of similaritywould be expected if observers tended to devote only asingle fixation to each object during study More impor-tant accuracy again declined with the number of fixa-tions occurring after gaze left the target [F(630) 493MSe 7738 p 001] Note however that this recencyeffect is less pronounced than that in the intervening ob-ject data We attribute this pattern to a special role thatobjects play in producing recency behavior Differencesbetween the intervening object and intervening fixationfunctions are caused by the occasional multiple fixationof individual objects during study Given that these fix-

MEMORY FOR OBJECTS IN SCENES 681

ations on the same object were less detrimental to mem-ory than were fixations on different objects (as indicatedby the slightly better accuracy in the intervening fixationdata) it appears that objects not fixations are primarilyresponsible for the accuracy decline that defines recencyin our task Moreover if one were to replot such an object-based recency effect as a function of intervening fixa-tions the result should be a stretching of the object-basedrecency effect over more intervening events (ie fixa-tions) which is the exact pattern appearing in Figure 2AThis dilution of recency in the intervening fixation datasuggests one factor contributing to Hollingworth andHendersonrsquos (2002) failure to find a recency effect in theirstudy As recency is stretched over more of the interven-ing event function the slope becomes shallower and theeffect more difficult to observe For this reason we be-

lieve that the preferred method of quantifying recencyduring scene viewing is to serialize the data by objectsrather than by fixations

As a working interpretation of our data we hypothe-size that retroactive interference (RI) is a likely cause ofthe observed recency effect Consistent with a classic RIexplanation for STM forgetting (Waugh amp Norman 1965see also Intraub 1984 for RI applied to scenes) each ob-ject fixated after the target in our task might have inter-fered with memory for the target eventually driving recog-nition performance into the prerecency baseline Howeverinterference might be proactive as well as retroactive (Kep-pel amp Underwood 1962 Wickens Born amp Allen 1963)and given that a relatively small set of objects was beingreused in each trial of this experiment the potential forproactive interference (PI) influencing the data wouldseem high Previous research using a serial presentationparadigm has found that between-trial PI effects on pic-ture recognition memory are greatest early in an experi-ment rather than late (Jitsumori Wright amp Shyan 1989Wright 1998) particularly under conditions of immedi-ate testing (eg a 1-sec retention interval) It might thenbe the case that observers were able to accurately re-member the target over a large intervening object rangeearly in the experiment but as a result of PI accumulat-ing over trials this high level of accuracy gradually de-clined in the later trials to chance performance If theforgetting function was changing during the course ofthe experiment the recency and prerecency componentsin Figure 2A might be mere artifacts of averaging overtrials To explore this possibility we evenly divided the378 trials into ldquoearlyrdquo and ldquolaterdquo conditions If PI wasbuilding up throughout the experiment prerecency accu-racy in the early-trial condition should be higher than inthe late-trial condition However as can be seen from Fig-ure 3 this was clearly not the case If anything accuracyactually improved somewhat over the 3ndash7 intervening ob-ject range in the late trials Rather than suffering from abuildup of PI observers benefited slightly from theirgrowing familiarity with the set of stimulus objects

Although between-trial PI appears not to have playeda role in producing the Figure 2A intervening objectfunction there still exists the possibility that anotherform of PI might have influenced our data Given ourfree-viewing paradigm the number of nontarget objectsfixated before gaze first shifted to the target necessarilyvaried with the observersrsquo own idiosyncratic scanningpreferences If observers happened to inspect more pre-target objects in the 3ndash7 intervening object conditionsand if the fixation of these objects interfered with targetmemory the buildup of PI within a trial might result in thelower accuracy observed in the prerecency component

To evaluate this potential for within-trial PI we firsttested the premise that pretarget fixations were less fre-quent in the 1ndash2 intervening object conditions comparedwith the 3ndash7 intervening object conditions The resultsfrom this analysis are shown in Table 1 As expected onthe basis of our random selection of targets the number

Figure 2 Memory forgetting functions relating accuracy to thenumber of different events following viewing of the target Linesextending from the markers indicate one SEM (A) Data from Ex-periment 1 in which a single nine-object study scene was pre-sented on each trial and memory accuracy was defined in termsof either the number of intervening objects fixated between tar-get and test or the number of intervening fixations (B) Data fromExperiment 2 in which each trial consisted of nine one-objectstudy displays presented sequentially over time Square markersindicate data from a 1000ndash0 condition in which individual objectswere presented for 1 sec each without an interstimulus interval(ISI) Data from the 500ndash500 (500-msec presentation 500-msecISI) and 500ndash0 (500-msec presentation 0-msec ISI) conditions areshown by the triangle and circle markers respectively

682 ZELINSKY AND LOSCHKY

of distractor objects fixated before the target in a trialdid not meaningfully differ between the 1ndash2 and the 3ndash7intervening object conditions [Pearson χ 2 (8) 325p 918 N 1457] Our failure to find an overrepre-sentation of pretarget distractor fixations in the 3ndash7 in-tervening object trials therefore weakens the argumentthat within-trial PI could account for our recency dataWe also directly looked for a within-trial PI effect bycomparing trials in which the target was fixated veryearly in the sequence of fixations (with 0 1 or 2 objectsfixated before the target) with trials in which the targetwas first fixated later (three or more objects fixated be-fore the target) If within-trial PI played a role in deter-mining the recency effect accuracy should be higherwhen there were fewer pretarget fixations Table 2 indi-cates that this too was not the case Trials in which thetarget was fixated very early (ie few pretarget fixations)were no more accurate than trials in which the target wasfixated much later (ie several pretarget fixations) Wecan therefore conclude that pretarget fixations and what-ever within-trial PI they may have exerted cannot explainthe recency effects reported in Figure 2A

Unresolved QuestionsSince we dispatched PI as a factor shaping the Fig-

ure 2A forgetting functions an explanation appealing toRI becomes more attractive However two remaining al-

ternative explanations cannot be ruled out based on theExperiment 1 data First it is possible that the recency be-havior reported in Figure 2A reflects a decay processrather than RI from objects fixated after the target Decay-based explanations for STM forgetting enjoyed consider-able popularity in the early days of modern memory re-search (eg Peterson amp Peterson 1959 see also Altmannamp Gray 2002 for a recent formulation of this view) butwere largely replaced by interference-based explanationsafter several studies demonstrated little or no forgettingover time when item interference was controlled (Loess ampWaugh 1967 Waugh amp Norman 1965 Wickens et al1963) Although the latter studies certainly reduce theplausibility that decay was responsible for the currentlyobserved recency behavior this factor cannot be ruledout for the specific presentation and testing conditionsused in Experiment 1 A second factor potentially af-fecting the Figure 2A forgetting data is specific to the si-multaneous presentation paradigm Our use of eyemovements to serialize object processing assumes thatprocessing is restricted to only the currently fixated ob-ject during free viewing Several studies however havealso demonstrated processing benefits for extrafoveallyviewed objects (Henderson amp Anes 1994 PollatsekRayner amp Henderson 1990) If multiple objects werebeing processed during each display fixation this oppor-tunity for extrafoveal processing of the target might ex-plain the above-chance level of accuracy observed in theFigure 2A prerecency data Although such extrafovealprocessing benefits have been reported only for the in-tended target of a saccade and not every object in a dis-play (Henderson Pollatsek amp Rayner 1989) the poten-tial for a more generalized form of extrafoveal processingbenefit affecting our data should be considered

EXPERIMENT 2

Experiment 2 directly addressed the possibilities thatdecay or extrafoveal processes may have influenced theExperiment 1 data The question of whether decay or in-terference underlies the Figure 2A recency effect was re-solved by holding constant the number of interveningobjects appearing between study and test but varying theretention interval If decay was responsible for the ob-served decline in accuracy lengthening the retention in-terval should accelerate the accuracy decline and con-strict the expression of recency on the forgetting function

Table 1Percentage of Trials With a Given Number of Pretarget Object Fixations

(0ndash8) as a Function of Two Intervening Object Groups (1ndash2 vs 3ndash7)

Number of Pretarget Objects Fixated

Number of Intervening Objects 0 1 2 3 4 5 6 7 8

1ndash2 112 134 112 125 90 112 103 106 1063ndash7 103 113 124 120 102 104 109 106 120

NotemdashData exclude trials in which the target object was refixated n = 554 and 903cases for the 1ndash2 and 3ndash7 intervening object groupings respectively

Figure 3 Intervening object functions derived from an early-versus-late segregation of the Experiment 1 data Open markersshow data from the first 189 trials of the experiment closedmarkers show data from the final 189 trials

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

MEMORY FOR OBJECTS IN SCENES 681

ations on the same object were less detrimental to mem-ory than were fixations on different objects (as indicatedby the slightly better accuracy in the intervening fixationdata) it appears that objects not fixations are primarilyresponsible for the accuracy decline that defines recencyin our task Moreover if one were to replot such an object-based recency effect as a function of intervening fixa-tions the result should be a stretching of the object-basedrecency effect over more intervening events (ie fixa-tions) which is the exact pattern appearing in Figure 2AThis dilution of recency in the intervening fixation datasuggests one factor contributing to Hollingworth andHendersonrsquos (2002) failure to find a recency effect in theirstudy As recency is stretched over more of the interven-ing event function the slope becomes shallower and theeffect more difficult to observe For this reason we be-

lieve that the preferred method of quantifying recencyduring scene viewing is to serialize the data by objectsrather than by fixations

As a working interpretation of our data we hypothe-size that retroactive interference (RI) is a likely cause ofthe observed recency effect Consistent with a classic RIexplanation for STM forgetting (Waugh amp Norman 1965see also Intraub 1984 for RI applied to scenes) each ob-ject fixated after the target in our task might have inter-fered with memory for the target eventually driving recog-nition performance into the prerecency baseline Howeverinterference might be proactive as well as retroactive (Kep-pel amp Underwood 1962 Wickens Born amp Allen 1963)and given that a relatively small set of objects was beingreused in each trial of this experiment the potential forproactive interference (PI) influencing the data wouldseem high Previous research using a serial presentationparadigm has found that between-trial PI effects on pic-ture recognition memory are greatest early in an experi-ment rather than late (Jitsumori Wright amp Shyan 1989Wright 1998) particularly under conditions of immedi-ate testing (eg a 1-sec retention interval) It might thenbe the case that observers were able to accurately re-member the target over a large intervening object rangeearly in the experiment but as a result of PI accumulat-ing over trials this high level of accuracy gradually de-clined in the later trials to chance performance If theforgetting function was changing during the course ofthe experiment the recency and prerecency componentsin Figure 2A might be mere artifacts of averaging overtrials To explore this possibility we evenly divided the378 trials into ldquoearlyrdquo and ldquolaterdquo conditions If PI wasbuilding up throughout the experiment prerecency accu-racy in the early-trial condition should be higher than inthe late-trial condition However as can be seen from Fig-ure 3 this was clearly not the case If anything accuracyactually improved somewhat over the 3ndash7 intervening ob-ject range in the late trials Rather than suffering from abuildup of PI observers benefited slightly from theirgrowing familiarity with the set of stimulus objects

Although between-trial PI appears not to have playeda role in producing the Figure 2A intervening objectfunction there still exists the possibility that anotherform of PI might have influenced our data Given ourfree-viewing paradigm the number of nontarget objectsfixated before gaze first shifted to the target necessarilyvaried with the observersrsquo own idiosyncratic scanningpreferences If observers happened to inspect more pre-target objects in the 3ndash7 intervening object conditionsand if the fixation of these objects interfered with targetmemory the buildup of PI within a trial might result in thelower accuracy observed in the prerecency component

To evaluate this potential for within-trial PI we firsttested the premise that pretarget fixations were less fre-quent in the 1ndash2 intervening object conditions comparedwith the 3ndash7 intervening object conditions The resultsfrom this analysis are shown in Table 1 As expected onthe basis of our random selection of targets the number

Figure 2 Memory forgetting functions relating accuracy to thenumber of different events following viewing of the target Linesextending from the markers indicate one SEM (A) Data from Ex-periment 1 in which a single nine-object study scene was pre-sented on each trial and memory accuracy was defined in termsof either the number of intervening objects fixated between tar-get and test or the number of intervening fixations (B) Data fromExperiment 2 in which each trial consisted of nine one-objectstudy displays presented sequentially over time Square markersindicate data from a 1000ndash0 condition in which individual objectswere presented for 1 sec each without an interstimulus interval(ISI) Data from the 500ndash500 (500-msec presentation 500-msecISI) and 500ndash0 (500-msec presentation 0-msec ISI) conditions areshown by the triangle and circle markers respectively

682 ZELINSKY AND LOSCHKY

of distractor objects fixated before the target in a trialdid not meaningfully differ between the 1ndash2 and the 3ndash7intervening object conditions [Pearson χ 2 (8) 325p 918 N 1457] Our failure to find an overrepre-sentation of pretarget distractor fixations in the 3ndash7 in-tervening object trials therefore weakens the argumentthat within-trial PI could account for our recency dataWe also directly looked for a within-trial PI effect bycomparing trials in which the target was fixated veryearly in the sequence of fixations (with 0 1 or 2 objectsfixated before the target) with trials in which the targetwas first fixated later (three or more objects fixated be-fore the target) If within-trial PI played a role in deter-mining the recency effect accuracy should be higherwhen there were fewer pretarget fixations Table 2 indi-cates that this too was not the case Trials in which thetarget was fixated very early (ie few pretarget fixations)were no more accurate than trials in which the target wasfixated much later (ie several pretarget fixations) Wecan therefore conclude that pretarget fixations and what-ever within-trial PI they may have exerted cannot explainthe recency effects reported in Figure 2A

Unresolved QuestionsSince we dispatched PI as a factor shaping the Fig-

ure 2A forgetting functions an explanation appealing toRI becomes more attractive However two remaining al-

ternative explanations cannot be ruled out based on theExperiment 1 data First it is possible that the recency be-havior reported in Figure 2A reflects a decay processrather than RI from objects fixated after the target Decay-based explanations for STM forgetting enjoyed consider-able popularity in the early days of modern memory re-search (eg Peterson amp Peterson 1959 see also Altmannamp Gray 2002 for a recent formulation of this view) butwere largely replaced by interference-based explanationsafter several studies demonstrated little or no forgettingover time when item interference was controlled (Loess ampWaugh 1967 Waugh amp Norman 1965 Wickens et al1963) Although the latter studies certainly reduce theplausibility that decay was responsible for the currentlyobserved recency behavior this factor cannot be ruledout for the specific presentation and testing conditionsused in Experiment 1 A second factor potentially af-fecting the Figure 2A forgetting data is specific to the si-multaneous presentation paradigm Our use of eyemovements to serialize object processing assumes thatprocessing is restricted to only the currently fixated ob-ject during free viewing Several studies however havealso demonstrated processing benefits for extrafoveallyviewed objects (Henderson amp Anes 1994 PollatsekRayner amp Henderson 1990) If multiple objects werebeing processed during each display fixation this oppor-tunity for extrafoveal processing of the target might ex-plain the above-chance level of accuracy observed in theFigure 2A prerecency data Although such extrafovealprocessing benefits have been reported only for the in-tended target of a saccade and not every object in a dis-play (Henderson Pollatsek amp Rayner 1989) the poten-tial for a more generalized form of extrafoveal processingbenefit affecting our data should be considered

EXPERIMENT 2

Experiment 2 directly addressed the possibilities thatdecay or extrafoveal processes may have influenced theExperiment 1 data The question of whether decay or in-terference underlies the Figure 2A recency effect was re-solved by holding constant the number of interveningobjects appearing between study and test but varying theretention interval If decay was responsible for the ob-served decline in accuracy lengthening the retention in-terval should accelerate the accuracy decline and con-strict the expression of recency on the forgetting function

Table 1Percentage of Trials With a Given Number of Pretarget Object Fixations

(0ndash8) as a Function of Two Intervening Object Groups (1ndash2 vs 3ndash7)

Number of Pretarget Objects Fixated

Number of Intervening Objects 0 1 2 3 4 5 6 7 8

1ndash2 112 134 112 125 90 112 103 106 1063ndash7 103 113 124 120 102 104 109 106 120

NotemdashData exclude trials in which the target object was refixated n = 554 and 903cases for the 1ndash2 and 3ndash7 intervening object groupings respectively

Figure 3 Intervening object functions derived from an early-versus-late segregation of the Experiment 1 data Open markersshow data from the first 189 trials of the experiment closedmarkers show data from the final 189 trials

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

682 ZELINSKY AND LOSCHKY

of distractor objects fixated before the target in a trialdid not meaningfully differ between the 1ndash2 and the 3ndash7intervening object conditions [Pearson χ 2 (8) 325p 918 N 1457] Our failure to find an overrepre-sentation of pretarget distractor fixations in the 3ndash7 in-tervening object trials therefore weakens the argumentthat within-trial PI could account for our recency dataWe also directly looked for a within-trial PI effect bycomparing trials in which the target was fixated veryearly in the sequence of fixations (with 0 1 or 2 objectsfixated before the target) with trials in which the targetwas first fixated later (three or more objects fixated be-fore the target) If within-trial PI played a role in deter-mining the recency effect accuracy should be higherwhen there were fewer pretarget fixations Table 2 indi-cates that this too was not the case Trials in which thetarget was fixated very early (ie few pretarget fixations)were no more accurate than trials in which the target wasfixated much later (ie several pretarget fixations) Wecan therefore conclude that pretarget fixations and what-ever within-trial PI they may have exerted cannot explainthe recency effects reported in Figure 2A

Unresolved QuestionsSince we dispatched PI as a factor shaping the Fig-

ure 2A forgetting functions an explanation appealing toRI becomes more attractive However two remaining al-

ternative explanations cannot be ruled out based on theExperiment 1 data First it is possible that the recency be-havior reported in Figure 2A reflects a decay processrather than RI from objects fixated after the target Decay-based explanations for STM forgetting enjoyed consider-able popularity in the early days of modern memory re-search (eg Peterson amp Peterson 1959 see also Altmannamp Gray 2002 for a recent formulation of this view) butwere largely replaced by interference-based explanationsafter several studies demonstrated little or no forgettingover time when item interference was controlled (Loess ampWaugh 1967 Waugh amp Norman 1965 Wickens et al1963) Although the latter studies certainly reduce theplausibility that decay was responsible for the currentlyobserved recency behavior this factor cannot be ruledout for the specific presentation and testing conditionsused in Experiment 1 A second factor potentially af-fecting the Figure 2A forgetting data is specific to the si-multaneous presentation paradigm Our use of eyemovements to serialize object processing assumes thatprocessing is restricted to only the currently fixated ob-ject during free viewing Several studies however havealso demonstrated processing benefits for extrafoveallyviewed objects (Henderson amp Anes 1994 PollatsekRayner amp Henderson 1990) If multiple objects werebeing processed during each display fixation this oppor-tunity for extrafoveal processing of the target might ex-plain the above-chance level of accuracy observed in theFigure 2A prerecency data Although such extrafovealprocessing benefits have been reported only for the in-tended target of a saccade and not every object in a dis-play (Henderson Pollatsek amp Rayner 1989) the poten-tial for a more generalized form of extrafoveal processingbenefit affecting our data should be considered

EXPERIMENT 2

Experiment 2 directly addressed the possibilities thatdecay or extrafoveal processes may have influenced theExperiment 1 data The question of whether decay or in-terference underlies the Figure 2A recency effect was re-solved by holding constant the number of interveningobjects appearing between study and test but varying theretention interval If decay was responsible for the ob-served decline in accuracy lengthening the retention in-terval should accelerate the accuracy decline and con-strict the expression of recency on the forgetting function

Table 1Percentage of Trials With a Given Number of Pretarget Object Fixations

(0ndash8) as a Function of Two Intervening Object Groups (1ndash2 vs 3ndash7)

Number of Pretarget Objects Fixated

Number of Intervening Objects 0 1 2 3 4 5 6 7 8

1ndash2 112 134 112 125 90 112 103 106 1063ndash7 103 113 124 120 102 104 109 106 120

NotemdashData exclude trials in which the target object was refixated n = 554 and 903cases for the 1ndash2 and 3ndash7 intervening object groupings respectively

Figure 3 Intervening object functions derived from an early-versus-late segregation of the Experiment 1 data Open markersshow data from the first 189 trials of the experiment closedmarkers show data from the final 189 trials

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

MEMORY FOR OBJECTS IN SCENES 683

However if our recency effect was caused by interferencevarying the retention interval should have no meaningfuleffect on accuracy To address the question of whether ex-trafoveal processing was contributing to prerecency per-formance in Experiment 1 we adopted in Experiment 2 astandard sequential presentation methodology If objectsare presented one after the other in time rather than simul-taneously as scenes there could be no opportunity for ex-trafoveal processing to affect memory performance Bycomparing prerecency accuracy between Experiments 1and 2 we can therefore assess any extrafoveal influences

MethodUnlike Experiment 1 in which we inferred serial encoding from

the observersrsquo pattern of eye movements to the objects Experi-ment 2 more closely replicated past object memory studies by usinga sequential presentation paradigm Although we used the samereal-world objects and backgrounds in Experiment 2 as in Experi-ment 1 a sequential paradigm required that each of these objectsnow be presented separately over time (Figure 4) Instead of show-ing observers a single multiobject display we decomposed each Ex-periment 1 display and presented the component objects singly oneafter the other Because a nine-object scene was always presentedin each Experiment 1 trial for each trial in Experiment 2 we pre-sented a sequence of nine single-object displays The identity andlocation of these objects were identical to a counterpart scene froma trial in Experiment 1 Thus if the nine Experiment 2 displays fora given trial were superimposed the corresponding Experiment 1scene would be obtained The presentation duration of each single-object display and the interstimulus interval (ISI) between the off-set of one object and the onset of the next were independent vari-ables manipulated in three conditions In the 500ndash0 condition eachobject was presented for 500 msec followed immediately (0-msecISI) by the next object in the sequence the 500ndash500 conditionshowed each object for 500 msec followed by a 500-msec ISIshowing only the ldquoemptyrdquo background scene surface and the1000ndash0 condition showed each object for a full second without anaccompanying ISI6 Thirty-six observers from Stony Brook Uni-versity 12 in each of the above three conditions participated for

Table 2Memory Accuracy as a Function of the Number

of Objects Fixated Before the Target

XX Objects Fixated Correct SEM XX

0 703 8051 712 5682 727 434

XX 3+ 708 383 XX

NotemdashData in the 0 row indicate cases in which the target was fixatedby the first saccade following display onset SEM standard error ofthe mean

Figure 4 Events composing a typical trial in Experiment 2 (A) A sequence of nine one-object scenes was presentedto observers (B) Following the presentation of this sequence a spatial probe appeared at the target objectrsquos location(C) The observer then had to select the target from among four alternatives

A

B C

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

684 ZELINSKY AND LOSCHKY

course credit There were 378 trials per observer (as in Experi-ment 1) which were completed in one 25-h session Because eyemovements were not monitored in this experiment the observersnow registered their selection of a memory target by moving a com-puter mouse cursor to the desired object in the response grid (Fig-ure 4C) and clicking a button7 All other aspects of the design andprocedure including the presentation of the memory probe and theidentities and locations of objects in the response grid were iden-tical to those in Experiment 1

Results and DiscussionThe data from Experiment 2 are shown in Figure 2B

Despite dramatically different presentation methods therecency effects for the three Experiment 2 conditionswere remarkably similar to the pattern observed in Ex-periment 1 Accuracy in the 500ndash0 condition was 78after one intervening object 59 after two and 41after three accuracy in the 500ndash500 condition was 8164 and 39 respectively The prerecency level of ac-curacy in both the 500ndash0 and 500ndash500 conditions wasapproximately 36 A somewhat different pattern wasfound in the 1000ndash0 data Accuracy after one interven-ing object was 88 with this initially high level of per-formance declining sharply over the second (69) andthird (53) intervening objects Prerecency accuracy inthe 1000ndash0 condition was 47 18 lower than whatwas found in Experiment 1 and roughly 11 higher thanin the other Experiment 2 conditions To quantify thesepatterns we performed a two-way mixed design ANOVAwith number of intervening objects as a within-subjectsfactor and presentation condition (500ndash0 500ndash500 and1000ndash0) as a between-subjects factor As expected wefound a highly significant main effect of intervening ob-jects [F(42814126 GreenhousendashGeisser adjusted) 17575 p 001] and a significant main effect of pre-sentation condition [F(233) 500 p 013] but thesevariables did not significantly interact [F(85614126GreenhousendashGeisser adjusted) 103 p 418] The ef-fect of presentation condition was driven by significantdifferences (Sidak corrected) between the 1000ndash0 con-dition and the 500ndash0 [t(22) 279 p 026] and 500ndash500[t(22) 269 p 033] conditions The two 500-msec pre-sentation conditions were not significantly different[t(22) 0105 p 999]

Is the recency component a result of decay or of in-tervening objects We used the dissociation logic de-scribed by Waugh and Norman (1965) to tease apartdecay and interference factors in Experiment 2 varyingthe presentation rate of consecutive objects while hold-ing constant the number of items appearing in each in-tervening object condition Objects in the 500ndash0 condi-tion appeared at a rate of two per second objects in the500ndash500 condition appeared at a rate of one per secondThese conditions therefore allow a straightforward testof the decay versus interference hypotheses over thetimeframes relevant to object memory during free view-ing Assuming that a decay process was responsible forthe observed recency effects and that the rate of this decay

was constant the recency component in the 500ndash500 datashould be twice as steep as that in the 500ndash0 data Thisprediction follows from the fact that a decay process inthe 500ndash500 condition would have had twice as long todegrade object representations compared with the 500ndash0condition8 However if the observed recency effectswere due to interference introduced by the objects ap-pearing between the target and probe accuracy shouldnot differ between these two conditions Figure 2B pro-vides clear support for the interference hypothesis Thedata patterns in the 500ndash0 and 500ndash500 conditions over-lap and suggest no interaction between retention intervaland intervening objects Consistent with Waugh andNorman we therefore conclude that a decay process wasnot a meaningful contributor to the reported recency ef-fects even at the relatively brief exposure durations usedin this study Rather the recency component is better de-scribed by an interference process that accumulates withthe addition of each intervening object to the stimulussequence

Do extrafoveal processes contribute to prerecencymemory The sequential presentation paradigm adoptedin Experiment 2 provides a straightforward test of whetherextrafoveal processing contributed to the above-chancelevel of prerecency memory reported in Experiment 1According to this argument if observers in Experiment 1were extracting information from multiple objects whilefixated on an individual item the target (and many of thenontargets) would have received considerably more pro-cessing than what was previously assumedmdashprocessingthat might translate into an above-chance prerecencycomponent This hypothesis therefore predicts a signifi-cantly higher prerecency component for Experiment 1compared with the three Experiment 2 viewing conditionsin which extrafoveal processing was precluded by se-quential presentation

To evaluate this hypothesis we analyzed the prere-cency data from Experiments 1 and 2 and found a sig-nificant difference [t(40) 475 p 001] between thegaze-contingent Experiment 1 data (M 6417 SD 947) and the data from Experiment 2 collapsed acrossthe three presentation conditions (M 4082 SD 1138) Multiple comparisons (Sidak corrected) furtherrevealed significant differences between the Experi-ment 1 prerecency level of memory and each of the threeprerecency components from Experiment 2 [t(16) 231 p 022] Experiment 2 therefore provides partialsupport for the existence of extrafoveal benefits shapingperformance in our task Consistent with this proposalprerecency accuracy in all three of the Experiment 2 con-ditions was well below the level observed in Experi-ment 1 meaning that the high prerecency componentfrom Experiment 1 may have been due in part to ex-trafoveal processes benefiting target encoding Howeversuch extrafoveal processes cannot explain why prere-cency accuracy was higher in the 1000ndash0 condition rela-tive to the 500ndash0 and 500ndash500 conditions Because a se-

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

MEMORY FOR OBJECTS IN SCENES 685

quential presentation paradigm was used in all three of theExperiment 2 conditions those differences cannot be ex-plained in terms of an extrafoveal processing advantage

Consistent with previous work (Hollingworth amp Hen-derson 2002 Phillips amp Christie 1977a) we believe thatour Experiment 2 data suggest that longer viewing timeslead to better memory for the target regardless of thenumber of intervening items Such a relationship be-tween viewing duration and target memory might ex-plain in part the high prerecency level of accuracy ob-served in Experiment 1 The average first-pass targetgaze duration in Experiment 1 was 683 msec over the3ndash7 intervening item range intermediate to the 500-msec and 1000-msec presentation durations used in Ex-periment 2 Given that the target objects in Experiment 1were processed an average of 183 msec longer than thosein the short-duration Experiment 2 conditions it is rea-sonable to speculate that viewing duration may have con-tributed to the Experiment 1 prerecency componentHowever because the average Experiment 1 target view-ing time was considerably shorter than in the 1000ndash0condition from Experiment 2 we must also concludethat viewing duration is not the sole determinant of pre-recency performance Additional work will be needed toweight the contribution of these factors but for now itappears that both viewing duration and extrafoveal pro-cessing may have played roles in determining prerecencyaccuracy in Experiment 19

GENERAL DISCUSSION

Two clear data patterns emerged from this study Firstrecency effects exist for objects presented in scenes withthis memory advantage declining steadily with fixationof the first three intervening objects (Experiment 1) Asfor the cause of this recency advantage we were able torule out both between- and within-trial PI The forget-ting functions for early and late trials did not differ andcases in which the target was fixated early in a trial pro-duced no accuracy advantage over those in which the tar-get was fixated later in a trial By shifting to a sequentialpresentation paradigm and varying the retention interval(Experiment 2) we were also able to exclude decay as apossible cause of this recency effect The serial buildupof RI from fixations on intervening objects appears to bethe only plausible explanation Regardless of whetherthis serial order of processing is imposed on the memoryset by the experimenter (as in a sequential presentationparadigm) or imposed by the observerrsquos own eye move-ments (as in our simultaneous presentation paradigm)the recency memory advantage rapidly declines with thenumber of objects viewed between target and test

The second salient pattern emerging from our data isthe constant and relatively high level of accuracy overthe 3ndash7 intervening object range Multiple factors likelycontribute to this above-chance prerecency level of accu-racy One such factor is the amount of time that observershave to view the display objects Whereas viewing dura-

tion was necessarily uncontrolled in Experiment 1 andconfounded with simultaneous presentation Experi-ment 2 suggested that this factor may have contributed toprerecency memory in our task with observers exploitinglonger viewing times to improve accuracy over the prere-cency component of the forgetting function We also be-lieve that the extrafoveal availability of objects in Exper-iment 1 may have played a role in elevating this prerecencylevel of performance We base this assertion on the fact thatdifferences in presentation duration in Experiment 2 couldnot account for all of the variation in prerecency perfor-mance between the simultaneous and sequential presenta-tion conditions

At this point however the nature of this extrafovealcontribution remains unclear One possibility is that ob-servers were able to process multiple items during eachobject fixation in Experiment 1 and they then used thisextrafoveal information to improve their prerecency ac-curacy However given Henderson et alrsquos (1989) findingof extrafoveal processing benefits being limited to thetarget of a saccadic eye movement we would expect suchan influence to be relatively small A potentially more im-portant form of extrafoveal influence may involve the for-mation of spatial codes relating one object to anotherWhen an observer is free to control the order in whichitems are encoded into memory as would be the case inExperiment 1 memory for this scanning order may en-able the targetrsquos identity to be better retrieved upon pre-sentation of the spatial probe Future work will vary theorderliness with which objects are sequentially presentedin the study scene in an attempt to isolate and weight thecontribution of this factor to prerecency performance

Relating the present data to the object memory lit-erature Both the recency and prerecency behavior ob-served in this study are in near-perfect agreement withthe patterns reported by Irwin and Zelinsky (2002) Con-sistent with Irwin and Zelinskyrsquos post hoc analysis of re-cency behavior we found a three-object recency benefit(extending over the target and the first two interveningobjects) using a completely object-based gaze-contingentmethodology Irwin and Zelinsky also reported a highand relatively constant 65 level of accuracy when test-ing occurred farther back in the viewing sequence a pat-tern again very similar to the above-chance prerecencycomponent that we found in our study The consistencybetween these two studies is notable given their very dif-ferent display-termination criteria and suggests that thereported data patterns are indeed real and quite robust

The present data agree less well with the nonscene ob-ject memory literature that typically reports recencybenefits for only the final pattern presented in a series (fora review see Phillips 1983) However one should be care-ful when generalizing these one-object recency estimatesto object memory in scenes Objects in scenes appear inspatially diverse configurations and this spatial informa-tion can be used to help access and individuate objects inmemory as assumed by OFT Most object memory stud-ies however present objects one after the other in the

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

686 ZELINSKY AND LOSCHKY

same spatial location thereby preventing the use of spa-tial cues to assist memory and potentially underestimat-ing our memory ability The representation of identityand spatial information for objects in scenes is also ablend of visual and semantic codes that depends criti-cally on the stimulus task and the observerrsquos ability tochunk object features into meaningful units (Avons ampPhillips 1987 Cowan 2001 Gordon amp Irwin 2000Logan 1995) The object memory literature to the extentthat it has relied on abstract visual patterns minimizesthe contribution of these nonvisual codes and thereforeagain potentially underestimates our ability to rememberobjects in scenes Although teasing apart visual verbaland semantic contributions to object memory is a worth-while topic for future research when it comes to repre-senting the identity and location of real-world objects insimple scenes it is probably safe to say that our three-item recency estimate provides a better indication ofhuman memory ability

The present data also stand in sharp contrast to the ab-sence of a recency effect reported by Hollingworth andHenderson (2002) We attribute this puzzling discrep-ancy in part to the fact that these authors quantified theirforgetting function in terms of intervening fixations ratherthan intervening objects If observers in their study tendedto devote more than one fixation per visit to an object asmay well have been the case given that they were engagedin a change-detection task any object-based recency ef-fect would be diluted over a wider range of the forgettingfunction and might therefore become more difficult to dis-cern It might also have been the case that their post hocanalysis lacked an adequate number of trials per interven-ing fixation condition to observe a recency effect If soour manipulation of intervening objects as an independentvariable may simply have provided our study with thepower needed to observe recency behavior

Unlike recency memory which may be sensitive tostimulus type and methodological factors there is nodisagreement regarding our above-chance level of prere-cency memory ability With Hollingworth and Hender-son (2002) and Irwin and Zelinsky (2002) three studieshave now shown that memory exists for objects in scenesindependent of when these objects were fixated in theviewing sequence This finding has an important impli-cation for scene representation in intentional memorytasks Although we may have very good memory foronly the last three objects that we view in a scene wenevertheless have a reasonably good memory for manymore scene objects

The implications of recency and prerecency memorybenefits may also extend beyond standard intentionalworking memory tasks and are potentially relevant toany visuocognitive task involving the free viewing ofmulti-item displays For example consider our oftenprofoundly impaired ability to detect changes to objectsin scenes (for reviews see Rensink 2002 Simons 2000and Simons amp Levin 1997) Change detection difficulty

has been traditionally attributed to sparse representationarising from attentional limitations (Grimes 1996 Levinamp Simons 1997 OrsquoRegan Rensink amp Clark 1999 Ren-sink 2000a 2000b Rensink OrsquoRegan amp Clark 1997)According to this view limits on selective attention pre-vent the representation of one or both of the pre- or post-change objects thereby resulting in a change detectionfailure The present data suggest a different perspectivefrom which to view this literature one focused on work-ing memory constraints rather than attentional limitationsThe stimuli used in change detection studies range fromrealistic scenes (OrsquoRegan et al 1999 Rensink et al1997) to video clips (Levin amp Simons 1997) to natural-istic events (Simons amp Levin 1998) and in all of thesestimuli there were potentially dozens of objects that ob-servers might have inspected with dozens of fixationsOn the basis of the forgetting function reported here andelsewhere (Irwin amp Zelinsky 2002 Zelinsky amp Loschky1998) newly fixated objects in the scene should inter-fere with memory for previously fixated scene objectsThe best detection performance should therefore be ob-served when the changed object was one of the last threeobjects fixated in the scene If the change target was fix-ated farther back in the viewing sequence detection ofthis change would have to rely on a less accurate prere-cency memory (Hollingworth amp Henderson 2002) re-sulting in a higher frequency of change detection failureRather than an extremely sparse representation (OrsquoRe-gan 1992) change detection might therefore be servedby a fallible but relatively dense representation extendingover the last seven objects fixated during viewing and pos-sibly many more In this sense our memory-constrainedview is consistent with recent explanations of change de-tection suggesting that information exists for many ob-jects in a scene but that this information is impoverishedand not sufficient to affect performance on every mem-ory task (Angelone Levin amp Simons 2003 SimonsChabris Schnur amp Levin 2002 Zelinsky 2003) If ob-servers fail to fixate both the pre- and postchange objectsduring scene viewing even prerecency memory would beunavailable to the task and the probability of detectionshould approach chance (Henderson amp Hollingworth1999b Hollingworth Schrock amp Henderson 2001 butsee Zelinsky 2001)

Reconciling the intervening object function withobject memory theory Can OFT be reconciled withthe clear patterns of recency and prerecency memory re-ported in this study According to transsaccadic OFTour STM for objects in a scene is limited to only 3ndash4property lists at any given moment in time (Irwin 19921996 Irwin amp Andrews 1996 Irwin amp Gordon 1998Irwin amp Zelinsky 2002 see also Cowan 2001 for a per-spective from the working memory literature) Assum-ing that each object fixated in our task resulted in thecreation of a new object file transsaccadic OFT wouldpredict a 3ndash4 item recency benefit very much like theone observed in the present study Note however that

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

MEMORY FOR OBJECTS IN SCENES 687

OFTrsquos suggestion of a capacity limitation makes its STMfunction analogous to the operation of a fixed-capacityconveyor belt (Zelinsky 2001) Once the belt is filledwith objects each new file linked to an object must beaccompanied by the deallocation of an existing objectfile OFT therefore predicts a straightforward pattern ofobject displacement from memory good memory shouldexist for the 3ndash4 most recently viewed objects still on thebelt and poorer memory should exist for those objectsthat have fallen off the belt as a result of being fixatedfarther back in the viewing sequence

To account for above-chance prerecency memory ob-ject file theorists have had to amend their theories to in-clude mechanisms capable of longer term object repre-sentation For example in addition to a short-term objectfile representation Irwin (1996) posited the existence ofa semantic network containing information about objectproperties Similarly Hollingworth and Henderson (2002)recently suggested a dual-store modification to OFTThese authors proposed that short-term object file rep-resentations can consolidate into a more permanent recordof an objectrsquos properties This LTM representation isthought to remain available following the withdrawal ofattention and the dissolution of the objectrsquos spatially in-dexed property list (see Henderson 1994 1997 Hender-son amp Anes 1994 Henderson amp Siefert 2001 Irwin ampZelinsky 2002 and Pollatsek et al 1990 for relatedthoughts regarding the relationship between object filesand long-term representation) Such a structural divisionof theoretical labor is attractive in that it offers a clear-cut explanation for both recency and prerecency patternsof object memory Much like the theories proposed forobject memory during sequential presentation (Kerret al 1999 Phillips 1983 Phillips amp Christie 1977a)by assuming separate STM and LTM systems each withtheir own processing limitations and capacity con-straints it becomes possible to explain recency and pre-recency behavior simply by assigning each componentof the forgetting function to a different memory system

However despite the intuitive appeal of such a dual-store model of object memory restraint should be ex-erted before adopting it Given that a dual-store modelhas two free parameters and that only two componentsof the forgetting function require explanation applyingsuch a model to the present data is as much a descriptionas it is a theoretical advance What is needed is the in-troduction of new alternative theories of object memoryin scenes followed by critical evaluations of these theo-ries to determine which offers the best and most parsi-monious account of the data The present study was in-tended to help bring about this crucial step in theorydevelopment There currently exists a schism in the work-ing memory community with the visual cognition litera-ture embracing capacity-limited models and dual-storetheories (Hollingworth amp Henderson 2002 Irwin amp An-drews 1996 Luck amp Vogel 1997 Rensink 2000a 2000b)and the serial memory literature explaining performance

limitations in terms of distinctiveness (Knoedler Hellwigamp Neath 1999 Neath 1993) interference (Wright 1998Wright et al 1985) and related forms of processing con-straint By using eye movements to serialize the encod-ing of scene objects into memory and then deriving fromthese data an intervening object forgetting function it isour hope to bridge these communities and enable serialorder memory theory to begin informing our memory forobjects presented simultaneously in scenes

REFERENCES

Altmann E M amp Gray W D (2002) Forgetting to remember Thefunctional relationship of decay and interference Psychological Sci-ence 13 27-33

Angelone B Levin D amp Simons D (2003) The roles of repre-sentation and comparison failures in change blindness Perception32 947-962

Avons S E amp Phillips W A (1987) Representation of matrix pat-terns in long- and short-term visual memory Acta Psychologica 65227-246

Awh E Jonides J amp Reuter-Lorenz P A (1998) Rehearsal inspatial working memory Journal of Experimental Psychology HumanPerception amp Performance 24 780-790

Baddeley A (1986) Working memory (Vol 11) Oxford Oxford Uni-versity Press Clarendon Press

Bridgeman B amp Mayer M (1983) Failure to integrate visual infor-mation from successive fixations Bulletin of the Psychonomic Soci-ety 21 285-286

Bridgeman B Van der Heijden A amp Velichkovsky B (1994) Atheory of visual stability across saccadic eye movements Behavioralamp Brain Sciences 17 247-292

Broadbent D E amp Broadbent M H (1981) Recency effects in vi-sual memory Quarterly Journal of Experimental Psychology HumanExperimental Psychology 33A 1-15

Carlson-Radvansky L A (1999) Memory for relational informationacross eye movements Perception amp Psychophysics 61 919-934

Carlson-Radvansky L A amp Irwin D E (1995) Memory for struc-tural information across eye movements Journal of ExperimentalPsychology Learning Memory amp Cognition 21 1441-1458

Courtney S M Petit L Maisog J M Ungerleider L G ampHaxby J V (1998) An area specialized for spatial working memoryin human frontal cortex Science 279 1347-1351

Cowan N (2001) The magical number 4 in short-term memory A re-consideration of mental storage capacity Behavioral amp Brain Sci-ences 24 87-185

Gordon R D amp Irwin D E (2000) The role of physical and con-ceptual properties in preserving object continuity Journal of Exper-imental Psychology Learning Memory amp Cognition 26 136-150

Gottesman CV amp Intraub H (2002) Surface construal and themental representation of scenes Journal of Experimental Psychol-ogy Human Perception amp Performance 28 1-11

Greene R L (1986) Sources of recency effects in free recall Psy-chological Bulletin 99 221-228

Grimes J (1996) On the failure to detect changes in scenes across sac-cades In K Akins (Ed) Perception (pp 89-110) New York OxfordUniversity Press

Hale S Myerson J Rhee S H Weiss C S amp Abrams R A(1996) Selective interference with the maintenance of location in-formation in working memory Neuropsychology 10 228-240

Hayhoe M Bensinger D amp Ballard D (1998) Task constraintsin visual working memory Vision Research 38 125-137

Henderson J M (1994) Two representational systems in dynamic vi-sual representation Journal of Experimental Psychology General123 410-426

Henderson J M (1997) Transsaccadic memory and integration dur-ing real-world object perception Psychological Science 8 51-55

688 ZELINSKY AND LOSCHKY

Henderson J M amp Anes M D (1994) Roles of object-file reviewand type priming in visual identification within and across eye fixa-tions Journal of Experimental Psychology Human Perception ampPerformance 20 826-839

Henderson J M amp Hollingworth A (1999a) High-level sceneperception Annual Review of Psychology 50 243-271

Henderson J M amp Hollingworth A (1999b) The role of fixationposition in detecting scene changes across saccades PsychologicalScience 10 438-443

Henderson J M McClure K K Pierce S amp Schrock G (1997)Object identification without foveal vision Evidence from an artifi-cial scotoma paradigm Perception amp Psychophysics 59 323-346

Henderson J M Pollatsek A amp Rayner K (1987) Effects offoveal priming and extrafoveal preview on object identificationJournal of Experimental Psychology Human Perception amp Perfor-mance 13 449-463

Henderson J M Pollatsek A amp Rayner K (1989) Covert visualattention and extrafoveal information use during object identifica-tion Perception amp Psychophysics 45 196-208

Henderson J M amp Siefert A B [C] (1999) The influence of enan-tiomorphic transformations on transsaccadic object integrationJournal of Experimental Psychology Human Perception amp Perfor-mance 25 243-255

Henderson J M amp Siefert A B C (2001) Types and tokens intranssaccadic object identification Effects of spatial position andleftndashright orientation Psychonomic Bulletin amp Review 8 753-760

Hollingworth A (2004) Constructing visual representations of nat-ural scenes The roles of short- and long-term visual memory Jour-nal of Experimental Psychology Human Perception amp Performance30 519-537

Hollingworth A amp Henderson J M (2002) Accurate visualmemory for previously attended objects in natural scenes Journal ofExperimental Psychology Human Perception amp Performance 28113-136

Hollingworth A Schrock G amp Henderson J M (2001) Changedetection in the flicker paradigm The role of fixation position withinthe scene Memory amp Cognition 29 296-304

Hollingworth A Williams C C amp Henderson J M (2001) Tosee and remember Visually specific information is retained in mem-ory from previously attended objects in natural scenes PsychonomicBulletin amp Review 8 761-768

Intraub H (1980) Presentation rate and the representation of brieflyglimpsed pictures in memory Journal of Experimental PsychologyHuman Learning amp Memory 6 1-12

Intraub H (1984) Conceptual masking The effects of subsequent vi-sual events on memory for pictures Journal of Experimental Psy-chology Learning Memory amp Cognition 10 115-125

Intraub H (1999) Understanding and remembering briefly glimpsedpictures Implications for visual scanning and memory In V Colt-heart (Ed) Fleeting memories Cognition of brief visual stimuli(pp 47-70) Cambridge MA MIT Press

Irwin D E (1991) Information integration across saccadic eye move-ments Cognitive Psychology 23 420-456

Irwin D E (1992) Memory for position and identity across eye move-ments Journal of Experimental Psychology Learning Memory ampCognition 18 307-317

Irwin D E (1996) Integrating information across saccadic eye move-ments Current Directions in Psychological Science 5 94-99

Irwin D E amp Andrews R (1996) Integration and accumulation ofinformation across saccadic eye movements In T Inui amp J McClel-land (Eds) Attention and performance XVI Information integrationin perception and communication (pp 125-155) Cambridge MAMIT Press

Irwin D E Brown J amp Sun J (1988) Visual masking and visualintegration across saccadic eye movements Journal of ExperimentalPsychology General 117 276-287

Irwin D E amp Gordon R D (1998) Eye movements attention andtrans-saccadic memory Visual Cognition 5 127-155

Irwin D E amp Zelinsky G J (2002) Eye movements and scene per-ception Memory for things observed Perception amp Psychophysics64 882-895

Jitsumori M Wright A A amp Shyan M R (1989) Buildup and re-lease from proactive interference in a rhesus monkey Journal of Ex-perimental Psychology Animal Behavior Processes 15 329-337

Jones D Farrand P Stuart G amp Morris N (1995) The func-tional equivalence of verbal and spatial information in serial short-term memory Journal of Experimental Psychology Learning Mem-ory amp Cognition 21 1008-1018

Kahneman D amp Treisman A (1984) Changing views of attentionand automaticity In R Parasuraman amp D R Davies (Eds) Varietiesof attention (pp 29-61) New York Academic Press

Kahneman D Treisman A amp Burkell J (1983) The cost of vi-sual filtering Journal of Experimental Psychology Human Percep-tion amp Performance 9 510-522

Kahneman D Treisman A amp Gibbs B J (1992) The reviewing ofobject files Object-specific integration of information CognitivePsychology 24 175-219

Keppel G amp Underwood B J (1962) Proactive inhibition in short-term retention of single items Journal of Verbal Learning amp VerbalBehavior 1 153-161

Kerr J R Avons S E amp Ward G (1999) The effect of retentioninterval on serial position curves for item recognition of visual pat-terns and faces Journal of Experimental Psychology LearningMemory amp Cognition 25 1475-1494

Knoedler A J Hellwig K A amp Neath I (1999) The shift fromrecency to primacy with increasing delay Journal of ExperimentalPsychology Learning Memory amp Cognition 25 474-487

Lawrence B M Myerson J Oonk H M amp Abrams R A (2001)The effects of eye and limb movements on working memory Memory9 433-444

Levin D T amp Simons D J (1997) Failure to detect changes to at-tended objects in motion pictures Psychonomic Bulletin amp Review4 501-506

Loess H amp Waugh N C (1967) Short-term memory and inter-trialinterval Journal of Verbal Learning amp Verbal Behavior 6 455-460

Logan G (1995) Linguistic and conceptual control of visual spatialattention Cognitive Psychology 28 103-174

Luck S amp Vogel E (1997) The capacity of visual working memoryfor features and conjunctions Nature 390 279-280

McConkie G W amp Rayner K (1976) Identifying the span of the ef-fective stimulus in reading Literature review and theories of readingIn H Singer amp R B Ruddell (Eds) Theoretical models and pro-cesses of reading (2nd ed pp 137-162) Newark DE InternationalReading Association

McConkie G W amp Zola D (1979) Is visual information integratedacross successive fixations in reading Perception amp Psychophysics25 221-224

Neath I (1993) Distinctiveness and serial position effects in recogni-tion Memory amp Cognition 21 689-698

Neath I (1998) Human memory An introduction to research dataand theory New York BrooksCole

OrsquoRegan K (1992) Solving the ldquorealrdquo mysteries of visual perceptionThe world as an outside memory Canadian Journal of Psychology46 461-488

OrsquoRegan K Rensink R amp Clark J (1999) Change-blindness as aresult of ldquomudsplashesrdquo Nature 398 34

Peterson L R amp Peterson M J (1959) Short-term retention of in-dividual items Journal of Experimental Psychology 61 12-21

Phillips W A (1983) Short-term visual memory PhilosophicalTransactions of the Royal Society of London Series B 302 295-309

Phillips W A amp Christie D F M (1977a) Components of visualmemory Quarterly Journal of Experimental Psychology 29 117-133

Phillips W A amp Christie D F M (1977b) Interference with visu-alization Quarterly Journal of Experimental Psychology 29 637-650

MEMORY FOR OBJECTS IN SCENES 689

Pollatsek A Rayner K amp Henderson J M (1990) Role of spa-tial location in integration of pictorial information across saccadesJournal of Experimental Psychology Human Perception amp Perfor-mance 16 199-210

Potter M C (1976) Short-term conceptual memory for picturesJournal of Experimental Psychology Human Perception amp Perfor-mance 2 509-522

Potter M C amp Levy E I (1969) Recognition memory for a rapidsequence of pictures Journal of Experimental Psychology 81 10-15

Potter M C Staub A Rado J amp OrsquoConnor D H (2002)Recognition memory for briefly presented pictures The time courseof rapid forgetting Journal of Experimental Psychology HumanPerception amp Performance 28 1163-1175

Rao S M Hammeke T A McQuillen M P Kharti B O ampLloyd D (1984) Memory disturbance in chronic progressive mul-tiple sclerosis Archives of Neurology 41 625-631

Rayner K amp Pollatsek A (1983) Is visual information integratedacross saccades Perception amp Psychophysics 34 39-48

Rensink R (2000a) The dynamic representation of scenes VisualCognition 7 17-42

Rensink R (2000b) Visual search for change A probe into the natureof attentional processing Visual Cognition 7 345-376

Rensink R (2002) Change detection Annual Review of Psychology53 247-277

Rensink R OrsquoRegan K amp Clark J (1997) To see or not to seeThe need for attention to perceive changes in scenes PsychologicalScience 8 368-373

Shimozaki S Hayhoe M Zelinsky G Weinstein A Meri-gan W amp Ballard D (2003) Effect of parietal lobe lesions onsaccade targeting and spatial memory in a naturalistic visual searchtask Neuropsychologia 41 1365-1386

Simons D J (2000) Current approaches to change blindness VisualCognition 7 1-15

Simons D J Chabris C F Schnur T amp Levin D T (2002) Ev-idence for preserved representations in change blindness Con-sciousness amp Cognition 11 78-97

Simons D J amp Levin D T (1997) Change blindness Trends in Cog-nitive Sciences 1 261-267

Simons D J amp Levin D T (1998) Failure to detect changes to peo-ple during a real-world interaction Psychonomic Bulletin amp Review5 644-649

Smyth M M amp Scholey K A (1996) Serial order in spatial im-mediate memory Quarterly Journal of Experimental Psychology49A 159-177

Treisman A (1988) Features and objects The fourteenth Bartlettmemorial lecture Quarterly Journal of Experimental Psychology40A 201-237

Walker P Hitch G J Doyle A amp Porter T (1994) The devel-opment of short-term visual memory in young children Interna-tional Journal of Behavioral Development 17 73-89

Walker P Hitch G J amp Duroe S (1993) The effect of visual sim-ilarity on short-term memory for spatial location Implications forthe capacity of visual short-term memory Acta Psychologica 83203-224

Waugh N C amp Norman D A (1965) Primary memory Psycho-logical Review 72 89-104

Wickens D D Born D G amp Allen C K (1963) Proactive inhi-bition and item similarity in short-term memory Journal of VerbalLearning amp Verbal Behavior 2 440-445

Wright A A (1998) Auditory and visual serial position functionsobey different laws Psychonomic Bulletin amp Review 5 564-584

Wright A A Santiago H C Sands S F Kendrick D F ampCook R G (1985) Memory processing of serial lists by pigeonsmonkeys and people Science 229 287-289

Zelinsky G J (2001) Eye movements during change detection Im-plications for search constraints memory limitations and scanningstrategies Perception amp Psychophysics 63 209-225

Zelinsky G J (2003) Detecting changes between real-world objects

using spatiochromatic filters Psychonomic Bulletin amp Review 10533-555

Zelinsky GJ amp Loschky L (1998) Toward a more realistic assess-ment of visual short-term memory [Abstract] Investigative Oph-thalmology amp Visual Science 39 S224

Zelinsky G J amp Loschky L (2003) Fuzzy object file theory Aframework for understanding recency effects for objects in scenes[Abstract] Journal of Vision 3 644a

Zelinsky GJ amp Murphy GL (2000) Synchronizing visual and lan-guage processing An effect of object name length on eye move-ments Psychological Science 11 125-131

Zelinsky GJ Rao RP Hayhoe MM amp Ballard DH (1997)Eye movements reveal the spatiotemporal dynamics of visual searchPsychological Science 8 448-453

NOTES

1 Note that there is also a literature in which scene stimuli are pre-sented in rapid sequence and observers have to indicate whether a par-ticular test scene was included in that series (Intraub 1980 Potter1976 Potter amp Levy 1969 Potter Staub Rado amp OrsquoConnor 2002 fora review see Intraub 1999) These studies deal primarily with the rep-resentations and processes required to differentiate one scene from an-other and not the representation of individual objects within a particu-lar scene In the context of the present investigation we therefore seethese studies as basically serial presentation memory studies that havesubstituted scene stimuli for isolated objects

2 We define a scene as one or more objects obeying a lawful rela-tionship to a background context such as tools sitting on a workbenchWe refer to our stimuli as simple scenes to acknowledge the fact that ob-ject placement in these displays was less relationally and contextuallyconstrained than what is typical for fully realistic scenes See Gottes-man and Intraub (2002) for more discussion of what is and is not ascene

3 Because OFT assumes that object properties can be indexed by theirposition in space (Kahneman et al 1992) we adopted a task that en-couraged observers to encode both the featural properties of an objectas well as the objectrsquos location According to OFT upon presentation ofthe spatial probe observers should be able to use the probe position toaccess and retrieve any object properties on file at that location In thissense a spatial probe recognition task is ideally suited to evaluate thetenets of OFT In contrast predictions from OFT would be less clear inthe case of a nonspatial recognition test (eg choosing from among fouralternatives an object that appeared anywhere in the study scene) In theabsence of a spatial probe it would be difficult to know which of the ex-isting object files was contributing information to the recognition judg-ment and how observers were accessing this information

4 Note that our 25 estimate of chance in this 4AFC task assumes thatneither the target nor any of the three lures were being remembered dur-ing test and that observers were truly guessing from among the four al-ternatives Of course if observers did remember one or more of thelures estimates of chance would increase above 25 because observerswould be able to exclude these lures from the recognition decisionHowever one would have to assume a perfect memory for more than six(of the eight) nontargets per study display in order for exclusionaryguessing to account for the 65 level of prerecency accuracy in therecognition test (Zelinsky amp Loschky 2003) Given that such a dispro-portionately high level of nontarget memory is unlikely we believe thatthe prerecency level of accuracy obtained in Experiment 1 reflectsmemory for the target rather than guessing

5 Note that fixations and fixated objects were highly correlated inour data (r 85) meaning that the two data sets plotted in Figure 2Aare not independent For this reason we could not use inferential sta-tistics to test for significant differences between the intervening objectand intervening fixation functions

6 Our selection of the 500-msec and 1000-msec presentation timesused in Experiment 2 was intended to bracket the range of first-passgaze durations on objects observed in Experiment 1

7 Note that the hand movements associated with mouse usage oc-curred only at test replacing the ldquoeye cursorrdquo used by observers to entertheir selections in Experiment 1 Although it is reasonable to ask whetherthese different motor behaviors might differentially affect recency weconsider this an unlikely possibility As argued by Lawrence MyersonOonk and Abrams (2001 p 433) ldquoall spatially directed movements ap-pear to have similar effects on visuospatial working memoryrdquo regard-less of whether they are performed by the eyes or hands (see also Haleet al 1996) The present data would seem to support this claim If mem-ory was subject to motor interference in this study it probably did notvary with the mode of response

8 Because lengthening the retention interval also increases the op-portunity for rehearsal it is conceivable that a detrimental effect ofdecay might be offset by a rehearsal benefit thereby resulting in no netdifference between the 500ndash0 and 500ndash500 forgetting functions Weconsider this possibility unlikely for two reasons First one would haveto assume a very fortuitous set of conditions to achieve such a perfectoffset between decay and rehearsal at each of our intervening object lev-els Second our 500-msec manipulation of retention interval offeredonly a minimal opportunity for rehearsal Observers would likely be

able to verbally recode and rehearse only one object at most duringthis interval (Zelinsky amp Murphy 2000) making it impossible for a re-hearsal explanation to account for the similarity in accuracy over therange of intervening object levels reported in this study

9 While the present study was under review another study similar toour Experiment 2 came to press (Hollingworth 2004) The author useda moving dot to impose a temporal sequence on the objects encodedfrom a scene Consistent with our findings the results revealed both re-cency and prerecency memory components and a lack of proactive in-terference from repeatedly presentated objects and scenes Note that theldquofollow the dotrdquo paradigm while important in offering converging evi-dence for serialization of object memory in scenes differs from our Ex-periment 2 ldquoone object at a timerdquo paradigm in that all of the memoryitems remained visible during the dot sequence thus leaving open thepossibility that extrafoveal information influenced memory for the tar-get object

(Manuscript received July 7 2003revision accepted for publication August 30 2004)

690 ZELINSKY AND LOSCHKY