ANRV339-CP04-01 ARI 6 March 2008 0:12

Ecological MomentaryAssessmentSaul Shiffman,1 Arthur A. Stone,2

and Michael R. Hufford3

1Department of Psychology, University of Pittsburgh, Pittsburgh, Pennsylvania15260; email: Shiffman@pitt.edu2Psychiatry and Behavioral Sciences Department, State University of New York,Stony Brook, Stony Brook, New York 11794-8790; email: Arthur.stone@sunysb.edu3Cypress Bioscience, Inc., San Diego, California 92121;email: mhufford@cypressbio.com

Annu. Rev. Clin. Psychol. 2008. 4:1–32

First published online as a Review in Advance onNovember 28, 2007

The Annual Review of Clinical Psychology is onlineat http://clinpsy.annualreviews.org

This article’s doi:10.1146/annurev.clinpsy.3.022806.091415

Copyright c© 2008 by Annual Reviews.All rights reserved

1548-5943/08/0427-0001$20.00

Key Words

diary, experience sampling, real-time data capture

AbstractAssessment in clinical psychology typically relies on global retro-spective self-reports collected at research or clinic visits, which arelimited by recall bias and are not well suited to address how behav-ior changes over time and across contexts. Ecological momentaryassessment (EMA) involves repeated sampling of subjects’ currentbehaviors and experiences in real time, in subjects’ natural envi-ronments. EMA aims to minimize recall bias, maximize ecologicalvalidity, and allow study of microprocesses that influence behavior inreal-world contexts. EMA studies assess particular events in subjects’lives or assess subjects at periodic intervals, often by random timesampling, using technologies ranging from written diaries and tele-phones to electronic diaries and physiological sensors. We discussthe rationale for EMA, EMA designs, methodological and practicalissues, and comparisons of EMA and recall data. EMA holds uniquepromise to advance the science and practice of clinical psychologyby shedding light on the dynamics of behavior in real-world settings.

1

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Contents

INTRODUCTION. . . . . . . . . . . . . . . . . 2WHAT IS ECOLOGICAL

MOMENTARYASSESSMENT? . . . . . . . . . . . . . . . . . 3Momentary, Real-Time

Assessment . . . . . . . . . . . . . . . . . . . . 4Real-World Data . . . . . . . . . . . . . . . . . 4Repeated Assessment . . . . . . . . . . . . . 4

ECOLOGICAL MOMENTARYASSESSMENT SCOPEAND HISTORY . . . . . . . . . . . . . . . . . 5Historical Roots . . . . . . . . . . . . . . . . . . 6

AUTOBIOGRAPHICAL MEMORYAND LIMITATIONSOF RECALL . . . . . . . . . . . . . . . . . . . . 7

THE USES OF ECOLOGICALMOMENTARYASSESSMENT . . . . . . . . . . . . . . . . . . 8Individual Differences . . . . . . . . . . . . 8Natural History . . . . . . . . . . . . . . . . . . 8Contextual Associations . . . . . . . . . . . 9Temporal Sequences . . . . . . . . . . . . . . 9

DATA COMPARINGRETROSPECTIVE ANDREAL-TIME REPORTS. . . . . . . . . 10Comparison of Aggregated

Recall-Based Dataand EMA . . . . . . . . . . . . . . . . . . . . . 10

Correlations BetweenDisaggregated EMAand Recall . . . . . . . . . . . . . . . . . . . . . 11

Construct Validity of Real-TimeVersus Recall Assessments . . . . . 12

ECOLOGICAL MOMENTARY

ASSESSMENT DESIGNSAND APPROACHES . . . . . . . . . . . . 13Event-Based Monitoring . . . . . . . . . . 14Time-Based Designs. . . . . . . . . . . . . . 14Combination Designs . . . . . . . . . . . . 15Use of Recall in EMA . . . . . . . . . . . . 16Sampling Versus Coverage

Strategies . . . . . . . . . . . . . . . . . . . . . 16Daily Diaries: A Special Case

of EMA . . . . . . . . . . . . . . . . . . . . . . . 17USE OF ECOLOGICAL

MOMENTARY ASSESSMENTIN TREATMENT . . . . . . . . . . . . . . . 17For Assessment . . . . . . . . . . . . . . . . . . . 17For Intervention . . . . . . . . . . . . . . . . . 18

MEASUREMENTCONSIDERATIONS FORECOLOGICAL MOMENTARYASSESSMENTSELF-REPORTS . . . . . . . . . . . . . . . . 19

METHODOLOGICALCONSIDERATIONS INECOLOGICAL MOMENTARYASSESSMENT STUDIES . . . . . . . 20Reactivity . . . . . . . . . . . . . . . . . . . . . . . . 20Compliance . . . . . . . . . . . . . . . . . . . . . . 20

SPECIAL POPULATIONS . . . . . . . . . 22ANALYSIS OF EMA DATA . . . . . . . . . 22PRACTICAL ISSUES IN

ECOLOGICAL MOMENTARYASSESSMENT . . . . . . . . . . . . . . . . . . 23EMA Hardware and Software . . . . . 23Other Practical Considerations . . . . 24

CONCLUSION . . . . . . . . . . . . . . . . . . . . 24

INTRODUCTIONClinical psychologists, along with behavioral,social, and health scientists and practitionersof every stripe, are interested in people’s ev-eryday real-world behavior. This interest isperhaps especially marked for clinical psy-chologists because psychopathology and itsfunctional impairments are expressed in real-

world settings: No one is diagnosed or treatedbecause of how they behave in a laboratoryor consulting room. Yet, behavior is seldomstudied, assessed, or observed as it unfolds inthe real world. Instead, both clinicians and re-searchers rely on global, summary, or retro-spective self-reports of behavior: We ask pa-tients how often they experience anxiety, on

2 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

average, how many panic attacks they had dur-ing the past week or month, how intense theirpain generally is during the day, or how de-pressed their mood has been. Moreover, theemphasis on global assessments can keep usfrom seeing and studying dynamic changesin behavior over time and across situations,from appreciating how behavior varies, and isgoverned, by context, and from understand-ing cascades of behavior, or interactions withothers or with our environments that play outas a sequence of events over time. Thus, ourfrequent reliance on global, retrospective re-ports seriously limits our ability to accuratelycharacterize, understand, and change behav-ior in real-world settings and misses the dy-namics of life as it is lived, day-to-day, hour byhour. In this review, we discuss an alternativeto static retrospective reports—EcologicalMomentary Assessment (EMA, Stone &Shiffman 1994), which allows subjects and pa-tients to report repeatedly on their experi-ences in real-time, in real-world settings, overtime and across contexts.

WHAT IS ECOLOGICALMOMENTARY ASSESSMENT?

EMA is not a single research method; it en-compasses a range of methods and method-ological traditions, which we discuss below.However, to provide a quick sense of howEMA can be used to inform our understand-ing of clinical psychology, and as a referencefor its paradigmatic characteristics, we de-scribe a prototypical EMA study. Our exam-ple is a study of cigarette smoking cessationand relapse (see Shiffman 2005). Smoking is agood target for EMA, as it involves a behav-ior with clearly discernible small-scale events.Tracking experience over time allows the re-search to track the process of quitting and re-lapsing over time. In this study, smokers whohad recently quit were asked to monitor theircigarette craving, nicotine withdrawal symp-toms, mood, and activities over several weeks,using palm-top computers as electronic di-aries. Since episodes of smoking (“lapses”)

EcologicalMomentaryAssessment (EMA):methods usingrepeated collectionof real-time data onsubjects’ behaviorand experience intheir naturalenvironments

were of key interest, subjects were asked torecord any episodes of smoking as they hap-pened, and were then prompted to completebrief assessments of their craving, mood, andactivities during the episode. On top of this,about five times each day, at random times, theelectronic diaries also prompted, or “beeped,”subjects and administered a similar assess-ment. These assessments captured not onlythe events associated with lapses, but the flowof mood, behavior, and events in the hoursand days before and after lapses.

These EMA data, captured using theelectronic diaries, allowed the investiga-tors to answer a variety of questions (seeShiffman 2005), including: How much crav-ing do smokers experience when they quitsmoking? (Surprisingly little, except in dis-crete episodes; Shiffman et al. 1997a.) Doescraving intensity vary with individual charac-teristics, such as nicotine dependence? (Yes;Shiffman et al. 2004.) Are day-to-day changesin craving intensity associated with variationsin subsequent risk of smoking? (Yes, espe-cially craving experienced first thing in themorning; Shiffman et al. 1997a.) Does experi-encing emotional distress predispose smokersto lapse? (Yes, but only acute distress, over aperiod of hours; Shiffman & Waters 2004.)Do situational factors affect lapse risk? (Yes,especially emotional distress, others smok-ing, and alcohol consumption; Shiffman et al.1996b.) Do lapses diminish self-efficacy? (Yes;Shiffman et al. 1997b.)

These are just some of the kinds of ques-tions and answers that can be examined usingEMA. They illustrate the potential for EMAdata to address questions about individual dif-ferences, about particular episodes or situa-tions, about the unfolding of processes overtime, and about the interactions among thesefactors. In this way, they illustrate both therichness and the complexity of EMA data.

This study illustrates several key fea-tures common to EMA approaches (Stone &Shiffman 1994, Stone et al. 2007a):

� Data are collected in real-world envi-ronments, as subjects go about their

www.annualreviews.org • Ecological Momentary Assessment 3

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Autobiographicalmemory: memoryprocesses involved inrecalling one’s ownexperience

lives. This is the “ecological” aspectof EMA and allows generalization tothe subjects’ real lives, i.e., ecologicalvalidity.

� Assessments focus on subjects’ currentstate; for example, self-reports ask aboutcurrent feelings (or very recent ones),rather than asking for recall or summaryover long periods. This is the “momen-tary” aspect of EMA and aims to avoidthe error and bias associated with retro-spection.

� Moments are strategically selected forassessment, whether based on particu-lar features of interest (e.g., occasionswhen subjects smoked), by random sam-pling (to characterize subjects’ experi-ences through representative sampling),or by other sampling schemes.

� Subjects complete multiple assessmentsover time, providing a picture of howtheir experiences and behavior variesover time and across situations.

This study is only illustrative. The par-ticular design, assessment schedule, assess-ment content, and even technology will varyacross studies, depending on the behavior re-searchers are studying, their aims, and theirtheoretical frameworks. But what EMA stud-ies have in common is the collection of as-sessments of subjects’ current or recent states,sampled repeatedly over time, in their naturalenvironments. With this illustration as back-ground, we now move to highlighting the keyaspects of EMA methods.

Momentary, Real-Time Assessment

EMA methods developed in part in responseto the limitations of retrospective recall. Al-though we all feel confident in our own mem-ories, research on autobiographical memoryteaches us that memory can be quite un-reliable (Bradburn et al. 1987, Tourangeau2000). Our recollections are not just inac-curate: They are often systematically biased.That is, the errors made in recalling informa-tion are not just random noise; rather, they

change the data in systematic ways. For ex-ample, people are more likely to retrieve neg-atively valenced information when they are ina negative mood, thus introducing substantialbias (Clark & Teasdale 1982). Because the dy-namics of recall are so important to justifyingand structuring EMA methods, these issuesare reviewed in more detail below.

Real-World Data

If one is interested in how subjects feel atwork, there is no point asking them how theyfeel in the research clinic—or at home, forthat matter. EMA recognizes that many be-haviors and experiences can be affected bycontext. Therefore, in order for assessed ex-perience or behavior to be representative, ithas to be sampled in the contexts in which itnaturally occurs. Stated more simply, EMAemphasizes ecologically valid observations.Because EMA data are collected in sub-jects’ natural environments—in real life—they should be generalizable to real-world,real-life experience.

Repeated Assessment

EMA studies involve many repeated mea-sures, covering various extents of time withvarying intensity of assessment. Some imple-ment a dense schedule of assessment, assessingsubjects as often as every 30 minutes (Shapiroet al. 2002) over a period of days. At the otherextreme, subjects may be assessed less fre-quently (e.g., daily) over periods as long as ayear ( Jamison et al. 2001). Some EMA stud-ies are primarily focused on using these manymeasures to characterize the subject’s “typi-cal” state, aggregating over the repeated as-sessments to better characterize the subject’saverage state across situations. More often,EMA studies use the temporal resolution af-forded by multiple measures to focus on thewithin-subject changes in behavior and expe-rience over time and across contexts, address-ing how symptoms vary over time or how sit-uational antecedents influence behavior.

4 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

In summary, EMA aims to assess the ebband flow of experience and behavior over time,capturing life as it is lived, moment to mo-ment, hour to hour, day to day, as a wayof faithfully characterizing individuals and ofcapturing the dynamics of experience and be-havior over time and across settings.

ECOLOGICAL MOMENTARYASSESSMENT SCOPEAND HISTORY

EMA is sometimes mistakenly associated nar-rowly with particular designs, such as theuse of randomly scheduled prompts to col-lect assessments, or with certain technolo-gies, such as the use of palm-top comput-ers. However, EMA is not a single method,much less a particular technology, but rathera collection of methods that share the charac-teristics described above. EMA includes tra-ditional diaries, whether they use paper andpencil (Green et al. 2006), palm-top com-puters (Shiffman et al. 1996b), or telephones(Perrine et al. 1995). It encompasses inter-personal interaction diaries (Reis & Wheeler1991), ambulatory physiological monitoring(Kop et al. 2001), and collection of medi-cation compliance data by instrumented pillbottles (Byerly et al. 2005). The technologiesdiffer, the targets of assessment differ, theschedules of data collection differ, but all ofthese methods focus on collecting data repeat-edly, in close to real time, and in subjects’ nat-ural environments. EMA aims to bring thesediverse methods under a common frameworkin order to define higher-order methodologi-cal principles, identify commonalities acrossmethods, and thus provide a framework inwhich researchers and clinicians can select theappropriate methods for their particular re-search studies or interventions.

Although the term “EMA” was only coinedin 1994 (Stone & Shiffman 1994), EMA re-search, as defined here, has been an active areafor decades. A search using the terms “diary,”“experience sampling,” and “ecological mo-mentary assessment” yielded over 3000 cita-

tions in the past 25 years on PsychLit, and196,000 “hits” on GoogleTM Scholar. This in-dicates a substantial volume of research activ-ity using EMA methods.

The emergence of EMA as an importantresearch method is also indicated by a num-ber of books and reviews that have appearedon the topic, including books on EMA meth-ods and findings by Stone et al. (2007b),Hektner et al. (2007), and Fahrenberg &Myrtek (2001); applications to mental healthby DeVries (1992); and discussions of dataanalysis by Walls & Schafer (2006). Otherreviews include Wheeler & Reis (1991) onsampling schemes for EMA methods, Scol-lon et al. (2003) on the promise and chal-lenges of EMA methods, Bolger et al. (2003)on various uses of diary methods, and Pi-asecki et al. (2007) on applications of EMAmethods to clinical treatment. A recent spe-cial issue of the Journal of Personality (Tennenet al. 2005) described several research pro-grams applying EMA methods to issues rel-evant to personality, clinical, and health psy-chology, and several review papers discussedtheir application to particular domains: Thieleet al. (2002) reviewed the application of EMAmethods to clinical psychology, Moskowitz &Young (2006) discussed applications to psy-chopharmacology, and Beal & Weiss (2003) toindustrial psychology. The reader is referredto these sources for more detail than can beincluded here.

Finally, EMA methods are being used tostudy a very wide range of behaviors, expe-riences, and conditions. In a review of pub-lished diary studies, Thiele and colleagues(2002) found large groups of studies on pain,mood, anxiety and anxiety disorders, eating,sleep, gastrointestinal disorders, and alcoholconsumption. But even this is an incom-plete list: EMA studies include studies of de-pression, social support, initiate relationships,diet, work activity and satisfaction, sexualbehavior, psychotherapy, drug use, allergies,psychological stress, adverse effects of medi-cations, self-esteem, and asthma, to name justa few. Clinical disorders studied with EMA

www.annualreviews.org • Ecological Momentary Assessment 5

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

include addictive disorders, eating disorders,anxiety disorders, depression, bipolar disor-der, schizophrenia, sexual dysfunction, andADHD—in other words, the full range ofpsychopathology. Beyond clinical syndromesand symptoms, EMA is also widely used tostudy basic adaptation processes relevant toadjustment, such as coping, self-esteem, andsocial support, as well as behaviors central tohealth psychology and behavioral medicine,such as coping with illness and treatment,medication compliance, exercise, relaxation,and safe sexual practices, among other behav-iors. In sum, EMA methods are used in mostdomains of concern to clinical psychology.

Historical Roots

EMA draws together several historical tradi-tions, including diaries, self-monitoring, ex-perience sampling, ambulatory monitoring,and others. The oldest is the use of writtendiaries for research (Verbrugge 1980), whichwas systematically deployed in clinical re-search in the 1940s. Self-monitoring of par-ticular behaviors or experiences (Korotitsch& Nelson-Gray 1999) has a long history inresearch and particularly in behavioral treat-ment. This includes simple counts of clinicallyrelevant events, but also collection of dataabout their antecedents and contexts, whichprovided data for “naturalistic functional anal-ysis” (Schlundt et al. 1985). Related methodsincluded self-monitoring of particular targets;a prominent example is the Rochester Inter-action Record (Reis & Wheeler 1991), whichsubjects used to record every social interac-tion. Self-monitoring approaches often aimedto capture all relevant events, and as a resultdid not focus on sampling issues or collect dataoutside of target events.

Other historical streams that fed into thedevelopment of EMA focused on broad de-scriptions of subjects’ behavior. One camefrom the focus of the Kansas School of Eco-logical Research (Barker 1978) on continuousobservation of behavior through the day in thenatural environment. Another derived from

the ethnographic method of describing indi-viduals’ allocation of time, often to describedifferences among societies (Szalai 1966). Avariant of this within schools of manage-ment and business examined the behavior ofworkers in the workplace to understand howthey used their time and what activities theyengaged in.

More central to the development of mod-ern EMA methods was the developmentby Czikszentmihalyi and colleagues (DeVries1992, Hektner et al. 2007) of the ExperienceSampling Method, demonstrating the inno-vation of randomly sampling experience, ini-tially using pagers to “beep” people at ran-dom times to prompt them to complete diarycards reporting their activity, mood, and/orthoughts. The development of electronic di-aries, based upon the emerging technology ofhandheld computers, opened up new oppor-tunities for more complex and sophisticatedEMA protocols that incorporated, combined,and expanded upon the various approaches tocollecting EMA self-report data.

Enabled by technological developments,ambulatory monitoring of cardiovascularfunction, which became possible with the de-velopment of wearable cardiac monitors, hasbeen used for several decades as a means ofunderstanding the link between experienceand cardiovascular health (Turner et al. 1994).Ambulatory monitoring did not rely on self-report (although it was often accompanied bywritten diaries) and also enabled continuousor near-continuous recording. Recent devel-opments have expanded physiological moni-toring to other parameters, such as galvanicskin response, temperature, motion, and oth-ers (Wilhelm et al. 2003). Often little atten-tion was paid to sampling, in part because themonitors could collect data almost continu-ously (e.g., actigraph or heart-rate recording)or at very high and stable frequencies. Somephysiological monitoring devices (e.g., mon-itoring of blood-glucose, pulmonary func-tion) require subjects to actively make pe-riodic assessments, and thus raise many ofthe sampling issues that arise with self-report.

6 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Technological developments have also en-abled automated EMA assessment of behav-iors (e.g., pill taking; Cramer et al. 1989) andeven of the physical environment (e.g., airsampling; Saito et al. 2005).

These diverse traditions developed in dif-ferent disciplinary contexts, often with dis-tinct assessment targets. For example, theExperience Sampling Method focused on sub-jective states (Hektner et al. 2007 refer to itas systematic phenomenology), whereas self-monitoring focused on behaviors, and ambu-latory monitoring focused on physiologicalparameters. Moreover, because of their par-ticular disciplinary histories and content foci,these methods have been discussed in differ-ent literatures, and investigators in one tradi-tion often seem unaware of the others. Yet,these methods share many goals, concerns,and approaches. The object of EMA, whichattempts to encompass all of them, is to unifythese diverse approaches under a commonmethodological framework.

AUTOBIOGRAPHICAL MEMORYAND LIMITATIONS OF RECALL

A major motivation for EMA is to avoid thepitfalls and limitations of reliance on autobio-graphical memory. In this section, we expandon the issues and concerns regarding autobi-ographical memory, both to explain the ratio-nale for EMA and to provide an essential foun-dation to considerations of the uses of EMAand specific EMA designs.

Research on autobiographical memory(Bradburn et al. 1987) indicates that recall isnot just subject to random error but also isfraught with systematic bias, which can dis-tort recall even after relatively short inter-vals. Modern cognitive science considers thatmuch of what we “recall” is actually recon-struction, pieced together from fragmentaryinputs through the use of various heuristicstrategies. Many experiences are not retainedin memory, so often the information we areasked to provide simply is not available fordirect retrieval. Experiences are particularly

likely to be encoded and retrieved if they areemotionally salient or are unique; routine ex-perience is less likely to be encoded and harderto retrieve. Moreover, the process of retrievalitself is subject to bias because the accessi-bility of particular content in memory varieswith the subject’s mental state at the time ofretrieval.

Importantly, research inquiries usually asksubjects not to just retrieve but also to aggre-gate and summarize their experiences (e.g.,“How intense was the pain, on average, to-day?”). When trying to answer such questions,subjects do not recall, enumerate, and then ag-gregate their experience over time (Bradburnet al. 1987). Rather, they use a variety ofheuristics to estimate the answer. The use ofcognitive heuristics and the processes of re-trieval account for much of the bias in recalldata.

A key example of a biasing cognitiveheuristic is the “availability heuristic.” In itsoriginal form, as described by Tversky &Kahneman (1973), this refers to how peo-ple make judgments about the frequency ofevents. When deciding how often an event(say, a fight with a spouse) occurs, a persontries to retrieve an example. If an example iseasy to think of (i.e., it is easily “available” inmemory), the event is considered to be fre-quent. One can see how this heuristic wouldwork much of the time—rare events should beharder to “find” in memory. But one can alsosee how biased it can be: if a fight was partic-ularly memorable because it was intense, if itis easily recalled because it was recent, or be-cause one was recently reminded of it (say, bya previous question in the assessment), thenthe frequency of fights will be overestimated.

Importantly, the process of memory re-trieval is itself subject to bias by the person’scontext and mental state at the time of re-call. It has been shown, for example, that sub-jects in a negative mood more easily recallnegative information than positive (Kihlstromet al. 2000). Similarly, subjects who are in painfind it easy to remember past pain, but harderto recall pain-free states; accordingly, it’s been

www.annualreviews.org • Ecological Momentary Assessment 7

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Recall bias:systematic errors indata values (asdistinct from randomerror) introduced byprocesses ofautobiographicalmemory

shown that subjects who are in pain at the timeof recall will overestimate their past pain (Eichet al. 1985). This has profound implicationsfor research because it suggests that the sub-ject’s state and situation at time of reportingwill influence what is reported.

A dramatic demonstration of the biases inrecall—and an indication of how quickly thesebiases can set in—was reported by Redelmeieret al. (2003). Summary ratings of pain by sub-jects who had undergone a colonoscopy 20 to30 minutes earlier were found to be undulyinfluenced by the peak level of pain (presum-ably because it was most salient) and the painintensity at the end of the procedure (mostrecent). In other words, recall did not accu-rately represent the average pain over the in-terval, because it was based on a few of themost memorable moments, essentially ignor-ing most of the experience. This shows thepotential for bias even over short intervals.

Besides being distorted by the operation ofheuristic recall strategies, memory is also in-fluenced by what we know and believe ratherthan actually recall. People unconsciously re-organize their “memories” to make them fita coherent script or theory of events or toreconcile events with what transpired subse-quently (Ross 1989). These biases are particu-larly pernicious because they tend to producerecalled patterns that are coherent and thatmay conform to theoretical predictions, evenif they are false.

It is important to understand that these bi-asing processes operate involuntarily and un-consciously. They do not represent distortionby uncooperative or defensive subjects—thisis simply the way memory operates. Researchinquiries often maximize the potential for biasby asking about routine events, asking sub-jects to summarize their experience, and so-liciting recall in unusual settings (i.e., researchlaboratories) and contexts (e.g., after beingasked other questions) that can bias recall.Thus, heuristics that serve well enough to ad-dress the demands placed on autobiographicalmemory in everyday life can break down whencalled upon to produce accurate research data.

THE USES OF ECOLOGICALMOMENTARY ASSESSMENT

EMA data are collected for a variety of pur-poses (Bolger et al. 2003). We categorize theseinto four classes: (a) characterizing individ-ual differences, (b) describing natural history,(c) assessing contextual associations, and (d )documenting temporal sequences. We illus-trate each with an example.

Individual Differences

When used to characterize individual differ-ences, EMA data are aggregated to obtain ameasure of the subject that is collapsed acrosstime (i.e., across multiple EMA measures); forexample, the average intensity of pain experi-enced by a pain patient. As an extension ofthis, aggregated EMA data might be used toquantify subjects’ characteristics at two dif-ferent time points; e.g., pain before and aftertreatment administration. As estimates of sub-ject characteristics, aggregated EMA data areexpected to provide assessments of individualsthat are more reliable (because of aggregation)and more valid (because of avoidance of recallbias, representative sampling, and ecologicalvalidity). Of course, if the variable is very sta-ble over time, if recall bias were not present,and if contextual factors did not influence thevariable, then there would be no advantage inusing EMA.

Natural History

To describe natural history, EMA measuresare analyzed for trends over time. In this case,the within-subject variation over time itself isthe focus, and time is the independent vari-able, the X-axis in a graphical representationof the data. For example, McCarthy et al.(2006) documented the trajectories of variouswithdrawal symptoms that smokers experi-enced after quitting. The EMA data demon-strated that some symptoms peaked immedi-ately when smokers quit and then decreasedover time, while others increased and per-sisted, and still others increased only gradually

8 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

over time. These patterns contradicted widelyheld notions about the course of the with-drawal syndrome and were associated withdifferences in treatment outcome. Basic de-scriptive information about the natural his-tory of symptoms over time can often be animportant foundation for understanding clin-ical disorders and outcomes.

Contextual Associations

Studies that examine contextual associationslook at the association or interaction betweentwo (or more) phenomena that co-occur intime. Analyses of contextual associations areoften cross-sectional, even when data are col-lected longitudinally, in that they examinethe co-occurrence of events or experiences,not their sequence. In these analyses, timeis not explicitly represented—it is more of astage against which the events of interest playout. For example, Myin-Germeys et al. (2001)examined emotions accompanying stressfulevents as a way to test a diathesis-stress modelof schizophrenia. They postulated that vul-nerability to schizophrenia would be reflectedin excess emotional responses accompanyingstress. Schizophrenics, their first-degree rel-atives (who are genetically vulnerable), andnormal controls were assessed 10 times dailyabout stressful events and mood. An exam-ination of individual differences in averagemood showed that the schizophrenics re-ported more negative affect and more stress-ful events, whereas vulnerable individuals andnormal controls did not differ. But a look atstressor-mood associations revealed that thefirst-degree relatives reacted more stronglythan did controls. Thus, examination of theassociation between stressors and mood atparticular moments was key to understandingwhat vulnerabilities might be conferred by agenetic predisposition to schizophrenia.

Understanding the momentary cross-sectional associations between different as-pects of experience has also been importantfor foundational studies of the structure ofbehavior and experience; for example, data

on the covariation of momentary emotionshave been central in the debate about whetherpositive and negative emotions are polar op-posites or are independent dimensions andcan be experienced simultaneously. Feldman-Barrett & Russell (1998) used EMA data toaddress the argument that although one couldbe both happy and distressed over some in-terval of time, in a particular moment, onecould be either happy or distressed but notboth.

Although most designs examine associa-tions between different variables within thesame person, an interesting variation consid-ers how one person in a relationship affects theother (Bolger & Laurenceau 2005). For exam-ple, Larson & Richards (1994) asked membersof families to track their experience in paralleland examined how the mood of each affectedthe other. They found, for example, that a hus-band’s mood when he comes home from worksignificantly influences his wife’s mood, butnot vice versa.

Temporal Sequences

Finally, the longitudinal nature of EMA data isused to explicitly examine temporal sequencesof events or experiences, to document an-tecedents or consequences of events or be-haviors, or to study cascades of events. Inthese analyses, unlike those above, the order ofevents or assessments is explicitly consideredand is a key focus. The previously cited studyof quitting smoking (Shiffman et al. 1997b) as-sessed smokers’ affect and self-efficacy beforeand after lapses to smoking, and their effectson subsequent progression toward relapse, totest Marlatt’s theory (Curry et al. 1987) thatthe psychological response to lapses is whatdrives progression toward relapse. Compar-ing assessments obtained before the lapse andafterward confirmed the theory’s hypothesisthat lapses would result in increased negativeaffect and decreased self-efficacy (Shiffmanet al. 1997b). Continued EMA monitoring,however, contradicted the theory’s predictionthat increases in negative affect and decreases

www.annualreviews.org • Ecological Momentary Assessment 9

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

in self-efficacy would predict the risk ofsubsequent progression to another lapse orrelapse (Shiffman et al. 1996a). Importantly,retrospective analyses had appeared to showa relationship between initial responses to alapse and progression to relapse (Curry et al.1987), but later comparisons with EMA data(Shiffman et al. 1997c) showed that retrospec-tive reports of lapse episodes were inaccurateand biased: Subjects recalled their mood asworse than it actually had been, and those whohad returned to smoking at the time of re-call exaggerated how demoralizing the initiallapse had been. Thus, prospective assessmentsof the flow of behavior and experience, and ofthe antecedents and consequences of events,can enable a more valid and more detailed un-derstanding of behavior.

These examples illustrate the use of EMAdata to evaluate hypotheses regarding thedynamic interactions among processes overtime. Data provided by EMA studies may belikened to a movie, in which dynamic rela-tionships emerge over time, whereas global orrecall measures are analogous to a still pho-tograph, a single static snapshot of time. Byproviding temporal resolution, EMA meth-ods allow investigators to examine sequencesof events and experiences and enable them todescribe and analyze cascades of events andinteractions between events that shape behav-ior over periods of minutes, hours, or days.Insight into microprocesses—the interplay orcascade of cognitive, affective, and behavioralvariables over short intervals of time—is par-ticularly important because many theories ofpsychopathology and treatment focus on howthese processes unfold over time. Evaluationof microprocesses also facilitates developmentof interventions because an understanding ofhow affect, cognition, and behavior interactand unfold over time helps identify leveragepoints for timely, and at least potentially moreefficacious, clinical interventions. The abilityto shed light on dynamic processes and situa-tional influences is potentially the most criti-cal contribution of EMA methods to clinicalpsychology.

DATA COMPARINGRETROSPECTIVE ANDREAL-TIME REPORTS

Research suggests that recall measures, andespecially summary measures, are often biaseddue to the use of mental heuristics to recallinformation. This implies that there will bediscrepancies between EMA-based and recall-based assessments of the same period. Rel-atively few studies have examined such dis-crepancies. In discussing this literature, wedistinguish comparison with aggregated EMAmeasures from comparisons of disaggregatedtime-specific estimates.

Comparison of AggregatedRecall-Based Data and EMA

A number of studies have evaluated the rela-tionship between EMA- and recall-based as-sessments, comparing EMA assessments av-eraged over some interval with recall-basedmeasures for the same interval. In some in-stances, the two methods yield similar esti-mates (Shrier et al. 2005), and in some othercases, recall methods produce lower estimatesof intensity or frequency than do EMA meth-ods (Carney et al. 1998, Litt et al. 2000). How-ever, in many domains, recall-based assess-ments tend to yield higher estimated levelsthan diary ratings of the same target events:That is, symptoms tend to be described asmore frequent, more intense, and longer last-ing, sometimes dramatically so (Brodericket al. 2006, Houtveen & Oei 2007, Shiffman2007, Shiffman et al. 2006; see review in Vanden Brink et al. 2001). Behavior frequency isalso often overestimated in recall (Hommaet al. 2002, Shiffman & Paty 2003). Thisphenomenon may be the result of the un-due influence of more salient experiences inrecall: Intense pain is more salient than nopain, headaches are more salient than non-headaches, and so on, leading recall data tooverestimate clinical symptoms. An exampleof this is the finding that subjects who hadthe most intense headaches also overestimated

10 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

headache frequency (Van den Brink et al.2001).

Studies have also examined the correlationbetween aggregated EMA and recall-baseddata, which assesses whether the ranking ofindividuals is similar across the two sources,and can be high even when there are differ-ences in the two means. Like the findingson mean differences, the findings from cor-relational studies are also variable. Studies ofdrug use report good correspondence for fre-quency or quantity used (Carney et al. 1998,Shiffman & Paty 2003), but very poor cor-respondence when characterizing situationalpatterns of drug use, which involve more com-plex judgments (Shiffman 1993, Todd et al.2005). In one study of pain among chronicpain patients ( J. Broderick, J. Schwartz, andA.A. Stone, unpublished observations), corre-lations in the 0.70s were seen between EMAand recall-based measures of pain intensity,whereas Van den Brink (2001) reported cor-relations around 0.20 for headache frequency,intensity, and duration.

Thus, findings on correspondence be-tween aggregate EMA data and recall-basedestimates are inconsistent and variable. Onelikely reason is that the magnitude and direc-tion of recall bias can differ across subjects andsettings. Individual differences can affect re-call accuracy. Feldman-Barrett (1997) foundthat recalled distress was exaggerated amongmore neurotic subjects and positive affect wasexaggerated among extraverted subjects, andVan den Brink (2001) describes other individ-ual differences that moderate recall bias. Be-liefs can also moderate recall bias. McFarlandet al. (1989) found that women who believedthat their menstrual cycle influenced theirmood reported exaggerated negative mood inretrospect, but only for menstrual days. Incontrast, women who did not believe men-struation influenced mood did not show anybias, either for menstrual or nonmenstrualdays.

Variation in the pattern of target symptomsthemselves can moderate bias. Stone et al.(2005) showed that subjects whose pain was

relatively constant over time were able to es-timate their pain more accurately since, in ef-fect, they could easily know their typical painlevel or estimate accurately from any giventime point. In contrast, subjects whose painwas variable demonstrated considerable bias.Furthermore, Broderick et al. (2006) and oth-ers have found that recalled pain was particu-larly exaggerated when subjects were in pain atthe time of recall and understated when theywere not.

Thus, the validity of recall data likely varieswith characteristics of both the samples andthe setting. It is likely that the validity of re-call is also influenced by the duration of therecall interval and the variability, salience, anduniqueness of what is being assessed: Somekinds of end points may be well estimated,whereas others are not. Better understand-ing of these relationships could help estab-lish when EMA is likely to add the most valuecompared with aggregated recall measures.

Correlations Between DisaggregatedEMA and Recall

The studies reviewed above assess the cor-relation between EMA- and recall-based es-timates of experience when EMA data areaggregated over time to broadly character-ize person-level effects, and compared withsimilar data based on global recall. How-ever, EMA data are most often used not justto characterize between-person differences,but also to characterize within-person vari-ations in experience over time, so that thequestion is whether recall measures are ableto accurately reflect time-specific data. Theavailable evidence suggests that they cannot.Even when global recall data correlate wellwith aggregated EMA data, as in the stud-ies cited above ( J. Broderick, J. Schwartz, andA.A. Stone, unpublished observations; Carneyet al. 1998; Shiffman & Paty 2003), theydo not adequately capture time-varying data,with correlations for recall of specific dayshovering in the range of 0.20–0.40. Moreover,analyses show that the cross-day correlation

www.annualreviews.org • Ecological Momentary Assessment 11

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

also varies widely between subjects, withsome subjects actually showing negative cor-relations between their recalled reports andreal-time reports of the same target events(Carney et al. 1998, Searles et al. 2000).In other words, even when recall appearsadequate to characterize aggregate experi-ence, it is not typically adequate to charac-terize day-to-day changes in cognitions andbehaviors of interest, which are typically thefocus of EMA research. This highlights one ofthe unique contributions of EMA to the studyof processes that unfold over time.

Construct Validity of Real-TimeVersus Recall Assessments

In addition to comparisons of EMA-based andaggregated recall-based assessments to eachother, comparisons can be made based on theirconstruct validity—in other words, their re-lationships with other theoretically relevantconstructs. Several studies illustrate cases inwhich EMA data show incremental validityover and above retrospective or global as-sessments of similar constructs. Kamarck andcolleagues (2007) directly contrasted the ef-fect of job strain on cardiovascular outcomeswhen strain was measured by EMA versusglobal measures. Subjects filled out a stan-dard global job strain questionnaire and usedan electronic diary to collect EMA data onexperienced strain every 45 minutes for sixdays (data from the EMA was averaged tocreate a single EMA summary variable). Thestudy’s outcome measure was a prospectivelyassessed biological outcome: the progressionof blockage in the carotid artery (which corre-lates with blockage of coronary arteries) overthe subsequent three years. EMA-based mea-sures of job strain predicted progression ofarterial occlusion; however, traditional globalquestionnaire measures of job strain did not.Moreover, heart rate assessed by EMA inthe natural environment also independentlypredicted progression of carotid blockage,whereas heart rate measured in the lab didnot. This illustrates the potential of EMA

data to shed light on relationships that aremissed when relying on global retrospectiveself-reports.

In a study of smoking, Shiffman and col-leagues (2007) similarly compared global andEMA-based measures, in this case, measuresof “negative-affect smoking”—the tendencyto smoke when distressed. The EMA-basedmeasure estimated negative-affect smokingby comparing negative affect on smokingand nonsmoking occasions; the global assess-ment used standardized and validated ques-tionnaires. Shiffman et al. tested the predic-tion that smokers engaged in negative-affectsmoking would be more vulnerable to relapseafter they quit smoking. This was found to betrue for EMA-assessed negative-affect smok-ing, but not for assessments based on standardrecall-based questionnaires.

The literature also suggests that EMAmeasures may sometimes mirror the findingsof recall measures, but may capture the targetconstructs with less noise and greater sensitiv-ity. In a study of the efficacy of analgesics inthe treatment of rheumatoid arthritis, Nivedand colleagues (1994) reported that diary datadifferentiated active treatment from controlafter only four weeks of treatment, whereas ittook 24 weeks for the differences to becomeapparent in recall measures collected at clinicvisits. In a related vein, analyses have shownthat real-time diary-based methods resultedin decreased error variance when capturingevents (McKenzie et al. 2004) and scaled rat-ings (Pearson 2004), making such data moresensitive to treatment effects.

In contrast to these studies favoring EMAdata, J. Broderick, J. Schwartz, and A.A.Stone (unpublished observations) recentlyfound that EMA and recall-based measures ofpain were equally correlated with medication-taking and social impairment. Importantly,other studies have illustrated cases in whichrecall measures were actually better predic-tors of subsequent behavior than were EMAdata. These cases are illuminating. In a studyof bias in recall of pain, Kahneman and col-leagues (Redelmeier et al. 2003) showed that

12 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

participants who were randomized to un-dergo a longer colonoscopy, with more overallpain, but who experienced less intense painnear the end of the procedure actually re-called lower overall pain. A follow-up revealedthat subjects who had undergone the longerprocedure—which was experienced as morepainful in real time—were more likely to re-turn for a repeat colonoscopy. In other words,their future behavior was related to the ret-rospective summary of pain, not the momen-tary experience. Similarly, Oishi & Sullivan(2006) found that break-up of dating cou-ples was better predicted from their retro-spective summaries of relationship dissatis-faction than from their daily ratings. Thesefindings make sense, because people shapetheir subsequent behavior—whether to returnfor a colonoscopy, whether to proceed witha relationship—by reference to their storedsummary memory of the experience, not theactual experience at the time.

These examples illustrate a key point: Evenif they do not match momentary experience,retrospective impressions or global beliefs canexercise greater control over subsequent be-havior, since they represent the informationthat people use to make subsequent decisions.In this sense, the momentary and retrospec-tive reports represent different perspectiveson the same event. If one is interested in whatwas experienced in the moment (e.g., to un-derstand the physiological effects of the painor to evaluate the effects of an analgesic) orwishes to understand what are the proximaldeterminants of a specific event, then the mo-mentary data are likely to provide a more validpicture. But if one is interested in the person’simpressions of an event or in the predictionof future behavior, then the retrospective im-pressions may well be better predictors thanEMA. This suggests that researchers evalu-ating the validity and utility of EMA and re-call data must consider whether they are con-cerned with understanding the experience aslived or with understanding subjects’ impres-sions of those experiences. As we have arguedwith regard to assessments of psychological

Event-basedsampling: a methodof data collectionwhereby a recordingis made each time apredefined eventoccurs

Time-basedsampling: a methodof data collectionwhereby a recordingis solicited based on atime schedule, oftenbased on randomtime intervals

coping (Stone et al. 1998), confusion aboutwhat is being assessed can misdirect theoryand research.

The discrepancy between momentary ex-perience and global summary recall also sug-gests that it would be worthwhile for researchto help us understand how momentary ex-periences are integrated in the developmentof global judgments. More research is alsoneeded on the circumstances in which globalor retrospective judgments are valid and cir-cumstances in which EMA data are neces-sary. In the interim, EMA is likely to provideunique insights about processes that charac-terize dynamics of behavior, cognition, andaffect over time.

ECOLOGICAL MOMENTARYASSESSMENT DESIGNSAND APPROACHES

In this section, we describe and categorizeEMA designs, by which we mean the schemethat dictates the scheduling, arrangement, andtemporal coverage of EMAs. In global assess-ments such as personality questionnaires, theresearcher assumes that the assessment cap-tures the subject’s entire experience in one fellswoop, rendering it unimportant when the as-sessment is made. In EMA, one assesses mo-ments or periods of time, raising the issue ofhow to ensure that the moments or periodsassessed are representative of the subject’s ex-perience. In many cases, the assessments canbe conceptualized as a sample of the person’sexperience or behavior. Thus, designing anEMA protocol can essentially amount to de-signing a sampling scheme for moments inan individual’s life. The most important influ-ence on the design must be the aims of thestudy.

EMA sampling and assessment schemescan be roughly divided into event-basedsampling and time-based sampling schemes(Shiffman 2007, Wheeler & Reis 1991).Event-based schemes do not aim to charac-terize subjects’ entire experience, but ratherto focus on particular discrete events or

www.annualreviews.org • Ecological Momentary Assessment 13

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

episodes in subjects’ lives—e.g., headaches(Niere & Jerak 2004) or drinking episodes(Todd et al. 2005)—and organize the data col-lection around these events. Time-based sam-pling typically aims to characterize experiencemore broadly and inclusively—e.g., observinghow mood varies over time—without a prede-fined focus on discrete events.

Event-Based Monitoring

In many disorders, the clinical and researchinterest is in particular events or episodes, e.g.,instances of drinking, violence, or panic at-tacks. These cases lend themselves to event-based monitoring, in which assessments aretriggered by the occurrence of a predefinedevent of interest to the investigator. For ex-ample, subjects might be asked to completean assessment when they have a panic at-tack (Taylor et al. 1990), engage in a so-cial interaction lasting more than 10 minutes(Reis & Wheeler 1991), or take a medica-tion ( Jonasson et al. 1999). Typically, the sub-jects themselves determine when the event hasoccurred and initiate an assessment (thoughsome events can be automatically detectedby devices; see Kop et al. 2001). Such pro-tocols require clear definitions of the event,which can be surprisingly thorny to delin-eate: If subjects are to make a record everytime they eat, does chewing gum count aseating? Sometimes, target events are definedas episodic flare-ups of otherwise continu-ous experiences, for example, a pain episodein which pain is experienced more intensely(McCarberg 2007), or an episode of intensecigarette craving (Shiffman et al. 1997a).Defining the algorithm for declaring an eventis particularly difficult—and important—inthese cases.

If one only needs a record of events, for ex-ample, to ascertain their frequency and timedistribution, subjects need only note that anevent has occurred, e.g., press a button ona recording device. More often, investiga-tors wish to collect data about the event:its duration, intensity, antecedent mood, etc.

(Schlundt et al. 1985, Shiffman et al. 1996b).If the events are too frequent, it may not berealistic to assess each event, in which casea subset of them can be sampled at random(Shiffman et al. 2002).

An important limitation in the use ofevent-based monitoring is that there is oftenno way to independently assess or verify com-pliance; i.e., there is no way to know whetherevents occurred that were not entered or (lesslikely) entries made for events that did not oc-cur. In a few studies, diary entries have beencompared to records made by separate elec-tronic devices (e.g., instrumented medicationdispensers); these have suggested that medica-tion compliance is exaggerated in self-report(see Hufford 2007). Event entries can some-times also be roughly confirmed by biochemi-cal measures; e.g., Shiffman & Paty (2003) re-ported that subjects’ electronic diary recordsof cigarettes were consistent with biochemicalmeasures of smoking. In any case, event re-ports are subject to error resulting from poorcompliance or falsification.

Time-Based Designs

Some clinical phenomena—e.g., mood,pain—vary continuously and are not easilyconceptualized in an episodic framework.In some instances (e.g., actigraphy, heartrate, skin conductance), the phenomenoncan be monitored continuously. In the moretypical case, where this is not possible, EMAprotocols rely on time-based sampling. Thereare many varieties of time-based samplingschemes, which vary in schedule, frequency,and timing (see Delespaul 1995 for moredetail).

The frequency of time-based assessmentswill determine the resolution the study willhave. The resolution needed depends on thegoal of the study, what is known about the be-havior studied, and the theoretical frameworkof the study.

A variety of time-based assessment sched-ules are used in EMA. Some administer assess-ment at fixed intervals. This has been common

14 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

in ambulatory monitoring of blood pressure(e.g., every 30 to 45 minutes in Kamarck et al.1998). Assessment at equal intervals allowsthe time block to serve as the unit of analy-sis and supports analyses that require evenlyspaced assessments, such as simple autocorre-lation analysis and time series analysis. (Dailydiaries are a special case discussed below.)Some studies have used somewhat irregu-lar intervals, typically defined by social pa-rameters: For example, Hensley et al. (2003)had subjects complete an asthma diary in themorning and evening of each day. Because thetime blocks are vaguely defined, they give sub-jects considerable discretion in the timing ofassessments, and this has the potential to in-troduce bias. For example, subjects might re-member to complete their mood assessmentswhen they are reminded by extreme affect,thus biasing the sample of mood.

An alternative to fixed intervals is a vari-able schedule, which usually administers as-sessments at random times in order to achievea representative sampling of subjects’ state. Avariation is stratified random sampling, fromwithin strata defined by blocks of time withina day. For example, Affleck and colleagues(1998) assessed subjects once in each pre-defined time window in the morning, after-noon, or evening, with assessments scheduledat random within each interval. This guaran-tees that the sample of assessments will evenlysample time across the day and ensures (sub-ject to missed assessments) that each timeblock includes an assessment, which allows thetime block to serve as the unit of analysis.

Setting the frequency of assessments in-cludes considerations of subject burden as wellas of how rapidly the target phenomenon isexpected to vary. Although assessing subjects3 to 5 times per day is common, some studieshave succeeded with as many as 20 or moreassessments per day (Goldstein et al. 1992;Kamarck et al. 1998, 2002, 2005, 2007). As-sessment frequency may be varied to collectmore assessments at certain times of day ifthose are of particular interest (Shiffman et al.2000). Assessments should ideally be sched-

PDA: personaldigital assistant (ahandheld computer)

uled throughout the waking day. Some stud-ies have limited assessment to a narrow rangeof hours (e.g., 10am to 10pm in Kimhy et al.2006), which misses early-morning and late-night hours that may encompass important—and substantially different—experiences andbehaviors.

When using time-based assessment sched-ules, especially with variable intervals, EMAstudies require some method of signaling sub-jects when an assessment is scheduled. This istypically accomplished through the use of adevice—a beeper, phone, wristwatch, or per-sonal digital assistant (PDA)—programmedto signal the subjects at the appropriate times(see Shiffman 2007 for a discussion of devices).

Thus, although events can be captured atthe subject’s initiative, continuous phenomenatypically have to be sampled using a suitabletime-based sampling scheme.

Combination Designs

Different sampling approaches can be fruit-fully combined to test particular hypotheses.When the researcher is interested in the cir-cumstances that are associated with a targetevent, it can be particularly helpful to com-bine time- and event-based assessments in or-der to provide a context for interpreting theevent data. Data on the situational contextof clinically meaningful events, such as panicattacks (Margraf et al. 1987) or binge-eatingepisodes (Engel et al. 2007) are hard to inter-pret without comparison to base-rates (Patyet al. 1992). For example, Greeno et al. (2000)found depressed affect among bingers at thetime of a binge. However, this is difficult tointerpret: Does it mean that negative moodshelp trigger binges, or just that binge eatersare generally unhappy, even when not binge-ing? By comparing affect reported during abinge with affect reported on random occa-sions outside of binges, Greeno et al. (2000)were able to establish that binges were specif-ically associated with more distress. This de-sign is modeled after the case-control design,with the events as “cases” and the nonevent

www.annualreviews.org • Ecological Momentary Assessment 15

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

data collected by time-based methods consti-tuting the “controls,” and is often referred toas a case-crossover design (Maclure & Mittle-man 2000).

In EMA designs combining event- andtime-based assessments, time-based assess-ments can also document the antecedents orsequelae of events. For example, Shiffman &Waters (2004) used time-based data to showthat ex-smokers were experiencing escalatinglevels of affective distress in the hours preced-ing a smoking lapse (the event) and to showthat self-efficacy decreased following a lapse,but not after occasions when smokers success-fully resisted a temptation to smoke (Shiffmanet al. 1997b). Time-based assessments can beused to follow up on the sequelae of a recordedevent. For example, to test how quickly mi-graine medications delivered pain relief andhow long the relief lasted, Sheftell and col-leagues (2005) had subjects record the onsetof migraines and then scheduled a series ofassessments as follow-ups.

Combining different schedules of time-based assessments can also be useful. For ex-ample, Muraven et al. (2005) assessed socialdrinking in subjects throughout the day butalso scheduled an assessment each morning toask about hangovers from the previous night’sdrinking. This illustrates how decisions aboutdesign and scheduling of assessments need tobe driven by the research questions and byknowledge about the natural history of thetarget behavior.

Use of Recall in EMA

Although some EMA studies truly focus onthe moment, many assessments involve somedegree of retrospection. For example, Affleckand colleagues (1998) asked fibromyalgia sub-jects to report on their pain and fatigue overthe past 30 minutes. Event-based diaries of-ten involve some retrospection if the entryis made after the event: For example, studieswith the Rochester Interaction Diary (Reis &Wheeler 1991) ask subjects about social in-teractions they had just concluded. Thus, the

focus on momentary experience is not abso-lute; it might more liberally be thought of asa focus on recent experience. Of course, someevents and experiences are likely to be recalledmore accurately, particularly if they are un-common and/or highly salient (e.g., a majormarital spat) or recent. However, since the lit-erature shows that recall can be biased overeven a short interval, the researcher needsto carefully consider the use of recall meth-ods and the potential for bias even over shortintervals.

Sampling Versus Coverage Strategies

When subjects are assessed intermittently andthe focus is on their momentary state, the as-sessments do not provide complete coverageof their daily experience. That is, even thoughwe may have assessed their mood 12 times in aday, and found them happy each time, it is pos-sible that they experienced moments of mis-ery in between the assessments. A samplingstrategy recognizes that only some momentsare assessed, but relies on the idea that, overmultiple days and multiple assessments, bothmoments of happiness and moments of miserywill be sampled, and the mix will be represen-tative of the subjects’ average mood.

In lieu of a sampling strategy, investiga-tors sometimes adopt a coverage strategy,which aims to cover every moment of the day.For some kinds of objective data, continu-ous measurement (e.g., actigraphy) is possi-ble, enabling true coverage (e.g., Tulen et al.2001). For self-report measures, continuousassessment is not possible, so investigatorstry to obtain coverage by having subjects re-call and summarize the period between as-sessments. This is sometimes implemented byasking subjects several times a day to recalland summarize their experience since the lastassessment, providing piecemeal coverage ofthe entire day. For example, Van den Brinket al. (2001) assessed pediatric headache pa-tients four times each day, and at each as-sessment, asked them to recall and character-ize headaches experienced since the previous

16 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

assessment. Besides being vulnerable to biasesintroduced by recall, this strategy requiressubjects to accurately maintain a timeline ofexperience and focus on experience since thelast assessment. Research suggests that peo-ple are particularly poor at keeping track oftime and identifying when events took place(Sudman & Bradburn 1973), suggesting thatthis approach needs to be treated with cau-tion. Coverage strategies may be useful whenthe investigator needs an absolute estimate orcount of events; otherwise, sampling strate-gies can yield representative estimates.

Daily Diaries: A Special Case of EMA

Daily (i.e., once-a-day) diaries are particularlypopular with researchers and clinicians be-cause of their ease of administration and rela-tively low subject burden. Within the frame-work that we have been discussing, dailydiaries are fixed-interval assessments with anassessment frequency of once per day, em-ploying a retrospective coverage strategy.Even though daily diaries rely on recall of theentire day and therefore are not momentary,we consider them as falling into the family ofEMA designs because they are administeredrepeatedly and provide a dynamic look at thevariables investigated, although with less res-olution than that of within-day assessments.

Daily diaries bring some of the benefits ofEMA in terms of illuminating process overtime, but they also carry with them some of thelimitations of retrospective assessments, ask-ing participants to summarize the entire day’sexperience. This may introduce some bias be-cause moods can vary substantially from onemoment to the next, and true recall is likelyto stretch the capabilities of autobiographi-cal memory. Moreover, research on autobi-ographical memory suggests that biases dueto heuristic recall strategies operate even overshort time frames. The typical scheduling ofdiary completion at the end of the day mayitself become a source of bias, since this tendsto be a highly unrepresentative moment, whensubjects may be particularly fatigued, for ex-

ample, which could bias their recall of the day’sexperience. Accurate recall may be more likelywhen subjects are asked to recall less frequent,more notable and salient events, such as theoccurrence of a migraine headache. Compar-isons of daily diary data to real-time data willenable researchers to determine whether theformer is a reasonable proxy for the latter. Re-search is needed to shed light on when ac-curate recall is and is not likely. Thus, dailydiaries require a cautious approach.

Besides concern about retrospective bias,daily assessment also provides limited tempo-ral resolution in which to observe behavior;that is, within-day processes may be criticallyimportant for some phenomena. This was il-lustrated in a study by Shiffman & Waters(2004) on the antecedents of smoking lapses.Daily diary data showed no increase in nega-tive affect in the days leading up to a lapse andno association between daily negative affectand lapse risk. However, examination of hour-by-hour affect data revealed a steep increasein negative affect in the hours immediatelypreceding lapse episodes. Thus, day-level datamay miss important sources of dynamic varia-tion that drive behavior. However, when usedcarefully, especially for highly memorable andslow-moving targets, daily diaries can never-theless be a tremendous asset for studying cer-tain questions and are often an improvementover relying on global retrospective recall.

USE OF ECOLOGICALMOMENTARY ASSESSMENTIN TREATMENT

For Assessment

Although we have been addressing the use ofEMA for research data, it also seems naturalto use EMA for ongoing assessment duringtreatment. The treatment context includesmany features that lend themselves to EMAmethods: The client is likely motivated todevote energy to assessment, and there areusually clear target behaviors or experienceson which to focus, dictated by the client’s

www.annualreviews.org • Ecological Momentary Assessment 17

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

presenting problem, the nature of the pathol-ogy, and/or the clinical formulation of thecase. EMA methods lend themselves to idio-graphic n = 1 analysis; even formal statisticaltesting can be done by using a single client’sdata to guide therapy and assess progress (e.g.,Delespaul 1995). Properly structured EMAdata lend themselves to the sort of micro-analysis of process that is often the subject oftherapeutic discussion, and that is seen as re-vealing opportunities for intervention (“Let’sfigure out what led up to your anxiety attackso we can understand it and think about howto prevent it”). Because change is expectedduring treatment, ongoing assessment can beinformative. EMA could also prove useful inshedding light on processes and mediators ofpsychotherapy-induced change.

EMA assessments are only beginning tosee application in clinical settings. Norton andcolleagues (2003) conducted a small outcomestudy of CBT for binge eating, in which pa-tients were randomized to use EMA-basedassessment or not. There were no signifi-cant differences in the primary outcome, butthere were strong trends favoring the pa-tients with EMA, suggesting that a largerstudy would have demonstrated reliable dif-ferences. Protocols to make optimal use ofEMA in treatment have yet to be developed.Widespread use of EMA in treatment willalso have to overcome both real and perceivedbarriers of the technical and financial bur-den of these methods. We expect that EMAmethods will see increasing use in clinical set-tings over the next decade (see Piasecki et al.2007 for a review of EMA applied to clinicalassessment).

For Intervention

Although using EMA for clinical assessmentwould be a minor extension of its use inresearch, applying EMA methods directlyto intervention—that is, implementing real-time in-the-moment interventions as sub-jects go about their daily lives—may openup more radical developments. Bridging the

gap between patients’ here-and-now experi-ences and behaviors in the real world and theremoved, confined, and limited contact withthe clinician in the consulting room has al-ways been a major challenge for psychologicaltreatment. Only a few papers have describedmomentary interventions, and fewer still haveevaluated them.

Newman et al. (2003) reviewed a vari-ety of palm-top-assisted treatments for psy-chological disorders. Newman et al. (1997)reported that a brief palm-top-assisted mo-mentary intervention for panic disorder wasequivalent in efficacy to a longer therapist-administered treatment. The control groupin this study also engaged in EMA andmay have benefited from that, which high-lights the importance of identifying theincremental benefits of EMA interventionversus EMA. Momentary interventions havealso been described for eating disorders(Norton et al. 2003) and addictive disorders(Riley et al. 2002). Carter et al. (2007) describea particularly sophisticated and interactiveEMA-based treatment program for smoking(still under evaluation) and present a concep-tual foundation of EMA-based treatments.

At this stage, then, EMA-based momen-tary intervention remains a promising, butonly partly proven, idea. However, its poten-tial seems enormous. The idea of momentarytreatment, delivering intervention immedi-ately on-the-spot and as needed in real-worldsettings—a “therapist in your pocket”—holdspromise for addressing behavior at crucialmoments in the patient’s life. EMA-basedtreatment can provide behavioral guidance orother interventions (e.g., relaxation stimuli)throughout a patient’s day as well as just at themoment they are needed. By learning froman individual’s history, algorithms could, forexample, tailor coping suggestions based onwhat has worked before for this patient in thissituation. Furthermore, by observing the pa-tient over time, predictive algorithms couldanticipate and respond to challenges beforethey gain strength, for example, noting risingstress levels and intervening (or contacting a

18 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

counselor) before they result in maladaptivebehavior. In-the-moment interventions haveonly begun to be explored, but they have thepotential to revolutionize clinical treatment.

MEASUREMENTCONSIDERATIONS FORECOLOGICAL MOMENTARYASSESSMENT SELF-REPORTS

When self-report assessments are used inEMA, some special considerations are needed.Many questionnaires were not designed forassessing momentary states; the instructionsmay have a longer recall time frame or notime frame at all. Whether adapting an ex-isting instrument for EMA or creating a newone, it is important to consider whether theitem makes sense in the new time frame—i.e.,momentary, hourly, etc.—both from the in-vestigator’s conceptual perspective and fromthe subject’s perspective. For the latter, cog-nitive testing of proposed assessments (Willis2005), as well as pilot testing of assessmentsin the field, is usually prudent.

An unusual aspect of EMA studies is thatsubjects will typically encounter each assess-ment many, many times. This requires thatassessments be well tuned: Something that isjust a small annoyance when seen once canbecome a real irritant when encountered fivetimes a day for weeks.

Construction of EMA assessments mayalso be influenced by the device or mediumused. For example, PDAs have limited screendisplay space, requiring that items and re-sponse options be reasonably succinct. Wheninteractive voice response systems are used,the length, complexity, and number of op-tions needs to be severely limited, keeping inmind that subjects must listen to and remem-ber all the response options and their corre-sponding keypad numbers. Simply moving aquestionnaire from paper to a screen-basedelectronic device should not be problematic:A meta-analysis showed that paper-based andscreen-based questionnaires were equivalent(Gwaltney et al. 2007).

Use of programmable electronic devicesto administer assessments provides oppor-tunities to change the order in which as-sessment items are administered. This canbe used to present items in random orderto counterbalance any effects of item se-quencing (for free access to such software,see http://www.experiencesampling.org/).This programmability could be used to im-plement computer-adaptive testing (Chang &Reeve 2005), which provides efficient assess-ment of a construct with a minimum num-ber of items, by dynamically changing whichitems are administered to particular subjectsbased on their responses to prior items.

Although most EMA studies collect struc-tured quantitative data (using structuredscales to quantify responses), EMA meth-ods can also be used to gather qualitativedata. Diaries can allow write-in responses, andO’Connell et al. (2002) have supplementedPDAs for quantitative assessment with taperecorders for collecting qualitative narrativesfor later coding. Automatically activated taperecorders have also been used to collect qual-itative data (i.e., conversations) without theneed for subject intervention (Mehl et al.2001). Assessments of physiological parame-ters usually bring with them a set of techni-cal concerns specific to the system being mea-sured and the method used.

Besides keeping in mind the special mea-surement issues that apply to EMA, it isequally important to recognize that many ofthe measurement issues that apply to globalpsychometric assessment also apply to EMA.That is, it is important that assessments be re-liable and valid; in EMA, reliability can some-times be achieved through aggregation acrossmultiple assessments rather than across mul-tiple items within a single assessment. Themeaning of EMA assessments has to be care-fully considered; Schwarz (2007) has spec-ulated that focusing self-report on immedi-ate experience might shift the individual’sfocus to very small events at the cost of the“big picture.” It is also important to keepin mind that self-report data collected via

www.annualreviews.org • Ecological Momentary Assessment 19

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

EMA methods are still self-report data andare subject to many of the limitations inher-ent to self-report. Like any self-report data,EMA data can be adversely influenced by theeffects of subjects’ psychopathology (Kessleret al. 2000) and by deception or self-deception.

METHODOLOGICALCONSIDERATIONS INECOLOGICAL MOMENTARYASSESSMENT STUDIES

Reactivity

Reactivity is defined as the potential for be-havior or experience to be affected by theact of assessing it. Behavioral studies of self-monitoring, in which patients were asked tomonitor the behavior they were trying tochange, often demonstrated reduction in theproblem behavior due to monitoring alone—so much so that self-monitoring came to beconsidered a part of behavior-change treat-ment, not just assessment. However, reactivityis not universally observed. In particular, stud-ies showed that reactivity emerged when sub-jects were trying to change the target behaviorand when the behavior was recorded prior tobeing executed (e.g., recording a meal beforeeating it)—in other words, under conditionsthat provide both the motivation and the op-portunity to exert control over the behavior(Korotitsch & Nelson-Gray 1999). This sug-gests that EMA researchers need to be con-cerned about reactivity under such conditions,although there has not yet been a direct testof this idea.

When such conditions are not present,several studies of EMA find little or noevidence of reactive effects (Cruise et al.1996; Hufford & Shields 2002; Hufford et al.2002a,b; Hufford & Shiffman 2002; Littet al. 1998; Stone et al. 1998). In the mostcontrolled study of reactivity to EMA, Stoneand colleagues (2003a) assigned subjects withpain syndromes to complete either no EMAmonitoring or sampling of their pain using

electronic diaries 3, 6, or 12 times daily. Noevidence was found that the pain ratingswere systematically reactive to the EMAmonitoring.

Besides traditional reactivity, one mightalso be concerned that the burden of moni-toring, including being interrupted by “beep-ing” throughout the day, might cause distressthat would bias the data. Again, the evidencesuggests this is not typical. The Stone et al.(2003a) study found no trend over time in painratings and no systematic effect on pain rat-ings of increased prompting, despite a delib-erately aggressive prompting and assessmentschedule.

Given the earlier literature suggestingthat reactivity can occur, the literature doesnot completely resolve when reactive effectsmight or might not be observed. EMA re-searchers should be alert to the potential forreactivity while recognizing that little evi-dence has been found to support the concernthat EMA engenders significant reactivity.

Compliance

While EMA methods promise to capturedata closer in time to events and symptomsof interest, they also place on subjects theburden of making timely recordings as theygo about their daily lives. Missing assessmentshave the potential to bias the obtained sampleof behavior and experience, especially if themissing data are nonrandom. As a result,EMA studies place a premium on obtaininghigh levels of subject compliance with theassessment protocol.

The methods used to collect diary data canaffect the investigator’s ability to obtain anddocument compliance. Studies using paper di-aries often report that about 90% of diarycards are returned (Hufford & Shields 2002).However, timely compliance is often inferredfrom the date and time subjects have recordedon the diary card. This opens the door for sub-jects to hoard and backfill diaries (that is, tocomplete entries, perhaps en masse, after thefact) while seeming compliant.

20 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Concern about hoarding and backfillingis heightened by the findings from studiesin which subjects completed written diarieswhile also using electronically instrumenteddevices (e.g., inhalers, blood glucose moni-tors) that independently tracked the actualtime and date of events, such as medica-tion taking or glucose monitoring. Acrosseight studies (Hufford 2007), objective datafrom the devices indicated that compliancehad actually been much lower than was in-dicated by the written diaries: Many of thewritten entries had been falsified. Moreover,the written entries were systematically biased,for example, showing better glycemic con-trol than had actually been achieved (Mazzeet al. 1984). The written diary records un-derstated problems of glucose control and ac-centuated the concern about backfilling andfaked compliance. These concerns were fur-ther accentuated by reports from paper di-ary studies in which high proportions of sub-jects later admitted to backfilling entries, evenwhen they had been electronically promptedat the appropriate times and had recordedthose times on their diary cards (Litt et al.1998, Moghaddam & Ferguson 2007).

To assess true compliance with written di-aries, Stone et al. (2002) used an instrumentedpaper diary (IPD), which was covertly instru-mented with photosensors that recorded theopening of the diary booklet—a prerequisiteto completing diary cards. Chronic pain sub-jects were assigned to use either the IPD ora PDA-based electronic diary to record theirpain at three scheduled times daily for threeweeks. Subjects were trained on the diary,and steps were taken to encourage motivationand compliance. The IPD allowed compar-ison of reported compliance—based on thetime and date handwritten on returned diarycards—and actual compliance—based on theelectronic records. Reported compliance was90%, as in many other paper diary studies.However, actual compliance was estimated at11%. In other words, almost 90% of the sub-mitted diary cards had been falsified. Evidenceindicated that the diaries had been hoarded

for days and backfilled. More disturbing still,many subjects also showed evidence of fill-ing their paper diary cards in advance of thestated time (Stone et al. 2003b), which ob-viously renders the data invalid. Brodericket al. (2003) subsequently showed that addingan audible reminder to complete the diaryboosted compliance only modestly.

The other 40 pain sufferers in the studyby Stone et al. (Stone et al. 2002, 2003a,b;Stone & Shiffman 2002) had been given anelectronic diary, which prompted for assess-ments and did not allow entries outside thedesignated time windows. These subjects ac-tually did complete 94% of their assigned as-sessments on time.

These compliance findings by Stone andcolleagues have stirred debate among diaryresearchers (Green et al. 2006, Tennen et al.2006), with critics noting, for example, thatvarious procedures (e.g., having subjects re-turn diaries daily by mail, so that postmarksprovide some degree of time-stamping) mightimprove timely compliance with written di-aries. In any case, the Stone et al. study (Stoneet al. 2002, 2003a,b; Stone & Shiffman 1994,2002), along with the prior literature docu-menting falsified paper diary entries (Hufford2007), raises critical questions about timelycompliance. Paper-and-pencil diaries are vul-nerable to being falsified, and both forward-and backfilling are possible and potentiallyfrequent, making verification of timely com-pliance both essential and challenging. Thus,the burden of proof falls to the EMA inves-tigator to establish, by whatever means, thetimeliness of recording.

The most common way to ensure and doc-ument timely compliance is the use of elec-tronic diaries, which tag each record with timeof entry and thus allow for detailed analy-sis of compliance, particularly with interval-or signal-contingent recordings. Many elec-tronic diary studies document compliancerates around 90% (Hufford & Shields 2002).However, good compliance is not univer-sal even with electronic diaries; rates as lowas 50% have been reported (e.g., Jamison

www.annualreviews.org • Ecological Momentary Assessment 21

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

et al. 2001), suggesting that the mere useof electronic diary technology is not, by it-self, a panacea. A variety of suggestions ofways to enhance compliance with electronicdiary studies has been presented elsewhere(Hufford & Shiffman 2003). As noted above,compliance with event-oriented protocols isdifficult to verify.

SPECIAL POPULATIONS

Because of the burdens they place on subjects,EMA methods often raise concerns about ap-plication to special populations. These con-cerns are often magnified when EMA stud-ies employ electronic devices, such as PDAsor cell phones, for data collection. A com-mon concern is that elderly subjects willbe unable or unwilling to operate high-techdevices. However, data on subject prefer-ence for electronic versus more traditionalpaper diaries do not support this assump-tion. In a meta-analysis of patient prefer-ence, older subjects are actually less likelyto prefer paper diaries (in Shiffman et al.2007). Similarly, although there have beenconcerns about the use of such technologyin low-education, low-socioeconomic-statussubjects, Finkelstein et al. (2000) demon-strated good success with just such a sample.

Different concerns are sometimes raisedabout the use of diaries with children. Sev-eral studies have demonstrated that childrenare able to use electronic diaries, and goodcompliance and results have been seen withchildren as young as 7 (see Van den Brinket al. 2001). Of course, the use of EMA meth-ods does not obviate, and may increase, somemore general issues about assessment of chil-dren, such as comprehension of the question-naires used.

Concerns also arise about whether someclinical populations might be unable to per-form in EMA studies precisely because of theirpsychopathology. In this regard, it is notablethat several studies have been successfullycarried out among schizophrenics (Delespaul1995, Myin-Germeys et al. 2001), and good

compliance using cell phones for EMA re-porting has been reported in a sample ofhomeless crack addicts (Freedman et al. 2006).These examples suggest that, with appropriatesensitivity, EMA methods and related tech-nology may be useful in a wide array of patientpopulations and that investigators should notmake decisions about feasibility without test-ing their assumptions.

A subtler concern that is not addressedby the performance of special populationsin EMA studies is that subjects who volun-teer for and complete demanding EMA stud-ies may not be representative. There havebeen no reports of particular recruitment dif-ficulties or notable deviations in EMA studysamples, but this has not been systematicallyevaluated. It seems likely that personal andenvironmental factors may discourage somesubjects’ participation in EMA studies. Oc-cupational demands may preclude availabil-ity for momentary assessment (consider sur-geons and assembly line workers, although seeGoldstein et al. 1992). Also, some work en-vironments may be so noisy as to precludehearing prompts or using a phone. Motor im-pairments, impaired vision or hearing, and il-literacy may preclude participation in EMA(and in many non-EMA) studies. Finally, sub-jects who are unfamiliar with or fearful oftechnology may be put off by EMA studiesusing high-tech devices. The limits of EMAand their impact on studies should be contin-ually evaluated.

ANALYSIS OF EMA DATA

EMA data usually consist of a large number ofobservations from each subject, with the num-ber and timing of observations often vary-ing between subjects. EMA data do not lendthemselves to the basic approaches most clin-ical psychologists learned in graduate-schoolstatistics classes, which require independentobservations. Nor do they fit easily into tradi-tional repeated-measures designs, which typi-cally require the same number of observationsfor each person and no missing data. However,

22 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

a variety of methods—essentially extensionsof the regression framework most psycholo-gists are familiar with—can handle such data.Known variously as random effects models,multilevel or hierarchical analysis, and gen-eralized estimating equations, these methodsimplement regression models while account-ing for the interrelationships of observationswithin subjects. Extensions of these methodsgo beyond linear regression to cover logisticregression and survival models, among oth-ers. Detailed exposition about these methodsis beyond the scope of this review; the readeris referred to Schwartz & Stone (1998, 2007)for detailed and practical guidance on appli-cations of these models to EMA data. Thebottom line is that EMA researchers can useeasily available software to apply the concep-tual models they are familiar with from moretraditional between-subjects analyses to EMAdata. Besides these relatively familiar statisti-cal models, a variety of novel approaches canfruitfully be applied to EMA data (Walls &Schafer 2006).

PRACTICAL ISSUES INECOLOGICAL MOMENTARYASSESSMENT

The implementation of EMA places uniquedemands on both researchers and subjects. Adetailed discussion of the practical aspects ofimplementing EMA is beyond the scope ofthis review. The interested reader is referredto Hufford (2007). Here we touch only brieflyon considerations of hardware and software,and management of subjects and data.

EMA Hardware and Software

An investigator planning an EMA study facesmany practical issues and decisions. One keydecision is what type of mechanism to use tocapture data. Although EMA is most closelyassociated with the use of palm-top comput-ers (Hufford & Shields 2002), the method ishardware agnostic and encompasses a rangeof high- and low-tech devices, ranging from

paper and pencil to futuristic wireless sen-sors (Intille 2007). The key factor to con-sider in selecting a hardware platform is thesupport provided for key functions, includingthe ability to prompt subjects to complete as-sessments, manage prompting schedules (in-cluding randomization and contingencies),present assessment content to subjects, man-age assessment logic such as branching, ac-cept and store subjects’ self-report data, time-stamp entries to avoid faked compliance,and/or directly collect physical or physio-logical data (see Shiffman 2007). Investiga-tors should also consider the suitability of aparticular device or medium to their subjectpopulation: For example, a small screen or atelephone system may be less suitable for anelderly population with visual or hearing im-pairments, respectively.

The cost of devices and of software devel-opment is often an additional consideration.Investigators should avoid being overly influ-enced by initial start-up costs and also shouldconsider downstream costs, such as the costof cell phone service or of data entry from pa-per diaries. Unfortunately, high fixed costs oftechnological solutions can make it very hardto initiate small studies or pilot studies.

EMA approaches using electronic devicesrequire software for basic system functional-ity and for the specifics of the study proto-col. The success of such studies depends oncorrect and robust performance of software,which can often be complex. Both freewareand commercial versions of EMA software arenow readily available. Le et al. (2006) have re-viewed a number of freely available programs,and a number of commercial vendors also pro-vide programming and support services forEMA studies. These base systems have to beconfigured or customized to the needs of aparticular study. Researchers should carefullyconsider capabilities and limitations of varioussoftware systems. For example, some systemscan only accommodate signaling during speci-fied periods of the day, which can bias the sam-ple by, for example, omitting early-morningand late-night events, which may be highly

www.annualreviews.org • Ecological Momentary Assessment 23

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

salient. Some programs may support event- orsignal-driven assessments, but not the combi-nation of the two. In light of the availabilityof software from multiple sources, investiga-tors are well advised to use existing software orservices.

Other Practical Considerations

Investigators should also consider proceduresand resources related to managing studysubjects. Training of subjects on the protocol,the assessment, and any device used is an es-sential aspect of EMA studies. Data manage-ment is an additional consideration in EMAstudies. EMA studies can produce volumi-nous datasets with hundreds of thousands ofobservations, possibly of diverse kinds (e.g.,momentary, daily, events). Additional compli-cations include the fact that the records areordered in time and the temporal relation-ships need to be maintained, and there may benondata records that do not represent subjectassessments (e.g., records of missed assess-ment prompts) that also have to be maintainedwithin the database. Data management facili-ties for maintaining these databases and pro-ducing analytical datasets are an essential re-source for EMA studies and should be plannedinto the study.

CONCLUSION

EMA studies help us appreciate the impor-tance of change and context in everyday be-havior and experience. The dynamic influenceof context is demonstrated not only for con-structs that we might expect to be variable,such as affect, craving, and pain, but also formany constructs that are typically conceptu-alized as stable trait-like characteristics, suchas extraversion (Fleeson 2001), self-esteem(Kernis 2005), coping style (Schwartz et al.1999), and self-efficacy (Gwaltney et al. 2005).EMA’s microscopic focus and temporal res-olution is beginning to help us see the un-derlying dynamics behind even trait-like con-

structs. EMA studies suggest that behaviorand experience are much more dynamic, andmuch more influenced by its immediate con-text, than we usually envision. Even individualdifferences can sometimes be best understoodthrough analysis of within-person variation.For example, differences in how persons reactto situational stimuli, such as stressors, are re-lated to personality (Feldman-Barrett 1997),psychopathology (Myin-Germeys et al. 2001),and clinical outcomes (Kamarck et al. 2007),and within-person variation in response maysometimes be as important as the mean level(Gwaltney et al. 2005, Kernis 2005). Despitetheir limitations, EMA methods promise toenhance our understanding of the dynamicinteractions between individuals and theirenvironments.

Clinical psychology—and psychology ingeneral—is prone to overestimate the role ofstable traits in determining behavior and tounderestimate the influence of the local set-ting on behavior (Mischel 2004). Our behav-ior in daily life is influenced not just by ourpredispositions, but also by where we are andwhom we are with, by how we are feeling andwhat situation we are in, by what has recentlyhappened and what we have done or felt inthe minutes and hours preceding the presentmoment. One context can elicit dysfunctionalbehavior while another elicits a healthier re-sponse; one can fuel psychopathology or setus on a course toward improvement. Under-standing life as it is lived, up close, will helpus better understand both health and pathol-ogy and help us see where there are oppor-tunities to intervene on the side of health.This requires methods that examine behav-ior at the appropriate level of granularity:“Research methods that examine this land-scape from 10,000 feet cannot shed light onhow this landscape is shaped at ground level”(Shiffman 2005, p. 1743). EMA methods pro-vide an important tool to help clinical psy-chology explore the dynamic nature of behav-ior, thought, and feeling as they unfold overtime.

24 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

SUMMARY POINTS

1. Autobiographical memory processes can introduce bias into retrospective self-reports,which form the bulk of clinical assessments.

2. EMA is a collection of methods for obtaining repeated real-time assessments of sub-jects’ behavior and experience in their natural environments, to minimize recall bias,maximize ecological validity, and document variation over time.

3. EMA encompasses a variety of diary approaches and technologies used to collect dataon a schedule (e.g., daily diaries or assessments scheduled at random) or in responseto clinical events (e.g., symptom episodes or behaviors).

4. The collection of frequent assessments makes EMA studies well suited to studymicroprocesses—i.e., how behavior and experience vary over time and acrosschanging contexts.

5. Although sometimes retrospective self-reports correspond well with those based onaggregated EMA data, recall of behavior and experience over time does not correspondwell to real-time EMA assessments.

6. The application of EMA methods for assessment and intervention in treatment ispromising but is in its infancy.

7. EMA methods have great potential to advance the science and practice of clinicalpsychology by providing more valid and more detailed data about real-world behaviorand experience.

FUTURE ISSUES

1. Further prospective longitudinal analyses of EMA data are necessary to shed light ontheoretically relevant microprocesses in studies of psychopathology and treatment.

2. Further improvements in technological tools for EMA studies would enhance ease ofuse, reduce cost, and expand capabilities.

3. An improved understanding is needed of the relationship between recall and real-timeEMA data, the circumstances under which recall may be accurate, and the processesby which persons develop retrospective accounts of their experience.

4. Models are needed for application of EMA methods to clinical assessment andintervention.

ACKNOWLEDGMENTS

The authors are grateful to Julie Mickens for invaluable help in preparing this manuscript.

DISCLOSURE STATEMENT

S. Shiffman is a founder and Chief Science Officer of invivodate, Inc., which provides electronicdiary software and services for research. A.A. Stone and M.R. Hufford own stock in invivodata,Inc.

www.annualreviews.org • Ecological Momentary Assessment 25

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

LITERATURE CITED

Affleck G, Tennen H, Urrows S, Higgens P, Abeles M, et al. 1998. Fibromyalgia and women’spursuit of personal goals: a daily process analysis. Health Psychol. 17:40–47

Barker R. 1978. Habitats, Environments and Human Behavior: Studies in the Ecological Psychologyand Ecobehavioral Science of the Midwest Psychological Field Station: 1947–1972. San Francisco,CA: Jossey-Bass

Beal DJ, Weiss HM. 2003. Methods of Ecological Momentary Assessment in organizationalresearch. Organ. Res. Methods 6:440–64

Bolger N, Davis A, Rafaeli E. 2003. Diary methods: capturing life as it is lived. Annu. Rev.Psychol. 54:579–616

Bolger N, Laurenceau J-P. 2005. Using diary methods to study marital and family processes.J. Family Psychol. 19:86–97

Summarizes biasesrelated toautobiographicalmemory, and howthey can biasresearch data.

Bradburn N, Rips L, Shevell S. 1987. Answering autobiographical questions: the impactof memory and inference on surveys. Science 236:157–61

Broderick JE, Schwartz JE, Shiffman S, Hufford MR, Stone AA. 2003. Signaling does notadequately improve diary compliance. Ann. Behav. Med. 26:193–248

Broderick JE, Schwartz JE, Stone AA. 2006. Context ( pain and affect) influences recall pain ratings.Presented at Annu. Meet. Am. Pain Soc., San Antonio, TX

Byerly M, Fisher R, Whatley K, Holland R, Varghese F, et al. 2005. A comparison of electronicmonitoring vs clinician rating of antipsychotic adherence in outpatients with schizophre-nia. Psychiatry Res. 133:129–33

Carney MA, Tennen H, Affleck G, del Boca FK, Kranzler HR. 1998. Levels and patterns ofalcohol consumption using timeline follow-back, daily diaries and real-time “electronicinterviews.” J. Stud. Alcohol 59:447–54

Carter BL, Day SX, Cinciripini PM, Wetter DW. 2007. Momentary health interventions:Where are we and where are we going? See Stone et al. 2007b, pp. 289–307

Chang C, Reeve BB. 2005. Item response theory and its applications to patient-reported out-comes measurement. Eval. Health Prof. 28:264–82

Clark DM, Teasdale JD. 1982. Diurnal variation in clinical depression and accessibility ofmemories of positive and negative experiences. J. Abnorm. Psychol. 91:87–95

Cramer JA, Mattson RH, Prevey ML, Scheyer RD, Ouellette VL. 1989. How often is medi-cation taken as prescribed: a novel assessment technique. J. Am. Med. Assoc. 261:3273–77

Cruise CE, Broderick JE, Porter L, Kaell A. 1996. Reactive effects of diary self-assessment inchronic pain patients. Pain 67:253–58

Curry S, Marlatt GA, Gordon JR. 1987. Abstinence violation effect: validation of an attribu-tional construct with smoking cessation. J. Consult. Clin. Psychol. 55:145–49

Delespaul P. 1995. Assessing Schizophrenia in Daily Life: The Experience Sampling Method. Maas-tricht, The Netherlands: Maastricht Univ. Press

DeVries M, ed. 1992. The Experience of Psychopathology: Investigating Mental Disorders in TheirNatural Settings. Cambridge, UK: Cambridge Univ. Press. 429 pp.

Eich E, Reeves J, Jaeger B, Graff-Radford S. 1985. Memory for pain: relation between pastand present pain intensity. Pain 223:375–79

Engel SG, Boseck J, Crosby RD, Wonderlich SA, Mitchell JE, et al. 2007. The relationship ofmomentary anger and impulsivity to bulimic behavior. Behav. Res. Ther. 45:437–47

Fahrenberg J, Myrtek M. 2001. Progress in Ambulatory Assessment: Computer-AssistedPsychological and Psychophysiological Methods in Monitoring and Field Studies. Seattle, WA:Hogrefe & Huber

26 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Feldman-Barrett L. 1997. The relationships among momentary emotion experiences, per-sonality descriptions, and retrospective ratings of emotion. Personal. Soc. Psychol. Bull.23:1100–10

Feldman-Barrett L, Russell JA. 1998. Independence and bipolarity in the structure of currentaffect. J. Personal. Soc. Psychol. 74:967–84

Finkelstein J, Cabrera MR, Hripcsak G. 2000. Internet-based home asthma telemonitoring.Chest 117:148–55

Fleeson W. 2001. Toward a structure- and process-integrated view of personality: traits asdensity distributions of states. J. Personal. Soc. Psychol. 80:1011–27

Freedman M, Lester K, McNamara C, Milby J, Schumacher J. 2006. Cell phones for EcologicalMomentary Assessment with cocaine-addicted homeless patients in treatment. J. Subst.Abuse Treat. 30:105–11

Goldstein I, Jamner L, Shapiro D. 1992. Ambulatory blood pressure and heart rate in healthymale paramedics during a workday and a nonworkday. Health Psychol. 11:48–54

Green AS, Rafaeli E, Bolger N, Shrout PE, Reis HT. 2006. Paper or plastic? Data equivalencein paper and electronic diaries. Psychol. Methods 11:87–105

Greeno CG, Wing R, Shiffman S. 2000. Binge antecedents in obese women with and withoutBinge Eating Disorder. J. Consult. Clin. Psychol. 68:95–102

Gwaltney CJ, Shields AS, Shiffman S. 2007. Equivalence of electronic and paper-and-penciladministration of patient reported outcome measures: a meta-analytic review. Value Health.In press

Gwaltney CJ, Shiffman S, Sayette MA. 2005. Situational correlates of abstinence self-efficacy.J. Abnorm. Psychol. 114:649–60

Describes thedevelopment of theExperienceSampling Methodand reviewsfindings fromstudies applying itto a variety ofdomains.

Hektner JM, Schmidt JA, Csikszentmihalyi M. 2007. Experience Sampling Method: Mea-

suring the Quality of Everyday Life. Thousand Oaks, CA: SageHensley MJ, Chalmers A, Clover K, Gibson PG, Toneguzzi R, Lewis PR. 2003. Symptoms of

asthma: comparison of a parent-completed retrospective questionnaire with a prospectivedaily symptom diary. Pediatr. Pulmonol. 36:509–13

Homma Y, Ando T, Yoshida M, Kageyama S, Takei M, et al. 2002. Voiding and incontinencefrequencies: variability of diary data and required diary length. Neurourol. Urodyn. 21:204–9

Houtveen JH, Oei NYL. 2007. Recall bias in reporting medically unexplained symptoms comesfrom semantic memory. J. Psychosom. Res. 62:277–82

Hufford MR. 2007. Special methodological challenges and opportunities in Ecological Mo-mentary Assessment. See Stone et al. 2007b, pp. 54–75

Hufford MR, Shields AL. 2002. Electronic diaries: an examination of applications and whatworks in the field. Appl. Clin. Trials 11:46–56

Hufford MR, Shields AL, Shiffman S, Paty J, Balabanis M. 2002a. Reactivity to Ecological Mo-mentary Assessment: an example using undergraduate problem drinkers. Psychol. Addict.Behav. 16:205–11

Hufford MR, Shiffman SS. 2002. Methodological issues affecting the value of patient-reportedoutcomes data. Exp. Rev. Pharmacoecon. Health Outcomes 2:119–28

Hufford MR, Shiffman S. 2003. Patient-reported outcomes: assessment methods. Dis. Manag.Health Outcomes 11:77–86

Hufford MR, Stone AA, Shiffman S, Schwartz JE, Broderick JE. 2002b. Paper vs. electronicdiaries: compliance and subject evaluations. Appl. Clin. Trials 11(8):38–43

Intille S. 2007. Technological innovations enabling automatic, context-sensitive EcologicalMomentary Assessment. See Stone et al. 2007b, pp. 308–37

www.annualreviews.org • Ecological Momentary Assessment 27

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Jamison RN, Raymond SA, Levine JG, Slawsby EA, Nedeljkovic SS, Katz NP. 2001. Electronicdiaries for monitoring chronic pain: one-year validation study. Pain 91:277–85

Jonasson G, Carlsen K, Sodal A, Jonasson C, Mowinckel P. 1999. Patient compliance in aclinical trial with inhaled budesonide in children with mild asthma. Eur. Respir. J. 14:150–54

Kamarck TW, Janicki DL, Shiffman S, Polk DE, Muldoon MF, et al. 2002. Psychosocial de-mands and ambulatory blood pressure: a field assessment approach. Physiol. Behav. 77:699–704

Kamarck TW, Muldoon M, Shiffman S, Sutton-Tyrrell K. 2007. Experiences of demandand control during daily life are predictors of carotid atherosclerotic progression amonghealthy men. Health Psychol. 26:324–32

Kamarck TW, Schwartz J, Shiffman S, Muldoon M, Sutton-Tyrrell K, Janicki D. 2005. Psy-chosocial stress and cardiovascular risk: What is the role of daily experience? J. Pers.73:1749–74

Kamarck TW, Shiffman S, Smithline L, Goodie JL, Paty J, et al. 1998. Effects of task strain,social conflict, and emotional activation on ambulatory cardiovascular activity: daily lifeconsequences of recurring stress in a multiethnic adult sample. Health Psychol. 17:17–29

Kernis MH. 2005. Measuring self-esteem in context: the importance of stability of self-esteemin psychological functioning. J. Personal. 73:1–37

Kessler R, Wittchen H-U, Abelson J, Zhao S. 2000. Methodological issues in assessing psy-chiatric disorders with self-reports. See Stone et al. 2000, pp. 229–55

Kihlstrom J, Eich E, Sandbrand D, Tobias B. 2000. Emotion and memory: implications forself-report. See Stone et al. 2000, pp. 81–100

Kimhy D, Delespaul P, Corcoran C, Ahn H, Yale S, Malaspina D. 2006. Computerized expe-rience sampling method (ESMc): assessing feasibility and validity among individuals withschizophrenia. J. Psychiatr. Res. 40:221–30

Kop W, Verdino R, Gottdiener J, O’Leary S, Bairey Merz C, Krantz D. 2001. Changes inheart rate and heart rate variability before ambulatory ischemic events. J. Am. Coll. Cardiol.38:742–49

Korotitsch WJ, Nelson-Gray RO. 1999. An overview of self-monitoring research in assessmentand treatment. Psychol. Assess. 2:415–25

Larson R, Richards MH. 1994. Divergent Realities: The Emotional Lives of Mothers, Fathers andAdolescents. New York: Basic Books

Le B, Hat NC, Beal DJ. 2006. Pocket-sized psychology studies: exploring daily diary softwarefor PalmPilots. Behav. Res. Methods 38:325–32

Litt MD, Cooney NL, Morse P. 1998. Ecological Momentary Assessment (EMA) with treatedalcoholics: methodological problems and potential solutions. Health Psychol. 17:48–52

Litt MD, Cooney NL, Morse P. 2000. Reactivity to alcohol-related stimuli in the laboratoryand in the field: predictors of craving in treated alcoholics. Addiction 95:889–900

Maclure M, Mittleman MA. 2000. Should we use a case-crossover design? Annu. Rev. PublicHealth 21:193–221

Margraf J, Taylor CB, Ehlers A, Roth WT. 1987. Panic attacks in the natural environment.J. Nerv. Ment. Dis. 175:558–65

Mazze RS, Shamoon H, Pasmantier R, Lucido D, Murphy J, et al. 1984. Reliability of bloodglucose monitoring by subjects with diabetes mellitus. Am. J. Med. 77:211–17

McCarberg BH. 2007. The treatment of breakthrough pain. Pain Med. 8:S8–13McCarthy DE, Piasecki TM, Fiore MC, Baker TB. 2006. Life before and after quitting smok-

ing: an electronic diary study. J. Abnorm. Psychol. 115:454–66

28 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

McFarland C, Ross M, DeCourville N. 1989. Women’s theories of menstruation and biases inrecall of menstrual symptoms. J. Personal. Soc. Psychol. 57:522–31

McKenzie S, Paty J, Grogan D, Rosano M, Curry L, et al. 2004. Trial power: how eDiarydata can enhance the power and reduce the sample size in clinical trials. Appl. Clin. Trials13:54–68

Mehl MR, Pennebaker JW, Crow DM, Dabbs J, Price JH. 2001. The Electronically ActivatedRecorder (EAR): a device for sampling naturalistic daily activities and conversations. Behav.Res. Methods Instrum. Comput. 33:517–23

Mischel W. 2004. Toward an integrative science of the person. Annu. Rev. Psychol. 55:1–22Moghaddam NG, Ferguson E. 2007. Smoking, mood regulation, and personality: an event-

sampling exploration of potential models and moderation. J. Personal. 75:451–78Moskowitz D, Young SN. 2006. Ecological Momentary Assessment: what it is and why it is a

method of the future in clinical psychopharmacology. J. Psychiatry Neurosci. 31:13–20Muraven M, Collins RL, Morshimer ET, Shiffman S, Paty JA. 2005. The morning after: limit

violations and the self-regulation of alcohol consumption. Psychol. Addict. Behav. 19:253–62Myin-Germeys I, van Os J, Schwartz JE, Stone AA, Delespaul PA. 2001. Emotional reactivity

to daily life stress in psychosis. Arch. Gen. Psychiatry 58:1137–44Newman MG, Erickson T, Przeworski A, Dzus E. 2003. Self-help and minimal-contact ther-

apies for anxiety disorders: Is human contact necessary for therapeutic efficacy? J. Clin.Psychol. 59:251–74

Newman MG, Kenardy J, Herman S, Taylor CB. 1997. Comparison of cognitive behavioraltreatment of panic disorder with computer-assisted brief cognitive behavior treatment.J. Consult. Clin. Psychol. 65:178–83

Niere K, Jerak A. 2004. Measurement of headache frequency, intensity and duration: compar-ison of patient report by questionnaire and headache diary. Physiother. Res. Int. 9:149–56

Nived O, Sturfelt G, Eckernas S-A, Singer P. 1994. A comparison of 6 months’ complianceof patients with rheumatoid arthritis treated with tenoxicam and naproxen: use of patientcomputer data to assess response to treatment. J. Rheumatol. 21:1537–41

Norton M, Wonderlich SA, Myers T, Mitchell JE, Crosby RD. 2003. The use of palmtopcomputers in the treatment of bulimia nervosa. Eur. Eat. Disord. 11:231–42

O’Connell KA, Gerkovich MM, Bott M, Cook MR, Shiffman S. 2002. The effect of anticipatorystrategies on the first day of smoking cessation. Psychol. Addict. Behav. 16:150–56

Oishi S, Sullivan HW. 2006. The predictive value of daily vs. retrospective well-being judg-ments in relationship stability. J. Exp. Soc. Psychol. 42:460–70

Paty JA, Kassel JD, Shiffman S. 1992. Assessing stimulus control of smoking: the importanceof base rates. In The Experience of Psychopathology, ed. M DeVries, pp. 347–52. Cambridge,UK: Cambridge Univ. Press

Pearson J. 2004. Better, faster and cheaper diary trials? Evaluating the potential scientific benefitsof e-diary technology in a chronic primary insomnia trial. Presented at Annu. Meet. DIA,Arlington, VA

Perrine MW, Mundt JC, Searles JS, Lester LS. 1995. Validation of daily self-reported alcoholconsumption using Interactive Voice Response (IVR) technology. J. Stud. Alcohol 56:487–90

Piasecki TM, Hufford MR, Solhan M, Trull TJ. 2007. Assessing clients in their natural en-vironments with electronic diaries: rationale, benefits, limitations, and barriers. Psychol.Assess. 19:25–43

Redelmeier DA, Katz J, Kahneman D. 2003. Memories of colonoscopy: a randomized trial.Pain 104:187–94

www.annualreviews.org • Ecological Momentary Assessment 29

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Reis HT, Wheeler L. 1991. Studying social interaction with the Rochester Interaction Record.In Advances in Experimental Social Psychology, ed. MP Zanna, pp. 270–318. San Diego, CA:Academic

Riley WT, Jerome A, Behar A, Weil J. 2002. Computer and manual self-help behavioral strate-gies for smoking reduction: initial feasibility and one-year follow-up. Nicotine Tobacco Res.4:S183–88

Ross M. 1989. Relation of implicit theories to the construction of personal histories. Psychol.Rev. 96:341–57

Saito M, Kumano H, Yoshiuchi K, Kokubo K, Yamamoto Y, et al. 2005. Symptom profile ofmultiple chemical sensitivity in actual life. Psychosom. Med. 67:318–25

Schlundt DG, Johnson WG, Jarrell MP. 1985. A naturalistic functional analysis of eatingbehavior in bulimia and obesity. Adv. Behav. Res. Ther. 7:149–62

Schwartz JE, Neale J, Marco C, Shiffman S, Stone AA. 1999. Does trait coping exist? Amomentary assessment approach to the evaluation of traits. J. Personal. Soc. Psychol. 77:360–69

Schwartz JE, Stone AA. 1998. Data analysis for EMA studies. Health Psychol. 17:6–16Schwartz JE, Stone AA. 2007. Analysis of real-time momentary data: a practical guide. See

Stone et al. 2007b, pp. 76–113Schwarz N. 2007. Retrospective and concurrent self-reports: the rationale for real-time data

capture. See Stone et al. 2007b, pp. 11–26Scollon CN, Kim-Prieto C, Diener E. 2003. Experience sampling: promises and pitfalls,

strengths and weaknesses. J. Happiness Stud. 4:5–34Searles JS, Helzer JE, Walter DE. 2000. Comparison of drinking patterns measured by daily

reports and timeline follow back. Psychol. Addict. Behav. 14:277–86Shapiro D, Jamner LD, Davydov DM, James P. 2002. Situations and moods associated with

smoking in everyday life. Psychol. Addict. Behav. 16:342–45Sheftell FD, Dahlof CG, Brandes JL, Agosti R, Jones MW, Barrett PS. 2005. Two replicate

randomized, double-blind, placebo-controlled trials of the time to onset of pain relief inthe acute treatment of migraine with a fast-disintegrating/rapid-release formulation ofsumatriptan tablets. Clin. Ther. 27:407–17

Shiffman S. 1993. Assessing smoking patterns and motives. J. Consult. Clin. Psychol. 61:732–42Shiffman S. 2005. Dynamic influences on smoking relapse process. J. Personal. 73:1715–48Shiffman S. 2007. Designing protocols for Ecological Momentary Assessment. See Stone et al.

2007b, pp. 27–53Shiffman S, Balabanis MH, Gwaltney CJ, Paty JA, Gnys M, et al. 2007. Prediction of lapse

from associations between smoking and situational antecedents assessed by EcologicalMomentary Assessment. Drug Alcohol Depend. 91:159–68

Shiffman S, Elash CA, Paton S, Gwaltney CJ, Paty JA, et al. 2000. Comparative efficacy of24-hour and 16-hour transdermal nicotine patch for relief of craving and withdrawal.Addiction 95:1185–95

Shiffman S, Engberg J, Paty JA, Perz W, Gnys M, et al. 1997a. A day at a time: predictingsmoking lapse from daily urge. J. Abnorm. Psychol. 106:104–16

Shiffman S, Ferguson SG, Gwaltney CJ, Balabanis MH, Shadel WG. 2006. Reductionof abstinence-induced withdrawal and craving using nicotine replacement therapy.Psychopharmacology 184:637–44

Shiffman S, Gwaltney CJ, Balabanis M, Liu KS, Paty JA, et al. 2002. Immediate antecedents ofcigarette smoking: an analysis from Ecological Momentary Assessment. J. Abnorm. Psychol.111:531–45

30 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Shiffman S, Gwaltney CJ, Shields AL. 2007. Validation of ePRO measures and using ePRO toenhance validity. Presented at Annu. Meet. New Clin. Drug Eval. Unit, 47th, Boca Raton,FL

Shiffman S, Hickcox M, Paty JA, Gnys M, Kassel JD, Richards T. 1996a. Progression froma smoking lapse to relapse: prediction from abstinence violation effects and nicotine de-pendence. J. Consult. Clin. Psychol. 64:993

Shiffman S, Hickcox M, Paty JA, Gnys M, Kassel JD, Richards T. 1997b. The abstinenceviolation effect following smoking lapses and temptations. Cogn. Ther. Res. 21:497–523

Shiffman S, Hufford M, Hickcox M, Paty JA, Gnys M, Kassel JD. 1997c. Remember that? Acomparison of real-time vs. retrospective recall of smoking lapses. J. Consult. Clin. Psychol.65:292–300

Shiffman S, Paty JA. 2003. Using an electronic diary to capture frequent events. Presented at FromQuality of Life to Patient Outcomes Assess.: Res. Agenda for a Paradigm Shift, Drug Inf.Assoc., Baltimore, MD

Shiffman S, Paty JA, Gnys M, Kassel JD, Hickcox M. 1996b. First lapses to smoking: within-subjects analyses of real-time reports. J. Consult. Clin. Psychol. 64:366–79

Shiffman S, Waters AJ. 2004. Negative affect and smoking lapses: a prospective analysis.J. Consult. Clin. Psychol. 72:192–201

Shiffman S, Waters AJ, Hickcox M. 2004. The Nicotine Dependence Syndrome Scale: a multi-dimensional measure of nicotine dependence. Nicotine Tob. Res. 6:327–48

Shrier LA, Shih M-C, Beardslee WR. 2005. Affect and sexual behavior in adolescents: a reviewof the literature and comparison of momentary sampling with diary and retrospectiveself-report methods of measurement. Pediatrics 115:573–81

Stone AA, Bachrach CA, Jobe JB, Kurtzman HS, Cain VS, eds. 2000. The Science of Self-Report:Implications for Research and Practice. Mahwah, NJ: Erlbaum

Stone AA, Broderick JE, Schwartz JE, Shiffman S, Litcher-Kelly L, Calvanese P. 2003a. In-tensive momentary reporting of pain with an electronic diary: reactivity, compliance, andpatient satisfaction. Pain 104:343–51

Stone AA, Schwartz JE, Broderick JE, Shiffman S. 2005. Variability of momentary pain predictsrecall of weekly pain: a consequence of the peak (or salience) memory heuristic. Personal.Soc. Psychol. Bull. 31:1340–46

Stone AA, Schwartz JE, Neale JM, Shiffman S, Marco CA, et al. 1998. A comparison of copingassessed by Ecological Momentary Assessment and retrospective recall. J. Personal. Soc.Psychol. 74:1670–80

Initially statedEMA principles,describing therationale for andcharacteristics ofEMA.

Stone AA, Shiffman S. 1994. Ecological Momentary Assessment in behavioral medicine.Ann. Behav. Med. 16:199–202

Stone AA, Shiffman S. 2002. Capturing momentary, self-report data: a proposal for reportingguidelines. Ann. Behav. Med. 24:236–43

Stone AA, Shiffman S, Atienza A, Nebeling L. 2007a. Introduction to Ecological MomentaryAssessment (EMA). See Stone et al. 2007b, pp. xi–xiv

Lays out rationaleand foundation ofEMA methods,including design,practicalities, anddata analysis, withexamples of EMAstudies.

Stone AA, Shiffman S, Atienza A, Nebeling L, eds. 2007b. The Science of Real-Time Data

Capture: Self-Reports in Health Research. New York: Oxford Univ. Press

Reports a studyevaluating actualcompliance withwritten diariescompared withelectronic diaries.

Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. 2002. Patient noncom-pliance with paper diaries. Br. Med. J. 324:1193–94

Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. 2003b. Patient compliancewith paper and electronic diaries. Controlled Clin. Trials 24:182–99

Sudman S, Bradburn NM. 1973. Effects of time and memory factors on response in surveys.J. Am. Stat. Assoc. 68:805–15

www.annualreviews.org • Ecological Momentary Assessment 31

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

ANRV339-CP04-01 ARI 6 March 2008 0:12

Szalai A. 1966. The multinational comparative time budget research project: a venture ininternational research cooperation. Am. Behav. Sci. 10:30

Taylor BC, Fried L, Kenardy J. 1990. The use of a real-time diary for data acquisition andprocessing. Behav. Res. Ther. 28:92–97

Tennen H, Affleck G, Armeli S. 2005. Personality and daily experience revisited. J. Personal.73:1–19

Tennen H, Affleck G, Coyne JC, Larsen RJ, DeLongis A. 2006. Paper and plastic in dailydiary research: comment on Green, Rafaeli, Bolger, Shrout, and Reis. Psychol. Methods11:112–18

Thiele C, Laireiter A-R, Baumann U. 2002. Diaries in clinical psychology and psychotherapy:a selective review. Clin. Psychol. Psychother. 9:1–37

Todd M, Armeli S, Tennen H, Carney MA, Ball SA, et al. 2005. Drinking to cope: a comparisonof questionnaire and electronic diary reports. J. Stud. Alcohol 66:121–29

Tourangeau R. 2000. Remembering what happened: memory errors and survey reports. SeeStone et al. 2000, pp. 29–48

Tulen J, Volkers A, Stronks D, Cavelaars M, Groeneveld W. 2001. Accelerometry in clinicalpsychophysiology. In Ambulatory Assessment, ed. J Fahrenberg, M Myrtek, pp. 207–31.Seattle, WA: Hogrefe & Huber

Turner JR, Ward MM, Gellman MD, Johnston DW. 1994. The relationship between labora-tory and ambulatory cardiovascular activity: current evidence and future directions. Ann.Behav. Med. 16:12–23

Tversky A, Kahneman D. 1973. Availability: a heuristic for judging frequency and probability.Cogn. Psychol. 5:207–32

Van Den Brink M, Bandell-Hoekstra E, Abu-Sadd H. 2001. The occurrence of recall bias inpediatric headache: a comparison of questionnaire and diary data. Headache 41:11–20

Verbrugge LM. 1980. Health diaries. Med. Care 18:73–95

Describes a varietyof methods forstatistical analysisof EMA data.

Walls TA, Schafer JL, eds. 2006. Models for Intensive Longitudinal Data. New York:Oxford Univ. Press.

Wheeler L, Reis H. 1991. Self-recording of everyday life events: origins, types, and uses.J. Personal. 59:339–54

Wilhelm FH, Roth WT, Sackner MA. 2003. The LifeShirt: an advanced system for ambulatorymeasurement of respiratory and cardiac function. Behav. Modif. 27:671–91

Willis G. 2005. Cognitive Interviewing. Thousand Oaks, CA: Sage

32 Shiffman · Stone · Hufford

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

AR339-FM ARI 5 March 2008 23:50

Annual Review ofClinical Psychology

Volume 4, 2008Contents

Ecological Momentary AssessmentSaul Shiffman, Arthur A. Stone, and Michael R. Hufford � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �1

Modern Approaches to Conceptualizing and Measuring HumanLife StressScott M. Monroe � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 33

Pharmacotherapy of Mood DisordersMichael E. Thase and Timothey Denko � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 53

The Empirical Status of Psychodynamic TherapiesMary Beth Connolly Gibbons, Paul Crits-Christoph, and Bridget Hearon � � � � � � � � � � � � � 93

Cost-Effective Early Childhood Development Programs fromPreschool to Third GradeArthur J. Reynolds and Judy A. Temple � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �109

Neuropsychological RehabilitationBarbara A. Wilson � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �141

Pediatric Bipolar DisorderEllen Leibenluft and Brendan A. Rich � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �163

Stress and the Hypothalamic Pituitary Adrenal Axis in theDevelopmental Course of SchizophreniaElaine Walker, Vijay Mittal, and Kevin Tessner � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �189

Psychopathy as a Clinical and Empirical ConstructRobert D. Hare and Craig S. Neumann � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �217

The Behavioral Genetics of Personality DisorderW. John Livesley and Kerry L. Jang � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �247

Disorders of Childhood and Adolescence: Gender andPsychopathologyCarolyn Zahn-Waxler, Elizabeth A. Shirtcliff, and Kristine Marceau � � � � � � � � � � � � � � � �275

vii

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.

AR339-FM ARI 5 March 2008 23:50

Should Binge Eating Disorder be Included in the DSM-V? A CriticalReview of the State of the EvidenceRuth H. Striegel-Moore and Debra L. Franko � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �305

Behavioral Disinhibition and the Development of Early-OnsetAddiction: Common and Specific InfluencesWilliam G. Iacono, Stephen M. Malone, and Matt McGue � � � � � � � � � � � � � � � � � � � � � � � � � � �325

Psychosocial and Biobehavioral Factors and Their Interplayin Coronary Heart DiseaseRedford B. Williams � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �349

Stigma as Related to Mental DisordersStephen P. Hinshaw and Andrea Stier � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �367

Indexes

Cumulative Index of Contributing Authors, Volumes 1–4 � � � � � � � � � � � � � � � � � � � � � � � � � � �395

Cumulative Index of Chapter Titles, Volumes 1–4 � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �397

Errata

An online log of corrections to Annual Review of Clinical Psychology chapters (if any)may be found at http://clinpsy.AnnualReviews.org

viii Contents

Ann

u. R

ev. C

lin. P

sych

ol. 2

008.

4:1-

32. D

ownl

oade

d fr

om a

rjou

rnal

s.an

nual

revi

ews.

org

by O

ntar

io C

ounc

il of

Uni

vers

ities

Lib

rari

es o

n 04

/16/

08. F

or p

erso

nal u

se o

nly.