Deﬁning the “Disruptive” in Preschool Behavior: What ...devepi.duhs.duke.edu/library/pdf/19724.pdf · Deﬁning the “Disruptive” in Preschool Behavior 185 including structured

Clinical Child and Family Psychology Review, Vol. 8, No. 3, September 2005 ( C© 2005)DOI: 10.1007/s10567-005-6664-5

Defining the “Disruptive” in Preschool Behavior:What Diagnostic Observation Can Teach Us

Lauren S. Wakschlag,1,6 Bennett L. Leventhal,1 Margaret J. Briggs-Gowan,3

Barbara Danis,1 Kate Keenan,2 Carri Hill,1 Helen L. Egger,4

Domenic Cicchetti,3 and Alice S. Carter5

This paper presents the clinical/developmental framework underlying a new diagnostic ob-servational tool, the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS). Thespecial importance of observation for clinical assessment during the preschool period is de-lineated. The developmental rationale for a multi-dimensional assessment of disruptive be-havior in young children, including problems in modulation of negative affect and low com-petence is discussed. The ways in which the DB-DOS will elucidate distinctions betweennormative and atypical behavior during this developmental period via (a) the integrationof qualitative and quantitative dimensions of behavior within a clinically-sensitive codingsystem, (b) the observation of child behavior both within, and outside of, the parent–childcontext and (c) the use of specially designed tasks to “press” for clinically salient behaviorsare addressed. The promise of this new method for yielding a more precise, developmen-tally based description of the phenotype of early onset disruptive behavior problems and forproviding a standardized clinical tool for observational assessment of disruptive behavior inyoung children is presented. Large-scale validation of the measure is currently underway.

KEY WORDS: preschool disruptive behavior; behavior problems; developmental methods; develop-mental psychopathology; observational assessment.

INTRODUCTION

Disruptive behavior problems are the most com-mon reason for mental health referral of preschoolchildren and, these problems are often persistentand impairing (Campbell, 2002; Wakschlag & Danis,

1Institute for Juvenile Research, University of Illinois at Chicago.2Department of Psychiatry, University of Chicago, Chicago,Illinois.

3Epidemiology and Public Health, Yale University, New Haven,Connecticut.

4Psychiatry and Behavioral Sciences, Duke University, Durham,North Carolina.

5Department of Psychology, University of Massachusetts atBoston, Boston, Massachusetts.

6Address all correspondence to Lauren S. Wakschlag, Ph.D., Insti-tute for Juvenile Research, Department of Psychiatry, Universityof Illinois at Chicago, 1747 W. Roosevelt Rd, Room 155, MC 747,Chicago, Illinois 60608; e-mail: [email protected].

2004).7 However, because this developmental periodis marked by developing autonomy and a great dealof behavioral change, most preschoolers exhibit atleast some of the behaviors that fall under the rubricof disruptive behavior. This makes clinical assess-ment of preschool behavior particularly subtle andcomplex. Consequently, there has been a lack of con-sensus about how to validly assess clinical problems

7Historically attention deficit/hyperactivity disorder (ADHD) hasbeen included within the broader rubric of externalizing prob-lems. However, DSM-IV differentiates Disruptive Behavior Dis-orders (DBDs, i.e., Oppositional Defiant Disorder (ODD) andConduct Disorder (CD)) from ADHD, with DBDs being funda-mentally social rather than attentional in nature. This distinctionis supported by research showing different developmental path-ways and correlates for DBDs and ADHD (Hinshaw & Lahey,1993; Wakschlag & Hans, 1999). We will use the term “disrup-tive behavior” to refer to oppositional and conduct problems, theprimary focus of this paper.

183

1096-4037/05/0900-0183/0 C© 2005 Springer Science+Business Media, Inc.

184 Wakschlag et al.

during this period (Campbell, 2002). This is espe-cially problematic because behavior problems arethe most common reason for referral of preschool-ers to mental health clinics. Thus, the developmentof standardized methods that provide a metric fordevelopmentally sensitive clinical decision-makingwith preschool children is crucial. Such methods willsubstantially improve our ability to appropriatelydistinguish transient and/or normative behaviors inpreschoolers from behavior disorders requiring treat-ment and to delineate the nature of early emergingpathways to disruptive behavior disorders.

The goal of this paper is to elucidate how diag-nostic observation may be used to advance scientificand clinical knowledge in the area of preschool dis-ruptive behavior.

Measurement of Disruptive Behavior in YoungChildren: Challenges and Opportunities

The development of self-control is a hallmark ofthe toddler and preschool periods (Kochanska, Coy,& Murray, 2001; Sroufe, 1996). Frustration tolerance,delay of gratification, the use of verbal negotiationstrategies, internalization of standards, and behav-ioral flexibility are skills that emerge and are con-solidated during this period. Developmental contextsand demands also shift during this period, includ-ing expansion of the social world to include peers,heightened parental expectations, and limit setting.As these many processes converge, there is a nor-mative increase in behavioral disruption (Campbell,2002; Tremblay, 2000). Distinguishing typical fromatypical behavior during this developmental periodthus presents significant diagnostic and methodologicchallenges.

Despite these complexities, dramatic progresshas been made in the conceptualization andmeasurement of social–emotional problems and psy-chopathology in young children over the past decade(DelCarmen-Wiggins & Carter, 2004). A largebody of work has established that problem behaviorpatterns begin in the first years of life (Carter, Briggs-Gowan, Jones, & Little, 2003). Most recently, prelim-inary validation efforts have demonstrated that clin-ical problems can be reliably identified in preschoolchildren, based on parent report (Task Force, 2003).Though these studies have focused primarily onconcurrent validity (Keenan & Wakschlag, 2002),a few studies have examined the predictive validityof DBDs identified in preschoolers (Lavigne et al.,

1998; Speltz, McMellan, DeKlyen, & Jones, 1999).These clinical studies, as well as a large body of devel-opmental research on preschool behavior problemsassessed with behavior checklists (e.g., Campbell,2002; Dumas, Serketich, & LaFreniere, 1995;Gardner, Ward, Burton, & Wilson, 2003; Shaw et al.,1998), have provided a strong empirical foundationfor the study of early emerging psychopathology.

However, advancing clinical science in this arearequires supplementing information from parentreport with data derived from clinically informativeobservational methods. Observational methods arevitally important for elucidating the boundariesbetween normal and abnormal behavior in youngchildren and for characterizing the phenotype ofdisruptive behavior in young children in all itssubtlety and complexity. Observational methods areimportant supplements to parent report becausealthough parents are good historians about children’sbehavior, they are often less able to make judgmentsabout whether behaviors are normative or atypical(Lord, 1997). In addition, parents’ ability to report“accurately” is influenced by a variety of factors,including threshold for misbehavior, developmentalknowledge, stress, and parental psychopathology(Briggs-Gowan, Carter, & Schwab-Stone, 1996;Hay et al., 1999). Further, distinguishing normativevariation from clinical problems in early childhoodrequires systematic, nuanced observation of arange of behaviors, and their qualitative features.Historically agreement between direct observationand parent report of preschoolers’ behavior has beenmodest at best, suggesting that each provides uniqueinformation (Gardner, 2000; Kagan, 1998). In thediscussion that follows, we present diagnostic obser-vation as a promising and innovative methodologydesigned to supplement parent-report methods.We introduce the Disruptive Behavior DiagnosticObservation Schedule (DB-DOS), a method we haverecently developed for observational assessment ofdisruptive behavior in young children, providepreliminary data on its reliability and validity, andhighlight its potential for advancing the science ofdevelopmental psychopathology.

The Role of Diagnostic Observation in theAssessment of Young Children

Definition and Potential Utility

Observational methodologies are a longstand-ing tradition in developmental and clinical research,

Defining the “Disruptive” in Preschool Behavior 185

including structured laboratory-based parent–childinteraction and behavioral analogue methods andhome-based naturalistic observation methods (e.g.,Buckley, Klein, Durbin, Hayden, & Moerk, 2002;Elardo & Bradley, 1981; Haynes, 2001; Hubbardet al., 2002; Kochanska, 1998; Pelligrini, 2001; Shaw,2003; Zelenko, 2004). Each of these methods hasunique strengths and weaknesses, which we compareand contrast to diagnostic observation below.

In developmental research, laboratory-basedobservational methods have generally focusedon structured assessments (e.g., free play, familydiscussions) designed to elicit characteristic styles ofparent–child interactions (Kerig & Lindahl, 2001).Such methods allow for assessment of discrete childbehaviors as well as parenting. Typically, a rangeof tasks is used that include both “demand” andfree play tasks. Within the context of these tasks,parents are often encouraged to respond as naturallyas possible. One of the most widely used preschoolparadigms is the Dyadic Parent–Child InteractionCoding System (D-PICS; Robinson & Eyberg, 1981),a 15 minute paradigm that includes three structuredsituations: a child-directed interaction, a parent-directed interaction, and a cleanup task. Frequencycounts of a range of child behaviors are obtainedand then summed to create three total scores: childdeviance (sum of whining, yelling, crying, physicalnegative, aggression, and smart talk), noncompliance(percent of noncompliance in response to parentalcommands), and child negative affect (Webster-Stratton & Lindsay, 1999). Multiple studies havedemonstrated that D-PICS conduct problem scoresare higher for referred versus nonreferred preschool-ers (Eyberg, Boggs, & Algina, 1995; Speltz, DeKlyen,Greenberg, & Dryden, 1995; Webster-Stratton &Lindsay, 1999). However, in these and otherstudies, parent and child behavior ratings have beentreated as correlates of DBDs rather than used togenerate diagnoses (e.g., Speltz et al., 1995;Webster-Stratton & Lindsay, 1999; Wakschlag & Keenan,2001).

Another type of laboratory-based observationis the “behavioral analogue” or “performance-based”task, designed to tap into very specific clinically rele-vant processes (Haynes, 2001). For example, in thestudy of DBDs in older children, analogue meth-ods have been used to study cheating, social in-formation processing, and response to provocation(Criss, Shaw, & Ingoldsby, 2003; Frick & Loney,2000). Behavioral analogue tasks include simulations(e.g., computer pinball game in competition with

an alleged peer), tasks with the child alone, andtasks where the parental behavior is scripted (e.g.,parents are instructed to use a standardized set ofcommands). Although more commonly used witholder children, similar methods have been used withpreschool children, including the use of delay tasks toexamine anger regulation and impulsivity (Campbell,Pierce, March, Ewing, & Szumowski, 1994; Gilliom,Shaw, Beck, Schonberg, & Lukon, 2002).

In contrast, home-based observations are de-signed to more naturalistically capture child behaviorwithin the context of routine activities of daily life.Such observations generally involve multiple visitsto a home to obtain a varied sampling of behaviors.Families are observed engaging in “typical dailyactivities.” Although virtually all home-basedobservations include an unstructured component,some also include structured tasks in the home(e.g., cleanup, family meals) (Bradley, Mundfrom,Whiteside, Casey, & Barrett, 1994; Buckley et al.,2002; Gardner, 2000; Webster-Stratton & Lindsay,1999). Coding methods have varied, ranging fromevent-based coding to q-sort methodology (Buckleyet al., 2002; Gardner, 2000). Rates of noncomplianceobserved in the home distinguish parent-ratedproblematic preschoolers from normal controls(Gardner, Sonuga-Barke, & Sayal, 1999).

Standardized diagnostic observation provides adirect examiner-based method to systematically elicitthe full range of behaviors relevant to the diagnosisof a specific disorder. Diagnostic observation is in-tentionally structured to “press” for salient behav-iors, thereby increasing the likelihood that a range ofclinically relevant behaviors will be observed (Lordet al., 2000). The strength of diagnostic observationmethods is that they inherently combine research andclinical utility. That is, they are designed to be admin-istered in a standardized fashion, clinically sensitiveand relatively brief, and thus feasible for clinicians touse (Mash & Foster, 2001).

By providing a semi-structured standardizedmethod of direct assessment, diagnostic observationsalso allows for the use of clinical judgment. Clini-cal observation goes beyond observation of discretebehaviors per se, offering an integrated examinationof multiple facets of the child’s behavior and func-tioning (Wakschlag & Danis, 2004). Building on suchobservations, clinical judgment is an overall assess-ment of the atypicality of the child’s behavior, whichweights the salience of particular behaviors based onage-appropriateness and context. The clinician’s ex-perience of, and with, the child is a vitally important


dimension of this process (Benham, 2000; Wakschlag& Danis, 2004; Zelenko, 2004).

Standardized diagnostic observation has beenused very effectively in research and clinical practicewith autism-spectrum disorders. The Autism Diag-nostic Observation Schedule (ADOS; Lord et al.,2000) is an interactive schedule that uses examiner“presses” to elicit behaviors fundamental to thediagnosis of autism (i.e., communication, reciprocalsocial interaction and stereotyped behavior). In con-junction with a standardized diagnostic interview,the ADOS has become an essential part of a “goldstandard” battery for diagnostic assessment of autismand has contributed to broad-based consensus aboutdiagnostic validity (Lord, Rutter, & Le Couteur,1994). In particular, it has been crucial for resolvingthe controversy about whether autism can be reliablyidentified in preschool children (Lord, 1995). Fur-ther, by providing a method with which to character-ize phenotypic variations, it has been a cornerstoneof genetic research and studies of diagnosis, treat-ment and developmental course (Kim et al., 2002;Lord, Leventhal, & Cook, 2001; Owley et al., 2001).

Diagnostic Observation in Context

Diagnostic observations have the capacity toprovide very important, unique information but theyalso have limitations. They assess current, not life-time symptomatology. The information yielded willnecessarily be “de-contextualized” relative to natu-ralistic or multiple, repeated observations. Further,since low incidence behaviors may not occur in a rel-atively brief observation, their presence is importantbut their absence cannot be interpreted as the ab-sence of a symptom (Lord, 1997). Thus, diagnosticobservations should not be used in isolation. Rather,they are designed to be used as companion meth-ods to parent interviews, which place these obser-vations in historical context. Assessment of histori-cal context includes interviewing parents about thedevelopmental course of the behavior, life eventsthat may have precipitated the onset of the behav-ior, and events and stressors within the broader socialcontext. Parent-interview data also provide a win-dow into the subjective meaning of the child’s be-havior to the parent; data which may be informativeabove and beyond “objective” ratings (Campione-Barr & Smetana, 2004). Information from other keyinformants in the child’s life (e.g., teachers) is alsocrucial for interpreting whether behavior observed

represents enduring patterns (i.e., is “typical” for thechild) or is a perturbation due to normative develop-mental transitions or recent disruptions in the child’ssocial environment.

Distinguishing typical from atypical behavioralso requires interpretation of behaviors observedwithin developmental context (Wakschlag & Danis,2004). As such, standardized evaluation of the child’sdevelopmental functioning is also a critical compo-nent of a developmentally sensitive assessment bat-tery (for a more extensive discussion, see Carter,Briggs-Gowan, & Ornstein Davis, 2004; Wakschlag& Danis, 2004). This enables control for develop-mental level when examining the discriminative andpredictive utility of observational data as well asdetermining atypicality based on both chronologi-cal and developmental age. For example, noncompli-ance and negative affect may be interpreted differ-ently if language delays are present.

Varying Observational Methods: Tradeoffsand Opportunities

As we have noted above, each of the varioustypes of observational methodology have unique ad-vantages and disadvantages and such trade-offs mustbe acknowledged and thoughtfully examined. Usinga range of methodologies, studies have correlated ob-served disruptive behavior with parent-reported be-havior problems in preschool children. Such meth-ods have also highlighted dimensions of parentingbehavior central in pathways to psychopathology.Their clinical utility is limited, however, becausethey: (a) measure these behaviors in terms of pres-ence/absence or frequency and do not take qualita-tive dimensions into account; (b) do not comprehen-sively assess the constellation of behaviors relevantto a diagnosis of a DBD; (c) are not clinically sensi-tive or specific, and (d) do not include a method ofdirect examiner-based clinical assessment. Further,since most methods have relied exclusively on obser-vation of child behavior within the context of parent–child interaction, the extent to which the child’s be-havior is pervasive or specific to that relationshipcannot be systematically observed. With the growingfocus on elucidating the nature and pattern of earlyemerging psychopathology (Task Force, 2003), diag-nostically informative observational methodology isincreasingly imperative (Carter et al., 2004).

One of the primary advantages of diagnosticobservation (i.e., that it is structured to increase the


likelihood of eliciting behaviors of interest) is also itsdisadvantage (i.e., it is not naturalistic). As Gardner(2000) has noted, the validity of observationalmethods may be affected by both type of task(e.g., structured versus naturalistic) and locationof observation (e.g., home versus lab). As a result,choice of an observational method must carefullyconsider the unique advantages and disadvantagesof each method and the appropriateness of a methodwithin a particular research study must be drivenby the particular questions the study is designed toanswer. Thus, if the goal is to enhance the accuracyof diagnostic decision-making and characterizethe phenotype of disruptive behavior disorders inyoung children, structured diagnostic observationhas significant advantages because the full rangeof behaviors are unlikely to be observed in anunstructured naturalistic context, unless repeated ob-servations are conducted. Second, although there areclearly significant research advantages to naturalisticobservations, they are not practical as clinical tools.

The Disruptive Behavior Diagnostic ObservationSchedule (DB-DOS)

The Disruptive Behavior Diagnostic Observa-tion Schedule (DB-DOS; Wakschlag et al., 2002) isan observational clinical research tool for assessingdisruptive behavior in young children. The aim of theDB-DOS is to provide a developmentally informed,examiner-based method for the clinical assessment ofdisruptive behavior in young children that will yieldinformation essential for characterizing the pheno-type of early emerging disruptive behavior and serveas a companion to parent-interview methods. In thepresent paper, we focus primarily on the conceptualframework and methodology of the DB-DOS andhighlight the ways in which it has been designed toadvance clinical science in the area of preschool dis-ruptive behavior. We have recently been funded bythe National Institute of Mental Health to conducta large-scale validation of the DB-DOS measure (L.Wakschlag, PI). This study, which is designed to es-tablish the reliability and validity of the DB-DOS,will include extensive psychometric analysis; thus adetailed discussion of reliability, validity, and psycho-metric properties of the measure is premature. Herewe provide preliminary evidence of the DB-DOS’ re-liability and validity, including evidence from a pi-lot sample of referred and nonreferred preschoolers(N = 35) (Wakschlag & Danis, 2004).

DB-DOS Method

Using the ADOS paradigm as a model, theDB-DOS (a) uses tasks developed to systematically“press” for behaviors relevant to DSM diagnosticcriteria, (b) combines structured guidelines for exam-iner behavior (designed to standardize the contextand allow salient behaviors to unfold) and the use ofclinical judgment (to allow for flexibility and respon-siveness to individual differences amongst children),(c) is relatively brief to administer (approximately50 min), and (d) rates behaviors along a continuumof atypicality in order to assess behavior in a clin-ically informative manner. However, by definition,diagnostic observations must have components thatare unique to the disorder in question becausethey are specifically designed to press for behaviorssalient to this constellation of problems. As a result,the DB-DOS paradigm and coding system also differfrom those of the ADOS in several fundamentalways that reflect methodologic issues unique to theassessment of disruptive behavior problems.

DB-DOS Structure

The DB-DOS is structured into three mini-contexts, called modules (see Fig. 1). Autism spec-trum behaviors tend to be pervasively impairing;as such the ADOS diagnostic assessment is withthe examiner alone. In contrast, disruptive behav-iors may vary substantially across context, with par-ticular salience of the parent–child context. Assess-ment of the child in interaction with the parent ismost reflective of day-to-day interactions, but alsomakes it difficult to systematically assess the child’scapacity for self-regulation since parents vary sub-stantially in their skillfulness and, in fact, may ex-hibit problematic behaviors that amplify child behav-ior problems (Patterson, Capaldi, & Bank, 1991). Incontrast, interactions between a child and the exam-iner lack this “history” and can be more easily stan-dardized across children. As a result, the DB-DOSis designed to press for contextual differences in be-havior and includes three modules: two with the ex-aminer (Modules 1 and 2) and one with the parent(Module 3).

Modules 1 and 2, the examiner modules, aredesigned to systematically assess the way in whichexternal structure may affect the child’s capacity toadaptively manage behavior and emotions via sys-tematic variation in the level of support provided


Fig

.1.

DB

-DO

Ssc

hem

atic

.


by the examiner. Across all tasks, the examiner re-sponds to the child in a low-key, warm, and naturalway. The goal is to put the child at ease in order toallow the child’s natural behavior to unfold. Thus,the examiner seeks to strike a balance by making thechild feel comfortable without altering or obscuringthe child’s typical presentation. Within these guide-lines, it is emphasized that the examiner’s behaviornot be rigidly scripted. Rather, clinical judgment isused in determining the nature, pacing, and qualityof examiner responses.

The key difference across examiner modules isthe extent to which the examiner seeks opportuni-ties to interact with the child. The goal of Module 1(“Examiner Engaged”) is to allow the child’s behav-ior to unfold with a responsive examiner. In thismodule, the examiner is sitting at the child’s sideand is involved and attentive to the child during alltasks to create “active” opportunities to respond tothe child (e.g., smiling, commenting). During thismodule, the examiner uses normal social responsesto child’s positive social- and task-related behaviors.Within these positive social interchanges, the ex-aminer’s responses should be matched to or mirrorthose presented by the child in terms of modality andintensity, rather than attempting to amplify or mod-ify child behavior.

Module 2 (“Examiner Busy”) is designed to pro-vide a “withdrawal of attention” context and to pressfor covert behaviors (e.g., lying). During this module,the examiner does not actively seek opportunities torespond to the child and only engages minimally withthe child in response to child initiations. In order tonaturally create this structure within this module, thechild is given tasks to do independently while the ex-aminer is “busy” during this module with his/her own“work.” The examiner may respond to active initia-tions by the child in a warm and natural manner butwith the goal of encouraging the child to resume in-dependent activity.

In contrast, examiner response to child disrup-tive behaviors occurs in essentially the same mannerin both modules. Within the DB-DOS context,disruptive behaviors are defined as oppositional, ag-gressive, and destructive behaviors and/or moderate-to-high levels of irritable/angry affect. Responses todisruptive behavior are guided by the fundamentalgoal of the DB-DOS to allow the full range of thechild’s disruptive behavior to emerge, while at thesame time providing scaffolded support as a means oftesting “what it takes” to help a child exhibiting dis-ruptive behavior to modulate his/her response. Since

scoring is based, in part, on the child’s responsive-ness to examiner support, a hierarchy of well-pacedprompts is used to guide examiner responses.

A Level 1 prompt serves as a reminder; thus,Level 1 prompts are designed to redirect the childto the task at hand, and typically utilize restatementof the directions (e.g., “remember you have to doyour job before you can do the next thing”). Level 2prompts provide the child with encouragement. (e.g.,“let’s see how fast you can do it” “you can play withthose later but you need to wait”). Level 3 promptsincrease the level of examiner involvement by pro-viding a contingency (e.g., showing child the next toyto be played with once the current task is completed),giving a warning (e.g., “if you can’t do it, I can helpyou”), or trying to join with the child around the tasks(e.g., “let’s try it together”). A Level 4 prompt is de-signed to de-escalate very intense behaviors and mayinclude physical or verbal support for task comple-tion (e.g., helping the child to complete the task, stat-ing directions in a firm tone, and/or moving on to thenext task).

Examiners move through the hierarchy ofprompts in response to escalating, persistent or se-rious disruptive behavior. Examiners are trained notto step in too quickly; prompts should be paced to al-low the child sufficient time to modulate his/her ownbehavior. In the event of unsafe or very rapidly esca-lating behavior (e.g., throwing things, trying to leavethe room, aggression towards the examiner), examin-ers use their judgment to move through the promptsmore quickly or begin at a higher level; thus, the ex-aminer may skip earlier levels and provide a Level 3or 4 prompt as his/her first response.

In contrast to the examiner modules, structur-ing in the parent module occurs via the variation oftask demands rather than by scripting the parent’sbehavior. This allows the parent’s behavior to unfoldas naturally as possible. The parent module also pro-vides a standard context with which to assess parentalbehavior in a clinically informative manner.

DB-DOS tasks are activities within the mod-ules that are designed as “presses” to elicit salientbehaviors. In developing these tasks, we drew ona number of widely used developmental paradigmsto create a series of presses that would be likely toelicit the full range of clinically relevant behavior.Such tasks included compliance “do” and “don’t”tasks, cleanup, withdrawal of attention, social play,and frustration tasks; tasks that have been validatedacross a range of samples of varying ethnicity andsocioeconomic status (Campbell, Szumowski, Ewing,


Table I. Overview of DB-DOS Tasks by Module

Behavioral press Task description

Module 1: Examiner EngagedCompliance “Do” Three consecutive sorting tasksFrustration Bubble toy demonstrated but does not work for childSocial play Examiner and child play with a marble construction toy

Module 2: Examiner BusyCompliance “Don’t” Boring crayon task with prohibited toys nearbya

Frustration Trying to complete puzzle to win prize with rigged taska

Internalization of rules Child prohibited from touching novel toy when examiner briefly leaves the roomModule 3: Parent

Compliance “Do” Coloring + Clean-upFrustration Multi-step puzzle taskCompliance “Don’t” Parent completes questionnaire, child may read book but prohibited from touching toys on shelfSocial play Parent and child play together with shelf toys

aChild is permitted to play with these toys at the close of the session and is always awarded a prize for making “good effort.”

Gluck, & Breaux, 1982; Gardner et al., 1999; Garner& Power, 1996; Gilliom et al., 2002; Kochanska &Aksan, 1995; Robinson & Eyberg, 1981; Wakschlag& Keenan, 2001; Webster-Stratton & Lindsay, 1999).Thus, the innovation of the DB-DOS does not lie inthe originality of the tasks per se. Rather, its uniquecontribution is in the compilation of a battery of tasksthat collectively press for the broad range of behav-iors clinically salient in the assessment of DBDs and,in the DB-DOS rating system designed to capturethese behaviors in a clinically informative fashion.

There are three tasks in each of the examinermodules and four tasks in the parent module, eachapproximately 5 min in length. Thus, the entire DB-DOS paradigm takes approximately 50 min. Taskswere designed to be parallel across examiner andparent modules (Table I summarizes DB-DOS tasksby module).

Rating Child Behavior on the DB-DOS

In the development of the DB-DOS, we wereguided by two fundamental principles. One was the

importance of clinical judgment for distinguishingbetween typical and atypical behavior in young chil-dren (Gilliam & Mayes, 2004; Wakschlag & Danis,2004). As discussed above, clinical judgment is usedin DB-DOS administration in terms of determin-ing the timing, pacing, and nature of examiner re-sponses to child behavior. For DB-DOS scoring onthe other hand, clinical judgment is built into thecodes rather than via independent inference by theexaminer. That is, the identification of clinically rel-evant items and clinical inference (i.e., judgmentabout the atypicality of a behavior based on an in-tegrated view of its quantitative and qualitative fea-tures) were used to construct the codes. Thus, for ex-ample, clinical expertise drove the selection of “easeof elicitation,” “rapid escalation,” and “difficulty re-covering” as clinically relevant dimensions of prob-lems in modulation of negative affect. Consistentwith the framework of clinical observation (Gilliam& Mayes, 2004), the rating system goes beyond dis-crete behaviors by taking their qualitative featuresinto account. For example, for the item “easy toelicit negative affect” (see Table II), negative af-fect elicited in response to positive social stimuli is

Table II. Illustrate DB-DOS Item: Easy to Elicit Negative Affect

None (0) Low (1) Moderate (2) High (3)

Negative affect not easily elicited. Thismay include children who take a lotbefore they get frustrated and thenonly display low-level negative affecton 1–2 occasions.

Negative affect eliciteda few times inresponse to a buildup of frustration,limits or demands.

Negative affect elicited severaltimes in response to even lowlevel frustrations, limits ordemands, but not in responseto social interactions

Negative affect elicitedfrequently, and is inresponse to both low levelfrustrations, limits, demandsand to social interactions

Note. This variable assesses the ease with which negative affect is elicited. “Ease” is based on the frequency but also takes into accountthe nature of the precipitant. Thus, for example, children whose negative affect is elicited at the beginning of demanding tasks beforefrustration builds and children who often exhibit negative affect during fun tasks would be rated in the moderate to high range for thiscode. This variable assesses elicitation of any negative affect, regardless of intensity.


weighted more heavily along the atypicality scale(coded as a 3) than negative affect elicited to frustra-tion or demands (codes of 1 or 2), even if these occurat the same frequency. As such, while clinical knowl-edge is an important prerequisite for the clinical useof the DB-DOS, the actual rating of behaviors ob-served is specified along the continuum of each code,rather than via clinician inference. This structure wasdesigned to get the “best of both worlds,” i.e., tomake use of clinical expertise while at the same timeobtaining equivalent data from one case to the next(Westen & Weinberger, 2004).8

The second guiding principle was the impor-tance of a developmentally sensitive and -informedmethodology. This included creating a paradigm:(a) informed by the central developmental tasksof the preschool period; (b) with developmentallyappropriate tasks, and; (c) testing and validatingthe measure with preschool populations so thatindividual differences across this age period could becaptured and atypical behaviors could be identifiedrelative to normative behaviors in this age group. Itwas this principle that led us to design the DB-DOSto examine deficits within the context of develop-mental domains (rather than as isolated symptoms).We used DSM-IV DBD nosology as a starting pointfor delineating the core developmental domainsimpaired in children with disruptive behaviorproblems. In delineating these core domains, we alsowent beyond DSM nosology to include a domain thathas been conceptualized by developmental theoristsas playing a fundamental role in disruptive behaviortrajectories (i.e., problems in competence) (Dodge,1993; Flanagan, Bierman, & Kam, 2003; Webster-Stratton & Lindsay, 1999). This enabled us to takeadvantage of the benefits of both a “top-down” ap-proach (i.e., applying the well-validated constructs ofDSM DBDs for school-age children to preschoolers)and a “bottom-up” approach (i.e., defining problemsin relation to developmental domains). We believedthat this combined approach would enable us to bestidentify the defining features of disruptive behaviorin preschoolers, including unique developmentalfeatures, and identification of clinically relevant items.

Drawing on clinical and developmental research(Calkins & Dedmon, 2000; Campbell, 2002; Cole,Michel, & Teti, 1994; Dodge & Crick, 1990; Flanaganet al., 2003; Keenan, 2000; Kochanska et al., 2001;

8Clearly, clinicians must then use their clinical judgment to“weight” information derived from this rating system with otherinformation gathered in the diagnostic decision-making process.

Luthar, Burack, Cicchetti, & Weisz, 1997; Shaw,Gilliom, & Giovannelli, 2000), we identified threecore areas of behavioral and socio-emotionalfunctioning of particular salience for understandingthe development of disruptive behavior disordersin young children: behavioral control, emotionmodulation, and social orientation. In each area, weconceptualized behavior along a continuum fromproblematic to competent.

Behavioral control reflects the child’s capacityto regulate his/her behavior in response to social de-mands and emotional experiences. For children withdisruptive behavior, problems in behavioral controlmay include a tendency to respond aggressively whenangry and intransigence. In contrast, competencies inbehavioral control may include the presence of be-havioral coping strategies and assertiveness. Emotionmodulation reflects the child’s capacity to modulatethe intensity, duration, and appropriateness ofhis/her response to emotionally arousing situationsand stimuli. Children with disruptive behavior oftenhave problems in modulation of negative affect,including responding intensely to frustration, diffi-culty recovering when distressed, rapid escalationof upset and chronic negative mood. Competenciesin emotion modulation include positive affectivityand the ability to maintain emotional equilibriumin the face of frustration. Social orientation reflectsthe child’s responsiveness to, interest in, concernfor, and active engagement with his/her socialenvironment. Children with disruptive behavioroften have problems in social orientation such as lackof empathy, with concomitant poor internalizationof social norms (e.g., lying, deliberate rule breaking),a tendency to respond in a hostile, irritable mannerto social bids, and antisocial behaviors (e.g., spitefuland deliberately annoying behavior). In contrast,competencies in social orientation include a high levelof responsiveness to, interest in, and initiative withthe social environment, use of social strategies forcoping (e.g., asking for help), and prosocial behaviors(e.g., kind, thoughtful and empathic behaviors).

Using this theoretical framework as a guide,we then “deconstructed” DBD symptoms and othercore behaviors into their salient qualitative andquantitative dimensions to develop the items forthe DB-DOS rating system. This “deconstruction”and subsequent development of integrative codesis fundamental to the goal of the DB-DOS, i.e.,capturing clinically salient behaviors in an integratedmanner. Coding is done globally via review of video-tapes, with items rated separately for each module.


Preliminary evidence indicates good inter-raterreliability, with an average intraclass correlation of.84 (Wakschlag & Danis, 2004).

Items on the DB-DOS are organized withinthree primary domains, each of which have severalsub-domains (see Fig. 1). The Disruptive Behaviordomain is comprised of three sub-domains: oppo-sitional, aggressive, and deceptive behavior. TheModulation of Negative Affect domain has twosub-domains: negative emotional tone and escalationof negative affect. The third domain assesses childCompetence and is comprised of two sub-domains:social competence and coping/mastery. Preliminaryevidence suggests that the DB-DOS domains areinternally consistent (Cronbach’s alphas rangingfrom .76 to .85) and distinguish between referredand nonreferred children, with referred childrenexhibiting higher Disruptive Behavior and problemsin Modulation of Negative Affect and lower levels ofCompetence (F(1,34) = 12.94, p < .001) (Wakschlag& Danis, 2004).

Each DB-DOS item is scored along a continuumof either atypicality or competence. As a result, in as-sessing problematic behaviors, a broad range of typ-ical behaviors is collapsed within the zero category.Thus, in the Disruptive Behavior and Modulation ofNegative Affect problem domains, a score of “0” in-dicates no evidence of deficit or atypicality, “1” indi-cates a mild form of the behavior, which may or maynot be atypical, and scores of “2–3” are clearly atyp-ical (2: clearly atypical, 3: markedly atypical). Con-versely, in the Competence domain, a broad range ofproblem behaviors are included in the “0” category, a“1” indicates a mild form of the behavior, which mayor may not be competent, and scores of “2–3” reflectclear evidence of competence. Notably, although fre-quency of behavior is generally taken into account,its relative weight varies across different items. Forexample, serious behaviors such as hitting an adult,may be coded as problematic even if they only oc-cur once, whereas noncompliance in response to taskdirectives is more normatively expectable and mustoccur more frequently in order to be coded as prob-lematic. DB-DOS items are listed in Fig. 1, Table IIprovides a more detailed illustration of a DB-DOSitem.

DB-DOS Scoring

The DB-DOS was developed as a versatile mea-sure, with the potential for use as both a diagnostic

tool and as a research method for characterizing thephenotype of disruptive behavior in young children.Thus, our goal is for it to generate a range of scoresincluding: (a) a clinical algorithm that will establishcut-points to distinguish cases, subclinical cases andnon-cases; (b) continuous domain scores, and(c) pro-file scores (i.e., cross-domain combinations).

Validation of the DB-DOS

Validation of the DB-DOS is centered on estab-lishing its clinical utility at multiple levels, includingdiagnostic utility (i.e., enhancing precision of diagno-sis), predictive utility (i.e., enhancing prediction overtime, including the incremental utility of the measureover and above existing methods), and conceptualutility (i.e., advancing understanding of a particulartype of psychopathology (Vasey & Lonigan, 2000) aswell as ecological validity and validity across culturalcontexts.

Diagnostic Utility

By comprehensively assessing the constellationof behaviors that comprise DBD symptoms, the DB-DOS is designed to generate a clinical algorithmthat can distinguish cases from noncases with highsensitivity and specificity. Data from our pilot sam-ple indicate very good to excellent sensitivity andspecificity (92.9 and 85.7% respectively) in distin-guishing between referred and nonreferred children(Wakschlag & Danis, 2004). However, extensive psy-chometric work will be necessary to fully validatethe DB-DOS. This will include establishing clinicalcut-points and generating a diagnostic algorithm thatweights the clinical significance of specific behaviors,incorporates information about pervasiveness acrossDB-DOS modules, and takes age and sex differencesinto account. Validation will require an iterative pro-cess in which psychometric findings and clinical con-siderations inform each other.

Another central dimension of this process is de-termining the manner in which behaviors will becombined across modules. There are many questionsto be resolved in this process, such as: (a) Should abehavior be “weighted” equally if it occurs in oneversus multiple modules; and, (b) are behaviors ob-served with examiner or parent differentially salient?Preliminary analyses have indicated substantial het-erogeneity in children’s pattern of problems across


the DB-DOS contexts. For example, of the chil-dren exhibiting problems during the DB-DOS, ap-proximately 40% exhibit problems across both par-ent and examiner modules, 40% exhibit problems inthe parent module only, and 20% in the examinermodules only (Wakschlag, Leventhal, Hill, Danis, &Keenan, 2004). Analyses that examine the signifi-cance of these patterns for determining caseness andfor predictive validity will critically inform the devel-opment of the clinical algorithm. Since the DB-DOSis specifically designed as a companion to interview-based methods, developing mechanisms for weight-ing and integrating information across informantsand contexts (i.e., school, home and clinic) is also fun-damental (Carter et al., 2004; Gardner, 2000).

Conceptual Utility

Although there is an increasing support for theidentification of clinical problems in young children,the phenotype of disruptive behavior has not yetbeen well characterized; symptom counts alone arerelatively uninformative for predicting individual dif-ferences in presentation or persistence (Greenberg,Speltz, DeKlyen, & Jones, 2001). For example, lit-tle is known about sex differences in young childrenwith disruptive behavior. By taking a theoreticallydriven approach with emphasis on qualitative pat-terns of behavior, the DB-DOS is designed to have ahigh level of conceptual utility. For example, prelim-inary analyses suggest that disruptive girls are morelikely to exhibit problems in Modulation of NegativeAffect and disruptive boys are more likely to dis-play problems with Oppositionality and Aggression(Wakschlag et al., 2004). Further, the DB-DOS wasdesigned to enhance understanding of the role thatparenting plays in early emerging disruptive behav-ior trajectories by providing a standard method forassessing disruptive behavior problems both withinand outside of the parent–child relationship context.

Predictive Utility

One of the greatest knowledge gaps in thearea of preschool disruptive behavior is that ofpredictive validity. Studies using diagnostic methodswith preschoolers are relatively recent and thus,little is known about the predictive validity ofDBDs in young children. Existing data suggestmoderate stability of disruptive behavior problemsin preschool children (Briggs-Gowan & Carter,

1998; Lavigne et al., 1998; Mathiesen & Sanson,2000; Pierce, Ewing, & Campbell, 1999; Speltz et al.,1999). In order to advance scientific understandingin this area, explorations of predictive validitymust go beyond establishing stability per se toexamining individual differences in these patterns.The DB-DOS was designed to address this issue atseveral levels. First, by “deconstructing” symptomsto include qualitative features, the DB-DOS lendsitself to examination of the predictive value of thenature and pattern of particular types of behaviors inyoung children. For example, there is evidence thatdifferent types of behaviors within the constellationof ODD symptoms differentially predict stability(Speltz et al., 1999; Stormshak, Bierman, & Group,1998). Thus, the more qualitative and fine-grainedassessment of such behaviors on the DB-DOS islikely to be useful predictively. Second, by examiningbehaviors across multiple domains, the DB-DOSis uniquely suited for person-oriented analyses thatcan predict individual differences in patterns ofpersistence and desistance over time.

Ecological Validity

Ecological validity establishes that behavior ob-served in the laboratory is reflective of “real life” be-havior. The DB-DOS is structured to enhance eco-logical validity by using relatively simple tasks thatare reflective of typical experiences of preschoolers(e.g., having to wait, following directions). Data fromour pilot sample indicate that parents generally feelthat child behavior on the DB-DOS is representa-tive of the child’s usual behavior. Nearly two-thirdsof parents in our pilot sample rated their child’s be-havior on the DB-DOS as typical, with the major-ity of other parents rating their child’s behavior as“typically somewhat worse.” Interestingly, no par-ents rated their child’s behavior as “typically better,”suggesting that observations on the DB-DOS are notover-identifying problems.

Coding Parental Behavior DuringDB-DOS Module 3

Parallel to the DB-DOS child behavior codingsystem, the Parenting Clinical Observation Schedule(P-COS; Wakschlag, Hill, Danis, Grace, & Keenan,2003) was designed to assess parental behavior dur-ing the DB-DOS in a clinically sensitive manner. TheP-COS draws on parenting coding systems widely


used in developmental research in defining key con-structs, including the centrality of both positive andnegative dimensions of parenting behavior (e.g.,Denham, 1993; Kochanska, 1998; Olson, Bates, &Bayles, 1990; Patterson et al., 1991; Pettit, Bates,& Dodge, 1993; Shaw & Bell, 1993; Wakschlag &Hans, 1999). It uses the clinically oriented structureof the DB-DOS to operationalize these constructs,with problems and competencies coded within eachdomain. Salient behaviors are also deconstructed soas to “unpack” maternal behavior in a manner thatwill be informative for treatment. For example, re-sponse to misbehavior is coded in terms of firm-ness, use of anticipatory versus reactive managementstrategies, and the flexibility with which these strate-gies are employed. Similarly, we intentionally codebehavioral responsiveness and behavior managementseparately because the use of anticipatory strategiesmay be uniquely important in reducing the risk thatbehaviors will escalate (Gardner et al., 1999). Wealso include low incidence but highly pathognomicparenting behaviors in order to capture behaviorsthat are often “red flags” to the clinician of seriousproblems (e.g., power struggles). Parenting behaviorsrated on the P-COS are illustrated in Table III.

In summary, the DB-DOS, and its companioncoding system, the P-COS, were developed in re-sponse to both methodologic and substantive gapsin the study of early onset disruptive behavior. Wenow delineate their potential for advancing clinicalscience in this domain.

How the DB-DOS Can Inform Central Questionsin the Study of Early Emerging Disruptive Behavior

Question 1: What are the BoundariesBetween Typical and Disruptive Behaviorin Preschool Children?

The defining feature of DBDs is a pattern ofnegative behaviors that interfere with social interac-tions with others (DSM IV, APA, 1994). The hall-marks of ODD are defiance and negative emotional-ity. The essential features of CD are aggression andrule violation. On the surface, it would appear thatthese patterns of disordered behavior are virtuallyidentical to the normative behavioral disruption ofthe preschool years. As a result, in contrast to olderchildren, the presence of such behaviors is neces-sary but not sufficient for determining whether be-haviors are problematic at this age. As such, taking

Table III. The Parenting Clinical Observation Schedule (DB-DOS Module 3)

DB-DOS parenting domains1. Competence

1A. BehavioralBehavior management

1. Firmness2. Use of positive behavior strategies3. Flexibility and modulation of behavior management

strategiesBehavioral responsiveness

4. Scaffolding5. Responsivity to compliance and other positive

behaviors1B. Emotional responsiveness

6. Affectionate behavior7. Positive engagement8. Labeling

2. Maternal problem behaviors9. Spiteful/hostile behavior

10. Verbal aggression toward child—threats11. Verbal aggression toward child—cursing12. Physical aggression toward child13. Engagement in power struggles14. Emotional misattunement

3. Affective tone15. Intensity of positive affect16. Pervasiveness of positive affect17. Intensity of irritable/angry affect18. Pervasiveness of irritable/angry affect19. Anxious/tense20. Sad/depressed

both quantitative and qualitative features into accountin an integrated manner is crucial for distinguishingbetween typical and disruptive behavior in preschoolchildren (Campbell, 2002; Hay, Castle, & Davies,2000; Wakschlag & Danis, 2004). While developmen-tally sensitive parent interview methods can assesseach of these features as separate components of be-havior, unlike clinical observation they cannot assessthe way in which these features “fit together.” Wesuggest below clinical principles that we have devel-oped based on extensive clinical experience with dis-ruptive preschoolers and drawing on developmen-tal research (Wakschlag & Danis, 2004, Campbell,2002). A primary goal of the DB-DOS is to provideempirical support for these distinctions by generatingparameters that identify specific qualitative dimen-sions of behavior that are discriminative.

Clinical Principle A: Normative Noncompliancein Preschoolers is not Pervasive

Normative or “assertive noncompliance”(Crockenberg & Litman, 1990) is frequent but not


characteristic during the preschool period. Suchbehavior generally reflects assertions of autonomyand is rarely elicited during interactions that do notinvolve limit setting. In contrast, clinically significantoppositionality and defiance tend to be pervasiveacross settings and/or within social relationships andtransactions. This is “negativism for its own sake”or a “reflexive no” (Wenar, 1982). On the DB-DOS,we examine pervasiveness of noncompliance inseveral ways. First, using the code “pervasivenessof noncompliance” we rate the extent to whichnoncompliance is elicited mostly during tasks thatinvolve limit setting or whether it predominatesacross all types of tasks, including enjoyable ones.The DB-DOS also measures assertive noncompli-ance via an “assertiveness “variable. Second, we canexamine the extent to which the child exhibits highlevels of noncompliance pervasively across modulesor whether noncompliance manifests primarily inone module (e.g., only with parent).

Clinical Principle B: Normative Aggressionis Primarily Instrumental Rather than Hostileand Proactive and Largely Reflects ImmatureConflict Resolution Skills

Normative aggression during this period tendsto reflect an immature strategy for resolving conflictwith peers rather than deliberate efforts to hurt an-other (Hay et al., 2000; Tremblay, 2000) and is gener-ally ameliorated with adult intervention. In contrast,clinically significant aggression is proactive and per-sistent, has a deliberate, driven quality to it, and maybe nasty and/or spiteful. On the DB-DOS, we assessqualitative dimensions (e.g., severity, deliberateness)of a range of aggressive behaviors including physicaland verbal aggression, spiteful behavior, and aggres-sion towards objects.

Clinical Principle C: Normative Expressions ofNegative Affect are Relatively Well-Modulated

Normative expressions of negative affect arelikely to occur in response to frustration or fatigue,are generally of mild-moderate intensity, and, evenwhen high in intensity, have a relatively rapid pe-riod of recovery. In contrast, children with disruptivebehavior problems tend to have frequent tantrumsthat are easily elicited, prolonged, and poorly mod-ulated. Thus, on the DB-DOS, we code multiple di-mensions of the child’s ability to modulate negative

affect (e.g., pervasiveness, ease of elicitation, highestintensity). For example, the code “easy to elicit” in-corporates both the frequency of negative affect andthe type of task that elicits it (Table II). We also cap-ture qualitative dimensions of emotion modulationin terms of competencies, such as the use of “posi-tive coping strategies” in the face of challenge or frus-tration. Here the emphasis is on both the frequencywith which coping strategies are displayed as wellas their quality (e.g., the presence of varied copingstrategies).

Clinical Principle D: Normative Misbehavior andNegativity are Generally Responsive to Adult Input

In contrast, disruptive behaviors are often en-trenched, intransigent, and persistent in the face ofadult support. On the DB-DOS we capture this byintegrating the intensity of the behavior with its re-sponsiveness to adult input. For example, the codes“highest level of defiance” and “difficulty recoveringfrom negative affect” rate as most problematic be-havior that persists in the face of adult prompts orsupport.

Question 2: Can DBDs in Young Childrenbe Distinguished from Parent–ChildRelationship Problems?

This is a vexing problem, which essentially boilsdown to the question of whether the constellation ofbehaviors identified as “disruptive behavior” are ac-tually a problem that resides within the child (i.e.,a DBD) or whether they reflect behavioral difficul-ties that are a reaction to other problems within thechild’s social context. We have previously suggested(Keenan & Wakschlag, 2002), that when a patternof behavior is pervasive, persistent, and developmen-tally impairing it is a clinical problem, even whenproblematic parenting or other difficulties in the socialcontext are present. This reflects our theoretical view-point that problem behaviors that impair a child’sability to successfully master normative developmen-tal tasks and hinder participation in developmentallyappropriate activities are clinically significant, irre-spective of etiology. This viewpoint does not discountthe central role that parents play in the developmentof behavioral regulation in young children, however(e.g., Kochanska et al., 2001). This framework alsohighlights the fact that there are multiple pathways tothe development of early emerging psychopathology,


including the potent contribution of child character-istics (e.g., temperamental vulnerability) to the de-velopment of disruptive behavior (Keenan & Shaw,2003).

The question of preschool psychopathologyversus disordered parent–child relationship is hotlydebated within the field (Carter et al., 2004; Emde,2003; Jensen & Hoagwood, 1997; Zeanah, Boris &Scheeringa, 1997). A primary goal of the DB-DOSis to provide data to help answer this questionempirically. First, embedding standard assessment ofparenting behavior within the diagnostic observationparadigm allows for systematic control of problem-atic parenting behavior in analyses examining thediscriminant, predictive, and clinical utility of childbehavior on the DB-DOS. For example, we canexamine whether child behavior on the DB-DOSpredicts impairment in the school setting, onceparenting behavior has been controlled.

Second, standard clinically oriented assessmentof parental problems and competencies via theP-COS ratings will enable us to begin to charac-terize individual differences in parenting patternsamongst the parents of children with disruptive be-havior problems. (In contrast, much previous workhas compared the parenting received by childrenwith and without behavior problems.) Recently, weconducted preliminary analyses to examine whetherhighly problematic parenting behavior distinguishedchildren who exhibited problems on the DB-DOSparent-module only. Interestingly, while half of thesechildren had parents displaying highly problematicparenting styles, nearly one-third of this group hadparents who exhibited highly competent parenting(Wakschlag et al., 2004). This would suggest thatwhile parent–child relationship problems may be acontributing factor for a sub-group of children withdisruptive behavior, there may also be distinct sub-groups of children whose behavior problems derivefrom other pathways, including (a) temperamen-tal vulnerabilities and (b) the interaction of child-specific and parenting factors. Understanding suchheterogeneity in causal pathways is a fundamentalnext step for the field. The opportunity the DB-DOSprovides to examine child disruptive behaviors bothwithin and outside of the context of parent–child in-teraction is likely to be especially informative in thisprocess, because it allows for objective rating of childbehavior that is not dependent on parents’ subjectiveexperience of the child.

The role of parent behavior in diagnostic con-ceptualization must also be considered. For our

purposes, problematic parenting is considered a riskfactor rather than a clinical indicator. Thus, we do notexpect to include parenting behavior within the diag-nostic algorithm. On the other hand, as noted above,P-COS data on quality of parenting may also beused to delineate sub-types (e.g., disruptive behav-ior with or without parenting problems). In addition,we will examine whether taking P-COS profile scoresinto account enhances the predictive utility of theDB-DOS (e.g., examining whether competent par-enting predicts desistance).

Question 3: Can the Heterogeneity of DisruptiveBehavior in Young Children be Characterizedin a Manner that Enhances Predictionand Informs Etiologic Studies?

The predictive validity of DBDs in preschoolersis a critical, but relatively untested area of inquiry(but see Speltz et al., 1999). We hypothesize thatobservational methods will add incremental valuein diagnostic classification and prediction over timefor several reasons. First, the clinical presentationof young children with DBDs is very heterogeneous((Wakschlag & Danis, 2004). For example, somechildren with oppositional-defiant symptoms exhibitchronic negative mood, whereas others present onlywith emotional dysregulation when upset. It is notknown what this heterogeneity “means” in termsof persistence and desistance of disruptive behaviorproblems. Further, it is not yet clear whether thereare distinct disruptive behavior constellations (i.e.,separate oppositional and conduct patterns) at thisage or whether there is a more nonspecific disruptivebehavior disorder in preschoolers. By observing pat-terns of behavior, rather than isolated symptoms, wewill be able to systematically characterize individualdifferences in manifestations (e.g., sex differences,how constellations of behaviors “hang together”) ina manner that will enable us to better understand andcharacterize multiple trajectories over time.

We also contend that the development of a stan-dardized observational method is fundamental tobreaking new scientific ground in research on earlyemerging DBDs, independent of its incrementalpredictive value. As emphasized in the blueprint forDSM-V, recent advances in developmental sciencecreate unparalleled opportunities to broaden themethods and approaches used in refining taxonomiesof developmental psychopathology (Pine et al.,2002). A standardized observational method has


unique potential for advancing this line of inquiryin several ways. First, observational methods areuniquely suited to enhancing phenotypic descriptionof disruptive behavior in young children. Advancesin genetic research have led to a shift in focus from asearch for genetic causes for discrete categorical dis-orders to one on genetic contributions to dimensionalrisk factors involved in the multifactorial origins ofdisorder (Rutter, 1997). Phenotypic description atthis level requires “deconstructing” the grosser levelsof behavior captured by symptoms into their variouscomponents. “Mapping” neuroscientific findings onstructural and functional brain deficits onto varioustypes of psychopathology also requires this level ofdescription (Pine et al., 2002). Examining disruptivebehavior as a more continuous phenomenon iscentral to this line of inquiry. All of these approachesnecessitate a high level of clarity of description (Pineet al., 2002; Sroufe, 1997; Wakefeld, 1997), whichobservational methods are especially well-suited toprovide.

The DB-DOS may contribute to enhanced phe-notypic description in several ways. First, symptomsare “deconstructed” into their various components.As such, one DSM symptom such as “touchy/easilyannoyed” is measured by three more fine-grainedcodes on the DB-DOS: easy to elicit negative affect,its rapid escalation, and difficulty recovering fromit. Similarly, the symptom “often defies or refusesto comply with adult requests/rules” is capturedwith four DB-DOS codes assessing intensity, type,and pervasiveness of these behaviors. Second, theinclusion of a set of behaviors that are not part ofcurrent DSM nosology, but have frequently beenidentified as deficient in children with DBDs (i.e.,problems in social competence), will enable us toexamine whether these behaviors are, in fact, adefining feature of DBDs in young children, therebyexpanding beyond the limits of current taxonomies.Further, organizing behavioral assessment in theDB-DOS within developmental domains (i.e.,disruptive behavior, modulation of negative affectand competence) rather than in diagnostic categorieslends itself to person-oriented analysis. This willenable us to move beyond categorical grouping totypologies of behavior and to assess whether theseclusters vary across patterns of disruptive behavior,such as persistence of symptoms, diagnostic status,and level of impairment over time. As has beendemonstrated by the significant role the ADOShas played in research on autism, such precisecharacterization is not only useful for, but also may,

in part, drive advances in epidemiologic, genetic,developmental neuroscience, and pharmacologicresearch (Pine et al., 2002; Vitiello, 2001).

Conclusion: Where Do We Go From Here?

We have highlighted the potential of diagnosticobservation methods in general, and the DB-DOSin particular, to advance scientific inquiry about thenosology, patterns, and clinical significance of earlyemerging disruptive behavior. We conclude by out-lining fundamental next steps in this process.

Validation of the DB-DOS

As noted above, we are currently conductinga large-scale validation of the DB-DOS measure.The primary goals of this study are: (1) psychome-tric analysis and refinement of the DB-DOS instru-ment including (a) developing a clinical algorithmthat optimizes sensitivity and specificity for classify-ing children along a continuum of disruptive behav-ior problems and (b) testing the incremental valueof DB-DOS ratings over and above information de-rived from parent interview for predicting impair-ment and persistence of DBDs over time; (2) exam-ining whether DB-DOS scores distinguish childrenwith disruptive behavior problems from childrenwith developmental problems and children who arenot pervasively disruptive but are disruptive in thecontext of problematic parent–child interactions and;(3) examining the predictive value of DB-DOS pro-files for disruptive behavior trajectories over time.The validation sample will consist of 360 preschool-ers along a disruptive behavior continuum, includingchildren referred for disruptive behavior problems,nonreferred children whose parents have behavioralconcerns and nonreferred children without behav-ioral concerns. Laboratory observations of child be-havior on the DB-DOS will be validated in relationto parent and teacher report of child behavior andobservations of the child in a real life (i.e., school)setting.

Demonstration of the reliability and validity ofthe DB-DOS within this study is an important firststep towards demonstrating the potential contribu-tion of diagnostic observation to the study of clinicalphenomena in young children. This work will lay thefoundation for future studies that will examine thevalidity and reliability of the DB-DOS within socialand cultural context. First, examination of behaviorsobserved on the DB-DOS to child behavior observed


in other real life contexts, such as the home, will bea crucial test of its ecological validity. As has beennoted (Gardner, 2000), even assessments with highface validity may not necessarily reflect “real life”behavior. Second, a vitally important next step willbe examination of the reliability and validity of theDB-DOS across varying socioeconomic and ethniccontexts. Although the importance of context iswidely acknowledged in clinical and developmentalresearch, measurement and model equivalenceacross contexts is often assumed and rarely tested(Carter et al., 2004; Cicchetti & Aber, 1998; Raver,2004). As has been noted, a contextual approach iscrucial for identifying individual differences in andcontextual influences on, risk processes (Cicchetti& Aber, 1998; Raver, 2004). Standardization ofthe DB-DOS within a representative sample ofpreschoolers is also an important area for futureresearch.

Clinical Utility

In addition to the scientific issues to be ad-dressed, testing whether the DB-DOS will have sub-stantial “added value” to clinicians is critical to estab-lishing its clinical utility. As noted above, cliniciansface the pragmatic issue that there is currently nostandardized method for direct clinical assessment ofpreschool children’s behavior, despite high rates ofclinical referral (Wakschlag & Danis, 2004). Estab-lishing clinical utility requires demonstrating that: (a)clinical administration is feasible; (b) real-time scor-ing is reliable and valid; and (c) that individuals witha range of clinical expertise and experience can bereliably trained. Administration of the DB-DOS isrelatively brief and is consistent with the length ofthe ADOS as well as most diagnostic interviews. TheDB-DOS was also designed to be a clinically usefulmeasure—such that tasks are as naturalistic as pos-sible, do not feel highly “contrived,” and do not re-quire costly or sophisticated equipment or laboratorysetups. Testing of the reliability and validity of real-time clinical scoring of the DB-DOS is currently un-derway. Our ultimate goal is to have an instrumentthat can be widely used by both research and gen-eral clinicians. However, we note that the DB-DOS isnot meant to train individuals in clinical observationbut rather to provide trained observers with a stan-dard method of clinical observation. Thus, we antici-pate that a prerequisite for its use will be training andexperience in clinical observation and assessment aswell as knowledge of normative development.

With the benefits of diagnostic observation, alsocome new challenges. By adding an additional sourceof clinical information, the possibility that thesesources will provide discrepant information is in-creased. Thus, systematic methods for reconcilingdifferences across methods will be needed (Carteret al., 2004). Further, while “going beyond” DSMsymptoms may enhance our understanding of indi-vidual differences in clinical presentation and trajec-tories, it also raises the question of whether and howthese new dimensions should be incorporated intodiagnostic nosology. Whether or not the DB-DOSis sensitive enough to capture treatment gains overtime is another question that remains to be testedempirically.

In conclusion, although the DB-DOS holdsmuch promise, the importance of more precise char-acterization of preschool disruptive behavior extendswell beyond a particular diagnostic observation mea-sure. The past decade has taught us much about theemergence and identification of mental health prob-lems in the first few years of life. Delineation of theboundaries between typical and atypical behavior inyoung children now appears to be possible. Whilethere are many challenges that lie ahead, advancingscientific inquiry in this manner is likely to deepenour understanding of the origins of psychopathology,in all its diversity and complexity, in a manner withdirect implications for prevention, assessment, andtreatment of clinical problems in young children.

ACKNOWLEDGMENTS

The development of the DB-DOS and the writ-ing of this paper have been supported by NIMHgrant 1R01 MH068455-01 to Dr. Wakschlag and1 R01 MH62437-01 to Dr. Keenan, support to Drs.Wakschlag, Carter, Briggs-Gowan & Egger from Na-tional ZERO TO THREE (via a grant to ZERO TOTHREE from the Robert Wood Johnson Founda-tion) and ongoing support by the Walden and JeanYoung Shaw Foundation and the Irving B. HarrisCenter for Developmental Studies to the Depart-ment of Psychiatry at the University of Chicago.Dr. Catherine Lord’s seminal work on the ADOSwas the inspiration for the DB-DOS, her continu-ing incisive and generous guidance has been vitallyimportant to its development. We are indebted toDr. Chaya Roth, whose teachings laid the founda-tion for this work. Dr. Edwin Cook’s ongoing criti-cal feedback is gratefully acknowledged. We are also


indebted to Dr. Adrian Angold for his critical com-ments. We are very appreciative of the contribu-tion of our research team and clinical students whosefeedback has greatly enriched this effort. Finally, theDB-DOS is dedicated to the memory of our belovedstudent and colleague, Kathleen Kennedy Martin,Psy.D., a consummate clinical observer who embod-ied the principles upon which the DB-DOS rests.

REFERENCES

Benham, A. (2000). The observation and assessment of young chil-dren including use of the Infant–Toddler Mental Status Exam.In C. Zeanah (Ed.), Handbook of infant mental health (2nded., pp. 249–266). New York, NY: Guilford Press.

Bradley, R. H., Mundfrom, D. J., Whiteside, L., Casey, P. H.,& Barrett, K. (1994). A factor analytic study of the infant–toddler and early childhood versions of the HOME inventoryadministered to White, Black, and Hispanic American par-ents of children born preterm. Child Development, 65, 880–888.

Briggs-Gowan, M., & Carter, A. (1998). Preliminary acceptabil-ity and psychometrics of the Infant–Toddler Social and Emo-tional Assessment (ITSEA): A new adult report question-naire. Infant Mental Health Journal, 19, 422–445.

Briggs-Gowan, M. J., Carter, A. S., & Schwab-Stone, M. (1996).Discrepancies among mother, child, and teacher reports: Ex-amining the contributions of maternal depression and anxiety.Journal of Abnormal Child Psychology, 24(6), 749–765.

Buckley, M., Klein, D., Durbin, C., Hayden, E., & Moerk, K.(2002). Development and validation of a Q-sort procedureto assess temperament and behavior in preschool children.Journal of Clinical Child and Adolescent Psychology, 31, 525–539.

Calkins, S. D., & Dedmon, S. E. (2000). Physiological and be-havioral regulation in two-year-old children with aggres-sive/destructive behavior problems. Journal of AbnormalChild Psychology, 28, 103–119.

Campbell, S. (2002). Behavior problems in preschool children:Clinical and developmental issues (2nd ed.). New York, NY:Guilford Press.

Campbell, S., Pierce, E., March, C., Ewing, L., & Szumowski, E.(1994). Hard-to-manage preschool boys: Symptomatic behav-ior across contexts and time. Child Development, 65, 836–851.

Campbell, S. B., Szumowski, E. K., Ewing, L. J., Gluck, D. S.,& Breaux, A. M. (1982). A multidimensional assessment ofparent-identified behavior problem toddlers. Journal of Ab-normal Child Psychology, 10, 569–592.

Campione-Barr, N., & Smetana, J. (2004). In the eye of thebeholder: Subjective and observer ratings of middle-classAfrican American mother-adolescent interactions. Develop-mental Psychology, 40(6), 927–934.

Carter, A. S., Briggs-Gowan, M. J., Jones, S. M., & Little, T. D.(2003). The Infant-Toddler Social and Emotional Assessment(ITSEA): Factor structure, reliability, and validity. Journal ofAbnormal Child Psychology, 31, 495–514.

Carter, A., Briggs-Gowan, M., & Ornstein Davis, N. (2004). As-sessment of young children’s social-emotional developmentand psychopathology: Recent advances and recommenda-tions for practice. Journal of Child Psychology, Psychiatry andAllied Disciplines, 45, 1–26.

Cicchetti, D., & Aber, J. L. (1998). Contextualism and develop-mental psychopathology. Development and Psychopathology,10, 137–142.

Cole, P., Michel, M., & Teti, L. (1994). The development of emo-tion regulation and dysregulation: A clinical perspective. InN. Fox (ed.), The development of emotion regulation: Bio-logical and behavioral considerations (serial no. 240, vol. 59,pp. 73–102). Chicago: University of Chicago Press.

Criss, M. M., Shaw, D., & Ingoldsby, E. M. (2003). Mother-sonpositive synchrony in middle childhood: Relation to antisocialbehavior. Social Development, 12, 379–401.

Crockenberg, S., & Litman, C. (1990). Autonomy as competencein 2 year olds: Maternal correlates of child defiance, com-pliance, and self assertion. Developmental Psycholology, 26,961–971.

DelCarmen-Wiggins, R., & Carter, A. (2004). Handbook of infant,toddler and preschool mental health assessment. New York:Oxford University Press.

Denham, S. (1993). Maternal emotional responsiveness and tod-dlers’ social-emotional competence. Journal of Child Psychol-ogy and Psychiatry, 34, 715–728.

Dodge, K., & Crick, N. (1990). Social information-processingbases of aggressive behavior in children. Personality and So-cial Psychology Bulletin, 16, 8–22.

Dodge, K. (1993). Social-cognitive mechanisms in the develop-ment of conduct disorder and depression. Annual Reviews ofPsychology, 44, 559–584.

Dumas, J., Serketich, W., & LaFreniere, P. (1995). “Balance ofpower”: A transactional analysis of control in mother-childdyads involving socially competent, aggressive, and anxiouschildren. Journal of Abnormal Psychology, 104, 104–113.

Elardo, R., & Bradley, R. (1981). The Home Observation for Mea-surement of the Environment Scale: A review of research. De-velopmental Review, 1, 113–145.

Emde, R. (2003). Commentary: RDC-PA: A major step forwardand some issues. Journal of the American Academy of Childand Adolescent Psychiatry, 42(12), 1513–1516.

Eyberg, S., Boggs, S., & Algina, J. (1995). New developments inpsychosocial, pharmacological, and combined treatments ofconduct disorders in aggressive children. Psychopharmacol-ogy Bulletin, 31, 83–91.

Flanagan, K., Bierman, K., Kam, C., & Conduct Problems Pre-vention Research Group (2003). Identifying at-risk childrenat school entry: The usefulness of multibehavioral problemprofiles. Journal of Clinical Child and Adolescent Psychology,32, 396–407.

Frick, P., & Loney, B. (2000). The use of laboratory andperformance-based measures in the assessment of chil-dren and adolescents with conduct disorders. Journal ofClinical Child Psychology (Special Section: Laboratory andperformance-based measures of childhood disorders), 29, 540–555.

Gardner, F. (2000). Methodological issues in direct observation ofparent-child interaction: Do observational findings reflect thenatural behavior of participants? Clinical Child and FamilyPsychology Review, 3, 185–198.

Garner, P., & Power, T. (1996). Preschoolers’ emotional controlin the disappointment paradigm and its relation to temper-ament, emotional knowledge, and family knowledge. ChildDevelopment, 67, 1406–1419.

Gardner, F., Sonuga-Barke, E. J., & Sayal, K. (1999). Parents an-ticipating misbehaviour: An observational study of strategiesparents use to prevent conflict with behaviour problem chil-dren. Journal of Child Psychology and Psychiatry and AlliedDisciplines, 40, 1185–1196.

Gardner, F., Ward, S., Burton, J., & Wilson, C. (2003). The role ofmother-child joint play in the early development of children’sconduct problems: A longitudinal observational study. SocialDevelopment, 12(3), 361–379.

Gilliam, W., & Mayes, L. (2004). Integrating clinical and psy-chometric approaches: Developmental assessment and in-fant mental health evaluation. In R. Del Carmen Wiggins &


A. Carter (Eds.), Handbook of infant, toddler and preschoolmental health assessment (pp. 185–203). New York: OxfordUniversity Press.

Gilliom, M., Shaw, D. S., Beck, J. E., Schonberg, M. A., & Lukon,J. L. (2002). Anger regulation in disadvantaged preschoolboys: Strategies, antecedents, and the development of self-control. Developmental Psychology, 38, 222–235.

Greenberg, M. T., Speltz, M. L., DeKlyen, M., & Jones, K.(2001). Correlates of clinic referral for early conduct prob-lems: Variable- and person-oriented approaches. Develop-ment and Psychopathology, 13(2), 255–276.

Hay, D. F., Castle, J., & Davies, L. (2000). Toddlers’ use of forceagainst familiar peers: A precursor of serious aggression?Child Development, 71, 457–467.

Hay, D., Pawlby, S., Sharp, D., Schmucker, G., Mills, A.,Allen, H., & Kumar, R. (1999). Parents’ judgements aboutyoung children’s problems: Why mothers and fathers dis-agree yet still predict later outcomes. Journal of ChildPsychology, Psychiatry and Allied Disciplines, 40, 1249–1258.

Haynes, S. (2001). Introduction to the special section on clinicalapplications of analogue behavioral observation. Psychologi-cal Assessment (Special Issue on Clinical Applications of Ana-logue Behavioral Observation), 13, 3–4.

Hubbard, J., Smithmyer, C., Ramsden, S., Parker, E., Flanagan, K.,Dearing, K., Relyea, N., & Simons, R. (2002). Observational,physiological, and self-report measures of children’s anger:Relations to reactive versus proactive aggression. Child De-velopment, 73, 1101–1119.

Jensen, P., & Hoagwood, K. (1997). The book of names: DSM-IVin context. Developmental Psychopathology, 9, 231–250.

Kagan, J. (1998). Biology and the child. In W. Damon &N. Eisenberg (Eds.), Handbook of child psychology: Social,emotional and personality development (Vol. 3, pp. 177–236).New York: Wiley.

Keenan, K. (2000). Emotion dysregulation as a risk factor forpsychopathology. Clinical Psychology Science and Practice, 7,418–434.

Keenan, K., & Shaw, D. (2003). Starting at the beginning: Explor-ing the etiology of antisocial behavior in the first years of life.In B. Lahey, T. Moffit, & A. Caspi (Eds.), Causes of conductdisorder and juvenile delinquency (pp. 153–181). New York:Guilford.

Keenan, K., & Wakschlag, L. (2002). Can a valid diagnosis ofdisruptive behavior disorder be made in preschool children?American Journal of Psychiatry, 59, 351–358.

Kerig, P., & Lindahl, K. (2001). Family observational coding sys-tems: Resources for systemic research. Mahwah, NJ: Erlbaum.

Kim, S., Young, L., Gonen, D., Veenstra-VanderWeele, J.,Courchesne, R., Courchesne, E., Lord, C., Leventhal, B.,Cook E. Jr, & Insel, T. (2002). Transmission disequilib-rium testing of arginine vasopressin receptor 1A (AVPR1A)polymorphisms in autism. Molecular Psychiatry, 7, 503–507.

Kochanska, G. (1998). Mother-child relationship, child fearful-ness, and emerging attachment: A short-term longitudinalstudy. Developmental Psychology, 34, 490–490.

Kochanska, G., & Aksan, N. (1995). Mother-child mutual positiveaffect, the quality of child compliance to requests and prohi-bitions, and maternal control as correlates of early internal-ization. Child Development, 66, 236–254.

Kochanska, G., Coy, K., & Murray, K. (2001). The developmentof self-regulation in the first four years of life. Child Develop-ment, 72, 1091–1111.

Lavigne, J., Arend, R., Rosenbaum, D., Binns, H., Christoffel, K.,& Gibbons, R. (1998). Psychiatric disorders with onset inthe preschool years: I. Stability of Diagnoses. Journal of theAmerican Academy of Child and Adolescent Psychiatry, 37,1246–1254.

Lord, C. (1995). Follow-up of two-year-olds referred for possibleautism. Journal of Child Psychology and Psychiatry, 36, 1365–1382.

Lord, C. (1997). Diagnostic instruments in autism spectrum disor-ders. In D. Cohen & F. Volkmar (Eds.), Handbook of autismand pervasive developmental disorders (2nd ed., pp. 460–483).New York: Wiley.

Lord, C., Leventhal, B., & Cook, E. (2001). Quantifying the phe-notype in autism spectrum disorders. American Journal ofMedical Genetics: Neuropsychiatric Genetics, 105, 36–38.

Lord, C., Risi, S., Lambrecht, L., Cook, E., Leventhal, B.,DiLavore, P., Pickles, A., & Rutter, M. (2000). The AutismDiagnostic Observation Schedule-Generic: A standard mea-sure of social and communication deficits associated with thespectrum of autism. Journal of Autism and DevelopmentalDisorders, 30, 205–223.

Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism DiagnosticInterview-Revised: A revised version of the a diagnostic inter-view for caregivers of individuals with possible pervasive de-velopmental disorders. Journal of Autism and DevelopmentalDisorders, 24, 659–685.

Luthar, S., Burack, J., Cicchetti, D., & Weisz, J. (Eds.). (1997).Developmental psychopathology: Perspectives on adjustment,risk and disorder. New York, NY: Cambridge UniversityPress.

Mash, E., & Foster, S. (2001). Exporting analogue behavioral ob-servation from research to clinical practice: Useful or cost-defective? Psychological Assessment, 13, 86–98.

Mathiesen, K., & Sanson, A. (2000). Dimenisons of early child-hood behavior problems: Stability and predictors of changefrom 18 to 30 months. Journal of Abnormal Child Psychology,28, 15–31.

Olson, S., Bates, J., & Bayles, K. (1990). Early antecedents ofchildhood impulsivity: The role of parent-child interaction,cognitive competence and temperament. Journal of Abnor-mal Child Psychology, 18, 317–334.

Owley, T., McMahon, W., Cook, E., Laulhere, T., South, M.,Mays, L., Shernoff, E., Lainhart, J., Modahl, C., Corsello,C., Ozonoff, S., Risi, S., Lord, C., Leventhal, B., & Filipek,P. (2001). Multi-site double-blind, placebo-controlled trial ofprocine secretin in autism. Journal of the American Academyof Child and Adolescent Psychiatry, 40, 1293–1299.

Patterson, G., Capaldi, D., & Bank, L. (1991). An early startermodel for predicting delinquency. In D. Pepler & K. Rubin(Eds.), The development and treatment of aggression in child-hood (pp. 139–168). Hillsdale, NJ: Erlbaum.

Pelligrini, A. (2001). The role of direct observation in the assess-ment of young children. Journal of Child Psychology and Psy-chiatry and Allied Disciplines, 42, 861–870.

Pettit, G., Bates, J., & Dodge, K. (1993). Family interaction pat-terns and children’s conduct problems at home and school: Alongitudinal perspective. School Psychology Review, 22, 403–420.

Pierce, E., Ewing, L., & Campbell, S. (1999). Diagnostic status andsymptomatic behavior of hard-to-manage preschool childrenin middle childhood and early adolescence. Journal of ClinicalChild Psychology, 28, 44–57.

Pine, D., Alegria, M., Cook, E., Jr, Costello, E., Dahl, R.,Koretz, D., Merikangas, K., Reiss, A., & Vitiello, B.(2002). Advances in developmental science and DSM-IV. InD. Kupfer, M. First, & D. Regier (Eds.), A research agenda forDSM-V (pp. 85–122). Washington, DC: American PsychiatricAssociation.

Raver, C. (2004). Placing emotional self-regulation in socioculturaland socioeconomic contexts. Child Development, 75, 346–354.

Robinson, E. A., & Eyberg, S. M. (1981). The dyadic parent-child interaction coding system: Standardization and valida-tion. Journal of Consulting and Clinical Psychology, 49, 245–250.


Rutter, M. (1997). Nature-nurture integration: The example of an-tisocial behavior. American Psychologist, 52, 390–398.

Shaw, D. (2003). Innovative approaches and methods to study ofchildren’s conduct problems. Social Development, 12, 309–314.

Shaw, D., & Bell, R. (1993). Developmental theories of parentalcontributors to antisocial behavior. Journal of AbnormalChild Psychology, 21, 493–518.

Shaw, D., Gilliom, M., & Giovannelli, J. (2000). Aggressive behav-ior disorders. In C. Zeanah (Ed.), Handbook of infant mentalhealth (2nd ed., pp. 397–411). New York: Guilford Press.

Shaw, D., Winslow, E., Owens, E., Vondra, J., Cohn, J., & Bell, R.(1998). The development of early externalizing problemsamong children from low-income families: A transforma-tional perspective. Journal of Abnormal Child Psychology, 26,95–108.

Speltz, M., McMellan, J., DeKlyen, M., & Jones, K. (1999).Preschool boys with oppositional defiant disorder: Clinicalpresentation and diagnostic change. Journal of the AmericanAcademy of Child and Adolescent Psychiatry, 38(7), 838–845.

Speltz, M. L., DeKlyen, M., Greenberg, M. T., & Dryden, M.(1995). Clinic referral for oppositional defiant disorder: Rela-tive significance of attachment and behavioral variables. Jour-nal of Abnormal Child Psychology, 23, 487–507.

Sroufe, L.A. (1997). Psychopathology as outcome of development.Development and Psychopathology, 9, 251–268.

Sroufe, L. A. (1996). Emotional development: The organization ofemotional life in the early years. New York: Cambridge Uni-versity Press.

Stormshak, E., Bierman, K., & Group, C. P. P. R. (1998). The im-plications of different developmental patterns of disruptivebehavior problems for school adjustment. Development andPsychopathology, 10, 451–468.

Task Force on Research Diagnostic Criteria: Infancy andPreschool (2003). Research diagnostic criteria for infants andpreschool children: The process and empirical support. Jour-nal of the American Academy of Child and Adolescent Psychi-atry, 42, 1504–1512.

Tremblay, R. (2000). The development of aggressive behavior dur-ing childhood: What have we learned in the past century?International Journal of Behavioral Development, 24, 129–134.

Vasey, M., & Lonigan, C. (2000). Considering the clinical utilityof performance-based measures of childhood anxiety. Journalof Clinical Child Psychology (Special Section: Laboratory andperformance-based measures of childhood disorders), 29, 493–508.

Vitiello, B. (2001). Psychopharmacology for young children: Clini-cal needs and research opportunities. Pediatrics, 108, 983–989.

Wakefeld, J. (1997). When is development disordered? Develop-mental psychopathology and the harmful dysfunction analy-sis of mental disorder. Development and Psychopathology, 9,269–290.

Wakschlag, L., & Danis, B. (2004). Assessment of disruptive be-havior in young children: A clinical-developmental frame-work. In R. DelCarmen-Wiggins & A. S. Carter (Eds.), Hand-book of infant, toddler and preschool mental health assessment(pp. 421–440). New York: Oxford University Press.

Wakschlag, L., & Hans, S. (1999). Relation of maternal respon-siveness during infancy to the development of behavior prob-lems in high risk youths. Developmental Psychology, 37, 569–579.

Wakschlag, L., Hill, C., Danis, B., Grace, D., & Keenan, K.(2003). Parenting Clinical Observation Schedule (P-COS) forthe DB-DOS. (Version 1.1). Unpublished manual, Chicago:University of Chicago.

Wakschlag, L., & Keenan, K. (2001). Clinical significance and cor-relates of disruptive behavior symptoms in environmentallyat-risk preschoolers. Journal of Clinical Child Psychology, 30,262–275.

Wakschlag, L., Leventhal, B., Danis, B., Keenan, K., Hill, C.,Egger, H., & Carter, A. (2002). The Disruptive Behavior Di-agnostic Observation Schedule (DB-DOS). (Version 1.2). Un-published manual, Chicago: University of Chicago.

Wakschlag, L., Leventhal, B., Hill, C., Danis, B., & Keenan, K.(2004, October). Observed disruptive behavior in the parent-child context and impairment. Paper presented at theMeetings of the American Academy of Child and AdolescentPsychiatry, Washington, DC.

Webster-Stratton, C., & Lindsay, D. (1999). Social competenceand conduct problems in young children: Issues in assessment.Journal of Clinical Child Psychology, 28, 25–43.

Wenar, C. (1982). On negativism. Human Development, 25, 1–23.Westen, D., & Weinberger, J. (2004). When clinical description be-

comes statistical prediction. American Psychologist, 59, 595–614.

Zeanah, C., Boris, N., & Scheeringa, M. (1997). Psychopathologyin infancy. Journal of Child Psychology, Psychiatry and AlliedDisciplines, 38, 81–99.

Zelenko, M. (2004). Observation in infant-toddler mental healthassessment. In R. DelCarmen-Wiggins & A. Carter (Eds.),Handbook of Infant, Toddler and Preschool Mental HealthAssessment (pp. 205–222). New York: Oxford UniversityPress.

Documents

Deﬁning the “Disruptive” in Preschool Behavior: What ...devepi.duhs.duke.edu/library/pdf/19724.pdf · Deﬁning the “Disruptive” in Preschool Behavior 185 including structured