Best Practices in the Systematic Direct Observation of ...persingj/systematicobservation.pdf · tioners, Wilson and Reschly (1996) found that of the ... deemed eligible for special

Best Practices in the SystematicDirect Observation of StudentBehavior

63

OVERVIEW

Direct observation is one of the most widely usedassessment procedures by school psychologists. In asurvey of more than 1,000 school psychology practi-tioners, Wilson and Reschly (1996) found that of the26 different types of assessment instruments listedacross seven different assessment categories (e.g.,ability/intelligence, social/emotional, visual/motor),structured observational methods were ranked high-est in terms of frequency of use. Overall, practition-ers report that they conduct more than 15 behavioralobservations of student behavior during the course ofa typical month.

Systematic direct observation refers to observationof behavior other than behavior that has been explic-itly elicited by a predetermined and standardized setof stimuli (i.e., test behavior (Salvia & Ysseldyke,2001)). School psychology practitioners commonlyuse both naturalistic and systematic direct approachesto observing student behavior. Briefly, naturalisticobservation approaches refer to those observationalprocedures where the observer enters specific situa-tions (e.g., a classroom) and observes all that is goingon with no predetermined behaviors in mind. Here,the most common way of recording observations is tokeep an anecdotal record of the behaviors that seemimportant to the observer. In summarizing the infor-mation, the observer provides a complete descriptionof the many behaviors and the context in which theyoccurred.

In contrast, systematic direct approaches to behav-ioral observation are distinguished by five character-istics (Salvia & Ysseldyke, 2001). First, the goal ofobservation is to measure specific behaviors. Second,the behaviors being observed have been operationallydefined a priori in a precise manner. Third, observa-tions are conducted under standardized proceduresand are highly objective in nature. Fourth, the timesand places for observation are carefully selected andspecified. Fifth, scoring and summarizing of data arestandardized and do not vary from one observer toanother.

Both naturalistic and systematic direct approachesto observing behavior have proven useful in develop-ing theory and practice related to the assessment andintervention of student behavior. This chapter pro-vides an overview of (a) naturalistic observationalprocedures, (b) systematic direct observational pro-cedures, and (c) observational instruments as theyrelated to the observation of student behavior in theclassroom.

BASIC CONSIDERATIONS

Rationale for the Systematic Direct Observation of Behavior

With the reauthorization of the Individuals with Dis-abilities Education Act (IDEA, 1997), now more thanever it is absolutely incumbent that school psycholo-gists adopt assessment and evaluation practices that

John M. HintzeUniversity of Massachusetts at AmherstRobert J. Volpe and Edward S. Shapiro

Lehigh University

993

Best Practices in School Psychology IV

994

allow for gathering relevant functional informationon student behavior and performance patterns.Specifically, sections 300.532(b) and (c) under theProcedures for Evaluation and Determination of Eli-gibility of IDEA indicate that a variety of assessmenttools and strategies be used to gather relevant func-tional and developmental information about a childand that they be validated for the specific purpose forwhich they are used. Moreover, section 300.532(d)requires the use of tests and other evaluation materi-als that include those tailored to assess specific areasof educational need and not merely those that aredesigned to provide a single general intelligence quo-tient. Similarly, sections 300.533(a)(1)(ii) and (iii)indicate that as part of the initial evaluation for spe-cial education, evaluation data include current class-room-based assessments and observations conductedby teachers and related services providers. Oncedeemed eligible for special education, section300.347(2)(ii) requires the specification of measur-able annual goals in the development of the individ-ualized education program that meet each of thechild’s needs as they pertain to the disability classifi-cation. The spirit of IDEA is also seen in the Princi-ples for Professional Ethics and Standards for theProvision of School Psychological Services of the Pro-fessional Conduct Manual (National Association ofSchool Psychologists, 2000). Here, school psycholo-gists are required to use assessment techniques andinstruments that have established validity and relia-

bility and to adopt assessment practices that increasethe likelihood of developing effective educationalinterventions.

The call for the increased use of systematic directobservational procedures and a movement towardmore ecologically sensitive functional assessmentpractices contrast directly with those of a more tra-ditional approach that have long been the standardin school psychological assessment practices. Table1 provides a comparison between traditional andbehavioral assessment across a number of key fun-damentals. Central to the difference between thetwo is how they view the role and causes of behav-ior and the type of inferences that can be made fromobserving behavior. From a behavioral perspective,all behavior is a result of the dynamic interplaybetween an individual and the environment. Thus,observing a person’s behavior is useful only to theextent that it can be used to summarize a personunder a specific set of circumstances. Behavior isthought to be specific to a situation. For this reason,assessment needs to be ongoing and direct until acertain level of stability or generalizability isobserved. This is in contrast to a more traditionalapproach whereby observed behavior is assumed tobe enduring and stable across all situations andintrapsychic or residing within the individual. Forthis reason, assessment can be global and static; thatis, occurring just once or enough to measure theinternal construct.

Table 1. Comparison between behavioral and traditional assessment

Behavioral Traditional

Causes of behavior Interaction between individual and Enduring underlying states or traitsenvironment

Role of behavior Important as a sample of an Behavior is important only to the extent that it individual’s behavior in a specific represents the underlying trait.situation

Consistency of behavior Behavior thought to be specific Behavior assumed to be consistent across time and to a specific situation settings.

Use of data To describe and quantify behavior To describe personalityunder specific conditions. To select Functioning and etiology. To diagnose and classify. appropriate interventions. To make prognosis or prediction. To monitor effectiveness of interventions.

Level of inference Low. High. Comparisons made Both within and between individual. Between individual. Timing of assessment Ongoing. Before, during, and after Before and occasionally after intervention.

intervention.

Note. Adapted from Goldfried and Kent (1972) and Hartmann, Roper, and Bradford (1979).

Systematic Observation

995

BEST PRACTICES IN SYSTEMATIC DIRECTOBSERVATION

Methods of Systematic Direct Observation

NATURALISTIC OBSERVATION

Naturalistic observation refers to the recording ofbehavioral events in their natural settings at the timethey occur, using trained or impartial observers, wheredescriptions of behaviors require little if any inferencebeyond that which is observed and recorded (Jones,Reid, & Patterson, 1979). Similar to descriptive, nar-rative, or anecdotal observations, naturalistic obser-vation employs the recording of salient behaviors anddiscriminative stimuli as they are observed chrono-logically in time. For example, a school psychologistmight observe a student in the classroom and note thesequence of behaviors or activities that are hypothe-sized to be important or serve a reinforcing functionin maintaining patterns of behavior. If a student isobserved to refuse the request of a teacher, then thisdatum would be noted and recorded as a behavioralobservation or event.

According to Wilson and Reschly (1996), natural-istic observation is the most frequently used type ofdirect observation, used nearly twice as often as othermore systematic approaches to observation. The rea-son for this practice most likely lies in its ease in useand minimal training requirements. In practice, nat-uralistic observation usually takes the form or makesuse of one of two recording methods. The easiestform involves simply observing and noting behaviorsand events descriptively or anecdotally as they occurin the natural setting. Typically, such behaviors andevents would be recorded in a written fashion, listedchronologically as they appeared in real time. Inter-pretation is limited to a descriptive account of thetypes of behaviors and events observed and their tem-poral ordering in time. Because interpretation is lim-ited, such anecdotal and descriptive accounts cannotbe used for making high-stakes decisions. In fact, oneof the limitations posed by naturalistic observation isthe inclination to “over interpret” the data, or makeinferences regarding student behavioral patternsfrom a limited and unstandardized sample of behav-ior. Likewise, observers are prone to use confirmatorysearch strategies whereby increased attention isdirected toward those behaviors that confirm theobservers’ original hypotheses. In both situations,

bias in the selection and interpretation of the obser-vation is present.

A second and equally popular method of naturalis-tic observation involves the use of A-B-C (Antecedent-Behavior-Consequence) observation and recording.Similar to anecdotal or descriptive observational tech-niques, here the practitioner makes careful note ofthose environmental arrangements, behaviors, orevents occurring just before the behavior of concern isobserved (i.e., the antecedent) and what behaviors orevents are observed as a result of the behavior (i.e., theconsequence). By using the example of refusing ateacher request above, the antecedent would mostlikely be some type of teacher request or directive, thebehavior would be the act of refusing, and the conse-quence would most likely be a brief description of howthe teacher responded to the student refusal. Figure 1presents an example of the type of recording schedulethat is typically used during A-B-C observations. Thistype of recording schedule can be easily constructedby dividing a sheet of paper into three columns, eachof which corresponds to one of the three conditions(i.e., antecedents, behaviors, and consequences). Onceconstructed, the observer provides a brief narrative ofeach condition as they are observed in the natural set-ting. Although listed as A-B-C, the behavior columnis generally completed first and then followed by theantecedents and consequences. The reasoning behindthis rests on the understanding that without the pres-ence of some salient recordable behavior, there is lit-tle use in recording antecedents and consequences.While numerous behaviors will obviously occur, onlythose of clinical importance are noted. Given this, it isoften helpful to specify beforehand those behaviorsthat will be recorded during the observation period.

Naturalistic observation techniques have becomeincreasingly popular owing in part to their utility aspart of an overall functional assessment strategy.Here, descriptive or anecdotal and A-B-C analysesare used as a preliminary step in data collection andserve the purpose of developing testable hypothesesregarding the motivation and maintenance of studentbehavior. For example, O’Neill et al. (1997) use nat-uralistic observation strategies as the first step in anoverall functional assessment procedure that allowsthe clinician to operationally define target behaviorsand formulate preliminary hypotheses regarding thefunction of behavior. Once salient behaviors andenvironmental events are observed naturalistically,

they are observed systematically by using time-sam-pling procedures.

The advantages of using naturalistic observationprocedures to identify target behaviors in this mannerare that (a) their importance or social validity ofbehaviors can be assessed by noting the frequency oftheir occurrence in the natural setting, (b) their rela-tionship to important environmental antecedents andconsequences can be examined systematically, (c) datacan be used to develop testable hypotheses regardingthe function of behavior, and (d) the data gatheredserve an important step in allowing the clinician tomake decisions regarding the function of behaviorrather than focusing on topographical and descriptiveaccounts of what is observed. As was previouslynoted, one of the restrictions of natural descriptiveaccounts of behavior is their limited utility in decisionmaking. Because the data gathered are purely descrip-tive in nature, decisions are restricted to summarystatements of what was observed and little else. For themost part, such data prove limited in a more problem-solving assessment orientation. Nonetheless, natural-istic descriptive accounts can be extremely usefulwhen used as a preliminary step in a problem-solvingfunctional assessment paradigm. Here the data gath-

ered form the basis for developing initial hypothesesthat can be subsequently observed in a more system-atic fashion. The next two sections describe such sys-tematic observational methods and the way in whichpractitioners can use such methods to quantify behav-ior from an ecological perspective.

OBSERVATIONAL PROCEDURES

Observational procedures refer to a set of techniquesthat school psychologists can use to quantify behav-ior along one or multiple dimensions (Kratochwill,Alper, & Cancelli, 1980). For example, a school psy-chologist might be interested in assessing the fre-quency in which a referred student is out of his seat.After operationally defining the behavior of interest,the target child would be directly observed for a spec-ified length of time with the number of times he gotout of his seat noted. Additionally, the length of timespent out of his seat for each occurrence might benoted. If out-of-seat behavior does not appear to bethe main issue of concern, then another behavior canbe specified and observed in a similar manner. Like-wise, multiple behaviors can be identified andobserved concurrently. Although the example is over-simplified, here the advantages of using observational


996

Figure 1. Example of naturalistic observation using A-B-C observation and recording

A-B-C Observation and Recording Sheet

Antecedent Behavior Consequence

Teacher asks students to take out Target student does not take Teacher reprimands target paper and pencil. out paper and pencil. Plays with student.

toy car on desk instead.

Teacher takes paper and pencil Target student pushes paper Teacher removes request. out of target student’s desk. and pencil off desk and onto floor. Teacher picks up paper and

Target student puts head on desk pencil and places on desk. Tells with arms folded around head. target student he can begin

work when he calms down.

Teacher continues lesson with Target student begins to play Teacher stops lesson and takes rest of class. with car on desk again. car away from target student.

Teacher directs student to put Target student kicks desk away. Teacher ignores target student. name on top of paper. Sits in chair with arms folded

across chest.


997

procedures are that they are flexible and can be tai-lored to suit the specific needs of the assessment situ-ation.

Measuring and Recording Behavior Systematically.There are various types of data that can be collectedduring systematic direct observation. A workable def-inition of a target behavior is one that provides anaccurate description of the behavior that clearly definesthe parameters of its existence and nonexistence(Heward, 1987). As such, constructs and reificationsdo not lend themselves well to direct observation. Forexample, raising one’s hand to be called on is anobservable and measurable behavior. Behaving “offthe wall” is not something that can be directlyobserved. In developing explicit behavioral definitionsHawkins and Dobes (1977) offer the following sug-gestions:

1. The definition should be objective, referring onlyto observable characteristics of the behavior andenvironment.

2. A workable behavioral definition is readable andunambiguous such that an experienced observercould read it and readily paraphrase it accurately.

3. The definition should be complete, delineating theboundaries of what is to be included as an instanceof the behavior and what is to be considered a non-instance of the behavior.

As such, operational definitions must be objective,ensuring that specific instances of the defined targetbehavior can be readily observed and recorded. Inaddition, an operational definition is a technologicaldefinition that enables others to use and replicate it(Baer, Wolf, & Risely, 1968). A complete operationaldefinition identifies what is not the target behaviorand aids observers in discriminating the target behav-ior from similar responses.

Once behavior is defined, the calibration of theoperational definition is determined by the nature ofthe data; that is, the frequency of its occurrence andthe particular interests of the observer (Hintze &Shapiro, 1995). In addition, practical considerationssuch as the availability of observers, the amount oftime the student is accessible, or any combination ofthese factors, all dictate the type of data collected.

Because each of these data may yield different results,the method of data collection must be clearly under-stood.

Frequency or Event Recording: The type of dataknown as frequency or event recording involvescounting the number of occurrences of behaviorobserved during a specified time period. When thetime periods in which the behavior is counted vary,frequencies are converted to rates of behavior per unitof time. For example, an observer may report that atarget child raised a hand at an average rate of onetime per minute during three separate observationsconducted over the course of three days, even thoughthe actual duration of each observation period var-ied. By using rate of behavior allows the practitionerto compare the occurrence of behavior across obser-vational periods (Shapiro, 1987).

Frequency or event recording is most useful whenobserving behaviors that have a discrete beginningand ending. Throwing paper airplanes, hitting, andthe raising of a hand are all examples of such behav-iors. Behaviors that are continuous or persist forlonger durations sometimes prove difficult to observeby using event recording. For example, pencil tap-ping, talking, or on-task behavior would be difficultto observe by using such an observational system. Aswith all recording schedules, a very clear operationaldefinition of the target behavior helps ensure thataccurate frequency data are being collected. Ininstances such as pencil tapping and talking, episodesof the behavior may be defined by using a time dimen-sion as well as a topographical description of thebehavior. For example, pencil tapping may be definedas the continuous tapping of the pencil against a phys-ical surface that produces audible noise for at leastfive consecutive seconds. In this way, the frequencycount for pencil tapping is easily recorded rather thanthe actual number of times the pencil is tappedagainst a physical surface.

A second consideration when using event recordinghas to do with the actual length of time each episodeof the behavior occurs for. Generally speaking, eachepisode of the behavior should take approximately thesame amount of time for each instance of the behav-ior (Barton & Ascione, 1984). For example, if anobserver is observing occurrences of “noncompli-ance,” the length of time might vary widely from asimple refusal to follow a direction to a knock-down


998

drag-out tantrum that lasts 20 minutes. Because eachepisode would be coded as the occurrence of one actof noncompliance, each would be weighted equally interms of the way in which it was quantified. Obvi-ously, the variability in the duration of each episode islost. For this reason, when the length of time a behav-ior occurs is important, event recording might not bethe most appropriate recording schedule. It may, how-ever, be combined with other recording options (e.g.,duration) to capture both frequency and time dimen-sions of a particular behavior.

Another instance in which frequency or eventrecording is particularly useful is when behavioroccurs at a relatively low rate. Such behaviors oftenoccur infrequently but are of interest because of theirintensity or seriousness. For example, running out ofthe classroom may occur once or twice a day, but mayrepresent significant difficulties for the student. Theadvantage here is that with low frequency behaviors,observational periods can be continuous and designedin a fashion to be relatively unobtrusive and at lowtime and cost to the observer. The disadvantage, how-ever, is that if any instance of the behavior is notobserved and recorded, the reliability of the observeddata is sacrificed.

As can be seen, the methods for frequency or eventrecording are quite varied. Commonly, the frequencyin which behavior occurs is recorded in a written for-mat (e.g., tallies on a piece of paper) with the begin-ning and ending time of the observational sessionnoted. In addition to simple paper-and-pencil record-ing, hand-held mechanical recorders such as those

used to keep track of attendance at a social function,golf wrist counters, or a wrist abacus can be used. Inthe end, any device capable of keeping a cumulativefrequency count can be used to perform event record-ing. Figure 2 presents an example of a paper-and-pen-cil frequency recording schedule.

Duration Recording: Another type of behavioralresponse that can be recorded is the duration of thebehavior. Duration measures may be very helpfulwith certain types of school-related behaviors. Study-ing, temper tantrums, social isolation, and aggressiveoutbursts are good examples of behaviors in whichduration is generally important. Duration is alsoappropriate in cases where changing the duration ofthe behavior is an important target for intervention.As in the case of event recording, behaviors that havediscrete beginnings and endings may be assessed inthe length of time the behavior lasts.

The duration of a behavior is usually standardizedin two ways (Salvia & Ysseldyke, 2001). First, theaverage duration of each occurrence may be com-puted. For example, Gary got up and out of his seatfive times during a 20-minute observation. The dura-tion of each episode was 2, 3, 4, 5, and 6 minutes,respectively. In this case the average duration was 4minutes (i.e., [2 + 3 + 4 + 5 + 6]/5). Second, the totalduration may be computed. In this same example,Gary was out of his seat a total of 20 minutes.

The most precise nonautomated instrument for col-lecting duration data is a stopwatch. The procedure forrecording total duration with a stopwatch is to start

Figure 2. Example of frequency recording. Target behaviors include getting out of seat (OS), calling out (CAL),and teacher redirections (TR)

Frequency Observation and Recording SheetDate: May 13, 2000

Observer: A. VanDelay

Time OS CAL TR

9:00 a.m.–10:00 a.m. X X X XXXXX XXXXXXX

10:00 a.m.–11:00 a.m. X XXX XXXX

11:00 a.m.–12.00 p.m. XX X XXXXXX

the stopwatch as the behavior begins and stop the tim-ing at the end of the episode. Without resetting thestopwatch, the observer starts the stopwatch again atthe beginning of the second occurrence of the behav-ior and stops timing at the end of the second episode.The observer continues to accumulate the durations oftime in this fashion until the end of the observationperiod and then transfers the total duration of timeshowing on the stopwatch to a record sheet. Figure 3presents an example of duration recording for thumbsucking.

When observation sessions are consistent in length(e.g., 20 minutes), total duration can be comparedacross sessions. However, when observation sessionsvary in length, a percent ratio of total duration toobservation length must be computed before com-parisons across observational sessions can be made.For example, if the total duration of Gary’s out-of-seat was 10 minutes in each of three observationalsessions but the observational sessions varied from20-, to 30-, and 40-minute time frames, then the totalduration percent that he was out of his seat would benoted as 50, 33, and 25%, respectively.

The procedure for recording duration per occur-rence with a stopwatch is to start the stopwatch asthe behavior begins and stop the timing at the end ofthe episode. The observer transfers the duration of

time showing on the stopwatch to a data sheet andresets the watch. The stopwatch is started again at thebeginning of the second occurrence of the behaviorand is stopped at the end. The duration of time foreach episode is then summed and divided by the num-ber of occurrences yielding an average duration perepisode.

Latency Recording: Latency recording is the mea-surement of elapsed time between the onset of a stim-ulus or signal (e.g., a verbal directive) and theinitiation of a specified behavior (Cooper, 1987).Latency recording should be used when the majorconcern is the length of time between an opportunityto elicit a behavior (e.g., after the presentation of theverbal directive) and the actual time it takes to beginperforming the behavior. In this case, the responselatency would be the length of time between the endof the teacher’s directive and initiation of the student’scompliance with the directive. As with event andduration recording, when latency is assessed, both thesignal and the behavior of interest must have discretebeginnings. The procedure for latency recording issimilar to that for duration recording. The observeruses a stopwatch and begins timing immediately afterthe signal or stimulus is delivered and stops timing atthe instant the target behavior is initiated. For each


999

Figure 3. Example of duration recording for thumb sucking

Duration Observation and Recording Sheet

Student: Alex Observer: B. MatthewsBehavior: Thumb SuckingDate: August 5, 2000

Time start: 10:10 Time stop: 10:30

Thumb sucking Elapsed time per episode(separate incidents) (in minutes and seconds)

1 1' 17"2 6' 42"3 2' 11"4 7' 26"5 52"

Total: 18' 28"

Average duration per episode, 3' 42"

episode, the summary datum is the time lapsebetween the signal and the behavior. The actual timethat it takes to complete the target behavior does notfigure into latency recording. This is because behav-iors can vary widely in the time it takes to completethem. For example, the directive of “pick up yourpencil and put your name on the top of the paper”and “pick up your pencil and complete the New YorkTimes Sunday crossword puzzle” would each beexpected to have similar latencies. Obviously, theduration of these two behaviors would vary signifi-cantly. If completion times of these two behaviorswere of primary interest, then duration would be anappropriate recording schedule. Moreover, like dura-tion, both average and total latency can be used tosummarize observed behaviors.

Time-Sampling Interval Recording: Whereas fre-quency, duration, and latency recording are able toaccurately capture the dimensions of behavior eachrepresents, oftentimes it is difficult to use any one ofthe recording schedules because of practical or mea-surement concerns. Issues dealing with availability ofobservers, lack of time, or operational issues (e.g.,complications in determining exact beginning andending of behavior) all contribute to the difficulty inobserving behavior continuously. The essential char-acteristics of time sampling interval recording involveselecting a time period for observation, dividing theobservational period into a number of equal inter-vals, and recording whether or not a specified targetbehavior has occurred during each interval (Merrell,1999). For example, a 30-minute observation periodmight be broken down into 120 15-second intervals.Within each interval, the presence or absence of oneor multiple behaviors might be assessed. Presence orabsence of the behavior will be determined by one ofthree, or any combination, of three recording sched-ules discussed below (i.e., whole, partial, or momen-tary time sampling recording).

Also, unlike event, duration, and latency recordingthat provide exact frequency or time dimensions ofobserved behavior, time sampling interval recordingprovides only approximates for the behavior as itoccurs. That being so, for situations where the exactnumber of occurrences or time spent engaged in thebehavior, or latency of the behavior is of concern, timesampling interval recording might not be the bestoption. Nonetheless, time sampling interval recording

provides an excellent alternative when conditionswarrant observing a number of behaviors simultane-ously, or for behaviors that occur at a moderate tohigh rate or steady state.

Whole-Interval Recording: In whole interval record-ing, the target behavior is scored as having occurredonly when it is present throughout the entire interval(intervals are scored usually with a plus sign or othermark to indicate the presence of the behavior, andempty intervals generally denote the absence of thebehavior). Since the behavior must be present for theentire interval, whole-interval recording lends itselfquite well to behaviors that are continuous or inter-vals that are of a short duration (Shapiro & Skinner,1990). One of the drawbacks of whole-intervalrecording, however, is that has a tendency to under-estimate the presence of the behavior in real time.Consider for example, a whole-interval recordingschedule where each interval is 15 seconds long. If,for example, off-task behavior were our target behav-ior and it was observed for 13 of the 15 seconds dur-ing the interval, then the interval would not be scoredfor the presence of off-task behavior since it did notoccur for the entire 15-second interval. In essence, itwould appear as if the target student was on-task forthe entire 15 seconds. Because of this, whole-intervalrecording is well suited for behaviors targeted forincrease through intervention efforts (Sulzer-Azaroff& Mayer, 1991).

Partial-Interval Recording: In contrast to whole-inter-val recording, with partial-interval recording anoccurrence of the behavior is scored if it occurs dur-ing any part of the interval. Thus, if a behavior beginsbefore the interval begins and ends within the inter-val, then an occurrence is scored. Similarly, if thebehavior starts after the beginning of the interval, thenan occurrence is scored. Finally, if multiple occur-rences of the behavior are observed within the sameinterval, then the interval is simply scored as if thebehavior occurred once. Again, in comparison towhole-interval recording, partial-interval recording isa good choice for behaviors that occur at a relativelylow rate or behaviors of somewhat inconsistent dura-tion. Also, partial-interval recording tends to overes-timate the actual occurrence of the behavior. By usingthe example above, if the target student were observedto be off-task for only 2 seconds of the 15-second


1000

interval, then the interval would be scored for thepresence of the behavior as if it occurred for the entire15-second interval. Because of this, partial-intervalrecording is well suited for behaviors targeted fordecrease through intervention efforts (Sulzer-Azaroff& Mayer, 1991).

Momentary Time-Sampling: Finally, with momen-tary time-sampling, a behavior is scored as present orabsent only during the moment that a timed intervalbegins. With this technique, the observer notes eitherthe presence or absence of the behavior at a briefinstant during the interval. By using the above exam-ple, the target student would be considered off-taskif at the moment of observation (e.g., very beginningof a 15-second interval) he was observed to be off-task, irrespective of any behavior observed during therest of the interval. Salvia and Hughes (1990) havesummarized a number of studies investigating theaccuracy of these time-sampling procedures. As waspreviously noted, both whole-interval and partial-interval sampling procedures provide inaccurate esti-mates of the behavior in real time. Momentarytime-sampling, although based on the smallest sam-ple of behavior, provides the least biased estimate ofbehavior as it actually occurs (Suen & Ary, 1989).

OBSERVATIONAL INSTRUMENTS

In contrast to observational procedures, observa-tional instruments have been developed to assess aspecific range of behaviors. For example, a schoolpsychologist might choose to use an observationalinstrument designed specifically to quantify the per-centage of time a student is academically engaged oroff-task or the frequency with which a teacher pro-vides directives, provides opportunities to respond,or positively reinforces student efforts. Unlike moregeneric observational procedures, however, the flexi-bility of observational instruments is typically lim-ited. With standardized administration and scoringprocedures, practitioners cannot alter the operationaldefinitions to suit their individual needs. However,because they have been developed with a specific pur-pose in mind, observational instruments tend to pro-vide a more detailed account of a student’s behavioralpattern across a variety of behaviors of commoninterest to the observer. What is lost in flexibility isgained in breadth of behaviors observed. Whileschool psychologists could certainly develop their

own observational instruments individually tailoredto particular behavioral constructs, the time spent indevelopment may be cost and labor prohibitive.

Nonetheless, the use of observational instrumentscontinues to gain in popularity. The increased interestin such techniques is due in part to the optimizationof laptop computers and hand-held data recordingdevices. What follows are two examples of publishedsystematic observation codes. The Behavior Observa-tion of Students in Schools (BOSS) (Shapiro, 1996)can be a useful part of an academic assessment inclassroom settings. It may also be used in assessmentsof disruptive behavior, if aggressive behavior is not ofprincipal concern. The Attention Deficit Hyperactiv-ity Disorder School Observation Code (ADHDSOC)(Gadow, Sprafkin, & Nolan, 1996) was specificallydesigned to assess disruptive child behavior across anumber of school settings and includes various mea-sures of aggressive child behavior. These specific codeswere selected for review owing to their ease of use, andtheir differential purposes.

Behavior Observation of Students in Schools. TheBOSS is a useful observation code for assessing childacademic behavior in the classroom environment. Itis a relatively easy code to learn and use. In simpleterms, the BOSS assesses levels of “on-task” and “off-task” behavior. The amount of time children areengaged in academic tasks appears to be an impor-tant instructional variable (see Gettinger, 1986, for areview). Although several existing observation codesinclude behaviors that represent academic engage-ment, the BOSS is unique in that it divides engage-ment into two categories: (a) active engagement (e.g.,writing, raising hand, answering a question) and (b)passive engagement (e.g., looking at a worksheet, lis-tening to teacher directions). Furthermore, off-taskbehaviors are assorted into three categories: (a) off-task motor (e.g., out-of-seat, fidgeting, playing withpencil), (b) off-task verbal (e.g., calling out, talking toa peer when prohibited), and (c) off-task passive (e.g.,looking around, looking out the window). Finally, theBOSS also includes a measure of teacher directedinstruction (TDI), which provides an estimate of theamount of time the teacher is engaged in directinstruction. For example, the TDI category would bescored as present if the teacher were lecturing to theclass and absent if the teacher were sitting at the deskcorrecting papers.


1001

The BOSS is administered in 15-second intervalsfor a period of at least 15 minutes. The on-taskbehaviors (i.e., active and passive engagement) arescored at the beginning of each interval by usingmomentary time-sampling. For the remainder of eachinterval, the off-task behaviors (i.e., motor, verbal,and passive) are noted by using partial interval scor-ing. In addition, during every fifth interval, ratherthan observing the target student, the behavior of arandomly selected peer is observed and noted. Indoing so, comparisons can be made between the tar-get student and a peer composite that represents “typ-ical” behavior during the observational period. Oncethe observation is completed, scoring summaries arecomputed for on- and off-task behaviors of both thetarget student and the peer composite, as well as anoverall estimate of how much time the teacher wasengaged in direct instruction.

Attention Deficit Hyperactivity Disorder SchoolObservation Code. According to its authors, theADHDSOC was developed as both a screening mea-sure and as a tool for evaluating the effects of inter-ventions for children diagnosed with attention-deficit/hyperactivity and related disorders (e.g., oppo-sitional defiant disorder). The ADHDSOC can be usedacross a number of school settings (e.g., classroom,lunchroom, playground). For example, the followingseven behavior categories are scored in classroom sit-uations: (a) interference (e.g., target student calls outwhen it is inappropriate to do so), (b) motor move-ment (e.g., target student gets out of seat without per-mission, (c) noncompliance (e.g., target studentignores verbal direction from teacher or aide), (d) ver-bal aggression (e.g., target student curses), (e) sym-bolic aggression (e.g., target student takes anotherstudent’s pencil), (f) object aggression (e.g., target stu-dent kicks chair or desk), and (g) off-task (e.g., targetstudent stops working on assignment and stares outthe window). Though the various aggression scores(i.e., verbal, symbolic, object) may be coded individu-ally, the authors suggest collapsing them into a singlecategory termed “nonphysical aggression.” For obser-vations conducted in the lunchroom or on the play-ground the following categories are coded: (a)appropriate social behavior (e.g., target student isobserved talking to another child appropriately), (b)noncompliance, (c) nonphysical aggression (thisincludes both object and symbolic aggression), (d) ver-

bal aggression, and (e) physical aggression (e.g., tar-get student trips another child). Across all settings, tar-get behaviors are scored on a partial interval every 15seconds.

For diagnostic purposes, the authors recommendselecting three or four average peers to observe forcomparison. Selected peers and the target student arethen observed in alternating 1-minute intervals. Theobserver rotates through each peer until all have beenobserved, and then returns to the initial peer. Specificguidelines for collecting observation data as well asstatistical guidelines for comparing the scores of thetarget student to the peer composite are provided. Inaddition, guidelines for incorporating the ADHD-SOC into treatment evaluation procedures are alsopresented. Validation of the ADHDSOC is presentedin several studies of school-based medication evalua-tion studies (see Gadow, 1993; Gadow, Nolan, Poal-icelli, & Sprafkin, 1991).

Both the BOSS and the ADHDSOC are relativelysimple school-based observation codes to learn anduse. They offer the opportunity to assess both within-student features of behavior (e.g., across various set-tings that vary with respect to task demands), andbetween-student features of behavior (e.g., target stu-dent as compared to peer composite). These are onlytwo of the many coding systems available to schoolpsychologists. Other available instruments are listedin Table 2.

General Issues in Systematic Direct Observation

CONSIDERATIONS IN SELECTION OF

TARGET BEHAVIORS

Considering the Social Significance of the Behavior.In daily practice, it is not uncommon for teachers andother school personnel to describe a litany of targetbehaviors that present as possible candidates forchange. As most school psychologists are all too wellaware, to target each for observation and interven-tion is likely to prove time and cost inefficient in thelong run. As such, assessment information gatheredbefore systematically observing behavior (e.g., inter-views) is absolutely crucial to designing a soundobservation strategy. In doing so, the school psychol-ogist must determine which elements of a student’sbehavioral repertoire might serve as socially signifi-cant and ecologically valid target behaviors (Hintze& Shapiro, 1995).


1002

Procedures that affect behavior of one or more per-sons in ways that increase the probability or magni-tude of benefits for any one or all of the personsinvolved are considered to be socially valid targets forbehavior change (Hawkins, 1986). Teaching childrento read, pay attention, or make friends with peers arelikely to be socially valid, because these efforts gen-erally increase benefits and decrease costs for both thetargeted individual and the rest of society (Hawkins,1991). As such, a goal, outcome, or procedure is validonly to the extent that choosing it as a target forchange improves the benefit-to-cost ratio for the indi-vidual, for others, or both. A consumer’s or profes-sional’s opinion about a targeted behavior for changeis only valid to the extent that it is consistent withsuch improved benefit-to-cost ratio (Hawkins, 1991).

Although conceptually the notion of social validitymakes sense, in practice it may be difficult to opera-tionalize. At least one objective validation strategy isto use normative data from same age peers as the tar-get student to identify which behaviors are likely to beadaptive, and the extent to which such behaviors areexpected to be mastered. For example, observing therates of academic engaged time of peers of the targetchild can help establish whether or not a discrepancyis present, in addition to the magnitude of the dis-crepancy and goals for change (Hawkins, 1991). Sim-ilarly, observing those behaviors considered to be

associated with targeted behaviors (e.g., escape oravoidance) from a normative reference group mayassist in determining which behaviors are of criticalimportance for success (Hawkins, 1991). Finally, thebest validation of which behavior is most adaptive isto test experimentally the outcomes produced by dif-ferent behaviors and different levels of their perfor-mance (Hintze & Eckert, 2000). By definition, thosestrategies that yield the greatest benefit at the least costare the most adaptive (Hawkins, 1986).

Prioritizing Possible Target Behaviors. In manyassessment situations, decisions must be made regard-ing the relative priority of possible target behaviors(Hintze & Shapiro, 1995). In a review of the research,Nelson and Hayes (1979) offer four suggestions thatcan help guide the practitioner faced with a multitudeof potential target behaviors:

1. Alter the behavior that is most irritating to the per-son who has identified the problem (Tharp & Wet-zel, 1969).

2. Alter a behavior that may be relatively easy tochange (O’Leary, 1972).

3. Alter behaviors that will produce beneficialresponse generalization (Stokes & Baer, 1977).


1003

Table 2. Other examples of school-based observation protocols

Length of Recording intervals,

Code Published Age group Setting schedule seconds

Systematic Screening of Sopris West Behavior Disorders: Peer (pre-school Social Behavior Code adaptation (Walker & Severson, availablea) Elementary Playground • Partial interval 10 1990)The Preschool See Preschool Classroom • States 30 Observation Code references (momentary (Bramlett, 1993; Bramlett time-sample)& Barnett, 1993) • Events

(frequency of response)

State-Event Available from Elementary Classroom • States 15 Classroom author (momentary Observation System time-sample)(Saudargas, 1997) • Events

(frequency of response)

aSinclair, Del’Homme, and Gonzalez (1993).

4. When behaviors exist as part of a longer responsechain, alter the behaviors at the beginning of thechain (Angle, Hay, Hay, & Ellinwood, 1977).

In addition, Hawkins (1986) suggests that:

1. Targeted behaviors should be those that represent“keystone” or pivotal behaviors within a behav-ioral response hierarchy.

2. Behaviors that have a “general utility” should beconsidered prior to those with highly specific func-tions.

3. The construction or acquisition of behavioralresponse repertoires should take precedence overthe pure elimination of specific behaviors.

4. Behaviors that gain a student access to reinforce-ment in the natural environment should be givenhigh priority.

5. Student choice should be considered when select-ing possible target behaviors.

Antecedents and Consequences of Behavior. Both nat-uralistic observation and systematic direct observationstrategies allow the observer to examine interdepen-dencies among functional antecedents, behaviors, andconsequences (Alessi, 1988). Careful examination ofsuch functional response chains allow practitioners todevelop hypotheses such as:

1. Are there avoidance, escape, or termination behav-iors evident, contingent upon requests or demandsmade by the teacher?

2. Does the magnitude of the behavior change as afunction of varying task demands?

3. Does the target behavior lead to accessing socialattention or preferred tangibles or activities?

4. Are there particular setting or temporal character-istics associated with the target behavior?

Once developed, such hypotheses can be testedexperimentally in a more controlled experimentalanalysis of the behavior (Iwata, Dorsey, Slifer, Bau-

man, & Richman, 1982). By using a brief func-tional analysis (Wacker et al., 1990), hypothesizedbehavioral functions can be assessed using prede-termined analogue assessment strategies. Mostassessments include alone, escape, and attentionconditions. During the first phase of the analysis,reinforcement (e.g., social attention, withdrawal ofa demand) is provided contingent on the occurrenceof the target behavior. During the second phase ofthe brief functional analysis each condition is repli-cated; however, the reinforcement contingency (i.e.,escape, attention) is provided for appropriatebehavior rather than inappropriate behavior. Byusing such a methodology allows the practitioner tovalidate specific A-B-C chains and target specificenvironmental contingencies for ongoing assess-ment and intervention.

SUMMARY

Significant effort has been made over the pastdecade to shift the role of the school psychologistfrom one of problem identifier to one of problemsolver (Deno, 1995). With a focus on hypothesistesting and scientific accountability, the school psy-chologist as problem solver seeks to use assessmentinstruments that provide clear links between assess-ment and intervention. Systematic direct observa-tion provides one of the most useful strategies foraccomplishing this goal. With a focus on sociallysignificant and meaningful behavior change, sys-tematic direct observation changes the approach ofthe observer from passive to active and reflects tech-nological advances in both the study of humanbehavior and how behavior is recorded and sum-marized. Moreover, systematic direct observationprocedures are in line with the reauthorization ofIDEA and current standards for test use amongschool psychologists.

As with any new skill, it takes time to become flu-ent in the use of systematic observations. School psy-chologists should not be fooled by its apparentsimplicity and should expect to devote as much timein training to use such procedures as they typicallywould in learning any standardized assessmentinstrument. With continued use, however, school psy-chologists will find systematic direct observation tobe a crucial component of just about any of the ser-vices they offer.


1004

REFERENCES

Alessi, G. (1988). Direct observation methods for emo-tional/behavior problems. In E. S. Shapiro & T. R. Kra-tochwill (Eds.), Behavior assessment in schools:Conceptual foundations and practical applications (pp.14–75). New York: Guilford.

Angle, H. V., Hay, L. R., Hay, W. M., & Ellinwood, E. H.(1977). Computer assisted behavioral assessment. In J.D. Cone & R. P. Hawkins (Eds.), Behavioral assessment:New directions in clinical psychology (pp. 369–380).New York: Brunner/Mazel.

Baer, D. M., Wolf, M. M., & Risely, T. (1968). Currentdimensions of applied behavior analysis. Journal ofApplied Behavior Analysis, 1, 91–97.

Barton, E. J., & Ascione, F. R. (1984). Direct observation.In T. H. Ollendick & M. Herson (Eds.), Child behaviorassessment: Principles and procedures (pp. 166–194).New York: Pergamon.

Bramlett, R. K. (1993). The Preschool ObservationCode. Unpublished manuscript, University of CentralKansas.

Bramlett, R. K., & Barnett, D. W. (1993). The developmentof a direct observation code for use in preschool settings.School Psychology Review, 22, 49–62.

Cooper, J. O. (1987). Measuring and recording behavior.In J. O. Cooper, T. E. Heron, & W. L. Heward (Eds.),Applied behavior analysis (pp. 59–80). Columbus, OH:Merrill.

Deno, S. L. (1995). The school psychologist as problemsolver. In A. Thomas & J. Grimes (Eds.), Best practicesin school psychology III (pp. 471–484). Washington,DC: National Association of School Psychologists.

Gadow, K. D. (1993). A school-based medication evalua-tion program. In J. L. Matson (Ed.), Handbook of hyper-activity in children (pp. 186–219). Boston: Allyn &Bacon.

Gadow, K. D., Nolan, E. E., Poalicelli, L. M., & Sprafkin,J. (1991). A procedure for assessing the effects ofmethylphenidate on hyperactive children in public

school settings. Journal of Clinical Child Psychology, 20,268–276.

Gadow, K. D., Sprafkin, J., & Nolan, E. E. (1996). ADHDSchool Observation Code. Stony Brook, NY: CheckmatePlus.

Gettinger, M. (1986). Issues and trends in academicengaged time of students. Special Services in the Schools,2, 1–17.

Goldfried, M. R., & Kent, R. N. (1972). Traditional ver-sus behavioral personality assessment: A comparison ofmethodological and theoretical assumptions. Psycholog-ical Bulletin, 77, 409–420.

Hartmann, D. P., Roper, B. L., & Bradford, D. C. (1979).Some relationships between behavioral and traditionalassessment. Journal of Behavioral Assessment, 1, 3–21.

Hawkins, R. P. (1986). Selection of target behaviors. In R.O. Nelson & S. C. Hayes (Eds.), Conceptual foundationsof behavioral assessment (pp. 331–385). New York:Guilford.

Hawkins, R. P. (1991). Is social validity what we are inter-ested in? Argument for a functional approach. Journal ofApplied Behavior Analysis, 24, 205–213.

Hawkins, R. P., & Dobes, R. W. (1977). Behavioral defin-itions in applied behavior analysis: Explicit or implicit?In B. C. Etzel, J. M. LeBlanc, & D. M. Baer (Eds.), Newdevelopments in behavioral research: Theory, method,and application (pp. 167–188). Hillsdale, NJ: Erlbaum.

Heward, W. L. (1987). Selecting and defining target behav-ior. In J. O. Cooper, T. E. Heron, & W. L. Heward (Eds.),Applied behavior analysis (pp. 36–58). Columbus: Mer-rill.

Hintze, J. M., & Eckert, T. L. (2000). The use of functionalassessment and analysis strategies to reduce the non-compliant behavior of a child with autism. Proven Prac-tice: Prevention & Remediation Solutions for Schools, 3,9–15.

Hintze, J. M., & Shapiro, E. S. (1995). Systematic obser-vation of classroom behavior. In A. Thomas & J. Grimes(Eds.), Best practices in school psychology III (pp.


1005

651–660). Washington, DC: National Association ofSchool Psychologists.

Individuals with Disabilities Education Act: Amendmentsof 1997 (PL 105-17). 20 USC Chapter 33, Sections 1400et seq. (Statute).

Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., &Richman, G. S. (1982). Toward a functional analysis ofself-injury. Analysis and Intervention in DevelopmentalDisabilities, 2, 3–20.

Jones, R. R., Reid, J. B., & Patterson, G. R. (1979). Natu-ralistic observation in clinical assessment. In P.McReynolds (Ed.), Advances in psychological assess-ment, 3, 42–95. San Francisco: Jossey-Bass.

Kratochwill, T. R., Alper, S., & Cancelli, A. A. (1980).Nondiscriminatory assessment in psychology and edu-cation. In L. Mann & D. A. Sabatino (Eds.), Fourthreview of special education (pp. 229–286). New York:Grune & Stratton.

Merrell, K. W. (1999). Behavioral, social, and emotionalassessment of children and adolescents. Mahwah, NJ:Erlbaum.

National Association of School Psychologists. (2000). Pro-fessional conduct manual (4th ed.). Bethesda, MD:Author.

Nelson, R. O., & Hayes, S. C. (1979). Some current dimensionsof behavioral assessment. Behavioral Assessment, 1, 1–16.

O’Leary, K. D. (1972). The assessment of psychopathologyin children. In H. C. Quay & J. S. Werry (Eds.), Psy-chopathological disorders of childhood (pp. 234–272).New York: John Wiley.

O’Neill, R. E., Horner, R. H., Albin, R. W., Sprague, J. R.,Storey, K., & Newton, J. S. (1997). Functional assessmentand program development for problem behavior: A prac-tical handbook (2nd ed.). Pacific Grove, CA: Brooks/Cole.

Salvia, J., & Hughes, C. (1990). Curriculum-based assess-ment: Testing what is taught. New York: Macmillan.

Salvia, J., & Ysseldyke, J. E. (2001). Assessment (8th ed.).Princeton, NJ: Houghton Mifflin.

Saudargas, R. A. (1997). State-Event Classroom Observa-tion System (SECOS). Unpublished manuscript, Univer-sity of Tennessee-Knoxville.

Shapiro, E. S. (1987). Behavioral assessment in school psy-chology. Hillsdale, NJ: Erlbaum.

Shapiro, E. S. (1996). Academic skills problems workbook.New York: Guilford.

Shapiro, E. S., & Skinner, C. H. (1990). Best practices inobservation and ecological assessment. In A. Thomas &J. Grimes (Eds.), Best practices in school psychology II(pp. 507–518). Washington, DC: National Associationof School Psychologists.

Sinclair, E., Del’Homme, M., & Gonzalez, M. (1993). Sys-tematic screening for preschool behavioral disorders.Behavioral Disorders, 18, 177–188.

Stokes, T. F., & Baer, D. M. (1977). An implicit technologyof generalization. Journal of Applied Behavior Analysis,19, 349–367.

Suen, H. K., & Ary, D. (1989). Analyzing quantitativebehavioral observation data. Hillsdale, NJ: Erlbaum.

Sulzer-Azaroff, B., & Mayer, G. R. (1991). Behavior analy-sis for lasting change (2nd ed.). New York: HarcourtBrace.

Tharp, R. G., & Wetzel, R. J. (1969). Behavior modifica-tion in the natural environment. New York: Academic.

Wacker, D., Steege, M., Northup, J., Reimers, T., Berg, W.,& Sasso, G. (1990). Use of functional analysis and accept-ability measures to assess and treat severe behavior prob-lems: An outpatient model. In A. C. Repp & N. N. Singh(Eds.), Perspectives on the use of nonaversive and aver-sive interventions for persons with developmental dis-abilities (pp. 349–359). Pacific Grove, CA: Brooks/ Cole.

Walker, H. M., & Severson, H. H. (1990). Systematicscreening for behavior disorders: Observer training man-ual. Longmont, CO: Sopris West.

Wilson, M. S., & Reschly, D. J. (1996). Assessment inschool psychology training and practice. School Psy-chology Review, 25, 9–23.


1006

Documents

Best Practices in the Systematic Direct Observation of ...persingj/systematicobservation.pdf · tioners, Wilson and Reschly (1996) found that of the ... deemed eligible for special