DIRECT RECORDING IN EDUCATION

Eric HaughtonRe search TraineeBureau of Child ReasearchUniversity of KansasMedical Center

Introduction

Contemporayy science is characterized by the widespread use of

direct data-recording techniques. Gone are the hurnan observers of the

eighteenth and early ninteenth centuries. Ffuman observers have been re-

placed by e.tremely sophisticated equipment that provides a complete quant-

itative record of specific events. Continuously operative electric counters,

a wide range of speciallzed photographic films, and graphic recording de-

vices fill most scientific laboratories whether they study psysical events

or luman behavior. Education is the notable exception to this general con-

dition in research.

Direct data-recording and automatic instrumentation aqe taken for

granted in physiological, rnedical and most psychological laboratories.

Educatisn uses none of the available devices. Nor has education devel-

oped its own specialized recording devices. The consequences of the failure

to employ continuous and direct-recording procedures in education are

enormous and have seriously retarded needed developmerts in one of our

most important cultural and social endeavors.

It inay surprise no one that few refinements have been introduced in

regular grade-school testing programs in the last fortv or fifty years.

Certain rnechanical changes have occurred, for example separ4te and ma-

chine scorable answer sheets are now used widely. Yet the questions, their

Haughton DRE Z

answers, and the content issues of intermittent testing are still being

debated in School of Education and professional journals. One the other

hand, one would expect highly sophisticated measurernent techniques in

Special Education. For when people are suffering from medical problems,

we attack the problem with our most sophisticated recording devices. No

such specialized direct recording techniques have been developed by pro-

fessional Special Educators. In fact, they often use fewer of the stand-

ardized rneasurement procedures, relying completely on highly personal

and subjective evaluations. Thus, an area where one would expect to find

maximurn precision one finds gross imprecision. Not only is continuous,

direct recording not systernaticalym empl-oyed, but even the crude mea-

surement tools available are not in use !

The distinction between rneasurement and direct-recording requires

clarification. Of course, there is no absolute difference between the two

quantitative procedures. However, direct recording is always closer to,

or more directly related to, a particular event wh"ile measurement is less

precise and may involve indirect assessment.

Websterts New International Dictionary (L946) fifth defintion of the

noun measure is: I'actpr process of ascertaining the extent, dimensions,

quantity, degree, capacity, or the like, of a thing"rr As a verb, lTreasurers

second meaning is t'to estimate. " In other words, measurernent involves

Haughtor DRE 3

the cornparison or estimation of the properties of an object with reference

to a scale or standard. A tablers length and height may be measured in feet

and inches, while its weight is rneasured in pounds and ounces. On the

other hand, the sixth meaning of recgrd isttsomething rnade by perforating,

indenting, or otherwise transforrning an original blank. I' Clearly, dinosaur

footprints are the beastrs record, while palentological measurernents are in

centimeters and meters. A photograph of a dinosaur imprint that includes

a ruler or other standard:measurement unit is a better record than a de-

tailed tabd of measurements.

It is clear that the distinction between measurement arrd recording

involoves a continuurn, and this rnust not be forgotten. We are discussing

degrees of acuracy and precisio:, quantitative proximity to the actual event,

not exclusive properties of either measurement or recording.

The distinction between rneasurement and recording is clear in ed-

ucation. Measurement it involoved in such practices as IQ assessment,

achievement testing, percentage correct summaries, personality evalu-

ation, attitude appraisal, and various vocational and occupational interest

measures. These attempts to rneasure usually involve paper and pencil

tests in which the testee answers a number of questions, These questions

may involve such techniques as the semantic differential, sentence comp-

letion, or rnultiple choice. 'Whatever the form of the question, the marks

made by the testee are generally different from the topic or target behavior

Haughton DRE 4

the test is designed to appraise. ALso the test situation is very different

from the situation being assesed, unless one is studying test-taking be-

havior.

Intelligence scales are not conoidered tests of vocablulary, but

rather assessments of some intellectual capacity. The final step, after

cornpleting an intelligence test, is to add scores from disparate tests;

then, through a variety of transformations, the tester ends with an est-

imate of a characteristic of human behavior recently called rrintelligence"

or ttg". Plus or minus a standard error of measurement.

On the other hand, direct data-recording systerns produce data re-

,P-rye.aentative of a specific target response. Though these data are usually

limited in scope, imprecision in the record is also limited to defined,

quantifiable espects of the recording system. Typically, rnechanical and

automatic d.evices are employed because of their extrerne accuracy. An

additional property of automatic devices is that they can run continuously

for extended periods.

Direct recoruding has often been used in motor skills research.

Automatic timing and counting are standard procedures in pursuit rotorrfacking, endurance, and fine motor coordination experirnents. Contern-

poraty experiments on the acceleration and deceleration of a variety of Frurnan

responses have used hand switches, pedals, levers, and other electromech-

Haughton DRE 5

anical devices to produce electrical impulses. These electrical impulses

have been fed into automatic programing and recording equipment. Per-

rnanent graphical records are produced by event recorders and cumulative

recorders. Although graphic recorders are sensitive devices counters

usually provide an accu.racy check on the data. More recently, transistor-

ized equipemnt has been adopted for use in behavioral experimentsr pro-

cedures. In any case, the continu.ous record is rnore charateristric of the

behavior than an arithmetic or simple numerical summary.

While drarnatic changes have been taking place in many professions

which study and modify human behavior, the field of Education still employs

remarkably prirnitive and ig.qensitive forms of measurenrent. Some types

of measurement attempts, such as achievernent testing, IQ evaluation and

vocational assessment, served a purpose before the development of rnore

refined procedures. Ffowetrer, new d.irect and continuous and. remarkably

sensitive records of human performance are now available. These new

recording procedures should be irnplimented rapidly to improve the educa-

tional tailoring, planning, and implementation required for each studentrs

efficient academic progre s s.

The Issues

Three main topics will be presented in the following sections. Central

Haughton DRE 6

to these topics is ther problem of irnprecise rneasurernent in education.

Educational rneasurement has been distracted and impeded by: A) the failure

to include time as a basic of measurement. B) fixation on intermittent

testing programs and C)a steadfast emphasis on a dichotorrlolrs classif-

ication system. These three rnain thernes have subordifrate implications.

Although it is difficult to rnake a causal statement with such cornplex inter-

woven variables, I believe the weight'"-.id.r.e will indicate that the failure

to introduce continuous, direct recording has been an act of cornission

rather than one of omission, much less a function of ignorance. The curn-

ulative weight of the evidence leaves only one conclusion: Education has

failed to take advantage of major behavior recording innovations.

Certain administative procedures, quite divorced from the welfare

of pupils and individual academic planning, have played their part in retard-

ing innovation. I will not attempt to establish complicity between educa-

tional psychologists and adrnisitrators, but it will be clear that their distance

from the classroom and the teachersr concerns have contributed to contin-

uiling rather than alieviating, classroom problems. Yet these people should

be in the forefront of educational innovation. Nor have badly required records

that would immediately aid our educational planning and prediction been col-

lected. Immediate change is necessa,.rl t possible and demanded by the neces-

sity for increased efficie.-'ncy in education.

Haughton DRE 7

Measurement VS. Direct Recording in Education

If direct recording is so important in science and the understanding

of human behavior why has such data not been collected in education?

Perhaps the first question is: Has continuous direct recording been ernployed

in special or regular classrooms? A thorough search of over 50 text books in

educational psychology, special education, special applications and techniques

(see appendix) revealed neither use nor explanation of any type of direct re-

cording of childrents acadernic performance or classroom behavior. Many

studies used scaling techniques, various obsevation and coding procedures

or intermittent achievernent test as primary data. But none used direct

recording of pupil performance. In addition to the texts, I checked over

80 articles in various professional journals. No article reported the im-

plimentation of continuous direct, recording of classroom target behavior.

One study (Stainer, L9?91 recorded number of problems completed during

a standard time several times a day. These data were collected on 532

British grade-school children. Performance rate was not computed nor were

the data collected continuously. Group averages were derived frorn the data

and presented to reprosent the grouprs performance. Ffowever, this study

is the sole classroom study where direct recording was even partially im-

plemented.

A) 'Why no continuous recording?

Haughton DRE 8

There are rnany separate and interlocking aspects of this complicated

problem, but'I will discuss five major issues. The specific issues are:

1) early estimation atternpts

Zl rise of standardized achievernent tests

3) ernphasis on quality of perforrnance

4l speed vs power controversy

5) time as irnplict rather than explicit in performance

Each of these issues coexisted and interacted, but each made a distinct

contribution to delaying the i.ntroduction of continuous performance rate re-

cording.

Estimation vs precise prediction:

One of the first attempts to describe children objectively was the

Binet-Simon scale (1910). Binet was not a teacher and because he was ap-

pointed by the French Government did not have direct access to classrooms.

Therefore his stategy was to develop a rapid and objective estimate of

pupilrs intellectual characteristics to determine special or regular class

placernent. Binet and Simon recognized. the problem of bias in the subjec-

tive evaluation that then determined pupil placement. Binet (1910) *r" ol,,

well aware of the multiple factors involved in classroom performance and

was one of the first to suggest a comprehensive and objective t.esting pro-

grarr. of physical, visual, hearing, firernory, and personality character-

istics. However, his contribution lost a great deal in translation. Only

Haughton DRtr 9

one part of his overall strategy was imphasized and implirnented by educational

rneasurernent specialists. Emphasis on intellectural assessment by a behavior

sampling technique was an important historical antecedent because it deter-

mined the course of educational assessment and evaluation for the next 50

years.

Binet and Simonrs work led to the second issue, th,at is the emphasis

on standardized achievement tests. Educational rneasurement specialists,

following the original idea of standarlzed test, worked exclusively on ob-

jective tests as an alternative to teacherts grades. Considerable research

was done during the early 1900rs indicating the teachers grades were extrerne-

ly variable and subjective (Brooks L922, Starch and Elliot,Lgl?l. Therefore

a substitute, in the form of an objectively constructed and scored test was

conside red impo rtarfi and nece s sary.

'It is fair to say that achivement best development, construction and use

is a major phenornenon in Arnerican education. Various estirnates are made,

but there is general agreement that over 150 .million standardized achivement

tests are given each school year. To what end? Occasionally theOe test

results help the teacher assign grades. But rnainly tests are given for ad-

ministrative reasons" Smedsland (1964) sumarized certain educational

biases with the term rrtest happiness", I'Test happinessrr is reflected in

centrally coordinated testing. WhiLe the results of school-wide testing pro-

grams rnay be useful for centraLLized decisions they are proving less use-

Haughton DRE 10

ful to classroorn teachers and are antagonizing rnany parents.

Concern over methods of quantifying the quality of performance one

of the outgrowths of the ernphasis on standardized testing that has swarnped

education. The major concern has been with accurate topograph) rather than

with topography and rate. This is the third major educational issue. Per-

centage of correct answers has been chosen as the sole and representative

summary of a pupilts output. However a more complete summary would

be computed by the equation: Perforrnance-fiwre

sently we are told the exact percentage of correctLTrcr\ e OF

the pupilr s^ performance.A

The main problem with percentage is that is has an arbitrary max-

irnurn that is independent of pupil performance. In other words,l0O% is

not really an adequate description of a pupilts work but a lirniting condition

of our scoring procedures. It is as though we started a race, but failed to

record time. Everyone who completed the 100 yards would receive 100%.

In the testing situations, pupil might know much more than the test

tests, or much less. Clearly, two students are very differenct in their

efficency and competance if both get 100/0, but one takes 3 hours while

the other finishes the test in one hour. Oners perforrnance rate is three

times that of the other. A record of excellence is necessary but this record

performance rate. Pre-

answers, but nothing about

should not be at the expense of sensitivity to other important properties of

Haughton DRE 11

the pupilf s perforrnance.

The importance of timing has had a varied history in educational

testing. There are many discussions in the literature about the inclusion

or exclusion of time when testing especially under the topic of speed and

power tests. When discussing this issue Pressey, Robinson and Horrocksn,tni

(1959) state that one should be carefullnto "introduce a speed test at a

point where it is inappropriate....furtherrnore, there are sorne tasks

where speed is neither desirable nor importan.t, and in such cases there is

little point including speed as an aspect of achievement.'r But how can itr ;

ime.r1 Butbe that rate of work is unirnportant? Behavior occurs during t

differences between tirned and untirned tests confused experirnenters.

Since they didnrt like the confusion they chose ong of the possible alterna-

tives. They cl*qse to split tests into tirned and untimed assessments. They

could have decided to place all performance on a tirne base, thus eliminating

the necessity of intermittent test:*" {fr.y could have had a "orrirrroorr"^;"";;; '

1\

for each pupil. Instead, they chose to haggle over the distinctions between

speed and power tests-when in fact performance is made up of both num-

ber and tirne.

Inadequate recording has been responsible for many educational

misunderstandings and/rnisinterpretatioqs. For example, educationai psy-

cholgists have failed to recognize t}re effects of testing on performance.

The issue of interrnittent tirnlng has been conf used by sorne ernotional be-

Haughton DRE LZ

havior associated with testing. Timing has always been added to testst1a. \".' \..\ri

thereforeOthe issue of rnotivation frustration-Pressey, Robinson and

Horrocks (1959) conclude that timing on tests will produce rrunfortunate

motivation or frustration effects.'r So timing has been avoided because of

hypothesized negative effects.

B) Interrnitten'r; te sting:

Related to inadequate recording is the i.ssue of intermittent rather

than continuous assessment. Binet and Simon had to use a single exposure

to each child because of their position. This is certainly not the case with

full-time educat ors. Everyone has a feeling for the effect of tests. They

worry people. Just how much tests disrupted behavior was not recorded

because childrenrs daily performance was not recorded. Highly synthetic

experiments have been conducted to study the effects of rtanxietyttupon test

performance. But still the main issue was avoided.

'tiMhy dg(ve not have continuous recording in the classroom? Why is

there no data relavant to each teacher on each pupil concbrning the work each

pupil was assigned? Here, the acts of comission are c1ear. It was argued

by Pressey et aI (L959') that tirning pupils would introCuce undesired notiva-

tional problems. Flowever there are no data to indicate that continuously

recording time in pupil performance records introduces such variables.

These statements are highly biased and greatly oversimpified erCcrapolations.

One consequence of these inadequate extroplations was that formal

achievement testing remained intermittent. Although teachers themselves

Haughton DRE t3

take continuous records of performance, educational psychologists and achie-

vement testers kept their distance from current, dynamic and extremelyr..a

individual academic problerns by using, standardized achievernent test. They

compuulld thqproblern even further by resorting to large group summaries

whereas the teacher demands individually sensitive data.

Here again, we see that distance from actual classroorn needs and

individual pupil requirements delayed irnportant educational progress.

Educational planners continually refer to rreach pupil progressing at his own

rnaximal rate of progress'r but the data on which such planning could be based

is not collected. Intermittent achievernent testing does not provide adequ-

ate information because of the long and arbitrary times between tests.

This is an excellent example where the distinction between measure-Cl. .l a. , t. ..;.

ment and direct recording can be presentedO The arbitTary testing period,

each spring, deslgned to assess academic achievement is obviously on the

measurement end of the continuum compared to daily, performance rate re-

cording. A continuous daily record indicates exag!f1. when the pupil reaches

a certain level of competance or completes a specified amount of work. Pre-c!

cise data tells the teacher,rthe cornpetance of each pupil, thus allowing com-:

vidual data and therefore individual academic planning. Stand-

ardized achievement tests cannot discriminate between a child who reached

a specified 1eve1 of academic work early in the year and a pupil who reached

ptetJ/inai

the sa::te level just before the test, No wonder predictions based on achie-

Haughton DRE L4

rnent data are so inad,equate. Achiernent data are not sensitive to the rate

at which the pupi.l works since they are not continuous performance records.

Only performance records based on a combination of number of problems

completed divided by time taken to cornplete the problerns given an accr.lr-

ate record of a pupilrs academic performance

C) Classification vs functional analysis and description:

W'e have seen that failure to record perforrnance over time marke-

cily reduces the sensitivity of educational measurefirents. This problem

seems to be the outgrowth of another central educational theme - that is

the emphasis on classification. Binet and Simonrs work was concerned with

classifying children into those who could benifit from special class placement.

Essentially this was a reflecti-on of an Aristotelian philosophic infl'qence.

Much early and currect measurement emphasis is on a classification score

,or label. Education is rife with classificatry terms such as: educable and

trainable retarded; underachievers; overachievers; emotionally disturbed;

anxiety; aggression and intelligence. These terms ^''".r."rr1 things. Usu-

ally they produce a pigeon- holing of the child. A rrtr\ainablerr retarded. child

may be intitutionalized and therefore not exposed to important educational

experiences. An rroveractive and agqressive" child may be placed in a

special classroom where teaching proced*"es are not very different from

the regular classroom, but the child suffers the effects of social dislocation

and must go through re-entry when proJ/r"sis or therapsis is judged complete.'A

Haughton DRtr 15

I'indings over the past fifty years have indicated that each newly

identified pupil rrcharacteristic'r was unstable as measured in the face

o{ minor changes in environmental events. I8 scores change over tirne,

new environments produce changes in a variety of test scores, and a fore-

ign attack can reverse rating scale data. Test score changes are the rule

rather than the exception in education. Since behavior is dynamic, one

would have thought that expenmentalists would change their strategies--

from one forrnulated by Binet over fifty years ago--toward a rrrore func-

tional analysis of hurnan behavior. They did not. Aristotetion-like class-

ification systems have expanded and become more detailed (eg. the MMPI).

Now I arn supporling a change in ernphaEisr a shift frorn separtate class-

ificatio:r:rs to a more functionally oriented approach in the recording of

academic progress.

The Immediate Future: Perform ance Rate Recording

Emphasis on the rate of ernission of behavior as an important charac-'

teristic of human behavior is indicative of this shift in emphasis. InterestiLELy,

an emphasis on behavior recording was predicted by Adkins (1958) in an art-

icle on educational measurement. She considered that precise measurement

would be Skinnerrs contribution to education and not programed instruction or

extrapolations from his animal work. This predicti.on rnay well be accurate

as the possibility of direct recording seems to offe r great promise as a

quantitative innovation of significant proportions in education. However, the

Haughton DRtr L6

issue was not raised again. until Lindsleyrs article on direct rneasurernent

(Lindsley L964). In this article he laid out some of the strategi es and irn-

plications of a functional analysis of trehavior cornbined with contin'wous,

direct recording procedtlres.

Although direct recording of acadernic perforrnance has not been

ernployed widely yet, the procedure appears to have great promise in educa-

tion. Since rate statements require precise recording of the nurnber of

problem and the time in which they were cornpleted a new and corrlpre-

hensive recording procedure will be introduced to education. Direct re-

cording uts across and potentially eliminates a variety of rnajor educa-

tional rneasurelT)ent issues. Continuous and direct acadernic records

are obviously valid. For they are a direcil record of the pupilrs day to

day acadernic perforrnance" The data rnay be fairly variable (a major

problern in traditional tests of reliability) but the wariability will be aXg-

cord of the pupil and not an interaction between the pupil and the test. A

final point, though this does not cornplete an exhaustive analysis, is that

since there are no artificial limits inherent is such a record. Rate record:

provide an extrernely sensitive record of behavior. Percertage correct

suffers frorn artificial upper and lower lirnitE which make these data in-

sensitive. Also percentage values include a rrcorrectnessrr evaluation, which

increase the subjectivity of this procedure and make it less representative

of the pupilrs output.

Haughton DRE 17

Obviously nurnber or frequency must include time before precise

comparative statements can be rnade. Performance rate can be computed,

only when both number and time are recorded. Rate is an excellent summ-

ary statement. It is extrernely precise and cornpletely sensitive to indiv-

idual differences. While there has been rnuch lip-service to individual

differences in education the topic has received litle detailed attention. In

fact group analysis and group statistical data are comrnon to educational

research "upordwh ile in-dividual analysis is not.

I believe that individual performance rates will markedly amplify

our understanding and sensitivity to individual pupil differences. Educational

progress of each child is the basic product of our school. To date our

across pupil quality checks have been negligent. Too much subjective eva-

luation is the rule rather Lhan the exception. Current descriptive and

predictive staternents about pupils contain many sources of imprecision and

variance. Yariance data should be one of the quantified characteristics of

the pupil and should not be confounded by imprecion in our recording or Elea-

surernent procedures. Our duty is to make accurate descriptive and meaning'

ful1 predictive statements about children. This is our responsibilty and we

must revamp our evaluative procedures in the light of new recording d.eve-

loprnent. Or education will continue to be the only legally required and

significant influence on masses of chiLdren in our culture that remains

scientifically retarded. It is growing increasingly uncomfortable in the

Haughton DRE 18

20th centruy to use pre-scientific tools. We can expect a major revolution

in education when our professional topic,%upiI,s acadernic progress, is

clarrified by accurate inforrnation on each pupil supplied by continuous

performance rate recording.