DOCUMENT RESUME 95 TM 0O$ 257 NOTE 130p. - ERIC · 2014-01-14 · ED 102 2on. AUTHOR. TITLE. INSTITUTION. SPONS AGENCY. PUB DATE. NOTE AVAILABLE FROM. EDRS PRICE *DESCRIPTORS. DOCUMENT

ED 102 2on

AUTHORTITLEINSTITUTION

SPONS AGENCY

PUB DATENOTEAVAILABLE FROM

EDRS PRICE*DESCRIPTORS

DOCUMENT RESUME

95 TM 0O$ 257

Andrews, Theodore E., Ed.Assessment.New York State Education Dept., Albany. Div. of.Teacher Education and Certification.; State Univ. of.New York, Albany.National Center for Improvement of EducationalSystems (DHEW/OE), Washington, D. C.; Office ofEducation (DHEW), Washington, D.C. Teacher Corps.74130p.National Dissemination Center for PerformanceEducation, Syracuse University, Syracuse, New York13210

MF-$0.76 HC-$6.97 PLUS POSTAGECertification; Costs; *Educational Assessment;Educational Research; Inservice Teacher Education;Lesson Plans; *Measurement Techniques; Models;Objectives; *Performance Based Teacher Education;Performance Criteria; Preservice Education; ProgramEvaluation; State Departments of Education; StudentEvaluation; *Teacher Evaluation; Teaching Methods

ABSTPACT"The Role: of the State in Performance-Based Teacher

Education-Certification" by Robert Roth creates a context for viewinghow state agencies are approaching performance education. PeterAirasiam then explores the value questions that are at the heart ofevaluation issues. Fred McDonald looks at "The State of the Art inPerformance Assessment," and Barak Rosenshine lists recommendationsconcerning the research dilemmas. James Popham, in three papers,touches on the problems of selecting assessment systems, developingperformance 'tests, and identifying minimal competencies. Del Schalockthen details what occurs when "Moving from Conceptualization toPractice in Assessment." Concerning costs and teacher concerns, twoeducators, Bruce Joyce and Herbert Hite present their conclusions.Beatrice Ward discusses the cost factors involved in developmentalwork at the Far Vest Regional Laboratory. Finally two teachers,Sandra Feldman and Bernard McKenna, note their interest i'n the

/ potential of performance-based teacher education and reveal theirmost serious concerns. (RC)

U S 00 PA THE NT 01 HI At TH*DUCA ON & VVk I. F ARE14ASI0NAt hsTITutt of

I DU TIC 'dtt 5, .t , 11 t . tt t . t , utti VI . ,..% ti .41 t % %A ,.. t 1 t . t or .4. . .,

..4,Cf- . % .I pi . .1 wiof NI' *. , A kiA .1.1. ..' :t 1ti t % r

BEST COPY AVAILABLE tf\

rS

ASS SSMENTto

THE UNIVERSITY OF THE STATE OF NEW YORKTHE STATE EDUCATION DEPARTMENT

DIVISION OF TEACHER EDUCATION AND CERTIFICAN'and S

.

MULTI -STATE CONSORTIUM ON PERFORMANCE4AiEDTEACHER EDUCATION

ASSESSMENT

PETER AIRASIAN SANDRA FELDMAN JESSE H. GARRISON HERBERT HITEBRUCE R. JOYCE BERT Y. KERSH FREDERICK McDONALD BERNARD McKENNA

W. JAMES POPHAM BARAK ROSENSHINE ROBERT A. ROTHH. DEL SCHALOCK BEATRICE A. WARD

Theodore E. Andrews, Editor

MULTI-STATE CONSORTIUM ON PERFORMANCE-BASEDTEACHER EDUCATION

Peter W. AirasianBoston CollegeBoston ' 'ssachusetts

Sandra FeldmanUnited Federation of TeachersNew York C. New York

Je%se H. CaribouOregon College of EducationMonmouth, Oregon

Herbert HiteWestern Washington State CollegeBellingham, Washington

Bruce. R. JoyceTeachers CollegeColumbia UniversityNew York City. New York

Bert Y. KershOregon College of EducationMonmouth, Oregon

Frederick J. McponaldEducational Testing ServicePrinceton, New Jersey

CONTR I BUTORS

Bernard H. McKennaNational Education AssociationWashington, D.C.

W. James PophamUniversity of. California, Los Angeles

andThe Instructional 9,ectives Exchange

Robert A. RothCoordinator of Performance Education ProjectNew Jersey Department of Education

Barak RosenshineBureau of Educational ResearchUniversity of Illinois at Urbana

H. Del SchalockTeaching ResearchA Div'. *an of the Oregon State System of

Higher EducationMonmouth, Oregon

Beatrice A. Wax!Associate Director, Teacher EducationFar West Laboratory for Educational

Research and Development

ii

Foreword

The Multi-State Consortium is deeply indebted tothe authors who helped shape this publication. Theirwillingness to share their experiences and opinionsboth with the consortium and with the readers of thisvolume is greatly appreciated. The consortium believesthat the ideas contained herein, will not only interestreaders but also influence policy decisions..

The consortium is particularly indebted to theUnited States Office of Education for its support. Cre-ated by a Title V.srant, it has benefited both from ad-ditional fiscal resources and the direct involvement ofrepresentatives of the United States Office of Education.In partictrar, Teacher Corps (James Steffensen), Na-...

ttonal Center for the Improvement of Educational Sys-tems (Allen Schmieder), and Title V (Stuart Dean)

supportive: of its eitotui anu ftetptulin planning the meeting that led to the conference, thesource for most of the ,)apers appearing in this publi-cation.

The Conference Committee included James Steffen-sen and Paul Collins (both of Teacher Corps), BruceJoyce of Teachers College and the director of the con-: ..ot, -. The success of the conference and, it is

hoped, of this publication, is in large measure a reflec-tion of their efforts.

ti

i.

iii

MULTISTATE CONSORTIUM ON PERFORMANCE- BASED TEACHER EDUCATION

fl

. STATE REPRESENTATIVE

Washington Edwin L. LyleOregon James WallaceUtah Roger C. MouritsenArizona John Potts

= Texas Tom T. WalkerMinnesota Patricia J. GoralskiVermont Robert VailNew York Vincent C. GazzettaFlorida Nell KannwischerNew Jersey Ward Sinclair

Administering State: New York

Project DirectorTheodore E. Andrew's

New York State Education Department

Stuart E. DeanUnited States

Office of Education Representative

Syracuse University CoordinatorJames Collins

Fundedby

United States Office of EducationTitle V, Section 505, Grant

and

Syracuse University ProjectsSupported

byTeacher Corps

andNational Center For the Improvement of Educational Systems

C

iv

Introduction

Tbe Multi-State Consortium is pleased to present tointerested reatitt.mthe following papers.

We have titled this volume "Assessment" becauseof the importance we attach to the assessment prob-lems. However, not all of the papers focus solely onassessment. In fact, it is difficult, probably jto diuss assessment without quickly touching on suchareas as research, costs, teacher evaluation, and pro-gram development issues.

Many of the articles .were presented by the authorsto the consortium' at a series of meetings held in con-junction with the 1973 American Educational Re-search Association's annual meeting in New Orleans inFebruary. Several of these have been significantly re-vised, and we have included other chapters thatappeared' appropriate in the context of this publication.

The authors' audience is primarily state educationagency personnel and, in a larger sense, political deci-sion makers. However, the logic and appeal of theselections is not limited to that group; anyone inter-ested in the problems and potential of performance ed-ucation should be engaged by these learned and per-sonal opinions.,

The first selection, "The Role of the State inPerformance-Based Teacher Education-Certification'by Robert Roth, creates a context for viewing how var-ious state education. agencies are approaching perform-ance education. Peter Airasian then explores the valuequestions that are at the heart of evaluation issues.

Fred McDonald looks at the "State of the Art in Per-formance Assessment," and Barak Rosenshine makes aseries of recommendations concerning the research di-lemmas. James Popham in. three separate papers,touches on the problems of selecting assessment sys-tems, developing.. petformance tests, and identifyingminimal competencies. Del Schalock dim details whatoccurs when "Moving From Conceptualization toPractice in Assessment."

The assessment emphasis now shifts to related top-ics: costs and teacher concerns. Little definitive infor-mation exists on the costs of developing and imple-menting PBTE programs and assessment systems. Twoeducators (Bruce Joyce and Herb'n Hite) who haveprepared cost analyses for their respective states (NewYork and Washington) present their conclusions. Bea-trice Ward also discusses the cost factors involved inthe developmental work at the Fair West Regional Lab-oratory. Their collective conclusions are that programswill cost more, but that using what has already beenproduced will keep the costs within manageable limits.

Finally we offer two papers prepared by teacher rep-resentatives, Sandra Feldman and Bernard McKenna.Each notes their...Interest in the potential of pBTEwhile also revealing their most serious conccips. Mc-Kenna's paper concluded the AERA symposium andmost fittingly concludes this volume as he recapitulateswhat has come before and reacts to what he has heard.

be

0

The Role of the State inPerfoi mance-Based Teacher Education-Certification

By

ROBERT A. ROTH

Certification mid EducationThe Tovem:nt toward development of perform-

ance-based teveher education and certification programshas expe.riereed rapid growth within the past few)ears. Cumntly there are approximately 30 states thatare actively involved in the study of either perform-ance-based teacher education or certification.' Thepurpose of this paper is not to discuss the 'pros andcons of a performance-based program, but to describeand examine the issues facing a state that has electedto move in this direction. Germane to the discussion,.are the changing roles and relationships that must beconsidered in order to plan effectively fo implementa-tion. Any scheme tor devc.:ktement that has not consid-ered and accounted for the essential underlying issues'risks being nugatory in.nature;

Initially it may he of value to recognize the tradi-tional distinction between the prticess of teacher educa-tion and that of certification. Basically, teacher:educa-tion serves a preparatory function, whereas certificationselects those who are eligible for employMent andProvides them with a license: Certification traditionallyhas been a screening device, and it has been assumedthat the state is the best agency to carry out this func-tion. Discussion here will focus primarily on teachercertification as it has been a state responsibility.

It is interesting to note that there has been a greatdeal more resistance to tperformance-based certificationthan to performance-based teacher education. At a re-cent conference of the Regional Interstate Project heldin Denver, the consensus appeared to support this dis-tinction. Sandra Feldman, vice president of New YorkState Local AFT, stated "We do not oppose Perform-

1 Robert A. Roth. 'Performance-Based Teacher Certifica-tion: A Sur.ey of the States." .1 renton: New Jersey State De-partment of Et:million. Mision of Field Services, December1972.

ance-Based Teacher Education. The concept is a wel-come one . . . We oppose, however, a changeover toPerfollhance-Based Certification at this time." 2 DavidDarland, with NEA, added,

6 Most would agree to the importance ofthe performance' dimension of educatingteachers, but to establish one, prototype ofteacher education as the sole route to legallicensure is pure folly, especially in the ab-sence of established evidence through via-ble research. To base advanced creden-flailing or renewing certifications on sucha singular notion is eve; more upsetting.This is not to decry experimentation withperformance- base=d teacher education. Al-recdy some developmental approaches toperformance-based teacher education ap-pear promising, if not highly successful.3

The difference between certification and teacher edu-cation, however, varies significantly depending uponthe particular certification model. Since there are manyways in which performance-based certification can bestructured, criticisms shiould be centered around howthe issues pertain to a given structure or definition ofperformance-based certification.

If one views certification (particularly the perform-ance type) to be a testing procedure, then the distinc-tion is clear and the meaning of the skepticism is more

Sandra Feldman. "Performance -Based Certification: itTeacher Unionist's View," Periipt mance-Based Education andCertification, report of the Regional Interstate Project Pro-gram. Denver. Colorado. July 18-20, 1972, Denver: ColoradoState Department of Education, January 1973, p. 66.

'David Darland. "The Role of Professional Organizationsin Performance-Based Teacher Education," Performance-BasedF.ducation and Certificatior, report of the Regional InterstateProject Program, Denver, Colorado. July 18-20, 1972, Denver:Colorado State Department of Education, January 1973, pp.69, 70.

2

apparent. In this paradigm, the preparing institutions- tare responsible for developing competencies in their

:teacher candidates and the state certifies a Candidate'scompetence by testing him before issuing a license.Many difficulties associated such a licensing pro-cedure have been pointed out. One can argue, for CA-ample, that there exists no empirical base on which toconstruct a valid testing technique, particularly in viewof varied teachinklcontexts. The problem is not thesame with' performance-based teacher -aucation be-cause there is- a diversity of programs and flexibility toconstantly develop and change the performance stan-'dards. Performance-based certification, it is argued,mandates only one way of teaching, seems more of afinality, and is less respoasive to change.

In the approved program approach the distinctionbetween certification and teacher education becomesless clear. When utilizing an approved program ap-proach to Certification in conjunction with perform-

. ance-based criteria, as frequently the case, the dis-tinction may become even more nebulous. Supportingperformance-based teacher education but opposing per-formance certification then becomes a tenuous position.Interestingly enough, the approved program approachis the predominant system in use. In 1971 it was re-ported that "at the present time:36 states report exten-sive use of the approved program appfoach to certifi-cation, and in fact it has become the vehicle whereby

. forward-looking states have found the freedom to movein many promising new directions."

it seems that the approved program approach toperformance -based certification would be less suscepti-ble to criticism than th3 state examination approach. Inaddition, it is the more common certification systemcurrently in use and it provides a certain degree offreedom to explore new directions such as the perform-ance-based model. The project, Improving State Lead-ership in Education, issued a report which concludedthat "It would.appear that the effective administrationof a state-wide performance-based teacher certificationsystem would depend almost entirely upon an effectivesystem for program approval." in view of these fac-tors, the models for performance-based certification to

I Improving State Leadership in Education, "Planitiog anclEffecting Improvements in the Preparation and Certification ofEducators," report of a Special Study, Denver, Colorado,April 1971. p. 7.

Ibid., p.

a

follow will he within the context of. the appioved pro-gram approach.

Issues

In selecting a particular model, a number oftam issues need to he considered. An'essential'questionis what the role of the state should he in the certifica- ,

tion process. There are at .two opposingviewpoints concerning the state's function. On the onehand, there are those who see the state as an adminis-trative and regulatory body.

The belie, is that the state must improveits guardianship cf the public interest bysetting. ever higher standards and develop-ing more efficient systems of management.In one sense the state knows what isbest."

This view of the state's role , in certification is thepredominant one currently in practice. It is a central-ized approach with uniformity and standardizationbeing the emphasis. Even. an approved program ap-proach could fit into this scheme if regulations con-cerning program content are specified. A performance-based certification system structured .-on the abovetenets would specify teacher performance criteria foreertnicattim at the state revel,

The opposin' viewpoint on certification emphasizeda decentralized system with more local control and, abroader base for' decision leaking and social change. Inthis strategy, "the state must promote change ratherthan mandate it and accept diversity ,as more respon-sive to the state's needs than mandated singlestandards."

The competency approach could easily fit into thisphilosOphY also by allowing teacher education pro-grams or other professional agencies to develop theirown particular sets of competencies. In fact, as An-drews points out ,in some places the competencymovement "has been adopted as an attempt to reformthe educational system by changing the locus of au-thority and- thereby the way in which decisions aremade." One result of this is that a variety of stand-:

" Theodore E. Andrews, "Its Wisdom and Its Folly," paperpiesented to the Workshop on Problems of Competency-Basedteacher Education, Teacher . Corps. State University of NewYork at Albany, May 1972, p. 7.

Ibid.

" Ibid., p. 6.

ants appear. replacing the single set of state standards.The implementation of specific viewpoint of. a

state's role results in a number of rannfietaions inherent in the particular position. Theseconsequence.: are,in effect, the underl}ing issues whiCh impinge upon the

'decision to select a particular state role and thereforeshould be carefully considered.

In the centralized view of the state's role, a set ofperformance criteria would be established at the statelevel. These c. teria may be developed by a stateagency or thro h statewide improvement, the meritsof which will he dist:eked at a 'later point. This stand-ard set of statewide cribariecan be utilized in an ap-proved program approach or can be developed into astate testing instrument. Since the feitiver has been de-,termined ft: he possibly more advantageous, immediatediscuision will follow in this context,

The approved program approach itself has beenevaluzitvQy sonic educators as being restrictive. Lierheimer has pointed outfhat,

the' colleges approved Program must fol-low exactly the courses prescribed forstate certification. Such a curricular re-quirement does; not provide the freedomwhich colleges must have if they are alsoto be held responsible 'for the qualifica-tions of the teachers they prepare.

His remarks are made particularly pertinent to a,competency-based program by substituting "perform-ance -criteria" tor "courses" in his statement. Thus,tack of curricular freedom may result from a central-ized state role with statewide performance criteria.

Curricular freedom extends beyond the right to de-cide on a particular 'set of courses. The freedom .to ox-perinreht with innovative curricula also appears to beprecluded by a rigid set of state performance criteria.The right of colleges to experiment becomes ate impor-tant issue in the selection of a performance-based certi-

4fication model.The project. Improving State Leadership in Educa-

tion, reported that critics of certification structures isgeneral complain that "The rigidity of state require-ments discourages flexibility and creativity 'in teacherpreparation programs." "' Further, 'Ideally, the ap-proved program approgh woult)- allow institutions toexperiment and develop creative programs of teacher

. _

Alvin P. Lierheimor. "Give Up the Ship." paper presentedto the Massachusetts Advisory Council on Education. BostonUniversity, February 3, 1968. p. 4.

1" Improving State Leadership, p. 3.

11

3

preparation and encourage innovation in teacher edu-cation within the framework of generally agreed upongoals.- " An important part of this last statement isthe word "generally:: Generally agreed upon goalsmay still provide the freedom that Lierheimer is con-cerned about. .

It would seem that the centralized view of the state's'role with a stant.ard set of specific performance criteriawould he' contrary to the intent of the approved pro-gram approach. Yet, performance-bdsed certificationappears to depend "almost entirely ,upon pn effectivesystem for program approval." '2, An approved pro-gram approach without highly specific criteria is an al-ternative.

Curricular freedom, the right to experiment, flekibil-ity, innovation, and creativity in programs are issuesrelated to the state's role that directly affect the teacherpreparation institutions Other issues relate to the indi-vidual and the restrictions imposed by a specific set ofperfOrmance criteria exisling as state standards for cer-tification.

McDonald relates that "The specifics of teachingcompetence will differ markedly depending on ilow wedecide about the freedom each person will be given tochoose the goes and means for his personal develop.,inent and his life style?? 1" At one extreme theteach-er's services are sought requiring social skills, but atthe other enii he is an expert strategist requiring tech-nical skills. A specific set of state standards may onlypermit one of these philosophies to prevail, as optionstray' be impractical or even contradictory. Yet, oneMay argue than without state Control contradictorystandards cold

McDonald also raises a related issue. "Should wenot consider whether a teacher hasrthe freedom to de-fine the nature of his service to students? Does he havethe freedom to decide what will be. required ofhim?" " Decisions on these questionpclearly have im-plications for standardization of competencies. and therole of the:state.

An overriding concern with the performance criteriaapproach is that students will be boxed-in, forced toconform to a particular mold. It is argued by somethat ': ertification must provide for flexibility in person-

11 Ibid., p. 7.

12 ibid., p. 4.

1 Frederick J. McDonald, "The Philosophical Problems ofCompetency-Based Teacher Education," Teacher Corps, StateUniversity of New Yoreat Albany, May 12, 1972, p. 5.

11 Ibid., p. 7.

4

4

L.

a

6

4 1'

ality, method, and phil'os6phy. (open classrooms, tradi-tional, etc.). A specific set of standards at the statelevel does not provide for this flexibility. The decen-tralized state role Ooes, as it allows diversity in pro-grams and performance criteria.

The American Association of Colleges for TeacherEducation in .Evaluative Criteria. for AccreditingTeacher Education, A Source Book on Selected Issues,asserts that "there are and should continue to be sev-eral philosophies of teacher education." 15 Will a cen-tralked state role and specified performance criteriapki.lude varied philosophies of teacher education?Each state must examine its particular structure to de-termine whether or not this would occur.

Several other questions must be considered in rela-tion' to the development of a set of formance crite-ria at the. state level. Can luch aerie readily bechanged? Can a standard set of competencies be devel--oped to fit all teaching situations ,or must a number ofsets of criteria be designed? In relation to the affectivedomain, Elam believes

'File competencies that are easier to *-scribe and to evaluate are likely, to domi-nate . . . The skills of teaching and thebehaviors of a .teacher which are difficultto learn and to evaluate often focus on thehuman aspects of teacher-pupil eotitacts:m

t

Can these performance criteria be established in theaffective domain on a statewide basis,.or are they situa-tion specific and thus call for multiple standards devel-oped at local levels? Will decentralization make .theproblem any easier to solve?

The arguments suggesting a need for an empiricalbase for performance -based certification but notteacher education were presented earlier. These argu-ments pertain to a certification System with a uniformset of standards at the state level, the centralized 'v;ew

of the state's role.At a recent meeting of the American Federation of

Teachers. the following statement was issued in a re-port.

25 American Association of Colleges for Teacher Education,Evaluative Criteria for Accrediting Teacher Education: ASource Book on Selected Issues, Washington, D.C.: AACTE,1967, p. 26.

18 Stanley.. Elem. "Performance -Based Teitcher Education:What is the Stat.: of the Art?" Performance-Based TeacherEducation Series. 'No. 1.* Washington, D.C.: AACTE. 1971,p. 19.

If state agencies Llgin to require themastery of specific competencies as a pre-requisite for certification, two dangerswould exist. The would be thatpointed out earlier:..not..v2Idated knowl-edge and skill competencies as well as per-sonal cliara'ctetistics 'unrelated to trueteaching effectiveness may be required,leading to certification standards perhapseven more non-relevant than those nowexisting. Second; pressure groups may be

. able to legislate requirements that attemptto define teachers and teacher behaviors..into unacceptable Patterns. A. candidatecould be required to fit the mold or not becertified.17

Perhaps general 'guidelines or a variety of standardsdeveloped by local groups or institutions would be lesssusceptible to*these dangers. On the other hand, thesegroups may be just astiikelysto comn.it these errors.

In reference to 'establishing a.ininimutiliset of com-petencies at the state level, Andrews surmises that

Evsluating theccompetencies demands.aframe of reference, at its heart a set ofvalues: I worry about states establishing ,value systems, thus the frame of referencemust be diversified and most, likely local,

.ized . . Since we have a diverse populartion with.. varied philosophies,- I believe a .state should promote a,:ertification systemthat expects diversity aid challenges all tomeet the highest level of accomplish:ment."

Those who favor a uniform set.of quality standardsthroughout the state, however, would seek the morecentralized decision-making state role. 'Inequitiesamong programs would thus be eliminated and em-ployers would be 'assured that all certified pasonnelpossess at least a mini unrclet of com.letencies.

In analyzing the models in terms of the issues,' animportant question should always remain in sight. Inmost eases it will not be a matter of whether or not acondition exists, but t6.what extent it exists. For exam-ple, to state that curricular freedom does or does notexist is merely an opinion that does not 'focus on theissue. The real issue is whether or not there is suf-

ficient curricular freedom to satisfy those involved.Carrying the example to the other extreme, there may

I/ American Federation of Teachers, AFT-QuEST Consor-tium Yettrhook. Washington. D.C.: AFT, April 1972, p..30.

28 Artilrews, "Its Atisdom," p. 12.

be circumstances. that permit curricular freedoin: (orother conditions ) to exist to such an extent that it de-Stroys another essential or desirable element of a certi-fication structure. The models must he scrutinized todetermine if conditions arc sufficiently provided for,but not overindulged.

Models .F.

There are.mimy ways in which a perfolmance-basedteacher certification system can be designed withip theapproved program approach. At one end of a matin-uum we have.a very open system with maximum flex'-

. bility, whereas at tic. other end we hive a highly-struc-tured and centralized approach (figure ) There are,

"of course, many possibilities in between. Some of themodels have bzen alluded to in the discussion: ofissues.

, 4

Process Informational Facilitation

.5,

these standards. preparatio rograms are to be devel-oped and implemented b a consortium of agencies..Each agency designates its own representative(s) andolarifies with that (those) representative(k) his (their)authority in acting in behalf' ofs the agency:' The agen-cies in a :consortium are colleges and universities,.'school organizations, Ad professional associations.

The profesiional association, deterthined by the totalfaculty 6f certifiei employeeS in a school organizationin accordance with state law election procedures, hasthe responsibility of proiding opportunity .for inputfroni all other speCialized and subject matter associa-

,.ti9ns. The school organi4ation represents parents, Incalhoards, and achninisiration.

. The consortium is charged with describing roles tohe. assumed by the person to be grat4ed a. specific cer.

4Guidelines . Prescriptive

decentralized

A

STAyE,,ROLE

Es, I. ,

Continthim,,of Models,for

Performance:Based Tgacher CprtifieationApptoved Program Approach

The open-ended approach may be called the priaZeismodel, In thissystein, tbe state.does not determinefliecontent of the teacher education program. Perforthancecriteria are not established at the state level. The pri-mary. role of the gtatel is to define the process for de-velopment of teacher education programs, stating whois to be involve.f and the..nature of

moreinvolvement. In

this model, the state plays a more decentralized rolikwith more loca1J.:ontrOI and a broader base for deci-sion making.

.

Some states are now operating a competency-basedcertification syi'tem consistent with this model., Thestate of Washington is a priMary example was the firststate to adopt competency-based certification, and crowhas an . operational program. A new set of standardsfor approval of teacher preparation prdgrams became'effectiVe in Washington in September 1971.19 Under

8"'State of Washington. "Guidelines and Standards for the

Development and Approval of Programs of Preparation Lead-ing to the Certification of School Professional Persognel,"Olympia. WaShington: Superintendent of Public' instrukion,adopted Jury 9. 1971.

110

1111 .

centralized.

. .

tificate and with identifying and stating the rationaleA for the Icompetencies required of persolis who plan, to

perform the described' roles. The certificates will be is-sued'by thestate through an approved. consortium pro-gram: Thee standards are themselves:process and per-formance Sandaids.

in reference to this model. it would be of littlemeaning to support performance-based teacher educa-tion but not performance-based certification. One

' merely provides for the other; hence,Ithey become partof the same process. As noted earlier, The necessarytask is to examine the various certification models 'interms of the issues rather than compare certificationwith teacher education.

Clearly.' this. state has moved toward a decentralizedstructure with more local control, a broader base fordecision making, and diversity of standards. Perform-ance standards -arc more readily changed with feed-back. and probably less resistance 'would be encoun-tered in.the state. This model values optimum freedomfor the preparing institution in relation to curriculardecisions, flexibility, and creativity. In terms of the in-

a

r'

3

6

dividual. there is the possibility, depending on the pro-gram, :or freedom to define goals, for flexibility in per-sonality, method, and philosophy. Reflecting thisviewpoint, William Drummond, a former associate in

the Washington State Mpartment of Education, urgedthat "State departments of education, therefore, shouldfoster creativity and intellectual freedom and promoteprograms of teacher education which support and cher-ish uniqueness an individualism." 2"

Ttb:. Washingty n meg, therefore, also rejects theregulatory role of a state department of education.Wendell Allen, as Washington's assistalit superintend-ent of public instrwtion, concluded

To emphasize this regulatory role is toprotect the status quo. When the rule isthe thing, change must come before there

. can be a new rule. There is danger in thiscircumstance that the major energies ofthelagency will be spent on administrativerattier than leadership functions."

An essential point to note is the prevalence of multi-ple standardS, lack of uniformity, less legal need for anempirical base, and no single set of standards. Shouldall of the above factors be deemed advisable, then aparticular state might select this model.

New York has envisioned a very similar type ofprogram.2 Four process standards have been estab-lished to be utilized for the development of pilot proj-ects. The standards require the establishment-of a pol-icy board made up of represenrativel of teachers,school districts,. colleges, and teacher education stu-dents. This group considers the objectives of theschools involved, the competencies teachers need to besuccessful in that environment, as well as those quali-ties desirable for all teachers, and acceptable evidencefor attainment of competencies. The policy board thenwill establish individualized programs for the prepara-tion of teachers to meet these criteria. Finally, a man-agement system must be esUblished. Trial projects maybe designed for initial or continuing certification or

2" William H. Drummond, "Conference Commentary" inThe Seattle Conference: The Role of the State Department ofEducation in Teacher Education, OlyMpia, Washington: StateSuperintendent of Public Instruction, 1967, p. 74.

21 Wendell C. Allen, "State Government and Teacher Edu-cation A Different Role for the State Education Agency," in

The SeattOConference. p. 78.

2.2 New York State Education Department, "A New Style ofCertification." Albany, New York: NeW York State Depart-ment of Education, Division of Teacher Education, March 15,1971.

both. The State Department of Education will exerciseits legal responsibility for program approval. Note thedecentralized role and the belief that performance cri-

teria are mostly situation specific.Vermont has expanded the decision-making base

to local school districts. A local school district may de-velop a program kir the inservice training and .profes-sional advancement of its staff and may apply to theState Department of Education for approval to recom-mend issuance and renewal of all certificates at thelocal level. The appropriate certificate will be issued bythe State Department of Education.

The ideal district must submit evidence that theteachers, se\41ool board, and administrative personnelhave particip ted in the planning and development ofthe program. e local program must include provisionfor job descriptiort task analysis, and performance cri-

teria for all education personnel. An approved programapproach is in effect for college teacher preparationprograms.

Washington, New York, and Vermont areicase snit-ies that. fall into the process model. Local decisionmaking characterizes these attempts, assuming what is

acceptable in one situation may be unacceptable in an-other.

Moving slightly along the continuum away from the.4: .protess model but within the local decision-making

framework, there is a model suggested by Lierheimer 24which we may call the informational model. The central'thesis of Lierheimer's proposal is that the state's role

is not to make judgments but to maintain records. Hesuggests the students be tested over a multitude offactors including actual teaching performance. Thereis a possibility here for utilizatign of performancecriteria, but the testing is not don by the state.

Decentralization is emphasized in*s.atfproach withlocal school teams conducting the evaluation of thecompetence of potential teachers. Ultimately, theagency to decide on teacher performance for licensingpurposes would be the school. The function of thestate is to monitor the local evaluation but not imposestate standards. Although evaluation systems would beapproved by the state there would be no uniform tech-niques for verification of classroom performance. Thestate office would maintain a data bank on all teachingpersonnel in the state.

:!' Vermont State Department of Educition, "RegulationsGoverning the Certification of Educational Personnel," Mont-peli:r: Vermont State Hoard of Education. Department of Ed-ucation. July 1. 1971.

=' Lierheimer, "Give Up The Ship."

A unique feature of this model is that the state ac-cumulates information on an individual but makes nodecision in refereace to competence. I he major role ofthe state is to provide resources. The local district isprovided with the information, and it is at this levelthat decisions are made as to whether the individual'scompetence fits the particular situation. The underlyingassumption is that values and competencies are situa-tion specific and hence require local evaluation. Cur-rently, there are no states utilizing this informationalmodel. Again, analysis of the model should he made interms of all the issues identified earlier.

This model can he modified to interject more statecontrol and greater uniformity. Minimum standardscould be set by the state for the various competenciesor groups of competencies. These minimum standardswould he .established for certification purposes. Thestate would still maintain its individual data bank andlocal districts could use the information for hiring pur-poses. This modified model would he farther along thecontinuum in terms of state control and decision mak-ing:.

Another open-type model which does not providequite as broad a decision-making ba9 is being devel-oped by the state of Florida. In this case consortia arenot designed for purposes of initial certification al-

* though inservice nrograms are developed by local dis-tricts. This "facilitation model" utilizes the college ap-proved prOgram approach commonly in practice amongthe states.

The program approval regulations are somewhatprocess it nature indicating prescribed activities, butthey are content standards as well, identifying coursesnecessary for certification. There are alternatives to thecontent regulations which provide for performance-based programs.

An institution may. instead, specify thecompetencies which its graduates will beexpected to demonstrate, identify the pit-cedures by which those competencies willbe measured. and then develop a programwhich leads to those competencies. Oncesuch a program is approved, its graduateswill receive regular teaching certificateswith no penalties. Institutions are nowbeing encouraged to develop competency-based programs.-.

In this model control is in the colleges, but direction

Florida Department of FJocation, "The Florida Programfor Improving the Training. Evaluation. and Lieensur.: of Ed-ucational Personnel." draft number 2. Tallahassee: Educa-tional Reseatch and Development Section. April 7, 1971.

7

is provided' by the state. The colleges de% clop theirown competencies, but these are consistent with statecourse requirements. There is additional direction andstimulus provided by the state, however, which facili-tates development of such programs. The state is com-piling a catalog of teaching competencies which willeventuall be validated through research. These compe-tencies, or performance criteria, will he provided to thecolleges to facilitate their program development. Theseparticular criteria, however, will not necessarily bemandated and certain!. all will not be required of agiven institution. Other facilitating procedures by theState are assembling of training materials based onperformance criteria and staff development for teachertrainers. The emphasis is on facilitation. Decision mak-ing is soinewhat diffused but the role of the state isstronger than in previous models. The facilitationmodel presents different responses to the issues.

The remaining two models 'to he discussed can begrouped under a heading of central decision making.The first two models, you may recall, were local deci-sion-making types, with the facilitation model beingsomewhere between. These last two nxtelss.are at theother extreme end of the continuum.

One approach to performance-based certification isto establish performance criteria at the state level. Thisapproach supports a strong state role and a uniformset of standards. It guarantees that each certificated in-dividual has at least a minimum set of competencies.These criteria could he utilized as a state test or partof an approved program. The focus here, however, ison the approved program approach.

The manner in which these criteria are stated signifi-cantly affects the impact they will have on teacher edu-cation programs and the role of the state. The per-formance .criteria can he stated in generic terms whichthen serve as guidelines for further specification byteacher preparation institutions. This guidelines modelincreases centralized authority yet does provide a cer-.tain degree of participation on the part of the collegesor consortia.

As an example of competencies consistent with theguidelines model, it might he required that the teachercandidatedemonstrate the ability to diagnose areas ofstudent deficiency. maintain a classroom environmentwhich motivates students to learn, plan an instructionalunit, employ a variety of instructional techniques, etc.

Utah ;': recently adopted at the state level a set of

Utah State Board of Education," Recommended Profi-ciency cittidelines ft r Media 1:ndot semen's." Salt Lake City:Utah State Board of Education. 1972.

8

BEST COPY AVAILABLE

Robert A. Rotb

performance criteria for instructional media, some ofwhich approximate the guidelines model type of crite-ria. Prerequisite to a Basic Media Endorsement are abachelor's degree and a teaching certificate. An exami-nation foi proficiency conducted by a recommendinginstitution (with an approved certification program) isthen administered. The recommending institution is

free to determine how the competency will be demon-strated or ascertained, but a candidate may request anopportunity to demonstrate a competency whenever hefeels he is ready. Competencies may be demonstratedone at a time. Candidates who perform satisfactorilywill he considered as having met the endorsement re-quirement regardless of the route taken to obtain the

competency.Proficiency must be demonstrated in five areas.

Some examples of performance standards are as fol-

lows:

--Using media selection tools of his choice, the can-didate will identify the tools he has selected andinclude a rationale for the choke of each.

The candidate will explain what one would do toselect new subject headings for materials which

are n.. considered in "Sears List of Subject Head-ings."

The candidate will demonstrate proficiency in

mounting pictures by producing one acceptableexample of the following:

(I) dry mount on a hard surface, using. drymounting tissue

(2) dry mount, using dry mounting cloth

( 3) rubber cement mount IC A

(4) laminate with thermo copy machine, adhe-sive acetate, or heat press.

At the extreme end of the continuum we have what

can he termed the "prescriptive model." In this systemthe state provides very specific performance criteria(behavioral objectives) which are utilized by the col-leges as objectives and evaluative criteria. This is themost dominant of the state roles within an approvedprogram approach with an emphasis on the administra-tive and regulatory function of a state educationagency. Uniformity in certification with a single set ofstandards is the essential feature.

The state of New Jersey is currently studying thefeasibility of such a performance-based certificationsystem. Specific performance criteria are being devel-oped for use on a statewide basis as certification stand-ards. How specific these will be has not'as yet been de-termined. There are two unique aspects to the NewJersey approach, however, that broaden the base of de-cision making. The performance criteria are being de-veloped by task forces composed of a crosssection ofeducators from across the state, representing teachers,administrators, college students, college professors, theState Department, and the various professional associa-tions. These criteria, therefore, are not developed bythe State Department but represent a consensus of pro-fessional educators in the state. In addition, evaluationof prospective teachers may involve schools, colleges,and professional associations, a resemblance to theconsortium idea.

Clearly, the statewide involvement in development ofcriteria adds considerable power to the approach. Itpresents a decided advantage over development of cri-teria by a state department or even a college or univer-

sity. it appears to have greater validity and is morelikely to find statewide acceptance. Significantly, "it hasbeen generally agreed that whoever determines certifi-

cation requirements controls the program ofpreparation." 1' Thus, in this instance, control is morein the hands of the total profession.

The guidelines model can be developed by the samemethod. The difference between the two models thenlies in the specificity of the criteria. How does this dif-ference relate to the issues, and how do these twomodels compare with the open end of the continuumin terms of the issues?

27 Improving State Leadership, p. 5.

In the process model, teacher preparation institu-tions have maximum curricular freedom. The guide-lines model allows the institutions the opportunity todevelop the specific performance criteria while the pre-scriptive model does not provide for this. A compari-son of performance criteria with traditional course liststandards may be of valu.: at this point. A course intests and measurement is a familiar requirement i.t thecourse list system. The guidelines model would requirecompetencies that are somewhat more specific, such asabiliy to evaluate student performance and ability todevelop tests. The prescriptive model, however, wouldlist a number of specific performances such as abilityto formulate essay (multiple choke, etc.) test itemsand analyze tests for validity. Also, the evidence ac-cepted that the performance had been achieved wouldhe provided. Continuing our comparison, if used in acourse list system, a prescriptive model would list thethings that should be taught in a tests and measure-ment course rather than leaving this to the college.

Andrews has stated that "a required set of perform-ance criteria could be just as moribund as rigid courserequirements have been in the past." 28 It appears thatthe more specific the criteria the less freedom that ex-ists. Recall that the approved program approach works"within the framework of generally agreed upongoals." 2" The possibilities for creativity through inno-vative programs can be achieved in the design ofmeans to achieve the objectives, but not through alter-native objectives. Two basic questions are at hand.First, is curricular freedom seen as being of value; andsecond, does a prescribed set of sped& performancecriteria significantly limit this freedom? A relatedquestion is whether or not the guidelines model offers agreat deal more freedom than the prescriptive model.

A concern si:iar to the question of freedom is di-versity. The process model allows, and even encour-ages, diversity among programs. Those in favor of di-versity argue that there are varied philosophies ofeducation requiring different teaching models. Any setof performane: criteria is based on a theory of teach-ing and the teaching-learning process. Although not al-ways articulated the purposes of :caching are inherentin the criteria.

In the process model several teaching philosophiesexist simultaneously with validation and developmentbeing ongoing processes. A set of specific criteria, how-

Theodore Andrews.. New Directions in Certification.Washington. D.C.: Association of Teacher Educators. 1971, p.10.

;:" Imp.uving State Leadership, p. 7.

9

ever, reli;s on one teaching model and also establishesa particular value system. The problem here is thatthere is no empirical. base to lead us to the correctmodel. As noted earlier, lack of an empirical base is aprimary concern with performance-based certification.With a variety of program types, it can be argued, werecognize the developmeatal state of our knowledgebase, whereas a single model seems a finality and de-mands emphical validation before being adopted. Thisaccounts for the suppor: of performance-based teachereducation instead of certification.

Another point made by those favoring diversity isthat performance criteria are situation specific. Thereare numerous contexts for teaching, both in terms ofenvironment and educational philosophy. This requiresdifferent sets of competencies, at least in terms of thegeneral situations (not for every school, etc.). Theremay not be enough in common to establish at least aminimum core of competencies at the state level.Washington, Vermont, and New York appear to be-lieve in this as evidenced by their process models.

All of the above factors suggest multiple standardsand diversity of programs. The initial question iswhether these are valid concerns. The other positionargues for more standardization and quality assurance.Inequitis among programs are diminished. Certainly,the prescriptive model adheres to the latter viewpoint.The guidelines model does provide a certain degree ofvariability in that each institution can define the spe-cific criteria to fit its needs. The prescriptive model in-sists on a single standard; the guidelines model offerssome degree of multiple standards although minutewhen compared to the process model.

A frequent criticism of competency-based programsis the problem of writing performance criteria in theaffective domain. This problem becomes amplified aswe move across the continuum toward the prescriptivemodel. As an example, the guidelines model might re-quire competence in developing teacher-student rap-port. Each ieacher preparation institution would beprovided the freedom to determine not only how thismight he developed but how it might be judged toexist. The prescriptive model, however, would specifythe performance criteria necessary to achieve this,such as "uses student names," or "smiles or acknowl-edges student responses by nodding." The question iswhether or not such criteria can be written on a state-wide level. Ignoring the affective domain and con-centrating on the cognitve and psychomotor would nothe a viable alternative.

The reader may recall the issues raised concerning

the rights of the inOividual as suggested by Mc-Donald. Are there opportunities for flexibility in per-sonal ty. method, and philosophy? What about thelight of the individual to define his own goals? Rack ley

and Miller, as members of the Pennsylvania State De-

partment of Education, stated that

Individual differences arc not taken Intoaccount in blanket certification standards.We are convinced that the improvement ofteacher preparation must take place at thepoint of initial preparation . . . with atten-tion directed to individual needs within thecontext of general certification require-ments.''

The proLess model provides for individual flexibility.and, there arc functioning programs which operate onthese premises. The prescriptive model precludes muchof this, at least in terms of the specific criteria requiredby the state. The individual does not have the freedomto define his own goals, but he may have the opportu-nity to select his own method of achieving the objec-tives. Again, those favoring a uniform set of standardswould find individual selection of goals to he undesira-ble and detrimental to certification.

The guidelines model may provide a certain degreeof individual choice but within the boundaries definedat the state level. The general objective must be ac-complished. but the specifics can vary with the individ-ual. The manner in which one wishes to develop teach-er-student rapport. or plan for a lesson can varysignificantly from another individual's method. Thebasic question is not just one of uniform standards ver-sus flexibility but the degree cf each that is desirable.

Alternatives to Approved ProgramsThe discussion of issues and alternatives has thus far

been limited to the approved program approach tostate certification. Approved programs referred to thosedeveloped by colleges alone or by consortia. The evi-

dence presented earlier ;n this paper suggest that ap-proving programs is the more viable approach to per-formance-based teacher certification, and some specificcriticisms of the state testing approach were described.

There are some teaching areas, however, that findthemselves less rigidly tied to college preparation pro-grams and thus arc more amenable to alternative ap-proaches. The area of vocational education, for exam-

McDonald, "Philosophical Problems."

-" .1. R. Rackky and Norman Miller, "Broad Policy Con-cerns and Direoion for a State Demo-went of 1:dilation inTeacher Education," in The Seattle. Conference, p. 15.

pie, is somewhat unique in that it usually relies onexperienced professionals in the various trades to enterthe teaching profession. There are other areas, such asmusic, that also require specific skills unique to theparticular profession. Educational fields such as these

warrant consideration of alternative approaches thatare not necessarily bound by college degree programs.These different approaches are not necessarily limitedin application to the special teaching areas mentioned,however, as the alternatives may be utilized for anyteaching field if desired.

A commonly discussed filternative to reacher certifi-

cation is the establishment of a state testing procedure.Time are several ways in which this can be imple-mented, some of which will he described here. A co-gent argument against this approach (which waspointed out earlier) is that there exists no empiricalbase on which to construct a valid testing technique,particularly in view of varied teaching contexts. Thepredictive validity of any such examination devicewould have to be established.

It is again important to consider how the state test-ing models reflect the various issues, Questions aboutcurricular freedom, ,,individual freedom, and variedteaching philosophies should not be forgotten. Thestate testing approdch to certification offers radicallydifferent responses to the issues when compared to themodels within the approved program approach.

The informational model suggested by Lierheimercan easily be modified to fit a state testing procedure.A set of behaviorally stated competencies could beformulated as certific..tiJn descriptors. A teacher candi-date's degree of accomplishment of each of the criteriamulti be indicated to Corm his competence profile.Minimum :standards established for certification couldbe set by the state for each criterion or group of crite-ria. A system could he established (total score,weighted scores, etc.) to determine the individual's eli-gibility for certification. The state would still maintainits individual data bank and local districts could usethe information for hiring purposes.

An important modificatioh of the Lierheimer infor-mational model is that not only are minimum levels es-tablished for certification, but the testing of the candi-date to determine his ackevement of each criterion isdone by the state, not Mough an approved programapproach. The control of standards and verification ofaccomplishment reside in the hands of the state.

The modified informational model is but one varia-tion of the state testing concept. Any outside agency orgroup of evaluators could be designated by the state to

carry out the testing function. There is an opportunityto imoke members of the profession in both dc elop-

II lent of ciitcria and scr% icc on e% aluating boards orteams who certify individuals. Instead of a profile. veri-fication of ininimum competence might be all that is

necessary. Differentiated certification could he basedon different degrees of accomplishment or even differ-ent types of criteria. Evaluating boards or teams couldagain be used throughout the entire process.

It is generally assumed that the evaluation for certi-fication would be done in a live classroom situation.

alternative would be to establish testing centerswhere specific skills would be evaluated such as thosefound in microteaching. This might he particularly use-ful for initial certification due to the inequities in stu-dent teaching situations. Students could also he used intest centers similar to the laboratory schools. Thiswould provide a more controlled situation and fewervariables would- enter into the evaluation.

A combination of evaluation in student teaching set-tings and controlled laboratory situations is also an al-ternative. This might be built into a system where arecommendation from a preparing institution (collegeor consortium) in addition to testing in a center wouldhe necessary parts of the process for certification. Thevariations to this testing approach are too numerous tobe included in this discussion.

11

EpaogueEach model must be considered carefully in terms of

the issues identified. Certainly, there are other issues tobe accounted for which were not discussed here. Theidea of certification levels was not presented in this

paper and could by itself be an emir: area of discus-sion with direct bearing on the selection of models.Another important question is whether or not to usestudent outcomes as an indication of teaching compet-ence. Concerns of a practical nature such as cost, over-all feasibility in terms of management, state size, diver-sity, and available resources are ,examples of otherissues. The questions raised here were more of a philo-sophical nature and are pertinent to decision making.

The models described were identified as being alonga continuum. This implies that there are many othermodels which can he considered, but they most likelywill differ from these models in degree rather thanbasic type. Perhaps a system can be developed withpositive elements from several of the models describedhere. it may also be possible that more than one modelcan be in operation at a given time, particularly if oneaccepts the notion that certain areas require or morereadily fit into a state testing approach while all otherareas fit one of the appr\Oved program models. Theoverriding concern is which ntodel or models best servethe purposes of certification.

12

Performance-Based Teacher Education:Evaluation Issues*

By

PETER W. AIRASIAN

IntroductionIn this paper I shall consider two levels of evalua-

tion in performance-based teacher education models.The first concerns the evaluation of the basic conceptof performance-based teacher education. The secondrelates to evaluation issues within the context of anongoing performance-based approach. I shall argue thatthe important evaluative issues are not those of select-ing appropriate and objective measuring techniques butrather involve the types of judgments which dictatewho will he evaluated and on the basis of what per-formances. Those who are seeking cookbook, practical,"how to" answers from my presentation will be disap-pointed. My purpose is to argue that "how to" prob-lems. are predicated upon a prior set of evaluativedecisions, decisions ultimately more important andpowerful than any set of evidence gathering techniques.no matter how objective or complete these are.

The majority of evaluation issues raised by perform-ance -based teacher education are not new. What is newand significant about performance-based teacher educa-tion is that for the first time, systematic evaluation ofteacher training models and products is called for. It isthe required synthesis of evaluation problems whichheretofore have been treated largely individually whichmakes evaluation an issue in performance-basedteacher education.

EvaluationWe must start with a definition of evaluation, since

the term has multiple meanings according to the con-text within which it is considered. Perhaps the mostcommonly understood meaning of the term is "a judg-ment of the extent to which learners have mastered theobjectives of instruction." While this definition is ade-

. .

Invited paper. Multi-State Con..ortium on Performance-Based Teacher Fducation. New Orlean!. Louisiana, February26, 1973.

quite within. the context of many instructional pro-grams, it is limited in three senses. First, it narrowsattention to the intended outcomes of instruction. Sec-ond, it largely ignores information gathering about theprocess of instruction. Finally, it makes the learner thesole object of evaluation. To limit evaluation of per-formance-based programs to student evaluation is tooverlook many of the nonlearner aims and assumptionsinherent in the approach. It is important, particularlyin the early stages of acceptance and implementation,to articulate all the evaluative issues inherent in per-formance -based programs. To undertake this task,evaluation must be recognized as encompassing a moregenral purpose than simply determining learner mas-tery of course outcomes. In the context of this paper,then, evaluation wil! refer to "a value judgment ofmerit or worth." Clearly this is a more general defini-tion than that discussed above. It is meant to be. Omit-ting referents such as the learner, the curriculum mate-rials, or the instruction from the definition focusesattention upon the process involved in performing anevaluation; that is, making a value judgment. It alsosuggests that whenever judgments are made, regardlessof the referent, evaluatitin has taken place.

The heart of the evaluation process, then, is valuing.Data gathering, be it "hard," objective data or "soft,"impressionistic data, is not evaluation. Evaluation takesplace when data are compared to some standard ornorm and a decision about pass or fail, accept or re-ject, or good or bad is made. For example, the fact[ha. about 85 percent of the age cohort in the UnitedStates completes secondary school does not, in and ofitself, convey value. Some of my colleagues, when pre-sented with this fact say "Isn't it good that more youngpeople in America graduate from secondary schoolthan in any other 'country in the world?" Other col-leagues respond "Only 85 percentand I thought wewere doing much better." Or, to select an instance

closer to performance-based teacher education, con-skier a prospective teacher who teaches 60 percent of agroup of sixth graders the difference between simpleand compound sentences. Is mastery by 60 percent ofthe class grounds for congratulating or chastising theteacher? It is only when data are judged in terms ofstandards or norms that one can ascribe value to per-formance. There is, in essence, a difference betweenmeasurement (gathering data) and evaluation (placinga value on the performance).

Despite our wishes to the contrary, decisions aboutwhat is good, valuable, worthy, and desirable are madenot in the research domain, but rather spring from ourindividual philosophies and frames of reference. if I

were to ask ydu to cite three examples of school prac-tices which are based firmly in established researchfindings, I dare say that you would be hard put to re-spond. Arguments for and against performance-basededucation reside, and will continue to reside, in thevalue domain. The questions asked are not whetherone can state performances in behavioral, measurableterms but whether one should state them in any terms;not whether teachers can be evaluated on the basis oftheir students' performance, but whether they ought tobe. These are issues which are based in philosophy andvalue orientations, and it is in these frameworks thatevaluation centers. As a consequence, it is important toconsider evaluation issues as they relate to argumentsfor and against the basic idea of performance-basedteacher education.

Evaluation of the Performance-Based ModelThe antecedents of performance-based teacher edu-

cation are many, but they all revolve around a singlecentral theme: accountability. There is a serious break-down in the interest and ability of American citizens tosupport education. Educators are being forced to ac-cept responsibility for their activities and products. Ev-idence of successand if not success at least efficiencyis becoming a prerequisite for continuing financialsupport. The "cult of efficiency and rationality" is withus .gain. "Demonstrated value for dollars expended"and "It is time to make education scientific" are thewatchwords.

Now, my experience tells me that these are thewatchwords of legislators and administrators, not ofclassroom teachers. Accountability is, of course, alwaysthreatening to those who are to be held accountable.There is growing evidence (e.g.. Jackson, 1968; Goodand Brophy, 1973) that classroom teachers simplydon't think of accountability in the same rational,

13

efliciency-oriented terms as their administrative superi-ors. Moreover, classroom teachers perceive accounta-bility, rightly or wrongly, primarily as a method ofpointing the finger cf blame at individuals, rather thanat a myriad of interrelated and .diflicutt -tt.- define fac-tors. )

Performance-based teacher education is here todaymit as the result of a ground-swell of support fromteachers. but because administrators and politicians be-lieve it is a more rational, efficient, and accountablemethod of training and certifying teachers, as well itmay be. Performance-based programs are with us be-cause they are believed to he better by those who havesome say in the matter, not because research evidenceoverwhelmingly indicates their superiority. The deci-sion has proceeded from the top downwards. In short,the values implicit in a performance-based approach fitclosely with the values of legislators and administra-tors. An implicit evaluation has, therefore, alreadytaken place. The evaluation was not based upon harddata, but rather upon a comparison of the perceivedvalues inherent in performance-based versus other ap-proaches to teacher education. In a number of states itis clear that the performance-based approach has beenevaluated as superior.

The benefits claimed for a performance-based ap-proach clear and public specification of ends, emphasison exit not entry behaviors, individualized instruction,accountability, etc. are well articulated. It is the expec-tation of these outcomes which affords the justificationfor curret,t programs. Or, perhaps more appropriately,it is the value placed upon these outcomes which af-fords the justification. However, in accepting the per-formance-based approach, one is making a series of as-sumptions about-teaching, teacher training, and teacherevaluation. It is important to identify these assumptionsbecause when performance -based teacher education iscriticized and discussed, it will be primarily on thebasis of its inherent assumptions

Before elaborating these assumptions, it is appropri-ate to pause here to reemphasize the main po;nt of myargument thus far. In the social sciences, arguments forand against innovations and practice are based primar-ily upon the perceived values embodied in theinnovations and practices. Because the social sciences

. have few, if any, paradigms or models which are uni-ersally accepted, social scientists wage their battles atthe level of first causes or starting points. The physicalscience., are different from the social sciences insofar asthe physical sciences contain laws, theories, and princi-ples to which all physical scientists subscribe. One phy-

14

sicist may not agree with another's application or re-search in the area of relativity theory, but both acceptthe basic axior..s of the thcory. Such is hot the ease inthe social sciences. We are not at the stage of arguingabout applications, but rather about staring points. Wehave five theories of personality, not one; three theo-ries of what the "good" teacher is like, not one; fourlearning theories, not one. Each of us accepts differentstarting points about the nature of education, of man,or of the good. However, despite our rush to emulatethe physical sciences, one of the most powerful aspects

fl of the social sciences may be its pluralism. When weargue and criticize it is more often about assumptionsthan about application or practice. To a humanistic ed-ucator pertbrmance-based teacher education is anath-ema regardless of whether it succeeds or fails becauseit emphasizes, the wrong things. The point of thisrather lengthy digression is that evaluation of perform-ance-based teacher education by those outside the foldwill center upon the assumptions inherent in the per-formance-based approach as viewed from their valuepremises. Adherents to the movement require no evalu-ation of the central assumptions. They have alreadyevaluated i.nd endorsed them as good. Note that whileperformance-based proponents acknowledge the needfor research about teaching activities and outcomes toimprove the approach, few argue for a test of the basicidea itself. The basic idea has already been evaluatedand accepted.

What, then,' are the assumptions inherent in the per-formance-based approach? Major assumptions numberfive and are as follows:

1. There exists a set of performances which is im-portant for all teachers to possess.

2. It is reasonable to identify, define, and set stand-ards for the relevant performances.

3. Once identified, we possess the knowledge andskill to teach the relevant performances.

4. Teaching can be characterized as the sum of thedefined performances.

5. The performances are measurably related to stu-dent learning.

The fact thatthat.I have called these five propositionsassumptions does .rot imply that proponents of per-formance-based teacher educatioii consider these to beassumptions. In fact, the extent to which any state, lo-cale, or university has adopted a Performance-basedapproach is the extent to which these propositions arenot assumptions, but givens. While -thoughtful advo-cates of performance based teacher education recognizethe heed for more research in areas touching on these,

five .:real, most advocates accept the statements asfacts awaiting certificatior by research. To critics ofthe performance-based notion, however, the five state-ments qre regarded as assumptions and will he thefocal points of criticism and attack.

To illustrate my point, consider an article by ArthurCombs, director for humanistic education at the Uni-versity of Florida, which appeared in a recent issue of"The Journal of Teacher Education" (1972). The titleof the article is "Some Basic Concepts for Teacher Ed-ucation." I have selected this article as an example notbecause I favor the arguments it advances, but ratherbecause it happened to be sticking out kitty-cornerfrom a heap of similar articles on my desk. Strictlyrandom access to the 'heap would have served the samepurpose. Here arc two assertions advanced.

I. "The production of an effective teacher is ahighly personal matter, dependent primarilyupon the development of an appropriate systemof beliefs." (p. 286)

2. "Effective teacher education must concentrate itsefforts upon meunings rather than behaviors."(p. 287)

Combs goes on to argue, "It is conceivable that requir-ing a teacher education program to define precisely thebehaviors it hopes to produce may be the surest way todestroy the effectiveness of its products by concentrat-ing everyone's attention on the wrong dynamics" (p.288). It is clear that Combs' value orientation placeshim in a teacher education camp fat different from thatof performance-based adherents. If a performance-based approach could be shown to achieve its aims, Iwould expect Combs' reaction to be, "Fine, but that'snot what teaching is all about."

Returning to the five assumptions stated above, it ispossible to consider each individually and to raiseqiiestions a out its appropriateness. One could ques-tion the extent to which teaching is actually a iciencewhich can be analyzed and described in precise terms.Given current knowledge about teachers and teaching,one could raise questions about the ethical issues in-volved in identifying relevant teacher competencies and:eVels of performance. Even if certain performances/ex known to be related to teaching succes.>, the ques-den of whether such performances can be taught toprospective teachers is not at all clear. Within the con-text of performance-based teacher education, such is-sues are tagged as "problems to be solved," not as"considerations to be weighed in determining whetherwe should have performance-based programs."

Performance-based ter,cher education advocates have4

proceeded from a belief system which accepts theselive assumptions as givens. As a result, what is perhapsthe most important tnaluutke question about perform-ance-based teacher educationwhether or not it is bet-ter than current practice., and should he implemented

has been answered in many states. The likely impactof this value judgment on states, teacher education in-stitutions. prospeetie teachers, and students outweighsthe impact of evaluations carried out within any ongo-.ing performance-based program. Once one's belief sys-tem accepts the need to go ahead with perf.)rmance-based teacher eduea.ion and certification, probtinsreduce to those concerned with implementation and"spreading the gospel.- It was possible to have ap-proached the issue of performance-based teacher edu-cation as a societal experiment, emphasizing plannedintervention and evaluation of the basic notion itself(Campbell, 1969) . However, it should be noted thatthis experimental approach also proceeds from a beliefsystem, one which accepts 4 rational, planned, gradualapproach to societal change.

Regardless of the basis on which the decision to in-stitute a performance-based approach is made, how-ever, one; it is agreed to commence, issues related toevaluutic n within ongoing programs arise. It is to theseproblems I snail now direct attention.

Evaluation in Performance-Based Teacher°Education Programs

A multitude of models whi,:h integrate both stu&\ntand curriculum evaluation concerns of instructionalprograms have been advanced in recent years (e.g:,Airasian, Mactaus, and Rakow, 1972; Provus, 1969;Stuffleheam, 1971). These models describe varioustypes of evaluation, the points in instruction where thetypes arc most relevant, and the kinds of judgmentswhich can be Made on the basis of the evaluations.Most models arc specifically designed to help indivi-dualize instruction within the context of programswhich emphasize predefined objectives or competen-cies. Because such models exist, I shall not attempt tosuggest a full-blown evaluation model for perform-ance-based programs. Rather, I shall direct may atten-titin to a few evaluative issues which I feel are ofmajor importance for any performance-based teachereducation model.

Defining Teacher CompetenciesWhile some observers-sive that the most powerful

individuals in a performance-based approach are thosewho ultimately certify performance or competency, 1

15

would argue that the most powerful individuals arethose who frame the competencies to he attained.these are the individuals who explicitly define what isa good tetteht -. The decisions of these individuals colorthe selection of learning experiences as well as theevaluative techniques and criteria. In thyerformance-based approach. which proceeds from identification ofends to selection of means to. obtain these ends, it isthe ends which are paramoOnt. The rationale for aprogram. its learning experiences, standards, and certi-fication practices rest upon the performances defined asneeded by the good teacher. Parenthetically, there is ?vast literature on the pros (e.g., Bloom. Hastings, a.. .dMadaus, 1971; Block, 1971; Popham, 1968; Tyler,1934, 1950) and eons (e.g., Atkin, 1968; Doll, 1971;Eisner, 1966) of an approach to education which pro-ceeds from clear articulation of ends to selection ofmeans. Seldom does this literature appear in the con-text of discussions about performance-based teachereducation.

It is at this first stage involving the definition ofcompetencies that evaluation should play a major role.Just as controversy about performance-based teachereducation generally will center upon its'assumptions, sowill controversy about any given performance-basedprogram center on its specific definition of the goodteacher. In short, one of the primary domains of evalu-ation within a given program will be its goals; i.e., thecompetencies it seeks to teach. That this will inevitablyhe the case is a function' of the fact that not everyonedefines a good teacher in the same way.

Wise administrators of performance-based programswill build evaluation of their program's defined compe-tencies.into their planning efforts. One useful model forsuchtevaluationis suggested by the National Assess-ment of Educational Progress study staff. The NAEPsought to provide nationwide census-like data aboutthe educational attainment of Americans in a variety ofsubject fields. In planning the assessment, the staffquickly came face to face with the problem of definingwhat the educational system was trying to achieve. Inessence the problem was to define the competenciespossessed by the gOoti student in areas ranging fromarithmetic to citizenship. A tentative list of objectiveswas determined for each content field by subject matterspecialists. These objectives and suggested proceduresfor gathering evidence about their mastery were sub-mitted for evaluation and revision to panels composedof educators, concerned citizens, and scholars repre-senting various points of view and sections of thecountry Some of the panel sessions were heated; some

16

resulted in objectives being deleted because consensusregarding their appropriateness or importance couldnot be reached. In the end, however, a general consen-

, sus was reached and a list of objectives acceptable to6 representatives of diverse segments of the populace was

advanced in each subject area.Thus, once a performance-based program has gener-

ated an initial specification of the competencies it seeksto instill in prospective teachers, numerous interestedand affect(' d publics such as parents, teachers, students,administrators, scholars, and others should have an Op-portunity to judge, criticize, and revise the competen-cies. Not only does such a screening process have apublic relation value, but more importantly, it affordsinput into the process by publics who are likely to bevitally concerned and affected by the performance-based approach. it is likely that reconciling the inpu sand judgments of, varied constituents will be difficult,time consuming, and often frustrating. However, theimportance of the specified competencies in directingall aspects of the performance-based program man-dates such an evaluation at this early stage.

r Evaluating Student ProgressOnce Some degree of consensus regarding the 'ends

of performance-based teacher education is reached, theproblem of evaluating student progress through theprogram becomes relevant. There are three domainswhich call for student evaluation. The first involvesknowledge of processes, theories, techniques, etc., in-formation which is best evaluated by paper and pencilmeans similar to those utilized in' nonperformancecredit aad course-centered approaches. The second do-main involves teaching practices and activities, demon-strations that the student can adtually perform and uti-lize various strategies, modes, and techniques. Finally,students will be evaluated in terms of the extent towhich their knowledge and performance capabilities re-sult in improved classroom learning.

The third domain, which may be termed the productor output domain, will be discussed in a later sectionof this paper. In this section I will consider the knowl-edge and practice domains. Three evaluative issues ap-pear relevant to the discussion: the specification ofentry behaviors, the mode of grading student perform-ance, and the problem of obtaining an adequate sample'of student behavior.

While performance-based advocates stress the exit.behaviors of students who pass through their programs,it may be appropriate to raise some questions relatedto student cntry behaviors. Note that emphasis on exist

behaviors implies an acceptance of the proposition thatenough is known or can be known about instructionaltechniques' to insure that most individuals can attainthe prespecified competencie,s of a program. Despitesome personal reservations about such an assumption,I am willing to concede that it may be appropriate andtrue. However, one might question whether it is worththe time, effort, and expenditure for a- particular stu-dent to take 8 years.to obtain ultimate initial certifica-tion when the majotity of studens,'reach the samestage in 2 years Should, then, proipective candidatesfor performance-based programs be 'evaluated with re-spect to their attainment of a minimum set of entrycompetencies before being accepted into the program?

Undoubtedly some initial screening of candidates isalways done. Candidates who do not possess adequateacademic records or test scores are rejected. Capdidateswho do not manifest particular interest, pe?sonality,and value characteristics are not accepted. But evi-dence about the cognitive .and affective status of indi-viduals is typically gathered by means of global,imprecise, impressionistic means, the very meansperformance-based programs strive to eradicate. Itwould appear worthwhile, then, to evaluate prospectivestudents in terms of a set of entry competencies whichare specified in the sa. ae manner as the exit competen-cies. Certainly it is impossible to.. identify a completelist of entrance competencies at. this point in time.However, some entrance criteria can be identified andmade explicit. Such criteria would require monitoringand revision over successive groups of students to ar-rive at a more complete and relevant set, but the im-pact of well defined entry competencies may be worththe effort insofar as they may reduce the frustration ofunqualified candidates as well as serving as a means ofplacing accepted candidates into the optimum instruc-tional starting point (Airasian and Madaus, 1972a).

Once students are into the instructional process, it isnecessary to determine whether or not they have at-tained mastery of the required competencies. The firstissue here is, of course, what is meant by mastery.That is, one must identify the standard of performancestudents must attain to be certified as possessing acompetency. Obviously, there is no source, other thanjudgment, to which one can refer to select appropriatestandards. The question of standards is one whichplagues all evaluation efforts. However, the nature ofthe competency, its relevance to instruction, its sus-pected impact on classroom learning and other suchconsiderations should be weighed in setting the stand-ard. For example, based upon consideration of the

above areas, 1 have set a mastery standard of 85 per-cent in my secondary level course in tests and measure-ments. A student is' certified, as bmastering tests andmeasurements wlifn he demonstrates competency on85 percent of my course' objectives. Were I teachingphysical education to-secondary education majors con-centrating in areas other than. physical education,would set a lower mastery standard. At any rate, onemust have some criterion whichcenables him to d :ter-mine whether a given performance is indicative ofcompetency.

With 'the prior specification of competencies andstandards, the problem of grading students is relativelystraightforward. The crucial judgment to be made iswhether the student has attained mastery of a compe-tency, not how he has performed relative to his peers(Airasian and Madaus, 1972b; Popham, 1971). Asvarious authors concerned with* performance-basedteacher .education have indicated, criterion-referencedas opposed to norm-referenced evaluation is needed.With respect to each required competency, wewill wantto know whether the student has achieved criterion-level performance, not how high or low he stands rela-tive to other students. Criterion-referenced evaluation,which many argue is more relevant, humane, and"fair" than norm-referenced evaluation, is possible onlywhen criterion .performance is predefined. One of themajor advantages of performance-based models is thatthey dp incorporate prespecified, well defined exit be-haviors. As an added benefit, the availability of clearlystated exit behaviors which serve as evaluative criteriaenhances diagnosis of student learning difficulties. Un-like the course grade method of evaluating perfAnance,a criterion-referenced approach related to identifiedcompetencies can provide the student with more pre-cise diagnostic direction than "study more" or "workharder." In a sense, all evaluation taking place rior toactual certification of competence is formativ diag-nostic evaluation.

Actual data gathering techniques to evaluate knowl-edge and practice competencies are not complex. Forknowledge competencies, paper and pencil tests, oralexaminations, and the like are appropriate. For prac-tice competencies, studies' performance in classroom,microteaching, or other similar situations can beevaluated by one, or preferably more, judges on thebasis of checklists or overall performance. The basisfor judgment, be it judgment of knowledge or practice,is always performance relative to the predefined com-petencies. The extent to which evidence gathering situ-ations permit students to manifest the behaviors inher-

17

ent in the competencies is the extent to which' theevaluation is valid:

One note of (minion must be introduced. however.Any testing situation provides only a sample of a stu-dent's behavior. Relatively slight differences in testitems or classroom situations from those utilized in anevaluation may engender different student performance.Thus, in judging competency, particularly of 'teachingpractice behaviors where evidence gathering anci evalu-ation age more time consuming, it will be important toinclude more than a single measurement. Studentsshould be required to demonstrate competence in class-rooms of varying types Aefore they are certified ascompetent.

Competencies EraluatidIn addition to helping judge curricular adequacy and

student learning, evaluation plays another powerfulrole in any instructional system: it defines for thosebeing evaluated what the important or "real" aims c*.the instructional 'program are. We have all heard stu-dents remark, "lrget what he tells you in class. If youwant an A, memorize the book." The implication ofsuch a statement is that the grading processthat'which ultimately counts -,-is based upon a very limitedsubset of the totality of instructional activities andaims.

''he relevance 'of evaluation's role in defining thereal aims of instruction for performance-based teachereducation relates primarily to affective performance.No performance-based approach worth' its salt willargue that teaching competence is comprised-solely ofcognitive and psychomotor skills. Attitudes, values, in-terests, preferences, and the like are at least as impor-tant to the competent teacher as cognitive and psy-chomotor abilityperhaps more so. However,instructional programs based upon the clear identifica-tion of objectives or competlecies have a lengthy his-tory of paying lip service to the importance of affectiveoutcomes, but concentrating evaluation efforts on more%Laity measured nonaffective areas. By following such apractice, evaluators define the.cognitive and psychomo-tor aims as the really important objectives of the.progrNm, regardless of what the program designersclaim about the importance of affective aims. It doesn'ttake long for students to "psych out" what is requiredof them and to shape their behavior to comply withexpectations.

Thus, if affective performances are important, theymust be evaluated. Unquestionably the state of the artof affective assessment lags behind cognitive or psy-

18

chomotor assessment. In the end, interpretive judg-ments based upon both formal ami informal observa-tag and discussions will probably provide 4thoptimum means of gathering affective evaluative dataabout student progress...the lack of, objectivity associ-ated with such techniques in Qomparison to more for-mal paper and pencil techniques should not deter eval-"aation. One method of stressing -the importance ofaffective aims is to diagnose and evaluate them.

Data Maintenance

In the course of ev aluation in performance-basedteacher education, much/date about individual studentswill be gathered and assessed. Two issue s related tothe maintenance of such data must be considered.First. one must consider the form in which individualstudent data will be kept and disseminated: Second,

"one must consider to whom data will be disseminated.Each of these isstlts will be considered in turn.

Perlormancy-based educational progt ams start witha clear definition of the capabilities possessed by thegood teacher. The capabilities are specified in behav-ioral terms, not in terms of number of credit hours,grades in courses, and the likC. it seems reasonable,

,given differences between performance-based and moretraditional programs, that student data whin perform-.ance-hased approaches be maintained in a differentmanner than in terms, of course gradeS or creditsearned. Rather, student records should, contain an indi-cation of student attainment of every relevant perform-ance.. While the prior statement may hardly soundinsightful or rey .olutionary, recordkeeping. and dis-semination in practice are likely to become time-consuming and unwieldy. One of the few advantages ofthe letter grade or course credit recordkeeping systemis that it permitsits to describe a student in a short, ef-ficient manner. An ihdex Lard suffices to characterize astudent over 4 college years. In an attempt to over-come the lack of specification inherent in grades andcredits, performance-based approaches turned to- theidentification of specific behaviors related to goodteaching. To characterize student leirning in the light ofthe extensive lists of performances identified in mostperformance-based programs, pages and pages of rec-ords are needed for each student. It is important thatsuch records be kept and disseminated in unshortened,unexpurgated form to state certification boards, pro-spective employers, and to the students themselves.

Individuals responsible for recordkeeping in per-i'ormance-based approaches must resist the inevitable

C

t.

pressures from school administrators and certificationboardS to provide a number or a brief statement tocharacterize a student. fo succumb to such entreaties,no matter how attractive they appear or strongly theyare urged, will result in reducing the importance of theprespecified performances, which represent after all,the basic appeal of performance-based approaches.Further, to succumb to such entreaties will 'return certi-fication dtd hiring practices to a "highest numberwins" basis, with the result that specific relevant capa-bilities will be overlotik,:d or ignored.

Adequate recordkeeping represents only one aspectof the data maintenance problem. To a much greaterextent than more traditional approaches, performance-based instructional programs will have on file explicitaffective judgments about students. While our schoolsand society accept cognitive information about individ-uals as a mime currency of worth or merit, there issome reluctance to judge individuals 'tire the basis ofpersonality, value, or other affective .clutracterisitics.There is a feeling that these arb mote priitate and leisnecessary for others to know. That affective qualitiesare important to good teaching, few would argue.However, a crucial question to be answered within thecontext of performance-based programl is "To whomshould judgments about a given student's values, per-sonality, interests, and preferences be disseminated?"Clearly some individuals and agencies will require suchevaluative information; but which, under what circapt-stattices, in what form, with what guidelines, and forhsw.4 long are only a feiv of the concerns which shouldgifide dissemination, There are no right or wrong, ethi-cal or unethical answers to these questions. It is impor-tant, howevqr, that every performance-based teachereducation approach find answers to these questions,answers which prott..4.the privacy rights of its studentsyet afford information access to those to whom it is es-sential.

Judging Teachers by Their Students' .

. Achievement

The final evaluative issue to be raised is undoubtedly 4

the most emotion-charged aspect of performance-basedteacher education: the evaluation of teacher ,compe-tence in terms of student learning. I suppose that in areal sense, my own dual responsibility as a professionaleducator and a parent reflects the problems involved injudging teachers' performance by their students' learn-ing. As a profcssiouid educator, I recognize the myriadof factors outside Ithe teacher's control which affect

4

a

/

4.

learning. Rut as a parent, 1 won&r whether not:hold-ing teaeherc eAen minimall accountable for studentlearning is an acceptance of the position that the rea-son for i:l! failure rots upon the student's shoulders.

The dzi!a which must be gathered to evaluate teach-ers,' effects upon student Warning ay not at all clear.Personality and cognitive differences between teachersteachiQg the same type of classes as oftet as note resultin the same oyerall class performance for both teachertypes. R4.setirch indicates that a substantial portion ofthe variance in student ability and achievement is at-tributable to environmental factors whose major impacton the student occurs before the student reaches theschool. The effect of failure in prior years up On subse-quent performance remains the subject of discussionamong educators, Although none of us relish the impli-cation;;, it may he that prolonged failure, In school ex-periences ultimately results in students who are simplynot reclaimable, regardless of any teacher's competenceor perseverance. We might even questiCip whether fail-ure or poor performance in some courses is not a ben-

t efit "to students,' especially if it prevents them frompursuing careers or areas where they-ifiav little proba-bility of success. Finally, and perhaps .most impor-tantly, there is the problem of the criteria\ which indi-cale successful student learning. ''The possibilities here.are

In sum, while it is always possible to evaluate teach-ing competency by measuring student learning, the is-sues remaining to be settled before such evaluation canhe undertaken in an intelligent manner, fair to b.pth

191

teachers and students, suggests that student learningmeasures not he used to evaluate individual teachers atpresent. Rather, research on variables and measureshypothesized to relate to student achievement should beundertaken. Particular teagher competencies shouldfirst he validated against,measures of student behavior.Note, however, that the most accurate and extensiveresearch Alata - will not remove the ultimate need formaking evaluative judgments concerning. the type andform kif student behavior which will be considered in-dicative of competent teaching.

Conclusion

This paper has argued that the 'real evaluation ques-tions inherent in performance-based teacher educationrelate not to constructing objective evidence gatheringtechniques or measures, but rather to judgments aboutwhat data are to he gathered, from whom, and underwhat conditions2The selection of instruments and tech-niques is an outgrowth of these. prior evaluative deci-sions. Those whose primary interests are concernedwith how questions, need only consult most measurement texts or the curriculum evaluative, literatute. If Ihave left some of *my audience unsatisfied because Idid not address more practical how to questiots, mysole defense is that I believe identification and resolu-tion of the what questions is the surest manner of ..nak-

, ing perfonnlince-bakd teacher. educ4ion a mote via-hie, effective, and humane .approach to the problem ofeducating teachers.

References

Airasian, P. W. and Madaus, G. F, Criterion-refer-enced' testing in the classroom, NCME Measure-ment in Edfieation, 3, 4, May, 1972a.

Airasian, 0. W. and Madaus, G. F., Functional typesof studt..0 evaluation, Measurement and..Evalua-

.don in Guidance, 4, 4, 1972b, pp. 221-233.

Airasian, P. W., Madaus, G. F., and Rakow, E. A.,An instructional evaluation monitoring system,Curriculum Theory Network, Monograph. Supple-ment, Curriculum Evaluation: Potential and Real-ity, 8/9, 1972, pp. 74-95.

11.

Atkin, Al. Using behaviorally stated objectives fordesigning the curriculum: a cautionary note.__Paper delivered at the , annual meeting of theAmerican Educational Research Association, Chi-cago, Ill., Februar3. 1968.

Block, J. IL, Mastery learning: theory and practice,New York: Holt, Rinehart & Winston, 1971.

Bloom, B. S., Hast:ngs, J. T., and Madisus, G. F.,Handbook of formative and summative evaluationof student learning, New York: McGraw-Hill,1971.

V

20

Campbell, D. T., Reforms as experiments, AmericanPsychologist. 24, 4, 1969, pp. 409.429.

Combs, A. W., Some basic concepts for teacher educa-tion, The Jouenal of Teacher Education, 23, 3,1972, pp. 286-290.

Doll, W. E., A methodology of experience: an alterna-tive to behavioral objectives. Paper delivered atthe annual meeting of the American EducationalResearch Association, New York, N. Y., February1971.

Elsner. E. W., Educational objectives: hebp or hidrance. Paper delivered at the annual meetithe American Educational Research Assoc'Chicago, Ill., February 1966.

Good; T. L. and Brophy, J. E., Looking in classrains,New York: Harper and Row, 1973.

Jackson, P. W., Lifo.in elassnioms, New York: Holt,Rinehart & Winston, 1968.

Popham, W. J., Probing the validity of argumentsagainst behavioral goals. Symposium presentation

41

at the annual meeting of the American Educa-tional Research Association, Chicago, Ill., Febru-ary 1968.

Popham, W. J. (ed.), Criterion-referenced measure-ment, Englewood Cliffs, N.J.: Educational Tech-pology Publications, 1971.

Provos, M. M., Evaluation of ongoing programs in thepublic school system. In R. Tyler (ed.), Educa-tional evaluation: new roles, new means, sixty-eighth yearbook of the National Society for theStudy of Education, Part 11. Chicago: Universityof Chicago Press, 1969, pp. 242-283.

Stu &bears, D. I., et al., Educational Evaluation anddecision making; Itaska, Ill.: F.' E. Peacock Pub-,

971,

IP

Tyler, Et. W., Basic principles of curriculum and in-struction, Chicago: University of Chicago Press,1950.

Tyler, R. W., Constructing achievement tests, Colum-bus, Ohio: Ohio State University ,Press, 1934.

The State Of The Art In Performance AssessmentOf Teaching Competence

By

FREDERICK J. McDONALD

"N To describe and evaluate the state of the art in per-'t formanze assessment of teaching competence, we

..r should a swer three questions: (1) What are the goalsof the sy tems 'to measure teaching competence? (2)Whit prog have we made in developing the proce-dures required for systems measuring teaching compe-

,. tence? (3) What problems must be solved to create..% effective and efficient systems for assessing teaching

competence?Before attempting to answer these questions, I

should first openly acknowledge the assumptions thatinfluence my ideas Ind those that do not. I assume. thateven small prog ) s made in assessing teacher compe-tence will be a eat improvement over our presentevaluations. Because I assume this, I, am willing to

, ozge the use of procedures and systems which at thepAsent time are limited or even defectivesince I alsoassume that as we use what instruments and techniquesare now available we shall learn more about the natureof teaching competence and progressively improve .ourmethods for evaluating it. I further assume that thecost of measuring Teacher competence will fibt neces-sarily be prohibitive, despite the fact that many peopleseem to assume the contrary. It is obvious thit institut-ing a measurement systerit where there za.4 none be-fore will impose heavy costs. But until we have a com-plete cost/benefit analysis, .I am not willing to assumethat these costs will be unmanageable. Further, I donot assume that the evaluation system will necessarilybe complex because we do not know 'as yet what infor-mation is needed to make reliable and valid judgmentsabout teaching competence. I choost, therefore, to beoptimistic about both the cost and complexity of anevaluation system

It seems to me that the most reasonable position totake at this time is to acknowledge the many problemsand to regard their solution as dependent upon..the or-

I

f - S

21

dinary proCesses of research and development. It isself-defeating and destructive. to so negatiVely prejudgethe situation that work which might resolve the prob-leW'is either impeded or premattirely terminated.

The Goals of an Evaluation SystemLet us now return to the three questions with which

I began this presentation. The first is obviously themost important: what should be the goals of a per-formance assessment system? The ultimate goal of sucha system is to provide information that enables the de-cision makersteacher educators, certification officers,and administratorsto decide whether a teacher candi-date has' sufficient competence to be permitted toteach. Everything else that we will want to say aboutthis goal is either a refinement of what is meant bymaking a judgment about competence or' a problemthat must be solved if such judgments are to be made.I will therefore list a set of conditions that must be sat-isfied for this goal to be attained.

First, measurement procedures used in the evalua-tion of teaching competence must have hich validity.Validity in .this case means we have demonstrated asubstantial relationship between a teaching skill or per-formance and its effects upon students.

At this-point in time we know very little about whatskills or performances have demonstrable effects onstudent behavior.. The highest priority in a research

'program on teaching competence must be given tosolving this problem. The conclusion, however, shouldnot be drawn that we must defer the development ofan evaluation system until all the relevant research hasbeen done. We already have an abundance of ideas onpertthent teaching competencies which we can begin tomeasurea necessary first step; and one whose-Cffecton teaching performance can be studied systematicallyas part of the process of developing evaluation systems.

22

We cannot or ought not, however, make definitivejudgments about teaching competence until the validi-ties of specific teaching skills or performances have

been established.Second, we must establish the reliability of assess-

ment procedures that will be used, in evaluation sys-tems. Reliability in this case means the stability of ateaching skill or performance. Does the teaching skill

vary considerably from day to day, class to class, topicto topic? We need to know the conditions under whichit is most likely to be variable and those under which itis most likely to be constant.

While considerable attention is given to the problemof validity, practically no attention has been given tothe , performance characteristics of teaching skills.

Therc is some intuitive understanding that a variety ofconditions may affect a teacher's skill, but formal re-search on the problem is nonexistent. It makes no dif-ference how research on this aspect of teaching per-formance 'c Imes out because we need all kinds Ofinformation in making judgments about competence.If, for example, a particular teaching skill requires spe-cial conditions to elicit it. we must ask whether thoseconditions were present when the teacher's possessionof the skill was being evaluated.

Third, we need to know what, for lack of a betterterm, I will call the "learnability" of a skill. It is con-ceivable that a particular skill, even though it has beendemonstrated to have highly desirable effects uponchildren's learning, can be learned by only a few indi-viduals. We do not expect everyone to become theoiet-lc& physicists or concert pianists or gold medal-win-ning athletes because we know that some specialaptitudes may be required for achievements of thisorder. Similarly, there may be forms of teaching whoseacquisition demands unusual aptitudes. Thus, weshould avoid the trap of assuming that every skill dem-onstrated to have desirable effects on si dents will nec-essarily be required of every person who wishes toteach.

We must develop the kinds of information describedin these three pointsrelating to the reliability, valid-ity, and learnability of teaching skills. We cannotachieve our goals unless we can define what is to bemeasured, under what conditions it is to be measured,and how the information yielded by the procedure is tobe interpreted. But there are some other conditions be-yond these three that must be met by any system formeasuring teaching competence.

A fourth condition, for example, is that the informa-tion gathered about teachers must be as uncontami-

nated by subjective biases and political processes as is

humanly possible. Any observation of human beli.1%ioris, of course, influenced to a degree by the characteris-tics of the observer and by the characteristics of themeasurement procedure itself. We shall need to de-velop new strategies for collecting information onteaching competence if we are to minimize the prob-lems of gathering such information.

Fifth, the conditions of measurement must providecomparable information on groups of teachers. Wemust standardize the conditions under which teacherbehavior is measured. Otherwise, it is impossible to tellwhether variations in teacher behavior arc due to situa-tional differences or to differences in teacher compe-.tence, thus exposing the candidate to the vagaries ofidiosyncratic favoritism.

The conclusion should not be drawn, however, thatthe assessment system should, ignore variations in theconditions under which teacher performance Nears. AsI stated earlier, it is essential that we determine thoseconditions under which a teaching skill may be ex-pected to vary. Such information is necessary in evalu-ating teaching competence. But we cannot treat teach-ing as if it were so different on each separate occasionthat we can never evaluate it. The conflict between es-tablishing reasonable expectations for teaching per-formance and the variety and complexity of the situa-tions in which teaching occurs is one of the mostimportant problems we have to solve. Until it is solved,our decisions about competerWe must necessarily betentative.

Sixth, we must develop an evaluation system thatmakes due allowance for the teachers' opportunities toacquire the relevant teaching skills. A decade may wellpass before teacher education programs have becomeperformance-based and before a reasonable amount ofperformance assessment has taken place. To demandcompetence of individuals who have not had the op-portunitites to acquire that competence is immoral, andto make judgments that affect their careers on such aninsubstantial basis is equally immoral. Assessmentshould not be used for evaluative purposes when theindividuals being assessed have not had the opportunityto learn those things for which they are being evalu-

ated.In listing these constraints, I have outlined the major

problems that must be solved if a system of assessingteaching competence is to be developed and used in apractical way; indeed the solution of the problems de-pends on our meeting these conditions.

The State of Measurement Technologyand Knowledge

Despite popular opinion to the contrary, measure-ment technology follows the wants of its users. Forthree-quarters of a century, decision makers of one kind'or another have wanted to assess what candidates forteaching positions know. Measurement technology forassessing academic knowledge thus became highly de-veloped. Consequently, we now have widely availabletests of knowledge of subject matter and of knowledgeabout teaching methods. But until recent years therehas been no deniand of any consequence for the devel:opment of performance assessment. As a consequence;this aspeet.of measurement is largely undeveloped.

We have developed observation scales which permitus to report what teachers do under ordinary teachingconditions:. but none of these observation schedules ormethods meet any of the constraints which I outlinedpreviously. The conclusion should not, however, bedrawn that the use of observation schedules should beterminated: nevertheless, they and any other measure-ment procedure must be subjected to the kinds of re-search and development necessary to make them usefulwithin the stipulated constraints.

The basic problem of performance assessment tech-nology is to create a controlled situation or to use anordinary teaching situation with sufficient knowledge ofits characteristics so that their influence on the per-formance can be assessed. Observation schedules fallinto this second group of procedures. The problem ofassessing the influences in a wide variety of teachingconditions and performances is enormous. I have cometo the conclusion that a more profitable line of re-search should use controls, simulated teaching situa-tions.

Rather than! describe the adequacy of availablemeasurement technology, a rather depressing picture, Ishall describe the work in progress on the assessmentof performance: using simulated teaching conditions.Thii work is relatively new in the field of teachingcompetence, and it is conceivable that it may never de-velop into a practicable technology. But some of theproblems must Ice solved and are more likely to besolved.efficiently and satisfactorily by using simulatedteaching conditions. The relationships between teachingperformance and student learning, for example, can bestudied very efficiently by using microteaching sessions.Similarly, the reliability of teaching performancesunder controlled conditions can also be easily assessed.

We are attempting to build a system that provides

BEST COPY AVAILABLE

23

,fara

4+4124.401,-*Matik412

as

..a..

Frederick 3. McDonald

information about basic teaching skills, teaching per-formances, and teaching strategies. This system is de-signed to use simulated teaching situations of varyingdegrees of complexity. Initially, both the teaching skillmeasured and the conditions under which it is meas-ured are relatively simple. The prospective teachermoves through the system from this simple initial levelto one where he or she must use a variety of skills,performances, and strategies in combination and mustenact them over a period of time. The critical featureof the component systems is that the conditions arecarefully controlled. In what ways are they controlled?For any given component, the teaching objectives andteaching performances are. specified; for example, if weare attempting to assess the teacher's ability to rein-force students participating in a discussion, the objec-tive of the lesson is set as increasing the amount ofstudent participation in the class. The trainee is toldthat he will be evaluated on the degree to which stu-dents participate in a class discussion. Initially, wewant to measure only his use of reinforcement proce-*dures to facilitate participation in clasiroom discussion.In other components and at other times, the teachingsituation will be constructed to evaluate the trainee'sability to present a stimulating problem for discussionor to ask questions which elicit comments from stu-dents or facilitate discussion by highlighting differencesof opinion.

In order to make assessments at this time, certainfactors have to be controlled. For example, if we are to

24

study the teacher's use of reinforcement techniques, wedo not want the teacher preoccupied with the prepara-tion or development of the topic or with the choice ofthe topic, nor do we want a group of teachers to havedifferent objectives. Therefore, as I noted, we stipulatethe objective, and we also provide the teaching mate-rials for the lesson. Further, we control variability ofthe students by reassigning the students to a teachingassignment and by providing the preparation periodfor familiarizing themselves with the material to be dis-cussed. It should be noted, and I wish to emphasize,that the factors which are controlled in one type of sit-uation may be systematically varied in another in orderto assess the degree of the teacher's skill.

What can be learned by using this technology?These mi oteaching sessions can be used to assess ateacher's s 'II under a variety of different teaching con-ditions. We ave tried several formats of microteachinglessons to iermine what can be learned by havingteachers tea different lessons and different groups ofstudents. In one forinat, for example, we asked theteacher to teach four different topics to the same groupof students; in another, a teacher teaches different stu-dents the same topic; and in yet another, the teackerwill teach different topics to the same students andthen the same topics to another set of students. Inthese ways we can test how the teacher's skills changeunder the different teaching conditions. We have al-ready found that inexperienced teachers' behavior var-ies from topic to topic, to some extent, even thoughthe topics appear to be highly similar in nature andwould seem to offer similar opportunities for usingteaching skills.

Eventually, we would.expect to have a series of rep-resentative teaching situations in which it had beenshown that certain kinds of teaching skills are highlylikely to be useful. An assessment battery would 'con-sist of a series of microteaching sessions covering theseteaching situations.

Because the microteaching format has several limita-tions, such as relatively short lessons and small num-bers of students used, we developed a mini-courseformat to use for more complex teaching situations, Amini-course in this context is a short course lasting 1or 2 weeks. The teachers teach this course an hour aday for one of these periods of time. For these courseswe have also developed a complete statement of objec-tives, course materials, and achievement tests to betaken by the students at the end of the course. Almost300 preservice teachers have taught these courses sothat we have gathered considerable data on teaching

performance and experience with the technology' itself.The results of the analyses of these teaching per-

formances indicate that the microteaching perform-ances are relatively poor predictofs of the teaching per-formances in the mini-courses. But, for several reasons,I am not willing to offer this conclusion as solidly sub-stantiated. First, the subject-matter of the short coursesis more complex, and segments of any one lessonwould be similar to the teaching situations created bythe microteaching session. But their combination pre-sents a different kind of teaching problem. Also, in thesituation in which we have gathered data, relatively lit-

. tle performance training took place bet cen the initialmicroteaching and the subsequent teaching in the shortcourses. Systematic training using the assessment gath-ered in the microteaching might influence substantiallythe relationships between the two kinds .of perform-ances; that is, where the teacher has sufficient skills, as-revealed in the microteaching performance, the instruc-tors would reinforce these skills so that they were morelikely to be used by trainees in the more complexteaching. There is still reason to believe that we havecreated two sufficiently distinctive types of performancesituations which will yield equally valuable but differentkinds of information. My own opinion is that the mi-croteaching is more useful for assessing the degree towhich a teacher has basic skills; the mini-course is mostuseful in assessing how teachers integrate these skillsinto complex teaching performances.

As I said earlier, otir goal is to build a system thatwill assess the simple components of teaching compe-tence and also its more complex aspects. Wevisualizethe coordinated assessment system as made up of pe-riods of microteaching used primarily for assessmentpurposes and short courses or units that can later beembedded in internship and student-teaching experi-ences. These components would be arranged so thatthe training institution could be given information con-tinuously about the level of competence being reachedby the trainee.

The student teaching and internship experiences canbe used to assess daily performance under uncontrolledconditions. They are useful for providing informationon what teachers are likely to do in contrast to whatthey are able to do. The simulation-type teaching situa-tions that I have described should be thought of asteachers assessing what teachers are capable of doing;whereas on-the-job performance tells us what he or sheregularly does. We can also use on-the-job observationto assess such factors as teaching style.

I am convinced, after a year's experience with ad-

ministering systems such as these and another year'sanalysis of the data gathered, that it is entirely feasibleto build a useful performance assessment .system thatcan he embedded in any teacher-training program. Assuch systems arc developed, state departments of edu-cation .:an learn what criteria ought to he applied tothe assessment of teaching competence. The prototypethat we are developing in our research program can beused to generate concepts about the appropriate meansto evaluate each training institution's procedures for as-sessing the competence of their students.

It is obvious to me that the type of techniques beingdeveloped can be used with beginning teachers prior tothe granting of tenure, and for periodic reassessment ofteaching skill. We should not, however, use these pro-cedures at the present time for continuing certificationschemes, any more than we ought to use the currentobservational procedures and administrator evaluationsfor that purpose; but there is good reason to believethat, in time, assessment strategies can be developedwhich will be useful both in preservice and inserviceassessment of teaching competence.

One question asked frequently is, "How expensivewill performance assessment be?" As I observed ear-lier, there is no adequate way not to answer this ques-tion because we do not know what information weshall need to make judgments about teaching compe-tence. My experience indicates that we can gather largeamounts of data about teaching performance relativelyinexpensively: for example, three 20-minute micro-teaching sessions yield an hour of teaching perform-ances. A videotape of these performances can be ob-served to assess many different skills. In fact, the shortcourse yields almost more information than can bequickly processed. We have tesorted to audiotapingand videotaping about a half hour of each hour sessionfor the very practical reason that this half hour coversthree-fourths of flu. teaching that occurs. I am not con-vinced that even this much information is needed: Weare studying this problem by taking different samplesof the videotape performances that we have and find-ing :he intercorrelations among teaching performancesacross the samples. This technical work will eventuallyhelp us find the optimum amount of observation timethat we must use in order to make reliable judgmentsabout teaching skills.

While many people are concerned about the cost ofperformance testing, few discuss 'the economies thatmay be achieved by using them. It is very apparentthat, when one looks at videotapes of teaching per-formances, a diagnosis of basic teaching skills can be

25

made very easily, and this information can be used tospeed the learning of these teaching skills. We alreadyhave sufficient information on feedback and modelingtechniques to know that the information generated in aperformance assessment system can be incorporatedinto feedback and ,modeling systems to increase theeffectiveness o" training. Thus, we may expect the insti-tution to benefit by achieving economies in trainingtime and the trainee to benefit by achieving compe-tence sooner when he enters upon actual teaching.

Reactive Simulation Devices

have been describing performance assessment pro-cedures which require the teacher to interact with stu-dents in a simulation of actual teaching. There aresome other procedures that appear to be promising inassessing potential teaching competence. We have de-veloped a filmed simulation test that portrays a teacherconducting a class. The film is stopped periodically andthe viewer is asked to say what he or she would do inthis situation; in other parts of the test the viewer isasked to explain what is occurring in the class, and, insome places, he is asked what advice or suggestions hewould give to the teacher. We do not yet have data onthe predictive validity of this measurement, but it ob-viously taps the prospective teacher's attitudes towardstudents, his perceptions of the problems of teaching,and his understanding of some phases of teaching strat-egies. The test situation is sufficiently lifelike to creategreat interest on the part of those who view the film,and invariably the film produces some intense discus-sions about what the teaching portrays, thus suggestingto us that the film also has great potential for trainingpurposes.

Another type of. simulation device has been createdto depict the kinds of strategies teachers use. This pro-cedure is a gamelike device in which an experimenterplays the role of student in a highly controlled way.The teacher arranges the subject matter in the form ofpresentations or questions, and the experimenter re-sponds whenever the teacher asks a question. Thisgamelike situation does discriminate sharply betweendeductive and inductive teaching styles. It seems likelythat the role of the experimenter can be computerizedso that the game can be used by large numbers oftrainees in a more efficient manner.

Both of these techniques are illugratfve of the kindsthat are likely to be developed to assess teacher com-petence. Obviously, their validity in predicting teachingperformance has to be established. But the work to

26

date suggests that they are probably of sufficient valid-ity to be useful in assessing what a teacher is likely todo in a classroom.

The Problems To Be SolvedThe work that I have described here is meant to be

illustrative of the possibilities of assessing teachingcompetence. I emphasized that the problems are pri-marily technical and partially theoretical. Wes'have notyet developed an adequate technology of performanceassessment. but the reason for this is related more tothe lack of experience with these technologies than to alack of knowledge of what to do.. Theo theoretical problems are more complex and willrequire extensive research and development. The kindsof procedures that I have been talking about here canbe adapted to experimentsfor example, assessing therelationship of teaching skills to pupil learning. Care-fully controlled experiments can be designed in the mi-croteaching format to study hypotheses about these re-lationships. The mini-course data can be used toinduce hypotheses about the relationship of teachingcompetence to pupil learning. But until the basic ques-tions about what competencies make n difference havebeen resolved, the design of assessment technology will

be incomplete.Each of the conditions governing an assessment sys-

tem, that I outlined previously, presents a technicalproblem to be solved. These problems are highly. ame-nable to research, and there is no reason to believethat they cannot be solved in a relatively short periodof time.

A basic problem is political in character. Many indi-viduals believe that competence cannot be measuredbecause they do not want competence to be measured:.The solution to this problem is obviously complex.Certainly the development of efficient and demonstMbly useful assessment systems will weaken the ..argu-ments against assessment. Still, there will probably al-ways be those who will say that the most importantaspects of teaching are unmeasurable. But I, for one,have never been able to understand how this conclu-sion can be reached before anybody has tried seriouslyto measure the more elusive aspects of teaching.

ConclusionWhat should state departments of education, who

wish to move to performance-based certification, doabout the assessment problem? Obviously, an enor-mous amount of research and development work re-

Vs

mains to be done, and even though we are optimisticabout the pace at which the research and developmentcan proceed, we cannot at this time offer state depart-ment personnel finished performance assessment sys-tems. For that matter, we all know that just selectingthe competencies that will be the criterion perform-ances cannot be dote at the present time.

I have the impression that state department person-nel are largely concerned, and justifiably so, about thislatter question. I propose that rather than think aboutassessment in terms of what competencies should bemeasured, state department personnel think about thecriteria for evaluating assessment systems from whichjudgments about competence are made. If you agreewith me that it will he several years before we have agood body of research established on the relationshipof teaching competence to pupil learning, then it seemsto me that the most practical course to take is to setstandards for the evidence about competence that is to

'be provided, however that competence may be defined.I urge your consideration of the approach because I

believe that once a faculty in an institution becomesconcerned with the assessment problem, the definitionof competence and the demonstration of its relevanceto student learning will become major concerns of thefaculty and will more likely be tackled quickly and ef-ficiently. But it also seems to me that what state de-partment personnel ought to be asking is what thebases are to be on judgments about competence aremade. If attention is focused on these decision-makingprocesses and the information is used to make decisionsabout competence, we shall at least have guaranteedthat the judgments are not being made subjectiVelythat a multiplicity of factors affecting competence havebeen inadequately controlled or that the range of ateacher's skill has not been appropriately evaluated.The problem for the next several years can be stated-as a cinestion addressed to the training institution,"Given your conception of what constitutes compe-tence, what evidence have you gathered thht demon-strates that teachers have acquired these competen-cies ?"

In my opinion, if state departments would adoptcriteria for the evidence to be presented to supportteacher certification, the performance-based movementwould move forward quickly. You would also avoidthe trap of falling back into approving programs whichhave a performance base but cannot produce evidencethat the trainees who go through the program are infact competent teachers. This approach would focusthe institution's attention on the development of the as-

sessment of competence and would give each institu-tion greater freedom in the development of programs.

Those of us who are interested in the performance-based movement in teacher education accept a basicprinciple: we do not care how a teacher becomes coin-

a.

tQ

..111Y.

, to. 4P.

BEST COPY AVAILABLE

r

27

petent v.ovided that he or she can demonstrate thatcompetence. If we believe in this principle, as I knowwe do, then our efforts should be directed to decidingwhat evidence we will assess to determine whether ateacher is competent.

./ f tt

:1(reiabiemA.

,-.

Conference Participants

I

28

PBTE: Proceed With Caution

By

BARAK ROSENSHINE

As review the current state of knowledge aboutPBTE, ' am overwhelmed by the certainty expressedby state legislatures and state departments of educationwhen they mandate teacher performance criteria as ananswer to educational problems. It is equally over-whelm;ng to read the lists of behavioral competenciesdeveloped by teacher educators or to read the trainingpackages developed to instill' these competencies. Dothese educators and legislators know something that Idon't know, and why hasn't this message gottenthrough to me?

When I look at research on teaching, I am over-. whelmed with uncertainties. This is because systematic,

cumulative research on teaching behaviors and studentlearn ^ barely begun. The results of the research,to date, are best seen as providing suggestions for fu-ture research, not future practice. Research to datetells us very little on effective questioning skillq, for ex-ample, and even less about whether effective question-ing skills are similar across grade levels and subject

areas.Until informed otherwise, l' assert that it is illusion-

ary to mandate teacher performance criteria and expectthat pupils will benefit from teachers achieving a setcriteria on these skills. We simply know too little, and

our research is too contradictory to support such man-dates. Therefore, I would caution leaders in PBTE. to

proceed slowly and with caution. Teaching competen-cies are not matters that can be decided by a poll ofconcerned citizens or be legislated by state legislators.

At the same time, the interest generated by PBTE,and the training which is being provided under PBTE

programs both provide settings and possibilities to cor.-duct research to answer some of the questions which

are becoming increasingly important. The purpose ofthis paper. then. is to attempt to specify research prob-

lems and suggest ways in which such research can be

conducted.

Current Knowledge on T :aching Competencies

Anyone who writes. about teaching competencies is

aware of the lack of research which links teaching be-haviors to beneficial effects upon pupils. My own view

is that it is premature to use this research to derive im-plications for practice or teacher training. This is be-cause there have been relatively few studies of teachingbehavior and student -learning, and most of these stud-ies were done on a small budget with a small sample"of teachers and a small number of observations andoutcome measures. At present, the conclusions are not

ready for translation into teacher training competen-cies. At present, we have only the faith that increased

research.,and improved research will yield knowledgethat can be translated into teacher competencies.

The following is my encapsulated view on the stateof research on teaching competencies and studentgrowth:

a. The research base for building teachingcompetencies is4 extremely thin, because therehave been only a small number of studies whichattempted to relate. teaching behaviors to pupillearning and very few reviews of these studies.For example, although one extensive review waswritten on the correlations between teaching be-haviors and pupils' cognitive 'gain (aosenshine,1971), similar reviews apparently 10e not ap-peared on the correlations between teaching be-haviors and pupil social or cognitive growth. Re-views on experimental studies on teachingbehaviors and pupil learning have yet to appear.

Because of the lack of research cited aboveand the unevenness of the quality of researchwhich has been done, the results of the best ofthese studies are not sufficiently clear to betranslated into performance modules

b. The correlational studies which have been completed and reviewed (see Rosenshine, 1971; Ro-senshine and Furst, 1971) are best seen as pro-viding ideas for future research, not futurepractice. In each of the reviews which I authoredor coauthored there is no section on implications

for practice. Rather, most of the space is de-voted to ideas which might he considered in fu-ture research studies. In the most comprehensiveof these reviews ( Rosenshine, 1971) the presen-tation of the results on each variable is followedby suggestions for future research. These sugges-tions are made not only for results which wereconsistently statistically significant or consistentin trend, but also for results which are, to date,nonsignificant. Thus, investigators who wish tobuild upon current research will have no troublefinding suggestions for such work. Whether thesesuggestions have merit is still to be decided.

c. Although a variety of teacher-training skillspackages have been produced by educationallaboratories, research and development centers,private industry, and private individuals, few, ifany of these packages are accompanied b tech-nical reports which show. that training t chersin these skills (or a 'combination of skit ) re-sults in greater student learning.

The above paragraph is particularly distressingwhen one learns that many of these training ma-terials have been available for 5 years or more,and thousands of preservice and inservice teach-ers have received training in these skills. Yet,when one looks for reports on the effectivenessof such training for helping pupils growth, thereis a lack' of reports. Those few reports which are

29

available were usually dissertations which fo-cused upon 15 to 20 teachers in one location.

'fills lack of research on products developedby agencies other than colleges of education isdistressing for a number of reasons. First, it per-petuateg a familiar educational problem of de,velop and disseminate and let validity lie in theeye of the beholder. If well-funded organizationshave not adequately tested theill products, whatcan be expected from colleges of education?Second, the lack of research on pupil growthdeprives the developer of vital information whichcan be used in modifying the product or advisinghow the product can best be used. Third, itleaves colleges of education in a quandry whenthey have to select appropriate training pro-grams.

d. The lack of research in the past makes it verydifficult to design research for the future becausewe have little to build on. Currently, when ex-perimental studies are conducted and nonsignifi-cant results are obtained, there are any numberof explanations: the design may have been inap-propriate, the tests may have been jnadequato,the treatment itself was not very important, orany of a number of additional possibilities. With-out a research base to which we can refer, wecan barely t.egin to guess which alternative ex-planation ;4 most likely.

Will Research Take Place in PBTE?

Performance-based teacher education' offers `an ex-cellent opportunity for colleges of education to' aid inexpanding and clarifying our knowledge base aboutteaching competencies and student growth. A criticalquestion, however, is whether there, will be an exten-sive research effort or whether the resources \v411 bedevoted to development and a vague promise fol re-search at some future time.

This section is written in the hopes of facilitatingsuch research. It begins with a listing of existingteacher-train ng materials, raises issues about the selec-tion oftieafh:ng competencies, discusses those agencieswhich might cc.nduct such research, and concludes witha few suggestims on research settings and outcomemeasures.

Sources for Teacher Training ProductsAs I survey current work in PBTE, it seems that a

great deal of time is being given to developing trainingmaterials. I question whether this time is necessary. In

my opinion, we already have a large number of devel-oped products which can be used either in presentform or with slight modification in PBTE. Scale ofthese products can be identified fairly easily, and cur-rent work will identify even more.

A major source for identifying training products is"Resources for Performance-Based Education" (Hous-ton, 1973),. a joint product of the New York State Ed-ucation Department and the Multi-State Consortium onPBTE. The book is the most comprehensive compila-tion of materials I have seen. It includes over 3,000.teacher-training products which were developed by pri-vate individuals, universities, publishing houses, andgovernment-sponsored organizations. Anothir source.of teacher training materials is the third edition of theCEDaR Catalogue (1972). This two-volume editioncontains descriptions of products which have been de-veloped by the federally sponsored educational labora-tories and research and development centers. It alsocontains descriptions of products under development,

30 BEST COPS AVIKRBLE

and elaborate cross-references of the products. TheCEDaR Catalogue, however, is currently limitca to theproducts of the 20 organzations in the Council for Ed-ucational Development and Research (CEDaR). Inaddition, the Program on Teaching Effectiveness at the

StanfordResearch and Development Center has been

collecting information on teacher-training products de-veloped by all sources and a publication of their find-ings should be available in late 1973.

.Dr. k

1^.

s-74A --

(t.4tir

Barak Rosenshine

A

If such a large number of teacher-training productsare already available- it would seem unwise' to developstill more. A more efficient 4trategy would be to use ormodify-41ase to meet institutional needs. An exampleof such a program is the one developed at Florida In-ternational University (Sobol, 1973) in which. all ofthe modules represent modifications of existing, acces-

sible material.Because there already exist a large number of

teacher-training products, and because we have littleinformation on the effectiveness of these products forenhancing pupil growth, it seems reasonable to advo-

cate that research on existing products take a higherpriority than the development of still more unvalidated

products.Yet, the evidence to date strongly suggests that still

more development is outpacing research in PBTE. Thisis not surprising because most educational develop-ments of the last decade have been characterized by"implement and develop on a large scale and then do

research." Unfortunately, the research is seldom done,

and by the time we are ready to do serious research,the developers and their public havd become occupiedwith still another "new" but unvalidated innovationand return to more development with another promiseof research at some future time.

Issues in Selecting Training ProductsThe availability of a large number of different

teacher-training products (there appear to be over1,000 modules in "Resources for Performance-BasedEducation," and a larger number of audio tapes, pro-gramed texts, kits, games, and guises) makes the prob-lem of selection extremely difficult. Some questions andissues in selection might be framed as follows:

1. How does one select teacher competencies?2. How does one distinguish between competencies

which are useful and those which are trivial andmisleading?

3. Are these competencies appropriate for all sub-ject areas, all age levels; and all types of stu-dents? If not, how does one decide the most ap-propriate competencies for the particular setting?

4. Which competencies are most appropriate fordifferent contexts (e.g., types of student, differ-ing school environments, differing instructionalmaterials, differing grade levels, and geographicalareas)?

5. Are the same competencies useful for cognitive,social, and emotional growth in pupils? If not,how does one decide the most appropriate blend?

The common practice for answering these questionshas been for people regarded as knowledgeable in thefield to use their experience and knowledge to make abest guess. Because there is a lack of research, theguesses represent the best thing we have. The critical.question then, is whether there will be future, system-atic, cumulative efforts to attempt to answer the aboveissues.

Who Will Do the Research?One obvious locus for research is those institutions

doing the development, particularly the educationallaboratories. My guess would be that if educationallaboratories are also involved in research on the effec-tiveness of their teacher-training products upon pupils,the results of this research would feed back and im-prove the development of present and future materials.

But the major place for research on the teacher-training performance-based packages would be in thoseinstitutionswhich are preparing teachers. Those institu-

Jitions preparing 500 to 1,000 teachers a year wouldhave more than a cursory interest in the effects of theirofferings, not only upon their trainees, but upon thepupils which the trainees will he teaching.

Collaborative research would seem appropriate, par-ticularly, within a state or across a national organiza-tion such as the Teitcher Corps. Within the pages ofthe PBTE newsletter there are a least two examples ofproposed collaborative research. In one, involvingTeacher Corps trainees, a study is planned for the1973-74 school year involving 400 interns in 20 insti-tutions. In this study, approximately 10 measures ofcognitive, emotional, and social groWth of childrentaught by these interns will be collected in the fall andagain in the spring. For each intern, information willbe collected on the extent to which they possess theskills, attitudes, and knowledge that are believed to fa-cilitate the learning and growth of minority group andlow-income elementary school children. A major re-search question will be the relationship of intern skills,attitudes, and knowledge to the learning and growth oftheir pupils.

Another program described in the PBTE newsletterpresents a two-step process.. First, statewide institutionsare to implement a PBTE program which includes as-sessment of whether trainees achieve desired skills,knowledge, and attitudes. In the second step, the insti-tutions are required to_ demonstrate whether the desiredskills, knowledge, and attitudes are appropriate. It isexpected that both steps will be operative before 1980.

Funding PBTE ResearchThe funds for the proposed research can come from

Many sources. One source should certainly be thoselegislatures which are advocating teacher certificationbased upon performance. Another source could bethose educational . institutions which plan to develop-performance-based training modules. Even if modulesweren't available, one would expect that for every $2spent on development and dissemination, $1 would bespent on research to validate the training behaviorsagainst desirable educational outcomes. Another sourcecould be the Federal. Govenment, which, to my mind,has not been allocating a sufficient proportion of fundsfor research into the developments they are funding.

Settings for Research on Performance-Based Skills

The appropriate settings for research on perform-ance-based skills is a tricky question which probably

tt

31

will not be answerable until a good deal of researchhas been done. The major issue is the extent to whichresults obtained in.special settings are generalizable toregular school settings. Special settings have been usedin a number of studies, and these range from 10t-min-ute lessons which student teachers present to speciallyassembled groups of pupils to unique 2-week unitswhich regular teachers present to their regular pupils.The differences in settings, instructional materials, andoutcome measures are such that research will beneeded to determine the extent to which results ob-tained in one setting are generalizable to another.

At present, a case can be made for the utility of allsettings. Small. controlled settings offer possibilities-forintensive study, and findings obtained in these settingscan be verified in more natural situations. Thus, al-though teacher-training institutions would have diffi-culty conducting research in regular classrooms, theycould conduct research by assembling smaller groupsof pupils and noting the effects of trained and un-trained preservice teachers who taught mini-lessonslasting from 10 minutes to five 1-hour lessons.

To date, the number of such studiei which havetaken place on the preservice level is extremely smallless than one per state. It is hoped that colleges ofeducation will recognize the advantage of combiningthe implementation of PBTE with research on the per-formance-based skills.

Matching Outcome Measures to Training

11One of the biggest proble s in conducting researchon teaching, whatever the se ting or the performanceskills, is selecting the appropriate criterion. measures.In many studies, the evidence,is limited to whethertrainees modified their instructional behavior in desiredways. Research which contains trainee outcomes is im-portant, but it cannot substitute for research on whetherthese trainee outcomes increase pupil-learning.

If someone is conducting validation research, it pres-ently seems extremely important to spend a eat dealof time inspecting the match het4pen the to cher per-

. formante skills , and the criterion measu es. Suchinspection and matching can take place two ways. Oneway would' be to start with specific performance skills(e.g., asking higher order .questiont) and make rea-soned guesses on the kind of pupil learning one. can-reasonably expect if these specific skills are imple- ..mented. (One might also consider the student leavingwhich would not be expected.) Then one could focateor develop appropriate learning measure's.

32.. .

A second procedure would be to select pupil out-comes (e.g., social. or emotional growth), inspect theavailable measures in this area, and then develop a ra-tionale for the performance skills`which seem appropri-ate for enhancing such pupil learning. This second pro-cedure hag been'en used to reanalyze correlational andexperimental studies in whi,-)1 cognitive measures arethe outcome. The current evidence suggests a fairly.high specificity of effect. That is, there is a strong cor-

. respondence between the cognitive activites emphasizedduring instruction and those assessed on the cognitiveoutcome measure. If further analyses continue to sup-port specificity of effect for cognitive measures, suchspecificity may also hold for afi 've outcomes. If so,then it would seem wise to inspect the affective out-

. . come measures used in a validation study to see ifthe e is sufficient correspondence between the affective'be aviors stressed in the instructional modules andth se assessed on th' outcome measures.

Currently; we spend relatively little time inspectingt e match between the performance skills' .and. the out-c me measures. Rather, the more common practice haseen to select performance skills from a variety of

. ources, train teachers in these skills, and run some ex-riment to see' whether the pupils of the trained

eachers show greater Icarning than the pupils of the. untrained teachers. The usual result has been.no signif-

. leant differences.(The reader who is interested in further study of the

design of validation studies might consult Flanders(1970, chapter 12), Flanders (1971), anciRoseashineand Furst (1973 ).)

Irrefutable HypOthesesI ,- One hopes that when validation research occurs in

PBTE, investigators and reviewers-will allow for thepossibility that certain pet ideas will not be validated.Unfortunately, there has been a tendency in educationto hang on to all ideas, no matter what the research te-

suits. When faced with studies which do not supportour pet ideas, we frequently claim that if the studieshad been better designed, or if the treatment had lastedlonger, then our ideas would have been validated.When all else fails, we invoke The Educators' Creed:

These are things that tests can't measureThese may be the .most important things of 'allAnd the experimental group did best on these.

When training teachers in performance skills doesnot yield differences in pupil growth there are any num-ber of possible explanations, and these explanations aresources for future research. It seems unreasonable andunpro,'Ictive to develop lines of argument which allowus-to,u.sert the validity of all performance skills whichwe consider important no matter what empirical resultsare obtained. Indeed, given the large number of teach-ing skills which have generated, one would hope thatresearch would help us reduce the number of skillscurrently con.sidqted important.

Summary

In performance-based teacher education (as in otherareas of education such, as the teaching of reading), weare faced with a large number of training and instruc-tional materials, a great deal of interest, and a limitedamount of research. As such, PBTE appears no betterand no worse than the rest of the educational field.

Whether PBTE will be limited to development anddissemination, or wither it will also inclUde necessaryresearch on pupil learning is a critical question whichwill be answered by actions in the next few years.. Theteacher-training materials which have been developedoffer excellent opportunities for experimental research(as contrasted to the correlational research whicr hasdominated the field to date.) One hopes that institu-tions engaged in training teachers will cooperate hi de-veloping a systematic and cumulative knowledge base.

r°

4

14

ReferencesCEDaR Catalogue (3d edition). Commercial-Educa-

tional Distributing' Services, P.O. Box 3711,Portland, Oregon. 1972.

Flandyrs, N. A. "Analyzing Teaching Behavior." los-tont Addison-Wesley, 1970.

Flanderst N. A. A national coordinated progranr of re-search on teaching effectiveness. In "How Teach-ers Make a Difference." U.S. GovernmentPrinting Office, Washington, D.C. Stock No.1780-0813. 1971.

Houston, W. R. and 'others. "Resources for pektOrm-ance-Based Eductition." The State EducatioiiDe-partment, Ilivision of Teacher Education and Cer-

tification. Albany, MX. 1973.

33 S--

Rosenshine, B. "Teaching Behaviors and StudentAchievement." New York City: Humanities Press,1971.

Rosenshine, B. and Furit, N, Research on teacher per-formance criteria. In B. 0. Smith (Ed.) "Re-search in Teacher Education: A Symposium."Englewood Cliffs, N.J.: Prentice-Hall,. 1971. pp.%37-72.

Rosenshine, B. and Furst, N. The use of direct obser-vation to study teaching; in R. M. W. Travers(Ed.) "Second Handbook of Research .on Teach-ing." Chicago: Rand Mc*Nalljt and Co., 1973.pp. 123=184.

Sobol, T. T..."Stildent Handbook for General TeachingSkills Clinic." Miami: College of Education, nor-

International University. 1972.

3

O

C. .

by

34

ry

Alternative Teacher Assessment Strategies*

By

W. JAMES POPHAM

,

At a time when the .Nation.'%entire educational en-terprise is being subjected to increasing public scrutiny,

it is not surprising that the classroom teachergener-ally conceded to be the pivotal figure in most instruc-tional settingsis being evaluated more frequently and

more rigorously than ever before. , Perhaps this

increased stress on teacher evaluation stems frimi thewidespread concern about accountability in education.Possibly it is the predictable concomitant of a gluttedteacher market where employers can finally be choosy,when in former years their chief concern was to get aninstructor, any instructor, to cover every classroom.

Whatever the cause, concerns about teacher evalua-tion have become far more pronounced in the past Jewyears than at any time during this century, even thougheducational researchers have been continually carryingout teacher effectiveness investigationt for well over70 years. The difference in the focus of theseEctivitiesprovides the key to our understanding of why today'stypical teacher starts to perspire a bit when someonemention teacher competence assessment. In the old

days, host toucher effectiveness researchers weresearching for a suitable criterion variable which, if lo-cated, woulsi permit them to isolate independent varia-bles (such as teachers' personality traits, educationalexperience, or instructional styles) that would contrib-

ute to more effective instruction. Such investigationswere accurately perceived by teachers as research in-quiries and, as such, were not viewed as particularlythreatening.. Even in- those instances where the atten-tion of the investigator was clearly directed towardteacher evaluation; few teachers were very concerned.After all, even if defensible assessment techniques werediscovered, it was generally held that teacher evalua-tion efforts would be directed toward helping teachers,never firing them. The American public had great conzfidence in the Nation's public schools; and, although

vto"'* An invited working paper for a meeting of the Multi-State

Consortium on Performance-Based Teacher Evaluation, NewOrleans, February 25-28, 1973.

everyone knew there were differences in the abilities ofteachers, a tenured teacher's position was next to in-

violate. For example, a recent search of California'steacher employment records revealed that during thelast 40 yeare, not one California teacher has been dis-missed on the grounds of incompetence. It is smallwonder that in the past our teachers have not been toothreatened by teacher evaluation activities.

But they are threatened nowand with good rea-son. Dissatisfied legislators in a good many states arebeginning to -enact penalty-laden laws which call formore stringent teacher-accountability. The most cele-brated of these recent teacher accountability laws is theso-called Stull Act (named after its author, Assembly-

man John Stull) passes by the California legislature.during the 1971 legislative session (Assembly Bill293). The Stull Act has 'generated an immense amountof educational activity .among California school peo-.ple,2 for its implications are serious indeed. The new10/ calls for the annualcevaluation of all probationaryteachers and the biennial evaluation of all nonproba-tionary teachers. The evaluations must be made on thebasis, as stipulated by law, of pupil progress according

to district-explicated standards of achievement in allareas of. study. What has happened in California as aresult of the tull Act is that an attempt has beenmade to ope tionalize incompetence so that even ten-ured teachers can be dismissed if they are evaluatedadversely. We can expect to see other state legislatureienacting comparable teacher evaluation laws in thenext few years, particularly if .the California experimentseems to. be working.

But :even if no more states establish teacher ap-praisal; systems, there is still a strong likelihood thatlocal :*districts, perhaps buffeted by school board pres-sures, will set up some sort of teacher evaluation sys-

Personal communication, 'Research Department, CaliforniaTeachers Association, Burlingame, Calif.

c2 See W. J. Popham, "California's New Precedent-SettingTeacher Evaluation Law;' Educational Research, Vol. 2, No.7, July 1972, pp. 1341

tem. In view of these developments at the state. andlocal levels it does not require Irma prescience to beable to forecast an increasing need for the technical de-vices and procedures required for effective teacher tip-

,praisal systeins.

Although it is genc rally assumed by most laymen(and many legislators) that educators currently possessadequate devices fot use in evaluating a teacher's in-structional effectiveness, nothing could be farther fromthe truth. The history of teacher effectiveness researchis replete with failure after failure in efforts to devisedefensible indicators of how well a teacher teaches.Space limitations preclude an exhaustive analysis " ofthe limitations of previously tried assessment schemes.but each of the chief contenders, that is, ratings, obser-vations, and pupil test performance,' have fatal defects.

RatingsBriefly, the difficulty with ratings of teacher effec-

tiveness (characteristically supplied by administrators,but also obtainable from students, peers, etc.) is thatdifferent raters have different notions regarding what itis that constitutes. good teaching. The same teacherwho is rated high by one individual because of su:hfactors as "flexible interaction with learners" and "per-sonable, infornSul rapport with class" may be rated lowby another individual because of "poor discipline" and"classroom anarchy." We all use our private value ma-trix in judging whether good teaching has taken place,and when we try to pool these disparate sets of raterexpectations, chaos is the characteristic result. Howmany times, f4#r example, has a classroom teacher beenrated negatively'by a principal because the"acher wasconducting class in a way other than the manner inwhich the principal recalled his/her lustrous days inthe clasSroom. Yet that same teacher may receive apositive rating by the district office supervisor who hasa different idea of how teaching should be carried on.It has been observed that one person's humorist is an-other person's smart aleck. Similarly, one rater's Mr.Chips is another rater's Mr. Peepers.

ObservationsWith respect to systematic observations of the .teach-

er'S classroom behavior, we encounter an interesting

3 For a more detailed examination_ of the strengths andweaknesses of various teacher effectiveness assessment ap-proaches. see J. D. McNeil and W. J. Popham. "The Assess-ment of Teacher Competence," Chapter 7, second edition, TheHandbook of Research on Teaching, R. M. W. Travers ed.,Macmillan, 1973.

359.

assumption. It runs as follows: if certain process varia-bles can be found to correlate positively with desiredoutcome variables, then by ascertaining whether thoseprocess variables are present, on that basis alone wecan make judgments regarding the desired outcomes.For example, if it is discovered that a teacher's provi-sion of practice opportunities for learners generally re-sults in desirable learner attainment; then proponentsof observational teacher evaluation schemes would Con-tend that we can, at least in part, evaluate teachers onthe basis of the degree to which they provide practiceopportunities.

The trouble with the logic of this approach is itstendency Fe force one to the position that the processvariables scrutinized by classroom observers are notonly necessary for securing worthwhile results withlearners, but that they .are essentially sufficient. For ifthe phenomena observed; e.g., amount of teacher talk,are viewed as means to an end, why not assess.the'endresults directly without encountering the measurementnoise associated with the extra assessment step. For al-though no upstanding classroom observation devoteewill ever assert that those behaviors observed are with-out exception associated with desired outcomes inlearners (such as important cognitive or affectivechanges), the logic of the observation strategy pushesus to place greater reliance on means-end predictive re-lationships than the current sophistication of our obser-vational techniques permits. If we are really interestedin the ends, why not focus our assessment energy onthem?

A second difficulty with observation-based ap-proaches to _teacher appraisal is that although a teachermay display optimal use of the classroom behaviorscalled for in the observation system, there may be dele-terious factors present, factors not built into the obser-vation structure, whose presence will essentially cancelout the positive features of the teacher's classroom be-havior. The only way to head off this assessment diffi-culty would be to build an observation system so ex-haustive that it could pick up all (or most) negativeprocess variables, but by that time the system would betoo vast to be practical..

Another difficulty with observational approaches tothe assessment of teacher effectheness is that whereasthey might prove useful in identifying some classroompractices which in general will yield beneficial resultswith learners, the teacher evaluation game *demandspersonal and particular decisions, not general guide-lines. A particular teacher working toward particulargoals with particular students in a particular setting

A'

36

may break all the process guidelines and yet achievesuperb result.. The particularized interaction effects aretoo subtle for ouf currently unsophisticated observationsystems. There have been several outstanding pro foot-ball quarterbacks whose passing form looked abysmal,yet when thb receiver arrived at the appointed spot theball was always there waiting.

Finally, there is considerable danger that when thestakes are high enough (and job security represents abig bet), many teachers will "fake good." Observationevaluation systems are particularly susceptible to suchfaking, for in these days of openly described criteriawe can expect teachers to know what factors will beinvolved in the observation system. Indeed, any dili-gent and legally informed teachers organization shouldbe easily able to unearth the observation dimensionsinvolved. Having been apprised of % p actices yieldpositive eimluathins, is it so unrealistic to expect thatteachers will tend toward the use of those practiceswhen under observation? Of course, if one wished toemploy constant monitoring of classroom behaviorthrough such devices, as closed circuit television, thensuch fakeability fears would be vitiated, but by thenmost schools would have been closed permanently In-cause of the anti - I984 teachers' strikes.

Pupil Test PerformanceThe chief deficiency with the use of student test per-

formance as an index of teacher proficiency has gener-ally been that the wrong kinds of tests were employed.Since 1900 most teacher effectiveness research inwhich pupil test performance was employed as a crite-rion variable involved the use of standardized achieve-ment tests. Since most standardized tests were designedto serve a different purpose; namely, to permit compar-isons among individual learners (not among teachers)they invariably resulted in a "no significant different"outcome.

The difficulties with standardized or norm - referencedtests, particularly for teacher evaluation, have beentreated elsewhere,' but their two most visible defectscan be briefly identified. First, since commercially de -'veloped standardized tests must serve students through-out an entire nation, the generalized naiure of theircontent coverage is often inconsistent with local curric-

See. for example, W. J. Popham, "Domain-ReferencedMeasurement and Teacher Evaluation," Education Technol-ogy, in press: Robert Glaser, "A Criterion-Referenced Test,"Criterion-Referenced Measurement: An Introduction. W. J.Popham, ed., Educational Technology Publications, EnglewoodCliffs, N.J., 1971, pp. 41-51.

ular emphases. Incongruent measurement and curricu-lum results in misleading data. Second, certain psy-chometric properties of norm-referenced tests (such astheir heavy reliance on producing among-learner vari-ance) leads to tests which are sometimes insensitive todetecting the results of high quality instruction.

In the past few years the development of criterion-referenced (or mastery) tests offers teacher evaluatorsan alternative to standardized tests for assessing an in-structor's impact on learners. The judicious employ-ment of criterion-referenced tests for teacher evaluationpurposes is only beginning to be seriously investigated.

Teaching Performance TestsIn the mid-sixties, the writer had reached a point of

frustration regarding teacher effectiveness assessmentdevices and, after a reappraisal of alternative assess-ment strategies, had proposed the development of analternative approach to solving this problem; namely,through the use of a teaching performance test. Twoseparate projects n were supported by the U.S. Officeof Education, each designed to develop and attempt tovalidate teaching performance tests indifferent subjectmatter fields. While the rationale underlying the teach-ing performance jest strategy, as well as the detailedresults of these two projects are supplied elsewhere,e a

Nief description of the performance test approach can110upplied here.

One of the major difficulties in comparing teachersfor purposes of iflstructor evaluation is that differentteachers have different instructional emphases, therebymaking across-the-board comparisons misleading. Theteaching performance test counteracts this problem byproviding an identical task for different instructors;namely, the ability to accomplish prespecified instruc-tional objectives. The teaching performance test is builton the general premise that one chief reason for ateacher's existence in the classroom is to bring aboutworthwhile changes in students; that is, changes intheir knowledge, attitudes, skills, etc. To the extentthat this is true, then one criterion by which a teachershould be judged is his or her ability to bring aboutsuch changes. By providing identical instructional

Performance Tests of Instructor Competence for Trade orTechnical Education. USOE Cooperative Research ContractNo. 0E-5-85-051; Development of a Performance Test ofTeaching Proficiency, USOE Cooperative Research ContractNo. 3200.

W. J. Popham, "Performance Tests' of Teaching Profi-ciency: Rationale. Development, and Validation." AmericanEducational Research Journal, January 1971, 8 (1), pp. 105-117.

objectives for'teachers, then giving the teachers an op-portunity to accomplish those objectives using whateverinstructional techniques they wish, a measure of theteacher's ability to accomplish given objectives can beprovided. One might wish to argue that the betterachiever of given objectives will also be the betterachiever of his /her own objectives, but this is a ques-tion which can be answered empirically. If one simplydecides that an important criterion of teaching is theability to accomplish instructional objectives, thenteaching performance tests would appear to have someutility in a data-based evaluation matrix.

The steps involved in a teaching performance testare these: (1) the teacher is provided with an explicitinstructional objective (and sample test item) alongwith any background information necessary to becomefamiliar with the subject matter related to that objec-tive; (2) the teacher plans a lesson designed to accom-plish the objective; (3) the teacher instructs a group oflearners, typically a small group of learners for a shortperiod of time; (4) the learners are posttested with anexamination based on the objective. The examinationhas not previously been seen by the teacher but its na-ture is readily inferable from the objective (and sampletest item) previously given to the teacher.

In the USOE-supported research studies describedabove, the purpose of developing the performance testswas primarily research-oriented; that is, it was antici-pated that these devices would be employed principallyfor research purposes such as the identification of rele-vant independent variables. Consistent with that intent,the performance tests involved in those investigationsconsumed a fairly large amount of learner instructionaltime, ranging from 4 to 10 hours. At the conclusion ofthose investigations, it became. clear that if teachingperformance tests were to prove practical for teacherevaluation or instructional improvement purposes. theywould have to be cleveloped for much shorter periodsof instructional time. As a consequence, the writer's re-

.

cent development work with performance tests has fea-tured instruments which take only 15 minutes of in-struct'onal time and are designed to be used with smallgroups o; adults or younter learners. These teachingperformance tests, frequently referred to as instruc-tional mini-lessons, superficialiy appear comparable tothe microteaching exercises developed at Stanford Uni-versity some years back. In rationale, however, theyare quite different. The Stanford microteaching lessonsemphasize the teacher's acquisition of process skills;e.g., good questioning techniques. The instructionalmini-lessons referred to here, on other hand, focus

37

more heavily on the results of the teaching than uponthe instructional procedures themselves.

During the past few years teaching performance testshave been employed both in preservice and inserviceteacher education settings! Generally speaking, theseperformance tests have been of the short duration al-luded to abovt; i.e., 15 to 20 minutes in length. Butwhile these.- devices appear useful in instructional set-tings, for example, in helping prospective teachers be-come more facile at accomplishing prespecified instruc-tional objectives, their utility for purposes of teacherevaluation has been largely unstudied.

In a recent paper Glass.. has proffered the notionthat teaching performance tests may have insufficientreliability to permit their effective use in teacher evalu-ation enterprises. Glass cited several investigations inwhich the reliability of teaching performance tests wasclearly inadequate. Several of the investigations cited,however, had been conducted as doctoral dissertationsor by novice researchers. The reliability of teachingperformance tests is as yet a seriously unstudied mat-ter. For one thing, the teaching performance tests usedin these investigations have been constructed on an al-mo3t opportunistic basis, that is, whatever topics,objectives, etc., have come to the investigator's mind.No attempt has been made to carefully delineate tietruly critical dimensions in teaching performance tests.Beyond that, only one investigator 9 has carefully at-tempted to study the reliability of even these ill-definidperformance tests. Results of is investigation will bereported by Millman at the 1973 meeting of the Amer-ican Educational Research Association. Examination ofthe Millman findings suggests that the reliability evi-dence, once again, is not encouraging. But, as indicatedabove, the nature of the performance test employed inthat investigation was not rigorously explicated.

When this paper was solicited as one of several deal-ing with the "state of the art" in the assessment strate-gies suitable for performance-based teacher education,I had just completed the final draft of an American

1 W. J. Popham, Applications of Teaching PerformanceTests to lnservice and .Preservice Teacher Education. A paperpresented at the annual meeting of the American EducationalResearch Association. New Orleans. February 26-March 1,

1973.

Gene V. Glass. Statistical and Measurement Problems inImplementing the Stull Act, Stanford University InvitationalConference on the Stull Act, October 1972. Palo Alto, Calif.

" Jason Millman, Psychometric Characteristics of Perform-ance Tests of Teaching Effectiveness. A paper presented at theannual meeting of the American Educational Research Asso-ciation, New Orleans, February 1973.

38

Educational Research Association (AERA) paper de-scribing a set of minimal competencies for a perform-ance-based teacher education program.'". I had evensketched alternative assessment procedures for each ofthe competencies. Now I just couldn't bring myself to

10 W. J. Popham, Identification and Assessment of MinimalCompetencies for Objectives-Oriented Teacher Education Pro-grams. A paper presented at the annual meeting of the Ameri-can Educational Research Association, New Orleans, February1973.

BEST COPY AVAILABLE

14,, ra

--14'IPO

ti.tikt

Conference Participants

rewrite the paper or even to subtly paraphrase my orig-inal paper. I try to restrict my paraphrasing talents tothe writing of others, not my own.

Accordingly, in the present effort I have attemptedto focus exclusively on the major assessment alternativesfor teacher competence appraisal. Since if perform-ance-based teacher education programs cannot demon-strate that their competency-armed products are indeedbetter teachers, then the performance-based teacher ed-ucation folk had best fold up their competencies andslip away into the night.

4

" :

1

Applications Of Teaching Performance Tests ToInservice And Preservice Teacher Education*

By

W. JAMES POPHAM

Irrespective of whether the programs are referred toas "performance-based," "competency-based," or"skill - focused," the advocacy of teacher preparationschemes designed to promote measurable capabilitiesof instructors, is beginning to be quite fashionable'among teacher educators. A recent survey ' of Califor-nia teacher education institutions found that two-thirdsof the institutions .participating in the survey indicatedthey were engaged at least partially in competency-based teacher preparation: There is even a multi-stateconsortium 2 on performance-based teacher educationfeaturing such forward-looking statesas' Arizona, Flor-ida, Minnesota, New York, Oregon, Texas, Utah, Ver-mont, and Washington. Surely, if most teacher educa-tors are not caught,up in the competency-based game,then we are at least witnessing the pregame warmups.

Whether competency-based teacher education willbecome something other than one of those ephemeralfads. so common in teacher education temains \to beseen. Although many teacher educa 3rs are quick tojoin the competency-based movement at the verballevel, few are willing to devote the requisite energy todevising the criterion measures without which the ap-proach is only rhetoric. In the recent survey 3 of Cali-fornia teacher education colleges and universities, noinstitution reported satisfactory assessment proceduresfor a competency-based program. As with so many ed-ucational innovations, advocacy is less expensive thanworkable implementation procedures.

A paper presented at the annual meeting of the AmericanEducational Research Association, New Orleans, February 26-March 1, 1973.

I Performance-Based Teacher Education, Vol. 1, No. 5, De-cember 1972, pp. 1, S.

2 Multi-State Consortium on Performance-Based TeacherEducation. c/o Bureau of Teacher Education, New York StateEducation Department, Albany, N.Y.

s Op. cit.

39

Teacher Competencies as Enabling SkillsBut there is another aspect of the competency-based

teacher education movement which is equally intrigu-ing. If you sit in on almost any discussion among pro-ponents of competency-based teacher education, youwill hear them describing the teacher competenciesthey are trying to promote as though they were ends inthemselves. As with many recent religious converts, thefervor of these teacher educators for competency iden-tification has become so ab-coniuming that they losesight of what the competencies are really supposed toaccomplish.

Reducing the problem to its essentials, we can seethat whatever competencies a teacher acquires must beviewed as vehicles for making that teacher more effec-tive. More effective, in this instance, means a teacherbetter able to help learners. Thus, the competencietimost frequently identified by performance-proneteacher educators

toreally en route skills which

should contribute to the _terminal skill of being able tohelp learners. For example, suppose one of the teachercompetencies we are trying to promote involves theteacher's ability to view real or simulated instructionalsituations and identify the extent to which certain in-structional tactics have been employed. We assumethat the teacher who can master such a skill will subse-quently be able to apply this skill in real instructionalsituations. Such skilli should thus be viewed as precur-sive to one's becoming an effective teacher.

Now the point of this distinction between en routeand terminal competencies is that the bulk of compe-tencies currently viewed as the staples of performance-based teacher education are well removed from thosewhich might legitimately be viewed as terminal. Unlessen route skills are constantly verified as ,being actualcontributors to terminal skills, then we have little as-surance that defensible competencies are being pro-moted.

40

Some of the competencies sought by teacher educa-tors should be closer to the terminal proficiencies wewish teachers to display. The purpose of this paper isto explore the teacher education applications of onemeasurement. approach designed to assess such a near-terminal competency. The measurement approachunder consideration is the teaching performance test.

Teaching Performance Tests:Description and Rationale

In brief,' teaching performance tests work as fol-lows: An instructor is presented with one or more ex-plicit instructional objectives (plus a sample test item)and is directed to prepare a short lesson designed toaccomplish the objective(s). If the objective\deals witha topic presumed to be unfamiliar to the teticher, thenrelevant background information is made available.After planning the lesson, tie teacher instructs a group-of learners (either children or adults) for a short pe-riod of time: e.g., 15 minutes. The number of learnedcan be as few as a half dozen or as many as an entireclass. At the conclusion of thejesson the learners aregiven a posttest'

based on the objective. While notpreviously seen by the teacher, the nature of the test isreadily inferable from the objective the teacher hasbeen attempting to achieve. Characteristically, learnersare also asked to rate how interesting the lesson was. Ifsuch an interest rating is employed, the teacher is ap-prised of the forthcoming rating and-encouraged to de-sign a lesson which not only accomplishes the cognitiveobjective, but also promotes positive learner interestratings.

Using these two indicators, an estimate is, providedof the teacher's ability to promote prespesified objec-tives, both cognitive (as reflected by posttest perform-ance) and affective (as reflected by the interest rat-ings). Now it may be argued that an unrepresentativeestimate of instructional prowess is yielded by measur-ing a teacher's ability to promote learner attainment ofprespecified objectives for a small group of learnersduring a brief time period. Yet, while perhaps not asrepresentative of the real teaching world as we mightwish, there are sufficient parallels with reality atsuch an assessment procedure may have utility orteacher educators. In particular, since it more clos ly

4 J. D. McNeil, and W. 3. Popham, "The Assessor ofTeacher CoMpetenie," Chapter 7, The Handbook of Researchon Teaching, R. M. W. Travers, ed., Macmillan, in press.

5 If novel subject matter is employed, no pretest is typicallyemployed. With less esoteric topics, a pretest may be utilizedto identify sufficiently naive learners.

approximates a terminal teaching skill than many ofthe competencies currently being fostered by perform-ance-based teacher educators, it may have advanta-geous instructional and evaluational dividends. The re-mainder of this analysis will (1) set forth threedistinctive applications of teaching performance tests inteacher education operations, (2) &Scribe actual inser-vice and preservice situations In which performancetests have been employed, (3). identify usage guidelinesderived from these experiences, and (4) discuss certainproblems which have arisen in the use of teaching per-formance tests.

Application One:A Focusing Mechanism

It is the writer's belief that much of the edbcationalineffectiveness which exists in our schools can be at-tribute directly to teachers' preoccupations: with in-structional process. Far too many teachers" are caughtup with concerns about devising new and exciting waysof teaching, without ever verifying what. effects thoseprocedures have on children. Some teachers pridefullyannounce that they strive to "teach their class differ-,ently every year," never recognizing that they may beabandoning one year an approach that was truly effec-tive the previous year. Innovations are adulated fortheir own sake. For instance, open schools are cur-rently in vogue. Ten years ago, it was nongradedschools: A decade earlier,, we were praising the rap -'tures of the core curriculuin.

Not that there is anything intrinsically wrong withthese new instructional approaches, for surely theypossess many meritorious features. ,It's just .that toomany educators succumb to the lure of an attractiveinstructional prom without checking the quality of itsimpact on learners: And that, after all, should be thereason we search for better instructional procedures.

HCOCC, as a method of counteracting what appearsto be an almost hereditary concern about instructionalprocess, frequent use of teaching performance tests canprovide a mechanism to focus the teacher's attentionon the effects of instruction. Since in a performancetest situation, the quality of a teacher's efforts is predi-cated on results achieved with learners, both cognitiveand affective, it is difficult to discount the effects of in-struction on Pupils. For example, if you are a prospec-tive teacher who during a semester is obliged to teacha half dozen or more mini-lessons (as short durationteaching performance tests are sometimes called), andthe first concern after your lesson is a determination of

its effects on learners. it is difficult to see how youwould not soon begin to view as important a lesson'simpact on pupils.

This initial application, therefore, is instructional innature. More specifically. it is designed to foster a dis-position on the part of the teacher; namely, a disposi-tion to view' as important the effects of instruction onlearners.

Application Two:A Setting for Testing the Value ofInstructional Tactics

liven though the foregoing application; i.e., as amechanism for focusing teachers' attention on the con-sequences of instruction, may have suggested that at-tention to instructional procedures was somehow repre-hensible, such is surely not the case. We can onlysecure good results with learners if we 'use appropriateinstructional processes. The trick is to apply instruc-tional techniques judiciously in such a way that we caneither verify their efficacy (in terms of effects on learn-ers) or at least be able to make high probabilityguesses that a given technique will yield desirable re-sults with pupils.

A second application of teaching performance testsinvolves their use as a method of allowing teachers totest the differential effectiveness of various instructionaltechniques. Teachers can complete a .series of mini-les-sons attemptjng to incorporate different instructionaltactics, then judge their worth in terms of the resultsyielded with learners. For instance, suppose a teachertaught the same mini-lessons to two different groups ofcomparable learners, the lesson being essentially thesame except that one lesson provided much opportu-nity for learners to practice the skills called for in themini-lesson's objective, while the other lesson providedno such practice. The teacher could thgn contrast theposttest results of both _lessons and begin to reach aconclusion regarding, for certain kinds of instructionalobjectives, the efficacy of providing relevant practice.

Not that a tactic-present versus tactic-absent designmust be employed in this second; application of per-formance tests, fora teacher can often gain insights re-garding the value of a given instructional procedurefrom using the procedure even without the controltreatment. This is particularly true when for a particu-lar performance test there arc some .normative datawhich, even in rough terms, yield an estimate of howwell teachers typically perform on the lesson. Thispoint will be treated in more detail later.

41

An important aspect of this application of teachingperformance tests is that a teacher need not be the ac-tual instructor in a mini-lesson to profit from themini-lesson teacher's experience. A typical format forthe conduct of mini-lessons, either those taught royounger learners or to a grour of one's peers, is thepostlesson analysis session. During this session, theteacher's instructional approach, is appraised in termsof its effects on learners. Many teachers report theylearn as much from watching the mini-lesson teacher'slesson, then analyzing its strengths or shortcomings. asthey do from actually teaching the lesson themselves.

Frequent teacher performance tests, either for teach-ers as mini-lesson instructors or as an objfct for groupanalysis of other's instructional efforts, can provide thefocus of a consequence-oriented preservice or inserviceteachers education program.

Application Three:A Formative or Summative ProgramEvaluation Device

Developmental work with teachinglerformance testsis still at such an early state that it may be imprudentto employ them for the evaluation of individual.teach-ers. The only exception might be for isolating instruc-tors who arc extremely weak or strong in their abilityto accomplish prespecified goals. Nonetheless, as a pro-gram evaluation assessment technique, *performancetests may have considerable utility. Indeed, the mostimportant use of teaching performance tests may be asinstruments to aid in the appraisal of inservice' or pre-service teacher education enterprises.

The argument, briefly, is that if a teacher educationprogram sets out to promote, teachers' abilities to ac-complish prespecified objectives, then the program can,be legitimately evaluated in terms of its abilizy to doso. Here's how the evaluation strategy might work. At,the outset of a teacher education program,; e.g., a pre-service credential sequence or an extended staff devel-opment institute, a representative sample (or all) ofthe participating teachers could complete one of twodifferent performance tests (e.g., test X and test Z,with half the teachers completing test X 'and half testZ). At the close of the program, the teachers wouldcomplete the other performance test. The predictionwould be that for both tests the teachers' post-programefforts would produce markedly better results than thepre-program .efforts. As a summative evaluation strata-gem, such an approach could yield devastating infor-mation. Whathappens, for instance, if a teacar educa-

42

don program discovers its teachers are no better ableafter intensive instruction to promote learner masteryof objectives than they were prior to instruction?Surely dra'stic changes in the program seem warranted.Perhaps, if such failures are recurring phenomena. theprogram should be terminated.

If performance tests are emplOyed at. earlier stagesof the program, with a view to guiding program modi-fications during t 'Se course of the instruction, thenformative evaluation benefits can also be derived fromthe use of teaching performance tests.

In review, then, we have briefly examined three pos-sible applications of teaching performance tests inconnection with either preservice or inservice teachereducation efforts. There will, of course, be other usesof perfOrmance tests in teacher education. For exam-ple, a professor might use teaching performance testsas a motivating mechanism, showing %vice studentsthat more skilled teachers can outperform beginners insail instructional situations. But focusing for the mo-ment on the three application strategies described thusfar, we can examine some actual utilizations of teach-ing peormance tests.

Actual lzrrice and Preservice ApplicationsThe I n Complex. One of the first reported uses

col teaching performance tests to aid practicing teachersOak place during 19%1 in the Jordan Complex, an af-filiated group of urban schools in the Los Angeles CitySchool. District. Under 'the leadership of La,VerneParks, complex director, groups of teachers met weeklyafter regular school hours to witness each other takingturns teaching mi ii-lessons, then discussing the meritsof the teaching approaches. Lessons were dividedabout evenly between those designed for young chil-dren (volunteer pupils were used as learners) andthose designed for adults (teachers participating in theprogram took turns serving as learners).

Reactions os the program, extending over severalmonths, were quite positive. Merle Williamson andJoyce Cooper, the class leaders, gathered anonymouslysupplied course evaluation data indicating that 84 per-cent of the participating teachers felt the class hadhelped them (1) understand the role of instructionalobjectives, (2) plan lessons for given periods of time,(3) develop alternative instructional strategies, and(4) critique lessons on the basis of posttest results.Eighty-seven percent of the participants reported thatthe class had helped them personalize instruction. Mrs.Parks remarked, in reviewing the experience, that

One of the most exciting outcomes. from this coursewas that teachers began to look more critically atthemselves and their peers in terms of factors contrib-uting to successful pupil results."

UCLA. As might be anticipated because of the writ-er's affiliation, use of teaching performance tests in theteacher education program at UCLA over the pastyears has been fairly extensive. All three of the appli-cations discussed in the prezious section have been em-ployed, with attention generally given to the use ofperformance tests as an instructional intervention.

Typically, teaching performance tests have bqnused with preservice candidates, usually involving Xi-sons taught to other members of the class. The settingfor these lessons is ordinarily referred to as a mini-les-son clinic and features the customary teaching-testing-analysis model. Usually one or two mini-lessons aretaught during a 2-hour clinic session. During someterms we have tried to squeeze three mini-lesson as-signments into a single 2-hour lab period. On other oc-cessions, we have required the prospective teachers togenerate. their own objectives and tests, then try to ac-complish the objectives in a short-term lesson, as anadditional exercise in. promoting learner goal attain-ment.

The most recent procedure we have employed forusing teaching performance tests is described in some-detail in the appendix. Briefly, it involves the use ofweekly nine-student mini-lesson clinics. during whichone preservice credential candidate teaches six class-mates while two classmates plus a teaching assistantserve as instructional analysts. In addition, each cre-dential candidate is obliged to teach at least threemini-lessons outside of class time to small groups ofadults. Thus, in a 10-week academic quarter, prospec-tive teachers have about two dozen opportunities toserve as mini-lesson teachers, analysts, or learners.

As our use of mini-lesson clinics has increased, therehas naturally been great interest in the manner. inwhich the teacher education students were receivingthem. At the end of the fall quarter, a quarter duringwhich mini-lessons were employed according to thescheme presented in the appendix, students were askedto supply anonymous evaluations of the course at itsconclusion. Little structure in the evaluation form waspresented to the students, only requests to isolate partsof the course they liked most, liked least, etc. Of the58 students who mentioned the mini-lesson clinics, 32were positive and 26 were negative. In view of the fact

-

6 Personal communication to the writer, May 1.5, 1972.

that none of the mini-lesson clinic leaders possessedany prior experience with mini-lesson clinics, either asparticipants or supervisors, hence were probably lessskillful as clinic leaders than might have been wished,these results are not distressing.

One of the hopes in setting up mini-lesson assign-ments so that students were obliged to teach the samemini-lesson a second time (outside of class time) wasthat the mini-lesson, teacher would profit from theclinic critique session. Hopefully, the insights gainedfrom the analysis of the teacher's lesson would lead to'instructional improvements when the mini-lesson wasretaught and a comparable posttest form was itser toassess learner. .achievement.

Table 1. Results of Mini - lessons RetaughtAfter Initial Mini-lesson Clinic Analysic

. . .._. .

FirstLesson

Superior

_

NoDifference

SecondLesson

Superior

Cognitive Posttest

Interest Rating

n=18 n=6' n=31

n=13__,

n=9 n=33

Happily, this appears to have been the case in thefall 1972 UCLA situatioI for, from the reports sup-plied by class members (to which no grade credit wasapplied), both cognitive and affective performances in-creased more frequently on the retaught 'lessons thanthey decreased, as can be seen in table 1. Of course,these data are only suggestive in view of the fact thatthere . were no controls exercised over the ability oflearners in the retaught or initially taught lessons.

Recently we have also used performance tests inconnection with program evaluation. During the fall1972 quarter, six mini-lessons were randomly assignedto approximately a dozen credential candidates (froma particular teacher education course) at the beginning.of the quarter and to a different group of credentialcandidates at or near the close of the quarter. In grossterms, the prediction was that the performance of theseprospective teachers would be better later in the quar-ter, presumably after the impact of the teacher educa-tion program had worked its "beneficial" effectg.

Since the mini-lessons were to be taught to class-mates, care had to be taken not to involve any studentsin mini-lessons at the first of the quarter (either vsteachers or learners) which they would encounter atthe;lose of the quarter. Siij,ce six different mini-lessonswefe used, this posed no problem. Althot(gh it hadbeen planned to haveihree teachers per mint-lesson on

BEST COPY AVAILABLE

43

, $

W. James. Popham

both an early-in-course and late-in-course basis, withstudent attrition, missed assignments, etc., a less bal-anced performance ;distributibn resulted. The mini-les-sons were taught to groups of approximately six learn-ers (according to the procedural scheme described inthe appendix), with the early-in-course mini-lessonsoccurring during the second week on the quarter. Ide-ally, the first week would have been used for the pre-test mini - lesson, but course organizational requirementsdittated that the second week was a more reasonablechoice. Mini-lessons completed during weeks eight,nine, or 10 of the 16-week quarter constituted thelate-in-course measures. Since all mini-lessons werecompleted on Fridays, this means that prior to the latecourse mini-lessons the credential candidates had typi-cally experienced a minimum of about 8 weeks of in-struction. The teacher education program variationunder consideration involved daily 2-hour class ses-sions, including lectures, discussions, observations inpublic schools, and the mini-lesson clinic activities.

Results of the early course versus late course meas-urement arc presented in table 2 for both the cognitivemeasure (posttest percentage correct) and affectivemeasure (interest rating). All data included in table 2are mean results for a given mini-lesson teacher, typi-cally based on an n of five or six classmate- learners: Itshould be noted that since six different mini-lessonswere employed, with different revels of difficulty andinterest associated with each, interpretation of the datashould focus on the columns of the table.

44

Table 2. Mean Results of Early-in-Course and Late-in-CourseMini-lessons Taught by Credential Candidates in a

UCLA Teacher Education Program

Cognitive Results

Early-in-Course

Late-in-Course

\A4-ective Results

Mini-lessons

F-; 1 -3 i 4 5 6

Posttest Percentages Correct

7472 90 36 84

9082 60

7480 96

88 73 34. 86 82 8290 95 "50 9792 65

67

Minielessons

2 3 4 I 5 6

Ratings of Lesson's Interest(5.hi. 1. low)

Early -in -Corse

1.44e-in-Course

2.7 3 . 8 3.8 3.6 3.0 .2..43.2 3.8 3.83.4 4.0

3.0 3 . 3 3.t 4.0 3.2 4.63.5 /I. 5 5.04.0 4.1

An examination of table 2 will reveal that the pre-dicted results were supported by the data. For both in-dices; i.e., mean posttest percentages correct and inter-est ratings, the late-in-course performances exceededthe early-hi-course performances.

By converting the results for each performance testto standard scores for that measure, With a mean of 50

0

and a standard deviation of 10, it was then possible topool the data from all six performance tests and com-pute separate t tests for both the cognitive and affectiveresults. A summary of these analyses is presented intable 3 where it can be seen that a significant differ-ence was present in both instances, with'a mean differ-ence of .83 standard deviation unit present in the caseof the cognitive measure and .97 standard deviationunit for the affective measure. Both differences, ashoped. favored the late -in- course teaching performancejests.

This represents the initial time, at least to the writ-er's knowledge, that teaching performance .ests havebeen used in this manner as a teacher education pro-gram evaluation technique. It is obvious, even from ex-amining the data table, that some refinements are inorder; e.g., the disparate n's in the columns, etc. Never-theless, as an illustration of the use of performancetests for program evaluation purposes, the foregoingdescription may be of some utility.

California State University, Northridge. During sum-mer 1972, two inservice workshops for approximately50 teachers and Aninistrators were sponsored by Cal-ifornia State University, Northridge in coordinationwith Administrative Area K of the Los Angeles CitySchools., Clare Rose, the instructor for both workshop,reports that teaching performance tests were used inthe workshops as a vehicle for achieving one of theworkshop goals; namely, that participants would beable to supervise objectives-based instructional im-provement programs.

A demonstration mini-lesson was presented in eachworkshop, followed one week later by having all work-shop participants serve either, as mini-lesson teachersor students, as particular mini-lessons were taughttwice by six volunteers. Prior to the second teaching of

Table 3. Analysis of Early-in-Course and Late-In-Course Mini - lessonResults, Pooled on the Basis of Standard Score Transformations

Cognitive ResultsEarly-in-CourseLate-in-Course

Affective ResultsEarly-in-CourseLate-in-Course

* One-tailed.

n

1113

1113

45 . 4653.77

44.7354 . 46

S.D.

7.869.28

5.5910.90

Xdift.

8.31

9.73

t

2.25

2.57

Ps

< .025

< .01

6 8

each mini-lesson, an analysis of the initial lesson wascarried out, Professor Rose reports that 90 percent ofthe mini-lesson teachers were able to promote higherposttest results on the second lesson. She indicated that100 percent of the workshop participants reported, onanonymous end-of-session evaluation forms, that themini-lesson activities had been valuable.

Professor Martin Levine has also employed teachingperformance tests in preservice teacher education forseveral years at California State University, Northridge.Most recently, Professor Levine reported the followingformat for his use of.perforkance tests: .

tai preservice secondary education meth.ods course meets qn the campus of a par-ticipating junior high school, usually forthree hours weekly (e.g., Wednesday 9-12). One or two classes of secondary pu-pils are assigned to the college methodsclass by the principal. Usually pupils inthese classes are classified as "low abilitylearners",in need of more individual atten-tion. Pupils:are assigned at random to col-lege trainees who are responsible forachieving, prespecified instructional objec-tives issued by the- instructor. Each traineeteaches his miniciass of three pupils forone hour during each weekly meeting ofthe course. Objectives deal with generalstudy skills and thus are appropriate forboth _pupils who need to master this kindof objective and for college trainees whocome from all of the different academicareas commonly,. etaught in secondaryschools. Trainees have a week to prepareto teach each objective. A teaching timelimit of thirty minutes or less is usuallyset. The instructor administers a ',pretestand posttest. Trainees analyze their teach-ing effectiveness in terms of how well theirpupils achieve the objectives. Traineesmay, re-teach objectives during subsequentMeetings in 4.ases which warrant it. Fromtime to time, trainees combine miniclasses,with one member teaching to a prespeci-fied objective while his peers observe thelesson for use of such instructional princi-ples as practice and feedback. A post-observation conference is held immediatelyafter the lesson. Usually instruction takesplace in a large area stich as the oral artsroom or the school cafeteria where the in-structor can monitor the entire process as-sisted by the master teachers.

User Guidelines

Based on Our limited experience to uate, thereap-pear to be a few guidelines which might be of value to

teacher educators considering the utilization of teachingperformance tests.. Some general suogestOns regardingthe use of petten fiance tests are available elsewhere."

First, it has become apparent that the capability ofthe mini-lesson analyst (when group, use of perform-ance tests is involved) is far more critical than hadbeen anticipated. Unless the person supervising theanalysis session is both convinced of the value of theactivity and able to provide instructional insights(when, for example, poor learner performance oc-curs), then the mini-lesson play be far less profitablethan possible. Too many teachers will write off mini-lesson teaching efforts, particularly if poor perform-ances occur; as unrepresentative of their performancein a real 'teaching situation. TR'S, one-of the supervi-sor's missions is to clarify the parallels .of the mini-lesson activity and regular'classroom teaching. Further,a deft instructional analysis can provide an unsuccessfulmini-lesson jeachei with a promise of future successduring retaught or other subsequent lesson's. The mini-lesson supervisor must eliminate the frustration thatfollows failure if no improvement plan is presented.Skilled supervisors are so important that, in the writer'sview, unless they are available in sufficient numbers(or can be trained in time) group-type mini-lesson ac-tivities should not be undertaken.

A second point relates to the use of mini-lessonsawith children or adults. Reports to, date suggest thatwhile a steady diet of mini-lessons for, young learnersis palatable, exclusive use of adult learners (el.,teacher educator classmates or colleagues)As less ac-ceptable. Teacher educators who, for a variety of prac-tical reasons, may prefer- to rely on mini-lesstmsloradult learners, should strive to provide, as a change ofpace, a few mini - lesionsns involving younger learners.

Third, there seem to be some discernible dividendsassociated with providing some type of normative data,even roughly displayed, against which to interpret one'sperformance as a mini-lesson teacher. Without suchcomparative data, the teacher or supervisor is hard putto tell whether a given performance is good or bad.Referring back to table 2, it can be seen that there areclear differences, both in ifficulty and probable inter-

. est, in certain of the ini-lessons. How is the mini-lesson teacher to know whether his/hers was a welldesigned and execute lesson if no interpretiveframeWork is provided? We need to supply mini-lessonteachers with what any golfer needs to make the game

In the teaching improvement kits distributed by Instruc-tional Appraisal Services, Box 24821, Los Angeles, Calif.90024.

t

46

more meaningful, a rough notice of what constitutes apar performance."

Of course. normative data are not absolutely indis-,

pensable, for when a mini-'esson tsaeher discovers thathis or her learners are all scoring around 50 percenton a task for which 90-100 percent proficiency hadbeen anticipated, then the teacher at least has theknowledge that expectations have not been attainedand that's better than nothing. But comparative dataare really a big help.

Problem's .

Teacher preformance tests, when employed as re-sources for teacher education evaluation or instruction,are in their infancy. Not surprisingly, therefore, a scoreof diaper - related difficulties have already arisen. Anduntil a definitive Dr. Spock volume arrives to deal withour dilemmas, we'll have to do some trying and, unfor-tunately, some erring.

One of the most basic problems facing wouldlbeSsers of performance tests stems from the instrumentsthemselves. We have not yet made an acceptable effort

to delineate the defining dimensions of performancetests, in terms of their content, objectives, posttest na-ture, background information, difficulty level, etc. Al-most all of the recently developed performance' testshave been devised more or less on the basis of experi-ence and intuition. This situation needs to be rectifiedwithout delay.

A related problem is the reliability of the' teacherperformance tests themselves. We do not yet have suf-ficient data to know how many mini-lessons a teachermust attempt before we can assess the teacher's overalllevel of competence on such tasks. It is expected that ateacher will not rate consistently high, say, on allmini-lesson attempts because his teaching score will de-pend in part on the subject matter of the lesson, inpart on his own teaching approach to that lesson, andin part on how well other factors have been controlled.We ate optimistic that teachers on the ends of thecompetency continuum can be isolated for special assist-ance or special commendation.

In the absence of clearly posted signs indicating pars (perhole) of three, four, or five, the writer's early experiences ona golf- course would have suggested that an acceptable numberof strokes per hcile was something closer to 10, 15, or 20.

cit

A minor problem, but one which can yield trouble-

some implementation difficulties, is the necessity to se-lect topics for mini-lessons which will be viewed asimportant, both by the teachers who carry out themini-lesson and by the students who are taught. In aneffort to identify novel topics, thereby eliminating theneed for pretesting, a few topics have been chosenwhich are so esoteric as to yield atypical (or all tootypical) student apathy. Perhaps it may be wiser to se-lect some main line curricular objectives and go to thetrouble of pretesting prospective students in order tolocate a suitably unknowledgeable learner group. Suchmini-lesson topics might then be viewed as more'mean-ingful by both teachers and learners.

Finally, the logistics problem associated with properuse of performance tests should be anticipated. An ex-amination pf the gep-by-step details given in the ap-pendix will .reveal the level of organizational planningneeded to head off confusion. For instance, it is highlydesirable to provide teachers of unsuccessful mini-lessons with an opportunity to replan and then reteadi?the lesson to a different group of learners. But providling these reteaching opportunities takes a good deal ofplanning time. Faced with such planning frustrations,many teacher, educators will be tempted to return tothe less tasidg, but perhaps less effective, lecture-dis-cussion classroom format. Anticipating logistical prob-lems can help avoid them. A competent secretary orteaching assistant ,can alleviate logistical distractions by.

working out the organizational requirements in ad-vance.

ReviewIn retrospect; an attempt has been made in this

paper to discuss possible applications of teaching per-formance tests to the activities of teacher educators. If

a bias in favor of such applications was reflected in thepaper, this was only natural, for such is the writer'sbias. Although still a rather primitive tool, the per-formance test may be a valuable instrument to teachereducators. As we look at ancient man's hand axes, wemay view them as incredibly simple devices, yet theirimpact was enormous. No strict analogy is being pro-posed here, only a plea to consider teaching perform-ance tests as an additional tool in our teacher educa-tion kits. Think of how many sabre-toothed tigers we

might slay.

a

tr

Annotated Bibliography ofTeaching Performance Test Articles

Baker, E. L., "Relationship Between Learner Achieve-ment and Instructional Principles Stressed DuringTeacher Preparatiob," Journal of Educational Re-search, November 1969, 63(3):99-102. (A studyis reported in which teaching performance testsare employed as a dependent measure in relation-ship to several independent instructional varia-bles.)

, Teaching Performance Tests as DependentMeasures in Instructional Research. A paper pre-sented at the annual meeting of the American Ed-ucational Research Association, New Orleans,February 25-March 1, 1973. (Techniques forusing teacher performance tests in instructionalresearch are described along with illustrative data

. typifying such applications.)

Belgard, M., Rosenshine, B., and Gage, N. L., "Explo-rations of the teacher's effectiveness in explaining,in Be4 lack, A.A. and Westbury, L" (Editors),Researcih Into Classroom Processes: Recent De-velopments and Next Steps. New York, N.Y.:Teachers College Press, 1971, pp. 175-217. (For-ty-three teachers taught two different 115-minutelessons to the same group of pupils. The correla-tion between residual class means was .47.)

Connor, A., Cross-validating Two Performance Testsof Instructional Proficiency. University, of Califor-nia, Los Angeles. USOE Project .No. 8 -1 -174,1969. (This investigation, conducted as part of apost-doctoral fellowship project by Connor, exam-ines the correlation between the performance of17 experienced teachers on two different perform-ance tests. A low, positive relationship was re-ported.)

Glass, G. V., Statistical and Measurement Problems inImplementing the Stull Act. A presentation at theStanford Invitational Conference on the Stull Act,October 1972. (In this paper, Glass critiques anumber of previous studies regarding teachingperformance tests and offers the view that theircdrrent reliability is not adequate to warrant usein the evaluat;on of teachers.)

instiz, T. B., A Method of Identifying the Effective

47

Teacher. Doctoral thesis: University of California,Los Angeles, 1969. (Univ. microfilms. No. 29-3022-A.) (This investigation assessed the degreeof correlation between the performances of nineteachers on two 'different performance tests in-volving different pupils. A rank order correlationof .64 was reported. A second study was also de-scribed in which five teachers taught two differentperformance jests to the same students. A highrank order correlation was reported for the hitteranalysis.)

Levine, M.. V., "The pct on Pupil Achievement of aCriterion-Refereed Instruction41 Model Usea byStudent Teachers," The Journal of Teacher Edu-cation, Vol. 23, No. 4, Winter 1972, pp. 477-481: ( A 75-minute teaching performance test wasused as a criterion measure to contrast theefficacy of alterntive teacher education ap-proaches.)

McNeil, 3. D., "Performance Tests: Asliessing Teach-ers of Reading." The Reading Teacher, 1972, pp.622-627. (In this article, a teaching fair is de-ssribed in which prospective elementary teacherstaught different groups of three elementary schoolpupils in consecutive teaching situations. The in-vestigator reported a tendency for teachers to dis-play 'relatively stable proficiency on such tasks.)

D., )and Popham, W. J., "The Assessmentof Teacher Competence," Chapter 7, The Hand-book of Research on Teachilit, R. M. W. Trav-ers, Ed., Macmillan, in press. (1 his is a compre-hensive xeview of research and evaluationstrategies involving teacher effectiveness. Teachingperformance tests arc examined along with com-monly employed teacher competency" indicatorsaccording to various criteria for use in researchand evaluation.) v

Millman, J., Psychometric Characteristics of Pqrform-ance Tests of Teaching Effectiveness. Pap& pre-sented to the American Educational Research As-sociation, New Orleans, February 1973i(Describes a number of small scale investigations .

-carriers out in Pasadena, Calif., in 1971-72 in

48

effort to study the psychometric properties ofteaching performance tests.)

, "Teacher Effectiveness: New Indicators for anOld Problem," Educational Horizons, Spring1973. (A readable rationale statement endorsingthe use Of performance-based measures of instruc-tor skill.)

Popham, W. L, Evaluating Instruction. EnglewoodCliffs, N. J.: Prentice-Hall, Inc., 1973. (Twochapters of this book are self-instruction programsdesigned to provide the reader with skills neededto construct and to use teaching performancetests. ), "Performance Tests of Teaching Proficiency:RatiOnale, Developntent, anti Validation," Ameri-can Educational Research Journal, 1971, 8, 105-117. (This article describes the background of

.e.

teaching performance tests and the development\ of such tests in the fields of social studies and vo-

cational education during the mid-sixties. At-tempts to validate the tests are reported and thevalidation strategy analyzed.)

0

"Teaching Performance Tests as a Vehicle forInstructor Self-Evaluation," The National Elemen-tary Principal, Vol. 52, No. 5, February 1973.i(This article describes alternative techniques ofemploying teaching performance tests as instru-ments foOnstructor self - appraisal.)

, "Teaching Skill Under Scrutiny," Phi DeltaKappan, Vol. 52, No. 10, June 1971, pp. 599-602. (This article examines alternative 'techniquesfor assessing teacher skill and describes .the possi-ble 'role of teacher performance tests as an alter-native to ratings, observations, and pupilperformance on standardized tests.)

Appendix

TEAM S MINI-LESSON

CLINIC GUIDELINESUCLA Teacher Education Laboratory

Fall 1972

Commencing with the second week of the quarter,each student enrolled in team S will be required toparticipate either as an instructor or a learner in aweekly, mini-lesson clinic. The procedural elements ofthat participation will be outlined below.

One Booklet RequiredDuring the first week of the quarter, when enroll-

ments have settled down to some extent, a mini-lessonclinic assignment list will be distributed. Nine studentswill be assigned to a particular mini-lesson clinicgroup. Three of the students in each group will be dim-ignated- as the A' group, three as the B group, :andthree as the C group. (These arbitrary designations donot reflect the instructors' grading vision and will haveno relationship to the final grade earned in thecourse.) Each student should, without delay, then goto the student bookstore and purchase one copy of the

>.

adult form Teaching Improvement Kit, which bears thesame letter as the one he or she was assigned. For ex-ample, a student in mini-lesson clinic No. 13 who isdesignated as a member of the B group should. pur-chase a copy of Form B of the Adult Teaching Im-provement Kit. Please note that there are some optiOnalTeaching Improvement Kits available in the bookstorefor use with children, but that the required TeachingImpro 7ement Kit, one of the three. forms available(i.e., A, B, C), is an adult kit. With these materials inhand, the mini-lesson clinics will get underway the sec-ond week of the course in the assigned rooms.

One mini-lesson will be taught at each clinic, thusthe individual assigned to teach on a given date mustbe present and prepared to teach. Failure to do so wiltresult in a severe grade penalty. Yet, excused absenceswill occur. Therefore, all students should be ready toteach one week early in case of an unanticipated ab-sence by the regularly assigned teacher.

General Nature of the ParticipationTo provide a brief overview, an individual student's

responsibilities will he described. First, a student willbe obliged to instruct the six students in his mini-lessonclinic group who have been assigned other letters thanhis/her own. For example, a C member of the mini-les-son group wIluld be obliged to, teach a lesson from theC., Teaching Improvement Kit toJthe,six A and 1.3 stu-dents in the group. Seconcideach student must aft as a

.critic for two mini- lessons taught by the members. ofthe mini-lesson group who possess the same letters. Toillustrate, a B member of mini-lesson clinic group No.7 would not only teach one B mini-lesson him/herself,but would serve (along with another B member) as acritic while two other B lessons are being taught to thesix A and C students. Finally, each student will beobliged to locate one or more groups of at least ?kmadults to whom each of the three mini-lessons in his orher teaching kit can be taught, using a form of theposttest other than that employed in the mini-lessonclinic. Thus, in summary, each student will teach onemini-lesson to team S classmates, critique two othermini-lessons being taught to team S classmates, teachthree mini-lessons to adults other than team S class-mates, and serve as a learner for six mini-lessons.

Procedural SpecificsNow, in more detail, here are the step-by-step pro-

cedures to be followed by each student. First, consultthe mini-lesson assignment sheet to note which group,letter, and which week you have been assigned. Nowread the first three chapters of the Teaching Improve-ment Kit (pp.I-15). You may wish to examine themini-lesson assignment 'information in chapter 4. Donot examine the posttests. (on green and blue sheets)which are included 'n the Teaching Improvement Kits.The mini-lesson clip e sessions have not been assignedto influence your gr de in the course, but are specifi-cally intended to hcl you improve your instructionalskills. As a conseque ue, examination of the posttestsprior to your tcachin the lesson would reduce thelikelihood that the mini-lesson would be beneficial toyou.

1.On the day designated on the assignment sheet, you

will be obliged to teach one of the three lessons inyour improvement kit (1, 2, or 3). On that day youwill instruct (for no more than 15 minutes) the sixmembers of your group who have been assigned otherletters; e.g., for the B students this would he the A andC students. The two members of your group who have

49

been assigned the same letter will sit somewhat apartfrom the student group and will attempt to analyze thequality of your teaching plans, activities, etc. At theconclusion of the 15-minute lesson, the perforated cop-ies of posttest 1 (green paper) should be removedfrom your Teaching Improvement Kits and distributedto your six students. You should not have seen the post-test prior to this moment, These tests should bequickly completed and Stored by the mini-lesson clinicgroup leader. On the basis.. of average scores on(1) the interest rating and (2) the posttest, diseussionof the teaching should be conducted in terms of the re-sults produced. That is, good learner performanceshould result in a discussion focused on instructionaltactics which seemed effective; poor results should leadto a discussion focused on instructional procedureswhich might be altered. To assist you in judging howeffective a given teaching performance was, at the rearof each Teaching Improvement Kit data are availableregarding how successful other teachers have been witheach mini-lesson. Results 'of each student's perform-ance will be turned in, but not for grading purposes,by the mini-lesson clinic leader.

On two occasions, therefore, each student will beserving not as a learner but as a critic. On those twooccasions the student (having access to the TeachinglitImprovement Kit under consideration) will e hadan opportunity to examine the mini-lesson ass mentprior to the clinic. He or she will. undoubtedly havesome thoughts regarding an appropriate instructionalprocedures This may be beneficial during the post-lesson analysis session. It will be useful to have two peo-ple, other than the clinic leader, who are as conversantwith the mini-lesson requirements as the teacher. Dur-ing the analysis discussion, it is anticipated that themini-lesson clinic leader and the two critics will takeprimary responsibility for isolating elements of the les-son that were particularly effective or ineffective. Re-member that the appraisal of instructional meansshould be made chiefly in terms of learner results; thatis, the averaged interest ratings a *I posttest scores.

The final responsibility for each m S student is toteach all three of the mini-lessons in his or her kit toanother group of adults; that is, someone other thanteam S students. Since each of the mini-lessons in theTeaching Improvement Kits has two equivalent post-tests, the team S students should not, either deliber-ately or inadvertently, examine posttest 2 (bluesheets). Ideally after the original teaching of thatmini-lesson in the mini-lesson clinic, each student .,

should locate a group of at least three adults (friends,

50

relatives, or people off the street) who would be will-ing to serve as students for the approximately 20 min-utes involved for this assignment (15 minutes of teach-ing plus 5 or so minutes of testing). The mini-lessonshould he taught:and results summarized on the Mini-lesion Posttest 2 Summary Report Forms which havebeen provided (see attachment). The informationcalled for is brief, yet should be completed in its en-tirety. These Mini-lesson Posttest 2 Report Formsshould be turned in weekly during the 9 o'clock classmeetings. Iii other .words, an individual team S stu-dent should have an opportunity to reteach the mini-lesson he originally taught to team S students after acritique of the first lesson. Hopefully, this analysis willbe useful in promoting improved learner performance.The other two lessons in the kit will, for that student,be taught for the first time. Ideally, the discussion of aclassmate's teaching of that same lesson will prove use-ful in helping devise an effective lesson.

Other ConsiderationsResults of the weekly mini-lesson clinics will be

made available so that students who wish to compare

Your name

their performance with that of other team S membersmay do so. As indicated previously, there are other,optional Teaching 1.raprovement Kits available in thebookstore for use with younger learners (elementiryschool age). Furthermore, mini-lessons in the adultkits can be used with mature secondary school stu-dents. Thus, it is possible to teach mini-lessons fromanother kit (other than the letter assigned) to highschool age students. Either of these activities; that is,teaching mini-lessons to elementary or secoschool .youngsters, should be considered option forteam S students. Remember, the whole purpose of the,.mini-lesson clinic operation is to improve the team Smember's skill in accomplishing prespecified objectiveswith teaching procedures which are also interesting tothe learners. The amount of time that yoil can devoteto promotion of this particular competency will un-doubtedly yield great benefits to your future students.As a consequence, please approach the mini-lessonclinics and outside mini-lesson assignments as a realopportunity to increase your instructional skills.

MINI-LESSON POSTTEST 2 REPORT FORM

Date mini-lesson taught

Mini-lesson

Today's date

Setting fOr teaching and types of learners (one or two sentences)

Number of students taught

Results: average interest rating

average percent correct on posttest

Was this the second time you taught the mini-lesson? )"es No

If this was the second time, supply)he average interest rating

and average percent correctmini-lesson.

Comments (optional):

for the first time you taught the

Below give the names and phone numbers (if available) of at least three of thestudents you taught.

(1"

Identification And Assessment Of Minimal CompetenciesFor Objectives-Oriented Teacher §ducation Programs*

By

W. JAMES POPHAM

Classroom teachers have hundreds of things to do. Itshould follow, then, that prospective classroom teach-ers have hundreds of things to learn. But too manywell meaning teacher educators have used these twopremises to draw the conclusion that they thereforehave hundreds of things they must teach, and that ,rep-resents a serious error.

Given the instructional time available in typicalteacher education programs, we must be more modestin our aspirations. When we ask teacher education can-didates to swing an axe at every tree in the forest, theymay fail to fell even a sapling. Far too many teachereducation programs are predicated on a cover-the-waterfront concept: that is, give 'em the works in cul-tural foundations, educational psychology, and instruc-tional methods. And the use of the verb "cover" isquite deliherate. Most- teacher educators feel compelledto cover content in their courses that they perceive as'germane to the teacher's responsibilities. When theseprofessors have covered such content, they sleep easierat night. Few of these coverage-culprits ever verifywhether their extensive coverage of subject matter everresults in any payoff for the teacher candidate, otherthan the ability to pass a memory-oriented final exami-nation. And as in so many content-coverage courses,not just those in a teacher education sequence, whatwas covered one semester has faded from the student'smemory by the first week of the next term.

Particularly at a time when teachers are beingweighed more scrupulously on the public's accountabil-ity scales, teacher educators who persist in covering allrelevent topics are probably doing an injustice to theteachers, they arc responsible for preparing. It makesmore sense for the teacher educator to select a limitednumber of competencies which teachers should ac-quire, then focus the program's resources on makingcertain these skills are acquired.

Now even if this point of view (i.e., a focus on theattainment of a modest set of competencies) were as-

51

siduously followed, there would still be considerabledisagreement regarding which competencies to promoteor, in more general terms, what kinds of content toemphasize. Some would prefer to focus on the teach-er's attainment of a wide repertoire of instructionaltechniquq.s. Others might attend more directly to theteacher's becoming a more integrated hump being.Still others would emphasize the teacher's taquisitionof subject matter expertise. The alternative emphasesare myriad.

An Objectives OrientationThe remainder of this analysis will describe a partic-

ular orientation which can be described in generalterms as an outcomes-focused -pproach. An out.:comes-focused approach emphasizes the results that ateacher's efforts produce in modifying the behaviors oflearners and can be contrasted with more process-focused strategies which attend to the instructionalploys a teacher utilizes with pupils. Because instruc-tional objectives can serve as a convenient way of de-scribing the intended results a teacher wishes toachieve with learners, we may refer to one variant ofan outcomes-focused approach to teacher education asobjectives-oriented. An objectives-oriented strategy foreducating teachers will be treated here.

The rationale for an objectives-oriented approach toteacher education characteristically rests on a centralassumption; namely, that the raison d'être for a class-room teacher is to bring about worthwhile changes inlearners, i.e., important kinds of improvements in theirknowledge, attitudes, skills, etc. Proponents of anobjectives-oriented teacher education program believethat even if a teacher lectures with consummate skill, butthe students are left unchanged, the teacher has failed.

* A paper presented at the annual meeting of the AmericanEducational Research Association, New Orleans, Fcbruary 26-March 1, 1973.

52

Similarly, they contend that even if the teacher has leda nondirective discussion with the artistry of Carl Ro-gers, but the students are t asically unaffected. then theteacher has failed. The criterion, quite clearly, is notwhat the teacher does, but what happens to pupils as aconsequence of what the teacher does. Few objectives-oriented teacher education programs are not somehowwedded to this basic view of a teacher's mission.

But how do objectives enter the picture? Well, theirchief value is in helping teachers identify more clearly,prior to instruction, the kinds of changes whidh shouldbe promoted in the learners. Statements of instructionalobjectives are nothing more than that, convenient de-scriptors of intended changes in learners. In the early1960's, advocates of the oft-maligned behavioral objec-tive endorsed such formulations vigorously because oftheir focus on the learner's post-instruction behavior,not on what the teacher was.eng to do or the content,that. the course would cover., It is unfortunate thatthat,some educators have become so entangled with behav-ioral objectives they have made them a fetish. Preciseinstructional objectives, in the main, are simply state-ments of what teachers want to happen to learners as aresult of instruction. The more explicitly these inten-.tions can be formulated, the better we can tell whetherthe intentions haverbeen realized, and it is for that rea-son that most proponents of_ objectives strive formeasurability as the sine qua non of an acceptableobjective. But remember the central purpose of an in-structional objectiveit is to help an instructionalplanner conceptualize the kind of changes to be pro-moted in learners.

Proponents of instructional strategies featuring meas-urable objectives should forthrightly admit that theirconception of the instructional process is generally onebased on rational decision making. Some critics of anobjectives-oriented approach denigrate such strategiesas "industrial models" of education and thereforesomehow unworthy of man's truly humanistic capabili-ties. They would prefer less systematic and intellectual-ized approaches, favoring instead more intuitive, dy-namic models. But when Aristotle isolated the essenceof man as his rational animality and held that a per-son's potentials were realized to the extent that thoserational powers were actualized, he offered objectives-oriented teacher educators a satisfactory counter-argu-ment. To plan one's actions on the basis of the action'slikely consequences is less industrial, than it is rational.To be clearheaded is not to be mechanistic. To defineanticipated outcomes in advance does not relegate oneio an assembly line mentality. On the contrary, to be

rational in our education decisions will give our stu-dents the best chance of prospering from the educationwe provide them.

Are Instructional Techniques Unimportant?With most objectives- oriented teacher education pro-

grams, it is proper to assert that a distinction is drawnbetween instructional means and instructional ends,with the stress typically on ends. But as anyone whohas attempted to achieve a significant end will agree, itis brought about by employing appropriate means.Hence, an outcomes-focused teacher educator must beparticularly attentive to instructional .techniques, en-hancing the teacher's skill in employing a wide reper-toire of teaching tactics, for It is only through the, judi-cious use of such procedures that significant kinds ofresults in learners n be attained.4

Minimum CompetencieS.Programed instruction specialists are familiar ith

an approach to the development of instructional mate-rials known as lean programing. In sulh a strategy theprogramer tries to accomplish a given instructio alobjective, with the least possible amount of instructio ofstimulus material. Aside from its.obvious economic ad-vantages, lean programing girlies with it a dividendwhen an early version program is unsuccessful. It isunsuccessful.

to improve a low density' program b su le-menting it. than it is to delete segments of a hig den-sity program, for in the latter approach we may be ex-cising the very ingredients that contributed to whatevereffectiveness the program possessed.

Similarly, what is being proposed here may bedescribed as lean competency promotion, for onlythree competencies of an objectives-oriented teachereducation program will be recommended. Now surelyteacher education candidates will learn other things asthey complete their preparation programs; they mayeven learn some of the hundreds of things referred toat the outset of this paper. Since it will be easier tosupplement a few minimal competencies than to deletefrom a, more diverse array, it is proposed that onlythree such skills be emphasized in an objectives-ori-ented teacher education program.

The remainder of this discussion will isolate thesethree minimal competencies, offer some support fortheir importance, and describe alternative methods ofassessing the degree 'to which each has been attained.These three competencies may be used as the guidinggoals of either a preservice or inservice teacher educa-

tion effort. The differences in strategies for promotingthe competmies for experienced or beginning feedersare only superficial and the differences in assessmenttactics almost nonexistent.

Competency Number OneSince the chief assumption of an objectives-oriented

program is that teachers should prothote worthwhilechanges in learners, it is not surprising that the initialcompetency to be fostered deals with that basic skill:

1. Teachers must be able to achieve prespecifiedinstructional objectives with diverse kinds oflearners.

This competency implies that itt skilled teachershould, when presented with clear statements of in-tended changes in learners, ye able to devise instruc-tional sequences which will work; that is, which willbring about the sought-for changes in the learners.Further, the competency indicates that this skill bemanifest with different kinds of learners; for example,children of differing ages, ability levels, ethnic bdck-grounds, socioeconomic status, etc.

The truly professional teaclier not only will need tobe conversant with tested instructional principles inorder to design such instructional plans but will haveto discover what kinds of teaching tactics personallyprove effective. Not all violinists can get good musicfrom the same fiddle. Different people must adapt dif-ferent teaching styles. For some teachers a nondirectiveapproach will work beautifully, while for other teacherssuch a strategy would be a disaster. It is imperativethat a teacher discover what communication style, cou-pled with

theinstructional principles, typically re-

sults in the attainment of prespecified instructionalobjectives for that teacher.

Assessment TacticsThere are two prime contenders for assessing the de-

gree to which this initial competency has been attained.The first of these involves the use of teaching perform-ance tests (or instructional mini-lessons) whose ration-ale and applications are described elsewhere.' Briefly,

1 W. James Popham. "Performance Tests of Teaching Profi-ciency: Rationale, Development. and Validation," AmericanEducational Research Journal

Aj January 1971, 8(1). pp. 105417:W. James Popham, Applications of Teaching PerformanceTests to Inservice and Preservice Teacher Education. Paperpresented at the annual meeting of the American EducationalResearch Association. New Orleans, February 26-March 1.1973.

53

a teacher is given a measurable instructional objective( typically dealing with a novel topic) along with anynecessary background information needed to under-stand the objective. The teacher plans a short lesson(e.g., 15-20 minutes) designed (1) to accomplish theobjective and (2i to be interesting to the learners. Thelesson is then taught, either to a small group of six tei,eight learners or to an entire class: At the conclusionof the lesson, a posttest is admigistered to the learners.The posttest has not been previously seen by theteacher,- but its nature is readily inferable from theclearly stated objective. A form requesting the learnerto "rate how interesting the lesson was" is also pro-vided. The teacher is judged on the basis of whetherboth the cognitive intention (the objective as measuredby the posttest) and the affective intention (the pro-motion of learner interest as measured by the ratingform) have been achieved.

The recency of serious research attention given toteaching performance tests as an evaluative tool proba-bly renders them inappropriate at the present time forassessing individuals other than those at the extremesof a distribution; i.e., the particularly good or particu-larly poor goal achievers. Performance tests may alsobe used to evaluate the efficacy of a teacher educationprogram by administering them on a preprogram andpostprogram basis to the teachers involved. For ex-.ample, suppose two teaching performance teats (X andY) were employed. One of each could be administeredto half the teachers (or teacher candidates) at thebeginning and at the close of the program. The pre-diction would be that Mg,r,. < and V <prr < - poet.

Although the reliability of different teaching per-formance tests has not yet been established With suffi-cient precision to warrant their use for individual eval-uation. with more systematic delineation of the keyelements constituting such tests we may find that in thefuture they can be used for more fine-grained analysesof individual teacher's mastery of competency numberone.

A second approach to the assessment of the initial -+competency is to allow teachers to posit their own in-structicaal objectives, develop a congruent mastery ex-amination, then instruct a group of learners in order toattain the objective. Interest ratings can also be em-ployed here. Because an objective generated by ateacher can be less readily compared with objectivespursued by other teachers, there is the 'additional re-sponsibility of thc; educator to appraise the quality ofthe teacher's objective, not to mention the consonanceof the test with the objective. The advantage of this

54

second approach is that the teachers do most of thework in generating the objectives, tests, etc. Further,because the topic need not be novel, the objective maybe designed for longer periods of instructional time aspart of the ongoing curriculum activities. With topicswhich fall within the learner's probable experiencebase, a pretest must be administered to establish entrybehavior level.

There are, of course, a number of en route skillswhich a teacher should master on the way to attainingthis initial competency, but by employing one, or possi-bly both, of these assessment tactics the teacher educa-tor should be able to determine whether competencynumber one has been satisfactorily promoted.

Competency Number Two

It has been observed elsewhere that one of the con-soling features of conventional, instruction is that it ischaracteristically so impotent we need not worry toomuch about what its goals sc. Similarly, if a teacher isnot particularly proficient at accomplishing instruc-tional objectives, then we need not be too concernedabout what the teacher is attempting to do. Just sup-pose that a teacher has achieved competency numberone, that is, has become skilled in promoting the learn-ers' accomplishment of prespecified objectivesthen itbecomes extremely important to have the teacher directthis instructional prowess toward the proper goals. Ac-cordingly, the second minimal competency of an objec-tives-oriented teacher education program bedomes:

2. Teachers must be able to both select andgenerate defensible instructional objectives.

Since teachers who are skilled goal-achievers mustbecome able to either generate or select worthy goals,it is fortunate that curriculum specialists are finally discarding their customary intuitive approaches in favorof more practical goal-determination procedures. Forexample, the current refinement of large scale educa-tional needs assessment approaches can be translatedinto practical guidelines for teachers who wish to de-termine educational objectives in a more rational faih-ion. Screening of goals by the use of various taxono-mies of educational objectives, such as those devised byGaga, Mechner, Bloom, and Krathwohl, also can leadto the adoption of more appropriate goals. Withoutgoing into those technical procedures more intensively,it can be established that these are schemes now avail-

able which, albeit imperfect, can aid a teacher in theattainment of competency number two.

Since there are now available to teachers an increas-ing number of extant pools of instructional objectives,thereby permitting teachers to select objectives ratherthan be obliged to generate them personally, it seemswise to develop the teacher's proficiency in objectivesselection as well as objectives generation.

Assessment TacticsThere are several procedures available for assessing

the teacher's mastery of competency number two. Oneprocedure would require teachers to generate a set ofmeasurable objectives, then have theie judged by oth-ers (using criteria of significance, suitability for learn-ers, etc.). A description of real or fictitious learnerscould be given as part of the goal-generating task, andthen descriptions could be examined by judges prior tothe appraisal of the goals. Teachers could be asked togenerate such.objectives at the beginning of the teachereducation program and at Its conclusion. These paperscould be coded and rated by judges without knowingthe time at which the papers had been written. Theprediction, of course, is that the end-of-course objec-tive would be rated higher.

Another approach" to assessment might ccinsist ofhaving a teacher select a specified number of objectivesfrom a larger pool of such objectives, then have theselections appraised by others. As with the previous as-sessment approach, subsequent judgment of the teach-er's objectives (either generated or selected) can berendered according to very general or very specific cri-teria.

Additional assessment tactics might involve theteacher's describing, in an exam-like setting, alternativeprocedures for selecting or generating defensible objec-tives. These descriptions, as with the. first two assess-ment tactics, might then be evaluated by judges, and ifdesired, on s preprogram and postprogram basis.

Variations of these approaches are possible, ofcourse, such as having teachers themselves rate the ad-equacy of objectives selected by other teachers, suchratings being subsequently appraised. .

Competency Number ThreeThe first two competencies have been highly related

to instructional objectives, their determination and ac-complishMent. The third minimal competency of anobjectives-oriented teacher education program is, un-

like the first two, .quite unrelated to objectives. In fact,it is almost antithetical to a concern about objectives:

3. Teachers MIMI be able to detect the unanticipatedeffects of their instruction.

In spite of good intentions, even combined withgood ,intention-achieving skills, a teacher's efforts willoften produce unforeseen detrimental and beneficial re-sults with students. Hence, all of the outcomes of in-struction must be considered. The teacher must beskilled in determining the totality of 'what happened tostudents, including of course what was supposed tohappen.

There are several different techniques a teachermight employ to discern unanticipated effects of in-

.struction, such as the use of (a) relatively unstructuredanonymous student questionnaires (e.g., "List the bestand worst things that happened to you because of thiscourse."), (b) strucred anonymous questionnaireswhich attempt to isolate the positive and negative of-'fects, cognitive as well as effective, which might occurbecause of instruction, (c) quasi-projective techniquessuch as the assignment of an essay to the class dealingwith topics such as "My reactions to Biology I" or"Autobiography of a U.S. History Student," and (d)the investigation of the results of a teacher's efforts bya colleague) who follows Scriven's Goal Free Evalua-tion strategy; that is, Who. attempts to discover (with-out even knowing what the. teacher's objectives were)what happened to the students.

It is imperative that objectives-oriented teacher edu-cation programs promote this third competency, forwithout it there is too much dangbr that teachers mayhave marvelods intenlons, accomplish them beauti-fully, but at the same time promote harmful side effectswhich more than cancel out the anticipated results.

Assessment TacticsEven more clearly than the first two competencies,

this third is heavily dispositional in nature; that is, wemust strengthen teachers' dispositions to attend to theunanticipated effects of instruction.

One rather primitive method of getting at this dispo-sition is to employ an inventory such as that presentedin the appendix. The rationale and scoring scheme ofthis inventory, "Looking at Teaching," is suppliedalong with the inventory. In brief, a student is asked to

55

register various degrees of agreement with a series ofstatements regarding instruction, some of which dealwith the use of unanticipated side effects.

On the skill side of this competency, we could al-ways ask a teacher to describe as many ways as possi-ble whereby a teacher who wishes to can detect sucheffects,

Perhaps simulation approaches offer the greatestpromise with respect to ascertaining whether this com-petency has been mastered. Instructional situationscould be presented to the teacher, either on paper, vi-deotape, or filth, in which there are clearly intendedobjectives plus some evidence as to the- degree towhich they had been achieved. In addition, therewould be some subtly identified unanticipated effects ofinstruction. The teacher would-be asked to evaluate theworth of the instruction, and a record would be madeof the extent to which attention to the unanticipatedside effects had been incorporated in that evaluation.

Geuing teachers to describe their general evaluationstrategies is another alternative, for one could then in-spect such descriptions to see if, in response to thislargely unstructured stimulus, unanticipated side effectswere built into the teacher's analysis plan.

A BeginningIn review, an attempt was made in this analysis to

defend the proposition that, fewer competencies shouldbe used as the organizing structure for teacher educa-tion programs. An objectives-oriented teacher' educa-tion-approach was ;described and three minimal compe-tencies for such a strategy were isolated, along withalternative assessment tactics for each.

These three competencies were ici.tntified on thebasis of the writer's experience with outccmes-focusedteacher education programs. They are predicated onthe belief that teachers who pOssess such skills will beable to do a better job for the learners they attempt toserve.

The assessment tactics, however, are certainly not assophisticated as one would wish. Hopefully, this deline-ationiof possible assessment ploys may stimulate otherobjeClives-oriented teacher educators to share their petassessment devices. More importantly, perhaps, it mayencourage teacher educators, both objectives-orientedand those of other persuasions, to scrutinize tho ade-quacy of minimal skills offered by their programs andthe schemes which they employ for their assessment.

56

Appendix ,

Looking at Teaching

Directions.This inventory consists of four brief descriptions of

instructional situations, each of which is followed byfive statements. Please register the extent to which youagree or disagree with each of the five statements bycircling the appropriate letters to the left of each state-ment according to the following scheme:

SA = Strongly AgreeA = AgreeU = UncertainD = Disagree

SD = Strongly Disagree

0 There are no right or wrong answers to this inven-tory. It represents an effort to secure your reactions tovarious views of instruction. Therefore, please be ascandid as possible in your responses.

Situation I.Mr. Hill is a junior high school history teacher who

believes very strongly in "open education." He designsclass sessions so that they are relatively unstructured,with a heavy emphasis on. discussions plus individualreports of resource projects students have initiated be-cause of their personal interests. Mr. Hill finds thatstudents are generally responsive to his approach, butsome of them register dissatisfaction that they are notlearning enough to prepare them for serious highschool history classes.

SA A U D SD 1. Mr. Hill has no right to emphasize°open education if it deprives students

of standard course coverage.

SA A U D SD

SA A UDSD

SA A U D SD

2. The type of instruction Mr. WV isproviding will generally be uninterest-ing to students.

3. It is impossible to combine anyform of open education with adequatecontent coverage.

4. Mr. Hill should have devised ex-plicit instructional plans, almost day-by-day details, prior to the beginning ofthe semester.

SA A U D SD 5. Mr. Hill should not try to detectany effects of his instructional schemeother than those he guessed mightemerge.

Situation II.An elementary school teacher, Mrs. Price, usually

works with third- or fourth-grade children. Normally,most students who come jo her class can read quitewell, but 20-25 percent cannot. She devises specialself-instruction karning centers for these poor readersand encourages them to go to the centers during un-scheduled class time so that they can improve theirreading abilities. Although the performance of thesechildren indicates they have became somewhat betterreaders, they are subjected to considerable verbal.abuse by the good readers in the class whenever theyparticipate in the learning centers.

SA A U D SD

SAAUDSD

SA A U D SD

SA A U D SD

SA A U D SD

6. Self-instructional materials can be avaluable resource for any teacher.

7. Mrs. Price should have done noth-ing special for the poor readers comingto her class because their deficiencieswere the responsibility of previousteachers.

8. Even though it was not foreseen,Mrs. Price should realize that the neg-ative effects of the abuse they receivedmay have been more harmful to thepoor readers than whatever progressthey mace in reading.

9. It is normal for 20-25 percent ofchildren to read badly; so any gainsMrs. Price can get will be all the morevaluable.

10. Poor achievers must always expectto experience a certain amount of de-risiveness from normal and highachievers.

Situation III..Mr. Cohen is a high school English teacher who

e

plans his instruction with inordinat6 care. Prior to eachclass he details every significant level of achievementhe believes students should make as a consequence ofhis course. He also attempts to spell out any major at-titudinal or interest shifts he is attempting to promotewith the pupils. At the close of the academic year heevaluate his English class totally in terms of whetherthese intended changes, both intellectual and attitudi-nal, have been produced in the learners..

SA A U D SD 11. Mr. Cohen should certainly deter-mine whether, at the beginning of theacademic year.his pupils can alwaysdisplay the intended behaviors.

:SAAUDSD

SAAUDSD

SA AUDSD

SAAUDSD

12. If learners are informed Of theclear expectations of an instructor, suchas those which Mr. Cohen appears tohave, they will tend to be less anxiousabout the learning situation.

13. Beyond the clearly delineated be-havioral changes which Mr. Cohen hasidentified, he should discern whetherthere were any adverse or beneficialeffects of students which hi had notconsidered prior to-instruction.

14. Mr. Cohen's careful planning, al-though commendable in the abstract,will 'probably take too., much valuableenergy from his actual instruction.

'15. In general, humanity subject fieldssuch as English are the least amenableto an instructional approach dependentin the prespecification of educationalgoals.

Sititatkin IV.Ms. Harold is an elementary school's music instruc-

tor who must work instructionally with children at alllevels. She feels terribly overburdened with the numberof youngsters she is obliged to service, taus devises ex-tremely intensified music lessons for each grade level.Although there is little doubt that the children arelearning about music, there are a number of indicationsthat they are becoming antagonistic to music in theprocess. Ms. Harold behaves as though she were bbliv-ious of these negative attitudes.SA A U D SD 16. Ms. Harold is probably required

to undertake instruction beyond whatmight be expected of a typical teacher.hence we should excuse any negativeattitudes she might be creating.

57

SA A U DSD 17. The negative attitudes the childrenare developing are relatively unimpor-tant, particularly because the childrenarc learning a great deal about music.

SAAUDSD 18. Only in esthetic fields such asmusic and art will intensified instruc-tion lead to student negativism.

SA A U D SD 19. Ms. Harold should abandon anyemphasis on music skills and focus in-stead on promoting positive attitudestoward music.

SA A U D SD 20. Ms. Harold must recognize thatunanticipated effects of ,instruction arepotentially more important than in-'tended effects and should strive to iden-tify such effects as the negative atti-tudes seen here.

Scoring Directions

Looking at Teaching

This inventory is designed to detect how predis-posed teachers are to detecing the unanticipated effectsof instruction and use them in evaluating the quality ofinstruction. Of the 20 statements with which respond-ents are to indicate agreement or disagreement, onlyfive deal with this question. The other 15 items are in-cluded only to camouflage the real purpose of the in-ventory sothat respondents are not readily able to de-tect the socially desirable way to answer the items.

The five items and the scores associated with eachresponse are given below. Omitted items should begiven a score of 3.

KEY ,

ItemNumber SA A

PointsU' D SD

5 1 2 3 4 58 5 4 3 2 1

13 5 4 3 2 1

1.7 I 2 3 4 55 4 3 2 1

Since a person might earn a maximum of 25 pointson the basis of these five items, scores approximating25 should he considered to reflect a predisposition-toconsider unanticipated side effects important in evalu-ating the quality of a teacher's instructional efforts.

58

From Commitment to Practice in

Assessing the Outcome of Teaching:A Case Study'

H. D. SCIIALOCK

By

JESSE H. GARRISON

As experience with performance-based teacher edu-cation has accunarlated, the interrelated problems ofcompetency definition and competency assessment havecome increasingly into focus. On the one hand it is4.0recognized that teaching competence is something morethan the mastery of knowledge and simple teachingskit's or behaviors, but on the other it is recognizedthat as soon as the -definition of competency extendsbeyond the knowledge aicti skill level, the matter of as-sessment becomes inordinately complex. In fact, in theeyes of many, it takes on properties that demand Morefrom the technology of measurement. and evaluationthan that technology has at the moment to give:

As a mutt of this circumstance the designers ofteacher education programs face a difficult Choide. If ateacher education .program is to be performance- orcompetency-based, and not be a mockery of those con-.cepts, provision must he made for the collection of ac-ceptable evidence of competency demonstration. Yetthe technology of assessment is such that when teaching .

competency is defined in terms of performance inongoing school settings, or in terms of the outcomesexpected to be achieved through teaching (Ttinrer'scriterion levels 3, 2, and 1), the wherewithal! to meas-

ure such outcomes simply is not available.. When con-.1fronted with this circumstance most program designers

adopt the simpler definitions,,, of teaching competencyand proceed with program development as if such defi-

nitions were acceptable.This circumstance, in our opinion, helps account for

the large number of 'teacher education programs in ex-istence today that label themselves as being compe-tency- or performance-based, but choose to define

This paper was piesented in outline form at the meetingsof the Multi-State Consortium on Performance-Based TeacherEducation, New Orleans, February 25-28, 1973.

BERT Y. KERSHow,

"competency" at the knowledge or simple skill. level.Since SQ few. institutions haVe moved to implement apestormance-based teacher education program that. isbas6d upon higher order definitions of competency,and those that have moved in this direction have onlybegun to solve the problems of assessment that are as-sociated with it, persons planning to implement such aprogram have no models to pattern and no access toinstrumentation 'that will permit. them to carry dut sucha program even if they are courageous enough to at-.tempt it. How else can one account for the fact thatonly %handful of programs define teaching competencyin terms of complex skill outcomes, and probably nomore than half a dozen programs in the entire UnitedStates define teaching competency in terms of the abil-ity to perform\ the functions of a certificated teachingposition or the ability to bring about the outcomes ex-peered from the successful execution of a teaching po-sition?

The intricate relationship between competency defi-nitiOn and assessment has been recognized for a longwhile. In 1970, in a paper entitled "The Focus of Per-formance-Based Certification: Knowledge, Teaching Be-havior or the Products that Derive from a Teacher's .Behavior," the first author pointed to three levels ofcompetency definition that performance-based. teachereducation programs could adopt (SchaloCk, 1970). Thesewere referred to in general terms its knowledge outcomes,skill or "behavioral" outcomes, and product outcomes,with product outcomes referring to the outcomes to beachieved through the performance of teaching func-tions in ongoing school settings. Questions were alsoraised in that paper as to the implications of the vat-ious levels of competency definition for assessment.The authors of the elementary models made use of

similar distinctions in describing alternative foci forperformance-based teacher education programs (De-

Vault. et at, 1973), and Turner's refinement of thisthree-level distinction to one that involves .six levels(Turner. 1973) has highlighted the issue even further.

Up to now, however, little progress has been madein developing assessment strategies that parallel thethree- or six-level distinctions that have been made inco , tency definition. Turner's efforts to measuretea ' ing skills (1965), the evolution of microteaching(Men and Fortune, 1967), and the recent efforts ofPopham in the use of what he terms performince testsof teaching proficiency (1967, 1971) provide a begin-ning to the kind of assessment technology tnat would

I" provide for such parallelism, but taken singly or .to-gether these developments do not as yet provide Whatis needed to implement a performance-based .teachereducation program that defines-teaching competency inits more complex for%

The purpose of this paper is to describe an effort byfaculty of Oregon Collegefpf Education and the Teach-ing. Research Division of the Oregon State System ofHigher Education to develop a system for assessingteaching competency that accommodates the higherorder definitions of competency. The system that' isbeinedeveloped rests within the context of the elemen-tary teacher Aim& program at OCE, and takes. asits point of departure the following definition: "Teach-

. ing competency is the ability to bring about the out-comes expected of an elementary teacher in a certifi-cated teaching position." °

Background NILDefining teaching competency in terms of outcomes

9 or product criteria is consistent with the specificationsof the ComField model for elementary teacher educa-tion, one of' the nine elementary models developed inthe late 1960's under the sponsorship of the U.S.Office of Education (Schalock, et al, 1968; Schalock,.et al, 1970), and with the, specifications set forth in the

. new process standards for educational personnel devel-opment that have been ackipted recently by the OregonBoard of Education ("Process Standards," 1973).Using the "Process Standards" as th document of ref-erence

'i.a teaching competency is de d as:

The demonstrated ability to bring aboutthe, expected outcomes of a role or func-tion included in a job definition;

and a competent teacher is defined as: .

One who has acquired and demonstratethe essential competencies of a profe -sional position and integrates and utili

59

them effectively in meeting the require-ments of that position in accordance withits level and certificaiion status. At eachcertification level, tilt teacher must alsoprovide evidence that he has mastered theknowledge and skills assumed to be re-quired for the development of his ditchingcompetence at that level. (p. 48 of theApril 12 draft of the document).

The implications of this set of definitions for the designand operation' of teacher education programs havebeen spelled out in detail in a paper used by. the Boardof Education in the review process given the newstandards (Schalock, 1973a).

In the fall of 1972, OCE implemented an experi-mental elementary teacher education program that wasto serve both as a test of the feasibility of the "OregonProcess Standards," and a test of the soundness of theprinciples of the ComField model. It was also to serveas a context fur research and development, taking asits primary objective for the first year of operation thedevelopment of an assessment system that would meetthe demands of the most exacting definition of teachingcompetence that is possible (levels 2 and 1 in Turner'scriteria), and do so within the additional constraintsestablished "by the ComField model. These include thepersonalization of instruction and assessment, as de-fined by Schalock and Garrison (1973), the systemati-zation of program operation, as defined by DeVault(1973) and the operation of the program within thecontext of a teacher education consortium, as definetby the "Oregon Process Standards."

Forty-three students entered the program. Two full-time education faculty, six quarter- to full-time facultyfrom related subjcct matter areas, 43 school supervi-sors, and an equivalent of one full-time specialist inmeasurement and evaluation\ staffed the program. Theprogram was limited to the measurement and evalua-tion'staffed the program. The program was limited tothe prestudent teaching aspects of Pfessional prepara-tion, and extended crier a period of 6vo terms (fall andwinter). Students received 36 hours of college creditwhen they met the requirements of the program.

A relatively limited set of developmental goals wereset for the assessment system during the first year ofprogram operation. The decision was made to concen-trate on the development of those aspects of the systemthat wouldipermit the assessment of teaching compe-tencies in ongoing school settings at the precertificationlevel, moving if time permitted to the development ofcompetency assessment prociedures at tbeievel of initial

60

certiftbation.2 As the year progressed, this turned outto mean, operationally, the development of an assess-ment system that functioned at two levels of compe-tency demonstration: 1) lesson teaching; and 2) shortterm (2 to 5 days), full-responsibility teaching. Lessonteaching is the first and simplest context within which,.teaching competency is to be demonstrated in the pro-gram. Short-term, full-responsibility teaching is thenext simplest context for competency demonstration,and serves as the staging context for student teaching.Short-term teaching can be engaged in only after com-petency has been demonstrated in lesson teaching; andstudent teaching can be engaged only after competencyhas been demonstrated in short-term teaching.

A third demonstration context received some atten-tion during the year, but not as much as the first twocontexts that have been described. This was a studentteaching equivalency demonstration context. It requiredfull-responsibility teaching for a 5- to 10-day period,and could be entered only under conditions of excep-tional performance in short-term, full-responsibilityteaching. Successful performance in the student teach-ing equivalency context was accepted as evidence ofthe level of teaching competency required to receiveinitial certification.

As the assessment system Clow stands, it representslittle more than a beginning of the system that ulti-mately must evolve. Two major components of the sys7tem have been developed, and a third started, but allof these have undergone major revision in preparationfor the second year of .program operation. Undoubt-edly, they will undergo at least one more major revi-sion before they stabilize. In addition to the revision ofwhat has already been developed, however, the systemmust be extended to cover the assessment Of compe-tency for purposes of initial, basic, and standard certi-fication. This represents a major developmental under-taking for as the "Process Standards" now reads, initialcertification requires competency demonstration in a 2-to 5-wee.k full-responsibility teaching situation (studentteaching); basic certification requires competencydemOnstratibn in a one- to three-term full-responsibil-ity teaching situation (intern or protected first-yearteaching); and standard certification requires compe-

:The recently adopted "Process StAaards" for educationalpersonnel development in Oregon call for three levels of certi-fication: initial. basic, and standard. Competency demonstra-tion is required at all three levels of certification. As level ofcertification progresses the competencies to be demonstratedincrease :.s number and kind, and performance standards in-crease in difficulty.

r.

tency demonstration in a 2..to 3-year full-responsibilityteaching situation after the basic certificate has beenreceived.

Finally, the system must be extended to cover theknowledges and skills assumed to be needed to per-form effectively as a teacher. This includes knowledgeand skills in the various subject matter areas of profes-sional education as well as those in the subject matterareas to be taught. .

It can be seen from this brief outline that the workthat remains on the assessment syitem far exceeds thework that has been done. What has been accompliihedthus far represents only the foundation of the systemthat will be needed in the long run to implement thekind of competency -based teacher education programthat' is desired at the college, or that reflects fully thespecifications of the Com Field model or the new "Ore-gon Process-Standards." The work that has been donerepresents a beginning, however, for the basic outlineof the system has been established and several of itsmany components have been developed and tested. Weare at least on the way, and that is more than could besaid a year ago. Because Of this, and because so littleelse exists that can be .drawn upon in .implementingteacher education programs that incorporate higherorder definitions of competency, making public whathas been done thus far at OCE may be of some benefitto others. Hopefully it will be of benefit to OCE aswell, for the response to what has been done may helpclarify problems that are not seen or have not been an-ticipated at point to the work of others that could beof benefit.

Implementing a teacher educatinn program that isgenuinely representative of the idealt of a model ascomplex as ComField or as demanding as the "ProcessStandards" that have been adopted in Oregon is amonumental task, and any institution that attemptssuch a venture needs all the help it can get.

Bore proceeding with the description of the workthat s been done on, the assessment system thus far,it is worth noting that the experimental program withinwhich the assessment system is being developed wasjudged to be sufficiently successful in its first year ofoperation that OCE faculty, cooperating school super-visors and administrators, and students recommendedthat it be installed as the "regular" program in the ele-

mentary division of the college. This recommendationhas been accepted. Operationally this means that be-tween 275 and 300 students will move through theprofessional component of the program during thecoming year, that approximately tho same number of

.4

school supervisors and i 5 or so college supervisors willhave to be trained to employ the assessment system thatis to be used in the program, and that 20 or more sub-ject matter specialists will have to be at least informedof the system so that they will be able to relate

atit

meaningfully. The implications of this decision at thelevel of system. 9ycration are treated in some detailelsewhere in the paper."

An Initial View of System RequirementsWhen initially planned. the system for assessing

teaching competence at OCE was designed to accom-modate a number of special conditions. These includeda particular definition of assessment; a particular defi-nition of teaching competency; a commitment to theprinciple of gradualism in the demonstration of teach-ing competency; a commitment to the principle thatmeasures coming from the system would have utility isdecisions about instruction, certification, and hiring; acommitment to the principle that measures comingfrom the system would be of suffiCient quality that afirst-rate program of research could be built aroundthem; and a commitment to develop an assessment sys-tem that could be operated within theuconstraints ofthe resources available to the college through regularfunding channels. Since each of these items had major.impact upon the nature of the system that evolved,each is discussed briefly in the paragraphs that follow.

The OCE Definition of AssessmentWhen planning the experimental teacher education

program at OCE, the assessment system that was toaccompany it was to be more than a measurement orevaluation system, if by measurement one means theassignment of numerals to observations and if by eval-uation one means.the assignment of value to numerals.It' was to incorporate these two sets of operations, butinclude As well the concept of an information manage-ment system that serves particular decision-makingfunctions. Assessment was seen in the context of theOCE program, therefore, as a mechanism that supportsdecision making. Put in other terms, it was a targetedinformation system. Two major classes of decisions

3 A list of the products that have been developed within theOCE examental- program, including a description of theprogram per se. the teaching competencies pursued within theprogram, and the system that has been developed for the as-sessment of those comcztencies is available upon icquest.Those interested in obtaining this list, or any of the items re-ferred to on it, may do so by contacting the authors.

61

were to be served by the system, instructional decisionsand program adaptation or design (management andpolicy) decisions.

Given this concept of assessment, the system was toinclude measures ot. teaching competency; pereformancestandards for competency demonstration at particularlevels of certification or prccertification experienq;.7specifications as to the decisions to be served by par./ticular measures of competency.;"specifications as to thestructure or mechanisms to be employed in arriving atparticular classes of decisions; specifications as to theform which the data Were to assume to facilitate eachparticular class of decisions; and an information reduc-tion, storage, and: distribution/retrieval system thatpermits the efficient handling of the data that

andfrom the system. The rationale for the system, and a de-scription of its -various pieces and parts, appears in aforthcoming book entitled "Exploring Competency-Based Education" (Schalock,. 1973b).

The OCE Definition of Teaching Conipetency.The basic definition teaching .competency that

guided the development .of the assessment system atOCE has already been cited (see page 59). It has alsobeen pointed out that the definition of competencyadopted by OCE is consistent with the definition pro-posed in the ComField model -for elementary teachereducation, and the definition inoposed in the .ncwlyadopted "Oregon Process Standards" for the accredita-tion of educational personnel development programs.What has not bden pointed out is the host "of surphismeanings that such a definition carries.

Perhaps the most troublesome Of its various .surplusmeanings is the fact that as defined, competency is al-'

,ways situation specific. The performance of instruc-.

tional planning and preparation functions] for example,of the performance of instructional 'functions br assess-ment functions, are always specific to a particular'group of children in a partial* subject matter .in aparticular educational setting, .dud require thereby aparticular set of performance standards! The meaningof com,petence is also always dependent uppn the coin--plexityrof the teaching task to be performed. The dem-onstration of teaching competence in the context of les-son teaching, for example, has a considerably differentmeaning than the demonstration of teaching compet-ence in the context bf short-term, full-responsibility orintern teaching. Finally, the OCE definition cif teachingcompetence requires that competence be demonstrated

.

62

in ongoing school settings, and that it meet designatedperformance standards.

While such meanings may sound complex andstrange, they are not at all arbitrary, for they follownecessarily from the basic definition of competence asthe ability to bring about the outcomes expected of acertificated job position. This is so because jobs are sit-uation specific. If such meanings are hard to under-stand the reader is referred again to the technical paperthat spells out in considerable detail the implication ofa competency definition of this kind for programoperation (Schalock, 1973a).

The OCE Definition of Gradualism in theDemonstragon of Teaching Competency

As indicated previously, the "Oregon Process Stand-ards" call for competency demonstration ai the level ofinitial certification to take place in a full-responsibilityteaching situation of no less than 2 weeks duration.Th, assumption underlying this expectation is that ademonstration context of any less demanding naturewould be insufficient as a source of evidence aboutability to bring about the expected outcomes of teach-ing. A year of full-responsibility teaching is suggested

as the demonstration context for competency assess-,ment at the level of basic certification and 2 to 3 yearsof full-responsibility teaching. after basic certificationhas been achieved, is suggested as the demonstrationcontext for competency assessment for the standardcertificate.

The successful performance of teaching functionsunder conditions of full-responsibility for a 2-week pe-ri.xl is a demanding task. It is also one that is notlikely to be performed successfully without prior expe-rience in teaching. The experimental program at OCErecognized this fact, and was planned to incorporate anextensive system of precertification teaching in whichteaching competency could be demonstrated within aseries of graduated competency demonstration contexts.Two demonstration contexts were to be provided prior

to the 2-week context in 'which competence was to bedemonstrated for purposes of initial certification, les-son teaching, and short-term (2-5 days full-responsi-bility teaching. The lesson teaching context was to re-quire the preparation and resentation of three lessonson three separate days and in three different subjectareas.

Both demonstration contexts were to be providedwithin what amounts to the first two terms of a three-term professioaal preparation sequence. Obsci vation

and informal lesson teaching were to precede the prep-aration and presentation of the three lessons that wereto receive formal assessment. Performance standardsfor lesson teaching were to be based upon a summaryof performance in the three lessons, and were to bemet before a student was to Lngage in 'short-term, full-responsibility teaching. The ability to bring about de-sired learning outcomes in pupils was not to be re-quired as a competency at the level of lesson teaching,though the assessment and display of the learning out-comes achieved in lessons were.

After competence was demonstrated in lesson teach-ing, a student was then to demonstrate his competencein short-term, full-responsibility teaching. Essentiallythe same competencies demonstrated in lesson teachingwere to be demonstrated in the new context, but withmore demanding performance standards. Some addi-tional competencies were also to be demonstrated. Per-formance standards were to assume much \the sameform as they did in lesson teaching, including not beingheld accountable for bringing about desired learningoutcomes. That was to be a requirement, however, forcompetency demonstration at all levels of.certification.

The demonstration of competence at the level ofshort-term, full-responsibility teaching was to permit astudent to move on to student teaching, to internteaching, or to a student teaching equivalency examina-tion. The basic assumption underlying the design of thegraduated demonstration contexts to be included in theprogram was that if students were to perform effectivelyin a 2- to 5-week, full-responsibility teaching situationstudent teaching) that. provides the context for com-

petency demonstration at the level of initial certifica-tion they would have to have a carefully tailored set ofprecertification teaching experiences that would preparethem to do so.

The Utility of Competency MeasuresFor Instruction and Job Placement

From its inception the assessment system at OCEwas seen as serving the purposes of instruction, certifi-cation, and job placement. Accordingly, in each step inits development the system was t. influenced by the,needs of instructional staff, the rtifying agency, andthe personnel officers of school districts. Informationobtained through the assessment system was to serveonline instructional (supervision) decisions as well ascertification and hiring decisions. The latter two sets ofdecisions were to be facilitated by the preparation of"competency profiles" that depict competencies demon-

oa. see

,r

strated in each demonstration context. To make thepreparation of such profiles feasible they were to beprepared by compter.

The Utility of Competency MeasuresFor Research

One of the major problems that has confronted re-search on teacher education, or for that matter re-search on the effectiveness of instruction, has been thelack of adequate measures on the outcome side ofteaching. One of the major aims for the OCE assess-ment system was the development of a system of meas-urement that would eliminate this problem, and sodoing enable a first-rate program of research to b l es-tablished at the college: in order for this to happen along-range. systematically. designed program If re-search was planned on the quality of the me suresthemselves. The results of the first year of t is re-search, and the lilies which it is to take during thecoming year, are 4scribed in subsequent sections ofthe paper. i

The Feasibility of the Assessment Systeiii;Given the Resources Regularly Availab eto the Teacher Education Program at CE

While h:.1p was to be available fos the dof the proposed assessment system throughTeacher Corps and the Natiusni Center f

velopmentrants fromr the Im

provement of Educationist Systems, US. Offipe of Edu-cation, the system was seen as having to opekate withinthe constraints of the resources ordinarily a ailable tothe teacher education program at OCE once 't was de-veloped. Since the syst-m obviously would r quire re-sources to operate, this Team that ways woul have tobe found to utilize resourceo presently availabl in dif-ferent ways. or to find as yet untapped resour es andenlist them in support of system application fiThisstance was consciously adopted for two reason.: First,OCk did.not wish to ctevelop an elaborate assess ent,yste and then 'find itself in a position of being na-ble to operate it because, It, was too costly. Second, ifthe system was to have general utility to the field itwould have tb be functional within the resources ordi-narily available to most. teacher education programs. \

1

The OCE System fox Assessing TeachingCompetency, Year

1

Starting with the view of system requirements thathas just been outlined, work was begun on the actual -

63

developpent of the system in September 1972. Workon the system progressed simultaneously with the de-velopment of the experimental program as a whole,and as with any interdependent effort, steps taken inone arena both influenced and were influenced by stepstaken in the other. Only the results of this developmen-tal process will be described In the present paper. Theprocess itself. is being chronicled in a monograph thatdescribes the development cif the experimental programat OCE historically (Schalpck. Kersh, and Garrison, inpreparation).

Teaching Competencies to be DemonstratedIn order to develop a system for assessing teaching

competency one has to be clear about the teachingcompetencies to be assessed. Fortunately, the instruc-tional staff at OCE were reasonably clear as to thecompetencies that they wished to see demonstrated atthe level of initial certification, so this step in the de-velopmental prcess was accomplished with reasonableease.

It will be recalled that the definition of competencyadopted at OCE,and in Oregon generally, has to dowith the ability to bring about the outcomes expectedin the performance of a role or function within a cer-tificated teaching position. As used in this definition,Joie or function refers to the largest meaningful classi-fication used describing units of work :within ateaching position" (p. 19 of the April 12 draft of the"Oregon Process Standards"). The teaching functionsaccepted by die OCE staff as a point of departure indeveloping the assessment system were as follows:

Defining the objectives of instruction

Adjusting instruction for the individuals involved(teacher and pupil)Selecting appropriate. materials and proceduresfor instruction, given the objectives and individu-als involved

Orgabizing the learning environment to supportinstruction

interacting with pupils (for pupil success) in theprocess of instruction

Evaluating student growth (cognitive and attitu-dinal)

Defining next learning steps, and the instruc-tional procedures that attend them, given all ofthe above.

These seven functions were propbsed initially by Dr.Herbert' Hite, now chairn.an of the Department of,Ed-ucation at Western Washington College, as content for

4

BEST COPY AVAILABLE

64

thervComld

model (Sehalock and Hale, 1968). Theystd

as the rimary organizers for instruction and as-sessment in th experimental program at OC'E duringits first year of operation.'

In developing the assessment system around theseseven teaching functions (competencies to be demon-strated), the decision was made to obtain evidence asto how well the functions were planned as well as howwell they were carried out in actual teaching. This gaverise to two reasonably distinct sets of assessment pro-cedures, one focusing upon the assessment of lesson orcurriculum plans and the other upon lesson or curricu-lum presentations.

H. D. *halo&

.6'

Competedey Demonstration Contexts.

In a program that defines. competency in terms ofthe performance of teaching functions in an ongoing

school setting the identification of the t ontexts ,inwhich competencies are to be demonstrated becomes ascritical as the identification of the competencievthem-selves. In fact, it has been the OCE experience that it

is not possible to think in terms of competency withoutthinking in terms of the context in which competencywas to be d6monstrated! Until these ,two aspects of

+ Teaching functions, of course, are always imbedded in asubject matter context. and as :t consequence their assessmentrequires an accompanying assessment of the adequacy of thecontent they carry.

tti

competency definition were sorted out, little prograswas able to be made in the development of the assess-ment system.

As indicated previously, the first year of work onthe system saw the development of assessment capabil-ity in two demonstration contexts: lesson teaching andshort-term (2-5 days), full-responsibility teaching.Some work was also done in relation to a studentteaching equivalency demonstration context, but thiswas not completed. The first two of these demonstra-tion contexts deal with precertification competencydemonstration, and are designed to prepare studentsgradually to demonstrate the leVel of teaching compe-tence required for initial certification. The third is de-signed as the context in which competence at the levelof initial certification may be demonstrated.

Reasonably detailed specifications surround each ofthe competency demonstration contexts. Those sur-rounding lesson teaching illustrate the form and sub-stance that they tend to take.

In partial satisfaction of requirements for completingthe two-term block sequence that ,somprises the 1972-73 Experimental Teacher Education program at OCE,each student is to demonstrate competence in teachingelementary pupils in appropriate school settings. EachETE student will accomplish this objective by assum-ing full teaching responsibility for two to five succes-sive school days in an ongoing program of instruction,in such fashion that their performance is judged ade-quate by the student, the college supervisor, and theclasstoom supervisor in accordance with the standardsset for performance in such a situation.

In preparation for the full-responsibility teachingdemonstration, each student will prepare and teachthree lessons, one in each of three different subjectareas. The preparation and presentation of each lessonshall be done under direct supervision and in strehfashion that a complete assessment can be made ofstrengths and' weaknesses, and that the student mayprofit from the assessment. Satisfactory completion ofall three lessons is a prerequisite to entry into the full-responsibility demonstration. The three teaching les-sons are to meet the follosying condititt:

1. At least one lesson shall involve the teaching ofreading, and at least one the teaching of art,physical education, or music;

2. At least one of the lessons shall enable the stu-dent to demonstrate ability to teach childrenfrom diverse cultural or ethnic blckgrounds;

3. At least one of the lessons shall have as its pri-mary or secotidary Instructional objective thelearning of career awareness;

4. The desired learning outcomes for each lessonshould be limited so that e lesson normally willnot require more than sixt :' (60) minutes of in-struction and not less than t senty (20);

S. Each lesson shall be taught to a group, of stu-dents numbering five or more, and at least onelesson shall involve an entire classroom or itsequivalent;

6. The student will be evaluated on the basis of aspecified set of teaching tasks or functions (com-plex skills) within each lesson, ranging from theplanning of the lesson through the assessment ofpupil learning from the lesson;

7. Formal lessons may be prepared and taught,. atany time during the two terms allotted the exper-imental program, though the student needs to re-member that if all requirements of the programare to be met within the two-term period, thethree teaching lessons will have to be completedin time to permit the student to arrange for andcomplete the full-responsibility teaching demon-stration;

8. An objective summary of a student's perform-ance in the preparation and presentation of les-sons will be provided to the student as soon aspossible after the completion of either %the plan-ning or presentation of each lesson. The studentwill be permitted to offer any reaction or rebut-tal to the evaluation which has been made intime for it to be of bengfit.to all concerned inthe preparation of the 'next teaching lesson,, orthe full- responsibility demonstration; and

9. Each student will be permitted to offer any reac-tion or rebuttal to the final assessment made ofhis performance in the full-responsibility teach-ing demonstration before it is finally determinedwhether or not the student has adequately dem-onstrated teaching competence in terms of pre-viotisly agreed upon standards.

Performance Standards for Competency 4Demonstration

Another aspect of the meaning of competency thathad to be unraveled before progress could be made inthe development of the assessment system was the mat-ter of performance standards. This was a particularlytroublesome concept for it was imbedded in both thenature of the competency to be demonstrated and thecontext in which it was to be demonstrated. For exam-ple, defining the objectives of instruction was a compe-tency to be demonstrated, but there is nothing inherentin that competency descriptor that speaks to the qual-ity expected (standard) in its performance. It alsomakes no reference to the context in which perforManceis to take place. This is equally troublesome since theperformance standards for defining the objectives of in-struction in the context of lesson teaching may be con-siderably different than in the context of short-term,full-responsibility teaching. Because of this interdepen-dency of competency descriptor, the context in which a

65

competency is to be demonstrated, and the perform-ance standard set for its demonstration the task of be-coming clear as to what the assessment system was todo and how it was to do it was more difficult than an-ticipated.

Another level of subtlety and complexity emerged inisdation to performance standards as the assessmentsystem developed. This was the distinction that had tobe drawn between performance ratings and perform..ance standards. As the system was planned initially itwas anticipated that performance standards wouldapply to each competency that was being assessed. Asthe system evolved it was discovered that applying theconcept of performance standards at that level of detailwas simply not functional. Ratings of performance hadto be applied at that level; i.e., at the level of eachcompetency descriptor, but it turned out that perform-.ance standards seemed to apply best to performancewithin a particular demonstration context. Thus; as thesystem evolved during the first year, of program opera;tion, perforttance standards came to apply to perform-ance .patterns across . .competencies within particular

.demonstration contexts, rather than to individual com-petency demonstrations. ,

Such an arrangement in no way lessens the impor-tance of individual competency assessments. Theses arestill the central focus of the system, and their assess-ment provides the basis for much of the instructionthat occurs in field settings. Their assessment also pro-vides the basis for arriving at a judgment about per-formance standards, for these are defined in. terms ofindividual competency assessments within a particulardemonstration context. Seen in this way performancestandards 'serve operationally as tfie exit requirementsfrom a particular demonstratiOn context, or entrancerequirements .to another. They also serve as the crite-rion 'measure for certification. The performance stand-ards that 4vere established during the,first year of pro-gram operation fort. lesson teaching and short-term,full-responsibility teaching appear as attachment A.

The Approach Taken to Measurement. The approach taken to the measurement of individ-ual teaching competencies was one of obtaining care-fully delimited professional judgments, in the form ofrating scale placements, as to the adequacy of a stu-deit's performance in a particular demonstration con-text. At least two separate professional judgments wereobtained in relation to each competency demonstration,one from a student's college supervisor and one from

66

his school supervisor. An evaluative judgment was alsoobtained from a content specialist if a student re-quested it. The ratings were designed so as to accom-modate the impact of setting differences on competencydemonstration.

The rating scales assumed somewhat different formsfor evaluating plans to evaluation presentations. In eval-uating plans a three-point scale was used; in evaluatingpresentations. a five-point scale. This difference oc-curred because ratings were applied only to items in aplan that were acceptable; i.e.. if an item was unac-ceptable it had to be modified until it was. As a conse-quence the ratings applied to the plan were really onlythe upper three scale positions of the five-point scaleapplied to teaching presentations.

In addition to" a judgment as to individual competen-cies, raters' were asked to provide an overall judgmentas to the quality of a plan or a presentation. These lat-ter judgments were to be made only after all individualcompetencies had been assessed: To illustrate the na-ture of the overall rating system the forms used in theapplication of the scales to lesson teaching are ap-pended as attachment B.

Two other features of the rating system> are worthyof note, namely, the listing of items under .each compe-tency descriptor that give focus to the descriptor andthe requirement that the behavioral or product indica-tors relied upon by a rater in responding to an item berecorded. In the evaluation of plans. each of the focus-ing items was rated (which turned out to be a poorpractice). In the evaluation of presentations each fo-cusing item was attended to, but only the competencywas rated. This latter strategy proved to he reasonablylunctiOnal, and provided the basis for an important re-vision pf the forms for the coming year.

They decision to ask raters to record the indicatorsthey relied upon in making a.particular judgment grewout of the need to make the ratings as objective aspossible. It was felt that the inclusion of the request torecord indicator statements would help bring about ob-jectivity in two ways. First, it woulq force raters to atleast think about, and hopefully idedtify. the indicatorsused. Second, by compiling a listing of the indicatorsused by different raters in different settlings, a guide onindicator use could be prepared that could he of. valuein training raters. Unfortunately, not enough care wasdirected to this aspect of the rating process during thefirst year of program operation, and a relatively spot-ted record of indicator use was the result. In the comingyear the rating methodology has been changed in this

regard, however, and indicator usage assumes a moreprominent place within it.

Data Management and UtilizationOnly the most rudimentary system was developed

during the first year of program operation for the man-agement and utilization of the data coming from thecompetency assessment system:. A work-stUdy studentwas responsible for the distribution of the various eval-uation forms to students and college supervisors, Stu-dents were then responsible for getting the forms theyreceived to their school supervisor, and for returningthe completed school supervisor's forms to the collegesupervisor. A record of completed forms was main-tained in each college supervisor's office for' each of thestudents they were sponsoring, and a data summarysheet was prepared from each set of forms for use in apermanent file. The data recorded on these summarysheets were then put in computer storage for purposesof competency profile preparation and research: Forillustrative. purposes, the summary data form., for lessonteaching appears as figure 1.. The translation of thisinformation to the computer permitted a series of meth-odological studies to be undertaken oil die adequacy ofthe measures coming from the sAtem (see the nextsection of the paper.), grid preliminary work was ableto be done in the preparation of competency profiles.This task was sufficiently complex, however, that itscompletion was not possible during the first year ofprogram operation.

Data management procedures that served Mine de-cision making in relation to lesson 'and short-term,full-responsibility teaching were left largely to theringe-nuity of the college and school. supervisors involved.Several guidelines to data use were provided, hoWever.First, all conferencing with students about plans orperformance in the classroom was_to be data based, atleast at point of departure. Second, a student had theright and obligation to quarrel with ratings or indicatorstatements whenever he or she felt they were inaccu-,rate or unfair. Third, discussion of performance in the

s. Two assessment systems were actually developed iq sup-;fort of the experimental program at OCE during its first yearof operation. The first is the sompetency assessment system:that is d:scribed in the present paper. The second is a pro-gram assessmmt system that is designed to systematically col-lect data on all aspects of program operation and make thatdata available at a time and in a form that facilitates programadaptation decisions. The program adaptation system is de-signed on the same principles as the competency assessmentsystem (see pp. )1 to 14) and should therefore have the samedegree a transportability.

RE

SEA

RC

H S

UM

MA

RY

: LE

SSO

N P

RE

PAR

AT

ION

AN

D P

RE

SEN

TA

TIO

N

NO

TE

: The

dat

a su

mm

ariz

ed b

elow

are

take

n di

rect

ly f

rom

LE

SSO

N P

LA

N a

nd L

ESS

ON

PR

ESE

NT

AT

ION

eva

luat

ion

form

s. F

orso

me

elem

ents

the

data

tran

sfer

is d

irec

t, th

at is

, the

ele

men

t app

eari

ng o

n th

e pr

esen

t for

m h

as a

n ex

act

coun

terp

art o

n an

eva

luat

ion

form

, and

thus

the

ratin

g th

atap

pear

s fo

r th

e el

emen

t her

e is

take

n di

rect

ly f

rom

the

eval

uatio

n fo

rm: I

n ot

her

case

s, h

owev

er, t

he d

ata

tran

sfer

req

uire

s m

anip

ulat

ion,

for

exa

m-

ple,

fur

ther

red

uctio

n or

fur

ther

sum

mar

izat

ion.

Whe

re th

is is

the

case

, exi

stin

g ra

tings

are

alw

ay u

sed

in th

e m

anip

ulat

ions

. NE

W R

AT

ING

SA

RE

NE

VE

R O

BT

AIN

ED

OR

MA

DE

FO

R P

UR

POSE

S O

F A

RE

SEA

RC

H S

UM

MA

RY

.

- -

__

-

Stud

ent's

Nam

e

SEL

EC

TO

R A

: LE

SSO

N P

LA

N

I.. O

BJE

CT

IVE

S (a

vera

ge o

f th

e fo

urra

tings

re

obje

ctiv

es)

2. A

DA

PTIN

G O

BJE

CT

IVE

S T

O L

EA

RN

ER

CH

AR

AC

TE

RIS

TIC

S (d

irec

t tra

nsfe

r)

Les

son

Num

ber

**

*

3. S

EL

EC

TIN

G I

NST

RU

CT

ION

AL

MA

TE

RIA

LS

AN

D P

RO

CE

DU

RE

S (a

vera

ge o

f fo

urra

tings

re

inst

ruct

ion)

4. E

VA

LU

AT

ION

I a

vera

ge o

f th

e th

ree

ratin

gs r

e ev

alua

tion)

5. P

LA

NN

ING

NE

XT

ST

EPS

(av

erag

e of

the

two

ratin

gs r

e ne

xt s

teps

)

6. T

HE

FIT

BE

TW

EE

N I

NST

RU

CT

OR

, LE

SSO

N.

AN

D C

ON

TE

XT

(av

erag

e of

the

two

ratin

gsre

fit)

7.SU

MM

AR

Y E

VA

LU

AT

ION

OF

TH

EL

ESS

ON

PL

AN

(av

erag

e of

the

ratin

gs a

ppea

ring

abo

ve)

Pers

on W

ho P

rovi

ded

The

Rat

ing

SEL

EC

TO

R B

: LE

SSO

N P

RE

SEN

TA

TIO

N

I.

OB

JEC

TIV

ES

(dir

ect t

rans

fer)

2. A

DA

PTIN

G T

HE

LE

SSO

N T

O L

EA

RN

ER

CH

AR

AC

TE

RIS

TIC

S (d

irec

t tra

nsfe

r )

3. C

LA

SSR

OO

M M

AN

AG

EM

EN

T (

dire

ct tr

ansf

er)

4. I

NST

RU

CT

ION

(:ii

rect

tran

sfer

)

5. E

VA

LU

AT

IO.4

(av

erag

e of

thre

esu

mm

ary

ratin

gs)

6. A

CH

IEV

EM

EN

T O

F D

ESI

RE

D L

EA

RN

ING

OU

TC

OM

ES

(dir

ect t

rans

fer)

7. P

LA

NN

ING

NE

XT

ST

EPS

(di

rect

tran

sfer

)

8. T

HE

FIT

BE

TW

EE

N I

NST

RU

CT

OR

, LE

SSO

N,

AN

D C

ON

TE

XT

(di

rect

tran

sfer

)

9.SU

MM

AR

Y E

VA

LU

AT

ION

OF

TH

EL

ESS

ON

PR

ESE

NT

AT

ION

(av

erag

eof

the

ratin

gs a

ppea

ring

abo

ve)

Figu

re I

. Dat

a su

mm

ary

form

for

less

on p

lann

ing

and

prep

arat

ion.

*

68

classroom was to omit as close as 1.:4)ssible in time toactual performance. Fourth, the college and school-su-pervisor would confer as to the overall adequacy of theperformance of a student before a student was permit-ted to advance to the next level of competency demon-stration, even though the college supervisor was theperson to make that judgment ultimately.

Within these broad guidelines all possible combina-tions of procedures and schedules were followed. Noreduction or synthesis of the data on the evaluationforms occurred before or after discussions with stu-dents, except for the summary that was prepared forpermanent file and computer use. As the reader willsee in the next section of the paper the failure to directgreater attention to the management and utilization ofthe data coming from the assessment of lesson andshort-term teaching had its consequences.

Data management procedures were more exacting inrelation to the student teaching equivalency demonstra-tion, even though much less work had been done de-velopmentally on that demonstration context than theothers. This undoubtedly reflected the fact that certifi-cation decisions were involved. The procedure that wasdeveloped to handle this level of decision required thata jury of college faculty and classroom teachers whohad not served as the student's supervisors examine alldata available on a student's performance and arrive ata decision concerning the match between performanceand performance standards. The student's sponsor pre-sents the data called for in this regard and provides thenecessary context building that permits the jury to viewthe data in perspective. A segment of video tape show-ing the student teaching is included as a part of thestudent record.

This procedure was tested on five occasions duringthe first year of the program and found to be reasona-bly satisfactory. It needs to be pointed out, however,that firm performance standards were not operating onthose occasions and so our experience with these pro-cedures did not represent a full test of them.

Data on System OperationIn order to determine how well the assessment sys-

tem was working in the context of the experimental pro-gram, analyses were made of the various studentapplications of the system, the completeness of infor-mation coming from the system, the trustworthiness ofthat information, etc. Studhs of this nature were car-ried out midway through the program and then againat its completion. The data reported in the paragraphsthat follow are based on these analyses.

Of the 43 students that enrolled formally in the ex-perimental program, 42 attempted lesson teaching and38 of these met the performance standards set for thatdemonstration context. All 38 of these students thenattempted short-term, full-responsibility teaching, and34 of them met the performance standards set for thatcontext. On the basis of these figures approximatelythree-fourths of the students who entered the programmet competency demonstration requirements for exitfrom it, and thereby met entry requirements for stu-dent teaching. Five of the 34 students, however, chal-lenged student teaching through the student teachingequivalency demonstration, and three of those five

were judged competent at the level of initial certifica-tion.

Taken at face value these data would suggest thatthe assessment system was working well. In some re-spects that is a fair judgment. A larger proportion ofstudents dropped or were dropped from the experimen-tal program than is typically the case in the nonexperi-mental form of the program. College and school super-visory staff also reported that the evaluations that theywere able 'to make of student's performance weresharper and more detailed than they had ever beenable to make before. Staff also reported that the database provided by the ratings greatly facilitated instruo:tional activities that accompanied the superAsory proc-ess.

Two other s of data, however, force caution in in-terpreting how well the system worked. The first is aset of data that has to do with the conscientiousness ofperformance rating and documentation by various eval-uators. These data are summarized in tables 1 and 2,and are informative on a number of counts. First, it isimmediately clear that the rating forms were' not ap.plied in all cases, and often times when applied theywere not attended to completely. Second plans arerated more consistently and more compiltely thanpresentations. Third, the conscientiousness of ratingand documentation varied by class of rater, with theschool supervisors generally being the most conscien-tious about filling out the forms completely. Finally,very few content specialists from the college faculty ap-plied the forms to either plans or presentations, andwhen they did they were not overly conscientious abouttheir use.

If taken at face value these data would suggest thatthe assessment system was essentially nonfunctional.,This would be an over-interpretation, however, forwhile the application of the system obviously left muchto be desired the data that appear in tables 1 and 2

Table 1. CompletenessLesson Planning

Elements Rated_

of Ratingsand Presentation

CollegeSupervisor

on Competency

SchoolSupervisor

in

ContentSpecialist

Plansall elements rated

(14 68) ( N 96) 6(N 20)

Lesson 1 20 .30 3Lesson 2 20 23 7Lesson 3

some elements rated17 27 4

Lesson 1 7 g 3 1

Lesson 2 1 6 2Lesson 3

no elements rated1 2 0

Lesson 1 2 3 1

Lesson 2 0 1 2Lesson 3 0 . 1 0

Pwsentatimisall elements rated

(N=69) (N=94) (N=20)

Lesson 1 16 20 1

Lesson 2 7 19 2Lesson 3

some elements rated3 17

Lesson 1 3 4 0Lesson 2 4Lesson 3

no elements rated1 3

Lesson I 11 10 4Lesson 2 12 8 8Lesson 3 15 9 4

Table 2. Completeness of Ratings for Sh.i.,4Full-Responsibility T-

Elements Rated

Plans

CollegeSupervisor

(N=24)

SchoolSupervisor

(N= 29)

ContentSpecialist .

(N=1)all elements rated 18 28some elements rated 4 1

no elements rated 2Presentations. (N= 24) (N16= 28) (N= 1)

all elements rated 1 1

some elements rated 3 9no elements rated 20 4

have a number of explanations. First, the system wasinstigated with essentially no staff Reparation. Second,some items within the system were badly in need of re-vision, and as a consequence many evaluators simplychose to omit them and deal only with those that madesense or were able to be managed. Third, the supervi-sory load on the college staff became so heavy ilea. theend of the program that they were essentially unable tomeet the demands that were placed upon them. This isreflected in the high proportion of ratings missing 'nthe second andthird lesson presentations, and in theshort-term, full-responsibility teaching situation. Fi-nally. no expectations were established nor held in the

69

prograin for content specialists to apply the system toeither plans or presentations. Students were free to asktheir participation in the rating process if &shut': orcontent specialists could ask to become involved in as-sessing a plan or the performance of a particular stu-dent, but this was not a matter that received a greatdeal of attention in the program.

In some respects, then, given the circumstances thatsurrounded the development and application of the sys-tem, it is possible to be delighted with the extent of thesystem's application and the conscientiousness withwhich it was applied. The data are particularly encour-aging in this regard for school supervisors.

Three additional sets of data support a sense ofhopefulness about the system and its operation in thecontext of the experimental program. The first dealswith a set of analyses that were. carried out to deter-,mine the sensitivity of the ratings. Two kinds of sensi-tivity indicators were.used, the extent to which compe-tency performance measures varied for an individualstudent, and the extent to which performance profit svaried across students. The assumption underlying boanalyses was that in most cases variability should befound in individual competency demonstrations withinthe profile of any single student, and that there shouldbe variations in competency profiles across students. Itwas further assumed that if such variability were ob-served this could be taken as evidence of the sensitivityof the measures.

The results of these analyses were in the directiondesired. While some students were found to vary rela-tively little in the 'competencies demonstrated, moststudents varied considerably. More ifimportantly, theytended to vary in all possible ways. For example, somestudents were consistently high across performancemeasures, some consistently low, and some both highand low. Similar variability was found between stu-dents.

The second set of data that arc encouraging of thesystem's potential deals with the extent of agreementon ratings of student performance between independentraters. A number of analyses of this kind were made,though obviously they were limited by the incomplete-ness of the data as reflected in tables 1 and 2. Never-theless, by the close of the first term of the program22 lesson plans and 12 lesson preScenfations were foundthat were sufficiently complete to permit interrateragreements to be calculated. On the basis of thesecalculations, level of agreement was approximately 80percent for the items rated in lesson plans and 75 per-cent for the items rated on lesson presentations. No

a

70

interrater agreements were calculated for short-termteaching.

The third set of data that are encouraging deals withrating patterns of evaluators. In these analyses, evalua-tor ratings across students were the basis for compari-son, and were analyzed independently of the studentson which the ratings were made. Pattern analyses wererun that compared a) college and school supervisorsratings. h) one college evaluator's ratings with the rat-ings of another, c) ratings provided by school supervi-sors in one school with those of another, and d) rat-ings provided by content specialists with thoseprovided by both college and school supervisors Byand large these analyses showed that while there wassome tendency on the part of all evaluators to skewthe ratings toward the upper scale values, and sometendency ft.r rating patterns to reflect the individualsdoing the rating (for example, one college supervisorwill tend to rate higher or with less variability than an-other), overall rating patterns tend to be roughlyequivalent. This is especially the case as ratings beingcompared increase in their generality or larger numbersof ratings are compared.

This last point is illustrated by the histograms pre-sented as figures 2 and 3. Figure 2 shows the ratingpatterns of two college supervisors for competence inclassroom management in the context of lesson teach-ing. Figure 3 shows the rating patterns of all collegesupervisors and all It hool supervisors for the samemeasure. Even though one would expect greater simi-larity between the two college supervisors than betweencollege and school supervisors, this was not the case.The greater similarity in rating patterns reflected infigure 3 can best be accounted for by the effect of largenumbers entering the picture.

There tended to be leis variability in rating patternsaround plans on the part of all evaluators than aroundpresentations.

As indicated throughout this discussion, the data onsystem performance are both encouraging and 4scour-aging. On the encouraging side there is evidence thatthe measures provide reasonably sensitive discrimina-tions in relation to pupil performance, and that raterstend to provide reasonably similar ratings when observ-ing the same students and reasonably similar patternsof ratings when observing across students. Add to thisthe opinions of both college and school supervisorsthat the data that come from the system help themmake better judgments about competence and providebetter instructional help than has heretofore been pos-sible, there is reason to be hopeful about the potential

of the system. It should also be noted on the hopefulside that both college and school supervisors indicatedstrong support for the continued use and further devel-opment of the assessment system, and that its use isnot impossibly expensive. The resources invested in thedevelopment, operatic,a, and adaptation of the experi-mental program as a whole 'this past year are summa-rized in figure 4. Costs most directly. attributable to theassessement function within the program are those as-sociated with increased school supervisory time,development costs, adaptation costs, and researchcosts.

All things considered, these are reasonably encour-aging data. On the discouraging side, however, there isevidence that if the system is to function effectively,and if the measures arc to be of a quality that permitsa great deal .of confidence to be placed in them, thereis still much to be done. Major revisions within thesystem itself must be made and an effective procedurebe devised to assure care in its application. It is to theproposed modifications in the system for the comingyear that we now turn.

An Expanded View of System RequirementsOn the basis of the data just reviewed. and recom-

mendations from college and school supervisors, stu-dents. and assessment personnel, major changes arebeing made in the competency assessment system forthe second year of program operation. These changeswill be reviewed in some detail since they representwhat appears to be major advances in the methodol-ogy. J.o provide continuity with the description of theassessment system provided previously, the headingsused in that description will be used again.

Teaching Competencies To Re DemonstratedIt will be recalled that seven teaching functions or

competencies constituted the core of the experimentalprogram during the first year of operation (see p. 64).For the coming year the list has been expanded andorganized into clusters of competencies. As the listpresently 'stands, four competency clusters are to bedemonstrated in the context of lesson teaching: Plan-ning and Preparing for Instruction;, Performing Instruc-tional Functions; Performing Assessment Functions;and Displaying Pupil Achievement. A fifth cluster ofcompetencies has been Aided to these four basic clus-ters for demonstration in the short-term, full-responsi-hility teaching context and in student teaching. This isa set of competenCes that has to do with Interpersonal

r-1

.

I

13

45

Col

lege

Sup

ervi

sor

01N

=15

ME

AN

I =

3.2

7ST

AN

DA

RDERROR OF MEAN 1 = .26

STANDARD

DE

VIA

TIO

N O

F 1

= 1

.00

12

.3

45

Col

lege

Sup

ervi

sor

02N

=17

ME

AN

1 =

3.3

5STANDARD ERROR OF MEAN 1 = .23

STANDARD DEVIATION OF 1

= .97

Figu

re 2

. The

dis

trib

utio

n of

cla

ssro

om m

anag

emen

t rat

ings

on

less

on p

rese

ntat

ions

obs

erve

d by

col

lege

sup

ervi

sors

01

and

02--

-(d

istr

ibut

ions

hav

e be

en n

orm

aliz

ed)

1

r

r

r I

34

5

Col

lege

Sup

ervi

sors

01,

02

N =

32MEAN 1

= 3

.47

STANDARD

ER

RO

R O

F M

EA

N 1

= .1

4ST

AN

DA

RD

DE

VIA

TIO

N O

F 1

= .7

9

12

34

5

Scho

ol S

uper

viso

rs, A

ll-

N =

67M

EA

N 1

= 3

.73

STA

ND

AR

D E

RR

OR

OF

ME

AN

1 =

.10

STA

ND

AR

D D

EV

IAT

ION

OF

1=

.78

Figu

re 3

. The

dis

trib

utio

n of

cla

ssro

om m

anag

enie

nt r

atin

gs o

n le

sson

pre

sent

atio

ns o

bser

ved

by a

ll co

llege

and

sch

ool s

uper

viso

rs(d

istr

ibut

ions

hav

e be

en n

orm

aliz

ed)

;01

72 t

RESOURCES UTILIZED IN THE FIRST YEAR OF THE ETE PROGRAM(44 students enrolled in the program,

Regular"Junior-Block"

Program

37 completed it)_ _ - - -- --_

Experimental"Junior-Block"

Program

Fall Winter Spring Fall Winter Spring

Program OperationEducation FacultySubject Matter

1.5 FTE1.5 FTE

1.5 FTE1.25 FTF

1.5 FTE1.75 FTE

i . 5 FTE1.50 FTE

School Supervisors 2 hrs.p/wk.

2 hrs.p,'wk.

3 hrs.p/wk.

5.25.hrs.p/wk.

Secretary:: Clerical Normal Normal +50% +50%Supplies,. Services.Administration

99

ff

11) Normal+10%

Normal+10%

Program DevelopmentEducation Faculty .5 FTE .5 FTEAssessment Faculty 1.0 FTE 1.0 FTESecretary/ Clerical .75 FTE .75 FTESupplies/Services $750 $750Administration +10% +10%

Program AdaptationEducation Faculty

4 .50 FTESchool Supervisors 6 ® 5 hrs. p/wk.Students 12 ® 5 hrs. S/wk. .

Assessment Faculty 1.0 FTESecretary/Clerical 1.0 FTEStipp lies/Services $500

.7

Program Related Re-search $500 $750 $500

- - -Figure 4. Resources utilized in the first year of the experimental program at OCE (43 students enrolled in the program,

34 completed it

Interaction. The Separate competencies to be demon-strated within the'se five clusters are as follows:

Competency Clustir I.Planning and Preparing for Instruction

Defining learning objectives and the indicators oftheir achievementPlanning instructional activities, materials, andproceduresPlanning for the assessment of learning

Competency Cluster II.Performing Instructional Functions

Conveying the objectives of instructionAdapting instruction to contextManaging the instructional processManaging unexpected events

Competency Cluster III.Performing Assessment Functions

Assessing learning before instructionAssessing learning during instructionAssessing learning after instruction

Competency Cluster IV.Displaying Pupil Achievement

Displaying prelesson and pditlesson achievementDisplaying learning gains that result from in-struction

Competency Cluster V.Enhancing Interpersonal Relationships

Acting responsibly in terms of the feelings,needs, and wishes of othersWorking constructively in task-oriented situa-tions with others

It is expected that this list of competencies will con-tinue to expand or be refined as the assessment systemis applied and teswd in student teaching and interncontexts, or in contexts designed to give evidence ofcompetency demonstration for purposes of standardcertification.

Competency Demonstration Contexts'lo changes are planned for the demonstration con-

texks that were established in the first year of programopekation. In fact, with the formal acceptance of the"Pro\Fess Standards" for the Preparation of educationpersOpnel by the Oregon Board of education, the dem-onstra ion contexts thus far established in the programappea to he more appropriate than initially. antici-pated; he competency demonstration contexts that arenow pr posed for the program, and their relationshipto level certification, is as follows:

Contexts or Competency DemonstrationPrior to C rfification

Less teachingShort- erm (2 to 5 days), full-responsibility.teachi

Contexts for'Competency DemonstrationFor Purposei. of Certification

Initial certification: student teaching (2 to 5weeks), :pr student teaching equivalency demon-stration (5 10 days)Basic . cCrtification: intern teaching (10-30.weeks) of; ptotected first-year teachingStandard certification: 2 to 3 years of full-titneteaching after the basic certificate has been re-ceived.

Performance Standards for CompetencyDemonstration \

It will be recalled that performance standards wereestablished during the first year of the program fol.competency demonstration in lesson teaching and inshort-term, full-responsibility teaching only. It will alsobe recalled that these standards received very little for-mal testing. As a consequence, with one importantchange, the second year of the program is being en-tered with the same 'performance standards for lessonand short-term teaching that were set in the first yearof the program. The change is to require a summativecompetency assessment for exit from the prcertifica-tion phase of the program. The. assessement procedureproposed is much like that outlined' for assessment atthe level of initial certification (exit from studentteaching). for it is also to involve apjury of independ-

73

cat judges. These judges 'are to consist of college andschool faculty that did not carry supervisory responsi-

6 hility for a student and they are to decide whetherdemonstrated performance in the short-term, full-re-sponsibility teaching situation meets the performancestandakis that haite been set for it. A segment of videotape .showing the student teaching is also to be in-cluded as part of the student's record:

One of three decisions will be reached as a conse-quence of this summary assessment:

the student has passed the teaching competencyrequirements at the precertification level, and mayenroll in student teaching or internshipthe student is eligible for student teaching or in-ternship, but may enroll only after completingspecified tasks or submitting additional evidenceof teaching competence

the student is not eligible for student teaching orinternship, and must remain in . the prestudentteaching part of the program until evidence canbe provided that all teaching. competency require-ments for entry into student or intern teachinghaVe.been met.

Specific criteria have not as yet been established as abasis for any one of these decisions.

It is anticipated that with use, the performancestandards that have been set for lesson teaching andshort-term, full-responsibility teaching will be sharp-ened, or in some other way modified. It is also expectedthat the jury system that is being proposed for sum-mary judgment. of competence. in short-term teachingwill he applied to all certification judgments within theprogram. Finally, it is expected that all certification de-cisions will rest heavily on evidence as to the ability ofa prospective teacher to bring about the learning out-comes expected for pupils. Almost no thinking has:been . done within the program about the nature ofcompetency demonstration requirements for purposesof certification beyond those that have been mentioned.

The Approach Taken to Measurement

Major refinements have been made in the approachis still one that involves ratings. Seven major changeshave been made in the rating system:

all ratings are made in terms of a five-point scale(during the first year teaching plans were ratedon a three-point scale whi:c. teaching perfonnandewas rated on a five-point scaleratings are provided only for competencies andcompetency clusters (ratings were required during

6

.,74

the first year of the program for the items thatelaborated a competency as well)all ratings are to he recorded by entering a nu-merical score in a box opposite the competency orcompetency cluster to be rated, rather than mark-ing a position on a continuous scale (it is antici-pated that this procedure will force more care orattention to be given each individual judgmentand its recording)the rating scale positions are more carefullyanchored in the attributes that define each scalepositionexamples of the indicators that can be reliedupon in arriving at a particular rating scale judg-ment have been provided on the rating formthe rating forms have been revised so that theyinvite easier recording of the indicators used in

jarriving at a particular judgmentall competency statements, and the items thatarc intended to define them, have been edited andfield tested for their clarity and meaning.

In combination these changes are designed to makethe rating process both more manageable and discrimi-nating.

The forms as a .whole have not as yet been fieldtested, but initial reactions to them by college andschool supervisory staff have been most encouraging.To illustrate the content and format of the new assess-ment forms, sample page:; from the assessment batteorfor lesson teaching are appended as attachment C.Complete copies of the Corms can be obtained uponwriting the authors.

One further change has been made in the assessmentforms that is of major consequence. This is their organ'!"ization by the source of indicators relied upon inmaking judgments about the competencies being rated.Accordingly, Planning and Preparation Functions, andAchievemdnt 13; 'lay Functions, are judged in terms ofproducts of a tt.... her's behavior; Performance orAs-sessment Functions are judged on the basis of a teach-er's behavikir per se; and Performance of InstructionalFunctions and the Enhancing of Interpersonal Rela-tions are judged both on the basis of, teacher behaviorand pupil behavior. These distinctions will he noted inboth the directions given to evaluatofs and in the sam-ple indicators.

V

Data Management and UtilizationAs yet (July 1973 ) the specific data management

and utilization procedures to he implemented duringthe second year of the program have not been estab-lished, though a set of quality assurance procedures

have been agreed to and an extensive program of re-search on the quality of the measures coming from theassessment system is being prepared. Specific proce-dures for the distribution and collection of forms, forthe utilization of the information contained on the /forins for instructional purposes, and for the utilization/of that information for decision purposes relative to/movement from one demonstration context to the next,are still to be defined.

Taking steps to insure the quality of the aids resobtained through the assessment system is panic arlycritical in the coming year because of expande use.Fifteen college supervisors, up to three hundred stu-dents and school supervisors, and a dozen or so con-tent specialists will he applying the system throughouta half dozen school districts. Well-defined quality as-surance procedures must be implemented if the data'coming from the various users of the system are to beat all trustworthy.

Two strategies make up the quality assurance plan.First, it calls for a careful inservice education programto be provided on system usage. Second, it calls forsystematic checks on the quality of ratings being made.These checks will be made midway through each termthat the program is offered, and at the end of eachterm. inservice programs will be designed on the basisof the information obtained through these checks, anddata management procedures will be elaborated asneeded. The research that is planned on the quality ofthe measures follows the same general lines as. the re-search pursued in the first year of the program, thoughit will be extended in quality and "scope. One addition

'will he the systematic study of indicator usage. Thecomputer programs needed to carry out such researchhaiie been developed and tested so the results of thesestudies will be able to be acted upon as the programprogresses.

The OCE Assessment System in Perspective

To those who have. managed-. to work their waythrough the paper it must be abundantly.clear that theassessment system being developed at OCE is a longway from completion. The parts of the system thathave been developed will obviously undergo further re-finement and the more complex parts of the system areyet to be developed. Problems of behavior and productsampling within demonstration contexts, performancestandards we demanding demonstration contexts,and the development of measures of competence thatare trustworthy are still to be confronted. As planned

75

The assessment effort at ocE needs to be viewedwithin still another context, however, and that is thecontext of,rescarCh on education and teacher education.One of the great handicaps of research on the effec-tiveness of teachers has been the lack of strong meas-ures of effectiveness. WithoUt such measures, no matterhow g? the design or elaborati the analysiS, signifi-cant relationships are not likely to be found. A weakdependent or7critcrion measure will defeat a strong re-,search design and analysis everytime.

In the authors' judgment the work °that has been ini-tiated It OCE in the 'dlea of competency assessmentrepresents a major step toward the resolution of thecriturionplem in tea; her effectiveness research. Ifall goes according to plan, 3 years froth now the workbegun last year will be completed, and for be firsttime a measurement system may be available that willmeet die deMands of research that can make a differ-ence. When that time comes there can be a hopefulnessabout educational research that has been missing for along white.

now, the completion of the total system in a form thatwill per:flit its use with known confidence is targeted forthe close of the 1975-76 academic year.

Whip much remains to he done on the system, andmajor problems are yet to her resolved, a good deal;progress has been made. the bnsic outline of the sys-teM is complete; the basic constructs, dimensions, andmethodologies of the system have been defined and im-plemented;*datamanagement and utilization strategies,thbugh primitive, have been 'established; quality assur-ance mechanisms have been developed and tested; andthe system in its broad outline has been found to beboth acceptable and useful to college faculties, schoolsupervisors, and students. The system has also been

.found to be manageable in terms of cost, particularlywhen developmental costs are differentiated from sys:tem operations costs. Whip there. is obviously muchthat remaing., to he done, the rudiments of the systemhave at least been developed and tested. In the judg-ment of the authors this in itself represents a reasonable

,gain for the woad of education.

Reference§'

Allen, D. W, and Fortune, J. C. An analysis of micro-teaching: a new procedOre in teacher education..In "Micro-Teaching: A Description." Stanford,Calif.: School of Education, Stanford University.1967. Section III, pp. 1-11.

DeVault, M. V. Systems approach applications in de-signing teacher education programs. In Cooper, etal. (Ed.) "A Systems Approach to Program De-sign: Vol. II." San Francisco: McCutchan Pub-lishing Corporation:` 1973(a). pp. 19-32.

DeVault, M. V., et al., "Competency-Based TeacherEducation: Problems and Prospects for the Dec-ades Ahead." Vol. 1. Berkeley,. California: Mc-Cutchan Publishing Corporation, 1973 ( b).

P9pham, W. J. The performance test: A nevi approachto the assessmere of teaching proficiency. "Jour-nal of Teacher Educatiop," 1968, pp. 19, 216-222.

Popham, W. J. Performance tests of teaching pro-ficidncy: rationale, development, and validation."American Educational Research Journal," 1971,pp. 8, 105-117.

"Process Standardy for Educational Personnel Develop-ment Programs." The 0regQn Board ,of Educa-tion, April 12 draft copy, 1973. .

Schalock, U... D. -YThe Foct of Performance-BasedCertication: Knowledge, Teaching Behavior, orthe Products that Derive from a Teacher's Behav-ior." Procedures of the Conference on Perform-aneeBased Certification Florida State Departrment of Edikation,' May 19,70, Miami Beach.Mimeographed. Also available in the ERIC sum-mary of th Inference.

Schalock, H. D. ; implications of the Oregon Board ofEducation.% Proposed -Process. Standards for theDesign and Operation of Thcher Education Prn.pms." Interpretive Paper No. 1: Defining andAssessing. Teacher Competence. April 1973 (a).

-Mimeographed .

Schalock, H. D. Notes on the concept of assessment inthe context of performance-based teacher edu-cation. In R. Houston (RC) "ExploPipg Com-petency-Based Education." (In preparation),1973(b).

76

Schalock, H. D. and Garrison, J. H. The personaliza-tion of teacher education programs. In DeViult,et al. (eii. ) "New Directions In Tpacher Educa-tion: Problems and Prospects fot the DecadesAhead." 'San Francisco: McCutchan PublishingCo., 1973. op. 33-34.

Schalock, H. D. and Hale, J. R. "A Competency:.Based, Field - Centered Systems Approach to Ele-mentary Teacher Education. Volume 1: Overviewand Specifications. U.S. Office of Education, Bu-reau of Research, Project No. 89022. Portland,Oregon: The Northwest Regional Educational Re-search Laboratory, 1968.

Schalock, H. D., Kersh, B. Y., and Horyna, L. L, "APlan for Managing the Development, Impiementa7tion. and Operation of a Model ElementaryTeacher Education Program." Vols. I & 11. Super-

intendent of Documents, U.S. Government Print.:inn Office, Washington, D.C., 1970.

Schalock, H. D., Kash, B. V , and Garrison,. J. H."Teacher Education in Transition: A History ofCBTE at Oregon College of Education. Vol. I:The Experimental Program." In preparation.

Turner, R. L. "The Acquisition of Teaching Skills inElementary School Settings." Bulletin of theSchool of Education, Indiana University, 1965.

Turner, R. L. Rationale for competency-based teachereducation and certification:. levels of criteria, InB. Rosner (ed.) "The Power of Competency-Based Teacher Education." Prepared by the Com-mittee on National Program Priorities in TeacherEducation. New York: Allyn and Bacon, 1973.pp. 3-8.

Attachment A. Performati,ce Standards

PERFORMANCE STANDARDSFOR LESSON TEACHING

The first, and simplest context within which teachingcompetency is to be demonstrated in the ETE programat OCE is lesson teaching. Competency at this levelof teaching must be demonstrated before a student isfree to engage in full-responsibility teaching. At leastthree lessons must .te taught for purposes of formalevaluation, and standards must be met for both theirpr7aration and their presentation.

Standards for Lesson Planning

In preparing for lesson teaching a reasonably detailedlesson plan must be prepared and reviewed by bothcollege and school supervisors. The standards set, forthe preparation of plans are item specific standards,that is, both the college and school supervisor must indi-cate, independently, that every item to be attended toin the plan has. been dealt with satisfactorily. Thisstandard must be met before the lesson can be pre-sented to children. If a plan does not meet this standardupon its initial review, it must be revised until it does.

Standards for Lesson PresentAtion

The standards set for performance in the presentationof lessons are pattern standards, that is, they apply tothe pattern of performance demonstrated in the presen-tation of three or more lessons. Two standards are tobe applied td the performance record of a student onthe three or more lessons presented:

evidence of favorable performance on each of the. teaching functions assessed in at least one of the

three lessons presented;

evidence of favorable performance on the prepon-derance of teaching functions assessed in the threelessons presented. Preponderance is defined hereto mean at least 75 percent of the functions as-sessed in the course of the three less@ resentedwill reflect evidence of favorable perfo nce.and no more than 25 percent of the functions as-sessed will reflect evidence of unfavorable per-formance.

77

PERFORMANCE STANDARDSFOR SHO T-TERM (2-5 DAYS)FULL-RES ONSIBILITY TEACHING

NIVIren studens, in the ETE program meet perform-ance requirements in lesson teaching they are free toenter the first f II-responsibility teaching experiencethdt is provided in the program. This is what istermed a short -te?n, full-responsibility teaching con-text. This experierine requires a student in the pro-gram to assume full responsibility for planning andcarrying out instruction in the schools for a minimumof 2 days and a maximum of 5.

Three kinds of standards are applied to a student'sperformance in short-term, full-responsibility teaching.Two of these correspond to the standards applied tolesson teaching, The third set of standards pertains tothe utilization and management of affect.

Standards for Curriculum Planning

As in the case of individual lesson teaching, a cur-riculum plan for short-twin, full-responsibility teachingmust be approved before teaching can be undertaken.This requires that a reasonably detailed curriculumplan be prepared for a 2- to 5-day demonstration (asused here a curriculum plan consists of a number ofindividual lesson plans, and the relationships if anybetween them), the curriculum plan be reviewed byboth college and school supervisors by lesson plan, andthat both the college and school supervisors must in-dicate, independently, that inch of the items to be:mended to in the plan as a whole has been dealtwith satisfactorily. In keeping with the generally moredemanding requirements of ithe short-term, full-re-sponsibility demonstration all lessons to be presentedwithin the 2- to 5-day teaching period must net ac-ceptable standards before teaching can begin.

Standards for Curriculum Presentation

Standards for curriculum presentation in short-term,full-responsibility teaching arc more demanding, andcover more aspects of teaching, than do the standards .for individual lesson presentation. The first standardassumes the same form as one of the standards set for

78

the presentation of individual lessons. This is the pat-tern standard that requires evidence of favorable per-formance on the preponderance of teaching functionsassessed in the sum of the lessons presented in. the2 to 5 days of full-responsibility teaching. The secondstandard for lesson presentation is also a pattern stand-ard, and requires that over the course of the lessonspresented in the period of full-responsibility teaching:

variety in learning activities will be provided;variety in cognitive functions and levels in pupils

will be exercised;variety in affective expressions will be employed

in teaching; andpositive feelings in pupils, such as excitement and

interest, will be utilized in their learning.

Standards for Affect ManagementThe standards set. for the management of affect in

short-terM, full-msponsibility teaching take as theirfocus four dimensions of affective expression:

teacher responses to instances of pupil affect;the management of pupil responses to instances of

pupil affect;the anticipation of pupil upsets and disruptions,

and their redirection; andthe management of pupil upsets and disruptions

when such occur.

Performance standards in relation to these dimen-sions of affective expression require that during thecourse of the full-responsibility teaching experience astudent need only to perform effectively three of thefour dimensions specified (any three will do), andthat he or she needs to perform to this level on onlyone of the 2 or more days that he engages in full-timeteaching. Such a. standard reflects, the view that themanagement of affect In a classroom is a complexmatter, and that in an initial full-responsibility teachingsituation, performance standards set for it should notbe particularly demanding.

79

Attachment BDEMONSTRATION CONTEXT: 1.ESSON PREPARATION AND PRESENTATION

LESSON PLAN(Attach to your Lesson elan and Lesson Plan Evaluation Form)

Student's Name Lesson Number

RECORD OF NEGOTIATIONAPPROVAL TO NEGOTIATE THE LESSON APPROVAL TO PREPARE A FORMAL PLAN .WITH A SCHOOL SUPERVISOR FOR THE LESSON

College Supervisor Date Sch_ool Supervisor Date

CONTEXT DESCRIPTION

STUDENTS To BE TAUGHT CONTENT TO BE TAUGHTSchool ._ Area__ _

Grade _ _ Expected Learning Outcomes_ _______ .

Number __ _

Special Characteristic:. _

DATE(S) OF LESSON PRESENTATION TIME(S) OF LESSON PRESENTATION

* *

SUMMARY EVALUATION OF LESSON PLAN(Obtain only after all elements of ;Amtr plan have been evaluated)

Q

I judge the plan as Ef whole to be of(circld one)

ACCEPTABLEQUALITY

OUTSTANDINGQUALITY

School Supervisor

College Supervisor'

I- I

Content Specialist

*

Field Test Format 2Experimental Elementary Teacher Education Program

Oregon College of EducationDecember 1972

80

LESSON PLAN EVALUATION FORM

Students Name Lesson Number

Have the evaluators that check your plan initial each of the items listed that meets with their approval. If the treat-ment of an item is thought to be outstanding, have the evaluator draw a circle around his or her initials. Be sure to attachthis sheet to your lesson plan.

. . . . .

ELEMENTS OF THE PLAN

OBJECTIVES

Are the learning outcomes expected from the lesson clearlystated? °

Are they appropriate and worthwhile outcomes; gives thecharacteristics of the pupils to be taught?

Are the indicators that are to be used as evidence of suc-cessful outcome achievement identified?

Are the procedures to be used in obtaining evidence of out-come achievement identified?

ADAPTING OBJECTIVES TO LEARNERCHARACTERISTICS .

Are there provisions for modifying the objectives of thelesson to meet individual pupil characteristics?

SELECTING INSTRUCTIONAL MATERIALS ANLPROCEDURES

Are the instructional materials to be used in the lessonclearly identified?.

Are they appropriate to the learners to be taught and thelearning outcomes to be achieved? . .

Are the organizational and instructional procedures to beused in the lesson clearly identified?

Are they appropriate to the learners to be taught and thelearning outcomes to be achieved?

EVALUATION

Are there provisions for determining where pupils standwith respect to the desired learning outcomes of thelesson before it is presented?

Are there provisions for :edback to pup% about their per-formance during the ti. le the lesson is being presented?

Ire there provisions for determining where pupils standwith respect to the desired learning outcomes of the lessonafter it has been presented?

PLANNING NEXT STEPS

Is there some indication in the plan of what would be donenext with the pupils if the learning outcomes expectedfrom the lesson materialize?

Is there some indication in the plan of what would be donenext with the pupils if the learning outcomes expectedfrom the lesson did not Materialize?

MATCHING INSTRUCTOR. LESSON AND CONTEXT

Does the lesson as planned appear to i feasible and ap-propriate to the school setting in which it is to be pre-sented?

Does the lesson as planned appear to; be feasible aid ap-propriate to the student who is to present it?

EVALUATORS OF THE PLAN

SchoolSupervisor

CollegeSupervisor

Field Test Format 2Experimental Elementary Teacher Education Program

Oregon College of EducationDecember 1972 17

ContentSpecialist

DEMONSTRATION CONTEXT: LESSON PREPARATION AND PRESENTATION

Student's Name.

SUMMARY EVALUATION OF LESSON PRESENTATION

Lesson Number Date

We judge each of the lesson _ -elements that follow to be of UNACCEPTABLE ACCEPTABLE

(circle one) QUALITY QUALITY

OBJECTIVES

ADAPTING THE LESSON TOLEARNER CHARACTERISTICS 1

QUALITY OF PRESENTATION

Classroom Management(Transition, Termination, andStudent Attention)

Instruction(Materials, Procedures, andOrganization)

EVALUATION

Preassessment

Feedback during the lesson - :-

Feedback after the lesson

Achievement of desired learningoutcomes

PLANNING NEXT STEPS

THE FIT BETWEEN INSTRUCTOR,LESSON, AND CONTEXT

------ ..... --

---._

School Supervisor

81

OUTSTANDINGQUALITY

Student Presenter

College Supervisor

Content Specialist Student Recorder$ * *

Field Test Format 2Experimental Elementary Teacher Education Program.

Oregon College of EducationDecember 1972

.

DE

MO

NST

RA

TIO

N C

ON

TE

XT

: LE

SSO

N P

RE

PAR

AT

ION

AN

DPR

ESE

NT

AT

ION

LE

SSO

N P

RE

SEN

TA

TIO

N E

VA

LU

AT

ION

FO

RM

_St

uden

ts N

ame

Con

tent

Are

aD

ate

Scho

ol o

r C

olle

ge S

vper

viso

r

Les

son

Ele

men

t

OB

JEC

TIV

ES

Wer

e th

e le

arni

ng o

utco

mes

exp

ecte

d fr

om th

e le

sson

cle

arly

ex-

.pl

aine

d to

the

pupi

ls?

Wer

e th

e le

arni

ng o

utco

mes

exp

ecte

d fr

om th

e le

sson

und

erst

ood

by th

e pu

pils

?

Did

the

pupi

ls v

iew

the

expe

cted

lear

ning

out

com

es a

s ap

prop

ri-

ate

and

wor

thw

hile

?

AD

APT

ING

TH

E L

ESS

ON

TO

LE

AR

NE

RC

HA

RA

CT

ER

IST

ICS

Was

the

less

on c

arri

ed o

ut in

a m

anne

r th

at r

efle

cted

ada

ptat

ion

to p

upil

char

acte

rist

ics?

QU

AL

ITY

OF

PRE

SEN

TA

TIO

N

Was

the

tran

sitio

n be

twee

n th

e pr

evio

us le

arni

ng a

ctiv

ity a

ndth

e pr

esen

t les

son

effe

ctiv

e?

Was

the

atte

ntio

n of

the

pupi

ls m

aint

aine

d th

roug

hout

the

less

on?

Wer

e in

stru

ctio

nal m

ater

ials

, pro

cedu

res,

and

org

aniz

atio

nal

stra

tegi

es u

sed

adva

ntag

eous

ly?

Was

the

term

inat

ion

and

:'wra

p up

" of

the

less

on e

ffec

tivel

yha

ndle

d?

Eva

luat

or's

Jud

pien

tT

he B

ehav

it ,r

Of

The

Stu

dent

and

or

Sugg

estio

ns(c

heck

0,..

Pupi

ls T

hat L

ed Y

ou T

o M

ake

The

for

Judg

men

t You

Hav

e M

ade

Impr

ovem

ent

Insu

ffic

ient

'.

i

,E

vide

nce

Yes

No

To

Judg

eg

!i

'I

t r

Fiel

d T

est F

orm

at 2

Exp

erim

enta

l Ele

men

tary

Tea

cher

_E

duca

tion

Prog

ram

Ore

gon

Col

lege

of

Edu

catio

n*

Dec

embe

r 19

72

LE

SSO

N P

RE

SEN

TA

TIO

N E

VA

LU

AT

ION

FO

RM

(co

ntin

ued)

Les

son

Ele

men

t

EV

AL

UA

TIO

N

Was

the

stud

ent's

pre

less

on a

sses

smen

t of

pupi

ls e

ssen

tially

ac-

cura

te?

Was

the

feed

back

pro

vide

d to

pup

ils a

bout

thei

r pe

rfor

man

cedu

ring

the

less

on a

dequ

ate?

Was

the

feed

back

pro

vide

d to

pup

ils a

bout

thei

r pe

rfor

man

ceaf

ter

the

less

on a

dequ

ate?

Wer

e th

e de

sire

d le

arni

ng o

utco

mes

fro

m th

e le

sson

rea

lized

?

Wer

e th

ere

unex

pect

ed o

utco

mes

fro

m th

e le

sson

that

over

-sh

adow

ed th

e de

sire

d le

arni

ng o

utco

mes

?

PI A

NN

ING

NE

XT

ST

EPS

In ta

king

with

the

stud

ent a

fter

the

less

on w

as h

e or

she

abl

e to

iden

tify

next

app

ropr

iate

ste

ps f

or p

upils

?

TH

E F

IT r

7T

WE

EN

IN

STR

UC

TO

R,

LE

SSO

N, t

i'I

CO

NT

EX

T

Did

the

!ess

oa a

s pr

esen

ted

fit w

ell i

nto'

the

ongo

ing

inst

ruct

iona

lpr

ogra

m o

f th

e sc

hool

?

Did

the

less

on a

s pr

esen

ted

fit w

ell t

he s

tude

nt w

ho p

rese

nted

it?

r.

Eva

luat

or's

Jud

gmen

t(c

ited:

one

)

Yes

IIn

suff

icie

ntE

vide

nce

No

To

Judg

e

The

Beh

avio

r O

f ne

Stu

dent

and

/or

Sugg

estio

nsPu

pils

Tha

t Led

You

To

Mak

e T

hepr

Judg

men

t You

Hav

e M

ade

.Im

prov

emen

t

0.11

....

._

a.

1

Fiel

d T

est F

orm

at 2

Exp

erim

enta

l Ele

men

taiy

Tea

cher

Edu

catio

n Pr

ogra

mO

rego

n C

olle

ge o

f E

duca

tion

Dec

embe

r 19

72

FIE

LD

TE

ST F

OR

MA

T #

3

CO

MPE

TE

NC

Y D

EM

ON

STR

AT

ION

CO

NT

EX

T: L

ESS

ON

TE

AC

HIN

G

CO

MPE

TE

NC

Y C

LU

STE

R I

, PL

AN

NIN

G A

ND

PR

EPA

RIN

G F

OR

IN

STR

UC

TIO

N

Stud

ent

Les

son.

Num

ber

Dat

e

Eva

luat

or

The

fir

st f

orm

al d

eino

nstr

atio

n of

teac

hing

com

pete

ncy

in th

e el

emen

tary

teac

her

educ

atio

n pr

ogra

m a

tO

CE

take

s pl

ace

in th

e co

ntex

tof

less

on te

achi

ng. T

hree

less

ons

are

taug

ht f

or p

urpo

ses

of c

ompe

tenc

y as

sess

men

t, an

d th

ree

clus

ters

of

com

pete

ncie

s ar

eas

sess

ed in

each

less

on: P

lann

ing

and

Prep

arin

g fo

r In

stru

ctio

n,-

Perf

orm

ing

Inst

ruct

iona

l Fun

ctio

ns, P

erfo

rmin

g A

sses

smen

tFu

nctio

ns. A

fou

rth

clus

ter

of te

achi

ng c

ompe

tenc

ies,

Dis

play

ing

Pupi

l 'A

chie

vem

ent,

is a

sses

sed

in a

t lea

st o

ne o

f th

e th

ree

less

ons.

Perf

orm

ance

sta

ndar

dsan

d st

atem

ents

of

proc

edur

e th

at a

ccom

pany

com

pete

ncy

dem

onst

ratio

n in

the

cont

ext o

f le

sson

s ar

e de

scri

bed

in "

The

OC

E G

uide

toC

ompe

tenc

y A

sses

smen

t in

Les

son

Tea

chin

g."

TH

E F

OR

MS

AT

TA

CH

ED

AR

E T

O B

E U

SED

IN

EV

AL

UA

TIN

G P

LA

NS

FOR

EA

CH

LE

SSO

N T

HA

T I

S T

O B

ET

AU

GH

T

The

Ele

men

tary

Tea

cher

Edu

catio

n Pr

ogra

mO

rego

n C

olle

ge o

f E

duca

tion

Mon

mou

th, O

rego

nJu

ly 1

973

p

fa CD

CO

MPE

TE

NC

Y D

EM

ON

STR

AT

ION

CO

NT

EX

T: L

ESS

ON

TE

AC

HIN

GC

OM

PET

EN

CY

CL

UST

ER

I. P

LA

NN

ING

AN

D P

RE

PAR

ING

FO

R I

NST

RU

CT

ION

(Atta

ch to

you

r L

esso

n Pl

an a

nd L

esso

n Pl

an E

valu

atio

n Fo

rm)

Stud

ent's

Nam

e

RE

CO

RD

OF

NE

GO

TIA

TIO

N

Les

son

Num

ber

APP

RO

VA

L T

O N

EG

OT

IAT

E A

FO

RM

AL

LE

SSO

N P

LA

N W

ITH

AA

PPR

OV

AL

TO

PR

EPA

RE

A F

OR

MA

L L

ESS

ON

PL

AN

.SC

HO

OL

SU

PER

VIS

OR

Col

lege

Sup

ervi

sor

Dat

e

CO

NT

EX

T D

FSC

RIP

TIO

N.

Scho

ol S

uper

viso

r

STU

DE

NT

S T

O B

E -

TA

UG

HT

CO

NT

EN

T T

O R

E T

AU

GH

T'

Scho

olA

rea

Gra

deE

xpec

ted

Lea

rnin

g O

utco

mes

Num

ber

Spec

ial C

hara

cter

istic

s

DA

TE

(S)

OF

LE

SSO

N P

RE

SEN

TA

TIO

N

Dat

e

TIM

E(S

) O

F L

ESS

ON

PR

ESE

NT

AT

ION

SUM

MA

RY

EV

AL

UA

TIO

N O

F L

ESS

ON

PL

AN

(Obt

ain

only

aft

er a

ll el

emen

ts o

f yo

ur p

lan

have

bee

n ev

alua

ted)

I ju

dge

the

plan

as

a w

hole

to b

e of

the

follo

win

g qu

ality

e e

nter

the

num

ber

in th

e bo

x be

low

you

r na

me

that

bes

t des

crib

es y

our

judg

men

t as

to th

e ov

eral

l qua

lity

ofth

e pl

an):

(1)

MIN

IMA

L

The

ele

men

ts o

f th

e pl

an a

re a

ttend

ed to

at a

leve

lof

det

ail.

inve

ntiv

enes

s. a

nd c

are

that

sug

gest

s a

bare

ly a

dequ

ate

mas

tery

of

the

'kno

wle

dge

and

skill

need

ed to

pre

sent

,the

less

on.

Scho

ol S

uper

viso

r

(2)

(3)

(4)

AD

EQ

UA

TE

The

ele

men

ts o

f th

e pl

an a

re a

ttend

ed to

at a

leve

lof

det

ail,

inve

ntiv

enes

s, a

nd c

are

that

sug

gest

s a

suffi

cien

t mas

tery

of th

etkn

owle

dge

and

skill

nee

ded

to p

rese

nt th

e le

sson

.

Col

lege

Sup

ervi

sor

(5)

OU

TST

AN

DIN

G

The

ele

men

ts o

f th

e pl

an a

re a

ttend

ed to

at a

leve

lof

det

ail,

inve

ntiv

enes

s. a

nd c

are

that

sug

gest

s an

exce

ptio

nal m

aste

ry o

f th

e kn

owle

dge

and

skill

need

ed to

pre

sent

the

less

on.

Con

tent

Spe

cial

ist

(

Stud

ent's

Nam

e

C

DE

MO

NST

RA

TIO

N C

ON

TE

XT

: LE

SSO

N T

EA

CH

ING

LE

SSO

N P

LA

N E

VA

LU

AT

ION

FO

RM

(Be

sure

to a

ttach

to y

our

Les

son

Plan

and

Sum

mar

y E

valu

atio

n Fo

rm)

4L

esso

n N

umbe

r

DIR

EC

TIO

NS

TO

EV

AL

UA

TO

RS.

Whe

n ev

alua

ting

a le

sson

pla

n ra

te e

ach

of th

e B

OL

D F

AC

E it

ems

that

app

ear

on th

is f

orm

that

mee

tw

ith y

our

appr

oval

. If

an it

em d

oes

not m

eet w

ith y

our

appr

oval

. the

stu

dent

is o

blig

ated

-to

mod

ify

it un

til it

doe

s. A

ll ra

tings

are

to b

e en

tere

d in

the

boxe

spr

ovid

ed. R

AT

ING

S FO

R A

LI.

IT

EM

S T

HA

T A

PPE

AR

ON

TH

IS F

OR

M A

RE

TO

BE

BA

SED

UPO

N P

RO

DU

CT

S O

F A

STU

DE

NT

S B

EH

AV

IOR

, RA

TH

ER

TH

AN

UPO

N B

EH

AV

IOR

PER

SE

. Eac

h ra

ting

is to

ref

lect

one

of

the

follo

win

g ju

dgm

ents

: 1 I

)

MIN

IMA

L.

The

ele

men

t of

the

plan

is a

ttend

ed to

at a

leve

l of

de-

tail.

invf

ntiv

en,-

,,. a

nd c

are

that

sug

gest

s a

bare

lyad

ecir

acte

.pia

ster

of

the

know

ledg

e an

d sk

ill n

eede

d to

pres

ent t

he le

sson

.

43)

AD

EQ

UA

TE

The

ele

men

t of

the

plan

is a

ttend

ed to

at a

leve

l of

de-

tail.

inve

ntiv

enes

s, a

nd c

are

that

sug

gest

s a

suff

icie

ntm

aste

ry o

f th

e kn

owle

dge

and

skill

nee

ded

to p

rese

ntth

e le

sson

.

14)

:5)

OU

TST

AN

DIN

G

The

ele

men

t of

the

plan

is a

ttend

ed to

at a

leve

l of

de-

tail,

inve

ntiv

enes

s, a

nd c

are

that

sug

gest

s an

exc

ep-

tiona

l mas

tery

of

the

know

ledg

e an

d sk

ill n

eede

d to

pres

ent t

he le

sson

..

0E

LE

ME

NT

S O

F T

HE

PL

AN

EV

AL

UA

TO

R O

F T

HE

PL

AN

Scho

olSu

perv

isor

Col

lege

Supe

rvis

orC

onte

ntSp

ecia

list

CO

MM

EN

TS

P

OB

JEC

TIV

ES

AN

D I

ND

ICA

TO

RS

Are

the

lear

ning

out

com

es e

xpec

ted

from

the

less

on c

lear

ly s

tate

d?(E

xpec

ted

outc

omes

nee

d no

t be

stat

ed in

the

form

of

"beh

avio

ral"

obje

ctiv

es)

Are

they

wor

thy.

hile

out

com

es, g

iven

the

char

acte

rist

ics

of th

e pu

pils

to b

etau

ght?

Are

the

indi

cato

rs o

f ou

tcom

e ac

hiev

emen

t ide

ntif

ied?

'

INST

RU

CT

ION

AL

AC

TIV

ITIE

S. M

AT

ER

IAL

S, A

ND

PR

OC

ED

UR

ES

Are

the

lear

ning

act

iviti

es to

be

purs

ued

in th

e le

sson

cle

arly

iden

tifie

d?A

re th

ey lo

gica

lly r

elat

ed to

the

lear

ning

out

com

es d

esir

ed f

rom

the

less

on?

Are

they

app

ropr

iate

ly s

eque

nced

?N

. Are

the

inst

ruct

iona

l mat

eria

ls a

nd p

roce

dure

s to

be u

sed

In th

e le

sson

cle

arly

.iden

tifie

d?A

re th

ey a

ppro

pria

te to

the

lear

ners

to b

e ta

ught

and

the

outc

omes

to b

eac

hiev

ed?

f

- C

OM

PET

EN

CY

DE

MO

NST

RA

TIO

N C

ON

TE

XT

: LE

SSO

N T

EA

CH

ING

CO

MPE

TE

NC

Y C

LU

STE

R I

I; P

ER

FOR

MIN

G N

STR

UC

TIO

NA

LFU

NC

TIO

NS

CO

MPE

TE

NC

Y I

ND

ICA

TO

RS:

ST

U E

NT

BE

HA

VIO

R

Snid

eln.

.t117

1eL

esso

n N

umbe

rD

ute

Pres

ente

dT

une

Pres

ente

d.

.

DIR

EC

TIO

NS

TO

EV

AL

UA

TO

RS.

Whe

n es

alua

ting

a le

sson

pre

sent

atio

n-ra

te e

ach

of th

e in

stru

ctio

nal f

unct

ions

tbol

d fa

ce A

VM

, tha

t app

ear

on th

is f

orm

. All

Fatin

g; a

re to

11e

ente

red

in th

e bo

xes

pros

ided

. RA

TIN

GS

FOR

IT

EM

SL

IST

ED

ON

TH

E F

IRST

TW

O P

AG

ES

OF

TH

E F

OR

M A

RE

TO

BE

BA

SED

UPO

N S

TU

DE

NT

BE

HA

VIO

R. T

hese

ratin

gs a

rc to

ref

lect

one

of

use

poss

ible

judg

men

ts:

, 1)

:21

UN

AC

CE

PTA

BL

E.

The

beh

avio

r of

she

stu

dent

in p

rese

ntin

g th

e le

sson

The

beh

avio

r of

the

stud

ent i

n pr

esen

ting

the

less

onre

tne

insu

ffic

ient

leve

l of

thor

ough

nes

;'. in

vent

ive-

refl

ects

an

adeq

uate

leve

l of

thor

ough

ness

, inv

entiv

e-,

rim

.d

care

in r

elat

ion

to th

e fu

nctio

n.-,

ness

. and

car

e in

rel

atio

n to

the

func

tion.

'.

...._

(31

AC

CE

PTA

BL

E

INST

RL

CT

IO N

AL

FU

N-c

m N

CO

NV

EY

ING

TH

E O

BJE

CT

IVE

S O

F IN

STR

UC

TIO

N

Was

ther

e al

auc

mpt

to c

oney

the

obje

ctiv

es o

f th

e le

sson

. to

pupi

l's?

AD

APT

ING

IN

STR

UC

TIO

N T

O C

ON

TE

XT

sr UW

ere

the

obje

ctis

es

of th

e le

sson

ada

pted

to f

it pu

pil

resp

onse

s to

the

less

on?

Wer

e th

e ob

ject

ives

of

the

less

on a

dapt

ed to

fit

the

chir

acte

rist

ics

of th

e in

stru

ctio

nal s

ettin

g ge

nera

lly?

Wer

e in

stru

ctio

nal p

roce

dure

s an

d ac

tiviti

es a

dapt

edto

fit

pupi

lre

spon

ses

to th

em?

Wer

e in

stru

ctio

nal p

roce

dure

s an

d ac

hviti

es a

dapt

edto

fit

the

char

acte

rist

ics

of th

e in

stru

ctio

nal s

ettin

g ge

nera

lly?

Wer

e m

ater

ials

ada

pted

to f

it th

e ch

arac

teri

stic

s of

the

inst

ruc-

tiona

l set

ting?

S.4.

11PL

E I

ND

ICA

TO

RS

k.15

)SU

PER

IOR

The

beh

avio

r of

the

stud

ent i

n pr

esen

ting

the

less

onre

flec

ts a

n ex

eept

ivna

l lev

el o

f th

orou

ghne

ss. i

ns e

ntiv

e-ne

ss, a

nd c

are

in r

elat

ion

to th

e fu

nctio

n.

Posi

tive

to N

egat

ive

Exa

mpt

ei

Obj

ectiv

es o

f th

e le

sson

are

des

crib

edSt

eps

are

take

n to

insu

re o

bjec

tives

are

und

erst

ood

Ref

eren

ce is

mad

e to

obj

ectiv

es d

urin

g th

e le

sson

Obj

ectiv

es a

re n

ot s

tate

dO

bjec

tives

are

sta

ted

once

but

not

ref

erre

d to

aga

inO

bjec

tives

are

sta

ted,

but

lear

ning

act

iviti

esar

e no

t cle

arly

re-

late

d to

them

Posi

tive

to N

egat

ive

Exa

mpl

es

O6j

ectiv

es a

re m

odif

ied

or d

iffe

rent

iate

d fo

r pu

pils

on th

e ba

sis

ofpu

pil r

espo

nse

to th

e le

sson

Obj

ectiv

es o

r pf

oced

ures

mod

ifie

d to

acc

omm

odat

e un

ex-

pect

ed e

vent

s; e

.g.,

a sn

owf

II o

r a

fire

mea

rby

Mat

eria

ls o

rigi

nally

pla

nefo

r us

e in

the

less

on a

re d

isca

rded

or m

odif

ied

Obj

ectiv

es, m

ater

ials

, or

toce

dure

s ar

e no

t mod

ifie

d,ev

enth

ough

pup

il re

spon

se to

t-1

indi

cate

s th

ey a

re in

appr

opri

ate

Obj

ectiv

es, m

ater

ials

, or

proc

edur

es a

re n

ot m

odif

ied

toac

com

-`m

odat

e un

antic

ipat

ed e

vent

.

IND

ICA

TO

RS

TH

AT

SE

RV

ED

AS

.4 B

ASI

S FO

R J

UD

GM

EN

T

c7

CO

M M

EN

T

CO

MM

EN

T

PER

FOR

AM

G-

INST

RU

CT

ION

AL

FU

NC

TIO

N-P

AG

E 4

TH

E R

AT

ING

TH

AT

IS

TO

BE

PR

OV

IDE

D O

NT

HIS

PA

GE

IS

STIL

L T

O R

EFL

EC

T O

NE

OF

TH

EFO

LL

OW

ING

JU

DG

ME

NT

S:

tl)U

NA

CC

EPT

AB

LE

No

mor

e th

an h

alf

the

pupi

ls b

eing

taug

ht b

ehav

ed in

way

s w

hich

indi

cate

d th

e in

stru

ct?

nal f

unct

ion

was

bein

g ca

rrie

d"ou

t eff

ectiv

ely.

(2)

%(4

)(5

).

AC

CE

PTA

BL

ESU

PER

IOR

v

r!

Thr

ee-f

ourt

hsor

so

of th

e pu

pils

bei

ng ta

ught

beh

aved

All

or n

earl

y al

l of

the

pupi

ls b

eing

taug

ht b

ehav

ed in

in w

ays

whi

ch in

dica

ted

the

inst

ruct

iona

l fun

ctio

n w

asw

ays

whi

ch in

dica

ted

the

inst

ruct

iona

lfu

nctio

n w

as

bein

g ca

rrie

d ou

t eff

ectiv

ely.

bein

g ca

rrie

d ou

t eff

ectiv

ely.

INS

TR

UC

TIO

NA

L F

UN

I:110

1V

° IN

DIC

AT

OR

S T

HA

T S

ER

VE

D A

SS

AM

PLE

IND

ICA

TO

RS

A B

AS

IS F

OR

JU

DG

ME

NT

MA

NA

GIN

G U

NE

XPE

CT

ED

EV

EN

TS

Wor

e po

tent

ially

dis

rupt

ive

even

ts e

ffec

tivel

y m

anag

ed?

Posi

tive

to N

egat

ive

Exa

mpl

esU

nexp

ecte

d vi

sito

rs o

r un

usua

l eve

nts

do n

ot c

ause

und

ue d

is-

rupt

ion

in le

arni

ng a

ctiv

ities

Whe

n a

child

is d

isru

ptiv

e th

e re

spon

se o

f th

e st

uden

t to

him

doe

sno

t add

to h

is d

isru

ptiv

enes

s ,o

r ca

use

it to

spr

ead

toot

hers

A c

hild

who

dis

rupt

s te

nds

not t

o be

dis

rupt

ilit a

sec

ond

orth

ird

time

Whe

n a

child

is f

eelin

g an

gry

or u

pset

or

afra

id th

e st

uden

t'sre

spon

se to

hin

t doe

s no

t int

ensi

fy h

is f

eelin

gs, o

r ca

use

them

tosp

read

to o

ther

sN

egat

ive

indi

cato

rs a

re r

efle

cted

-in

beha

vior

that

is th

ere

vers

e of

the

abov

eC

OM

ME

NT

00 00

CO

MPE

TE

NC

Y D

EM

ON

STR

AT

ION

CO

NT

EX

T: L

ESS

ON

TE

AC

HIN

G

CO

MPE

TE

NC

Y C

LU

STE

R H

I, P

ER

FOR

MIN

G -

ASS

ESS

IVE

WFU

NC

TIO

NS

CO

MPE

TE

NC

Y I

ND

ICA

TO

RS:

ST

UD

EN

T. B

EW

MO

R

Stud

ent's

Nam

eL

esso

n N

umbe

rD

ate

Pres

ente

dT

ime

Pres

ente

dSu

perv

isor

or

C'o

;:ten

t Spe

cial

ist's

.Cam

eD

IRE

CT

ION

S T

O E

VA

LU

AT

OR

S. W

hen

eval

uatin

g a

less

on p

rese

ntat

ion

rate

eac

h of

the

asse

ssm

ent f

unct

ions

(bo

ld f

ace

item

s) th

at a

ppea

r on

this

for

m. A

ll ra

ting'

are

tobe

ent

ered

in th

e bm

es lo

iovi

ded.

RA

TIN

GS

FOR

AL

L I

TE

MS

TH

AT

APP

EA

R O

N T

HIS

FOR

M A

RE

TO

BE

BA

SED

UPO

N S

TU

DE

NT

BE

HA

VIO

R. T

hese

rat

ing.

, are

tore

flec

t one

of

five

poss

ible

judg

men

ts:

t:(

I i

i2)

(3)

(4)

(5)

UN

AC

CE

PTA

BL

EA

CC

EPT

AB

LE

SUPF

RI

3R-

:

The

beh

avio

r of

the

stud

ent i

n pr

esen

ting

the

less

onT

he b

ehav

ior

of th

e st

uden

t in

pres

entin

gth

e le

sson

The

beh

avio

r of

the

stud

ent

Er

pres

entin

g th

e le

ssor

.t l

iets

.an

insu

ffic

ient

leve

l of

thor

ough

ness

, inv

entiv

e-re

flet

- .3

an

adeq

uate

leve

l of

thor

ough

ness

,in

vent

ive-

refl

ects

an

exce

ptio

nal l

evel

of

horo

ughn

ess,

inve

ntiv

e-ne

ss. a

ild c

are

in r

elat

ion

to th

e fu

nctio

n.:le

ss. a

nd c

are

in r

elat

ion

to th

e fu

nctio

n.ne

ss, a

nd c

are

in r

elat

ion

W ti

e fu

nctio

n.

-.4

SSE

SSM

EN

T F

UN

CT

ION

SAM

PLE

IN

DIC

AT

OR

S

ASS

ESS

ING

LE

AR

NIN

G B

EFO

RE

TH

E L

ESS

ON

Wer

e st

eps

take

n be

fore

the

less

on b

egan

, or

earl

y in

the

less

on,

to id

entif

y w

here

pup

ils s

tood

in r

elat

ion

to th

e le

arni

ngou

t-co

mes

exp

ecte

d fr

om th

e le

sson

?

If s

uch

anm

ent w

as m

ade,

was

it c

arri

ed o

ut w

ith r

easo

n-ab

leie

ncy

and

accu

racy

?

Wer

e ob

ject

ives

, act

iviti

es, o

r pr

oced

ures

in th

e le

sson

ada

pted

tore

flec

t the

info

rmat

ion

that

cam

e fr

om th

eas

sess

men

t?

Posi

tive

to N

egat

ive

Exa

mpl

es

A p

rete

st o

ver

the

lear

ning

out

com

es e

xpec

ted

from

the

less

on is

adm

inis

tere

d pr

ior

to th

e le

sson

The

chi

ldV

fs r

egul

ar te

ache

r is

ask

ed f

orhe

r ju

dgm

ent a

s to

whe

re th

e-pu

pils

sta

nd r

elat

ive

to th

e ex

pect

ed le

arni

ngou

tcom

es\ O

ne o

f th

e fi

rst a

ctiv

ities

unde

rtak

en in

the

less

on is

an

"in-

1,

fari

na!"

ass

essm

ent o

f w

here

the

child

ren

stan

dre

lativ

e to

suc

hou

tcom

esW

hate

ver

the

form

of

asse

ss in

ent,.

it is

car

ried

out

in a

rel

ativ

ely

shor

t per

iod

of ti

me

and

with

out u

ndue

atte

ntio

nbe

ing

calle

d to

itIn

form

atio

n co

min

g fr

om th

e as

sess

men

t is

acte

dup

on in

des

ign-

ing

the

less

onN

eith

ei a

for

mal

or

info

rmal

ass

essm

ent i

s m

ade

ofpu

pil l

earn

ing

befo

re th

e le

sson

is u

nder

take

nIf

an

asse

ssm

ent i

s-ft

dle-

itis

inef

fici

ently

or in

effe

ctiv

ely

carr

ied

out;

for

exam

ple,

it r

equi

res

too

Muc

h tim

e, g

ains

too

muc

hat

ten-

tion,

or

the

info

rmat

ion

that

is o

btai

ned

isin

accu

rate

or

in s

ome

othe

r w

ay u

nsat

isfa

ctor

yIn

form

atio

n is

obt

aine

r; b

ut th

ere

is n

o in

dica

tion

that

itis

act

edup

on

.,

IND

ICA

TO

RS

HA

T S

ER

VE

D A

SB

ASI

S FO

R J

UD

GM

EN

T

CO

MM

EN

Tt

C

0

ti

v

a

CO

MPE

TE

NC

Y D

EM

ON

STR

AT

ION

CO

NT

EX

T:

LE

SSO

NT

1AC

HIN

G

CO

MPE

TE

NC

Y C

LU

STE

R I

V, D

ISPL

AY

ING

_PU

PIL

AC

HIE

VE

ME

NT

CO

MPE

TE

NC

Y I

ND

ICA

TO

RS:

PR

OD

UC

TS

OF

A S

TU

DE

NT

'SB

EH

AV

IOR

4

Iz C

Stud

ent's

Nam

eL

esso

n N

umbe

rD

ate

Pres

ente

d3n

perv

iirT

rolto

titer

ii Sp

erru

hsii7

CaT

ria7

DIR

EC

TIO

NS

TO

EV

AL

UA

TO

RS.

Whe

n ev

alua

ting

a le

sson

pre

sent

atio

n ra

teea

ch o

f th

e ac

hiev

emen

t dis

play

fun

ctio

n,bo

ldfa

ce it

enis

i tha

t app

ear

on th

is f

orm

. All

ratin

gs

are

to b

e en

tere

d in

the

bOxe

s pr

ovid

ed. R

AT

ING

SFO

R A

LL

IT

EM

S T

HA

T A

PPE

AR

ON

TH

IS F

OR

M A

RE

TO

BE

BA

SED

UPO

N P

RO

DU

CT

S O

E A

ST

UD

EN

T'S

BE

-H

AV

IOR

. RA

TH

ER

TH

AN

UPO

N B

EH

AV

IOR

PE

R S

E. T

hese

rat

ings

are

tore

flec

t one

of

tive

poss

ible

Alg

men

ts:

IU

NA

CC

EPT

AB

LE

The

pro

duct

s of

the

stud

ent's

eff

ort t

o di

spla

y le

arni

ngou

tcom

es r

elat

ed to

the

less

on r

efle

ct a

n in

suff

icie

ntle

vel o

f th

orou

ghne

ss. i

nven

tiven

ess.

and

car

e in

re-

latio

n to

this

teac

hing

fun

ctio

n.

3,:4

,A

CC

EPT

AB

LE

The

pro

duct

s of

the

stud

ent's

eff

ort t

o di

spla

y le

arni

ngou

tcom

es r

elat

ed to

the

less

on r

efle

ct a

n qd

equa

tele

vel o

f th

orou

ghne

ss. i

nven

tiven

ess,

and

car

e an

re-

latio

n to

this

teac

hing

fun

ctio

n.

5,SU

PER

IOR

The

pro

duct

s of

the

stud

ent's

eff

ort t

o di

spla

y le

arni

ngou

tcom

es r

elat

ed to

the

less

on r

efle

ct a

n ex

tern

:0nd

leve

l of

thor

ough

ness

. inv

entiv

enes

s. a

nd c

are

in r

e-la

tion

to th

is te

achi

ng f

unct

ion.

..

AC

HIE

VE

ME

NT

DIS

PLA

Y F

UN

CT

ION

SAM

PLE

IN

DIC

AT

OR

S

DIS

PLA

YIN

G P

R E

LE

SSO

N A

ND

PO

STL

ESS

ON

,Po

sitiv

e to

Neg

ativ

e-E

xam

ples

AC

HIE

VE

ME

NT

Wer

e tb

e re

sults

of

the

asse

ssm

ent b

efor

e th

e le

sson

rec

orde

dR

esul

ts o

f at

leas

t the

pos

ttess

on a

sses

smen

ts o

f le

arni

ng o

ut-

and

uisp

laye

d in

suc

h a

way

that

they

wer

e ea

sily

inte

rpre

ted?

com

es a

re r

ecor

ded

and

disp

laye

d so

that

any

one

look

ing

at th

e(P

leas

e at

tach

the

disp

lay

if o

ne h

as b

een

prep

ared

; sam

ple

fin

form

atio

n ca

n ea

sily

det

erm

ine

whe

te e

ach

pupi

l sta

nds

With

disp

lay

form

s ar

e fo

und

in th

e "O

CE

Gui

de to

Com

pete

ncy

resp

ect t

o th

e le

arni

ng o

utco

mes

exp

ecte

d fr

om th

e le

sson

Ass

essm

ent i

n L

esso

n T

each

ing"

);

Post

less

on le

arni

ng s

core

s ar

e su

mm

ariz

ed to

sho

w th

e pr

opor

-L

ion

of p

upils

who

lichi

eved

the

lear

ning

out

com

es e

xpec

ted

from

the

less

on, a

nd th

e pr

optir

tion

that

did

not

Wer

e th

e re

sults

of

the

asse

ssm

ent a

fter

the

less

on r

ecor

ded

and

Prel

esso

n an

d po

stle

sson

ass

esen

t inf

orm

atio

n is

not

rec

orde

ddi

spla

yed

in s

uch

a w

ay th

at th

ey w

ere

easi

ly in

terp

rete

d?or

dis

play

ed(P

leas

e at

tach

the

disp

lay

if o

ne h

as b

een

prep

ared

: sam

ple

Post

less

on a

chie

vem

ent d

ata

arei

disp

laye

d bu

t are

not

sum

mer

-di

spla

y fo

rms

are

foun

d in

the

"OC

E G

uide

to C

ompe

tenc

yiz

ed to

ref

lect

the

prop

ortio

n O

P pu

pils

who

ach

ieve

the

leai

ning

Ass

essm

ent i

n L

esso

n T

each

ing"

)ou

tcom

es e

xpec

ted

from

the

less

on, o

r th

e pr

opor

tion

of s

tude

nts

Iw

ho f

ail t

o do

so

IND

ICA

TO

RS

TH

AT

SE

RV

ED

AS

A B

ASI

S FO

R J

UD

GM

E\T

CO

MM

EN

T

a

X

a

Estimating Costs of a Competency-OrientationBy

BRUCE JOYCE

low do we build a 'conlpet :ney-oiicnted certificationand training ystem? Personally, I could never g$,I.

through all the necessary complicated steps, althoughthere are people here who could. I'm much more inter--ested in trying to find out wharteaching skills will payoff; whether.theyhave to be idiosyncratic; whether any-body ,can even train us all in the same ones. It's con-ceivable that we all may not he capable Ihlf the sameones; maybe in the long rtni we will have to do ourown thing as teacher competence, in the end, may turnout to be personal41r it may be external and train-able. Also, teachirg k elusive. You can measure 2,000reasonable-siiunding competencies and not have caughtthe essence of teaching. So far the research hasn't vali-dated many competencies, but I'm not prepared to giveit up.

To dete?mine competence, we also have to facesome rough questions. For example, we'r not evensure that as teachers we should be playing the roles wedo. I keep critizing th role of the self7contained class-room teacher mostly because it nearly killed,me. I hadto go into college teaching to keep alive, but I simplycouldn't teach six things to 36 kids who were that dif-ferent from each other. I didn't have the capacity forthe job, and I had to become a professor in orue. tosurvive. I'm not sure that I want to spend any timefinding competencies for the present roles teachersplay. We might do much better to design better schoolsand find out what competence we need to make themwork.

I know nothing about costs. OK? I couldn't cost-ac-count a major program, but last summer, some of theNew York State Education Department people askedme to help them estimate how much a competencyprogram would cost to develop. I agreed to do it, and1 wrote tnem the followingpaper.

Largely because of the massive research and devtg-opment effort entailed if competency -based trainingand certification are to he achieved, it has been sug-gested that development and implementation he cen-

91

tralized. An alternative to accommodate local needsand bring about a broad base of pariicipation iii devpl-opment is that the competency-orientation should berequired .and relatively small local consortia (probablymade up in eh case of a few school districts and col-leges) sta d.determine competency criteria, developprograms, and implement the programs and the certifi-cation procedures. The latter option is attractive be-cause it promises to involve so many people and to in-sure relevance to local school needs.

The centrally mandated system is inacceptable onboth political and substantive grounds: politically be-cause it would give .considerable power to a few per-sons (competensy-orientvd certification tin(' training, ismuch more likely to affect wh* will actually be perinit-Jed 10.1eacli than course-based certification and train-ing, however centralized), and substantively becausenot enough is yet known about the identification ofteaching competencies to permit anyone to develop awidely mandated set of competencies for any particularcaw:, i y of teachers with any great. confidence, thatthose competencies will stanl tile.

Centering development a.aund Many local centers isacceptable as a process, but is not Without- seriousproblems. The more local centers there are, the 'Moredifficult it will be to organize teams' of sufficient exper-tise to develop really strong training and certificationsystems. Major universities,"working with strong schooldistricts, have had trouble doing this. Yet, strong de-velopment teams are net essary. High.quality in a com-petency-orientation is essential because both certifica-ttc.a and training procedures will be more powerful andmistakes (such as emphasizing trivial ciuivetencies)will be magnified.

Many districts and smaller colleges are unaccus-tomed to freeing personnel for development, but a sus-tained effort by a large team is essential if the task ofidentifying competencies, organizing training andsessment, and implementing t,trtificidion proceou,to he done effectively.

92

Without coordination and quali.v control, broad par-ticipation in the program development could be anendless and ineffective process.

PurposeIn this Paper an attempt is made to identify the

tasks of creating a competency-based .:acher trainingand certification system and to estimai, the costs ofcompleting the tasks adequately. In addi ion, a base isprovided for estimating the effect on co. is of severalpossible options for organizing the proses; of develop-ment and implemental. This can be used as a basefor estimating cost when several possible )ptions fororganizing the process include broad local input into

the system. To estimate basic technical cost., an esti-

mate is made of the cost of d2colopment by a compact;independent team of experts who would siltelA, accom-plish the technical tasks of producing a rototypetraining-certification system. This option (referred toas process option A) is not a realistic choke, butyields a fairly reliable cost estimate from the technicalpoint of view,',Which permits an estimate of the cyst ifseveral possible organizational processes were em-.

ployed: strictly yea! developmehit (process B),local development with central technical assistance anddevelopment services (process C), and a general statesystem with provision for local options (process D).

Source of Cost EstimatesHow do we arrive at the costs of the tasks and sub-

sequently, the; costs of the process options? There arethree sources of our estimates. One is the experience ofthe Bureau of Research Teacher Education Projqt, es-petially.the costing procedures included in their feasi-bility studies. The second is the cost of developed per-formance-based materials such as those in "Materialsfor Modules" (appendix .13). These two sources can

help us determine therbase costs represented in processoption A (the least desirable process option in termsof creating a statewide training-certification systeni).The third source is the least reliable and represents es-timates of the probable increase in the base cost due tothe greater complexity and duplication of effort of thg

other process options.

Technical Taski and Process OptionsCertification and training are interrelated parts of

the same system which serves as the basis for pi eser-vice teacher education. inservice teacher education, and

the certifying and diagnostic procedures related to bothof them.

The purpose of the developmenteof contpetency-ori-ented systems is to unite preservice education, initialcertification, inservice education, and continuing or ex-tended certification around a systetit of teaching com-petencies which form the goals of preservice education,the standards for initial and extended licenses, and thebasis for diagnosis of performance of inservice teachersand prescription of inserviee training. It is probablynot possible or desirable to separate licensing or.train-ing at the preservice or inservice levels under the com-

petency orientation.The entire system for competency-oriented certifica-

tion and training depends on the creation of fourinterrelated storage and retrieval systems plus the orga-nizational and communications networks necessary tocreate and implement them. The creation of these sys-tems and networks constitutes the technical side of thedevelopment process. The creation and management ofthe organization necessary to create the systems consti-tute the process side. Both technical and process sidespresent options which affect costs greatly.

The Technical SideThe technical side of competency-based certification

and training involves the creation and validation offour systems, These are as follows:

A storage and retrieval system.of the specifications of

teacher competency.

Since teaching is complex the number of competen-cies which are likely to be specified is large indeed.The Bureau of Research Models averaged between2,000 and 3,000 competency specifications and theserepresented efforts by single t.,ified institutions orsmall consortia rather than statewide consortia of di-verse institutions and representation. It seems reason-able to suppose that the number of competencies willincrease as the political base for establishing them isbroadened. Hence, when teacher associations, repre-sentatives of school administrations, the public and stn=dents all contribute, as well as expert teams from uni-

versities and state departments of education, the

number of potential competencies of a teacher will belarge and the process of identifying the most importantcompetencies may be complex.

ii storage and retrieval system of mediated instruc-tional systems and agent-mediated components de-

cigned to prodm e the competencies.

P.

.

This storage and retrieval system represents themeans of teacher education. The number of itemsstored as .instructional systems will be equal to orsomewhat greater than the number of competencieswhich are specified. Also, the extensiveness of each in-structional system is much greater than the specifica-tion of any competency. Thus, the production of soft-ware to fill?this systeni'will be an extensive task. Patmore simply, it will he difficult to specify critical com-petencies, but developing, the program...elements toachieve them will btt.even more complex..

A system (4 assesstraPnt devices designed to determinethe effects of the .nsirutional systems, and agency-mediated components and to measure thee competenciesspecified in system A.

Without an assessment system competency-basedcertification would be impossible as would be the as-sessment of the effectiveness of 'program elements.Properly organized, the assessment system provides adiagnostic profile of the teacher candidate and providesalso the means for tracking his progress to determinewhen certification should be granted. In the e; of in-service teachers, it provides the means for diagnosingthe state of their competency and relating them to theinstructional systems to be uscd.in inservice education.

A management system for interrelating subsystems 1.2, and 3 above.

The magnitude, of the three other systems requiresthe use of a contemporary management system for di-agnosis, prescription, tracking and progress, and pro-viding feedback to teacher candidates, teachers; andprogram and licensing managers. Without a complexautomated management system individualization orpersonalization would be impossible in a program andso would he implementation, for the complexity ofspecifications and training devices is .so great as to de-feat any presently available option for program control.Without a modern information system chaos wouldsurely ensue. at the point of implementation. It is possi-ble to imagine a statewide managemegts'Rstem whichindividual teacher training institutes would time to guidethem In the identification of competencies. and theymight use a central system to withdrAw instructionalsystems and assessment devices for their particulartraining program.

The research and development effort needed to pro-duce the four interrelated systems is enormous. Theyrepresent the ost of the substance of competency-based teacher education and licensing.

.93

The Process SideIf it is desired to have participation by all relevant

groups in the creation of the competency-based teachereducation and certification system then suitable corn-inunications networks and organizations have to be setup to permit teachers, students, teacher organizations,representatives of the public, subject matter experts,experts on teaching, and representatives of state de-partments and colleges to participate in competencyidentification, the selection of instructional systems,and the creation of the assessment systems. The proc-ess of creation can reasonably be divided into threelevels: specification of the systems, development, andimplementation. It is possible to imagine a state or na-tionwide network providing participation by all rele-vant groups in the specification level of the creation ofthe basic system. Development will probatjA have tohe organized at a few major centers, funded to bringtogether talent to create the operating systems. It ispossible to imagine a network stretching across the stateso that many _local units might contribute somethingto the development. However, development is an ex-ceedingly complex tasks especially if the products areto be tested as they are developed. Thus, a few majorcenters probably will do most of the actual productionof the instructional systems and testing device. How-ever, broad-based groups can evaluate the pioducts,and a representative group can monitor the entireoperation.

Minimum, General, Statewide Competenciesand Extended Local Competencies

It is possible to use some of the critical competencyspecifications for minimum statewide certificationthese could represent minimum standards for preser-vice and inservice training. A much larger number ofspecifications might be prepared representing localneeds. These would be used by individual teachertraining centers for training purposes and for determin-ing local certification in addition to state certification.Imagine, for example, a network of field centers repre-senting the local school districts, higher education insti-tutions, community representatives, teacher associa-tions. etc. Suppose that there are about 20 of tilt* invarious parts of the state. They produce competencyspecifications and agree on those which will be usedfor minimum specifications for any given type ofteacher (elementary, secondary, special education,etc.). Each of the local centers, however; would pro-duce specifications which were deemed important by

94

the people in that center but which were not selectedby statewide repwsentatives as essential for statewidecertification. 1 hese additional specifications nonethelesscould be stored in thespecifications storage system andinstructional systems could be designed to achieve

them and assessment systems to assess them. Thus acomplete statewide system, if it is to take care of localneeds. would have to he much larger than the systemwhich would produce only state requirements. Clearly'

the greater the participation of interested parties andthe more numerous .the local centers, the more costly

the creation of the system. From the point of view oforganization and communication and also from thepoint of view of the size of resulting system participa-tion and with provision for local options costs will in-

crease.This cost would' -be nothing like the cost of asking

each local unit or center including several local units tocreate their own competency-based teacher educationand certification system. The cost of building anystrong. competency-based program with local optionswill he $10 to $12 million (and that is probably theminimum which should be anticipated). It would cost$5 to 37 million for each local unit to create its own..system\ thus if there were 10 systems in the state eachdeveloped by local units, the costwould be in theneighborhood of $60 to $70 million. It would seem farmore reasonable. in cost to create a statewide systemthat had provisions for special loAl needs and inter-ests.

Technical Tasks and Process OptionsThe four tasks involved in the creation of a compe-

tency-based program, plus the development of a certiti-

0

cation system, represent the five tasks required toactivate a statewide cpmpeteney-oriented training andcertification system. The cost of each of these will varycOlsiderably depending on the process options whichare selected. In table 1, four process options arc de-picted over the five tasks. 4.

Process A (developing one complete prototype sys-tem) is the lem costly option in terms of dollars, buthas the disadvantage that its statewide acceptancewould not be high, in all likelihood.

Process 13 (development of complete system by sev-

eral local centers) would be very costly. The cost ofeach complete system would he higher than that of asystem developed by an ."expert" team. (I estimate thatthe cost would be 50 percent higher) and that wouldhe multiplied by the-number of local centers.

Process C (local development supported by a fewdevelopment centers) would he somewhat less ,costly

than B because the development centers could reduce,duplication of effort.

Process Et (development of a statewide system with

local-optionsj would be much higher than A but muchlower than B or C and would result in an acceptablestatewide process and plan. It appears to be the bestcost option.

Process A is least costly because most compact. It isuseful, however, for cost estimates because it can yieldit base cost which can be multiplied by factors repre-senting the increased complexity and duplication of ef-fort required in the other options.

Process B results in several comparable systems. Itgives local needs the fullest play, but the use of nonex-pert teams escalates the cost of each system so that 4each local system would be very expensive and the cost

Table 1. Cost Factors of Technic41 'Casks by Process Options

C. Local Consortia D. Statewide

A. Prototype B. Local Consortia Plus Development System With

Sptem by Expert T'eam as Focus Assistance Local Options

1. Competency System 1.5 1.25 2

(no. of consortia) (no. of consortia)

2. Instructional System 1.5 1.25 2

3. Assessment System I 1.5 1.25 2

4. Management System 1 1.5 1.25

5. Certification System I 1.5 1.25 2

;1

within the state would be the cost of each system timesthe number of local systems.

Process C would he a hit less than II due to somecentralization of effort.

Process D, providing for wide local participation butcentral coordination to reduce duplication of effort,might only require about twice the investment of proc-ess A.

The Base Cost of the Technical TasksThe Bu 'eau of Research Models, especially the "fea-

sibility studies, provided cost estimates of many of thebasic technical tasks. These are supplemented by theactual Costs of devekving performance-based trainingmaterials. The Florida State University and Universityof Wisconsin estimates provide data .especially relevantto *the types of activities which New 7ork appears torequire..

General CostsThe Florida State team attempted to identify the

various costs of start* and implementing a competzn-cy-based teacher edtleation program that would pro-vide coniprehencive clinical training. That is, they weredealing with the professional components rather thanthe liberal urts components and the others that mightcontribute to the education of a teacher. They plannedabout 300 program units. They estimated about $21/4million to develop the drifts, about $75,000 to developand test the entry diagnostic system, about $400,000to develop and equip the computer management sys-tem, and about $200,000 to carry on a faculty trainingprogram, making a grand total of a little less than $3million.

My personal view is that this estimate is conserva-tive. For example, the faculty training program is notcosted very effectively by years. It is worthwhile not-ing, however, that they probably assume that many ofthe faculty would he persons who are involved in thedevelppment and who would, in the course of develop-ing the materials, 'train themselves to carry on ..thekinds of activities they would need to engage in as fac-ulty members. It is worth noting, that they did not ex-pect to implement a program in 1 year or 2 but ratherexpected to take 5 years fpr the development and im-plementation prbcess.

I think this is a very fair estimate of time. Develop-ing, testing, and integrating really significant, compe-tency-bAsed instructional systems will be atime-consuming and very expensive task. Furthermore,

BEST COPY- AIMIABLE

-rode!.12.f=l+la

I ,.,'.

fir,..

78-

Bruce Joyce

95

if one were to go to a competency-based licensing, onewould not want to do so until he had determined thegeneral effects of a competency-based teacher educa-tion program. In addition to the time for "start up"that was estimated by Florida, it might take 4 or 3rears before enough people had been through the pro-gram observed as teachers to lay a data base for revis-ing the certification procedure. Thus, to move from thepresent teacher education program structure to a com-petency-base could be somewhere between 5 and 10years, probably in the upper end of that range. Process65tions would not affect time much, because develop-ment of materials is the chief consumer of time andthe process options would affect only the other tasks..

The above costs did not include competency identifi-cation, which averaged a bit over $100,000 for each ofthe Bureau of Research studies. Since the Bureau ofResearch teams generally feel that the specificationwould have to be reworked before development couldbegin, probably another $100,000 would have to be in-vested in this task.

The Cost of Data Processing andManagement System

It is not possible to imagine competency-based edu-cation on a large scale unless it is supported by a com-puterized management system. The eventual programsmay he somewhat less complicated than those envi-toncd in the Bureau of Research models, but units in

96

a competency-based program will amount to a verylarge number. ( Probably more than 2.000, still underthe average number in the Bureau of Research pro-

grams.) In relating 500 to 1,000 teacher candidates toa program containing that many elements requires in-formation access, the coordination of support mate-rials, and the coordination of faculty of very greatcomplexity.

The cost of creating such a management system willhe considerable. The University of Wisconsin did avery thorough job of identifying the elements of amanagement system and the costsot., maintaining astaff sufficient to operate the system. They thought out

a system which would have the capacity of handlingover 700 students in an on-campus setting. They esti-mated an annual cost of awund $712,000 to maintain

the system (about $1,00( per student.) The rental ofhardware for the system would he in the neighborhoodof $350,000, .and the cost of the data processing staff

about the same. However, such a management support

system would have the capability of serving A greatmany more students at relatively low additional cost. Ifa state the size of New York decides to move intocompetency-based education, it would seem wise to de-

velop one or two prototype management systems. If

carefully dpigned they could 'accommodate the pro-gram units for a number of programs of differenttypes. This effort would reduce substantially the cost of

developing and maintaining management systems andinstitutions throughout the state. However, it is veryhard to imagine that even the greatest amount of shar-

ing among the centers for teacher education would re-duce the cost of maintaining a management system to

much under $500 per student. The system, however,

would permit an individualization of instruction, pro-gram planning, and assessment far beyond the capacityof any present teacher education program., Also thesystem envisioned by Wisconsin is a multimedia systemusingcomputer system instruction, instruction through

motion picture, television tapes, audio tapes, and pro-gramed units providing a variety of instructional modes

which very few instructional settings presently offer.

Thus the utility of the management system is general

it handles a large number of program elements for a

large number of students and increases the type as well

as number of instructional options.

On the Requirement of an Automated System

It is worth noting that none of the Bureau of Re-search models was conceived without the assumption

that it would he possible to operate a competency-based program without an automated management sys-tem. To relate 500 students to 2,000 instructional unitsin such a way that students have instructional options,

arc assessed and.made aware of results, and relate toprogram options in terms of developed competence, amanagement system is simply Accessary.

The expectation should he that part of the develop-ment of the competency-based system involves the cre-

ation of management systems. No program plan shouldbe accepted that does not ihclude the provision for thedevelopment as such an automated system.

Base Cost EstimatesUnder process option A, the costs of the tasks

would seem to break down as follows:

Iftveloping the Competency System $200,000

Developing Instructional Materials minimuth. $2,500,000

(including Assessment Devices)

Developing Diagnostic System $75,000

Developing Management System $400,000

Annual Cost of Maintaining Management $750,000

System (to serve one program)

These can be multiplied by the factors in table. I todevelop very rout) estimates of costs for the otherprocess options.

The experience of developers in recent years addssome specificity to the cost estimates and suggests cau-tions.

Development of SpetificadonsThe post the United States Office of Education of

having nine models developed by relatively expertteams in a very short period of time (about 8 months

probably not a sufficient period of time to do thejob right) was over $31/2 million. Relatively inexpertteams working over a longer period of time will proba-bly be much more costly. The cost could be reducedby havinn statewide organization in which state andlocal teams take responsibility for certain areas and the

whole effort is coordinated. Local participation, espe-cially to lay down initial preferences and to monitorthe results, could be encouraged.

The Development of the Training SystemThus, the training system area is where the real cost

actually. begins. The Bureau of Research project esti-mated between $5 and $7 million to develop their sys-tem, using a single team within a consortium or within-

L

a single institution of higher educaticin linked to localschool districts. The broader the base of participationand the more complex the model of teacher education,the higher will be the cost.

The cost of the mini-courses developed at the FarWest Laboratory is illustrative. The Far West Labora-tory. uses an extremely efficient development teamwhich creates and tests its product using standard re-search and development procedures. The teams arewell trained and stay together for long periods of time. .

They do not have responsibility for u wide diversity ofproducts but concentrate on the particular type of in-structional system known as the mini-course..The costof development of each siugle mini-course providingabout 30, hours of instruction to a teacher on oneteaching skill is over $100,000. At Teachers College.Columbia University. a series of six instructional sys-tems designed to teach three teaching skills and threestrategies was developed at a cost of a little over$100,000, or about $16,000 per instructional system.

It is probable that a comprehensive teacher trainingprogram at the preservice level will contain at least200 instructional systems (as in Florida's estimates ). Ifthat is the case .and the development cost of $15 or$16 thousand found by the Columbia team turns outto be an accurate one, the estimates from the Bureauof Research feasibitity sniffles In: trivia on tagut.

This will be the cost of developing one model ofteacher education by one centrally organized team. Asindicated earlier, the greater that development is de:centralized (process options 11, C, D) the greater willhe the cost unless the teams specialize and do not at-tempt to build the entire range of possible instructionalsystems.

What might he done would be to develop a master.plan of development and subcontra':t pieces of the de-velopment (process DI. However, it has to he remem-bered that there are very few places presently in thestate where much developmental productivity could heexpected for some time to come and it would take cen-ters a while to tool .up. Therefore, it seems wise tomake the assumption that a few places in the statewould have to he funded for major development ef-forts:

Further Notes on the Assessment System

The picture is somewhat different with respect to theassessment subsyStem. The developers of the instruc-tional systems could create many of the assessmentmeasures for specific competencies as a part of their

97

developmental effort. This would probably. be the mostefficient way of developing the specific measures andwas the proceddre proposed by the Bureau of Re-search team. However, in addition to the measures ofspecific competency, as they would relate to the in-structional systems, one 'needs measures of the generalcompetency of the teacher or situations in which hecan being together a group of competencies and performin a more effective way. It :was recommended in sev-eral of the Bureau of Research proposals that teachinglaboratories he used for this; that situations be set upin which a teacher could be brought together with asmall group (14 students. He would teach for an houror two, while his ability to set objectives and teach,and the gains in pupil knowledge were assessed at the

time. Some kind of assessment system like thiswould probably he desirable. ,,It is unlikely that ratingblanks or rating systems of observing and ratingteacher class behavior in the teacher's own classroomwill prove to he feasible. For more than 50 years per-sons have been trying to develop such measures, withnotable lack of success.

Saving Money by Sharing Materials

The technical nature of the competency-oriented sys-tem makes it nity denencinnt.competency-based software. Specifying the competencynecessary for any particular teacher role, building ap-propriate instructional systems, creating reliable assess-ment systems, and implementing a program all requireextensive research and development efforts and majorchanges in procedures. Statements of competency, in-structional, systems, and assessment devices are all ei-ther wholly (as in competency specifications) or partlyexpressed on paper, film, television tape, etc. Withouta very large development effort, there will be no implementation of CBTE.

Under some' circumstances, a solution to the needfor development might be to fund one or twa agenciesto create complete systems and disseminate them.

In this case, however, the actual implementationmust be closely allied to local needs and local person-nel must participate substantially in the whole processof determining directio% for change, defining teacherroles, -and selecting and implementing training systems.

To support local efforts without either controllingthem or engaging in a ruinously costly duplication ofdevelopment costs in every local agency, a large na-tional storehouse of competency-based products shouldbe developed. From this storehouse, local agencies can

98

draw much of what they will need to build theircompetency - bused traiiang system, while developingthe remainder locally. Fortunately, agencies across theNation have Keen prOducing materials which can getthe storehouse off to a good start.

The area of leading instruction provides a goodexample. There presently exist a number of approachesto reading and a number of systems for acquaintingteachers with them and training competatties relevantto those approaches. A local agency can draw fromthese in order to build the reading component of aprogram to train preservice or inservicc teachers.

This situation is duplicated in several other areas.ke developers represent a range of agencies, many

of .which are wholly or partially supported by Federalfunds:

I. Research and development centers and re-gional laboratories are available.

2. Curriculum projects in many areas, includingubject areas, approaches to the education of

young children, special education, correction, ca-reer opportunities, etc., are in ekistence.

3. N.C.I.E.S. projects. including the Teacher Cen-ters, Texas Education Agency, Protocols proj-ects. Florida State, are funded.

4. Teacher Corps programs have includeddevelopment officers for the last 2 years and

projects hay Wl 'auted products and processes.

5. Colleges and consortia which are in the processof reaching for the competency orientation haveproduced many materials.

The result is a beginning to a national storehouseprovided that the products arc brought together, madevisible, and demonstrated in preservice programs andinservice teacher centers. More products are needed, tohe sure, and many need transformation and testing, buta considerable start has been made and should not helost. Many agencies are presently operating fromscratch unaware of the considerable array from which

they could draw.What is necessary to make the storehouse useful?

First, potential users need to know what exists in thestorehouse and the potential use of each product.( Presently the Florida Catalog of Competencies. theFlorida Catalog of Competency-Based Materials, and"Materials for Modules," prepared at Teachers Collegefor the Teacher Corps, list many materials.)

Second. users need to know how Materials actuallyworkwhat an instructional system or assessment de-vice can he expected to achieve and what it takes touse it.

Hard. some Materials need further development tomake them adaptable to many local situationstheirtransportability needs to be improved.

Fourth. users need to experience competency-basedprograms, and 'Teacher Centers can Operate with prod-ucts largely drawn from the storehouse. This is ex-tremely important. The storehouse has now reachedthe point where energetic designers can, by using theproducts or others and developing some productsthenfielves, actually 'operate programs and TeacherCenters which would be at least three-fourths compe-tency-based.

The surprising fact is that there is presently avail-able much more software., especially.in terms of in-structional systems, than could possibly be. used in a2::year, full-time, teacslier preparation program. Not all

of the material is or high quality--and developmenthas been missing:in some important areasbut carefulselection from the storehouse, combined with local de-velopment, will enable program implementation fairly'soon, provided that the available materials are dissemL-nated and demonstrated and if program 'Models andTeacher Centers using the storehouse are also estab-lished as demonstration centers.

The Nature of the StorehouseMost of the ,products are in the form either of in-

structional systems or smaller units, generally calledmodules. Generally they consist of a competency specirfication, a set of activities and supporting mediateerma--terial. and an assessment device. Materials are avail-able in at least the following areas. (See following page.)

Imagine, for example. a r ogram to prepare teachersof the'soeial studies.

It mightin:.

(1 1 the study of teaching

include competency And adapt the following mate-rials:

Parsons' Guided Self-Analy-sis or Mini-course on .

Flanders System

(Grinder peogram r

Vincet t Popham)

Far West I .abora-t or y) Teachers College Units

Mini-coursesTeachers College 11,0s

Presentational skills (Gen-eral Learning Corporation)Teachers College Units

2) basic educationalpsychology

( 3 I basic instructionaldesign

(4.) curriculum alternativesin social studies

5) basic teaching skills

I 6 1 basic teaching strategies

4

Are

as o

f C

bmpe

tenc

e

Stud

y of

Tea

chin

g

Bas

ic T

each

ing

Skill

s

Mod

els

of T

each

ing

Inst

ruct

iona

l.

Dec

isio

n- M

akin

gSk

ills

Dia

gnos

tic S

kills

The

Stu

dy o

f th

eSt

uden

t

Cir

cula

rD

ecis

ion

Mak

ing

Inst

ruct

iona

lD

esig

n.

Edu

catio

nal

Philo

soph

y

Edu

catio

nal

Psyc

holo

gy

A

et.

AiR

.4.Y

OF

PRO

DU

CT

S

Typ

es o

f C

ompe

tenc

e

Skill

s in

Ana

lyzi

ng T

each

ing

of S

elf

and

Peer

sIn

trod

uctio

n to

Sys

tem

s'B

asic

Ski

lls f

orfo

r St

udyi

ng T

each

ing0

.St

udyi

ng T

each

ing

App

licat

ion

of S

- ki

lls in

feed

back

to T

each

ers

A v

ast v

arie

ty o

f te

achi

ng s

kills

are

em

bodi

ed in

inst

ruct

iona

l sys

tem

s. (

The

se a

re o

ften

exp

ress

ed in

jarg

on. a

s St

rnc-

luri

ng, M

odul

atin

g Fo

cusi

ng. R

einf

oxin

g, S

et I

nduc

tion,

Exp

lain

ing,

etc

.).

A c

onsi

dera

ble

vari

ety

of te

achi

ng s

trat

egie

s ar

e av

aila

ble.

(T

hese

can

be

in s

peci

fic

cont

ent a

reas

, in

Wal

len'

s in

stru

c-tio

nal s

yste

ms

in th

e re

adin

g ar

ea, I

vey

's c

ouns

elin

g, a

nd m

ini-

cour

ses

in a

rith

met

ic te

achi

ng.,

or th

ey c

an r

elat

e to

curr

icul

um s

trat

egie

s, a

s th

e in

stru

ccon

al s

yste

ms

to te

ach

teac

hers

the

stra

tegi

es o

f IP

I, o

r hO

w to

impl

emen

t the

Eng

elm

ann-

Bec

ker

appr

oach

to e

arly

chi

ldho

od e

duca

tion.

)

Setti

ng O

bjec

tives

Sele

ctin

g A

ppro

ache

sSe

lect

ing

Eva

luat

ion

Dev

ices

.

The

re a

re m

any

type

s of

sys

tem

s fo

r, in

stru

ctio

nal d

esig

n, a

nd c

ompe

tenc

y-or

ient

ed d

evic

es to

teac

h th

em h

ave

prol

if-

erat

ed.

i.

Cre

ativ

itySe

lf-E

stee

ipA

chie

vem

ent

Soci

abili

ty

..

Rea

ding

Soci

al S

tudi

esSc

ienc

eM

athe

mat

ics

(Mos

t of

the

deve

lopm

ent i

n th

is a

rea

cons

ists

of

mod

ular

ized

.cur

ricu

lum

cou

rses

dev

elop

ed b

y lo

cal p

rogr

ams.

How

-.

ever

, pro

duct

s lik

etth

e In

form

atio

n U

nits

fro

m th

e Fa

r W

est L

abor

ator

y ar

e pr

ofes

sion

ally

pac

kage

d. T

he lo

cal p

rod-

ucts

var

y in

qua

lity

but i

nclu

de s

ome

exce

llent

mat

eria

ls.)

Arc

hite

ctur

e..

Staf

fU

tiliz

atio

ll"C

urri

cula

r D

esig

nT

echn

olog

y(T

hese

are

as a

re r

elat

ivel

y un

touc

hed

by d

evel

oper

s.)

Pers

onal

ists

Soci

al k

ealis

tsA

cade

mic

sC

yber

netic

s(T

here

are

som

e m

odul

ar m

etho

di.c

ours

es in

thes

e ar

eas,

but

muc

h ne

eds

to b

e do

ne.)

0In

this

are

a th

ere

are

a va

riet

y of

pac

kage

dcou

rses

. Dev

elop

ers

tend

. to

diff

er in

poi

nt o

f vi

ew, s

o co

nten

t var

ies.

)

.46

Using the already-developed materials (includingcompetency specifications, instructional materials, andassessment devices), one could easily organize a fac-ulty to implement a competency-based program to pre-pare teachers of the social studies. Most teacher educa-tion agencies would probably prefer to create some oftheir own materials, but to. develop all they would needwould probably require an investment of $400 or $500thousand, whereas if they draw froth the nationalstorehouse t "he cost becomes feasible.

A Teacher Center s

To build ti Teacher Center would involve much thesame process. A Ti.tcher Center should offer teachersthree types of service:

Teacher

S

0).RE

H

SE

1. Teachers Option. When the tettcher selects COM-peteneies according to his interest d self-diagno-sis.

2. External Diagnosis - Option. When the teacher'speers or supervisors determins .::at he has aneed by analyzing his teaching.

3. Cureicaltint 'Thrust Option. When a team,school, or agency inauguratis a new program re-quiring new competewes.

To create such a center one needs two offer teachers agreat many options. which can be selectiod on any of .the three service bases described above, For example,instructional systems could be classified on a basis

somewhat like .the Array of Products and made ,avail-able with a inicroteaching laboratory. Thtis:

EE

I D ElEE

Teacher Centers could be designed around the roles asuited ttr the various roles. For example:

of the teacher. A good many of the existing products are

°

A

General Competencies

-Study of Teaching

Instructional Design

4

Basic Teaching Skills

1

Skills

Strategies

Curriculum

Psychology

Etc.

Role.Related Instruction

Skill Builder(IPI Training System)(SRA System in Reading) .

Productive Thinker(Synectics Material)

Community Leader(Teachers College Units)

Counselor(Micro-counseling Skills)

Academician(McCrel Units, University ofTexas Units, Teachers College Units)

1'

O

More .than enough material presently exists in thestorehouse to permit the creation of Teacher Centerssuch as this. Cidup led with the types of workshop cen-ters. used in the English Teacher Centers, .they provide.a great many possibilities for teacher self-training.

Cost and QualityEven if a local program. built all its own staff,

1 think CBTE is dirt cheap even at the level of cost 1hav estimated. 1 think it is extremely inexpensive.What is 'really expensive is what we are doing now.' Wepour hundr&is of millions of dollars: into ineffectiveteacher: education programs every year. Thai's *lostpure waste: The inefficiency and waste of the presentsystemis just terrible.

.

A great deal of the cost of development will be in-.1tind costs. View- a teacher edueation program as a de-veloping nation. You have to deyeldp capital somehowand there are a couple forms of capital: one is ideas;the other, software. You hivest faculty time in order to

101

get these. We simply heed to put a greater proportionof staff costs into development than we have.

What bothers .me about the stance th,lt it will beeasy and cheap is when development simply becomesmodularizing the present stuff of,. teacher educatiod.That is easy and cheap lAit it doesn't result in muchimprovement

t,

1fhave never personally experienced, a major univer-sity ( maybe Houston,. trtayhe Toledo) that was capableof doing the whole development job by itself. The Na-tional Consortium is essential for just pooling the tal-ent to do the job. For example, without theMini-courses produced at tote Far West Lab and Ethermaterials produced by dozens of others, 1 wouldn'tdream of rhe type of program we run 'at Columbia al-though vsfl: 'kuild a lot of instructional systems our-selvek. We mast share materials if weare to keen costsin hounds and `draw on each other for program. ideas. /CBTE should be a national development effort with amyriad of local' variations.

Bibliography °

I. Models fgr Elementary Teacher Fducation,

The Florida State University, it Model for the Prepara-4 ti Olt of Elementary ScHbol Teachers ( OE-58018;

two volumes), Washington, D.C.: U.S. Office of' Education Bureau of Research, 1968.

Michigan Stalk University, Behavioral Science Elemen-tary Teacher Education Program (0E-58024,three volumes), Washington, D.C.: U.S. Office ofEducation Bureau of Research, 1968. -1.

Northwest Regional Educational Laboratory, A Com-,. potency Based,Field Centered, Systems Approach

to Elementary "Teacher Education (0E-58020,three volumes), Washington, D.C.: U.S. Office.ofEducation Bureau of Research, 1968.

.4tacuse University, Specifications for a Comprehen-sive Undergraduate and. Inseloviee Teacher Educa-lion Program for Elementary Teachers (0E-58016, two volumes), Washington, D.C.: U.S.Office of Education Bureau of Research, 1968.

Teachers Uollege, Columbia University, The Teacher.

.

Innovator A Program to Prepare Teachers(OE-58021), Washington, D.C.: U.S. Office ofEducation Bureau of Research, 1968.

University of Georgia, Educational Model Specifications.. - for the Preparation of Elementary Teachers

(0E-580) 9 ), Washington, D.C. : U.S. Office ofEducation Bureau of Research, 1968.

University of Massachusetts, Model ElementaryTeacher Education Program (0E-58022), Wash-ington, D.C.: U.S., Office of Education Bureau. ofResearch, 1968.

University of Pittsburgh, A Model of Teacher Trainingfor the lis("vidualization of Instruction (0E-58018), Washington, D.C.: U.S. Office of Educa-tion Bureau of Research, 1968..

University of Toledo, Educational Specifications for aComprehensive Elementary Teacher EducationProgram (OE-58023, two volumes), Washington,D.C.: U.S. Of Education Bureau of Re-search, 1468.

,.r

University of Wisconsin. Flementnry Teacher Educa-tion Project (OF-58025, four volumes), Wash-ington, D.C.: U.S. Office of Education Bureau ofResearch, 1969.

II. Feasibility Studiesl'he Florida State Univeriity, A Feasibility Study of

the Florida State University Model for the Prepa-ration of .letnentary School Teachers, Washing-ton. D.C.: U.S. Office of Education Bureau ofResearch, 1969.

Michigan state University, Feasibility Study: Behav-ioral Science Teacher Education Program, Wash-ington, D.C.: U.S. Office of Education Bureau ofResearch, 1970.

Ohio Consortium, The Feasibility of Educational Spec-ifications for the Ohio Comprehensive ElementaryTeacher Education Program (Phase II), Wash-ington, D.C.: U.S. Office of Education Bureau of

Research, 1970.

Oregon College of Education, A Plan for Managingthe Development, Implementation and Operationof a Model Elementary Teacher Education Pro-

gram, (three volumes), Washington, D.C.: U.S.Office of Education Bureau of Research,1970.

Syracuse University Ptotocooperative, A Study of theFeasibility of the Refined Syracuse UniversitySpecifications for a Comprehensive Undergraduateand Inservice Teacher Education Program forElementary Teachers, Washington, D.C.: U.S.

Office of Education Bureau of Research, 1969.

University of Georgia, The Feasibility of the GetIrgiaEduca:ional Model for Teacher PreparationEle-mentary, (three volumes), Washington, D.C.:U.S. Office of Education Bureau of Research,1970.

University of Massachusetts, A Feasibility Study on the-Model Elementary Teacher Education Program(Phase LI), Washington, D.C.: U.S. Office :4 Ed-ucation Bureau of Research, 1970.

The University of Toledo, The Feasibility of Educa-tional Specifications 'for the Ohio ComprehensiveElementary Teacher Education Program (Phase11), Washington, D.C.: U.S. Office of EducationBureau of Research, 1969.

f1,1

tlnhersity of Wisconsin, Wisconsin ElementaryTeacher Education Project Feasibility Study, (twovolumes), Washington, D.C.: U.S. Office of Edu-cation Bureau of Research, 1970.

III. Other SourcesCruikshank, Donald R., et al., The Ohio State Univer-

sity Analysis of the Nine Comprehensive Elemen-

tary Teacher Education Models (CETEM),Washington, D.C.: U.S. Office of Education, Na-tional Center for Educational\Researcl- and De-velopment, 1970.

Engbretson, William E., Analysis and Evaluation of

Plans for Comprehensive Teachet EducationModels, Washington, D.C. U.S. Office of Educa-tion, 1968.

Joyce, Bruce R., "Variations on a Sysiems Theme:Comprehensive Reform hi Teacher Education,"Interchange, Vol. 1, No. 3 (1970), pp. 83-95.

Klatt, Judith and Walt Le Baton, System DevelopmentCorporation, A Short Summary of Ten ModelTeacher Education Programs, Washington, D.C.:U.S. Office of Educatickn, National Center for Ed-

. ucational Research and Development, 1969.

Le Baron, Walt, System Development Corporation,Systems Analysis and Learning Systems in theDevelopment of Elementary Teacher EducationModels, Washington, D C.: U.S. Office of Educa-tion, National Center for Educational Researchand Development, 1969.

Proceedings of 22d Annual Georgia Teacher Educa-tion Conference: TEACHER EDUCATIONMODEL, Journal of Research and Developthent.in Education, Vol. 2, No. rSpring 1969).

A Reader's Guide to the Comprehensive -Models forPreparing Elementary Teachers, Washington,D C.: ERIC Clearinghouse on Teacher Educationand the American Association of Colleges forTeacher Education, 1969.

System Development Corporation, Analytic Summa-ries of Specifications for Model Teacher Educa-tion Programs, Washington, D.C.: U.S. Office ofEducation, National Center for Educational Re-search and Development, 1969.

Funding Competency-Based Teacher EducationBy.

HERBERT HITE

Competency-based teacher education costs differfrom the costs of htandatd teacher education -becausethe two programs differ. There are differences in therequired performances of students and also in the rolesof faculty. Program materials are different and so are

wssessment procedures. The sources of funds forCBTE, however, are not different from the sources fortraditiOnal, of standard, programs. The same sources ofrevenue for higher education which ate always in shortsupply must implement the CBTE programs. There-fore, the analysis of costs for CBTE is most meaning:ful when these costs are expressed in terms of fundingformulas for.oiher higher education programs.

Among the program variables of CBTE which arereflected in Merin costs from standard programs arethe following:

1. Instructional materials, units, and techniquesare individualized. If the criterion of CBTE isthe success of the individual student of teach-ing, then CBTE materials must allow for differ-ent rates and different options ny these individ-ual learners. These materials .constitute a highercost factor than standard program materials.

2. For the most pact, the CBTE faculty wo.-1-, in aone-to-one relationship with studentsascounselors, evaluators, explainers, analysts. Thechanged role of faculty has a higher cost thanfaculty in the traditional lecturer role.

3. The process of admission to CBTE appears tobe a more complicated and hence a more costlyprocess than admission to standard programs.Prerequisites for success, which shOuld consti-tute admission criteria, seem to be more 'diffi-cult to assess than the usual examination ofstudents' grade-point averages.

4. Complex management systems may be a signifi-cant cost item.

5. Students in the CBTE programs are usuallyfull-time interns or observers or work overvarying periods of time with individualized

These 10 factors are described in a paper given for theTeacher Corps in June 1972. -The Economics of Competency-Based Teacher Education," Herbert Hite, Research Foundationof the State of New York, Albany, N. Y., 1972.

103

study materials. Full-time students rather thannumber of .elasses arc the bases for measuringcosts. This is not necessarily a hightr cost fac-tor, just different.

6. In CBTE, school personnel have a significantlylarger role in the instructional process and thecost of their participation is relatively higher.

7. More instructional time is involved in assess-ment of student progress, and this process isusually more complex than grading in courses.Evaluation costs are usually higher.

8. Decision making regarding all phases of theCBTE program involves not only college per-sonnel but also school and community mrm-bers. The large amounts of these people's timewhich is needed are a significant cost factor,and no fund have been available for this func-tion in the past.

9. Specialist or consultant services to assist stateagencies in program approval may result in ad-ditional costs for CBTE.

10. Competency-based programs leading to ad-vanced certification will be centered in schooldistricts, and the development and administra-tion of these new programs will be an addi-tional cost factor eventually.

In a study of costs for the new CBTE certificationstandards in the state of Washington, which was re-quested by the state's legislature, two CBTE programswere analyzed.'' The 'study suggests that for the timebeing, the following factors are additional costs ofCBTE relative to 'standard teacher education programs"in that state:

I. Ada 50 percent to the tosts of the standardteacher education to account for the variable ofindividualized mode of instruction.

2. An additional 50 percent will be required for de-veloping each new CBTE program.

3. An additional 50 percent will be needed to payfor released time of school personnel involved inconsortium arrangementspolicy making, pro-gram planning, securing program evaluation, etc.

One of the programs analyzed in the cost study was des-cribed in the paper previously cited.

r1

104

The recommendations to the Washington State Leg-islature are for the colleges and universities to receivedouble funding for new CBTE programs and theschool districts to receive finds for consortium activi-ties.

Some of the costs for CHIT are the same if the pro-gram includes one student or 1,000 students. Thesecosts include the development of policy councils, thecertification process, the development of basic compe-tencies and assessment processes. and a basic set of in-dividualized materials. Some costs vary according tothe number of students: the number of faculty per stu-dent, the number of laboratories or public school class-rooms, the number of sets of materials. Some costsrepresent quality judgments, such as the number of op-tions to be made available to students, the ratio of fac-ulty to students, the degree of refinement of structionalmodules and assessment procedures.

Obviously, a good deal of the difference in costs ofCBTE and standard programs depends upon decisionsas to the nature of specific CBTE programs. For exam-ple, the definition of competency can make a differencein program costs. If compete yis- ed as a set ofteaching performances, the the focus win u onhighly refined and tested innitructional devices to elicitteaching behaviors. If the e phasic is upon products ofteaching; i.e., pupil achie ments, then the emphasiswill he upon the field setting. for working with learnersand evaluating the outcomes of`t\his activity.

The CBTE model at Western' Washington State Col-lege is based upon product.ixiteria. The candidate isrequired to demonstrate competency by hi inging about"appropriate" changes iv the behavior of elementary orsecondary pupils. The thajor components of the modelare an entry phase, in/which the candidate meets ad-mission criteria, a kifoledge phase in which he ac-quires a repertoire :of information about the substftweof what he will teach, a laboratory in which he ac-quires and demonstrates knowledge and skill aboutteaching and learning principles and applies theseskills, and a practicum in which he demonstrates hisability to apply principles of teaching and learning in :tvariety of situa ms. The laboratory and practicumconstitute two quarters of full-time study and practicein a teaching center.

The program depends upon ,close working relation-ships in the laboratory and practicum among the stu-dents. the professors, and cooperating teachers who gothrough a special training program to prepare as field

supervisors. lbe basic essentials of the program areevaluations of entry into the program, diagnosis of

knowledge necessary tc, succeed in the laboratory, atest of instructional competency as an entry into thepracticum and modified T.I.C.'s as evidence of masteryof the basic model and hence certification. Study mate-rials in the laboratory consist of 52 modules and as-sorted readings, films, filmstrips, tapes, observation as-signments. and mini-teaching arrangements. Thestudents design their own study programs leading tothe T.I.C., which is the basic measure of competency,by negotiating learning contracts with the clinical pro -fessors.. Study contracts vary.

After 4 years of partial success and occasional fail-ures, the clinical faculty has arrived at a workingmodel. The costs of this working model arc heavierthan the costs for standard teacher education at West-ern. The cost studies which have been completed sug-gest that even after initial years of program deviisp-ment have been completed, tadd ft) the costs of t standard programs at Western:

I. About . percent additional is needed to fundextr' support costs, such as visits by campus-ha ed faculty to the teaching centers, and speciali dividualized learning modules.About 25 percent additional is needed for thecontinuous development of the program compo-nentsevaluation instruments, adaptations of'Modules, processes for o_ rienting experiencedteachers to the model, etc.

3. About 25 percent is needed for the administra-tion of each teaching center, including negotiations with school personnel concerning consor-tium arrangements.

This is a modest proposal for extra funding com-pared to other suggestions for funding CBTE. TheWestern model is still developing, but it is based uponmodest resources. Compared to the ElementaryTeacher Education models, funded in 1968 by USOE,this is a Model T compared to a brand new LincolnContinental. The present program does produce the re-quired demonstrations of competency, however, andthere are considerable affective gains over the standardprogram in the views.of faculty and students.

Still, even a modest program needs somewhere be-tween a 50 and a 100 percent increase in funding forteacher education. Where does the funding come from?In the real world, deans of education do not confrontthe legislature or the hoards of regents with demandsfor so ninny extra millions to fund a new progrsmi.Colleges and universities have only a few alternatives.

These are the possible alternatives a college has forfinding the additional resources for C'BTE:

t.

I. The college might maintain the existing level offaculty and instructional support funding butlimit the enrollment of students into the pro-gram.

2. The college might require more time or credithours by students who were candidates for a cer-tificate under the CUTE plan, thus increasing ei-ther tuitions or student credit hours and conse-quent funding.

3. The faculty might seek an adjustment in theformula for support of programs. Field-based or

...luboratory programs might he funded at twicethe level of campus-based programs.

4. The college faculty might seek outside fundinggrants or fellowships.

5. Schools might contribute by sharing the coststhrough support of a complementary inserviceCUTE program.

6. Colleges might develop graduate programs, withtheir higher rates of funding, as adjuncts to thecompetency-based undergraduate program.

In Washington. master's level instruction in state in-stitutions is funded at 2.3 times the level of upper-divi-sion student instruction. Clearly this level of fundingwould provide what the faculty at Western have de-fined as their need for CBTE programs. This adjust-ment would require action by the state legislature or amajor and radical change in programing teacher educa-tion within the college. Neither possibility seems likely.

Outside funding is possible, but not very dependableas a base for a continuing extra program cost. The col-lege could reduce enrollment and maintain the existinglevel of faculty and support programs, but this .is'politi-catty impossible at Western, although this may be pos-sible at other institutions. A few school districts willshare costs. In fart, one school district reportedly has abudget of $150,000 for advanced certificate programsunder the CBTE model. These school funds," however,are usually restricted to payments to school personnelfor released time. This leaves the college administratorwhere he always iswith the same old system to try tomanipulate. Basically the game is to generate sufficientcredit hours to secure the necessary entitlement for fac-ulty and instructional support.

The CUTE program at Western operates in fourteaching centers. Each center consists of a team ofclinical professors, students, and cooperating teachers.In a center with four clinical professors, here is a strat-egy for generating the necessary student credit hoursand getting the necessary work accomplished.

Two clinical professors will have eight to 10 newstudents each quarter. As the studeats remain for twoquarters. this means that the instructional load for each

BEST COPY AVAILABLE 105

of he two professors he ' K full -tim students, at.d.:rgraduate Tht,c two professors between

h:m generate three and one -third full-time facultyequivalent entitlements.

Herbert Hite

One clinical pro-- or takes on half as many stu-dents, or a total of nine, which is about two-thirds of afull-time equivalent faculty load. He is also responsiblefor the administrative matters di the center, includingnegotiations with school personnel.

One clinical professor will direct 25 part-time gradu-ate students in a four-credit practicum. The studentsare teachers and volunteer to act as cooperating ad-junct faculty in the CBTE program. The credits theyearn may apply to an advanced certificate. Their prac-ticum is concerned with development of the processesfor working with students in the program, includingevaluation criteria and techniques. The major responsi-bility of this clinical professor is program development.The 25 students in the four-credit graduate practicumgenerwe one full-time equivalent faculty load.

The four clinical prJf:ssors, then, generate fivei l'.E. units, which entitle., the center to the needed in-:41'0101ml materials and service: which are additionalto.those required in campus-based programs, Also, two

the: professors earn student credit hours in the proc-ess of administering and developing the CBTEgram.

The four clinical professors may modify their ownassignments. They are jointly responsible '11)r 45 full-

106

time undergraduates and 25 part-time graduates, how-evvr they divide the responsibilities of the teaching cen-ter. These figures are only illustrative.

A vat ation on this model is a center in which theundergraduate intern. in his final quarter of demon-strating competency, relieves an experienced teacher,who becomes a full-time master's candidate. The internis hacked up by a first-quarter laboratory .student whoacts as a teaching assistant part time. Both are closelysupervised by the clinical professor. In this model, in-stead of supervising 18 full-time undergraduates, theclinical professor is responsible for four full-time grad-uate students, four interns, and four laboratory stu-dents. in four classrooms.

These figures and load descriptias represent a fewof the ways in which the CBTE faculty may generatethe amount of credit hours needed to support the costsof CUTE which are above the costs of standard pro-grams. The experience at Western indicates that thesekinds of loads are feasible, given the nature of the-CUTE program which has been developed at that insti-tution.

Summary

Competency-based teacher education is more expen-sive than the standard programs. The variables withinCBTE programs which affect costs are many, and, ad-ditionally, 'there are many kinds of value judgmentsabout programs which will affect the level of fundingneeded for a given CBTE program. Funding for amodest approach to CUTE, which is largely field-based, is possible within the usual formulas for gener-ating support for higher education. The experience ofclinical professors indicates that the credit hours gener-ated by full-time undergraduate students and part-timegraduate students (cooperating teachers) are sufficieto fund CBTE. State officials could simplify the fund-ing problem, however, by supporting CBTE as 'a"high-cost" college program, equivalent to graduatestudy or to study in the other professional fields suchas health sciences or law.

It seems clear from initial experiences with CBTEthat costs are greater than with regular programs. Howmuch greater depends upon a number of judgmentswhich are made by the CBTE program designers.

The high cost factor which is common to all CBTEprograms is the individualized mode of instruction,which is necessary if teacher candidates are to demon-strate competency criteria. Other expenditures, particu-larly expenditures for program development and for

management, canean vary wildly from CBTE.program toCUTE program,

The cost decision may hinge upon the way the fac-ulty define competency. If competency is defined interms of specific teaching behaviors which are assumedto be related to changes in pupils, then the modulesand other training materials will need to he refined tothe point That it is possible to predict a given level ofperformance on the part of the consumer of thosemodules. It follows from that decision that teachingmodules and strategies must be highly refined. Costscan be very high to achieve this end.

Competency may be defined, however, as changes inthe performance of the .teaching candidate's clients Inthat case the specific modules or strategies will beeffective inasmuch as they assist an individual candi-date to implement growth on the part of the candi-date's pupils. The emphaiis in program developmentwill then be upon the field setting, the education of co-

opera lifg-kehool lkissOntel, the evaluation of the prod-_

ucts of teaching. The costs for these components ofCBTE.are more likely to be absorbed within the ongo-ing school program for pupils and the inservice educa-tion for teachers.

If the CBTE faculty choose to design all their ownlearning modules, the costs will be relatively highthey could be astronomical. The cost factor seemscomparable to the costs of developing programed in-struction or instructional systems. There are choices tohe made concerning the quality levels of the CBTEprogram; e.g., the ratio of faculty to students; the num-ber of alternative instructional strategies, etc.

'New state certification standards which specify com-petency or performance by candidates also require thatthe new teacher education programs be managed by aconsortium of agencies, including not only colleges oruniversities but also school administrators, professionalassociations, and the public. Released time for teachersto participate fully in training programs and in policydecisions is a new cost for teacher education.

The realistic level for funding CBTE might be tocompare teacher education to graduate education or toother high-cost programs such as in the laboratorysciences. 3 The rationale for shifting the funding from alevel comparable to academic college instruction' tothat of funding graduate or laboratory science instruc-

A The higher level of funding per student does not meanthat the total cost of teacher education necessarily would be'ligher: preparation agencies could limit enrollment in their::BTE programs.

tion is that CBTE is like othei high-cost collegiate in-struction. ('BEE is a highly individualized programand should he supported in the same way as other col-legiate programs which are conceded to require ahighly individualized approach.

Ultimately, the preparation of teachers under a com-peteney-bawd model will need sustained financing inthe same way as teacher education programs are pres-ently funded. Specific CBTE programs must earn col-lege credits. The credits must meet the quality stand-

,

C. 4

5

107

arils other college credits must meet. The credits musthe sufficient to provide for the necessary time of fac-ulty and for support programs. Thc additional fundingneeded to implement the individualized CBTE pro-grams must come through the existing system of highereducation. Therefore, the fate of CBTE seems to restwith the decision makers who allocate funds to collegesand universities, and specifically CBTE will dependupon a more generous formula than is now used tofund undergraduate teacher education.

tt.

108

Assessment of Teacher Performance:What is Involved? What is the Cost?*

By

BEATRICE A. WARD

When the term "performance-based" is used as adescriptor for teach..r training and/or teacher certifica-tion, certain essential conditions apply, What theteacher does is as important as what he knows. Duringtraining, skill development receives equal or greater at-tention than knowledge acquisition. At the end oftraining, mastery is used as the evaluative criteriarather than a list of courses completed and grades re-ceived. Information about the teacher's performance isobtained throughout the training period and subse-quently in the operable classroom in order to deter-mine whether the specified teaching skills are being dc-quired and used. Student performance also is studiedin order to determine whether teacher use of certainskills relates to the level and quality of student learn-ing. -

Within such a framework, assessment of teacher per-formance is multi-faceted. It is conducted for many dif-ferent reasons, focuses upon a wide variety of teacherand student behaviors, and involves a diversity of indi-viduals and institutions. Inquiry into several of thesefacets of assessment is the purpose of this paper. Thediscussion centers around choices that may be maderegarding assessment and the costs associated with par-ticular options. It builds upon the information obtainedby the Teacher Education Division at the Far WestLaboratory: for Educational Research and Developmentduring 5 years of developing and testing performance-based teacher training materials.

Why Assess Teacher PerformanceAssessment of teacher performance can be under-

taken for a variety of purposes. Among them are thefollowing:

To determine the average type of performancethat can be expected of experienced or inexperi-

*_

Invited. paper, Multi-State Consortium on Performance-Based Teacher Education, New Orleans. Louisiana, February27, 1973.

\enced teachers at a local, regional, state, or na-ional levelo identify the types of teaching skills a majority

o`f inservice or preservice teachers perform welland those which they need to improve or acquireTo establish a minimum acceptable level of per-formance for success as a teacher; to identifythose teaching skills that relate to successful stu-dent performanceTo determine what an individual's level of per-formance is at the time he enters inservice orpreservice teacher trainingTo establish that an individual has mastered oneor more teaching skilli as a result of training

Two types of data are associated with these forms ofassessment. Assessment conducted to determine aver-age teacher performance, identify how well teachersperform certain specific skills, or establish a minimumacceptable level of performance builds upon informa-tion regarding teaching in general. Samples of teachingand teachers may be used as a data base. The informa-tion obtained serves as a guide for policy decisionssuch as the selection of the content (skills and knowl-edge) to be included in a preservice and/or inservicetraining program and the specification of, the skill andknowledge requirements for certification. General in-formation of this type also aids in determining how ex-tensive an inservice retraining program should be; i.e.,the potential number of teachers needing training in agiven skill and the geographic areas in which particulartypes of training are needed. .

On the other hand, the data base for planning an in-dividual teacher's training program and/or verifyingthitt a particular individual has accwired. certain speci-fied teaching skills requires detailed information re-garding that person's performance,. Sampling proce-dures cannot be used. Rather, a profile of skill usagealong with a judgment as to the quality of use mustbe available for each individual teacher. The specificityof the profile may vary depending upon whether thepattern of skill use will be used to outline a training

sequence or verify competence prior to certification,Pupil learning outcomes may or may not he part of theprofile depending upon the inservice or preserviec sta-tus of the teacher. But, regardless of the informationincluded, the assessment process must help each indi-vidual teacher identify his areas of strength and weak-ness and plan a training program that will improve histeaching performance.

Assessment of the individual teacher or assessmentof a sample of the teacher population, therefore,evolves from the intended use of the assessment infor-mation. Policy decisions can build from a different database than training decisions. Obviously, given sufficienttime and financial resources, assessment of individualteachers could serve policymaking as well as trainingpurposes. However, since time and money are limited,study of a representative sample of teachers appears tohe the most cost/effective means for obtaining the dataneeded to make policy decisions. A variety of tech-

109

niques may be used to arrive at the teaching sample.For example, an adaptation of matrix sampling (Husekand Sirotnik, 1968; Lord and Novick, 1968) mightprovide a means for obtaining information about howa broad sample of teachers' use of a wide range ofskills at the same time keeping costs similati.to thoseassociated with an indepth study of a small sample.

Based upon these guidelines, the ultimate cost ofteacher..assessment is determined, in part, by the pur-poses for which the findings will be used. A first steptdward designing and financing asn assessment effort isto outline the,questioas the data are to answer.

What Teaching Performance Will Be Assessed?Determining wl.qt teaching performance will be as-

sessed also involved several steps (see- figure 1). Thefirst is to identify the potential list of skills to be in-cluded in the assessment effort.

Figure 1

Steps in Selecting Teaching Performance To Be Assessed

Use alreadydeveloped

lists

Define levelof specificity

Classroom

Step 1

Identify PotentialList .* Skills

Step 2

Define Rangeof Skill Use

Step 3

Select Setting in WhichUse Will Be Measured

Developownlist

Select subjectmatter areas

Manipulated(microteach)

110

Identifying skills. The diversity of knowledge andskills encompassed within. this teaching act has beendocumented by numerous experts in the field: i.e.,Smith, 1971; Houston, 1972; Joyce & Weil, 1972. Yet,to date, research has shown few positive relationshipsbetween teaching skill and student outcomes (Rosen-shine, 1971). As a result, selection of the skills to beassessed: for the most part, consists of combining whatlittle evidence exists regarding teacher effects with the-oretical and practical opinion regarding desired teacherperformance.

Several efforts to identify critical teaching skills havealready been undertaken by individuals and agenciesSuch as the developers of the models for elementaryteacher education (Elementary Teacher EducationModels, 1969), the designers of competency-basedteacher training programs (Dodl, 1972), and the de-velopers of performance-based teacher training mate-rials (the Far West Laboratory); In each instance, athorough review of the literature related to a particularaspect of teaching is undertaken, inclass observationsare made to verify that certain skills are used, andsx-perienced teachers, school administrators, parents, andothers are asked to judge the relative importance of theskills.

As suggested by Joyce (1973), .persons.responsiblefor teacher assessment may opt to use these alreadydeveloped skill lists or they may choose to developtheir own. If the latter option is taken, the costs pre-sented in table I indicate the expenditures in personneland other operating expenses such as travel and pur-chase of supplies and materials that may be required.The figures are based upon costs incurred by theTeacM.-Education Division of the Far West Labora-tory while identifying critical teaching skills in the

areas of pupils' language development, mathematics tu-toring, independent learning, and higher cognitive

F

questioning. On the average, skill identification cost$14,670 for each set of skills.

The efficacy of using and/or building upon existingskill lists is apparent given these costs. Since teachingincorporates multiple sets of skills of the type ;dentifiedby the Laboratory, the total cost of skill identificationcould be expected to exceed $100,000 if started fromthe beginning. This would be the cost even if thesearch were to be limited to available research infor-mation, and theoretical and experiential opinion. if fur-ther research to identify relationships between teacherand pupil performance were included, the costs wouldbe much higher. Thus, if assessment costs are to bekept within reason, the skills to be assessed should betaken from the skill lists being compiled as part .of thebroad performance-based teacher education effort.Each assessment agency should not attempt to developits own unique skillslist.

Range of Skill Use. The second step in determiningwhat teacher performance will be assessed is to definethe range of skill use to be studied. By this,4I mean thelevel of specificity at which the skill will be assessedand the variety of., subject matter areas in which theskill samples will be taken.

Many areas of teaching can be described in terms ofgeneral performance levels as well as specific categorieswithin these levels. For example, the general teachingskill of asking higher cognitive questions can be di-vided into the more specific skills of asking analysis,synthesis, and evaluation questioni (see table 2).Within any assessment process, whether it is for policy-making or training purposes, measures of this' skillprobably would focus upon the general performancelevel. Previous research indicates that higher cognitivequestions generally represent less than half the questionsasked by teachers (Gallagher, 1965; Davis & Tinsley,1967; Guszak, 1967). In order to set training priorites

Table 1.'7

Costs of Identifying

.

NPupils'

a Set

Skill Area

of Teaching Skills.7

Language Indepen- HigherDevdop- Math dent Cognitive

ment Tutoring . Learning Questioning Mean

Personnel Costs $ 5,584 $10i06R $ 4,953 $ 5,981 $ 6,647

Direct Costs 6,938 11,836 6,156 7,163 8,023

TOTAL $12,522 $21,904 $1,) ,109 $13,144 $14,670

General Skill Level:

Specific .Skill

Table 2Higher Cognitive Questioning*

Use higher cognitive questions

1. Use three types of analysis questions.questions requiring students to think of motives or

causes--questions requiring students to make inferences

questions requiring students to find evidence to supportgeneralizations, interpretations, or conclusions

2. Use two types of synthesis questions.questions asking students to make predictionsquestions asking students to develop solutions to prob-

lems

3. Use three types of evaluation questions.questions asking students to take- a stand on a con-

troversial issuequestions asking students to judge truth or validityquestions asking students to judge beauty or worth

and/or verify competence in this area, therefore, thesingle measure of percentage of higher cognitive ques-tions asked is adequate. If a teacher (preservice or in-service) asks more than. 60-70 percent higher cognitivequestions, he is performing much better than the aver-age teacher. If he is using considerably less than 50percent higher cognitive questions, he probably wouldbenefit from training. Given this general piece of as-sessment information. many decisions can be made. Itis only after a decision to design and instigate traininghas been made that the specific skill levels become im-portant. Learning to ask analysis, synthesis, and evalu-ation questions can help increase a teacher's overalluse of higher cognitive questions. Specificity at CI:-training stage is essential. For assessment purposes,general measures of skill use can provide adequate in-'formation and at the same time keep measurement ef-forts and costs within reasonable limits.

As outlined in figure 1, defining the range of skilluse to be assessed also includes selection of the subjectmatter areas in which teacher performance will bestudied. Again, this is a decision that affects assessmentcosts. Single examples of teaching are less costly to ob-tain than multiple examples.

For some teaching skills, the decision is obvious.Skills related to teaching reading should be assessed inthe context of a reading lesson. LikeW6e skills specificto the teaching of mathematics need to be measuredwhile mathematics is being taught.

Taken from Mini-course 9. Higher Cognitive Questioning.developed by Far West Laboiatory.

111

Other skills are more general in nature. Probihg, forexample, can be used in any subject area. It employsthe same type of teacher-pupil interaction regardless ofthe content of the lesson. A teacher's use of probing inany s bject area probably would be representative ofhis us of the, skill in general.

Still\ other skills differ in use depending upon thecontext in which they are applied; for example, re-.sponse \ to pupils' errors in reading probably differsfrom reiponse to errors in science.

The problem within an assessment effort is to deter-mine w 'ch subject areas are most likely to offer thebest opp rtunity for -teachers to exhibit . their use ofeach par cular skill. During the initial phases of as-sessment, inquiry probably will be limited. to generaliz-able skills or to specific content area skills. Because ofdata colle ion complexity, information regarding skillsthat apply to several content areas but differ in theiruse within each area will follow later.

Perform ce Setting. The third step in selecting theteaching pe ormance to be assessed is to select thesetting in w ich skill use will be measured. Based uponTurner's (1 ,72) six criteria of teacher performance, - ,inclass as contrasted with manipulated examples of be-havior may considered; the most common mani-vulated exam le being a microteach lesson. The salient

_question to be considered is whether a reasonablesample of skill\ can be obtained in a microteach setting.

Since much teaching occurs in small group or one.to-one tutoriallsituations, and critics of the classroom(such as Silberman, 1970) contend that large-group

vf

112

rfE

BEST COPY. AVAILABLE

Beatrice Ward

instruction should seldom take place, a small-groupsetting, e.g., a microteach lesson . may be an appropri-ate information base for studying a high proportionof,teaching skills. The disadvantage of such a perform-ance sample is that few student outcomes of any conse-quence could be expected to occur in a single 10-20mipute lesson. Thus, if study of teacher effect uponstudent learning is to be included in the. assessmentprocess, at least some examples of inclass performanceshould be included. It should be noted, however, thatgiven the present state of the art of assessment, re-searchers (Smith, 1971; Herbert, 1971) question theappropriateness of using student outcomes to measureteacher performance. Further, since the restricted .envi-ronment of a microteach lesson, whether live or onvideotape, also facilitates concurrent -scoring of anumber of teaching skills, this instructional settingwarrants serious consideration as an assessment tool.

Cost of AssessmentThe Teacher Education Division of the Far West

Laboratory has conducted a number of studies ofteacher change resulting from minicourse training. Inthese studies, an assessment procedure has been usedthat includes

assessment of already identified skills (skills wereidentified during development of training mate-rials),

assessment of both general and specific skill use(e.g., both percent of higher cognitive questions

4 4 a A

4

4*

411*

dames Steffeason

and use of analysis, synthesis, and evaluationquestions were assessed),

assessment within . a certain prescribed contentarea,

assessment within a microteach setting.

Using this procedure, microteach lessons were recordedon videotape and critiqued at some later date. bytrained observers. Only limited, if any, measures ofstudent performance were obtained.

Sample costs for conducting the assessment are pre-sented in table 3. They vie based on the study. of fiveteachers because this is the number of teachers whocan conveniently be scheduled into a microteach facil-ity during a 9:00 a.m. to 3:30 p.m. school day. Theaverage cost per teacher for five teachers is $180.Since critiquer training, 'research design, and data prep-aration costs would not be repeated with a larger sam-ple of teachers, the cost to assess 30 teachers would be$2,160; gr $72 per teacher.

If the skills to be measured and the lesson contentwere carefully selected, such an assessment effort couldprovide a large amount of information- about teacherperformance. It could be used for many of the pur-poses outlined earlier in this paper.

Who Will Do the Assessing?Assessment of teacher performance generally will be

conducted by three types of individuals.1. Teachers. Peer observations (one teacher ob-

serving another and vice versa) .can be used tostudy a wide variety of teaching skills.

Table 3.Cost of Assessing Teacher Performance

Using a Microteach Setting*

Collecting Performance Sample5 1/2-hour videotapes @ $15/each7.5 hours of research intern to monitor collection of

video tape examples (0 $9.450/year + benefits)

Ciiiiiquikg'PerformanceCritiquer training;1 hour training per behavior

12 behaviors X 2 critiquers X 1 hour24 hours .X .t 3.50/hour

12 hours of research intern to do training

Critiquers to score recorded lessons12 behaviors = 6 passes to score6 passes @ 30 min. each = 3 hours per tape3 hours X 5 tapes = 15 hours @ $3.50 per hour15 hours @ $3.50 per hour X 2 critiquers

Research design and data interpretation1 week research design planning@ $17,000/year (Note:If mere teachers were involved, this cost would bedispersed over theentire population)

p

D.ata preparationKeypunch I day clerk/analyst at $9,000/year

benefits 40.00Run and analyze 1 day 40.00

$ 75.00

50.00

84.0077 . 00I

105 . 00

$120.00

266.00

434.00 434.00

TOTAL COST (Assessment of 5 teachers)

* Based on assessment' of 5 teachers.

0 .1%

2. Critiquers. Analysis of audio and/or video taperecordings of ;caching (e.g., a microteach les-son) may be done by specially trained critiquers.They also may be teachers but more often theywill be graduate or undergraduate students in ed-ucation hired to carry out a particular critiquingtask.

3.' Inclass observers. Whenever an assessment pro-gram demands high accuracy in recordingteacher inclass performance, specially trained in-dividuals will be needed. Inclass scoring ofteacher performance requires simultaneous moni-toring of multiple variables and instantaneousrecognition of the skill(s) to be assessed. Con-

8Q.00

$900.00

a

*

113

siderable training and p,ractice must be com-pleted in order to achieve this degree of observerskill.

Information obtained through beer observations willbe the least accurate form of assessment data. None-theless, when the assessment is being done to identifyinservice teachers' training needf, this type of informa-tion is sufficient to establish a tentative skill profile.

In our work at the Far West Laboratory,. we testeda form of peer assessment in the responsive skills area.Six observations were made (in this case, four were -

done by teacher peers and two were completed by the

114

teacher's students). Gross information was obtained re-garding such skills as teacher response to pupil ideas,teacher 'use of positive and/or negative responses to/ student performance, andteacher use of nonverbal

,, 1 reinforcement. Given .the observation findings, theteachers were alerted to skill areas in-,.which they neededto improve. Decitions could .be made regardingwhere to begin the inservice trtning program for thatgroup of teachers. Tffe gross information received fromthe peer observations allowed the training to proceedwith some degree of individual focus..

Critiquer analysis of audio or video tape recordingsof teaching samples provides more precise .information

. about the skills used. The data are more reliable thanpeer observations. Many skills can be. scored from asingle teaching sample by rerunning the recording sev-eral times. On the other hand, since the teaching situa-tion being studied is manipulated, questions (must .beasked regarding the generalizability of these findings tothe teacher's regular performance in the classroom.The best time to use this form of assessment may be atthe end, of training when a teacher needs to be surethat he has acquired and is using a partigplar skill orset of skills and when nuances in skill use need to bepursued.

Overall, thesinost comprehensive obseWittion prom-, dure is to have highly skilled observers score teacher

and pupil performance in the classroom setting.' Asnoted above, this requires the observer to identify spe-cific skills at the moment they occur. Teacher-pupil in-teraction in an on-going classroom setting .cannot be

-rerun to check the observer's scoring. Further, sincethe observer can code the use of only a small numberof skills at a time, several observations may be neces-sary in order to code the same number. of skills thatcould be scored in a single 30- minute video taped les-

son. 1nclass observations appear to be most usefulWhen pupil outcomes are to be assessed as well as.teacher performance, when a particular teaching skill isexpected to occur only in a natural setting, when dataon teacher performance must be analyzed as soon asthey are obtained, and when equipment to recordteaching samples is in limited supply. Extensive inclassobservations are not essential to five of the six assess-

. ment purposes listed eadier in this'paper.The cost of assessment depends upon both the type

of .performande to be measured and the assessmentprocedure to, be used. Peer observations are the leastexpensive; inclass observittions the most costly unlesslarge amounts of equipment must be purchased to'in-stall critiquing of recorded lessons. Table 3 presented

tx

ft*

S.

J

an exmple of the costs associated with assessment ofmicroteach samples of teaching. Estimates of °the costof trained observer scoring of the same number ofteaching skills in the classroom would be higher thanthe $72 per teacher projected for the microteach sam-ple. Because of this increased cost, inclass observationsprobably should be used only when no other 'assess-ment method will provide the needed information.

Developing the Teacher's SkillA basic assumption of teacher assessment is that

once data are availableregarding a teacher's perform-ance, training will be offered to improve that teacher'scompetence. Using his skill profile to identify areas ofstrength and weakness, the teacher may waive trainingin one skill area to emphasize improving another. Tobe successful, a program that alloWs these options in-corporates training packages that cover diversity ofskills. Within a performance orientation, each of thesepackages generally will include models of the skills tobe acquired,, opportunities to ptactiee the skiffs, andevaluative' feedback regarding the teacher's perform-ance-during practice. .

As noted by Joyce (1973), development of suchtraining materials is costly. Time and effort, repre-sented by dollars, must be spent to identify the skills tobe included in the package, develop the initial form ofthe materials, test the materials to be sure teachers ac-quire the specified skills, and revise the materials basedupon 'testing results. Table 4 presents cost datp for thedesign of..one such type of training package, the mini-courses developed at the Far West' Laboratory. Thedata represent average costs for each development taskbased upon 'the design and testing of four minicourses.

The merit e: building a training system .tliot uses asmany already developed materials as possiN is ob-,vious given such development costs.

Table.4. Cost of Developing Training Materials

Development TaskConceptualizatioh and skill-identificationProduct developmentProduct testing (one test with 30-50 teachers)Revision after testing

TOTAL

Cosa$ 14,670

28,96246,27912,627

$102,538

An --additional cost factor that should be consideredaspart of a complete assessment cycle is the cost ofproviding training. Table 5 lists the estitrfted costs fora 5-week training sequence' that includes microteach-

/Table S. Cost oaf Training*

Starr-up castsMaterials..puichase16mm /limed models di witching skillsteacher handbooks 30 $2.50video tapes for microteaching 30 1;2 hour titlesEquipment purchase -VTR equipment. 3 sets

Training costsCoordinator of training.6 weeks for planning and monitoring

*Substitutes to release teachers.1 substitute per 10 teachers X 5 weeksOf training = days $35 /day

$ 1.0(X)75

,450

6,000

Subtotal $7,5256 .

2,025 .

1,050

Subtotal 3.075

TOTAL $10,600

Based upon minicourse training costs. -Optional; other ways may be used to release inservice teacherst 'or microteaching; not required in preservice.

ing. Start-up costs such as purchase of training mate-rials and video equipment are included as well as 'thecost of conducting the training itself. Once the trainingmaterials and equipment. are available, this type oftraining runs approximately $100 per teacher. Given a,shorter training period, a larger number of teachers towork with the coordinator, or less microteaching, the

p

I.

115

costs w'mpisbe, less. Ultimately, the cost would dependupon whether the training was tiart of a larger per-formance-based assessment add %training systtin of a

,single, one-shot, effort.

Summary

In This paper questions haye been raised regardingthe purposes of assessment;--the teacher performance to be assessed; andthe individuals who will do.the assessment, partic-

ularly the level of training and skill they must havebased upon the assessment approach biing used.

In each questioearett, s6veral optional. courses ofaction were presented. Different cost leveli were associ-ated with each option. Thus, the task for. the individ-ual, or agency, responsible for assessing teacher per-formance is to determine which options within eachdecision area best meet the policy-making and traininginformation needi of that particular program (agency).Central') to this decision will be the quantity and qualityof data required given the ,purposes for which the 'find-ings willbe used. A reasonable balanc6 must beachieved between the desire to obtain detailed, highlyreliable data and the total dollars to be spent.

a

4 v.,

o

i.V.

.3

116

Bibliography

Davis, 0. L. and Tinsley, D. C. Cognitive objectivesrevealed by classroom questiong asked by socialsimiles teachers. Peabody Journal of Education,1967, 45, 21-26? a

`Doll, Noiman, et al. "A CatOlog of Teacher Compe--, tencies-4A Working- Document." Tallahaisee:

Florida State Univers*, 1972.

Elementary Teacher Education Models, A Summary.. Journal of Risearch and 'Development In Educa-

tion, Vol: 2, No. 3. University of Georgia, Athens,Georgia;"Spring 1969.

. ..

Gallagher, J. J. Expressive though t by gifted children,. 4'..-1in tht classroom. Elementary English, 1965, 42,

''' ;568-559,4.

.

.

Ginza, F. -J. Teacher questioning and, reading. The; Reading Teacher, 1967, 21,227 -234.'

Iliseriaert, John. .A research base for accreditation ofteacher pi epara*n programs. Accreditatiot andResearch Problems, J. L. Burdin. and M. T. Rea-gan, eds. ERIC Clearinghouse on Teacher Edu-.calk)°, Washington, D.C., 1971.

Houston, W; Robert. "Strategies and ReSources forDeveloping a Competency-Based Teacher Educa-tion Program." University of Houston, New York °State Eilucation Department and Multi-State Con-sortiumi on Performance-Based Teacher Educa-tion, 1972.

Husek, T. R.. and Sirotnik, .Ken. Matrix sampling ineducational research: an empirical investigation.Paper presented at the 1968 convention of theAmerican Educational Research Association, Chi-

cago,

Joyce, Bruce Re Estimating costs of. a competency ori-entation., Paper presented at MUlti-State Consor-tium on -Performahce-Based Teacher 'Education,New Orleans, February 1973.

Joyce, Bruce R. and Well, Marsha. "Models of Teach-ing." Englewood Cliffs, N.J$: PrenticeHall, 1972.

Lord, F. M.' and Novick, M. R.-"Statistical Theories ofMental Test' Scores." Reading, Mass.: Addison-Wesley, 1968.

B. "Teaching Behaviours and StudentAchievement." Lopdon: National Foundation forEducational Reseirch in England and Wales,1971.

Suberminv C. E. "Crisis in the Classroom." NewYork: Random House, Inc., 1970.

Smith, B. 0. Certification of educational personnel.University of South Florida, unpublished paper,1971.

Turner, R. L. Relationships betwcin Teachers for theReal World and the eleMentary models program-matic themes and mechanisms payoffs, mechanismand cost. In B. Rosner (ed.), "The Power ofCompetency-Based Teacher Education." Boston:Allyn and Bacon, Inc., 1972.

s

4.

The Teachers Vie-Wpoi tBy

SANDRA FELDMAN

-1- have been asked to address my comments specifi-cally to the concerns of State Education Agency per:.sonnet will have been and will be making significantpolicy decisions concerning performance education. Iintend to do justriliatang I hope you'll forgive me.

Although it doesn't seem to be, it should be a sourceof sorry to state education departments that teachersarc suspicious and wary of them, that they are notlooked upon7by the professibn as allies, that they arenot looked to .as a resource for the solving of educa-tional problems, that they are not thought of as provid-ing educational leadership. .

19hould be food- for thought, at least, that often thepolicy7nakers in state educatioti departMents, in the

',business of prescribing for the schools, find themselveson opposite sides of the barricades from teachers inlegislative and.-other controversies on school matters,

/raging from certification to teacher education to "al-ternative" school plans or performance contracting, orvnuchcrs, km teacher evaluation, or. professional prac-tices acts, or questions Of school structure and gover-

-nanceor a host of other issues.Why?Why don't state education departmentsrhave a

healthier respect for classroom teachers, for their accu-mutated experienct and expertise, for -their organiza-tions as,an expression of teacher concerns?

. Why aren't teachers more significantly involved inthe process of decision making at slate 'education de-eartment levels. (And by teachers I do not mean thestate education department personnel *ho once. were''*the classroom and .feel themselves able, therefore, tospeak for teachers. Nor do I niean selected individualteachers, handpicked by school administrations. 1 meanteacher leaders selected democratically by the teachersthemselves, through their organisation.)

I Would like to hear what you think the reasons are:I think 'they are manifold and I will tell you what 1think first.

One reason, in my opinion, is an incomprehensibleinability of education administration to accept the fact

-S-

117

that times have changed, that teachers are organized,that they insist on speaking for themselves and on hav-ing a voice in, the decisions which affect not just theirworking lives, but the education of the children theyteach.

Insofar as your invitation to this conference ofteacher representatives demonstrates some conscious-ness of that, 4 congratulate youand 1 intend to takefull advantage of this opportunity. But I do so with acertain skepticism; as a participant in many confer-ences such as this one, I seldom see the fruits of suchexchanges in concrete application back home.

Another reason for a lack of 'real cooperation be-tween teache'rs and their state education departments isa result, 1 thinkl of some state education departmentofficials seeing themselves as in the vanguard of educa-tional change and seeing teachers as defenders of thestatas quo. To teachers, this characterization is ludi-crous. They do not see their derhands for drasticallyreduced class size, for massively -inkreased social, psy-chological, and health services for "children, for themaintenance of high professional standards as "oldhat." They do not see performance contracting, "ac-countability," legislation like the Stull Act, differen-tiated staffing, and merit pay schemes as revolutionaryimprovements, but as backward steps and 'attacks onthe integrity of the profession. They are not- inipressedby paper-wei3hty master plans or glossy brochures ad-vertising "bad new steps" which usually do nothingwhatsoever to help thenfin the classroom.

There is yet another reason, in my opinion, for the"credibilitgap" between teachers and state educationdepartments; and this one I will give a little more timeto, for it will lead me to the subject at hand. That is, Ifear that many sate' educttion department officials inpolicymaking roles (1 )-are under great pressure to re-spond to the now famous "crisis of confidence" in theschools, without the knowledge and resources to dosomething real to end failure, and (2) share in thegrowing public attittode-that teachers are responsible

II

BEST COPY AVAILABLE

118

for school failure and either don't know how to do

something about itor don't want to.So, if we are to succeed in ehieving the very wor-

.thy goal of this consortium hich is supposed to be

"devoted to improving the q ality of State Educationpersonnel and their decision-making abilities," we haveto first establish that one important, essential way ofimproving that policy decision making is to listen tothe teacher organizations. Then we have to talk aboutthe crisis in confidence that spurred the performance-based movementwhy we have it and what we oughtto do about it.

8

V

4.

ofi-;

ialffit* .1411

Sandra Feldthin

Crisis in Confidence .0

The schools do not appear to he meeting the de-mands of a changed economy, of a job market whichrequires mote and more skills and increasingly higherlevels of training. Whether or not the schools in thepast were the vehicle for upward mobility, for provid-ing the path cut of poverty, most people believe thatthey wereand expect them to continue that function.This problem of higher expectations is compounded bythe/ sociai turbulence of the sixties and by the racialconflict which so often was most visible around school

issues.Second, despite the recent "revelation" by Christo-

pher Jencks that schooling cannot end economic ine-quality among the adult populationand despite the

fact that this was undoubtedly always true, that eco-nomic equality would have to be legislated as a matterof social policypublic schooling has undeniably pro-duced a literate citizenry. Where in the pastand es-pecially through the thirties, forties, and fiftiesschoolteachers were among the most educated people in oursociety, .even when they only had a high school ortraining school degree, today, the country is full of ed,ucated people. Thousands- upon thousands of parentsand citizens in white collar government employment, inadvertising, in media jobs, accounting, computers, in awide variety of nonteaching academic pursuits, feelthat if they were not working at their present occupa-tion, they could teach. Today, although teachers arestill among the most educated; they do' not have thestatus that comes,with having knowledge that others donot have. While a holder of a Ph.D. in economics feelsthat a doctor's or a lawyer's skills are very special,even somewhat mysterious and intimidating, he doesnot feel that wa$, about a teachereven if the teacheralso holds a Ph.D.

As a profession, education must, like other profes-sions, assemble its experience and skill into a concrete,coherent body of knowledge as tangible as what exists

in law or medicine.That is why we must have performance-based

teacher education. Not only will It make teacher educa-don more relevant to classroom needs and thus im-prove education, but it will, if developed properly, pro-vide us with that concrete knowledge without which wecannot much longer defend the public school system.

And that is why I believe, unshakeably, thatdeveloping a body of knowledge about the teaching-learning process ought to be the initi.' thrust andprime, concern at present of the performance-basedteacher education movement. We should be settingabout doing what Fred MacDonald is proposing bedone by a National Commission on Research and De-velopment. We should be making a coordinated, long-term commitment to validating teacher competenciesnot what is being done, which is a short-term com-mitment to listing them. We should be working atproving what teacher behavior, what teaching strate-gies, effect what learning and how.

We should not, as the New York State EducationDepartment is doing, be telling teacher education insti-tutions to "Pool! Change over to a 'performance-basedteacher education program for. certification approvalpurposes." At a meeting several months ago, to whichrepresentatives of teacher education institutions and thestate teacher organization were invited, there were as

many different notions of what performance-basedteacher education is as there were deans of educationat the meeting. And the leadership in the state educa-tion department is not providing a model or a researchdesign .or any r..:t1 guidanceunless you call require-ments listed on a forth for program approval, guidance.We will have as many different, and as many irrelevantand inadequate teacher education programs as we tioright nowbut New York State will "brag" about itsperformance-based teacher education program at con-ferences like these throughout the country.

Just recently, at a meeting of sophisticated chairmenof education.departments of one of our latgest teachereducation institutions, I saw education department ,rep-resentatives of. a college struggle with a paper descrip7'tion of what they hoped would be considered a per-formance-based program; and they had listed nok.teacher competencies, but behavioral objectiveswhichis fine to do, but from which, as Bunny Smith has em-phasixed. over and over, you learn nothing about theteaching process.

I have grave doubts as to whether the states can au-tonomously develop meaningful performance-basedprogramsbut I have no doubt that most teacher edu-cation institutions, without resources for research andwithout access to a large. variety of classrooms, cannotdo it.

If the performance-based teacher education move-ment is going to be meaningful, it cannot be handledas a public relations gesture by state education depart-ments and tossed back to the colleges jwhich have beenfloundering all akng. Nor should the effort to developa knowledge base of teacher competencies be tied tocertification at this time, and certainly not to continu-ing certification schemes, because teachers will notstand for the destruction of tenure and job rightsthrough renewable certificationespeclally when a re-view of performance is based on unvatidated lists ofcompetencies.

If performance-based teacher education is going tobe meaningful, it cannot beas every tiducation "inno-vation"' usually isdoomed, to success. In any otherprofessiontake medicine-Lstrategies !are tried andtested and discarded if they don't work ',and the knowl-edge gained from systematic research into those strate-gies, whether they cure the disease or not, provides acontinuous buildup of intelligence in the field: In edu-cation, we scurry to hideor ignorefailure. Thatthe research we've done so far is inconclusive, becausesophisticated instruments to successfully interpret andmeasure teacher behavior are still in the early stages of

119

development -- and I know you've heard MacDonald,Rosenshine, and Schalockshould not be a source ofembarrassment to us. It should provide us with the re-solve to push that important work forward.

Instead, each 'state goes its own way, without a pool-ing and sharingnot just of knowledge, but of politi-cat to ge. a commitment for funds to do thetsearch we all know needs to be done. In New YorkState we have a very definite timetable. Byis itI984?we will have performance-based teacher edu-cation, so-called, and performance-based teacher certi-fication throughout the State. But will we know anymore about education or about teaching? I think notunless the National Commission for Research andDevelopment is ready to share their work with usand they haven't got nearly enough money to do thework that has to be done. l that is the direction weshould be tasting.

What vitt7.'should be doingand 'actually the multi-state consortium might well be an excellent vehicle forthisis cO4dinating a single effort, a national effort todevelop a knowledge base. We should be collating thelimited knowlEdge we now have, and we should bebuilding a small number of teacher models based onavailable research and the opinion of experiencedclassroom educators. We should be developing in abroad cross-section of schools throughout the Nation asystematic assessment and data-gathering machinerywhich would enable us to compare and study 'teachingbehavior and its effects on learning response in a pre-scribed variety of school environments. We shouldteach prospective teachers in the models creatednothundreds of different ones, but a fewso that we cancontrol for effects, and we should have a research de-sign built in so we can validate the competencies wehave agreed on by studying those new teachers, on thejob, where their education should continue during aninternship-probationary period.

While I believe that the main thrust of perform-ance-based teacher education should, in the beginning,be preservice, there must be involvement of experi-enced classroom teachers because they have a greatdeal of expertise to offer and because they will learn agreat deal in the process of participating.

This past summer, The City University of New Yorkcame to UFT to seek cooperation in the de4lopmentof competencies for teacher-training models. In a shorttime, we recruited over 300 interested teachers (whowere paid a fee for their participan, as they shouldbe). They provided a valuable contribution to thework, and they learned 4 great deal:

12Q

"I was forced to think out in a concrete role my role

and skills as a teacher.""1 was forced to analyze myself and think of what

'really makes a good teacher.""It is high time an emphasis was put on specific

competencies rather than abstract theories.""I came away more optimistic about the future of

education."(And school critics arc always seeking a way to bet-

ter teacher "attitudes"!)State education departments should be joining with

the teacher orginizations to demand that National In-stitute of Education and its funds be put at our dis-

posal; that it be used as a resource for the profession,not in the form of hundreds of diverse grants awardedin a scattered way throughout the country, but in a

aft

so"

ti

massive, concentrated effort to develop the profession

in the way rhave just briefly described.As state education department personnel, you should

not he plunging forward blindly, as I think jtli are,

because of the great public pressure to "do something."

You shoulti insist on your right and your need to base

your decision making on substantive, proven, proles-

iii-mal knowledge. Whether you do or don't, the

reacher organizations will insist that you do; r itthere is a struggle between those demandin icicle

public relations solutions and the teachers who are de- *.manding substance, you will be squeezed in the addle- --unless you have already chosen sidesand thepromise of performance-based teacher education may

die in the process,

121

Some Substantive issues And Political ConsiderationsIn Performance-Based Teachet Education

What Have We Heard?

By

BERNARD H. MCKENNA*

11. finally we've heard that measuring specific per-formances at various criterion levels will becostly.

There are a lot of other lesser problems we've heardabout, and likely some additional major ones I'veomitted.

We've heard here over the past 4 days that

1. the state of the art is little advancedin factsomeone termed it "primitive";

2. there is little data on what makes for compe-., tenceas some put it"We know too little

about what's worth teaching";3. the most difficult job, will be defining competen-

cies;4. adequaLy of assessment apparatus is the most

critical, issue and will be most 'difficult to de-velop;

5. the definitions of behaviors are, at this' point,almost totally hypothetical;

6. we la& summative evaluation tools; i.e., thereare rating forms for making global judgments,and there are research tools containing specificskill forniulationbut nothing in between;

7. there is a complex interaction between the set-ting and performance, an interaction that af-fects performance in a major way, one that wehaven't much taken into account, let alone ana-lyzed;

8. the affective domain is being neglected in per-formance -based teacher education;

9. the tools for measurement in the social sci-ences, including teacher education, are in aniiifant stage; i.e., even with the sophisticationdeveloped in measurement on cognitive skillssuch as reading, we're still not able to diagnosecauses of all reading difficulties, let alone pre-scribe behaviors that will correct themso, ithas been asked, how will we soon be able tomeasure attitudes, values, and the like?

10. there is a monumental problem in matchingteaching styles to learning styles;

* Dr. McKenna is professional associate, Instruction andProfessional Development, National Education Association.

What Does What We've Heard Say To Us?One thing it might say is "forget it." My starting off

with such a laundry list could cause you to believe.that's where I am"forget it," or that I'm launchhiginto a defensive diatribe.

Actually, I believe that performance-based teachereducation is a promising and viable concept and shouldhe pursued.

O

My listing these concerns is for the purpose of put-.ting a framework around what I'm about to respondto, and that is the followilis questidn.

What Are the Main Issues in Performance-BasedTeacher Education That Researchers,Developers, Teachers, Professors, andOthers Should Be Emphasizing?

In my judgment, a first line of activity should bearound tools for evaluation. Peiformance-based teachereducation will rise or fall on our ability to evaluate.That is, a first-order question is 'How shall/ we deter-

, Itmine that teacher candidate A is now ready to practicethe profession and should be licensed to do so and thatcandidate X shotild ."go back to the drawing board"?'

Another way of saying it is that until we can meas-ure performance in such a way that we are able toconfidently identify minimal levels of performance topractice the profession) we will not have performance-based teacher education.

Some have said at this conference that prior to iden-tifying minimal levels of performance there are themore difficult jobs of

1.3

122

(a) establishing goals and objectives. for education((h) identifying perforniances (competencies') re4

quired for achieving the goals and objectives.

I don't believe those to be the most difficult tasks;and even if they were, we're further along in accom-plishing them than wc'are on the evaluation matter.

For examrlo, there are available1. some Well-developed needs-assessment systems;

2. several quite complete taxonomies;3. the works of Mager, Alkin, Popham, and others-

on getting goals and objectives into more nearlymanageable performance-based terms;

4.. "The Florida Catalog of ConVetences" and"Sigtir

6ofGood Teaching"' (the latter is less

known but highly developed);"5. ihe Seven Cardinal P.rinciplei or Ten Purposes

of Secondary Education;6. the techniques of most recent vintage reported at

this conference by Adele Thomas of TheUniversity of New York in which they were ableto gain high consensus on which skills should be.most valued;

7. the Northwest Regional Laboratory materialsand those of the Teacher Corps.

I'm not that .concerned about our ability to arrive atgoals and objectives.

Some in this conference have argued that "instruc-tional systems," as BrUce Joyce termed them, will bemost difficult to develop. I'd argue that we have madequite satisfactory progress on these also.

Whether one goes with those toward the affectiveend of the continuum like "Man a Course of Study"and the Taba Social Studies Curriculum or the morebehavior-modification oriented types reflected in someof the packaged curriculums, there is a variety ofquite well-developed instructional systems to choosefrom which include both content and teaching strate-gies.

So that brings us back to evaluation devicesand I

repeatperformance-based. ..Lacher education will riseor fall on our ability to evaluate performance in rela-tion to agreed-upon standards.

Past research on evaluation leaves much to be de-sired; and I'm convinced that too little emphasis isbeing given to research and development -in this keyarea in present efforts with 'performance -based teachereducation.

.There are places to start, and I'm surprisol thatmore of the programs haven't attended to those prom-ising possibilities. For example, the .performances iden-

tilled by, Rosenshine and Furst, such as clarity, varia-bility, enthusiasm, have shown some promising..relationships to outcomes with students and are oneplace to begin. Ova

There are also others that should be consideredwhether or not they show direct relationship to learn-ing outcomes. The point here is that there are someprocesses which may not lead directly to agreed-upongoals (or at least cannot be demonstrated to det so)which .should be promoted and which. are worthy ofevaluation for their own sake. One brief illustration ofthis. A recent study of several thousand students repre-sentative. of the American high school population con-cluded that great numbers of students may be develop-ing little affinity to the democratic process simplybecause they have little opportunity to experience itduring their school career. If this is so, is it not impor-tant that the process of education in the schools be-come, in as much as possible, a microcosm of the bestof democracy as' it is practiced in the greater society?And isn't this important whether or not such processescan be definitively shown to contribute to specificlearning outcomes? Evaluation systems for perform-ance-based teacher education should incorporate such

criteria.I believe, as Bruce Joyce -predicted earlier in this

conference, that it will take 20 or more years to de-velop reliable and valid evaluation tools. It will proba-bly cost more than his suggested $100 million. Whenasked if it were possible to develop,an accountabilitymodel for New York City, Henry Dyer of EducationalTesting Service, responded that it could be done butwould be more difficult than getting to the moonandit is well known that NASA had for several years forthat project, almost unlimited resources and unlimitedpower.

Three years later Fred .Mcponald and his peoplehave made some progress on The New York City ac-

- countability mandate. I've just read the description oftheir model. It appears promising on first .look. But italmost totally skirts the problem of relating perform-ance to student achievement. What might be the costand the time line if this issue were responded to?

Nat Gage told us .on Sunday that Lindquist gavewarning 30 years or more ago that there are some 18or 20 variables other than teacher performance whichcontribute to student achievement. Gene Glass put itwell recently when he concluded front WS research that"aside' from the irrelevance of much of the °content ofstandardized achievement tests, their use .in evaluatingteachers is unjust.

BEST COPY MINABLE

It' you don't think that kind of thing is being pro-moted, you have only to read the Stull Act in Califor-nia. It's like directing the medical profession to find acancer cure in a tear or mandating the Nation's econ-omists to correct the balance-of-payments conundrumin a'few months. Or look at the FIcischmann _Commis-sion Report in New York State, or some of the Manragement by Objectives programs that are being laid onschool statTs across the country.

1

123

skillsnot sameness. (An orchestra of all piccoloplayers would be pretty dull and would certainly re-ceive a strange feview from the music critics.) Therehas been considerable research done on this team con-

., cept of evaluation over a period of 30 years in the In-stitute of Administrative Research, Columbia Univer-sity. "Indicators of Quality" is n promising tool for thispurpose.

Bernard IL McKenna

Two other quick things on the evaluation issue. Thefirst is that where work is being done' on evaluation re-lated to performance-based teacher education, thereappears to be overemphasis on evaluation of didactic *

teaching styles. One hears the terms "teaching lessons"and "collecting student papers" and other lecture-typeactivities referred to all too frequently. I thought BettyWard was very hold when she raised the question yes- ?

terday about the desirability of assessing'skills of totalgroups since the teaching process is becoming'more ofa team 4arrangement, more interpersonal, more a reflec-tion of the open-classroom concept, more guided inde-pendent study. I say she was hold, since the mini-courses she has been so involved in developing to aconsiderable degree assume a didactic teaching style.

The second point is that there should be more ton-sideration of evaluating staff as a team. I think it wasBetty Ward also who suggested "don't test every skillin every teacher." In this 'regard, we might look on thestaff as an orchestraevaluating for complementary

How Does Perforinance-Based TeacherEducation Relate to Inservice?

If inservice education had ever amounted toanything, we'd have had respectable performance-basedteacher education long ago.

We've had inservice education a long time. We'vealso had performance evaluation inservice a long time.In fact, one might expect most inservice evaluation tohe performance-based and for the purpose of determin-ing corrective measures (inservice. eeds).

Instead this is the way it has been: inservice evallia-tion of school staffs has been poorly' conceived, littledeveloped, only haphazardly implemented and the re-sults often inappropriately used.

In our work with I- million plui teachers across thecountry, we 'note some quite commonly held attitudeson their, part about inservice education:

1. They find it threatening.2. They assume its main purpose wilt, be to deter-, mine change of status-- retention, tenure, promo -

tiun.3 They feel uninforined about the criteria by

which they are evaluated.4. They assume evaluation will be accomplished

through brief observations (too brief) and basedon some kind of checklist:

5. They expect it' will be done to them by some-body else, someone in a more elevated position.

6. They say, the feedback they get from it is inade-quate and often absent.

7. They believe there will be few concrete recom-mendations for improvement as a result of theirbeing evaluated.

8. They expect that following evaluation there wiltbe little or no help to improve their performancethrough inservice programs geared specifically tocorrect the lacks identified through evaluation.

That's the Way it is according to our members. So ifperformance-based teacher education is to be carriedover to inservice education, it will need to respond toall the deficiencies of present inservice evaluation andinservice education. And that's a big order.

If performance-based teacher education works, withits proposed field orientation, hospital-schools concept,

124

internships, and career ladders, then the transitionfrom preservice to inservice both in terms of the proc-ess and the evaluation of it should become so gradual,

bniooth, that one can't tell where one leaves off andthe other starts. It would then approach what the lawprofession is getting underway in the new Antioch LawSchool in the city where 1 live, and what Western Re-serve University got underway in medical educationmore than 15 years ago.

That's short shrift for the inservice aspects, but thereis some other ground to cover.

What Is Recommendable?A. Involve on a parity-basisnot just tokenall

groups within the profession who will be affectedby the program. You might begin with some of thestates in this consortium.

B. Assure that goals and objectives for PBTE arebased on goals and objectives for schooling.

C. Assure that goals and objectives for schooling arecooperatively developed by parents, educators, andstadentS.

D. When objectives are turned into performance cri-teria, provide for macro as well as micro perform-ances.

E. Assure that program and conditions are evaluatedconcurrently with performance and that they arerelated to it. It's futile to evaluate a teacher's per-formance without also evaluating all these condi-tions that make competent performance possible.

F. Involve all those who will be affected by it, in de-veloping the evaluation planprospective teachers,teacher supervisors, their students, and some par-ents.'

G. Use multiple indexes for evaluation.H. Involve a variety of personnel in conducting evalua-

tionself, 'peers, supervisors, students.I. Thoroughly train those who will evaluate. This may

be the most important.J. Develop concurrently plans for correcting both

those program and condition elements and perform-ances which are shown to be deficient as a resultof evaluation. This may be the most costly, partic-ularly on the inservice

K. Assure that there will be ample opportunity to im-prove (including direct assistance, time, and mate-rials) for those judged to require improvement as aresult of evaluation.

L. Provide for full-blown, written, agreed-upon ar-rangements for both substantive and procedural dueprocess when evaluation systems are to be used fordecision making lin other than determining .itkserv-ice; that is, the retention, tenure, promotion. 'K

M. Secure research funds commensurate with the large-,ness of the task.

N. Develop time lines at allow for realistic accom-plishment of what's a complishable, taking into ac-count the state of the, art and the complex inter-

.

relationships of the many variables as well as thefeebleness of tools in social science measurement.

0. Researchers get in there and testify before legisla-tures on what's possible and what's not.

P. Stay close to the schools. ( i couldn't disagree morewith Fred McDonald when he said, "You can'tlearn anything in the messy classroom situation."That may be why ETS was so long in recognizingthe injustices of the National Teachers Examina-tion.)

Q. And finally, provide sufficient periods for test andtryout, evaluation, and recycling before dissemina-tionbefore making extravagant promises about allthe ills the new program is sure to correct. Let usnot allow this innovation to suffer the same set-backs as:

1. the teacher aide concept which got underway inBay City, Michigan, 20 years ago, but was prob-ably delayed 10 or more years by poor projectdesign and overstated promotion;

2. or educational television which suffered some ofthe same problems partially as a result of theHagerstown experiment in the midfifties;

3. or differentiated staffing which appears to havesuffered in a similar way in the last 2 or 3 years.

What Are The 'tics Of The Matter?My final charge was to deal with the politics of -per-

formance-based teacher education.As you have doubtless noted by now, I assumed I

had several charges.Ted Andrews told me by phone that the symposia

were to be on assessment in CBTE. Later in a letter hesuggested I deal with its relation to inservice education.And when the program came out, the term "politics"appeared in the title of my presentation.

According to someone's definition, politics is "theart of the possible." If that's so, I've already coveredwhat I think is possible and what conditions I believeare necessary to accomplish the possible. Just an addi-tional thought on political considerations.

I am weary of power struggles. Unlike Pat Goralski,I don't accept them as givens. I hope soon we'll get be-yond them so we can concentrate our energies on get-ting the priorities of the American people in betterbalance in order to obtain th., cvnditions and resourcesthat are so badly needed for schooling, all the wayfrom prenursery school to graduate programs.

This can be accomplished, at least partially, if wework toward parity in decision making and then con-centrate on issues rather than on which individuals orgroups will have the Most power.

Redistribution of power so that it will be more equi-table among all groups concerned will in the end giveus all more power.