APPLYING THE TONGRUENCE"PRINCIPLEOE BLOOM'S TAXONOMY TO DESIGNINGONLINE INSTRUCTION

Seung-Youn (Yonnie) Chyung and Donald StepichBoise Stale Universiiv

Bloom and his colleagues developed the laxonomy of educaliona! objcclivcs in 1936, and 45 years later agroup of psychologists and educatoi s irvised the one-dimensional taxonomy lo a two-dimensional Cixonomy,The developers offhi; laxonomy lhcori/.i:d Ihal ihc taxonomy ofcducalional objectives could be u.' ed wilh anysuhjcci matter and for any levels of learners. However, lillie has been written aboiii how ihe laxonomy can belilfeeiivcly used in designing asynchronously-delivcred online inslmetion. We have found Ihat Ihe laxonomyis an efleclive guideline for designing graduale-lcvel online inslmclion, because il helped us maintain ihe con-gruence among inslmclional components. In this article, we present a case study that explains how the use olthe taxonomy of educational objectives was instrumental in the developinenl of graduate-level online insinic-tion.

INTRODUCTION

Instructional design (ID) is a prescriptive sci-ence in that its fundamental purpose is to iden-tily instructional methods that are likely tolead to desired learning goals in a given situa-tion (Reigeluth. 1983). One principle thathelps instructional designers accomplish thispurpose is the principle of congruence, whichsays that, in any situation, learning goals,instructional strategies, and assessment tneth-ods should be carefully matched, or congruent(Dick, Carey, & Carey, 2()OI; Smith & Ragan,

1999). To achieve ihis congruence, manyinstructional design models suggest identify-ing the intended learning outcomes (objec-tives) and determining the types of learningrepresented by those objectives (Dick. Carey,& Carey, 2(K)I; Morrison, Ross. & Kemp.2001; Smith & Ragan. 1999). In this way,objectives serve as a cornerstone for theinstructional design process, helping to deter-mine the instructional strategies and assess-inent methods that will be used which, in turn,helps to ensure the congruenee of the instruc-tion.

• Stuns-Youii (Yoniiiel Chyung, Department of Instruclional & Performance Technology, College ol" Engineering. BoiseState Universily. 1910 University t)r.. Boise. ID 83725-2070. H-mail: ychung@boisestate.edu

TheQuaiierly Review of Dislance Hducation. Volume 4(.l). 2(KI\ pp. 317-130 ISSN 1,528-3518Copyright © 2003 Inlormalion Age Publishing, Inc. All rights of reproduction in any form resei-ved.

The Quuiicrly Rcviuw iil' Disiaiicc hducaiian Vol. 4. No. .1, 2(K)3

Various classification schemes have beendeveloped lo describe dirfeieni types ol Icurn-inj; outcomes (Gagnc. 1985; Morrison. Ross.& Kemp, 2(K)I). One of the most well knownis Bloom's Taxonomy of Educational Objoc-tivcs (Bloom. Engclhart. Furst. Hill. & Krath-wohl. 1956: Anderson & Krathwohl. 2(K)I).When it was deveUtpcd. Blooin and his col-leagues iheori/ed that the taxonomy ct>uld beused wilh any learners and in any content area.Because of its broad appllcabilily, the taxon-omy has been widely used as a guide lordesigning inslruclion in traditional face-to-face learning environments (e.g.. Ainsworth,1994: Calhoun. Davidson. Sinioris. Vincent, &Gritlith. 2(K)2; Chcethan & Chivers. 2(K)I:Niehoff & Whitney-Bammerlin. 1995).

However, ihe growth and development ofthe World Wide Web as an instructional deliv-ery system has brought a new dimension lo thedesign of instruction. In contrast to traditionalface-io-facc inslruelion. asynchront)us onlinelearning environments: provide a "reducedcues environment" {Haythornthwaitc. Ka/incr.Robins. & .Shoemaker, 2000) in which facialexpressit)n, vocal inllection. and olher nonver-bal cues are absent: involve mostly asyncbro-nous communication among participants,which means that participants do not interactin real time: and ask students to dcmonstraletheir understanding of course content mostlyvia writing. These characteristics have a num-ber of benefils. For example, students can par-ticipate at a time and place that is convenientfor them; there is redueed time pressure andstudents can think about others" messages, aswell as their own. before participating: and it Ismore dilTieult for one person to dominate theinteraction. At the same time, asynchronousonline learning presents the instructionaldesigner with a number of ehallengcs. One ofthese challenges is developing instructionalactivities that will engage and inlorm students,who have different knowledge and experiencelevels, at a distance. While Bloom's taxonomyhas been used as a guide in the design offacc-tolace instruclion. little has been writtenabt)ut its use in the design o\ asynchronous

online inslruction. This article describes onepossible approach. I'he article contains twomain sections: a brief overview of BlcHim'staxonomy, and a description of how we haveused Bloom's taxonomy in the design of theinstructional materials for a graduate-levelonline course.

BLOOM'S TAXONOMY

The original purpose for developing a taxon-omy of edueational objectives was to "providefor classification ol the goals of our educa-tional systotn" {Blootn. et al., 1956, p. 1). Thisgrew out of a need, expressed by a group ofeollcge examiners, fur a iheoretical frameworkthat would allow educators to eommunicatcand share testing-relaled information with itne;mother. The resulting discussions, heldbetween 1949 and 1953. lead to the conclusionthat evaluation criteria and testing materialsshould be based on well defined educationalobjectives. Bloom and his colleagues deliber-ately cbose the term "taxonomy" for their clas-sification system to emphasize the hierarchicalnature oi learning. From their research, theyconcluded that, "there is an unmistakable trendpointing toward a hierarchy of classes ofbehavitsr which is in aecordancc with ourpresent tentative classifieation of Ihese behav-iors" (Bloom, ct al.. 1956. p. 19). As a result,they idcntilled six levels of cognitive skill,from siinple to complex, and argued thatinstructional objoetives should be ordered totbilow that hierarchical sequence (see Table1). Their hypothesis was that learning complexcognitive skills, such as the ability to synthe-size interrelated intormution, would be basedon learning simpler cognitive skills, such asunderstanding concepts and principles, wbichwould be built based on learning even muchsimpler cognitive skills, such as rememberingspecific facts.

Ultimately, three taxonomies were devel-oped: a cognitive taxonomy deseribing intcl-leclual abilities (Bloom, et al.. 1956), anaffective taxonomy describing values and atti-

Applying ihe 'Congruence" Prim iple of Bloom' s Taxonomy lo Online Instruction 319

TABLF. I

The Original Taxonomy of Ehe Cognitive Domain (Bloom, cl al.. 1956)

Uvel DescriptionEvaluation Makingjinlgmciits aboui ihe value of ideas, works, solutions, rnclhods, materials, etc.Synthesis Putting logoiher of elements and piuts so as to form a whole.Analysis Breakdown of the material into its eonstitiicnt p;«1sand deteetion of the relationships of the pads

and of the way they are organized.Applieation The use of abstraciions in particular and concrete silualioiis.Comprehension An understanding of (he literal message contained in a eommunication. Three lypcs of

comprehension behavior are translation, inlerprelalion, and extrapolalion.Knowledge The recall ot specifie. and universal^, the recall of methmls and processes, or the reeall of a

pattern, structure, or setting,

tudes (Kralhwohl, Bloom, & Masia. 1964);and a psychomotor taxonomy describing mus-cular movements and the manipulation ofobjects (Simpson, 1972; Harrow. 1972). Thedevelopment of three taxonomies was ba.sedon the premise that there are three qualitalivelydifferent types, or "domains." of learning. Inthis view, learning a cognitive skill (for exam-ple, reading music) is nol the same as learninga physical .skill (for example, plucking thestrings of a guitar) or learning an afiective skill(for example, appreciating music played on aguitar). Each domain includes a different hier-archy of component skills and requires differ-ent instruetionai strategies.

While three taxonomies were developed,the cognitive taxonomy has received the mostattention, because it is often seen as the majorfocus of secondary and post-secondary educa-tion. In a recent revision (Anderson & Krath-wohl, 2001), the levels have been reorderedslightly. More importantly, a second, knowl-edge dimension has been added to provide amore comprehensive system for classifyingedueationat objectives. Definitions for theknowledge and cognitive process dimensionsare provided in Table 2.

While the taxonomy has been widely used,it has also been criticized (Bereiter & Searda-malia. 1998; Moore, 1982; Sugrue. 2002), Atthe heart of the criticistns is the idea that ihetaxonomy is analytic, meaning that it attemptsto hreak learning down into component partsand identify prerequisite relationships amongthose components. Critics argue that learning is

synthetic or holistie and thatcognitive skills aredynamically interdependent rather than hierar-chically related. For example, understanding aconcept or principle (the understand level) mayoccur in conjunction with learning how to use it(the apply level) or determining its value in agiven situation (the evaluation level).

These criticisms reflect changes in the waylearning is eonceptualized (Cunningham,1992: Jonassen. 199!: Jonassen, Peck. & Wil-son. 1999) and instructional designers havebegun to incorporate these changes into theinstructional design process (Tennyson. 1995;Wilson. 1997: Young, 1993), However, webelieve that Bloom's taxonomy still has meritas a guide for instructional planning for twospecific reasons. First, it reminds us that thekey 10 effeetive instruction is the congruenceor '"degree of correspondence among theobjectives, instruction, and assessment"(Anderson & Krathwohl, 2001. p. 10), Second,because it is analytical, it helps remind us thatlearning is made up of a complex array of cog-nitive skills. At the same time, it doesn't pre-vent us from designing instruction in a moredynamic way, in which a low-level cognitiveskill can be learned in conjunction witb ahigher-level cognitive skill.

In the next sections, we describe our effortsto use Bloom's taxonomy as a guide fordesigning and developing graduate-levelonline instruction and how the taxonomyhelped us maintain the congruence atnong thecomponents of online instruction.

The Quarterly Review oi" Disiajiec Hducacion Vol. 4. No . , 2003

cc

o>i

BooXrt

E -•a

H

-^ c

I - ^ 15

U 0-

y '-

•c CLo «< 'P

,r -j H 3 =

S -3O =

< a: C 5 a

< u

T3 SC —•3 '.J

::> u

P c? >a. '5b

-5 E

? /? "71 .tS

2 I

<:2. 3. °

Applying the 'Congruence"Primiple of Bloom' v Tctximoniy to Onlim- Insininiou

BACKGROUND OFTHE COURSE

(Gagne. 1985). collapsed into ihi: loMowingfive major sections:

The Instruciional & Performance Technology(IPT) Depanmenl at Boise State Universiiyoffers both online and on-campus coursesleading lo a master's degree. AM onlinecourses are delivered asynchronousiy via theInternet using Lotus Notes software. Duringihc fall 2()02 semester, both authors taught acourse titled. "Introduction to IPT." The goalof the course was to help new students acL|uireinformation ahout the historical and theoreticalIbundations of the IPT field. Enrollment was13 students in each class.

Within Lotus Notes, the database holdingall of the course documents is referred to as a"course room."' Unlike other Weh-basedcourse management systems. Lotus Notes doesnot structure a course according lo a pre-dcter-mined set of placeholders. Instead, instructorscan construct their course rooms in whateverway they choose. We used a calendar approachto construct our course rporn. With thisapproach, the semester was divided into mod-ules with each module designed to last 7 days,from Sunday to Saturday. Each mcxJule wasbased on Gagne's "nine events of instruction"

A "Read Me First" section. This sectionwas used to post announcements and topresent the instructional objectives for themodule, as a way of telling students whatthey were expected to accomplish duringthe week. In addition, students were givenan advanced organizer designed to pro-vide them with a clear direction for theweek and help them construct relation-ships between information from previousweeks and the current week's topics.A section for new content. This sectionwas used to post multimedia instructionalmaterials, designed to serve as "lecturenotes." along with multimedia demonstra-tion liles and links to additional learningresources on the Web.One or more sections for planned learningactivities. These sections were used topresent the learning activities for themodule. Learning activities included avariety of think-out-loud exercises, ongo-ing threaded discussions, small groupprojects, or individual assignments. Forexample, students might be asked to useassigned readings to post answers to spe-

«rPTsnfc 4K-'-lse ' AllDoEvmenti

'tf» »

;oi , '•.: IPT'>;;f,4'i:i.irii -AM Dru.nnBnr "

11*1536 •»W/4M

_JlmiiuOa'iV«_J My FavwM!

t _J BSU Catipus

Main TspK nei(>a»e - flsipanigt Priwaie Discussion Area tut Gmii

Kennedy

WfDk I Read Me FirstWeRk I Tectinnlnqy-HiHd 1 rinll

• 5. Private discussion area

• 1. Wockiy 'Road Me Fir^f area

•2. Weekly mstruciionai material nioa

* I 'i Systemic d Synlemntif Apf)lif:oti(in^»i^3, I/Vcckty learning activity aroarntry Kniiwledigu AssessmHMlWook t. Wiap Up 4 * * r •*• *'^*^^'' " " P - " P afoa

FtCURK 1The Bask Weekly Structure of ihe hitrtxluctory IPT Course Room.

322 The Quarterly Review of Disiance Hducaiioii Vol. 4. No. 3. 2<K)3

TABLE 3

A Summarv of the Inslrudional Modules

Module sections Irislnitiioiial liveiils Types of InsimvlUmal Aiih'iiies

I. Wcokly "Read Me First" area

2.Weeklv instiiiciional material area 4,Present new content

3.Weekly learning aetivity area

4.Wcckfy wrap-up area

1. Gain attention Weekly annimncemcms2. Intorni !e;irners olohjeetive Inslriiciional topics and objeciives3. Stimulate recall of prior knowledge Ferlbrmanee criteria

Advanced orgutii/ersLecture nolesMultimedia denionstratiotisLinks to additional leaining resources onthe WebThink-oui-loud aetivitiesThreaded, collaboraiive discussionsIndividual assignments

5.Provide learning guidance6.Blicil perlbrmanee

9.Hnhance retention and transfer

.Private discussion area for each sliident7.Provide feedback

A sumimu^y of weekly aetivitiesPosI organizersWeekly pcrfonnanee evaluations

cific questions. Or they might be asked tocommenl on a short posted sccnurio atidthen comment on the ideas others post asthe discussion progresses.

4. A wrap-up section. This section was usedto present a sutnmary of [he key pointsfrom the tiiodulc.

5. A private discussion section for each stu-dent, aceessihie only to that student andthe instructor. These private areas wereused each week to present students withindividual performance feedback. Thisfeedback was designed to help studentstiionitor their t)wn learning progress andto eoaeh them toward higher-level, morecritical thinking about the content of thecourse.

Table }< summarizes these five major sec-tions and the events o\ instruction that are rep-resented within each section. While Gagne's"'nine events ol" instruction" and Bloom's tax-onomy were combined to guide the design ofthe mcxluies. our foeus in this article is onBloom's taxonomy. Readers who would likemore infoniialion about the "nine events ofitistruction" are referred to Gagnc (1985) andGagne and Medsker (1996).

By combining this calendar approach withGagne's nine events of instruction, new stu-dents received a consistent instructional struc-

ture each week, which helped them developsell-regulative online behaviors. See Figure 1.

USING BLOOM'S TAXONOMY TODEVELOP CONGRUENTINSTRUCTION

Onee we had established the overall structureof the online environment and identified theweekly modules, our next steps were todevelop objectives, learning activities andmaterials, and evaluation criteria for eachmodule. This is the point at which the c(ingru-ence principle becomes particularly important.Maintaining the congruence among the objec-tives, learning activities, and evaluation crite-ria is critical to the effectiveness of theinstruction. Congruent instruction means thatlearning activities are designed to support Ihcobjectives and that the evaluation methods aredesigned to assess important learning out-comes represented by the objectives. Bloom'staxt)nomy of educational objectives wasinstrumental in making sure there was congru-ence among the components of each module.In this case study, the Week 4 tnodule will beused to illustrate how we designed our onlineinstruction to maintain the congruence amonginstructional eomponents (see Figure 2).

Applying the Vongrufiur'Trinciple of Bloom' s Taxonomy to Desif-iiinf- Online hisinuiiim 323

_ J ByAulho.Ji B Dale_ ] Iniliucloi'sVien

I Mv Favomes» _ l BSU CawKis

Pnvote Discussion Area lor GeorgeKennedy

Week 4 Read Me First ^

Week 1: Behavior Engineering Model

' 4-1 Worthy Pcrtormancc - ThorLConomir:Vniue

' 4-? BEM as u Diaqnustic Tinil

• A~3 The DiMusion ol Effect

' Weekly ovaluatien •

• Woohiy instructionalobjectives > - Congment

yWeekly learningactivities

FrGURE 2Making the Congruence Among Instructional Components.

Developing Objectives

We used Bioom's original taxotiomy lodetermine the levels of the objeetives for eachmodule and to design learning aetivitiesthrough which students would accomplishthose ohjectives. [Note: The introductory [PTcourse was originally developed by one of theauthors of this article in 1997 using Bloom'soriginal taxonomy, and has been revised everysemester. After the revised Bloom's taxonomywas published in 2001. the authors of this arti-cle adopted some of the new concepts of therevised taxonomy in their course design; how-ever, they continued to use Bloom's originaltaxonomy as the framework of their coursedesign because they did not fmd compellingreasons to modify their course using thetwo-dimensional taxonomy.] Depending on

the weekly instruetional topic, the levels of theobjectives ranged from remembering facts andunderstanding new concepts to synthesizingand evaluating information. For example, thetopic of the Week 4 module was Thomas Gil-bert's Behavior Engineering Model and weidentified four instructional objectives for thatmodule. Prior to developing learning aetivities.we determined the levels in the taxonomy foreach ohjective. Because we acknowledge thatlearning sequence and processes are interde-pendent, we listed the highest level from thetaxonomy, in conjunction with lower, support-ing levels. The objectives and levels for thismodule are summari/.ed in Table 4.

After we developed the objectives and cate-gorized them using Bloom's taxonomy, wecompared those objeetives against the entryknowledge levels of our students. At the begin-

TABLK 4

Week 4 Objectives and LevelsObjectives Levels in Bloom's li

1. To explain "worthy pcriormance" ac-corcJing lo Gilbcn'sBehavior Engineering Model

2. To iilili/,c (he liehavior engineering model as a diagnosticloo! 10 analyze ihe sources tor improving hum;uieompeienee

3. To apply the coneepl of "ihe greatest leverage" lo behaviorengineering silualions

4. To predici "JilTusion of etfeet" in various behaviorengineering situations;

Knowledge and Comprehension

Application and Analysis

Appiiealion and Analysis

Analysis, Synthesis, and t^valuation

334 The Quarterly Review of Dislanw: hducaiion Vol. 4. No, 3. 2(X)3

ning ol" the scmosler, we adminislcrcd anen try-knowledge assessment to the students inorder to assess their existing knowledge onspecific instructional topics. This was donewith a seit-assessment questionnaire, made upol' 40 items (hat asked students to rale howfamiliar they were with information that wouldbe covered in the course. A 3-point scale wasused:

1. 1 have never heard of it.2. I have heard of it, hut I can't explain to

other people exactly what it is.3. I'm Janiiliai" with it and I can explain to

other people exactly what it is.

Scores were summarized hy the topic. Thisinformation about students* existing knowl-edge helped us jusiify the levels of thepre-determined weekly instructional objec-tives or modify them if necessai^. For exam-ple, data obtained from the entry knowledgeassessment showed that students' entry knowl-edge levels on the three items related to theWeek 4 module (#5 about Thomas Gilhen. #6ahout the Behavior Engineering Model, and #7about the concept of worthy performance)were extremely low (see Figure 3). Since mostof the students had not even heard of the infor-mation (81%, 657c. 88% respectively), weconcluded that the prescribed objectives for

the Week 4 module were appropriate for thetargeted learners and decided to use the objec-tives without modification.

Developing Learning Activities

We posted instructional materials for eachmodule on Friday and students participated inthe learning activities anytime during the fol-lowing week. The learning activity areas foreach module provided various activities thatwere designed to facilitate learning and helpstudents achieve the instructional objectivesfor (hat module. For example, we includedthree learning activities for (he Week 4 moduleon Gilbert's Behavior Engineering Model:activity 4-1 for objective #1, activity 4-2 forobjectives #2 and #3, and activity 4-3 forobjectives #3 and #4 (see Figure 4 ).

A Learning Activity for the Knowledgeand Comprehension Levels

The first learning activity during week 4(4-1: Worthy Performance—The EconomicValue) was designed to help students under-stand Gilbert's use of the term, "worthy perfor-mance." Students were asked. "What doesGilbert mean by producing 'worthy perfor-mance'?" The following messages illustrate

t%of sludents) 1tH>%

Q5: Thomas Gilbert 06:BEM U7: Worthy Perf,

b # 1 Never heard of il

CJ #2 Heard ot il

» #3 Cai exi^an i(

35%

FlGtJRK 3Results olEhc Enlry Knowledge Assessment on Questions 5, 6. and 7.

Applying the 'Conf-nwiite "Principle of Bloom' ,v Ttixonumy to Online Insirinlinn

IT

J

^ By Author- I B y t l - .

V _1 BSU C

Week 4learningacttvrtios

(5)N».H«.Iwie (Jl^j

©

Main I npiE - naupDnia - Ftiitpatiis

Kennedy

Week. 4 Rund Me FirslWeek 1 Ilithnviiir Enginnnnnc) Mndel

» ^ - l " Worthy''"fl^^'^Q'icH Iho ^rKrsinomirVnIuH

> 4-2 BEM n i n OioqnDslii: Tiiiil -4- '

WeekiWrnp up

knowledgo & comprehension

iipplicatior) & analysisllor tho obicctivos 12 and >3|

-—b. analysis, syitlhcsts & evaluationd, (lorlhc otJicclivo M and Mf

FiGUKh 4Learning Acliviiies Used in ihe Wcuk 4 Module.

how studenLs iJemonsiratcd their knowledge

and comprchensitin. (Please niHe that the

names of the students have been changed in all

of the messages that follow. In addition, the

messages arc written as they were posted hy

the students, with errors in spelling and syntax

left intact.)

Lisa: Worthy performance is the cost effec-tive function of accompMshnicni over be-havior, where behavior is the combinationof environment and a person's behaviorreperlory. This is based on :i managementor business concept and stated as his 3rdLeisurely Theorem or Management.Translation: Worthy perfitrmance is themost efficient use of the individual inter-acting with their environment and its re-sources lo achieve a desired yoai. Gilbertextends this to say that wt)rthy perftir-mance is also the ultimate responsibilityof management systems, and the absenceof worthy performance and accomplish-ment is a failure of the management sys-tem.

Lisa's message can be divided into two sec-tions, corresponding to the knowledge andcotiiprehension levels of Bloom's taxonomy.Lisa first recalls what "worthy performance" is.using terms that arc taken largely from Gilbert(the knowledge level). She then elaborates on

that dctlnition and translates it into terms thai

are more her own {the comprehension level).

Eugene: Any accomplishment is the result ofsome effort expended by an individual orteam who have some level of competencein the sequence of events either from priorexperience, training, or even leamini!while completing the task. Additionally,the activities leading up to the accom-plishment has to take plaee in some lo-cale, presumable well-suited lor theendeavor. These two things combinedyield the final product. Gilbert recognizesthis fact and makes them the basis of"wiirthy performance" . . . . For example,trying to have ."i year olds packing Hourinto bags outside during a high wind isclearly not a worthy performance. Provid-ing a stable interior environment for awell-trained, mature staff to pack thosebags using automated equipment is farmore worthy. It is this disparity that Gil-bert is describing and providing insightson improving.

Eugene's message also shows comprehen-sion, but in a slightly different way. Like Lisa.Eugene begins with an explanation of whatworthy performance is. pointing out that twothings "yield the final product." He then adds asimple example that highlights importantaspect.s of the concept. For boih students, ihc

The Quarterly Kevicw of Oisiancc Hducalion Voi, 4, No. 3. 2(H)3

learning activity cliciled the knowledge andcomprehension thai was the inlent of theobjective.

Although this first learning activity wasdesigned to facilitate students" knowledge andcomprehension, we have observed examplesof "slep-up learning" in ihe students'responses, in which students demonstrated theintended level ol' learning (for example, com-prehension) and then went beyond that to dem-onstrate an unanticipated higher level oflearning {for example, application, analysis,synthesis, or evaluation). The following mes-sages illustrate this step-up learning.

Ken: 1 have held training sessions forihemiogo, but thi;y don't attend. Thai is one ofthe other reasons I gel Trustraled. So Iguess I need lo use Gilbert's BEM andfind (he root cause of their absence at mytraining .sessions. Maybe these peoplewho don't attend need some son of incen-tive to attend - like a stipend or some typeof "freebie". I have to look into this a littledeeper.

Sandra: I think ihe BEM model would be auseful iool in (hat one can visualize andweigh what will happen if you change oneor more ol the parameters.

Both students go beyond eomprebension. Inbis message. Ken is beginning lo apply Gil-bert's "Behavioral Engineering Model"(BEM) lo a particular situation. Sandra's mes-sage redeets her evaluation of the usefulnessolGilherl's model. This phenomenon reflcclsihe kind of "sell-directed learning" that isfacihlatcd by ibe self-directed, relleclivenature of ihc asynchronous online learningcnvironnicnl. Because there are fewer limeconstraints than in (raditional. faec-to-faceinstruction, students can think about the mes-sages thai have previously been posted, alongwith whal they wanl to add. before postingtheir message. This encourages students loreflect on ihcir growing knowledge as a part ofthe discussion.

A Learning Activity for the Applicationand Analysis Levels

The seeond learning aelivity during Week 4{4-2: BEM as a Diagnostic Tool) asked stu-dents to:

Look around al your workplaec andohsent:' a typical pcrlormer and an exem-plary performer in ierms of iiccomplishingaspecitle Usk. Howdifftircnt ;irc ihey.' If i(is desirable, inierview wiih your typieal andexemplary performers lo see their own per-cepiions aboui ihe 6 issues. Report wha(yoL Ibund and brainsiorm/discuss wha(needs lo be done in order lo help ihe typiealperformer reduee his/her PIP (Polenlial forImproving Perlornuuice).

To formulate a response, students would haveto apply Gilbert's mode! to a particular situa-tion and analyze the data they obtained fromtheir observations and interviews. The follow-ing messages illustrate this application andanalysis ot Gilbert's model.

(lary: The situation I have observed includestwo individuals in a sales/account man-agement setting I was unable to inter-view the Exemplary Performer. bu( haveinteracted enough wiih ihe person to un-derstand their curren( performance. Mydiscussions were primarily with the Typi-cal Performer.. . . In a brief synopsis, itappears that the Typical Performer is notperforming in an exemplary manner be-cause he does not I'eel thai a support andfeedback system is in place. Greater iacil-italion by management of a communica-tion and feedback system may help loremedy this situation.

,|uliet:... I think the lack of incentive shownfor both performers by ihe diagnostic ta-ble is lypieal '.'>\' military medicine. Thishas been discussed frequently among.ststaff. There is no reward for performingadequately let alone well. The only incen-tive seems to be internal professional sat-islaetion . . . . The knowledge differencebetween performers points to training as asolution. A training program thai provid-ed instruction and opportunities to siari

Applying ihe 'Congruence "Principle of Bloom' s Tiixonomy lo Oesi_ii>ii>ii; Oniine Insiniciion 327

IV's on the very young and old wouldlikely elose this perfortTiance gap. How-ever, my impression is that if the gap werecalculated, it would not be worth the de-velopment costs of the training program.

Gary and Juliet have eonsidered differentsituations. But they have followed a similarprocess of applying Gilbert's model and sepa-rating the results into components of Ihemodel. In Gary's situation, the major problemappears lo be the amount of feedback ihcemployees reeeive. Juliet describes incentivesand knowledge as the primary problem areaswithin her situation.

When we used learning activities designedto help students aequire analysis skills, weoften asked them to provide feedback to oneanother. This method helped students considerdifferent views of the same situation. The fol-lowing example of a threaded discussionshows how this kind of inter-student feedbackpromoted collaborative analysis.

Susan: Here is an example from my office . .. The exemplary performer finds contactinformation on the web and contacts thesubmitter for additional information. Ex-emplary performers use feedback to en-hance their own performance. The typicalperformer places missing information or-ders at ihe back of the pile and only dealswith them when they absolutely have todo so. Exemplary performers receive payincreases to reward them for their work.Exemplary performers seek ways to speedup their own processing and prevent in-complete orders from even reaching them.Typical performers simply process or-ders. Exemplary performers process or-ders faster ihan the expected rate. Typiealperfonners process orders at or below Iheexpected rate.

Dan: vSusan. Did you do interviews? \i seemsthat motivaiion may be an issue here. Youdidn't indicate if incentives were avail-able. The task seems simple enough andassuming they have the tools, motivationand incentives could be the problem.What do you think?"

Kristi: I agree Dan that incentive and motivecould be the problem. Also, the data fac-tor indicates that a typical performer putsincomplete orders in the back ot the pile.This behavior must be eliminated and re-placed with a more aggressive approach.Perhaps an order rejection process that re-sult.s in properly submitted orders. Thiswotild force ihe number of iniprojierlysubmitted orders to decrease.

Dan: Kristi. an intervention is needed to stopthe workers from putting incomplete or-ders in the back of the pile. Your processof "order rejection" would work and thesystem (instrument) could force that tohappen. Just don't accept it unlit it's right.The incentive for having the most com-plete orders monthly sounds good. Butcould other factors cause the same personto always win this contest? Such as typingspeed. You would need to evaluate Iheoutcome.

Susan: Kristi, this is a great idea. As Dan hasstated, some intervention is required. Ilike the incentive idea and agree Ihat wewould need to evaluate who wins eachmonth so we can try lo learn from iheirmotivation and their process and imple-ment everywhere. Remember, we wantworthy performance!"

In this excerpl. the students are discussinghow to interpret ihe results Susan obtained inher example. In her initial analysis, she com-pares different behaviors exhibited by theexemplary performer and the typieal per-former. Dan and Kristi analyze the situationand propose a possible cause of the differentbehaviors, suggesting ihat incentives may bean important issue. Susan agrees and begins loconsider ihe practical implications of ihis rec-ommendation.

A Learning Activity for the Analysis,Synthesis, and Evaluation Level

The third learning activity during Week 4(4-3: The Diffusion of Effect) was designed tohelp students synthesize information about

The Quarterly Review oI'Dislanee Hducation Vol. 4. No. 3. 2003

Gilbert's Behavior Engineering Model and

assess the value of Gilbert's model. Students

were asked to respond to the fallowing ques-

tion: "What does Gilbert mean by "the dilTu-

sion of effect'?' Is it important lor pt:rlbrniancc

technologists to be aware of the diffusion of

effect? If so, why? If not. why not? How do

performance technologists apply ihis principle

to their practice?" The following messages

illustrate student synthesis and evaluation.

Colleen: It's very imporlanl for HP Technol-ogists to he aware ol" Ihc diffusion of ef-fect. Throughout the article, Gilbertexplains that the behavior engineeringmodel doesn't apply scientific methods,bui engineering methods, which meansevery HP Technologist w'\\\ have a differ-ent style when using Ihe model. Gilbertlater states, "the behavior engineeringmodel cannot tell whether or not a moti-vational effect is present, but lead him toask certain qiie.stion,s." These questionsare important in getting to the true perfor-mance opportunity and also detenniningthe most effective and co.st efficient solu-tion.

David: As far as PTs applying this principleto their practice. I think it is important tobe aware t)f the phenomena. However, 1question its usefulness, particularly if thediffusion results in positive change in thePIP. It seems to me that Gilbert feci.s thatas long as he is getting results, he doesn'treally eare how it is happening. Anythoughts?

The evaluation level involves making Judg-ments about the value or u.scfulncss of ideas,solutions, inctbods. etc. The centerpiece ofboth of these messages is an evaluation of Gil-bert's model, though the students come Lo dif-ferent conclusions about the model. Colleennotes that ibc questions tbc model asks are crit-ical to a "true" understanding of a particularsituation. In contrast, David questions thevalue of tbc model because it doesn't, in hisview, adequately address issues of process.

Evaluating Weekly Online LearningOutcomes Based on Bloom's Taxonomy

Bloom's taxonomy also guided us in settingevaluation criteria that would be congruentwith the obicctives for each module (see Table5).

At the end of each weekly module, we eval-uated individual students' pcrlbrtnancc andprovided feedback to tbcm using their privatediscussion area. Two types of feedback wereprovided to tbc learners: the number of pointsthai they earned and qualitative, constructivefeedback about their pcribnnancc during theweek. Below are examples of this qualitativefeedback that we provided to students:

Instructor: Your BEM example |Note: adocument link to the student's originalmessage is inserted in the feedback state-ment so that the student will review herown message and reflect on her perfor-mance as comparing it to the instructor'sfeedback.] about increasing referralsshows that you cleariy undersiand the dif-ference between behavior and accom-plishment. You did a nice job identifyingseveral solutions and indicating whichone would have the greatest impact, at thelowest cost.

In this message, tbc instructor provides con-firmation to tbc student thai she has acquiredthe basic knowledge and comprehension levelsof the objective. In addition, the instructorcompliments the student for the analyticthought she shows in her response. This servesto reinforce the student for her analysis andincrease the confidence she feels in her abilityto master the objectives.

Instructor: Dan, you've demonstrated inyour messages that you have a goixJ un-derstanding aboul Gilbert's ideas [a link tohis document is inserted here]. You alsodid a good job analyzing your work situa-tion using the HEM [a link to his docu-ment is insertetl)—the daia you collectedis prelly interesting, ll's a great start inyour needs assessment and PI processes. I

Applying ihe 'Omgruence "Principle of Bloom' s Tcvconomy lo Deslf;ttii\i' Online Instrm-tian 329

TABLH 5

Week 4 Ohjeetives, Learning Activity, and Evaluiilion Criteria

Instriiclional OhjertivesLearning Aciivity & the Inieiided

Level ol

Evitluation iij Studenls' Performance onAcliieviiif; ihe Hiiihes! Level ofLeuming

in the Learning Aciivity

1. To explain "worthy performance"according lo Gilbert's HchuviorHngineering Model

2. To utilize ihe behavior engineeringmode! as a diagnostic tool to analy/cIhc sources tor improving humancompeicnce

3. To apply the concepi of "the greatestleverage" lo behavior engineeringsituations

4. Ti) predict "diffusion of effect" invarious behavior engineeringsituations

Learning aciivity # 4-1Comprehension

Leiiming activity #4-2Analysis

Learning activity #4-3B valuation

Accurate description of the meaning of"wonhy performance" in the contextof performance improvement

Concrtite and coherent application of thesix elements in the behaviorengineering model lo analyze realperformance impmvement situations

Logieat analysis of selecting cosi-effeetive solutions lo the performanceproblems

Analysis and synthesis of various factorsthai contribute lo systemic effects of asolution

Jusiifieation of their selection as thecost-effective solution

Judgment on the usefulness ofbehavioral engineering model

can see that a successful intervenlion onthis stop will have a domino effect {tippleeffect) on other end results since ihe workHow is such that it has lo be done correcl-ly from the beginning to the end.

In the above message, the instructor pointsout to the student that he has demonsiratcdbasic knowledge and analytic skills about thetopic, [n addition, the latter part of the tnessageis intended to reinforce the student for his syn-thesis oithe data and to help hitn predict possi-ble effects of his suggested intervention.

With weekly feedback from the instructor.students were able lo self-monitor their learn-ing progress and build close rapport with theinstructor.

CONCLUSION

Designing instruction can be a challenge toinstructional designers for a number of rea-sons. The recent proliferation of the WtsrldWide Web as an instructional delivery inediutTihas brought new challenges to the task ofdesigning instruction. However, in the designol Internet-based online instruction, thectueial

instructional design principle still applies;instructional components such as instructionalobjectives, instructional activities, and assess-ment methods should be carefully matched tohelp students achieve the intended learningouteomes. We have found that Bloom's taxon-omy can be a helpful guide in achieving thiscongruenee in online instruetion.

REFERENCES

Ainsworlh. P. (1994). Restructuring the introduc-tory aceounling courses: The Kansas Slate Uni-versity experience, .foumul of AceounlingEducation. /2(4). 305-32.1

Anderson. L.. & Krathwohl. D. (Eds.). (2(K)I). Alaxonoiny for learning, teaching, and as.'ifs.sing:A revision of liloom 's taxonomy of cdttcalionalohjvciives. New York: Longman.

Berciter. C . & Scardamalia. M. (1998). BeyondBloom's taxonomy: Rethinking knowledge forthe knowledge age. In A. Hargreaves. A, Lieber-man. M. Fullcn, & D. Hopkins (Eds.). Inierno-lioiml hamilwok of educational change. Boston:Kluwer Acadetiiie.

Bloom. B.. Engelhart. M.. Furst, E., Hill. W . &Kralhwohl. D. (19,56). Taxonomy of editcaiionalobjeelives: The classification of educational

TheQuancrly Kcvicw ofDislancc Education Vol. 4, N»i, 3. 2003

goals tHandbook I: Cognitive domain). NewYork; McKay.

Calhoun. J.. Davidson. P., .Sinioris. M.. Vincenl. E..& GrilTiih, J. (2002). Toward an understandingof competency identificalion and assessment inhcallh care managemenl. Qualiry Mamigemeniin Heallh Can: //(D. 14-38.

Cheeihan. G.. & Chivcrs. G, (2001), How protes-sionals learn: An invesiigaiion ofinformal learn-ing amongst people working in professions.Journal of European Indcislrial Training. 25{5).248-292.

Cunningham, D. J, (1992). Beyond educational psy-chology: Steps toward an educational semiotic.Educalional Psychology Review. 4{1). [65-194.

Dick. W., Carey. L.. & Carey. J. (). (2001). The sys-lemaiic design of inslniclion (5th ed.). NewYork: Longman.

Gagne. R. (1985). The conditions of learning (4thEd.). New York: Holt. Rinehart and Winston.

Gagne- R.. & Medsker. K. (1996). The conditions of(earning: Training applications. Forl Worth.TX: Harcourt Brace College Publishers.

Harrow. A. (1972). A taxonomy of the psxchomoiordomain: A guide for developing behavioralobjectives. New York: McKay.

Haythomthwaile, C. Kazmer. M. M.. Robins, J.. &Shoemaker, S. {2000). Community developmentamong distance learners: Temp(ual and techno-logical dimensions. Journal of Computer-Medi-ated Communication. 6( I). Retrieved October 2,2(X)0, from the World Wide Web: http://www,ascusc,org/icmc/vol6/issuel/haythomth-waiie.html

Jonassen. D. H. (1991). Objectivism versus con-structivism: Do we need a new philosophicalparadigm? Educational Technology Researchand Development. 39{}). 5-14.

Jonassen. D. H.. Peck. K. L.. & Wilson. B. G.(1999). Learning with teclmologv: A construc-tivist perspective. Upper Saddle River. NJ: Mer-rill.

Krathwohl. D.. Bloom, B.. & Masia. B. (1964). Tax-onomy of educational objectives: The cla.ssifica-lion of educational goals (Handbook If:Ajjective domain). New York: David McKay.

Moore. D. (1982). Reconsidering Bloom's taxon-omy of educational objectives, cognitivedomain. Educational Theory, 32( 1). 29-34.

Morrison. G., Ross. S.. & Kemp. J. (2001). Design-ing effective instruction (3rd Ed.). New York:Wiley.

Nichoff. B.. & Whitney-Bammerlin. D. (1995).Don't let your training process derail your jour-ney to total qualiiy management. S.A.MAdvanced Management Journal. 6()( I). 39-45.

Reigeluth. C. M. (1983). Instructional design: Whatis it and why is it? In C. M. Reigeluth (Ed.).Instructional-design theories and models: Anoverview of their current status. Hiilsdale. NJ:Erlbauni-

Simpson. E. (1972). The classification of educa-tional objectives in the psychomotor domain:The psychomotor domain (Vol. 3). Washington,DC: Gryphon House.

Smith. P. L.. & Ragan. T. J. (1999), Instructionaldesign (2nd cd.). Upper Saddle River, NJ: Mer-rill.

Sugrue, B. (2002). Problems with Bloom's taxon-omy. Retrieved December 14. 2002. from theWorld Wide Web: http://www,perfonTiancex-press.org/()212/

Tennyson. R. D, (1995). The isnpact of the cognitivescience movement on instmctional design fun-damentals. In B. B, Seels (Ed.I. Instructionaldesign jundametitals: A reconsideration. Engle-wood Cliffs. NJ: Educational Technology Publi-calit)ns.

Wilson, B. G. (1997), Rejlections on constructivismand instructional design. Retrieved March 4,2003, from the World Wide Web: http://car-bon.cudenver.edu/~bwiison/construct.html

Young, M. (1993), Instructional design for situatedlearning. Educational TechnoU>gy Research andDevelopment. 410). 43-58,