critical thinking DELPHI METHOD

8/7/2019 critical thinking DELPHI METHOD

1/111

' \. ,, -. v ;, :: - ~ '; '; ,' ' - ,' " ._, J. - . '

D OC UM EN T R ESU MEED 315 423 TM 014 423

SPONS AGENCYPUB DATENOTEPUB TYPE

Facione, ~eter A.Critical Thinking: A Statement of Expert Consensusfor Purposes of Educational Assessment andInstru~tion. Research Findings andRecommendations.bmtlr1can Philosophical Association, Newark, Del.90l12p., Some pages have broken type.Reports - Research/Technical (143)

AUTHORTITLE

IDENTIFIERS

MF01/PC05 Plus Postage.Cognitive Ability, *Critical Thinking, ~CurriculumDevelopment; Cu. rr cu ru n E va ll 1a ti on ~ ~ De lp hiTechnique; ~Educational Assessment; QualitativeResearch~Experts; *Panel Consensus Technique

EDRS PRICEDESCRIP'l'ORS

ABSTRACT Using a qualitative research methodology, kn~wn asthe Delphi MeUlod, an interactj,ve panel of experts was convened towork toward a consensus on the role of critical thinking (CT) ineducational asse~sment and instrUction. In Delphi research, expertsparticipate in several rounds of ~uestions that require thoughtfuland detailed responses. Panelists work toward consensus by sharingreasoned opinions and reconsidering the opinion~ with regard tocomments, objections, and arguments offered by other experts. A totalof 46 scholare, educators, and leading figures in CT theory and CTassessment res~arch were gathered for the panel meetings. About halfof the ranelists were primarily affiliated with philosophydepartments: the others were affiliated with education, socialsciences, or physical sciences. Recommendations resulting from thediscussion rounds address the cognitivp skill dimension of CT, thediSpositional dimension of CT, and specific recommendations on CTinstruction and assessment, including development of a CT curriculum.A discussion of commercially available CT assessment tools, abibliography With an emphasis on assessment, and a set of letterswhich chronicle the progcess of the Delphi research group areappended. (TJH)

*~~~~~~*~~*~~~~~~~~~~*~~~~~~~~~*~***~*~*~**~~~*~~~*~~~~**~~*~*~~~~~***** Reproductions supplied by EDRS are the best that can be madefrom thp. originaJ document.*~~~~~~~~~~~~~~~~*~~*~~~~**~~*~*~~~~~*~~*~~~**~~~*~~~~***~*~**~~~~***~*


2/111

, - '" 1 \

u.s. D EP AR T rt IIH T O F E D UC AT IO NOffice of Educa',onal Aaseareh and Improvemen,eDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)rTills document l1li8 bean raptoouced asrecolved fr om 'lie person or or llan'za"ol '0"11,"8,,"g "rMono r changes neve been made '0 Imp rovoreproductIon Qualo)y

" PERMIS SION TO REPRODUCE TH ISM ATE RIA L H AS B EE N G RA NTE D B Y~1(:.~ 8 . ;' /CIOtv[

FOlnts of v Iew o roponlonsstated In t ll l$ cocu-men t 110nO ' n fl ceSSI l" ly rep resent o floc la lOERI pOsIt ion or pol iCyT O T HE E DUCAT IO NA L R ESOURCESINFORMATION CENTER (ERIC),"

FOR PURPOSES OF

CRITICAL TIIINKING:

A STATEMENT OF EXPERT CONSENSUS

EDUCATIONAL ASSESSMENT AND INSTRUCfION

R e sear ch F in din gs a nd R e com m en da tion sP r ep ar ed for theCommittee on Pre-College Philosophyof theA m e r ica n P h ilo so ph ica l A s so cia tio n

byP eter A . F acione,Ca l ifo r nia S ta te Un i ve rsit y, Fullerton

(c) 1990 P. A. Pac-ione

~,

B E S T C O p y A V A i L A b L t .


3/111

' . y 1> . . . J .... '

. Q _ O N T E N T SI The Critical Thinkillg Movement and CT Assessmer",t 1II Research Methodology and Purpose 4III The Cognitive Skill Dimension of Critical Thinking 8IV The Dispositional Dimension of Critical Thinking 20Procedural, Laudatory and Normative Uses the Term "CT" ..21Dispositions of the Good Critical Thinker 27V Further Recommendations on CT Instruction and Assessment ..27The CT Goal , , , . I , , 28The CT Curriculum. e 0 e. e 00 eo 30The CT Assessment Tool 0 0 30

'fhe CT Instructor II 32VI The Delphi Research Panel .. 0 e 33

TABLESTABLE 1 Consensus Statement Regarding Critical Thinkingand the Ideal Critical Thinker 0 0 e 3TABLE 2 Project History 7TABLE 3 Consensus List of CT Cognitive Skills and Sub-Skills 121ABLE 4 Consensus Descriptions of Core CT Skills and S~b-Skills.13TABLE 5 Affective Dispositions of Critical Thinking 2 6TABLE 6 Consensus Statement on Teaching and Assessing CT Skills.29TABLE 7 Participating Critical Thinking Experts 35

APPENDICESAppendix A: Commercial CT Assessment Tools 00 36Appendix B: CT Bibliography with Emphasis on Assessment 40Appendix C: The Delphi Research Letters 51


4/111

: ~ : t: /~ ? ! ~ f' (: r ', . V t_--' . " _ : .

CR IT ICAL TH INK ING : A STATEMENT OF EX !"FRT CONSENSUSFOR PURPOSES OF EDUCATIONAL ASSESSMENT . IU~D INSTRUCTION.1 ~ Cr iti cal 7hin kin g H gyem ent an d ~ As~ essm entFrom NewJersey to Califernia, and from Newfoundland to Florida,

leaders in the critical thin~dng movement have advocated majoreducational reform. They have argued that effective and meaningfuleducation requires that curricular, pedagogical and asses$ment strategiesat all levels of edl.tcation be coordinated 50 as to foster in studentsthose cognitive skills and habits of inquiry associated with criticalthinking. They have made the case that educatinQ students to be criticalthinkers is vital for the student~ themselves and for society in genera~(Ennis, 1962, 1981, 1980; Pa.ssmore, 1907; Schievella, 19~B; Shef'fler,1973; Lipman, 1977; Siegel, 1980, 1988; Gardner, 1993; Arons, 1983;Beyer, 1985; Costa, 1985; Quellmalz, 1983, 1995; Scriven, 1985;StE:!rllberg, 1985; Ruggiero, 19a5; Paul, 1988 (a) and (b); etc.).

The arguments for critical thinking have been successful.After decades of rela ti ve neglect, the eighties witnessed a growing

accor-d that the heart of education lies exactly where traditionaladvocates of a liber&l education always said it was -- in the processes

disjointed skills and senescent information. The c:ritical thinkingof inquiry, learning and thinking rather than in the accumulation of

movement gained momentumthroughout the decade. Conferences and positionpapers led to the development of college level critical thinking (eT)courses. In elementary and secondary schools (K-12) teachers revisedlesson plans to incorporate CT objectives. In the span of a few yearspublishing CT textbooks and offering CT staff development programs becamegrowth industries. The CTmovement enjoyed major success when

4


5/111

._.,;-~,,::::._r. __:~; ~.::~.~-'~': _ '_ ~ 'v ., .: ", ,_ . . ~ .~~ ' . -

universities introduced CTrequirements into their general educationprogramsandstate departments of educcationtargeted CTin theircurricular frameworksand their standardized testing programs. By thedecade'sendCTcould no longer be chaloacterizedas a cottage industry.

Withsuccess comequestions: Not:newones necessarily, but, becauseof the e.:pectations whichhave beenri!lised and the investments beingproposed, vel(ingones. IntUitively, CTinstruction should focus on howstudents approacha. qLlestionand reason about it. CTpedagogyshoulddevelop in sbJdents those cognitive skills andaffective dispositionswhichcharacterize the goodcritical thinker. Rather than or in additionto targeting whether a given answer is correct, CTassessmentshouldtarget the qLlali t y of the critical thinking the students put intoarriving at that answer. Thuf:;,fer all OTtheir succeosses,CTexpertsfind they mL(stccmtinue to address somefundamentalacadem:lcconcen,s.Whntexactly are those skills and dispositions whic:hcharacterize CT'?Whatare someeffective ways to teach CT? Andhowcan CT,particularlyif it becomesa campus-wide,district-wide or statewide requirement, Ueassessed?

Whenthese academicquestions are askedby the individual professorar teacher ~eekingto introduce CTinto her ownclassroom, they aredifficult ':-f"lough.But t:hequestions take on social, fiscal, andpolitical dimensionswhenaskedby cempuacurriculum committees, sc:hooldistrict offices, boards of education, and the educational testing andpublishiwJ indLl~tr'es. This is not to say that the experts find theseql..lestiunsinslll'""lnoLlntc\ble.On the contrary, CTexperts have workedwiththeir" colleagues in the education communityon someremarkableprojects.For- e;.:ample,California a.ndNewJersey have established ....ys ofintradLU:ingCTinto their curr-icular frameworksand statewide testing

""'v "_"


6/111

' - > ..!~:.:..~;~(~.:.~: ...r : ~ ' : " ;- " \ ~ ; - . . . .. . -: - . .~ , . - :: -- :: '< - ,. . . . .~..

programs. The twenty-clampus California State University system, whichenrolls hundreds of thousands of students, has established a process ferthe approval ef CTCOLtrSesfor its general education requirement.

Givan the central role played by philosophers in art.ic:ulating thevalue, both individual clnd social, of CT, in analyzing the concept of CT,in designing college lev6l1 academic programs in CT, and in assisting withefforts to introduce CT into the K-12 curriculum, it is little wonderthat the American Philosophical Association, through its Committee 01'Pre-College Philosophy, has ta'~en an interest in the CTmovement and itsimpact en the profession. In December of 1987 that committee asked thisinvestigator to make a systematic inquiry into the current state of CTand CT assessment.

TABLEtCONSBNSUS STATBMBNT RBGARDIN"G CRITICALTHINKING AND THB IDEAL CRITICAL THINKER

We undentand critic:al thinking to be purposeful, self-regulatory judgmentwhich results in interpretation, analysil, evaluation, aad inference, as weU asexplanation of the evidential, conceptual, methociologic:ai, criteriological, orc on teztu ai co nsid eratio ns UPODwhich th at ju dgmeD t is based. CT is essential as a toolof inquiry. As such , CT isa lib era tin g fo rc e in education a nd a p ow erfu l re so urc e hiODe' .personal and civic: life. While not synonymous witb good thinking, CT is a pervasiveand self-rectifying human phenomenon. The ideal critical thinker is habituaDyinquisitive, weU-informed,lrusUul of reason, open-minded, flexible, fair-minded inevaluatioD, honest infacing pe rsoD8 l b ia se s, prudent i . a making judgments, wiI1iDgtoreconsider, cleu about issues, orderly incomplex malten, diligent ill seeking relevantinformatioa, reasonable in the selection of criteria, focused in inquiry, and penistentin seeking resu'ta which are 81preciseuhe subject and the circumstances of inquirypermit . Thus, educating good critical thinken means working toward this ideal. Itc omb in es d ev elo pin g CT skills with nurturing those dispositions which consistentlyyield useful irudghts and which are the basis of a rational and democratic society.

As Table 1 st.lggests, a key result of inquiry is the artiCUlation bya panel of CT experts of a conceptualization of CT it terms of twodimensions: cognitive skills and affective dispositions. Section II of


7/111

this reper t describes the Delphi research methodology_ Section IIIaddress the skill dimension of CT, and Sect.ion IV the dispositionaldimension of CT. Research findings are presented throughout the report,both in the text and in tab'.llar form. Six recommendations are presentedin Sections III and IV so they can be reloted most sensibly with their- . . .rationale. Nine c.\ddltionc\l recommendations which pertain specifically toCT instruction and assessment care presented in Section V.

II -- Researr:JJ_ Methodology and PUr:I3ose

The Committee on Pre-College Philosophy suggested several personswith special e~~pertis"e in CT and CT whomthis investigator might contactas plJrt of the inquir'y into the controversial issues known to lie at theheart of the professl.on's concern. This investigater decided to employthe power"ful q~lalitative research methodology known as the Delphi Method.The Delphi Methed reql.lil'ss the formaticn of an interactive panel efexperts. Thes:;e persons must be willing to share their expertise and worktoward Q consensus r~solution of ma.tters of opinion. Using the fir~ ~group o f ~~perts to nominate othurs, the Delphi panel soon teck s h a p p .In all forty-six persons, widely recognized by their professionalcoUeagLles to have special e)~perienc:e and expertise in CT ~nstruction,,l\ssessment or theory, made tt.e commitment to partj~lpate in this Delphiproject. If it were not f(.)r their cOI,scientir:~s effort, (for' which thisil1ve'5.tiga.tor is e:.:tremeiy dPprec:ia.tivp~, the consensus exprp.ssed in thi.s"repor"t could :lot h~VE~been re;>~ched.

III Delphi research eHperts participate in several rounds ofq\.testions which call for~ thoughtful and detailed responses. Achieving a,:on'sensus of expert opinion using the Delphi Method is not a matter of

4"...,


8/111

voting cr tabulating quantitative data. Rather the expert panelists worktoward consensus by sharing their reasoned opinions and being willing toreconsider them in the light of the comments, objections and argument~offered by other experts. In Delphi research, once an expert ~xpressesan opinion, even a dissenting one, itbecomes a factor !.~I the mixand. . . .flow of all sL,bsequent argLlment and thought. TC'tcircumvent undueinfluence carising from any given experts p.-cfessional status, each roundQf questions is initiated by the p.r:.;ject director and all responses arecoordinated through that p~~'$on. The project director circulates to the. . .~.entire panel directq.:.;.,6tations and synthesized responses, with the name!:;of their aut.hpys removed.

Th~ expert panelists t.hemselves, through the thoughtfulness andp,t:rsuasiveness of their written responses, shape the line of inquiry.The project director endeavors to frame questions which respond to thedirection panel debate is taking and lead the conversation towardfruitful resolution. As the inqLliry proceeds, the project directorassists the panelists with bibliographies and alerts them to other usefulsources of relevant informaticn. As areas cf accord or disagreement.:.nerge these are presented to the panel in the form of drafts ofpreliminary findings or crucial follow-!..,pQuestions. The processtenninatr:?s when the project director c..~tGrcninesthat sufficient accordhas been reached for ar'eas of consens~ oJ to be made public. Delphifindings als~ include descriptions o.f residual disagreement andstatemp.nts of minority opinion.

A clear and ciccurate conceptualization of CT is absolutely essentialfor the development of valid CTassessment tools and effective CTill~trllctional programs. 'Ht.h this in mind, and recognizing thatdiv~rgent conctOlptualizations of CThave hindered curricular and

8


9/111

....... -- .. ,' ." . . . . . . . . . . .

assessmf~nt efforts, early in the Delphi process the panel decided itsmost worthwhile ~Qntribution could be the articulation of a clear andcorrect conceptualization of CT. The expert panelists devoted theirmajor effort toward that end. The eHperts hope~ that by comingto,:onsensus they could offer educators interested in CT assessment or. . . .instruction a conceptualization of Cl' of sufficient clarity, accuracy andrichness to warrant their serious attention.

To balance the theoretical with the practical, the experts askedthemselves what a generally educatad college lower division levelcri tical thinker should be able to do. However, they did not attempt todescribe the typical college level critic:al thinker. It soon becamef~"ident that the experts were actually articulating an ideal. It may bethat no person is -{:'-tllyadept at all the skills and sub-skills theexperts found to be central to CT. It may be that no person has fullycultivated all the affective dispositions which characterize a goodcritical thinker. Also humans compartmentalize their lives in ways thatCT is more active and evident in some areas than in others. This givesno more reason to abandon the effort to infuse CT into the educationalsystem than that knowingno friendship is perfect gives one reason tode'Spair of having fr~iends. 1'Me experts' purpose in putting the idealbefore the educatiol' community is that it should serve as a rich andworthy goal guiding CT assessment and curriculum development at alleducational levels.

69


10/111

.i.~ / : . - : : - ; r . ~ ~. " ~ . . . " . ' .~ - " ' ' ' ' . :.. .> ~ : . . ~ ~ ~~-~,,/t.;-,.;". ": : ~ . t . ' . .

_ ,_ , _ . ~ _ _ . . ; . . . . . o . . _ _ _ '_ _ " " " " " " " . "T "~~ ""___ _ ~

TABLB2PROJECT HISTORY

Round 1 (P"b. 11, 1988) and Round 2 (Mar. 14, 1988) initiated the Delpbiprocess. Inboth roundB panelists w ere inv ited to nom b18t~ other cre xp erts to jo inin this research p ro ject. T he experts re ac hed co nse nsu s o n tb e w o rk in g a ssumptio n th at" th e COD cep tof cr could bem ade operational to the e~eDt that important parts of CTcould be auessed &lidly and reliably." The exper ts agreed to b e ! i D their analysis01 CT by "identifying the core elem ents of cr which might reuonabl, be expectedat tb e freshm an an d so ph om o re g en eral ed ucatio n co lleg e lev el." T he ratio nale fo r thisd ec is io n wa s tb et th e co lleg e le vel t heo re tic al c :o ru stmctof CT c ou ld reasonably be usedto guide w bat m ight be sa id a bo ut cr at th e K -IZ lev el. Also th e p an elists D o ted th atmos t of the participatibl ezperts had greater ezpedence at the college level than in Ie-12 educat ion.Round 3 (MI . 'Y4 , 1988)wuan opea-ended inv ita tion for experts to w rite th eir ow nlist o f th e o pe ra tio ns wh ic h th ey c on ce iv ed ~ f, Jfs cen tra l to C T. T he first syntbesis of th i sinput w as presen ted for expert rev iew in R ound 4 (Sept. 23, 1988). This synthesisfocused on the skill d im ension of C T . R ound 4 inv ited responses regard ing each skUIand sub -sk ill identified, a p ropos ed [and u lt ima tely re jec ted ] in pu t/o utp ut m o de l o f CToperations, a list of closely rela ted cognitiv e operations w hich m ight or m ight not bedistinguished from cr, a general statement regarding what a skiU is and how one istaught , and ill list o f cav eats an d cau tio ns reg ard in g CT in st ru ct ion aud a ss es smen t.Rount l SA (Feb. 28, 1989) re viewe d th e d efin itio ns and c la ss if ic at ion o f CTc og nitiv e sk iD sin th e lig ht o f e xp ert re sp on se s to R o ua d4 . RD u udS8 (also P eb .28 , 1 98 9)

proposed sblltements r ega rd ing the d ispos it iona l dimension of C T and about its possiblenormative Cf)nnotations. RoundSC (Mar. 10, 1989) aslced forspeclfic recommenda t ionsregarding (:f instruction ana assessm ent, and offered a rev ision of the generalstatement (tn teaching and assessing 8 cogYli t ive skill. RdUDd S included severalquotatioDScuiled from the panelists' earlier responses and invited comments andreactions.T he f~Xperts'comm ent s r eg ar ding the " ar iou s quo ta tion s included in each roundadded greatly t o t he p ro je ct directors understanding o f th e experts' overaU views. Fromthese and the responses to specific Round SA , SB an d SC q ue stio ns, th e p ro je ct d ire cto rassem bled u draft report of all D elphi findinl~, including recommendations. R ound 6,

(S ept . 2 5, 1!'89) circulated that draft an d g av e th e C T exp erts th e o pp ortu nity to expressth eir v iews o r m a ke c omme nts fo r in clu sio n in th e final re po rt, w h ic h we nt th ro ug h its lastre visio ns in N o v. 1989.

7


11/111

III -- The Cogp.ltlve Skill Dimension of Cnticsl ThiokiDRF I N D I N G : A s i n d i c a t e d i n T a b l e 1 , t h e e x p e r t ~ f i n d g o o d c r i t i c a lt h i n k i n g t o i n c l u d e b o t h a s k i l l d i m e n s i o n a n d a d i s p o s i t i o n a ld i m e n s i n n . T h e e x p e r t s f i n d C T t o i n c l u d e c o g n i t i v e s k i l l s i n ( 1 )i n t e r p r e t a t i o n , ( 2 ) a n a l y s i s , ( 3 ) e v a l u a t i o n , ( 4 ) i n f e r e n c e , ( 5 )e x p l a n a t i o n a n d ( 6 ) s e l f - r e g u l a t i o n . E a c h o f t h e s o s i x i s a t t h e c o r eo f C T . A s s o c i a t e d N i t h e a c h a r e c r i t e r i a b y w h i c h i t s e x e c u t i o n c a n b em e a n i n g f u l l y e v a l u a t e d . H o w e v e r , n o a t t e m p t i s m a d e h e r e t o s p e c i f yt h o s e c r i t e r i i s i n c e a m p l e c r i t e r i o l o g i c a l d i s c u s s i o n s e x i s t i n t h el i t e r a t u r e .Concernednot to generate misunderstandings,the experts offer many

cautions about the analysis of CTin terms of skills andsub-skills. Theexperts warn that goodCTis not rote, mechanical,unreflec:tive,disconnectede){scution of sundry cognitive processes. Theycaution notto lose sight of the wholewhile attempting to attend well to its manyparts.

R E C O M M E N D A T I O N 1 : A l l C T i n s t r u c t i o n s h o u l d a i m a t d e v e l o p i n g g o o dc r i t i c a l t h i n k e r s - - p e r s o n s w h o c a n i n t e g r a t e s u c ~ e s s f u l e x e c u t i o n o fv a r i o u s s k i l l s i n t h e C T e n h a n c e d c l a s s r o o m N i t h t h e c o n f i d e n c e ,i n c l i n a t i o n ~ n d g o o d j u d g m e n t t o u s e t h e s e p a w e r f u l t o o l s i n t h e i ro t h e r s t u d i e s a n d In t h e i r e v @ r y d a y l i v e s . P e r s o n s w h o h a v ep r o f i c i e n c y i n C T s k i l l s b u t f a i l t o u s e t h e m a p p r o p r i a t e l y a r e m o s tu n l i k e l y t o b e r e g a r d e d a s g o o d c r i t i c a l t h i n k e r s .R E C O M M E N D A T I O N 2 : T h o s e N n o s e e k t o i n f u s e C T i n t o t h e e d u c a t i o n a ls y s t e m t o b e g u i d e d b y a h o l i s t i c c o n c e p t u a l i z a t i o n o f w h a t i t m e a n s t ob e a 9 0 0 d c r i t i c a l t h i n k e r . T h a t s o m e a s p e c t s o f C T , p a r t i c u l a r l yf e a t u r e s H i t h i n i t s s k i l l d i m e n s i o n , a r e m o r e r e a d i l y t a r g p . t e d b ye x i s t i n g e d u c a t i o n a l a s s e s s m e n t s t r a t e g i e s s h o u l d n o t d i s t o r t t h ec o n c e p t u a l i z a t i o n o f C T n o r t r u n c a t e f u l l - b l o w n C T i n s t r u c t i o n .

Theexpert.scharacterize certain cognitive skills as central or coreCT s~dlls. The moreone achieV'es proficiency in these skills, the more

worthy one is of being regarded as adept at CT. The experts are not,however, scayirltJ trat a person must be proficient at every skill to beperceived c..~S h,,~viflgCTability. Consideringthe panel's pl~"poses andIIlC'thorJology, trying to an~lyzeCTin terms of necessary a.ndsuf ficientconditions wouldhavehau strcJngnegativf~ utility. Thus, in viewof the

8

11


12/111

pl"ec.:isifm ~~hic:ht.he questicJn permits, the panel, ear-ly in the Delphip,."oc.:ess,dec:ided to strive for a c:onsensus on the core skills. The panelwas not asked to name skills without whic:h a person is surely not acr"itical thinker.

Responses to Rounds 4 and 5A reveal the experts to be virtuallyunerumcus (N)95Y.)on ir":luding analysis, evaluatic:m, and inference ascepJ ral. to CT. BI_\t in response to Round 6 one assessment expert stror.I:Jldissented regarding th:= rnctu-srcn of interpretation, arguing that it wasprop~rly a pdrt of c:ommunication, ~ot CT. The same expert ncted thatanal ysis, a'6 defined in this repor-t, overlaps with reading and listening.These pointE' r'C:liseobvious difficulties for CT assessment, particularlyas on~ alt~mpts to make finer differenliations between CT andcUlnmllnice\tion or- between an.alysis-in-the-CT-sense and analysis-in-the-n:~adin'J-'5ens~. Regat"cl.ing self-r"egLll~tion the e~:pert noted, the meta-cognitive~Sf'l'?ct of self--rsfJ,-\la.tion iflr~kesit e:


13/111

F I N D I N G : T h e r e i s c o n s e n s u s t h a t o n e m i g h t i m p r o v e a n e ' s a w n C T i ns e v e r a l N a y s . T h e e x p e r t s a g r e e t h a t o n e c o u l d c r i t i c a l l y t U a m i n e a n de v a l u a t e o n e ' s a w n r e a s o n i n g p r o c e s s e s . O n e c a u l d l e a r n h o ~ t o t h i n km a r e o b j e c t i v e l y a n d l o g i c a l l y . O n e c o u l d e x p a n d o n e ' s r e p e r t o i r e o ft h o s e m a r e s p e c i a l i z e d p r o c e d u r e s a n d c r i t e r i a u s e d i n d i f f e r e n t a r e a so f h u m a n t h o u g h t a n d i n q u i r y . O n e c o u l d i n c r e a s e o n e ' s b a s e o fi n f o r m a t i o n a n d l i f e e x p e r i e n c e .

It was readily apparent that the experts do not regard CTas a bodyof l::nowledgeto be delivered to students as one more school subject alongwith others. The panel sees CT, like reading and writing, as havingapplications in all areas aT life and learning. And,as several pointedout, CTinstruction, like reading and writing, can occur in programs richwith discipline-specific: content or in programs ~Jhichrely on the eventsin everyday life as the basis for developing one's CT.

F I N D I N G O n e i m p l i c a t i o n t h e e x p e r t s d r a w f r o m t h e i r a n a l y s i s o f C Ts k i l l s i s t h i s : " w h i l e C T s k i l l s t h e m s e l v e s t r a n s c e n d s p e c i f i c s u b j e c t sa r d i s c i p l i n e s , e x e r c i s i n g t h e m s u c c e s s f u l l y i n c e r t ~ i n c o n t e x t sd e m a n d s d o m a i n - s p e c i f i c k n o w l e d g e , s a m e o f w h i c h m a y c o n c e r n s p e c i f i cm e t h o d s a n d t e c h n i q u e s u s e d t o m a k e r e ~ s o n a b l e j u d g m e n t s i n t h o s es p e c i f i c c o n t e x t s . II

Although the identification and analYSis of CTskills transcend, insignificant ways, specific subjects or disciplines, learning and applyingthese skills in manycontexts requires domain-specific ~~nowledge.Thisdemain-specific knowledge includes understanding methodologicalprinciples and competence to engage in norm-regulated practices that areat the core oT reasondble judgments in those specific-contexts. Theexplicit mention of "evidential, conceptual, methodological,I.:r'j,t.er"iologiLcd, or c:onte)ttuC\l"considerations in connection withexpla.natiun rE~infon:es this point. Too muchof value is lost if CTisr,:onceivedof simply a.s a list of logical operations and domain-specific:knowledgeis clJf"lceivedof simply as an aggregation of informationInquir'y into the nexus of reasonable judgment and actual application can

10

1 3, . . -


14/111

prcdl.\c:enew appreciations of the nec:essity of robust c:onc:epts of both CTand domain-specific knowledge in educ:ation.

R E C O M M E N D A T I O N 3 1 S i n c e b e c o m i n g a d e p t a t C T i n v o l v e s l e ~ r n i n g t o u s eC T s k i l l s e f f e c t i v e l y i n m a n y d i f f e r e n t c o n t e x t s , t h e e x p e r t s i n s i s tt h a t " o n e c a n n a t o v e r e m p h a s i z e t h e v a l u e o f a s o l i d l i b e r a l e d u c a t i o nt o s u p p l e m e n t t h e h o n i n g o f o n ~ ' s C T s k i l l s a n d t h e c u l t i v a t i n g o fo n e ' s C T d i s p o s i t i o n s . "

The experts c:aution that CTskills c:anusefully be grouped and sub-c:lassified in a number of legitimate ways. Hence, the sub-c:lassificationwhic:hresulted from this Delphi research should net be interpreted asnecessaril y e>:cludingall others. Indeed, while dec:laring themselves tobe in agreement with this sub-c:lassification, various participatingexperts have also published their ownsub-classific:ations. There areareas of overlap in the c:lassifit:ation system which emerged from theDelphi r-esearch. However, while charac:terizing eac:h skill and sub-skUlis important, c~reating ar-bitrary differentiations simply to forc:e eac:hand every sub-skill to bec:omec:onceptually disc:retlrt from all the othersis rJeither l1eceStsary nor useful. In prac:tic:al c:ontexts the exec:ution ofs.oloe skill~ or sub-skills maypresuppose others. Thus, order of theDelphi Hsting is nCltintended to imply the endorsement of anypsychologic:al, logical or epistemological or-der or skill-sequenc:e, nor asprescribing any educational taxonomy or skill-hierarc:hy.

Table 3 lists the skills and sub-skills whic:hthe experts identifyas being at the core of CT. No claim is being made that the liste~;hal.lsts the concept of CTin either breadth or detail. Beyond theirinclu!iion in CT,ma.nyof the skills a.ndsub-skills identified arevaluClble, if not vital, for other important ac:tivities, suc:h asc:ommunicatingeffectively_ Also CTskills can be applied in c:oncert withother technical or ifIter-personal skills to any number of spec:ific:

11

14


15/111

concerns such as programming comput.ers, defending clients, developing awinninc;Jsales strategy. managing an office, or helping a friend figureaut what might be wrong with his car. In part this is what the expertsmean by characterizing these CT skills as pervasive and purposet-ul. It.is also fair to say that a particular skill, such as evaluation, or aparticular sub-skill, such as developing reasons, is essential forsuccess in a given endeavor, such as properly diagnosing illness. Theexpert.s are not concerned that various skills and sut'''skills are widelyused. It is not a problem that the skills might be essential elements inother endeavors. On the contrary, it would be extrl:!mely disconcerting ifthey were not, since t.he case for infusing CT into the educational systemdepends un eT's utility across almQst all areas of life and learning.

The experts are clear on the point that not every useful cognitive

; .

TABLB3CONSBNSUS LIST orCRITICAL THINKINGCOGNlTIVB SKILLS AND SUB-SKILLSt. Interpretation Categorization D e c : o d .iD & Significance CarifyiD& Meaning

Bxamiaingldeu Identifyir.g Arguments ADa lyzir .& Arguments Assessing Caims AssessiDg Arguments Queryinl Bvidence Conjecturing Alternatives Drawina Conclusions

2. Analysis

3. Bvaluation

4. Inference

S.Expianation Statial Results Ju stify in g P ro ce du re s P re se ntia& Arguments6. Self-Regulat iOJl . Self-examination

Seif-eorrec :t ioD.t5


16/111

process should be thought of as CT. Not every valuable Lhinking skill isCT skill. CT is one among a faPaily of clasel y rela ted forms of higher-order' thinking, along with, for e>:ample, problem~sol"ing, decisionmaking, cmd creative thinking. Unfortunately the conceptual overlaps andcomplex relatior'ls=hips ,ollsnongll the vari.ol.~s forms of higher-orderthinking ha.ve yet to be examined satisfactorily. Howevern, that does notimply that one cannot develop a careful and accurate conceptualization ofthe target, CT -- a conceptualization fully adequat~

In addiUofl to acccrd on the listings in Table

to '~trpCse,

3, the Delphi e>:pertswhich is to guide CT assessment and instruct.ion.

find remar-kable consensus on the descriptions of ea.ch of the skills andsub-:;kills. These descriptions a.re presented in Table 4. The e:,a.mplesas':>c.Jciatt;ld 4~ith ea.ch sub-skill i:\re intended as clarifications. Somereader~ ~ight see in the~ suggestions of possible instructional orassessment strategies. Others might see in them the tools to initiatestaff development conversations i~bout the curri


17/111

~ ' ~ ; { _ i

w i t h o u t p r e j u d i c e t o i n q u i r y ; t o d e t f r m i n e a u s e f u l w a y o f s o r t i n ga n d s u b M c l a s s i f y i n g i n f o r m a t i o n ; t o m a k e a n u n d e r s t a , ' d a b l e r e p o r to f w h a t o n e e x p e r i e n c e d i n a g i v e n s i t u a t i o n ; t o c l a ~ s i f y d a t a ,f i n d i n g s o r o p i n i o n s u s i n g a g i v e n t l a s s i f i c a t i o n s c h e m a .

1 .2 DE CO DIN G S IG NI FI CA NC E:* to deteot, attend to, and describe the informationalcontent, affective purport, directive functions, intentions,motives, pur~oses, social significance, values, views, rules,procedures. cr.iteria, or inferential relationships expressed inconvention-based oommunication systems. suoh as in language,social behaviors, drawings, numbers, graphs, tables, charts, signsand symbols.

F o r e x a m p l e : t o d e t e c t ~ n d d e s c r i b e a p e r s o n ' s p u r p o s e s i n a s k i n ga g i v e n q u e s t i o n ; t o a p p r e c i a t e t h e s i g n i f i c a n c e o f a p a r t i c u l a rf a c i a l e x p r e s s i o n o r g e s t u r e u s e d i n a g i v e n s o c i a l s i t u a t i o n ; t od i s c e r n t h e u s e o f i r o ~ y o r r h e t o r i c a l q u e s t i o n s i n d e b a t e ; t o 'i n t e r p r e t t h e d a t a d i s p l a y e d o r p r e s e n t e d u s i n g a ~ a r t i c u l a r f o r mo f i n s t r u m e n t a t l o n .

1.3 C LA RI FY IN G H EA NI N' G:* to paraphrase or make explicit, through stipulation,description. analogy or figurative expression, the oontextual.conventional or intended meanings of words, ideas, concepts,statements, behaviors, drawings, numbers, signs, charts, graphs,symbols, rules, events or ceremonies.* to use stipulation, description, analogy or figurativeexpression to ~emove confusing, unintended vagueness or ambiguity,or to design a reasonable procedure for so doing.~ o r e x a m p l e : t o r e s t a t e w h a t a p e r s o n s a i d u s i n g d i f f e r e n t w o r d so r e x p r e s s i o n s w h i l e p r e s e r v i n g t h a t p e r s o n ' s i n t e n d e d m e a n i n g s ;t o f i n d a n e x a m p l e w h i c h h e l p s e x p l a i n s o m e t h i n g t o s o m e o n e ; t od e v e l o p a d i s t i n c t i o n w h i c h m a k e s c l e a r a c o n c e p t u a l d i f f e r e n c e o rr e m o v e s a t r o u b l e s o m e a m b i g u i t y .

2. ANALYSIS: To identify the intended and actual inferentialrelationships among statements, questions, concepts, descriptionsor other forms of representation intended to express beliefs,judgments, experiences, reasons, information, or opinions.2.1 EX AMINING IDEAS:* to determine the role various expressions play or areintended to play in the context of argument, reasoning orpersuasion.

* to define terms.* to compare or oontrast ideas, concepts, or statements.* to identify issues or problems and determine theircomponent parts, and also to identify the conceptual relationshipsof those parts to each other and to the whole.F o r e x a m p l e : t o i d e n t i f y a p h r a s e i n t e n d e d t o t r i g g e r as y m p a t h e t i c e m o t i o n a l r e s p o n s e w h i c h m i g h t i ~ d u c e a n a u d i e n c e t oa g r e e w i t h a n o p i n i o n ; t o e x a m i n e c l o s e l y r e l a t e d p r o p o s a l s

14 17


18/111

r e g a r d i n g a g i v e n p r o b l e m a n ~ t o d e t e r m i n e t h e i r r O i n t s o fs i m i l a r i t y a n d d i v e r g e n c e ; g i v e n a c o m p l i c a t e d a s s i g n m e n t , t od e t e r m i n e h O H i t m i g h t b e b r o k e n u p i n t o s m a l l e r , m o r e m a n a g e a b l et a s k s ; t o d e f i n e a n a u s t r a c t c o n c e p t .

2.2 DETECTING ARGUMENTS:* given a set of statements, desoriptions, questions orgraphic representations, to determine whether or not the seteK presses, or is intended to express, a reason or reasons J.nsupport of or contesting some olaim, opinion or point of view.

F o r e x a m p l e , g i v e n a p a r a g r a p h , d e t e r m i n e w h e t h e r a s t a n d a r dr e a d i n g o f t h a t p a r a g r a p h i n t h e c o n t e x t o f h o w a n d w h e r e it i sp u b l i s h e d . w o u l d s u g g e s t t h a t i t p r e s e n t s a c l a i m a s w e l l a s ar e a s o n o r r e a s o n s i n s u p p o r t o f t h a t c l a i m ; g i v e n a p a s s a g e f r o m an e w s p a p e r e d i t o r i a l , d e t p r m i n e i f t h e a u t h o r o f t h a t p a s s a g ei n t e n d e d i t a s a n e x p r e s s i o n o f r e a s o n s f o r o r a g a i n s t a g i v e nc l a i m o r o p i n i o n ; g i v e n a c o m m e r c i a l a n n o u n c e m e n t , i d e n t i f y ~ n yc l a i m s b e i n g a d v a n c e d . a l o n g w i t h t h e r e a s o n s p r e s e n t e d i n t h e i r~ u p p o r t .

2.3 ANALYZING ARGUMENTS:* given the expression of a reason or reasons intendedto support or oontest some claim, opinion or point of view, toidentify and differentiate: (a) the intended main conolusion, (b)the premises and reasons advanced in support of the mainconclusion, (c) further premises and reasons advanced as backup orsupport for those premises and reasons intended as supporting themain oonolusion, (d) additional unexpressed elements of thatreasoning, such as intermediary conclusions, unstated assumptionsor presuppositions, (e) the overall structure of the argument orintended chain of reasoning, and (f) any items contained in thebody of expressions being examined which are not intended to betaken as part of the reasoning being expressed or its intendedbackground.

F o r e x a m p l e : g i v e n a b r i e f a r g u m e n t , p a r a g r a p h - s i z e d a r g u m e n t , o ra p o s i t i o n p a p e r o n a c o n t r o v e r s i a l s o c i a l i s s u e , t o i d e n t i f y t h ea u t h o r ' s c h i e f c l a i m , t h e r e a s o n s a n d p r e m i s e s t h e a u t h o r a d v a n c e so n b e h a l f o f t h a t c l a i m , t h e b a c k g r o u n d i n f o r m a t i o n u s e d t o~ u p p o r t t h o s e r e a s o n s o r p r e m i s e s , a n d c r u c i a l a s s u m p t i o n si m p l i c i t i n t h e a u t h o r ' s r e a s o n i n g ; g i v e n s e v e r a l r e a s o n s o rc h a i n s o f r e a s o n s i n s u p p o r t o f a p a r t i c u l a r c l a i m , t o d e v e l o p ag r a p h i c r e p r e s e n t a t i o n w h i c h u s e + u l l y c h e r e c t e r i e s t h ei n f e r e n t i a l f l o w o f t h a t r e a s o n i n g .

3. EVALUATION: To assess the credibility of statements or otherrepresentations which are acoounts or descriptions of a person'sperception. experience, situation, judgment, belief, or opinion;and to assess the logical strength of the actual or intendinferential relationships among statements, descriptions,questions or other forms of representation.3.1 ASSESSING CLAIMS:* to recognize the factors relevant to assessing the

15

~:~\,:

18


19/111

degree of credibility to ascribe tQ d source of information oropinion. * to assees the contextual relevance of questions,information, princi.ples, rules or procedural directions.* to asseES the acceptability, the level o~ confidenceto place in the probability or truth of any given representationof an experience, situation, judgment, belief or opinion.F o r e x a m p l e : t o r e c o g n i z e t h e f a c t o r s w h i c h m a k e a p e r s o n ac r e d i b l e w i t n e s s r e g a r d i n g ~ g i v e n e v e n t o r c r e d i b l e a u t h o r i t y o na g i v e n t o p i c ; t o d e t e r m i n e i f a g i v e n p r i n c i p l e o f c o n d u c t i sa p p l i c a b l e t o d e c i d i n ~ w h a t t o d a i n a g i v e n s i t u a t i o n ; t od e t e r m i n e i f a g i v e n c l a i m i s l i k e l y t o b e t r u e o r f a l s e b a s e d e nw h a t o n e k n o w s o r c a n r e a s o n a b l y f i n d o u t .

3.2 ASSESSING ARGUMENTS:* to judge whether the assumed acceptability of thepremises of a given argument justify one's acoepting as true(deductiy~ly certain), or very probably true (inductivelyjustified), the expressed conclusion of that argument.* to antioipate or to raise questions or objections, andto assess whether these point to significant weakness in theargument being evaluated.* to determine whether an argument relies on false ordoubtful assumptions or presuppositions and then to determine howcrucially these affect its strength.* to judge between reasonable and fallacious inferences;* to judge the probative strength of an argument'spremises and assumptions with a view toward determining theacceptability of the argument.* to determine and judge the probative strength of anargument's intended or unintended consequences with a view towardjudging the acceptability of the argument;* to determine the extent to which possible additionalinformation might strengthen or weaken an argument.

F o r e x a m p l e : g i v e n a n a r g u m e n t t o j u d g e if i t s c o n c l u s i o n f o l l o w se i t h ~ r w i t h c e r t a i n t y o r w i t h a h i g h l e v e l o f c o n f i d e n c e f r o m i t sp r e m i s e s ; t o c h ~ c k f o r i d e n t i f i a b l e f o r m a l a n d i n f o r m a l f ~ l l a c l e ~ lg i v e n a n o b j e c t i o n t o a n a r g u m e n t t o e v a l u a t e t h e l ~ ~ ; : ~ 1 f e r c e o ft h a t o b j e c t i o n ; t c e v a l u a t e t n ~ q u a ! l ~ ~ ~ n j a p p l i c a b i l i t y o fa n a l o g i c a l a r g u m e r l ! . s l 'r : ~'.ldCJe the l o g . i ; : ? 1 strength o f ugumentsb a s e d C ,' r~ ...0 Q t ~~t 1 C c1 1 's 1 t '. 1 e1 t i o n s C ' r causal rea -:;c nin 9 ; l0 jLId 9 e i fa g i v e n ~ r g u m e n t !~ r e l e v a n t o r d p p l i c a b l e o r h a s i m p l i r a t i o n s f o rthe sit u e t io n c lt t:and; to d e t e r m i n e ho a possible new data mlyh~l e a d l o g i c ~ l l y t o t h e f u r t h e r c o n f i r m a t i o n o r d i s t o n f i r m a t i o n o f ag iIien: p 1rl12~I

4: INFERENCE: To identify and secure elements needed to drawreasonable conclusions; to form conjectures and hYrotheses; tocOIlsider relevant information and to educe the consequencesflowing from data, statements, principles, evidence, judgments,b el ief s, o pi nio ns, concepts, descriptions, questions, or otherforms of r~presentation,

16 19


20/111

4.1 QUE RYING EVI DENC E:* in particular, to recognize premises which requiresupport and to formulate a strategy for seeking and gatheringinformation which might supply that supPort.* in general, to judge that information relevant todeciding the aooeptability, plausibility or relative merits of agiven alternative, question, issue, theory, hypothesis, orstatement is required, and to determine plausible investigatorystrategies for aoquiring that information.

F o r e x a m p l e ; w h e n a t t e m p t i n g t o d e v e l o p a p e r s u a s i v e a r g u m e n t i ns u p p o r t o f o n e ' s o p i n i o n , t o j u d g e w h a t b a c k g r o u n d i n f o r m ~ t i o n i tw o u l d b e u s e f u l t o h a v e a n d t o d e v e l o p a p l a n w h i c h w i l l y i e l d ac l e a r a n s w e r a s t o w h e t h e r o r n o t s u c h i n f o r m a t i o n i s a v a i l a b l e ;a f t e r j ~ d g i n g t h a t c e r t a i r . m i s s i n g i n f o r m a t i o n w o u l d b e g e r m a n e i nd e t e r m i n i n g i f a g i v e n o p i n i o n i s m o r e o r l e s s r e a s o n a b l e t h a n ac o m ~ e t i n g o p i n i o n , t o p l a n a s e a r c h w h i c h w i l l r e v e a l i f t h a ti n f o r m a t i u n i s a v a i l a b l e .

4 .2 C ON JE CT UR IN G A LT ER NA TI VE S;* to formulate multiple alternatives for resolving aproblem, to postulate a series of suppositions regarding aquestion, to project alternative hypotheses regarding an event, todevelop a variety of different plans to aohieve some goal.* to draw out presuppositions and projeot the range ofpO~5ible consequences of decisions, positions, policies. theories,or beliefs.

F o r e ~ a m p l e : g i v e n a p r o b l e m w i t h t e c h n i c a l , e t h i c a l o r b u d g e t a r yr a m i f i c a t i o n s , t o d e v e l o p a s e t o f o p t i o n s f o r a d d r e S S i n g a n dr e s o l v i n g t h a t p r o b l e m , g i v e n a s e t o f p r i o r i t i e s w i t h w h i c h o n em a y o r m a y n o t ~ g r e e , t o p r o j e c t t h e d i f f i c u l t i e s a n d t h e b e n e f i t sw h i c h are l i k e l y t o r e s u l t i f t h o s e p r i o r i t i e s a r e a d o p t e d i nd e c i s i o n m a k i n g .

4.3 DRAWI NG CON CLUSI ONS:* to apply appropriate modes of inference in determiningwhat position. opinion or point of view one should take on a givenmatter or issue.* given a set of statements, descriptions, questions orother forms of representation, to educe, with the proper level oflogical strength, the~r inferential relationships and theconsequences or the presuppositions which they support, warrant,imply or entail.* to employ successfully various sub-species ofreasoning, as for example to reason analogically. arithmetically,dialectically, scientifically, etc.* to determine which of several possible conclusions ismost strongly warranted or supported by the evidence at hand, or which should be rejected or regarded as less plausible by thein for ma ti on gi ve n.

F o r e x a m p l e : t o c a r r y o u t e x p e r i m e n t s a n d t o a p p l y a p p r o p r i a t e~ t ~ t i s t i c a l i n f e r e n c e t e c h n i q u e s i n o r d e r t o c o n f i r m o r d i s c o n f i r ma n e m p i r i c a l h y p o t h e s i s ; g i v e n a c o n t r o v e r s i a l i s s u e t o e x a m i n e

17


21/111

i n f o r m e d o p i n i o n s , c o n s i d e r v a r i o u s o p p o s i n g v i e w s a n d t h e r e u s o n sa d v a n c e d f o r t h e m , g a t h e r r e l e v a n t i n f o r m a t i o n , a n d f o r m u l a t eo n e ' s o w n c o n s i d e r e d o p i n i o n r e g a r d i n g t h a t i s s u e ; t o d e d u c e at h e o r e m f r o m a x i o m s u s i n g p r e s c r i b e d r u l e s o f i n f e r e n c e .

5: EX PLANATION: To state the resul ts of one 's re.asoning; tojustify that reasoning in terms of the evidential, conceptual,methodological, criteriological and contextual considerations uponwhich one's results were based; and to present one's reasoning inthe form of oogent arguments.

5.1 STATING RESOLTS:* to produce accurate statements, descriptions orrepresentations of the results of one's reasoning activities so asto analyze, evaluate, infer from, or monitor those results.F o r e x a m p l e : t o s t a t e o n e ' s r e a s o n s f o r h o l d i n g a g i v e n v i e w ; t ow r i t e d o w n f o r o n e ' s o w n f u t u r e u s e o n e ' s c u r r e n t t h i n k i n g a b o u ta n i m p o r t a n t o r c o m p l e x m a t t e r ; t o s t a t e o n e ' s r e s e a r c h f i n d i n g s ;t o c o n v e y o n e ' s a n a l y s i s a n d j u d g m e n t r e g a r d i n g a w o r k o f a r t ; t os t a t e o n e ' s c o n s i d e r e d o p i n i o n o n a m a t t e r o f p r a c t i c a l u r g e n c y .

5 .2 J UST IF YI NG P RO CE DU RES :* to present the evidential, conceptual, methodological,criteriological and oontextual considerat~ons which one used informing one's interpretations, analyses, evaluation or inferences,so that one might accuratelY record, evaluate, descrihe or justifythose processes to one's self or to others, or so as to remedyperceived deficiencies in the general way one executes thoseprooesses.F o r e x a m p l e : t o k e e p a l o g o f t h e s t e p s f o l l o w e d i n w o r k i n gt h r o u g h a l o n g o r d i f f i c u l t p r o b l e m o r s c i e n t i f i c p r o c e d u r e ; t oe x p l a i n o n e ' s c h o i c e o f a p a r t i c u l a r s t a t i s t i c a l t e s t f o r p u r p o s e so f d a t a a n a l y s i s ; t o s t a t e t h e s t a n d a r d s o n e u s e d i n e v a l u a t i n g ap i e c e o f l i t e r a t u r e ; t o e x p l a i n h o w o n e u n d e r s t a n d s a k e y c o n c e p tw h e n c o n c e p t u a l c l a r i t y i s c r u c i a l f o r f u r t h e r p r o g r e s s o n a g i v e np r o b l e m ; t o s h o w t h a t t h e p r e r e q u i s i t e s f o r t h e u s e o f a g i v e nt e c h n i c a l m e t h o d o l o g y h a v e b e e n s a t i s f i E d ; t o r e p o r t t h e s t r a t e g yu s e d i n a t t e m p t i n g t o m a k e a d e c i s i o n i n a r e a s o n a b l e w a y ; t od e s i g n a g r a p h i c d i s p l a y w h i c h r e p r e s e n t s t h e q u a n t i t a t i v e o rs p a t i a l i n f o r m a t i o n u s e d a s e v i d e n c e .

5. 3 PR ES ENT IN G A RGU ME NT S:* to give reasons for accepting some claim.* to meet objections to the method, conceptualizations,evidence, criteria or contextual appropriateness of inferential,analytical or evaluative judgments.

F o r e x ~ m p l e : t o w r i t e a p a p e r i n w h i c h o n e a r g u e s f o r a g i v e n~ o s i t i o n o r p o l i c y ; t o a n t i c i p a t e a n d t o r e s p o n d t o r e a s o n a b l ec r i t i c i s m s o n e m i g h t e x p e c t t o b e r a i s e d a g a i n s t o n e ' s p o l i t i c a lv i e w s l t o i d e n t i f y a n d e x p r e s s e v i d e n c e a n d c o u n t e r - e v i d e n c ei n t e n d e d a s a d i a l e c t i c a l c o n t r i b u t i o n t o o n e ' s o w n o r a n o t h e rp e r s o n ' s t h i n k i n g o n a m a t t e r o f d e e p p e r s o n a l c o n c e r n .

18 21


22/111

'

8: SELF-REGULATION: Self-consciously to monitor one's cognitiveactivities, the elements used in those activities, and the resultseduced, particularly by applying skills in analysis and evaluationto one's own inferential Judgments with a view toward questioning,confirming, validating, or corr~cting either one's reasoning orone's results.8.1 SELF-EXAMINATION:* to reflect on on~'s own reasoning and verify both theresults produced and the corr~ct application and execution of. thecognitive skills involved.* to make an objective and thoughtful meta-cognitiveself-assessment of one's opinions and reasons for holding them.* to judge the extent to which one'e thinking isinfluenced by deficiencies in one's knowledge, or by stereotypes,prejudices, emotions or any other factors which constrain one'sobjectivity or rationality.* to reflect on one's motivations, values, at~itudes andinterests with a view toward determining that one has endeavored

to be unbiased, fair-minded, thorough, objective, respeotful ofthe truth, reasonable, and rational in coming to one's analyses,interpretations, evaluations, inferences, or expressions.F o r e x a m p l e : t o e x a m i n e o n e ' s v i e w s o n a c o n t r o v e r s i a l i s s u e w i t hs e n s i t i v i t y t o t h e p o s s i b l e i n f l u e n c e s o f o n e ' s p e r s o n a l b i a s o rs e l f - i n t e r e s t ; t o r e v i e w o n e ' s m e t h o d o l o g y o r c a l c u l a t i o n s w i t h av i e w t o d e t e c t i n g m i s t a k e n a p p l i c a t i o n s o r i n a d v e r t e n t e r r o r s ; t or e r e a d s o u r c e s t o a s s u r e t h a t o n e h a s n o t o v e r l o o k e d i m p o r t a n ti n f o r m a t i o n ; t o i d e n t i f y a n d r e v i e w t h e a c c e p t a b i l i t y o f t h ef a c t s , o p i n i o n s o r a s s u m p t i o n s o n e r e l i e j o n i n c o m i n g t o a g i v e np o i n t o f v i e w ; t o i d e n t i f y a n d r e v i e w o n e ' s r e a s o n s a n d r e a s o n i n gp r o c e s s e s i n c o m i n g t o a g i v e n c o n c l u s i o n .

6.Z SELF-CORRECTION:* where self-examination reveals errors or deficiencies,to design reasonable procedures to remedy or correct, if possible,those mistakes and their causes.F o r e x a m p l e : g i v e n a m e t h o d o l o g i c a l m i s t a k e o r f a c t u a l d e f i c i e n c yi n o n e ' s w o r k , t o r e v i s e t h a t w o r k s o a s t o c o r r e c t t h e p r o b l e ma n d t h e n t o d e t e r m i n e if t h e r e v i s i o n s w a r r a n t c h a n g e s i n a n yp o s i t i o n , f i n d i n g s , o r o p i n i o n s b a s e d t h e r e o n .

19 22


23/111

IV -- T he Dispos itional Dimensi on of Cri tical Tbinking

As is evident, particularly in the descriptions of self-examinationand self-correction, there are di5pOs~.tional components to criticalthinking. Indeed each cognitive skill, if it is 'to be exercisedappropriately, can be correlated with the cognitive disposition to do so.In each case a person who is proficient in a given skill can be said tohave the aptitude to execute that skill, even if at a given moment theper'son is not using the skill. But there was a great deal more manyexper-ts wished s...y in regard to the personal traits, habits nf mind,


24/111

that various experts mean different things when they used the term "eT" inreference to its possible dispositional components.

The ~eepest division is between the nearly two-thirds majority whohold that the term neT" includes in its meaning a reference to certainaffective dispositions dnd the roughly one-third minority who hold thatneTIl t-efet-s only to cognitive skills and di~positions, but not to affectivedispositions. The project director put this issue to the panel in severaldiffer-snt ways, sometimes directly and at other times more obliquely.Responses, comment.sC\ndarguments wer-e shared, as were the objections ancounter-arguments which they engendered. In the end the panel remaineddivided both numerically and in depth of feeling, with opPOSingpositionsb"='!comingore strident and entr'enc:hed as the debate continued.

In Round 513, of those expr-essing an opinion, the majority (611..)Inaintairl t.hat the affective dispositions constitute part of the meaning of"CT.II They drgue tha.t these dispositions flew from, and are implied by,the ver"y concept of CT, much as the cognitive dispositions are. Thesee~


25/111

to persuade the 0ther third of their expert colleagues to view thesedi:.positio.1s as essential to the concept I.Jf CT. The majority was, howeverpersuasive in bringing about virtual unanimity regarding l.\5ing theaffective dispositions to describe the paradigm critical thinker. (SeeTable 1.)

In Round :S8 C \ minority (30~) insist on using "eT" in a strictprocedural ~ense, that is as referring only to a certain judgmentalprcc~ss. They distinguish sharply between what: is true of criticalthir)/dng from what is true of good criticdl thinkers. Their primaryc.oncern iswith the CTskills. They argue that good ct-itical thinkers arepeople whohave those skills and certain valuable habits as well. If theyare good cl'-itical thinkers, then they llse their' CT skills appropriatelybecause good cr-itical thinkers also have some or all of the affectivet1ispositions listed in Table 5. But those dispositions are not what ismednt by 'leT.'1 They argue that one would not want to say a sophist is na critical I:hinker simply b~caLlse the sophist uses CT skills fer deceptiveor sel.'f-iJlter'~sled ends. The sophist, they would lTI~intain,is a c:riticalthinl-:er -- but not an good one


26/111

"Howwell is this person using CT?n The secorld sense applies to thethinker's morality and responds to the question, "Is this person's use' ofCT ethical?" In order to clarify whic.:hsense of ngood" the experts wishedto convey, Round 4 asked the panel to respond to a proposal that CT mighthave a normative dimension in addition to a skill dimension and a affec:t.ivedisposi tional dimension.

F I N D I N G : T h e m i s t a k e n n o t i o n t h a t C T h a s a n o r m a t i v e c o m p o n e n t i sr e j e c t e d by t h e e x p e r t p a n e l i s t s . I t i s a n i n a p p r o p r i a t e u s e o f t h et e r m t o d e n y t h a t s o m e o n e i s e n g a g e d i n C T o n t h e g r o u n d s t h a t o n ed i s a p p r o v e s e t h i c a l l y o f " I h a t t h e p e r s o n i s d o i n g . W h a t " C T n m e a n s ,w h y i t i s o f v a l u e , a n d t h e e t h i c s o f i t s u s e a r e b e s t r e g a r d e d a st h r e e d i s t i n c t c o n c e r n s .

The majority of experts er;utor who use~ CT skills to contrive a way to mislead a gulliblejll(Y into corivrc t.inq and punishing an innocent psrson is not a criticalthiph:~r. Sirrce nt:?itl-~erSLtfficiently valLte truth cg.nd since both appear tolad:: the ,nor'al fiber to e!-~chewdelibE:?rate deception in the practice oftheir socially Import.an t; professions, neither attorney should be accorded

2 6


27/111

:" : "

the moral approval which calling them c:ritical thinkers would imp1y.The debate turned out to be instructive in another way as well. The

consensus (74h in support and 4h opposed) was that this report shoulds)(press the experts' fullest support and appreciation of the immensepersonal and social importance of CT.

The panel shares a solid consensus about the importance of CTas atool of inquir'y, as a liberating force in education, as a powerfulreSOLlrce in one's personal life, and as a vital component in a rationaldemoc("atic society. It is 9>:tremely unlikely that any panelist wouldcondone using CT for immoral, deceptive, or unjust purposes. However thepersonal and civic value of CT and sensitivity to the morality of its usedre not acceptable grounds far building a normative dimension into themeaning of the term nCT". Some aven saw such an effort as misguided andpotentially destr-uctive of the CTmovement. Giving "eT" a normativetwist could, the'l" argue, lead to unwarranted limitations on open inquiryand b:l unjustifie.\ble ideological restrictions 011 the very conc:ept ofbeing a "thinking" person. The totalitarian specter this conjures up ist.he antithe'Eli.s of the liberating critical spirit described earlier.

Dispo~"iti()"s of' the Good Critical Thinker

F I N D I N G : T o t h e e x p e r t s , a g o o d c r i t i c a l t h i n k e r , t h e p a r a d i g m c a s e ,i s h a b i t u a l l y d i s p o s e d t o e n g a g e i n , a n d t o e n c o u r a g e o t h e r s t o e n g a g ei n , c r i t i c a l j u d g m e n t . S h e i s a b l e t o m a k e s u c h j u d g m e n t s i n a w i d er a n g e o f c o n t e x t s a n d f o r a w i d e v a r i e t y o f p u r p o s e s . A l t h o u g h p e r h a p sn o t a l ~ a y s u p p e r m o s t i n m i n d , t h e r a t i o n a l j u s t i f i c a t i o n f o rc u l t i v a t i n g t h o s e a f f e c t i v e d i s p o s i t i o n s w h i c h c h a r a c t e r i z e t h ep a r a d i g m c r i t i c a l t h i n k e r a r e s o u n d l y g r o u n d e d i n e T ' s p e r s o n a l a n dc i v i c v a l u e . C T i s k n o w n t o c o n t r i b u t e t o t h e f a i r - m i n d e d a n a l y s i s a n dr e s o l u t i o n o f q u e s t i o n s . C T i s a p o w e r f u l t o o l i n t h e s e a r c h f o rk n ~ " l e d g e . C T c a n h e l p p e o p l e o v e r c o m e t h e b l i n d , s o p h i s t i c , o ri r r a t i o n a l d e f e n s e o f i n t e l l e c t u a l l y d e f e c t i v e o r b i a s e d o p i n i o n s . C Tp r o m o t e s r a t i o n a l a u t o n o m y , i n t e l l e c t u a l f r e e d o m a n d t h e o b j e c t i v e ,

24


28/111

.~"\1 '-- ,... :'

r e a s o n e d a n d e v i d e n c e - b a s e d i n v e s t i g a t i o n o f a v e r y N i d e r a n g e o fp e r s o n a l a n d s Q e ~ a l i s s u e s a n d t o n e s r n s .

Thus, in addition to possessing CTskill~ the goed critical thinkerc:anbe charac:ter"izsd by certain affective dispositions or habits of mind.These dis po si ti ons , li st ed in Table 5 belew, flew from two sources:characteristics which the experts judge to hold true of good criticalttllnkers, and t.he affective dispositions the expert:i judge to be part ofCT in its fullest. realization. The majority of the experts (61X)regardthe dispositions listed in Table 5 as part of the conceptualization ofCT. A consensus eHists (8::3Xn favor) that good critical thinkers can becharacterized as exhibiting thesE: dispositions.

TABLBSAFFBCTIVB DISPOSITIONS OF CRITICAL THINKINGAPPROACHBS TOLIPS ANDLIVING INGBNBRAL: iDquis it iveness with regard to a w ide range of issues, concem to becom e 8JId remain gene ra lly weU- in fo rmed , a le rtn es s to o pp ortu nitie s to use CT, trust in the proc:eues of reasoned inquiry,

se lf -confidence inODe '. OWDabU i ty to re as on , opea-mindedneu re prd in& d iv erg en t world views, ftezibiUty illconsidering a l temaUves and opinions, u nd erstaad iD& o f the o pin io ns o f o th er p eo ple , fair-miDdedaess illappraiaillg reuonin& honesty in faciag on e'. ow n b iases, prejudices,s te re oty pe s, e lo c: en trie o r s oc io ce ntr iL te nd en cie s, prudence insuspen diag , m aking or altering judgments, w iU iapess to reconsider and rev ise v iew s w here honestreOection sugests that change is warranted.APPROACHBS TO SPBClFIC ISSUBS, QUBSTIONS OR PROBLBMS: cladty in statiol the question or concem, orderliness inworking with comple :Ki ty , d ilig en ce in seek in g relev an t iD fo rm atio n, reuouableaess in selectinl and applying criteria, care in focusiD S attention on tbe co ncem at hand, persisteDc:e thou ... difficulties are encouatered, prectlion to the depee perm itted by subject and circum stances

0 2S, . . .

28


29/111

The experts are not saying that a person whose metaphysical,epistemological, political, cultural or religious view of the world isdifferent from ane's own is, ipso 'facto, net a good critical thinker.Beliefs are not atoms each of which is at any moment subject to beingreconsidered independently. Beliefs form intricately interconnectedsystems of thought. To focus critical attention on any of them,particular'ly those mor'e central or fundamental to one's own view of theworld, ca.r. cause reverberations throughout cne's entire belief system.Thus in ad\iocating CT the panelists are not urging ideologicalconforinity. Indeed, just as many experts argued that an over-emphaSis onthe values of CT could lead to trQuble, others warn that an over-emphasison the skills dimension of CT to the exclusion of the affectivedispositions might have the unfortunate result of making some studentsclosti-minded, intellectually inflexible and dogmatic.

R E C O M M E N D A T I O N 5 : J u s t a s w i t h t h e c o g n i t i v e d i m e n s i o n o f C T , w h e nc o n c e i v i n g o f t h e e d u c a t i o n o r a s s e s s m e n t o f c r i t i c a l t h i n k e r s , i t i si m p o r t a n t t o c o n s i d e r w a y s o f d e v e l o p i n g m a t e r i a l s , p e d a g o g i e s , a n da s s e s s m e n t t o o l s t h a t a r e e f f e c t i v e a n d e q u i t a b l e i n t h e i r f o c u s o nt h e s e a f f e c t i v e d i s p o s i t i o n s . T h e c u l t i v a t i o n o f t h e s p . d i s p o s i t i o n s i sp a r t i c u l a r l y i m p o r t a n t t o i n s u r e t h e u s e o f C T s k i l l s o u t s i d e t h en a r r o w i n s t r u c t i o n a l s e t t i n g . P e r s o n s w h o h a v e d e v e l o p e d t h e s ea f f e c t i v e d i s p o s i t i o n s a r e m u c h m o r e l i k e l y t o a p p l y t h e i r C T s k i l l sa p p r o p r i a t e l y i n b o t h t h e i r p e r s o n a l l i f e a n d t h e i r c i v i c l i f e t h a n a r et h o s e w h o h a v e m a s t e r e d t h e s k i l l s b u t a r e n o t d i s p o s e d t o u s e t h e m .

As with the listing of cognitive skills ea.rlier, the panel does notintend that each disposition be considered .a necessary condition. Theexperts are characterizing the ideal. In setting forth the concept cfthe paradigm c:rP:icdl thinker, they intend to e>:press a gOnl taw~l~d whichall might strive. These '.irtLteS require a meaSLlr-e of matLlrity io=t.fldper SOl ,al development not commcmlv found in college saphomor-esor twelfthgradt:?(s. Yet to u~l


30/111

" . : ' ; ' , ' ; :~.

will lead to these vir-tL,es WOL\ldbe an even moreprofound mistah:e.

R E C O M M E N D A T I O N 6 : F r o m e a r l y c h i l d h o o d p e o p l e s h o u l d b e t ~ u g h t , f o re x a m p l e , t o r e a s o n , t o s e e k r e l e v a n t f a c t s , t o c o n s i d e r u p t i o n s , a n d t ou n d e r s t a n d t h e v i e w s o f o t h e r s . I t i s n e i t h e r i m p r a c t i c a l n o ru n r e a s o n a b l e t o d e m a n d t h a t t h e e d u c a t i o n a l s y s t e m t e a c h y o u n g p e o p l et h e h a b i t s o f m i n d w h i c h c h a r a c t e r i z e t h e g o o d c r i t i c a l t h i n k e r ,r e i n f o r c e t h o s ! p r a c t i c e G , a n d m o v e s t u d e n t s w e l l d o w n t r e p a t h t o w a r dt h e i r a t t a i n m e n t .

Y -- Further Recommendations on CT Instruction and AssessmentSevera.lpedagogic:alandassessment implications follow from the

dispositional dimensionof CT, implications whichmight not be apparentif educ:at.orsfocused only on the skill dimensionof CT. Theeducation ofgoodcritical thinkers is more than training students to execute a set ofc:ognitive skills. For example,in terms of pedagogy,modelinghowtoevaluate critically that information whichstudents wouldnormally acceptuncritically and encouragingthemto do the samecan do wondersfordevelopingtheir' confidence in their CTability. With this confidencestudents are muchmorelikely to try thinking for themselves. Just asinstn.ll:tion should not focus on shills only, assessmentwhichfocus ons~dlls onl)r' may give a misleadingor inc:ompletepictu.re of someone's,=,trengt.hsas a critkal thinker.

The CT Goal

R E C O M M E N D A T I O N 7 : B r c a u s e C T h e l p s s t u d e n t s w l t h a w i d e r a n g e o fe d u c a t i o n a l , p e r s o n a l a n d CIVIC c o n c e r n s i n a r a t i o n a l w a y , t h ea c a ~ e m i c g o a l o f C T i n s t r u c t i o n , r e g a r d l e s s o f t h e e d u c a t i o n a l l e v e l ,s h o u l d b e f u r t h e r i n g s t u d e n t s i n t h e d e v e l o p m e n t o f t h e i r C T c o g n i t i v es k i l l s a n d a f f e c t i v e d i s p o s i t i o n s .

27


31/111

TABLB6CONSENSUS STATBMBNT ON TBACH ING AND ASSBSSING CT SK ILLSA CT skill, like any skill, is the ability to engage in aD activity, process orprocedure. Ingeneral, bav iD l a ,kill includes being able to do the right th ing at theright time. So, being sk ille d a t CT involves knowin&. perhaps impUcidy or witbout tbeability to articu late this know lecl& e, bo th a set of procedures and wbeu to apply thosep ro ce du res. B eiDg sk illed also involves having some degree of profic iency ineJ:ecutingthose proc:edures and beiD g w iU iag to do so wben appropriate. ReSecting on andimprov iDl one's CT skills involves judging.ben one is or is not performing weD , or 81weU u possible, a nd c on sid ed a& way. o f im prov ing o ne's perfOrDIaDce. Leam iD g C Tinvolves acquiring the ability to make su ch self-reD ectiv e judgments.Skills, particularly CT captive skiDs, can be taugbt in' a variety of ways, sucbaa by m aking the p roc: edur es expUc: it , describiag how they are to be applied andexecuted, explaininland modeMI their correct use, and justifying their app6c:atioD.Teaehiaa c:opitive skills also involves exposing leamen to situatioDi wbere there are

good reuons to exerc: ise the desired, proc:edures, jud&ing their performanc:e, andproYidia& the leamen with constructive feedback t'ep rd ia& b oth th eir p ro fic ie ncy andways to improve it. Instruction might start with situatic.ns that ale aditidaUy simple,but should culminate insituations that are realisticaUy complex. Padicularly illthe caseof CT, the leamen must contribute a solid measure of personal effort, attention, practice,desire, and, u they learn how, self-monitoring. Teaching sk iD s in vo lv es mo tiv atin glearnen to achieve higher le~els of p ro ficien cy an d, p articu la rly in the case of CT ,independence. It also in vo lv es c oa ch in g le arn ers on how tbey can achieve those goals.In tbeory there are several ways persons can be judged to be more or leuproficient in a given L"'Tskill or at the integrated use of related CT sIriUs. One way

is to observe a person over tim e perform ing those activ ities, processes or Pnk.'"'eduresgenerally regarded as p resuppos ing that skiD (o r p ro pe r execution. ODe th en mDlo... (ju dgmen t re gard in g the degree to wbieb tbe person possesses the general skill inquestion. A second way is to com pare the outcom es (if any) that result from elecutinga given skiDapillit some set of criteria. A third way is to query persons and receivetheir descriptions of the p roc: edur es aDd judgments they are using as they exercise thatsk iU, wou ld use if tbey were to perform that sk iD , or did use when tbey performed thatskill. A fourth w.y is to campa'ie tbe outcomes (i f any) tbat result from performinganother tu k against some set of criteria, where the perform ance of that otber taskhas been sbown to correlate strongly with exercising the skill of interest. However,that such correlations exist between any otber task and CT, or any of its sub-skiUs, basyet to be established in the research literature.

Bach of the four ways of CT assessment has limitations as well 85strengths. Nomatter which ways are used, it is important to ensure that the assessment conditionsfoster an attitude inwhich the subjects are disposed to use their skills as well as theyean, and are not constrained or inhibited from doing so . Inour view it is highlyadvantageous to gatber ev idence regarding CT perform ance in many situ atio ns, u sin gseveral assessment methods, so as to compile a composite picture of the subject and toc ro ss chtN : k the results of an one wa of assessment.


32/111

_r:, ~"

Either to transform CTinto one subject field amongothers, or tonarrow the range of CT applications strictly to domain-specific subject:content, would be to truncate its utility, misapprehend its nature anddiminish its valu~~. Within the overall curriculum the goal of learningCT can be clearly distinguished from the goal of learning domain-specificcontent. Andyet,: while these two goals can b' distinguished, theexperts do not wish to deny one of the best ways to learn CT is within asubject c:ontext.

R E C O M M E N D A T I O N 8 : D i r e c t i n s t r u c t i o n i n C T a n d a s s e s s m e n t o f C T s h o u l db e a n e K p l i c i t p a r t s o f an y c o u r s e g r a n t e d a p p r o v a l f o r p u r p o s e s o fs a t i s f y i n g C T r e q u i r e m e n t s , w h e t h e r t h a t c o u r s e i s a C T c o u r s e per seD r a c o u r s e i n a g i v e n s u b j e c t f i e l d . T h e p r i m a r y a c a d e m i c c r i t e r i o ni n t h e e v a l u a t i o n o f a p r o p o s e d i n s t r u c t i o n a l p r o g r a m f o r p u r p o s e s o fa c h i e v i n g t h e C T g o a l s h o u l d b e w h e t h e r t h e p r o g r a m w i l l f u r t h e r t h ed e v e l o p m e n t o f s t u d e n t s ' C T s k i l l s a n d d i s p o s i t i o n s .

The CT Curriculu.Given that CT has, in many cases, became a college genera.l education

reqLdrement, secondary schools can be expected to begin to developcollege preparatory CT programs. However, the value of CT extends wellbeyond its importance as a university-level inquiry tool. CT is vitallyimpor"tant in the personal and civic life of all members of society. Asignificant: pet-centage of the citizenry will not gr-aduate from highschool, or ii they graduate, will not have the benefit of post-secondaryeducation.

R E C O M M E N D A T I O N 9 : T h u s , C T i n s t r u c t i o n s h o u l d n o t b e r e s e r v e d o n l y f o rt h o s e w h o p l a n t o a t t e n d c o l l e g e . h o r s h o u l d i t b e d e f e r r e d u n t i lc o l l e g e , s i n c e it i s n o t l i k e l y t o b e e f f e c t i v e i f i t w e r e .R E C O M M E N D A T I O N 1 0 : E x p l i c i t a t t e n t i o n t o t h e f o s t e r i n g o f C T s k i l l s a n dd i s p o s i t i o n s s h o u l d b e m a d e a n i n s t r u c t i o n a l g o a l a t a l l l e v e l s o f t h eK-12 c u r r i c u l u m . T h e c u l t i v a t i o n o f C T d i s p o s i t i o n s a n d a n i n s i s t e n c eo n g i v i n g a n d e v a l u a t i n g r e a s o n s , s h o u l d b e a n i n t e g r a l p a r t o fe l e m e n t a r y s c h o o l e d u c a t i o n . I n m i d d l e s c h o o l s a n d h i g h s c h o o l s ,


33/111

i n s t r u c t i o n o n v a r i o u s a s p e c t s a n d a p p l i c a t i o n s 0 4 C T s h o u l d b ei n t e g r a t e d i n t o a l l s u b j e c t a r e a i n s t r u c t i o n . S p e c i f i c c o u r s e s i n C Ta n d a n a d v a n c e d p l a c e m e n t e x a m i n a t i o n p r o g r a m i n C T f o r c o l l e g e b o u n ds t u d e n t s s h o u l d b e d e v e l o p e d . A l t h o u g h f o r g D o d r e a s o n s a t t h e p o s t -s e c o n d a r y l e v e l C T p r o g r a m s a r e g e n e r a l l y a s s o c i a t e d w i t h d e p a r t m e n t so f p h i l o s o p h y , n o a c a d e m i c u n i t s h o u l d b e r e s t r i c t e d i n p r i n c i p l e f r o mp a r t i c i p a t i n g i n a n i n s t i t u t i o n ' s C T p r o g r a m , p r o v i d e d t h a t t h e o v e r a l li n s t i t u t i o n a l p r o g r a m i n C T e q u i p s s t l l d e n t s t o a p p l y C T t o a b r o a dr a n g e o f e d u c a t i o n a l , p e r s o n a l a n d c i v i c s u b j e c t s , i s s u e s a n d p r o b l e m s .

There is growing evidence of the successes, both scientific andeconomic, of those industrialized democracies which emphasize demandingacademic assessment and ~et firm educational standards for career andprof,~ssional advancement. Assessment that counts is unquestionably a keyfac:blr in promoting academic achievement.

R E C O M M E N D A T I O N 11: T h u s , m l n l m u m C T p r o f i c i e n c y e K p e c t a t i o n s s h o u l d b es e t f o r e a c h e d u c a t i o n a l l e v e l , i n c l u d i n g p r o m o t i o n i n g r a d s , h i g hs : h o o l g r a d u a t i o n , c o l l e g e e n t r a n c e , a n d g r a d u a t e s c h o o l a d m i s s i o n .

The development of valid and reliable assessmeot strategies fromwhich teachers can dr.:lwreasonable inferences about; students' CT, incontrast to their domain-specific knowledge or other ~cademic:abilities(such as reading or writing>, is essential. CTassessment strategies,whether for Llse in the individual classroom or" for broader pL~rposes,must!lot simply rewar'd arriving at correct answers. They must. howeverrecognize achieving correct answers by way of good CT. The challenge ofCT assessment is not to let what is easily meaSLlred restrict our sense othe .fullness of CT. It would be shameful if those a.ssessment instruments",."hieh fl-lCUS CJnl on CT skills dr-ove OLlrCT cur r...cular design and causedthe d1sposition.:'\1 co-npcmerrt e of goad CT to be neglected.

R E C O M M E N D A T I O N 12: I n e v a l u a t i n g t h e a c c e p t a b i l i t y o f a C Ts t r a t ~ q y o r i n ~ t r u m e n t o n e s h o u l d c o n s i d e r c o n t e n t v a l i d i t y ,v a l i d i t y , r e l i a b i l i t y , a n d f a i r n e s s .a s s e s s m e n tc o n s t r L l c t

30


34/111

(I) Content Validity: The strategy or instrument should be based011 an appropriate conceptualization of CTand a clear understanding ofwhich aspects of CTthe assessment targets. Each task or question shouldbe evaluated to insure that correctly responding to that item is not amatter of rote learning or information recall. Whether for' the classroomor for broader educational purposes, CTassessment should includestrategies fer targeting CT's dispositional dimension as well as itsccgnitive skills dimension.

(2) Construct Validity: In acceptable CTassessment each task orqLtestian should have been evaluated to insure that students whoanswercorrectly do so on the basis of geod CT and that inadequate or wrongresponses euoethe result of weak or inadequate CT. Entire strategies orspeci fie: item~"ion whichgood CT leads to wrong answers, or poor CTtoright answers, should not be ,-,sed.

(3) Reliability: In acceptable CTassessment each task or questionshould l1ave been evaluated to insure that good critical thinkersgenerally do better on that item than weak critical thinkers. Ifdifferent persons are involved in evaluating the results, for examplegr-r:tdingessC\ys or judging presentations, the evaluations of the differentjudges should be cross-checked to assure that their findings arer-sliable, that is, generally consistent with one another. However, it isan open question whether the levels of achievement associated with thedHfer-ent CTsub-skills and affective dispositions are positivelycon-elated. Empirical r-esearch on how the sub-skills correlate with eachoUler and with vari.aus dispo'Sitions has yet to be undertaken. Thus, atthis lime, due cQLltionshould be exercised regarding howto interprett..~.:hnic:alflleCl'.iLtrl~Sf t:est-form reliability in the case of paper and

3l


35/111

pencil CT 4=1SSeSsmentnstruments.(4) Fairness: CT~ssessment should not unfairly disadvantage or

advantage groups of students on the grounds of reading ability, domain-specific knowledge [broadly understood as including the evidential,conceptual, methodological, critericlogical, contextual considerations,or familiarity with technical vocabularyl, gender or age related lifee)~periellce, ethnicity or socio-economic status, differences in socialnorms, or differences in cuttLlral assumptions. CTassessment locates CTtasks and questions in some assumed context, either subject-specific,ever-yday life, or fictional. Thus, guaranteeing that all students,regardless of their individual backgrounds, will ceme to the CTassessment on a perfectly equal basis in terms background knowledge,reading ability, lii=ee)~perienc:est etc. is impossible. However,e}~


36/111

i n s t r u m e n t s s h o u l d b e e m p l o y e d , d e p e n d i n g e n w h i c h a s p e c t o f C T i sb e i n g t d r g e t e d a n d w h e r e s t u d e n t s a r e i n t h e i r l e a r n i n g t h ei n t r o d u c t o r y s t a g e , t h e p r a c t i c e s t a g e , t h e i n t e g r a t i o n s t a g e o r t h eg e n e r a l i z e d t r a n s f e r s t a g e . A l t h o u g h t h e v e t e r a n C T i n s t r u c t o r i s a b l ~t o a s s e s s s t u d J n t s c o n t i n u o u s l y , C T a s s e s s m e n t s h o u l d b e m a d e e x p l i c i tt o r e i n f o r c e i t s w o r t h i n t h e e y e s o f t h e s t u d e n t s , t h e i r f a m i l i e s , a n dt h e p u b l i c . It s h o u l d b e m a d e e x p l i c i t t o s u p p o r t t h e g o a l s o fe d u c a t o r s s e e k i n g t o i m p r o v e t h e c u r r i c u l u m . A n d i t s h o u l d b e m a d ee x p l i c i t t o p r o p e r l y i n f o r m e d u c a t i o n a l p o l i c y f o r m a t i o n .

T h e C T I n s t r u c t o rR E C O M M E N D A T I O N 1 4 : T e a c h i n g C T i s m o s t e f f e c t i v e i f t h e i n s t r u c t o rm o d e l s C T d i s p o s i t i o n s a n d t h e p r o p e r u s e o f C T s k i l l s i n t h e v e r yp r o c e s s o f i n s t r u c t ~ o n . R e g a r d l e s s o f t h e s u b j e c t a r e a , s t u d e n t ss h o u l d b e e n c o u r a g e d t o b e c u r i o u s , t o r a i s e o b j e c t i o n s , a s k q u e s t i o n s ,p o i n t o u t d i f f i c u l t i e s i n t h e i n s t r u c t o r ' s p o s i t i o n . T h e s e o b j e c t i o n sa n d q u e s t i o n s s h o u l d b e c l a r i f i e d , i n t e r p r e t e d , a n d e x a m i n e do b j e c t i v e l y . S t u d e n t s s h o u l d b e g i v e n r e a s o n s f o r d o i n g t h i n g s ac e r t a i n N a y , r a t h e r t h a n b e i n g d o g m a t i c a l l y t o l d h o w t o d o t h e m .I n s t r u c t i o n s h o u l d b r i d g e t h e g a p b e t w e e n t h e s u b j e c t a n d t h e s t u d e n t ' so w n e x p e r i e n c e . I n t h e c a s e o f C T i n s t r u c t i o n , t h e t o p i c s o fd i s c u s s i o n s h o u l d n o t b e r e s t r i c t e d t o f a c t u a l m a t t e r s D r a c a d e m i cs u b j e c t s , b u t s h ~ u l d i n c l u d e i s s u e s w h i c h h a v e n o r m a t i v e , m o r a l ,e t h i c a l o r p u b l i c p o l i c y d i m e n s i o n s .

The ideal CT instruc:tor will integrate instruc:tion in CT in avariety of subject areas. She will teac:h specific CT skills directlyusing these subjects as content for the application of those skills. Shewill help students elaborate, transfer and generalize these skills to tE lvar"iety of contexts. She will create tE l classroom and school environmentwhich is supportive of CT. She will model CT in her teaching and herinteractions with colleagues. She will provide her stUdents withthought-provoking sL(bjects to learn about, and projec:ts to undertake.She will engage student's in social activities requiring them to reflecton, artic:ulate, shdF'e and di;planations andcontrasts in how they e>:ecuted var-ious CT tasks. She will evaluate eachstudeltt'!:i PI'"gress, ~chievement or proficiency in CT continuously.

, .R E C O M M E N D A T I O N 1 5 : F o r C T t o i n f u s e t h e K - 1 2 a n d c o l l e g e c u r r i c u l u m ,t e a c h e r " t r a i n i n g " s h o u l d g i v e w a y t o t e a c h e r " e d u c a t i o n . " I f t e a c h e r s

3 3'


37/111

~ r e t o m o d e l C T , s o m u s t t h o s e w h o h a v e a n i n s t r u c t i o n a l r o l e i nt e a c h e r p r e p a r a t i o n o r s t a f f d e v e l o p m e n t . I n a l l i n s t r u c t i e n , a n dp a r t i c u l a r l y i n C T i n s t r u c t i o n , b o t h f a c u l t y a n d l e a d e r s o f f a c u l t yd e v e l o p m e n t s h o u l d m o d e l C T . T h e y s h o u l d f o s t e r t h e s t u d e n t s 'c o n f i d e n c e i n t h e i r o w n p o w e r s o f r e a s o n , r a t h e r t h a n d e p e n d e n c y o nr e t e l e a r n i n g . T h e y s h o u l d n u r t u r e i n s t u d e n t s o p e n - m i n d e d n e s s ,a t t e n t i o n t o a l t e r n a t i v e s , a n d a s m u c h p r e c i s i o n o f t h o u g h t a s t h es u b j e c t a n d c i r c u m s t a n c e s p e r m i t .

VI -- The Delphi Research panel

The Delphi research findings reported here result from theparticipation of forty-six scholars, educators and leading figures in CTtheory and Cl' assessmant research. Roughly half the panelists areprimarily affilia.ted academically with Philosophy (S2X), the others areaffiliated with Education (22'Y.), the Social Scienc,es (207.),or thePhysical Sciences (6/.).

It would be a mi.stake to construe participation in this researchproject as implying that a person agrees with all the findings. Thus,where r..:onsensus is rsported a minority of panelists hold divergent views.Where near unanimity is reported a some panelists may may net be in fulla cc .o rd w iL h hew t,he specifics a r e expressed. In t h e e n d , however, afterreviewing the draft Delphi findings presented in Round 6, only one of thef(Jrty-si~ explicitly optecl to be listed as a pa,,..tic:ipant only, but not aswuppor t; ing the d o c u m e n t .

These D e l p h i findings fi>: an important moment in time. It is amoment w h e n the efferts of forty-six e x p e r t s p o s s e s s i n g s p e C i a lexperience and k n o w l e d g e i n m a t t e r s relating t o C T converged with a v i a wtOWai""'lj discovering i.f SCllne lne~asure of general accord could be found. Aswe mQ"e from t!If:?S'-tcc(~sses of the eighties into the decade of theni:l'~ties;, the person';; wl'\o par'ticipdted in this project hope that thefinding'-:,>f e;,~pe:-'tconsenSLtS reporled her'ein will advance criticalthinkiny and h~lp sh~pe the futUre of CT instruction and CT asseSSlilent.

'_) ~1.. .J ,


38/111

TA8LB1PARTICIPATING CRITICAL THINKING EXPBRTS

Jonathan A dlerDavid AnnisAmoldAroDlJames BellBarry K. BeyerCharles BlatzRob BradyNeil BrowneB ez C emmellS ODA rthur L CostaStan DundonRobert H. EnnisJames B . F re em anJac k F urlo ngE ugene G arverH. Scott HestevoldDa vid H itch co ckJohn HoaglundKenneth HoweRalpb H. JohnsonStuart KeeleyA nthony L aw sonM atthew L ipm anDavid S. MartinJohn MartinGary MatthewsS tu art M i lle rB rooke N oel MooreWayne NeukbergerStephen NorrisR ichard P ark erR ichard D . ParryR ichard P aulPhilip PecorinoWilliam RapaportPasqual Sc:hievellaZ ack S eec:bAnita SilversR icb ard S tig gin sRober t J. SwartzSteven TignerCaro l T uck erPerry WeddleRobert WengertMa rk We in steinP eter WmoQ ard

Philosophy B rooklyn C oU egePhilosophy Ball State UniversityPbysics University of WashingtonPsychology How ard C ommunity College, MDBducatioa G eorge M ason UniversityPhilosophy University of ToledoPhUosophy S te ts on Un iv en ityEconomics B owHn g G reen State UniversityCT Assessment American College Testing (ACT)Bduc:at ion Sacramento State UniversityPhilosophy Cal. Polytechnic Uaiversity, SLOBducat ion University of DlinoisPhilosophy Hunter CoUele, CUNYFreshman Studies Transylvania UniversityCritical Thinking Saint John'. UDivenityPhilosophy University of AlabamaPhilosophy M cM aster UniversityPhilosophy_, Chris topher Newport CollegeEducation U niversity of ColoradoPhilosophy University o f Wmdso rPsychology B ow ling G reen S tate UnivenityZoology Arizona State UniversityPhilosophy Montclair State CollegeEducation GaDaudet UniversityPbilosopby University of C"mcinnatiPhilosopby U. Massachuset ts , AmherstPsychology Towsen State UniversityPhilosophy CSU L'1ticoA ssm t. and Bval. O regon D epartm ent of EducationEducation Memorial University of NewfoundlandPhilosopby CSU ..OticoPhilosophy A gnes Scott C oU egePhilosophy Sonoma State UniversitySocial Sciences Oueensborough C. CollegeComputer Science SUNY BuffaloCouncil of Critical Analysis, Port Jefferson, NYBehavioral Science P alom ar C olle gePhilosophy San Pranclsc:o State UniversityNo rthw est R eg io nal Educational Laboratory, PortlandPhilosophy U. Ma ssachusetts , B ostonPhilosopby University of ToledoCT Assessment Bducational Testing Service (BTS)Philosophy Sacramento State UniversityPhilosophy University of IllinoisInstitute for CT Montclair State CollegeEducation' University of KentuckY

t 35 38


39/111

~.: _ ., :1 . ~.

APPENDIX AC om m er c ia l l y A ya i la b le OT A s se s sm e n t To o ls

Prepared for the APA Committee on Pre-College PhilosophyDelphi Researoh Project on CT Assessment

The Education Testing Service (ETS)AcademicProfile Test measuresthe academic abilities of CT, reading, wrtting, and using mathematicaldata, all within the context of three major academic areas --humanities, social sciences and natural sciences. This is a multiplechoice instrument with an optional critical essay (locally scored). Itassumes that studel1ts have completed most or all of their generaleducation -- that is, completed the sophomore college year.

ETSalso has sections targeting logical reasoning and analyticalthinking on several of its widely used instruments such as the (a) LawSchool Admissions Test, (LSAT)ilLogical Reasoning" section, (b) GraduateRecol'"dE>,amination, (GRE)-- General Test, the Analytical sec:tion, (c)AdvanceeJPl~~c:ementTest, subject-matter based CT questions, (d) NationalAssessment of Educational Progress, higher order thinking andlr.:lboratory-based questions, (e) Forei.gn Service Test, in-basket porti.on,~\nd(f) Graduate Record E~


40/111

1'1i5sion, I

8/7/2019 critical thinking DE

Documents

critical thinking DELPHI METHOD