The Validity of Writing Two-Item Rules

8/6/2019 The Validity of Writing Two-Item Rules

http://slidepdf.com/reader/full/the-validity-of-writing-two-item-rules 1/18

The present s tudy involved the ~est ing of two commonmult ip le -choice i te~ wri t ing ru les . A recent review of researchrevealed tha t much of the advice given for wr it ing mult i? le -choicetes t i tems is based on experience and wisdom rather than on empir icalresearch . The rules assesseG in th is study inc lude: (1) t he ph ra si ng

of the s tem in the form of a ques t ion versus a par t ial sen tence ; and(2) the use of the inc lus ive "none of the a! : )ove"opt ion ins tead of aspec ir~c content option . Limited empirica l research suggests tha tu : : : ingthe par t ial sentence format and thp. i nclus ive "none of theabove" , - 'pt ionmay ' .ead t o undes i rab le i tem and tes t charac ten .s t ics ,whi le l~xtbook authors essent1a l ly are div ided on the i r opinionsabout t va: id i ty of each ru le. The items used in th is s tudy weref rom the inst ructor 's manual for D. Myer ' s (1986) te xt e nt it le d"Psychology." Items w~re randomly ass igned to be rewri t ten to ref lectthe exper imenta l condi t ions under invest igat ion. T. . . . o ins truc tors ofan in t roductory psycho:ogy course se lected 32 mUlt iple -choice i temsfor the s tudy . The rewri t ten tes ts wer n a dm in is ter ed to 228 s tudentsenro lled in two sect ions of an in t roductory psychology class. Aboutha l f of the s tudents in each sec t ion rece ived Form A and the otherha l f rece ived Form B, resu l t ing in 115 Form A and 113 Form Bresponses. The same manipula ted i tems were combined wi th 18 di ffe ren tnon-manipula ted i tems in a th i rd sect ion of the c lass to compr iseFo~ms C and D, whose adminis tra t ion resul ted in 59 Form C and 59 FormD responses . Resul ts of fer no eV1dence to support the use of e i thertype of s tem and limi ted evidence to caut ion agains t use of the "noneof the above" opt ion, Two data tab les and examples of the four i te:mformats used are provided . (TJH}

ED 307 298

AUTHORTITLEPUB DATENuTEPUB TYPE

EDRS PRIcr ;DESc. . :RIPTORS

IDENTIFIERS

ABSTRACT

DOCUMENT RE8UME

TM 013 3).5

Crehan , Kevin ; Haladyna, Tl lomas M.The Val id i ty of Two I tem-WriLing Rules .89l ap .Rep or ts - R es ea rc h/ Te ch ni ca l (143)

MFOl/PCOl Plus Postage.Col lege Students ; Higher Education; 1O:Mult1ple cnoaceTe st s; P sy ch ol og y; 1O: Te stC on st ru ct io n; Tes t Fo rm at ;Tes t Val id it y1O:Item rit ing Rules Para l lel Test Forms; StemAnalys is

1 0 : 1 0 :eproduct ions suppl ied by EDRS are the bes t tha t can be madef rom th e o ri gj : la l do cume nt .0 :

1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : ~ 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 t 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 t 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 : 1 0 :

1 0 :



__ __

The Val id i ty of Tvo I te s -Wrl t lnq Rule s

,

Kevin Crehan

Univ e r s i t y o f Nevada , as Vegas

a "d

Thomas H. Hal adyna

Ar i :ona S ta te Univer s i ty wes t campus

U.S DEPARTMENT OF EDUCATIONOffICE'of Educ a ti o na l Re se a rc h a nd Impr ov emen t

EDUCATIONAL R~SOLIRCES INFORMATION

CE'lTER (ERIC>~hlS doc\Jmenl has been reproduced as

r ec eiv ed fr om the p er son o r o rg an iz at io no ng l" )C lll nQ 1 1

[ ' Minor change~ have heen made 10Improver ep roduct io n Qua li ty

"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

k'FIJl IV - : : D .Ctc"I!1N

TO THE EDUCATION. \L RESOURCESINFORMATION CENTER (ERIC)"

• POints of View or opinions stated In Ih,sdocu-ment do n01 necessari ly represent off ic ia lOERI posu.on or coney

, ,cB ES T C O p y A V A ilA R !J : "

--- ~~--- -



The ~alldlty of ?vo I t ea -Wr l t l nq Ru l es

A re ren t r ev i ev o f re~ea r ch revea l ed t ha t auch of th e adv i ce

g iv en fe r vYi t i ng mul t i p l e - cho i ce t e s t i t ems i s based on

ex~e r i e nc~ and w isdom ra t~e r th an ea t r i ca l r e seaLch . The

pres en t s tudy i nvo l ved th e t e s t i ng o f two co~ac r i t em wr i t i ng

r u l e s : (1) the phr as ing of the s t em in the to ra o f a ques t i on

ve r s us a pa r t i a l s en t ence and (2 ) t he u se o f the inc l us iv e "none

of t he above " op t i on In s t ea r l o f a spec i f i c con t en t op t i on .

L imi t ed empi r i ca l r e se a rch sugges t s t ha t us i ng t he pa r t i a l

s en t ence fo rma t and the i nc lu s ive ' none o f t hese ' op t ion may lead

t~ undes i r ab l e i t em and t e s t ch a rac t e r i s t . i c s , wh i l e t e x tbook

au tho r s e s sen t i a l l y a re d iv i ded on the i r op in ions abou t the

va l id i t y o f each r l ~ l e . Resu l t s o f th i s expe r i . en t a l s tudy o f fe r

no ev i dence to suppor t t he us e o f e i t he r t ype o f s t em and l i a i t ed

ev id ence to cau t ion aga i ns t u s e t he op t i oL "none o f t he above .~

j



Tvo Ite. Wri t i ng Q~le5Pa~e 3

The Va l td l t y o f ! vo I t ea - I r l t i nq Rules

A nu .be r o f wr i t e r s in the f i e ld o f educa t i ona l ae asu r eaen t

h ave co • • ~n t ed tha t mu l t i J l e - c ho i ce (Me) i t e a wr i t i n g , desp i t e

i t s w id esp read popu la r i t y and u se , ha s r e ce i ved l i t t l e s cho l~ r ly

a t t e n t i on in t he pas t (C ronbach , 1970; Ebe l , 1951; Hi l l . an &

prac t i c e s , t he op t iAa l nu .be r o f op t ions and th e des i r a b i l i t y o f

k ey ba l anc i ng . Hos t i t em-wr i t i n g ru l e s have been s tud i ed fewer

t han 10 t i . e s . Thus th e eap i r i ca l f ounda t ion fo r the va l i d i t y o f

G reen , i n p re s s ; N i t ko , 1984; Ro i~ and Ha l adyna , 1982 ; Wes . a n ,

1971; Wood , 1977). In a r ev i ew o f emp i r i ca l r e se a rch on i t em

wr i t i n g , Ha l adyna and Do¥n ing (1989a ) r epo r t ed f i na~n9 96

eap i r i ca l s t ud i e s o f wh ich S3 dea l t w i t h on ly two I t em-wr i t i n g

. any i t em-wr I t i ng ru l e s i s weak , and t he bas i s fo r . any r u l e s i s

o f t en au tho r i t a t i ve wi sdoa pas s ed on th rough t ex t books and o th e r

I n a rev i ew o f 46 r e fe r enc es de a l i ng wi th the top i c He

pro fe s s ion a l pub l i ca t i ons and p re s en t a t i o ns .

The s t udy repo r t ed he re add re s se s two i t ee -wr i t i ng ru l e s

wh i ch a r e popu l a r l y p re s c r ib ed in t r ea t . en t s on He i t em wr i t i n g

in t ex t~ooks and o th e r sou rce s i n t he educa t i ona l measu re . en t

l i t er a tu r e (Ha l ady na & Downing , 1989b ). The f i r s t ru l e i s :

"Don ' t u se ' n one o f the above ' a s an op t ion~ i th e s econd ru l e Is

"Use e i t he r the que s t i on f o r . a t o r th e coap l e t t on fo rma t when

phr a s i ng the s t em. "

None of the A~Qye



Tvo I t e •• r l t l ~g Ru le sPage 4

E .p i r i ca l r e sea r ch on th i s iter vr t t i ng ru l e has b l en

th ese re f e r ences s t a t ed suppor t o r l ack o f suppor t fo r the "Don ' t

use ' non e o f the above ' a s an ~p t lon~ ru l e . Th i s vas the t en th.o s t o f t en aen t ioned ru l e , aad t h i s su r vey vas t aken as ev idence

o f the i apa r t ance o f the r u l e fo r i t em Yr i t e r~ . Hoveve r l au tho r s

ve re d iv i ded on t he i r suppor t fo r th i s ru l e , w i th 19 fo r and 1~

aga in s t . Obv ious ly soae con t rove r sy ex i s t s i n the va l i d i t y of

t he ru l e .

l i . i t e d to on ly t en s tud i e s (Boyn~ 'n , 19S0i Dudycha , ca rpen~e r ,

1973; For sy th & Spra t t , 1980; Hughes & Tr i ab l e , 1965; Muel l e r,

1915, Oos te rho f & Coa t s , 1984; Rimland , 1960; SchRe i se r &

Whi tney, 1975; Wes .an & Benne t t , 1946; Wil l i a . son & Hopk ins ,

1967). Al l o f the se s tud i e s i nvo lve J tho I t e . cha rac t e r i s t i c o f

d i f f i c u l t y , bu t on ly f ive s tud i ed I t ea d l s c r i . i na t i on and

re l i ab i l i t y , and on ly tvo va l i d i t y . In a l l i n s t an ces , t he u se o f

" none o f the above" op t ion aade i t e . s ao re d l f f i c u l t , t he aean

~f f e c t ac r03S n ine s tud i e s whe re re su l t s v e re ag9r eqab l e was

4.8\ . 11th d i sc r l a ina t i on , a vo id ing the i nc lu s i ve "none o f t he

above" op t ion . ade i t ea s s l i gh t l y ao re d i sc r l . i na t i nq , .03, whi l e

r e l i ab i l i t y v as i . p roved by a fac to r o f .04.

~ues t i on Fo r . a t V~rauB~p le t i on Fo rma t

One of th e .o s t funda . en t a l r egu l r6men t~ in HC i t em

wr i t i ng I s tha t one s t a t e s t he i t ea in a que~ t ion fo r . a t o r a

co . p l e t1on fo r . a t . On t he su r f ac e the re appea r s to be n~ r eason



Two I t e . Wr i t ing Ru le sPage 5

ia pr ov ea en t v as a aed ian .065, which i s a r educ t i on o f 6.5\ e r ro r

(1989a)~ the I '~ le I s one of th~ Bos t COBBon q iven In t r ea t . en t s

on He i te . w ri ti n9, 41 o f 46 ! e f e r ence s . en t i oned i t , and a l l 41

suppor t the ~8e of e i t he r fo r . a t . Pa radoxi ca l l y , t he a.all body

o f e . p i r i ca l re sea r ch l eads to the oppos i t e conc lus ion .

S tud ie s of t~ i s i t e . wr i t ing ru l e i nc lude : Boa rd and

Whi tney (1972) , Oudycha & ca rpen te r (1973 ), Dunn & Golds t e i n

( 195 9) , S c ha e is e r & Whi tney (1975a ; 1975b ), and Sch rock & Muel l e r

(1982 ). ~hese s ix s tud i es obse rved e f f ec t s on i t ea d i f f i cu l t y in

each in s t ance , d i sc r l a i na t i on in th r ee ca se s , r e l i ab i l i ty fC lu r

t i a es , and va l id i t y twice . I n gene ra l , t he ques t i on for aa t

~ppear s to have an advan tage ove : the sen t ence coap le t i on for . a t

wi th re spec t to Bak ing I t ea s s l i gh t l y eas ie r , hav ing l i t t l e o r no

e i fe c t on i te a d i s c r ia ina t ion , and aak lnq t es t scor es based on

such i t ea s ao re re l i ab le and va l id . Fo r re l i ab i l i t y , t he

va r i ance in te s t s co re s . Va l id i t y was i . p roved by . 06 in t vo

s tud ie s (Boa rd & Whitney , 1972 ; Schae i s e r & . h i~ney, 1975b) .

Based on tbe se few s tud i es , i t appear s t he ev idence favo rs the

use o f t he ques t i on for . a t over t he coap l e t ion for . a t In ph ' . a s i ng

the Me s t e a .

The p re sen t s tudy fu r t he r inves t i ga te s these two i t e . -

wr i t ing ru l es .

-b



TVo I tea WrIt Ing RulesPage 6

The i te~ used in this s tudy were f roa the ins t ruc tor ' s

. anua l for "yer ' s (1986) text en t i t led Psychology. Tvo

ins t ruc tor5 of an Int roducto r? 1~ycho loqy course se lec ted 32 HC

Iteas for the s tudy. Bach 1te . was keyed to the object ives of

the course and ae t the s tandard r~qul re . ent s for MC I te . wr it i ng.

Bach i te . a lso had adequate perfo raance characte r is t ic s as judged

f ro . previous uses . ! t e .s vere rando . ly ass igned to be rewri t ten

to ref lec t the exp~r i .en ta l aan ipula t lons as o ut l ine d bel ow:

Ho. of Vers ion 1 Version 2

l.te.s

8 coaple t ion coaple t ion

8

opt ion 'e ' (CI)

quest ion

none of these (CN)

coaple t lor.

8 quest ion quest ion

opt ion 'e' (Q8) opt ion 'e ' (CE)

none of these (OM) opt 1 on ' e ' (OR)

quest ionco.ple t lon

none of these (CN) none of thes~ (ON)

Figure 1 provides an exampl e o f one I tem wri t ten in al l four

var ia t ions .

- - - - - - - - - - - - - - - - ~ ~ ~ ~ ~ ~ - - ~ - -Inse r t Plqure Ina~~ut here

- - - - - - - - - - - - - - - - - - - - - - - - - - - -

. . . .,



Tvo I tea Wri t Ing RulesPaqe 7

~he aan ipu la t lons vere ba l anced both wi th in and be tween the tva

vers ions . V~r8ion 1 i t eas vere coab ined ~i th e igh teen non -

aan ipu la ted I te .a to coapr i se Po ra A of the f ina l exaa for tvo

~ec t lons of an in t roduc tory psychology c lae s ~hl1e Vers ion 2

i t eas vere eoab ined wi th the saMe eigh teen i teas to co .pr i se Fora

B. Tes t fo ras were key ba lanced wi th the opt i on 'none of the se '

be tnq keyed three t i . e s In s ix teen ap~earance8 o r approx iaa te ly

one- f i f t h of the t ! . e .

The tes t s vere ad . ln i . t e red to two sec t ions of the c lass

wi th approx laa te ly one-ha l f the s tuden ts in each sec t ion

~ece iv inq Fora A and the o ther ha l f rece iv ing For . B r~su l t ing in

,-,o

115 Fora A and 113 For . B responses . In addi t ion , t he saae

aan lpu la ted i te as vere coab ined wi th e igh teen di f fe ren t non-

aan lpu la t sd i taaa in a th i rd sec t lon of the c la ss to co .pr i s e

Fo ras C and D. Foras vere key ba lanced as abcve and t es t

ad . in i s t rQt ion 1n th i s c lass resu l ted in 59 Fara C and 59 Fora 0

responses .

~h is des ign vas chosen to a l low coapar i son of i t a . fo r . a t

aan l~u la t lons cont ro l l i ng for exaa inee ab i l l ty . Tha t I s , when

Vers ion 1 CE l tcas are co .b ined wi th Vers ion 2 OR Iteas , ve hav~

s ix teen i teas no t eap loy ing the opt ion 'none of these ' . When

Vers ion 1 O M l t eas a re co .b ined wi th Vers ion 2 CN It e .~ we have

these sa . e s ix teen i t e•• e .~ loy lnq the opt lon 'none of these ' .

I t ea charac te r i s t i cs can be c~ .pared be tween these s ix teen I te .



Tvo I te . Wr i t ing RulesPage 8

other of the e igh t l t ea subsca les under each condi t ion . S ince ,

a t bes t , aaa l l e f fec t s izes were an t ic ipa ted hypothes i s tes t inq

va . conduc ted wi th a lpha se t a t the .10 l eve l fo r each

s ta t i s t i ca l t es t .

RESUL%S

~able 1 resen t s the aeans and 3tandard dev ia t ions of i t ea

d i f f i cu l t i e s , aean pOin t -b i se r la l s and the KIJc le r -Rlchardeon 20

ze l l ab l11ty es t i aa t$s of each subsca le for the four foras of the

tes t .

Inse r t t ab le 1 about here

In order to tes t fo r d i f fe rences 1n di ff i cu l ty and

disc r la ina t lon for the ques t Ion versus coap le t ion for . a t I t~ .

s t a t i s t i c s fo r the For . A-OR ! t eas were coab ined wi th i t e .

~ ta t l e t i c s fo r the Pora B-QK Ite f t lSand were coapared to the 'o ra

A-eN i t ems coab lned wi th the Fora B-eB I t eas . S i_i l a r ly i t em

s ta t i8t i cs fo r the saae ! te . types on For . s C and D were

co .b ined . In order to tes t fo r d i f fe rences i~ dI f f i cu l ty and

dlsc r la ina t lon fo~ the inc lus ive versus spec i f i c op t ion

hypothes i s I t ea s ta t i s t i c s fo~ Fora A-CB I tG.s were c020ined wi th

Fora B-OK and ver~ co .pared to the Fora A-ON i teas co .b ined wi th

~ .



Tvo I te . Wrl t lnq RulesPage 9

'O~B B-CH I t eas . S ia l l a r ly~ 1tea s ta t i s t i c s fo r t h e s ane l t ~~

types vere co .bined on Poras C and D. SuaaaTY sta t i s t i cs fo r the

c~ .b lned I teas are presen ted in ?ab le 2.

- - - - - - - - - - - - - - - - - - - - - - - - -

Inse r t Tab l e 2 about here

il l

- - - - - - - - - - - - - - - - - - - - - - - - -

DI 'FICULTY

The observed d i f fe rence in d i f f i cu l ty vas .02 h iqher fo r the

ques t ion foraat . A cor~e la ted one- ta i l ed t - t es t shoved non

Bi9n i f i cance a t the . 10 l~ve l (t m . 56 , df = 1S , r •• 70, P c

. 29 ) . !h~ t - t es t fo r the saae co .par i son on Poras C and D shoved

s i . l l ar re5u l t s wi th a aean d i f fe rence of .003 and a non-

819ni f i can t t s t a t is t i c (t•• 10, df E 15 , r = .76, P = .46) .

Dif fe rences be tween u~ing and not us ing the op t ion tno~e of

these ' vas tes ted by co .b in ing Fora A CE wi th rora B QR i te .

d i f f i cu l t i e s a r id cOllpar ing these v i th Por . . A Q and Po~. B C i tea

d i f f i cu l t i e s . The d i f fe rence in .ean d i f f i cu l ty vas .027 vi th

use of 'none of these ' be ing lover . The depen4en t t - t e s t vas

s i~n i f i can t a t the .1 leve l (t ~ 1.44, df • 15 , r • . 916, ~ =

.OSS) . The saae tes t fo r For . s C and D had s i . l1a r r~su l t5 wi th

a aean 4if fe~ence of .043 (t = 1 .59 , df a 15 , r a . 67, p a .065) .

QISCRIHUfATIOft

Dif fe rencas in .ean pOint -b i se r ta l s be tween the ques t lon and



?VO !te . Wri t ing RulesPage 10

Differences In Bean point -blser ia ls hetveen using and not us ing

the Inclu81v~ 'none of these ' opt ion vere . 034 and . 033 for Por .1 va rora B and Pora C vs Fora D respect ively and f~vored not

aSiDCJ the Inc ' ,usive opt ion in both ins tances . , .he observed

dif .~erences , however , fa i led to reach signif icance at the .10

level . The corre la ted t - tests for rora A versus 'ora B and 'or .

DISCWfSIOI

C versus Pora D had p valuea of . 18 and .20 respect ively.

While th is s tudy fai ls to offer suppor t to a reco . . ~ndat ion

regarding use of ei ther the quest ion or coaple t ion foraat ovor

the other , observed resul ts regarding use of the -none o~ these-

opt ion are consis tent wi th previous f lndin~8 in direct ion and

aagnl tude. Dif!erences 1n dif ficul ty vere sta t i s t ica l ly

s ignif icant and in 3 to 4' range favor ing the speci f ic opt ion

over the inclus ive opt ion foraat . I te . d iscr i . lna t lons were also

observed to be s l19ht ly over . 033 hlgh~r for the speci f ic npt ion

for .at . This resul t , whi le not s ta t i s t ica l ly s ignif icant , Is a t

the saae level as observed in previous research. Lack of

s ta t i s t ica l s iC)nlflcance aay be at t r ibutable to the lOY power to

detect a di f ference of this 'aagni tude with s1xteen subjects

(l teas) ~nd the lov corre la t ions bety~en the ite . d lscr l .1nat lons

between for .s (.183, .488). I t i s noted that di f ferences ' In itea

J 1



~o I te . Wll t lnq Rul~8Pa<)e .&.1

dif ferences in re l i abi l i ty of about .04 favor ing use of the

spoci f lc opt ion ovor use of -none of these- . ru ture research ont h i s shou ld a~e the knovledge of th is ef fec t s ize t o det er . ine

the saaple s ize necessary to detec t ~ .03 or 9reater e f fec t v i th

reasonable pover .



10.

11.

12 .

Tvo t te. wr i t ing RuleoPaqe 12

- -.....a......-~.1 . Board, C. , Whitney, O. R. (1912)•. ~ effec t of se lec ted

poor Ite.- vJ: i t inC}pr,~ct ices on te at di ff ic ult y,

re l iabi l i ty , and val ic! ' , ty .ilmll;na}__of SdUCat_1o_··Ull.

t=D.JICeMot , L 125-233.

2. Baynton, H. (1~50). ~nclus lon of wnone of these- aakesapell tn~ i te .s aore di ff icul t . Idgcat lonal_6 PsychQlogicalHOllgrclCDt, ~ 431-432.

Cronbach, L. J. (1970). Reviev of on the ~heory ofachieve .ent t~s t I teaa by J .R . Bor.u th . fAYcboaotr lka , ~,509-511.

4. Our-n , Y . P. , I Goldste in , L. o . (1959). fe st d if fi cu lty ,valid i ty , and re l lahi l i ty as a function of a se l~cted.u lt ip le-ch~!ce i to. construct ion pr l rc l~lea . Idgclt lgnaland Psycholog ica l HAaBuro.ent , ~ 17~~179.

Dudycha, A. L., • carpenter, - J. 8. (1973). Bffects of i te •foraa ts on i te a d is ck ia in at io n and diff icul ty. Journa l ofAppl ied ~sycbQ1Q~ ~ 116-121.

Bbel , I . L . ( lS51) . wri t ing the te~t Ite •• In S. F.Lindquis t (Bd.) Bducatlooal "e.sure .~ (1st eel.) ( pp .185-249). Vashin9ton, DC: A•• r ican Counci l on Iducat i~n.

Feldt , L. 8. (1969)~ A t~8t of the hypothes i . tha tCronbach 's alpha or Kuder-Rlchardson coeff icient twenty isthe sa• • for test tests . 2 I J~bq . . t ' I ' a , ~ 363-373.

Forsyth , I . A. , Sprat t , I . P . (1980). Measur ing proble .solvlnq abi l i ty in .. the .. t l~ wi th .ul t lp le chol~ iteas.Yhe ~ffec t of i t•• foraa t on selec ted t te . and textcharacter ist ics. Jqgrnal o!_Jdgcat loDl l Mea'ure.ont , ' ~31-43.

,.. ,

6.

7.

8.

9. Guil ford , J . P. (1965). rDndl .~t l l s ta t i s t ic s lUfsycbology And lduCi t ion . l rd .d~r . ev York~ N.Y. , McOrav-Hil) . .

Hiladyna , T . M. ~ Dovnlny, 8. H. (1989. ) . The val id i ty ofa taxonoay of ' aul t l 'ple-cholce ita . wr i t ing rules . A»RlledMe.lgre.eDt In IdJu;:au..sm, 1..

Haiadyna, Y. H. , Downing, 8. H. (1989b) . A taxonoay of.u l t ip le-cholce I tem wri t ing rules. ApRl led MeasureMent ~ICdUCtlt 100., 1..

Hughea, H. H. p , Triable , . , a.' (1965) . f i le uae of cOIIplex·al tern .~ives In .v , l t ip le-cholce i tems. Idvcat lQoal and

13



Tvo Ito. I r l t ing RulesP.ge 13

Psycholggicai Macnu:eaanL 2~, ii7-iib.

13 . Nueller. D. J . ~197S).of co.plex alte~natlves

1 te . . . lCIucat lona! Iud141.

An assesa.eat of t he effect1venessl 'n aultlple choice achteV'IlMnt test

bycbo1gqical Ke .e qr e. at . . H, 135-

14. Killaan, J ., a Greene, J . (In press). !be speci f ica t ionand develo~nt uf t.st~ of achleveaent and abilities. InR. L. '"lnu (Ed.), lc2uc.tlooaL.lfeu_urellAD.t. (lrd ed.).W._sh!ngton, DC: &serlcan Council on Idncat lon.

15 . Hye r s , D . (1986)0 f ly cho lggx . "~York, •• Y •• ~~r thPubl l ihers .

23. We._h, A. G. i1S171). writing the t e a t Iteil. Irr I . L.'1' hoEmJl te (l Id .), lduc"ttoMI,tte".greaent (pp."-111).Washington, DC: Aaerlcan Coancl l on !dueation.

24. We a a a n . A. G., , Bennett, O. K. (1946). !be use of ' n o n f t

16. Mitto, A. J . (1984) . ~ook r ev i ev of Rold an~ H. l a~yna t 8 A, technolog1 for tes t - l te . vr l t lng . ~curnal of Iduc.tlonalMeasure.oot, ~ 201-204.

17. OOsterhof, A. C • .a Coats , P . K . (198.) . eoaparlsOft ofdiff icul t ies and rel lab111ty of quant i ta t ive vord proble . .in cOlIPle t lon and aultlple-cholee Ite. foraats. Appl iedbyeHlogleal MMlur_~-. L 8'7-2 '4.

18. Ri . land, B. (1960). "he'."ffects of varyln9 tiae ll.its andof ustng -right anaver not 9iven- in expert.ental for •• ofthe U. S. hvy ' l r i th .e t lc ! 'ellt . lduCtltioDll__andP8ych~c~1 NeI.uro.eDt~ ~ 5~3-539.

19. aold, G. }I~, , Haladyna, if. M. (1982). I tecbnol_o_a_y____f_Qz_t e e t - i t o . yr lt ing. Mev !or t , . 7: Acade. le Pres ••

20 • Schl ie iser, 'c. a., 'Whttney, ". It. ('1'154)• effect of tvoselected I t .a -wrl t lng practices on test difficulty,~l .cr l . lna t lon, and reliability. iOuroal of 'XQl r laen tAlUK.tiQD, ll, 30-34.

~1. Schaels.r, Cw Bv, , Whltney, D. a . t1975b) . Ybe effect ?fInCOMplete ate_ and -none of tbe above- fo1ls on test ah.J·it.a CMractez: 'atla. Paper presented at the elnnu,l lleetlnqof the .. tional Council on Measure.ent i n aducl :t tl on ,'.uhlngton, D.C.

22. 'Schrock, t'. J . , and Muel ler , D. J. (1982) . Bffects ofvtolatlnc) three aultlpl.-cholce t tea constz:uctioJ~principles. f be Journal ' of l duca \l oDa !_loaca rCb , ~ 314-311.



.._

es.

26.

Tvo Ite . Wxlt ln9 lulesPaqe 14

~f t b==c · eS .h Gp t l v " 1" t e . t conDt~vc t lo " . ~wir.l G'Iduc.tlon.l r eycbo logy . ~ 541-54'w

'Wl111a_oft, .... '·L., , RopkinlS', K. fl'. (1967). the UIIe of· noQ.-of- these~ versus hoaogeneous . l ternat ives on aul t ip le-choice tes ts ' : . • xperlaental r et la bl ll t, . a rJ v. ll dl tycOllparlsona. Jogr,.! of I dgca t l oo . l tleoaU(t:HaL .L. 53'-58.

food, R. (1917). Mult ip le choice: 1 s tate of the ar trepor t . IyllultloD in Iduc.tloDl I nt er na ti Qn al P rog re sa ,L 191-210.

15



- - - - - - - - - - - - - - - - - - - - - - - - - - ~ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Tvo Itea Wrl t ing RulesPa<j . 15

ftIL8 I.

.... (.) ... !Jt.DIHtt "vi.ta_ (.,. d cttlt-te.l't7laHce.,.,_,totat-.I"l'l,',:. to) -ftIIt .. 21 "Itabl ttty k) -tOI' nee. , l~••• alJcc:alo'acne" tnt' 'fo~.;

s ta t i s t lc Ite. 'or. A Ita. ,"01:. • rte. POl'. C Ite. Por. D7ype !'ype Type 'I'ype

P cs .105 eM .636 CB .733 CIt .634S .180 .209 . 136 01320 .379 .401 . 450 .38Sr . 535 . 591 . 619 .627

p QI .792 cs .798 QI .7 '8 CI .178S .140 . 130 .149 .143P > . 352 . 415 .355 .405r .469 .568 .528 . .66

P O if .806 aK .790 ON .731 01 .718S .092 .097 .134 .131D . 319 .to , ' .401 .402r .31e . 489 .611 .572

.p CII .622 'OIl . ' 66 CIt .667 Q It .653S . 217 .151 . 202 .1920 . 328 .419 .395 .41~r .385 .580 .549 .53'

1 6



~ I te . Vrltln9 Rulesraqe 16

.. _ .ad .taR'ltd' ... IKt_ 'foC' t .. 41fflewltta .... ~'.,.• bclt.1Mtl __ . ...s W'ltIl~ ... t_ted '~.lleMtty _ tIM ~l"

alxte_ It.. aal.. ~_...,. Ia. tJ'lN"

'rons f t • • Mean stand.Ed Mean Standard ttellabl1 t ty*7ype Diff . Dev la t,! ('J!'. Disc . Deviation

AI8 Q . 12 ' . 159 .316 . 11" .74l aB c· .710 . 1 '5 . 3 71 . 106 . 72CID Q .125 . 111 .384 . 12" . 14CaD c· . 722 . 171 .400 . .42 . 75

". I '.'1t1 ..t'U .39" .107 . 75AI . N .720 .1.79 . 360 .111 . 70caD I . 126 . 129 .426

.16 ' . 7 'c a D it .682 •.38 .393 . 125 . 75

-Rel iabi l i ty est laa te bI~ed on average poln t -b l se r l~ l s fo rs ix teen i teas af te r Gul1f~d (1965). ,

17



Two Ite. W%ltlng Rulesp~,.11

rt~. 1

fte toI1.. ,... ,••• ~•...,1. d \;1Ie her At.. loneta ...... 1.. ·_~tIIIe eel"ct_ l_t~ab.

(at) In the i r clanlc nln ... -year study, F rl ed . . n and Ro.eun foun4t l t a 1 : co.pet 1 t 1ve; h a r d - d r 1v 1ftc), ' l a p a t i - e n t , a n e t e a e l1yanqere4-1ndlvidaalB are especia l ly suscept ib le tor

.a. s ta . .ch ulcers •b . c a n C a r .

* c. bent a t tacka .d. accidents .e. non. of these

(OM) In thei r c l a s s i c nine-year .~ud7, r r iedaan and Roaeaanfound that coapet l t lve , hard-dr lvln9, i .pat lent , andeasi ly angered Indlvldu.l~ are especial ly saac@ptlb l~-to which of th.'-followll\CJ?

*

a . s toMach ulcersb . cancerc . s trokesd. ac:c tdentse . Done of t h e s e

(ca) In their c lasa lc nlne-year s tudy, Prledaan 'aDd Roseasn foundthat coapet l t lve, hard-drLvln9, i . ,a t l . . t , and eas i lyangered individual . are especial ly s u s c e p t i b l e to :

•• s toaach ulcers.b. cancer.

• c. heart a t tacks.d O . a cc ide nt ••e•• t ro ke ••

(QB) In their cla •• lc nine-yeer study, r r iedaan and 108e . . n foaDd'tha t co.petl t ive , "ha~d-dr lvin9, tapat lent , and eas i lya ng ere d I nd ivi du al s a r e e s p e c i a l l y s u s c e p t i b l e t o which o fthe fol lowing?

a . s toaacb ulcersb. cancer

* c. hear t a t tacks ·d. ' ace Id .n tse. s t roke .

13

Documents

The Validity of Writing Two-Item Rules