Writing Effective Tests

8/14/2019 Writing Effective Tests

1/28

Writing Effective Tests

The following slides summarize the workof Lucy Jacobs and Clinton Chase fromtheir book Developing and Using Tests

Effectively: A Guide for Faculty

The book is a compilation of years of

research on how to develop effectivetest instruments.


2/28

Writing Tests

Most higher education faculty never took acourse in how to write tests. Their experiencecomes from the many tests they have taken overtheir many years in school. This personal

experience is not a reliable way to learn todevelop tests that will determine so manyimportant outcomes for our students.

Even with diligence, time and practice writingtests, most tests given in college haveweaknesses and are not highly reliable.

However, the following presentation can

significant help to improve the quality of the tests


3/28

Writing Fair and EffectiveTests

Purpose ofTests

Grades Planning content

presentations

Facilitate learning

Motivate students

Benefits toStudents

Learn how theyare tested

Engage in deep orsurface learning

based on tests Feedback on

learning


4/28

Test Benefits to Instructors

Focus on essentialcourse content

Message to

students on whatsimportant

Valuable feedback

on learning

Feedback if courseobjectives wereachieved

Feedback oneffectiveness ofinstruction

A motivationalstrategy


5/28

Problems with College Tests

Dont focus onwhats important

Too little feedback

given to students Questions

ambiguous or

unclear

Too short tosample contentfairly

Purpose of test notmade clear-sostudents see it as

penalToo few tests-

dont have a broadview of students

knowledge


6/28

How Long Should a Test Be?

True-False 30sec.

Multiple choice 1

min. Completion 1

min.

Short answers 2min.

Multiple choice90sec. higherlevel thinking

Matching 30sec.

Short Essay

10-15 minutes.

Extended essay

30 minutes


7/28


8/28

Test Item Difficulty

Tests shouldproduce a spreadof scores

Test items shouldbe of averagedifficulty

Items that 50-70%of the studentscan answercorrectly

If they didnt study

Difficulty can be:

A. content it asks

B. cognitive skilllevel beingmeasured

C. Construction ofthe

item


9/28

Test Reliability

Long enough tosample contentwell

Most studentshould be able tofinish

Score rangeshould be wide

Items free ofambiguity and

tricks

Directions shouldbe clear.

Scoring procedure

needs to be free oferrors.

All students should

take the same test


10/28


11/28

Multiple Choice Tests

Write a stem in theform on anincomplete

statement orquestion thatcould be answeredwithout look at the

four choices Always put a verb

in the stem

Do not pad thestem-write themsimple and clear.

Distracters mustbe plausible

Avoid using humor

Make certain thereis only one bestanswer


12/28


Make distractersfairlyhomogeneous-this

will force studentsto bediscriminating

Dont giveirrelevant clues ortricks-thesereduce validity

Make answers

Avoid givinggrammar clues

careful with

specific modifiers--ie. Never, always

Limit use of all of

the above andnone of the above


13/28


Arrange options inlogical(alpha )order

Keep correctresponse letter( number)balanced

Dont lift stemsverbatim from text

Arrange answers

in vertical columns

Avoid asking trivia

Ask students tochoose the best

answer not thecorrect answer.


14/28

True-False Tests

Advantages

A. Sample manymore bits of

information B. Ask more

question in 50

minute timeperiod helpsreliability

C. Research

affirms True-False

Limitations

A. Difficult to writeso there not

ambiguous orobvious

B. Difficult to make

clearly true orclearly falsestatements

C. 50-50 chance

D. Educated guess


15/28

True-False Tests

Avoid use ofspecificdeterminers ( all,

never) Avoid use of

qualifiers(sometimes, usually)

Avoid use ofindefinite terms(long time ago)

Write questionsthat are clear andsimple to read.

Make it clearwhere the answersare to be placed--dont use + or -


16/28

True-False Higher LevelThinking Use propositional

logic--if thenapproach.

Example-If thegross nationalproduct goes upand inflation is3.4%then thestock market willlikely go up.

Setting up aproblem thatrequires analysis.

Example --Johnbought a caritwont start. Whichare possiblereasons?


17/28

Short Answer or CompletionItems Requires recall

rather thanrecognition of

information. Doesnt test

higher levelthinking

Best used to seewhat informationstudents have

collected.

Questions shouldhave a single wordanswer.

Make certain thereis only one rightanswer

Delete only keywords

Make blanks samelength

Dont lift


18/28

Matching Tests

Stimulus columnon the leftnumbered--

response columnon right lettered.

Provide space toright answer to theleft of stimuli.

Use onlyhomogeneous

material

Put stimuli inalphabetical order

Use 10-15 items

Keep entire test onone page.

Have at least five

more stimuli thanresponses


19/28

Essay Tests

Advantages

1. Can assesscomplex learning

2. Easy to write 3. Emphasize

communication

skills asfundamental

4. Cant simplyrecognize ananswer

5. Doesnt allowfor guessing--except bluffing

6. Can see howstudents select,organize andevaluate ideas and

apply them


20/28

Essay Tests

Limitations

1. Difficult to score

2. Scores are less

reliable 3. Sample only a

limited amount of

content

4. Scoreinfluenced byreaders overall

impression of thestudent

5. Not favorable todeveloping goodwriting skills


21/28

Essay Reliability Concerns

Less reliable thanobjective tests

Environmental

factors influencescoring ie. Time ofday, mood ofreader where

paper is in thestack Previous paper can

influence current

paper

A second readingoften results in adifferent grade.

Expectations ofstudentsperformance

Physical elements

of the paper--handwriting

Limited number ofquestions luck of

the draw


22/28

Making Essay Tests Better

Restrict essays toassess onlyoutcomes that

require complexhigher levelcognitivefunctions.

Examples 1. Compare

-Contrast

2.Present

Keep questionfocused on asingle objective

All students shouldrespond to thesame questions-dont give choices

Grammar andspelling count onlyif taught as an

objective in the


23/28

Making Essay Tests Better

Directions need tobe very clear andinclude type of

responseexpected--Exampleoutline, list,discussion

List the point valueof each question


24/28

Scoring Essay Tests

Conceal studentsnames

Skim a few before

starting to gradeto get a sense ofresponses

Read only oneitem across allpapers

Reshuffle the stack

Use a prescribedreading procedure

A. Key procedure

lays out ideas thatshould have beenin the essay

B. RankingProcedure

Makes piles basedon quality


25/28

Open-Book Tests

Advantages

Used in classeswhere formulas,

tables or graphsare needed to findsolutions

Studentdemonstrates theyknow how to usereferences

Reflect real life

Limitations

Students tend notto prepare well

No evidence tosuggest studentsdo any better


26/28

Take Home Tests

Advantages

1. More thoroughexploration of

ideas can takeplace

2. No time

restrictions 3. Usually better

written

Limitations

1. Question rarelysample adequately

course material 2. Much more

grading time

3. Students oftenunsure of howthorough to be.


27/28

Re-takes of Tests

Advantages

Retakes do resultin better learning

Lower test anxiety Students learn

from their

mistakes It eliminates the

need for make-uptests

Limitations

A large test bankis needed

Grading timeincreases

Students dont

often prepare wellfor the first test


28/28

Helping Students Prepare forthe Tests

Review the scopeof the test

Use a practice test

Clarify time limits Clarify what

material will be

needed andallowed

Review gradingprocedure

Review make-uppolicy

Provide a review

session Make provisions

for last minute

questions Allow for breaks

during long exams

Documents

Writing Effective Tests