Microsoft PowerPoint - TSL752-ch4.pdf

7/27/2019 Microsoft PowerPoint - TSL752-ch4.pdf

http://slidepdf.com/reader/full/microsoft-powerpoint-tsl752-ch4pdf 1/45

Trends in Testing, Assessmentand Evaluation

Chapter 4Test Construction and

Administration



Guiding Principles for Evaluation• Evaluation should relate directly to instructional

objectives

• Each evaluation activity should be designedto promote student growth

– The actual activity should be useful practice in itself

– Feedback should be useable by the student• Multiple evaluation strategies should be

provided to master achievement of X

objective/competency

• Student should clearly understand themethods of evaluation for X test or activity



Questions to Ask yourself inDesigning a Test

• What objectives will (should) I

be testing?

• What types of items will be

• How long will the test be in

terms of time and number ofitems?

• How much will each objectivebe worth in terms of weighting

and number of items?



Tests as Diagnostic Tools

• Students demonstrate

learning• Instructor effectiveness –

modify teaching strategies

or activities• Assignment of letter

grades



Different Types of Tests & Learning

• Paper & Pencil/ WebCT

Testing – Limited Choice Questions

(MC, T/F, Matching)

– Open-Ended Questions(Short Answer, Essay)

• Performance Testing

– Acquisition of skills that canbe demonstrated through

action (e.g., music, nursing,etc.)



Planning a Test

First step: Outline learning

objectives or major conceptsto be covered by the test

– Test should be representative of

objectives and material covered – Major student complaint: Tests

don’t fairly cover the materialthat was supposed to be

canvassed on the test.



Second Step: Create a testblueprint

Planning a Test

Third Step: Create questionsbased on blueprint – Match the question type

with the appropriate level of

learning



Fourth Step: For each check

on the blueprint, jot down(might use 3x5 cards) 3-4alternative question on

Planning a Test

which will get at the sameobjective

Fifth Step: Organize questionsand/or ideas by item types



Sixth Step: Eliminate similar

questionsSeventh Step: Walk away

from this for a couple of

Planning a Test

daysEighth Step: Reread all of the

items – try doing this from

the standpoint of a student



Ninth Step: Organize

questions logicallyTenth Step: Time yourself

actually taking the test

Planning a Test

and then multiply that byabout 4 depending on thelevel of students

Eleventh Step: Analyze theresults (item analyses)



Sample test blueprint



Translating CourseObjectives/Competencies into

Test Items• Syllabus

– Specification table- what was taught/weight areaso e es e

• Creating a Test Blueprint – Blueprint- this is the test plan, i.e., which questions

test what concept

– Plotting the objectives/competencies against somehierarchy representing levels of cognitive difficultyor depth of processing



Thinking Skills• What level of learning

corresponds to the course

content

• Bloom’s Taxonomy of

– Knowledge (see handout)

– Comprehension

– Application

– Analysis

– Synthesis

– Evaluation



Practical Considerations

• Representative sample of

the course content notrandom– purposeful based

on blueprint

• Representative sample ofskill or cognitive levelsacross content

• Analyze results by levelAND content area



Question Arrangement on a Test

• Group by question type

– Common instructions will save reading time

• Limit the number of times students have

• Patterns on test must be logical

– Arrange from a content standpoint

– Keep similar concepts together

• Group by difficulty (easy to hard)



Selecting the Right Type ofevaluation

• How do you know what type of question

to use and when?• It depends on the skill you are testing.

• Evaluation should always match as

c ose y as poss e e ac ua ac v yyou’re teaching. – Examples: Teaching Speech, should

evaluate an oral speech

– If testing ability to write in Spanish, better give an essay.

– Testing reading –MC, TF

– Wouldn’t use MC to test creative writing



Question Types versesCognitive Levels of Learning

Knowledge

Comprehension

Application Analysis

Synthesis

Multiple Choice (MC)

True/False (TF)

MatchingCompletion

Short Answer

MC

Short Answer

ProblemsEssay

Performance

MC

Short Answer

Essay



Constructing the Test

• Types of Test Questions:

– Multiple-Choice Items

– True-False Items

– Fill-In-blank

– Short-Answer Items

– Essay Questions



Multiple Choice Items• Advantages:

– Extremely versatile-can measure the higher level mental processes (application,

analysis, synthesis and evaluation)

–

answer/essay and T/F item – Can cover a wide range of content can be

sampled by one test

• Disadvantages – Difficult to construct plausible alternative

responses



Types of Multiple Choice Items

• Four Basic Types

– Question Type – Incomplete Statement Type

– Right Answer Type

– Best Answer Type• Which Type is Best?

– Question Type vs.

Incomplete Statement – Right Answer vs. Best Answer

Type



Multiple Choice Items

1. Writing the stem first:

A. Be sure the stem asks a clear question

B. Stems phrased as questions are

C. Stems should not contain a lot ofirrelevant info.

D. Appropriate reading level/terms

E. Be sure the stem is grammaticallycorrect

F. Avoid negatively stated stems



• Writing the correct response

– Use same terms/reading level

– Avoid too many qualifiers

Assi n a random osition in the


answer sequence• Read the stem and correct

response together

• Generate the distractors/alternative responses



Other Tips for Constructing MCItems:

– Items should have 3-4 alternatives.

– Stem should present a single, clearlyformulated problem


– Simple, understandable, excludeextraneous words from both stemand alternatives

– Include in the stem any word that

are repeated in each response – Avoid all of the above (can answer

based on partial information)

– Avoid none of the above



• Alternative responses/distractors should be

plausible and as homogeneous as possible• Response alternatives should not overlap

– Two synonymous terms (arithmetic average/mean)


– None of the following are part of the brain exceptwhich one?

• Emphasize negative wording

• Each item should be independent of other items in the test

– Information in the stem of one item should NOT helpanswer another item.



True-False Test Items• Best suited for testing 3 kinds of

info.• Knowledge level learning

• Understanding of misconceptions

• When there are two logical responses

• van ages: – Sample a large amount of learningper unit of student testing time

• Disadvantages:

– Tends to be very easy – 50-50 chance of guessing

– Tends to be low in reliability



Tips for Constructing True/FalseItems

– Tips for constructing True-False Items

• Avoid double negatives

• Avoid long or complex sentences

• Specific determiners (always, never, only,etc.) should be used with caution

• Include only one central idea in eachstatement

• Avoid emphasizing the trivial

• Exact quantitative (two, three, four)language is better than qualitative (some,few, many)

• Avoid a pattern of answers



Objective Test Item Analyses

• Evaluating the Effectiveness of

Items. – Why?

• Scientific way to improve the quality of

• Identify poorly written items whichmislead students

• Identify areas (competencies) ofdifficulty

– Item analyses provided info. on:• Item difficulty

• Item discrimination

• Effectiveness of alternatives in MC Tests



Short-Answer Items• Two Types: (Question and Incomplete

Statement)

• Advantages:

– Easy to construct

– Excellent format for measurin who, what,

when, and where info. – Guessing in minimized

– Student must know the material- rather than simply recognize the answer

• Disadvantages: – Grading can be time consuming

– More than one answer can be correct



– Better to supply the term and require a

definition – For numerical answers, indicate the

degree of precision expected and the

Tips for Constructing ShortAnswer Items

.

– Use direct questions rather thanincomplete statements

– Try to phrase items so that there is onlyone possible correct response

– When incomplete statements are used,do not use more than one blank withinan item.



Essay QuestionsTypes of Essay Questions

• Extended Response Question

– Great deal of latitude on how to respondto a question.

– Example: Discuss essay and multiple- c o ce ype es s.

• Restricted Response Question

– More specific, easier to score, improvedreliability and validity

– Example: Compare and contrast therelative advantages of disadvantages ofessay and multiple choice tests withrespect to: reliability, validity, objectivity,& usability.



Essay Items• Advantages:

– Measures higher learning levels(synthesis, evaluation) and is easier to construct than an objective testitem

Students are less likely to answer anessay question by guessing

– Require superior study methods

– Offer students an opportunity to

demonstrate their abilities to:• Organize knowledge

• Express opinions

• Foster creativity



• May limit the sampling of

material covered• Tends to reduce validity of the test

• Disadvantages

Essay Items

Subjective unreliable nature of

scoring

• “halo effect” – good or bad student’sprevious level of performance

• Written expression• Handwriting legibility

• Grammatical and spelling errors

• Time Consuming



• Give students a clear idea of thescope & direction intended for the answer – Might help to start the question with

the description of the required

Essay Questions

. ., ,

• Appropriate language level for students

• Construct questions that requirestudents to demonstrate acommand of background info,but do not simply repeat thatinfo.



• If question calls for an opinion, besure that the emphasis is not onthe opinion but on the way itspresented or argued.

Essay Questions

• Use a larger number of shorter,more specific questions rather than one or two longer questions

so that more information can beassessed.



• You might

– Give students a pair of sampleanswers to a question of thetype you will give on the test.

Essay Questions

– Sketch out a rubric (gradingscheme) for each questionbefore reading the papers ORrandomly select a few to read

and make up the gradingscheme based on thoseanswers

– Give students a writing rubric



– Detach identifying informationand use code numbers instead toavoid letting personality factorsinfluence you.

Essay Questions

– After grading all the papers onone item, reread the first few tomake sure you maintainedconsistent standards

– Be clear to student the extend towhich factors other than content(e.g., grammar, handwriting, etc.)will influence the grade.



– Provide reasonable time limits for each

question

• “thinking and writing time”

– Avoid permitting students a choice of

uestions

Tip for constructing EssayQuestions

• Will not necessarily get a representative sample ofstudent achievement. Only be requiring allstudents to answer all questions can their achievement be compared

– A definite task should be put forth to thestudent

• Critical words: compare, contrast, analyze,evaluate, etc.



Scoring Essay Items

• Write an outline of the key points (use outline

to design a rubric)• Determine how many points are to be

assigned to the question as a whole and to thevar ous par s w n .

• If possible, score the test without knowledge ofthe student’s name

– Face Sheet

• Score all of the answers to one question beforeproceeding to the next question

– Consistent standard



Scoring Essay Exams

• If possible, score each setof answers within the same

time frame

• Handwriting, spelling &Neatness

– Two separate grades?

• Mastery of material• Other



Alternative Methods of Assessment

• Research/Term Papers

• Research Reviews• Reports

• Case Studies

• Portfolios

• Projects

• Performances

• Peer evaluation

• Mastery

• Simulations



Preparing rubric



Item Analysis



Item difficulty



Cheating

• Preventing Cheating

– Reduce the pressure(multiple evaluations)

– Make reasonable demandseng con en o exam

– Use alternative seating

– Use alternative forms

– Be cautious with extra copies



Using Assessment & Evaluationto Improve Student Learning

Outcomes

• Providing feedback to

• Closing the assessment &evaluation loop

• Maximizing student

learning

Documents

Microsoft PowerPoint - TSL752-ch4.pdf