Validity And Reliabilty

This presentation was developed for the exclusive use of students enrolled in:

Educational Testing & Grading, Professor Gregory E. Stone.

© 2004 Gregory E. Stone. All rights reserved. This presentation may not be reproduced in any form, in part or as a whole, without the express written permission of the author.

Validity & Reliability

Exam ScoreQuality

No matter how well theobjectives are written,or how clever the items,the quality and usefulnessof an examination ispredicated on Validityand Reliability

Validity & Reliability

Examination ScoreQuality

Evaluative

Statistical

Validity & Reliability

Examination ScoreQuality

Validity

Reliability

Validity & Reliability

BOTH CONCEPTSRELATE TO THE

SCORE NOTTHE TEST!

Validity

Reliability

Validity & Reliability

Validity Reliability

Validity & Reliability

We don’t say “an exam is valid and reliable”

We do say “the exam score is reliable andValid for a specified purpose”

KEY ELEMENT!

Reliability“Was the measure of studentability reached by accident orchance, or was it reached througha clear and stable, meaningfulexamination?”

Validity & Reliability

ReliabilityAnswers questions relative tothe stability and clarity of anexamination.

Validity & Reliability

Validity & Reliability

ReliabilityReliability is a STATISTIC.

Specifically, a correlationranging from

0 = Completely Unreliable1 = Perfectly Reliable

Validity & Reliability

ReliabilityReliability is also a measure of

error within an examination.

Poor wording, ambiguity,correct answer issues, failure

to link with objectives, etc.

Validity & Reliability

Content/Action + Error

Validity & Reliability

Content/Action + Error

The information we seek and our best hope for obtaining it.

Validity & Reliability

Content/Action + Error

The information we seek and our best hope for obtaining it.

Our human frailty and inability to write effective questions.

Validity & Reliability

ReliabilityReliability comes in multiple

forms, including:

Test-Retest ReliabilitySplit-Half Reliability

Validity & Reliability

ReliabilityTest-Restest

Administer to students.After time, administer again.

Are the students performing ina consistent manner?

Validity & Reliability

ReliabilitySplit-Half

Create 2 smaller contentBalanced tests and administer.

Do students perform similarlyon each test half?

Validity & Reliability

Validity“Did the test we created reallymeasure what we hoped itwould measure?”

Validity & Reliability

ValidityRelates to questions about thereasonability of content,criterion and constructassociated with the learningobjectives and assessment.

Validity & Reliability

Validity & Reliability

Validity & Reliability

ValidityContent Related Validity

Does the content we have selectedappropriately represent ourconstruct (top level domain)?

Validity & Reliability

ValidityConstruct Validity

Is there really such a concept aswhat we created as a construct?(e.g. Does “reading ability” reallyexist?)

Validity & Reliability

ValidityCriterion Related Validity

Does our test adequately link thecontent to the construct? Is the test a good measure of the construct?

Validity & Reliability

Examination Quality

Validity

Reliability

Validity & Reliability

Validity & Reliability

RELIABILITYis a necessary but

not sufficient condition forVALIDITY