Validity And Reliabilty

This presentation was developed for the exclusive use of students enrolled in:

Educational Testing & Grading, Professor Gregory E. Stone.

© 2004 Gregory E. Stone. All rights reserved. This presentation may not be reproduced in any form, in part or as a whole, without the express written permission of the author.

Validity & Reliability

Exam ScoreQuality

No matter how well theobjectives are written,or how clever the items,the quality and usefulnessof an examination ispredicated on Validityand Reliability


Examination ScoreQuality

Evaluative

Statistical


Examination ScoreQuality

Validity

Reliability


BOTH CONCEPTSRELATE TO THE

SCORE NOTTHE TEST!

Validity

Reliability


Validity Reliability


We don’t say “an exam is valid and reliable”

We do say “the exam score is reliable andValid for a specified purpose”

KEY ELEMENT!

Reliability“Was the measure of studentability reached by accident orchance, or was it reached througha clear and stable, meaningfulexamination?”


ReliabilityAnswers questions relative tothe stability and clarity of anexamination.



ReliabilityReliability is a STATISTIC.

Specifically, a correlationranging from

0 = Completely Unreliable1 = Perfectly Reliable


ReliabilityReliability is also a measure of

error within an examination.

Poor wording, ambiguity,correct answer issues, failure

to link with objectives, etc.


Content/Action + Error



The information we seek and our best hope for obtaining it.



The information we seek and our best hope for obtaining it.

Our human frailty and inability to write effective questions.


ReliabilityReliability comes in multiple

forms, including:

Test-Retest ReliabilitySplit-Half Reliability


ReliabilityTest-Restest

Administer to students.After time, administer again.

Are the students performing ina consistent manner?


ReliabilitySplit-Half

Create 2 smaller contentBalanced tests and administer.

Do students perform similarlyon each test half?


Validity“Did the test we created reallymeasure what we hoped itwould measure?”


ValidityRelates to questions about thereasonability of content,criterion and constructassociated with the learningobjectives and assessment.




ValidityContent Related Validity

Does the content we have selectedappropriately represent ourconstruct (top level domain)?


ValidityConstruct Validity

Is there really such a concept aswhat we created as a construct?(e.g. Does “reading ability” reallyexist?)


ValidityCriterion Related Validity

Does our test adequately link thecontent to the construct? Is the test a good measure of the construct?


Examination Quality

Validity

Reliability



RELIABILITYis a necessary but

not sufficient condition forVALIDITY

Business

Validity And Reliabilty