Reliability of Selection Measures

Reliability Defined

The degree of dependability,

consistency, or stability ofscores on measures used inselection

Error of Measurement

Obtained Score

True Score

Error Score

Errors of Measurement

-Physical characteristics = Higher reliability (HR)

-Psychological attributes = Lower reliability (LR)

==================================Large amounts of measurement

error = LRSmall amounts of measurement

Error = HR

Methods of Estimating Reliability

Examines two sets of measure

If scores from the two measurements are similar, reliability will be raised

If scores from the two measurements are dissimilar, reliability will be lowered

Methods of Estimating Reliability

• Statistical procedures calculated reliability coefficients range from 0.00 to 1.00

• Higher the coefficient, the less the measurement error, and the higher the reliability estimate

• Lower the coefficient, the higher the measurement error, and the lower the reliability estimate

Methods of Estimating ReliabilitySelecting a method: Dependability of data

collected today be reflective of the same person in the future

To what degree do evaluations vary from one another

Accuracy of scores on measuring the true ability

Dependability of an assessment of a measure at a given moment

Methods of Estimating Reliability

Test-Retest Same measure is used tocollect data from the same person at two different times• The higher the coefficient, the

greater the approximation to the true score: closer to 1.00

• The lower the coefficient, the greater likelihood of error: closer to .00

Methods of Estimating Reliability

Test-Retest – Sources of error

Memory Learning“True change”

Methods of Estimating Reliability

Parallel or Equivalent FormsDoes not use the same measuretwice but will instead use equivalent versions of the

measure Each has same number of

questions Each has same level of difficulty Averages of scores are the same

Methods of Estimating Reliability

Internal Consistency EstimatesHow similar are different parts of

the measure (e.g. different questions on a test) in what they measure?

Split-half reliability Kuder-Richardson Formula 20 Coefficient Alpha (α) – most

commonly reported

Factors Influencing The Reliability of a Measure

Method of Estimating Reliability

Individual Differences Among Respondents

Length of a Measure• Test Question Difficulty• Administration of a

measure

Standard Error of Measurement

Useful applications:Forces us to think of

scores on a measure not as exact points

Aid in decision making in which only one individual is involved

Determine the magnitude of how scores differ significantly from one another