Upload
mkhuram
View
219
Download
0
Embed Size (px)
Citation preview
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 1/25
What makes a selection What makes a selection
system work system work Reliability and Validity Reliability and Validity
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 2/25
A selection device (SD) is: A selection device (SD) is:
Anything used to pick one person over another Anything used to pick one person over anotherfor a job, including:for a job, including:
ResumeResume
Interview Interview
Intelligence testIntelligence test
Drug testDrug test
Skills testSkills test And a bunch of other things And a bunch of other things
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 3/25
A hiring system generally uses 2 A hiring system generally uses 2
or more selection devicesor more selection devices A typical selection system might include: A typical selection system might include:
Review resumeReview resume
Phone interview Phone interview
Interviews with HR & SupervisorInterviews with HR & Supervisor
Intelligence testIntelligence test
Drug testDrug test
Background check Background check
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 4/25
To use this system and these To use this system and these
selection devices they must be:selection devices they must be: LegalLegal ² ² no ¶unjustified· adverse impactno ¶unjustified· adverse impact
ReliableReliable ² ² produce consistent resultsproduce consistent results
Valid Valid ² ² measure what they are supposed tomeasure what they are supposed tomeasuremeasure
PracticalPractical ² ² cost effectivecost effective
Acceptable to the applicant Acceptable to the applicant ² ² lets not make thelets not make theapplicant madapplicant mad
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 5/25
CorrelationCorrelation ² ² used to ¶evaluate·used to ¶evaluate·
reliability and validityreliability and validity CorrelationCorrelation ² ² compares 2 sets of numberscompares 2 sets of numbers
(height & weight)(height & weight)
FromFrom ² ²1.00 to 0.00 to +1.001.00 to 0.00 to +1.00 Generally, we are interested in the 0.00 to +1.00Generally, we are interested in the 0.00 to +1.00
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 6/25
Perfectly positive correlationPerfectly positive correlation
(+1.00)(+1.00) +1.00 means that the two numbers are+1.00 means that the two numbers are perfectly perfectly
positively correlatedpositively correlated
If If X X goes up thengoes up then Y Y also goes upalso goes up If If X X goes down thengoes down then Y Y also goes downalso goes down
If If X X stays the samestays the same Y Y also stays the samealso stays the same
We We NEVER NEVER see this in the real world.see this in the real world.
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 7/25
Positive correlationPositive correlation
> +0.00 to +1.00> +0.00 to +1.00
If If X X goes up thengoes up then Y Y goes upgoes up
Statistical significance is strongly influenced by Statistical significance is strongly influenced by sample size:sample size:
A sample of <30 is questionable when calculating A sample of <30 is questionable when calculating
correlationcorrelation
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 8/25
What is reliability What is reliability
Degree of dependability, consistency or stability Degree of dependability, consistency or stability
of scoresof scores
UnreliableUnreliable ² ² I got a 32 on the ACT
in May andI got a 32 on the ACT
in May and21 in June. Something is wrong.21 in June. Something is wrong.
ReliableReliable ² ² get a 32 in May and a 31 in June.get a 32 in May and a 31 in June.
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 9/25
What causes errors What causes errors
The test itself The test itself
The situation (hot room) The situation (hot room)
The person (sick, bad day) The person (sick, bad day) Scoring errorsScoring errors
Administrative errors in giving a test Administrative errors in giving a test
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 10/25
Estimating reliabilityEstimating reliability ² ² externalexternal
methodsmethods Test Test ² ² retestretest
Take the same test at time1 and time2 Take the same test at time1 and time2
Correlate the scores at t1 and t2Correlate the scores at t1 and t2
ProblemsProblems ² ² time consuming & learning time consuming & learning
Parallel or equivalent formsParallel or equivalent forms
Take forms A and B of same test Take forms A and B of same test
Correlate the scores of A & BCorrelate the scores of A & B
ProblemsProblems ² ² time consuming & equivalency of formstime consuming & equivalency of forms
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 11/25
Estimating reliabilityEstimating reliability ² ² internalinternal
methodsmethods SplitSplit--half (could divide test into 2 sets of half (could divide test into 2 sets of
questions, or people into 2 sets, then correlatequestions, or people into 2 sets, then correlate
Half1 with Half2Half1 with Half2
IssuesIssues ² ² need sufficient sample size, and must splitneed sufficient sample size, and must split
truly randomly, so that halves are equivalenttruly randomly, so that halves are equivalent
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 12/25
Estimating reliabilityEstimating reliability ² ² internalinternal
methodsmethods K uderK uder--Richardson & Cronbach·s AlphaRichardson & Cronbach·s Alpha
For different types of measuresFor different types of measures
Basically Basically ² ² they calculate the average of all possiblethey calculate the average of all possiblesplitsplit--halveshalves
Requires a statistical package (SPSS)Requires a statistical package (SPSS)
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 13/25
Estimating reliabilityEstimating reliability ² ²interraterinterrater
agreementagreement Compares the agreement between (for example)Compares the agreement between (for example)
3 interviewers3 interviewers
If we rate on a scale of 1 to 10, 3 raters canIf we rate on a scale of 1 to 10, 3 raters candiffer, but we should not see a 9, a 5 and a 2 fordiffer, but we should not see a 9, a 5 and a 2 for
the same candidate.the same candidate.
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 14/25
R eliability and validityR eliability and validity
If a selection device is reliableIf a selection device is reliable
It may or may not be validIt may or may not be valid
If a selection device isIf a selection device is notnot reliable itreliable it cannotcannot bebe valid valid
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 15/25
V alidity V alidity
DefinitionDefinition ² ² the degree to which availablethe degree to which available
evidence supports inferences made from scoresevidence supports inferences made from scores
on selection deviceson selection devices
Hi ACT scores predict success in collegeHi ACT scores predict success in college
Meaning Meaning ² ² does the device measure what it isdoes the device measure what it is
supposed to measure?supposed to measure?
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 16/25
V alidating selection measures V alidating selection measures
CriterionCriterion--related validity related validity
PredictivePredictive
Concurrent
Concurrent
Content validity Content validity
Construct validity Construct validity
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 17/25
CriterionCriterion--related validityrelated validity ² ²
predictive validity predictive validity Give a test to applicants when hired andGive a test to applicants when hired and
compare to performance on the jobcompare to performance on the job
Correlate test with performance
Correlate test with performance
ProblemsProblems
Subjectivity of performance measuresSubjectivity of performance measures
T
ime lag from test to performance T
ime lag from test to performance
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 18/25
CriterionCriterion--related validityrelated validity ² ²
concurrent validityconcurrent validity Give a test to current employees whose jobGive a test to current employees whose job
performance is already knownperformance is already known
Correlate test with performance
Correlate test with performance
ProblemsProblems
Are current employees similar to applicants Are current employees similar to applicants
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 19/25
CriterionCriterion--related validityrelated validity ² ² To use To use
this the jobs must be:this the jobs must be: Reasonably stable over timeReasonably stable over time
Must have a bias free criterionMust have a bias free criterion
Must be a representative sampleMust be a representative sample Sample must be large enough to be stableSample must be large enough to be stable
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 20/25
Content validityContent validity
Based on the content of the jobBased on the content of the job
Must be representative of the content of the jobMust be representative of the content of the job
Needs a complete job analysisNeeds a complete job analysis It is a miniIt is a mini--job samplejob sample
Develop using Subject Matter Experts (SME·s)Develop using Subject Matter Experts (SME·s)
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 21/25
Construct validityConstruct validity
Measuring an abstract concept such as:Measuring an abstract concept such as:
intelligence or sociability intelligence or sociability
We may measure intelligence with an IQ test, We may measure intelligence with an IQ test,but does it really measure the constructbut does it really measure the construct
(intelligence)?(intelligence)?
This creates a complex situation to analyze This creates a complex situation to analyze ² ² letslets
leave it at that.leave it at that.
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 22/25
V alidity generalization V alidity generalization
Applying a validity study from one situation (job Applying a validity study from one situation (job
or company) to another.or company) to another.
Burden of proof for applicability is on the userBurden of proof for applicability is on the userof the selection device.of the selection device.
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 23/25
Is the selection device practicalIs the selection device practical
Refers generally to cost effectiveness of theRefers generally to cost effectiveness of the
measuremeasure
Depends greatly on the nature of the jobDepends greatly on the nature of the job
8/8/2019 What Makes a Selection System Work
http://slidepdf.com/reader/full/what-makes-a-selection-system-work 24/25
Is the selection device acceptableIs the selection device acceptable
to job applicantsto job applicants Less crucial in a ¶tight· job marketLess crucial in a ¶tight· job market
If I find a drug test insulting will I turn downIf I find a drug test insulting will I turn down
your job if I have a second offer that does notyour job if I have a second offer that does notrequire a drug test?require a drug test?