View
215
Download
0
Category
Preview:
Citation preview
Intelligence testing: A matter of life and death?
Chad W. BuckendahlBuros Center for Testing
University of Nebraska, Lincoln
Buros Center for Testing• Oscar K. Buros, 1935; UNL, 1979• Improve the science and practice of testing• Buros Institute of Mental Measurements
(BIMM): commercially available tests– Mental Measurements Yearbook (MMY)– Test in Print (TIP)
• Buros Institute for Assessment Consultation and Outreach (BIACO): proprietary tests– Psychometric services for educational
licensure, certification, admissions, and employment testing
Case Background
• Defendant (Erick Vela) participated in a bank robbery in Norfolk, NE in September, 2002
• Five people were killed during the robbery
• One of four defendants convicted• Motion asserting mental retardation
Legal Background
• Atkins v. Virginia (2002)– Prohibited execution of persons with
mental retardation• Additional Nebraska statute
– “An intelligence quotient of 70 or below on a reliably administered intelligence quotient test shall be presumptive evidence of mental retardation.”
Diagnosing Mental Retardation
• Relies on multiple criteria• American Psychiatric Association and
American Association of Mental Retardation:– Sub-average intellectual functioning,– Significant limitations in adaptive skills
such as communications, self-care, and self-direction, and
– Manifest before age of 18
Defining primary criteria
• Sub-average intelligence:– IQ 70 or below (2 std. dev. below mean)– Considers measurement error
• Adaptive functioning:– Coping with common life demands– Standards of personal independence– Socio-cultural background– Community setting
Intelligence tests
• Tests administered by three different defense experts:– Wechsler Abbreviated Scale of
Intelligence (WASI): July 2003– Wechsler Adult Intelligence Scales (3rd
ed.) (WAIS-III): November 2003– Stanford-Binet (5th ed.) (SB5): July 2004
Adaptive Behavior Tests
• July 2003: none• November/December 2003: none• 2004-05: Vineland Adaptive Behavior
Scales – Defense: defendant’s sister– State: two friends of the defendant
Malingering tests• Administered when there are concerns
about motivation/effort in performance– July, 2003: none– November, 2003:
• Test of Memory Malingering (TOMM)• Victoria Symptom Validity Test (VSVT)• Validity Indicator Profile (VIP)• 21-item test
– July, 2004: none
Professional Standards
• Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999)– Validity– Reliability– Administration– Scoring– Score Interpretation
Validity Argument
• “Ultimately, the validity of an intended interpretation of test scores relies on all the available evidence relevant to the technical quality of a testing system. This includes . . .appropriate test administration and scoring. . .”(Standards, p. 17)
Administration and Scoring
• “The usefulness and interpretability of test scores require that a test be administered and scored according to the developer’s instructions. . . Without such standardization, the accuracy and comparability of score interpretations would be reduced.” (Standards, p. 61)
Scores
• WASI: 87 (Full Scale IQ)– 82 (Verbal IQ), 94 (Performance IQ)
• WAIS-III: 75 (FSIQ)– 75 (VIQ), 78 (PIQ)
• SB5: 66 (FSIQ)– 56 (VIQ), 79 (Nonverbal IQ)
Interpreting scale scores
• Full scale score relative to average (mean) score for population (i.e. 100)
• Full scale score relative to the statutory cut score (i.e. 70)
• Expected range of full scale and sub-scale scores given defendant’s observed scores
Comparison of subject's observed WASI scores with the general population
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
50 60 70 80 90 100 110 120 130 140 150IQ Score
Den
sity
Overall WASI distributionFull Scale IQVerbal IQPerformance IQ
Men
tal R
etar
datio
n
Comparison of subject's observed WASI FSIQ score with a score of 70
0.00
0.05
0.10
0.15
0.20
50 55 60 65 70 75 80 85 90 95 100FSIQ
Dens
ity
Men
tal R
etar
datio
n
Given an observed WASI FSIQ score of 87, one would have an approximately 0.0000002% chance of having a true score of 70 or lower.
WASI caution
“[The WASI] should not be used alone to make diagnosis or educational placement decisions. It is not meant to replace more comprehensive measures of intelligence, such as the WISC-III and the WAIS-III."
(Technical Manual, p.8)
Comparison of subject's observed WAIS-III scores with the general population
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
50 60 70 80 90 100 110 120 130 140 150IQ Score
Den
sity
Overall WAIS-IIIFull Scale IQVerbal IQPerformance IQ
Men
tal R
etar
datio
n
Comparison of subject's observed WAIS-III FSIQ score with a score of 70
0.00
0.05
0.10
0.15
0.20
50 55 60 65 70 75 80 85 90 95 100FSIQ
Dens
ity
Men
tal R
etar
datio
n
Given an observed WAIS-III FSIQ score of 75, one would have an approximately 1.7% chance of having a true score of 70 or lower.
“. . .a low score on the WAIS-III does not necessarily reflect a low level of intellectual functioning. . . other factors such as the following may be implicated: cultural and linguistic discrepancy from the test’s standardization sample, disabling distractibility or anxiety, severe psychopathology, deafness, poor motivation or inadequate persistence, and extremely oppositional behavior or very poor rapport with the examiner. Before diagnosing low intellectual functioning or mental retardation, the examiner must rule out these factors.” (emphasis added)
Administration and Scoring Manual (pp.7-8)
Interpreting WAIS-III Scores
Comparison of subject's observed SB5 scores with the general population
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
50 60 70 80 90 100 110 120 130 140 150IQ Score
Den
sity
Overall SB5Full Scale IQVerbal IQNon-Verbal IQ
Men
tal R
etar
datio
n
SB5 VIQ/NVIQ Differences
• The defendant had a…• Full scale IQ (FSIQ) score of 66• Verbal IQ (VIQ) score of 56• Nonverbal IQ (NVIQ) score of 79• 23 point difference between VIQ and NVIQ
• Only 1.9% of the standardization sample had a difference of 23 or more between VIQ and NVIQ
“… when significant differences exist between the NVIQ and the VIQ, examiners should be cautious about evaluating FSIQ as a summary of an individual’s general ability level. . .When the context of the assessment and examinee’s background is influenced by such factors as communication disorders, learning disabilities, autism, or non-English background, the NVIQ may be a better indicator of global cognitive potential.”(emphasis added)
Examiner’s Manual (p. 134)
Interpreting SB5 Scores
Comparison of subject's observed SB5 NVIQ score with a score of 70
0.00
0.05
0.10
0.15
0.20
50 55 60 65 70 75 80 85 90 95 100NVIQ
Dens
ity
Men
tal R
etar
datio
n
Given an observed SB5 NVIQ score of 79, one would have an approximately 0.1% chance of having a true score of 70 or lower.
Full Scale Scores Over Time
50
60
70
80
90
100
110
120
130
140
150
WASI, 07/200
3
WAIS-III, 1
1-12/200
3
SB5, 07
/2004
Full
Scal
e IQ
Verbal Scale Scores Over Time
50
60
70
80
90
100
110
120
130
140
150
WASI, 07/200
3
WAIS-III, 1
1-12/200
3
SB5, 07
/2004
Verb
al IQ
Nonverbal Scale Scores Over Time
50
60
70
80
90
100
110
120
130
140
150
WASI, 07/200
3
WAIS-III, 1
1-12/200
3
SB5, 07
/2004
Perf
orm
ance
/Non
verb
al IQ
TestWASI FSIQ <.00001 Verbal IQ 0.06 Performance IQ <0.00001WAIS-III FSIQ 1.7 Verbal IQ 2.7 Performance IQ 1.9SB5 FSIQ 95.9
Verbal IQ 97.0Nonverbal IQ 0.1
Chance of true score being 70 or lower (%)
Summary of Intelligence Tests
Adaptive Functioning Information
• Vineland Adaptive Behavior Scales• School records
– Course grades, remedial programs• Observations
– Family members, teachers, inmates– Employment record, corrections officials
Defense approach
• Strict interpretation of the statute (i.e. reliably administered intelligence test)
• Suggested that accuracy (validity) could be obtained without precision (reliability)
• Consider the standard error in determining the range of possible true IQ scores
• Suggested that observed scores across tests were different due to the Flynn effect (i.e. intergenerational influence on norm sample)
Judge’s Ruling
• Presumption of MR rejected• Validity of presumptive score:
– 3rd intelligence test administered by defense
– Probability of 66 on SB5 was remote– No malingering tests administered on it– Examiner did not follow interpretation
directions in test publisher’s manual
Judge’s Ruling (cont.)
• Sub-average intelligence– Considered standard error– WAIS-III score met the criterion
• Adaptive functioning– Preponderance of evidence did not support
limitations in this area• Age of onset
– NE’s statute does not specify age• Defense motion denied
Future Research Questions• Measurement
– Application of standard setting methodology for cut score– Validity of score interpretations given intended uses– Instrument development (e.g., content, norm sample)
• Legal– Implications for future cases/legislation– Court’s application of current validity theory
• Psychological– Criterion definition of mental retardation (rather than norm)– Interpretation of intelligence as unique from achievement– Stability of intelligence as a construct
Recommended