NEO-PI-R Validity Scales

8/10/2019 NEO-PI-R Validity Scales

http://slidepdf.com/reader/full/neo-pi-r-validity-scales 1/9

Research Validity Scales for theNEO–PI–R: Additional Evidence

for Reliability and Validity

M. Scott Young

James A. Haley Medical Center

Tampa, Florida and

Department of Psychology

University of South Florida

John A. Schinka

James A. Haley Medical Center

Tampa, Florida and

Department of PsychiatryUniversity of South Florida School of Medicine

We examined the reliability and validity of the research validity scales (Schinka,

Kinder, & Kremer, 1997) for the NEO–Personality Inventory–Revised (NEO–PI–R)

in a clinical sample. The Negative Presentation Management (NPM) and Positive

Presentation Management (PPM) scales were found to have satisfactory internal con-

sistency reliability.Support for thevalidity of these scaleswas providedby thepattern

of convergent and discriminant correlations with respective Personality Assessment

Inventory(PAI) validity scales. Finally, PAI profiles of individuals with invalid NPM

scoreswere found to differsignificantly from those with valid NPM scores. Compari-

sons of the invalid profiles with profiles from other clinical samples provided addi-

tional support for the use of the NPM scale as a measure of negative impression

management.

Emerging from a tradition of factor-analytic research of normal personality

descriptors and trait termsfromthelay languagelexicon (Digman,1990;Goldberg,

1990; McCrae & Costa, 1997), the Five-factor modelof personalityhasbecome the

dominant model fortheinvestigationofpersonality.This modelproposes thata par-simonious description of basic personality should include measures of neuroticism

and stability, extraversion, intellect and openness, agreeableness, and conscien-

tiousness.TheinfluenceoftheFive-factormodelhasbeensubstantiveinareasofap-

JOURNAL OF PERSONALITY ASSESSMENT, 76 (3), 412–420Copyright © 2001, Lawrence Erlbaum Associates, Inc.



plied psychology. Not surprisingly, five-factor measures are used in those settings

andactivities that have typically employed inventories of normal personality struc-

ture, such as counseling centers andin job selection. In addition, Costa andMcCrae(1992) presented a persuasive argument for the clinical use of personality invento-

ries that provide measures of the Five-factor model.

Several inventories have been developed or revised to provide measures of the

five primary personality dimensions. The most comprehensive, and perhaps the

best validated, of these is the revision of the NEO–Personality Inventory

(NEO–PI–R; Costa & McCrae, 1992). The NEO–PI–R domain measures are titled

Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness. Six

facet subscales provide more detailed examination of personality within each of

the five domains. A series of studies employing the NEO–PI–R (e.g., Costa & Mc-Crae, 1988; McCrae & Costa, 1989a, 1989b) have established the existence of

substantive relations between the inventory’s five-factor structure and other mod-

els of individual differences.

However, the NEO–PI–R has been criticized (Ben-Porath & Waller, 1992a,

1992b) for its failure to include traditional validity scales that would provide mea-

sures of response bias and distortion, such as those provided on the Minnesota

Multiphasic Personality Inventory–2 (Butcher, Dahlstrom, Graham, Tellegen, &

Kaemmer, 1989) Scales L and F p. This concern received support by the finding that

the NEO–PI–R or NEO Five Factor Inventory (a brief version of the NEO–PI–R;Costa&McCrae,1992)domainscalescanbeinfluencedbyinstructionalsetsinsim-

ulated feigning studies (e.g., Paulhus, Bruce, & Trapnell, 1995). To address this is-

sue, Schinka, Kinder, and Kremer (1997) developed a set of validity indexes for the

NEO–PI–R. Rather than introducing new items to the already validated measure,

Schinka et al. used the existing NEO–PI–R item pool to develop 10-item validity

scalesfordetectingpositivepresentationmanagement(denial ofcommonfaults, at-

tribution of uncommon virtues) and negative presentation management (denial of

commonvirtues, attributionof uncommon faults) response styles. Anadditional in-

consistencyscale wasalso developed to detect randomand thoughtless responding.Initial examinationof the Positive PresentationManagement (PPM),Negative Pre-

sentation Management (NPM), and Inconsistency (INC) scales revealed satisfac-

tory internal consistencyandinterscale relations. The validity of thescales was next

examined by contrasting five sets of NEO–PI–R protocols: from a separate sample

of working adults, from a sample of 100 NEO–PI–R protocols with randomly pro-

duced responses, and from three samples of undergraduates completing the

NEO–PI–R under different instructional sets (feign positive, feign negative, re-

spond honestly). Analyses revealed significant differences among the groups on

both thevalidity scalesandthe NEO–PI–Rdomain scales, suggesting that the valid-ity scales are sensitive to impression management strategies.

In this study we examined the scale characteristics of the NPM and PPM re-

search validity scales in a clinical sample. We further investigated the validity of

RESEARCH VALIDITY SCALES FOR THE NEO–PI–R 413



these scales by examining their relationships to the validity scales of the Personal-

ity Assessment Inventory (PAI; Morey, 1991), a psychopathology inventory that

has received positive reviews in terms of scale development (Helmes, 1993;Helmes & Reddon, 1993). The PAI contains four validity scales, providing mea-

sures of Infrequent Responding, Inconsistent Responding (ICN), Positive Impres-

sion Management (PIM), and Negative Impression Management (NIM). Support

for the utility of the PAI validity scales has been provided in several studies (e.g.,

Morey, 1991; Morey & Lanier, 1998). The Schinka et al. (1997) INC, PPM, and

NPM validity scales thus parallel the PAI ICN, PIM, and NIM scales in assessing

response impression management. It was expected that the Schinka et al. PPM and

NPM scales would show a significant pattern of convergent and discriminant cor-

relations with the corresponding PAI validity scales. The measures’ inconsistencyscales were not expected to correlate highly because the scales measure careless-

ness, random responding, and test rejection, and it is likely that these response pat-

terns would not be the same given the difference in item content (normal

personality vs. psychopathology) for the two inventories.

The clinical sample for this study consisted of diagnosed alcoholics voluntarily

participating in a Veterans Affairs substance abuse treatment program. In addition

to the direct benefits of treatment, several other factors are known to influence ad-

mission to such programs and may serve as motivational sources for impression

management. These most frequently include the possibility of both short-term(e.g., housing, entrance to compensated work programs) and long-term (e.g., dis-

ability benefits) gain, both of which might produce negative impression manage-

ment because these benefits are provided to patients with poorer prognoses or

handicapping conditions. Less frequently, entry into voluntary treatment can be

expected to have a positive influence on outcomes of legal proceedings (e.g., for

charges of driving under the influence of alcohol or domestic violence). In these

situations, positive impression management might be expected because

minimization of adjustment or psychiatric problems associated with alcoholism

would be expected to provide more favorable legal outcomes. Because these fac-tors could be expected to influence impression management, the sample was be-

lieved to provide an excellent opportunity to study the characteristics of the

Schinka et al. research validity scales.

METHOD

Participants

Participants were 118 male veterans voluntarily seeking treatment for alcohol de-

pendence.The veteransaveraged46.81(SD = 8.67) years ofage with12.87 years of

education (SD = 2.57). The sample was primarily White (81.4%), although African

414 YOUNG AND SCHINKA



Americans (11%), American Indians (3.4%), and other races (4.2%) were also rep-

resented. The majority of participants (62.8%) did not meet criteria for any Diag-

nostic and Statistical Manual of Mental Disorders (4th ed. [ DSM–IV ]; AmericanPsychiatric Association, 1994) disorder other than alcohol dependence, although

9.3% met DSM–IV criteria for major depression, 11% were classified as bipolar

disorder, and 16.9%were diagnosed with another psychiatric disorder. Most veter-

ans (64.4%) did not meet criteria for a secondary substance-related disorder, but

13.6% were cocaine dependent, 9.3% were cocaine abusers, and 12.7% abused or

were dependent on substances other than alcohol or cocaine.

Procedure and Measures

All veterans who consented to participate in the study completed a comprehensive

evaluation that included demographic and historical information, structured diag-

nostic interviews (e.g., Kessler et al., 1994; Sheehan et al., 1997), and self-report

personality and psychopathology inventories. All interviews were conducted with

participants approximately 1 week after they were voluntarily admitted to the sub-

stance abuse treatment program of a metropolitan Veterans Affairs hospital for

treatment of their alcohol dependence.

Participants completed both the NEO–PI–R (Costa & McCrae, 1992) and the

PAI (Morey, 1991). All NEO–PI–R domain and facet scales were scored as well asthe research validity scales developed by Schinka et al. (1997). For the purposes of

this study only the PAI validity, clinical, and substance abuse problems scales

were scored. NEO–PI–R and PAI profiles with validity scale T scores greater than

or equal to 70 (2 SD above the mean) were designated as invalid. Although Morey

(1991) suggested several different cutoff scores for the PAI based on empirical

studies, no cutoff scores have been recommended for the research validity scales

for the NEO–PI–R. To use equivalent portions of the distributions of validity

scores for both the PAI and NEO–PI–R, we used traditional cutoff scores of 70,

which would exceed the scores of approximately 98% of the distribution of stan-dardization sample scores for these instruments.

RESULTS AND DISCUSSION

The internal consistency of the validity scales was examined first before we ex-

amined their criterion validity. The alpha coefficients, means, and standard devi-

ations of the PAI and research validity scales are presented in Table 1. The PPM

scale exhibited an acceptable internal consistency of .70, and the NPM had aslightly higher coefficient alpha of .75. The internal consistency of the INC scale

was not examined because the items do not share a similar content. The internal

consistency reliabilities of the validity scales were consistent with those origi-




nally reported by Schinka et al. (1997, Table 3; PPM = .56; NPM = .67) for an

adult sample.

Correlations between the PAI and research validity scales are presented in Ta-

ble 2. Examination of the correlation between the PAI NIM and PIM scales (–.43)

is consistent with that reported by Morey (1991) for a clinical sample (–.45, Table

10–11), replicating the nature of the relation of the two scales.

ThePPMscalewasfoundtohaveapositivecorrelationwiththePAIPIMscaler =

.62, p < .001) and a negative correlation with the PAI NIM scale r = –.41, p < .001).

Similaranalysis for theNPM scale revealed a positivecorrelationwith the PAI NIM

scale r =.31, p <.001)andanegativecorrelationwiththePAIPIMscale r =–.29, p <

.01). Results for the INC scalerevealed insignificant correlations with no meaning-

ful pattern, consistentwiththe fact that the scale isa measure ofcontentless response

style and not of an underlying construct.


TABLE 1

Research and PAI Validity Scale Alpha Coefficients, Means, and Standard Deviations

Scale Alpha Scale M Scale SD

PPM .70 39.40 12.93

NPM .75 61.88 12.72

INC — 44.39 14.21

PAI PIM .90 31.75 15.19

PAI NIM .83 57.00 17.01

PAI ICN — 53.69 7.50

Note. PAI = Personality Assessment Inventory; PPM= Positive PresentationManagement; NPM=

Negative Presentation Management; INC = Inconsistency; PIM = Positive Impression Management;

NIM = Negative Impression Management; ICN = Inconsistent Responding.

1Table 10–1 presents the value as .45; however, the value should be –.45 (L. Morey, personal com-

munication, December 1999).

TABLE 2

Correlations Among PAI and Research Validity Scales

PAI PIM PAI NIM INC NPM PPM

PAI ICN .10 .04 .02 –.10 .07

PAI PIM –.43* .03 –.29* .62*

PAI NIM .15 .31* –.41*

INC .06 –.03

NPM –.39*

Note. PAI = Personality Assessment Inventory; PIM = Positive Impression Management; NIM =

Negative Impression Management; INC = Inconsistency; NPM = Negative Presentation Management;

PPM = Positive Presentation Management; ICN = Inconsistent Responding.

* p = .01.



The extent of agreement between the corresponding PAI and research validity

scales was then examined. If individuals gave invalid responses on one of the per-

sonalitymeasures, thentheywouldbeexpectedtorespond inasimilarmannerontheothermeasure.Using thesecriteria, noprofilewas found tohaveaninvalidPAIPIM,

and onlya singleparticipant had an invalid PPM score.This is not a surprising find-

ingbecausethesampleconsistedofvoluntaryadmissionstoasubstanceabusetreat-

ment program, and the following analyses were therefore restricted to an

examination of the NPM scale.Classification ofvalid and invalidcases using T ≥70

cutoff scores for the NPM and NIM scales revealed agreement on 70% of the cases,

where both scales agreed that 14 cases were invalid and that 68 were valid.

In a final analysis we conducted a multivariate analysis of variance

(MANOVA) to compare the PAI validity and clinical scale profiles of participantswith NPM scores in valid and invalid ranges. The overall MANOVA was signifi-

cant, F (15, 102) = 4.5, p < .001, indicating that there were significant differences

in PAI profiles based on the validity of their NPM scores. The PAI profiles for the

respective groups are plotted in Figure 1. Univariate follow-up analyses of vari-

ance indicated that individuals with invalid NPM scale scores had significantly

higher scores on the following PAI validity and clinical indexes: Infrequency,

NIM, Somatic Complaints, Anxiety, Anxiety-Related Disorders, Depression,

Paranoia, Schizophrenia, Borderline, and Drug Problems. Alternatively, individu-

als with a valid NPM score had significantly higher scores on the PAI PIM scale.The groups scored equivalently on only the PAI ICN, Mania, Antisocial, and Al-

cohol scales, providing additional support for making a distinction between indi-

viduals based on the validity of their NPM scores.

The meaning of this distinction is clarified by comparison with previously re-

ported samples. Figure 1 also displays profiles plotted from data of large clinical

and alcoholic samples provided by Morey (1991; Tables 4 through 8 and Figures 9

through 15, respectively). The mean clinical scale elevation for the current NIM-

valid sample of alcohol-dependent patients is approximately 4 T scores higher than

that reported for Morey’s alcoholic sample and 6.5 T scores higher than that re-ported for Morey’s clinical sample. In contrast, the mean clinical scale elevation

for the current NIM-invalid sample of alcohol-dependent patients is approxi-

mately 13 T scores higher than that reported for Morey’s alcoholic sample and 15 T

scores higher than that reported for Morey’s clinical sample. Both the pattern of

the PAI profile and the level of mean clinical scale elevation of the NIM-invalid

sample suggest that the profile reflects not only psychopathology, but also the ef-

fects of negative impression management.

CONCLUSIONS

The results of ourstudy provide additional evidence in support of the reliabilityand

validity of the research validity scales for the NEO–PI–R. The NPM and PPM




scales were found to have internal consistency reliability consistent with previous

findings in a nonclinical sample (Schinka et al., 1997) and consistent with those in

clinicalsamplesfor the PAI (Morey, 1991).Furthermore, support for the validity of

the NPM and PPM scales was provided both by the pattern of convergent and

discriminant correlations with respective PAI validity scales and by the substantial

agreement of the NPM andPAI NIM scales in identifying invalid profiles. Because

only a single case in our self-referred clinical sample produced a PPM-invalid pro-

tocol, we were unable to explore fully the characteristics of the scale. Future re-

search efforts should be undertaken with nonclinical samples to examine thepsychometric features of the PPM scale in greater detail.

Finally, PAI profiles of individuals with invalid NPM scores differed signifi-

cantly from those with valid NPM scores on several of the PAI clinical and validity


FIGURE 1 Mean Personality Assessment Inventory (PAI) profiles for Negative Presentation

Management–Invalid (NPM–Invalid), Negative Presentation Management–Valid

(NPM–Valid), andMorey (1991) samples. Data for thealcoholic andclinical samples are repro-

duced by special permission of the Publisher, Psychological Assessment Resources, Inc.,

Odessa, FL 33556, from the Personality Assessment Inventory by Leslie Morey, PhD, Copy-

right 1991 by PAR, Inc. Further reproduction is prohibitedwithout permission of PAR, Inc. INF

= Infrequent Responding; ICN = Inconsistent Responding; NIM = Negative Impression Man-

agement; PIM =Positive Impression Management; SOM = Somatic Complaints; ANX = Anxi-

ety; ARD = Anxiety-Related Disorders; DEP = Depression; MAN = Mania; PAR = Paranoia;

SCZ = Schizophrenia; BOR = Borderline; ANT = Antisocial; ALC = Alcohol Dependency;

DRG = Drug Problems.



indexes. Comparisons with profiles provided by other large samples suggested

that the NPM-invalid profile reflects not only psychopathology, but also the ef -

fects of negative impression management.

REFERENCES

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th

ed.). Washington, DC: Author.

Ben-Porath, Y. S., & Waller, N. G. (1992a). Five big issues in clinical assessment: A rejoinder to Costa

and McCrae. Psychological Assessment, 4, 23–25.

Ben-Porath, Y. S., & Waller, N. G. (1992b). “Normal” personality inventories in clinical assessment:General requirements and the potential for using the NEOPersonality Inventory. Psychological As-

sessment, 4, 14–19.

Butcher, J.N.,Dahlstrom, W. G.,Graham, J. R.,Tellegen,A.,& Kaemmer,B. (1989). MMPI–2: Minne-

sota Multiphasic Personality Inventory–2: Manual for administration and scoring. Minneapolis:

University of Minnesota Press.

Costa, P. T., Jr., & McCrae, R. R. (1988). From catalog to classification: Murray’s needs and the five-

factor model. Journal of Personality and Social Psychology, 55, 258–265.

Costa, P. T., Jr., & McCrae, R. R. (1992). Professional manual for the revised NEO Personality Inven-

tory (NEO–PI–R) and NEO Five Factor Inventory (NEO–FFI). Odessa, FL: Psychological Assess-

ment Resources.

Digman, J. M. (1990). Personality structure: Emergenceof thefive-factor model. Annual Review of Psy-chology, 41, 417–440.

Goldberg,L. R. (1990). An alternative “descriptionof personality”:The BigFive factor structure. Jour-

nal of Personality and Social Psychology, 59, 1216–1229.

Helmes, E. (1993). A modern instrument for evaluating psychopathology—The Personality Assess-

ment Inventory. Journal of Personality Assessment, 61, 414–417.

Helmes, E., & Reddon, J. R. (1993). A perspective on developments in assessing psychopathology: A

critical review of the MMPI and MMPI–2. Psychological Bulletin, 113, 453–471.

Kessler, R. C., McGonagle, K. A., Zhao, S., Nelson, C. V., Hughes, M., Eshleman, S., Wittchen, H. U.,

& Kendler, K. S. (1994). Lifetime and 12-month prevalence of DSM–III–R psychiatric disorders in

the United States: Results from the National Comorbidity Study. Archives of General Psychiatry,

51, 8–19.McCrae, R. R.,& Costa,P. T., Jr. (1989a). Reinterpreting theMyers–Briggstype indicator from theper-

spective of the five-factor model of personality. Journal of Personality, 57, 17–40.

McCrae, R. R., & Costa, P. T., Jr. (1989b). The structure of interpersonal traits: Wiggins’ circumplex

and the five-factor model. Journal of Personality and Social Psychology, 56, 586–595.

McCrae, R.R.,& Costa,P.T., Jr. (1997). Personality trait structureas a human universal. American Psy-

chologist, 52, 509–516.

Morey, L. C. (1991). PersonalityAssessmentInventory:Professional manual. Odessa, FL:Psychologi-

cal Assessment Resources.

Morey,L. C., & Lanier, V. W. (1998). Operating characteristics of six response distortion indicators for

the Personality Assessment Inventory. Assessment, 5, 203–214.

Paulhus, D. L.,Bruce, M. N.,& Trapnell, P. D. (1995). Effectsof self-presentation strategieson person-

ality profiles and their structure. Personality and Social Psychology Bulletin, 21, 100–107.

Schinka, J.,Kinder, B.,& Kremer, T. (1997). Research validity scales for theNEO–PI–R:Development

and initial validation. Journal of Personality Assessment, 68, 127–138.




Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Janavas, J., Weiler, E., Keskiner, A., Schinka, J. A.,

Knapp, E., Sheehan, M., Stoker, M. J., & Dunbar, G. C. (1997). The validity of the Mini Interna-

tional Neuropsychiatric Interview (MINI) according to the SCID–P and its reliability. European

Journal of Psychiatry, 12, 232–241.

John A. Schinka

Psychology Service (116B)

James A. Haley VA Medical Center

13000 Bruce B. Downs Boulevard

Tampa, FL 33612

Received December 10, 1999

Revised August 31, 2000


Documents

NEO-PI-R Validity Scales