An Argument for Context-SpecificPersonality Assessments

ERIC D. HEGGESTAD AND HEATHER L. GORDONUniversity of North Carolina at Charlotte

Some scholars within our field (e.g.,Morgeson et al., 2007; Murphy & Dziewec-zynski, 2005) have begun to lament as dis-appointing the criterion-related validityevidence for personality assessments in theprediction of job performance. In meta-ana-lytic work, for example, Barrick,Mount, andJudge (2001), Dudley, Orvis, Lebiecki, andCortina (2006), Hurtz and Donovan (2000),and Tett, Jackson, and Rothstein (1991) havereported rather small (.12, .15, .13, and .12,respectively) uncorrected mean validitycoefficients between Conscientiousness,the most predictive of the Big Five traits,and job performance. In their article, HoughandOswald (2008) argued that a more care-ful consideration of situational demands onbehavior is essential to understanding thepersonality–performance relationship andto the identification of conditions underwhich larger criterion-related validities canbe expected. We could not agree more.

In his now classic book Personality andAssessment, Mischel (1968) reviewed thepersonality literature and made two keyobservations: (1) that self-report measures ofpersonality rarely, if ever, correlate withbehavior at a level above .30 and (2) that

correlations of people’s behaviors across sit-uations are typically small in magnitude.Based on these observations, Mischel sug-gested that ‘‘the concept of personality traitsas broad predispositions is thus untenable’’(p. 140), a conclusion that would cripple per-sonality psychology for some time. Thedefense made by personality psychologiststo Mischel’s strong conclusion came in theform of the principle of aggregation (Epstein&O’Brien, 1985). Stated simply, the principleof aggregation suggests (a) that the accumu-lation of behavioral information from a largenumber of different situations can result ina reliable indication of trait standing and (b)that trait assessments can be expected to pre-dict (well in excess of .30) behaviorwhen thatbehavior has been aggregated over a numberof specific instances. Fundamentally, there-fore, theprincipleof aggregation suggests thatpersonality assessments are poor predictorsof specific instances of behavior (becausethese behaviors are strongly influenced bythe demands of the particular situation) butcan be reasonably good predictors of lawfultendencies in behavior that emerge acrossdifferent situations.

The disappointingly low criterion-relatedvalidities observed for personality assess-ments are likely a result, at least in part, ofour lack of understanding of the principle ofaggregation and its application to the contextof selection. Many personality assessmentsused in personnel decision making have notbeen created specifically for use in selectionsettings but rather as general assessments ofthe respondent’s personality. As such, the

Correspondence concerning this article should beaddressed to Eric D. Heggestad. E-mail: edhegges@email.uncc.edu

Address: Department of Psychology, University ofNorth Carolina at Charlotte, 9201 University CityBoulevard, Charlotte, NC 28223-0001

Eric D. Heggestad, Department of Psychology, Uni-versity of North Carolina at Charlotte; Heather L.Gordon, Organizational Science, University of NorthCarolina at Charlotte.

Industrial and Organizational Psychology, 1 (2008), 320–322.Copyright ª 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

320

items and instruction sets for these measuresare general and do not focus on a particularsituational context. In other words, theseassessments ask people to report their trait-related behaviors on average across the vari-ety of situations they typically encounter.Working from the principle of aggregation,scores on such assessments should best pre-dict behavioral tendencies that arise whenbehavior is aggregated across a wide varietyof situations and settings. As noted by Houghand Oswald, and consistent with this notion,personality trait measures have been found tobe good predictors of major life outcomes,such as health behaviors and occupationalattainment (Bogg & Roberts, 2004; Roberts,Kuncel, Shiner, Caspi, & Goldberg, 2007).

We, as industrial–organizational psy-chologists, however, are generally moreinterested inmaking predictions about a per-son’s behavior at work than about his or herbehavior in general. As such, our focusshould be on the prediction of behavioraggregated across work-related situations.Thus, it can be reasoned that assessing anapplicant with a general personality mea-sure results in a mismatch between whatour predictor is designed to predict (i.e.,aggregated levels of general behavior) andthe scope of the criterion we are interestedin predicting (i.e., aggregated levels of workbehavior). By using general personalitymeasures in selection contexts, we are tak-ing a predictor-centric approach ratherthan a more appropriate criterion-centricapproach (Bartram, 2005; Guion, 1998).

As mentioned briefly by Hough andOswald, several researchers (Bing, Whan-ger, Davison, & VanHook, 2004; DeGroot& Kluemper, 2007; Hunthausen, Truxillo,Bauer, & Hammer, 2003; Schmit, Ryan,Stierwalt, & Powell, 1995) have investigatedcontextualized personality assessments aspredictors of context-specific outcomes(such as job performance). Contextualizedpersonality assessments provide informa-tion about behavioral tendencies within aspecific context, such as a workplace con-text. As such, the use of these measures re-presents a tighter predictor–criterion linkagethan does the use of general personality

assessments. Most often, contextualizationis accomplished by instructing respondentsto consider only their behavior within thework context when responding to the itemsor by adding contextual information, such asthe phrase ‘‘at work,’’ directly to the items.Research has demonstrated superior predic-tive capability of contextualized measuresover standard, noncontextualized versionsof the same measure when the criterion ofinterest is context specific (Bing et al., 2004;DeGroot & Kluemper, 2007; Hunthausenet al., 2003; Schmit et al., 1995). Forexample, Schmit et al. compared the crite-rion-related validities for a school-specificcontextualized measure of Conscientious-ness and a noncontextualized version ofthe same measure in the prediction of GPA.The criterion-related validity was found to benotably higher for the contextualized mea-sure (r¼ .41) than for the noncontextualizedmeasure (r ¼ .25; see Bing et al., 2004, forsimilar findings). As another example, Hun-thausen et al., who varied the degree of con-textualization through an instructionalmanipulation, found that contextualizationwas associated with an increase in concur-rent criterion-related validity (from r ¼ .10to .31) for a measure of Conscientiousnessin the prediction of performance ratingsamong customer service employees in theairline industry.

Consideration of the principle of aggrega-tion in the selection context suggests that theuse of contextualized measures will createa tighter linkage between the predictor andour criteria of interest and that, conse-quently, the criterion-related validity ofthose assessments should be stronger thanfor similar, noncontextualized assessments.Although the research base is still ratherlimited, existing empirical evidence doessupport the superiority of contextualizedassessments in the prediction of work-specific outcomes. Although we would liketo see additional research in this area (e.g.,how much contextualization is required inthe items and within the measure; a clearevaluation of construct equivalence;the impact of faking), we believe thatresearchers and practitioners alike should

Contextualization 321

begin consistently using contextualizedassessments of personality. Researchersinvestigating the relationships between per-sonality and organizationally relevantoutcomes are likely to find stronger associa-tions when contextualized measures areused. Likewise, practitioners are likely toobserve increased criterion-relatedvaliditiesandmakebetter quality decisionswhencon-textualized measures are used in employeeselection systems. In fact, with some addi-tional empirical support, we believe thatthe use of contextualized measures withinselection contexts could soon become a bestpractice in our field.

References

Barrick, M. R., Mount, M. K., & Judge, T. A. (2001).Personality and performance at the beginning ofthe new millennium: What do we know and wheredowegonext? International Journal of Selection andAssessment, 9, 9–30.

Bartram, D. (2005). The Great Eight competencies: Acriterion-centric approach to validation. Journal ofApplied Psychology, 90, 1185–1203.

Bing,M.N.,Whanger, J. C.,Davison,H.K.,&VanHook,J. B. (2004). Incremental validity of the frame-of-reference effect in personality scale scores: Areplication and extension. Journal of AppliedPsychology, 89, 150–157.

Bogg, T., & Roberts, B. W. (2004). Conscientiousnessand health behaviors: Ameta-analysis of the leadingbehavioral contributors to mortality. PsychologicalBulletin, 130, 887–919.

DeGroot, T., & Kluemper, D. (2007). Evidence of pre-dictive and incremental validity of personalityfactors, vocal attractiveness and the situational inter-view. International Journal of Selection and Assess-ment, 15, 30–39.

Dudley, N. M., Orvis, K. A., Lebiecki, J. E., & Cortina, J.M. (2006). A meta-analytic investigation of consci-entiousness in the prediction of job performance:Examining the intercorrelations and the incrementalvalidity of narrow traits. Journal of Applied Psychol-ogy, 91, 40–57.

Epstein, S., & O’Brien, E. J. (1985). The person-situationdebate in historical andcurrent perspective.Psycho-logical Bulletin, 98, 513–537.

Guion, R. M. (1998). Assessment, measurement, andprediction for personnel decisions. Mahwah, NJ:Lawrence Erlbaum Associates.

Guion, R. M., & Gottier, R. F. (1965). Validity of person-ality measures in personnel selection. PersonnelPsychology, 18, 135–164.

Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp,J. D., & McCloy, R. A. (1990). Criterion-relatedvalidities of personality constructs and the effect ofresponse distortion on those validities. Journal ofApplied Psychology, 75, 581–595.

Hough, L.M., &Oswald, F. L. (2008). Personality testingand industrial–organizational psychology: Reflec-tions, progress, and prospects.Industrial and Orga-nizational Psychology: Perspectives on Science andPractice, 1, 272–290.

Hunthausen, J. M., Truxillo, D. M., Bauer, T. N., &Hammer, L. B. (2003). A field study of frame-of-reference effects on personality test validity. Jour-nal of Applied Psychology, 88, 545–551.

Hurtz, G. M., & Donovan, J. J. (2000). Personality andjob performance: The Big Five revisited. Journal ofApplied Psychology, 85, 869–879.

Mischel, W. (1968). Personality and assessment. NewYork: Wiley.

Morgeson, F. P., Campion, M. A., Dipboye, R. L.,Hollenbeck, J. R., Murphy, K., & Schmitt, N.(2007). Are we getting fooled again? Coming toterms with limitations in the use of personality testsfor personnel selection. Personnel Psychology, 60,1029–1049.

Murphy, K., & Dzieweczynski, J. (2005). Why don’tmeasures of broad dimensions of personality per-form better as predictors of job performance?Human Performance, 18, 343–357.

Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A., &Goldberg, L. R. (2007). The power of personality:The comparative validity of personality traits, socio-economic status, and cognitive ability for predictingimportant life outcomes.PerspectivesonPsycholog-ical Science, 2, 313–345.

Schmit, M. J., Ryan, A. M., Stierwalt, S. L., & Powell, A.B. (1995). Frame-of-reference effects on personalityscale scores and criterion-related validity. Journal ofApplied Psychology, 80, 607–620.

Tett, R. P., Jackson, D. N., & Rothstein, M. (1991). Per-sonality measures as predictors of job performance:A meta-analytic review. Personnel Psychology, 44,703–742.

322 E.D. Heggestad and H.L. Gordon