A Meta-Analysis of Cognitive Ability and Team Performance

8/12/2019 A Meta-Analysis of Cognitive Ability and Team Performance

1/26

SMALLGROUPRESEARCH/October2001Devine,Philips/COGNITIVEABILITYINTEAMS

DO SMARTER TEAMS DO BETTER

A Meta-Analysis of Cognitive Ability andTeam Performance

DENNIS J. DEVINE

Indiana UniversityPurdue University, Indianapolis

JENNIFER L. PHILIPS

University of Akron

This study reports the results of several meta-analyses examining the relationship between

four operational definitions of cognitive ability within teams (highest member score, lowest

member score, mean score, standard deviation of scores) and team performance. The three

indices associated with level yielded moderate and positive sample-weighted estimates of

the populationrelationship(.21 to .29), butsamplingerrorfailed to account forenough vari-

ation to rule outmoderator variables.In contrast, theindexassociated with dispersion (i.e.,

standarddeviationof member scores) wasessentiallyunrelatedto teamperformance (.03),

andsampling error provideda plausible explanation for the observed variation across stud-

ies.A subgroupanalysisrevealed that mean cognitive abilitywas a much better predictorof

team performance in laboratory settings (.37) than in field settings (.14). Study limitations,

practical implications, and future research directions are discussed.

Cognitive ability is the capacity to understand complex ideas,

learn from experience, reason, problem solve, and adapt (Neisser

et al., 1996; Sternberg, 1997). After hundreds of empirical studies,

it is now clear that cognitive ability is one of the best predictors of

individual job performance (Hunter & Hunter, 1984; Schmidt &

Hunter, 1998; Wagner, 1997). Given that many tasks performed by

small groups or teams involve learning, reasoning, and problem

solving, it seemslikelythat thecognitiveability of team members is

related to team performance. However, the issue is not as straight-

507

AUTHORSNOTE: We thankBill Rogers,EricSundstrom,and Jane Williamsfor theirhelp-

ful comments on earlier versions of this article and Marc Fogel forhis assistance in collect-

ing and coding data.

SMALL GROUP RESEARCH, Vol. 32 No. 5, October 2001 507-532

2001 Sage Publications


2/26

forward as intuition suggests. It has been known for some time that

relationships at one level of analysis do not necessarily hold at

another (Kozlowski & Klein, 2000; Robinson, 1950; Thorndike,

1939).Klein,Dansereau, andHall (1994)offered a classic example

of this phenomenon in discussing the relationshipbetween popular

votesand outcomes inU.S. presidentialelections. At thestate level,

thecandidatereceiving themost popular voteswins allthe electoral

votes (i.e., winner takes all); however, as highlighted by the2000

U.S. presidential election, the winner at the national level does not

necessarily receive the most popular votes in the country as a

whole. Essentially, the relationship between popular votes and

election outcome differs across levels. Assuming that the state-levelrelationshipholds at thenational level would be akin to making

a cross-level fallacy (Rousseau, 1985).

Ina similar fashion,as far as workgroups are concerned, it is not

appropriate to assume a relationship between cognitive ability and

performance at the team level based on studies conducted at the

individual level. In other words, the strong positive relationship

between cognitive ability and performance at the individual level

does notcause (or imply) a relationshipat the team level. Consider,

for example, a decision-making team composed of members from

several functional areas of an organization. Individually, the mem-

bers may be intelligent and perform their specific roles well. How-

ever, this does not ensure that the team as a whole will do well. Forinstance, differences in perspective associated with the various

functional areas may prevent the effective integration of relevant

information (Hinsz, Tindale, & Vollrath, 1997; Larson & Christensen,

1993). In essence, the existence of a team-level relationship

between cognitive ability and performance is an empirical issue: It

could be positive, negative, nonexistent, or variable from situation

to situation. With the growing use of work groups and teams in

organizations, it would be beneficial for practitioners to identify

valid, low-cost, practical predictors of team performance. Cogni-

tiveability tests mayproveuseful in this regard, but their valuecan-

not be assumed.

508 SMALL GROUP RESEARCH / October 2001


3/26

Complicating assessment of the relationship between cognitive

ability and performance in work groups is the variety of ways in

which cognitive ability can be operationally defined at the team

level. In theempirical literature onsmall groups, three indices asso-

ciatedwith thefunctional amount(i.e., level)ofcognitive resources

available to the team have appeared most often: (a) the value of the

teamshighest scoring individual, (b)thevalueof theteamslowest

scoring individual, and (c) themean of team member scores. These

three operational definitions correspond to three task types identi-

fied by Steiner (1972), involving different functional relationships

between individual and group performance: (a) tasks where the

best individual performance determines the groups performance,(b) tasks where the worst individual performance determines the

groups performance, and (c) tasks where all individual perfor-

mances contribute to group performance in a summative fashion.

Steiner labeled the first type of taskdisjunctive, the second type

conjunctive, and the third typeadditive.

In addition to these three operational definitions associated with

the functional level of cognitive ability present in a work group, the

rise of diversity issues in the workplace has called attention to the

potential influence of team-level variation on team dynamics and

effectiveness (Jackson, 1996). This line of work suggests a fourth

way of treating cognitive ability at the team level: in terms of the

dispersion of member scores. Underlying much of the work ondiversity are thenotions that team memberdiversity is a good thing

and that heterogeneous groups should outperform homogeneous

groups(especially ontasksrequiringcreativityor innovation)because

they possess a largerpool of task-relatedresources(K.Y. Williams&

OReilly, 1998). Although there has been little systematic discus-

sion concerning theimpact of team memberdiversitywith regard to

cognitive ability, a number of recent studies have nonetheless

examined the association between the standard deviation of mem-

ber cognitive ability scores and team performance.

From a practical perspective, it would be advantageous to learn

if thefour operational definitions varywith regard to their criterion-

Devine, Philips / COGNITIVE ABILITY IN TEAMS 509


4/26

related validity. To the extent they do, this wouldsuggest strategies

for composing work groups. For example, if the best predictor of

team performance across a variety of tasks turned out to be the

score of the most intelligent member, this would suggest ensuring

that allwork groupshaveat least onevery intelligent team member.

Conversely, if mean cognitive ability score was found to be thebest

predictor of team performance, efforts could be made to select as

many intelligent members as possible without excessive concern

over finding a single genius.

Fortunately, althoughresearch oncognitive ability inworkgroups

is a relatively recent phenomenon, there is now enough data to

make a preliminary assessment of theteam-level relationshipusingmeta-analytic methods. In the next section, we report the results of

several meta-analyses concerning the relationship between team-

level cognitive ability and teamperformance using fouroperational

definitions of team-level cognitive ability (i.e., mean member

score, highest member score, lowest member score, and standard

deviation of scores). Given the strength and robustness of the indi-

vidual-level relationship, our primary research hypothesis was as

follows: Team-level indices reflecting the highest, lowest, and

mean cognitive ability scoreswithinwork groupswill each be posi-

tively related to team performance. Furthermore, given the critical

role of member interdependence in most work groups, it seems

likely that the performance of few (if any) work groups would bedetermined solely by the actions of a single member as implied by

Steiners (1972) disjunctive and conjunctive task types. Even in

work groups where one member is clearly most important, it is dif-

ficult to imagine that performance is notaffected to some degree by

other members. Therefore, we expected that the observed relation-

ship between team-level cognitive ability and performance would

be stronger when indexed by the mean member score as opposed to

the highest or lowest score on the team. Given the lack of theoreti-

caldevelopment in theliterature,we didnot haveany formalexpec-

tations with regard to the relationship between the dispersion of

cognitive ability scores and team performance.



5/26

METHOD

LITERATURE SEARCH

Before any meta-analysis can be conducted on antecedents of

team performance, it is necessary to address the issue of whether

work groups and teams will be treated as the same thing. Although

persuasive arguments have been made for treating them separately,

thedistinction is relativelyrecent, andthetwo termshavebeen used

synonymously throughout much of the literature on small groups

(Guzzo, 1996; Ilgen,1999).Furthermore, in any event, preliminary

inspection revealed that few empirical reports described the studycontext well enough to make fine distinctions. As a result, we use

the termsgroup,work group, andteaminterchangeably in refer-

ence to small collectivesof individuals that interact for thepurpose

of accomplishing one or more shared goals while operating with

some degree of interdependence.

Severalconvergent methods were used to searchforstudies with

usable data. First, we conducted a computerized search of the

PSYCHINFO database for the years 1967 to 1999. The following

terms (and relevant combinations) were used in the computerized

database search: team, group, workgroup, cognitive ability, mental

ability, generalability, ability, intelligence,performance, effective-

ness, efficiency,productivity, and outcome. The authors also manu-ally scanned the titles in each issue of the following journals for the

past 10 years:Journal of Applied Psychology,Personnel Psychol-

ogy,Academy of Management Journal,Journal of Management,

Journal of Organizational Behavior, Organizational Behavior and

Human Decision Processes, Small Group Research, Journal of

Personality and Social Psychology, andIntelligence. Abstracts and

method sections were consulted for any article possessing a title

that suggested team-level research. Reference lists for recent theo-

retical andempiricalpapersandbooks onwork groupor team com-

position were also examined (e.g., Barrick, Stewart, Neubert, &

Mount, 1998; Cohen & Bailey, 1997; Guzzo & Dickson, 1996;



6/26

Ilgen,1999; Milliken & Martins, 1996).Given thesmall numberof

published studies on this topic, special emphasis was placed on

acquiringunpublished data. In particular, after reviewingabstracts,

we obtained several unpublished theses and dissertations through

interlibrary loan; two of these eventually proved useful (Blades,

1976; OConnell, 1994). We also generated a list of researchers

known to conduct research on groups and teams and contacted

these individuals via e-mail. Data from four studies (i.e., Gully,

1997; Hollenbeck et al., 1999; Sundstrom & Futrell, 1999; Zukin,

1999) were obtained in this fashion.

INCLUSION CRITERIA

A study was included in the meta-analysis if it did each of the

following: (a) measured individual-level cognitive ability and

formed one or more team-level indices using some explicit aggre-

gation process, (b) measured team performance, and (c) empiri-

cally assessed the degree of association between team-level cogni-

tive ability and team performance using a Pearsonsrcorrelation.

The following were considered acceptable measures of cogni-

tive ability: (a) scores from an established measure of cognitive

ability (e.g., Wonderlic Personnel Test, 1992) or (b) general apti-

tude (e.g., American College Test Verbal, Scholastic Assessment

Test Quantitative) or multiaptitude test scores (e.g., compositescores on the General Aptitude Test Battery or Differential Apti-

tude Test). Although scores from an established measure of cogni-

tive ability are obviously preferable, general or composite aptitude

scores tend to be highlycorrelated with measures of cognitive abil-

ity because they measure one or more primary components (Wag-

ner, 1997). Team performance was defined as the degree to which

the team accomplished its goal or mission. We initially hoped to

conduct separate analyses for accuracy and quality criteria and

speed and quantity criteria, but only three studies were found that

reported associations between measures of team-level cognitive

ability andproductivity (i.e., Neuman& Wright, 1999; OConnell,

1994; Sundstrom & Futrell, 1999), so this was not possible. When

multiple criteria were reported (e.g., production countsandsubjec-



7/26

tive ratings), we chose the one that best appeared to capture overall

accomplishment of the teams mission. When subjective perfor-

mance measures were available for both supervisors and team

members, we used supervisor ratings because of (a) their common

use in administrative decision making in organizations, (b) their

availability in each study, and (c) their tendency to exhibit more

variabilitythan team memberratings. Thus, although both supervi-

sors and team members can provide useful information on team

performance related to their unique perspectives, we chose to use

supervisor ratings because of their greater salience, availability,

and variability relative to aggregated team member self-ratings.

All studies appearing to be relevant were read in entirety by oneof the authors,and a final judgmentwas thenmadeas towhether the

study provided usabledata. Themajority of empirical studies iden-

tified in the computerized search were discarded because they did

not analyze (or, in a few cases, report) the relationship between

cognitive ability andperformance at the team level. Because of our

focus on general cognitive ability, we didnot include studies solely

involving team-level indices of task-specific ability. These studies

used individual performance scores on some task as the basis for

constructing a group-level index of task-specific ability, often as a

benchmark for determining process gain or loss associated with

group performance on the same task. We also did not include stud-

ies that used a domain-specific aptitude test score instead of a mea-sure of general cognitive ability (e.g., Gurnee, 1937; Kabanoff &

OBrien, 1979). Finally, we wish to note that we were unable to

obtain usable data from three well-known studies of group compo-

sition (i.e., Spector & Suttell, 1957; Terborg, Castore, & DeNinno,

1976; Tziner & Eden, 1985) because cognitive ability scores were

dichotomized to allow formation of teams with varying degrees of

heterogeneity but no quantitative values at the team level were

calculated and/or reported.

In total,ourreviewof theliteratureyielded 19studies that metall

three inclusion criteria; Table 1 provides descriptive summaries of

each. Several studies reported data from several related experi-

ments or settings (i.e., Blades, 1976; OBrien & Owens, 1969;

Sundstrom & Futrell, 1999), resulting in a total of 25 independent



8/26

TABLE 1: Descriptive Information for Studies Included in One or More Meta-Analyses

Study Task Cognitive Ability Measure Performanc

Barrick, Stewart, Neubert, Small appliance and electronic Wonderlic Personnel Test Supervisor rating of

and Mount (1998) assembly; fabrication and maintenance (sum of 8 dimensio

Blades (1976) (1 and 2) Operating an army mess hall Henman-Nelson Mental Average of superviso

Ability Test (two raters)

Bottoms (1998) SouthEast Airlines top management Wonderlic Personnel Test Profit (from algorithm

simulation

Brandt (1998) Energy International top management Wonderlic Personnel Test Time taken to identif

simulation for position

Clayton (1998) SouthEast Airlines top management Wonderlic Personnel Test Profit (from algorithm

simulationColarelli and Boos (1992) Evaluation of a personnel program Composite of global ACT Average score on cou

and GPA (two raters)

Devine (1999) SouthEast Airlines top management Wonderlic Personnel Test Profit (from algorithm

simulation

Fiedler and Meuwese (1963) Operating antiaircraft artillery guns AGCT scores Commander ranking

Gully (1997) TEAM TANDEM military command Wonderlic Personnel Test Quality and quantity

and control simulation classification

Hollenbeck et al. (1999) Dynamic distributed decision-making Wonderlic Personnel Test Computer-generated

military command and control

simulation

LePine, Hollenbeck, Ilgen, TIDE2

military command and control ACT/SAT Accuracy of target ai

and Hedlund (1997) simulation

Neuman and Wright (1999) Human resource problem solving Thurstone Test of Mental Supervisor ratings (f

Alertness

514


9/26

OBrien and Owens (1969) (1) Collective letter writing for recruitment AGCT Composite rating of

(based on 5 dimens

OBrien and Owens (1969) (2) Constructing a chart AGCT Number of errors ma

OBrien and Owens (1969) (3) Generating a short fictional story ACT-English Composite ratings of

5 dimensions)


5 dimensions)


5 dimensions)

OConnell (1994) Automotive construction, paint and GATB composite Supervisor rating of

assembly; maintenance (based on 6 dimens

Stevens, Jones, Fischer, and Manual/technical jobs SRA verbal subtest Supervisor ratings of

Kane (1999)

Sundstrom and Futrell (1999) Automotive headlamp assembly IRT/DAT composite Number of units asse

Tziner and Vardi (1983) Operating military tanks IDF composite Commander ranking

W. M. Williams and Brainstorming Henmon-Nelson Test of Overall rating of solu

Sternberg (1988) Mental Ability

Zukin (1999) Solving murder mystery Wonderlic Personnel Test Correct answer (Yes

NOTE: ACT= American CollegeTest; GPA = gradepoint average;AGCT= ArmyGeneral Classification Test; SAT = ScholasticAssesGeneral AptitudeTestBattery;SRA = Science Research AssociatesSurveyof Basic Skills;IRT = Industrial ReadingTest; DAT = DiffeIDF = Israeli Defense Force battery based on Otis-Lenon, command of Hebrew, and interview.

515


10/26

samples. The majority of studies occurred in a controlled setting

(11 of 19) within the past 10 years (14 of 19).

CODING

Fivequantitativevariableswere codedfrom usable studies: (a)num-

ber of teams involved, (b) correlation between the mean cognitive

ability score within the team and team performance, (c) correlation

between the highest cognitive ability score within the team and

team performance, (d)correlationbetween thelowest cognitive abil-

ity score within the team andteam performance,and(e) correlation

between the standard deviation of member cognitive ability scoresand team performance. We did not code reliability estimates for

cognitive ability or teamperformance for several reasons. First, many

studies did not report reliability estimates for either individual-

level cognitive ability or teamperformance. Second, cognitive abil-

ity measures tend to be very reliable, and many studies used objec-

tive performance indices (i.e., counts or standardized algorithms)

that can be assumed to have near perfect interrater reliability

(Mento, Steel,& Karen,1987).Third, in many cases,correction for

measurement error has a minor impact on parameters estimated via

meta-analysis (Koslowsky & Sagie, 1994).

Table 2 contains effect-size information for each independent

sample for the four operational definitions of team-level cognitiveability examined in this study. Effect sizes ranged from strong and

positive toweakand negative. Samplesizes varied froma low of 16

(OBrien & Owens, 1969) to a high of 514 (Sundstrom & Futrell,

1999), with the average study involving 71 teams. With regard to

operational definitions, 24 samples reported a correlation between

themean cognitive ability score of team members and team perfor-

mance (i.e., all but Neuman & Wright, 1999), 16 samples provided

a correlation between the highest member score and team perfor-

mance, 17 samples provided a correlation between the lowest

member score and team performance, and 9 provided a correlation

between the standard deviation of scores and team performance.



11/26

PROCEDURE

We conducted four main analyses corresponding to the follow-

ing operational definitions of team-level cognitive ability: (a) arith-

metic mean of member scores (M), (b) score of the lowest scoring

member (LOW), (c) score of the highest scoring member (HIGH),


TABLE 2: Coding Information for Studies Included in Meta-Analyses

rxy

Study N M HIGH LOW SD Setting

Barrick, Stewart, 51 .23 .03 .02 .22 Field

Neubert, and Mount (1998)

Blades (1976) (1) 49 .02 . . . Field

Blades (1976) (2) 51 .22 . . . Field

Bottoms (1998) 50 .22 .24 .08 . Lab

Brandt (1998) 54.(52)a

.32 .14 .17 .05 Lab

Clayton (1998) 55 .21 .07 .21 .15 Lab

Colarelli and Boos (1992) 86 .24 . . . Field

Devine (1999) 52 .40 .19 .38 .09 Lab

Fiedler and Meuwese (1963) 24 .19 . . . Field

Gully (1997) 81 .38 . . . Lab

Hollenbeck et al. (1999) 76 .26 . . . Lab

LePine, Hollenbeck, Ilgen, 26 .34 .37 .30 . Lab

and Hedlund (1997)

Neuman & Wright (1999) 79 . . .33 . Field

OBrien & Owens (1969) (1) 20 .58 .48 .56 . Lab

OBrien & Owens (1969) (2) 20 .13 .12 .12 . Lab

OBrien & Owens (1969) (3) 16 .52 .32 .49 . Lab

OBrien & Owens (1969) (4) 16 .03 .15 .04 . Lab

OBrien & Owens (1969) (5) 16 .52 .19 .56 . Lab

OConnell (1994) 118 .13 .07 .16 .16 Field

Stevens (1999) 56 .29 . . .20 Field

Sundstrom and Futrell (1999) (1) 117 .43 .30 .28 .10 Lab

Sundstrom and Futrell (1999) (2) 514 .40 .26 .34 .05 Lab

Tziner and Vardi (1983) 115 .30 . . . Field

Williams and Sternberg (1988) 24 .65 .65 .43 . LabZukin (1999) 62 .26 .26 .15 .09 Lab

NOTE:N= number of teams included in the study.M= arithmetic mean of member scores;HIGH = score of the highest scoring member; LOW = score of the lowest scoring member;SD= standard deviation of member scores.a.N= 54 forMandSD, andN= 52 for the HIGH and LOW.


12/26

and (d) standard deviation of member scores (SD). For each main

analysis,a sample-weighted meancorrelation was calculatedusing

the corresponding distribution of effects along with the following

statistics: (a) chi-square test for the homogeneity of observed effect-

sizes (Rosenthal, 1991), (b) percentage of observed variance

accounted for by sampling error (Hunter & Schmidt, 1990), (c)95%

credibility intervalformed around the estimatedpopulation param-

eter using the corrected standard deviation (Hunter, Schmidt, &

Jackson, 1982),and(d)95%confidence interval formedaroundthe

estimated population parameter based on the standard deviation of

the sampling distribution (Whitener, 1990).

The first three statistics were used to evaluate the likelihood thatthe cognitive abilityperformance relationship is moderated at the

team level. The existence of one or more moderators is suggested

when (a) the chi-square test for coefficient homogeneity is statisti-

callysignificant,(b) thepercentageof observedvarianceaccounted

for by sampling error is less than 75%, and (c) the credibility inter-

val is large and/or includes zero. Conversely, the 95% confi-

dence interval assesses the amount of sampling error influencing

the sample-weighted mean correlation. When all of the observed

variation in a distribution of coefficients is accounted for by sam-

pling error, the credibility interval will be zero, and the confidence

interval will reflect the sampling error affecting the estimate of the

one true population relationship. When sampling error accountsfor a relatively small portion of the observed variation in a set of

coefficients, the credibility interval will be nonzero, and the confi-

dence intervalwill provide a measure of thesamplingerror inherent

in theestimate of themean relationship across multiple subgroups.

HunterandSchmidt (1990)noted that support fora particular mod-

erator exists when there is (a) a meaningful difference in the esti-

mated effect size across moderator subgroups and (b) reduced

within-subgroupvariation relative to theoverall distribution (i.e., a

high percentage of variation accounted for by sampling error, a

nonsignificant chi-square, anda small credibility interval that does

not include zero).



13/26

RESULTS

Results of the four main analyses are presented in Table 3. As is

evident from inspection of the table, theMoperational definition

yielded thestrongestestimatedrelationshipwithteamperformance

(r= .294) followed by the LOW (r= .246) and HIGH (r= .208)

indices, whereas theSDindex produced a very weak and negative

estimate (r= .026). Furthermore, 95% confidence intervals cre-

ated around the sample-weighted means for all three operational

definitions involving leveldidnot include zero, suggesting observed

criterion-related validity associated with these indices will gener-

ally be positive. However, the confidence intervals for all threeoperational definitions did overlap, so our expectation that the

mean score wouldproduce a stronger observed relationshipreceived

only limited support (i.e., from the point estimates). Overall, these

data are consistent with thenotion that the three operational defini-

tions involving level have positive relationships with team perfor-

mance,but none is clearly superior. Incontrast, the95%confidence

interval for theSDindex included zero, consistent with the notion

that there may well be no relationship between the dispersion of

member cognitive ability scores and team effectiveness.

Table 3 also contains evidence that other variables moderate the

strength of these team-level relationships. Specifically, the chi-

square tests for homogeneity in the observed coefficients associ-ated with theM, HIGH,andLOWanalyses were statistically signif-

icant, indicating that variability in the distribution of coefficients

was significantly greater than what would be expected by chance.

In addition, sampling error accounted for less than 59% of the

observed variance in sample-weighted effect sizes for these three

analyses, in all cases below the 75% value suggested by Hunter

et al. (1982) as sufficient to conclude that one true effect character-

izes the relationship in the population as a whole. Finally, 95%

credibility intervals created around each sample-weighted mean

correlationwere relatively large forall three estimatesandincluded

zero for the LOW index. In light of this consistency, there is good

reason to suspect that the magnitude of the team-level relationship



14/26

TABLE 3: Summaries for Main and Moderator Meta-Analyses

95%

K N rxy 2

%2,SE Credibility Interval C

Operational definition

HIGH 16 1,209 .208 27.35* 59.16 .027, .389

M 24 1,749 .294* 55.87*** 41.36 .042, .545

SD 9 1,079 .026 11.45 77.33 .124, .071

LOW 17 1,288 .246 39.54*** 40.73 .010, .505 Study-setting moderator

Lab 16 1,199 .368* 15.65 103.36

Field 8 550 .137 17.63* 44.31 .123, .339

NOTE:K = numberof samples inanalysis; N = numberof teamsin analysis;rxy= meansample-weighted correlation;2

= chi-squarevaof coefficients;%

2, SE= percentage of observed variance accounted forby sampling error;HIGH = score of thehighest scoring mem

mean of member scores;SD= standard deviation of member scores; LOW = score of the lowest scoring member.*p< .05. **p< .01. ***p< .001.

520


15/26

between cognitive ability and performance varies across situations

forall threeoperational definitions involvinglevel.Considering the

weak estimated relationship, the narrow confidence interval that

includes zero, and the relatively large amount of observed variance

accounted for by sampling error, it appears from these data that

work group diversity in terms of cognitive ability is essentially

unrelated to team performance in most situations.

Given these data, we proceeded to look for potential moderators

of the team-level relationship. We initially hoped to code the stud-

ies in our database for several task characteristics, but this proved

impossible because (a) most studies provided only a cursory

description of the task and (b) study characteristics that could bereliably coded from available information tended to be highly cor-

related, making it impossible to isolate their effects. In particular,

studies conducted in fieldsettings tendedto involve standing teams

engaged in familiar behavioral tasks with long time frames; con-

versely, labstudies tendedto usenovel intellectual tasks,short time

frames, and ad hoc groups of students who had little familiarity

with one another. As a result, we opted to code only one surface

characteristic: study setting.Lab studies were defined as those that

took place under controlled conditions wherein the focal task was

created solely for the purpose of scientific investigationandhad no

intrinsic importance or meaningfulness. Fieldstudies were defined

as any study that did not meet the definition of a lab study. Further-more, given theextremely smallnumberof studies conductedin the

field that reported correlation for the HIGH and LOW indices, we

onlyexaminedthestudy-settingmoderatorusing thedistribution of

coefficients associated with theMindex.

As seen inTable 3, thestudy-setting moderatorprovides a useful

starting point for understanding the nature of the team-level rela-

tionshipbetween cognitive ability and performance. With regard to

the magnitude of the team-level relationship, the two subgroups

exhibited a large difference: The estimated relationship was con-

siderablystronger in labstudies (r= .37) than actual organizational

settings (r= .14). Furthermore, all the observed variation in study

coefficients associated with lab studies was explained by sampling

error, and the chi-square test for heterogeneity was nonsignificant



16/26


17/26

sentativeness of the studies included in these meta-analyses. Thus,

these results should be viewed with caution, and parameter esti-

mates should be treated as preliminary.

IMPLICATIONS

The positive relationshipobserved for all three operational defi-

nitions associated with cognitive ability level within a team sug-

gests that the functional amount of cognitive ability in teams does

indeed predict team performance across a broad variety of team

contexts. All other things being equal, the cognitive ability of the

most intelligent member accounts for roughly 4.3% of thevariancein team performance, the cognitive ability of the least intelligent

member explains about 6.1%, and the mean cognitive ability of

team members captures approximately 8.6% of the variance in

team performance (i.e., twice as much as thecognitive ability of the

most intelligent member). Furthermore, in situations where it is

desirable to predict how well a team will perform, it appears more

valuable to know the mean level of cognitive ability of members

than the score of the highest or lowest scoring individual.

Perhaps the most important finding of this study, however, was

thestrong evidenceof moderation affecting the team-level relation-

ship forall three operational definitions associated with level. With

regard to composing work groups, the main analyses indicate therelationship between mean cognitive ability and team performance

varies across situations. Furthermore, the moderator analysis find-

ings suggest that in organizational settings, the predictive efficacy

of team-level cognitive ability will be weak in some team contexts

and nonexistent in others. Overall, there is little evidence here to

suggest that cognitive ability tests representa panacea forpractitio-

ners charged with assembling effective work groups.

This of course begs the question of which characteristics are

responsible for the variation in the team-level relationships. Few

empirical studies in the literature have examinedpotential modera-

tors, but theory and empirical research suggest a potential role for

the following: (a) task complexity, (b) degree of physical activity,

and(c) task familiarity. Taskcomplexityisa functionof thenumber



18/26

of information cues and behavioral acts required of a task, as well

as the coordination and adaptation required of members (Wood,

1986).Giventhat task complexity moderates therelationshipbetween

cognitive ability andperformance for individuals (Hunter & Hunter,

1984), team-level indices of cognitive ability maybe more strongly

related to team performance on complex tasks than simple tasks.

With regard to physical activity, intellectual tasks (e.g., planning,

decision making, problem solving) differ from behavioral tasks

(production, assembly, maintenance) in that the latter involve sub-

stantial movement and coordination of team members as well as

their tools or equipment (McGrath, 1984). Although information-

processing demands arecertainlypresent in both types of tasks, theteam-level cognitiveability shouldbe morestronglyrelated to team

performance in intellectual tasks than behavioral tasks due to the

strong influence of psychomotor and physical abilities in the latter.

Finally, regardingtaskfamiliarity, research byKanferandAckerman

(1989) suggests the relationshipbetween cognitive ability and task

performance decreases over time for individuals as they acquire

more experience with a task. Essentially, they arguedthat cognitive

ability is important in the early stages of learning a new task but

becomes progressivelylessimportantas knowledge is acquiredand

skills becomeproceduralized. Extending this argument to the team

level, the correlation between team-level cognitive abilityand team

performance shouldbe highest for novel tasks and shoulddecreaseover time (i.e., repetitions or cycles) as team members become

more familiar with their roles and the roles of other members.

Overall, to the extent several task moderators are operating,

there will likely be situations where team-level cognitive ability

indices are strongly correlated with team performance and other

situations where the two are unrelated or perhaps even negatively

related. Combining the empirical evidence of moderation and the

theoretical discussion above, team-level cognitive ability might be

expected to yield its lowest predictive validity for performance

when team tasks aresimple, familiar, andbehavioral.Applying this

logic to organizational settings, team-levelcognitive abilitymaybe

only marginally useful for predicting the effectiveness of standing

work teams engaged in repetitive, standardizedactivities (e.g., pro-



19/26

duction or assembly teams or maintenance crews). On the other

hand, given the strong relationship found with intellectual tasks in

the lab, team-level cognitive ability may be a better predictor of

performance forad hocteams facinga relatively complex task with

a finite life span (e.g., selection committees, research and develop-

ment teams, or quality circles). Furthermore, there may be higher

order interactions among the task characteristics such that the

effect of one task characteristic depends on the level of another. For

instance, theeffect of task familiarity could be more pronounced in

complex tasks as opposed to simple tasks.

Finally, the results of the main analysis for the standard devia-

tion indexsuggest that thedispersion ofcognitive ability in teamsisnot an important concern for researchers or practitioners. It should

be noted that the samplesize for the SD analysis wasthesmallestof

thefour main analyses,but these studies didshowreasonable heter-

ogeneity with regard to study setting, and there was relatively little

variation in the observed coefficients across studies. Combined

with the lack of a compelling theoretical rationale for why the dis-

persion of cognitive ability shouldbe important, it is unlikely that a

substantial relationship exists between cognitive ability dispersion

in teams and team performance in general. At the same time,

depending on the nature of the task, it is conceivable that variation

in member cognitive ability (or lack thereof) might be related to

team performance in some situations.

FUTURE RESEARCH DIRECTIONS

Given thepreviousdiscussion,a good firststep wouldbe to iden-

tify the team contexts where the cognitive ability of team members

is most strongly related to team performance. The hypotheses

implied above could be tested in controlled settings by having

teams engage in tasks that vary along key task dimensions, allow-

ing examination of potential interactions among task characteris-

tics and changes over time. Alternatively, given the complex con-

texts and multiple task responsibilities associated with real work

groups, it might be possible to agree on a taxonomy of work group

types found in organizational settings (e.g., Sundstrom, 1999) and



20/26

then examine the different types in field settings. This would sacri-

fice precision related to identifying operative characteristics in

favor of greater realism and enhanced generalizability. Takentogether,

the two strategies could further ourunderstanding of themicro task

characteristics that affect the team-level relationship as well as the

macro team contexts in which team-level cognitive ability is most

important.

In addition to identifying theconditions when it is most andleast

important to maximize the cognitive ability of work groups, it

would also be helpful to identify particular roles where cognitive

ability is most important once major team types have been distin-

guished. Borrowing Steiners (1972) emphasis on the criterion, itmight be useful to usecognitive ability measures topredict individ-

ualrole performancesandthen focus onbuildingcompilationmod-

els relating individual role performances to team performance in

specific contexts (Kozlowski & Klein, 2000). In particular, mem-

ber role performances could be represented by main effects and

dyadic and triadic performance-related exchanges represented by

higher order interaction terms (i.e., AB, ABC). Given the strong

link between individual-level cognitive ability and job (i.e., role)

performance, specific roles or role exchanges could be identified

where cognitive ability is most important in particular team types

with fairly standard roles (e.g., sports teams or top management

groups). For instance, if a strong effect were obtained for the ABinteraction term, this would suggest that the quality of the

task-related exchange between Members A and B was strongly

related to overall team performance. Given that cognitive ability is

a good predictorof individual performance,a manager or executive

could then try to ensure that two of the more intelligent members

occupied these roles on the team.

In addition to this focus on settings and specific member roles,

practitioners could also benefit from more research on how

team-level cognitive ability relates to team dynamics. In particular,

it would also be useful to have a better understanding of how the

cognitive ability of members affects team outcomes via process

variables such as task-related communication, behavioral coordi-

nation, interpersonal conflict, encouragement, and performance



21/26


22/26

CONCLUSION

Team-level cognitive ability appears to be positively related to

team performance for three operational definitions that focus on

amount, but themagnitudeof all three relationships appears to vary

as a function of one or more contextual variables. Although we

have highlighted several moderator candidates pertaining to thetask, there is currentlynotenoughdata available to identify the fac-

tors responsible for the observed variation in these relationships.

Most important, the findings from this study suggest that team-

level cognitive ability indices may not be good predictors of team

performance in some organizational settings. More research is

neededto identify the team contexts where cognitive ability is most

and least useful as a predictor of team performance.

REFERENCES

Barrick, M. R., Stewart, G. L., Neubert, M. J., & Mount, M. K. (1998). Relating member

ability and personality to work-team processes and team effectiveness. Journal of

Applied Psychology,83(3), 377-391.


Individual

Cognitive Ability

Team-Level

Cognitive Ability

LOW

HIGH

MEAN

SD

Team Outcomes

Effectiveness

Viability

Task Characteristics

Complexity

Physical Activity

Familiarity

Team Process

Information

Sharing

Information

Integration

Conflict

Coordination

Individual

Role Performance

Figure 1: A Conceptual Model of the Cognitive AbilityTeam Performance

Relationship


23/26

Blades, J. W. (1976).The influenceof intelligence,task abilityand motivationon group per-

formance (Unpublished doctoral dissertation, University of Washington).Dissertation

Abstracts International,37, 1463.

Bottoms, N. (1998).Group decision-making in a simulated business environment. Unpub-

lished bachelors honors thesis, Indiana UniversityPurdue University, Indianapolis.

Brandt, K. M. (1998).The effects of personality compatibility on team processes and team

performance. Unpublished masters thesis, Indiana UniversityPurdue University,

Indianapolis.

Clayton, L. D. (1998).The effects of gender composition and task complexity on group pro-

cesses and performance. Unpublished masters thesis, Indiana UniversityPurdue Uni-

versity, Indianapolis.

Cohen,S. G.,& Bailey, D. E. (1997). What makes teams work:Groupeffectivenessresearch

from the shop floor to the executive suite.Journal of Management,23, 239-290.

Colarelli, S. M., & Boos, A. L. (1992). Sociometric and ability-based assignment to work

groups: Someimplications for personnelselection.Journal of OrganizationalBehavior,

13, 187-196.

Devine, D. J. (1999). Effects of cognitive ability, task knowledge, information sharing, and

conflict on group decision-making effectiveness.Small Group Research,30, 608-634.

Fiedler, F. E., & Meuwese, W.A.T. (1963). Leaders contribution to task performance in

cohesiveand uncohesivegroups.Journalof AbnormalandSocial Psychology, 67, 83-87.

Gully, S. M. (1997).The influences of self-regulatory processes on learning and perfor-

mance in a team training context. Unpublished doctoral dissertation, Michigan State

University, East Lansing.

Gurnee, H. (1937). Maze learning in the collective situation. Journal of Psychology,3,

437-443.

Guzzo, R. A. (1996). Fundamental considerations about work groups. In M. A. West (Ed.),

Handbook of work group psychology(pp. 3-21). San Francisco: Jossey-Bass.

Guzzo,R. A.,& Dickson, M. W. (1996). Teams in organizations:Recent research on perfor-

mance and effectiveness.Annual Review Psychology,47, 307-338.

Hinsz, V. B., Tindale, R. S., & Vollrath, D. A. (1997). The emerging conceptualization of

groups as information processors.Psychological Bulletin,121, 43-64.Hollenbeck, J. R., Ilgen, D. R., Sheppard, L., Ellis, A., Moon, H., & West, B. (1999, May).

Person-team fit: A structural approach. Paper presented at the 14th Annual Meeting of

the Society for Industrial Organizational Psychology, Atlanta, GA.

Hunter,J. E.,& Hunter,R. F. (1984).Validity andutilityof alternatepredictors of jobperfor-

mance.Psychological Bulletin,96, 72-98.

Hunter, J.E., & Schmidt,F.L. (1990).Method of meta-analysis:Correctingerrorandbias in

research findings. Newbury Park, CA: Sage.

Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982).Meta-analysis: Cumulating research

findings across studies. Beverly Hills, CA: Sage.

Ilgen, D. R. (1999). Teams embedded in organizations: Some implications.American Psy-

chologist,54, 129-139.

Jackson, S. E. (1996). The consequences of diversity in multidisciplinary work teams. In

M. A. West (Ed.),Handbook of work group psychology(pp. 53-76). Chichester, UK:

Wiley.

Kabanoff, B., & OBrien, G. E. (1979). Cooperation structure and the relationship of leaderandmember abilityto group performance.Journalof Applied Psychology, 64, 526-532.



24/26

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/

aptitude-treatment interaction approach to skill acquisition.Journal of Applied Psychol-

ogy,74, 657-690.

Klein,K. J.,Dansereau, R.G., & Hall, R.J. (1994). Level issuesin theorydevelopment,data

collection, and analysis.Academy of Management Review,19, 195-229.

Koslowsky, M., & Sagie, A. (1994). Components of artifactual variance in meta-analytic

research.Personnel Psychology,47, 561-574.

Kozlowski, S.W.J., & Klein, K. J. (2000). A multilevel approach to theory and research in

organizations: Contextual, temporal, and emergent processes. In S.W.J. Kozlowski &

K. J. Klein (Eds.),Multilevel theory, research, and methods in organizations: Founda-

tions, extensions, and new directions(pp. 3-90). San Francisco: Jossey-Bass.

Larson, J. R., & Christensen, C. (1993). Groups as problem-solving units: Toward a new

meaning of social cognition.British Journal of Social Psychology,32, 5-30.

LePine, J. A.,Hollenbeck, J. R.,Ilgen,D. R.,& Hedlund, J. (1997). Effects of individualdif-

ferences on the performance of hierarchical decision-making teams: Much more thang.

Journal of Applied Psychology,82(5), 803-811.

McGrath, J. E. (1984). Groups: Interaction and performance. Englewood Cliffs, NJ:

Prentice Hall.

Mento,A. J.,Steel, R. P., & Karren, R. J. (1987). A meta-analytic study of theeffectsof goal

setting on task performance.Organizational Behavior and Human Decision Processes,

39, 52-83.

Milliken, F. J., & Martins, L. L. (1996). Searching for common threads: Understanding the

multiple effects of diversity in organizational groups.Academy of Management Review,

21, 402-433.

Neisser,U., Boodoo,G., Bouchard, T. J., Jr.,Boykin,A. W.,Brody, N.,Ceci, S.J.,Halpern,D. F.,

Lohelin, J. C., Perloff, R., Sternberg, R. J., & Urbina, S. (1996). Intelligence: Knowns

and unknowns.American Psychologist,51, 77-101.

Neuman, G.A.,& Wright,J. (1999). Team effectiveness:Beyondskillsand cognitive ability.

Journal of Applied Psychology,84, 376-389.

OBrien, G. E., & Owens, A. G. (1969). Effects of organizational structure on correlations

between member abilities and group productivity.Journal of Applied Psychology,53,525-530.

OConnell, M. S. (1994). The impact of team leadership on self-directed work team perfor-

mance: A field study (Unpublished doctoral dissertation, University of Akron).Disser-

tation Abstracts International,55, 1699.

Robinson, W. S. (1950). Ecological correlations and the behavior of individuals.American

Sociological Review,15, 351-357.

Rosenthal, R. (1991).Meta-analytic procedures for social research. Newbury Park, CA:

Sage.

Rousseau, D. M. (1985). Issues of levelin organizational research. In L. L. Cummings &

B. Staw (Eds.),Research in organizational behavior(Vol. 7, pp. 1-37). Greenwich, CT:

JAI.

Schmidt,F.L.,& Hunter,J. E. (1998).The validityand utilityof selectionmethodsin person-

nel psychology: Practical and theoretical implications of 85 years of research findings.

Psychological Bulletin,124, 262-274.

Spector, P., & Suttell, B. J. (1957).An experimental comparison of the effectiveness of threepatterns of leadership behavior(Unpublished technical report, AIR-196-57-FR-164).

Washington, DC: American Institute for Research.



25/26

Steiner, I. D. (1972).Group process and productivity. New York: Academic Press.

Sternberg, R. J. (1997). The concept of intelligence and its role in lifelong learning and suc-

cess.American Psychologist,52, 1030-1037.

Stevens, M. J., Jones, R. G., Fischer, D. L., & Kane, T. D. (1999).Team performance and

individual effectiveness: Personality and team context. Paper presented at the 14th

Annual Meeting of the Society for Industrial and Organizational Psychology, Atlanta,

GA.

Sundstrom, E. (1999). The challenges of supporting work team effectiveness. In E. Sundstrom

(Ed.),Supporting work team effectiveness: Best management practices for fostering

high performance(pp. 3-23). San Francisco: Jossey-Bass.

Sundstrom, E.,& Futrell, D. A. (1999). Work group productivity andgeneral mental ability:

Two field studies of group composition. Unpublished manuscript.

Terborg, J. R.,Castore, C.,& DeNinno, J. A. (1976).A longitudinalfield investigationof the

impact of group composition on group performance and cohesion.Journal of Personal-

ity and Social Psychology,34(5), 782-790.

Thorndike, E. L. (1939). On the fallacy of imputing the correlations found for groups to the

individuals or smaller groups composing them.American Journal of Psychology,52,

122-124.

Tziner, A., & Eden, D. (1985). Effects of crew composition on crew performance: Does the

whole equal the sum of its parts?Journal of Applied Psychology,70(1), 85-93.

Tziner, A., & Vardi, Y. (1983). Ability as a moderator between cohesiveness and task crew

performance.Journal of Occupational Behavior,4, 137-143.

Wagner, R. K. (1997). Intelligence, training, and employment.American Psychologist,52,

1059-1069.

Weingart, L. R. (1997). How did they do that? The ways and means of studying group pro-

cess.Research in Organizational Behavior,19, 189-239.

Whitener, E. M. (1990). Confusion of confidence intervals and credibility intervals in

meta-analysis.Journal of Applied Psychology,75, 315-321.

Williams, K. Y., & OReilly, C. A., III. (1998). Demography and diversity in organizations:

A review of 40 years of research.Research in Organizational Behavior,20, 77-140.

Williams, W. M.,& Sternberg, R. J. (1988). Groupintelligence: Whysomegroups arebetterthan others.Intelligence,12, 351-377.

Wonderlic Personnel Test. (1992).Wonderlic Personnel Test & scholastic level exam users

manual. Milwaukee, WI: Author.

Wood, R. E. (1986). Task complexity: Definition of the construct. Organizational Behavior

and Human Decision Processes,37, 60-82.

Zukin, L. (1999).Individual differences and individual-in-team performance: An investiga-

tion of the impact of personality and ability in a decision making team with distributed

information. Unpublisheddoctoral dissertation,George MasonUniversity,Fairfax, VA.

Dennis J. Devine is an assistant professor of industrial and organizational psychol-

ogyat Indiana UniversityPurdue University, Indianapolis.He receivedhis Ph.D.in

industrial and organizational psychology from Michigan State University in 1996.

His research interests include team dynamics and effectiveness, jury decision mak-ing, expert-novice differences in job performance, and organizational recruitment.



26/26

Jennifer L. Philips is a 1st-year graduatestudent in the industrial and organizational

psychologyPh.D. program at the University of Akron.Her research interests include

the effects of team member composition on team effectiveness, group decision mak-

ing, legal influences on human resources decision making, organization culture or

climate, and organizational citizenship behaviors.


Documents

A Meta-Analysis of Cognitive Ability and Team Performance