Not as Smart as We Think: A Study of Collective Intelligence in Virtual Groups

Last updated February 10, 2014

ABSTRACT

Organizations are increasingly using virtual groups for many different types of work, yet

little research has examined factors that make groups perform better across multiple different

types of tasks. Recent research in Science (Woolley et al. 2010) proposed that groups, like

individuals, have a general factor of collective intelligence. Groups high in collective intelligence

perform well across multiple types of tasks while those low in collective intelligence do not. We

studied groups that used computer-mediated communication (CMC) to investigate whether the

concept of collective intelligence is similar or different when groups work in this important

context. A collective intelligence factor did not emerge among groups using CMC, suggesting

that, unlike individual intelligence, collective intelligence manifests itself differently depending

on context. Individual performance was correlated with individual intelligence, but virtual group

performance was disconnected from both individual intelligence and collective intelligence,

suggesting that task-specific group work processes trump inherent group characteristics when

using CMC. We also evaluate which factors were relevant for specific task types in computer-

mediated work, and suggest reasons that certain task characteristics are so important in

influencing group performance and inhibiting general collective intelligence.

Keywords: Collective intelligence; group performance; task types; intelligence; computer-

mediated communication

Not as Smart as We Think: A Study of Collective Intelligence in Virtual Groups

INTRODUCTION

Groups increasingly use information and communication technologies (ICT) to enable

virtual work (Chudoba et al. 2005; Montoya et al. 2011). Some groups are entirely virtual, being

composed of geographically dispersed individuals who communicate only through ICT, while

other groups use ICT to augment face-to-face work (Bell and Kozlowski 2002) or support

partially-distributed teams (Privman et al. 2013).

The effectiveness of groups using ICT is a major research stream in information systems

(Kahai et al. 2007). Such research generally focuses on one type of task (e.g., decision making,

idea generation), or, alternatively, compares performance on one type of task against another.

However, many groups are called on to complete multiple types of tasks. In some cases, teams

have multiple responsibilities that involve working together over time on a variety of tasks. Even

in cases where groups only work together on one project, complex projects can require the group

to perform well on many different types of sub-tasks within the project, such as generating ideas

and alternatives, making decisions among alternatives, planning, and so on. Little research

examines the traits of groups that lead to consistently high performance across the multiple types

of tasks that many groups perform, or whether such traits exist at all. That is, no research has

examined whether some virtual groups are inherently “smarter” than others, being able to

perform well regardless of task, or whether virtual group task performance is more dependent on

the task such that there are no “smarter” groups that excel across all types of tasks.

This paper uses the concept of collective intelligence to examine whether groups using

ICT can perform consistently across different types of tasks. In other words, we seek to know if

groups have some inherent characteristic (i.e., a “collective intelligence” factor) that can make

them perform consistently across multiple types of tasks when using ICT.

Individual intelligence is generally a good predictor of performance on a variety of

individual tasks (Deary 2000). Individual intelligence has been studied for decades, but we are

just starting to see research on the intelligence of groups. A widely publicized study by Woolley

et al. (2010) in Science reported that groups, like individuals, have a certain level of “collective

intelligence,” such that groups which perform well on one type of task will perform well on

others. In a pair of empirical studies of face-to-face groups, they found evidence of a collective

intelligence factor, a measure of consistent group performance across a series of tasks, which

was highly predictive of performance on a subsequent, more complex task. This collective

intelligence factor differed from the individual intelligence of group members, and was

significantly predicted by members’ social sensitivity – the ability to understand the emotions of

others based on visual facial cues (Baron-Cohen et al. 2001). Equality of participation and the

proportion of females in the group were also correlated with collective intelligence.

Researchers have called for more studies to more fully understand these findings (Deary

2012), to further distinguish the effects of individual intelligence and collective intelligence on

group performance (Deary 2012), and to investigate other possible predictors of collective

intelligence (Woodley and Bell 2010). One key question is whether the factor measured by

Woolley et al. (2010), which is highly correlated with visual processing (something that is not

available in many ICT used by groups), transcends media or is instead a measure of a group’s

capability to interact in one media where visual processing dominates, i.e., face-to-face

communication. Because groups routinely use ICT (Chudoba et al. 2005; Jones et al. 2008;

Montoya et al. 2011), it is important to understand collective intelligence when groups use ICT

as well as when they interact face-to-face. Is collective intelligence like individual intelligence,

an inherent factor that transcends media, or is collective intelligence constrained by media or

task type? While some groups working face-to-face seem to be more collectively intelligent than

others, does the same effect exist for groups that work virtually? If so, what factors can explain

this collective intelligence? If not, what are the implications for groups electing to use CMC for

multiple types of tasks?

We examined collective intelligence in an ICT environment in which groups used text-

based computer-mediated communication (CMC) because this form of ICT is quite different

from the face-to-face environment in which the original collective intelligence factor was

developed. If we find evidence of collective intelligence in this environment that is correlated to

the same factors in the Woolley et al. (2010) study of face-to-face interaction, then we can be

more confident that that this factor is indeed like general intelligence and transcends media. If we

do not find evidence of a collective intelligence factor in groups using CMC, then we question

the generalizability of previous findings and suggest that collective intelligence, as a concept,

differs depending on context, and that when using CMC, task requirements may trump any

inherent group characteristics. These contributions will help researchers and practitioners to

better understand across-task performance of groups using CMC. The findings also open up

several new opportunities for research on collective intelligence and efforts to make virtual

groups “work smarter.”

PRIOR RESEARCH AND THEORY

Collective intelligence

Woolley et al. (2010) proposed that groups, like individuals, have a certain level of

intelligence. That is, they propose that groups that perform well on one type of task also tend to

perform (generally) well on other types of tasks, much like individual intelligence measures

cognitive ability by measuring performance on a variety of individual cognitive tasks. Thus, we

define collective intelligence as the ability of a group to perform consistently well across

multiple types of tasks. This definition distinguishes the concept of collective intelligence from

several other constructs that have been studied in the virtual group literature.

A long line of research in information systems and related fields has considered various

constructs related to group cognition and group performance. These include transactive memory

systems (Choi et al. 2010; Lewis and Herndon 2011; Ren and Argote 2011), collaboration know-

how (Majchrzak et al. 2005), distributed cognition (Boland et al. 1994; Hollan et al. 2000), group

memory (Wittenbaum 2003), collective mindfulness (Weick et al. 1999), team mental models

(Resick et al. 2010; Thomas and Bostrom 2007), and others (Fuller et al. 2006; Hinsz et al. 1997;

Martins et al. 2012; Thomas and Bostrom 2010).

However, none of these studies have examined the intelligence of a group in terms of the

ability to perform well consistently across conditions such as media and task. While these

previously-studied constructs may be correlated with the intelligence of a group, or contribute to

a group’s collective intelligence, the conceptualization of collective intelligence is fundamentally

different from these constructs, which do not focus on consistently high performance across

different tasks. Much like creativity is a trait that is useful in brainstorming, but not necessarily

decision making, these previously-proposed group-level cognitive constructs have been shown to

be predictors of performance for some types of tasks, but not across a range of task types. Hence

they are not collective intelligence per se.

For example, the transactive memory system of a group is defined as the system of

processes and structures that group members develop to encode, store, and retrieve knowledge,

including knowledge about the skills and knowledge of each group member (Ren and Argote

2011). There is a clear distinction between the conceptualization of a memory system (the

structure and processes used by a group) and the conceptualization of intelligence (consistent

performance across a variety of tasks). While transactive memory as a skill is likely valuable to

groups in many situations, research suggests that transactive memory systems are not relevant

across all types of tasks (Akgun et al. 2005; Lewis and Herndon 2011).

“Collaboration know-how” is another virtual collaboration concept that has been studied

for different types of tasks and has also been found to have different effects on performance for

different task types (Majchrzak et al. 2005), suggesting that it is not a general intelligence

concept (which by definition, is consistent across task types). While collaboration know-how can

clearly benefit virtual groups, this is an individual-level construct and does not represent the

ability of a group as a whole to perform consistently well. Similar arguments could be made for

the other concepts listed above as to why they are fundamentally different from (though not

necessarily unrelated to, or perhaps even predictive of) the concept of collective intelligence.

In sum, no study to date has determined whether groups working virtually have a certain

level of intelligence analogous to the intelligence of individuals, one of the most-studied

individual-level factors in the social sciences.

Initial findings on collective intelligence

Woolley et al. (2010) carried out a pair of empirical studies with 192 groups of 2-5

people working face-to-face on a set of tasks. Similar to the measurement of individual

intelligence, they used a factor analysis on the performance scores across tasks to assess

collective intelligence. As with individual intelligence, they found that one dominant factor

emerged from the factor analysis, which they labeled collective intelligence. They also found,

like the correlation between individual intelligence and performance, that collective intelligence

was predictive of group performance on a separate, complex group task.

What makes a group collectively intelligent -- i.e., able to perform well on multiple types

of tasks? Like other models of complex behavior, Woolley et al. (2010) found that collective

intelligence was not necessarily a direct reflection of individual-level characteristics. The

collective intelligence factor was only slightly correlated with the individual intelligence of

group members (both average and maximum), and individual intelligence, unlike the collective

intelligence factor, was not related to group performance on the complex task. Variables that

were significantly correlated with collective intelligence were social sensitivity of group

members (Baron-Cohen et al. 2001), variance of speaking turns, and percentage of females in the

group. However, in a regression model with all three factors as independent variables, only

social sensitivity was a significant predictor of collective intelligence.

Social sensitivity is a measure of how well individuals can understand the emotions and

feelings of others based on visual cues (Baron-Cohen et al. 2001; Woolley et al. 2010). It is

measured by having participants look at photographs of individuals’ eyes and identify the

emotion expressed (Baron-Cohen et al. 2001; Bender et al. 2012; Kress and Schar 2012).

Therefore, the collective intelligence factor found by Woolley et al. (2010) depends heavily on

the visual understanding of emotions between group members as they work together, something

that is absent when groups use some forms of ICT, such as text-based CMC.

Collective intelligence in virtual work?

The key question of interest in our study is whether the factor claimed to be collective

intelligence by Woolley et al. (2010) is an inherent characteristic that transcends technology,

much like individual intelligence, or whether their findings apply only to face-to-face group

work, with collective intelligence emerging in a different manner (or not at all) when groups use

ICTs that lack visual cues. If a general collective intelligence factor like individual intelligence

exists, then it should appear when groups work face-to-face and when they work using ICT.

Virtual work could include the use of text-based tools (e.g., e-mail, instant messaging),

audio-based tools (e.g., mobile phone, voice-over-IP), video-based tools (e.g., video chat), or a

combination of any of these tools. For this study, we chose to focus on virtual work using a text-

based CMC, as it could be considered most different from face-to-face interaction because it

lacks visual and audio cues.

Group members working virtually using text-based CMC are separated spatially and

sometimes even temporally. These barriers add problems and cognitive challenges in addition to

those faced by groups working face-to-face. Several problems, including communication failures

and misunderstanding, lead to problems in maintaining mutual knowledge in virtual work

(Cramton 2001). CMC has distinct properties as compared to face-to-face discourse, including

different levels of interaction, organization, and language use (Abbasi and Chen 2008).

Groups communicating virtually refer less to personality cues and physical appearance,

and have increased opportunities to self-censor, than face-to-face groups, where behavior is

sometimes biased by physical appearance (McCauley 1998; Walther 1996). Groups working

virtually sometimes have limitations on visual and audio cues. This suggests that social

sensitivity, a measure of how well individuals understand the emotions of others based on visual

cues (Baron-Cohen et al. 2001; Woolley et al. 2010), may not be a relevant factor that makes

groups more intelligent (i.e., high performers across tasks) in a CMC context.

The collective intelligence factor found by Woolley et al. (2010) depends heavily on

social sensitivity, which is measured by having participants identify the emotion expressed in

photographs of individuals’ eyes (Baron-Cohen et al. 2001; Kress and Schar 2012). This visual

understanding of emotions can be difficult or impossible with CMC that does not provide visual

cues, such as textual chat or audio conferencing. Therefore, previously reported results of

collective intelligence may not be directly relevant to groups working virtually.

Previous research on virtual work suggests that both individual performance (Kang et al.

2012) and group performance (Maruping and Agarwal 2004) depend not only on what type of

task the group is performing, but how well the technology fits with, or is appropriated for, the

given task (Dennis et al. 2008; Fuller and Dennis 2009). This research shows that task type is an

important factor influencing the outcome of collaborative work but it has not examined whether

groups perform consistently across tasks when they use CMC (i.e., with some “intelligent”

groups generally outperforming others regardless of task). Though we know that task and

technology affect the performance of groups, we do not know if some groups perform more

effectively in both optimal and suboptimal conditions of task-technology fit.

If groups do possess a collective intelligence, then we would expect groups high in

collective intelligence to perform better than groups low in collective intelligence across a range

of tasks. This pattern should hold whether those groups meet face-to-face or use ICT, and

whether or not the ICT is a good fit for the task. (In fact, one might expect smarter groups to

emerge in tasks with poor technology fit because they might be better at adapting themselves, the

technology, or the task to excel.

A study of collective intelligence in groups using CMC should result in one of three

outcomes, each of which would provide an important contribution to research on collective

intelligence and to the understanding of virtual group performance across tasks.

First, if a collective intelligence factor emerges and is highly correlated with social

sensitivity and the other variables found by Woolley et al. (2010), then our research would

validate the factor proposed by Woolley et al. (2010) as a general form of collective intelligence

that transcends media. Thus, regardless of the media chosen, truly intelligent groups would be

high performers across multiple tasks whether they met face-to-face or via ICT. This outcome

would also suggest that visual social sensitivity is an important factor that can predict success

even in settings with limited visual indications of emotion.

Second, if a collective intelligence factor emerges, but is not correlated with social

sensitivity or the other variables found by Woolley et al. (2010), the results would limit the

generalizability of their findings. This would suggest that there is a general collective

intelligence factor inherent to groups but that other factors (such as a non-visual form of social

sensitivity) become more important for groups when they use CMC in which visual cues are

limited. In other words, the factor proposed by Woolley et al. (2010) may be important for face-

to-face group work, but collective intelligence may be related to different group characteristics

when groups use ICT.

Third, if a collective intelligence factor does not emerge when groups use CMC, we

would conclude that this factor, only previously validated in face-to-face group work, cannot be

considered general collective intelligence in the same sense as an individual intelligence factor.

Rather, this outcome would suggest that collective intelligence (i.e., the consistency of

performance across different types of tasks) may be inherent when groups work face-to-face, but

fails to appear when they work virtually – performance is not consistent across different tasks.

In other words, it may be that the collective intelligence of a group can only be developed

under certain circumstances. Some groups may be more “intelligent” (i.e., high performing) than

others, but only when the conditions are right. Such an outcome would demonstrate that for

CMC, an inherent disconnect exists between task performance and inherent group

characteristics, which makes performance more dependent on task type than on any inherent

group characteristic. This would lead us to conclude that different types of tasks have different

requirements, and thus the processes groups use to perform them are likely more important in

this setting than underlying group characteristics.

Any one of these conclusions do not depend on perfect replication of the Woolley et al.

(2010) studies. In fact, if we were to conduct a perfect replication of their studies (with the only

exception being that the study is carried out using ICT), we would not know if the collective

intelligence factor we studied was somehow unique to the tasks and subjects used, and whether

or not the findings could be generalized to other groups and tasks. Thus to investigate whether

the collective intelligence factor found by Woolley et al. (2010) is generalizable to groups using

CMC, we only need to show that when groups use CMC, performance on any set of diverse tasks

is either (1) consistent, with highly intelligent groups performing better, or (2) not consistent,

with no groups emerging as having any characteristic that is linked to higher performance across

multiple tasks. The results of our study when compared to the results published in Science are

sufficient to establish whether a collective intelligence factor as defined by both Woolley et al.

(2010) and us emerges in groups using CMC. Such a comparison, without perfect replication nor

new data collection of face-to-face groups, gives valuable insight into the patterns of

performance of groups using CMC and the question of whether such groups can be collectively

intelligent or must alternatively focus on task requirements to improve performance.

Factors predicting collective intelligence

In addition to understanding the collective intelligence factor itself, we examine how

various constructs are related to collective intelligence, in this case for groups using ICTs. If a

collective intelligence factor does not emerge when groups use ICT, it may be important to

understand which constructs are related to performance for specific task types so as to better

understand why across-task performance (i.e., collective intelligence) is not consistent. Such an

understanding is also important in order to help groups cultivate their collective intelligence to

perform better on a variety of tasks. We used the factors from Woolley et al. (2010) as predictors

of collective intelligence: individual intelligence, social sensitivity, variance of speaking turns,

and females as a percentage of group members.

Individual intelligence is the ability of an individual to perform well across a variety of

cognitive tasks (Devine and Philips 2001; Woolley et al. 2010). Individual intelligence is related

to individual performance across multiple tasks, so the average intelligence of group members

may have an effect on the overall performance of the group. A meta-analysis showed that

individual cognitive ability has a small to moderate effect on group performance, though this

effect was attenuated by familiarity with a given task (Devine and Philips 2001). Woolley et al.

(2010) had mixed results with individual intelligence. In Study 1, aggregated individual

intelligence was not correlated with collective intelligence, but the correlation was significant for

Study 2. Therefore, it remains unclear how individual intelligence is related to collective

intelligence. The effects of individual intelligence should not differ by media used, so we predict

that individual intelligence should have a small effect on the general performance of groups

working virtually. Based on prior research, it is unclear whether the effect of individual

intelligence on group performance varies by task (Devine and Philips 2001).

Social sensitivity is the ability to perceive and understand the feelings and viewpoints of

others (Bender et al. 2012). It is measured using the Reading the Mind in the Eyes test (Baron-

Cohen et al. 2001). The construct is significantly related to group performance in face-to-face

settings (Bender et al. 2012; Kress and Schar 2012). Although members of groups working

virtually may be able to detect other group members’ emotions without visual cues (e.g., through

interpretation of textual cues), the social sensitivity construct is measured through a test where

individuals must interpret emotion or intentions through visual cues (Baron-Cohen et al. 2001;

Woolley et al. 2010). Thus, the construct measured using this test may not be as salient when

groups use ICT that lack visual cues. Although research exists on the effects of emotion in text-

based environments, we know of no analogous text-based test of an individual’s ability to

interpret complex social or emotional cues in a text-based environment. Further, while research

has shown a correlation between ability to interpret emotion through visual cues and the ability

to interpret emotion through audio cues (Zuckerman et al. 1975), research has not shown a link

between ability to interpret visual cues with ability to interpret text cues.

Variance of speaking turns is a measure of the distribution of communication within a

group. Groups with a high variance of speaking turns are those in which some members of the

group dominate discussion while other members of the group contribute proportionally less.

Woolley et al. (2010) found variance of speaking turns to be correlated with collective

intelligence, but not a significant predictor of the intelligence factor in a regression model. Some

research has found distribution of speaking turns to be relevant to group success (Borge et al.

2012; Dong et al. 2012), with more equal distribution being preferred. Essentially, such research

argues that equal distribution of communication implies equal participation and contribution

from group members. However, depending on the makeup of the group and the task at hand,

variance in speaking turns may not be essential for a group to succeed. Use of text-based CMC

results in more equal participation (Garfield et al. 2001; Straus 1996), so this factor may have

less effect when groups work virtually.

Females as a percentage of group members is calculated for each group to measure the

effects of gender, which have been shown to be important in some research (Adams and Funk

2012; Apesteguia et al. 2012). The correlation of this variable with face-to-face collective

intelligence was a result of women scoring higher on social sensitivity. Again, groups with

limited visual cues may not see the same effect. However, gender does play an important role in

the way individuals use technology (Venkatesh and Morris 2000). Research on gender effects

often concludes that group success is less dependent on the presence of a particular gender than

on the positive effect of a mix of both genders (Adams and Funk 2012; Apesteguia et al. 2012).

METHOD

Participants

Participants were 324 undergraduate and graduate students at a large university business

school randomly organized into 86 groups of 3-5 members. Student samples are appropriate for

testing theories about phenomena that are theorized to hold true across the population (Compeau

et al. 2012). If a collective intelligence factor does not emerge in groups of students, previous

findings on collective intelligence cannot be generalized across all populations. In other words,

we aim to understand whether collective intelligence is, like individual intelligence, an inherent

factor regardless of media. Showing the emergence or non-emergence of this factor in any given

subset of the population (including students) will either confirm the generalizability of previous

findings on collective intelligence, or will show the need to better understand collective

intelligence and how it differs by context.

In some classes, participation in a research study was required. In other classes, students

were offered an incentive such as course extra credit or entry in a drawing for a cash prize. 42.9

percent of the participants were female, and the average age of participants was 21 years (s.d. =

1.85). As a comparison, the average age in the Woolley et al. (2010) studies was in the twenties.1

Tasks and Performance Measures

Groups used the Gmail chat software, which is a commonly used text-based CMC tool

that is similar to other text-based CMC tools. Participants communicated using only Gmail Chat.

Generic Gmail accounts were used for the study (e.g., “teammember.a1”) so the individuals were

not aware of which other individuals in the lab were communicating which messages.

Following Woolley et al. (2010), groups completed three initial tasks and one complex

task. The initial tasks were used in the factor analysis to find a collective intelligence factor,

while the complex task was used to have a single measure of group performance separate from

the collective intelligence tasks, and which would evaluate the performance-predicting

capabilities of the collective intelligence factor. Like Woolley et al. (2010), we selected the three

initial tasks from different quadrants of the McGrath group task circumplex (McGrath 1984),

which classifies and explains four different types of tasks that groups perform—brainstorming,

decision, negotiation, and execution—that each requires different cognitive processes. We chose

to use only three tasks—brainstorming, decision and negotiation—due to time constraints. The

fourth task type – an execution task – was not used in this study, as all tasks require some form

of execution. Tasks used in previous research classified as strictly execution tasks are busy work,

such as typing a document or replicating a set of patterns.

Brainstorming: Participants were given seven minutes to brainstorm ideas to increase tourism to

the city in which the university was located. The tourism task is a classic brainstorming task that

1 Woolley et al.’s Study 1 had 120 participants, whose average age was 32. Woolley et al.’s Study 2 had 579 participants; the average age of the Boston participants was 29 and the average age of the Pittsburgh participants was 23. Woolley et al. do not list how many participants were in Boston and how many were in Pittsburgh.

has been used in many studies (Dean et al. 2006; Dennis et al. 2013; Pinsonneault et al. 1999).

As in the Woolley et al. (2010) studies, performance on the brainstorming task was determined

as the number of unique ideas produced by each group. Three coders independently counted the

ideas from the transcripts. Inter-rater reliability calculations indicated adequate agreement among

raters (Fleiss’ kappa = 0.92).

Decision: The college admissions task, a hidden profile task, was used for the decision portion.

A hidden profile task is a task in which each group member first considers a set of incomplete

information, and then the group comes together to make a final decision. Incomplete information

includes both common information, known to every group member, and unique information,

known to only a subset of group members (Stasser 1992). A variation of the college admissions

task has been used in several studies (Fuller and Dennis 2009; Zigurs et al. 1988). Individuals

were given information about four hypothetical candidates applying to a university. Participants

were then asked to decide, as a group, which candidates to admit and which to deny. Groups

were allowed to admit up to two of the four candidates after discussing them for 12 minutes.

Groups received one point for each correct decision (admit or deny), resulting in a performance

score ranging from zero to four. The correct decision for each of the candidates was based on the

criteria for admitting students and validated by university admissions officers.

Negotiation: For the negotiation task, we used a slightly simplified version of the shopping plan

task used by Woolley et al. (2010). Groups had 20 minutes to create a shopping plan where they

decided which stores to visit to complete their individual shopping lists. Plans were scored

according to the number of items purchased, the time taken to complete shopping, the quality

and price of items, and whether or not items spoiled. Scores were calculated using the same

criteria used by Woolley et al. (2010).

Complex task: Complex tasks often require some aspect of each of the four types or modes of

activity (McGrath 1991), as groups need to brainstorm, negotiate, decide on, and execute

strategies to complete the task. For the complex task in our study, groups were asked to make a

distribution decision about a hypothetical candy production firm. Each group member was

assigned as a division manager with specific resources needs, and groups were given information

about the profit margins of each division. The groups were asked to make a decision of how to

distribute two key ingredients to the various divisions to maximize profits for the company

(Mennecke and Wheeler 2012; Raghavan 1990). Groups were given 25 minutes to complete this

task. The performance score for the task was the standardized maximum profit per division that

could be achieved by the hypothetical company based on the distribution of the ingredients.

Measures

Individual intelligence is the ability of an individual to perform well across a variety of

cognitive tasks (Devine and Philips 2001; Woolley et al. 2010). Participants completed the

Wonderlic Personnel Test, a cognitive ability exercise that has been validated by academic

researchers (Dodrill 1983; Dodrill and Warner 1988) and used in numerous studies (Blickle et al.

2010; Simon et al. 1996; Woolley et al. 2010). The test consists of 50 questions to be answered

in a period of 12 minutes. For each group, the average and maximum score was calculated.

Social sensitivity is the ability to perceive and understand the feelings and viewpoints of

others, measured using the Reading the Mind in the Eyes test (Baron-Cohen et al. 2001). The

multiple-choice task consists of 36 consecutive photographs of eyes, where participants respond

with the emotion that the photographed person is feeling. For each photograph, participants

decide among four given choices. For each group, the average and maximum score was

calculated.

Variation in speaking turns is a measure of the distribution of communication within a

group. Groups with a high variance of speaking turns are those where some members of the

group dominate discussion. We downloaded the Gmail chat transcript for each group and

calculated the percentage of messages contributed by each participant. We then calculated the

variance of these percentages for each group (Woolley et al. 2010).

Females as a percentage of group members. Participants were asked to self-report their

gender. For each group, percent female was calculated, as done by Woolley et al. (2010).

Procedures

Participants first completed an online survey before signing up for a lab session. The

survey contained (a) an informed consent statement as required by the local institutional review

board; (b) the Reading the Mind in the Eyes test (Baron-Cohen et al. 2001), measuring social

sensitivity; and (c) a set of questions designed to enable robustness and validity checks

(described below).

The remainder of the data was collected in a computer lab in the business school.

Participants sat at individual workstations where they could not see the computer screens of other

participants. Participants first completed the Wonderlic intelligence test, and then completed the

group tasks as randomly-assigned groups of 3-5 members. The three simple tasks—

brainstorming, negotiation, and decision-making—were completed in random order. Following

these tasks, the groups performed the complex problem-solving task. After the group tasks,

participants completed a survey soliciting demographic information.

Procedures and tasks were refined during four pilot sessions. Minor adjustments were

made to timing and task requirements to allow participants to complete the tasks comfortably

within a two-hour period to prevent fatigue.

Comparison to Individual Intelligence and Performance

In addition to the main data collection and analysis, we recruited 91 additional

participants from the same subject pool to complete the same tasks using the same measures and

procedures but working as individuals rather than in groups. We analyzed this individual level

data using the procedures described in the Analysis and Results section below. We found that an

individual intelligence factor emerged from the data and that this factor was significantly

correlated with scores on the Wonderlic intelligence test (r=0.318; p<0.01). We conclude that

individual intelligence influenced individual performance for these subjects, tasks, measures, and

procedures, and thus they constitute an appropriate context in which to investigate whether a

collective intelligence factor emerges when groups use CMC.

ANALYSIS AND RESULTS

Our data analysis involved three steps. First, we followed the procedures of Woolley et

al. (2010) to determine whether a collective intelligence factor would emerge from the data.

Second, we examined which of the variables we measured related to task performance on the

four tasks separately. That is, if certain factors are not related to performance across all tasks,

which factors are related to performance only on certain tasks but not others? Third, we

performed several robustness and validity checks to ensure the validity of our results.

Collective Intelligence Factor

Following the procedures of Woolley et al. (2010), we used group performance scores on

the set of simple tasks to determine whether a collective intelligence factor emerged from the

data. The first criterion is that the average correlation between task scores should be positive.

The pairwise correlations in the Woolley et al. (2010) studies were either significantly positive or

not statistically significant, with an average correlation of 0.28 in Study 1 and 0.23 in Study 2.

In our study, correlations between task scores were either not statistically significant or

significantly negative, as shown in Table 1. The average correlation was -0.12, indicating that

performance on one task was not correlated with performance on other tasks.

The method to identify an intelligence factor, whether individual or collective, consists of

a factor analysis of performance scores on a variety of tasks (Deary 2000; Woolley et al. 2010).

The intelligence factor in this factor analysis should (a) account for 30-50 percent of the

variance, with the next factor accounting for significantly less, (b) have an eigenvalue greater

than 1.38, and (c) demonstrate an obvious elbow in the scree plot (Woolley et al. 2010).

Table 1. Correlations between group tasks and traits. 1 2 3 4 5 6 7 8 9

1. Brainstorming task

2. Decision task -0.14

3. Negotiation task -0.25* -0.09

4. Complex task 0.02 0.01 0.15

5. Average intelligence 0.51** -0.06 -0.02 0.07

6. Max intelligence 0.47** -0.11 -0.05 -0.08 0.75**

7. Percent female -0.25* -0.11 0.08 -0.18 -0.37** -0.22*

8. Avg social sensitivity 0.34** -0.26* -0.09 0.04 0.44** 0.42** 0.01

9. Max social sensitivity 0.26* -0.23* -0.07 -0.09 0.22* 0.32** 0.19 0.63**

10. Speaking variance -0.19 0.04 -0.06 0.05 -0.15 -0.04 0.00 0.08 0.07

*p<0.05 **p<0.01

None of these criteria were met in our study. In our principal components factor analysis,

the first factor accounted for 42 percent of the variance, but the second factor accounted for 36

percent, suggesting two dominant factors, rather than an emerging intelligence factor. The first

factor’s eigenvalue was 1.26, and no drop-off elbow appeared in the scree plot, which is shown

in Figure 1. Taken together, these analyses indicate that a general collective intelligence factor

does not emerge when groups use CMC.

Because no collective intelligence factor emerged, we next examined which of the

variables were correlated with specific task scores (see Table 1). That is, because performance on

different tasks is not correlated, it would be useful to better understand which factors, such as

individual intelligence, relate to performance on specific task types. We collected the same

measures examined by Woolley et al. (2010): individual intelligence, social sensitivity, speaking

turn variance, percentage of females, and group performance on a complex task.

Figure 1. Scree plots comparing our factor analysis to the Woolley et al. (2010) studies

Differences in Task Types

Similar to the findings of Woolley et al. (2010), individual intelligence was not correlated

with performance on the decision task, the negotiation task, or the complex task, but was

correlated with performance on the brainstorming task (see Table 1).

Brainstorming performance was also correlated with visual social sensitivity (likely due

to positive correlation between individual intelligence and visual social sensitivity) and the

percentage of females in the group. However, groups with a smaller percentage of females

generated more ideas, opposite to the correlation found in the Woolley et al. (2010) studies,

where females scored higher on social sensitivity (also in contrast to our findings).

Unlike the Woolley et al. (2010) studies, social sensitivity and percentage of females in

the group were not significantly positively correlated with task performance for any tasks beyond

brainstorming. Social sensitivity was negatively correlated with the performance of decision

tasks, meaning that groups with a higher average social sensitivity made worse decisions.

Speaking turn variance was not correlated with performance for any task in our study, indicating

variance in turns may not be as relevant when groups use CMC.

To examine the effects of these variables on task performance beyond simple

correlations, we ran four separate regression models with respective performance on each of the

four tasks as the dependent variables, including group size and task order as control variables.

Table 2 summarizes the results of these regression analyses.

For the brainstorming task, both individual intelligence and social sensitivity were

significant predictors of performance (p<.05). Although percentage of females was correlated

with task performance, it was not significant in the regression model, suggesting that the

correlation was a statistical artifact due to correlations with other variables.

For the decision task, social sensitivity was significantly negatively related to

performance. For the negotiation task and the complex task, none of the variables were

significantly related to performance.

Table 2. Results of separate regression models on the tasks Variables Parameter estimate (p-value)

Task Brainstorming Decision Negotiation Complex Task

Intercept -21.414(0.002)

6.026(<0.001)

70.638(<0.001)

19.001(<0.001)

# members 1.238

(0.172)-0.289

(0.119)1.255

(0.550)-0.267

(0.508)

Task order 0.774

(0.279)-0.005

(0.973)-0.345

(0.836)N/A

Average intelligence

0.611**(0.006)

0.018(0.681)

0.219(0.662)

-0.006(0.948)

Avg social sensitivity

0.494*(0.026)

-0.099*(0.028)

-0.479(0.345)

0.031(0.753)

% female -0.887

(0.144)-0.075(.530)

1.070(0.452)

-0.429(0.114)

Variance speaking -0.006

(0.143)0.000

(0.877)-0.001

(0.890)0.000

(0.962)

*p<0.05 **p<0.01. In each model, the dependent variable is the task performance score.

Robustness and Validity Checks

Robustness and validity checks on task selection

The tasks were similar to tasks used by Woolley et al. (2010). Exact replication of their

tasks is not desired, as the goal was to examine consistent performance across different tasks, not

to perfectly replicate their study, which would raise concerns about generalizability to other

tasks. A collective intelligence factor should, by definition, predict consistently high

performance across multiple types of tasks. If the Woolley et al. findings were dependent on

specific tasks, this would further strengthen our argument that the factor they found was not truly

collective intelligence of groups. As we noted in the Methods section, our analysis of individual

performance of these tasks found that an individual intelligence factor emerged that was

correlated to the individual’s Wonderlic score, suggesting that these tasks were appropriate. Even

so, we completed a task robustness check and two validity checks to ensure that the differences

between our findings and those of Woolley et al. (2010) were not dependent on our task

selection, all of which were drawn from prior research.

First, we used several combinations of tasks for our analysis. Our main findings result

from the set of three simpler tasks—a brainstorming task, a decision task, and a negotiation task.

We ran a factor analysis on all other combinations of tasks (see Figure 2 for a list of these

combinations) and found that no matter what tasks we included in the factor analysis, a single

collective intelligence factor did not emerge. To ensure that the findings were not based on using

too few tasks, we also ran the procedures on the combination of all four tasks, with the same

results, as summarized in Figure 2 and Table 3.

Figure 2. Scree plots for factor analyses of all combinations of tasks

Table 3. Results of task combination robustness check

Brainstorm + Decision + Complex

Brainstorm + Negotiation +

Complex

Decision + Negotiation +

Complex

All four tasks

Average correlation -0.04 -0.03 0.02 -0.05Variance explained by first factor

38.15 42.99 39.14 32.64

Variance explained by second factor

33.25 33.81 34.12 27.69

First eigenvalue 1.14 1.29 1.17 1.31

Second, we compared the variation in our task performance scores to the Woolley et al.

(2010) studies. The mean coefficient of variation across our tasks (0.308) was not significantly

different from the mean coefficient of variation for their tasks (COV=0.345; t(17)=.36, p=.738),

suggesting that lack of variation in task performance is not an issue.

Third, we checked for ceiling effects. Our mean performance across the tasks was 69

percent of the optimal which was not significantly different from the mean performance in the

their studies (58% of optimal; t(17)=0.80, p=.468), suggesting that ceiling effects were not an

issue.

Validity checks for other constructs

Although only task performance scores are used in the factor analysis to detect an

intelligence factor, we also compared the descriptive statistics of our other measures to those of

(Woolley et al. 2010) to further ensure that our groups were not significantly different from the

groups in their studies. These statistics are shown in Table 4.

Table 4. Comparison of group level measures to Woolley et al. (2010)* Individual level Group level Individual

intelligence (Wonderlic

score)

Social sensitivity

Average individual

intelligence

Average social

sensitivity

% Female Variance in speaking

turns

Group size

Mean 27.4 ** 24.8 n/a 27.3 23.9 24.8 25.9 42.6 50.16 138.8 112.4 3.7 n/aSt Dev

6.1 ** 5.4 n/a 3.4 6.0

3.1 2.8 28.0 37.22147.7 97.2 0.7 n/a

Min 8.0 ** 7 n/a 20.3 8.0 14.7 18.5 0.0 0.0 5.2 0.7 3.0 2.0Max 43.0 39.0 35 n/a 35.7 39.0 31.0 35.0 100.0 100.0 858.3 355.0 5.0 5.0

*Values in the left columns represent the current study; values in the right columns refer to Woolley et al. Study 2 **These values were not directly reported in the Woolley et al. study. However, information in Woolley et al.’s published supplementary materials indicate that for Study 2, the mean Wonderlic score was between 22.9 and 24.4; the standard deviation was between 6.81 and 7.07; and the minimum score was less than or equal to 8. Robustness check for statistical methods

To ensure that our results were not dependent on the type of extraction used in our factor

analysis to find an intelligence factor, we repeated our analysis using four methods of extraction:

maximum likelihood, unweighted least squares, generalized least squares, and principal axis

factoring. Further, for each method of extraction, various methods of rotation (Varimax, Promax,

and no rotation) were attempted. In each case, no intelligence factor emerged. In all cases,

multiple factors explained over 30 percent of variance, no factor had an eigenvalue of 1.3 or

higher, and no scree plots showed any obvious elbow to indicate one dominant factor.

Use of these statistical methods to analyze the separate individual level data described in

the Methods section showed that when individuals completed the same tasks, performance was

consistent across tasks and an intelligence factor emerged. The average pairwise correlation of

performance scores was 0.17; the first factor in the factor analysis accounted for 45% of

variance, with the second factor being much lower (31%), creating an obvious elbow in the scree

plot. These results demonstrate the utility of our tasks and factor analysis procedures for

detecting a general intelligence factor. It was only when the tasks were performed by groups

using CMC that performance became inconsistent with no general intelligence factor emerging.

Robustness check for motivation

We completed an additional robustness check to ensure that random noise in the data

resulting from careless participants did not affect our results. As part of the pre-survey,

participants completed a set of survey questions designed to determine whether the participant

was paying attention and would earnestly complete the study. The set of questions was generally

irrelevant to the study but contained one question with an objective answer (i.e., “Please select

Disagree as the answer to this question”). In addition, the survey was designed to be completed

in 15-30 minutes. Participants who took less than 10 minutes on the study and/or answered the

objective question incorrectly were flagged. Analyses excluding the flagged participants

achieved the same results (i.e., no intelligence factor emerged, and individual intelligence was

related only to the brainstorming task). In addition, a research assistant read the chat transcripts

to verify that the teams were earnestly trying to perform well on the tasks. No team was

identified as failing to take the task seriously.

DISCUSSION

Interpretation of findings

First and foremost, the results of this study show that no collective intelligence factor

emerged when groups used text-based CMC. Yet when we used these same procedures and

measures to examine the performance of individuals drawn from the same subject pool

performing thee same tasks, we found an individual intelligence factor emerge that was

correlated with the individual’s Wonderlic score. An inherent collective intelligence factor

similar in nature to individual intelligence should transcend media and tasks, indicating that

collective intelligence does not manifest when groups use CMC.

Thus we conclude that the factor found by Woolley et al. (2010) is not a general factor of

collective intelligence inherent to groups under all conditions, but it is a measure of a group’s

general ability to work well in face-to-face settings (i.e., face-to-face collective intelligence).

Collective intelligence manifests itself differently in virtual settings; that is, groups using CMC

do not appear to have an inherent factor that makes some groups more intelligent (i.e., highly

performing across tasks) than others. We conclude that collective intelligence is a concept that is

unlike individual intelligence in that it emerges differently under different conditions. For some

forms of CMC, task characteristics have a more powerful effect on group performance than any

inherent group factor.

Stated differently, our main finding is that even though groups working face-to-face

display a consistent level of performance across tasks, dependent on the social sensitivity of

groups members, groups working through CMC with no visual cues did not have consistent

performance; rather, their performance differed by task type. Contrary to past research with

groups working face-to-face, we found that when groups used CMC, groups that performed well

on the decision making task did not perform well on the brainstorming, negotiation or complex

tasks (the same was true for other combinations of tasks) – performance on one task was not

linked to performance on other tasks.

This finding has rather disappointing implications in practice. We would like to think that

selecting “good” group members is important to performance when groups use CMC. Yet our

results show that on average, groups that did well in one task did not do well in others. In other

words, there was no consistent general tendency in performance. Regardless of the potential

reasons that face-to-face groups could outperform groups using ICTs (e.g., difficulty with

conflict management, difficulty understanding social cues, etc.), the findings indicate that even in

sub-optimal conditions, no groups performed consistently better than others. Truly intelligent

groups should perform well relative to other groups, across tasks and media, even (and

especially) in conditions where the task and the technology do not fit well together. Therefore,

the concept of collective intelligence is more complex than past research would suggest,

manifesting under some conditions but not others.

Second, given that no inherent general collective intelligence factor was found when

groups used ICT, we now turn to the factors that might explain performance on specific tasks

when groups interact using text-based CMC. Face-to-face collective intelligence was highly

correlated with social sensitivity, a measure related to visual communication, which is not

available to groups using text-based CMC. In our study, social sensitivity had a significant

positive effect on brainstorming performance, a significant negative effect on decision-making

performance, and no significant effect on the performance for the negotiation task or the

complex task. We conclude that researchers should take caution in claiming that social

sensitivity is an important factor influencing group performance without considering the media

groups use and the tasks they perform. Although social sensitivity influences performance when

using groups work face-to-face, it has no consistent effects when groups use CMC that lack

visual cues. The ability to understand non-visual social cues may be linked to group performance

for certain types of tasks (Walther and Tidwell 1995), but this skill is likely separate from social

sensitivity based on interpreting visual cues.

One of the most surprising results of our study and the Woolley et al. (2010) studies is the

general lack of impact of individual intelligence on overall group performance. Individual

intelligence is generally a good predictor of performance on individual tasks (Deary 2000; Deary

2012; Woolley et al. 2010), and was significantly correlated with individual performance for the

tasks we studied. However, something happens in group work that disconnects individual

intelligence from group performance. Our study suggests that group member intelligence may be

relevant only for tasks involving a sum of individual performance, such as brainstorming tasks.

Brainstorming is an additive task in which group performance is largely a function of the sum of

individual performance (i.e., the number of ideas produced by a group is the sum of ideas

produced by individuals) (Steiner 1972). As such, brainstorming is more akin to an individual

task than other types of group tasks that require group members to come to consensus on the

outcome(s). The effects of individual intelligence are not significant for more interactive tasks

that require a group to reach consensus.

Previous research and theory shows that group performance depends on the fit of the task

with the technology and on how the groups appropriate the technology to best fit the task at hand

(Dennis et al. 2001). Our results take this one step further to show that different group

characteristics have different effects on performance depending on the task. For brainstorming

tasks, the average intelligence of group members was significant; groups whose members had

higher average intelligence produced more ideas. Groups with a greater percentage of males

generated more ideas, contrary to the correlations found in the Woolley et al. (2010) studies,

which suggested that groups with a higher percentage of females generated more ideas. Social

sensitivity was also a significant predictor of performance on the brainstorming task. We believe

this was due to the high correlation between social sensitivity and individual intelligence (which

is itself an interesting finding), but more research is needed to understand this.

Also contrary to Woolley et al. (2010), social sensitivity was negativity related to the

performance of decision tasks. This surprising result is another indicator that visual social

sensitivity is not an appropriate measure to predict the performance of groups when they use

media other than face-to-face communication. Interestingly, neither average nor maximum

intelligence was correlated with group decision performance, indicating that the correlation

between decision performance and social sensitivity was not simply an artifact of the high

correlation between intelligence and social sensitivity. This finding suggests that those higher in

visual social sensitivity were more likely to be impacted by the lack of visual processing in the

text-based CMC environment, with their performance suffering when they could not use the

visual processing that helps in face-to-face collaboration.

Performance on the negotiation task was not highly correlated with any of the group

factors measured in our study. These results are much like those in the Woolley et al. (2010)

studies, where the only significant correlation was with speaking turn variance. Speaking turn

variance was not correlated with any of the tasks in our study, indicating variance in turns may

not be as relevant in CMC. Use of CMC has been shown to result in more equal participation

(Straus 1996), so this variable may have less effect in ICTs contexts, especially in text-based

media that allows simultaneous turn-taking.

Finally, performance on the complex task was not correlated with individual intelligence

or any of the other factors we measured. This result is consistent with our overall finding that

none of these factors predicted consistent performance across the simpler tasks. In other words,

because none of these factors predict performance on the entire set of brainstorming, decision,

and negotiation tasks, we are not surprised that none of these factors predicted performance on a

complex task requiring brainstorming, decision-making, and negotiation. In comparison,

Woolley et al. (2010) found the complex tasks to only be correlated with social sensitivity.

Limitations

One potential limitation of the current study is that the sample was limited to young

business students. Participants in our study averaged a slightly higher score on individual

intelligence than participants in the Woolley et al. (2010) studies and were on average two years

younger. Student samples are appropriate for testing theories about phenomena that are theorized

to hold true across the population (Compeau et al. 2012). If a collective intelligence factor does

not emerge in groups of students, previous findings on collective intelligence cannot be

generalized across all populations. In other words, we aimed to understand whether collective

intelligence is, like individual intelligence, an inherent factor related to social sensitivity

regardless of media. Showing the non-emergence of this factor in a subset of the population

suggested the need to better understand collective intelligence and how it differs by context.

Another limitation of both the current study and the Woolley et al. (2010) studies is that

groups had not worked together previously. As groups develop relationships and routines over

time, they may be able to better perform on a wider variety of tasks. However, for the purpose of

initial studies investigating collective intelligence in CMC, such as the current study, new groups

may be most appropriate in order to differentiate between inherent collective intelligence and the

effects of established routines.

Implications for Future Research

Despite these limitations, we believe there are six implications for future research. First,

more research is needed to understand the concept of collective intelligence. Our study suggests

some boundary conditions of the collective intelligence factor that emerges in face-to-face work.

More studies are needed to examine collective intelligence and the conditions under which it

emerges and influences group performance. In other words, while an inherent general collective

intelligence factor may not exist for all forms of group work, future research should continue to

assess why groups can develop a general ability to work well across tasks initially when using

media with visual cues, and whether groups can develop collective intelligence when using CMC

lacking such visual cues.

Research is also needed to determine whether collective intelligence should be

conceptualized and measured as an inherent factor within certain task types. It may be that, like

individual intelligence, groups perform consistently across all kinds of (for example) decision

tasks, even when using ICT. Perhaps the differences between group task types are greater than

the differences between individual task types when measuring intelligence, so that collective

intelligence emerges only within group task types. In this case, research is also needed on why

face-to-face groups can bridge the collective intelligence across the broad task types, but groups

using ICT cannot.

Future research could also determine how collective intelligence, rather than being an

inherent characteristic, may be developed over time in groups. It may that by establishing

routines and stability, some groups can become “smarter” than others across tasks. This would

establish collective intelligence as an acquired ability rather than as an inherent characteristic. It

may even be that collective intelligence is made up of a combination of inherent qualities and

qualities that groups develop over time; future research should thus further examine the true

nature of more “intelligent” groups.

Second, more research is needed to better understand the relationship – or rather the lack

of a relationship – between individual group members’ intelligence and group performance.

While previous research has produced mixed findings on this relationship (Devine and Philips

2001), our study suggests that group member intelligence may be relevant only for tasks

involving a sum of individual performance, such as brainstorming tasks, while the effects of

intelligence are not as important for more interactive group tasks that require consensus. What is

it about the need to reach consensus that weakens the relationship between individual

intelligence and group performance even though the relationship between individual intelligence

and individual performance on the same tasks is strong? Future research should focus on other

possible relationships (e.g., non-linear) between individual intelligence and group performance,

as well as potential moderators. It may be that individual intelligence is a necessary but not

sufficient condition for the effective performance of groups in non-additive tasks.

Third, more research is needed on the role of social sensitivity in group performance.

Although it may be the key factor related to across-task performance of groups working face-to-

face, we found it to have varying effects, depending on the task, on groups working virtually. It

makes sense that a factor that is dependent on visual cues would have little effect when using

CMC lacking visual cues. Nonetheless, we need more research to understand its effects on

groups working with other forms of CMC such as those offering audio cues and visual cues. It

may be that social sensitivity is linked to performance when groups use other forms of CMC that

include audio and/or visual cues (at least for those with high visual social sensitivity). More

research is needed on collective intelligence when groups use these other forms of CMC.

Because visual cues and emotional understanding are important to the performance of

groups working face-to-face, it would be fruitful to research the role of social cues in influencing

group performance when groups use media that lack visual and/or audio cues. Groups sometimes

substitute explicit text cues or nonverbal cues for the unavailable visual/audio cues (Walther and

Tidwell 1995). Studies are needed to examine the impact of such cues on virtual group

performance across different types of tasks. Although research has suggested that individuals

display and try to interpret such textual cues, we know of no validated measure of the ability to

interpret social and emotional cues in text (akin to visual social sensitivity in face-to-face

settings). A text-based version of the “Reading the Mind in the Eyes” test would be helpful in

understanding the effect of the social sensitivity trait in settings lacking visual cues. Such an

ability would likely not lead to higher performance across all types of tasks when using CMC

(otherwise, the performance scores in our study would have been correlated, regardless of

whether we measured such an ability), but it would be fruitful to understand in which situations

and tasks this ability would be helpful to groups.

Fourth, one key element of group work that is different from individual work is the social

interactions among group members. It may be that these interactions are more powerful than any

intelligence factor (individual or collective) for certain types of tasks (Bachrach et al. 2012;

Kosinski et al. 2012). This finding is consistent with other complex-systems phenomena where

the interaction among individuals emerges as more important than general individual or group

characteristics (Axelrod 1997; Watts 2002). More research is needed to understand the extent to

which group performance across tasks is dependent upon group characteristics versus on the

interaction processes of group members. Such interaction phenomena may be more useful in

explaining why groups are successful in some tasks and not others.

Fifth, future research could also examine groups that work in both face-to-face and CMC

environments. Many groups today work in both settings, and the ability to understand what

factors contribute to performance in both environments (and different CMC environments) is

important. It may also be important to understand what factors influence the performance of

established groups with prior histories of working together. The factors influencing performance

in these groups may be similar to or different from newly formed groups and may explain how

some groups using CMC can start to perform consistently across tasks.

Finally, another avenue for future research is the continued development of technology

and process interventions to improve group performance (Briggs et al. 2003; De Vreede et al.

2009; Zhang et al. 2011). Such action research benefits from the knowledge that group

performance depends on task type. Action researchers focusing on improving collaboration

should focus on tools to improve collaboration for context-specific tasks, or alternatively aim to

overcome the task-specific differences in virtual group work.

Implications for Practice

First, the findings of our study indicate a lack of inherent collective intelligence (i.e.,

consistent performance) in groups working together across multiple tasks using ICTs that lack

visual cues. This suggests that managers may not necessarily be able to rely on virtual groups

who have previously done well on one initial task, to perform well on a new type of task. There

may be some consistency within a type of task (creativity, decision making, negotiation), but

groups that are highly creative are not necessarily also good at making decisions or negotiating,

at least when using ICTs that lack certain social cues.

Second, individual intelligence was not related to virtual group performance in general.

Although there is an intuitive appeal to selecting highly intelligent group members in the hopes

of getting better performance when they work together, our research suggests that this approach

is no more likely to lead to good group performance on some virtual tasks than selecting group

members based on any other individual characteristic. Group member intelligence may or may

not be necessary for good group performance, but it is not sufficient, except for tasks that do not

require coming to consensus such as brainstorming. As a result, we caution managers that when

putting together a group of individuals to complete a task other than brainstorming, simply

choosing the brightest individuals will not necessarily result in the best group performance.

Group performance depends more on the requirements of the task and the nature of the ICT used

(e.g., face-to-face, CMC) more than on the traits of the individuals who make up the group.

Third, our research suggests that groups whose members are high in social sensitivity do

not perform well when they use text-based CMC that lack visual cues. It may that the lack of

visual social cues impairs the ability of these individuals to work. We suggest that groups of this

type when performing a decision-making task should elect to use a CMC tool that supports

visual cues, such as video conferencing, rather than a text-based CMC such as email or IM.

CONCLUSION

Our research suggests that collective intelligence is not a factor inherent to all groups

because it does not transcend media. While groups working face-to-face have a certain level of

intelligence because they perform consistently across tasks, group performance across tasks is

more complex for groups using CMC that lacks visual cues. In these cases, performance across

tasks is not inherently consistent, and more research is needed to understand the collective

intelligence of such groups.

Likewise, social sensitivity may enable groups to better manage face-to-face

communication and perform more consistently, but it does not consistently improve group

performance when groups work using CMC that lacks visual cues. Our study suggests that the

more complex interactions required for CMC-based group work makes the task-specific

processes by which group members work more important than inherent group characteristics or

interaction styles. Thus the key to consistent success across media may lie in helping group

members to identify better ways to work together for a given task rather than selecting group

members that have certain characteristics (e.g., social sensitivity) or relying on groups that have

previously performed well on other types of tasks.

REFERENCES

Abbasi, A., and Chen, H. 2008. "Cybergate: A design framework and system for text analysis of computer-mediated communication," MIS Quarterly (32:4), pp. 811-837.

Adams, R. B., and Funk, P. 2012. "Beyond the glass ceiling: Does gender matter?," Management Science (58:2), pp. 219-235.

Akgun, A., Byrne, J., and Keskin, H. 2005. "Knowledge networks in new product development projects: A transactive memory perspective," Information & Management (42:8), pp. 1105-1120.

Apesteguia, J., Azmat, G., and Iriberri, N. 2012. "The impact of gender composition on team performance and decision making: Evidence from the field," Management Science (58:1), pp. 78-93.

Axelrod, R. 1997. "The dissemination of culture: A model with local convergence and global polarization," Journal of Conflict Resolution (41:2), pp. 203-226.

Bachrach, Y., Graepel, T., Kasneci, G., Kosinski, M., and Van-Gael, J. 2012. "Crowd IQ - Aggregating opinions to boost performance," 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), Conitzer, Winikoff, Padgham and van der Hoek (eds.) (eds.), Valencia, Spain.

Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., and Plumb, I. 2001. "The "Reading the Mind in the Eyes" test revised version: A study with normal adults, and adults with Asperger Syndrome or High-functioning autism," Journal of Child Psychology and Psychiatry (42:2), pp. 241-251.

Bell, B. S., and Kozlowski, S. W. J. 2002. "A typology of virtual teams: Implications for effective leadership," Group & Organization Management (27:1), pp. 14-49.

Bender, L., Walia, G., Kambhampaty, K., Nygard, K. E., and Nygard, T. E. 2012. "Social sensitivity and classroom team projects: An empirical investigation," SIGCSE 2012, Raleigh, NC.

Blickle, G., Kramer, J., and Mierke, J. 2010. "Telephone-administered intelligence testing for research in work and organizational psychology: A comparative assessment study," European Journal of Psychological Assessment (26:3), pp. 154-161.

Boland, R. J., Tenkasi, R. V., and Te'eni, D. 1994. "Designing information technology to support distributed cognition," Organization science (5:3), pp. 456-475.

Borge, M., Ganoe, C. H., Shih, S.-I., and Carroll, J. M. 2012. "Patterns of team processes and breakdowns in information analysis tasks," 15th ACM Conference on Computer Supported Cooperative Work, ACM, Seattle, WA.

Briggs, R. O., De Vreede, G.-J., and Nunamaker, J. F. 2003. "Collaboration engineering with ThinkLets to pursue sustained success with group support systems," Journal of Management Information Systems (19:4), pp. 31-64.

Choi, S. Y., Lee, H., and Yoo, Y. 2010. "The impact of information technology and transactive memory systems on knowledge sharing, application, and team performance: A field study," MIS Quarterly (34), pp. 855-870.

Chudoba, K. M., Wynn, E., Lu, M., and Watson-Manheim, M. B. 2005. "How virtual are we? Measuring virtuality and understanding its impact in a global organization," Information Systems Journal (15:4), pp. 279-306.

Compeau, D., Marcolin, B., Kelley, H., and Higgins, C. 2012. "Generalizability of information systems research using student subjects - a reflection on our practices and

recommendations for future research," Information Systems Research (23:4), pp. 1093-1109.

Cramton, C. D. 2001. "The mutual knowledge problem and its consequences for dispersed collaboration," Organization Science (12:3), pp. 346-371.

De Vreede, G.-J., Briggs, R. O., and Massey, A. P. 2009. "Collaboration engineering: Foundations and Opportunities," Journal of the Association for Information Systems (10), pp. 121-137.

Dean, D. L., Hender, J. M., Rodgers, T. L., and Santanen, E. L. 2006. "Identifying quality, novel, and creative ideas: Constructs and scales for idea evaluation," Journal of the Association for Information Systems (7:10), pp. 646-698.

Deary, I. J. 2000. Looking Down on Human Intelligence: From Psychometrics to the Brain, New York, NY: Oxford University Press.

Deary, I. J. 2012. "Intelligence," Annual Review of Psychology (63:1), pp. 453-482. Dennis, A. R., Fuller, R. M., and Valacich, J. S. 2008. "Media, tasks, and communication

processes: A theory of media synchronicity," MIS Quarterly (32:3), pp. 575-600. Dennis, A. R., Minas, R. K., and Bhagwatwar, A. P. 2013. "Sparking creativity: Improving

electronic brainstorming with individual cognitive priming," Journal of Management Information Systems (29:4), pp. 195-216.

Dennis, A. R., Wixom, B. H., and Vandenberg, R. J. 2001. "Understanding fit and appropriation effects in group support systems via meta-analysis," MIS Quarterly (25:2), pp. 167-193.

Devine, D. J., and Philips, J. L. 2001. "Do smarter teams do better: A meta-analysis of cognitive ability and team performance," Small Group Research (44:3), pp. 507-532.

Dodrill, C. B. 1983. "Long-term reliability of the Wonderlic Personnel Test," Journal of Consulting and Clinical Psychology (51:2), pp. 316-317.

Dodrill, C. B., and Warner, M. H. 1988. "Further studies of the Wonderlic Personnel Test as a brief measure of intelligence," Journal of Consulting and Clinical Psychology (56:1), pp. 145-147.

Dong, W., Lepri, B., and Pentland, A. S. 2012. "Automatic prediction of small group performance in information sharing tasks," Collective Intelligence 2012, Boston, MA.

Fuller, M. A., Hardin, A. M., and Davison, R. M. 2006. "Efficacy in technology-mediated distributed teams," Journal of Management Information Systems (23), pp. 209-235.

Fuller, R. M., and Dennis, A. R. 2009. "Does fit matter? The impact of task-technology fit and appropriation on team performance in repeated tasks," Information Systems Research (20:1), pp. 2-27.

Garfield, M. J., Taylor, N. J., Dennis, A. R., and Satzinger, J. W. 2001. "Research Report: Modifying Paradigms—Individual Differences, Creativity Techniques, and Exposure to Ideas in Group Idea Generation," Information Systems Research (12:3), pp. 322-333.

Hinsz, V. B., Tindale, R. S., and Vollrath, D. A. 1997. "The emerging conceptualization of groups as information processors," Psychological Bulletin (121), pp. 43-64.

Hollan, J., Hutchins, E., and Kirsh, D. 2000. "Distributed cognition: Toward a new foundation for human-computer interaction research," ACM Transactions on Computer-Human Interaction (TOCHI) (7:2), pp. 174-196.

Jones, B. F., Wuchty, S., and Uzzi, B. 2008. "Multi-university research teams: Shifting impact, geography, and stratification in science," Science (332).

Kahai, S. S., Carroll, E., and Jestice, R. 2007. "Team collaboration in virtual worlds," SIGMIS Database (38:4), pp. 61-68.

Kang, S., Lim, K. H., Kim, M. S., and Yang, H.-D. 2012. "A multilevel analysis of the effect of group appropriation on collaborative technologies use and performance," Information Systems Research (23:1), pp. 214-230.

Kosinski, M., Bachrach, Y., Kasneci, G., Van-Gael, J., and Graepel, T. 2012. "Crowd IQ: Measuring the intelligence of crowdsourcing platforms," WebSci 2012, Evanston, IL.

Kress, G. L., and Schar, M. 2012. "Teamology - the art and science of design team formation," in: Design Thinking Research, H. Plattner, et al. (ed.), Springer-Verlag, Berlin, Germany.

Lewis, K., and Herndon, B. 2011. "Transactive Memory Systems: Current issues and future research directions," Organization Science (22:5), pp. 1254-1265.

Majchrzak, A., Malhotra, A., and John, R. 2005. "Perceived individual collaboration know-how development through information technology-enabled contextualization: Evidence from distributed teams," Information Systems Research (16:1), pp. 9-27.

Martins, L. L., Schilpzand, M. C., Kirkman, B. L., Ivanaj, S., and Ivanaj, V. 2012. "A contingency view of the effects of cognitive diversity on team performance: The moderating roles of team psychological safety and relationship conflict," Small Group Research (44), pp. 96-126.

Maruping, L. M., and Agarwal, R. 2004. "Managing team interpersonal processes through technology: A task-technology fit perspective," Journal of Applied Psychology (89:6), pp. 975-990.

McCauley, C. 1998. "Group dynamics in Janis's theory of groupthink: Backward and forward," Organizational Behavior and Human Decision Processes (73:2-3), pp. 142-162.

McGrath, J. E. 1984. Groups: Interaction and Performance, Englewood Cliffs, NJ: Prentice-Hall, Inc. pp. 53-66.

McGrath, J. E. 1991. "Time, interaction, and performance (TIP): A theory of groups," Small Group Research (22:2), pp. 147-174.

Mennecke, B. E., and Wheeler, B. C. 2012. "ISWorld Task Repository Index" Retrieved from https://scholarworks.iu.edu/dspace/bitstream/handle/2022/14355/ISWorld%20Task%20Repository.pdf.

Montoya, M., Massey, A. P., and Lockwood, N. 2011. "3D collaborative virtual environments: Linking collaborative behaviors and team performance," Decision Sciences (42:2), pp. 451-476.

Pinsonneault, A., Bariki, H., Gallupe, R. B., and Hoppen, N. 1999. "Electronic brainstorming: The illusion of productivity," Information Systems Research (10:2), pp. 110-133.

Privman, R., Hiltz, S. R., and Wang, Y. 2013. "In-group (us) versus out-group (them) dynamics and effectiveness in partially distributed teams," IEEE Transactions on Professional Communication (56:1), pp. 33-49.

Raghavan, V. 1990. "The impact of a computer network on group decision-making: An experimental investigation," Dissertation, University of Arizona).

Ren, Y., and Argote, L. 2011. "Transactive Memory Systems 1985-2010: An integrative framework of key dimensions, antecedents, and consequences," Academy of Management Annals (5:1), pp. 189-229.

Resick, C. J., Dickson, M. W., Mitchelson, J. K., Allison, L. K., and Clark, M. A. 2010. "Team composition, cognition, and effectiveness: Examining mental model similarity and accuracy," Group Dynamics: Theory, Research, and Practice (14), pp. 174-191.

Simon, S. J., Grover, V., Teng, J. T. C., and Whitcomb, K. 1996. "The relationship of information system training methods and cognitive ability to end-user satisfaction,

comprehension, and skill transfer: A longitudinal field study," Information Systems Research (7:4), pp. 466-490.

Stasser, G. 1992. "Information salience and the discovery of hidden profiles by decision-making groups: a thought experiment," Organizational Behavior and Human Decision Processes (52:1), pp. 156-181.

Steiner, I. D. 1972. Group Process and Productivity, New York, NY: Academic Press. Straus, S. G. 1996. "Getting a clue: The effects of communication media and information

distribution on participation and performance in computer mediated and face-to-face groups," Small Group Research (27:1), pp. 115-142.

Thomas, D., and Bostrom, R. 2007. "The role of a shared mental model of collaboration technology in facilitating knowledge work in virtual teams," 40th Annual Hawaii International Conference on System Sciences (HICSS'07), Ieee, pp. 1-8.

Thomas, D. M., and Bostrom, R. P. 2010. "Vital signs for virtual teams: An empirically developed trigger model for technology adaptation interventions," MIS Quarterly (34), pp. 115-142.

Venkatesh, V., and Morris, M. G. 2000. "Why don't men ever stop to ask for directions? Gender, social influence, and their role in technology acceptance and usage behavior," MIS Quarterly (24:1), pp. 115-139.

Walther, J. B. 1996. "Computer-mediated communication: Impersonal, interpersonal, and hyperpersonal interaction," Communication Research (23:1), pp. 3-43.

Walther, J. B., and Tidwell, L. C. 1995. "Nonverbal cues in computer-mediated communication, and the effects of chronemics on relational communication," Journal of Organizational Computing (5:4), pp. 355-378.

Watts, D. J. 2002. "A simple model of global cascades on random networks," Proceedings of the National Academy of Sciences (99:9), pp. 5766-5771.

Weick, K. E., Sutcliffe, K. M., and Obstfeld, D. 1999. "Organizing for high reliability: Processes of collective mindfulness," Research in Organizational Behavior, R.S. Sutton and B.M. Staw (eds.), Jai Press, Stanford, CA, pp. 81-123.

Wittenbaum, G. M. 2003. "Putting communication into the study of group memory," Human Communication Research (29), pp. 616-623.

Woodley, M. A., and Bell, E. 2010. "Is collective intelligence (mostly) the General Factor of Personality? A comment on Woolley, Chabris, Pentland, Hashmi and Malone," Intelligence (29:2-3), pp. 79-81.

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., and Malone, T. W. 2010. "Evidence for a collective intelligence factor in the performance of human groups," Science (330), pp. 686-688.

Zhang, X., Venkatesh, V., and Brown, S. A. 2011. "Designing collaborative systems to enhance team performance," Journal of the Association for Information Systems (12:8), pp. 556-584.

Zigurs, I., Poole, M. S., and DeSanctis, G. L. 1988. "A study of influence in computer-mediated group decision making," MIS Quarterly (12:4), pp. 625-644.

Zuckerman, M., Lipets, M. S., Koivumaki, J. H., and Rosenthal, R. 1975. "Encoding and decoding nonverbal cues of emotion," Journal of Personality and Social Psychology (32:6), pp. 1068-1076.