How the Science of Teams How the Science of Teams Can Inform Team ScienceCan Inform Team Science

How the Science of Teams How the Science of Teams Can Inform Team ScienceCan Inform Team Science

Nancy J. Cooke

March 13, 2015Team Science Retreat

Wake Forest School of Medicine ofWake Forest Baptist Medical Center

Overview• Why Team Science?• Update on NRC Study • My research and experience • A Multi-Level Systems Perspective

Micro Level: Challenges and SupportMeso Level: Challenges and SupportMacro Level: Challenges and Support

• Conclusion

Why Team Science?• Today’s problems require a team of multidisciplinary

individuals• Team Science is impactful (highly cited; Wuchty, et

al., 2007; Uzzi, et al., 2013)• Team Science is innovative (Uzzi, 2013)• Team Science is productive (Hall, et al., 2012)• Team Science has broad reach/uptake (Stipelman, et

al, 2014)

Why Team Science?But…•Not all science requires a team•Team science is difficult

Enhancing the Effectiveness of Team Science: Symposium at ICPS

March 13, 2015

Board on Behavioral, Cognitive, and Sensory SciencesDivision of Behavioral and Social Sciences and EducationNational Research Council

An Update on the NRC Study of Team Science

6

Study Background• Rationale: Clear need to provide research-based

guidance to improve the processes and outcomes of team science

• Sponsors: NSF, Computer and Information Systems and Engineering Directorate and Elsevier

• Goal: Enhance the effectiveness of collaborative research in science teams, research centers, and institutes.

• Audiences: NSF and other public and private research funders; the scientific community; the SciTS community; universities; research centers and institutes.

Committee ChargeConduct a consensus study on the science of team science to recommend opportunities to enhance the effectiveness of collaborative research in science teams, research centers, and institutes… Explore: •How individual factors influence team dynamics, effectiveness and productivity•Factors at the team, center, or institute level that influence effectiveness •Different management approaches and leadership styles that influence effectiveness •How tenure and promotion policies acknowledge academic researchers who join teams•Organizational factors that influence the effectiveness of science teams (e.g., human resource policies, cyberinfrastructure)•Organizational structures, policies and practices to promote effective teams

Committee• NANCY J. COOKE (Chair), Arizona State University• ROGER D. BLANDFORD (NAS), Stanford University• JONATHON N. CUMMINGS, Duke University • STEPHEN M. FIORE, University of Central Florida• KARA L. HALL, National Cancer Institute• JAMES S. JACKSON (IOM), University of Michigan• JOHN L. KING, University of Michigan• STEVEN W. J. KOZLOWSKI, Michigan State University• JUDITH S. OLSON, University of California, Irvine• JEREMY A. SABLOFF (NAS), Santa Fe Institute • DANIEL S. STOKOLS, University of California, Irvine• BRIAN UZZI, Northwestern University• HANNAH VALANTINE, National Institutes of Health

Study StatusReport expected in AprilMore information is available at:http://sites.nationalacademies.org/DBASSE/BBCSS/CurrentProjects/DBASSE_080231

Research Base for Informing Team Science

• SciTS – Science of Team Science (itself a multidisciplinary approach)

• Social Science• Complex Systems• Communications• Management• Medicine• Physical Sciences

The Foresight Initiative• National Geospatial-Intelligence Agency (NGA)

has awarded Arizona State University a grant of $20 million

• Five-year partnership known as the Foresight Initiative will examine how climate change affects resources and contributes to political unrest, as well as articulate sustainability and resilience strategies.

Foresight: A Science TeamApproximately 60 Investigators• 15 ASU Faculty from 8 ASU units• Post docs, research faculty, graduate students• Three National Labs• National Geospatial Intelligence Agency• Expertise in visualization, modeling climate

change, cognitive science, social media, human factors

Foresight: Team Science is Challenging

• Communicating across disciplines• Role confusion• Meetings• Remote participation• Goal conflicts• Sub-teams• Authorship• Resource Allocation

My Research and Experience Relevant to Team Science

Team = Heterogeneous and interdependent group of individuals

(human or synthetic) who plan, decide, perceive, design, solve problems, and act

as an integrated system (vs. group)

Cognitive activity at the team level=

Team Cognition

Improved team cognition Improved team/system effectiveness

Heterogeneous = differing backgrounds, differing perspectives on situation

(surgery, basketball)

Teams and Cognitive Tasks

I’ve Studied Team Cognition in These Tasks

Uninhabited Aerial Vehicle Command and ControlNaval Mission Planning

Cyber DefenseIntelligence Analysis

Human-Underwater Robot InteractionMedical Emergency Teams

Professional CookingHuman-Robot Search and Rescue

Methods: Synthetic Task EnvironmentsA compromise between field studies and laboratory experiments

16

Uninhabited Aerial Vehicle – Synthetic Task

Environment

MacroCog

Underwater Robots

CyberCog

What I’ve Learned

• Teams Learn• Teams Forget• Membership Matters• Team Training Matters

Teams LearnAs teams acquire experience, performance improves, interactions improve,

but not individual or collective knowledge

0

100

200

300

400

500

600

1 2 3 4 5 6 7 8 9 10

Mission

Team

Per

form

ance

Tm 1

Tm 2

Tm 3

Tm 4

Tm 5

Tm 6

Tm 7

Tm 8

Tm 9

Tm 10

Tm 11

• Individuals are trained to criterion prior to M1• Asymptotic team performance after four 40-min missions (robust finding)• Knowledge changes tend to occur in early learning (M1) and stabilize• Process improves and communication becomes more standard over time

40-min missionsSpring Break

Teams ForgetTeam forgetting is best predicted by interaction based measures, not

by individual forgetting (despite shared score components)

Regression model made up of individual decrements: F (4, 20) = 2.018, MSe=5880.23, p>.10, R2 = .29

Introduction of coordination and team SA: F (10, 14) = 2.71, MSe = 4011.88, p < .05, R2 = .66

8-10

wee

k re

tenti

on in

terv

al

Membership Matters• 117 males(92) & females(25) divided into 39

3-person (unfamiliar) Session 2 teams• Two between subjects conditions (retention

interval and familiarity) randomly assigned with scheduling constraints

• Participants randomly assigned to one of three roles

• Session 1: 5 40-min missions• Session 2: 3 40-min missions

10 Teams 10 Teams

9 Teams 10 Teams

3-5 weeks 10-13 weeks

Sam

eM

ixed

Com

posi

tion

Retention Interval

Mixed Condition

Session 1 Session 2

Retention

Interval

AVO PLO DEMPC AVO PLO DEMPC

Same Condition

Session 1 Session 2

Retention

Interval

AVO PLO DEMPC AVO PLO DEMPC

Team Retention and Composition

3-5

OR

10-1

3 W

eeks

All but Short-Intact teams suffer performance loss after the break

But a different story for Team Process (quality of team interactions)…

Team Process improves for mixed, but not intact teams after the break.

(There were no changes in knowledge after the break)

3-5

OR

10-1

3 W

eeks

Team Training MattersCross training (aligned with shared cognition) vs.

procedural/rigid training vs. Perturbation training (focused interactions)

Shared Mental Models

Assumptions• Individual is the unit of analysis• Measure individuals and aggregate• Increasing similarity or convergence over time is

associated with better teamwork • Focus on knowledge, static cognition (team mental

model, shared mental model)• A collection of knowledge experts should be an expert

team

+ +

Team Cognition =The collective knowledge of team members

Interactive Team Cognition

Team interactions often in the form of explicit communications are the

foundation of team cognition

ASSUMPTIONS

1) Team cognition is an activity; not a property or product

2) Team cognition is inextricably tied to context

3) Team cognition is best measured and studied when the team is the unit of analysis

US 2004 Olympic Basketball Team

"We still have a couple of days, but I don't know where we are," replied USA head coach Larry Brown … I've got a pretty good understanding of who needs to play. Now the job is to get an understanding of how we have to play."

A team of experts does NOT make an expert

team

Collaborative skill is not additive

US 1980 Olympic Ice Hockey Team

Herb Brooks and 20 young “no-names” won the 1980 Olympic Gold Medal in Ice Hockey

An expert team made up of no-names…

A Multi-Level Systems Perspective

• Micro -individual• Meso – team, group• Macro - organization, population

Borner, Contractor, Falk-Krzesinski, et al., 2010

Micro Level: Challenges

• Who should engage in team science?– Risks of early career tenure-track scientists

• Who should be on the team?– Team composition– Team assembly

• Faultlines and subgroups

Micro Level: Support

• Recommender systems • Research networking systems• Matching task to team assembly

Meso Level: Challenges

IPO Model (Hackman, 1987)

Four Phase Model of Transdisciplinary Research (Hall, Vogel Stipelman, 2012)

Meso Level: Challenges

Team Process Behaviors•Communication – shared mental models•Coordination•Conflict Resolution•Back-up Behavior•Situation Assessment

Meso Level: Support

•Training•Leadership•Technology•Tools for Team Science

– NCI Team Science Toolkit

Macro Level: Challenges

• Organizational rewards for team science• Disciplinary culture• Geographic dispersion• Complexity of multi-team systems• Mis-aligned goals

Macro Level: Support• Environment• Technology• Rewards• Collaboration Plans• Team Charters

Team Charters• Communication plan between teams

(modes, media, who to whom)• Plan for regular interactions• Plan for leadership – shared• Identify boundary spanning individuals

Asencio, Carter, DeChurch, et al., 2012

Conclusion• Team science is challenging• Team research has implications for

making science teams more effective• Challenges and support can be found at

the micro, meso, and macro levels