Team Output and Individual Productivity

Decoding Teams: Team Output and Individual Productivity

Mohammad Ahmadpoor and Benjamin F. Jones Northwestern University

May 22nd , 2018 Science of Team Science Conference

The Rise of Teams

Ref: Wuchty, Jones, Uzzi, The increasing dominance of teams in production of knowledge, 2007

MA1

Slide 2

MA1 Mohammad Ahmadpoor, 5/22/2018

Research QuestionsTeam Output & Credit Assignment• How does a set of individual qualities , , … map into the team’s

output, ?

• At one extreme, team output might be a “max” process, meaning max , , … )• At other extreme, team output might be a “min” process, meaning m , , … )

• How can individual productivities be inferred when people work in teams?

• Given series of observed outcomes how can we infer the individual productivities ?• High stakes, but current individual metrics are either “team blind” or use ad hoc sharing rules

(e.g., Hirsch 2005, Ellison 2013; NAS 2015; Perry and Reny 2016, Waltman 2016, Merton 1968)

Research QuestionsTeam Assembly• How should teams be organized? That is, what is the efficient match between

individuals and does this occur in practice?

• For example, consider a vs a process. These have fundamentally opposing implications for how you would organize teams and organizations (e.g., Becker 1973, Kremer 1993, Grossman and Maggi 2000, Topkis 2011)

Method: Key Idea

• The Hölder Mean• is a measure of impact, • is productivity of individual , • and is team size

• Special cases• max ( → ∞)• min ( → ∞) • arithmetic mean ( 1) • geometric mean ( 0)• harmonic mean ( 1)

1

Intuition: The lower , the greater influence of lower productivity team member

Potential Team Advantage,

• captures impact benefit associated with team of size , incl. advantages of aggregating effort, skill, marketing, or disadvantages via coordination costs (Wuchty et al. 2007; NAS 2015).

• Normalize by setting 1 for solo-authored work. • Thus for solo-authored work => individual productivity measured on

the scale of outcome metric. • interpreted as the impact advantage of teamwork over solo-work for

individuals that share a common individual productivity level

Estimation

• Use non-linear least squares to estimate , ,

• Run separately for each field of science, technological class of patents• All 185 science & engineering and social science fields with 500 papers• All 384 tech classes with 500 papers• Restrict to papers/patents with <= 8 team members

• 24 million journal articles, 13 million authors (WOS)• 3.9 million patents, 2.6 million inventors (USPTO)

• Use citations received within 8 years for

1

Result Set #1: Max or Min?

The Team Production Function

The Distribution of across Fields

• All fields of science, engineering, and social science, and all technology classes of patenting

• Centers between geometric and harmonic averages

• Implication: weak links. Greater influence of lower-productivity team members

The Distribution of across FieldsPapers Patents

1 for all WOS fields and 94% of patenting fields Implication: team advantage seen in prior literature not simply about higher-quality people working in

teams; exists conditional on quality of individual team members.

Result Set #2: Inferring Individual Productivity

Individual Productivity Estimates

log 1 Distribution for Patent Classes and WOS Fields

• Determined in light of the team-production parameters, and through for each field of science and patenting

• Implications: (a) Credit shared in non-linear ways ( 1); (b) outcome more informative for lower productivity team members ( 1)

• By construction, recall that measured on same metric as outcome metric,

• Interpretation: If everyone only did solo work, then would be arithmetic mean of an individual’s citations.

Accuracy of Individual Productivity Estimates• Consider capacity to predict outcomes for out-of-sample papers and patents

• Compare predictive capacity against alternative, commonly-used measures of individual productivity (Waltman 2016):i. mean citations (no adjustment for team size);ii. mean citations per collaborator (dividing citations to each work by number of

collaborators);iii. mean citations for individual’s solo works only.

• Run regressions by field, take as success metric

Prediction ResultsPapers Patents

Solo

Dual

Result Set #3: Team Assembly

Team Assembly

• A key organizational implication of 1 is positive assortative matching in team assembly• Do we see this kind of matching?

• Method:• Measure individual productivity purely using solo-authored work, giving

individual productivity estimates that are independent of coauthors’ productivity estimates

• Then ask who works with whom• Present ratio of (a) observed frequency of two-person pairings to (b) frequency

expected by chance.

Matching ResultsPatents

Papers

Summary

• Computationally-feasible method for analyzing team production, deployed across large paper and patent datasets.

• Universal regularities about team production• 1: Individuals typically combine between geometric and harmonic

average of individual productivities• 1: Large team advantage when members have similar individual

productivity• Positive assortative matching

• Tool for estimating individual productivity in team contexts

Extension / Future Directions• New metric for individual productivity. In follow-on project, comparing against multitude

of metrics (Waltman 2016) that have been proposed to evaluate individuals in science

• Richer team production functions (where there is data)• Multi-dimensional skills• Hierarchical teams• Effort

• Team production in other contexts• Arts/Music• Sports• Manufacturing

Further Intuition & Visualization of Model Fit • Consider two-person teams and normalized measure

• Model: ∑ , 

• Raw data version: Actual citations received instead of y Mean citations received to solo work instead of

12

Visualization:  Model Fit and Intuition (Papers)

Visualization:  Model Fit and Intuition (Patents)

Matching, Distribution across Fields

Calculate mean along diagonal for each field. Plot distribution of these means across fields.

Positive assortative matching in all fields

Team Assembly Implications?

Consistent with 1, teams tend to avoid weak links

Implications for stratification in science? Shift toward teams may ↑ stratification c.f., Jones et al. 2008

Implications for Matthew Effect? 1 runs against M.E. to extent that the top member of

the team receives less credit But PAM consistent with signaling aspect of M.E.: co-

authorship itself is a positive signal

Fitting Productivity Estimation Dist.