Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
Decoding Teams: Team Output and Individual Productivity
Mohammad Ahmadpoor and Benjamin F. Jones Northwestern University
May 22nd , 2018 Science of Team Science Conference
The Rise of Teams
Ref: Wuchty, Jones, Uzzi, The increasing dominance of teams in production of knowledge, 2007
MA1
Slide 2
MA1 Mohammad Ahmadpoor, 5/22/2018
Research QuestionsTeam Output & Credit Assignment• How does a set of individual qualities , , … map into the team’s
output, ?
• At one extreme, team output might be a “max” process, meaning max , , … )• At other extreme, team output might be a “min” process, meaning m , , … )
• How can individual productivities be inferred when people work in teams?
• Given series of observed outcomes how can we infer the individual productivities ?• High stakes, but current individual metrics are either “team blind” or use ad hoc sharing rules
(e.g., Hirsch 2005, Ellison 2013; NAS 2015; Perry and Reny 2016, Waltman 2016, Merton 1968)
Research QuestionsTeam Assembly• How should teams be organized? That is, what is the efficient match between
individuals and does this occur in practice?
• For example, consider a vs a process. These have fundamentally opposing implications for how you would organize teams and organizations (e.g., Becker 1973, Kremer 1993, Grossman and Maggi 2000, Topkis 2011)
Method: Key Idea
• The Hölder Mean• is a measure of impact, • is productivity of individual , • and is team size
• Special cases• max ( → ∞)• min ( → ∞) • arithmetic mean ( 1) • geometric mean ( 0)• harmonic mean ( 1)
1
Intuition: The lower , the greater influence of lower productivity team member
Potential Team Advantage,
• captures impact benefit associated with team of size , incl. advantages of aggregating effort, skill, marketing, or disadvantages via coordination costs (Wuchty et al. 2007; NAS 2015).
• Normalize by setting 1 for solo-authored work. • Thus for solo-authored work => individual productivity measured on
the scale of outcome metric. • interpreted as the impact advantage of teamwork over solo-work for
individuals that share a common individual productivity level
Estimation
• Use non-linear least squares to estimate , ,
• Run separately for each field of science, technological class of patents• All 185 science & engineering and social science fields with 500 papers• All 384 tech classes with 500 papers• Restrict to papers/patents with <= 8 team members
• 24 million journal articles, 13 million authors (WOS)• 3.9 million patents, 2.6 million inventors (USPTO)
• Use citations received within 8 years for
1
Result Set #1: Max or Min?
The Team Production Function
The Distribution of across Fields
• All fields of science, engineering, and social science, and all technology classes of patenting
• Centers between geometric and harmonic averages
• Implication: weak links. Greater influence of lower-productivity team members
The Distribution of across FieldsPapers Patents
1 for all WOS fields and 94% of patenting fields Implication: team advantage seen in prior literature not simply about higher-quality people working in
teams; exists conditional on quality of individual team members.
Result Set #2: Inferring Individual Productivity
Individual Productivity Estimates
log 1 Distribution for Patent Classes and WOS Fields
• Determined in light of the team-production parameters, and through for each field of science and patenting
• Implications: (a) Credit shared in non-linear ways ( 1); (b) outcome more informative for lower productivity team members ( 1)
• By construction, recall that measured on same metric as outcome metric,
• Interpretation: If everyone only did solo work, then would be arithmetic mean of an individual’s citations.
Accuracy of Individual Productivity Estimates• Consider capacity to predict outcomes for out-of-sample papers and patents
• Compare predictive capacity against alternative, commonly-used measures of individual productivity (Waltman 2016):i. mean citations (no adjustment for team size);ii. mean citations per collaborator (dividing citations to each work by number of
collaborators);iii. mean citations for individual’s solo works only.
• Run regressions by field, take as success metric
Prediction ResultsPapers Patents
Solo
Dual
Result Set #3: Team Assembly
Team Assembly
• A key organizational implication of 1 is positive assortative matching in team assembly• Do we see this kind of matching?
• Method:• Measure individual productivity purely using solo-authored work, giving
individual productivity estimates that are independent of coauthors’ productivity estimates
• Then ask who works with whom• Present ratio of (a) observed frequency of two-person pairings to (b) frequency
expected by chance.
Matching ResultsPatents
Papers
Summary
• Computationally-feasible method for analyzing team production, deployed across large paper and patent datasets.
• Universal regularities about team production• 1: Individuals typically combine between geometric and harmonic
average of individual productivities• 1: Large team advantage when members have similar individual
productivity• Positive assortative matching
• Tool for estimating individual productivity in team contexts
Extension / Future Directions• New metric for individual productivity. In follow-on project, comparing against multitude
of metrics (Waltman 2016) that have been proposed to evaluate individuals in science
• Richer team production functions (where there is data)• Multi-dimensional skills• Hierarchical teams• Effort
• Team production in other contexts• Arts/Music• Sports• Manufacturing
Further Intuition & Visualization of Model Fit • Consider two-person teams and normalized measure
• Model: ∑ ,
• Raw data version: Actual citations received instead of y Mean citations received to solo work instead of
12
Visualization: Model Fit and Intuition (Papers)
Visualization: Model Fit and Intuition (Patents)
Matching, Distribution across Fields
Calculate mean along diagonal for each field. Plot distribution of these means across fields.
Positive assortative matching in all fields
Team Assembly Implications?
Consistent with 1, teams tend to avoid weak links
Implications for stratification in science? Shift toward teams may ↑ stratification c.f., Jones et al. 2008
Implications for Matthew Effect? 1 runs against M.E. to extent that the top member of
the team receives less credit But PAM consistent with signaling aspect of M.E.: co-
authorship itself is a positive signal
Fitting Productivity Estimation Dist.