Management of Science, Serendipity, and ResearchPerformance: Evidence from a Survey of Scientists in

Japan and the U.S.

Kota Murayamaa,∗, Makoto Nireib, Hiroshi Shimizub

aDepartment of Economics, Northwestern University, 2001 Sheridan Road, Evanston, IL60208, United States

bInstitute of Innovation Research, Hitotsubashi University, 2-1 Naka, Kunitachi, Tokyo186-8603, Japan

Abstract

In science, research teams are increasing in size, which suggests that science

is becoming more organisational. This paper aims to empirically investigate

the effects of the division of labour in management and science on serendip-

ity, which has been considered one of the great factors in science. Specifi-

cally, in examining the survey of scientists conducted in Japan and the U.S.,

this paper treats the following questions: Does pursuing serendipity really

bring about better scientific outcomes? How does the division of labour

in science influence serendipity and publication productivity? The empir-

ical results suggest that serendipity actually brings about better research

quality on average. It also finds that if the managerial role is played by a

leading scientist in the team, this is positively associated with the quality

of the paper through allowing researchers to pursue serendipitous findings.

In contrast, if the managerial role and leading research role are played by

different members, this has a positive association with the number of papers

published, as the project size becomes larger. These results indicate there is

∗Corresponding author. E-mail: KotaMurayama2013@u.northwestern.edu

Preprint submitted to Research Policy October 11, 2014

a trade-off between serendipity and publication productivity in science via

who plays the leading role in research and management.

1. Introduction

Would Alexander Fleming have discovered penicillin if he had been part

of a large research team? Would he have changed his research plan on

influenza to explore a culture contaminated with a fungus in 1928, if his

research project had been managed by an efficient project manager? By fo-

cusing on serendipity and productivity in science, this paper aims to explore

the relation between the management of science and its research outcomes,

in three steps.

The first step explores the nature of serendipity in science. Serendipity

is regarded as one of the most important aspects of science. Fleming’s

discoveries of the enzyme lysozyme in 1923 and penicillin from the mould

Penicillium notatum in 1928 are frequently cited examples of serendipity.

The cosmic background radiation identified by the Bell Lab scientists Arno

Penzias and Robert Wilson; the circular structure of benzene discovered

by Friedrich Kekule; X-rays developed by Antoine Henri Becquerel; and

Hans Christian Ørsted’s finding that electric currents create magnetic fields,

are also well-quoted examples of serendipity. It seems that many major

discoveries have been made by people who were looking for something very

different.

Much of the anecdotal evidence suggests that serendipity does indeed

have a positive effect on the quality of research. However, is serendipity a

good thing to pursue? One might think that serendipity does not necessar-

ily bring about better results because unintended findings occur randomly.

2

Or one might think that serendipity brings about better results because a

scientist does not change their research plan and pursue unintended findings

unless they expect that the change would be worth pursuing. This might

also be thought because such anecdotes in the history of science only illus-

trate successful results. Thus, considering serendipity as a significant factor

in science might be biased. It is therefore necessary to make a neutral defi-

nition of serendipity, and conduct empirical investigations which specify its

relations to the quality of research; such studies have been lacking in the

literature. By following the definition of serendipity provided by Stephan

(2010) as “the act of finding answers to questions not yet posed,” this paper

investigates whether pursuing serendipitous findings has a positive effect on

the quality of research.

The second step considers the effect of the division of labour in research

and management on serendipity. How can unintended findings be explored

when management and coordination are of importance in science? As will

be reviewed in the next section, the size of research projects has been in-

creasing. Inter-disciplinary and inter-organisational research has been of

significance to the performance of research and development (Agrawal and

Goldfarb, 2008). Prioritising in scientific discovery has also increased (Elli-

son, 2002; Stephan and Levin, 1992). Research is increasingly accomplished

in teams across nearly all fields (Wuchty et al., 2007). This indicates that

management, such as setting a research goal, planning the research proce-

dure, organising the research team, coordinating the members’ efforts, and

managing a research schedule, is increasingly important so as to achieve the

research goals effectively and efficiently.

Serendipity apparently happens in a random manner, implying that it

is not manageable. However, management studies have indicated that cer-

3

tain managerial settings, such as a close relation between a corporate R&D

laboratory and the business divisions, transferring managerial power to the

on-site manager, and the use of external managerial resources, can promote

exploration and allow the flexible pursuit of business opportunities (Ches-

brough, 2003; Nonaka and Takeuchi, 1995). This suggests that a certain

managerial setting in science can promote a flexible pursuit of serendipitous

findings.

When a scientist encounters a serendipitous finding, they are faced with

an important choice: to be flexible and change the research plan to pursue

the serendipitous event, or to stick closely to the initial plan. A serendip-

itous finding comes unexpectedly in the form of a very crude and nascent

condition. Thus, the scientist is forced to make an intuitive decision whether

to pursue it or not. As is reviewed in the following section, this choice is

difficult, particularly when the scientist is working as part of a research team

managed by a competent and efficient project manager. This situation is

seen not only in science but also in business management. This issue is

related to the classical managerial challenge of whether to use top down

or bottom up management. If managerial power is transferred to the im-

mediate director, they can fully desterilise uncodified and tacit knowledge,

and use managerial resources in the context of the actual situation. How-

ever, if a hierarchical managerial role is played top down, findings based on

ground level intuition are seldom used. A centralised bureaucracy cannot

readily adopt new ideas or easily adapt to environmental changes, due to

its formalisation (Gouldner, 1954; Merton, 1957; Selznick, 1949). Directing

its attention to the allocation of managerial and leading research roles, this

paper explores the effects of the division of labour on serendipity.

The third step concerns the effect of the division of labour in research

4

and management on research productivity. One of the advantages of the

division of labour is the increased efficiency resulting from specialisation and

concentration on a single subtask. Thus, if a leading scientist is separated

from a managerial role, they can focus on research and increase productivity.

A specialised project manager can also be fully responsible for the progress

of a research project. Top down hierarchical management facilitates the

completion of the original research goal. In other words, the second and

third steps touch on the dilemma that exists in management: exploration

versus exploitation (March, 1991).

Through exploring a survey of scientists, this paper reports the follow-

ing results. First, serendipity has a positive association with citation of

papers. This suggests that the management of science needs to seriously

take serendipity into consideration because it is one of the important factors

in scientific discovery. Secondly, the integration of a managerial and leading

research role has a positive association with serendipity. This is consis-

tent with the coordination cost framework, which indicates that integration

reduces the costs of the coordination between management and actual re-

search, and provides scientists with flexibility in their research. Thirdly, the

separation of management from research has a larger, positive association

with the number of papers, as the project size becomes larger. It must

be noted that our empirical results are patterns of associations between

serendipity, research quality, and management. However, our empirical re-

sults suggest a trade-off between serendipity and productivity in science via

considering who plays the managerial and leading research roles in research

management. The findings of this paper provide managerial and policy im-

plications for the management of science. Since the size of research projects

in science has been growing, the role of the research manager is of increasing

5

importance to research performance. The findings of this paper suggest that

a bureaucratic and formalistic research manager can block a leading scientist

from approaching an initial research plan flexibly and pursuing serendipitous

findings, which are a source of quality scientific discoveries.

The remaining part of this paper is organised as follows. Section 2 de-

fines serendipity and reviews the previous literature on the management of

science, serendipity, and productivity. In Section 3, we introduce the hy-

potheses, data description, definitions of the variables, estimation models,

and some estimation issues. Section 4 presents the estimated results and

robustness checks. Section 5 summarises the findings, considers managerial

and policy implications, and discusses three limitations for future research.

2. Management and Serendipity

Research is rarely undertaken in isolation; it is increasingly carried out

by a team. The mean number of authors per paper has increased from 2.8

in 1981 to 4.2 in 1999 and team size in science has increased by 50% over a

19-year period (Adams et al., 2005).

There are several factors behind this trend in increasing team size. Sev-

eral studies have shown that collaborative research produces better out-

comes with higher citation rates (Andrews, 1979; Presser, 1980; Sauer, 1988;

Wuchty et al., 2007). The internet and institutional change have decreased

communication costs and promoted increasing team size (Agrawal and Gold-

farb, 2008). The increase of team size in scientific research in the U.S. has

been attributed to the deployment of the National Science Foundation’s

NSFNET and its connection to networks in Europe and Japan after 1987

(Adams et al., 2005). Advances in research equipment (e.g., cyclotron, parti-

cle accelerators, and high-flux research reactors) have increased both collab-

6

oration and team size. Experimental design has also changed from table-top

experiments to large-scale projects. This, too, accompanies changes in the

pattern of collaboration among researchers because the use of one of these

new experimental tools requires several different sets of expertise simulta-

neously.

Another trend in science, discussed in the literature on the management

of science, is related to diversity. Many researchers have suggested that di-

versity in a research team can lead to a greater level of creativity (Allen,

1977; Garvey, 1979; Kasperson, 1978; Pelled et al., 1999). Singh and Flem-

ing (2010) argued that collaboration reduces the probability of very poor

outcomes due to more rigorous selection processes and greater recombinant

opportunities in the creative searches. Zuckerman (1977) showed that nearly

two-thirds of the 286 Nobel Prize winners named between 1901 and 1972

were honoured for work they did collaboratively. By investigating the con-

ditions under which major discoveries or fundamentally new knowledge oc-

cur in science, Hollingsworth (2006) demonstrated that scientists are likely

to develop new and alternative ways of thinking when they interact with

other scientists with diverse areas of expertise and backgrounds. With the

advances in information and communication technology, and institutional

changes, scientists can obtain relevant but different knowledge by collabo-

rating with other scientists in areas outside their own specialties. Accessing

external complementary knowledge and expertise through networking be-

comes significant when promoting innovation, not only in business, but also

in science (Fleming et al., 2007; Hagedoorn, 2002; Heinze et al., 2009; Powell

et al., 1996).

Furthermore, competition in science becomes fiercer. The race to be first

in science has intensified (Ellison, 2002; Stephan and Levin, 1992). There is

7

competition not only for scientific priority in scientific discovery, but also for

research funding. Thus, it is increasingly important for a research team to

choose a research area and method and to set a research goal to minimise the

threat of being “scooped” (Dasgupta and David, 1994; Stephan and Levin,

1992).

As research becomes large scale, requires a high level of technical and

scientific knowledge, and competition becomes fiercer, the management of

science has become increasingly important. Managing and coordinating the

research processes and the different sets of expertise, and synchronising the

efforts towards a team goal, do not happen naturally (Barnard, 1938; Si-

mon, 1976). As research teams become larger, research becomes more inter-

disciplinary and inter-organisational, and competition becomes fiercer, the

role played by research management will be greater.

Even though the literature on the economics of science has grown (Stephan,

2010), the study of the management of science has been quite limited. The

management of science has not yet been well investigated in management

studies; nevertheless, studies have accumulated on organisational structure,

managerial patterns, and the performance of firms. One of the central points

in management studies involves the allocation of authority and responsibil-

ities. For example, an on-site manager has better access to information on

demand, created through the interaction between customers and front-line

workers. Therefore, if customer needs and market trends frequently change,

the decentralised allocation of authority and responsibilities are suitable for

adapting flexibly to a slippery demand. A decentralised and less formalised

management, which allows a high degree of flexibility, is suitable when an

organisation faces many exceptional problems and problem solving is not

easy (Perrow, 1967; Woodward, 1965). This suggests that the decision-

8

making should be carried out where the important information is gathered

and knowledge is created if environmental change is uncertain but highly

frequent.

Why is centralised and hierarchical management not considered to be ef-

fective in a less stable environment? This is related to the division of labour

and its coordination costs (Becker and Murphy, 1992; Lawrence and Lorsch,

1967; Thompson, 2003). If the division of labour and the specialisation of

individual units are highly developed in an organisation, this increases pro-

ductivity because it allows individuals to concentrate on narrowly defined

tasks and to accumulate specialised knowledge about them. However, ac-

cording to the coordination cost framework, the advantage of the extensive

use of the division of labour is diminished by environmental complexity and

coordination costs. If an environment frequently changes, it is necessary for

an organisation to re-engineer the nature of the subtasks and reconfigure the

relations between these subtasks in order to adapt flexibly to the environ-

mental change. If the division of labour and the specialisation of subtasks

are highly developed, the specialised subtasks need to be coordinated in or-

der to achieve overall efficiency. However, according to the coordination cost

framework, costs are increased if individuals do not have a general knowl-

edge of the subtasks complementary to theirs and of the hierarchical subtask

relations (Arora and Gambardella, 1994).

In spite of the fact that the division of labour and its management in

science have become more important since the sizes of research teams have

increased, this aspect of the management of science has not yet been ad-

dressed by research. Thus, taking into account the division of labour and

using the framework of coordination costs, this paper, which considers the

research team to be closely linked with specialisation and the division of

9

labour, explores the effects of specialisation in science and the role of man-

agement in serendipity and the productivity of scientific research.

The primary focus of this paper is on exploring the effects on serendipity

of the division of labour between management and science research. The

word “serendipity” was coined by the novelist Horace Walpole, who was

inspired by the Persian fairy tale, “Three Princes of Serendip”. Merton

and Barber (2004) explored how the word was unexpectedly popularised

without a clear definition from its 1754 coinage to the twentieth century.

In scientific circles, the word has been used since the nineteenth century,

when the importance of unplanned and accidental factors in the making of

scientific discoveries gained recognition. Serendipity has been noted for its

role in the work of inventors and entrepreneurs, by persons such as George

W. Merck, a president of Merck & Co., and Willis Whitney, a director of

research of the General Electric Laboratories.

In the colloquial sense, serendipity is the making of happy and unex-

pected discoveries. Many anecdotal stories reveal how unintentional find-

ings have yielded unexpectedly fortunate results. Many great discoveries,

such as penicillin, X-rays, celluloid, and artificial sweetener, have been ut-

terly fortuitous, making the concept of serendipity not well-operationalised

(Roberts, 1989; Shapiro, 1986). It is uncertain whether the accidental na-

ture of serendipity is linked to the nature of the discovery process or the

unexpected impact of the discovery. However, upon closer examination, it

is obvious that the unplanned and accidental nature of serendipity is con-

nected only with the discovery process. This is reflected in official defini-

tions of the word. For instance, The American Heritage Dictionary of the

English Language, fourth edition, defines serendipity as “the faculty of mak-

ing fortunate discoveries by accident.” Furthermore, distinguishing between

10

the unexpected and the accidental is difficult, especially when research in-

volves exploration of the unknown. In order to operationalise the concept of

serendipity, it is therefore appropriate to think of serendipity as “the act of

finding answers to questions not yet posed” (Stephan, 2010). This definition

focuses not only on the discovery process, but also on the relation between

discovery and the specific research question. Even though this definition

directs our attention to the extent to which the discovery answers a ques-

tion not yet posed, the present paper adopts this definition and explores the

relation between management and serendipity in science.

Serendipity has been considered one of the most important character-

istics of science. As mentioned above, many great discoveries in science

such as penicillin, X-rays, insulin, and the pulsar, have been regarded as

serendipitous findings. Since all images of serendipity have been created by

such great discoveries, serendipity is generally recognised as something that

scientists should explore further, when they encounter serendipitous find-

ings. In other words, it is thought that serendipity has a significant positive

effect on research quality.

Is serendipity good to pursue? One might suppose that great anecdotes

in the history of science cover only the successful results of serendipity. There

might have been many cases in which scientists explored unintended find-

ings and ended up with nothing. Furthermore, endogenous effects may exist

in the relation between research quality and reported serendipity. This is

because scientists explore a finding further only when they expect it will be

worth pursuing. Therefore, whether they obtained the finding as intended

or by chance may affect the likelihood of the eventual publication of the

finding. No study has empirically examined the extent to which pursuing

serendipitous findings brings better scientific outcomes. In addition to the

11

investigation of the effects of the division of labour on serendipity and pro-

ductivity, this study empirically investigates the extent to which pursuing

serendipity contributes to the quality of research by introducing instrumen-

tal variables.

3. Estimation Strategy

3.1. Hypotheses

This paper directs its attention to the managerial role in a research team

in order to explore the effect of management on serendipity and productivity

in science research, focusing on three points. The first point examines the

relation between serendipity and research performance. Based on anecdo-

tal evidence, we assume that serendipity improves the quality of research.

Hence, our first hypothesis to be explored is the following:

H-1: The existence of serendipity has a positive effect on the quality of

research.

The second point is related to the discussion about information asymme-

try and the coordination costs between management and research, which is

closely related to the discussion of serendipity. Scientists possess specialised

and domain-specific expertise. As the previous literature on scientific discov-

ery has explained, the nature of scientific discovery is highly unpredictable

(Polanyi, 1962), and tacit and uncodified knowledge plays an important

role in research, even though the outcomes of research are usually codi-

fied and published (Collins and Harrison, 1975; Polanyi, 1967). Learning is

highly situated in an on-site context (Brown and Duguid, 1991; Kogut and

Zander, 1992; Lave and Wenger, 1991). When scientists are committed to

actual research, they often encounter unexpected observations and findings.

12

Thus, if the managerial role in the research project team and the leading

role in the actual research are taken by different individuals, the research

project will have an information asymmetry between management and the

actual research. When a scientist observes unexpected but potentially cre-

ative serendipitous findings or encounters a serendipitous idea, they need to

encourage the person who plays a managerial role to change the initial re-

search plan in order to pursue the serendipitous. Presenting a serendipitous

encounter to a manager may be risky, particularly when the new idea or

observation is contrary to accepted ways of doing or thinking about things

(Pelz and Andrews, 1966). Thus, even if a surprising fact or relation is

observed, there may be a case in which it is not (optimally) investigated

by the discoverer (Barber and Fox, 1958; Van Andel, 1992). In contrast, if

a core scientist is also responsible for the management of the project, the

coordination and communication costs for shifting the research to pursue

the serendipitous findings will be decreased. Hence, this paper investigates

the following hypothesis:

H-2: Serendipity is positively related to the integration of core-scientists

into management.

However, if a core scientist plays a managerial role, the advantage of

the division of labour in science will not be fully realised. Efficiency is

increased by specialisation and concentration on a single subtask. Managing

a research team and conducting research require different sets of expertise.

Thus, it is possible that if a core scientist is separated from a managerial role,

they can focus on the research. This is important, particularly for a large

scale research project, which requires many bureaucratic procedures, much

paper work, and many managerial tasks. This paper, therefore, explores the

13

following hypothesis:

H-3: Research productivity is positively related to the separation of core-

scientists from management.

3.2. Data Description

We use data from the scientists’ survey conducted in Japan and the

U.S. in the time period 2009–2011 by Hitotsubashi University, the National

Institute of Science and Technology Policy (NISTEP) of the Ministry of

Education, Culture, Sports, Science, and Technology, and Georgia Institute

of Technology.1 The survey aimed to characterise the knowledge creation

process in science. It sampled a population of articles and letters which

had been recorded in the Web of Science database of Thomson Reuters

from 2001 to 2006 (database years). Review papers were excluded from the

population. Because this survey was particularly interested in characterising

high-performing research projects, it divided the population into two groups:

highly cited papers and a control group, based on the number of citations as

of the end of December 2006. The highly cited papers (H paper) consisted

of the top 1% of the highly cited papers in each journal field (22 fields in

total) and from each database year, while the normal papers (N paper) were

randomly selected from each journal field and from each database year from

the population of the survey, excluding highly cited papers. The sampling

rates were different between the two groups: roughly one-third of the samples

fell in the H group (the top 1% in the world). The survey sent a questionnaire

to the corresponding author of the sampled paper (“focal paper”) and asked

about the research projects which generated their paper. The respondent

1The full questionnaire can be found in Appendix 2 of Nagaoka et al. (2010).

14

was asked to define the project as an entire body of continuous research that

generated the focal paper and related papers. The response rate was 26%

in the U.S. (2,329 authors responded) and 27% in Japan (2,081 authors).

The questions asked by the survey include the following topics: the

knowledge sources which inspired the projects; uncertainty in the knowl-

edge creation process; research competition; composition of the research

team; sources of research funding; the research outputs, including papers,

patents, and licenses; and the profile of the scientists. The basic findings

from the survey characterised the research inputs, the knowledge creation

process, and the research outputs in H and N papers in both countries.2

One of the findings, for example, is that both the main result of the paper

and the research process were as initially expected or planned only for 11%

of the H papers in Japan and 14% in the U.S. (17% of the N papers in

both countries) (Nagaoka et al., 2011). This empirically demonstrates that

science confronts high uncertainty in general. By using the questions in the

survey about managerial roles and serendipity in particular, this paper aims

to explore the effects on serendipity of the division of labour in management.

In the Appendix, we provide the exact questions asked and the responses.

It must be noted that the survey was not constructed as a panel dataset,

even though the focal papers are sampled from a multi-year time period.

3.3. Definitions of the Variables

This section introduces the definitions of the variables used in our estima-

tions. Table 1 presents a complete list of the variables and their definitions,

and Table 2 shows summary statistics for all variables.

2See Nagaoka et al. (2011) for the detailed results.

15

3.3.1. Dependent Variables

The H-1 hypothesis uses the total number of citations observed by 2009,

which is denoted by number of citations, as a proxy for research quality.

Even though the division between H and N groups is determined by the ci-

tations in 2006, we chose the year 2009 so that the time window for citations

should be as long as possible in our database. The time window may still

be relatively short for some papers. However, since 70% of papers indicate

an increase in number of citations only by less than 10 from 2007 to 2008,

we believe that this variable is a reasonable proxy for research quality. Note

that the summary statistics in Table 2 show that the distribution of number

of citations is heavily skewed to the right. In order to deal with the skewed

distribution and the effects of outliers, we take the logarithm of number of

citations in the following estimations.

For the H-2 hypothesis, the dependent variable, serendipity, indicates

whether the respondent reported that the main finding of the focal paper

was obtained through serendipity. More specifically, this survey asked: “Has

the research output found the answers to questions not originally posed (in

other words, was the research output serendipitous)?” Approximately 55%

of the respondents answered yes. The highest was 61% in Computer Science,

and the lowest was 42%, in the Social Sciences.

For the H-3 hypothesis, the dependent variable, published papers, is the

number of refereed articles published by the entire research project. The

variation in published papers captures the variation in research productivity

in the regression where the research inputs such as project funds and the

number of researchers are controlled.

16

3.3.2. Independent Variables

Management structure is measured by two mutually exclusive variables.

Integration is a binary variable that takes the value one if the researcher

executed the central part of the research and contributed the most to the

research output, and at the same time, took a leading role in the research

management, designing the research project, organising the research team,

and/or acquiring research funds. Separation is a binary variable that takes

the value one if the researcher executed the central part of the research and

contributed the most to the research output, but took no managerial role.

There are observations that take zero for both integration and separation.

These observations, which exhibit a management structure in the middle

between clear integration and clear separation, account for 36% of the sample

for the H-2 model and 20% for the baseline H-3 model.

Other variables that describe research project characteristics are project

size, project duration, project funds, skill diversity, inter-lab community,

knowledge diversity, and competitor threat. Project size is the number of peo-

ple involved in the project, which includes collaborative researchers (includ-

ing coauthors), graduate students, undergraduates, and technicians. Since

not all projects had been terminated by the time of the survey, project du-

ration is calculated by subtracting the year when the project started from

the year of the most recent corresponding publication. project funds is the

total sum of research funding provided to the project. Since project funds is

heavily skewed to the right, we take its logarithm in estimations. Skill diver-

sity, inter-lab community, knowledge diversity, and competitor threat are all

binary variables. Skill diversity takes the value one if the researcher stated

that it was important for conceiving the research project to communicate

17

with researchers who have different research skills, for example, experimen-

tal researchers communicating with theorists. Inter-lab community takes

the value one if the researcher built a research community beyond their

own laboratory. Knowledge diversity takes the value one if the researcher

stated that it was important for conceiving the research project to communi-

cate with visiting researchers or postdoctoral researchers. Competitor threat

takes the value one if the researcher considered the possibility of competitors

who may have had priority in the research results.

Scientists are classified by the following variables: past publications, years

in paper, age, PhD, award, university, country, theory, and experiment. Past

publications measures the number of refereed papers published by the re-

searcher in the past three years.3 Years in paper measures the number of

years taken to publish the focal paper. Age is the respondent’s age at the

time of the survey. PhD, award, university, country, theory, and experiment

are all binary variables. PhD takes the value one if the researcher has a

Ph.D. or equivalent degree. Award takes the value one if the researcher re-

ceived a distinguished paper award or a conference award. University takes

the value one if the respondent works for a university. Country takes the

value one for respondents in the U.S. and zero for respondents in Japan.

Theory and experiment take the value one if the researcher specialises in

theory, respectively, experiments.

We control the respondent’s research field based on the survey’s classi-

fication. Table 3 shows the correspondence between its classification and

the 22 ESI journal fields. All scientific areas are divided into ten fields:

3The three year period corresponds to 2006–2008 in the Japanese survey and 2007–2009in the U.S. survey.

18

Chemistry, Materials Science, Physics & Space Science, Computer Science

& Mathematics, Engineering, Environment/Ecology & Geosciences, Clinical

Medicine & Psychiatry/Psychology, Agricultural Sciences & Plant & Animal

Sciences, Basic Life Sciences, and Social Sciences.4

3.4. Estimation Models

We employ the following estimation models to examine our three hy-

potheses. The H-1 model investigates the relation between serendipity and

research quality. In estimating the effect of serendipity on quality, we pay

particular attention to the possibility that serendipity has an endogenous

relation to the quality. As we reviewed above and will discuss in the fol-

lowing section, a researcher pursues publication of a scientific finding only

when the researcher believes that it is worth pushing forward, while the re-

searcher’s ex ante evaluation of the finding may differ depending on whether

the finding emerged from serendipity. If this is the case, the coefficient of an

ordinary least squares (OLS) regression may overestimate or underestimate

the effect of serendipity. To circumvent the endogeneity bias, we use a Two-

Stage Least Squares (2SLS) regression, in which we first regress serendipity

on the instrumental variables, and then the number of citations is regressed

on the predicted serendipity of the first stage. In this way, the coefficient

in the second stage captures the effect on the number of citations of the

exogenous variation in serendipity.

The H-2 model asks whether the integration of core-scientists with man-

agement affects serendipity. Since serendipity is a binary variable in our

dataset, we conduct a probit regression. An important issue with probit

4Social Sciences may be a fairly broad field compared to other fields. However, sinceabout 95% of the respondents are natural scientists, this makes no significant difference.

19

regressions is their fragility to any heteroskedasticity in the error terms.

Hence, we test whether our results are robust to misspecification of the

error term, and claim that our hypothesis still holds.

For the H-3 model, which examines the relation between management–

research separation and research productivity, we use a Negative Binomial

(NB) regression under the assumption that the variance of the dependent

variable takes a quadratic form. Since research productivity is measured

by the number of papers produced by the research project (controlled for

inputs), our dependent variable is necessarily discrete. Moreover, we only

have data on research projects that published at least one paper. For these

reasons, we use a zero-truncated NB regression model. An alternative would

be a Poisson regression, but that is not suitable in this case, since the equi-

dispersion hypothesis is strongly rejected at the 1% significance level. Since

the estimator of the NB regression coincides with that of quasi-maximum

likelihood, it is robust to misspecification of the distribution of the dependent

variable. That is, the NB regression yields a consistent estimator as long as

the specification of the conditional expectation of the regressand is correct.5

The unit of analysis for the H-3 model is the entire research project,

whereas the unit of analysis for the H-1 and H-2 models is the focal paper.

This is because the H-3 model is concerned with the effect of management

structure on the productivity of the entire project. In estimating the H-

1 and H-2 models, we restrict the observations to those cases where the

respondent is the first author of the focal paper, while for the H-3 model

we use observations for which the respondent was the researcher who took

5See Cameron and Trivedi (2005) for further discussion and Ding et al. (2010) for anapplication in a related context.

20

a central part in the research and contributed the most.6 Thus, the author

characteristics controlled in those two models are those of the first author,

while in the H-3 model they are those of the principal investigator. The

project funds variable is modified as the amount per paper for the H-1 and

H-2 models.

3.5. Estimation Issues

3.5.1. Sampling Bias

As a consequence of the survey method, one-third of the samples were

chosen from those researchers who wrote one of the top 1% highly cited

papers. Hence, our samples are not randomly drawn from the entire popu-

lation. We must consider this problem in order to obtain a consistent esti-

mator for the H-1 model, since the stratification depends on the regressand

(i.e., the number of citations).

A straightforward way to address this problem is to use the weighted least

squares method. In our regressions, we have two strata: highly cited papers

and others. Each observation i is weighted by the ratio between the popula-

tion frequency and the sample frequency of the stratum to which i belongs.

The weights are 0.032 for the highly cited papers and 1.433 for the others.

Under reasonable regularity conditions, this weighted least squares estimator

is consistent and asymptotically normally distributed (Wooldridge, 2010a,

2010b). Moreover, a consistent estimator of covariance matrix is obtained

by slightly modifying White’s (1980) heteroskedasticity-consistent covari-

ance matrix.

6In the H-1 and H-2 models, we excluded samples who are the corresponding but notfirst author of the focal paper to control the main author’s characteristics. About 30% ofthe sample was dropped after this selection.

21

3.5.2. Selection Bias

A number of anecdotes have casually reported that serendipity was im-

portant for successful research. Clearly, this cannot be taken as evidence

for a general tendency, since dramatic and successful anecdotes tend to be

highlighted and selected. As noted above, this paper adopts a definition of

serendipity, following Stephan (2010), as neutral as possible to the conse-

quential value of the serendipitous finding. Our survey contains many cases

in which scientists report that their finding was serendipitous in the sense

that it answered questions not yet posed, and yet the value of the finding,

measured by the number of citations, was not that large. By comparing

serendipitous findings to intentional findings with both successful and un-

successful findings, this paper aims to estimate the difference in the potential

values of findings that are found intentionally and accidentally from the per-

spectives of scientists. The neutral definition of serendipity enables us to

avoid the issue of selection bias that arises from overlooking the cases of

serendipitous but not highly valued findings.

3.5.3. Endogeneity Bias

The H-1 model concerns the population difference in the value of findings

between serendipitous findings and intentional ones. A regression of value on

serendipity may not yield a consistent estimator for this difference, however,

if the value difference affects the observation of serendipity in our data. In

other words, the estimate is biased if serendipity is an endogenous variable.

The serendipity observed in the data can be affected by the consequen-

tial value of the finding in the following sense. The value of the finding and

the serendipitous event are observable only conditional on the publication

of the finding. Moreover, a research team pursues publication only when

22

it considers it to be valuable. Hence, the observed citation rate might re-

flect not only the intrinsic value of a finding uncovered by a serendipitous

event, but also the research team’s evaluation of that discovery. There can

be two directions of endogenous effects from the quality of the finding to its

serendipity. The first possibility is that the researcher is less familiar with

the topic pertaining to the serendipitous finding than the topic originally

pursued; hence, the finding seems more novel to the researcher, who overes-

timates the value of the serendipitous finding. The second possibility is that

the researcher pursues their serendipitous finding only if it is highly valu-

able. This is because diverting the direction of research from the original

plan seems undesirable or risky. While we cannot determine which effect of

endogeneity is dominant ex ante, both effects imply that a simple regression

would result in underestimation or overestimation of the effect of serendipity

on research quality. Indeed, in the estimation of the H-1 model, a variable

addition test rejects the hypothesis that serendipity is an exogenous variable

at the 1% significance level (see Table 4).

We use instrumental variables to deal with the endogeneity bias. The

instrumental variables must correlate with the existence of serendipitous

findings, and they must not affect the ex ante evaluation of the findings. To

satisfy this criterion, we use two instrumental variables, skill diversity and

inter-lab community.

It is plausible that serendipity correlates with these two variables, since

complementarity in knowledge and skills are key to enhancing creativity.

For example, Heinze et al. (2009) pointed out that communication with spe-

cialists who have different knowledge or skills constitutes one of the most

important factors in inspiring a researcher’s creativity. Furthermore, we

assume that our instruments do not directly affect the ex-ante evaluation

23

of the serendipitous finding for the following three reasons. First, by con-

struction, these two instruments characterise the entire project rather than

the focal research paper. Second, even if the instruments affect the qual-

ity of the findings, the effect occurs mainly through enhancing creativity.

For example, skill diversity, which indicates whether communication with

researchers with different skills was important for conceiving the research

project, improves the value of findings by increasing the chance of valuable

serendipitous findings. In fact, we find that the variation of number of cita-

tions explained by skill diversity through serendipity in our 2SLS estimate

almost exhausts the correlation between number of citations and skill diver-

sity. Third, we conducted Sargan and Bassman over-identification tests to

see whether these two instruments are exogenous.7 The tests did not reject

the hypothesis that all the instruments are exogenous at the 20% significance

level. See Table 4 for the detailed results.

4. Results

4.1. Baseline Estimates

Table 4 summarises the estimation result for the H-1 model. We observe

that serendipity has a positive correlation with the logarithm of citation

counts at the 5% significance level in the 2SLS regression. This confirms our

hypothesis H-1: the findings through serendipitous events exhibit a higher

value in terms of the number of citations than findings which were made

intentionally. Note that this effect is insignificant in the OLS regression.

Namely, our sample does not exhibit a significantly positive correlation be-

tween serendipitous events and greater citation counts directly. In other

7Since our sample is drawn from two groups, highly cited and others, with differentsampling rates, we conducted over-identification tests for each subsample.

24

words, there are many observations which report serendipitous events and

low citations. When the serendipitous events are instrumented by exoge-

nous variables, however, a positive effect of serendipity is identified. We

interpret this estimation result as follows. Researchers tend to overestimate

the value of a serendipitous finding, and thus pursue the publication of the

finding with a less stringent criterion than intentional findings. As a result,

relatively more publication of serendipitous findings with low value are ob-

served, which masks the intrinsic positive quality of serendipitous findings.

However, the estimate may be open to other interpretations.

Among the control variables, we note that the existence of a threat from

competitors and the number of past publications of the first author have sig-

nificant positive effects. This is natural, as the competitor threat proxies the

potential value of the research topic, and the publication record represents

the researcher’s ability. We observe that the project size is negatively re-

lated to the quality on average. We robustly find that university researchers

exhibit less citation counts, which may reflect the fact that more of them

publish in a field with fewer researchers, than do researchers in industry.

The country effect shows that the average number of citations is higher in

the U.S. than in Japan.

The H-2 model examines the connection between management structure

and serendipity. In Table 5, the left column exhibits the result of the base-

line probit regression for H-2. We observe a positive correlation between

management–research integration and serendipity. The baseline model is a

heteroskedastic probit, because the homoskedastic specification is statisti-

cally rejected at the 1% significance level. The variance of the probit index

depends on project duration, which is likely to be an exogenous variable in

this model.

25

Table 5 also indicates that serendipity is reported more often when the

researchers were more engaged with an inter-laboratory community (a re-

search community beyond their own labs) or when the conception of the

project benefited from communication with researchers with different skills.

This result justifies our use of these variables as instrumental variables in the

estimation of the H-1 model. In addition, we note from the country dummy

that the Japanese respondents reported serendipity more often than the U.S.

respondents.

The regression results of the H-3 model are shown in Table 6. Our

hypothesis is that the separation of management from research increases

research productivity, measured in the number of publications generated by

the entire project. The first column shows that the separation exhibits a

positive effect on the number of publications. When the interaction with

project size is included as in the second column, separation has a negative

coefficient, while the interaction term exhibits significant positive effects.

Thus, the estimates imply that the separating management style yields a

higher productivity for a large project team. This result conforms to our

hypothesis. Quantitatively, the baseline estimate predicts that the adoption

of a separating management style increases the number of publications for a

project with more than three researchers. The average increase is 1.9 for a

project with 6 researchers and 4.8 for a project with 10 researchers, whereas

the projects with 6 and 10 researchers correspond to the median and the

75th percentile in project size, respectively. Considering that the median

number of papers published by a project is 7, the increase of publications

by adopting a separating management style seems non-negligible.

The coefficients for the control variables indicate that the number of pa-

pers generated by the project is large when the project fundsing is large,

26

when the project duration is long (though the effect is attenuated by the

squared term), when threatening competitors exist, and when the principal

investigator has a good publication record or a Ph.D. Projects led by univer-

sity researchers generate fewer papers than the other projects, indicating the

same pattern as for the H-1 model, where university researchers receive less

citations. U.S. projects also tend to generate fewer papers than Japanese, in

contrast with the result in H-1, which showed that U.S. researchers received

more citations than Japanese.

4.2. Robustness Checks

We conducted various robustness checks. First, we tested the validity of

our choice of instrumental variables. Three different choices were examined:

skill diversity (labeled “H-1-1”), knowledge diversity (labeled “H-1-2”), and

knowledge diversity and inter-lab community (labeled “H-1-3”). Table 7

shows the results. All three models suggest that serendipity and the number

of citations are positively correlated, so that our hypothesis is maintained.

Secondly, for the H-2 model, we tested different specifications of the error

term. The estimation for the homoskedastic probit is shown in the right

column of Table 5. We observe that the estimated coefficients are qualita-

tively similar to the baseline results. We also confirmed that an alternative

specification with variance depending on country as well as project duration

generates a similar pattern. Finally, we restricted the sample in the H-3

model to the projects which clearly stated their management structure, by

dropping the observations stating that management was not necessary or

choosing “other” in the survey. The estimates in Table 6 show that all the

major results are still qualitatively unchanged. Moreover, dropping the out-

liers (top 1%) of project size, published papers, and past publications in the

27

H-3 model did not alter our estimation result qualitatively.

5. Conclusion

This paper investigated the effects of division of labour between manage-

ment and research on serendipity and productivity in scientific research. The

major estimated results reveal the following three points. First, the estima-

tion shows that on average serendipity brings about better research quality.

Much of the anecdotal evidence has suggested serendipity plays a critical

role in science. The estimation from the survey empirically demonstrates

that serendipity is a key feature of science not only in great discoveries but

also in science in general. This finding suggests the importance of a man-

agement that gives scientists the flexibility to pursue a serendipitous finding

when they face the unintended and unexpected. Second, the integration of a

managerial role with a leading research role has a positive effect on serendip-

ity. Following the discussion of the division of labour and its coordination

costs, it can be interpreted that integration reduces the coordination costs

between management and research and provides scientists with flexibility in

research. When a scientist finds unintended and unexpected findings, the

findings are usually still too crude and uncertain to be articulated. Thus, if

the front-line scientist is delegated decision-making in the research project,

they can fully desterilise any uncodified tacit knowledge and use managerial

resources in the context of the actual situation. However, if a hierarchical

managerial role, which increases the information asymmetry and incommen-

surability between management and actual research, is implemented in a top

down fashion, it rebuffs research efforts to pursue a serendipitous finding.

Thus, the serendipitous findings based on ground level intuition are seldom

pursued. Thirdly, the results show that if a project is managed not by a

28

scientist who actually leads the scientific research, but by a person who spe-

cialises in research management, it increases the productivity of the research

when measured by the number of papers. These three findings imply that

there is a trade-off between pursuing serendipity and achieving research ef-

ficiency in science, via who plays the managerial role and who the leading

research role.

Returning to the example of penicillin, the findings of the paper suggest

that Fleming would have faced difficulties in changing his original research

plan to pursue his serendipitous finding if he had been working in a large

laboratory and his research had been led by a competent project manager.

In other words, Fleming would not have pursued the serendipitous finding,

but he would have delivered more papers concerning the original research

project if a managerial role had been played by a specialised director.

Serendipity plays an essential role in discoveries not only in science, but

also in technology, management, business practices, art, and daily life (Ja-

cobs, 2010; Svensson and Wood, 2005; Van Andel, 1992). The findings

of this paper have implications for corporate R&D and university research

administrators in particular. First, these results about the effects of the

division of labour between research and management on serendipity and

productivity in science are consistent with the contingency theory of firms

between the complexity of environment (e.g., demand, strategic positioning,

and technology) and their organisational structure (Burns and Stalker, 1961;

Lawrence and Lorsch, 1967; Scott, 1981), which have indicated that decen-

tralised and less formalised management allows a high degree of flexibility.

This is suitable when an organisation faces many exceptional problems and

problem solving is not easy (Perrow, 1967; Woodward, 1965). It suggests

that if corporate R&D is involved with embedded and uncodified knowledge

29

and if the firm needs alert responses to rapidly changing demand and supply

conditions, greater autonomy should be given to the R&D unit (Birkinshaw

et al., 2002). This may explain why it is difficult for corporate R&D over-

seen by a central business director to profit from serendipitous findings in

the laboratory. The findings of this paper suggest that decisions should be

made where the important information is gathered and knowledge is created

if unexpected findings are important.

Secondly, the findings have implications for university research adminis-

trators. Since the size of research projects is increasing and the competition

for priority of discovery in science is becoming fiercer, university research

administrators who are responsible for planning and managing research ac-

tivities and promoting research outcomes are serving very important roles in

science (Kaplan, 1959). According to the findings of this paper, the research

outcomes of the team depend highly on the extent to which the leading re-

search role and the managerial role are divided in the team. Operational

administrators are usually trained to complete the project’s goal. In fact,

they attempt to manage in a way that will eliminate uncertainty in their af-

fairs so that they can meet budgets and target deadlines (Udwadia, 1990). If

a managerial decision is made by such an operational research administrator,

it is highly likely that they would adhere closely to the initial research plan

even when scientists in the team encounter something serendipitous. Our

results do not suggest that the division of labour in research and manage-

ment is always inappropriate in science. The findings of this paper highlight

the way that the separation of research from managerial role allows the team

to achieve higher productivity as measured by the number of publications

for a given level of inputs. It would also work well if the research project

aims to make a thorough investigation of the possible combinations of ma-

30

terials with a door-to-door-check method, for example. Additionally, it is

quite important for a university research administrator to fully understand

the nature of discovery in science and the trade-off between serendipity and

productivity in science via who plays the managerial role and who the lead-

ing research role in research management (Kaplan, 1959; Kulakowski and

Chronister, 2006).

For the purpose of thinking about future research, it is important to note

three limitations to the present study. The first is related to the time scope

of the research project. A research project does not stand alone for a scien-

tist. Instead there is usually a sequence of continuous and related projects.

Even when the project is defined by the respondent themselves, as an entire

body of research that generated the focal paper and related papers, there

still remains, to some degree, a continuity of the research projects. This

problem is also related to the measurement of the quality of the research.

The scientists’ survey, which aimed to explore the nature of high performing

research projects compared with that of randomly selected research projects,

measured the quality of a research project by the number of citations that

the focal paper of the project received. Even though the measurement of

research quality by the number of citations has been widely used, there are

debates over the validity of this method. One of the issues related to the

present paper lies in the base year of the citation. Although it is necessary

to set a certain base point for measuring the number of citations, one might

always question whether some research might become highly valued after

the data cut-off point, for instance, due to the advancement of complemen-

tary technology. Since the scientists’ survey adopted a relatively short data

cut-off point because it aimed to distribute the questionnaires properly to

the corresponding authors, there is a possibility that this data cut-off point

31

underestimates the quality of research that takes a long time to be valued. In

addition, we used cross-sectional data, rather than a panel dataset. There-

fore, it should be noted that our empirical results are patterns of associations

between variables. Even though we carefully introduced instrumental vari-

ables in order to eliminate endogenous bias and tested our hypotheses based

on the patterns of correlations in the regression models, it is important to

collect panel data for deducing causal relations. These time scope issues are

an endemic problem in the study of science.

The second point is related to the country effects. The main aim of this

paper was to explore the effects of the division of labour between manage-

ment and research in science on serendipity and publication productivity.

Thus, although this paper introduces a country dummy variable to control

for country effects, it did not address the international comparison of the

management of science. However, as the estimations show, the country vari-

ables show significant effects on serendipity and publication productivity. It

is reasonable to assume that the ways in which scientific research is organ-

ised and managed are different across countries. Since the sample size of the

study did not allow robust estimation in the international comparison, it is

potentially important to collect a larger sample of the management of science

across countries so that one can make detailed international comparisons.

The third point is linked to the quality of the project managers. A

key result suggests that if scientific research is bureaucratically controlled

in a research organisation, serendipitous encounters will not be realised. In

other words, even when a managerial role and a leading research role are

played by different people, serendipity will be realised if a manager shares

tacit and domain-specific knowledge with leading scientists and understands

the nature of scientific discovery. This paper presupposes a certain degree of

32

incommensurability, which was proposed by “Kuhnian paradigm arguments”

(Kuhn, 1970) between a manager and leading scientists. However, the degree

of incommensurability depends on a manager’s expertise and capabilities.

Since the scientists’ survey does not allow the investigation of a manager’s

capabilities, this paper does not explore the quality of managers in a research

organisation. Organisations for university research administrators such as

SRA (Society of Research Administrators) and NCURA (National Council of

University Research Administrators) in the U.S., and ARAM (Association

of Research Managers and Administrators) in the U.K., were established

in the 1960s. Not only these organisations but also governments (e.g., the

Development of a Research Administration System program launched by the

Ministry of Education, Culture, Sports, Science, and Technology in Japan)

are beginning to understand that a managerial role should be played by a

specialist who can share tacit and domain-specific knowledge with leading

scientists: scientists could then focus on large-scale research projects, which

could have the managerial flexibility for realising serendipitous encounters.

The previous literature on how scientists with different sets of expertise

and paradigms communicate has indicated that scientists communicate in

groups called “trading zones,” where they can agree on the rules of exchange,

share the same learned languages, and share tacit knowledge (Collins et al.,

2007; Galison, 1997, 1999). However, since the extent to which managers

and scientists can reduce the degree of incommensurability depends on a

manager’s ability, it is important to explore the manager’s expertise and

capabilities for the research outcome in detail.

33

Table 1: Definitions of Variables

Variable Definitionnumber of citations Cumulative number of citations in 2009.serendipity Equals 1 if their research output found the answers to ques-

tions not originally posed.published papers The total number of refereed publications produced by the

research project.integration Equals 1 if the researcher executed the central part of the

research and contributed the most to the research output,and at the same time, took a leading role in the researchmanagement, designing the research project, organising theresearch team, and/or acquiring research funds.

separation Equals 1 if the researcher executed the central part of theresearch and contributed the most to the research output,but on the other hand, played no managerial role.

project size Sum of the number of collaborative researchers (includingcoauthors), graduate students, undergraduates, and techni-cians involved in the project.

project duration Years between the launch of the research project and thelatest publication by the project.

project funds The total sum of research funds prepared for the project.skill diversity Equals 1 if the researcher states that communication with

researchers who have different research skills was importantfor conceiving the research project.

inter-lab community Equals 1 if the researcher built a research community be-yond own laboratory.

knowledge diversity Equals 1 if the researcher states that communication withvisiting researchers or postdoctoral researchers was impor-tant for conceiving the research project.

competitor threat Equals 1 if the researcher considered the possibility of com-petitors who may have obtained priority in the researchresults.

past publications The number of refereed papers that the researcher publishedin the three previous years.

years in paper Years between the launch of the project and the publicationof the focal paper.

age Respondent’s age at the time of survey.PhD Equals 1 if the researcher had a Ph.D. or equivalent degree.award Equals 1 if the researcher received a distinguished paper

award or a conference award.university Equals 1 if the researcher works for universities.country Equals 1 for the respondents in the U.S. and 0 for respon-

dents in Japan.theory Equals 1 if the researcher specialised in theoretical work.experiment Equals if the researcher specialised in experiments.

34

Table 2: Summary Statistics

H-1 Model (Obs.= 1629) H-3 Model (Obs.= 1892)Variable Mean SD Min Med Max Mean SD Min Med Max

number of citations 51.0 102.0 0 12 1367serendipity 0.59 0.49 0 1 1published papers 19.3 42.5 1 7 590integration 0.58 0.49 0 1 1separation 0.07 0.25 0 0 1 0.06 0.23 0 0 1project size 9.66 25.21 1 6 603 9.95 21.7 1 6 601project duration 7.23 4.99 0 7 46skill diversity 0.29 0.46 0 0 1inter-lab community 0.45 0.5 0 0 1knowledge diversity 0.24 0.43 0 0 1project funds (M$) 0.14 1.12 0 0.02 31.3 0.15 1.17 0 0.02 31.3competitor threat 0.26 0.44 0 0 1 0.31 0.46 0 0 1past publications 20.6 41.5 0 10 750 26.4 50.8 0 12.5 750PhD 0.78 0.42 0 1 1 0.82 0.39 0 1 1award 0.34 0.47 0 0 1 0.41 0.49 0 0 1age 48.5 10.2 31 47 91 50.0 9.93 32 49 91years in paper 3.14 3.48 0 2 36 3.18 3.41 0 2 36university 0.73 0.45 0 1 1 0.77 0.42 0 1 1country 0.39 0.49 0 0 1 0.32 0.47 0 0 1theory 0.24 0.42 0 0 1 0.2 0.4 0 0 1experiment 0.62 0.49 0 0 1 0.68 0.47 0 1 1

Note: Sample statistics of H-2 model are similar to those of H-1 model, and hence omitted.

35

Table 3: List of Fields

22 ESI Journal Fields 10 Fields Obs. %Agricultural Sciences Agricultural Sciences & 349 7.9

Plant & Animal Science Plant & Animal ScienceBiology & Biochemistry Basic Life Science 910 20.6

ImmunologyMicrobiology

Neuroscience & BehaviourPharmacology & Genetics

Chemistry Chemistry 441 10.0Clinical Medicine Clinical Medicine & 710 16.1

Psychiatry/Psychology Psychiatry/PsychologyComputer Science Computer Science & 208 4.7

Mathematics MathematicsEconomics & Business Social Sciences 250 5.7Social Science, general

Engineering Engineering 368 8.3Environment/Ecology Environment/Ecology & 308 7.0

Geosciences GeosciencesMaterial Science Material Science 214 4.9Multidisciplinary (Journal field was assigned based on 13 0.3

the analysis of the backward citations)

Physics Physics & 639 14.5Space Science Space Science

Total 4410 100

36

Table 4: H-1 Model (Effect of Serendipity on Research Quality)

OLS 2SLSserendipity 0.043 (0.061) 1.066∗∗∗ (0.395)integration 0.112∗ (0.067) 0.037 (0.079)separation 0.013 (0.126) -0.108 (0.137)project size -0.002∗∗ (0.001) -0.002∗ (0.001)project funds (log) 0.024∗ (0.014) 0.015 (0.015)competitor threat 0.333∗∗∗ (0.077) 0.271∗∗∗ (0.087)past publications 0.002∗∗∗ (0.001) 0.002∗∗∗ (0.001)years in paper -0.019 (0.017) -0.038∗ (0.021)(years in paper)2 0.000 (0.001) 0.001 (0.001)age -0.060∗∗ (0.025) -0.047 (0.029)(age)2 0.000∗ (0.000) 0.000 (0.000)PhD 0.221∗∗∗ (0.082) 0.186∗∗ (0.091)award 0.004 (0.065) 0.014 (0.073)university -0.143∗∗ (0.072) -0.178∗∗ (0.081)country 0.309∗∗∗ (0.074) 0.543∗∗∗ (0.122)theory -0.059 (0.111) -0.035 (0.124)experiment 0.113 (0.094) 0.130 (0.106)field fixed effects Y YObservations 1629 1629F -statistic 7.759 6.587Variable addition test: F (1, 1602)=8.66∗∗∗

Sargan test: χ2(1)= 0.95, 1.29 (p-value = 0.33, 0.25)Basmann test: χ2(1)=0.93, 1.21 (p-value = 0.34, 0.27)

Standard errors in parentheses∗ p < 0.10, ∗∗ p < 0.05, ∗∗∗ p < 0.01Note: Over-identification tests are conducted separatelyfor N (normal papers) and H (highly cited papers).

37

Table 5: H-2 Model (Effect of Management Structure on Serendipity)

Heteroskedastic Probit Probitintegration 0.059∗∗ (0.025) 0.048∗ (0.026)project size -0.000 (0.000) -0.000 (0.001)project funds (log) 0.007 (0.005) 0.005 (0.006)skill diversity 0.081∗∗∗ (0.027) 0.097∗∗∗ (0.026)inter-lab community 0.113∗∗∗ (0.030) 0.131∗∗∗ (0.027)competitor threat 0.034 (0.029) 0.048∗ (0.028)past publications -0.000 (0.000) 0.000 (0.000)years in paper -0.002 (0.007) 0.011 (0.008)(years in paper)2 0.000 (0.000) -0.000 (0.000)age -0.003 (0.009) -0.016 (0.011)(age)2 0.000 (0.000) 0.000 (0.000)PhD 0.022 (0.034) 0.029 (0.033)award 0.011 (0.026) -0.008 (0.027)university -0.002 (0.028) 0.001 (0.029)country -0.245∗∗∗ (0.036) -0.274∗∗∗ (0.037)theory 0.027 (0.047) 0.029 (0.048)experiment 0.018 (0.039) 0.021 (0.042)field fixed effects Y Ylnσ2

project duration -0.061∗∗∗ (0.023)Observations 1474 1474Log Likelihood -915.7045 -919.700LR test for homoskedasticity: χ2(1) = 7.99∗∗∗

Standard errors in parentheses ∗ p < 0.10, ∗∗ p < 0.05, ∗∗∗ p < 0.01

Note: All coefficients report average marginal effects exceptfor project duration.

38

Table 6: H-3 Model (Effect of Management Structure on Research Productivity)

All Sample Projects Sample with Management Needsseparation 0.287∗∗ (0.132) -0.137 (0.212) 0.294∗∗ (0.131) -0.134 (0.208)sep. × project size 0.051∗∗ (0.022) 0.052∗∗ (0.022)project size 0.012∗∗∗ (0.002) 0.012∗∗∗ (0.002) 0.012∗∗∗ (0.002) 0.012∗∗∗ (0.002)project duration 0.152∗∗∗ (0.015) 0.152∗∗∗ (0.015) 0.141∗∗∗ (0.016) 0.142∗∗∗ (0.016)(project duration)2 -0.003∗∗∗ (0.001) -0.003∗∗∗ (0.001) -0.002∗∗∗ (0.001) -0.002∗∗∗ (0.001)project funds (log) 0.115∗∗∗ (0.012) 0.114∗∗∗ (0.012) 0.115∗∗∗ (0.012) 0.115∗∗∗ (0.012)competitor threat 0.304∗∗∗ (0.068) 0.284∗∗∗ (0.068) 0.265∗∗∗ (0.069) 0.245∗∗∗ (0.069)past publications 0.005∗∗∗ (0.001) 0.005∗∗∗ (0.001) 0.005∗∗∗ (0.001) 0.005∗∗∗ (0.001)age -0.018 (0.030) -0.017 (0.030) -0.024 (0.032) -0.022 (0.032)(age)2 0.000 (0.000) 0.000 (0.000) 0.000 (0.000) 0.000 (0.000)PhD 0.373∗∗∗ (0.094) 0.388∗∗∗ (0.094) 0.321∗∗∗ (0.098) 0.337∗∗∗ (0.097)award 0.111∗ (0.067) 0.103 (0.067) 0.074 (0.068) 0.066 (0.068)university -0.262∗∗∗ (0.073) -0.279∗∗∗ (0.073) -0.309∗∗∗ (0.074) -0.328∗∗∗ (0.075)country -0.486∗∗∗ (0.072) -0.491∗∗∗ (0.072) -0.505∗∗∗ (0.074) -0.511∗∗∗ (0.074)theory 0.125 (0.133) 0.142 (0.132) 0.012 (0.139) 0.030 (0.138)experiment -0.105 (0.111) -0.078 (0.111) -0.056 (0.114) -0.029 (0.113)field fixed effects Y Y Y YObservations 1892 1892 1708 1708Log Likelihood -6630.021 -6627.055 -6069.145 -6065.998

Standard errors in parentheses ∗ p < 0.10, ∗∗ p < 0.05, ∗∗∗ p < 0.01

Note: All coefficients are semi-elasticities.

39

Table 7: H-1 Model (2SLS with Alternative Instruments)

H-1-1 H-1-2 H-1-3serendipity 1.281∗∗ (0.574) 1.953∗∗ (0.930) 0.982∗∗ (0.478)integration 0.014 (0.086) -0.034 (0.108) 0.036 (0.079)separation -0.141 (0.150) -0.218 (0.188) -0.107 (0.138)project size -0.002∗∗ (0.001) -0.002 (0.002) -0.002∗∗ (0.001)project funds (log) 0.075∗∗ (0.036) 0.053 (0.047) 0.084∗∗∗ (0.032)competitor threat 0.237∗∗∗ (0.092) 0.202∗ (0.111) 0.253∗∗∗ (0.087)past publications 0.002∗∗ (0.001) 0.002∗ (0.001) 0.002∗∗∗ (0.001)years in paper -0.050∗∗ (0.022) -0.060∗∗ (0.029) -0.045∗∗ (0.021)(years in paper)2 0.001 (0.001) 0.001 (0.001) 0.001 (0.001)age -0.046 (0.031) -0.037 (0.038) -0.050∗ (0.029)(age)2 0.000 (0.000) 0.000 (0.000) 0.000 (0.000)PhD 0.177∗ (0.096) 0.154 (0.114) 0.187∗∗ (0.090)award 0.006 (0.077) 0.015 (0.090) 0.002 (0.072)university -0.148∗ (0.088) -0.180∗ (0.106) -0.134 (0.083)country 0.591∗∗∗ (0.154) 0.741∗∗∗ (0.237) 0.524∗∗∗ (0.134)theory -0.034 (0.129) -0.017 (0.151) -0.042 (0.122)experiment 0.111 (0.109) 0.126 (0.128) 0.104 (0.103)field fixed effects Y Y YObservations 1629 1629 1629F -statistic 6.490 4.801 7.348

Standard errors in parentheses ∗ p < 0.10, ∗∗ p < 0.05, ∗∗∗ p < 0.01

40

Appendix: Selected Survey Questions and Responses

Question: Has the research output found the answers to questions not origi-nally posed?

Response RatesAnswers Highly Cited NormalYes 59.9% 54.0%No 40.1% 46.0%

Question: Please indicate which of the following best describes your role in the man-agement of the research project.

Response RatesAnswers Highly Cited Normal(1) A leading role in the research management, design-ing the research project, organising the research team,and/or acquiring research funds

70.9% 69.2%

(2) A member of the research management, but a roleless than that of the leader

14.1% 14.8%

(3) No managerial role 7.2% 5.8%(4) Management was not necessary 5.8% 8.0%(5) Other 2.1% 2.3%

Question: Please indicate which of the following best describes your role in the re-search implementation.

Response RatesAnswers Highly Cited Normal(1) I executed the central part of the research and con-tributed the most to the research output

64.4% 65.5%

(2) I took part in the central part of the research, but mycontribution was not as substantial as that of the centralresearcher

20.8% 21.9%

(3) I implemented the research under the guidance of theabove members

2.1% 3.0%

(4) I contributed to the research through the provisionof materials, data, equipment, or facilities

2.7% 2.8%

(5) Other 10.0% 6.8%

41

References

Adams, J. D., Grant, C. B., Clemmons, R. J., Stephan, P. E., 2005. Scientific

teams and institutional collaborations: Evidence from U.S. universities.

1981–1999. Research Policy 34 (3), 259–285.

Agrawal, A., Goldfarb, A., 2008. Restructuring research: Communication

costs and the democratization of university innovation. American Eco-

nomic Review 98 (4), 1578–1590.

Allen, T. J., 1977. Managing the Flow of Technology: Technology Trans-

fer and the Dissemination of Technological Information within the R&D

Organization. MIT Press, Cambridge, MA.

Andrews, F. M., 1979. Scientific Productivity: The Effectiveness of Research

Groups in Six Countries. Cambridge University Press.

Arora, A., Gambardella, A., 1994. The changing technology of technological

change: General and abstract knowledge and the division of innovative

labour. Research Policy 23 (5), 523–532.

Barber, B., Fox, R. C., 1958. The case of the floppy-eared rabbits: An

instance of serendipity gained and serendipity lost. American Journal of

Sociology 64 (2), 128–136.

Barnard, C. I., 1938. The Functions of the Executive. Harvard University

Press, Cambridge, MA.

Becker, G. S., Murphy, K. M., 1992. The divison of labor, coordination costs,

and knowledge. Quarterly Journal of Economics 107 (4), 1137–1160.

42

Birkinshaw, J., Nobel, R., Ridderstrale, J., 2002. Knowledge as a contin-

gency variable: Do the characteristics of knowledge predict organization

structure? Organization Science 13 (3), 274–289.

Brown, J. S., Duguid, P., 1991. Organizational learning and communities-

of-practice: Toward a unified view of working, learning, and innovation.

Organization Science 2 (1), 40–57.

Burns, T. B., Stalker, G. M., 1961. The Management of Innovation. Tavis-

tock Publications, London.

Cameron, C. A., Trivedi, P. K., 2005. Microeconometrics: Methods and

Applications. Cambridge University Press.

Chesbrough, H., 2003. Open Innovation: The New Imperative for Creat-

ing and Profiting from Technology. Harvard Business School Press, Cam-

bridge, MA.

Collins, H., Evans, R., Gorman, M., 2007. Trading zones and interactional

expertise. Studies in History and Philosophy of Science 38, 657–666.

Collins, H., Harrison, R. G., 1975. Building a tea laser: The caprices of

communication. Social Studies of Science 5 (4), 441–450.

Dasgupta, P., David, P., 1994. Toward a new economics of science. Research

Policy 23 (5), 487–521.

Ding, W. W., Levin, S. G., Stephan, P. E., Winkler, A. E., 2010. The

impact of information technology on academic scientists’ productivity and

collaboration patterns. Management Science 56 (9), 1439–1461.

43

Ellison, G., 2002. Evolving standards for academic publishing: A Q-R the-

ory. Journal of Political Economy 110 (5), 994–1034.

Fleming, L., Mingo, S., Chen, D., 2007. Collaborative brokerage, generative

creativity, and creative success. Administrative Science Quarterly 52 (3),

443–475.

Galison, P., 1997. Image and Logic: A Material Culture of Microphysics.

University of Chicago Press.

Galison, P., 1999. Trading zone: Coordinating action and belief. In: Biagioli,

M. (Ed.), The Science Studies Reader. Routledge, New York, pp. 137–160.

Garvey, W. D., 1979. Communication, the Essence of Science: Facilitating

Information Exchange among Librarians, Scientists, Engineers and Stu-

dents. Pergamon, Oxford.

Gouldner, A. W., 1954. Patterns of Industrial Bureaucracy. The Free Press,

Glencoe, IL.

Hagedoorn, J., 2002. Inter-firm R&D partnerships: An overview of major

trends and patterns since 1960. Research Policy 31 (4), 477–492.

Heinze, T., Shapira, P., Senker, J. M., 2009. Organizational and institutional

influences on creativity in scientific research. Research Policy 38 (4), 610–

623.

Hollingsworth, R. J., 2006. The dynamics of American science: An institu-

tional and organizational perspective on major discoveries. In: Beckert, J.,

Ebbinghaus, B., Hassel, A., Manow, P. (Eds.), Transformationen des Kap-

italismus: Festschrift fur Wolfgang Streeck zum sechzigsten Geburstag.

Campus Verlag, Frankfurt, pp. 361–380.

44

Jacobs, B. A., 2010. Serendipity in robbery target selection. British Journal

of Criminology 50 (3), 514–529.

Kaplan, N., 1959. The role of the research administrator. Administrative

Science Quarterly 4 (1), 20–42.

Kasperson, C. J., 1978. An analysis of the relationship between information

sources and creativity in scientists and engineers. Human Communication

Research 4 (2), 113–119.

Kogut, B., Zander, U., 1992. Knowledge of the firm, combinative capabili-

ties, and the replication of technology. Organization Science 3 (3), 383–

397.

Kuhn, T. S., 1970. The Structure of Scientific Revolutions, 2nd Edition.

University of Chicago Press.

Kulakowski, E. C., Chronister, L. U., 2006. Research Administration and

Management. Jones and Bartlett, Sudbury, England.

Lave, J., Wenger, E., 1991. Situated Learning: Legitimate Peripheral Par-

ticipation. Cambridge University Press.

Lawrence, P. R., Lorsch, J. W., 1967. Organization and Environment:

Managing Differentiation and Integration. Division of Research Gradu-

ate School of Business Administration Harvard University, Boston, MA.

March, J. G., 1991. Exploration and exploitation in organizational learning.

Organization Science 2 (1), 71–87.

Merton, R. K., 1957. Social Theory and Social Structure. The Free Press,

Glencoe, IL.

45

Merton, R. K., Barber, E. G., 2004. The Travels and Adventures of Serendip-

ity: A Study in Sociological Semantics and the Sociology of Science.

Princeton University Press.

Nagaoka, S., Igami, M., Eto, M., Ijichi, T., 2010. Knowledge creation process

in science: Basic findings from a large-scale survey of researchers in Japan.

IIR Working Paper No.10-08, Hitotsubashi University, Kunitachi, Japan.

Nagaoka, S., Igami, M., Walsh, J. P., Ijichi, T., 2011. Knowledge creation

process in science: Key comparative findings from the Hitotsubashi–

NISTEP–Georgia Tech scientists’ survey in Japan and the US. IIR Work-

ing Paper No.11-09, Hitotsubashi University, Kunitachi, Japan.

Nonaka, I., Takeuchi, H., 1995. The Knowledge-Creating Company: How

Japanese Companies Create the Dynamics of Innovation. Oxford Univer-

sity Press.

Pelled, L. H., Eisenhardt, K. M., Xin, K. R., 1999. Exploring the black box:

An analysis of work group diversity, conflict and performance. Adminis-

trative Science Quarterly 44 (1), 1–28.

Pelz, D. C., Andrews, F. M., 1966. Scientists in Organizations: Productive

Climates for Research and Development. Wiley, New York.

Perrow, C., 1967. A framework for the comparative analysis of organizations.

American Sociological Review 32 (2), 194–208.

Polanyi, M., 1962. The republic of science. Minerva 1 (1), 54–74.

Polanyi, M., 1967. The Tacit Dimension. Routldge & Kegan Paul, London.

46

Powell, W. W., Koput, K. W., Smith-Doerr, L., 1996. Interorganizational

collaboration and the locus of innovation: Networks of learning in biotech-

nology. Administrative Science Quarterly 41 (1), 116–145.

Presser, S., 1980. Collaboration and the quality of research. Social Studies

of Science 10 (1), 96–101.

Roberts, R. M., 1989. Serendipity: Accidental Discoveries in Science. Wiley,

New York.

Sauer, R. D., 1988. Estimates of the returns to quality and coauthorship in

economic academia. Journal of Political Economy 96 (4), 855–866.

Scott, W. R., 1981. Organizations: Rational, Natural and Open Systems.

Prentice-Hall, Englewood Cliffs, NJ.

Selznick, P., 1949. TVA and the Grass Roots. A Study in the Sociology of

Formal Organization. University of California Press.

Shapiro, G., 1986. A Skeleton in the Darkroom: Stories of Serendipity in

Science, Harper & Row, San Francisco.

Simon, H. A., 1976. Administrative Behavior: A Study of Decision-Making

Processes in Administrative Organization, 3rd Edition. The Free Press,

New York.

Singh, J., Fleming, L., 2010. Lone inventors as sources of breakthroughs:

Myth or reality? Management Science 58 (1), 41–56.

Stephan, P. E., 2010. The economics of science. In: Hall, B. H., Rosenberg,

N. (Eds.), Handbook of the Economics of Innovation. Vol. 1. Elsevier,

Amsterdam, pp. 217–273.

47

Stephan, P. E., Levin, S. G., 1992. Striking the Mother Lode in Science:

The Importance of Age, Place, and Time. Oxford University Press.

Svensson, G., Wood, G., 2005. The serendipity of leadership effectiveness in

management and business practices. Management Decision 43 (7), 1001–

1009.

Thompson, J., 2003. Organizations in Action: Social Science Bases of Ad-

ministrative Theory. Transaction, New Brunswick, N.J., London.

Udwadia, F. E., 1990. Creativity and innovation in organizations: Two

models and managerial implications. Technological Forecasting and So-

cial Change 38 (1), 65–80.

Van Andel, P., 1992. Serendipity “expect also the unexpected.” Creativity

and Innovation Management 1 (1), 20–32.

White, H., 1980. A heteroskedasticity-consistent covariance matrix estima-

tor and a direct test for heteroskedasticity. Econometrica 48 (4), 817–838.

Woodward, J., 1965. Industrial Organization: Theory and Practice. Oxford

University Press.

Wooldridge, J. M., 2010a. Econometric Analysis of Cross Section and Panel

Data, 2nd Edition. MIT Press, Cambridge.

Wooldridge, J. M., 2010b. Asymptotic properties of weighted M-estimators

for standard stratified samples. Econometric Theory 17 (2), 451–470.

Wuchty, S., Jones, B. F., Uzzi, B., 2007. The increasing dominance of teams

in production of knowledge. Science 316 (5827), 1036–1039.

48

Zuckerman, H., 1977. Scientific Elite: Nobel Laureates in the United States.

The Free Press, New York.

49