View
212
Download
0
Tags:
Embed Size (px)
Citation preview
Program Level Assessment of Outcomes and Impacts of Research of CentersJuan D. RogersSchool of Public PolicyGeorgia Tech
Project: Assessment of 15 Nanotechnology Science and Engineering Centers’ (NSECs): Outcomes and Impacts: Their Contribution to NNI Objectives and Goals, NSF 0955089.
AEA, Annaheim CA 2
ObjectivesPublication output and impact of the
NSECsCo-authorship patterns change over
timeGeographical spread or concentration of
NSEC research collaborationEvolution of areas of research as
reflected in keyword changesMultidisciplinary footprint of the centersThe case for centers
11/5/2011
AEA, Annaheim CA 3
Methodology OutlineQualitative-quantitative concurrent design
◦Qualitative component Multiple embedded case studies of mechanisms for
center outcomes Follow up field level impact with quantitative analysis
◦Quantitative component Bibliometrics (productivity, citation, co-authors, etc.) Personnel and funding data analysis Intellectual property instances (patents, licenses, etc.) Links with business and commercial applications
◦Methodologically: Final results are generalizations to theory from qualitative analysis We offer analysis of combined quant-qual data Interpretive schemes for quantitative findings on
centers
11/5/2011
AEA, Annaheim CA 4
Data and methods Acquisition of 85 center annual reports from all 15 NSEC
centers Extraction and clean up (duplicate removal) of publication
lists Extraction of NSEC articles from Web of Science (n=3,500) Look up and extraction of articles citing NSEC articles
(n=75,000) Clean up and classification of collaborating organizations Clean up, look up, identification, and matching of NSEC
authors in author listings Analyses of different type, multiple tools:
◦ Growth, shares, and overall trends (tabular)◦ Networks and collaborations (Gephi)◦ Geographical spread, GIS (ArcGIS)◦ Keywords and topics (VantagePoint)◦ Multidisciplinarity and science maps (Pajek)
11/5/2011
AEA, Annaheim CA 5
NSEC publication activity grows in three waves
11/5/2011
Notes: *Publication data not reported by all NSEC centers; last column reports average annual change for rows with change data.Source: ISI-WoS publication data based on NSEC annual reports by center.
2001
2002
2003
2004
2005
2006
2007
2008
2009
-10*
-
100
200
300
400
500
600
700
800
-
5,000
10,000
15,000
20,000
25,000
NSEC publications (all centers)
Citing articles
NS
EC
Pu
blic
ati
on
s
Cit
ing
art
icle
s
2001 2002 2003 2004 2005 2006 2007 2008 2009-10* 2001-10*
NSEC publications (all centers) 66 133 221 262 499 515
715
737
361 3,870 ▪ Annual change 102% 66% 19% 90% 3% 39% 3% -51% 34% Citing articles 48 391 1,164 2,619 4,595 7,415 10,469 15,243 19,149 94,484 ▪ Annual change 715% 198% 125% 75% 61% 41% 46% 26% 12%
Publications
2001-04 2005-062007-08
AEA, Annaheim CA 6
One out of 10 NSEC publications has an industry co-author
11/5/2011
Total unique firms co-authoring articles with NSEC (2001-2010): 146Total unique firms maintaining other types of collaborations with NSEC (as of 2010): 275**
Notes: * Publication data not reported by all NSEC centers; last column reports average annual change for rows with change data. **The type of collaborations are not specified by centers (only number of industry partners was provided).Source: ISI-WoS publication data based on NSEC annual reports by center and lists of industry partners provided by NSEC centers.
2001 2002 2003 2004 2005 2006 2007 20082009-
10*2001-
10*
NSEC centers with publications 3 6 6 13 13 15 15 15 13 15
NSEC publications (all centers) 66 133 221 262 499 515 715 737 361 3,509
NSEC pubs. co-auth. with industry 12 13 16 17 35 52 76 65 34 360
▪ Annual change 8% 23% 6% 106% 49% 46% -14% -48% 22%
▪ Share industry co-auth / all pubs. 18% 10% 7% 6% 7% 10% 11% 9% 9% 10%
Unique co-author firms 11 13 9 16 31 29 50 43 22 146
▪ Annual change 18% -31% 78% 94% -6% 72% -14% -49% 20%
Industry collaborations
AEA, Annaheim CA 7
Comparative Impact of NSEC Papers (Citations Cohort 2001)2001 2002 2003 2004 2005 2006 2007 2008 2009
C_Mean
0.88 6.30 14.44 23.64 35.88 48.48 61.11 75.61 90.17C_Med
0 2 4 6 9.5 11 12.5 14.5 17.5C_Max
12 52 148 297 528 776 1062 1409 1760N_Mean
0.26 2.11 5.13 8.18 11.62 14.78 17.87 21.11 24.04N_Medi
0 1 2 4 5 6 8 9 10N_Max
39 163 376 747 1268 1803 2286 2902 3484
11/5/2011
AEA, Annaheim CA 8
Comparative Impact of NSEC Papers (Citations Cohort 2002)2002 2003 2004 2005 2006 2007 2008 2009
C_Mean
1.20 9.06 21.32 36.92 51.75 67.13 85.15 101.80C_Med
0 3 8.5 16 20.5 25.5 30.5 36.5C_Max
19 88 196 400 585 807 1063 1330N_Mean
0.28 2.32 5.30 8.83 12.10 15.36 18.79 21.92N_Med
0 1 2 4 6 7 8 9N_Max
50 153 340 661 1053 1499 2042 2587
11/5/2011
NSEC papers appear to have higher impact measured by citations: median and mean citations grow faster than the cohort with window length
AEA, Annaheim CA 9
Rank in Cohort of Top 20 NSEC Papers
1 2 3 4 5 6 7 8 9 10
2001 5 29 98 167 179 223 313 411 465 6292002 6 10 17 20 23 56 69 124 127 148
11/5/2011
11 12 13 14 15 16 17 18 19 20
2001 834 1345 1383 2077 2992 3202 3203 3283 3284 37552002 175 192 228 279 322 382 430 513 559 566
NSEC papers rank highly in their cohort of Nano papers. Total Cohort 2001: 30462 papers. NSEC Cohort 2001: 66 papersTotal Cohort 2002: 34971 papers. NSEC Cohort 2002: 128 papers
AEA, Annaheim CA 10
Comparative Impact of NSEC Papers (Citations Cohort 2003)
2003 2004 2005 2006 2007 2008 2009
N_mean 0.30 2.28 5.76 9.17 12.62 16.32 19.77
N_median 0 1 3 4 6 7 9
N_max 46 251 674 1144 1676 2299 2913
C_mean 1.09 7.75 19.29 31.08 43.12 58.35 73.32
C_median 0 4 10 15 21 26 30
C_max 13 80 209 377 573 855 1204
11/5/2011
The distributions parameters show NSEC high position in the field but the toppaper is still not by the centers. N = 40813 CN = 222
1 2 3 4 5 6 7 8 9 10
3 14 15 34 36 39 47 65 83 98
11 12 13 14 15 16 17 18 19 20
119 126 129 132 137 159 198 211 238 245
AEA, Annaheim CA 11
Comparative Impact of NSEC Papers (Citations Cohort 2004)
2004 2005 2006 2007 2008 2009
Xmean 0.28 2.56 5.78 9.21 12.96 16.50
Xmedian 0 1 3 4 6 8
Xmax 46 299 650 1076 1683 2334
NXmean 0.84 7.32 16.14 25.74 37.31 47.79
NXmedian 0 4 10 15 20.5 28
NXmax 19 80 154 225 306 392
11/5/2011
1 2 3 4 5 6 7 8 9 10
42 52 55 71 73 79 100 105 123 147
11 12 13 14 15 16 17 18 19 20
248 261 300 314 472 473 539 557 588 614
N = 48952; CN = 259
AEA, Annaheim CA 12
Comparative Impact of NSEC Papers (Citations Cohort 2005)
2005 2006 2007 2008 2009
N_mean 0.36 2.65 6.04 9.97 13.74
N_median 0 1 3 5 7
N_max 69 303 590 974 1553
C_mean 1.20 9.07 20.51 34.96 49.12
C_median 0 4 10 16 23
C_max 25 145 405 949 1553
11/5/2011
1 2 3 4 5 6 7 8 9 10
1 5 8 17 18 25 29 32 36 38
11 12 13 14 15 16 17 18 19 20
42 50 81 86 100 102 105 129 130 149
N = 55998; CN = 499
AEA, Annaheim CA 13
Comparative Impact of NSEC Papers (Citations Cohort 2006)
2006 2007 2008 2009 2010
N_mean 0.39 2.67 6.37 10.11 10.94
N_median 0 1 3 5 6
N_max 39 176 396 671 756
C_mean 1.06 7.53 18.04 29.43 40.77
C_median 0 4 9 14 18
C_max 20 94 270 574 980
11/5/2011
1 2 3 4 5 6 7 8 9 10
1 3 5 6 21 25 30 48 51 70
11 12 13 14 15 16 17 18 19 20
84 89 95 99 100 103 132 146 148 190
N = 62351; CN = 512
AEA, Annaheim CA 14
Increasing JIF of Target Journals
11/5/2011
AEA, Annaheim CA 15
Increasing JIF of Target Journals
11/5/2011
AEA, Annaheim CA 16
Leveraging Support Reflected in Publication Support Acknowledgment
11/5/2011
AEA, Annaheim CA 17
NSEC co-authorship networks grow and become more widespread
11/5/2011
Co-authorships2001-2006
Notes: Nodes represent authors. Node size represents number of publications for the period. Node colors represent 15 NSEC centers. Line colors are those of the centers that maintain each co-authorship as found in publication databases.Source: ISI-WoS publication data based on NSEC annual reports by center.
Co-authorships
Co-authorships2007-2010
AEA, Annaheim CA 1811/5/2011
Central and wellconnected authors Highly collaborative
projects
Multiple productive andcollaborative authorsNSEC collaboration networks
present diverse patterns of co-authorship
(2001-2010)
Co-authorships
AEA, Annaheim CA 19
Wide geographic spread of NSEC research with concentration in specific locations/regions
11/5/2011
Note: Number of NSEC publications from 2001-2010 = 3509; number of citing publications = 75335. Citing publications, 2001-2010 exclude all NSEC publications.
Collaboration and places
AEA, Annaheim CA 20
Co-authoring extends beyond the NSECs to nearly all US states
11/5/2011
Collaboration and places
Clusters of top keywords(21 cluster solution reported)
More specializedterms
More linkedterms
Research topics
AEA, Annaheim CA 22
Top 20 Terms Across 15 NSECs(relative position of 30 most common, 2001-2006 v. 2007-2010)
11/5/2011
FOUNDATIONAL•FABRICS•THIN-FILM•SELF-ASSEMBLED MONOLAYER•SPECTROSCOPY•CARBON NANOTUBE
STABLE•NANOCRYSTAL•OPTICAL PROPERTY•MOLECULAR-DYNAMICS
GROWTH•FIELD-EFFECT TRANSISTOR•LITHOGRAPHY•GOLD NANOPARTICLE•QUANTUM DOT•SENSOR•WALL CARBON NANOTUBE•IN-VITRO•SEMICONDUCTOR•MOLECULAR-DYNAMICS SIMULATION•BIOSENSOR•IN-VIVO•THIN-FILM-TRANSISTOR•GRAPHITE
MATURE•DNA•SCATTERERS•TRANSISTOR•DIP PEN NANOLITHOGRAPHY•SILVER NANOPARTICLE•AQUEOUS-SOLUTION•ATOMIC FORCE MICROSCOPY•WAVE-GUIDE•HETEROSTRUCTURE
Research topics
AEA, Annaheim CA 23
Neurosciences
Agriculture
Chemistry
Physics
Engineering
Environ Sci
Matls SciClinical Med
Computer Sci.
Geosciences
Ecology
General Med
Map of Science: Subject Categories within all Nano (2009)
Source: Rafols, I. and Meyer, M. (2009) Diversity and Network Coherence as indicators of interdisciplinarity: case studies in bionanoscience. /Scientometrics/, 81(2), in print; Leydesdorff, L. and Rafols, I. (2009) A Global Map of Science Based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348-362.
11/5/2011
AEA, Annaheim CA 24
Neurosciences
Agriculture
Biomed Sci
Chemistry
Physics
Engineering
Environ Sci
Matls Sci
Infectious Diseases
Clinical Med
Computer Sci.
Geosciences
Ecology
General Med
NSEC research is multidisciplinary with focus areas in materials science, chemistry and biomedical sciences
Map source: Rafols, I., Meyer, M. (2009) Diversity and Network Coherence as indicators of interdisciplinarity: case studies in bionanoscience. Scientometrics, 81(2), in print; Leydesdorff, L., Rafols, I. (2009) A Global Map of Science Based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348-362.
Multidisciplinarity
Global Map of Science, 2006175 SCI Subject Categories
11/5/2011
AEA, Annaheim CA 25
Neurosciences
Agriculture
Biomed Sci
Chemistry
Physics
Engineering
Environ Sci
Matls Sci
Infectious Diseases
Clinical Med
Computer Sci.
Geosciences
Ecology
General Med
A range of disciplines beyond focal areas cite NSEC works
Multidisciplinarity
Global Map of Science, 2006175 SCI Subject Categories
11/5/2011
AEA, Annaheim CA 26
Implications for Center Policy
NSEC publication growth rate indicates rapid take-off by new centers.
NSEC mechanism allows for greater involvement of authors over time in diverse locations.
NSEC research involves some foundational as well as some emerging (and some maturing) topical areas.
NSECs work in multiple disciplines and their citation influence involves many more disciplines (especially in Biotechnology related fields).
11/5/2011
Conclusions
AEA, Annaheim CA 27
The Case for CentersReporting by PIs on crucial role of centers:
◦ Unique incentive to go deeply across disciplinary boundaries Significantly reduced cross-disciplinary transaction costs
◦ Unique research experience for graduate students Rich and diverse research infrastructure enables more risk taking Exposure to greater number of high quality scientific contacts Exposure to unique industry contacts
◦ Accelerator of promising young researcher development Center as recruiting tool of top talent Diverse mentoring opportunities for rapid career development
◦ Unique infrastructure possibilities Leverage of resources for shared new facilities Design of new unique instruments and experimental
arrangements
11/5/2011
AEA, Annaheim CA 28
Some Program ChallengesSome scientific contributions are difficult to
explain to the lay public ◦ Important for long term support of the enterprise
The special benefits for development of faculty and graduate students should be scaled up to reach the rest of the university community
The pre- and extra- university education efforts are dependent on the center program and are difficult to institutionalize without it. ◦ Their sustainability should be a program concern
Similar sustainability issue is raised by specialized infrastructure that may go to waste if centers are discontinued
11/5/2011