Joerg Bjarne SchlaepferDoctoral StudentKOF Swiss Economic InstituteETH Zuerich
StockholmNordic Workshop on Bibliometrics and Research Policy29. October 2013
How Nobel Laureates Would Perform in the Handelsblatt Ranking
Publication counts of Nobel Laureates in economics
2How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Rank Name Award Counts1 Paul A. Samuelson 1970 106.992 Joseph E. Stiglitz 2001 71.563 Eugene Fama 2013 58.424 Vernon Smith 2002 49.805 Amartya Sen 1998 48.556 James Tobin 1981 48.277 Thomas J. Sargent 2011 47.948 George J. Stigler 1982 47.609 Kenneth J. Arrow 1972 46.7110 Robert M. Solow 1987 46.39
29 October 2013
29 October 2013 3How Nobel Laureates Would Perform in the Handelsblatt Ranking
Doctoral student in economics
At KOF Swiss Economic Institute at ETH Zuerich
Manager of the bibliometric database Forschungsmonitoring
Calculate the bibliometric Handelsblattrankings
Introduction
Introduction Handelsblattranking Nobel Laureates Discussion
Agenda Introduction
Forschungsmonitoring / Handelsblattranking
Nobel Laureates
Discussion
29 October 2013 4How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Bibliometric webportal forschungsmonitoring.org
5How Nobel Laureates Would Perform in the Handelsblatt Ranking
Forschunsmonitoring.org has been initiated by the Verein für Socialpolitik
Records 6‘0000 (business) economists in Germany, Austria and Switzerland and their 85‘000 publications in peer-reviewed journals
Relational database
Systematic update once a year
29 October 2013
Introduction Handelsblattranking Nobel Laureates Discussion
Login for researchers to validate data
29 October 2013 6How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Handelsblattranking
29 October 2013 7How Nobel Laureates Would Perform in the Handelsblatt Ranking
Almost yearly publication since the year 2005
Economists or business economists
In Germany, Austria and Switzerland
Very well-known and heavily debated
Introduction Handelsblattranking Nobel Laureates Discussion
Methodology of the Handelsblattranking
29 October 2013 8How Nobel Laureates Would Perform in the Handelsblatt Ranking
Research articles in peer-reviewed Journals 1524 Journals from the EconLit Index supplemented by Statistics journals Journal weight
Handelsblatt count depends on Journal weight Number of authors Length of the text is ignored Kind of article: research articles, comments, notes, replys, editorials
Handelsblatt count is the journal weight divided by authors 1 count can be thought of as equivalent to one single-authored research article published
in the American Economic Review (AER)
Sum up the counts of the individual article• Over economist: Individual ranking • Over economists at University: Department ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Pivotal role how the journals are weighted The Handelsblatt-journal weighting list for economics is based on a study by
Pierre-Philippe Combes und Laurent Linnemer (2010): Inferring Missing Citations - A Quantitative Multi-Criteria Ranking of all Journals in Economics
18 September 2012 9Das Handelsblattranking und seine Zeitschriftenliste
Category Weight NumberA+ 1 10A 0.6 26
B+ 0.3 46B 0.2 76
C+ 0.15 112C 0.1 165D 0.05 1089
Introduction Handelsblattranking Nobel Laureates Discussion
Handelsblatt Journal list
A+ JournalsAmerican Economic ReviewEconometricaJournal of FinanceJournal of Financial EconomicsJournal of Monetary EconomicsJournal of Political EconomyNatureQuarterly Journal of EconomicsReview of Economic StudiesScience
29 October 2013 10How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Handelsblattranking 2013 for economists
29 October 2013 11How Nobel Laureates Would Perform in the Handelsblatt Ranking
Rank Name University Age Counts A+ A+, A # Publ.1 Bruno S. Frey Zeppelin University 72 29 4.08 10.4 290
2 Martin Hellwig Max-Planck Institute 64 23.32 7 19.15 73
3 Roman Inderst Frankfurt 43 21.6 6 16.8 72
4 Oded Stark Klagenfurt - 21.24 1.75 6.75 125
5 Ernst Fehr Zürich 57 20.26 11.65 17.2 96
6 Kai A. Konrad Max-Planck Institute 52 18.73 0.75 9.4 117
7 Helmut Lütkepohl Berlin FU 61 17.96 0.33 10.43 95
8 Hans-Werner Sinn ifo Institute 65 16.43 2 7.7 117
9 Gerard van den Berg Mannheim 51 15.94 2.67 13.17 72
10 Enno Mammen Mannheim 58 15.88 1 12.17 84
Introduction Handelsblattranking Nobel Laureates Discussion
Handelsblattranking 2013 for economic departmentsRank 13 Rank 11 University Counts A+ A+, A # Prof. Counts Prof.
1 1 Mannheim 81 17.5 52.4 32 2.22 4 Bonn 78 19.7 46.3 26 2.5
3 3 Munich 73 4.6 33.9 27 2.5
4 2 Zürich University 71 25.1 49.4 17 3
5 5 Zürich ETH 61 0.8 16.2 10 3.5
6 8 Cologne 53 4.7 24.2 20 2.27 6 Frankfurt 49 12 27.3 18 2.4
8 - Lausanne 42 10.9 26.9 14 2.28 7 Wien 42 2.3 20.9 14 2.3
10 16 Berlin HU 38 4.5 19.7 13 2.8
29 October 2013 12How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Controversy
29 October 2013 13How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Bibliometrics Handelsblattranking Nobel Laureates Discussion
Rankings are debated heavily and controversly
Convexity of the journal weight plays an important role
Neglects within journal heterogeneity
Provides compact information about research output
Does the responsability lye with the user of the information?
Shift of incentives Holmstrom & Milgrom (1991): If you incentivize tasks that are easy to monitor, people will
put more effort into these tasks.
How Nobel Laureates Would Perform in the Handelsblatt Ranking Matthias Krapf (University of Zuerich) Jörg Schläpfer
Regional and Sectoral Economic Studies Volume 12-3
AbstractWe compile a ranking of the research output of all Nobel Laureates in economics using the Handelsblatt methodology and compare the outcome to the Handelsblatt ranking of economists in the German-speaking area. Our analysis focuses on whether the overall rating scores of the Nobel Laureates are indicative of their high achievements. We discuss the role of the convexity of the employed journal weights and draw conclusions on the Handelsblatt’s methodology.
29 October 2013 14How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Previous studies on Nobel Laureates Rablen and Oswald (2008) estimate that receiving a Nobel Prize raises life
expectancy by between one and two years.
Jones (2010) and Jones and Weinberg (2012) find that innovations that resulted in Nobel Prizes are made increasingly later in life.
Weinberg and Galenson (2005) distinguish between two types of innovators and show that experimental scholars do their most important work later in life than conceptual laureates.
19 October 2012 15How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Publication table of Nobel Laureates in economics
29 October 2013 16How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Rank Name Award Lifetime Pre Award
1 Paul A. Samuelson 1970 106.99 66.752 Joseph E. Stiglitz 2001 71.56 71.563 Eugene Fama 2013 58.42 58.424 Vernon Smith 2002 49.80 45.815 Amartya Sen 1998 48.55 43.936 James Tobin 1981 48.27 37.137 Thomas J. Sargent 2011 47.94 47.948 George J. Stigler 1982 47.60 44.909 Kenneth J. Arrow 1972 46.71 26.1010 Robert M. Solow 1987 46.39 36.43
Observations Skewed distribution also among Nobel Lareates
Paul Samuelson – an economist with a superior reputation - leads the table by an impressive lead.
15 of the 74 Nobel Laureates in economics earned lower Handelsblatt counts than the top 20 researchers in the German speaking area.
27 Nobel Laureates have earned higher counts than the top researcher in the German-speaking area, who has not won a Nobel Prize. Hamermesh and Pfann (2012): conditional on its impact, the quantity of output has no or
even a negative effect on being awarded honors and prizes in economics.
Our table is not to be understood as a ranking of Nobel Prize winners’ scientific quality. Rather, our goal is to provide examples of different kinds of publication behaviors.
29 October 2013 17How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Publications in A+ and A journals Two very convex weighting schemes: restricted on A+ and A
Reflecting the tradeoff between publication quantity and publication quality
The correlation between counts from A+ and A journals and total counts is almost perfect with a Spearman coefficient of rank correlation of 0.98
In contrast for the Top of the German speaking area this rank correlation is only 0.63
29 October 2013 18How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Conclusion For the very top researchers measuring publications in top journals is a good
signal for overall output, but this restriction only applies to the very top researchers, i.e. Nobel Laureates.
Using Nobel Laureates as a benchmark is of interest because, as Hirsch (2005) notes, “[f]or the few scientists who earn a Nobel prize, the impact and relevance of their research is unquestionable.” However, any such comparison necessarily involves comparing apples and oranges.
The Handelsblatt ranking is not a perfect measure of past research output.
Standard research ratings that give sufficient weight to quantity are a major source of motivation for aspiring researchers. And it is the level of activity of all economists, which determines how far those can reach who are at the top of the pyramid of the economics community.
29 October 2013 19How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion
Questions / Discussion
29 October 2013 20How Nobel Laureates Would Perform in the Handelsblatt Ranking
Introduction Handelsblattranking Nobel Laureates Discussion