28
The B.E. Journal of Economic Analysis & Policy Advances Volume 9, Issue 1 2009 Article 39 Where Should we Submit our Manuscript? An Analysis of Journal Submission Strategies Martin Heintzelman * Diego Nocetti * Clarkson University, [email protected] Clarkson University, [email protected] Recommended Citation Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript? An Analysis of Journal Submission Strategies,” The B.E. Journal of Economic Analysis & Policy: Vol. 9: Iss. 1 (Advances), Article 39. Available at: http://www.bepress.com/bejeap/vol9/iss1/art39 Copyright c 2009 The Berkeley Electronic Press. All rights reserved.

Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Embed Size (px)

Citation preview

Page 1: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

The B.E. Journal of EconomicAnalysis & Policy

AdvancesVolume 9, Issue 1 2009 Article 39

Where Should we Submit our Manuscript? AnAnalysis of Journal Submission Strategies

Martin Heintzelman∗ Diego Nocetti†

∗Clarkson University, [email protected]†Clarkson University, [email protected]

Recommended CitationMartin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript? AnAnalysis of Journal Submission Strategies,” The B.E. Journal of Economic Analysis & Policy:Vol. 9: Iss. 1 (Advances), Article 39.Available at: http://www.bepress.com/bejeap/vol9/iss1/art39

Copyright c©2009 The Berkeley Electronic Press. All rights reserved.

Page 2: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Where Should we Submit our Manuscript? AnAnalysis of Journal Submission Strategies∗

Martin Heintzelman and Diego Nocetti

Abstract

In this paper, we analyze the problem faced by impatient researchers attempting to balancethe considerations of journal quality, submission lags, and acceptance probabilities in choosingappropriate outlets for their work. We first study the case in which probabilities of submissionoutcomes are exogenous parameters and show that authors can find the optimal submission paththrough the use of journal ‘scores’ based only on the journals’ characteristics and the author’s de-gree of impatience. Then, we analyze a more realistic framework in which acceptance probabilityis determined by the quality of the manuscript, in which the reviewing process may be imperfect,and in which authors may not be certain of the manuscript’s quality. Throughout, we illustrateour analysis with data on actual economics journals. We also consider the problem of journalsfacing a large number of submissions, limited space, and limited resources to review papers and,in particular, we examine the relative effectiveness of using submission fees and reviewing lags toration article submissions.

KEYWORDS: economics journals, submission strategies

∗We are thankful to Jody Beauchamp, Edward Knotek, Bill Smith, two anonymous referees, andto seminar participants at Clarkson University for helpful comments. All errors are our own.

Page 3: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

“Start with a higher-quality outlet than your eventual target.... The profes-sional returns to choosing a better journal are higher. But a strategy of aiminghigh requires a thick skin; the acceptance rate at major economics journalsis around 10 percent. Thus, it pays to have a ‘submission tree’ in mind, asequence of alternative outlets for your work.” - Daniel S. Hamermesh (1992)

“Give each of your papers a shot or two at the top journals, such as theAER, JPE, or QJE. Even if you are not confident in the paper, it is worth atry for two reasons. First, as author, you are not in the best position to judgeits quality; some people are too fond of their own work, and some are too hardon it. Let the editors decide. Second, the editorial process is highly imperfect.The bad news is that some of your best articles may end up getting rejectedfrom the top journals. The good news is that you may get lucky, and some ofyour so-so articles may end up published in top journals simply because theyhit the editor’s desk when he is in a good mood.” - Gregory Mankiw1

1 Introduction

Researchers in all fields face the difficult problem of finding the appropriatejournal for their manuscript. In economics, this choice is particularly difficultgiven the large number of diverse journals. The advice provided by GregoryMankiw and Daniel Hamermesh is intuitive: develop a submission tree, aim-ing at the top journals in the first submission, and, if rejected, follow the pathestablished. Although simple in theory, the selection process could be of im-mense proportions. For instance, in a universe of only 10 journals there are3,628,800 possible paths. In economics, there are more than 300 journals toconsider. Understanding this problem is important both to researchers tryingto increase their productivity and to editors and others designing policies thataffect the efficiency of scholarly research production.

Given the importance of the publication process to academic economists,it is not surprising that an extensive and diverse literature has emerged inthis area in recent years (e.g. Blank (1991), Engers and Gans(1998), Elli-son (2002a,b), Azar (2004, 2005, 2007, 2008), Faria (2005 a, b), Leslie (2005)amongst others). In this paper, we bring together the two sides of the publi-cation problem. We first evaluate the problem faced by impatient researcherswho attempt to balance the considerations of journal quality, submission lags,and acceptance probabilities in maximizing their payoff. Then, we examine

1http://gregmankiw.blogspot.com/2007/02/advice-for-new-junior-faculty.html

1

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 4: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

how, given the behavior and incentives of authors, journals can respond toimprove the efficiency of the publication process.2

In section 2 we revisit the problem first analyzed by Oster (1980) and morerecently by Azar (2004, 2005, 2007) and Leslie (2005), that of the optimalsubmission order with exogenous probabilities of submission outcomes, andshow that it can be simplified greatly: instead of numerically considering allpossible paths, the author can create a single submission path through the useof journal ‘scores’ based only on the journals’ characteristics and the author’sdegree of impatience. In addition to finding a simpler solution to the basicproblem, we extend the model to consider uncertain reviewing times and thepossibility of ‘revise and resubmits.’ In today’s academic environment, theseextensions are critical given longer observed reviewing times (Ellison, 2002a),more arduous demands in the revision process (Ellison, 2002b), and anecdotalevidence of highly uncertain delays.

In section 3 we extend our framework to account for manuscripts of differ-ing quality, authors’ uncertainty about the actual quality of the manuscript,and an imperfect reviewing process (see e.g. Gans and Shepherd (1994), Os-wald (2007) and Mankiw’s quotation, above).3 Although this more realisticframework introduces a level of complexity which does not permit, in general,finding simple ranking rules that are optimal, we are able to study a few im-portant special cases in which the problem becomes manageable. First, whenauthors know the quality of their papers, but referee reports are imperfect,authors can still create a single submission path through the use of journal‘scores’. Second, when authors are uncertain of the quality of their work, butreferees perfectly observe and reveal the quality (with a referee report), theproblem boils down to a two period problem in which only the initial submis-sion has to be considered. For the more general case with unknown qualityand an imperfect reviewing process, we consider a simple rule and show thatit performs well when compared with the optimal submission policy (whichrequires evaluating all the possible paths).

2In focusing on the submission/publication problem, we are ignoring issues related tothe production of manuscripts. That is, academics must balance their time between teach-ing, research, and service, and must choose also how much effort to put into particularmanuscripts before deciding to submit them. It is likely that questions of journal policiesregarding lags and fees will also affect these decisions by researchers, but that is beyond thescope of this work.

3Gans and Shepherd (1994) provide a number of anecdotes from award-winningeconomists about the rate at which their articles, including some of their best, have beenrejected by journals. Oswald (2007) uses a more analytical approach and likewise shows thatthe best (most-cited) articles in middle-tier journals are often ‘better’ than the least-citedpapers in top-tier journals.

2

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 5: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

We show that, in general, the advice provided by Mankiw and Hamermeshis correct. Sufficiently patient authors should submit to the top-tier journalsinitially. Intuitively, they can always submit their manuscripts later to journalswith lower rewards and higher acceptance rates. Given the long reviewingtimes in most journals, however, this advice may not be well suited for young,untenured, professors who are more likely to be impatient and risk averse.These authors should instead consider submitting to lower tier journals first.We also show that it may sometimes be optimal to submit to the top journalsregardless of the quality of the paper if reviewing errors are likely, and evenwhen one believes that the probability of acceptance in a top-tier journalis zero if top-tier journals provide relatively fast and accurate referee reports.More likely, however, authors of papers that are not of the highest quality, andespecially those without an established reputation, will lean towards lower tieroutlets. This is specially relevant given the finding of Blank (1991) that thesingle-blind reviewing process may lead to favoritism.4 In addition to showingour results analytically, we also illustrate them with simulations developedwith data on the characteristics of a set of economics journals.

Section 5 evaluates the policy implications of the models, with an emphasison the relative merits of reviewing lags and submission fees as rationing de-vices. We show that, while both submission fees and reviewing lags can reducethe number of low quality submissions that top journals receive, submissionfees are a more effective rationing device. By reducing reviewing lag times andincreasing submission fees a journal can ration the number of submissions, in-crease the quality of the papers received, and increase the expected utility ofauthors that still submit their manuscripts there. Therefore, while a length-ening of reviewing lags given low submission fees may be socially beneficial assuggested by Azar (2008), our analysis supports Leslie’s (2005) argument thatlow lags and high fees may be more efficient. Section 5 concludes.

2 Optimal Submission Rules in a Simple Frame-

work

The basic problem of submitting manuscripts to journals is about maximizingone’s expected payoff given an uncertain review with very low acceptance ratesin most economics journals. As Oster (1980) showed, it is a process that, given

4Laband and Piette (1994) and Medoff (2003), however, reject the idea of favoritismand show that editors are, in general, better connected to top authors and so are simplypublishing those authors’ very good papers.

3

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 6: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

known reputation scores and other journal statistics can be relatively easilyquantified.

In this section we consider an extended version of the problem and showthat it can be solved more generally, without the need to calculate payoffsfor each of a huge number of possible orders. Specifically, there are m targetjournals for a manuscript. Each journal has a submission fee Fj and a lag tjuntil a first response. The lag may be uncertain to the author. We assumethat journal j can have either a short lag t−j with probability θ or a long lag t+jwith complementary probability.5 The probability of acceptance in this stageis P f

j . With probability PjR the journal may require a revision of the paper,after which it accepts the paper with probability PR

j .6 The author receivesthe decision on revisions after a lag tjR and there is a cost FjR for revising thepaper (e.g. in terms of effort exerted). We assume that, given the option torevise and resubmit, the author exercises it.7

Once a manuscript is submitted to a journal and rejected the author cannotsubmit again to the same journal. Acceptance to journal j gives a rewardRj. This reward should be interpreted as the discounted utility of the streamof benefits, in prestige and/or pecuniary. The author can obtain a fallbackreward, R0, from journal 0 which accepts the paper for sure and immediately.8

The dynamic programming problem is:

J(s) = Max[R0,Maxj∈ s[−F+j + δfj (P accept

j Rj + P rejectj J(s− j))]], (1)

where s is the subset of journals not yet submitted, F+j = Fj + δfj PjRFjR

is the submission fee plus the discounted expected cost of resubmission, δfj =

θe−βt−j +(1−θ)e−βt

+j is the expected discount factor for the first response (β ≥ 0

is the individual’s discount rate), P acceptj = P f

j + δRj PRj PjR is the “discounted”

probability of acceptance with δRj = e−βtjR , and P rejectj = (1 − P f

j − PjR) +

5Our results could be easily generalized to arbitrary distributions with compact support.6We assume, trivially, that all journals, other than the ‘fallback’ journal described below,

have a positive probability of rejecting any given paper.7This will be the optimal policy as long as the costs for revising the paper are not too

large and or the probability of acceptance of a revised paper is sufficiently high. Intuitively,when the author receives a request for a revision she will exercise it if the score of the journal,defined below, given the updated probability of acceptance, remains the highest. Explicitlyconsidering this option is not difficult, but doing so does not alter the main implications ofthe model and complicates the notation considerably.

8This fallback reward is simply a placeholder in the dynamic programming problem, andits value has no impact on the results.

4

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 7: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

δRj (1− PRj )PjR is the discounted probability of rejection.

This problem can be simplified greatly by drawing on general search theory(e.g. Weitzman 1979). Specifically, define the score of a given journal as

vj =−F+

j + δfj Pacceptj Rj

1− δfj Prejectj

. (2)

Therefore,

Proposition 1 The optimal policy is to submit first to the journal with thehighest score, then (if rejected) to the second highest, and so on. If there is nojournal with a score higher than R0 the author should submit the manuscriptto the journal 0

Proof. The expected value of submitting first to journal j and then to journali is

−F+j +δfj P

acceptj Rj+P

rejectj δfj (−F+

i +δfi Paccepti Ri)+δfj δ

fi P

rejectj P reject

i J(s−j−i)(3)

Consider, alternatively, a policy in which journals j and i are in the reverseorder and all the other journals are in the same order as before. The expectedvalue of this policy is

−F+i +δfi P

accepti Ri+P

rejecti δfi (−F+

j +δfj Pacceptj Rj)+δfi δ

fj P

rejectj P reject

i J(s−i−j)(4)

Note that, after submission to journals i and j, both policies give the samecontinuation value, i.e., P reject

j P rejecti J(s − j − i) = P reject

j P rejecti J(s − i − j).

The first policy will be better if its expected value is larger, a condition whichcan be written as,

−F+j + δfj P

acceptj Rj

1− δfj Prejectj

>−F+

i + δfi Paccepti Ri

1− δfi Prejecti

(5)

Since this must be true for all orderings, the optimal path must be the one inwhich the journals are submitted in order of vj. �

We have thus transformed a problem of comparing all possible values of thepermutations of the journals not yet submitted into a simple problem of rank-ing each journal according to their characteristics and the rate of impatienceof the author. The score of a journal increases if reviewing times (either at thefirst stage or the second stage) become shorter, of if the probability of a shorterlag increases. In addition, a mean preserving spread in the distribution of the

5

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 8: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

lag increases the score of the journal.9 Also, journals with higher rewards,higher probabilities of acceptance (in the first or second round), lower submis-sion fees, and lower costs of revising the paper (less extensive revisions), havea higher score. Note, however, that the optimal policy is very different fromthe policy of submitting to the journal with the highest expected discountedreward. For instance, consider two journals with the same distribution of re-viewing times and the same fees; then, for the same expected reward (utility),the author should submit to the journal with the highest probability of re-jection. Intuitively, given the same expected rewards, the journal with lowerprobability of acceptance also has higher rewards. Given that the author cansubmit the manuscript to the lower reward journal if it is rejected, it is alwayspreferable to submit first to the ’better’ journal.

The effect of the probability of receiving a revise and resubmit on the scoreis ambiguous. To see why this is so, suppose first that revising a paper has nocosts and the second round decision is immediate. In this case, an increase inthe probability of a revise and resubmit will increase the score if the probabilityof acceptance after the revision is higher than the probability of an outrightacceptance in the first round, which is likely to be true in practice. However,higher costs of revising the paper or longer second round lags may offset thebenefits of a higher probability of acceptance by increasing the expected totallag time or expected total costs. In this case, a high probability of a reviseand resubmit could lower the score.

As expected, the score decreases with the authors’ degree of impatience.More importantly, less patient authors will clearly prefer journals with shorterlags, ceteris paribus. If we think of untenured faculty and faculty close toretirement as individuals who discount heavily the rewards received in thefuture, they will be more inclined to submit to journals with faster reviewingtimes than young tenured faculty. In addition, if reviewing times are positivelycorrelated with the journals’ rewards (e.g. Ellison, 2002a), untenured facultyand faculty close to retirement may give higher scores to lower-tier/shorter-lagjournals (see e.g., Oster and Hamermesh 1998).

Implicitly, we have so far assumed risk-neutrality with respect to payoffs -the utility from a given journal reward is simply the reward itself. We couldjust as easily assume that authors are risk averse by making utility a concavefunction of the reward. This serves to reduce the utility payoff (and score) gap

9Note that the discount factor is convex in the reviewing lag (discounted expected utilityimplies risk loving behavior with respect to the timing of events). Therefore a mean pre-serving spread in the distribution of the lag increases the discount factor. In other words,if the distribution of journal A’s time lags is second order stochastically dominated by thedistribution of journal B’s time lags, journal A’s discount factor will be higher.

6

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 9: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

between different journals.10 The practical effect then is that more risk averseauthors will see less of a gain to submitting to high reward/low probabilityjournals and will be more likely to start their submission trees at a lower level.Therefore, risk aversion will tend to strengthen the effect of impatience foruntenured faculty.

2.1 Example

To provide a simple example, we obtained data, presented in Table 1, on thecharacteristics of twenty general interest and top field journals. The meansubmit-accept time, which we use as a proxy for the actual lag (i.e. thereis no uncertainty and no revisions), comes from Ellison (2002a).11 The perperiod reward is taken to be equal to two different standardized citation scoresprovided by Kalaitzidakis et al. (2003): first, the scores calculated from theJournal Citation Reports(JCR) and then those scores adjusted by impact, age,and self-citation.12 Finally, we use the acceptance rate provided by the journal,whenever available, or the (mid-point)acceptance rate in the Cabells Directoryof Publishing Opportunities in Economics and Finance (2004). Although werecognize the faults of these variables, we believe that they nevertheless providegood proxies for the variables that authors use in their submission decisions.Note also that this example is a simplified version of the model describedabove. Because it is not straightforward to convert dollar submission fees anda normalized journal ‘reward’ to a common metric we exclude submission feeseven though such data is available. There is, unfortunately, not enough datato include uncertain lags in this analysis either.

10This is true since the score is linear in the reward and utility is a concave function ofthe payoff. Note that an uncertain reviewing time does not interact with risk-aversion overthe payoff because discounted utility is linear in the discount factor.

11Data are for 1999 and are adapted from Table 1, page 953 of that paper.12According to Kalaitzidakis et. al. (2003), the JCR score for each journal is based

on the number of citations received by those journals in articles published in 1998, withthe top-ranking journal, American Economic Review, normalized to 100. These scores aresubsequently adjusted to account for the impact of citations (citations in better journalscount more), the age of the journal (older journals will have accumulated more citations)and self-citations are excluded.

7

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 10: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Table 1: Journal CharacteristicsJournal Mean Lag1 Acceptance Rate2 JCR Reward3 Adjusted Reward4

General InterestAmerican Econ. Review 21.1 7.5 100 100

Econometrica 26.3 7.7 88.27 71.59J. of Political Economy 20.3 8.0 74.4 75.86

Quarterly J. of Economics 13 8.0 46 69.83Review of Econ. and Stat. 18.8 15.5 26 24.44

Economic Journal 18.2 8.0 28.2 20.49Review of Econ. Studies 28.8 8.0 26.8 34.76

Intl. Econ. Review 16.8 15.5 12.4 18.49Economic Inquiry 13 29.0 7.6 6.92

Canadian J. of Econ. 16.6 25.0 6.2 4.47

Field JournalsJ. of Financial Econ. 14.8 11.6 29.8 12.62

J. of Econ. Theory 16.4 8.0 28 50.02J. of Monetary Econ. 16 8.0 20.7 34.14

J. Environm. Econ.Manag. 13.1 13.7 12.8 12.83J. of Law and Econ. 14.8 8.0 17.6 5.9

RAND Journal of Econ. 20.9 15.5 11.6 12.98J. of Development Econ. 17.3 20.0 7.9 7.14

J. of Applied Econometr. 21.5 15.5 4.6 9.74J. of Mathemat. Econ 8.5 25.5 4.6 4.57J. of Comparat. Econ. 10.1 15.5 2.8 5.48

1 Average time from submission to acceptance. Based on Ellison (2002a).2 Acceptance Rate for submitted papers. For American Economic Review, Econometrica, Economic Journal, Economic Inquiry,

Canadian Journal of Economics, Journal of Financial Economics, and Journal of Environmental Economics and Management, data

comes directly from the journal’s themselves or associated Editor’s Reports for the most recent year available. Otherwise,

data is from Cabell’s Directory of Publishing Opportunities in Economics and Finance, 2004. Cabell’s did not have acceptance

rate data for Journal of Political Economy so we assume the same acceptance rate as Quarterly Journal of Economics. Likewise,

we assume the acceptance rate for Review of Economics and Statistics to be the same as International Economic Review.3 Reward per period. Kalaitzidakis et al. (2003) JEEA. Standardized citation index based on journal of citation reports.4 Reward per period. Kalaitzidakis et al. (2003) JEEA. Standardized citation index, impact, age, and self-citations adjusted.

8

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 11: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Table 2: Optimal Submission Orders

JCR Optimal JCR Impatient, Adjusted Adjusted Impatient,Journal Order1 Risk Averse Order2 Optimal Order1 Risk Averse Order2

American Econ. Review 1 7 1 7Econometrica 2 9 4 9

J. of Political Economy 3 6 3 6Quarterly J. of Economics 4 2 2 2Review of Econ. and Stat. 5 3 5 4

Economic Journal 6 8 8 8Review of Econ. Studies 7 10 6 10

Intl. Econ. Review 8 5 7 3Economic Inquiry 9 1 9 1

Canadian J. of Econ. 10 4 10 5

Field Journals3

J. of Financial Econ. 5 2 9 5J. of Econ. Theory 6 6 5 6

J. of Monetary Econ. 7 7 5 6J. Environm. Econ.Manag. 8 2 9 3

J. of Law and Econ. 8 6 11 10RAND Journal of Econ. 9 6 9 6J. of Development Econ. 10 6 10 5

J. of Applied Econometr. 11 10 9 8J. of Comparat. Econ. 11 7 10 4J. of Mathemat. Econ 11 2 10 1

1 Assumes a discount rate of 5%.2 Assumes a discount rate of 50% and a log-utility function on the reward.3 The optimal order for the field journals is relative to the general interest journals.

9

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 12: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Table 2 presents the optimal submission order for four different scenarios.Sufficiently patient risk-neutral authors should target the top journals andonly go for the lower level general interest or field journals after being rejectedfrom those journals that provide higher rewards. This is true for both ad-justed and unadjusted rewards. As suggested above, when authors are highlyimpatient and risk-averse, since reviewing lags do seem to be correlated withcitations/rewards, the optimal order tends to shift towards initial submissionsto lower ranked (safer and faster) field and general interest journals. For ex-ample, for these authors, the journal Economic Inquiry, which has a relativelyshort reviewing lag and a relatively high acceptance rate, would be the optimalfirst choice.

3 Manuscripts with Uncertain Quality and an

Imperfect Reviewing Process

The main drawback of the preceding framework is the assumption that ac-ceptance in a given journal is independent of the quality of the paper. Yet,although the reviewing process may be imperfect, not every paper has a tenpercent chance of acceptance in AER or JPE. In this section we consider amore realistic situation in which manuscripts differ on their quality, the qual-ity of the manuscript may be uncertain to the author, and referees obtainnoisy representations of the true quality of the manuscripts, which they use toaccept, reject, or require revisions of the paper.

In particular, a manuscript has one out of m+1 possible qualities qi ∈(q0, q1, ..., qm). We denote journal j as one which accepts papers of (perceived)quality equal or greater than qj.

13 The author assigns subjective prior proba-bilities over the distribution of possible qualities P (qi). The reviewer may mis-judge the quality of the manuscript. Specifically, the referee receives a noisysignal q̃ of the paper’s quality, which is retransmitted to the author in the formof a referee report. If the true quality of the manuscript is qi, the probabilitythat a referee in journal j assigns a quality qz is denoted P z,i

j = Prob(q̃zj | qi).It is likely, and we assume, that the modal perceived quality of manuscriptsis correct (e.g.P i,i

j > P k,ij ∀k 6= i). As we argue later, the author’s reputation

13Incorporating multiple quality dimensions (e.g. importance of the ideas, fit, craftman-ship) is fairly straightforward. We would simply need to define an index of quality dimensionsand the threshold index for the different journals (e.g. the threshold could be defined interms of a minimum level for each dimension). Another consideration related to fit thatcould be easily included is a preference for domestic or home-country journals, as describedin Faria (2005b).

10

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 13: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

may affect these probabilities.As in the previous section, we also incorporate the possibility that a paper

may go through a round of revisions. In contrast with the previous section, theprobability of obtaining a revision is likely to depend on the paper’s quality.If the required revisions of a paper are just “cosmetic changes” of papers thatwill be accepted anyway, the uncertain timing of the first-round decision wouldbe a good way to capture the effect of revisions. However, we also considerthe possibility that revisions allow referees to better observe the quality of thepapers. In particular, we assume that journal j asks the author to revise andresubmit the paper if the perceived quality is at least j-w (and lower than j,in which case the paper is accepted without revision). If the author submitsthe revised paper, the actual quality is observed and the decision made. Asbefore, there is a second round reviewing time tjR and a cost FjR for revisingthe paper.

Finally, acceptance to journal j gives a reward Rj. This reward may dependon the distribution of signals (e.g. journals that publish manuscripts withhigher average quality - but not necessarily higher thresholds- have a higherreward), but is exogenous to the author. In addition, the reward may dependon the actual quality of the paper and the journal itself (Oswald 2007). If thisis the case one can redefine the expected rewards as R̃j ≡ (Rj|i)

∑m−jx=0 P

j+x,ij .

The reward in this case has two components: a journal specific reward and areward that is constant across journals. Although learning introduces a levelof complexity which does not permit finding simple ranking rules, there are afew special important cases in which this problem simplifies greatly.

3.1 Imperfect Referee Reports with Known Quality

When the author knows the quality of the manuscript, neither referee reportsnor rejection provide new information. Therefore, under the assumption thaterrors in referee reports are independent across journals, the solution to thisproblem is similar to the one obtained above. In particular,

Proposition 2 Suppose that the author knows that the quality of the manuscriptis i. For authors with i < j define the score of journal j as

v−j =−Fj + δfj ∆jRj

1− δfj (1−∆j). (6)

where ∆j = Prob(q̃ > qj) =∑m−j

x=0 Pj+x,ij is the probability of acceptance. For

11

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 14: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

authors with i ≥ j define the score of journal j as

v+j =

−F+j + δfj ∆+

j Rj

1− δfj (1−∆j−w). (7)

where F+j = Fj + δfj Prob(qj−w ≤ q̃ < qj)F

rj , with Prob(qj−w ≤ q̃ < qj) =∑−1

x=−w Pj+x,ij , is the fee plus the expected cost of resubmission, ∆+

j = Prob(q̃ >qj) + δRj Prob(qj−w ≤ q̃ < qj) is the “discounted probability” of acceptance, and(1−∆j−w) = Prob(q̃ < qj−w) is the probability of rejection.

The optimal policy is to submit first to the journal with the highest score,then (if rejected) to the second highest, and so on.

Authors with i < j would never resubmit their papers because second-round reviews are error-free. Therefore, for them, the probability of accep-tance equals the probability of ‘type II’ errors (the probability of accepting apaper below the threshold quality) in the initial submission. The larger thisprobability is, the higher the score of the journal.14 Similarly, moving theprobability mass of signals to the left, such that the probability of ‘type I’errors (reject when the quality is equal or higher than the threshold) is higher,decreases the score of any journal for authors of high quality papers.

Changing the revise and resubmit threshold while maintaining the journalthreshold unchanged only affects the score of high quality papers. As men-tioned in the previous section, an increase in the probability that a paper willgo to a second round (i.e. a decrease in the revise and resubmit threshold)has ambiguous effects, depending on the costs of revising the paper relativeto the increase in the probability of acceptance that such change implies. Forinstance, for authors with very high-quality papers (relative to the journalthreshold), and especially for those who are highly impatient, the probabilityof acceptance is unlikely to change much while the costs of revising the papermay still be substantial. Of course, all authors of high quality papers wouldbe better off with more accurate first round reviews and no revisions.15

Top journals are likely to have a relatively lower probability of type IIerrors, discouraging submissions by authors of low quality papers. However,

14Of course, the reward of a journal may depend on the quality of the papers publishedrather than on the announced quality threshold. In such cases, increasing the probabilityof type II errors may reduce the score of a journal.

15At least one journal (Economic Inquiry) has begun to offer a ‘no revisions’ policy. Inthis case, authors can opt-in to a system where they receive a simple yes or no answer totheir submission, but will never be asked to revise the paper as a condition of publication.Based on our analysis above, this will attract submissions of high-quality manuscripts, andmanuscripts from more impatient authors.

12

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 15: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

these journals are also likely to have a relatively higher probability of type Ierrors (especially those journals with a large number of desk rejections), dis-couraging authors of good quality papers. Again, this effect will be particularlyrelevant for impatient untenured faculty for whom lag times are especially im-portant.16

Errors may also be related to reputation. In particular, our results mayhelp explain the findings of Blank (1991) in terms of single versus doubleblind reviewing processes.17 She presented evidence that, when the reviewingprocess is double blind, acceptance rates are in general lower than when theprocess is single blind. She further showed that authors in top-tier universitiesare largely unaffected, but authors in near-top ranked universities are affectednegatively by the double-blind policy.

In terms of the setup in this section, consideration of an author’s reputationin the single-blind process can be regarded as a rightward shift of the proba-bility mass of the signals’ distribution (possibly an involuntary shift: “if JohnDoe wrote it, it must be correct and it must be important”) for higher reputa-tion authors. Our interpretation of Blank’s (1991) findings is that for authorsin top tier universities prestige is less important in the refereeing process be-cause faculty in these universities produce papers of very high quality that arelikely to be accepted in both types of refereeing systems. Instead, in near-topranked universities, prestige still plays a role but there is more variability inactual quality. Therefore, these authors are more likely to submit manuscriptsand be accepted in journals that have a single-blinded system. The oppositeholds for those without a strong reputation since the consideration of reputa-tion will presumably manifest itself for these authors as a leftward shift of theprobability distribution of signals.

As a simple numerical example of the optimal submission rules in thissetup, we group the journals for which we have data into 3 categories (top-tier, second-tier, third-tier) and use the average statistics for each group tocalculate the scores.18 The average characteristics of these categories of jour-

16Formally, ∂vj

∂∆j∂δj< 0.

17See also Laband and Piette (1994) and Medoff (2003) for evidence on favoritism inacademic publishing. Hodgson and Rothman (1999) explore the causes and implicationsof the intense concentration of authors and editors at the top journals at a few top-tierinstitutions.

18Top-tier journals are American Economic Review, Econometrica, Quarterly Journal ofEconomics, and Journal of Political Economy. Second-tier journals are: Economic Jour-nal, Journal of Economic Theory, Journal of Environmental Economics and Management,Journal of Financial Economics, Journal of Law and Economics, Journal of Monetary Eco-nomics, RAND journal of Economics, Review of Economic Studies, and Review of Eco-nomics and Statistics. Third-tier journals are: Canadian Journal of Economics, Economic

13

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 16: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Table 3: Average Characteristics of Journal Categories

Journal Category1 Mean Lag Acceptance Rate JCR RewardFirst-tier 20.175 7.8 79.32

Second-tier 16.625 11.0375 21.6775Third-tier 14.82857143 20.85714286 8.115714

nals are presented in table 3. We consider multiple scenarios under differentassumptions on the quality of the paper, the quality of referee reports, the rateof impatience of the authors, and the shape of the utility function. Further-more, we assume that top-tier journals only accept the highest quality papers,second-tier journals accept manuscripts of quality “1” and “2”, and so on.19

Results of the numerical simulation are presented in table 4. The first fourcolumns define the manuscript’s quality (known to the author), the probabilitythat the referee receives an accurate signal, the author’s level of impatience,and whether or not the author is risk neutral or risk averse, respectively. Thelast column presents the optimal submission order where ‘1’ represents top-tier journals, ‘2’ second-tier journals, and ‘3’ third-tier journals. There area number of interesting implications. First, sufficiently-patient risk-neutralauthors should follow a “quality path” (i.e. start at a top-tier journal, followedby a second-tier, and so on), regardless of the actual quality of the paper.Second, impatient and/or risk averse authors of lower quality manuscriptsshould in fact disregard the conventional wisdom and not submit initially totop-tier journals but instead submit first to second-tier or third-tier journals.Finally, when reviewing errors are more likely, authors of lower quality papersshould shift towards higher quality outlets (type II errors) but impatient risk-averse authors of high quality papers may prefer lower tier journals (type Ierrors).

Inquiry, International Economic Review, Journal of Applied Econometrics, Journal of Com-parative Economics, Journal of Development Economics, and Journal of Mathematical Eco-nomics.

19To avoid the unrealistic assumption that third-tier journals accept papers with certaintywe assume that there are 4 quality levels and that there is an additional tier of journals thatprovide very low (zero) rewards.

14

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 17: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Table 4: Case Comparison: Author knows quality, Referee Report Imperfect

Probability of OptimalManuscript Quality1 Accurate Signal2 Discount rate Utility Submission Order1

1 0.7 0.05 Linear 1231 0.7 0.5 Linear 1231 0.4 0.05 Linear 1231 0.4 0.5 Linear 1231 0.7 0.05 log 1231 0.7 0.5 log 1231 0.4 0.05 log 1231 0.4 0.5 log 2132 0.7 0.05 Linear 1232 0.7 0.5 Linear 2132 0.4 0.05 Linear 1232 0.4 0.5 Linear 1232 0.7 0.05 log 2132 0.7 0.5 log 2312 0.4 0.05 log 1232 0.4 0.5 log 2313 0.7 0.05 Linear 1233 0.7 0.5 Linear 1323 0.4 0.05 Linear 1233 0.4 0.5 Linear 1233 0.7 0.05 log 1233 0.7 0.5 log 3213 0.4 0.05 log 1233 0.4 0.5 log 3214 0.7 0.05 Linear 1234 0.7 0.5 Linear 1234 0.4 0.05 Linear 1234 0.4 0.5 Linear 1234 0.7 0.05 log 1234 0.7 0.5 log 3214 0.4 0.05 log 1234 0.4 0.5 log 231

1 Quality/Journal Designations - 1: First-tier, 2: Second-tier, 3: Third-tier.2 This is the probability that the referee’s signal equals the actual quality. Conditional

on being inaccurate, the signal is distributed uniformly over the other qualities.

15

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 18: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

3.2 Perfect Referee Reports with Uncertain Quality

Another special case occurs when the quality of the manuscript is uncertainto the author but referees can identify the true quality (i.e. the matrix oferror probabilities is an identity matrix) and they retransmit this quality tothe author. Leslie (2005) considered this problem under the assumption thatδj = 1 and that the cost of a longer reviewing lag was part of Fj.

20The followingproposition establishes the optimal strategy:

Proposition 3 Suppose that referee reports provide a perfect signal of themanuscript quality. Define the value of journal j as

δjRj

m∑i=j

P (qi)− Fj + δj(1−m∑i=j

P (qi))

j−1∑k=0

P (qk|q < qj)(δkRk − Fk) (8)

where the term∑j−1

k=0 P (qk|q < qj)(δkRk − Fk) is the continuation value given

that the quality is lower than j and P (qk|q < qj) = P (qk)(1−

∑mi=j P (qi))

is the proba-

bility of that the manuscript is of quality k given rejection from j. The authorshould submit the manuscript to the journal for which equation (8) is largest.If the paper is rejected, the author should submit the paper to the journal withthe highest discounted reward given the (by that time known) quality of thepaper.

As in the previous cases with the scores, when referee reports are veryinformative the optimal policy only requires comparing m possible values.Consider the difference between the expected values from submitting first tothe best journal (m) or to the second-best journal (m−1). Using the equationabove this difference is:

V (m,m− 1)− V (m− 1) + (δm − δm−1)m−2∑k=0

P (qk)(δkRk − Fk) (9)

where,V (m,m−1) = [δm(P (qm)Rm+δm−1P (qm−1)Rm−1)−(Fm+δmP (qm−1Fm−1)]V (m− 1) = [δm−1Rm−1(P (qm) + P (qm−1))− Fm−1].These first 2 terms represent the difference in expected returns from a)

submitting to the best journal and then to the second-best if the quality ism−1 and b) submitting directly to the second-best journal. The comparative

20In addition, Leslie (2005) does not evaluate the learning process that occurs whenauthors receive a report from a referee that identifies the quality

16

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 19: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

statics for this difference are the same as that for the scores (e.g. the differenceincreases (decreases) with the probability of being accepted in the best (second-best) journal) and represent the trade-off between the higher probability ofacceptance of the lower level journal and facing the gamble of obtaining thereward from the best journal and then, if rejected, submitting to the second-best journal.

The third term is the difference in continuation values given that the qual-ity is lower than m − 1. The sign of this difference depends only on thesubmission-acceptance lags from the two journals. Looking at the three terms,it is possible that if the best journal has a substantially shorter reviewing timeit will be optimal to submit there first even if its probability of acceptance iszero. This occurs if the third term is sufficiently large and positive. Intuitively,if information arrives faster by submitting to the best journal, the expectedcost of the lost time in the best journal if the quality is m − 1 may be lowerthan the expected benefit of being accepted faster if the quality is lower thanm−1. This is more likely to occur if the author believes the paper to be of lowquality (i.e. this term increases with the prior probability that the paper is oflow quality). However, since in general better journals have longer reviewingtimes we generally expect the last term to be negative.

To illustrate the optimal submission policy in this setup we use again theaverage statistics for each journal category to calculate the payoffs. Table 5presents the optimal initial submission under a few different scenarios. Whenthe reviewing process is highly accurate, even patient and risk-neutral authorswill tend to submit initially to journals that are more in line with the believedquality of the paper. For example, given a discount rate of five percent, au-thors should submit initially to a top-tier journal if the prior distribution isuniform, should submit to a second-tier journal if it is highly unlikely that themanuscript is of the highest quality but has equal prior probability over theremaining qualities, and should submit to a third-tier journal if the manuscriptis likely to be below the top-tier and second-tier thresholds. As before, highlyimpatient and risk-averse authors will lean towards the faster and safer outlets.

3.3 The General Case

When authors are uncertain of the papers’ quality and the reviewing process isimperfect, the probability of acceptance in a given journal depends on the his-tory of submissions. That is, authors learn about the quality of their work fromrejection letters and referee reports, and this learning affects their perceptionabout the probability of having a paper accepted at a journal. Unfortunately,it is infeasible to solve this general problem with a simple rule as we have done

17

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 20: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Table 5: Optimal 1st Submission, Perfect Referee Reports

Prior Distribution1 Discount rate Utility Optimal Initial SubmissionA 0.05 Linear firstA 0.5 Linear firstA 0.05 log firstA 0.5 log secondB 0.05 Linear secondB 0.5 Linear secondB 0.05 log secondB 0.5 log thirdC 0.05 Linear thirdC 0.5 Linear thirdC 0.05 log thirdC 0.5 log third

1Distribution A: Uniform over all 4 possible qualities; Distribution B: Zero probability of top-quality,

uniform over lower three; Distribution C: Zero probability of top two qualities, uniform over lower two

in the cases above. Therefore, the only feasible way to find the optimal sub-mission strategy is by calculating the expected value of all m! possible paths,an arduous process. Given the difficulty in solving this problem, we considerthe performance of the score, vj =

δj∆jRj

1−δj(1−∆j), where ∆j is the prior probability

of acceptance in journal j (i.e.∆j = Prob(q̃ > qj) =∑m

i=0 P (qi)(∑m−j

x=0 Pj+x,ij ),

as an alternative ‘rule of thumb’ for choosing publication outlets.The score takes into account authors’ uncertainty about the quality of the

manuscript and it also takes into account the possibility of reviewing errors.However, because the score does not take into account the useful informationprovided by a rejection (the scores are calculated as if the probability of accep-tance to a given journal is the same regardless of the submission history) it maynot give the optimal solution. For instance, since rejection from lower-leveljournals provides more information than rejection from higher level journals,the score will be biased towards submission to higher quality journals. Like-wise, the score will do better when reviewing errors are more likely because inthis case a rejection will not provide as much information.

The advantage of the score is that one only needs to know the charac-teristics of the journals to calculate m scores instead of the daunting task ofcalculating the value of m! possible paths together with all the posterior prob-abilities given the different paths. It is an empirical matter whether the rule

18

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 21: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

does well when compared to the optimal policy, calculated numerically. Thus,to evaluate the performance of this rule we calculate the optimal submissionpolicy under different scenarios in the case where there are four classes of jour-nals to consider and we compare this policy to that suggested by our simplescore.

Table 6 compares the initial submission as determined first by the score(column 5) and then by the numerical calculation of all possible paths (column6). In no case that we consider does the advice of the score differ from theoptimal choice. While we know that the score is not necessarily optimal, theseresults indicate that it may be a good ‘rule of thumb.’

In line with our previous results, when the reviewing process is imper-fect, authors that are sufficiently patient and risk-neutral should submit theirmanuscripts initially to a top-tier journal without much regard to the likeli-hood that the paper is of high quality. Impatient risk-averse authors shouldstart from the safer and faster second-tier outlets if the author is completelyuncertain about the manuscript’s quality and from a third-tier journal if themanuscript is likely to be below the first-tier and second-tier journals thresh-olds. When it is highly likely that the manuscript’s quality is below the top-tierjournals’ threshold, authors should also start from the safer outlets if the re-viewing process is fairly accurate but should submit to a top-tier journal if thereviewing process is believed to be highly inaccurate.

4 Policy Implications

Our analysis has examined the problem economists face in choosing outlets fortheir manuscripts from a number of different angles. This analysis is helpful toauthors in clarifying the incentives they face in choosing an appropriate outletfor their research. It also has implications for journal editors in looking toimprove the standing of their journal and, more generally, for understandingthe efficiency of our current system for the dissemination of scholarly research.

19

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 22: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Table 6: General Numerical Case

Prior Probability of Initial Submission - Initial Submission -Distribution1 Accurate Signal2 Discount rate Utility Score Optimal

A 0.7 0.05 Linear first firstA 0.7 0.5 Linear first firstA 0.4 0.05 Linear first firstA 0.4 0.5 Linear first firstA 0.7 0.05 log first firstA 0.7 0.5 log second secondA 0.4 0.05 log first firstA 0.4 0.5 log second secondB 0.7 0.05 Linear first firstB 0.7 0.5 Linear second secondB 0.4 0.05 Linear first firstB 0.4 0.5 Linear first firstB 0.7 0.05 log second secondB 0.7 0.5 log third thirdB 0.4 0.05 log first firstB 0.4 0.5 log second secondC 0.7 0.05 Linear first firstC 0.7 0.5 Linear first firstC 0.4 0.05 Linear first firstC 0.4 0.5 Linear first firstC 0.7 0.05 log first firstC 0.7 0.5 log third thirdC 0.4 0.05 log first firstC 0.4 0.5 log third third

1Distribution A: Uniform over all 4 possible qualities; Distribution B: Zero probability of top-quality,

uniform over lower three; Distribution C: Zero probability of top two qualities, uniform over lower two2 This is the probability that the referee’s signal equals the actual quality. Conditional

on being inaccurate, the signal is distributed uniformly over the other qualities.

20

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 23: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

Given the low submission fees at most journals, it is clear that, if de-cisions came sufficiently fast, the optimal strategy for all authors would beto submit to journals in descending ranking, regardless of the quality of theirmanuscripts. This is consistent with Leslie (2005) and Azar (2005, 2007). As aresult, well-ranked journals would receive thousands of manuscripts to review.Ellison (2002a) found that the top general interest journals have longer reviewprocesses (by 6 months, on average) than lower ranked journals, perhaps forexactly this reason. Given the large number of submissions that good qualityjournals still receive, and despite complaints by most authors, one could postu-late, as Azar (2005) does, that current reviewing times may be too short. Onthe other hand, Leslie (2005) argues that journals should use submission feesas a rationing device.21We will now use the results of our analysis of authorbehavior, above, to shed light on this discussion.

To evaluate the relative merits of different rationing policies we returnbriefly to our model where authors can use journal scores to rank journals.Consider the threshold probability of acceptance in the top journal (P T

m) thatwould make a representative author indifferent between submitting there andsubmitting to the next best alternative, which has a score v−1. This thresholdprobability is:

PTm =Fm + δmv−1

δm(Rm − v−1)(10)

The threshold probability is increasing in the fee for journal m (and de-creasing in the fees for the next best alternative). It is also increasing in the lagof journal m and on the degree of impatience of the author (and is decreasingin the lags of the other journals). Therefore, by increasing reviewing times orsubmission fees, top journals reduce the number of low quality submissions, atleast from authors that are similar to the representative author.

Currently, most economics journals utilize reviewing lags as a rationingdevice.22 As Leslie (2005, p.412) points out ’delays have become the marketequilibrating mechanism to quality rank the increased supply of research pa-pers.’ Of course, the reduction in the editorial load at the top journals comesat a cost. First, longer lags reduce the expected utility from publication of

21Of course, an alternative to rationing would be to increase the ‘space’ in journals. Thatis, quite simply, to accept more papers. For an individual journal, however, this would likelyresult in a decrease in the quality of papers published, and, if not matched by other journals,a decrease in reputation (and reward).

22Top journals in finance and accounting, by contrast, have relatively higher submissionfees and faster average response times. Some of the new journals of the Berkeley ElectronicPress (B.E. Press) also have high fees and very short reviewing times.

21

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 24: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

all authors, not just those with low quality papers. Second, a reviewing timethat is too long diminishes the contribution of the manuscript, by making itobsolete and by reducing the time others can benefit from it. Third, authorsof high quality papers that are more impatient than the representative author(so they have higher thresholds) may prefer to switch to the lower level outlets,especially if those journals provide similar rewards or if the rewards dependon the quality of the paper itself.23

Would a change from the current rationing mechanism to one with highersubmission fees and shorter lags increase the efficiency of the system? Wecould certainly conclude that this is the case if 1) the editorial load doesnot increase, 2) the quality of the papers submitted and published increases(or does not decrease), and 3) the expected utility of the authors submittingpapers to the journal increases.24 We will show that this is indeed the case.Specifically, suppose that under the current system the top journal sets thereviewing lag times such that only authors with P T

m ≥ x submit their papers.Now suppose that the journal simultaneously reduced reviewing times andincreased submission fees in such a way that the journal score remains thesame for the marginal author. The following proposition establishes the effectof this policy on those with other probabilities of success.

Proposition 4 For a given decrease in reviewing times and an increase insubmission fees such that the score (and the expected utility) of the journalremains constant for the marginal author (P T

m = x), the score (and the expectedutility) of the journal increases [decreases] for those authors with a higher[lower] probability of acceptance.25

Proof. The new policy assumes that for the marginal author, expectedutility does not change. Therefore, for him the necessary change in submission

23The fact that better journals have longer reviewing times is particularly detrimentalif reputation plays a role in the publication process. Untenured economists who have notbuilt reputation are more likely to be impatient and thus more likely to submit first tolower-quality journals with faster reviewing times (especially those faculty in departmentsoutside of the top tier where 2nd tier publications will be valued more highly). Because ofreputation effects, this may hamper future publication outcomes and handicap these authorsin the later stages of their careers.

24Presumably, the journals’ objectives are to publish high quality research and to publishresearch in a timely manner subject to an editorial load constraint. See Engers and Gans(1998), McCabe and Snyder (2004), Faria (2005a), and Basancenot et al (2009) for formalmodels of the editorial process

25this proposition holds for any journal, not just the top journal. Therefore, this policymay help lower level journals to increase the quality of the manuscripts received

22

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 25: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

fees is ∆δ(P TmRm + (1−P T

m)v−1) = ∆Fm. Since ∆Fm increases with the prob-ability of acceptance, authors with a higher (lower) probability of acceptancewould require a larger (smaller) increase in fees to remain indifferent.�

Importantly, a decrease in submission fees and an increase in reviewingtimes has the opposite effect (i.e. it decreases (increases) the score for authorswith higher probability of acceptance). In essence, editorial lags and submis-sion fees are imperfect substitutes; while the former has a larger effect on highprobability authors, the latter has a larger effect on low probability authors.26

A fair concern with high submission fees is that researchers (or researchinstitutions) with tight budgets may not be able to afford them. One alter-native, used by some of the B.E. Press journals, is to require payments ’inkind’: an author that submits a paper must provide a certain amount (andquality) of refereing services to the journal when required. Another possiblesolution would be to price discriminate - offer reduced fees to submissions fromdeveloping countries, much like many journals offer reduced subscription fees.

It is also sometimes the case that submission fees are not incurred by theauthors themselves. If authors pay fees out of a research budget provided bytheir institution the effect of the fee may be reduced if the opportunity costof using that money is low. If institutions pay fees outright (authors do nothave research budgets, but are reimbursed on a case-by-case basis) authorsmay simply disregard submission fees. That said, this effect should disappearin the long-run if institutions notice many expensive submissions with fewacceptances.

5 Conclusion

The journal submission process is a controversial and stressful part of academia.There are many dimensions of uncertainty, and bad decisions could greatlydelay publication of important results and harm one’s career. This paper pro-vides new evidence that, on the whole, the advice supplied to young facultymembers by veterans of academia is correct. Authors largely have an incen-tive to submit first to the best journals and then subsequently, work their waydown a schedule of journals. The exceptions to this simple rule occur when

26Some journals, including the Journal of Financial Economics, make the submission feesconditional on acceptance (i.e. if the paper is accepted the fee is refunded). Such a policywould likely amplify the quality effects of the simple fee system analyzed in this paperas this imposes a differential expected cost on high quality vs. lower quality submissions.(e.g. increases in fees would be negligible for those papers with a very high probability ofacceptance).

23

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 26: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

authors are particularly impatient or risk-averse.We also note, however, that the efficiency of the system may be improved

by a system in which journals reduce time lags, perhaps through incentive-based rewards for faster reviewing by referees, and increase submission fees.This system reduces the impact of time-lags on impatient or risk averse authorsand more efficiently rations submissions to journals - higher reward journalswill get more submissions of high-quality papers and fewer submissions oflow-quality papers. This also streamlines the publication process, shorteningthe time during which important results are sitting on a desk, waiting forpublication.

References

[1] Ofer H. Azar. Rejections and the importance of first response times.International Journal of Social Economics, 31(3):259–274, 2004.

[2] Ofer H. Azar. The review process in economics: Is it too fast? SouthernEconomic Journal, 72(2):482–491, 2005.

[3] Ofer H. Azar. The slowdown in first-response times in economics journals:Can it be beneficial? Economic Inquiry, 45(1):179–187, 2007.

[4] Ofer H. Azar. Evolution of social norms with heterogeneous preferences: Ageneral model and an application to the academic review process. Journalof Economic Behavior and Organization, 65:420–435, 2008.

[5] Damien Besancenot, Joao Faria, and Kim Huynh. Search and research:The influence of editorial boards on journals’ quality. Working papers,HAL, 2009.

[6] Rebecca M. Blank. The effects of double-blind versus single-blind review-ing: Experimental evidence from the american economic review. Ameri-can Economic Review, 81(5):1041–1067, December 1991.

[7] David W. E. Cabell, editor. Cabell’s Directory of Publishing Opportunitiesin Economics and Finance. Cabell Publishing Company, 2004.

[8] Glenn Ellison. The slowdown of the economics publishing process. Journalof Political Economy, 110(5), 2002.

24

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39

Page 27: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

[9] Glenn Ellison. Evolving standards for academic publishing: A q-r theory.Journal of Political Economy, 110(5), 2002b.

[10] Maxim Engers and Joshua S. Gans. Why referees are not paid (enough).American Economic Review, 88(5):1341–1349, December 1998.

[11] Joao Ricardo Faria. The game academics play: Editors versus authors.Bulletin of Economic Research, 57:1–12, 2005a.

[12] Joao Ricardo Faria. Is there a trade-off between domestic and interna-tional publications? The Journal of Socio-Economics, 34:269–280, 2005b.

[13] Joshua S. Gans and George B. Shepherd. How are the mighty fallen:Rejected classic articles by leading economists. Journal of Economic Per-spectives, 8(1):165–179, 1994.

[14] Daniel S. Hamermesh. The young economist’s guide to professional eti-quette. Journal of Economic Perspectives, 6(1):169–179, Winter 1992.

[15] Geoffrey M. Hodgson and Harry Rothman. The editors and authors ofeconomics journals: A case of institutional oligopoly. The Economic Jour-nal, 109:F165–F186, February 1999.

[16] Pantelis Kalaitzidakis, Theofanis P. Mamuneas, and Thanasis Stengos.Rankings of academic journals and institutions in economics. Journal ofthe European Economic Association, 1(6):1346–1366, December 2003.

[17] David N. Laband and Michael J. Piette. Favoritism versus search forgood papers: Empirical evidence regarding the behavior of journal editors.Journal of Political Economy, 102(1):194–203, 1994.

[18] Derek Leslie. Are delays in academic publishing necessary. AmericanEconomic Review, 95(1):407–413, March 2005.

[19] Mark J. McCabe and Christopher M. Snyder. Open access and academicjournal quality. American Economic Review: Papers and Proceedings,95(2):453–458, 2005.

[20] Marshall H. Medoff. Editorial favouritism in economics? Southern Eco-nomic Journal, 70:425–434, 2003.

[21] Sharon Oster. The optimal order for submitting manuscripts. The Amer-ican Economic Review, 70(3):444–448, 1980.

25

Heintzelman and Nocetti: Journal Submission Strategies

Published by The Berkeley Electronic Press, 2009

Page 28: Martin Heintzelman and Diego Nocetti (2009) “Where Should we Submit our Manuscript An Analysis of Journal Submission Strategies

[22] Sharon Oster and Daniel S. Hamermesh. Aging and productivity amongeconomists. Review of Economics and Statistics, 80(1):154–156, February1998.

[23] Andrew J. Oswald. An examination of the reliability of prestigious schol-arly journals: Evidence and implications for decision-makers. Economica,74:21–31, 2007.

[24] Martin L. Weitzman. Optimal search for the best alternative. Economet-rica, 47(3):641–654, May 1979.

26

The B.E. Journal of Economic Analysis & Policy, Vol. 9 [2009], Iss. 1 (Advances), Art. 39

http://www.bepress.com/bejeap/vol9/iss1/art39