We Don't Replicate Accounting Research-Or Do We?

We Don’t Replicate Accounting Research—Or Do We?*

STEVEN E. SALTERIO, Queen’s University

1. Introduction

The purpose of this introduction to this mini-forum on replication research in accountingis to explain the thought process behind the decision to publish two articles that have asubstantial overlap in motivation, data and analysis back-to-back in the same issue ofContemporary Accounting Research. Indeed, the two articles are definitely conceptual orin-principle replications of each other; albeit with differences in the design choices thateach set of authors made, leading to dramatic differences in sample sizes. However, despitewhat differences there are, the articles arrive at the same conclusions. The topic is the riseof analysts’ provision of cash flow forecasts and the decline of the accruals anomaly—cer-tainly issues that bear careful examination, given the controversy associated over the rela-tive sophistication of analysts’ cash flow forecasts and the changes over time in the mostwidely accepted and researched anomaly, the accruals anomaly. Nonetheless, given ouralleged lack of a replication culture in accounting, I decided that a brief introduction andjustification was in order as part of presenting these two articles and their associatedcommentary.

When I became Editor (in-Chief) of CAR, one of the questions that I heard frequentlywas “are you going to do anything to increase the amount of replication research inaccounting?” A good question indeed, for in the mid-1990s CAR had attempted to encour-age replications and extensions under then Editor Mike Gibbins. His establishment of aspecial section of the journal devoted to “improvements and updates” resulted in CARpublishing 12 articles under this heading between 1994 and 1996. However, only three ofthese articles were clearly replications and even then they did not advertise that feature inthe title or abstract. Thus, CAR’s call for replication research in the 1990s was met with aresounding “thud” by the academic community.

If, however, one was to accept the rhetoric of leaders of the academic accounting pro-fession (e.g., Waymire 2012; Basu 2012) accounting suffers from a distinct lack of replica-tion of important findings. Indeed, this nostrum is repeated in literature reviews (e.g.,Pomeroy and Thornton 2008; Callihan 1994), methodological criticisms and suggestionsfor improvement in accounting research (e.g., Gray, Kouhy, and Lavers 1995; Lindsay1995, 1997; Lillis and Mundy 2005; Cooper and Morgan 2008; Humphrey 2008), reflec-tions by editors of learned journals about what they publish and will publish (e.g., Guthrieand Parker 2012), and suggestions for increasing the relevance of research to practice(e.g., Taylor 2009). Furthermore, on the surface these claims about lack of a replicationculture seem to be warranted. I found “only” 25 accounting, auditing, or tax articles since1993 that used “replication” in the either the article title or abstract amongst the thou-sands of articles in the 83 accounting, auditing, and tax journals that ABI/INFORM Glo-bal database indexes.1 Indeed, amongst the top six substantive journals in accounting, as

* An introduction to the mini-forum on replication research in accounting prepared by the then CAR Editor

(in-Chief) to provide context for the forum.

1. I searched a standard bibliographic database (i.e., ABI/INFORM Global) for articles in accounting, audit-

ing, or taxation journals (i.e., any of those three words in the journal title) that contained the term “replica-

tion” in the title or abstract over the period 1993 to date (i.e., 20 years of academic accounting research).

Contemporary Accounting Research Vol. XX No. XX (XX XX) pp. 1–9 © CAAA

doi:10.1111/1911-3846.12102

proxied by their use by the Financial Times in ranking MBA and other graduate programs(i.e., FT45), only four articles advertise that they contain a “replication” (four journals,one article each). Indeed, in two of these top journals (Journal of Accounting Research andThe Accounting Review) no articles highlight the term “replication” in their title orabstract.2 Thus, it appears the critics are correct; we do not have a replication culture inaccounting research.

Nonetheless, as a new CAR Editor (in-Chief) I had much else on my plate and while Igave the issue of “lack of a replication culture in accounting research” some thought, inthe end it did not make it to my priority list of things to do (see Salterio 2013 for a sum-mary of those priorities). In the last few months of my term, however, the issue of replica-tion and reproducibility came to the foreground for two reasons. First, there was theretraction by The Accounting Review of Hunton and Gold (2010) where the editors posedsubstantive questions to the authors about their sample collection methods and theresponses were found wanting. Indeed, in a statement by the authors, they freely admittedthat only one of the two authors was involved in the data collection and that the secondauthor did not even know what firm was involved in the study. The publication of thisinformation led me to ask the authors of Hunton co-authored papers in CAR to elaborateon their involvement in the collection of those articles’ samples. In two of the three cases(Hunton, Hoitash, and Thibodeau 2011; Hunton, Arnold, and Reck 2010) the co-authorsresponded that they did not have any involvement with the collection of the data and thatHunton had collected the data and then recruited them to help write it up.3 This causedme to have some worries, for if even the co-authors on a study did not know where thedata came from or the details about how it was collected, how could the study be repli-cated, even in principle? Hence, the retraction of Hunton and Gold (2010) put replicationback on my radar screen.

Second, and directly on point to this article, I was faced with a conundrum aboutwhat to do with the two papers in this forum section (Mohanran 2014; Wu and Radha-krishnan 2014). The author teams had essentially reached the same conclusions about thesame subject matter using similar methods. The two papers were accepted by myself andanother CAR editor within days of each other in April 2013. These two papers focus onthe rise of cash flow forecasts provision by analysts and the decline of the accruals anom-aly. The papers had been submitted to CAR more than a year apart, assigned to differenteditors with different reviewers and had never been posted to a public working paperrepository (e.g., SSRN). Indeed, at the time of acceptance (but not in the versions pub-lished in this issue) neither paper referred to the other’s research and the authors confessedto being surprised at the existence of each other’s paper. Nonetheless, the results are clo-sely related and provide strong replications of each other’s work. There was substantialvariation in the advice I received about what to do with two such closely related papers.The advice included determining which paper is the better of the two then publishing onlythat paper, combining the author teams and having the authors co-write one jointlyauthored paper, and ignoring the known existence of such highly similar papers and pub-lishing them a couple of issues apart! From the advice I received, I assumed that I couldcount on no one noticing the replications! So much for the calls for more replication in

2. Interestingly, the largest number of articles that advertise replication are found in an accounting education

journal (six in Issues in Accounting Education). Besides the Journal of Business, Finance and Accounting that

had four such articles, no other accounting, auditing, or tax journal featured more than two such articles

over the twenty year period.

3. I should note that one of co-authors of Tan, Libby and Hunton (2010) other than Hunton received Experi-

ment 1’s data directly from the firm involved in the research and that all authors were informed of the firm

that was providing the data in Experiment 2. Further, it should be clear that in Hunton et al. (2011), the

involvement of only one author with the data collection was clearly stated.

2 Contemporary Accounting Research

CAR Vol. XX No. XX (XX XX)

accounting research. After all, these “solutions” essentially advocated either not publishingreplicated work or hoping that no one would notice the replication if published!

2. What is a replication?

One of the confusing aspects about replication research is—What constitutes a replication?Our discussants, Ecker and Schipper (this volume) state clearly that the two papers are notreplications of each other as they do not “attempt an exact-as-practicable retracing ofanother’s research, including all aspects of the research design.” However, statisticians havea slightly different take on replication. Lindsay and Ehrenberg in the American Statistician(1993, 220) state: “One reason for the low regard for replication is the mistaken view thatit merely involves repeating the original study exactly. Researchers often do not recognize,or even deliberately ignore, the fact that replication must always involve some variation inthe conditions of the study.” They go on the point out “an identical replication would inany case be pointless: if all conditions were literally or effectively identical, the resultswould have to be so too.” (220). They conclude with the consideration, “In general, themore explicit, differentiated, and/or deliberate such variations in the conditions of observa-tion are while still obtaining the same result the more telling and exciting the outcome.”They go on to point out that at least two types of replications are possible:4

1. Close replication (direct replication as it is called in psychology research per OpenScience Collaboration 2012) “attempts to keep almost all the known conditions ofthe study much the same or at least very similar” similar to what Ecker and Schip-per describe as their criteria for replication.

2. Differentiated replication (or an “in-principle replication” in psychology research)that “involves deliberate, or at least known, variations in fairly major aspects of theconditions of the study” (220).

The latter is how I conceptualize the two papers that follow. I find the argument ofLindsay and Ehrenberg (1993) very persuasive when they argue that the ability to triangu-late or have convergent validity of the results that depend on the conceptual level of thevariables in question, not the artifact of method employed to realize these variables, is theadvantage that differentiated replication has over close replication. Hence, this essay onreplication in accounting research as a lead in to these articles.

3. Replication in social science research

The assertions about the lack of replication in accounting research are consistent withreports of lack of replication in other business disciplines. Studies late in the last centuryreported that in marketing between 2.4 percent and 6 percent of articles are replicated,finance as having a 10 percent replication rate, and management a 5 percent rate (seeTable 1, Tsang and Kwan 1999). Updates to those statistics suggest things are not improv-ing in this century, with marketing reporting a decline to less than 1.2 percent replicationrate versus a 2.4 percent rate of replication in the original 1994 report (Evanschitzky,Baumgarth, Hubbard, and Armstrong 2007).5 In addition, and probably even more trou-bling, there was the report that between a quarter and a half of the research results thatwere identified as being attempted replications were inconsistent with the results in the ori-ginal articles (Tsang and Kwan 1999).

4. Tsang and Kwan (1999) have an even more refined framework that breaks replication research into six

cells.

5. I reference this report specifically as it uses the same methods to estimate replication and indeed has an

author of the initial study on the replication. Hence, we can be somewhat assured that the comparisons are

actually comparable.

Replication in Accounting Research 3


Economists (e.g., Wible 1991) and psychologists (e.g., Nosek, Spies, and Motyl 2012)attribute this lack of “replication” as being due to the “well known incentives” of academ-ics to concentrate on reporting novel findings rather than “less interesting” replicationstudies, as well as the reluctance of leading journals to publish replication studies. Indeed,academic associations in economics (Glandon 2011) and psychology (Open Science Col-laboration 2012) have supported efforts to create a higher investment in replication bylowering the cost of doing the same. There also appears to be an interest in creating sucha culture in sociology as well (Freese 2007). That these three disciplines are also strugglingwith replication is important for accounting researchers, as they form the theoreticalunderpinnings for much of modern academic accounting research.

In psychology, the recent emphasis on lack of replication reportedly came from twomajor events. The first was the publication in a mainstream psychology journal, theJournal of Personality and Social Psychology, of a paper on psychic effects on cognition(Bem 2011). There was great skepticism of the article’s findings about evidence consistentwith ESP (extra-sensory perception) and hence several research teams set out to replicatethese experiments. The authors of these replications reported great difficulty in findingleading academic psychology publications that would accept their replications thatchallenged Bem’s findings (see Yong 2012). Indeed, the journal that published Bem (2011)initially turned down replication papers (they did not replicate Bem’s findings) by leadingpsychology researchers without review on the grounds that they did not publish repli-cation studies (Yong 2012). These difficulties were taken as added evidence that editorialpolicies and practices also contributed to the lack of replication research.

The second incident in psychology that raised questions about the lack of replicationis the case of Derek Stapel. Stapel is a Dutch social psychologist who appears to have fab-ricated his data in at least 30 of his publications (Yong 2012). Stapel’s fraud was madepublic by an internal whistleblower (a member of his lab) rather than being challenged bypublished findings in academic journals reporting that his results would not replicate. Thisinability for social science to self-correct through replication what has been termed a “sci-entific fraud on a massive scale” troubled thought leaders in psychology. Hence, an APSjournal, Perspectives on Psychological Science, set up the Reproducibility Project that fea-tures volunteer faculty from around the world replicating articles from leading psychologyjournals (Open Science Collaboration 2012). The editors of Perspectives called for facultymembers at leading universities to replicate studies that were published in a six-month per-iod in four leading psychology journals. Every replication attempt is to be registered inadvance with the journal and features the original study’s authors reviewing the proposedreplication to ensure the procedures are the same as in the original experiment (Open Sci-ence Collaboration 2012). The goal of the project is to determine the severity of the prob-lem associated with lack of replication is in psychology research by determining howmany published articles in leading psychology journals cannot be replicated (Open ScienceCollaboration 2012).6

Even earlier than this concern that arose in psychology, economic journals had begunto deal with the lack of replication of economics-based research. Being economists, they feltthe obstacle must be the high costs associated with replication versus the low rewards(Wible 1991). Hence, they strove to reduce the costs of replication research believing thatreplications would then be more likely to be done. The American Economic Association’spremiere journal, the American Economic Review, as part of their efforts to reduce replica-

6. An entire issue of Science (December 2011) was devoted to the wider issue of the lack of replication in

many scientific areas (Jasny, Chin, Chong, and Vignieri 2011) including citing reports from private sector

research labs (e.g., Bayer Healthcare) that they could not replicate findings in biomedical research that they

were investigating for potential development of therapeutic drugs (see Begley 2012 for details of these

claims).



tion costs, required authors from March 2005 onward to deposit electronic copies of theirfinal data and statistical programs for analyzing the data as a condition for publication(Glandon 2011). In 2008, the editor of the journal had a then graduate student carry out an“audit” of whether authors were submitting the needed materials. Glandon (2011) reportedthat 95 percent of the articles could be replicated, in principle, without the assistance of theoriginal authors, a key goal of reducing the costs of replication. Further, Glandon and otherdoctoral students attempted to replicate nine of the articles’ results using the data and pro-grams provided. All nine were replicated successfully with only minor differences fromreported results found (Glandon 2011). However, as the AER report notes, the replicationstarted with the final data file submitted by the author, not the original raw data. Hence,while seen as good news, Glandon (2011) explicitly noted that this approach could not dis-cover if there were problems in the original data that had been part what he termed “exten-sive data cleaning” that is inherent in every archival econometric study.

Indeed, as of this writing there is a major upset in the economic research world.Authors who attempted to replicate the findings of Reinhart and Rogoff (2009) (hereafterR&R) on the subject of economic growth and its relationship to government debt levelswere not able to do so based on the publicly cited data sources that R&R stated theyemployed (Milner 2013). One team of three authors attempting to replicate the studyasked R&R for a copy of their data. The three authors were able to replicate the originalresearch results based on the data, but along the way found there were serious errors inthe original data files (Milner 2013).7 Once those errors were corrected, the reportedresults of R&R are said to disappear. As R&R findings had a heavy influence of publicpolicy debates in the United States, these policy prescriptions based on the original resultsare being called into question. This recent case, albeit with the caveat that the replicationstudy has not been subject to peer review (nor was the original manuscript that was inbook form rather than an article submitted for peer review), suggests that Glandon’s(2011) observations need to be taken seriously. That is, data repositories are not thepanacea that many research reform advocates call for and indeed were among the mostcommon suggestions that I received as to what I could do as CAR Editor (in-Chief) toencourage more replication.

4. Replication research in accounting

Based on my own experience as a researcher, the story about lack of replication in social sci-ence research did not ring true to me. Certainly, there are articles of mine that have neverbeen replicated but my more “influential” ones certainly have been, even when they are so-called “lab experiments” that require recruitment of managers to do a task. Specifically, myco-authored study with Marlys Lipe (Lipe and Salterio 2000) on the effects of common mea-sures on managerial decision making in the balanced scorecard context was subject to at least18 replications and extensions (see Salterio 2012 for a list of these articles under the heading“Counterattacks and Elaborations on a Theme”). Given this variance between reports of alack of replication culture and my own experience as an author, I decided to look into theaccounting research literature beyond examining article titles and abstracts.

Using the advanced search capabilities of ABI/INFORM Global bibliographicresearch I searched for accounting, auditing, or tax articles that contained the word “repli-cation” in the text of the article (not just in the abstract or title). I found an additional443 such articles in accounting, auditing, and tax journals published since 1992 containing“replication” over the 25 with the term in the title or abstract. I then read the 10 words

7. Columns not included in the analysis that were said to be included; rows not added together, etc. In addi-

tion there were a variety of design choices that were controversial but the basic data errors are plain to see

(Milner 2013).



around the term (both in advance and after) for a sample of 200 articles (the first 200articles presented by the database using its “relevance” criteria). Of those 200, I found thatthat 46 articles (23 percent) reported replication of all or part of the findings of at leastone other accounting research study (indeed one article replicated parts of the findings offive studies).8 Furthermore, if one allows for variations of the word “replication” in thefull text of the articles by employing the “*” wildcard to account for all endings (i.e.,searching on “replicat*”) I identified another 943 accounting, auditing, and tax articles.These articles potentially contain replicated results that are in addition to the 468 articlesfrom the initial searches. If the 23 percent replication rate I found in my sample of 200articles can be legitimately projected to this population of over 1,300 articles, there maybe more than 300 articles in accounting academic literature in the last 20 years that con-tain at least partial replications of one or more studies. So much for an academic account-ing culture that does not do replication research!

While this latter finding is more consistent with my own accounting research experi-ence, I carried out one more check on the data. I decided to see whether the 18 extensionsand replications of Lipe and Salterio (2000) that I reported on in Salterio (2012) wouldhave been discovered by this database search procedure. To my surprise, I found that only1 of the 18 articles employed the term “replicat*” in it. Hence, if one searched literaturedatabases following my approach they would have missed 17 of the 18 extension and rep-lication articles that I reviewed in Salterio (2012). Thus, if anything the method I used tolocate potential replications of prior accounting research may understate the amount ofreplication work being done.

5. Conclusion and lessons learned

While it might be conventional wisdom to bemoan the lack of replication research inaccounting, it appears that there is a lot of “hidden” replication going on. Certainly, Ifound that it was relatively rare in accounting research for authors to advertise (in thearticle title or abstract) that they are carrying out a replication as part of their study.Indeed, such articles averaged less than 1.25 a year for the last 20 years even employingthe 83 accounting, auditing, and tax journals covered by ABI/INFORM Global. Nonethe-less, I found that some of these hidden replications in accounting research can be foundby searching the full text of articles for terms like “replication” and its variants. These rep-lications are often being done as part of extending that prior research, or more rarely,challenging the findings. My search of the full text articles in the ABI/INFORM Globaldatabase results in an increase from 1.25 replications published a year to 3.55 a year overthe last 20 years. Further, if I may be allowed to make a na€ıve projection based on thesample of 200 articles I examined to the population of over 1,300 articles containing theterm “replicat*” there are potentially more than 300 accounting studies published in thelast 20 years that involve some element of replication of prior research. That is 17 pub-lished, at least partial, replication studies a year! Third, based on my own experience, rep-lications may not be that easy to find in accounting research even when they occur. Onlyone of the 18 studies that extended, and in doing so replicated at least in part Lipe andSalterio (2000) that I report on in Salterio (2012) would have been discovered employingthis database search procedure. Hence, even taking into account the “hidden” replicationsmy search of the database discovered, this approach may well understate the true amountof replication research being done in accounting, if my experience generalizes.

8. Beyond the use of the term I was interested in, I found the word “replication” in review articles, in calls for

replications by authors of original articles, as a technical term associated with options, and within manage-

ment accounting and accounting information systems articles associated with database replication.



Some may argue that only “significant” findings are being replicated in accountingresearch, but isn’t that really where replication should be done? Certainly not every articlepublished, even in a top-tier journal like CAR, has such an impact on practice or researchthat it needs to be replicated. But what determines whether articles have findings that are“significant enough” to call for replication? Based on my more than a decade of editingexperience at CAR and AAA section journals my observation is that significance is judged,at least in part, by one or more of:

• where the article was originally published (i.e., on the FT 45 list),

• whether the article challenges other research or accepted beliefs about how theaccounting world works,

• whether the article attracts the attention of regulators or standard setters,

• whether the article attracts attention from practitioners who try to use the findings“to make money.”

So do the two following articles warrant publication under these criteria? Both arebeing published in CAR at the same time so the first criterion is readily met. Further,these two articles provide evidence about what has been termed one of the most recog-nized and controversial anomalies in the capital markets literature, the accruals anomaly(e.g., Fama and French 2008; Shiller 2003). In addition, the association studied—betweenthe decline of the accruals anomaly and the rise of analysts’ cash flow forecasting—employed a construct (i.e., cash flow forecast) that is controversial in and of itself relatedto the sophistication of cash flow forecasts (e.g., Givoly, Hayn, and Lehavy 2009 versusCall, Chen, and Tong 2013). For if as some contend (Givoly et al. 2009) analysts’ cashflow forecasts are mindless extensions of earnings forecasts, they would not provide any ofthe insights that the authors of these two studies suggest are employed by the market,based on the increased provision of such forecasts. So the second criterion is readily met.The accruals anomaly has attracted much attention in the investment community, leadingseveral major investment banks and hedge funds to hire accounting researchers to eitherrun or assist in running various funds. So the last criterion is met. Given three out of fourmeasures of significance have been met (and it often only takes one of the four to warranta replication), these two studies meet or beat those criteria and hence warrant replication.

So enjoy this pair of articles and the accompanying commentary on the issue ofthe relationship between the rise of analysts’ cash flow forecasting and the decline ofthe accruals anomaly! Spend some time, too, thinking about whether you agree with theconventional wisdom that accounting research has a culture that does not replicate its sig-nificant findings. I hope you will find the evidence that I present here in this introductionwill stimulate your thoughts about your own experience with research being replicatedand hopefully be less accepting of conventional wisdom in this area.

References

Basu, S. 2012. How can accounting researchers become more innovative? Accounting Horizons 26

(4): 851–70.

Begley, S. 2012. More trial, less error—An effort to improve scientific studies. Reuters. Available

online at www.reuters.com/assets/print?aid=USBRE87DOI820120814, retrieved August 14,

2013.

Bem, D. J. 2011. Feeling the future: Experimental evidence for anomalous retroactive influences on

cognition and affect. Journal of Personality and Social Psychology 100 (3): 407–25.

Call, A., S. Chen, and Y. Tong 2013. Are analysts’ cash flow forecasts na€ıve extensions of their own

earnings forecasts? Contemporary Accounting Research 30 (2): 438–65.

Callihan, D. S. 1994. Corporate effective tax rates: A synthesis of the literature. Journal of Account-

ing Literature 13: 1–43.



Cooper, D. J., and W. Morgan. 2008. Case study research in accounting. Accounting Horizons 22

(2): 159–78.

Evanschitzky, H., C. Baumgarth, R. Hubbard, and J. S. Armstrong. 2007. Replication research’s dis-

turbing trend. Journal of Business Research 60 (4): 411–15.

Fama, E., and K. French. 2008. Dissecting anomalies. Journal of Finance 63 (4): 1653–78.

Freese, J. 2007. Replication standards for quantitative social science: Why not sociology? Sociologi-

cal Methods and Research 36 (2): 153–72.

Givoly, D., C. Hayn, and R. Lehavy. 2009. The quality of analysts’ cash flow forecasts. The

Accounting Review 84 (6): 1877–911.

Glandon, P. 2011. Report on the American economic review data availability compliance project.

American Economic Association. Available online at http://www.aeaweb.org/aer/

2011_Data_Compliance_Report.pdf, retrieved August 14, 2013.

Gray, R., R. Kouhy, and S. Lavers. 1995. Methodological themes: Constructing a research database

of social and environmental reporting by UK companies. Accounting, Auditing & Accountability

Journal 8 (2): 78–87.

Guthrie, J., and L. D. Parker. 2012. Reflections and projections. Accounting, Auditing and Account-

ability Journal 25 (1): 6–26.

Humphrey, C. 2008. Auditing research: A review across the disciplinary divide. Accounting, Auditing

and Accountability Journal 21 (2): 170–203.

Hunton, J., and A. Gold. 2010. A field experiment comprising the outcomes of three fraud brain-

storming procedures: Nominal group, round robin, and open discussions. The Accounting

Review 85 (3): 911–35.

Hunton, J., V. Arnold, and J. Reck. 2010. Decision aid reliance: A longitudinal field study involving

professional buy-side financial analysts. Contemporary Accounting Research 27 (4): 997–1023.

Hunton, J., R. Hoitash, and J. Thibodeau. 2011. The relationship between perceived tone at the top

and earnings quality. Contemporary Accounting Research 28 (4): 1190–224.

Jasny, B., G. Chin, L. Chong, and S. Vignieri. 2011. Again, and again, and again. Science 334

(December 2): 1225.

Lillis, A. M., and J. Mundy. 2005. Cross-sectional field studies in management accounting research-

closing the gaps between surveys and case studies. Journal of Management Accounting Research

17: 119–41.

Lindsay, R. M. 1995. Reconsidering the status of tests of significance: An alternative criterion of

adequacy. Accounting, Organizations and Society 20 (1): 35–53.

Lindsay, R. M. 1997. Lies, damned lies and more statistics: The neglected issue of multiplicity in

accounting research. Accounting and Business Research 27 (3): 243–58.

Lindsay, R. M., and A. S. C. Ehrenberg. 1993. The design of replicated studies. The American Stat-

istician 47 (3): 217–22.

Lipe, M., and S. Salterio. 2000. Balanced scorecard: Judgmental effects of common and unique per-

formance measures. The Accounting Review 75 (3): 283–98.

Milner, B. 2013. Bad math? Reinhart-Rogoff study on perils of debt called into question. Globe and

Mail, April 16, B-9.

Mohanran, P. 2014. Analysts cash flow forecasts and the decline of the accruals anomaly. Contempo-

rary Accounting Research (this issue).

Nosek, B. A., J. R. Spies, and M. Motyl. 2012. Scientific Utopia: II. Restructuring incentives and

practices to promote truth over publishability. Perspectives on Psychological Science 7 (6):

615–31.

Open Science Collaboration. 2012. An open, large-scale, collaborative effort to estimate the repro-

ducibility of Psychological Science. Perspectives on Psychological Science 7 (6): 657–60.

Pomeroy, B., and D. B. Thornton. 2008. Meta-analysis and the accounting literature: The case of

audit committee independence and financial reporting quality. European Accounting Review 17

(2): 305–30.



Reinhart, C. M., and K. S. Rogoff. 2009. This time it is different. Princeton: Princeton University

Press.

Salterio, S. E. 2012. Balancing the scorecard through academic accounting research: Opportunity

lost? Journal of Accounting and Organizational Change 8 (4): 458–74.

Salterio, S. 2013. Contemporary Accounting Research 2012 Editor’s Report. Available online at

https://www.caaa.ca/CAR/index.html, retrieved August 14, 2013.

Shiller, R. J. 2003. From efficient markets theory to behavioral finance. Journal of Economic Perspec-

tives 17 (1): 83–104.

Tan, H.-T., R. Libby, and J. Hunton. 2010. When do analysts adjust for biases in management guid-

ance? Effects of guidance track record and analysts’ incentives. Contemporary Accounting

Research 27 (1): 187–208.

Taylor, S. 2009. Capital markets regulation: How can accounting research contribute? Australian

Accounting Review 19 (4): 319–25.

Tsang, E. W. K., and K.-M. Kwan. 1999. Replication and theory development in organizational

science: A critical realist perspective. Academy of Management Review 24 (4): 759–80.

Waymire, G. B. 2012. Seeds of innovation in accounting scholarship. Issues in Accounting Education

27 (4): 1077–93.

Wible, J. R. 1991. Maximization, replication and the economic rationality of positive economic

science. Review of Political Economy 3 (2): 164–86.

Wu, S.-L., and S. Radhakrishnan. 2014. Analysts’ cash flow forecasts and accrual mispricing.

Contemporary Accounting Research (this issue).

Yong, E. 2012. Bad copy: In the wake of high profile controversies, psychologists are facing up to

problems with replication. Nature 485 (May): 298–300.



Documents

We Don't Replicate Accounting Research-Or Do We?