Trends in Scientific, Technical and Medical Publishing

Ganesha Associates 1

Trends in Scientific, Technical and Medical Publishing

Associação Brasileira de Editores Científicos VIII Workshop de Editoração Científica

10 a 13 Novembro 2014Campos do Jordão/SP

12/11/2014


CV• BSc Physics 1971, PhD Neuroscience 1976, post doc

Epidemiology 1975-1979• Visiting Researcher, UFPe 1978-79, 1984• Editor, Publisher, Director at Elsevier Science, 1979 –

2005• Pubmed systems expert, NCBI, NIH 2006-2007• STM business analyst, Outsell Inc, 2009-2011• Visiting Professor UFPe, 2006-2008, 2012-2014• Independent consultant Ganesha Associates 2006-2014• Consultant, European Bioinformatics Institute ELIXIR

impact project 201512/11/2014

13/08/2013 Ganesha Associates

www.ganesha-associates.com


Elephants in the room

• Primary scientific content is predominantly in the form of articles with an IMRD structure (and in the future, related data)

• ‘Grey literature’ (standards, guidelines, professional communications, etc) excluded.

• Science isn’t local or national• “Quality”: Many publishing problems stem

from lack of critical reading of the scientific literature, not ‘English’

12/11/2014

Ganesha Associates 712/11/2014

The growing importance of metrics


Ganesha Associates 10Source: International Comparative Activity Performance – Elsevier 2011

BRIC output: documents

12/11/2014


Citation quality is a problem

12/11/2014



EPI : English Proficiency Index by age

12/11/2014

12/11/2014 Ganesha Associates 18

The emergence of Open Access

Clockwise: Harold Varmus, Michael Eisen, Pat Brown and David Lipman


History of sequence info => open access

12/11/2014


Data growth curves of 5 major EMBL-EBI resources (European Genome-phenome Archive (EGA); European Nucleotide Archive (ENA); Proteomics data repository (PRIDE); Metabolomics resource (MetaboLights); and Functional genomics database (ArrayExpress) over the years 2005-2013. Source: EMBL-EBI.

12/11/2014



ELIXIR: the European Research Infrastructure for biological data

• ELIXIR connects national infrastructures and EMBL-EBI– Launched in 2014

• 11 Member states + EMBL signed agreement– Czech Republic, Estonia,

Denmark, Finland,Israel, Netherlands, Norway,Portugal, Switzerland, Sweden, UK

• 6 countries have signed MoU and prepare national signature– Belgium, Greece, France, Italy,

Slovenia, Spain

12/11/2014

Ganesha Associates

Sustainable archives - value of data reuse?

2412/11/2014


The future

• Publicly funded and large biomedical research funders are committed to open access publishing.

• Smaller charitable funders are supportive of the aims of open access, but are concerned about the practical implications for their budgets and their funded researchers.

• Biomedical research funders are now turning their attention to other priorities for sharing research outputs, including data, protocols and negative results.

• Publishing priorities of biomedical research funders [BMJ Open 17 Sept 2014]

12/11/2014


Summary

• To be of use information resources need to be interconnected and freely accessible

• The value of these resources will be constantly assessed and reappraised

• This is not a job that can be undertaken by publishers alone

• Funders and end-users will drive the big changes

12/11/2014


Scientific assessment

12/11/2014


Scientific assessment - 1• “Subjective assessments of the merit and likely impact of

scientific publications are routinely made by scientists during their own research, and as part of promotion, appointment, and government committees.”

• The San Francisco Declaration on Research Assessment (DORA, 2012) intends to halt the practice of correlating the journal impact factor with the merits of a specific scientist's contributions.

• But then what…?

12/11/2014


Scientific assessment - 2

• “In a large cohort of NHLBI-funded cardiovascular R01 grants, we were unable to find an association between better percentile ranking and higher scientific impact as assessed by citation metrics.” [Circulation Research 9 January 2014]

• “Peer review should be able to tell us what research projects will have the biggest impacts,” Lauer contends. In fact, we explicitly tell scientists it’s one of the main criteria for review. But what we found is that peer review is not predicting outcomes at all. And that’s quite disconcerting.” Michael Lauer, NHLBI, NIH, Science August 2014

12/11/2014


Scientific assessment - 3

• “Using two large datasets (F1000 and Wellcome Trust) in which scientists have made qualitative assessments of scientific merit, we show that scientists are poor at judging scientific merit and the likely impact of a paper, and that their judgment is strongly influenced by the journal in which the paper is published.

• We also demonstrate that the number of citations a paper accumulates is a poor measure of merit and we argue that although it is likely to be poor, the impact factor, of the journal in which a paper is published, may be the best measure of scientific merit currently available.” [PLoS Biology 2013]

12/11/2014


Scientific assessment - 5• “A complete citation network was constructed

from all PubMed indexed English literature papers addressing the belief that β amyloid, a protein accumulated in the brain in Alzheimer’s disease, is produced by and injures skeletal muscle of patients with inclusion body myositis.

• We found that citation was often used to generate inappropriate information cascades resulting in unfounded authority of claims.” [BMJ 2009]

12/11/2014



Irreproducibility - 1• Drug development: The biotechnology company Amgen was

unable to replicate the vast majority of published pre-clinical research studies - only 6 out of 53 landmark cancer studies could be replicated, a success rate of 11%. [Nature 28 March 2012]

• Cancer research: There are many technical reasons why experimental results, particularly in cancer research, cannot be reproduced, including unrecognized variables in the complex experimental model, poor documentation of procedures, selective reporting of the most-positive findings, misinterpretation of technical noise as biological signal and, in the most extreme cases, fabrication of data. [Nature Reviews Clinical Oncology, 1 October 2013]

12/11/2014


Summary

• Bibliometrics, citation network analysis and informatics can provide insight into how the publishing process is working and can be improved.

• Impact factors may help in predicting author submission preferences but scientific impact can probably only be inferred by human annotation of citation networks

12/11/2014


Publishing 3.0• Publishing cycle: Funder[Strategy]> Author[Proposal]> Funder[Peer

Review: Likely outcome]> Researcher [Manuscript]> Editor[Peer Review: Methodology+?]> Publisher[Standards]> User[Usefulness]

• The funder and the reader determine “purpose” and “usefulness”. • The editorial process checks that basic methodology is appropriate,

argumentation clear, rules of scientific writing met, etc.

• The publisher creates the XML, links, applies data standards

12/11/2014


Publishing 3.0

• Each field will support a small hierarchy of journals but only journals near the top and at the very bottom will be strong brands.

• And the one at the bottom will probably be PLoS ONE or another mega journal!

12/11/2014



Publishing 3.0

• Not all journals will thrive• Need more ambition in the aims and scope,

instructions to authors• Objective, transparent editorial selection

process and clearer feedback to authors could help increase IF.

12/11/2014


Aims and scope/Instructions to authors• Aims and scope: Our mission is to promote scientific

knowledge generated in the rigor of the research methodology and ethics. The purpose of *** is to publish the outcomes of original research to advance the practice of *** in the medical, surgical management, education, research and information technology and communication.

• Abstracts: The summary should be structured into five sections (Background, Objectives, Methods, Results, Conclusions) when it is an Original Article, avoiding abbreviations and considering the maximum number of words.

• Figures: Figure legends should be double-spaced, and be numbered and placed before the References.

12/11/2014


Referee’s comments

1. I do not see any substantial improvement over the current literature regarding the association between atopic diseases and childhood leukemia. The study design of this paper still suffers from the possibility of reverse causality and does not contribute anything novel to the literature.2. A key strength of this study is that it examines these associations in a different population with a different immune profile than has previously been examined - a population that per the authors in lines 166-167 has a high incidence of parasite infection. Important limitations include the sample size and the methods of exposure assessment.

12/11/2014


Publishing 3.0• Reasons for rejection. Hypothesis unclear and/or unoriginal,

submitted to the wrong journal, badly written (Portuguese/English).

• The metaphor of “dwarfs standing on the shoulders of giants” expresses the meaning of "discovering truth by building on previous discoveries".

• But where are the giants and how do you climb on their shoulders?

• The virtual library, science without borders: reading English, collaboration

12/11/2014


Conclusions

• The STM publishing environment is changing rapidly.

• Brazilian STM publishers have an opportunity to reinvent themselves in a form that can be globally competitive.

• In order to achieve this goal they must professionalise their services and partner with key funders and user groups

12/11/2014

Documents

Trends in Scientific, Technical and Medical Publishing