Presentation given at Open Science question and answer session hosted by the Institute for Quantitative Social Science (IQSS), and the Office for Scholarly Communication (OSC) at Harvard University, on July 16th 2014.
Text of Open science: your questions answered
OPEN SCIENCE: YOUR QUESTIONS ANSWERED Varsha Khodiyar Data Publishing Manger, F1000Research @vkf1000 Michael Markie Associate Publisher, F1000Research @mmmarksman f1000research.com @f1000research
OPEN SCIENCE PUBLISHING MEANS: Open access articles Open peer review Open data Open software Full research provenance Open science is the movement to make scientific research, data and dissemination accessible to all levels of an inquiring society, amateur or professional.
ABOUT F1000RESEARCH THE FIRST OPEN SCIENCE JOURNAL The Seer of Science Publishing Science 4 October 2013: Vol. 342 no. 6154 pp. 66-67 DOI: 10.1126/science.342.6154.66 http://www.sciencemag.org/conte nt/342/6154/66.full.pdf http://blog.f1000.com/2013/10/04/vitek- tracz-science-interview/
WHAT IS F1000RESEARCH? F1000Research is an open science journal for life scientists that accepts all scientifically sound articles, ranging from single findings, case reports, protocols, replications, and null or negative results to more traditional articles. Key features: Publication within a week Transparent, post-publication peer review All data included Accepts non-traditional article types
OPEN PEER REVIEW
TYPES OF PEER REVIEW Time of review: Before publication: mediated by each individual journal Cascading review: reviews carried over to the next journal after rejection Third-party review: the peer review is no longer coupled to a journal. Post-publication peer review: journal publishes the article, then reviewers look at it. Transparency of review: Single-blind: the reviewer knows who the authors are, but the authors dont know who the reviewers are Double-blind: authors and reviewers are both anonymous Open peer review: all names are public. See: http://www.britishecologicalsociety.org/publications/journals/ for examples of each
TRADITIONAL PEER REVIEW Time of review: Before publication: mediated by each individual journal Cascading review: reviews carried over to the next journal after rejection Third-party review: the peer review is no longer coupled to a journal. Post-publication peer review: journal publishes the article, then reviewers look at it. Transparency of review: Single-blind: the reviewer knows who the authors are, but the authors dont know who the reviewers are Double-blind: authors and reviewers are both anonymous Open peer review: all names are public. See: http://www.britishecologicalsociety.org/publications/journals/ for examples of each
TRADITIONAL PUBLISHING AND PEER REVIEW TIMELINE
ISSUES WITH TRADITIONAL PEER REVIEW SYSTEM Lack of transparency - Who are the reviewers? - What happened with this paper before it was accepted? Lack of accountability - Anonymous reviews - Editorial decisions may not reflect reviews Inefficiency - Re-reviewing the same work at different journals Delays incidental (reviewing takes time) deliberate (reviewers delaying competitor papers) Cartoon by Nick D Kim, strange-matter.net
A POST-PUBLICATION APPROACH TO PEER REVIEW
REFEREE SCORES Approved Approved with reservations Not approved Articles with sufficient positive evaluations are indexed in PubMed, Scopus, and Embase. or Minimal requirements for indexing
OPEN PEER REVIEW AND OPEN COMMUNITIES Referee reports and other comments are visible to anyone. Community input
BENEFITS OF TRANSPARENT REVIEW FOR AUTHORS AND READERS Visible discussion between referees and authors (and editors) puts paper in context. Referees are good at spotting broader significance of an article. Shows the back story of a paper. (e.g. Why did it take 3 rounds of review? Authors can demonstrate that their paper was reviewed by top people in their field. Reduces bias amongst referees Educational aspect of open peer review: Open referee reports can serve as examples. Demonstrates differences between reviewers
BENEFITS FOR REVIEWERS Get a DOI: Take credit for hard work Demonstrate experience as reviewer Shows reviewers informed opinion of the work as a peer in the field, and where they thought it could be improved. Especially relevant in borderline cases, where an article just barely passed review. Heres what the community think: http://vimeo.com/99777547.
WHY SHARE DATA AND CODE?
WHY SHARE YOUR DATA? Transparency and openness are cornerstones of the scientific method Not allowing reuse of data is scientific malpractice Royal Society; Science as an open enterprise, Final report 2012 http://royalsociety.org/about-us/history/
SHARING DATA CORRELATES WITH HIGHER CITATIONS We conclude there is a direct effect of third-party data reuse that persists for years beyond the time when researchers have published most of the papers reusing their own data...We further conclude that...a substantial fraction of archived datasets are reused, and that the intensity of dataset reuse has been steadily increasing since 2003. Piowar HA., Vision TA. Data reuse and the open data citation advantage. PeerJ (2013) doi: 10.7717/peerj.175
SHARING DATA ADDITIONALLY PROMOTES Diversity of analyses and opinion New research testing of new hypotheses new analysis methods meta-analyses to create new datasets studies on data collection methods Reduction of error and fraud Education of new researchers Increased return on investment in research
PUSH TOWARDS DATA SHARING
SHARING DATA ALLOWS REPLICATION [W]e evaluated the replication of data analyses in 18 articles on microarray-based gene expression profiling published in Nature Genetics in 20052006...We reproduced two analyses in principle and six partially or with some discrepancies; ten could not be reproduced. The main reason for failure to reproduce was data unavailability. Ioannidis JPA. et al. Repeatability of published microarray gene expression analyses. Nature Genetics 41, 14955 (2009)
RESEARCH BECOMES HARDER TO ACCESS WITH AGE We examined the availability of data from 516 studies between 2 and 22 years old The odds of a data set being reported as extant fell by 17% per year Broken e-mails and obsolete storage devices were the main obstacles to data sharing Policies mandating data archiving at publication are clearly needed Vines TH. et al. The availability of research data declines rapidly with article age. Curr Biol 24, 947 (2014)
MAKING DATA AND CODE TRULY OPEN
FULL DATA INTEGRATION WITH RESEARCH PAPERS
DATA ARTICLES A dataset (or set of datasets) together with the associated methods/protocol used to create the data. No analysis of the data, results or conclusions should be included. http://f1000research.com/author-guidelines#data-art-sub One goal we had for publishing this Data article in F1000Research was to quickly share some of our ongoing behavioral datasets in order to encourage collaboration with others in the field. Donald Cooper, University of Colorado, Boulder http://f1000research.com/articles/2-53/v1
SOFTWARE AVAILABILITY SECTION
DATA PLOTTING TOOL http://blog.f1000research.com/2013/11/11/data-plotting/
DATA PLOTTING TOOL
ONGOING PROJECTS AT F1000RESEARCH F1000Research is involved in discussions with institutional librarians, researchers and other journals concerning data publication and sharing issues: Increasing the value of shared datasets (Force11, RDA) Tracking the usage of datasets using altmetrics Making data and software accessible (documented access route, or conditions of access) Ensuring data and software is useable Facilitating appropriate recognition for scientists (e.g. citable peer review) Addressing data storage issues Repository accreditation (PREPARDE project) Data format and software depreciation
