Presentation given at Open Science question and answer session hosted by the Institute for Quantitative Social Science (IQSS), and the Office for Scholarly Communication (OSC) at Harvard University, on July 16th 2014.
Text of Open science: your questions answered
OPEN SCIENCE: YOUR QUESTIONS ANSWERED Varsha Khodiyar Data
Publishing Manger, F1000Research @vkf1000 Michael Markie Associate
Publisher, F1000Research @mmmarksman f1000research.com
@f1000research
OPEN SCIENCE PUBLISHING MEANS: Open access articles Open peer
review Open data Open software Full research provenance Open
science is the movement to make scientific research, data and
dissemination accessible to all levels of an inquiring society,
amateur or professional.
ABOUT F1000RESEARCH THE FIRST OPEN SCIENCE JOURNAL The Seer of
Science Publishing Science 4 October 2013: Vol. 342 no. 6154 pp.
66-67 DOI: 10.1126/science.342.6154.66
http://www.sciencemag.org/conte nt/342/6154/66.full.pdf
http://blog.f1000.com/2013/10/04/vitek-
tracz-science-interview/
WHAT IS F1000RESEARCH? F1000Research is an open science journal
for life scientists that accepts all scientifically sound articles,
ranging from single findings, case reports, protocols,
replications, and null or negative results to more traditional
articles. Key features: Publication within a week Transparent,
post-publication peer review All data included Accepts
non-traditional article types
OPEN PEER REVIEW
TYPES OF PEER REVIEW Time of review: Before publication:
mediated by each individual journal Cascading review: reviews
carried over to the next journal after rejection Third-party
review: the peer review is no longer coupled to a journal.
Post-publication peer review: journal publishes the article, then
reviewers look at it. Transparency of review: Single-blind: the
reviewer knows who the authors are, but the authors dont know who
the reviewers are Double-blind: authors and reviewers are both
anonymous Open peer review: all names are public. See:
http://www.britishecologicalsociety.org/publications/journals/ for
examples of each
TRADITIONAL PEER REVIEW Time of review: Before publication:
mediated by each individual journal Cascading review: reviews
carried over to the next journal after rejection Third-party
review: the peer review is no longer coupled to a journal.
Post-publication peer review: journal publishes the article, then
reviewers look at it. Transparency of review: Single-blind: the
reviewer knows who the authors are, but the authors dont know who
the reviewers are Double-blind: authors and reviewers are both
anonymous Open peer review: all names are public. See:
http://www.britishecologicalsociety.org/publications/journals/ for
examples of each
TRADITIONAL PUBLISHING AND PEER REVIEW TIMELINE
ISSUES WITH TRADITIONAL PEER REVIEW SYSTEM Lack of transparency
- Who are the reviewers? - What happened with this paper before it
was accepted? Lack of accountability - Anonymous reviews -
Editorial decisions may not reflect reviews Inefficiency -
Re-reviewing the same work at different journals Delays incidental
(reviewing takes time) deliberate (reviewers delaying competitor
papers) Cartoon by Nick D Kim, strange-matter.net
A POST-PUBLICATION APPROACH TO PEER REVIEW
REFEREE SCORES Approved Approved with reservations Not approved
Articles with sufficient positive evaluations are indexed in
PubMed, Scopus, and Embase. or Minimal requirements for
indexing
OPEN PEER REVIEW AND OPEN COMMUNITIES Referee reports and other
comments are visible to anyone. Community input
BENEFITS OF TRANSPARENT REVIEW FOR AUTHORS AND READERS Visible
discussion between referees and authors (and editors) puts paper in
context. Referees are good at spotting broader significance of an
article. Shows the back story of a paper. (e.g. Why did it take 3
rounds of review? Authors can demonstrate that their paper was
reviewed by top people in their field. Reduces bias amongst
referees Educational aspect of open peer review: Open referee
reports can serve as examples. Demonstrates differences between
reviewers
BENEFITS FOR REVIEWERS Get a DOI: Take credit for hard work
Demonstrate experience as reviewer Shows reviewers informed opinion
of the work as a peer in the field, and where they thought it could
be improved. Especially relevant in borderline cases, where an
article just barely passed review. Heres what the community think:
http://vimeo.com/99777547.
WHY SHARE DATA AND CODE?
WHY SHARE YOUR DATA? Transparency and openness are cornerstones
of the scientific method Not allowing reuse of data is scientific
malpractice Royal Society; Science as an open enterprise, Final
report 2012 http://royalsociety.org/about-us/history/
SHARING DATA CORRELATES WITH HIGHER CITATIONS We conclude there
is a direct effect of third-party data reuse that persists for
years beyond the time when researchers have published most of the
papers reusing their own data...We further conclude that...a
substantial fraction of archived datasets are reused, and that the
intensity of dataset reuse has been steadily increasing since 2003.
Piowar HA., Vision TA. Data reuse and the open data citation
advantage. PeerJ (2013) doi: 10.7717/peerj.175
SHARING DATA ADDITIONALLY PROMOTES Diversity of analyses and
opinion New research testing of new hypotheses new analysis methods
meta-analyses to create new datasets studies on data collection
methods Reduction of error and fraud Education of new researchers
Increased return on investment in research
PUSH TOWARDS DATA SHARING
SHARING DATA ALLOWS REPLICATION [W]e evaluated the replication
of data analyses in 18 articles on microarray-based gene expression
profiling published in Nature Genetics in 20052006...We reproduced
two analyses in principle and six partially or with some
discrepancies; ten could not be reproduced. The main reason for
failure to reproduce was data unavailability. Ioannidis JPA. et al.
Repeatability of published microarray gene expression analyses.
Nature Genetics 41, 14955 (2009)
RESEARCH BECOMES HARDER TO ACCESS WITH AGE We examined the
availability of data from 516 studies between 2 and 22 years old
The odds of a data set being reported as extant fell by 17% per
year Broken e-mails and obsolete storage devices were the main
obstacles to data sharing Policies mandating data archiving at
publication are clearly needed Vines TH. et al. The availability of
research data declines rapidly with article age. Curr Biol 24, 947
(2014)
MAKING DATA AND CODE TRULY OPEN
FULL DATA INTEGRATION WITH RESEARCH PAPERS
DATA ARTICLES A dataset (or set of datasets) together with the
associated methods/protocol used to create the data. No analysis of
the data, results or conclusions should be included.
http://f1000research.com/author-guidelines#data-art-sub One goal we
had for publishing this Data article in F1000Research was to
quickly share some of our ongoing behavioral datasets in order to
encourage collaboration with others in the field. Donald Cooper,
University of Colorado, Boulder
http://f1000research.com/articles/2-53/v1
SOFTWARE AVAILABILITY SECTION
DATA PLOTTING TOOL
http://blog.f1000research.com/2013/11/11/data-plotting/
DATA PLOTTING TOOL
ONGOING PROJECTS AT F1000RESEARCH F1000Research is involved in
discussions with institutional librarians, researchers and other
journals concerning data publication and sharing issues: Increasing
the value of shared datasets (Force11, RDA) Tracking the usage of
datasets using altmetrics Making data and software accessible
(documented access route, or conditions of access) Ensuring data
and software is useable Facilitating appropriate recognition for
scientists (e.g. citable peer review) Addressing data storage
issues Repository accreditation (PREPARDE project) Data format and
software depreciation
