Our Scholarship System is Broke. Can Open Access Fix It?

[email protected] of Psychology

@ceptional

Our Scholarship System is Broke.

Can Open Access Fix It?

http://www.slideshare.net/holcombea/

1Wednesday, 24 October 12

mailto:[email protected]




Academic knowledge is boxed in by expensive journals.

Scientist meets Publisher

http://www.youtube.com/watch?v=GMIY_4t-DR0




Claudio Aspesi at http://poynder.blogspot.com.au/2012/03/open-access-brick-by-brick.html

$3983 USD per article for Elsevier $1350 USD per article for PLoS ONE

operating profit company industry7% Woolworths supermarkets, pokies12% BMW automobiles22% Coca-Cola adding sugar to water23% Rio Tinto mining36% Apple premium computing34% Springer scholarly publishing36% Elsevier scholarly publishing42% Wiley scholarly publishing

Thanks to Nick Scott-Samuel


http://poynder.blogspot.com.au/2012/03/open-access-brick-by-brick.html


$10,780 per article (not including charges

for color figures)

$85 per page

$80 per page (introductory rate is even cheaper)

$1350 per article

JOURNAL / PUBLISHER COST ($USD) ACCESS

Subscription

Open Access

Open Access

Open Access


$10,780 per article (not including charges

for color figures)

$85 per page

$80 per page (introductory rate is even cheaper)

$1350 per article

JOURNAL / PUBLISHER COST ($USD) ACCESS

Subscription

Open Access

Open Access

Open Access

$99 per life Open Access


Monopoly + = = $Profit maximization

Broke


“Open Access Hulk”

OA HULK WANTS TO KNOW WHO TO OCCUPY!

ELSEVIER!? ACS!? HARPERCOLLINS!?

YOU NAME IT, OA HULK WILL OCCUPY AND SMASH!


Claudio Aspesi at http://poynder.blogspot.com.au/2012/03/open-access-brick-by-brick.html

$3983 USD per article for Elsevier $1350 USD per article for PLoS ONE

started January 2012




2007

2008

2009

2010

2011

2012


2007

2008

2009

2010

2011

2012


2007

2008

2009

2010

2011

2012


•Deposit your manuscripts in the university repository (http://ses.library.usyd.edu.au/

•Even with closed journals, you often have the right to deposit your final version (e.g. Word document before typeset by publisher)

•Funders, universities should mandate this.

•Publishers will adapt, as they have in physics.

GREEN ROAD

Stevan Harnad

I have the sense that researchers are under awful pressures to present two faces to the world, sometimes alternately, and sometimes simultaneously. On the one hand, they are supposed to be impartial, with no axe to grind. On the other hand, they are supposed to be, not only experts in experimental methodology, higher ed pedagogy, grant-writing, and the like, but also formulating strong a priori hypotheses that turn out to be robustly supported by evidence and, all other things being equal, happen to satisfy the interests of funding sources and journal editors. In other words, they are supposed to be excellent axe-grinders.

GOLD ROAD

Article Processing Charge

$3,000

$1,350


http://ses.library.usyd.edu.au

http://ses.library.usyd.edu.au

Requirements from funders that publications be OA

•NIH (US) within 12 months

•Wellcome Trust (UK) within 6 months

•final grant payment withheld if you don’t comply

•NHMRC (Australia) within 12 months

•“publications arising from an NHMRC supported research project must be deposited into an open access institutional repository within a twelve month period from the date of publication.”

•ARC (Australia)

•You can use DP funds to pay open-access fees, but must be taken from the funds you were awarded to pay for other things.

•“Strongly encourages” open access, no teeth. Compliance rate very low.


Open Data: the next step

NHMRC: The next steps will be improving public and other researchers’ access to publicly funded data.

https://theconversation.edu.au/all-research-funded-by-nhmrc-to-be-accessible-free-of-charge-5486




Open Data: the next step

NHMRC: The next steps will be improving public and other researchers’ access to publicly funded data.





The Replicability Crisis

Rule among early-stage venture capital firms that “at least 50% of published studies, even those in top-tier academic journals, can't be repeated with the same conclusions by an industrial lab” - Prinz, Schlange, & Asadullah. Nature Rev. Drug Discov. 10, 712 (2011)

Bayer HealthCare :only about 25% of published preclinical studies could be validated to the point at which projects could continue

Amgen Fi!y-three papers were deemed ‘landmark’ studies (see ‘Reproducibility of research "ndings’)... scienti"c "ndings were con"rmed in only 6 (11%) cases

Why science is self-correcting

There's no point in scientific misconduct; it is always found.

Published on August 10, 2010 by Art Markman, Ph.D. in Ulterior Motives

Because scientists are always repeating each other's experiments, it is hard for a

fictitious result to hang on for very long.


http://www.psychologytoday.com/experts/art-markman-phd










http://www.psychologytoday.com/blog/ulterior-motives









The File-Drawer Problem

unpublished results

files

http://www.flickr.com/photos/nickperez/2569423078 t. magnum




http://www.flickr.com/photos/nickperez/2569423078


The File-Drawer Problem•Difficult to publish non-replications and replications

•Most journals only publish papers that “make a novel contribution”

•Reviewers/editors tend to hold non-replicating manuscript to higher standard than original.

•Bem

•Little career incentive to publish a non-replication or a replication


unpublished results

files













Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes ina scientific field, the less likely the research findings are to be true. Flexibility increases the potential for transforming what would be “negative” results into “positive” results.

Corollary 6: The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.








Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes ina scientific field, the less likely the research findings are to be true. Flexibility increases the potential for transforming what would be “negative” results into “positive” results.

Corollary 6: The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.



“In summary, while we agree with Ioannidis that most

research findings are false...”









Barriers to publishing replications and failed-replications

• No glory in publishing a replication

• Few journals publish replications

• usually uphill battle even with those that do

• The wrath of the original researcher


























File-drawer fixes

• Journals that don’t reject replications for being uninteresting or unimportant

• Pre-registration of study designs and analysis methods

• Brief reporting of replications

✔•◦

✔

◦ ◦

◦◦

✔


File-drawer fixes

• Journals that don’t reject replications as being uninteresting or unimportant

• Pre-registration of study designs and analysis methods

• Brief reporting of replications

✔•◦

✔

◦ ◦

◦◦

✔


preregistered Replication Reports

1. Authors plan a replication study

2. They submit an introduction and methods section

3. Sent to reviewers, including author of to-be-replicated article

4. Editor decides whether to accept/reject, based on:

1. Reviewer comments regarding the proposed protocol

2. Importance of original study, judged by argument in the introduction, number of citations of original, reviewer comments

5. The Intro, Method and analysis plan, and reviewer comments are posted on the journal website

6. After the results come in, the authors submit a conventional results and discussion section and that together with the raw data are posted, yielding the complete publication

1. some sort of minimal peer review needed for that. What exactly?

✔

Dan Simons

✔✔


• Original author signed off on it, so can’t complain / hate the replication authors as much.

• Good way to start for a new PhD student, anyone planning to build on some already-published results

• Will post the raw data

• Will facilitate, publish meta-analyses when replications accrue

• Reduce the incentive to publish flashy, headline-grabbing but unreliable studies?

✔✔✔

preregistered Replication Reports

at Psychological Science?


Documents

Our Scholarship System is Broke. Can Open Access Fix It?