Advocating for change in the scientific enterprise

Advocating for change in the scientific enterprise

Jessica PolkaDirector, ASAPbio

Visiting Scholar, Whitehead InstituteVisiting Fellow, Harvard Medical School

Twitter: @jessicapolka

The ivory tower is not a bubble

• Interactions with other scientists can be intensely political

• Dependent on public support/funds

• Profoundly influenced by policies and infrastructure

Daniel Parks/flickr

The way we do science is driven by social and political incentives

What we measure

• “Impact” (ie Journal Name/IF)• # of papers• Prestige of institution/mentors

What we don’t measure

• Reproducibility• Openness/sharing• Mentoring

Are these structural forces driving us to do the best science possible?

How did we get here?

1945

The Wartime DeficitWith mounting demands for scientists both for teaching and for research, we will enter the post-war period with a serious deficit in our trained scientific personnel.….for it takes at least 6 years from college entry to achieve a doctor's degree or its equivalent in science or engineering.

WE MUST RENEW OUR SCIENTIFIC TALENT

Centers of Basic ResearchPublicly and privately supported colleges and universities and the endowed research institutes must furnish both the new scientific knowledge and the trained research workers.

Each year under this program 6,000 undergraduate scholarships would be made available to high school graduates, and 300 graduate fellowships would be offered to college graduates.

The US research enterprise is built on a model of continuous expansion

Times have changed

9

Academia is not diverse & inclusive enough

Gibbs et al 2014

Age at independence has increased

Grant success rates are going down

Ron Vale, bioRxiv/PNAS 2015

More data now required to publish a paper

Retractions are increasing

https://nsaunders.wordpress.com/2010/11/30/analysis-of-retractions-in-pubmed/

The environment is “hypercompetitive”

What has changed?

Less money, more people

Research funding grew, then shrank

Center for American Progress

“The doubling”

https://nexus.od.nih.gov/all/2016/05/31/how-many-researchers/

More applicants for research grants (but not more awardees)

And yet, more graduate students are enrolling than ever

Each PI is now graduating more students

Ghaffarzadegan et al 2015NSF data via FASEB

http://www.theatlantic.com/business/archive/2013/02/the-phd-bust-americas-awful-market-for-young-scientists-in-7-charts/273339/

https://loop.nigms.nih.gov/2010/09/measuring-the-scientific-output-and-impact-of-nigms-grants/

But big labs are inefficient

Is biomedical research in trouble?

Solutions

So, what should you & I do?

Scientific citizenship• Awareness

• Discussion

• Organizing

• Action

• Datahound• Drugmonkey• Small Pond Science• Woman of Science

• Local groups• Scientific societies

• Communicate with congress – CLS, AFS• NIH RFIs

• Twitter

Organizing - Postdocs taking a seat at the table

Future of Research (FOR)

• ~9 meetings across the US and Canada• Now a Massachusetts non-profit with a full-

time Exec Director, Gary McDowell• Developing resources for postdocs (FLSA, etc)

and students• Amplifying voices of postdocs through

advocacy

Organizing - Scientific societies• The forum for scientific discussion• Leaders of culture change (see DORA)• Centers for science policy and communication• Interested in modernizing & the capturing the next generation

Communicating with congress

Communicating with congress

NIH Request for Information (RFI)

195 responses!

Zach Chisholm, flickr

Picking a problem

Publication is essential to scientific progress

Adapted from http://asapbio.org/survey

Publishing has changed

Ron Vale, bioRxiv/PNAS 2015

What to do about it?

Problem: fast and open venues are not ‘impactful’ venues

A preprint is a manuscript posted online before journal-organized peer review

Preprints & journals are compatible

Berg et al Science 2016

arXiv: 100,000 manuscripts per year

Preprint servers have existed for 25 years

In Biology

Jan-03

Jul-03Jan

-04Jul-0

4Jan

-05Jul-0

5Jan

-06Jul-0

6Jan

-07Jul-0

7Jan

-08Jul-0

8Jan

-09Jul-0

9Jan

-10Jul-1

0Jan

-11Jul-1

1Jan

-12Jul-1

2Jan

-13Jul-1

3Jan

-14Jul-1

4Jan

-15Jul-1

5Jan

-16Jul-1

60

100

200

300

400

500

600

700

800

900

1000

arXiv (q-bio w/cross-lists, from arxiv.org stats)bioRxiv (from bioRxiv)PeerJ Preprints (bio/med/life)F1000 ResearchThe WinnowerNature Precedings (manuscripts, from search results)Preprints.org (articles/reviews in bio/life/med)figshare (filtered by PrePubMed)

Version 1 | asapbio.org

Preprints are taking off in biology

• Benefits of preprints• Concerns surrounding preprints• Taking action• Recent updates

Lack of access to literature

Current problem

Preprints are immediately available to everyone around

the world

Preprints are publicly disclosed work that can be evaluated for a

PhD thesis, postdoc positions, fellowships, or jobs.

Training periods are long; students/postdocs wait to publish

Current problem

Preprints are public documents that enable committees to see the most recent work of an

applicant.

Recent work is “invisible” to grant and promotion committees

Current problem

The immediate visibility of preprints enables invitations to meetings, new

collaborations, etc.

Your most recent work is also invisible to your colleagues

Current problem

Also more feedback on your manuscriptthan 2-3 anonymous peer reviewers

Lack of transparency and length of review creates difficulties for establishing priority of discovery

Current problem

Preprints have a time stamp and DOI number; public evidence of what work was done when

Information is being held within laboratories for longer periods of time

With preprints, new knowledge is immediately

accessible, allowing research overall to advance.

Current problem

Concern: We can’t be trusted to share our work before peer review

• Reputation is importantFlickr/NASA Goddard

Concern: Journals won’t accept my preprintNature and Nature journals, Science,PNAS,Cell,eLife,J. Cell Biology, EMBO,ASM journalsOxford Press journals,J. Biol. ChemistryMBoCGeneticsJ. Neuroscience…….

Search Wikipedia: list of academic journals by preprint policyContains links to original policies

Concern: How can we ensure ethical disclosure of data?Preprint servers should (and already do!):• Screen for human subjects

research• Ensure that authors agree to

posting• Expect that methods are

present and complete

Concern: How should preprints be covered in the media?

Cell phones & cancer Vaccines & autism

Concern: I’m going to get scooped

http://asapbio.org/preprint-info/preprint-faq

Paul Ginsparg, founder of arXiv on scooping:

“It can’t happen, since arXiv postings are accepted as date-stamped priority claims.

Eventually I came to understand that biologists do not use “scoop” in the standard journalistic sense… Instead “scooping” in the context of biology research appears to mean using information or ideas without proper attribution.”

http://asapbio.org/drafts/draft1

Draft statement on disclosing & crediting scientific work

“As responsible citizens of the scientific community, we...will fairly cite original work presented as a preprint in our own scientific papers, just as we would cite a journal publication. We will acknowledge such work, as appropriate, in our presentations at scientific meetings.”

ie: preprints are public but not obviously well-respected

Posting preprints is a good experience

392 responses. Results at asapbio.org/survey

Accelerating Science and Publication in

biology

Feb. 16/17, 2016 at HHMI Headquarters

Strong consensus that broader use of preprints could become a valuable addition to the journal

system(Organizers: Daniel Colόn-Ramos, Jessica Polka, Harold Varmus, Ron Vale)

Moving preprints forward

Scientists

University Promotion Committees

Journals

Funding Agencies

.org

Encouraging the productive use of preprints

• Visibility• Network effects • Easy to find

• Standards• Screening• Citation• Preservation, access, licensing

• Policies• Funders• Journals• Institutions

#ASAPbio

ASAPbio Ambassadors

Encouraging the productive use of preprints

• Visibility• Network effects • Easy to find

• Standards• Screening• Citation• Preservation, access, licensing

• Policies• Funders• Journals• Institutions

A new kind of marketplace for papersOctober 4, 2016

UCSC & The Rockefeller University job adsSept 26 2016

In the interests of accelerating scientific discovery, the Biohub will establish a publication policy for open and rapid dissemination of research results: all Investigators will be required to post manuscripts on Arxiv on the date of submission to peer-reviewed journals.

https://med.stanford.edu/rmg/funding/chan_zuckerberg.html

If a scientist wants to cite an interim research product in an NIH application or report, the citation should meet certain standards.  These standards might include:

• Ensuring the document is preserved, findable, and freely accessible to people and machines

• Links to other versions and associate data and resources

• Attribution and disclosure of authorship, funding, competing interests, licensing, and other issues used in high-quality scholarly publication

• A clear statement that the product is preliminary, and the level of peer-review it has received (if any)

Note, NIH does not intend to require awardees to create interim research products. 

Encouraging the productive use of preprints

• Visibility• Network effects • Easy to find

• Standards• Screening• Citation• Preservation, access, licensing

• Policies• Funders• Journals• Institutions

Proposing community-governed infrastructure (like PubMed Central) for preprints

Layers

Overlay journals

Annotation layers

Robots are going to steal our jobs (and this is wonderful)

Nanopub.org

Subject, object, predicate

Blog post on scholarly kitchen

Only ~6% of articles can be practically reused

Total articles (PubMed): 24.5 million articles

Free to read (PMC): 4 million articles

Open to download/reuse (OA PMC subset): 1.4 million articles

Thank youASAPbio Co-organizersRon Vale (UCSF)James Fraser (UCSF)Daniel Colόn-Ramos (Yale)Harold Varmus (Cornell)

ASAPbio FundingSimonsSloanArnoldMoore

Jessica.polka@gmail.com, @jessicapolka

Mentors etcPam SilverIain CheesemanASCB

FOR FundingOpen Philanthropy Project

FOR colleaguesKristin KrukenbergSarah MazilliGary McDowellDavid Riglar& many others