-
Trusted evidence.Informed decisions.Better health.
PeerreviewforCochraneReviewsJohnHilton,BryonyUrquhart,HarrietMacLehose,SallyBell-Syer,MonazMehta,SeraTort
CochraneEditorialUnit
-
Disclosures of conflicts of interest
JohnHilton,BryonyUrquhart,HarrietMacLehose,SallyBell-Syer,MonazMehta,SeraTort
EmployedbyCochrane
-
Overview
• Background:Why?How?When?
• Policydevelopmentprocess
• Whatthepolicycovers
• Somekeyareasofpolicy
• ReviewGroupsurvey
• What’snext
• What’shappeninginpeerreview?
-
Background•
CochraneEditorialUnitsupportsCochraneReviewGroups,andworks
toimprovequalityandtheCochraneLibrary
• CRGShavepolicies,butnoCochrane-widepolicyonpeerreview•
Policytobesupplementedwithsupportingguidance•
Peerreviewisahottopic:isitdoomedorthriving?
-
Whydoweneedapolicy?
FORPEER-REVIEWERS
Toprovideaframeworkforpeer-reviewersworkingacrossCRGs.
FORUSERS
ToprovideclarityaboutCochrane’speerreviewprocess,bothingeneralandforspecificreviews.
FOREDITORIALTEAMS
Toprovideclearexpectations,standardsandguidanceformanagingthepeerreviewprocess.
FORAUTHORS
ToprovideclarityaroundthepeerreviewprocessforauthorsembarkingonaCochraneReview.
-
What’sdifferentaboutCochrane?•
‘Peerreferee’versus‘peerreview’
• Consumers
• MultipleReviewGroups
• Peerreviewnotusuallyaccept/reject
• CochraneReviewsandlongandcomplex
• Evidence-based
Butingeneralthesameprinciplesandsamechallengesasfacedbyauthors,editorsandpeerreviewerseverywhere.
-
Editorialpolicydevelopment
Topicidentified Smallworkinggroup
Advisorygroup
Execs Sign-offandEPPR
-
Peer review policy developmentWorkshop
DraftpolicydevelopmentAdvisoryGroupformation
PolicyrevisionSurvey
DraftpolicycompletedConsultation
Revision,Sign-off,PublicationCommunicationImplementation
Audit
Oct 2015
Nov 2016
-
What’sincluded
• Typeofpeerreview
• Numberandexpertiseofpeerreviewers
• Declarationsofinterestforpeerreviewers
• Acknowledgementandcreditforpeerreviewers
• Peerreviewturnaroundtime
• Ensuringthatauthorsaddressreviewers’comments
• Feedbacktopeerreviewers
• Invitingpeerreviewerstobeauthors
• Editorialroles
-
What’sincluded(continued)
• Peerreviewchecklists
• Peerreviewerconductandethics
• Conflictresolution
• ReviewGrouppeerreviewpoliciesandprocedures
• Researchintopeerreview
• PLUSsupportingguidance
-
What’snotincluded
• Post-publicationpeerreview
• Open(published)peerreview
-
Open (named) peer
reviewShouldpeerreviewersidentitiesbeknowntoauthorsandotherpeerreviewers?
PROS CONS
Transparency Somereluctance
Openscience Lesshonest
Lessone-sided Biases(+and-)
Betterreviews? Blandreviews?
Interestsexposed Reluctance
Evidence?
Limitedandseemstovarybetweenjournalsandsettings.
-
Reviewers:howmanyreviewersandwhatexpertise?
•
Atleastoneclinical/topicspecialist,withaminimumofoneexternaltotheCRG.
•
PLUSonestatistician/methodologist(ifthereviewdeviatesfromstandardmethodsorusescomplexmethods).
•
Aimtoinclude(ifrelevant)atleastoneconsumer(oruser)peerreviewer
-
Peerreviewofupdates• Fullorselectivepeerreview?•
Alwaysneedpeerreview?• What’schanged?
>>Decisionflowchart
-
Publishedpeerreviewpolicies
-
Survey of Review Groups• Informimplementationofnewpolicy•
Helptoidentifywhereadditionalguidancewouldhelp• Sharebestpractice•
Informfuturepolicydevelopment
50ReviewGroupsresponded.
-
Survey:summaryoffindings•
AllCRGscarryoutpeerreview;abouthalfhaveadocumented
policy
• Open(named)peerreviewiscommonlyused
•
MostCRGsusethestandardCochranepeerreviewforms,oramodifiedversion
• MostCRGspublishthenamesoftheirpeerreviewers
•
Feedbackisoftenprovidedtopeerreviewers;opportunityforfurtherincentives
•
Themostchallengingaspectsofpeerreviewrelatetotime:gettingreviewerstorespondtotheinvitation,thetimeittakespeerreviewerstoreturncomments,findingspecialistpeerreviewers,andoverallthetimeittakestoorganisepeerreview.
-
ImplementationPublish &
communicate
Support
Monitor, feedback &
audit
Additional support or
modify policy
Policy updates
-
What’snext?Draftpolicyfinalisation
Policyconsultation,viaReviewGroupExecutives
Policyrevision andsign-off
CochraneEditorialandPublishingandPolicyResourceupdatedandcommunicated
Webinars,presentations,guidance
Planformonitoring,auditandfeedback
-
SupportingguidanceHowtofindpeerreviewers
Howtokeeppeerreviewers
Howtoimprovequalityofpeerreviews
Howtospeeduppeerreview
Howtoinvolveconsumersandusers
Howtohandleethicalissues
Introducingnewpeerreviewprocesses
Managingfeedback
-
Opportunities for research! Within groups; between groups;
comparing with non-Cochrane
Research
-
Thankyou
PeerReviewAdvisoryGroup:
DeirdreBeecher,RuthBrassingtonChrisEccleston,KarenRobinsonSusanWieland,CarolineStruthersMelinaWillson,RichardWormald
CRGstaffwhorespondedtothesurvey
-
Monument to an anonymous peer reviewer
https://www.kickstarter.com/projects/972533097/monument-to-an-anonymous-peer-reviewer/description
