View
57
Download
1
Category
Tags:
Preview:
Citation preview
Brighton and Sussex NHS LKS is part of Brighton and Sussex University Hospitals NHS Trust
Librarian co-authors correlated with higher quality reported search strategies in generalinternal medicine systematic
reviews
Tom RoperJournal Club, 15 July 2015
Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in generalinternal medicine systematic reviews. J Clin Epidemiol. 2015 Jun;68(6):617-26. doi: 10.1016/j.jclinepi.2014.11.025
Systematic review reporting standards • 1999: QUORUM (QUality Of Reporting Of Meta-
analyses)• 2000: MOOSE (Meta-analysis Of Observational
Studies in Epidemiology)• 2009:PRISMA (Preferred Reporting Items of
Systematic reviews and Meta-Analyses)• 2007: AMSTAR (Assessment of Multiple Systematic
Reviews)• 2011: IOM Finding What Works in Health Care• 1996 onwards: Cochrane Handbook Chapter 6 (last
revision 2011)• 2012: Methodological Expectations of Cochrane
Intervention Reviews standards (MECIR)
What did the authors want to do?• SR searches poorly reported, if at all, and
rarely reproducible
• The research question: The purpose of this study was to determine whether LIS authorship was associated with better SR search quality and reporting
How did they do it?
• Took highest impact journals publishing SRs: Annals of Internal Medicine, BMJ, JAMA, Lancet, and PLoS Medicine (not NEJM)
• All SRs published between 2008 and 2012 identified:
• (search*[tiab] OR meta-analysis[Publication Type] OR meta-analysis[tiab] OR MEDLINE[tiab] OR (systematic[tiab] AND review[tiab])) OR systematic [sb]) AND (‘‘Lancet’’[Journal] OR ‘‘JAMA’’[Jour- nal] OR ‘‘Ann Intern Med’’[Journal] OR ‘‘PLoS Med’’[Journal] OR ‘‘BMJ’’[Journal])
One of three reviewers scrutinised papers Criteria: – article states it is a SR in ti or ab OR– article states it is a meta-analysis and methods
include literature search OR– methods include reported extensive search of the
literature and list inclusion and exclusion criteria
Short form to record LIS involvement– no or unclear LIS participation– mentioned or acknowledged LIS– LIS author
• Checklist based on the 15 IOM Recommended Standards for the Search Process and
• Modified version of the Peer Review of Electronic Search Strategies (PRESS) instrument
• Blinding: strategies stripped and loaded into a Google doc
Flow diagram. IOM, Institute of Medicine; PRESS, Peer Review of Electronic Search Strategies.
Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviewsRethlefsen, Melissa L., Journal of Clinical Epidemiology, Volume 68, Issue 6, 617-626
Copyright © 2015 Elsevier Inc.
Results
Systematic reviews and searches
None or unclear
Mentioned or acknowledged
Author
Number of systematic reviews (%)
438 (69.5) 147 (23.3) 45 (7.1)
Number of reproducible searches (% of searches reproducible)
164 (37.4) 82 (55.8) 29 (64.4)
Analysis of means for proportions of search reproducibility by levels of librarians and information specialists (LIS) involvement. Search reproducibility proportions significantly differ from the mean proportion of reproducibility in all three levels of LIS participation. Where the incident rate lines cross the decision limits, significant difference from the mean is demonstrated.
Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviewsRethlefsen, Melissa L., Journal of Clinical Epidemiology, Volume 68, Issue 6, 617-626
Copyright © 2015 Elsevier Inc.
Box plot of Peer Review of Electronic Search Strategies score by level of librarians and information specialists involvement.
Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviewsRethlefsen, Melissa L., Journal of Clinical Epidemiology, Volume 68, Issue 6, 617-626
Copyright © 2015 Elsevier Inc.
• Overall, LIS co-authored SRs had significantly higher odds of meeting 8 of the 13 analyzed IOM Standards than SRs with no LIS
Discussion
• LIS co-authored SRs– much higher quality, both in terms of the search strategy
itself and search strategy reporting, than SRs with a mentioned LIS
• more likely to hand-search• search for non-English studies• search citation indexes• subject databases• regional databases• gray [sic] literature
– better use of natural language terms – better translation of question into search strategy
• one negative correlation: updating (IOM 3.1.7): update the search at intervals appropriate to the pace of generation of new information for the research question being addressed
• a large number of SRs are published without a strategy, much less a reproducible one.
• notable number of PubMed search strategies only use keywords with no truncation or field codes — completely eliminates reproducibility
• peer reviewers and journal editors don’t review published search strategies carefully
Recommendations
• Search strategy peer review• Narrative explanations of search choices and
construction • Journal editors to consider having search strategies
reviewed for SRs • Editors to suggest SR authors receive and
acknowledge LIS help with SR searches and use the presence of LIS involvement as quality criterion
• NB Five previously reproducible search strategies disappeared from publisher Web sites and other locations during study
Limitations
• only fve journals
• what’s an SR?
• what’s a librarian?
• no assessment of precision and recall of SR strategies
• disagreement between IOM and AHRQ on standards:– AHRQ said no evidence for LIS involvement in
review of search strategy
Why does this matter?
• How many SRs do we do? Search for ‘systematic review’ in title of evidence search in KS = 17 hits
• Can we extrapolate from SR searches to other searches?