Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
SYSTEMATIC REVIEW ACCELERATOR:
THE POLYGLOT SEARCH TRANSLATOR
Justin Clark
Centre for Research in Evidence Based
Practice (CREBP)
Bond University, Gold Coast, Australia
SR tasks – full list of tasks and toolsTask No. Classification Task Description Automation tool
1 Preparation Formulate review
question
Decide on the research question of the
review.
SRA – TRIP data analysis
2 Preparation Find previous or
upcoming SR
Search for a SR that answers the same
question.
Cochrane library, Epistemonikos, PROSPERO
3 Write up Write the protocol Provide an objective, reproducible, sound
methodology for peer review.
Reporting guidelines, existing protocol templates
4 Preparation Devise search strategy Decide on databases and keywords to find
all relevant trials.
SRA - Registered Trials Finder (in development), SRA - Word Frequency
Analyser
5 Retrieval Search Aim to find all relevant citations even if
many irrelevant ones included.
SRA - Polyglot Search Translator
6 Retrieval De-duplicate Remove identical citations. SRA - De-duplicator. Rayyan
7 Appraisal Screen abstracts Based on titles and abstracts, remove
definitely irrelevant citations.
SRA - Endnote Helper, Covidence, Rayyan
8 Retrieval Obtain full text Download, request copies from authors,
inter-library loan, etc.
Endnote - find full text feature, SRA – Endnote Helper
9 Appraisal Screen full text Exclude irrelevant citations. SRA - Endnote Helper, Covidence
10 Retrieval Citation analysis Follow citations, cited and citing, from
included trials to find additional trials.
SRA - SpiderCite (in development)
11 Synthesis Extract data Extract outcome numbers and associate
with trial arm
Robot Reviewer
12 Synthesis Synthesize data Convert extracted data to common
representation (usually average and SD).
Revman
13 Retrieval Recheck literature Repeat the search to find new literature
published since the initial search.
SRA - Polyglot Search Translator (future development)
14 Synthesis Meta-analyze Statistically combine the results from all
included trials.
RevMan, MetaXL
15 Write up Write up review Produce and publish the final report SRA – Revman Replicant
Polyglot Search Translator (PST)
• The PST translates either a PubMed or Ovid Medline
search strategy to the correct syntax for different
databases
Polyglot Search Translator
• Copy your PubMed or Ovid Medline search and paste it in to the ‘Your Query’
box
• To see how the search translates across databases, click the arrow next to
the name of the database.
Polyglot Search Translator
• For some databases you will need to convert the subject terms (e.g. change
MeSH terms to Emtree terms)
Polyglot Search Translator
• The PST is designed to make searching databases easier, BUT it does not
remove the need to have knowledge of the process
Medline Ovid to Cochrane Central
Evaluation of the Polyglot Search Translator a tool for
improving searches in systematic reviews: a multi-centre
randomized crossover trial
• Clark J1, Carter M1, Honeyman D2, Cleo G1, Auld Y3, Booth D4, Condron P5, Dalais C6, Dern S2, Linthwaite B3, May N7, Munn J8, Ramsay L9, Rickett K6, Rutter C10, Sanders S1, Smith A11, Sondergeld P10, Wallin M11, Beller E1
• 1 Centre for Research in Evidence Based Practice, Bond University, Australia
• 2 Bond University Library, Bond University, Australia
• 3 Gold Coast Health Library Service, Gold Coast University Hospital, Australia
• 4 University Library, University of Newcastle, Australia
• 5 University Library, University of Melbourne, Australia
• 6 University Library, University of Queensland, Australia
• 7 Sturt Library, Flinders University, Australia
• 8 Centre for Teaching and Learning, Southern Cross University, Australia
• 9 University Library, Australian Catholic University, Australia
• 10 University Library, Queensland University of Technology, Australia
• 11 Hunter New England Health Libraries, NSW Health
• 12 University Library, Southern Cross University, Australia
Background
• Searching the literature to identify relevant studies is an
important part of a SR
• Translating the search to run in multiple databases is
needed for a good SR
• To improve the speed of this the Polyglot Search
Translator (PST) was developed.
Objectives
• To determine whether a PST-assisted search translation
improves the speed without an increase in errors when
translating search strings from one database to another.
Methods
• Participants (n=20) were recruited, 4 dropped out leaving
16 (10 of whom completed all 10 translations)
• Participants were randomly allocated ten search
strategies from a pool of 20
• They were then randomly assigned to either use the PST
to help translate it, or to not use it
• We compared how long it took, how many errors were
made and how many search results they found
Results
• A PST assisted search strategy translation was faster
with, a mean time saving of 14 minutes and 18 seconds
per translation.
• If you approximate a translation time of 1 hour per search
this meant a 24% reduction in time.
Figure 1 Mean time taken (minutes) to translate searches
45.1
62.6
79.1
66.8
72.8
70.6
44.7
41.5
27.6
33.6
26.8
28.3
23.7
42.0
11.4
30.8
43.8
49.7
23.5
49.8
52.5
56.7
30.9
17.7
22.5
15.5
24.5
18.1
20.0
5.8
0 10 20 30 40 50 60 70 80 90
All databases
Ovid MEDLINE to All databases
Ovid MEDLINE to CINAHL
Ovid MEDLINE to Cochrane
Ovid MEDLINE to Embase
Ovid MEDLINE to Pubmed
Ovid MEDLINE to Scopus
Ovid MEDLINE to Web of Science
PubMed to All databases
PubMed to CINAHL
PubMed to Cochrane
PubMed to Embase
PubMed to MEDLINE
PubMed to Scopus
PubMed to Web of Science
PST-A method
Manual method
Search hits: median percentage difference from the expected number to
be found
PST-A method Manual method
Translation
Median %
DifferenceRange
Median %
DifferenceRange
PubMed-Web of Science -1.47 (-87.3 to 600.0) 22.8 (-77.2 to 625.7)
PubMed-Scopus 768.3 (-95.2 to 18700.0) 46.1 (0.8 to 28000)
PubMed-MEDLINE -12.9 (-66.7 to 82.4) -17.3 (-57.8 to 160.2)
PubMed-Embase -2.5 (-60.7 to 86.6) -17.9 (-37.9 to 19.8)
PubMed-Cochrane -1.1 (-100.0 to 486.9) 34.6 (-48.5 to 1467.2)
PubMed-CINAHL 13.8 (-38.6 to 1262.5) 43.0 (-4.6 to 300.0)
PubMed - All databases -1.5 (-100.0 to 18700) 2.1 (-77.2 to 28000.0)
MEDLINE-Web of Science 11.7 (-75.9 to 596.7) 7.0 (-78.3 to 731.9)
MEDLINE-Scopus 22.5 (-41.2 to 2012.5) 21.9 (-91.4 to 7424.8)
MEDLINE-PubMed 2.4 (-98.8 to 753.1) 20.4 (-97.5 to 4658.5)
MEDLINE-Embase -23.1 (-99.6 to 7.2) -12.4 (-89.7 to 13.7)
MEDLINE-Cochrane2.4 (-100.0 to 39716.7) 1.4 (-98.9 to 1562.9)
MEDLINE-CINAHL 40.6 (-84.9 to 563.0) 104.0 (-36.6 to 2984.7)
MEDLINE - All databases3.0 (-100.0 to 39716.7) 7.0 (-98.9 to 4658.5)
All databases1.7 (-100.0 to 39716.7) 6.3 (-98.9 to 4658.5)
Errors: mean number of total errors
14
15
7
27
13
15
17
2
9
6
8
18
13
9
0
12
14
1
16
17
23
3
5
7
0
7
15
6
2
0 5 10 15 20 25 30
Total
Ovid MEDLINE-total
Ovid MEDLINE-CINAHL
Ovid MEDLINE-Cochrane
Ovid MEDLINE-Embase
Ovid MEDLINE-PubMed
Ovid MEDLINE-Scopus
Ovid MEDLINE-Web of Science
PubMed-total
PubMed-CINAHL
PubMed-Cochrane
PubMed-Embase
PubMed-MEDLINE
PubMed-Scopus
PubMed-Web of Science
Mean total errors Polyglot (n=95) Mean total errors Manual (n=103)
Errors: Number of type of errors
Type of error All databases Polyglot (n=95) All databases Manual (n=103)
Wrong wildcard or truncation syntax - recall 28 21
Missing or added truncation or wildcard - recall 6 14
Missing or added truncation or wildcard - Precision 0 1
Truncated term at wrong point - recall 2 5
Truncated term at wrong point - Precision 0 3
Missing subject term - recall 3 3
Missing keyword term - recall 4 12
Restricting to major subject terms - recall 3 3
Not exploding subject terms - recall 7 8
Exploding subject terms - precision 3 17
Incorrect MeSH conversion - recall 3 5
Incorrect MeSH conversion - Precision 3 1
Incorrect adjacency syntax - recall 5 1
Used incorrect adjacency - recall 0 2
Used incorrect adjacency- Precision 3 6
Incorrect field syntax - recall 9 2
Incorrect field syntax - precision 0 1
Chose incorrect fields - recall 4 13
Chose incorrect fields - precision 7 13
Incorrect phrase translation - recall 6 3
Incorrect phrase translation - Precision 1 6
Extra out of place characters inserted - recall 8 4
Spelling mistake in search - recall 1 7
Incorrect boolean operator - recall 2 1
Incorrect search structure - recall 6 7
Extra components added - recall 0 0
Extra components added - Precision 3 0
Total unique errors 117 159
Conclusion
• For people skilled in conducting searches of the literature
the PST can increase the speed of translating search
strategies h which this can be done, without a decrease in
quality