Systematic Reviews - InfectionControl.tips · 2016-04-25 · MEDLINE EMBASE Pubmed 1. exp Femoracetabular Impingement/ 2. femoral adj acetabular adj impingement.mp. 3. femoroacetabular

InfectionControl.tipsJoin. Contribute. Make a Difference

Systematic Reviews Andrew Duong, M.Sc


The Systematic Review

Process

Identify Research Question/Objective

Development of Research

Question/PICO question

Development and Execution of Search

Strategy

Data AbstractionPresentation and

Synthesis of Results

Assessment of Study Quality and Publication Biases


1. Developing the Research

Strategy

Start with your research question

Population

Intervention(s)

Types of study design

Each database/systematic review may require a

different search strategy

Medline, EMBASE, PubMed, CENTRAL

(Cochrane), CINAHL


1. Developing the Research

Strategy

When doing background research, identify

key words and terms that are important

Exclusionary terms are just as important


2a. Electronic Database

Searching

Search optimization

◦ Truncation ($, *)

◦ Operators (AND, OR, ADJn)

◦ Syntax (title, abstract, subject heading, publication type)

◦ Subject headings

Automatic explosion (.exp) expands on topic and includes all predesignated subheadings

Searches should be kept broad


2a. Electronic Database

SearchingMEDLINE EMBASE Pubmed

1. exp Femoracetabular

Impingement/

2. femoral adj acetabular adj

impingement.mp.

3. femoroacetabular

impingement.mp.

4. femoracetabular

impingement.mp

5. Hip impingement.mp

6. Pincer impingement.mp

7. Cam Impingement.mp.

8. 1 or 2 or 3 or 4 or 5 or 6 or 7

9. Limit 8 to English language

July 31, 2015: 1070 results

1. exp Femoracetabular

Impingement/

2. femoral adj acetabular adj

impingement.mp

3. femoroacetabular

impingement.mp

4. femoracetabular

impingement.mp

5. Hip impingement.mp

6. Pincer impingement.mp

7. Cam Impingement.mp.

8. 1 or 2 or 3 or 4 or 5 or 6 or 7

9. Limit 8 to English language

July 31, 2015: 1800 results

1. Femoracetabul*

Impingement/

2. Femoro*acetabul*

impingement/

3. Femoral acetabul*

impingement/

4. Hip Impingement/

5. Pincer Impingement/

6. Cam Impingement/

7. 1 or 2 or 3 or 4 or 5

or 6

8. Limit 7 to 2015 [year]

October 21, 2015: 207

results


2b. Manual Searching

Manual search of journals

Consulting experts

Conference proceedings

New, unpublished data

Data not necessarily peer-reviewed

Dissertations and theses


2b. Manual Searching

Ongoing clinical trials

https://clinicaltrials.gov/

Websites

Useful for treatment guidelines American Medical Association, Center for

Disease Control

Guidelines may be anecdotal

Always review the primary literature

https://clinicaltrials.gov/


Biases

Systematic errors that consistently result in misattribution of findings

Avoid by establishing clear criteria a priori that do not discriminate based on the evidence presented Publication Bias

Misrepresentation of the population of studies

Reporting Bias Attrition Bias Detection Bias


3. Screening

The development of inclusion/exclusion

criteria is important for this process

Always done in duplicate

Title Screening Abstract Screening

Full Text Screening

Screening is typically done in

spreadsheets (Microsoft Excel)


3. Screening

Code your screening 1 = Include

2 = Maybe include

3 = Exclude (non-randomized control trial)

4 = Exclude (duplicate article)

5 = Exclude (cadaveric study)

Etc.


3. Screening

Before screening, test a limited number of

articles (~30) to ensure that both

screeners are clear about what is to be

included, and what is not.

If there is a lot of disagreement, it is owrth

going back to the original

inclusion/exclusion criteria and revising

this.


Statistical Analysis

Agreement: Calculation of variability

during screening

Kappa (Cohen’s Kappa, κ)

Intraclass Correlation Coefficient (ICC)


Statistical Analysis

Improve agreement by

Developing a clear research question a priori

Developing inclusion/exclusion criteria a priori

Testing screening agreement with a few articles to determine if both reviewers are interpreting criteria in the same way

Disagreements should be resolved at this point, and criteria should be changed to be more clear


Agreement Calculation of variability during screening

Kappa (Cohen’s Kappa, κ)

Statistic for the measure of inter-rater

agreement

Systematic review: Level of agreement

between screeners (dichotomous only)

Indirect measure of quality of

inclusion/exclusion criteria

Κ = Total agreements – expected agreements

Overall total – expected agreements


Agreement

Intraclass Correlation Coefficient (ICC)

Inter-rater reliability based on quantitative values (continuous variables)

Useful for evaluations where scores are given

E.g. radiographic analysis can deem a patient to have a score of 1-10, which can vary depending on radiologist

Detsky quality assessment score based on multiple variables evaluating quality


4. Data Abstraction and

Analysis

Following screening, data should be

abstracted from full text articles in

duplicate

Similar to screening, test a limited

number of articles at first to ensure that

the data abstraction form is clear

Analysis of information can then be

performed from this subset


4. Data Analysis

Meta-Analysis

Integration of a number of studies to

create a more comprehensive

conclusion

Reduce heterogeneity among studies

Pooled studies must be sufficiently similar to

allow for integration of results


4. Data Analysis

Meta-analysis

Forest Plot

Heterogeneity: Variation in outcomes between studies Cochran’s Q: Weighted

difference between individual studies and the pooled effect of all studies

I2 statistic: Percentage of variation across all studies due to heterogeneity (not chance)


5. Presenting the Data

Magic formula for systematic reviews

Figure 1: Screening results (PRISMA

Guidelines)

Table 1: Study characteristics

Appendix 1: Search strategy


Records identified through database searching

(n = )

Screen

ing

Included

Eligibility

Iden

tification

Additional records identified through other sources

(n = )

Records after duplicates removed(n = )

Records screened(n = )

Records excluded(n = )

Full-text articles assessed for eligibility

(n = )

Full-text articles excluded, with reasons

(n = )

Studies included in qualitative synthesis

(n = )

Studies included in quantitative synthesis

(meta-analysis)(n = )


5. Presenting the Data Table 1: Summary of the studies,

highlighting general study information

Author, year, number of patients, mean age, gender proportion, general study guidelines, overall results


5. Preparing the Data

Appendix 1: Search Strategy


6. Writing the Manuscript

Outline

This is the most important part of the paper

Plan out the road map of the paper,

including details of what needs to be

introduced when, what results will be

discussed, and how figures are presented.

Edits at this stage are extremely easy

The more detail that is put in here, the easier the

rest of the writing process will be



Introduction

Background information, current gap in

knowledge, and purpose of study

Methods

Be as specific as possible, especially

search strategy development, screening

process, descriptive statistics,


6. Writing the Manuscript Results

Describes all the meaningful results

from the study

Figures and tables are complementary

to the results. Not a reiteration



Discussion Synthesis of results, which aims to answer

the research question First paragraph Statement of all major findings

Include all important findings from analyzing the results Do not restate the results

Strengths and limitations Include aspects that affect the quality(both

positively and negatively)



References

Use a reference manager

Mendeley reference manager

Be sure to cite using the journal’s stated

format

http://scholar.google.com/

http://scholar.google.com/



Figures and Tables

Ensure that all figures and tables are

original

Typically, figures should be 300 d.p.i. or

higher

Abstract

Presented first, but written last


Tips for Performing

Systematic Reviews

Purpose of a Systematic Review is to

objectively summarize the data

Do not present every detail of what is

happening in each study

Raw data is to be avoided, especially

when descriptive statistics can be used

in its place


Tips for Performing

Systematic Reviews

Create a well defined research question, and refer to literature to help you refine it

Search strategy should be reviewed by all authors, and results should be screened to ensure appropriate articles are included

Searches should be as broad as possible

A priori inclusion criteria can be modified as you go. Test it against multiple articles before screening on a full scale


Tips for Performing

Systematic Reviews

All screening and data abstraction steps should be done in duplicate.

Disagreement can be resolved by a third party

Keep records of every step, including number of agreements, why articles were excluded, type of information abstracted and analyzed, and what methods/software were used for data and statistical analysis


Tips for Performing

Systematic Reviews

Prior to writing/creating figures, create a

paper outline

What will be included in each section,

and what order will things be presented?


Tips for Performing

Systematic Reviews

Try to put everything into a figure (when it

makes sense)

Figures/Tables and text should try to be

mutually exclusive

Figures/tables are supplementary to

understanding the information and the

focus should be to SUMMARIZE the

data


Tips for Performing

Systematic Reviews Discussion

First paragraph should summarize the major findings

Be explicit with limitations of the study

Improve quality

Methods to reduce bias, increase agreement, broaden search, perform steps in duplicate, robustness of article databases

Decrease quality

Limited number of studies, lack of meta-analysis, deviance from protocol, quality of evidence abstracted, limited scope, other limitations


Thank you

Documents

Systematic Reviews - InfectionControl.tips · 2016-04-25 · MEDLINE EMBASE Pubmed 1. exp Femoracetabular Impingement/ 2. femoral adj acetabular adj impingement.mp. 3. femoroacetabular