View
194
Download
2
Embed Size (px)
Citation preview
What does it take to perform a
Systematic Review of animal studies?
Carlijn Hooijmans
Central Animal Laboratory, SYstematic Review Centre for Laboratory animal Experimentation; SYRCLE
Radboud University Medical Centre
www.umcn.nl/SYRCLE
1
Systematic steps:
1. Phrase the research question
2. Define in- and exclusion criteria
3. Search systematically for ALL original papers
4. Select relevant papers
5. Assess study quality and validity
6. Extract data
7. Analyze results (when possible perform MA)
8. Interpret and present data
2
Focus of this presentation:
1. Phrase the research question
2. Define in- and exclusion criteria
3. Search systematically for ALL original papers
4. Select relevant papers
5. Assess study quality and validity
6. Extract data
7. Analyze results (when possible perform MA)
8. Interpret and present data
3
Focus of this presentation:
1. Phrase the research question
2. Define in- and exclusion criteria
3. Search systematically for ALL original papers
4. Select relevant papers
5. Assess study quality and validity
6. Extract data
7. Analyze results (when possible perform MA)
8. Interpret and present data
4
Goal of Searching systematically
• Detect the maximum amount of available information
Advantages of Searching systematically
• Prevent false or imprecise conclusions
• Prevent unnecessary experimentation
• Obtain new insights that may arise from aggregating
earlier work
5
How to search systematically?
• Formulate adequate and specific research question
Define: - Disease of interest/ health problem
- Population
- Intervention/exposure
- Outcome measures
Example: „What is the effect of [intervention/exposure] on [outcome
measures] in [population studied] for [disease of
interest/health problem]?
• Identify appropriate databases
• Create and run a comprehensive search strategy
6
How to search systematically?
• Transform research question into search strategy for PubMed
Critical search components (SC).
• Identify relevant search terms for each SC.
o Identify Medical Subject Headings (MeSH) terms
"fatty acids, omega-3"[MeSH Terms]
o Identify free-text terms (synonims)
Fish oil [tiab], Omega-3 [tiab] , PUFAs [tiab], DHA [tiab], EPA [tiab], omega 3 fatty acids
[tiab], omega 3 fatty acid [tiab], polyunsaturated fatty acids [tiab] etc…..
o Combine MeSH terms and free-text terms
"fatty acids, omega-3"[MeSH Terms] OR Fish oil [tiab] OR
Omega-3 [tiab] OR, PUFAs [tiab] OR etc
• Repeat step 1-3 for every SC
• Combine SCs
7
Example: Systematic Review about supplementation of omega-3
fatty acids in animal models for Alzheimer’s Disease
Quick search in PubMed:
8
Comphrensive search strategy in PubMed:
Omega-3
Animal models
Alzheimer‟s
Example: Systematic Review about supplementation of omega-3
fatty acids in animal models for Alzheimer’s Disease
9
More information about searching
systematically and finding all animal studies?
10
Focus of this presentation:
1. Phrase the research question
2. Define in- and exclusion criteria
3. Search systematically for ALL original papers
4. Select relevant papers
5. Assess study quality and validity
6. Extract data
7. Analyze results (when possible perform MA)
8. Interpret and present data
11
Why should we critically appraise the included papers?
• Low methodological quality often causes bias in the study
results
overestimation or underestimation
Macleod et al. Stroke, 2008
12
Why should we critically appraise the
included papers?
• The conclusions resulting from a SR/meta-analysis are
dependent of the quality of the original included studies
Transparancy >>>> improves interpretation/reliability
Invalid studies may produce misleading results
Garbage in = garbage out
13
How do we critically appraise the included
papers?
• 2 dimensions:
External validity
Internal validity
• External validity:
Generalizability of the study results
• Internal validity:
The extent to which the results of a study are correct for
the circumstances being studied (methodological quality)
Threatened by bias (systematic errors)
14
Type of bias Description Solution
Selection bias:
Systematic differences between baseline
characteristics of the groups that are
compared
Randomization
Performance
bias:
Systematic differences between groups
in the care or in exposure to factors other
than the intervention of interest
Allocation concealment
Detection bias:
Systematic differences between groups in
how outcomes are determined
Blinding
Attrition bias:
Systematic differences between groups in
“drop outs” from a study
Reporting drop outs
(reason and nr)
How do we critically appraise the included
papers?
15
0% 20% 40% 60% 80% 100%
1) Was it stated in the method section that the experiment was randomized?
2) In case the answer at 1) was yes: Was the method of randomization adequate?
2) In case the answer at 1) was no: Were the groups similar at baseline?
3) Was the allocation to the different groups during the randomization process concealed?
4) Were the caregivers blinded for the allocation of the animals to the specific groups
5) Was the outcome assessment blinded?
6) Methods for outcome assessment the same in both groups?
7) Is the timing of the intervention during the day similar in both groups?
8) Was the outcome assessment randomized across the groups?
9) Number of excluded animals specified per experimental group for each outcome measure?
10) Reason for exclusion mentioned for each excluded animal?
yes no unclear na
Yes=Low risk of bias. No= High risk of bias. ?=Unclear risk of bias. N.a.=not applicable
How do we critically appraise the included
papers?
16
17
Focus of this presentation:
1. Phrase the research question
2. Define in- and exclusion criteria
3. Search systematically for ALL original papers
4. Select relevant papers
5. Assess study quality and validity
6. Extract data
7. Analyze results (when possible perform MA)
8. Interpret and present data
18
Why conducting meta-analyses?
• To increase power
• To increase the precision of estimates of treatment
effects
• To obtain new information about safety and efficacy of
treatments that is not directly visible in the individual
studies
• To generate new hypothesis
19
0 -5 +5
Cell death Adapted data for MA
measure mean sd n mean sd n effect effect size (confidence interval)
Study A % 1.71 -0.05 6 1,9 0,07 5 significant -2.91 [-4.83, -0.98]
Study B ng/ml 104 -6.29 6 114 6,28 6 significant -1.51 [-2.86, -0.16]
Study C % 1 -0.17 12 1,3 0,35 12 significant -1.05 [-1.92, -0.19]
Study D nr of cells -241 14,7 8 -233 17,8 7 not significant -0.50 [-1.53, 0.54]
Study E nr of cells -190 15,1 6 -168 17,6 7 significant -1.24 [-2.47, -0.01]
Study F nr of cells -161 14,6 7 -181 10,3 7 not significant -0.19 [-1.24, 0.86]
Overall -1.02 [-1.61, -0.43]
Data from individual studies
Control groupExperimental group
How to conduct a meta-analyses?
20
Subgroup analyses (fictive dataset)
Overall -1.47 [2.33, -0.61]
21
Subgroup analyses (fictive dataset)
Overall -1.47 [2.33, -0.61]
Males -0.79 [-1.51, -0.06]
Females -3.64 [-4.78, -2.50]
22
Subgroup analyses (fictive dataset)
Overall -1.47 [2.33, -0.61]
Males -0.79 [-1.51, -0.06]
Females -3.64 [-4.78, -2.50]
Hippocampus -1.92 [-2.65, -1.19]
Cortex 0.37 [-0.24, 0.98]
23
Subgroup analyses (fictive dataset)
Overall -1.47 [2.33, -0.61]
Males -0.79 [-1.51, -0.06]
Females -3.64 [-4.78, -2.50]
Hippocampus -1.92 [-2.65, -1.19]
Cortex 0.37 [-0.24, 0.98]
24
So we discussed:
1. Phrase the research question
2. Define in- and exclusion criteria
3. Search systematically for ALL original papers
4. Select relevant papers
5. Assess study quality and validity
6. Extract data
7. Analyze results (when possible perform MA)
8. Interpret and present data
25
Narrative review versus systematic review
Feature Narrative review Systematic review
Research
question
Often unclear or broad Specified and specific
Sources and
search
Not usually specified Comprehensive and explicit
search strategy.
More than 1 database.
Study
selection
Not usually specified Explicit selection criteria
Risk of bias
assessment
Not usually present or only
implicit
Critical appraisal on the basis
of explicit criteria
Data
synthesis
Often a qualitative summary Often also a quantitative
summary (meta-analysis)
26
27