Upload
tri-rachmadijanto
View
34
Download
3
Tags:
Embed Size (px)
Citation preview
Gavin StewartCentre for Evidence-Based ConservationUniversity of Birmingham, UK
Meta-analysis
Talk overview
A word of warning Meta-analysis Data extraction for meta-analysis
Warning
Do you want or need a meta-analysis?
heterogeneity Sample sizes standardisation
Meta-analysis
Steps, usefulness and limitations of meta-analysis
Heterogeneity and methods for its investigation
Weighting and bias
Systematicreview Meta-
analysis
quantitativepooling of results of individual studies
Systematic review in relation to meta-analysis
What does Meta-analysis do?
Meta-analysis aims to quantitatively combine results of different studies
Pooled estimate should be a weighted average of all studies included in a meta-analysis
increase statistical power improves generalisability
Weight studies based on amount of information
The larger studies are given more weight than smaller studies
Sample size Given the same sample size, more weight
will be given to studies with smaller variance (Inverse of variance)
Scale and Pseudoreplication
Weight based on the level of validity
Quality scales Quality components e.g., randomisation method,
baseline, sampling methods SCALE and PSEUDOREPLICATION (again) Different weightings for different elements, so very
controversial and often for the purpose of sensitivity analysis
limited by sample size so maybe necessary to sum
Heterogeneity in meta-analysis
Variation in results across studies Distinguish between statistically significant
heterogeneity and ecologically important heterogeneity
Causes/sources of heterogeneity
Chance Variations in populations Variations in interventions Different methodological quality Different outcome measures
Why investigate heterogeneity?
to decide whether the results of individual studies could be combined (FE models)
to identify effect modifiers (Study-level variables that are associated with the results of studies) e.g. time, method, ex situ/in situ
Methods for investigating heterogeneity
Graphical methods Statistical testing Excluding outliers Subgroup analysis Meta-regression
Statistical testing for heterogeneity
Are the differences in result across studies greater than could be expected by chance?
Q statistic
Excluding outliers
Outliers are excluded one by one until the statistical test of heterogeneity is no longer significant
Should be used very cautiously or not at all
Subgroup analysis in meta-analysis
To separate studies according to certain study-level variables
Then, to conduct quantitative pooling separately for each subgroup of studies
Meta-regression
The estimate of study results is the dependent variable and one or more study-level variables are the independent variables (predictors)
Biases and errors in meta-analysis
Meta-analysis is basically retrospective Results of meta-analysis may be misleading Biases may be introduced if the
identification, inclusion and assessment of primary studies are not systematic
because of publication related biases
Publication bias
studies with significant, positive, results are easier to find than those with non-significant or 'negative' results. The subsequent over-representation of positive studies in systematic reviews may mean that our reviews are biased toward a positive result.
Also have time lag bias, language bias and citation bias
Funnel plot (se against es) asymmetry
Bad or inappropriate meta-analysis
Not systematic in study identification and assessment Inappropriate pooling of heterogeneous results, (FE models) No investigation of heterogeneity Lack of details of included studies Inappropriate weighting individual studies Failed to consider publication and related biases Lack of sensitivity analysis Inappropriate interpretation of the results of subgroup analysis Inappropriate interpretation of the pooled average
-
-
-
-
Data extraction
Extract data with synthesis in mind e.g. Mean, n, sd for treatment and control
Extract data on effect modifiers Use standardised piloted method and check repeatability Consider scale and pseudoreplication Contact authors for missing data where possible Do not get side tracked into extracting more than you need
References
Cooper, H. and Hedges, L.V. (1994) (eds.) The Handbook of Research Synthesis. Russell Sage Foundation, New York.
Deeks, J.J., Altman, D.G. and Bradburn, M.J. (2001) Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. Systematic Reviews in Health Care. Meta-analysis in Context. (eds M. Egger, G.D. Smith and D.G. Altman), pp 285-312. British Medical Journal Publishing Group, London.
DerSimonian, R. and Laird, N. (1986) Meta-analysis in clinical trials. Controlled. Clinical. Trials, 7, 177-188. Egger, M., Davey-Smith, G., Schneider, M. and Minder, C. (1997) Bias in meta-analysis detected by a simple
graphical test British Medical Journal, 315, 629-34. Gates, S. (2002). Review of methodology of quantitative reviews using meta-analysis in ecology. Journal of
Animal Ecology, 71, 547–557. Gurevitch, J. and Hedges, L.V. (1999) Statistical issues in ecological meta-analyses. Ecology, 80, 1142–
1149. Gurevitch, J. and Hedges, L.V. (2001) Meta-analysis. Combining results of independent experiments. Design
and Analysis of Ecological Experiments (eds S.M. Scheiner and J. Gurevitch), pp. 347–369. Oxford University Press, Oxford.
Hedges, L.V., and Olkin, I. (1985). Statistical Methods for Meta-analysis. San Diego: Academic Press, San Diego.
Hurlbert, S.H, (1984) Pseudoreplication and the design of ecological field experiments. Ecological. Monographs, 54, 187-211.
Osenberg, C.W., Sarnelle, O., Cooper, S.D. and Holt, R.D. (1999) Resolving ecological questions through meta-analysis: goals, metrics and models. Ecology, 80, 1105–1117.
Thompson, S.G. and Sharp, S.J. (1999) Explaining heterogeneity in meta-analysis: a comparison of methods. Statistics in Medicine, 18, 2693-708.