Computer Methods and Programs in Biomedicine 53 (1997) 187–190

EasyMA: a program for the meta-analysis of clinical trials

Michel Cucherat *, Jean-Pierre Boissel, Alain Leizorovicz, Margaret C. Haugh

Ser6ice de Pharmacologie Clinique, Hopital cardiologique, BP3041, 69394 Lyon, Cedex 03, France

Received 28 August 1996; received in revised form 24 February 1997; accepted 25 February 1997

Abstract

Meta-analysis of clinical trial data is an increasingly important method in clinical research, particulary in the fieldof therapeutic evaluation. This method uses some specific statistical techniques which are not all available on standardpackages and therefore require specific developments. This paper describes a program designed to help medicalresearchers perform meta-analyses of clinical trial data with dichotomous outcomes. This program includes thevarious statistical methods of meta-analysis and enables cumulative meta-analysis and sub-groups to be performed.A robustness index can be determined and the results obtained in table and graphic formats. Data-editing anddata-manipulating facilities are also possible. Much care has been taken to make the user interface as user-friendlyas possible, so that the program is within the reach of all medical researchers. © 1997 Elsevier Science Ireland Ltd.

Keywords: Meta-analysis; Clinical trials; Treatment effect; Dichotomous outcomes

1. Introduction

Meta-analysis is the process of pooling clinicaltrial results which can be used to confirm theeffect of treatment and obtain more accurate esti-mate of effect or to suggest a new hypothesis thatcan be examined in new studies. Such analyseshave become increasingly popular in clinical re-search over recent years. The program described

in this paper was designed to perform meta-analy-sis of clinical trial data with dichotomous out-comes. Standard statistical packages do not offerall the statistical techniques used in meta-analysis,and these packages are not easy to use by thosewho are not fairly experienced statisticians, as isthe case for many physicians or medical re-searchers. The aim of this project was to providethe medical researchers with a very convivial andglobal tool to assist them in the step necessary forthe realisation of a meta-analysis (e.g. from dataediting to the graphic representation of the re-sults).

* Corresponding author. Tel.: +33 0 472115249; fax: +330 478531030; e-mail: Michel.Cucherat@upcl.univ-lyon1.fr

0169-2607/97/$17.00 © 1997 Elsevier Science Ireland Ltd. All rights reserved.

PII S 01 9 -2607 (97 )00016 -3

M. Cucherat et al. / Computer Methods and Programs in Biomedicine 53 (1997) 187–190188

2. Statistical principles of the meta-analyse

The statistical theory of meta-analysis hasbeen extensively described elsewhere [1–3].Briefly, meta-analysis pools the results of severalclinical trials in order to obtain a global esti-mate of the treatment effect. For a dichotomousoutcome several measures of the treatment effectcan be used: the risk ratio or the odds ratio andthe risk difference. Two statistical models havebeen proposed: the fixed-effects model which as-sumes identical treatment effects in all the trials,and the random-effect model which assumes be-tween-study differences in treatment effects.

Two statistics tests are performed, a test ofassociation and a test of heterogeneity [3]. Theformer tests the existence of a effect treatmentand the latter tests the hypothesis of a homoge-nous treatment effect across the trials. The ac-ceptance or the refusal of effectiveness of thetreatment is based upon the signification degreeof the test of association. When the homogene-ity hypothesis is rejected, the use of a methodbased on a random-effect model is preferable.However, before using a random model, theorigin of the heterogeneity must be sought bysub group analysis according the main charac-teristics of the trials (patients’ baseline charac-teristics or the methodological features of thetrials) [4].

Several statistical methods have been pro-posed. The EasyMA software offers the Man-tel–Haenszel method [5] modified by Robins,Breslow and Greenland [6], the Cochran method[7], the Peto method [8], the method of the loga-rithm of the odds ratio [9], the method of thelogarithm of the relative risk [9], the risk differ-ence method [9], the DerSimonian and Lairdmethod [10,11] and a adaptation of this lastmethod to a multiplicative measure of the effect[12].

3. Pseudocount

When a number of events is zero, the oddsratio calculation is impossible. One way to over-come this problem known as ‘pseudocount’, is

to replace the zero by a small number (e.g. 0.5),but the most efficient method consists of adding0.25 to all the number of events and to thenumber of patients in each group [13]. Exclusionof trial with zero events is also possible whenappropriate.

4. Robustness index

The validity of meta-analysis results is partiallydependant on the existence of a publication bias.To assess the robustness of results for this possi-ble publication bias, we can calculate an indexthat is the number of ‘negative trials’ needed to beadded to the meta-analysis in order to reachnon-significant results [14]. By ‘negative trials’ wemean trials with no evidence of a treatment effector with evidence of a deleterious effect. To obtainthis index it is necessary to specify the characteris-tics of the trials added (number of patients, baserisk in the control group, size of the effect treat-ment). This can be done by adding the same trialuntil the meta-analytic result became non-signifi-cant or by using simulation techniques. The num-ber of patients, the base risk and the treatmenteffect of each trial are randomly generated with adefined probability distribution (e.g. a Gaussiandistribution with a known mean and variance)and the robustness index will be the mean ofseveral simulation replications (an estimation ofits accuracy is therefore possible).

Orwin’s formula, given by Einarson [15], is alsoused to compute the number of studies with agiven treatment effect required to be added to theexisting number of studies to produced a specifiedoverall effect size (usually zero). Begg and coll,proposed a formal statistical significance test toprovide quantitative evidence to refute the hy-pothesis that no publication bias exists [16]. Thistest completes the two previous approaches in theassessment of the publication bias.

When the result from a meta-analysis is notstatistically significant an a-posteriori statisticalpower can be computed [17]. This power is theprobability that meta-analysis results wouldbe statistically significant given the number oftrials and patients included in the meta-analysis.

M. Cucherat et al. / Computer Methods and Programs in Biomedicine 53 (1997) 187–190 189

5. Cumulative meta-analysis

The cumulative meta-analyses, technique de-scribed by Lau et al. [12,18], is available inEasyMA. This technique involves adding the tri-als into the meta-analysis one by one and updat-ing the pooled estimate of the treatment effect ateach step. If the trials are added in chronologicalorder, this is equivalent to updating the meta-ana-lytical estimate each time a new trial is published.The trials can also be sorted in ascending ordescending order of the base risk or magnitude ofthe treatment effect.

6. Program validation and calculation verification

The correctness of the implementation of calcu-lation methods were assessed using several meth-ods including: re-doing the examples given in thepapers about these methods, cross-verification us-ing the calculations performed with a standardspreadsheet (Excel®), verification with the SAS®

statistical package (for the Mantel–Haenszel pro-cedure and P value computation).

7. Description

The program was written in object orientedPascal language, using Borland Turbo Pascal 7.0®

and the Turbo Vision® library. This program isbased on a user-friendly interface: roll-downmenus, dialogue boxes and contextual help facili-ties.

The data are entered into a spreadsheet-likeinput screen. Several outcomes can be definedsimultaneously. For each trial, an identifier (e.g.name), the number of patients in the treated andcontrol groups and the number of events for eachoutcome are entered.

Once entered the trials can be sorted on the riskof an outcome, on the value of the relative risk orthe order can be imposed by entering an ordervariable (e.g. the date of publication).

Data manipulation includes also a pooling fa-cilities that can be used when several trials haveone or two null numbers of case. A grouping of

several trials by summing of effectives and num-bers of case produces a pseudo trial. Severalheuristics to do that are available.

After the realisation of the first calculation, thedata are locked and cannot be modified withoutan user’s confirmation. This security prevents un-intentional alteration of data.

The data can be entered directly or can beimported from a text file. An exportation facilityto a text file is also available.

The results of meta-analysis calculation are dis-played in an specific windows, one end-point andone method at a time. All the outcomes and allmethods can be displayed step by step, or can beprinted at a one time.

The content of the results tables are parametra-ble, and include: the common estimation of thetreatment effect and its confidence interval; thestatistical tests for association and heterogeneity;the results of the sequential analysis; and whenthe test of association is significant, the computa-tion of a robustness index. In the case of ananalysis conducted by sub-groups, these resultsare displayed for each sub-group and as a whole.A heterogeneity test between the sub-groups iscomputed.

For a given method, the graphical results foreach outcome can be presented individually, or allthe results can be given on one graph [19]. Theformat of these graphs can be personalised, andcan include the 95% confidence intervals for theindividual trials and for the common estimate; thenumerical value of common estimation; the re-sults of statistical tests; and the number of pa-tients and events for each group in each trial. Forsub-group analysis the individual sub-group re-sults can be displayed as well as the overall resultfor all the trials.

8. Hardware requirements and programavailability

The program runs on MS-DOS® systems andrequires at least 640 Kb of RAM. It requires agraphics card EGA, VGA or Hercules)and a numerical coprocessor is strongly recom-mended. Only Postscript printers are supported.

M. Cucherat et al. / Computer Methods and Programs in Biomedicine 53 (1997) 187–190190

The program and users guide are available bydownloading from internet sites: www.spc.univ-lyon1.fr/citccf/easyma/ or www.spc.univ-lyon1.fr/�mcu/easyma/

9. Discussion and conclusion

Our experience with this program has shownthat some physicians, who understand meta-anal-ysis methodology but who have no specific train-ing can conduct complete meta-analysis bythemselves. This self-sufficiency is very importantto promote the current interest in this process ofevidence synthesis. However this should not hidethe fact that the real difficulties associated withmeta-analysis do not really concern calculation,but are more involved with the problems such asidentification of unpublished trials and exhaustiveidentification of published trials, selection of thetrials, data extraction and in the correct interpre-tation of the results.

Acknowledgements

This work was supported in part by the ‘Asso-ciation pour la Promotion de la Recherche enTherapeutique’. The authors would like to thankF. Gueyffier, P. Nony, T. Poynard, N. Strang fortheir suggestions for program improvements.

References

[1] R.F. Galbraith, A note on graphical representation ofestimated odds ratio from several clinical trials. Stat.Med. 7 (1988) 889–894.

[2] P. Nony, J.P. Boissel, M. Lievre, M. Cucherat, M.C.Haugh, G. Dayoub, Introduction a la methodologiemetaanalytique. Rev. Med. Interne 16 (1995) 536–546.

[3] A. Whitehead and J. Whitehead, A general parametricapproach to the meta-analysis of randomized clinicaltrials. Stat. Med. 10 (1991) 1665–1677.

[4] S.G. Thompson, Why sources of heterogeneity in meta-analysis should be investigated. Br. Med. J. 309 (1994)1351–1355.

[5] N. Mantel and W.M. Haenszel, Statistical aspects of theanalysis of data from retrospective studies of disease. J.Nat. Cancer Inst. 22 (1959) 719–748.

[6] J.M. Robins, N.E. Breslow, S. Greenland, Estimators ofthe Mantel–Haenszel variance consistent in both sparsedata and large-strata limiting models. Biometrics 42(1986) 311–323.

[7] W.G. Cochran, The combination of estimates from differ-ent experiments. Biometrics 10 (1954) 101–129.

[8] S. Yusuf, R. Peto, J. Lewis, R. Collins, T. Sleight, Betablockade during and after myocardial infraction: anoverview of the randomized trials. Prog. Cardiovasc. Dis.27 (1985) 335–371.

[9] J.P. Boissel, J. Blanchard, E. Panak, J.C. Peyrieux, H.Sacks, Considerations for the meta-analysis of random-ized clinical trials. Summary of a panel discussion. Con-trolled Clinical Trials 10 (1989) 254–281.

[10] R. DerSimonian and N. Laird, Meta-analysis in clinicaltrials, Controlled Clinical Trials 7 (1986) 177–188.

[11] J.A. Berlin, N. Laird, H.S. Sacks, T.C. Chalmers, Acomparison of statistical methods for combining eventrates from clinical trials. Stat. Med. 8 (1989) 141–151.

[12] J. Lau, E.M. Antman, J. Jimenez-Silva, B. Kupelnick, F.Mosteller, T.C. Chalmers, Cumulative meta-analysis oftherapeuthic trials for myocardial infarction. New Engl. J.Med. 327 (1992) 248–254.

[13] W.W. Hauck, S. Anderson, F.J.I. Leahy, Finite sampleproperties of some old and some new estimators of com-mon odds ratio from multiple 2×2 tables, J. Am. Stat.Assoc. 77 (1982) 145–152.

[14] R. Rosenthal, The file drawer problem and tolerance fornull results. Psychol. Bull. 86 (1979) 638–641.

[15] T.R. Einarson, J.S. Leeder, G. Koren, A method formeta-analysis of epidemiological studies. Drug Intelli-gence Clin. Pharm. 22 (1988) 813–824.

[16] C.B. Begg and M. Mazumdar, Operating characteristicsof a rank correlation test for publication bias. Biometrics50 (1994) 1088–1101.

[17] A. Munoz and B. Rosner, Power and sample size fora collection of 2×2 tables. Biometrics 40 (1984) 995–1004.

[18] J. Lau, C.H. Schmid, T.C. Chalmers, Cumulative meta-analysis of clinical trials builds evidence for exemplarymedical care. J. Clin. Epidemiol. 48 (1995) 45–57.

[19] P. Nony, M. Cucherat, M.C. Haugh, J.P. Boissel, Criticalreading of the of clinical trials. Therapie 50(1995) 339–351.

.