Parametric modelling of cost data: some simulation evidence

HEALTH ECONOMICS

Health Econ. 14: 421–428 (2005)

Published online 31 January 2005 in Wiley InterScience (www.interscience.wiley.com). DOI:10.1002/hec.941

HEALTH ECONOMICS LETTER

Parametric modelling of cost data: some simulation evidence

Andrew Briggsa,*, Richard Nixonb, Simon Dixonc and Simon ThompsonbaUniversity of Oxford, UKbMRC Biostatistics Unit, Cambridge, UKcUniversity of Sheffield, UK

Summary

Recently, commentators have suggested that the distributional form of cost data should be explicitly modelled togain efficiency in estimating the population mean. We perform a series of simulation experiments to evaluate theusual sample mean and the mean estimator of a lognormal distribution, in the context of both theoreticaldistributions and three large empirical datasets. The sample mean is always unbiased, but is somewhat less efficientwhen the population distribution is truly lognormal. However the lognormal estimator can perform appallinglywhen the true distribution is not lognormal. In practical situations, where the true distribution is unknown, thesample mean generally remains the estimator of choice, especially when limited sample size prohibits detailedmodelling of the cost data distribution. Copyright # 2005 John Wiley & Sons, Ltd.

Keywords cost analysis; simulation; lognormal

Introduction

The appropriate analysis of cost data generated byclinical trials is problematic. While the usualoutcome of interest is the population mean costfor a particular treatment, the distribution of costdata is generally highly skew because a fewpatients incur very large costs.

A number of commentators have made recom-mendations for the analysis of cost data. Briggsand Gray [1] have argued against the use ofstandard non-parametric methods for analysingcost data, precisely because of the necessary focuson mean cost. While they considered that theremay be some merit in transforming cost data, theyemphasised the importance of presenting costestimates on the untransformed scale.

Thompson and Barber [2] however arguedstrongly that transformation of cost data is notappropriate. They recommended the use of thesample mean as an estimator, with confidencelimits derived either from standard asymptotictheory, or from non-parametric bootstrapping ofthe sample mean.

In a recent contribution, O’Hagan and Stevenshave criticised the unequivocal nature of Thomp-son and Barber’s recommendations [3]. They arguethat while the appropriate focus of cost analysis isclearly the population mean cost, where cost dataare not normally distributed the sample mean isnot necessarily the most efficient estimator. Theyargue that, if cost data truly follow a lognormaldistribution, there will be efficiency gains fromusing exp(lm+lv/2) as an estimator of the meancost in the population, where lm and lv are the log

Copyright # 2005 John Wiley & Sons, Ltd.Received 21 July 2003

Accepted 28 April 2004

*Correspondence to: Department of Public Health, Health Economics Research Centre, University of Oxford, Old Road Campus,Headington, Oxford OX3 7LF, UK. E-mail: [email protected]

scale sample mean and variance, respectively.While O’Hagan and Stevens used an example ofapparently lognormally distributed costs, they arecareful not to argue for making a lognormalassumption in general. Rather they argue that costdata should be appropriately modelled, leavingsomething of a question as to what that appro-priate distribution might be.

Other commentators have been less circumspect.In particular, Zhou, in a series of articles [4–7], hasfocused attention on the use of estimators basedon assuming that costs follow a lognormaldistribution. From the reported applied examples,this assumption is linked to the failure to reject anull hypothesis of normality on the log scale [8],with no consideration of whether there are moreappropriate distributions for cost data.

Although the issues related to estimating meancosts for health care interventions have recentlycome to the fore for health economists conductingeconomic evaluations alongside clinical trials,these issues are by no means new. Similar issuesin the risk-adjustment literature were raised inrelation to estimating costs in the RAND insur-ance experiment in the early 1980s [9,10], wherelog transformation was found to yield bettercompliance with modelling assumptions, butwhere prediction on the untransformed scale wasthe ultimate goal. This requirement led Duan topropose the ‘smearing estimator’ that could beused to correct for the bias that occurs ifexpectations on the transformed scale are backtransformed [11]. In developing this initial frame-work, commentators such as Manning and Mulla-hy have also focused much attention on the use oftransformation as a method for improving theestimation of mean cost relating to health carecosts and expenditures [12–15].

The purpose of this paper is to evaluatetwo alternative estimators of population meancost. First, we look at their relative performanceunder a number of assumed parametric distribu-tions for cost. Second, we repeat the comparison inan empirical context using three datasets wherelarge numbers of individual level costs are avail-able. The aim is to highlight not only the potentialefficiency gains to be obtained from choosing theappropriate estimator, but also any potentialproblems of choosing the incorrect estimator.We return to the link between the estimationof costs for cost-effectiveness analysis and theestimation of costs in the risk-adjustment literaturein the discussion.

Simulating from parametricdistributions

Two parametric distributions were employed togenerate cost data. These were the lognormal andGamma distributions, both of which are used inpractice to model positively skewed cost data. Theappropriate (maximum likelihood) estimator ofthe population mean of a Gamma distribution isthe sample mean, whereas the appropriate estima-tor of the population mean of a lognormaldistribution is exp(lm+lv/2), as described above.Estimation of a confidence interval based on thesample mean estimator is straightforward usingstandard asymptotic assumptions. Obtaining theconfidence interval for the lognormal estimator isa non-trivial problem since it is a function of twotransformed sample estimates. The method used inthis paper is that outlined in a review paper byZhou [8].

For each distribution, the population mean wasset to be 1000 and five choices of coefficient ofvariation (CoV, the ratio of the standard deviationto the mean = 0.25, 0.5, 1.0, 1.5, 2.0) were used todefine the parameters of the distribution. Theresulting distributions (Figure 1) are plausiblerepresentations of cost data. These are thepopulation distributions from which samples aredrawn in the simulation exercise.

Samples of five different sizes (n=20, 50, 200,500, 2000) were drawn from each distribution andfor each coefficient of variation. This represented atotal of 50 different simulation experiments. Foreach experiment, with 10 000 replications, thesample mean and the lognormal estimator wereused to estimate the population mean. For eachsimulation, the bias (average estimate minus 1000)and coverage probability (proportion of 95%confidence intervals containing 1000) were calcu-lated. In order to summarise both bias andprecision, the root mean square error (RMSE)was calculated, which equals the square root of themean [estimate� 1000]2 in each simulation.

As expected, the RMSE reduces with decreasingcoefficient of variation and increasing sample size,no matter which combination of underlyingdistribution or estimator is chosen (Table 1).When the data are truly from a lognormaldistribution, both the lognormal estimator andthe sample mean are unbiased, but the lognormalestimator is more precise – all of the cells of thelower-right matrix in Table 1 show a smaller

Copyright # 2005 John Wiley & Sons, Ltd. Health Econ. 14: 421–428 (2005)

A. Briggs et al.422

RMSE than those in the lower-left matrix.However, the gain in precision is onlymoderate; the ratio of RMSEs always remainsgreater than 84% in Table 1. By contrast,when a lognormal estimator is applied to datathat follow a Gamma distribution, the results aredisastrous. Although, for low coefficients ofvariation there is little to choose between theestimators, where the coefficient of variation is

large, the lognormal estimator performs verypoorly. This is because the lognormal estimatoris severely biased in these cases, while the samplemean remains unbiased.

These results are echoed by the coverageprobabilities of the confidence intervals for thedifferent estimators (Table 2). Increasing coeffi-cients of variation and decreasing sample sizegenerally lead to poorer coverage (less than 95%).

0.0020

0 500 1000 1500 2000 2500 3000

0.0000

0.0005

0.0010

0.0015

Gamma

Cost

Den

sity

21.510.50.25

0 500 1000 1500 2000 2500 3000

0.0000

0.0005

0.0010

0.0015

Log Normal

Cost

Den

sity

21.510.50.25

Figure 1. Parametric distributions used to represent population cost distributions: upper panel shows five lognormal distributions

and lower panel shows five gamma distributions. Parameters of the distributions are set so that the mean cost is 1000 with

coefficients of variation 0.25, 0.5, 1.0, 1.5 and 2.0


Parametric Modelling of Cost Data 423

Again the lognormal estimator exhibits improvedperformance over the sample mean if the data aretruly lognormal. However, the misapplication ofthe lognormal estimator to Gamma distributeddata leads to some very poor coverage results,again due to bias.

Simulations from observed costdatasets

Since it is unlikely that population cost distribu-tions really follow a well-behaved functional formwe repeat the comparison of estimators using threelarge datasets of patient-level cost data. These aretreated as population distributions from whichsamples are drawn in the simulation. The threedatasets used for empirical simulations are de-scribed below.

The CPOU data: These data were constructedfor an economic evaluation of a Chest PainObservation Unit operating within the A and Edepartment of a single large teaching hospital [16].Patients with acute chest pain, undiagnosed byclinical assessment, electrocardiogram and chestradiograph, were recruited and followed-up to6 months. Nine hundred and seventy-two patientswere recruited to the study. Total cost (2001/2002prices) was constructed from the initial 6 hours ofcare, length of stay, parenteral drug therapy,

diagnostic tests, reattendances and readmissions,outpatient attendances and cardiology procedures.

The IV fluids data: These cost data wereconstructed for a randomised controlled trial oftwo prehospital intravenous fluids protocols forparamedics in patients with serious trauma [17].1309 patients were entered into the study from twodifferent Ambulance Trusts and costs were con-structed for 1191 patients. Total cost (1997/1998prices) was calculated for individual patients up to6 months post-incident, and included ambulancecosts, fluid costs, A and E costs, inpatient costs,and ambulatory care.

The Paramedics data: These cost data wereconstructed from a controlled study comparingambulance technicians and paramedics in patientswith serious trauma [18]. 1852 patients wereentered into the study from three differentAmbulance Trusts. Total cost (1996/1997 prices)was calculated for individual patients up to 6months post-incident, and included ambulancecosts, ambulance treatment costs, hospital costs,and ambulatory care.

Summary statistics of the per-patient total costare presented in Table 3 for each example. Allthree datasets exhibit extreme skewness andkurtosis (compared to the values of 0 and 3respectively for a normal distribution). Of interestis the coefficient of variation which shows that thestandard deviation in the data is roughly twice themean for all three datasets – this is the upper rangeof the coefficient of variation examined in the

Table 1. Estimated root mean squared error by underlying distribution and estimator for different sample sizes andcoefficients of variation

RMSE for sample mean estimator RMSE for exp(lm+lv/2) estimatorSimulation sample sizes Simulation sample sizes

Distribution CoV 20 50 200 500 2000 20 50 200 500 2000

Gamma 0.25 56 35 18 11 6 56 35 18 11 60.50 112 71 35 22 11 114 73 38 25 161.00 221 141 70 44 22 400 304 241 226 2181.50 333 214 105 67 34 1388 1097 925 896 8782.00 440 284 141 89 45 2663 1914 1510 1420 1378

Lognormal 0.25 56 36 18 11 6 56 36 18 11 60.50 112 71 35 22 11 112 71 35 22 111.00 224 141 72 45 23 221 137 69 43 221.50 336 214 109 67 34 328 197 99 61 312.00 450 288 143 63 45 419 250 122 54 38

RMSE}root mean squared error; lm}log mean; lv}log variance; CoV coefficient of variation.

A. Briggs et al.424


previous simulation experiments based on para-metric distributions. Taking logs of the cost ineach of the datasets appears to make the distribu-tions more normal (Table 4, Figure 2).

The simulation experiment (again with 10 000replications) involved drawing values at randomwithout replacement from the cost datasets inorder to form samples of varying sizes (n=20, 50,200, 500). On the basis of these samples, theestimators of the population mean (sample mean,and exp(lm+lv/2)) and their corresponding con-fidence limits were calculated.

The RMSE and coverage probabilities (com-pared to the sample mean in all the data) arepresented in Table 5. As expected, the RMSEdecreases with increasing sample size. The RMSEsare similar for the two estimators at the lowersample sizes, but as the Central Limit Theoremtakes effect, the sample mean becomes the moreaccurate. This is echoed in the coverage resultswhere the rapid deterioration in the coverageprobabilities for the lognormal estimator assample size increases is striking. This serves toemphasise that while the Central Limit Theoremplays an important role in the validity of thesample mean as an estimator, increasing samplesize is no guarantee of performance for otherestimators based on parametric assumptions ifthose assumptions turn out not to hold.

It is commonly considered that the CentralLimit Theorem applies for samples with greaterthan 30 observations, whatever the distribution inthe underlying population. However, this rule of

thumb does not apply to non-symmetric distribu-tions typical of cost data. Cochran’s alternativeguideline of n > 25Z2 for situations where the‘principal deviation from normality consists ofmarked positive skewness’ [19] is based on Z, theskewness coefficient in the sample. The guidelinewas devised such that a 95% confidence interval

Table 2. Estimated 95% confidence interval coverage probabilities by underlying distribution and estimator fordifferent sample sizes and coefficients of variation

Coverage for sample mean estimator Coverage for exp(lm+lv/2) estimatorSimulation sample sizes Simulation sample sizes

Distribution CoV 20 50 200 500 2000 20 50 200 500 2000

Gamma 0.25 0.93 0.94 0.95 0.95 0.95 0.93 0.94 0.96 0.96 0.950.50 0.92 0.93 0.95 0.95 0.95 0.94 0.96 0.97 0.95 0.891.00 0.90 0.93 0.95 0.95 0.95 0.97 0.97 0.69 0.18 01.50 0.87 0.91 0.94 0.95 0.94 0.99 0.92 0.14 0 02.00 0.83 0.89 0.93 0.94 0.95 0.99 0.93 0.20 0 0

Lognormal 0.25 0.92 0.93 0.95 0.95 0.95 0.91 0.93 0.95 0.95 0.950.50 0.91 0.93 0.94 0.95 0.95 0.91 0.93 0.94 0.95 0.951.00 0.87 0.91 0.93 0.94 0.95 0.90 0.93 0.95 0.95 0.951.50 0.83 0.88 0.92 0.94 0.94 0.89 0.93 0.94 0.95 0.942.00 0.80 0.86 0.91 0.92 0.94 0.88 0.92 0.94 0.95 0.95

lm}log mean; lv}log variance; CoV coefficient of variation.

Table 3. Summary statistics for per patient total cost inthe three example datasets

CPOU IV Fluids Paramedics

n 972 1191 1852Mean 518 2693 4233Sd 1145 7083 7961Skewness 5.3 4.8 7.5Kurtosis 37 32 88CoV 2.2 2.6 1.9

CoV coefficient of variation.

Table 4. Summary statistics for the natural log of costfor the three example datasets

CPOU IV Fluids Paramedics

n 972 1191 1852Mean 5.37 6.51 7.70Sd 1.19 1.32 1.09Skewness 0.59 1.69 �0.05Kurtosis 3.73 4.72 4.76CoV 0.22 0.20 0.14

CoV coefficient of variation.



will have an error probability no greater than 6%.For the three example datasets this suggests that aminimum sample size of approximately 700 isrequired for the CPOU data, 560 for the IV fluidsdata and 1400 for the paramedics data. The resultspresented in Table 5 for the coverage probabilitiesin fact indicate the guideline to be rather

conservative, since the sample mean has coverageclose to 95% at sample sizes of 200, 200 and 500,respectively. Nevertheless, what is clear is that then>30 rule of thumb is totally inappropriate forcost data.

Comments and conclusions

It was George Box that famously quoted ‘Allmodels are wrong’ [20] and Nester has added, inhis Applied Statistician’s Creed that ‘No data arenormally distributed’ [21]. For cost data it is highlyunlikely that parametric distributions are anythingother than a simple approximation to the truedistribution. The very construction of cost data –as a weighted sum of different resource counts –emphasises that total cost distributions are reallymixtures of many other types of distribution. Evenwith moderately sized samples drawn from knowndistributions the form of that distribution canoften not be reliably ascertained from the dataalone.

The simulation experiments performed hereconfirm that when the appropriate distributionalform of cost data is known, a degree of efficiencycan be gained from using the estimator appro-priate for that distribution. However, applicationof estimators based on incorrect parametricassumptions can lead to totally misleading con-clusions.

The focus of the simulation experiment was theestimation of mean costs as might be generatedalongside a clinical trial. However, in the risk-adjustment literature, similar issues have arisenand other commentators have also presented largescale simulation experiments to address the issueof what form of model/estimator is appropriate forthe data. For example, Manning and Mullahy [12]presented a comprehensive simulation experimentto explore the use of log-transformation versusgeneralised linear modelling (GLM) of cost data.Their general conclusion was that the choice ofmodelling technique was an empirical issue andthey provided an algorithm to help analysts choosethe appropriate approach to their data. Morerecently Deb and Burgess have conducted similarexperiments but using real-life health expendituredata and they conclude much more strongly thatGLMs based on Gamma densities are moreappropriate for risk-adjustment models of cost

0

0.05

0.1

0.15

0

0.05

0.1

0.15

Fra

ctio

n

2 4 6 8 10

Natural log of cost

CPOU dataset

0

0.05

0.1

0.15

0.2

Fra

ctio

n

4 6 8 10 12Natural log of cost

IV fluids dataset

Fra

ctio

n

4 6 8 10 12Natural log of cost

Paramedics dataset

Figure 2. Histograms showing the distribution of log cost in

each of the three example cost datasets

A. Briggs et al.426


data than transformation of the dependent vari-able [22].

The link between what might be described as therisk-adjustment literature, where modelling of costdata is widespread, and the health economicevaluation literature is instructive. In general, therisk-adjustment literature has found that carefulparametric modelling of cost data leads to moreefficient estimators than simply using OLS(equivalent to the sample mean). However, in therisk-adjustment literature, there are typically manymore data points (Deb and Burgess [22] estimatedmodels based on 10,000–500,000 observations). Inthe health economic evaluation literature, wherecost observations are generally collected frompatients recruited to randomised controlled trials,the sample sizes are typically much less. It is withinthe context of such studies, where formal model-ling of cost data is much more challenging, that wehave tried to focus our simulation study. The realdanger, as our simulations show, is the use oftransformation based on parametric assumptionsthat cannot be sufficiently tested. For example,Zhou, in a review of analysing cost data generatedalongside clinical trials presents a series ofexamples where he emphasises the use of the logtransformation [8]. The only justification providedis failure to reject a hypothesis of normality on thelog scale.

In empirical cost analysis for health economicevaluation, the true form of the cost distributionremains unknown. Overall, the sample meanperforms well and is unlikely to lead to inap-propriate inferences. Only when there are sufficientdata to permit detailed modelling and choice of asuitable parametric distributions, is the use of

other estimators of the population mean war-ranted.

Acknowledgements

A previous version of this paper was presented at thethird Developing Economic Evaluation Methodology(DEEM) workshop on ‘The analysis of cost data’ held atAberdeen University in April 2003. We are grateful tocomments from participants, in particular, WillardManning and Paul McNamee. The paper was conceivedat a ‘Focus fortnight on cost data’ organised by theCentre for Bayesian Statistics in Health Economics atSheffield University in February 2003 and we areindebted to Anthony O’Hagan for providing criticalcomment on early ideas. Finally, we are grateful to HiaLee Lee for her assistance with the parametric simula-tions of cost data and to Steven Goodacre for makingavailable one of the cost data sets for the simulations.All errors remain our own responsibility. Andrew Briggsholds a Public Health Career Scientist award from theUK Department of Health.

References1. Briggs A, Gray A. The distribution of health care

costs and their statistical analysis for economicevaluation. J Health Services Res Pol 1998; 3(4):233–245.

2. Thompson SG, Barber JA. How should cost data inpragmatic randomised trials be analysed? Br Med J2000; 320(7243): 1197–1200.

3. O’Hagan A, Stevens JW. Assessing and comparingcosts: how robust are the bootstrap and methodsbased on asymptotic normality? Health Econ 2003;12: 33–49.

Table 5. Simulation results based on drawing samples without replacement from three large datasets

Sample mean estimator exp(lm+lv/2) estimatorSimulation sample sizes Simulation sample sizes

Statistic Dataset 20 50 200 500 20 50 200 500

RMSE CPOU 253 160 70 37 234 141 95 87IV fluids 1583 1015 480 249 1721 1233 1090 1083Paramedics 1915 1131 585 321 1863 990 501 334

Coverage CPOU 0.76 0.83 0.95 0.98 0.80 0.79 0.62 0.28IV fluids 0.77 0.86 0.94 0.98 0.60 0.47 0.12 0.00Paramedics 0.78 0.84 0.89 0.95 0.86 0.87 0.84 0.84

RMSE}root mean squared error; lm}log mean; lv}log variance; CoV coefficient of variation.



4. Zhou XH, Gao S, Hui SL. Methods for comparingthe means of two independent log-normal samples.Biometrics 1997; 53(3): 1129–1135.

5. Zhou XH, Melfi CA, Hui SL. Methods forcomparison of cost data. Ann Intern Med 1997;127(8 Part 2): 752–756.

6. Zhou XH, Tu W. Interval estimation for the ratio inmeans of log-normally distributed medical costswith zero values. Comput Statist Data Anal 2000; 35:201–210.

7. Zhou XH, Melfi CA, Hui SL. Methods forcomparison of cost data. Ann Intern Med 1997;127(8 Part 2): 752–756.

8. Zhou XH. Inferences about population means ofhealth care costs. Statist Methods Med Res 2002; 11:327–339.

9. Manning WG. Health insurance and the demandfor medical care: evidence from a randomizedexperiment. Am Econ Rev 1987; 77: 251–277.

10. Duan N. A Comparison of alternative models forthe demand for medical care. J Business Econ Statist1983; 1: 115–126.

11. Duan N. Smearing estimate: a nonparametricretransformation method. J Am Statist Assoc1983; 78: 605–610.

12. Manning WG, Mullahy J. Estimating log models: totransform or not to transform? J Health Econ 2001;20(4): 461–494.

13. Manning WG. The logged dependent variable,heteroscedasticity, and the retransformationproblem. J Health Econ 1998; 17(3): 283–295.

14. Mullahy J. Much ado about two: reconsideringretransformation and the two-part model inhealth econometrics. J Health Econ 1998; 17(3):247–281.

15. Mullahy J. What you don’t know can’t hurt you?Statistical issues, and standards for medical tech-nology evaluation. Med Care 1996; 34(12 Suppl):DS124–DS135.

16. Goodacre S, Nicholl J, Dixon S, Cross E,Angelini K, Arnold J et al. Cluster randomisedcontrolled trial and economic evaluation of a chestpain observation unit versus routine care. BMJ2004; 328: 254–260.

17. Turner J, Nicholl J, Webber L, Cox H, Dixon S,Yates D. A randomised controlled trial of prehos-pital intravenous fluid replacement therapy inserious trauma. Health Technol Assess 2000; 4(31):1–57.

18. Nicholl J, Hughes S, Dixon S, Turner J, Yates D. Thecosts and benefits of paramedic skills in pre-hospitaltrauma care. Health Technol Assess 1998; 2(17).

19. Cochran WG. Sampling Techniques. Wiley: NewYork, 1977.

20. Box GEP. Science and statistics. J Am Statist Assoc1976; 71: 791–799.

21. Nester MR. An Applied Statistician’s Creed. ApplStatist 1996; 45(4): 401–410.

22. Deb P, Burgess JF. A Quasi-Experimental Compar-ison of Econometric Models for Health CareExpenditures. Hunter College Department of Eco-nomics Working Papers, 212: 2003.

A. Briggs et al.428


Documents

Parametric modelling of cost data: some simulation evidence