A decision-theoretic framework for the application of cost ... · A decision-theoretic framework for the application of cost-eﬀectiveness analysis in regulatory processes Gianluca

A decision-theoretic framework for the application of

cost-effectiveness analysis in regulatory processes

Gianluca Baio

Department of Statistical Science, University College London (UK)Department of Statistics, University of Milano Bicocca (Italy)

[email protected]

(Thanks to Pierluigi Russo, Italian Medicine Agency, Rome, Italy)

Bayes 2012

Aachen, 9-10-11 May 2012

Gianluca Baio ( UCL) EVI in regulatory contexts Bayes 2012, 10 May 2012 1 / 23

Outline of presentation

1 Health economic evaluations

2 Example: The market for statins

– Cost-effectiveness analysis– Probabilistic sensitivity analysis & Expected Value of Information

3 Mixed strategy and non-optimal market configuration

– Application to the market for statins

4 Conclusions








4 Conclusions








4 Conclusions








4 Conclusions


Health economic evaluations

• Objective: Combine costs & benefits of a given intervention into a rationalscheme for allocating resources

– Recently, models have been built upon more advanced statistical foundations– This problem can be formalised within a statistical decision-theoretic

approach. Rational decision-making is effected through the comparison ofexpected utilities


Health economic evaluations

• Objective: Combine costs & benefits of a given intervention into a rationalscheme for allocating resources

– Recently, models have been built upon more advanced statistical foundations– This problem can be formalised within a statistical decision-theoretic

approach. Rational decision-making is effected through the comparison ofexpected utilities

• Increasingly under a Bayesian framework, especially in the UK– David Spiegelhalter (2006). Bayesian methods, health technology assessment,

and performance monitoring. Report on progress 2001-2006 for MRC Unit’sQuinquennial Review

◦ Specific focus on Bayesian decision-theoretic development of cost-effectivenessanalysis

– Contributions by several scholars and research groups

◦ Tony O’Hagan, Jeremy Oakley (University of Sheffield — Centre for BayesianStatistics in Health Economics)

◦ Karl Claxton, Mike Sculpher (University of York)◦ Giovanni Parmigiani (John Hopkins University), Gordon Hazen (Northwestern

University)◦ Simon Thompson, Chris Jackson and Richard Nixon (MRC Cambridge)


(Bayesian) Decision-making process

• Typically, we define a “health economic response” (e, c), where for eachintervention (treatment) t

– e represents a suitable measure of clinical benefits (eg QALYs)– c are the costs associated with a given intervention





• The variables (e, c) are usually defined as functions of a set of relevantparameters θt which represent some population-level features of theunderlying process

– Probability of some clinical outcome– Duration in treatment– Reduction in the rate of occurrence of some event





• The variables (e, c) are usually defined as functions of a set of relevantparameters θt which represent some population-level features of theunderlying process

– Probability of some clinical outcome– Duration in treatment– Reduction in the rate of occurrence of some event

• There are (at least) two sources of uncertainty

– Sampling variability is modelled using an intervention-specific distributionp(e, c | θt)

– Parametric uncertainty is modelled using a (possibly subjective) priordistribution p(θt | D), based on some background data D

– Sometimes, we can (should!) consider also structural uncertainty, ie about themodelling assumptions used



• In addition, we define a utility function to describe the quality of t

– The function u(e, c; t) describes the value associated with applyingintervention t, in terms of the future (uncertain) outcomes

– Uncertainty is expressed through p(e, c,θ) = p(e, c | θ)p(θ | D)






• NB: typically, the utility function chosen is the monetary net benefit

u(e, c; t) := ket − ct

– k is the “willingness to pay”, ie the cost per extra unit of effectiveness gained









• Decision making is based on

– Computing for each intervention t the expected utility

Ut = E[u(e, c; t)]

(computed with respect to both individual and population uncertainty)











Ut = E[u(e, c; t)]


– Treating the entire homogeneous (sub)population with the most cost-effectivetreatment, ie that associated with the maximum expected utility











Ut = E[u(e, c; t)]


– Treating the entire homogeneous (sub)population with the most cost-effectivetreatment, ie that associated with the maximum expected utility

– Performing sensitivity analysis (to parameter and/or structural uncertainty)to investigate the impact of underlying uncertainty on the decision process


Example: The market for statins

• Statins are a class of drug used to lower plasma cholesterol level by inhibitingan enzyme in the liver. This results in decreased cholesterol synthesis as wellas increased clearance of low-density lipoprotein from the bloodstream




• Currently, there are 7 statins on the market, worldwide

– Atorvastatin (AS; synthetic, first marketed in 1997)– Fluvastatin (FS; synthetic, 1994)– Lovustatin (LS; fermentation-derived, 1976)– Pitavastatin (PtS; synthetic, 2003)– Pravastatin (PS; fermentation-derived, 1991)– Simvastatin (SS; synthetic derivate of fermentation process, 1988)– Rosuvastatin (RS; synthetic, 2003)






• AS, RS and SS are market leaders, in Italy






• AS, RS and SS are market leaders, in Italy

• Extensive data are available from controlled studies comparing the clinicaleffectiveness of several statins against placebo

• We are interested in evaluating the efficiency of the market with respect tothe fact that all the different statins are available for prescription and are allreimbursed by the Italian NHS (albeit under different conditions)


Statistical model

Theoreticaleffectiveness

Compliance

Reduction ineffectiveness

Treatmentcosts

Hospitalisationcosts

e c


Statistical model


Compliance


Treatmentcosts


e c

• The model is based on the combination ofevidence from RCTs and observational dataavailable for Italy

• Statin s is evaluated against statin t using themonetary net benefit as utility measure, andby means of the Expected IncrementalBenefit

EIB(s, t) = Expected utility(s) - Expected utility(t) = Us−Ut

• Decision rule:If EIB(s, t) > 0 then s is more cost-effective (C/E) than t


Statistical model — theoretical effectiveness

• Based on published data on RCTscomparing statins to placebo

• Define ysj and nsj as the number ofNon Fatal Miocardial Infarction(NFMI) cases and of individualobserved in the j−th study on statin s





• Then for s = 1, . . . , S = 6 statins andj = 1, . . . , Ns studies, we model

ysj ∼ Binomial(psj , nsj)

logit(psj) ∼ Normal(γs, τs)





• Then for s = 1, . . . , S = 6 statins andj = 1, . . . , Ns studies, we model

ysj ∼ Binomial(psj , nsj)

logit(psj) ∼ Normal(γs, τs)

• The parameter γs represent a “pooledestimate” of the theoreticaleffectiveness for statin s


Statistical model — reduction in effectiveness

nsj ysj zsgr msgr

γs πsg

µg σ2 us δg−1

ξg ηsg TCs dg

ρg es cs δg

th HC φh wh

ih lh fh vh

αh λh βh

We use observational data on clinical practice to model the decrease in effectivenessdue to G = 4 levels of non compliance

a = Logical nodes



nsj ysj zsgr msgr

γs πsg

µg σ2 us δg−1

ξg ηsg TCs dg

ρg es cs δg

th HC φh wh

ih lh fh vh

αh λh βh


We define µg and σ2 to encode information fromthe literature and model

ξg := log ρg ∼ Normal(µg, σ2) g = 2, 3, 4

ρ1 =1

exp(ξ4), ρ2 =

exp(ξ2)

exp(ξ4),

ρ3 =exp(ξ3)

exp(ξ4), ρ4 =

exp(ξ4)

exp(ξ4)



nsj ysj zsgr msgr

γs πsg

µg σ2 us δg−1

ξg ηsg TCs dg

ρg es cs δg

th HC φh wh

ih lh fh vh

αh λh βh


We define µg and σ2 to encode information fromthe literature and model

ξg := log ρg ∼ Normal(µg, σ2) g = 2, 3, 4

ρ1 =1

exp(ξ4), ρ2 =

exp(ξ2)

exp(ξ4),

ρ3 =exp(ξ3)

exp(ξ4), ρ4 =

exp(ξ4)

exp(ξ4)

Compliance group HR (95% CI)

Very low users 1 —Low users 0.85 [0.73− 0.98]

Intermediate users 0.82 [0.71− 0.95]High users 0.80 [0.70− 0.93]

⇒ µ =

−0.1625−0.1985−0.2231

, σ2 = 0.0049


Statistical model

• Compliance levels, treatment and hospitalisation costs are also estimatedusing observational data

• All the relevant parameters are then combined to define

– e = a weighted effectiveness in terms of chance of (non) experiencing NFMI,based on RCT data and compliance

– c = a total cost of the treatment, accounting for the level of compliance andthe risk of experiencing hospitalisations for NFMI


Statistical model

• Compliance levels, treatment and hospitalisation costs are also estimatedusing observational data

• All the relevant parameters are then combined to define

– e = a weighted effectiveness in terms of chance of (non) experiencing NFMI,based on RCT data and compliance

– c = a total cost of the treatment, accounting for the level of compliance andthe risk of experiencing hospitalisations for NFMI

• These are then combined to define

– The incremental effectiveness: ∆e = E[e | s,θ]− E[e | t,θ]– The incremental costs: ∆c = E[c | s,θ]− E[c | t,θ]– The expected incremental benefit: EIB = kE[∆e]− E[∆c] = Us − Ut

• The economic analysis can be then performed to estimate which statin is themost “cost-effective”


Cost-effectiveness analysis

0 10000 20000 30000 40000 50000

010

0020

0030

00

Expected Incremental Benefit

Willingness to pay

EIB

Atorvastatin vs Pravastatin

• We (arbitrarily) use AS as thereference intervention andcompare it to all the otherstatins

• If EIB> 0 then AS is morecost-effective than thecomparator

• First, compare AS against PS:for all k ≥ e0, AS is moreC/E as the black line is alwaysabove 0

• Thus, PS is irrelevant, as it isdominated by AS



0 10000 20000 30000 40000 50000

010

0020

0030

00


Willingness to pay

EIB

Atorvastatin vs PravastatinAtorvastatin vs Lovastatin

• Then, include LS: for all k ≤e415, LS has a higherexpected utility compared toAS (the red line is negative)

• Thus LS is more C/E than ASfor k ≤ e415, while AS ismore C/E for k > e415

k = e415



0 10000 20000 30000 40000 50000

010

0020

0030

00


Willingness to pay

EIB

Atorvastatin vs PravastatinAtorvastatin vs LovastatinAtorvastatin vs Fluvastatin

• However, for all k ≤ e2 760,FS has an even higherexpected utility (the blue lineis negative and lower than thered one)

• Consequently, LS is irrelevanttoo: it is dominated by FS fork ≤ e2 760 and by AS fork > e2 760

k = e2 760



0 10000 20000 30000 40000 50000

010

0020

0030

00


Willingness to pay

EIB

Atorvastatin vs PravastatinAtorvastatin vs LovastatinAtorvastatin vs FluvastatinAtorvastatin vs Rosuvastatin

• However, for all k ≤ e3 890,RS has an even higherexpected utility (the magentaline is negative and lower thanboth the red and the blueones)

• Consequently, FS is irrelevanttoo: it is dominated by RS fork ≤ e3 890 and by AS fork > e3 890

k = e3 890



0 10000 20000 30000 40000 50000

010

0020

0030

00


Willingness to pay

EIB

Atorvastatin vs PravastatinAtorvastatin vs LovastatinAtorvastatin vs FluvastatinAtorvastatin vs RosuvastatinAtorvastatin vs Simvastatin

• However, when k ≤ e16 000,SS has an even higherexpected utility (the green lineis negative and the lowest inthat range)

• Consequently, RS is irrelevanttoo: it is dominated by SS fork ≤ e16 000 and by AS fork > e16 000

• The decision problem is thensolved with the outcome:choose SS for k ≤ e16 000,and choose AS when k >e16 000

k = e16 000


Probabilistic sensitivity analysis

The quality of the current evidence is often limited

• During the pre-market authorisation phase, the regulator should decidewhether to grant reimbursement to a new product — and in some countriesalso set the price — on the basis of uncertain evidence, regarding bothclinical and economic outcomes

• Although it is possible to answer some unresolved questions after marketauthorisation, relevant decisions such as that on reimbursement (whichdetermines the overall access to the new treatment) have already been taken


Probabilistic sensitivity analysis

The quality of the current evidence is often limited

• During the pre-market authorisation phase, the regulator should decidewhether to grant reimbursement to a new product — and in some countriesalso set the price — on the basis of uncertain evidence, regarding bothclinical and economic outcomes

• Although it is possible to answer some unresolved questions after marketauthorisation, relevant decisions such as that on reimbursement (whichdetermines the overall access to the new treatment) have already been taken

• This leads to the necessity of performing (probabilistic) sensitivity analysis(PSA)

– Formal quantification of the impact of uncertainty in the parameters on theresults of the economic model

– Standard requirement in many health systems (e.g. for NICE in the UK), butstill not universally applied


PSA to parameter uncertaintyParameters Model structure Decision analysis



Compliance

Costs

Statin s


Compliance


Treatmentcosts


e c

Statin t


Compliance


Treatmentcosts


e c

Statin s

Benefits Costs

Statin t

Benefits Costs

ICER =20 000

1QALY





Compliance

Costs

x

x

x

x

Statin s


Compliance


Treatmentcosts


e c

Statin t


Compliance


Treatmentcosts


e c

Statin s

Benefits Costs1 10 000

Statin t

Benefits Costs2 30 000

ICER =20 000

1QALY

⇒ ⇒





Compliance

Costs

x

x

x

x

Statin s


Compliance


Treatmentcosts


e c

Statin t


Compliance


Treatmentcosts


e c

Statin s

Benefits Costs1 10 0000 5 000

Statin t

Benefits Costs2 30 0003 20 000

ICER =20 000

1QALY

⇒ ⇒





Compliance

Costs

x

x

x

x

Statin s


Compliance


Treatmentcosts


e c

Statin t


Compliance


Treatmentcosts


e c

Statin s

Benefits Costs1 10 0000 5 000. . . . . .

2 15 0002.55 10 000

Statin t

Benefits Costs2 30 0003 20 000. . . . . .

4 40 0003.80 30 000

ICER =20 000

1.25

ICER= 16 004

⇒ ⇒



Cost effectiveness plane Atorvastatin vs Simvastatin

Effectiveness differential

Cos

t diff

eren

tial

0.00 0.05 0.10

−10

000

1000

2000

• ICER=16003.59

k = 25000


Cost Effectiveness Acceptability Curve

0 10000 20000 30000 40000 50000

0.0

0.2

0.4

0.6

0.8

1.0


Willingness to pay

Pro

babi

lity

of c

ost e

ffect

iven

ess

Atorvastatin vs FluvastatinAtorvastatin vs LovastatinAtorvastatin vs PravastatinAtorvastatin vs RosuvastatinAtorvastatin vs Simvastatin



0 10000 20000 30000 40000 50000

0.0

0.2

0.4

0.6

0.8

1.0


Willingness to pay

Pro

babi

lity

of c

ost e

ffect

iven

ess


• Use net benefit utilityu(e, c, s) = kes − cs, butconsider varying k

• CEAC representsPr(k∆e −∆c > 0 | Data) asa function of k

• Suggested as the standardtool for PSA by NICE



0 10000 20000 30000 40000 50000

0.0

0.2

0.4

0.6

0.8

1.0


Willingness to pay

Pro

babi

lity

of c

ost e

ffect

iven

ess


• Use net benefit utilityu(e, c, s) = kes − cs, butconsider varying k

• CEAC representsPr(k∆e −∆c > 0 | Data) asa function of k

• Suggested as the standardtool for PSA by NICE

• Summarises the probability ofcost effectiveness, as itdepends on the willingness topay parameter k

• Meaningful only if theparameters are consideredrandom, i.e. within theBayesian framework


Expected value of (perfect) information

0 10000 20000 30000 40000 50000

6080

100

120

140

160

180

200

Expected Value of Information

Willingness to pay

EV

PI

SS most C/E ←→ AS most C/E



0 10000 20000 30000 40000 50000

6080

100

120

140

160

180

200


Willingness to pay

EV

PI


• Compares the ideal decisionprocess (ie if the uncertaintyon the parameters wereresolved to the simulatedvalues) with the actual one (iewhen uncertainty is averagedout in the expected utility)

• Describes the maximumamount the decision makershould be willing to pay toresolve the uncertainty in theparameters



0 10000 20000 30000 40000 50000

6080

100

120

140

160

180

200


Willingness to pay

EV

PI


• Compares the ideal decisionprocess (ie if the uncertaintyon the parameters wereresolved to the simulatedvalues) with the actual one (iewhen uncertainty is averagedout in the expected utility)

• Describes the maximumamount the decision makershould be willing to pay toresolve the uncertainty in theparameters

• By construction, combines

a) how much we are likely tolose if we take the “wrong”decision

b) how likely it is that we takeit

• Drives the process ofgathering additional evidence


Cost-effectiveness analysis — summary

Cost-effectiveness analysis summary

Reference intervention: AtorvastatinComparator intervention(s): Fluvastatin

: Lovastatin: Pravastatin: Rosuvastatin: Simvastatin

Optimal decision: choose Simvastatin for k<16000 and Atorvastatin for k>=16000

Analysis for willingness to pay parameter k = 25000

Expected utilityAtorvastatin 22770Fluvastatin 22336Lovastatin 21002Pravastatin 21579Rosuvastatin 22415Simvastatin 22628

EIB CEAC ICERAtorvastatin vs Fluvastatin 434.63 0.786 2759.65Atorvastatin vs Lovastatin 1768.49 0.869 414.67Atorvastatin vs Pravastatin 1191.01 0.973 -228.97Atorvastatin vs Rosuvastatin 355.24 0.755 3989.22Atorvastatin vs Simvastatin 142.44 0.636 16003.59

Optimal intervention (max expected utility) for k=25000: Atorvastatin

EVPI 175.25


Mixed strategies

• Clinical practice and the regulator’s decisions in particular are generally notable to move towards a rapid substitution of the available therapeutic optionswith a new one that is more cost-effective

– Only rarely a new treatment proves to be cost-effective over the entirepopulation

– Irreversibility risks associated with implementing an intervention (ie thedecision maker might want to temporize, in order to have more reliableevidence on which to base the final decision)

– The market usually takes some time to “adjust” to the new configurationgenerated by the innovative drug just introduced


Mixed strategies

• Clinical practice and the regulator’s decisions in particular are generally notable to move towards a rapid substitution of the available therapeutic optionswith a new one that is more cost-effective

– Only rarely a new treatment proves to be cost-effective over the entirepopulation

– Irreversibility risks associated with implementing an intervention (ie thedecision maker might want to temporize, in order to have more reliableevidence on which to base the final decision)

– The market usually takes some time to “adjust” to the new configurationgenerated by the innovative drug just introduced

• Consequently, non-optimal interventions tend to remain active on the market

– The regulator is faced with the problem of balancing the optimal decision (ieimplementing the most cost-effective treatment) under the constraints thatthe market shares of the other molecules already present on the marketcan not be all set to zero


Mixed strategies (cont’d)

• Since many interventions are kept on the market, in this case, the overallexpected utility in the population is

U =

S∑

s=1

qsUs,

where

– qs represents the market share for statin s

– Us is the expected utility for statin s


Mixed strategies (cont’d)

• Since many interventions are kept on the market, in this case, the overallexpected utility in the population is

U =

S∑

s=1

qsUs,

where

– qs represents the market share for statin s

– Us is the expected utility for statin s

• The impact of uncertainty in the decision process, when the mixed strategy isactually chosen by the decision maker (ie when all the options s are on themarket, with shares q1, q2, . . . , qS , respectively) can be measured extendingthe analysis of the EVPI


Analysis of the Mixed strategy

0 10000 20000 30000 40000 50000

200

400

600

800


Willingness to pay

EV

PI

Optimal strategyMixed strategy: Atorvastatin=31.07% Fluvastatin= 4.20% Lovastatin= 4.35% Pravastatin= 6.84% Rosuvastatin=19.37% Simvastatin=34.18%



0 10000 20000 30000 40000 50000

200

400

600

800


Willingness to pay

EV

PI


• For each k, the impact of themixed strategy is an increasein the EVPI with respect tothe optimal scenario. The lossin expected value ofinformation reaches e690 fork = e50 000

• This depends on

a) The company that market anon-C/E alternative

b) The regulator that does notdisinvest from a non-C/Etreatment



0 10000 20000 30000 40000 50000

200

400

600

800


Willingness to pay

EV

PI


• For each k, the impact of themixed strategy is an increasein the EVPI with respect tothe optimal scenario. The lossin expected value ofinformation reaches e690 fork = e50 000

• This depends on

a) The company that market anon-C/E alternative

b) The regulator that does notdisinvest from a non-C/Etreatment

• The value of the extrauncertainty can be used to

a) Establish the amount ofinvestment for research thatwould be cost-effective toreduce the uncertaintyabout optimal decision

b) Determine/modify thereimbursed retail price

c) Represent the payback valuefrom the company to theregional provider


Conclusions

• PSA to parameter uncertainty is a fundamental part of each health economicevaluation, and it should complement the standard decision analysis


Conclusions


• The analysis of the expected value of information can be used to compare theideal decision process (when uncertainty on the parameters is resolved) to theactual one

• Therefore, it drives research prioritisation: if the value of acquiring furtherinformation to reduce uncertainty is too high, then the decision-maker shouldchoose the optimal treatment based on the current evidence


Conclusions




• The analysis can be extended to situations where the market generatesnon-optimal combinations of interventions

• The increase in the value of information can be considered as a paybackvalue which could then be invested in (other) research


Conclusions




• The analysis can be extended to situations where the market generatesnon-optimal combinations of interventions

• The increase in the value of information can be considered as a paybackvalue which could then be invested in (other) research

• NB: The R package BCEA (soon available on CRAN and for the momentdownloadable at www.statistica.it/gianluca/BCEA) allows to produce asystematic economic analysis based on the results of a suitable Bayesianmodel, including the mixed strategy analysis


Some references

Spiegelhalter D., Abrams K., Myles J. (2004). Bayesian Approaches toClinical Trials and Health-Care Evaluation. John Wiley and Sons, Chichester,UK.

Baio G., Russo P. (2009). A Decision-Theoretic Framework for theApplication of Cost-Effectiveness Analysis in Regulatory Processes.Pharmacoeconomics 27(8), 645-655

Baio G., Dawid A. P. (2011). Probabilistic Sensitivity Analysis in HealthEconomics. Statistical Methods in Medical Research. doi:0962280211419832

Baio G. (2012). Bayesian Methods in Health Economics. CRC-ChapmanHall, London, UK [out late 2012]


Thank you!


Documents

A decision-theoretic framework for the application of cost ... · A decision-theoretic framework for the application of cost-eﬀectiveness analysis in regulatory processes Gianluca