Optimal Drug Development Programs and Efficient Licensing and Reimbursement

Regimens

Neil HawkinsKarl Claxton

CENTRE FOR HEALTH ECONOMICS

Overview

• Societal and commercial value of Information• Decision rules incorporating value of information• Challenges

The Quantitative Estimate of the Value of Sample Information

The value of additional sample information is the value of the increased likelihood of selecting the optimum treatment arising from the reduction in uncertainty regarding treatment effects (and costs).

What is the optimum treatment?

The treatment with greatest expected net benefit in terms of costs and effects- Bayesian Decision Rule.

-20000 -10000 0 10000 20000

Net Benefit(tx) - Net Benefit(placebo)

Net Benefit(€)

Pro

ba

bili

tyFavours Placebo Favours Treatment

Decision Uncertainty

Expected Net Value of Sample Information (ENVSI)

Expectation over potential future samples of:

Net benefit from optimum decision made including the additional sample data —

Net benefit from optimum decision based on existing data—

Cost of collecting sample

Note: ENVSI < 0, if the optimum decision does not change due to extra sample data

Bayesian Simulation of Future Samples for Binomial Parameter

Sample P from current posterior distribution:

Pcurrent ~ beta(a,b)

Simulate Trial Data

rT ~ bin(n, PTx )

Calculate new posterior distribution

Pnew ~ beta(a+r,b+n-r)

Small Sample (n=1)

-20000 -10000 0 10000 20000

Net Benefit(tx) - Net Benefit(placebo)

Net Benefit(€)

Pro

ba

bili

ty

Favours Placebo Favours Treatment

Current SampleFuture Samples

Large Sample (n=2000)

-20000 -10000 0 10000 20000

Net Benefit(tx) - Net Benefit(placebo)

Net Benefit(€)

Pro

ba

bili

tyFavours Placebo Favours Treatment

Current SampleFuture Samples

Example Phase II Trial Results

Tx Placebo Response 16 8 Deaths 1 1 Total 49 45

χ2 Test: 2.0037, 1 df, p = 0.1569

Decision Analytic Model

Net Benefit =

P(Resp) x QALYs Gained | Resp x Monetary Value of a QALY - P(Death) x QALYs Lost | Death) x Monetary Value of a QALY -Treatment Cost

Example Parameter Estimates

QALYs Gained | Response : ~ N(0.7,0.12) x 4

QALYs Lost | Death : ~ N(0.7,0.12) x 4

Value of a QALY: £30,000

Treatment Cost Per Course :(<10% Response) £12,000 (≥10% & < 20% Response) £14,000

(≥20% Response) £16,000

Treatment Population: 20,000

Production Costs: £150,000,000

Trial Costs: (Fixed) £10,000,000 (per Patient) £20,000

Based on Current Data

• New treatment is cost-effective• New treatment would not get approval based on a

frequentist hypothesis test

Societal Value of Sample Information (Efficacy Trial)

0 100 200 300 400 500

05

00

01

00

00

15

00

02

00

00

25

00

0

Societal Value of Information Efficacy Endpoint

Sample Size

Va

lue

of I

nfo

rma

tion

Commercial Value of Sample Information

The value of increased sales due to the increased probability of regulatory and reimbursement approval arising from the extra information

ICH E9: Guidance on Statistical Principles for

Clinical Trials

Using the usual method for determining the appropriate sample size, the following items should be specified:

• probability of erroneously rejecting the null hypothesis • probability of erroneously failing to reject the null

hypothesis

ICH E1A: The Extent of Population Exposure to Assess Clinical Safety

• 100 patients exposed for a minimum of one-year is considered to be acceptable to include as part of the safety data base.

• It is anticipated that the total number of individuals treated with the investigational drug, including short-term exposure, will be about 1500.

Probability of Approval (Efficacy Endpoint – Current Regulatory Regimen)

0 100 200 300 400 500

0.0

0.2

0.4

0.6

0.8

Probability of Regulatory Approval Efficacy Endpoint

Sample Size

Pro

ba

bilit

y of

Ap

pro

val

Value of Sample Information (Efficacy Endpoint - Current Regulatory Regimen)

0 100 200 300 400 500

01

00

00

20

00

03

00

00

40

00

05

00

00

60

00

07

00

00

Societal and Commercial Value of Information Efficacy Endpoint

Sample Size

Va

lue

of I

nfo

rma

tion Commercial

Societal

What happens if we just use a Bayesian CE decision rule?

Value of Sample Information (Efficacy Endpoint - Bayesian CE Decision Rule)

0 100 200 300 400 500

01

00

00

20

00

03

00

00

40

00

05

00

00

60

00

07

00

00

Value of Information (Bayesian Decision Rule) Efficacy Endpoint

Sample Size

Va

lue

of I

nfo

rma

tion

CommercialSocietal

Societal Value of Sample Information (Utility Study)

0 20 40 60 80 100

02

00

40

06

00

80

01

00

0Societal Value of Information

Utility Endpoint

Sample Size

Va

lue

of I

nfo

rma

tion

Value of Sample Information (Utility Study – Current Regulatory Regimen)

0 20 40 60 80 100

02

00

00

40

00

06

00

00

80

00

0

Value of Information (Bayesian Decision Rule) Utility Endpoint

Sample Size

Va

lue

of I

nfo

rma

tion

CommercialSocietal

Implications

• Under current regulatory system we might expect a lack of outcomes and long-term data

• We need to consider uncertainty and resulting VOI when making decisions, not just expectations based on current data

• How should we do this?

Maybe we shouldn’t abandon frequentist hypothesis testing just yet?

The FDA view

“...A reasonable basis for a claim [of cost-effectiveness] depends on a number of factors relevant to the benefits and costs of substantiating a particular claim. These factors include: the type of product, the consequences of a false claim, the benefits of a truthful claim, the costs of developing substantiation for the claim ...”

Potential Industry Responses to Approval based on Value of Information

Reduce cost of uncertainty by research or price reduction

Trade-off between:

•Additional research-Cost, delay and uncertain outcome

-Entry and free rider

•Price reduction-Reduces EVI (for payoffs > 0) but reduces revenues

Approval Based on Expected Net Value of Sample Information

Approve new (more expensive?) treatment if expected net benefit of treatment is greater than existing treatment and expected net value of further sample Information is zero

Approval Based on Expected Net Value of Sample Information

• Hard to define set of endpoints, study designs and sample space over which we calculate value of sample information

• ENVSI is uncertain and will change as data become available. When is ENVSI defined?

• Many of the parameters required to estimate ENVSI are uncertain and may not be transparent

• Non-financial capacity restraints on further research• What decision do we make in the interim? - Sunk costs,

irreversibility and option value

Approval based on Expected Value of Perfect Information

Population EVPI for pricing decisions

£0

£500,000

£1,000,000

£1,500,000

£2,000,000

£2,500,000

£3,000,000

£3,500,000

£4,000,000

£4,500,000

£5,000,000

£0 £10,000 £20,000 £30,000 £40,000 £50,000 £60,000 £70,000 £80,000 £90,000 £100,000

Cost-effectiveness threshold

Po

pu

latio

n E

VP

I

Price = £24

Price = £12

Relative risk of progression for copaxone, Betaferon and rebif(22mg) (£14m, £13.6m and £7m respectively)

Also the cost of care, costs of relapse and quality of life (£10m, £7m and £6m respectively)

£86.2mRelapsing remitting and primary progressive multiple sclerosis (scenario 2)

Disease modifying therapies for multiple sclerosis

Specificity (£3.6m)£20mWomen aged 18 to 64 years (scenario 3)

Liquid Based Cytology

Relative risks of vascular and non vascular death

(£780m for ASA-MR-dipridamole compared to clopidogrel in the stroke subgroup)

£865m

£250m

£710m

£240m

Stroke

Transient Ischaemic Attack

Myocardial Infarction

Peripheral Arterial Disease

(scenario 2)

Clopidogrel and dipyridamole for secondary prevention

Quality of life with influenza, the effect of oselatimivir and amantadine (£44.3m, £0.43m and £0.23m respectively)

£66.7mOtherwise healthy adults not at elevated risk of complications

Neurominidase inhibitors

Relative risk of death for non acute PCI for GPA as medical management and for Clopidogrel(£85,041,000, and £68,137,000 respectively)

£171mAcute treatment following non-ST-elevation acute coronary syndrome (scenario 2)

Glycoprotein IIb/IIIa

Quality of life with and without PDT (£3,370,000 for 20/40)

£6.2m

£15.3m

Visual acuity 20/40

Visual acuity 20/80

AMD Screening

EVPI for parametersPopulation EVPIPatient GroupCase Study

Relative risk of progression for copaxone, Betaferon and rebif(22mg) (£14m, £13.6m and £7m respectively)

Also the cost of care, costs of relapse and quality of life (£10m, £7m and £6m respectively)

£86.2mRelapsing remitting and primary progressive multiple sclerosis (scenario 2)

Disease modifying therapies for multiple sclerosis

Specificity (£3.6m)£20mWomen aged 18 to 64 years (scenario 3)

Liquid Based Cytology

Relative risks of vascular and non vascular death

(£780m for ASA-MR-dipridamole compared to clopidogrel in the stroke subgroup)

£865m

£250m

£710m

£240m

Stroke

Transient Ischaemic Attack

Myocardial Infarction

Peripheral Arterial Disease

(scenario 2)

Clopidogrel and dipyridamole for secondary prevention

Quality of life with influenza, the effect of oselatimivir and amantadine (£44.3m, £0.43m and £0.23m respectively)

£66.7mOtherwise healthy adults not at elevated risk of complications

Neurominidase inhibitors

Relative risk of death for non acute PCI for GPA as medical management and for Clopidogrel(£85,041,000, and £68,137,000 respectively)

£171mAcute treatment following non-ST-elevation acute coronary syndrome (scenario 2)

Glycoprotein IIb/IIIa

Quality of life with and without PDT (£3,370,000 for 20/40)

£6.2m

£15.3m

Visual acuity 20/40

Visual acuity 20/80

AMD Screening

EVPI for parametersPopulation EVPIPatient GroupCase Study

Summary results of the NICE pilot study

Approval based on Expected Value of Perfect Information

Approve new therapy if Expected Value of Perfect Information is below a given threshold at an acceptable cost-effectiveness threshold

• Requires an arbritary EVPI threshold for approval• Parameters still uncertain. For example; relevant time

horizon, future technological change.

Approval based on Decision Uncertainty

Cost-effectiveness acceptability curves for pricing decisions

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

£0 £10,000 £20,000 £30,000 £40,000 £50,000 £60,000 £70,000 £80,000 £90,000 £100,000

Cost-effectiveness threshold

Pro

ba

bili

ty c

ost

-eff

ect

ive

Price = £24

Price = £12

Approval based on Decision Uncertainty

Approve new therapy if decision uncertainty is below a given threshold at an acceptable cost-effectiveness threshold

• Requires an arbritary uncertainty threshold for approval

Some Challenges

• How we consider uncertainty when decision making will influence the availability of evidence

• How do we frame explicit decision rules incorporating uncertainty?