CFFE IN ONTARIO - National Pharmaceutical Council · CFFE IN ONTARIO ISPOR Symposium Orlando May 15, 2009 Ron Goeree, Jean-Eric Tarride, Feng Xie, James Bowen, Robert Hopkins, Lisa

1

CFFE IN ONTARIO

ISPOR SymposiumOrlando

May 15, 2009

Ron Goeree, Jean-Eric Tarride, Feng Xie, James Bowen, Robert Hopkins, Lisa Patterson, Gord Blackhouse, Daria O’Reilly

Programs for the Assessment of Technologies in Health (PATH) Research Institute, St Joseph’s Healthcare HamiltonDepartment of Clinical Epidemiology and Biostatistics, McMaster University

Improving health through health technology assessment

Programs for Assessment of Technologies in Health (PATH) Research Institute

2

Disclaimer

• Conditionally funded field evaluations (CFFE) (aka CED), began in Ontario in 2003 with the formation of the PATH Research Institute

• Since then, the term ‘field evaluations’ has been expanded by the Ontario MOHLTC to also include other activities (some not research) done internally and by other centers and units

• This presentation reflects how PATH conducts its field evaluation program, our contributions to CFFEs in Ontario, and our lessons learned



3

Overview

• What are CFFEs?– Mostly from PATH’s perspective

• Some examples of CFFEs and policy impact• What are the challenges for government

when trying to implement CFFEs/CED?• What are the challenges for researchers

when trying to conduct CFFEs/CED?• Conclusions for moving forward



6

Request for funding MAS Prioritization & literature review (Assessment of safety, efficacy, cost-effectiveness – Internally by Ontario MOHLTC)

Not enough information for evidence-based recommendations (informational uncertainty is too high)

Collect new/additional data (‘Field Evaluation’)

(trial, registry, patient outcomes, resource utilization, costs, etc.)

Prepare/update HTA with interim or final ‘Field Evaluation’ data(model re-estimated by combining old and new evidence)

Build preliminary cost-effectiveness model

(based on systematic review ofclinical & economic literature)

1-3 Years3-6 Months

Review of Evidence (OHTAC) Policy Recommendations (to Deputy Minister of Health)

HTA

Fee

dbac

k Lo

ops

to D

ecis

ion

Mak

ers

Estimate the value of collecting additional information (VOI)Source: Goeree R & Levin L, Pharmacoeconomics 2006;24(11):1-14.

Policydecisionmaking

PATH’s Evidence-Based CFFE Process

Uncertainty and Types of CFFEsAttribute of Technology

Questions HTAs typically address

Typical Decision Making Uncertainty

Types of CFFEs used to address uncertainty

QualityIs the technology consistent and of

high quality?

Lack of quality evidence or inconsistency in quality of

the technology

Technology quality assurance assessments

SafetyDoes the technology harm patients/health care professionals?

Context-specific safety concerns

Safety assessments, development of guidelines or

standards of practice

EfficacyCan the technology work in experimental

trial conditions?

Poor quality evidence, lack of evidence, or conflicting

evidence of efficacyExplanatory RCTs

EffectivenessDoes the technology work in ‘real world’

practice?

Generalizability of efficacy data or transferability of

clinical evidence

Pragmatic RCTS, observational studies (e.g.

cohort, registries)

Value for money

Is the technology cost-effective?

Transferability of economic & patient preference

evidence

Effectiveness studies including economic and patient preference information

Source: Goeree, Levin, Chandra, et al, JACR 2009;6(5):332-42

8

Example: CFFE of DES vs BMS in Ontario(OHTAC concerned over ‘real world’ effectiveness)



9

DES CFFE Impact

• OHTAC recommended:– DES be offered to high risk patients only– Diabetes– Long lesions (greater than 20mm) – Narrow lesions (less than or equal to 2.75mm)

• Funding for DES provided (initially $12m/year)• Overall funding was based on restricted DES

use as recommended by OHTAC



10

Example: CFFE of EVAR vs OSR for AAA (OHTAC concerned about longer term safety and effectiveness of EVAR)



11

EVAR CFFE Impact• OHTAC recommended:

– Increased access to EVAR in high risk patients

• EVAR has been changed from an uninsured to insured service (fee code introduced)

• Several vascular programs in the province (& other provinces) were restructured to accommodate EVAR

• Hospitals started receiving EVAR-specific funding (on top of global hospital budgets)



12

Challenges for Government• Where do you find the resources for CFFE?

– Savings from removal of ineffective or obsolete technologies– Savings from reduced out-of-province procedures–during CFFE– Savings from controlled diffusion of technology (e.g. DES)– Technology manufacturer collaboration (unrestricted grants)

• SM procrastination - Wait, wait, wait and then hurry up!!– “We don’t need that yet” --- “We need the results yesterday”– Designing and implementing a high quality CFFE takes time

• Political relevance “SM suffers from Attention Deficit Disorder ”• Research spans across political cycles (LT perspective)• Demonstrating value of CFFEs - Savings simply absorbed

and spent elsewhere, not reinvested back into CFFE program



13

October 21, 2006

~90%

http://www.nytimes.com/�

IMPACT OF DES FINDINGS Controlled diffusion of technology ($15 to $38 million) 2007/08 savings (decreased expenditures) from using CFFE findings

in making policy decision ~ $20 million ($58 million vs $38 million)

0

10

20

30

40

50

60

70

95/9

6

96/9

7

97/9

8

98/9

9

99/0

0

00/0

1

01/0

2

02/0

3

03/0

4

04/0

5

05/0

6

06/0

7

07/0

8

03/0

4

04/0

5

05/0

6

06/0

7

07/0

8

Drug Eluting Stents (DES)Bare Metal Stents (BMS)

Start of Study

$23 million

Forecast using US uptake rates

31%

78%

86%86% 86%

$20 million

Expe

nditu

res

on s

tent

s ($

mill

ions

)

15

Challenges for Researchers• Ideally would like preliminary model and VOI to inform

study design and data collection needs– Due to time constraints, difficult to complete before CFFE starts

• Ideally VOI should inform future data collection– HTA is an iterative process, but pressure to ‘move on’– OHTAC – recommended mortality data collection for DES

• Rigor!!! – We want to do methodologically sound CFFEs – Planning takes time (Study Working Groups – key stakeholders)– Proper question/design (sample size, follow-up, range outcomes)– Protocol development, CRFs and REBs (one study 100 REBs)– Contractual and legal delays (sometimes 3rd party with industry)– Peer-review (self imposed) and QA delays (critical 4 acceptance)



Appropriate safe and

consistent use of technology

during evaluation

Development & implementation

of field evaluation

Conduct of the field

evaluation

Analysis, reporting of

results and KT activities

Pillars of PATH’s QA ActivitiesQAIC SOPs

17

Conclusions – Moving Forward• CFFEs take time, number are maturing • Limited completed studies, but to date Ontario

collaborative model has been very successful• Government, researchers and health care

professionals all working toward common goal• Local data collection carries significant weight

in gaining stakeholder support for CFFE results and for policy implementation & funding (100%)

• Funding & time challenges (do it quick vs more rigorous study – political vs scientific needs)



Documents

CFFE IN ONTARIO - National Pharmaceutical Council · CFFE IN ONTARIO ISPOR Symposium Orlando May 15, 2009 Ron Goeree, Jean-Eric Tarride, Feng Xie, James Bowen, Robert Hopkins, Lisa