The Alberta Health Technologies

Decision Process: Post Policy

Implementation Review

Presenter: Sarah Flynn, Alberta Health

Authors: Dr. Anderson Chuck, Institute of Health Economics; Sarah Flynn, Alberta Health; Dr. Nina Buscemi, Alberta Health; Dr.

Kathryn Ambler, Alberta Health

Disclosure

I have no actual or potential conflict of interest

in relation to this topic or presentation.

2

The Alberta Health Technologies

Decision Process (AHTDP)

• Provincial review process for health technologies

and services that provides evidence and

information to inform decision-making

• A collaboration with Health Technology

Assessment Partners.

• Reviews consider: – effectiveness

– safety

– cost-effectiveness and budget impact

– ethical and legal implications

– patient and provider perspectives

– Potential policy approaches and implications

3

How do we assess impact?

• Health care decisions may be influenced by

clinical impact, cost-effectiveness findings.

• Limited opportunities to actively monitor

implementation of AHTDP-informed policy

decisions.

• Monitoring and evaluation.

4

Post-Policy Implementation Review

(PPIR)

• PPIR is a review of a policy decision to

determine:

• PPIR offers evaluation, accountability

5

Did the policy achieve the

desired results?

Were costs and benefits in line

with expectations?

Were there any unintended or

unforeseen consequences?

What are learnings for future policy development?

Pro

spec

tive

Retro

spective

Retrospective PPIR

6

• Key prerequisites for PPIR:

– Access to information and data

– Readiness to apply the findings

• Some policies are better candidates than others

– Ideal conditions for PPIR

– Policy implemented 2 – 5 years ago

– Clear and logical policy goals

– Adequate studies/evidence available

– Access to original participants

– Commitment of all participants

PPIR Framework

7

Testing for Preterm Labour

• Preterm labour/delivery occurs between 20 – 37 wks

• Leading cause of neonatal mortality and morbidity

• Contributes to neurodevelopmental problems,

respiratory/pulmonary dysfunction, hearing and visual

impairment, and other long-term health problems.

• Interventions are available to reduce

morbidity/mortality

• Issue:

– Not all symptomatic women will deliver

– Some women experiencing false labour are being

transferred and admitted to hospital

• Solution: A test which is good at identifying false

labour • The TLiIQ

® System or fetal fibronectin (fFN) testing

8

2006 AHDTP Review

9

• The fFN test was reviewed through the AHTDP to

determine if it should be publicly funded.

• Finding:

– fFN test can aid in ruling out unnecessary

interventions for women in false labour.

– fFN testing would result in cost savings to the

provincial health system through the avoidance of

ambulance transfers and decreased length of

stay.

The Policy Decision (2006)

10

• Introduce fFN testing for preterm labour as a

publicly funded service available to all Alberta

women by 2008

– Regional Health Authorities (RHAs) to implement

their preferred service delivery models and tests.

– Funding from existing budget allocations

– RHAs encouraged to implement quality assurance

mechanisms, guidelines and standards.

• RHAs to chose between fFN and an alternate

test (Actim™ Partus)

PPIR Methodology

11

Is there new evidence on performance differences between fFN and Actim™ Partus?

Literature Review Update

How was implementation conducted?

Barriers, facilitators, and unintended consequences

Key Informant Interviews

Health system impact and costs.

Transfers, length of stay and admissions

Economic Analysis

• Institute of Health Economics conducted a 2015

retrospective PPIR:

Do performance differences exist

between the two tests?

• Both the fFN test and the Actim™ Partus test had

high specificity and negative predictive values.

• Both the fFN test and the Actim™ Partus test had

low sensitivity and positive predictive values.

• FFN was more accurate at predicting preterm

delivery than Actim™ Partus .

• No evidence to suggest that the system adopted

in Alberta (fFN testing) should be changed based

on diagnostic performance

12

Was PTL testing implemented

according to the 2006 policy?

All Regional Health Authorities (RHAs) fully

implemented testing by 2008, using the same test

(fetal fibronectin).

RHAs covered costs through existing budgets*

Staff trained using a variety of materials

Providers generally trust the fFN test and

consider test results as part of their routine for

managing patients with PTL symptoms

Most of the training occurred at policy

implementation; levels of training varied

Access to testing equipment varied

13

Has PTL testing reduced

unnecessary hospital transfers,

admissions, and length of stay?

• Physicians appear to place more significance on

positive test results when deciding to transfer or

admit women (inappropriate use of test).

• Testing did not reduce unnecessary ambulance

transfers or admissions (women in false labour). – Ambulance transfers

• In true labour. 1.91 / 7.45 times more likely to transfer if

+/-

• In false labour: 0.78 / 2.22 times more likely to transfer +/-

– Admissions

• In true labour. 1.68 / 0.44 times more likely to transfer if

+/-

• In false labour: 5.38 / 0.47 times more likely to transfer +/-

14

Has PTL testing had any unintended

consequences?

• More patients being transferred or admitted,

regardless of test results

• fFN testing did not reduce unnecessary

ambulance transfers or admissions for women in

false PTL

• Testing has increased the number of appropriate

ambulance transfers and admissions for preterm

pregnancies in true labour.

15

Has PTL testing resulted in net cost

savings for the system?

• Testing did not reduce unnecessary use of

healthcare services, and associated costs.

• Total health system costs increased due to test

purchases and increased appropriate care due to

testing (more women in true PTL receiving care).

• The increase in health service utilization resulting

from testing has cost the health system an

estimated $3,458,443 for appropriate utilization

and $730,724 for unnecessary utilization

16

What next?

• Shared results with the key provincial stakeholder

group:

– The Maternal, Newborn, Child and Youth (MNCY)

Strategic Clinical Network (SCN)

• Feedback gathered

• The Alberta Advisory Committee on Health

Technologies made a formal recommendation to

the MNCY SCN:

– Do not maintain the status quo

– Consider the PPIR and Policy Analysis results

– Develop a plan of action to address inappropriate use of

preterm labour testing.

• March 20 decision to discontinue testing

17

Reflections

18

First PPIR of an AHTDP Decision:

• Challenges of a retrospective approach

• Working from an AHTDP review

• Capitalizing on HTA competencies

• Lessons for future reviews

Lessons

19

• Strengthening AHTDP reviews:

– Draw on HTA findings to develop comprehensive

policy options and recommendations

– Leverage HTA findings to support strategic,

measured implementation

• Strengthening PPIR:

– Identify a key stakeholder or “client”

– Prospectively evaluate or review policy decisions,

where possible