Embed Size (px)
Citation preview
ORIGINAL PAPER
Public funding of pharmaceuticals in the Netherlands:investigating the effect of evidence, process and contexton CVZ decision-making
Karin H. Cerri • Martin Knapp • Jose-Luis Fernandez
Received: 4 July 2012 / Accepted: 20 June 2013
� Springer-Verlag Berlin Heidelberg 2013
Abstract The College Voor Zorgverzekeringen (CVZ)
provides guidance to the Dutch healthcare system on
funding and use of new pharmaceutical technologies. This
study examined the impact of evidence, process and con-
text factors on CVZ decisions in 2004–2009. A data set of
CVZ decisions pertaining to pharmaceutical technologies
was created, including 29 variables extracted from pub-
lished information. A three-category outcome variable was
used, defined as the decision to ‘recommend’, ‘restrict’ or
‘not recommend’ a technology. Technologies included in
list 1A/1B or on the expensive drug list were considered
recommended; those included in list 2 or for which patient
co-payment is required were considered restricted; tech-
nologies not included on any reimbursement list were
classified as ‘not recommended’. Using multinomial
logistic regression, the relative contribution of explanatory
variables on CVZ decisions was assessed. In all, 244
technology appraisals (256 technologies) were analysed,
with 51 %, of technologies recommended, 33 % restricted
and 16 % not recommended by CVZ for funding. The
multinomial model showed significant associations
(p B 0.10) between CVZ outcome and several variables,
including: (1) use of an active comparator and demon-
stration of statistical superiority of the primary endpoint in
clinical trials, (2) pharmaceutical budget impact associated
with introduction of the technology, (3) therapeutic indi-
cation and (4) prevalence of the target population. Results
confirm the value of a comprehensive and multivariate
approach to understanding CVZ decision-making.
Keywords Health technology assessment � HTA � CVZ �Payer decision-making � Reimbursement � Coverage �Pharmaceuticals � Appraisal � Dutch reimbursement
JEL Classification I180
Introduction
In the heavily regulated European pharmaceutical market,
one of the areas under close scrutiny is the public funding of
pharmaceutical technologies. Health technology assessment
(HTA) is a process that is prominent in several European
Union (EU) Member States to advise healthcare systems on
the appropriate use of a new technology (such as a pharma-
ceutical product) and whether it should be recommended for
public funding. Member States differ in the requirements that
they have put in place to manage patient access to technol-
ogies once marketing authorisation has been granted. Pricing
and reimbursement requirements are present in most EU
Member States, although they vary in terms of both the
appraisal of the evidence and the role of HTA.
In the Netherlands, the College Voor Zorgverzekeringen
(CVZ) has an important role in supporting and maintaining
the quality, accessibility and affordability of healthcare
through its HTA activities. The Commissie Farmaceutische
Hulp (CFH) is a part of the CVZ. It is tasked with the
assessment of the therapeutic value of new technologies for
inclusion in the medicine’s reimbursement system (Gene-
esmiddelen Vergoedings Systeem—GVS) or inclusion in
the various reimbursement policies of the Dutch Health
Authority (Nederlandse Zorgautoriteit, NZa). Collectively,
the role of the CFH and CVZ is to provide advice to the
Ministry of Health on the reimbursement list appropriate
for the funding of each technology—the GVS 1A or 1B
K. H. Cerri (&) � M. Knapp � J.-L. Fernandez
London School of Economics and Political Science,
Houghton Street, London WC2A 2AE, UK
e-mail: [email protected]
123
Eur J Health Econ
DOI 10.1007/s10198-013-0514-z
list, or specific reimbursement policies (e.g. the expensive
drug list, or list ‘2’) (Fig. 1) [1, 3, 4]. Thus, the Ministry of
Health has the final decision-making authority with regard
to the reimbursement of new technologies [1, 2].
In assessing eligibility for reimbursement, the therapeutic
value of a technology plays an important role in the CVZ
recommendation [1]. The CVZ takes into account several
factors, including the disease for which the technology is
indicated and whether or not there is a standard of care
already reimbursed for that system with the same indication,
based on clinical guidelines and clinical criteria. The review
then considers various aspects of the technology to determine
whether an incremental therapeutic benefit exists. This
includes, among other things, the review of efficacy and
safety, based on evidence from randomised controlled trials
(RCTs) and safety registries. The review also collects infor-
mation about experience with use of the technology to assess,
for example, the risk of unknown side effects and to reduce
uncertainty around the therapeutic benefit of the technology.
The financial implications or cost consequences of adoption
of the technology are also assessed. The importance of each
factor is weighed and technologies are compared to reach a
decision on the category of therapeutic benefit that should be
applied (lower benefit, comparable benefit or higher benefit).
For inclusion on the GVS 1B list, pharmacoeconomic eval-
uation has been a mandatory requirement since 2005 [3]. For
inclusion in the expensive drug list, specific conditions must
be met including demonstration of therapeutic benefit and
conditional on further real-life assessments being performed
[4]. Established in 2006, this process represents a type of
conditional coverage mechanism, where the technology is
introduced, observed and re-evaluated in a defined period of
time according to a defined process.
The CVZ also has a particular role, perhaps less common in
other HTA bodies, of reviewing technologies for unlicensed
indications upon the request of the health insurance bodies. In
this situation, the CVZ is asked to establish whether the
unlicensed indication is rare (prevalence of less than
1:150,000 population), whether there is a scientific basis for
the efficacy of the technology in this unlicensed indication and
whether there is any other alternative therapy available in the
Netherlands for the condition under review.
Given the multidimensional nature of CVZ decision-
making and the high degree of stakeholder involvement, it
could be hypothesised that decisions are influenced not
only by the evidence supporting the technology, but also by
the assessment processes used and the context in which
they operate. The aim of this study was to identify evi-
dence, process and contextual factors that significantly
impact on CVZ decision outcomes while adjusting for the
presence of other influences.
Methods
Hypothesised drivers of CVZ decision-making
It was hypothesised, based on the available literature, that
CVZ decisions were driven by the CVZ decision-making
process, evidence considered within that process, and the
Marketing Authorisation
granted
Extramural therapy
Therapeutic equivalence
GVS list 1A
GVS List 2 with(out) patient
copayment
Not reimbursed
Incremental Therapeutic benefit
GVS list 1B
GVS List 2 with(out) patient
copayment
Not reimbursed
Intramural therapy
Expensive Drug
“Expensive Drug List”
Hospital budget and no reimbursement
Not Expensive Drug
Hospital budget and no reimbursement
Classified as “recommended”
Classified as “not
recommended”
Classified as “restricted”
Fig. 1 Pharmaceutical
reimbursement system in the
Netherlands and classification
CVZ coverage decisions utilised
in the analysis
K. H. Cerri et al.
123
socio-economic and political contexts in which decisions
were made. Research on HTA decision-making has shown
that the evidence related to the medicine or other tech-
nology under review (whether clinical, economic or
otherwise) can influence decisions. As the therapeutic
value of the technology plays an important role in the CVZ
recommendation, it was hypothesised that the nature of the
comparator used in the clinical trial, whether active or
placebo, would have an effect on decision-making [1]. In a
previous discrete-choice experiment among Dutch health-
care professionals, the analysis of choices made suggested
that severity of disease was one of the most significant
factors driving coverage decisions [5]. This presents an
opportunity to further investigate the extent to which dis-
ease characteristics influence CVZ decision-making, when
taking into account other confounding factors.
With regards to the role of economic evidence, it was
hypothesised that both affordability and efficiency criteria
would have an impact on CVZ decision-making based on
the understanding that both these principles are part of the
review process and because previous studies of CVZ
decision-making have highlighted the importance of eco-
nomic criteria [3, 5–9].
The literature examining the HTA appraisal process
provided insights into a number of process-related factors
that can potentially influence HTA decisions [10–16]; these
too were therefore identified as of relevance for this
analysis.
Finally, reference in the literature was made to the
impact of overall healthcare and welfare characteristics on
HTA decision-making, such as healthcare spending per
capita, societal willingness to pay, the structure of the
healthcare system, as well as ethical and social consider-
ations [11, 12, 15, 17–22].
Thus, the hypotheses tested in this analysis of CVZ
decision-making were:
• Choice of comparator in clinical trials significantly
impacts on CVZ outcomes: use of active rather than
placebo comparators increases the odds of recommen-
dation versus non-recommendation or restriction.
• The therapeutic area for which the technology is
indicated impacts on CVZ decisions: therapeutic areas
differ in their impact on the odds of recommendation
versus restriction or versus non-recommendation.
• Pharmaceutical budget impact estimates significantly
impact on CVZ outcomes: increasing budgetary impact
is hypothesised to increase the log-odds of non-recom-
mendation or restriction relative to recommendation.
• The use of cost-effectiveness analysis (introduced in
2005) plays a role in CVZ decision-making: higher
incremental cost-effectiveness ratios (ICERs) are
hypothesised to be associated with increased odds of
restriction or non-recommendation relative to
recommendation.
• CVZ restrictions and non-recommendations are
increasing over time relative to recommendations.
Sample
The pharmaceutical technology appraisals performed by
CVZ formed the basis for the sample included in this
analysis. The composition of the sample was determined
through the following inclusion and exclusion criteria. The
sample included all pharmaceutical technology appraisals
(which are the only type of technology appraised by the
CVZ) conducted in the period 2004–2009 indicated for an
adult population. The rationale for the latter restriction was
related to the fact that marketing authorisation for paedi-
atric indications follows a specific centralised regulatory
pathway and that the nature of the clinical evidence
package is influenced by this process. Also, paediatric
indications are relatively infrequent and therefore the rel-
evance of including these was questioned, with priority
given to generating a sample of technologies that was
representative of the majority of decisions made by the
CVZ, and thus focusing on the patient population aged 18 and
over. A 5-year time horizon was used to capture a sufficient
number of appraisals for analysis. Technology appraisals were
excluded from the analysis for any of the following reasons:
(1) they focussed on a non-adult population (aged\18 years);
(2) the appraised technologies were non-pharmaceutical
interventions; (3) marketing authorisation was withdrawn; or
(4) the full guidance was not available.
Outcome variable
The analysis was designed to reflect as closely as possible
the way CVZ makes its decisions, while balancing this with
the need to utilise an outcome variable that could be
implemented within a multivariate analysis. To this end, a
three-category outcome variable was used where technol-
ogies were recommended, restricted or not recommended.
The technology was considered recommended if it was
placed in the basic package (‘basis paket’), i.e. lists 1A or
1B, without any restriction or patient co-payment. Where
the decision was to place the technology in list 2 or in the
basis paket, but only for use in a sub-population or with a
patient co-payment, this technology was considered as
restricted. Finally, the technology was considered as not
recommended when it was designated as ‘not recom-
mended’ and/or was not included on any reimbursement
list (Fig. 1). Ranking of the three potential outcomes was
not implemented as ranking of outcomes can vary
according to the perspective adopted.
Public funding of pharmaceuticals in the Netherlands
123
Explanatory variables
In line with the hypothesised drivers of HTA decision-
making, 29 variables were defined including those relating
to (1) the clinical and economic characteristics of the
technology under appraisal [such as characteristics of
randomised controlled trials (RCT), use of observational
data, ICER reported]; (2) the processes used to come to a
decision (whether or not the technology was included in the
expensive drug list (Dure geneesmiddelen lijst); (3) the
socioeconomic context in which these decisions were made
(including percentage of gross domestic product (GDP)
spent on healthcare, year of appraisal and whether the
disease targeted by the technology was identified as a
priority by the healthcare system).1 These variables are
shown in Table 1. A number of variables were included to
investigate our research hypotheses, each testing a specific
aspect of a given hypothesised association.
Data set
A database of information pertaining to CVZ decisions and
explanatory variables of interest was developed. To create
this database, several steps were implemented. Publicly
available sources of data containing information on the
variables of interest were identified. Data extraction was
performed by a single researcher (KC), using a form
developed for the study to extract information from the
different appraisals in a way that was as transparent,
reproducible and consistent as possible. Finally, the
resulting extracted data were coded and prepared for
analysis. Variable definitions and data sources are shown in
Table 1.
Statistical analyses
Descriptive statistics were calculated for each extracted
variable, stratified by outcome group (recommended,
restricted or not recommended). For categorical variables,
we used the chi-squared test to test for differences in
proportions across the three outcomes. For continuous
indicators, we used the ANOVA test to test for differences
between means for normally distributed indicators and the
Kruskal-Wallis test to identify differences between ranks
of means for not normally distributed indicators. For all
tests, the results indicate whether significant differences
were identified at the 0.05 level of significance.
A multinomial logit regression was used in the analysis
to model the probabilities associated with the three types of
technology appraisal outcome. The ‘recommended’ out-
come was selected as the referent category in the analysis.
The objective of the analysis was to identify, ceteris
paribus, the effect of a range of factors potentially asso-
ciated with CVZ appraisal decisions and to assess which
combination of factors best explains the pattern of CVZ
decisions. Given the wide range of factors considered in the
analysis (see Table 1), a process was developed to deter-
mine which explanatory variables would appear in the final
specification of the model. This process involved the fol-
lowing steps:
• First, bivariate regression models were run to ascertain
the degree of correlation between individual explana-
tory variables and appraisal decisions.
• On the basis of these models, a subset of indicators was
selected that included those variables that showed at
least moderate significance levels (indicators with
p value below 0.25). A preliminary model was
estimated including these indicators.
• The model was reduced by removing those variables
with significance levels above the 0.10 threshold. To
guarantee its stability, this ‘base’ model was re-
estimated by sequentially removing one variable at a
time and verifying the stability of the effects on the
coefficient and significance level of the remaining
estimates.
• The model was subsequently tested through alternative
model specifications to examine its robustness and to
assess the sensitivity of the results to different
assumptions.
• As a final step, the base-case model results were
presented to representatives of CVZ in 1-h telephone
interviews, to seek feedback on the variables identified
within the base-case model, the coefficient and level of
significance to assess the validity of the model.
The application of the model-specification process outlined
above facilitated the interpretation of the results of the
models whilst allowing the analysis to explore the impact
of the wide range of indicators collected in the study. A
step-by-step process was followed to look for evidence of
collinearity within the set of regressors.
While significant effort was made to identify informa-
tion relevant to the variables of interest, a limited propor-
tion of data could not be found. To maximise the sample
size, imputation techniques were used to estimate entries
for missing observations. Missing values were replaced
with regression imputation estimates using the ‘impute’
command in STATA software (Intercooled (IC) Stata
version 10.1). The imputed values obtained were then
checked manually to ensure their face validity. In addition,
1 This variable aims to capture whether the pharmaceutical product in
question is linked to a disease area that is prioritised by the Ministry
of Health, by examining government plans/health documents that
highlight national healthcare system focus.
K. H. Cerri et al.
123
Ta
ble
1C
VZ
dat
ase
t:in
clu
ded
var
iab
les,
thei
rd
efin
itio
n,
dat
aex
trac
tio
nru
lean
dd
ata
sou
rces
No
.V
aria
ble
des
crip
tor
Var
iab
len
ame
Un
ito
fm
easu
reD
efin
itio
n
Ev
iden
ce—
clin
ical
and
dis
ease
-rel
ated
var
iab
les
1N
um
ber
of
RC
Ts
con
sid
ered
in
dec
isio
n
No
RC
TC
ou
nt
Th
en
um
ber
of
dis
tin
ctra
nd
om
ised
con
tro
lled
tria
ls(R
CT
s)th
atp
rov
ide
dat
a
rela
ted
toth
eth
erap
euti
cin
dic
atio
nu
nd
erev
alu
atio
n.
Ex
clu
ded
:st
ud
ies
that
are
sin
gle
-arm
,h
ave
no
ran
do
mis
atio
n,
or
are
no
n-i
nte
rven
tio
nal
(1)
2S
ize
of
po
pu
lati
on
incl
ud
edin
RC
Ts
RC
Tsi
zeN
um
eric
Mea
nn
um
ber
of
pat
ien
tsp
erR
CT
(1)
3L
eng
th/e
xte
nt
of
foll
ow
-up
inR
CT
RC
Td
ura
tio
nN
um
eric
Mea
nn
um
ber
of
wee
ks
ov
erw
hic
hd
ata
are
coll
ecte
do
np
atie
nts
that
ente
red
the
RC
Ts
(see
var
iab
len
o.
1)
(1)
4S
tati
stic
ally
sig
nifi
can
tsu
per
ior
resu
lts
RC
Tsu
per
ior
Cat
ego
rica
l(s
up
erio
r/
no
tsu
per
ior/
inco
nsi
sten
tre
sult
s)
Pre
sen
ceo
fst
atis
tica
lly
sig
nifi
can
tsu
per
iori
tyo
fte
chn
olo
gy
ver
sus
com
par
ato
r
for
pri
mar
yen
dp
oin
t(s)
.If
the
tech
no
log
ysh
ow
edst
atis
tica
lly
sig
nifi
can
t
sup
erio
rity
ino
ne
tria
l,b
ut
no
tin
ano
ther
,th
ere
sult
sw
ere
con
sid
ered
tob
e
‘in
con
sist
ent’
.R
CT
sd
esig
ned
as‘n
on
-in
feri
ori
ty’
stu
die
sw
ere
clas
sifi
edas
no
t
dem
on
stra
tin
gsu
per
iori
ty(i
.e.
‘no
’)(1
)
5R
elev
ance
of
RC
Tto
pay
erd
ecis
ion
RC
Tco
mp
arat
or
Nu
mer
icP
erce
nta
ge
of
RC
Ts
wh
ere
acti
ve
com
par
ato
rw
asu
sed
,as
op
po
sed
top
lace
bo
(1)
6N
um
ber
of
ob
serv
atio
nal
stu
die
s
con
sid
ered
ing
uid
ance
Ob
sStu
die
sC
ou
nt
Nu
mb
ero
fo
bse
rvat
ion
alst
ud
ies
pro
vid
ing
info
rmat
ion
tosu
pp
ort
stu
dy
dru
g.
Ob
serv
atio
nal
stu
die
sar
ed
efin
edas
no
n-i
nte
rven
tio
nal
(i.e
.d
on
ot
exp
lici
tly
req
ues
tth
ep
atie
nt
tota
ke
par
ticu
lar
med
icat
ion
or
the
ph
ysi
cian
tofo
llo
w
par
ticu
lar
pro
toco
l)(1
)
7P
rio
rity
dis
ease
area
Pri
ori
tyC
ateg
ori
cal—
yes
/no
Th
isv
aria
ble
cap
ture
sw
het
her
the
ph
arm
aceu
tica
lin
qu
esti
on
isli
nk
edto
a
dis
ease
area
that
isp
rio
riti
sed
by
the
Min
istr
yo
fH
ealt
h.
Pri
ori
tyd
isea
sear
eas
wer
eid
enti
fied
by
exam
inin
gg
ov
ern
men
tp
lan
s/h
ealt
hd
ocu
men
tsth
ath
igh
lig
ht
nat
ion
alh
ealt
hca
resy
stem
focu
s(2
)
8O
rph
anst
atu
sO
rph
anD
esig
Cat
ego
rica
l—y
es/n
oT
his
var
iab
leca
ptu
red
info
rmat
ion
on
wh
eth
ero
rn
ot
the
tech
no
log
yw
as
reco
gn
ised
by
the
Eu
rop
ean
Med
icin
esA
gen
cy(E
MA
)as
ano
rph
and
esig
nat
ed
med
icin
e(3
)
9T
her
apeu
tic
area
BN
F1
-12
Cat
ego
rica
l—1
2
cate
go
ries
Th
eB
riti
shN
atio
nal
Fo
rmu
lary
(BN
F)
cate
go
ries
wer
eu
sed
tocl
assi
fyea
ch
tech
no
log
yin
toth
eco
rres
po
nd
ing
ther
apeu
tic
area
(4)
10
Siz
eo
fel
igib
lep
op
ula
tio
nfo
r
trea
tmen
t
Eli
gib
leP
op
Nu
mer
icR
epo
rted
nu
mb
ero
fp
atie
nts
elig
ible
for
trea
tmen
t,as
per
the
sum
mar
yp
rod
uct
char
acte
rist
ics
and
ind
icat
ion
of
the
med
icat
ion
un
der
eval
uat
ion
isin
dic
ated
(1)
11
Av
aila
bil
ity
of
alte
rnat
ive
ther
apie
s
incu
rren
ttr
eatm
ent
sett
ing
Alt
ern
ativ
eTx
Cat
ego
rica
l—y
es/n
oA
nal
tern
ativ
ew
asco
nsi
der
edto
be
avai
lab
leif
com
par
ato
rsw
ere
clea
rly
defi
ned
inth
ere
vie
wb
yth
eH
TA
agen
cy.A
nal
tern
ativ
ew
asco
nsi
der
edN
OT
avai
lab
le
ifit
was
stat
edas
such
inth
eap
pra
isal
,o
rif
‘bes
tsu
pp
ort
ive
care
’o
r‘p
alli
ativ
e
care
’w
assp
ecifi
edas
the
com
par
ato
r(1
)
Ev
iden
ce—
eco
no
mic
var
iab
les
12
Co
nsi
der
atio
no
fco
stu
tili
tyan
aly
sis
ing
uid
ance
CU
AC
ateg
ori
cal—
CU
A
per
form
edo
rn
o
CU
A
Pre
sen
ceo
rab
sen
ceo
fa
cost
-uti
lity
anal
ysi
s(5
)
Public funding of pharmaceuticals in the Netherlands
123
Ta
ble
1co
nti
nu
ed
No
.V
aria
ble
des
crip
tor
Var
iab
len
ame
Un
ito
fm
easu
reD
efin
itio
n
13
Incr
emen
tal
cost
-uti
lity
rati
oo
f
tech
no
log
yv
ersu
sco
mp
arat
or
in
bas
eca
se
ICE
RN
um
eric
ICE
R(c
ost
per
QA
LY
)in
clu
ded
inth
ere
po
rtfo
rb
ase
case
asac
cep
ted
by
the
CF
H.
Th
isis
defi
ned
asth
eIC
ER
that
isre
late
dto
the
reco
mm
end
atio
n.
Ifm
ore
than
on
eIC
ER
isp
rese
nte
d(a
sth
ere
com
men
dat
ion
cov
ers
mo
reth
ano
ne
po
pu
lati
on
)th
enth
eIC
ER
per
tain
ing
toth
ela
rger
of
the
po
pu
lati
on
sw
as
rep
ort
ed.
Ifte
chn
olo
gy
isre
po
rted
asd
om
inan
to
rd
om
inat
ed,it
was
reco
rded
as
such
on
the
dat
aex
trac
tio
nsh
eet
(5)
14
–1
5M
ult
iple
CU
A/C
EA
mo
del
sre
po
rted
Mu
ltip
leC
EA
—
Mu
ltip
leIC
ER
s
Cat
ego
rica
l—y
es/n
o;
if
yes
—p
rov
ide
ran
ge
Var
iab
le1
4re
po
rts
wh
eth
erm
ore
than
on
eco
st-u
tili
tyo
rco
st-e
ffec
tiv
enes
s
mo
del
was
con
sid
ered
du
rin
gth
eap
pra
isal
(yes
/no
).If
yes
,v
aria
ble
15
mea
sure
sth
era
ng
eo
fb
ase-
case
ICE
Rs
pre
sen
ted
bet
wee
nth
ed
iffe
ren
tm
od
els
rep
ort
ed.
Th
ed
iffe
ren
ceb
etw
een
the
low
est
and
hig
hes
tIC
ER
was
calc
ula
ted
(5)
16
Un
cert
ain
tyar
ou
nd
the
bas
eca
se
ICE
Rre
po
rted
insu
bm
issi
on
(pro
bab
ilis
tic)
Pro
bab
ilis
ticI
CE
RN
um
eric
Th
isv
aria
ble
mea
sure
sth
ep
erce
nta
ge
pro
bab
ilit
yo
fac
cep
tan
ceat
the
thre
sho
ld
use
db
yth
eag
ency
.F
or
the
CF
Hth
ep
rob
abil
ity
of
the
med
icat
ion
bei
ng
cost
-
effe
ctiv
ew
asre
po
rted
ata
EU
R€5
0,0
00
thre
sho
ld(5
)
17
Un
cert
ain
tyar
ou
nd
bas
eca
seIC
ER
rep
ort
edin
sub
mis
sio
n(u
niv
aria
te)
Un
ivar
iate
ICE
RN
um
eric
Th
isv
aria
ble
mea
sure
dth
era
ng
eo
fIC
ER
s(m
in–
max
)re
sult
ing
fro
mu
niv
aria
te
sen
siti
vit
yo
nth
eb
ase
case
(5)
18
An
tici
pat
edb
ud
get
ary
imp
act
of
intr
od
uct
ion
of
new
tech
no
log
yin
hea
lth
care
syst
em
Bu
dg
etIm
pN
um
eric
Est
imat
edan
nu
alb
ud
get
ary
imp
act
of
intr
od
uci
ng
new
med
icat
ion
into
the
curr
ent
trea
tmen
tse
ttin
g,
ifth
ep
har
mac
euti
cal
wer
eto
be
intr
od
uce
dw
ith
ou
t
any
rest
rict
ion
.D
rug
cost
on
ly(p
ery
ear)
(6)
Dec
isio
n-m
akin
g
pro
cess
var
iab
les
19
So
ciet
alp
ersp
ecti
ve
ado
pte
din
CE
anal
ysi
s
So
ciet
alC
ateg
ori
cal—
yes
/no
Th
isv
aria
ble
cap
ture
sw
het
her
aso
ciet
alp
ersp
ecti
ve
was
ado
pte
din
the
app
rais
al
pro
cess
20
Incl
usi
on
of
pat
ien
tsu
bm
issi
on
Pat
ien
tSu
bC
ateg
ori
cal—
yes
/no
Ap
atie
nt
sub
mis
sio
nw
asco
nsi
der
edto
hav
eb
een
incl
ud
edas
par
to
fth
e
app
rais
alp
roce
ssif
asu
bm
issi
on
fro
ma
pat
ien
tg
rou
pw
asp
ost
edo
nth
e
web
pag
ep
erta
inin
gto
the
gu
idan
ce(7
)
21
No
.o
fd
ecis
ion
-mak
ers
acco
un
tab
leN
oD
ecM
aker
sN
um
eric
Cap
ture
sth
en
um
ber
of
com
mit
tee
mem
ber
sac
cou
nta
ble
for
the
gu
idan
ceis
sued
,
asre
po
rted
inth
eC
FH
pro
cess
gu
idel
ines
(8)
22
Co
st-e
ffec
tiv
enes
sev
alu
atio
n
com
po
nen
tin
pro
cess
CE
Ap
roce
ssC
ateg
ori
cal—
yes
/no
Cap
ture
sw
het
her
or
no
tco
st-e
ffec
tiv
enes
sis
aco
mp
on
ent
of
the
dec
isio
n-m
akin
g
pro
cess
.If
cost
-eff
ecti
ven
ess
anal
ysi
sis
afo
rmal
par
to
fth
eap
pra
isal
pro
cess
,
this
var
iab
lew
asm
ark
edas
‘yes
’(5
)
So
cio
-eco
no
mic
and
syst
emco
nte
xt
var
iab
les
23
Dat
eg
uid
ance
was
issu
edD
ate
Nu
mer
icY
ear
wh
enco
ver
age
dec
isio
nw
asis
sued
(11
)
24
Hea
lth
care
exp
end
itu
reo
n
ph
arm
aceu
tica
ls
Hea
lth
Ex
pN
um
eric
(€)
Hea
lth
care
bu
dg
etsp
ent
on
ph
arm
aceu
tica
lsp
erp
atie
nt
per
yea
r,d
uri
ng
the
sam
e
yea
rin
wh
ich
the
app
rais
alw
asp
ub
lish
ed(1
2,
13
)
25
Ele
ctio
ny
ear
atti
me
of
dec
isio
nE
lect
ion
Cat
ego
rica
l—y
es/n
oT
his
var
iab
leca
ptu
res
wh
eth
erth
ep
ayer
dec
isio
nw
asm
ade
wit
hin
anel
ecti
on
yea
r.A
nel
ecti
on
yea
rw
asd
efin
edas
ay
ear
inw
hic
hei
ther
nat
ion
al
go
ver
nm
ent
or
reg
ion
alel
ecti
on
sto
ok
pla
ce(1
3)
K. H. Cerri et al.
123
dummy variables were created to identify observations
with missing data to test in the regression models whether
the lack of data was significantly associated with differ-
ences in the outcome variable.
A series of sensitivity analyses were performed on the
base-case regression model to help evaluate the robust-
ness of the results. The sensitivity analyses included: (1)
examining the impact of a binary rather than a three-
category outcome variable; (2) restricting the base-case
analysis to complete observations, thus excluding obser-
vations with imputed values; and (3) estimating the model
assuming ordinal properties of the outcome variable. CVZ
decisions were classified into three categories, which may
lead to over-simplification of the CVZ decision-making
process and underestimate the heterogeneity that may
exist across decisions. To examine the impact of an
alternative classification system on the model, a sensi-
tivity analysis was performed using a binary outcome
variable, which confirmed the role of several factors
identified in the base-case analysis. Statistical analyses
were conducted using Intercooled (IC) STATA (Version
10.1 2009).
Results
A total of 277 drug reviews issued between January 2004
and June 2009 were retrieved from the CVZ website. Of
these, 244 full submissions, representing 256 coverage
decisions, were included for analysis. Thirty-three drug
reviews were excluded from the analysis for the following
reasons: (1) full guidance was not available (n = 13) or (2)
they focussed on a non-adult population (n = 20). The
most common coverage decision by the CVZ (n = 256)
was to recommend new technologies (51 %), followed by
restriction of funding (33 %), while 16 % of coverage
decisions advocated not funding the technology.
Univariate analysis
Of the 29 explanatory variables examined, descriptive
analysis suggested that a subset of 16 variables may play an
important role in determining CVZ decision-making
(Table 2). Five variables related to the clinical evidence
supporting the technology under evaluation. There was a
statistically significant difference in the average size of the
population included in RCTs and mean trial duration
among the three outcome groups (p \ 0.05). The com-
parator used within the clinical trial programme was
assessed: in particular, the percentage of comparisons made
to ‘active’ comparators as opposed to placebo was recor-
ded. Interventions recommended for use had a statistically
significantly higher number of trials with activeTa
ble
1co
nti
nu
ed
No
.V
aria
ble
des
crip
tor
Var
iab
len
ame
Un
ito
fm
easu
reD
efin
itio
n
26
EM
Aau
tho
risa
tio
nE
MA
auth
or
Cat
ego
rica
l—y
es/n
oT
his
var
iab
lep
rov
ides
info
rmat
ion
on
wh
eth
ero
rn
ot
the
tech
no
log
yh
adre
ceiv
ed
mar
ket
ing
auth
ori
sati
on
fro
mth
eE
uro
pea
nM
edic
ines
Ag
ency
for
the
ind
icat
ion
un
der
rev
iew
(11
)
27
Po
st-a
pp
rov
alst
ud
yre
qu
est
Fu
ture
CE
AC
ateg
ori
cal—
yes
/no
Th
isv
aria
ble
pro
vid
esin
form
atio
no
nw
het
her
reim
bu
rsem
ent
was
gra
nte
dw
ith
the
con
dit
ion
that
real
-lif
eo
bse
rvat
ion
ald
ata
on
the
tech
no
log
yw
ou
ldb
e
pro
vid
edw
ith
ina
spec
ified
tim
ep
erio
d(1
1)
28
Ex
pen
siv
ed
rug
Ex
pen
siv
eDru
gL
ist
Cat
ego
rica
l—y
es/n
oA
tech
no
log
yw
asco
nsi
der
edto
be
anex
pen
siv
ed
rug
ifit
was
rep
ort
edo
nth
e
‘‘D
uu
reg
enee
smid
del
lijs
t’’
pu
bli
shed
by
the
CF
H(1
3)
29
Co
-pay
men
tP
atie
ntC
op
ayC
ateg
ori
cal—
yes
/no
Th
isv
aria
ble
iden
tifi
esth
ose
tech
no
log
ies
wh
ere
pat
ien
tsar
ere
qu
este
dto
pay
a
per
cen
tag
eo
fth
ed
rug
cost
ino
rder
toac
cess
the
tech
no
log
y(1
4)
CF
H-r
app
ort
,se
ctio
n2
a;F
arm
aco
ther
apeu
tisc
hra
pp
ort
,se
ctio
n3
,4
a–f;
(2)
Min
iste
rie
van
Vo
lksg
ezo
nd
hei
d,
Wel
zijn
enS
po
rt[2
6,
27
];(3
)E
uro
pea
nM
edic
ines
Ag
ency
[28
]:(4
)B
riti
sh
Nat
ion
alF
orm
ula
ry[2
9];
(5),
Far
mac
o-e
con
om
isch
rap
po
rtan
dV
raag
stel
lin
gd
oel
mat
igh
eid
sto
ets;
(6)
Ko
sten
pro
gn
ose
Rap
po
rt;
(7)
CV
Z[1
];(8
)C
FH
(20
11
);(9
)C
FH
;(1
0)
So
ren
son
etal
.
[30];
(11
)L
ette
rfr
om
the
min
iste
rv
anV
olk
sgez
on
dh
eid
,W
elzi
jnen
Sp
ort
;(1
2)
Cen
traa
lB
ure
auv
oo
rd
eS
tati
stie
k(C
BS
)S
tatl
ine;
GIP
dat
aban
k[3
0];
(13
)T
od
osi
jev
icet
al.
[31];
(14
)
Med
icij
nk
ost
en[3
2];
(15
)B
EL
EID
SR
EG
EL
CI-
89
1[4
]
Public funding of pharmaceuticals in the Netherlands
123
Ta
ble
2C
VZ
des
crip
tiv
est
atis
tics
:st
atis
tica
lly
sig
nifi
can
tv
aria
ble
s(p
B0
.05
)
CV
Zto
tal
(n=
25
6)
Rec
om
men
ded
(n=
13
0)
Res
tric
ted
(n=
86
)N
ot
reco
mm
end
ed
(n=
40
)
pv
alu
eb
yst
atis
tica
lte
st
Mea
n(9
5%
CI)
Mea
n(9
5%
CI)
Mea
n(9
5%
CI)
Mea
n(9
5%
CI)
v2A
NO
VA
Kru
ska–
Wal
lis
Evi
den
ce—
clin
ica
la
nd
dis
ease
-rel
ate
d
vari
ab
les
No
RC
T2
.6(2
.3,
3)
2.2
(1.8
,2
.5)
2.8
(2.2
,3
.5)
3.8
(2,
5.6
)0
.09
12
RC
Tsi
ze8
30
(49
4,
1,1
65
)5
25
(36
6,
68
4)
1,3
64
(44
7,
2,2
81
)5
88
(23
2,
94
5)
0.3
04
2
RC
Td
ura
tio
n3
9(3
3,
46
)4
4(3
4,
54
)3
1(2
4,
38
)4
4(2
4,
65
)0
.25
63
RC
Tsu
per
ior
Su
per
ior
39
%4
5%
35
%2
5%
0.0
46
No
tsu
per
ior
29
%2
0%
35
%4
5%
0.0
03
Inco
nsi
sten
tre
sult
s1
7%
18
%1
7%
15
%0
.92
2
RC
Tco
mp
arat
ora
44
%(3
8,
51
)5
1%
(42
,6
0)
45
%(3
5,
55
)2
1%
(8,
34
)0
.08
36
Ob
sStu
die
s6
0%
(40
,7
0)
60
%(3
0,
80
)6
0%
(20
,9
0)
50
%(1
0,
80
)0
.79
48
Pri
ori
ty5
5%
61
%4
3%
60
%0
.02
8
Orp
han
Des
ig9
%1
1%
9%
0%
0.1
Th
erap
euti
car
ea
BN
F1
card
iov
ascu
lar
syst
em9
%4
%1
5%
10
%0
.01
4
BN
F2
cen
tral
ner
vo
us
syst
em1
6%
15
%1
5%
23
%0
.52
6
BN
F3
ear,
no
sean
do
rop
har
yn
x1
%2
%0
%0
%0
.23
BN
F4
end
ocr
ine
syst
em6
%5
%8
%5
%0
.54
1
BN
F5
eye
2%
3%
0%
0%
0.1
4
BN
F6
gas
tro
-in
test
inal
syst
em5
%5
%5
%3
%0
.75
2
BN
F7
infe
ctio
ns
9%
6%
12
%1
0%
0.3
51
BN
F8
mu
scu
losk
elet
al,
join
td
isea
ses
10
%7
%1
9%
3%
0.0
05
BN
F9
nu
trit
ion
and
blo
od
7%
8%
8%
3%
0.4
71
BN
F1
0o
bst
etri
cs,
gy
nae
colo
gy
,u
rin
ary
-
trac
td
iso
rder
s
3%
1%
7%
3%
0.0
36
BN
F1
1re
spir
ato
rysy
stem
4%
2%
0%
18
%0
.00
01
BN
F1
2sk
in5
%5
%2
%8
%0
.38
2
Eli
gib
leP
op
94
,54
3(3
1,3
94
,1
57
,69
3)
€61
,81
6(-
€16
,08
1,
€13
9,7
13
)
41
,08
7(8
,93
2,
73
,24
2)
26
8,1
45
(11
,85
2,
52
4,4
40
)
0.0
02
7
Alt
ern
ativ
eTx
79
%7
6%
85
%7
8%
0.2
76
Evi
den
ce—
eco
no
mic
vari
ab
les
CU
A1
1%
12
%7
%1
8%
0.1
96
ICE
R€3
6,6
21
(€1
0,2
17
,
€63
,02
4)
€51
,85
4(€
2,0
05
,
€10
1,7
04
)
€17
,96
3(€
11
,33
9,
€24
,58
8)
€18
,50
2(€
9,1
03
,
€27
,90
1)
0.5
33
5
K. H. Cerri et al.
123
Ta
ble
2co
nti
nu
ed
CV
Zto
tal
(n=
25
6)
Rec
om
men
ded
(n=
13
0)
Res
tric
ted
(n=
86
)N
ot
reco
mm
end
ed
(n=
40
)
pv
alu
eb
yst
atis
tica
lte
st
Mea
n(9
5%
CI)
Mea
n(9
5%
CI)
Mea
n(9
5%
CI)
Mea
n(9
5%
CI)
v2A
NO
VA
Kru
ska–
Wal
lis
Mu
ltip
leC
EA
1%
2%
0%
0%
0.2
3
Mu
ltip
leIC
ER
s
Lo
wra
ng
e€1
00
,59
2(-
€23
4,9
54
,
€43
6,1
38
)
€10
0,5
92
(-€2
34
,95
4,
€43
6,1
38
)
––
–
Hig
hra
ng
e€2
61
,85
1(€
23
8,3
38
,
€28
5,3
63
)
€26
1,8
51
(€2
38
,33
8,
€28
5,3
63
)
––
–
Pro
bab
ilis
ticI
CE
Ra
0.6
6(4
2,
90
)0
.79
(43
,1
15
)0
.52
(-6
6,
17
0)
0.5
5(-
16
8,
27
7)
0.5
77
4
Un
ivar
iate
ICE
R
Lo
wv
alu
e€1
7,9
54
(€5
,12
8,€3
0,7
79
)€3
1,4
62
(-€1
9,4
06
,
€82
,33
1)
€13
,40
2(-
€16
,80
8,
€43
,61
1)
€10
,46
5(€
2,3
73
,
€18
,55
7)
0.5
53
8
Hig
hv
alu
e€1
09
,73
7(-
€7,5
14
,
€22
6,9
87
)
€19
2,7
09
(-€1
06
,72
5,
€49
2,1
42
)
€48
,13
2(-
€43
,19
2,
€13
9,4
55
)
€43
,73
9(-
€53
9,
€88
,01
7)
0.5
53
8
Bu
dg
etIm
p3
6.4
(6.9
,6
5.9
)6
.7(4
.3,
9)
66
(-1
7.6
,7
6)
77
(-2
6.5
,1
75
)0
.02
24
Dec
isio
n-m
aki
ng
pro
cess
vari
ab
les
So
ciet
al1
%1
%0
%0
%0
.62
3
Pat
ien
tSu
b4
%2
%2
%1
3%
0.0
09
No
Dec
Mak
ers
20
(20
,2
0)
20
(20
,2
0)
20
(20
,2
0)
20
(20
,2
0)
–1
CE
Ap
roce
ss6
7%
65
%6
4%
78
%0
.28
7
So
cio
-eco
no
mic
con
text
vari
ab
les
Dat
ea2
00
6(2
00
6,
20
06
)2
00
6(2
00
6,
20
06
)2
00
6(2
00
6,
20
06
)2
00
7(2
00
6,
20
07
)0
.13
49
Hea
lth
Ex
pa
€29
4(€
29
1,€2
96
)€2
93
(€2
89
,€2
96
)€2
92
(€2
87
,€2
96
)€3
02
(€2
96
,€3
09
)0
.13
49
Ele
ctio
n2
0%
23
%2
0%
8%
0.0
94
EM
Aau
tho
r9
2%
92
%9
4%
85
%0
.20
7
Fu
ture
CE
A9
%1
6%
1%
3%
0.0
00
1
Ex
pen
siv
eDru
gL
ist
12
%2
3%
0%
0%
0.0
00
1
Pat
ien
tCo
pay
4%
2%
9%
3%
0.0
19
aT
hes
ev
aria
ble
sd
isp
lay
an
orm
ald
istr
ibu
tio
n.
Th
ere
mai
nin
gv
aria
ble
sd
isp
lay
an
on
-no
rmal
dis
trib
uti
on
Public funding of pharmaceuticals in the Netherlands
123
comparators and demonstrated statistically significant
superiority (p \ 0.05).
The effect of economic evidence, particularly the impact
on pharmaceutical budget, was observed. The estimated
budget impact associated with the introduction of the
technology ranged from a mean of €7 million for the rec-
ommended interventions to €66 and €75 million for the
restricted and not recommended interventions, respectively
(p = 0.0001). Availability of cost-effectiveness evidence
did not appear to differ significantly between outcome
groups. Only 11 % of CVZ decisions were backed by the
use of a cost-utility analysis (CUA). For the interventions
supported by a CUA, the average incremental cost-effec-
tiveness ratio (ICER) across all groups was €36,620, and
differences between outcome groups were not statistically
significant. Information was also captured on whether
alternative cost-effectiveness models (non-CUA) were
considered in the decision-making process; 15 % of inter-
ventions were supported by non-CUA models. The use of
non-CUA analyses occurred in 15 % of recommended
interventions, 9 % of restricted interventions and 28 % of
interventions not recommended for use. The differences
observed between decision outcome groups were statisti-
cally significant (p = 0.016).
The effect of several process-related factors was found
to be significant. Overall, in 4 % of submissions patient
evidence was considered as part of the process, with 2 %
of recommended or restricted technologies supported by
patient evidence, and 13 % of technologies that were
not recommended (p = 0.009). Of the technologies
appraised, overall 55 % were indicated in a disease area
that was prioritised by the healthcare system. Across the
outcome categories, the proportion of technologies indi-
cated in prioritised disease areas ranged from 43 % to
61 %, and the difference was significant (0.028). Of
recommended technologies, 16 % were requested to
provide additional cost-effectiveness evidence as part of
conditional reimbursement scheme, while 1 % of
restricted technologies and 3 % of non-recommended
technologies were requested to provide such evidence
(p = 0.0001).
In the Netherlands, patients can be asked to provide a
co-payment for their prescription. This is determined by
whether or not a certain technology falls within a basic
packet of insurance—this ranged from 2 to 9 % in the
sample (p = 0.019).
With regards to socioeconomic context variables, the
year of appraisal, election in the year of appraisal and
healthcare expenditure on pharmaceuticals appeared to
vary significantly between outcome groups (p = 0.05). The
distribution of CVZ decisions over time is shown in Fig. 2.
The remaining socioeconomic variables did not differ
significantly between outcome groups.
Multivariate analysis
The multinomial model showed significant associations
(p = 0.10) between CVZ outcome and nine variables: (1)
whether statistical superiority of the primary endpoint in
the clinical trial was demonstrated by the appraised tech-
nology, (2) use of an active comparator in the clinical tri-
als, (3) missing information on the duration of the RCT, (4)
the estimated impact on the pharmaceutical budget due to
the introduction of the technology, (5–7) therapeutic area
in which the technology is indicated (cancer, cardiovas-
cular disease and obstetrics/gynaecology/urinary-tract dis-
orders), (8) size of the eligible population for treatment and
(9) inclusion of patient submission in the appraisal process
(Table 3). Both log odds and marginal effects are displayed.
Variables 1–3; 9 had an effect on the odds of non-recom-
mendation relative to recommendation, while variables 4–8
had an effect on the odds of restriction relative to recommen-
dation. No variables had an effect on the odds of both restric-
tion and non-recommendation relative to recommendation.
Specifically, the use of an active comparator in clinical
trials decreased the log odds of a restriction or non-recom-
mendation relative to recommendation, although this effect
was only statistically significant for the latter (p = 0.239 and
p \ 0.001 respectively). Demonstration of superiority in the
clinical trial also decreased the odds of a restriction or non-
recommendation relative to recommendation (p = 0.022,
p \ 0.001, respectively). The lack of information on the
duration of the RCT increased the odds of a non-recom-
mendation (p = 0.001) and restriction (not statistically sig-
nificant) relative to recommendation. The budget impact of
the technology appeared to impact significantly on CVZ
decision-making: a unit increase in the budget impact
increased the probability of a restriction (p = 0.051) relative
to recommendation, but was not statistically significant on
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2004 2005 2006 2007 2008 2009
Recommended Restricted Not Recommended
Fig. 2 CVZ coverage decisions between 2004 and 2009 (June), by
year (n = 256)
K. H. Cerri et al.
123
the log odds of a non-recommendation relative to recom-
mendation. Technologies indicated for the treatment of can-
cer decreased the log odds of a restriction or non-
recommendation relative to recommendation, and this impact
was statistically significant on the log odds of a restriction
(p \ 0.001) and not significant on the log odds of a non-
recommendation relative to recommendation (p = 0.177).
Technologies indicated for the treatment of cardiovascular
disease, and obstetrics/gynaecology/urinary-tract disorders
increased the probability of a restriction relative to recom-
mendation, and this was statistically significant. None of
these dummy variables, with the exception of missing data
related to clinical trial duration, appeared to have a significant
effect on the odds of CVZ decisions. The marginal effects
observed followed a similar pattern to the odds ratios
observed in the base case multinomial logit model. When
CVZ decisions were regressed on these nine variables, the
resulting pseudo R-squared was 0.17, suggesting that the four
variables explained approximately 17 % of the variability in
CVZ decisions.
Sensitivity analyses
A series of sensitivity analyses was performed on the base-
case regression model to help evaluate the robustness of the
results. The first sensitivity analysis assessed whether the
impact of the explanatory variables on decision-making
varied if a binary outcome variable was utilised instead of
the base-case three-category outcome variable. The logistic
regression results using a binary outcome variable (covered
vs. not covered) showed a similar effect for the clinical
variables (use of an active comparator in the clinical trial
and the demonstration of superiority remained significant
variables) and the effect of inclusion of patient evidence.
The budget impact, duration of the RCT and whether the
technology was indicated for cancer, cardiovascular dis-
ease or obstetrics/gynaecological disorders were no longer
significant as predictors. Variables that were not included
in the base-case model were found to have a significant role
in this sensitivity analysis—the year of appraisal and
technologies indicated for the treatment of infectious
diseases.
In the second sensitivity analysis, the regression analysis
was run for the subset of complete observations (n = 98/
256). This sensitivity analysis was implemented with the
knowledge that removing incomplete observations from
the analysis could bias the analysis. The pseudo R-squared
for this model was 0.12, suggesting that this set of variables
explains approximately 12 % of the variability observed in
CVZ decision-making, as opposed to 17 % in the base-case
Table 3 Multivariate analysis of CVZ coverage decisions 2004–2009: base case model results
Log odds p value 95 % confidence interval Marginal effects
Restricted versus recommended
Use of active comparator in RCT -0.49 0.239 -1.31 0.33 -0.0068
Demonstrated clinical superiority in RCT -0.82 0.022 -1.53 -0.12 -0.098
Budgetary impact 0.0069 0.051 -3.1E-05 0.014 0.0014
Cancer therapy -1.59 \0.001 -2.47 -0.72 -0.28
Therapies for cardiovascular diseases 1.4 0.017 0.25 2.55 0.29
Therapies for obstetrics, gynaecology, urinary-tract disorders 2.4 0.032 0.2 4.59 0.36
Size of eligible population -1.4E-06 0.091 -2.9E-06 2E-07 -3.04E-07
Inclusion of patient submission -0.15 0.879 -2.08 1.78 -0.15
Lack of data on duration of RCT 0.24 0.56 -0.58 1.06 -0.057
Constant 0.28 0.459 -0.46 1.03 –
Not recommended versus recommended
Use of active comparator in RCT -2.54 \0.001 -3.84 -1.24 -0.25
Demonstrated clinical superiority in RCT -1.85 \0.001 -2.82 -0.89 -0.17
Budgetary impact 0.0047 0.217 -0.0028 0.012 0.00021
Cancer therapy -0.7 0.177 -1.71 0.31 -0.02
Therapies for cardiovascular diseases 0.82 0.338 -0.86 2.51 0.0077
Therapies for obstetrics, gynaecology, urinary-tract disorders 2.2 0.141 -0.73 5.12 0.082
Size of eligible population -1.6E-08 0.982 -1.4E-06 1.4E-06 5.38E-08
Inclusion of patient submission 1.79 0.032 0.16 3.41 0.33
Lack of data on duration of RCT 1.78 0.001 0.74 2.81 0.25
Constant 0.066 0.878 -0.78 0.91 –
Recommended technologies are the reference case. Multinomial logistic regression, pseudo R2: 0.17
Public funding of pharmaceuticals in the Netherlands
123
model. In this sensitivity analysis, the explanatory vari-
ables observed to have a predictive effect maintained this
effect, with the exception of the following variables which
no longer showed a predictive value: patient submission,
cancer therapies, obstetrics/gynaecology diseases, duration
of RCT and size of target population.
In the third sensitivity analysis, the outcome variable
was assumed to be ordinal and ordinal logistic regression
was used. The results of this analysis show very similar
results to the base-case analyses.
Discussion
To our knowledge, this study represents the first multi-
variate analysis of CVZ decision-making, exploring a wide
range of potentially influential factors within a large sam-
ple of decisions over a 5-year period (2004–2009). The
most common coverage decision by the CVZ was to rec-
ommend new technologies (51 %). Our results suggest that
a combination of clinical, economic and process factors
explains decisions made by CVZ to recommend, restrict or
not recommend pharmaceutical technologies for use. The
internal validity of the results obtained in this analysis was
examined in two ways: first by comparing the results with
published analyses of CVZ decision-making where avail-
able and second by sharing the base-case model results for
review with representatives of the CVZ.
Impact of clinical evidence and disease characteristics
on CVZ decision-making
We hypothesised that the choice of comparator in the
clinical trials would have an impact on CVZ decision-
making, given the focus of decision-making on establishing
whether incremental therapeutic benefit exists. Our analy-
sis shows that the use of an active comparator in clinical
trials influenced decision-making by decreasing the prob-
ability of non-recommendation. Indeed, the majority
(51 %) of recommended technologies were supported by
RCTs with active comparator arms rather than placebo-
control arms, as opposed to 45 % of restricted technologies
and 21 % of technologies not recommended for funding.
The disease area for which the technology was indicated
played an important role in CVZ decisions. In the multi-
variate analyses, cancer therapies, which could be inter-
preted as representing severe disease, significantly
decreased the probability of restriction relative to recom-
mendation. These results support findings from a previous
discrete choice experiment amongst Dutch healthcare
professionals that suggested that severity of disease was a
significant criterion driving coverage decisions [5]. In
contrast, technologies for the treatment of cardiovascular
disease and for obstetrics/gynaecology/urinary-tract disor-
ders increased the probability of a restriction relative to
recommendation. These results suggest that the nature of
the disease influences decision-making.
Economic evidence and its impact on CVZ
decision-making
The analysis did not support the hypothesis that the intro-
duction of a cost-effectiveness component in the CVZ
process has impacted on its decision-making. Only 11 % of
appraisals reported an ICER, and among those technologies
for which ICERs were reported there was no statistically
significant difference between outcome variables. Expert
review by CVZ representatives suggests that these results
reflect the fact that CUA was first introduced in the CVZ
process in 2005 and that CUA is only utilised as a criterion
for inclusion of technologies on List 1B. The results of this
analysis contrast with results of a discrete choice experi-
ment [5] that suggested that cost-utility analysis is an
important criterion for Dutch healthcare decision-makers.
Plausible explanations for this difference could be the
choice of methodology used and that Koopmanschap et al.
[5] focussed on hypothetical reimbursement decisions as
compared with the actual decisions analysed in this article.
Our analysis confirmed the hypothesis that pharmaceu-
tical budget impact estimates significantly influence CVZ
outcomes. In the sample considered for analysis, the mean
estimated budget impact for technologies recommended by
the CVZ was €7 million, compared with €66 million for
restricted technologies and €76 million for non-recom-
mended technologies. The multivariate analysis of deci-
sion-making showed that an increase in the budget impact
increased the log odds of restriction and non-recommen-
dation, relative to recommendation. The effect of budget-
ary impact considerations did not have a statistically
significant association with the odds of non-recommenda-
tion relative to recommendation, suggesting there are other
factors that better explain non-recommendations than
budgetary impact considerations (including nature of clin-
ical evidence, size of target population). These results are
aligned with those of Koopmanschap et al. [5] which
suggested that, among other factors, budget impact was a
significant consideration driving coverage decisions. These
results may also reflect the objective of the Dutch reim-
bursement system to reduce growth in out-patient drug
expenditure while maintaining high-quality healthcare and
patient outcomes [23].
Pattern of CVZ decision-making over time
In terms of the socioeconomic context of CVZ decision-
making, the multivariate analysis did not support the
K. H. Cerri et al.
123
hypothesis that non-recommendations and restrictions are
increasing over time relative to recommendations. While
descriptive analyses suggested the date of appraisal varied
by decision (Fig. 2), the multivariate analyses did not find
that time had a significant effect on decisions (Table 3)
when adjusting for confounding factors.
Limitations
The database constructed for these analyses, incorporating
information on appraisals conducted by CVZ from 2004 to
2009, was dependent on publicly available information.
Data extraction was performed by a single researcher,
which may be associated with reviewer bias, although the
method and analysis were reviewed by senior researchers.
It is possible that subtle concepts or rationales discussed by
the appraisal committee orally were not captured in the
documentation of the appraisal. In addition, dependence on
publicly available information meant that in the situation
where such information was incomplete, it was not possible
to ascertain if this was because the information was never
considered in the appraisal or if it was considered but not
recorded in the documentation.
To analyse decision-making, CVZ decisions were clas-
sified into three categories; this may lead to over-simpli-
fication of the decision-making process and underestimate
the heterogeneity that may exist across decisions. To
examine the impact of an alternative classification system
on the model, a sensitivity analysis was performed using a
binary outcome variable, which confirmed the role of
several factors identified in the base-case analysis. How-
ever, variables such as budget impact were no longer found
to have a significant effect, while variables that were not
included in the base-case model were now found to have a
significant role. This suggests that the use of binary out-
come categories can yield an alternative perspective on
CVZ decision-making, but at the expense of reducing
visibility in relation to the impact of explanatory variables
on specific types of coverage decisions.
The R-squared value obtained from the multivariate
analysis reflects the set of variables that were identified as
having a significant association with the outcome of CVZ
decision-making. The variables included in this analysis
represent factors for which indicators could be defined and
coded, and for which evidence could be collected retro-
spectively. While not representing the complete set of
factors potentially taken into account by CVZ, they do
represent the largest set of variables hitherto collected on
CVZ decisions and represent a range of factors hypothe-
sised to explain CVZ decision-making. As the results show,
the variables included in the analysis explain a proportion
of the variance of CVZ decision-making, highlighting that
unexplained variance linked to factors not included in this
analysis remains or that some influences on decisions are
simply random.
With the CVZ operating within the EU, it is important to
assess its decision-making process within the context of the
EU Transparency Directive 89/105/EEC. The European
Commission has launched an update of this directive to
further emphasise the notion of transparency. The proposed
amendment from the European Parliament [24] now
includes direct reference to the concept of transparency and
proposes that determination of price and access requires:
‘‘transparent, objective and verifiable criteria’’ [24, p. 8].
This analysis of CVZ decision-making highlights the
opportunity for increasing transparency by providing pub-
lic access to manufacturer submissions, reporting with
more detail the evidence taken into account in decision-
making and emphasising the distinction between the
assessment phase and the appraisal phase of the decision-
making process. However, it is important not to confuse the
inherent complexity of some decision-making processes
with lack of transparency.
Our analysis of CVZ decision-making took into account
a range of variables hypothesised to impact on decision-
making. To our knowledge, this study represents the first
published quantitative analysis of real-world CVZ deci-
sion-making. A number of quantitative analyses of real-
world coverage decision-making have been conducted
[11], with studies differing in the nature of the variables
and methods adopted. The framework adopted in the
present analysis, notwithstanding its limitations, provides
an example of a comprehensive analytical framework for
understanding coverage decision-making. A similar
framework was adopted in analysis of decision-making by
the National Institute for Health and Clinical Excellence
(NICE) [33], which showed significant associations
(p B 0.10) between NICE outcome and four factors: (1)
demonstration of statistical superiority of the primary
endpoint in clinical trials by the appraised technology, (2)
the incremental cost-effectiveness ratio (ICER), (3) the
number of pharmaceuticals appraised within the same
appraisal and (4) the appraisal year. While recognising the
diversity in scope, objective and context in which different
HTA bodies operate, convergence towards an analytical
framework for the analysis of coverage decisions in future
research may be of value.
Conclusion
The objective of this analysis was to examine the factors
that influence decisions made by CVZ to recommend,
restrict or not recommend pharmaceutical technologies for
use in the Netherlands. The analysis provided a rich source
of data from which to examine the role of each factor on
Public funding of pharmaceuticals in the Netherlands
123
CVZ decisions and more importantly the contribution of
each factor while adjusting for the effect of confounding
variables. The results suggest that coverage decision-
making is a complex process involving numerous clinical,
disease and affordability considerations. This fits well with
the argument presented by Stolk and Poley [25] that an
understanding of CVZ coverage decisions requires a
holistic and comprehensive assessment of multiple factors.
From a clinical and disease perspective, CVZ decision-
making is influenced by the choice of comparator in clin-
ical trials and the nature of the disease for which the
technology is indicated. Affordability plays an important
role in CVZ decision-making, but the anticipated effect of
cost-effectiveness evidence on decisions was not observed.
CVZ decision-making outcomes have remained relatively
stable over time. This analysis therefore broadens the scope
of evidence available on factors driving European HTA
coverage decisions. Our results confirm the value of a
comprehensive and multivariate approach to understanding
CVZ decision-making.
Acknowledgments The authors would like to thank Michael
Drummond and Alistair McGuire for their valuable comments on the
methods used and interpretation of analyses that were performed.
References
1. College voor zorgverzekeringen: CVZ-criteria voor beoordeling
therapeutische waarde. http://www.fk.cvz.nl/ (2010)
2. College voor zorgverzekeringen: procedure beoordeling extra-
murale geneesmiddelen. http://www.fk.cvz.nl/ (2010)
3. Stolk E.A., Rutten F.F.: The ‘‘health benefit basket’’ in the
Netherlands. Eur. J. Health Econ. 6(Suppl), 53–57 (2005)
4. College Tarieven Gezondsheidszorg ZorgAutoriteit: Beleidsregel
ci-891, bijlage 2 ctg-zaio beleidsregel dure geneesmiddelen.
http://www.nza.nl/binaries/13755/14184/14621 (2006)
5. Koopmanschap, M.A., Stolk, E.A., Koolman, X.: Dear policy
maker: have you made up your mind? A discrete choice experi-
ment among policy makers and other health professionals. Int.
J. Technol. Assess. Health Care 26, 198–204 (2010)
6. de Wolf, P., Brouwer, W.B., Rutten, F.F.: Regulating the Dutch
pharmaceutical market: improving efficiency or controlling
costs? Int. J. Health Plan. Manag. 20, 351–374 (2005)
7. Niezen, M.G., de Bont, A., Busschbach, J.J., Cohen, J.P., Stolk, E.A.:
Finding legitimacy for the role of budget impact in drug reimburse-
ment decisions. Int. J. Technol. Assess. Health Care 25, 49–55 (2009)
8. Stolk, E.A., de Bont, A., van Halteren, A.R., Bijlmer, R.J., Poley,
M.J.: Role of health technology assessment in shaping the ben-
efits package in the Netherlands. Expert Rev. Pharmacoecon.
Outcomes Res. 9, 85–94 (2009)
9. van Oostenbruggen, M.F., Jansen, R.B., Mur, K., Kooijman, H.:
Penny and pound wise: pharmacoeconomics from a governmental
perspective. Pharmacoeconomics 23, 219–226 (2005)
10. Al, M.J., Feenstra, T., Brouwer, W.B.: Decision makers’ views
on health care objectives and budget constraints: results from a
pilot study. Health Policy 70, 33–48 (2004)
11. Fischer, K.E.: A systematic review of coverage decision-making
on health technologies-evidence from the real world. Health
Policy 107, 218–230 (2012)
12. Lee, E.K., Kim, B.Y., Lim, J.Y., Park, M.H.: Different policy
outcomes of the new drugs and currently listed drugs under the
positive list system in South Korea. Value Health 15, S100–S103
(2012)
13. Menon, D., Stafinski, T., Stuart, G.: Access to drugs for cancer:
does where you live matter? Can. J. Public Health 96, 454–458
(2005)
14. OECD (ed) Health Technologies and Decision Making, The
OECD Health Project. OECD Publishing, France (2005)
15. Rocchi, A., Miller, E., Hopkins, R.B., Goeree, R.: Common drug
review recommendations: an evidence base for expectations?
Pharmacoeconomics 30, 229–246 (2012)
16. Vuorenkoski, L., Toiviainen, H., Hemminki, E.: Decision-making
in priority setting for medicines—a review of empirical studies.
Health Policy 86, 1–9 (2008)
17. Bryan, S., Williams, I., McIver, S.: Seeing the nice side of cost-
effectiveness analysis: a qualitative investigation of the use of cea
in nice technology appraisals. Health Econ. 16, 179–193 (2007)
18. Buxton, M.: How much are health-care systems prepared to pay
to produce a qaly? Eur. J. Health Econ. 6, 285–287 (2005)
19. Dakin, H., Devlin, N., Odeyemi, I.: ‘‘Yes’’, ‘‘no’’ or ‘‘yes, but’’?
Multinomial modelling of nice decision-making. Health Policy
77, 352–367 (2006)
20. Owens, D.: Interpretation of cost-effectiveness analyses. JGIM
13, 716–717 (1998)
21. Packer, C., Simpson, S., Stevens, A.: International diffusion of
new health technologies: a ten-country analysis of six health
technologies. Int. J. Technol. Assess. Health Care 22, 419–428
(2006)
22. Ross, J.: The use of economic evaluation in health care: Aus-
tralian decision makers’ perceptions. Health Policy 31, 103–110
(1995)
23. Pronk, M.H., Bonsel, G.J.: Out-patient drug policy by clinical
assessment rather than financial constraints? The gate-keeping
function of the out-patient drug reimbursement system in the
Netherlands. Eur. J. Health Econ. 5, 274–277 (2004)
24. Committee on the Environment Public Health and Food Safety:
Draft report on the proposal for a directive of the European
parliament and of the council relating to the transparency of
measures regulating the prices of medicinal products for human
use and their inclusion in the scope of public health insurance
systems (com(2012)0084–c7-0056/2012–2012/0035(cod)). In:
Parliament, E. (ed.) (2012). http://www.europarl.europa.eu/sides/
getDoc.do?pubRef=-%2f%2fEP%2f%2fNONSGML%2bCOMPARL
%2bPE-491.292%2b01%2bDOC%2bPDF%2bV0%2f%2fEN.
Accessed 18 Nov 2012
25. Stolk, E.A., Poley, M.J.: Criteria for determining a basic health
services package. Recent developments in the Netherlands. Eur.
J. Health Econ. 6, 2–7 (2005)
26. Ministerie van Volksgezondheid W.e.S.: Langer gezond leven—
ook een kwestie van gezond gedrag 2004–2007, Publicatie
Postbus 51, Rijsoverheid (2003)
27. Ministerie van Volksgezondheid W.e.S.: Kaderbrief 2007–
2011—visie op gezondheid en preventie. Publicatie Postbus 51,
Rijsoverheid (2007)
28. European Medicines Agency: European public assessment
reports. http://www.ema.europa.eu/docs/en_GB/document_library/
EPAR_-_Product_Information/human/000481/WC500050870.pdf
(2011). Accessed 5 Jan 2011
29. Joint Formulary Committee: British national formulary, 60th edn.
British Medical Association and Royal Pharmaceutical Society,
London (2010)
30. Sorenson, C.D.M., Kanavos, P.: Ensuring value for money in
health care: the role of hta in the european union. Cornwall,
World Health Organisation, on behalf of the European Obser-
vatory on Health Systems and Policies (2008)
K. H. Cerri et al.
123
31. Todosijevic, B.A.K., van der Kaap, H.: Dutch Parliamentary
Election Studies 1971–2006: Cumulative Data-Set and Docu-
mentation: DANS Data Guide, vol. 7. DANS—Data Archiving
and Networked Services, The Hague (2010)
32. Medicijnkosten: Medicijnkosten—een themasite van het college
voor zorgverzekeringen (2009), http://www.medicijnkosten.nl/.
Accessed 6 June 2009
33. Cerri, K.H., Knapp, M., Fernandez, J.L.: Decision-making by
NICE: examining the influences of evidence, process and context.
Health. Econ. Policy. Law. (2013). doi:10.1017/S174413311300
0030
Public funding of pharmaceuticals in the Netherlands
123
