Avai lable onl ine at www.sc iencedirect .com

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ]

1098-3015$36.00 – s

Published by Elsevie

http://dx.doi.org/10.

The views exprebehalf of or reflectin

E-mail: michael.* Address correspo

journal homepage: www.elsevier .com/ locate / jva l

Improving the Contribution of Regulatory Assessment Reportsto Health Technology Assessments—A Collaboration betweenthe European Medicines Agency and the European network forHealth Technology AssessmentMichael Berntgen, PhD, MDRA1,*, Anne Gourvil, PhD2, Mira Pavlovic, MD2, Wim Goettsch, PhD3,Hans-Georg Eichler, MD, MSc1, Finn Børlum Kristensen, MD, PhD4

1European Medicines Agency, London, UK; 2La Haute Autorité de Santé, Saint-Denis La Plaine Cedex, France; 3National Health CareInstitute, Diemen, The Netherlands; 4EUnetHTA Secretariat, C/o Danish Health and Medicines Authority, Copenhagen, Denmark

A B S T R A C T

In response to a recommendation from the Pharmaceutical Forum, theEuropean Medicines Agency and the European network for HealthTechnology Assessment initiated a collaboration with the aim toimprove the contribution regulatory assessment reports can make tothe assessment of relative effectiveness of medicinal products by healthtechnology assessment bodies. This collaboration on improving Euro-pean Public Assessment Reports (EPARs) started in February 2010 andwas performed over 2 years. As a result, the templates for preparingEPARs were revised to better address the needs of heath technologyorganizations. The better understanding of information needs was a keyoutcome of the collaboration. To ascertain whether these templatechanges led to the inclusion of relevant information, a review of a small

ee front matter Copyright & 2014, International S

r Inc.

1016/j.jval.2014.04.006

ssed in this article are the personal views of the ag the position of the European Medicines Agency

berntgen@ema.europa.eu.ndence to: Michael Berntgen, European Medicines

set of EPARs for recently approved medicinal products was carried out inparallel by both the European network for Health Technology Assess-ment and the European Medicines Agency. This report provides anaccount of this project on improving EPARs, which is part of the ongoingdialogue between regulators and health technology assessment bodieson a European level to support policymaker decisions in the future.Keywords: European Medicines Agency (EMA), European network forHealth Technology Assessment (EUnetHTA), European PublicAssessment Reports (EPARs), relative effectiveness assessment.

Copyright & 2014, International Society for Pharmacoeconomics andOutcomes Research (ISPOR). Published by Elsevier Inc.

Introduction

Scientific evaluation of the clinical data produced during drugdevelopment can be intended to estimate the benefit/risk ratio ofthe product (recently in some jurisdictions also referred to as benefit/harm/uncertainty), for the purpose of marketing authorization, or toestimate the effectiveness of the new product as compared withexisting therapies, as part of the health technology assessment (HTA)process to support decision making on price and reimbursement.

The European Medicines Agency (EMA) is responsible for thescientific evaluation of applications for European Union (EU)marketing authorizations for medicinal products in the so-called centralized procedure. Under this procedure, a singlemarketing-authorization application is submitted to the EMAand once authorization is granted by the European Commissionon the basis of a pan-European scientific assessment of quality,efficacy, and safety by EMA committees, a centralized marketingauthorization is valid in all EU member states.

The subsequent evaluation and decision-making process lead-ing to decisions on the pricing and reimbursement of medicinal

products lie with the national bodies. HTA is a multidisciplinaryprocess that summarizes information about the medical, social,economic, and ethical issues related to the use of a healthtechnology in a systematic, transparent, unbiased, and robustmanner. Its aim is to inform the formulation of safe, effectivehealth policies that are patient focused and seek to achieve bestvalue [1]. Criteria for HTA vary between countries, but generallyHTA bodies in Europe use relative effectiveness assessment (REA),a European equivalent to comparative effectiveness research inthe United States, as part of the HTA process [2].

The European network for Health Technology Assessment(EUnetHTA) is a network of organizations appointed by EUmember states that produce or contribute to HTA to facilitateefficient use of resources available for HTA, to create a sustain-able system of HTA knowledge sharing, and to promote goodpractice in HTA methods and processes [1,3]. Originally started asa project in 2006, EUnetHTA was established to create an effectiveand sustainable network for HTA across Europe [4].

In October 2008, the Pharmaceutical Forum, a high-levelministerial platform for discussion between member states, EU

ociety for Pharmacoeconomics and Outcomes Research (ISPOR).

uthor(s) and may not be understood or quoted as being made onor one of its committees or working parties.Agency, 7 Westferry Circus, Canary Wharf, London E14 4HB, UK.

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ]2

institutions, industry, health care professionals, patients, andinsurance funds, agreed on conclusions and recommendations tofind relevant solutions to public health considerations regardingpharmaceuticals, while ensuring the competitiveness of theindustry and the sustainability of the national health caresystems [5]. One of these recommendations provided a politicalmandate to initiate collaboration between the EMA andEUnetHTA with the aim to improve the availability and best useof data relevant to HTA. Specifically, the objective of this jointproject of regulators and HTA bodies was to “consider howinformation in the European Public Assessment Report /…/ canfurther contribute to relative effectiveness assessment” [5].

The regulatory assessment reports, which are the basis forcentral marketing authorizations, are publicly available as so-called European Public Assessment Reports (EPARs). EPARs reflectthe scientific conclusions reached by the EMA’s Committee forMedicinal Products for Human Use (CHMP) at the end of theregulatory review process after deletion of commercially confiden-tial information. These are published on the EMA’s Web site forevery medicine authorized through the centralized procedure inthe EU. Because these contain the CHMP’s judgment on themethodological quality of the studies and the relevance andsignificance of results, in the perspective of the estimate of thebalance between favorable and unfavorable effects for the medic-inal product, they may also be used as source documents for REAsfrom the HTA perspective. In particular, such assessment reportsare used in the context of rapid REA of pharmaceuticals [6].

The collaboration between the EMA and EUnetHTA on EPARsstarted in February 2010 and was performed over 2 years. It has beensubject to a wider consultation within EUnetHTA as part of theWorkPackage 5 on REA of Pharmaceuticals of EUnetHTA Joint Action 1.This report provides an account of various activities as part of theproject to address the recommendation on improving the EPAR.

Methods

In the first phase, EUnetHTA provided input on the usefulness ofpublished EPARs and Summary of Product Characteristics (SmPC)in the context of REA of medicinal products based on the initialwork performed in 2009 by some partners (which are alsoEUnetHTA partners) of the Medicine Evaluation Committee(MEDEV), a standing working group of the European Social HealthInsurance Forum. This input was reviewed by the EMA and anaction plan was established, which was mutually agreed betweenthe EMA and EUnetHTA. Subsequently, the templates and guidancedocuments for the assessment of initial marketing authorizationapplications were amended by the EMA to implement the agreedimprovements through either specific sections or supportive guid-ance. The revised templates and guidance documents were pre-sented to the CHMP for adoption and subsequently published onthe EMA’s Web site (www.ema.europa.eu) for use by assessors inthe centralized procedure (4Home 4Regulatory 4Human medi-cines 4Pre-authorisation 4Templates for assessors).

The second phase was the critical review of EPARs to establishcompliance with revised templates. For this purpose, EPARs of thefirst 10 products, for which the assessment report has been preparedafter the revised templates came into operation, were analyzed forcompliance with specific aspects amended in the template. Thefollowing EPARs were reviewed: Esbriet (pirfenidone), Gilenya (fin-golimod), Halaven (eribulin), Jevtana (cabazitaxel), Pravafenix (feno-fibrate/pravastatin), Pumarix (pandemic influenza vaccine [H5N1]),Teysuno (tegafur/gimeracil/oteracil), Trobalt (retigabine), Xeplion(paliperidone), and Xiapex (collagenase clostridium histolyticum).The review took place in second and third quarters of 2011 on thebasis of the publicly available EPARs. A questionnaire with 36questions was developed by the EMA targeting the identified areas

for improvement. Two types of answers were attributed dependingon the nature of the question: either the option “yes/no/notapplicable” (13 questions) if the underlying question was simplywhether an item is included or the grading “excellent/good/could beimproved/no/not applicable” (23 questions) if a more differentiatedreview was considered more meaningful. Table 1 indicates for eachreview item the type of question. Numerical values were assignedagainst the answers (“1/0” and “3/2/1/0” for binary and gradedanswers, respectively) to allow for statistical analysis of complianceof the EPARs for each individual question. The review of the 10EPARs was performed in parallel by the EMA and by EUnetHTA,always involving two reviewers per organization and using the samequestionnaire and numerical scales. In case of divergent viewsbetween the two reviewers, consensus was formed. One additionalquestion on the benefit-risk assessment was added by HTA organ-izations at the time of EUnetHTA review. As regards EUnetHTAreview, each EPAR was evaluated by two HTA organizations fromEUnetHTA. Ten HTA organizations (see Acknowledgment) partici-pated in this exercise coordinated by the French National Authorityfor Health (HAS). The results were calculated as mean values interms of the overall compliance rate across all 10 EPARs andreviewers for each of the two organizations (i.e., if the answer fora binary question was “yes” [“1”] in 6 out of 10 EPARs, a 60% overallcompliance rate was attributed). For descriptive purpose, the com-pliance rates are clustered in three distinct categories: more than80% compliance (mean), 50% to 80% compliance (mean), and lessthan 50% compliance (mean).

Results

As a result of the project, the templates for preparing EPARs wererevised to address the comments received. Furthermore, a reviewof a small set of EPARs was carried out in parallel by bothEUnetHTA and the EMA to ascertain whether these templatechanges led to the inclusion of relevant information. The follow-ing sections present these results separately.

Revision of Templates for Preparing EPARs

Figure 1 displays the development of revisions to templates forregulatory assessments with the aim to improve the contributionsuch reports can make to the assessment of relative effectivenessof medicinal products. The initial analysis of a compilation ofcomments from EUnetHTA/MEDEV identified 34 topic areas forfurther discussion. These topics areas concerned the format andstructure, the scientific content, the evaluation criteria, and otheraspects of the assessment report. A detailed discussion of eachitem revealed opportunities for improvement of template andguidance as well as areas in which adherence to existing templateand guidance might need to be improved. Certain items concer-ned the methodology for conducting an assessment rather thandata presentation, such as the acceptability of certain surrogateparameters or the conduct of a wider literature search; theseitems were excluded from the exercise. Most of the items, whichwere addressed through revisions to the templates, concerned thescientific content of the assessment report, particularly, theclinical aspects. In most instances, the agreed items are relatedto a more detailed data presentation or more explicit reasoning ofconsiderations for the regulatory decision making. In addition,specific sections of the SmPC have been identified in which abetter substantiation through information and discussion in theassessment report would be beneficial.

The subsequent revisions concerned various parts of theexisting templates and guidance documents. In Figure 1, a tabularoverview of the revisions to the templates is provided, identifyingupdates sections and describing the changes introduced. As an

Table 1 – Outcome of the review of compliance of EPARs with the template/guidance (N ¼ 10; categorizationbased on mean values).*

Overall compliance(%) accounted toEUnetHTA review

Review item Overall compliance (%)accounted to the EMAreview

o50 50–80 480 o50 50–80 480

Format and scientific contentX Table of contents† X

X List of abbreviations† XX Clear referencing of data from publications† X

X Structural formula for chemical substances† XX Structural characteristics for biologicals† XX Description of standard treatments in the EU† XX Discussion of compliance with legal requirements† X

X Discussion of compliance with scientific guidelines (EMA/CHMP)† XX Discussion of compliance with scientific advice† X

X Discussion of outcome of any GCP inspection and its impact on data reliability† XX Discussion of key elements of the study design: Patient population‡ X

X Discussion of key elements of the study design: Comparators‡ XX Discussion of key elements of the study design: Duration of the study‡ X

X Discussion of key elements of the study design: End points and/or compositeend point‡

X

X Display of participant flow (graphically or tabular)† XX Summary of the main efficacy data in the template table† X

X Explanation for reasoning for additional analyses, if requested‡ XX Explanation if a subgroup data were considered of particular relevance for the

overall assessment of efficacy‡X

X Justification for waiver of study or replacement by literature data‡ XX Highlighting of shortcomings of the efficacy data including impact on the

assessment‡X

X Reflection of additional input from external experts (SAG, ad-hoc expert group,PDCO), if requested‡

X

X Rationale for deciding that the risk-benefit balance is positive is adequatelydiscussed‡,§

Support for summary of product characteristics (SmPC)||

X SmPC section 4.1: Reflection of approved therapeutic indication includingselection of patient population and age range, as applicable‡

X

X SmPC section 4.2: Substantiation of dose recommendations‡ XX SmPC section 4.3: Substantiation of contraindications‡ X

X SmPC section 4.4: Substantiation of warning/precautions for use‡ XX SmPC section 4.5: Substantiation of interaction statements‡ XX SmPC section 4.6: Substantiation of use during pregnancy and lactation‡ X

X SmPC section 4.7: Substantiation of effects on ability to drive and usemachines‡

X

X SmPC section 4.8: Substantiation of adverse drug reaction profile‡ XX SmPC section 4.8: Definition of ADRs consistent between SmPC and

Assessment Report‡X

X SmPC section 4.9: Substantiation of information on overdose‡ XX SmPC section 5.1: Information on approved therapeutic indication(s) in line

with information in the Assessment Report‡X

X SmPC section 5.1: Available data in the pediatric population‡ XX SmPC section 5.2: Substantiation of pharmacokinetic properties‡ X

ADR, adverse drug reaction; CHMP, Committee for Medicinal Products for Human Use; EMA, European Medicines Agency; GCP, good clinicalpractice; HTA, health technology assessment; EPAR, European Public Assessment Report; EU, European Union; EUnetHTA, European networkfor Health Technology Assessment; PDCO, Paediatric Committee; SAG, Scientific Advisory Group.* Because a separate reference listing is required only for EPARs with referencing to numerous publications and none of the EPARs subject tothe review did fulfill this criterion, the review item “separate reference listing” (planned to be reviewed using a binary question) was notapplicable and is therefore not reported in terms of compliance.

† Item reviewed using a binary question; i.e., the aspect is included “yes/no.”‡ Item reviewed using a graded question; i.e., the aspect is included “excellent/good/could be improved/no.”§ Item added by HTA organizations at the time of EUnetHTA review.|| Only aspects of sections 4.1, 4.2, and 5.1 (Information on approved therapeutic indication) were topics specifically addressed throughrevisions of template/guidance.

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ] 3

Fig. 1 – Development of revisions to templates for regulatory assessments with the aim to improve the contribution such reportscan make to the assessment of relative effectiveness of medicinal products. CHMP, Committee for Medicinal Products forHuman Use; EMA, European Medicines Agency; EPAR, European Public Assessment Report; EU, European Union; EUnetHTA,European network for Health Technology Assessment; PL, package leaflet; QC, quality control; SAG, Scientific Advisory Group.

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ]4

example, for the discussion of key elements of the study design, anew subsection was created in the “Discussion on clinical efficacy”of the assessment report. In this new subsection titled “Design andconduct of clinical studies,” a critical discussion should addressamong others whether the design of the studies is deemedadequate, the patient population was adequately selected, thecomparator considered appropriate, an active comparator wasrelevant in view of EU-approved treatment options, and the choiceof end points and duration of study was adequate considering

regulatory guidance. To address the request for better explanationof the reasoning for additional analysis, relevance of a particularsubgroup, and the appropriateness to waive studies or replacethese against literature data, specific guidance was provided in anew subsection “Efficacy data and additional analysis.”

A template table for a structured summary of the mainefficacy data was developed jointly by the EMA and EUnetHTAand has been newly introduced into the assessment reports.Figure 2 shows this template together with the respective

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ] 5

guidance. The objective was to provide a tabulated summary ofthe efficacy data that were considered most relevant for theregulatory decision making. For each of the main studies, aseparate table should be presented detailing the results accordingto the prespecified analysis as well as any other analysis that isdeemed relevant. Because it is foreseen to allow for a plain datapresentation, this table should be read in conjunction with thediscussion and conclusion on clinical efficacy, as well as thebenefit-risk assessment in later parts of the assessment report.

Parallel Review of EPARs

The outcome of the review is presented in Table 1. For eachreview item, the overall compliance with the template across all10 EPARs is displayed in three categories. This is done separately

Fig. 2 – Template for the summary of main efficacy data for theANOVA, analysis of variance; CHMP, Committee for Medicinal PRegulating Authorities Clinical Trials; ISRCT, International StandmITT, modified intention to treat PP, per protocol.

for the review by EUnetHTA and by the EMA to allow for anoverview of the ranking by each of the two organizations as wellas a comparison of the results between the two reviews.

With regard to the format of the EPAR, the two reviewsshowed that a table of contents was always included whereasindividual publications were not always clearly referenced. Withregard to the scientific content of the EPAR, a high com-pliance rate was assigned by both the EMA and EUnetHTA forthe presentation of patient flow as graphic or table as well as thenewly introduced summary table of main efficacy results andanalysis. However, low compliance rates were identifiedregarding the presentation of discussion on the key elements ofthe study design including patient populations, com-parators, duration of the study, and end points, as also shownin Figure 3.

regulatory assessment report with guidance (in italics).roducts for Human Use; EudraCT, European Union Drugard Randomized Controlled Trial; ITT, intention to treat;

Fig. 2 – continued

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ]6

The explanation for reasoning of additional data analysesrequested by the CHMP as well as the emphasis on shortcomingof the efficacy data were highlighted by EUnetHTA as critical areasof the clinical efficacy discussion; the compliance according toEunetHTA review was found below 50% on these two items. Inaddition, explanations when a subgroup analysis is considered ofparticular relevance, justifications for waiver or replacement of astudy by literature data, and reflection of additional input fromexternal expert were also considered of particular importance.These review items were moderately compliant from EUnetHTAperspective, contrary to EMA, which rated these items above 80%compliance, except for the justification for waiver or replacementof a study by literature data (o50%). Regarding the additional itemreviewed by EUnetHTA only on the benefit-to-risk assessment, acompliance rate indicating satisfactory adherence to templateguidance was documented.

The support of the SmPC through the assessment reports wasconsidered by EUnetHTA as appropriate for most of the reviewitems (between the range of 50% and 80%); however, none of themwas above 80%. Four items were reviewed below 50% of compliance,notably for the substantiation of contraindications. The EMA attrib-uted a compliance rate of 50% to 80% and more than 80% for 10 and3 of these items, respectively. For 7 of the 13 review items includingthe sections on therapeutic indication, dose recommendations(particularly deviations from standard dose), and warnings andprecautions for use and interactions, the two organizations con-cluded on 50% to 80% compliance. Specifically for the review itemsstemming from the initial EUnetHTA/MEDEV comments, namely,the reflection of approved therapeutic indication in section 4.1, thesubstantiation of dose recommendations in section 4.2, and theinformation on approved therapeutic indication(s) in section 5.1,both reviews assigned a compliance category of 50% to 80%.

0

10

20

30

40

50

60

70

Patien

t pop

ulatio

n

Compa

rato

rs

Study

dur

ation

End p

oints

Key elements of study design

Co

mp

lian

ce r

ate

(%)

EMA review

EUnetHTA review

Fig. 3 – Compliance of EPARs with guidance in templatesregarding the presentation of discussion on the keyelements of study designs (N ¼ 10). EMA, EuropeanMedicines Agency; EPAR, European Public AssessmentReport; EUnetHTA, European network for Health TechnologyAssessment.

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ] 7

Discussion

This joint project between the EMA and EUnetHTA aimed atimproving the contribution EPARs can make to the assessment ofrelative effectiveness by European HTA bodies. EPARs are pub-lished regulatory assessment reports supporting the licensing ofmedicinal products that are centrally authorized in Europe. Giventhe scope of the centralized procedure, almost all new therapeu-tics (new molecular entities and biologics) receive their market-ing authorization through this licensing route [7].

Based on observations on the usefulness of EPARs and SmPCsfrom an HTA perspective, the EMA and EUnetHTA agreed onspecific items for improvement of the presentation of data andinformation in the EPAR. The emphasis was on a better under-standing of the regulatory judgment, such as critical discussionsof study design as well as specific subgroup analysis. Further-more, a structured and harmonized presentation of main efficacydata was requested in view of clarity and transparency of thedata that is deemed most important from a regulatory perspec-tive. Through changes in the templates and guidance documentsfor assessment reports, these agreed topics were specificallytargeted. The revisions—if adhered to when writing the actualreports—were aimed to ensure that the relevant data andinformation are provided with the published documents.

The review of actual EPARs explored how well these revisionsare translated into practice. Using the same methodology, boththe EMA and EUnetHTA came to the view that the compliancediffers across various aspects in the assessment report. Giventhat the EPAR format and scientific content showed overall highrates of compliance with the revised templates, it can beassumed that the quality of the documents has been improvedeven if no such assessment of previous EPARs with the samemethodology was performed to quantify the difference. Anachievement was the introduction of the tabular overview ofmain efficacy data, which was widely adhered to. Also, morestructural aspects in the EPAR were implemented. There remains,however, space for further improvement in the critical discussionof the key elements of the clinical study design, that is, on patientpopulation (including subpopulation and special populations),comparators, duration of the study, end points, and/or composite

end-point use. Some of these aspects may be present in theclinical efficacy discussion but are not visible enough or notdiscussed enough. The substantiation of the SmPC was variable,with some areas requiring special attention mainly for thesections concerning the dose recommendations (particularlydeviations from standard dose), contraindications, warningsand precautions, and interactions.

The most divergent view between the reviews of the twoorganizations seemed to be regarding the discussion of theshortcomings of efficacy data as well as the identification andexplanation of additional analysis requested during the regulatoryreview. These items are considered as critical areas of the clinicalefficacy discussion from EUnetHTA perspective, as well as explan-ations when a subgroup analysis is considered of particularrelevance, justifications for waiver or replacement of a study byliterature data, and reflection of additional input from an externalexpert, which would benefit being more clearly identified in thecontent of the clinical efficacy discussion and individually dis-cussed. Looking into the responses, however, there might havebeen different interpretations of the questions between the tworeviews. Importantly, aspects such as the request for additionalanalysis, Good Clinical Practice (GCP) inspections, and externalexpert consultations are discussed in the EPAR only if theyoccurred and if they are recorded. EUnetHTA suggestion to theEMA was to increase the granularity in the structure of the clinicalefficacy discussion report template to address the main above-mentioned aspects and make the information more visible.

It was also noted that there are different expectations withregard to the referencing of literature because the regulatoryassessment is usually based on study reports rather than pub-lished literature. In general, regulatory-specific terminology mighthave been differently understood in some cases during the reviewby EUnetHTA; for example, the term scientific guidelines in theregulatory context usually refers to the guidance on the clinicaldevelopment program to support an application for marketingauthorization rather than to treatment recommendations. Anoverall perception during the review by HTA bodies was that theinformation was provided in the EPAR but on some occasions waseither difficult to locate or not sufficiently discussed.

A limitation of the compliance review was that it wasperformed with EPARs that were written immediately after therevised templates came into operation. This meant that theprimary assessment reports from assessors, which lead up tothe final CHMP assessment report supporting the outcome of theregulatory review and hence the EPAR, have not been preparedfollowing these new standards. Furthermore, the review waslimited to 10 EPARs and was performed by the two decisionmakers without involving a “third” party. Nevertheless, thereview highlighted important aspects that should be subject tothe quality review process of assessment reports in the future. Ifsubsequently areas are identified that can be improved further inthe templates and the guidance documents, these can be imple-mented through the regular revision processes.

Conclusions and Perspectives

With this collaboration the EMA and EUnetHTA responded to apolitical recommendation to consider how the regulatory assess-ment report about favorable and unfavorable effects of a medi-cine can best be used in the assessment of the relativeeffectiveness of new medicines for HTA purposes in the EUmember states. There is growing international experience regard-ing aspects to be considered to optimize the interface betweenregulatory approval and reimbursement decision [8]. The presentwork was the first joint project of such collaboration betweenregulators and HTA bodies on a European level. The remit of both

V A L U E I N H E A L T H ] ( 2 0 1 4 ) ] ] ] – ] ] ]8

parties was respected throughout the project and the focus wason the presentation of data and information without enteringinto a discussion about potentially divergent views on eviden-tiary standards. With the improved presentation of data andinformation in the EPAR, it is envisaged that this regulatorydocument through harmonized efficacy data presentation will bemore useful in the context of rapid REAs by HTA bodies whenthey inform policymakers and health care decision makers in thefuture.

The initiation of the dialogue between the EMA and EUnetHTAon the EPAR has demonstrated the opportunity to engage indiscussions about better exchange of data and information. Infact, the parallel review of EPARs has been useful for each of theorganizations to not only critically review the end-product“assessment report” using a predefined methodology but alsomutually identify areas for future improvement.

The project has shown the pursuit of one opportunity forregulators to proactively contribute information to the assess-ment of relative (comparative) effectiveness by HTA bodies,independent of jurisdiction. Beyond this project on EPARs, theEMA and EUnetHTA are continuing to explore other areas ofcollaboration or exchange of information. These include ways toobtain scientific advice or early dialogues involving regulatorsand HTAs/payers, exchange on scientific and methodologicalguidelines, topics in the area of postlicensing data generation,as well as the particularities of orphan medicinal products [9].The EMA’s Road Map to 2015 identified the improvement ofEPARs in view of their use for HTAs as well as the EMA’sengagement with HTA organizations—regarding drug develop-ment and evidential standards as well as postlicensing researchand data collection—as major initiatives [10]. Similarly,EUnetHTA has defined collaboration with EMA as a priority [11].A joint EMA-EUnetHTA three-year work plan 2013-2015 hasrecently been agreed [12]. Overall, it is expected that this dialoguebetween regulators and HTA bodies on a European level willcontinue to facilitate the bridging between regulatory approvaland access to market and will provide important contributions tovarious health policy topics.

Acknowledgments

We thank the following organizations for having performed thecompliance review on behalf of EUnetHTA: Andalusian HTAAgency (AETSA), Italian Medicines Agency (AIFA), Catalan Agency

for Health Information, Assessment and Quality (CAHIAQ), Cen-ter for Medical Technology Policy (CMPT) in Baltimore, DutchHealth Care Insurance Board (CVZ), National Institute for Healthand Care Excellence (NICE), Norwegian Knowledge Center for theHealth Services (NOKC), French National Authority for Health(HAS), Association of Austrian Social Insurance Institutions(HVB), and University Hospital A. Gemelli, Italy.

Source of financial support: The EUnetHTA contribution tothis article arises from the EUnetHTA Joint Action 1, which hasreceived funding from the European Union, in the framework ofthe Health Programme.

R E F E R E N C E S

[1] Kristensen FB, Mäkelä M, Allgurin Neikter S, et al. European network forHealth Technology Assessment, EUnetHTA: planning, development,and implementation of a sustainable European network for HealthTechnology Assessment. Int J Technol Assess Health Care 2009;25(Suppl. 2):107–16.

[2] Kleijnen S, Goettsch W, d’Andon A, et al. Relative effectivenessassessment (REA) of pharmaceuticals—background review, July 2011.Available from: www.eunethta.eu. [Accessed March 6, 2014].

[3] EUnetHTA. Organisation. Available from: http://www.eunethta.eu/about-us/organisation. [Accessed March 6, 2014].

[4] Kristensen FB. Development of European HTA: from Vision toEUnetHTA. Michael Q 2012;9:147–56.

[5] HLPF conclusions and recommendations. Available from: http://ec.europa.eu/enterprise/sectors/healthcare/files/docs/pharmaforum_final_conclusions_en.pdf. [Accessed October 30, 2013].

[6] EUnetHTA. EUnetHTA rapid REA core model. Available from: www.eunethta.eu. [Accessed October 30, 2013].

[7] European Medicines Agency. Marketing authorisations for new activesubstances reviewed in the centralised procedure and thedecentralised/mutual recognition procedure, respectively; data on file.

[8] Tsoi B, Masucci L, Campbell K. Harmonization of reimbursement andregulatory approval processes: a systematic review of internationalexperiences. Expert Rev Pharmacoecon Outcomes Res 2013;13:497–511.

[9] Minutes from the EMA-EUnetHTA meeting on 14 May 2013, London.Available from: www.ema.europa.eu and www.eunethta.eu. [AccessedOctober 30, 2013].

[10] European Medicines Agency. Road map to 2015—The EuropeanMedicines Agency’s contribution to science, medicines and health.Available from: www.ema.europa.eu. [Accessed October 30, 2013].

[11] EUnetHTA. EUnetHTA Joint Action 2 three-year work plan. Availablefrom: http://www.eunethta.eu/outputs/eunethta-joint-action-2-3-year-work-plan-2013. [Accessed October 30,2013].

[12] EMA-EUnetHTA three-year work plan 2013-2015. Available from: www.ema.europa.eu and www.eunethta.eu. [Accessed March 6, 2014].