Embed Size (px)
Citation preview
Elsevier Editorial System(tm) for Vaccine
Manuscript Draft
Manuscript Number:
Title: The ADVANCE system for evidence generation on vaccine coverage,
benefits and risks based on secondary use of electronic health care data
in Europe
Article Type: SI: ADVANCE
Keywords: distributed network, common protocol, common analysis, vaccine,
benefit-risk, real world evidence
Corresponding Author: Professor Miriam C.J.M. Sturkenboom, PhD
Corresponding Author's Institution: Utrecht University Medical Center
First Author: Miriam C.J.M. Sturkenboom, PhD
Order of Authors: Miriam C.J.M. Sturkenboom, PhD; Lieke van de Aa; Kaatje
Bollaerts; Hanne-Dorthe Emborg; Rosa Gini; Harshana Liyanage; Simon de
Lusignan; Lampros Stergioulas; Alena Khromava; Charlotte Switzer; Germano
Ferreira; Lina Titievsky; Daniel Weibel
Abstract: Introduction: The Accelerated Development of VAccine beNefit-
risk Collaboration in Europe (ADVANCE) project aimed to develop and test
a system to rapidly provide best-available evidence from existing
European healthcare databases on post-licensure vaccination benefits and
risks to support informed decision making, in Europe.
Methods: Metadata on available data sources in Europe were collected
using the ADVANCE International Research Readiness (AIRR) survey. The
ADVANCE system that was developed and implemented aimed to address study
questions on coverage, benefits, safety endpoints, and benefit/risk
assessment. The workflow process that included: feasibility assessment of
data sources; study outline; study team selection; protocol development;
data extraction and harmonisation; statistical analysis plans; data
transformation & transfer; data pooling and reporting, was tested.
Aggregated data were shared on a remote research environment (RRE) which
could be accessed for collaborative work.
Results: Twenty-four databases responded to the AIRR survey: 16 had
linkable individual level population data; 12 of these captured routine
immunisation data, 6 through linkage with vaccine register data and the
others through GPs input. A common data model (CDM), with three input
files (population, vaccines and events) was defined. Data access
providers (DAPs) extracted and transformed study-specific data into these
common input files. Data analytical tools were programmed by different
statisticians against the CDM in R, Java and/or SAS and sent to the DAPs,
first to assess if the data were fit-for-purpose and subsequently for the
different ADVANCE proof-of-concept studies. The output of the analytical
tools was aggregated data that were sent to the RRE for pooling. Ten DAPs
participated in fit-for-purpose assessment and shared aggregated data for
pooling and reporting, seven participated in the first ADVANCE proof-of-
concept studies.
Conclusions: The ADVANCE system to generate evidence on background rates,
coverage, vaccine benefits and risks, from distributed healthcare
databases was implemented and tested.
Dr Gregory A Poland
Editor-in-Chief, Vaccine
Soest, 18 January 2019
Dear Dr Poland
We are pleased to submit our paper “The ADVANCE system for evidence generation on
vaccines coverage, benefits and risks based on secondary use of electronic health care data
in Europe” to the journal of Vaccine for the ADVANCE supplement. This paper describes the
system of how we generated evidence from distributed health care data. It is the second of the
series of 10 papers that will be included in the supplement.
On behalf of all co-authors
Prof. dr. Miriam CJM Sturkenboom
Cover Letter
I, Dr. Miriam Sturkenboom declare that all authors have seen and approved the final version
of the manuscript being submitted. We warrant that the article is our original work that has
not been previously published and is not under consideration for publication elsewhere
Dr Miriam Sturkenboom
*Author Agreement
Name Institute email
Wan Tin Huang Taiwan CDC [email protected]
Richard Platt Harvard Medical School Department of Population Medicine, Harvard Pilgrim Health Care Institute
Olaf Klungel Utrecht Institute of Pharmaceutical Sciences [email protected]
Daniel Salmon The Vaccine Safety Institute, Johns Hopkins [email protected]
*Suggested Reviewers
Declaration of interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:
Lieke van der Aa, Kaatje Bollaerts, Hanne-Dorthe Emborg, Harshana Liyanage and Lampros Stergioulas declared no conflict of interests. Miriam Sturkenboom declared that she has received grants from Novartis, CDC and Bill & Melinda Gates Foundation for work unrelated to the work presented here. Rosa Gini declared that she has grants from Innovative Medicines Initiative (IMI) for the EMIF project, and grants from PHARMO Institute for work unrelated to the work presented here. Simon de Lusignan declared he is: Director of RCGP RSC and has university-based research (enhanced surveillance of influenza vaccines) funded by GSK, he is also a member of Seqirus and Sanofi Pasteur Advisory Boards for influenza. Alena Khromava declared that she employed by Sanofi Pasteur and holds company shares/stock options. Germano Ferreira declared that he was employed by Sanofi Pasteur, GSK Vaccines and P95 during the project. Lina Titievsky declared that she is employed by Pfizer and holds company stocks/shares. Daniel Weibel declared that he has received reports personal fees from GSK for work unrelated to the work presented here.
*Declaration of Interest Statement
Highlights
ADVANCE aimed to develop a system to provide evidence on vaccination benefits and risks
in Europe
The ADVANCE system was designed using a distributed network of health care databases in
Europe
Workflow was set-up to verify suitability of databases to provide reliable evidence
The system was used in proof-of-concept studies to estimate vaccine coverage, benefits and
risks
A tested workflow with methods, tools and templates is available for future implementation
Highlights (for review)
Abstract 1
Introduction: The Accelerated Development of VAccine beNefit-risk Collaboration in 2
Europe (ADVANCE) project aimed to develop and test a system to rapidly provide best-3
available evidence from existing European healthcare databases on post-licensure vaccination 4
benefits and risks to support informed decision making, in Europe. 5
Methods: Metadata on available data sources in Europe were collected using the ADVANCE 6
International Research Readiness (AIRR) survey. The ADVANCE system that was developed 7
and implemented aimed to address study questions on coverage, benefits, safety endpoints, 8
and benefit/risk assessment. The workflow process that included: feasibility assessment of 9
data sources; study outline; study team selection; protocol development; data extraction and 10
harmonisation; statistical analysis plans; data transformation & transfer; data pooling and 11
reporting, was tested. Aggregated data were shared on a remote research environment (RRE) 12
which could be accessed for collaborative work. 13
Results: Twenty-four databases responded to the AIRR survey: 16 had linkable individual 14
level population data; 12 of these captured routine immunisation data, 6 through linkage with 15
vaccine register data and the others through GPs input. A common data model (CDM), with 16
three input files (population, vaccines and events) was defined. Data access providers (DAPs) 17
extracted and transformed study-specific data into these common input files. Data analytical 18
tools were programmed by different statisticians against the CDM in R, Java and/or SAS and 19
sent to the DAPs, first to assess if the data were fit-for-purpose and subsequently for the 20
different ADVANCE proof-of-concept studies. The output of the analytical tools was 21
aggregated data that were sent to the RRE for pooling. Ten DAPs participated in fit-for-22
purpose assessment and shared aggregated data for pooling and reporting, seven participated 23
in the first ADVANCE proof-of-concept studies. 24
*Abstract
Conclusions: The ADVANCE system to generate evidence on background rates, coverage, 25
vaccine benefits and risks, from distributed healthcare databases was implemented and tested. 26
27
The ADVANCE system for evidence generation on vaccine coverage, benefits and risks 1
based on secondary use of electronic health care data in Europe 2
Miriam Sturkenbooma,b,c
, Lieke van der Aad, Kaatje Bollaerts
a, Hanne-Dorthe Emborg
e, 3
Rosa Ginif, Harshana Liyanage
g, Simon de Lusignan
g.h, Lampros K Stergioulas
g, Alena 4
Khromavai, Charlotte Switzer
i1, Germano Ferreira
a2, Lina Titievsky
j, Daniel Weibel
b,k 5
aP-95 Epidemiology and Pharmacovigilance, Koning Leopold III laan 1 3001 Heverlee 6
Belgium ([email protected]; [email protected]; 7
bVACCINE.GRID, Spitalstrasse 33, Basel, Switzerland ([email protected]; 9
cJulius Global Health, University Medical Center Utrecht, Heidelberglaan 100, The 11
Netherlands ([email protected]) 12
dSciensano, Rue Juliette Wytsmanstraat 14, 1050 Brussels, Belgium 13
([email protected]) 14
eStatens Serum Institut, Artillerivej 5, DK-23002300, Denmark ([email protected]) 15
fAgenzia regionale di sanità della Toscana, Osservatorio di epidemiologia, via Pietro Dazzi 1, 16
50144 Florence, Italy ([email protected]) 17
gUniversity of Surrey, Guildford, Surrey GU2 7XH, UK ([email protected]; 18
[email protected]; [email protected]) 19
Royal College of General Practitioners Research and Surveillance Centre, 30 Euston Square, 20
London NW1 2FB, UK ([email protected]) 21
iSanofi Pasteur, 1755 Steeles Ave W, North York, ON M2R 3T4, Canada 22
([email protected]; [email protected]) 23
1 Current address: McMaster University, Hamilton, Canada 2 Current address: Ferreira Pharmacoepidemiology Consultancy Rua Sao Joao Areias, 9, 3430-735 Parada,
Portugal
*Title Page
j219 East 42nd St, NY, NY 10017, USA ([email protected]) 24
kErasmus University Medical Center, PO box 2014, 3000 CA Rotterdam, The Netherlands 25
([email protected]) 26
Corresponding author: Miriam Sturkenboom, University Medical Center Utrecht, 27
Heidelberglaan 100, The Netherlands ([email protected]) 28
29
1
The ADVANCE system for evidence generation on vaccine coverage, benefits and risks 1
based on secondary use of electronic health care data in Europe 2
Miriam Sturkenbooma,b,c
, Lieke van der Aad, Kaatje Bollaerts
a, Hanne-Dorthe Emborg
e, 3
Rosa Ginif, Harshana Liyanage
g, Simon de Lusignan
g.h, Lampros K Stergioulas
g, Alena 4
Khromavai, Charlotte Switzer
i1, Germano Ferreira
a2, Lina Titievsky
j, Daniel Weibel
b,k 5
aP-95 Epidemiology and Pharmacovigilance, Koning Leopold III laan 1 3001 Heverlee 6
Belgium ([email protected]; [email protected]; 7
bVACCINE.GRID, Spitalstrasse 33, Basel, Switzerland ([email protected]; 9
cJulius Global Health, University Medical Center Utrecht, Heidelberglaan 100, The 11
Netherlands ([email protected]) 12
dSciensano, Rue Juliette Wytsmanstraat 14, 1050 Brussels, Belgium 13
([email protected]) 14
eStatens Serum Institut, Artillerivej 5, DK-23002300, Denmark ([email protected]) 15
fAgenzia regionale di sanità della Toscana, Osservatorio di epidemiologia, via Pietro Dazzi 1, 16
50144 Florence, Italy ([email protected]) 17
gUniversity of Surrey, Guildford, Surrey GU2 7XH, UK ([email protected]; 18
[email protected]; [email protected]) 19
Royal College of General Practitioners Research and Surveillance Centre, 30 Euston Square, 20
London NW1 2FB, UK ([email protected]) 21
iSanofi Pasteur, 1755 Steeles Ave W, North York, ON M2R 3T4, Canada 22
([email protected]; [email protected]) 23
1 Current address: McMaster University, Hamilton, Canada 2 Current address: Ferreira Pharmacoepidemiology Consultancy Rua Sao Joao Areias, 9, 3430-735 Parada,
Portugal
*ManuscriptClick here to view linked References
2
j219 East 42nd St, NY, NY 10017, USA ([email protected]) 24
kErasmus University Medical Center, PO box 2014, 3000 CA Rotterdam, The Netherlands 25
([email protected]) 26
Corresponding author: Miriam Sturkenboom, University Medical Center Utrecht, 27
Heidelberglaan 100, The Netherlands ([email protected]) 28
29
30
31
32
33
34
3
Abstract 35
Introduction: The Accelerated Development of VAccine beNefit-risk Collaboration in 36
Europe (ADVANCE) project aimed to develop and test a system to rapidly provide best-37
available evidence from existing European healthcare databases on post-licensure vaccination 38
benefits and risks to support informed decision making, in Europe. 39
Methods: Metadata on available data sources in Europe were collected using the ADVANCE 40
International Research Readiness (AIRR) survey. The ADVANCE system that was developed 41
and implemented aimed to address study questions on coverage, benefits, safety endpoints, 42
and benefit/risk assessment. The workflow process that included: feasibility assessment of 43
data sources; study outline; study team selection; protocol development; data extraction and 44
harmonisation; statistical analysis plans; data transformation & transfer; data pooling and 45
reporting, was tested. Aggregated data were shared on a remote research environment (RRE) 46
which could be accessed for collaborative work. 47
Results: Twenty-four databases responded to the AIRR survey: 16 had linkable individual 48
level population data; 12 of these captured routine immunisation data, 6 through linkage with 49
vaccine register data and the others through GPs input. A common data model (CDM), with 50
three input files (population, vaccines and events) was defined. Data access providers (DAPs) 51
extracted and transformed study-specific data into these common input files. Data analytical 52
tools were programmed by different statisticians against the CDM in R, Java and/or SAS and 53
sent to the DAPs, first to assess if the data were fit-for-purpose and subsequently for the 54
different ADVANCE proof-of-concept studies. The output of the analytical tools was 55
aggregated data that were sent to the RRE for pooling. Ten DAPs participated in fit-for-56
purpose assessment and shared aggregated data for pooling and reporting, seven participated 57
in the first ADVANCE proof-of-concept studies. 58
4
Conclusions: The ADVANCE system to generate evidence on background rates, coverage, 59
vaccine benefits and risks, from distributed healthcare databases was implemented and tested. 60
Key words: distributed network, common protocol, common analytics, vaccine, benefit-risk, 61
real world evidence 62
63
5
1. Introduction 64
The Accelerated Development of VAccine beNefit-risk Collaboration in Europe project 65
(ADVANCE) is a public-private consortium of 47 organisations, including the European 66
Medicines Agency (EMA) and the European Center for Disease Control and Prevention 67
(ECDC) (see Appendix for full list of ADVANCE consortium partners) [1]. The ADVANCE 68
consortium’s vision is to deliver the ‘best evidence at the right time to support decision-69
making on vaccination in Europe’, and its mission is to establish a prototype for a suitable-70
for-purpose and sustainable system that rapidly provides the best available scientific evidence 71
on post-marketing vaccine benefits and risks for informed decision making and effective post-72
marketing surveillance [2]. 73
To achieve the ADVANCE mission, a ‘big healthcare data’ ecosystem was designed and 74
tested. This comprised an integrated workflow for evidence generation from available and 75
heterogeneous health databases in different European countries (Figure 1). This workflow 76
was developed to deliver timely and actionable evidence based on stakeholder needs and 77
needed to abide by the ADVANCE principles of public health improvement, scientific 78
integrity, scientific independence, transparency and quality as laid out in the best practice 79
guidance and deliver suitable-for-purpose evidence for decision making for all stakeholders 80
[2, 3]. 81
The ADVANCE system was modelled on the American Vaccine Safety Datalink (VSD), 82
which is a collaboration between nine health maintenance organisations in the USA and the 83
Center of Disease Controls Immunisation Safety Office, in operation since 1990 [4]. The 84
pioneering accomplishments of the VSD demonstrated the synergy of collaboration for 85
methods and evidence generation, and the use of distributed data approaches for secure 86
analyses of de-identified data and the associated governance models [5]. 87
6
ADVANCE capitalised on the methods that were developed in Europe following the 88
pharmacovigilance reform that was recommended by the US Institute of Medicine (IoM) in 89
2007 after the rofecoxib safety concerns [6]. The IoM recommendations underlined the need 90
for large-scale analysis using existing big healthcare data and led to the creation of the US 91
FDA’s Sentinel and OMOP (Observational Medical Outcomes Project) projects [7, 8]. In 92
Europe the EU-ADR project (Exploring and Understanding Adverse Drug Reactions by 93
Integrating Mining of Clinical Records and Biomedical Knowledge) was funded under the 94
European Commission’s seventh framework programme in 2008 [9]. This project developed 95
methods, tools and a simple common data model (CDM) to leverage information from health 96
care databases in Europe. The ECDC decided to use the EU-ADR experience and apply it to 97
the field of vaccines to monitor pandemic vaccine safety in 2009 in the VAESCO project 98
(Vaccine Adverse Event Surveillance and Communication) [10]. Both the EU-ADR and 99
VAESCO projects were terminated in 2012 because no subsequent funding was available [10-100
12]. The Innovative Medicines Initiative (IMI) issued a call for proposals for a 5-year project 101
that would test a system for vaccine benefit-risk monitoring in the EU. The funding was 102
awarded to the ADVANCE consortium and in this present paper we will describe the system 103
that was developed to generate evidence from distributed health care databases. 104
2. ADVANCE system for vaccine benefit-risk monitoring in the EU 105
2.1. System description 106
The description of the system’s characteristics below is based on the general workflow shown 107
in Figure 1. This figure shows the general process of generating evidence, with the 108
accompanying information technology tools and analytical programmes for each step (at the 109
top), plus the document templates for each step (below the arrow) that have been made 110
available for re-use [13]. The workflow was designed to implement the principles of the 111
ADVANCE best practice guidance (Table 1) [3]. Each component of the workflow involves 112
7
multiple steps and decisions, which are detailed in the process map that is described in Figure 113
2 and the system will be described with reference to these different steps. 114
2.2. Pre-study readiness 115
According to our vision, the system should be ready to generate evidence quickly when 116
questions arise. Pre-study readiness to act can be created by having a catalogue of available 117
databases that are fit-for-purpose. This requires knowledge both on the type of data (meta-118
level), but also the detailed characteristics of the data, in the databases so that a study team 119
can quickly decide which databases are fit-for-purpose. We developed the ADVANCE 120
International Research Readiness (AIRR) instrument to identify and catalogue potential 121
healthcare databases in Europe [14]. This instrument collected information on generic health 122
database characteristics, provenance and coding of data for particular vaccines, as well as 123
information on the governance and modalities for data access. The AIRR survey was 124
disseminated within and outside the consortium and a total of 24 responses were received 125
(Table 1). 126
Data were extracted and mapped to a CDM to create pre-study readiness and assess 127
feasibility, enabling the use of one analytical computer program to transform data (Figure 3). 128
Instructions and programs were sent to the DAPs and they were asked to extract study-129
specific data from their local databases and to re-format them (with the programs and 130
algorithms agreed in the fingerprinting workflow) into the ADVANCE CDM format (Figure 131
3). The instructions contained a table with the local codes and keywords in the local language 132
that had to be extracted by DAPs to implement the CDM. 133
Consistent identification of events and vaccines is difficult in European health care databases 134
as the data are captured in different formats, with different coding schemes and languages 135
(Table 2). A semi-automatic CodeMapper tool was created to identify events uniformly [15]. 136
This tool created and logged recommended codes for specific events, which could be updated 137
8
by data access providers (DAPs). A vaccine ontology (VaccO) was created and tested to 138
support DAPs in data extraction and coding of vaccines, which were often in local languages, 139
for inclusion in the CDM [16]. Both CodeMapper and VaccO are available for future studies 140
[17, 18]. 141
A dedicated fingerprint (detailed data characterisation) workflow was created to assess 142
whether the data that were added to the common data model locally were ‘fit-for-purpose’ for 143
the proposed coverage, benefit and safety proof-of-concept (POC) studies [19-21]. The 144
fingerprint aimed to provide detailed descriptions of the population, vaccination and events 145
input files, by running an analytical program on the local data. Data characterisation programs 146
were available in Java (Jerboa for the population and event file checking) and R (for coverage 147
estimations), the output was transferred to a token-protected remote research environment 148
(RRE) (Figure 3). In ADVANCE, we used the previously described Octopus RRE [9] 149
Partners could remotely access all the data for checking and further analyses. Quality checks 150
were conducted by the operations teams on the output of the fingerprint programs on the input 151
files if novel data was extracted and DAPs were asked to review and validate the outputs. A 152
detailed description of the data in these input files is provided in another paper in this 153
supplement [22]. 154
2.3. Study planning 155
A study request can arrive from different organisations and have different aims. The first step 156
is a quick feasibility assessment using the study question, followed by the preparation of a 157
study outline. The study outline should include a description of the rationale, objectives, 158
population, vaccines and outcomes of interest. The study outline should approval by both the 159
study requester and the scientific committee; for the POC studies the Steering Committee 160
undertook the role of the scientific committee. 161
9
2.4. Study team 162
After approval of the study outline, a study team should be identified and the various roles 163
assigned. The ADVANCE code of conduct (CoC) describes the principles and roles in the 164
study team [3]. The process for the study team selection in the first round of proof of concept 165
studies involved an open call to the consortium. Interested persons for the various roles, i.e., 166
principal investigator (PI), epidemiologist, vaccinologist, disease experts, statistician, DAP 167
and study manager could apply by submitting their curriculum vitae and expression of 168
interest. All study team members had to sign and post a public declaration of interest, which 169
are available in the POC-1 study report (www.advance-vaccines.eu, results section D5.6). 170
Finally 68 individuals from all stakeholders formed the study teams for the first four POC 171
studies. Oversight was provided by an operations coordination team comprising three of the 172
authors (MS, LvdA and LT) that had regular team meetings as well as meetings with the 173
DAPs and statisticians. 174
Once the study team is established and roles assigned, a contract, and study governance model 175
and the CoC can be discussed and agreed, as described previously [23]. We used the 176
ADVANCE CoC, which allows public and private stakeholders to assume various roles under 177
conditions of full transparency. 178
2.5. Protocol and statistical analysis plan 179
An epidemiological study requires a protocol which is a document containing the 180
methodological details of the design, implementation, analyses, documentation and 181
publication of the study results. For the POC studies, the protocols were developed by the 182
study team members with the relevant expertise (i.e., clinical, epidemiological and statistical 183
expertise and expertise on specific clinical or methodological aspects of the study; data 184
privacy and ethics). We created a protocol template for epidemiological studies, based on the 185
EMA Post Authorisation Safety Study Template to comply with the Good Vigilance Practice 186
10
module VIII [24, 25]. A detailed document history was maintained to enable traceability and 187
to ensure transparency about the contributions of each member. Documented review and 188
approval of the protocol was required from the study team, the Steering Committee (acting as 189
the Scientific Review Committee), local governance and ethical clearance of the DAPs, as 190
well as internal review by the ECDC and the vaccine manufacturers. All amendments, 191
changes and rebuttals were tracked for transparency and could be consulted in the shared 192
work-space. The approved protocols with the ENCePP checklist were posted in the 193
publically-available EU PAS register [26-29]. 194
We used the standard definition of a statistical analysis plan, comprising a detailed description 195
for the data extraction, transformation and analyses, and the dummy tables for the reports, 196
based on the variables in the CDM. The CDM that was used was able to support the 197
fingerprinting, coverage, benefit and safety ADVANCE proof of concept (POC) studies 198
(Table 3). This CDM structure can be used as a base for future studies. If the study needs to 199
be expanded, compatibility with the previous programs should be verified so that they can be 200
reused. 201
The SAP was written by the study team statisticians and reviewed by the study and operations 202
teams. Based on the SAP, a detailed instruction document was developed by the operations 203
team for the DAPs. This described the data that had to be extracted, the programs that were to 204
be run, the intermediate files, the final output, and how the data were to be transferred. The 205
instruction documents, as well as the scripts, are available in the POC-1 report, and can be 206
used as templates for future studies [13]. More details on the programs are available below in 207
section 2.7. The SAPs were signed off after the data-analysis programmes were finalised, 208
prior to starting the report. The SAPs for the different studies are available on the ADVANCE 209
website and can be used as templates for future studies [13]. 210
11
2.6. Data (re-)extraction 211
Following approval of the POC study protocols, the DAPs were asked to re-extract (refresh) 212
and map their data locally into the common input files of the CDM according to the 213
harmonised processes described above in the pre-study readiness section. Extraction and 214
mapping programs were stored locally and not visible or modifiable by the study team. 215
2.7. Data transformation 216
The CDM input data files needed to be transformed locally into aggregated datasets/tables 217
that could be used to generate the evidence required to answer the study questions. We 218
developed a common modular programming approach for the data transformation steps. 219
Programs were written by the study team statisticians and supervised by the operations team. 220
R and SAS programs were created independently and their output verified against each other. 221
The programs contained general modules (e.g. vaccine input file cleaning (e.g. deduplication), 222
cohort selection) that can be varied with different parameter settings so that they can be 223
adapted for use in other studies. The programs were tested on real data, which were available 224
on the RRE, to enable rapid debugging. The final data transformation programs were sent to 225
the DAPs and run on the three CDM input files (Figure 3). Fully de-identified and minimised 226
analytical datasets (aggregated or individual record level, depending on the type of analysis) 227
were sent to the RRE (Figure 3). 228
2.8. Analyses and reporting 229
The aggregated data from the different databases were combined on the RRE using one-stage 230
pooling [30, 31]. For each POC study, the final statistical analyses were conducted remotely 231
by statisticians from different partners, on the RRE. The results were discussed with the DAPs 232
before the report was drafted. The interpretation and reporting of the results and the strengths 233
and limitations were the responsibility of the study team who followed The STrengthening the 234
Reporting of OBservational studies in Epidemiology (STROBE) guidance[32]. A summary of 235
12
the results has been published in the EU PAS register [33]. The full study report, including a 236
full document history, have been made available publicly on the ADVANCE website [13]. 237
Dissemination of evidence of scientific communication followed the best practice guidance 238
[34]. 239
3. Discussion 240
In this paper we described an integrated vaccine coverage, benefit and risk evidence 241
generation system that was designed and tested during the ADVANCE project. The system 242
was able to transform health data from multiple sources into answers. The steps followed the 243
five elements (define use cases, data ecosystem, modelling, workflow integration and 244
adoption) as outlined by the McKinsey Global Institute report for transforming health data 245
into evidence that can be used to inform businesses [35]. While putting the evidence 246
generation workflow together much effort was put into the data ecosystem: creation of 247
readiness of DAPs, fingerprinting, data verification, harmonisation and benchmarking to 248
ensure that the evidence can be perceived as reliable, since this is one of the main hurdles for 249
the adoption of big data evidence for decision-making in the health area [35]. Testing of the 250
ADVANCE system was conducted through a series of POC studies that each addressed 251
specific scientific questions; these are described in more details in other papers in this 252
supplement [19, 21, 36, 37]. 253
Through the AIRR survey we aimed to collect meta-data on databases, but unfortunately we 254
got only a few responses and the data were quickly out-of-date. A recent overview reported 255
that 13 of the 34 European databases, identified as being potentially suitable for regulatory 256
purposes, captured population, vaccination and vaccination-related event data [38]. Eight of 257
these databases were included in ADVANCE but the following were not: GePARD 258
(Germany), PHARMO (The Netherlands), and SNIRAM and IMS Disease analyser (France). 259
In the overview, IPCI (included in initial ADVANCE assessments) was one of the 13 to be 260
13
reported to have vaccine data, but as it contains data on influenza vaccination only it was 261
considered as not fit-for-purpose for general vaccine studies in ADVANCE. In contrast, 262
ADVANCE identified the Swedish National Registries (Sweden), The Health Improvement 263
Network (THIN) and Royal College of General Practitioners (RCGP) Research and 264
Surveillance Centre (RSC) (UK), Cremona (Italy), and FISABIO and BIFAP (Spain) as 265
databases that captured vaccination data, but this information was not identified in the 266
overview [38]. This emphasises that no catalogue is complete or sufficient, and therefore, 267
where updating and non-response may be an issue, we should focus our efforts on developing 268
a publicly-available catalogue of databases, rather than multiple project-specific catalogues. It 269
also shows that survey data alone are not sufficient and how important fingerprinting is to 270
really assess if databases are fit-for-purpose. 271
In order to put the ADVANCE system into a global context we compared the key 272
characteristics and principles of ongoing collaborative health care database networks in the 273
area of vaccines, based on publicly available information (Table 3). The identified initiatives 274
were: Sentinel, VSD (Vaccine Safety Datalink) and IMEDS (Innovation in Medical Evidence 275
Development and Surveillance) in the USA, CNODES (Canadian Network for Drug Effect 276
Studies) in Canada and I-MOVE+ (Integrated Monitoring of Vaccines in Europe), VENICE 277
(Vaccine European New Integrated Collaboration Effort) and DRIVE (Development of 278
Robust and Innovative Vaccines Effectiveness) in Europe [39-45]. 279
ADVANCE is limited to vaccines as are DRIVE, IMOVE+ and VSD, but it has the highest 280
number of partners in the vaccine area, with the most diversity, and includes all types of 281
stakeholders, e.g., vaccine manufacturers, public health agencies and regulatory authorities. 282
The other collaborative networks exclude vaccine manufacturers as collaborators (except for 283
DRIVE and IMEDS). ADVANCE considers all routine scheduled vaccines in vaccination 284
programmes and is not focused on specific vaccines unlike I-MOVE+ (influenza and 285
14
pneumococcal vaccines) and DRIVE (influenza vaccines). ADVANCE addresses a range of 286
study questions (coverage, effectiveness, safety and benefit-risk) whereas I-MOVE+ and 287
DRIVE focus on vaccine effectiveness alone. VSD, Sentinel, IMEDS and CNODES and 288
ADVANCE are restricted to the secondary use of electronic health care data and currently do 289
not cover primary data collection in contrast with I-MOVE+ and DRIVE. ADVANCE 290
follows an open science principle with publicly-available protocols, reports and tools, similar 291
to Sentinel and CNODES, although CNODES, I-MOVE+ and VENICE provide public access 292
to only selected documents. 293
In addition to the scope and partnerships, differences exist also in the use of CDMs. In 294
ADVANCE we used the ADVANCE CDM, similar to the CDM that was used in previous 295
EU-funded distributed network studies such as in EU-ADR, SOS, SAFEGUARD, ARITMO, 296
EMIF, VAESCO although we made slight adaptations to the drug file to capture vaccine 297
specific information [9]. The similarity in structure of the CDM (variable names) allowed for 298
the re-use of available open source software, such as Jerboa. The actual data content of the 299
input files that are in the CDM is study-specific and limited to the need of the studies. This 300
was done for efficiency reasons, since we aimed to check the quality of all the data in the 301
input files, and this would have been burdensome if data irrelevant for the study were 302
converted into the CDM format. The process of inputting study specific data in the CDM in 303
ADVANCE differs from that used, for example, in the Sentinel and VSD projects, which 304
involves data conversion of the whole database to a CDM. In the resultant generalised CDM, 305
all the local data are transformed to be used as a base for data processing. Transforming the 306
entire database contents into the CDM would be possible in Europe but it would require a lot 307
of resources and effort, which need to be structurally financed. In the absence of such 308
funding, a study-specific approach can be more cost-efficient. Moreover, the assessment of 309
15
the data quality is more focused in a study specific manner. To create readiness of people and 310
data, we need to fully engaged the DAPs to leverage their experience and local expertise. 311
ADVANCE currently relies on secondary use of sensitive data, which is protected by the 312
General Data Protection Regulation (GDPR) that was fully implemented on May 25, 2018. 313
The new GDPR rules have a significant impact on pharmacoepidemiological studies for 314
future vaccine benefit-risk studies, the rules require that data are protected by design and by 315
default, which necessitates that data protection is included in the business processes for data 316
processing and management (including physical and technical safeguards, privacy enhancing 317
technologies, minimisation of processing principles, and robust data access documented 318
procedures). Although ADVANCE designed and tested the system prior to the obligation to 319
GDPR, steps for privacy by design, such as minimisation of data (only sharing project-320
required data, which were also anonymised to minimise disclosure risk) were implemented. 321
DAPs are currently assigning data privacy officers and privacy impact assessments will have 322
to be done to ensure that the databases are GDPR ready. A second POC study will be 323
implemented in early 2019, allowing to test the GDPR implementation. 324
Although we have tested the ADVANCE evidence generation system, there are 325
recommendations and areas for improvement. For the further development of the system, a 326
formal description of the local data and the local programs that feed the CDM could be 327
developed and shared, for further transparency and reproducibility. In addition the remote 328
platform for data sharing should allow token-less access (preferable phone authentication), 329
and should be able to provide the required processing and storage capacity through a private 330
cloud-based system. 331
In summary, the ADVANCE project has developed and implemented a system that can 332
generate evidence on vaccine coverage, benefits and risk from distributed available electronic 333
health care database in Europe. We have implemented the ADVANCE CoC and created a 334
16
transparent process map, tools and templates which are ready to be adopted and go in 335
production in the post-ADVANCE VAC4EU (Vaccine monitoring Collaboration for Europe) 336
association, which will continue the work described here, after the ADVANCE project ends. 337
338
17
Acknowledgements 339
The authors would like to acknowledge all users of the system for their contributions, testing, 340
assessments and comments: Maria de Ridder, Benus Becker, Peter Rijnbeek, Mees Mosseveld 341
(Erasmus MC, The Netherlands), Olivia Mahaux, Francois Hanguinet (GSK, Belgium), Toon 342
Braeye (Sciensano, Belgium), Marius Gheorge (Oslo University, Norway), Chris McGee 343
(University of Surrey and RCGP-RSC, UK), Elisa Martin (BIFAP, Spain), Talita Duarte-344
Talles (SIDIAP, Spain), Lisen Arnheim-Dahlstrom (Karolinska Institute, Sweden), Klara 345
Berencsi, Johnny Kahlert, Lars Pedersen (Aarhus University, Denmark), Matti Lehtinen 346
(University of Tampere, Finland), Marco Villa, Silvia Lucchi (Agenzia Ospedaliero Padana, 347
Italy), Giuseppe Roberto, Claudia Bartolini (Agenzia Regionale Toscana, Italy), Gino Picelli, 348
Lara Tramontan, Giorgia Danieli (PEDIANET), Caitlin Dodd (University Medical Center 349
Utrecht, The Netherlands), Myint Tin Tin Htar (Pfizer, France), Tom de Smedt (P-95). 350
We also acknowledge Margaret Haugh (MediCom Consult, France) for medical writing and 351
editorial assistance 352
353
18
Disclaimer statement 354
The views expressed in this article are the personal views of the authors and should not be 355
understood or quoted as being made on behalf of or reflecting the position of the agencies or 356
organisations with which the authors are affiliated. 357
358
19
Funding source 359
The Innovative Medicines Initiative Joint Undertaking funded this project under ADVANCE 360
grant agreement n° 115557, resources of which were composed of a financial contribution 361
from the European Union's Seventh Framework Programme (FP7/2007-2013) and in kind 362
contributions from EFPIA member companies. 363
364
20
Declaration of conflicts of interest 365
Lieke van der Aa, Kaatje Bollaerts, Hanne-Dorthe Emborg, Harshana Liyanage and Lampros 366
Stergioulas declared no conflict of interests. Miriam Sturkenboom declared that she has 367
received grants from Novartis, CDC and Bill & Melinda Gates Foundation for work unrelated 368
to the work presented here. Rosa Gini declared that she has grants from Innovative Medicines 369
Initiative (IMI) for the EMIF project, and grants from PHARMO Institute for work unrelated 370
to the work presented here. Simon de Lusignan declared he is: Director of RCGP RSC and 371
has university-based research (enhanced surveillance of influenza vaccines) funded by GSK, 372
he is also a member of Seqirus and Sanofi Pasteur Advisory Boards for influenza. Alena 373
Khromava declared that she employed by Sanofi Pasteur and holds company shares/stock 374
options. Germano Ferreira declared that he was employed by Sanofi Pasteur, GSK Vaccines 375
and P95 during the project. Lina Titievsky declared that she is employed by Pfizer and holds 376
company stocks/shares. Daniel Weibel declared that he has received reports personal fees 377
from GSK for work unrelated to the work presented here. 378
379
21
References 380
[1] ADVANCE. Available at: http://www.advance-vaccines.eu/. Accessed on: 20 June 2018. 381
[2] Sturkenboom M, Bahri P, Chiucchiuini A, Grove Krause T, S. H, Khromava A, et al. Why 382
we need more collaboration in Europe to enhance post-marketing surveillance of vaccines. 383
Vaccine. 2018;Paper 1 in ADVANCE supplement. 384
[3] Kurz X, Bauchau V, Mahy P, Glismann S, van der Aa LM, Simondon F. The ADVANCE 385
Code of Conduct for collaborative vaccine studies. Vaccine. 2017;35:1844-55. 386
[4] Fahey KR. The Pioneering Role of the Vaccine Safety Datalink Project (VSD) to Advance 387
Collaborative Research and Distributed Data Networks. EGEMS (Washington, DC). 388
2015;3:1195. 389
[5] McNeil MM, Gee J, Weintraub ES, Belongia EA, Lee GM, Glanz JM, et al. The Vaccine 390
Safety Datalink: successes and challenges monitoring vaccine safety. Vaccine. 2014;32:5390-391
8. 392
[6] Medicine Io. The future of drug safety: promoting and protecting the health of the public. 393
Washington, DC: The National Academies Press; 2007. 394
[7] Platt R, Wilson M, Chan KA, Benner JS, Marchibroda J, McClellan M. The New 395
Sentinel Network — Improving the Evidence of Medical-Product Safety. N Engl J Med. 396
2009;361:645-7. 397
[8] Ryan PB, Madigan D, Stang PE, Overhage JM, Racoosin JA, Hartzema AG. Empirical 398
assessment of methods for risk identification in healthcare data: results from the experiments 399
of the Observational Medical Outcomes Partnership. Stat Med. 2012;31:4401-15. 400
[9] Trifiro G, Coloma PM, Rijnbeek PR, Romio S, Mosseveld B, Weibel D, et al. Combining 401
multiple healthcare databases for postmarketing drug and vaccine safety surveillance: why 402
and how? J Intern Med. 2014;275:551-61. 403
22
[10] Dieleman J, Romio S, Johansen K, Weibel D, Bonhoeffer J, Sturkenboom M. Guillain-404
Barré syndrome and adjuvanted pandemic influenza A (H1N1) 2009 vaccine: multinational 405
case-control study in Europe. BMJ. 2011;343:d3908. 406
[11] Romio S, Weibel D, Dieleman JP, Olberg HK, de Vries CS, Sammon C, et al. Guillain-407
Barre syndrome and adjuvanted pandemic influenza A (H1N1) 2009 vaccines: a multinational 408
self-controlled case series in Europe. PloS One. 2014;9:e82222. 409
[12] Sturkenboom M. Advancing collaborative vaccine benefits and safety research in Europe 410
via the ADVANCE code of conduct. Vaccine. 2018;36:194-5. 411
[13] ADVANCE. D5.6 Results of POC-phase 1studies, WP5 – Proof-of-concept studies of a 412
framework to perform vaccine benefit-risk monitoring. 2017. 413
[14] ADVANCE. Deliverable D3.2 First report on data access, linkage and integration for 414
POC studies [Internal document]. 2016. 415
[15] Becker BFH, Avillach P, Romio S, van Mulligen EM, Weibel D, Sturkenboom M, et al. 416
CodeMapper: semiautomatic coding of case definitions. A contribution from the ADVANCE 417
project. Pharmacoepidemiol Drug Saf. 2017;26(8):998-1005. 418
[16] Becker B, Sturkenboom M, Kors J, Martin Merino E, Roberto G. D5.5 Ontology for the 419
integration and extraction of vaccine-related information in Europe: a proof of concept 420
Available at: http://www.advance-421
vaccines.eu/app/archivos/publicacion/54/ADVANCE%20D5.5%20Vaccine%20Ontology_v1422
_clean_noappendix.pdf. Accessed on: 16 May 2018. 423
[17] ADVANCE. CodeMapper website (restricted access). Available at: 424
https://euadr.erasmusmc.nl/CodeMapper. Accessed on: 17 June 2017. 425
[18] ADVANCE. ADVANCE VaccO (restricted access). Available at: http://www.advance-426
vaccines.eu/?page=toolbox&id=vaccO. Accessed on: 22 December. 427
23
[19] Emborg HD, Berensci K, Braeye T, Bauwens J, Bollaerts K, Correa A, et al. ADVANCE 428
system testing: can coverage of pertussis vaccination be estimated in EU countries using 429
electronic health data: an example. Vaccine. 2018;Paper 4 in Supplement. 430
[20] Tin Tin Htar M, de Ridder M, Braeye T, Correa A, McGee C, de Lusignan S, et al. 431
ADVANCE system testing: vaccine benefit studies using multi-country electronic health data 432
- the example of pertussis vaccination. Vaccine. 2018:Paper 5 in this supplement. 433
[21] Weibel D, Dodd C, Mahaux O, Haguinet F, de Smedt T, Duarte-Salles T, et al. 434
ADVANCE system testing: can safety studies be conducted using electronic health data: an 435
example with pertussis vaccination? Vaccine. 2018;Paper 6 in Supplement. 436
[22] Sturkenboom M, Weibel D, van der Aa L, Braeye T, Gheorge M, Becker B, et al. 437
ADVANCE database characterization and fit for purpose assessment for multi-country studies 438
on the coverage, benefits and risks of vaccinations. Vaccine. 2018;Paper 3 in Supplement. 439
[23] Torcel-Pagnon L, Bauchau V, Mahy P, Htar MTT, Van der Sande M, Mahe C, et al. 440
Guidance for the governance of public-private collaborations in vaccine post-marketing 441
settings in Europe. Vaccine. 2018;Submitted. 442
[24] European Medicines Agency. Guidance for the format and content of the protocol of 443
non-interventional post-authorisation safety studies. Available at: 444
https://www.ema.europa.eu/documents/other/guidance-format-content-protocol-non-445
interventional-post-authorisation-safety-studies_en.pdf. Accessed on: 30 November 2018. 446
[25] European Medicines Agency. Guideline on good pharmacovigilance practices (GVP) 447
Module VIII – Post-authorisation safety studies (Rev 3). Available. Accessed on: 16 August 448
2018. 449
[26] EU PAS Register. ADVANCE Coverage POC. Available at: 450
http://www.encepp.eu/encepp/viewResource.htm?id=21742. Accessed on: 16 August 2018. 451
24
[27] EU PAS Register. ADVANCE POC I Benefit-Risk. Available at: 452
http://www.encepp.eu/encepp/viewResource.htm?id=21729. Accessed on: 16 August 2018. 453
[28] EU PAS Register. ADVANCE POC I Risk. Available at: 454
http://www.encepp.eu/encepp/viewResource.htm?id=21721. Accessed on: 16 August 2018. 455
[29] EU PAS Register. Benefit study on pertussis vaccination. Available at: 456
http://www.encepp.eu/encepp/viewResource.htm?id=21757. Accessed on: 16 August 2018. 457
[30] La Gamba F, Corrao G, Romio S, Sturkenboom M, Trifiro G, Schink T, et al. Combining 458
evidence from multiple electronic health care databases: performances of one-stage and two-459
stage meta-analysis in matched case-control studies. Pharmacoepidemiol Drug Saf. 460
2017;26:1213-9. 461
[31] Scotti L, Rea F, Corrao G. One-stage and two-stage meta-analysis of individual 462
participant data led to consistent summarized evidence: lessons learned from combining 463
multiple databases. J Clin Epidemiol. 2018;95:19-27. 464
[32] STROBE Initiative. STrengthening the Reporting of OBservational studies in 465
Epidemiology Statement: Guidelines for reporting observational studies. Available at: 466
https://www.strobe-statement.org/index.php?id=available-checklists. Accessed on: 12 467
December 2018. 468
[33] ADVANCE. Executive summary POC-1 studies on pertussis coverage, benefits, risks 469
and B/R analysis. Available at: 470
http://www.encepp.eu/encepp/openAttachment/studyResult/21740. Accessed on: 12 471
December 2018. 472
[34] ADVANCE. Deliverable 1.12 Developing communication strategies on vaccine benefits 473
and risks: Guidance for public-private collaborations Available at: Deliverable 1.12 474
Developing communication strategies on vaccine benefits and risks: Guidance for public-475
private collaborations Accessed on: 12 December 2018. 476
25
[35] Institute; MG. The age of analytics: competing in a data-driven world 477
Available at: 478
https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Analyt479
ics/Our%20Insights/The%20age%20of%20analytics%20Competing%20in%20a%20data%20480
driven%20world/MGI-The-Age-of-Analytics-Full-report.ashx. Accessed on: 13 December 481
2018. 482
[36] Bollaerts K, Ledent E, de Smedt T, Weibel D, Emborg HD, Correa A, et al. ADVANCE 483
system testing: benefit-risk analysis of a marketed vaccine using multi-criteria decision 484
analysis and cohort modelling. Vaccine. 2018;Paper 9 in this supplement. 485
[37] Htar MTT, de Ridder M, Braeye T, Correa A, de Lusignan S, Duarte-Salles T, et al. 486
ADVANCE system testing: vaccine benefit studies by using multi-country electronic health 487
data: An example on pertussis vaccination. Vaccine. 2018;Paper 5 in Supplement. 488
[38] Pacurariu A, Plueschke K, McGettigan P, Morales DR, Slattery J, Vogl D, et al. 489
Electronic healthcare databases in Europe: descriptive analysis of characteristics and potential 490
for use in medicines regulation. BMJ Open. 2018;8:e023090. 491
[39] The Sentinel System. Available at: https://www.sentinelinitiative.org/. Accessed on: 15 492
May 2018. 493
[40] Centres for Disease Control and Prevention. Vaccine Safety Datalink (VSD). Available 494
at: https://www.cdc.gov/vaccinesafety/ensuringsafety/monitoring/vsd/index.html. Accessed 495
on: 15 May 2018. 496
[41] Administration; FaD. Innovations in Medical Evidence Development and Surveillance. 497
Available at: https://www.federalregister.gov/documents/2015/04/15/2015-498
08613/innovations-in-medical-evidence-development-and-surveillance-methods-research-499
agenda. Accessed on: 13 December 2018. 500
26
[42] Suissa S, Henry D, Caetano P, Dormuth CR, Ernst P, Hemmelgarn B, et al. CNODES: 501
the Canadian Network for Observational Drug Effect Studies. Open medicine : a peer-502
reviewed, independent, open-access journal. 2012;6:e134-40. 503
[43] I-MOVE+ (Integrated Monitoring of Vaccines in Europe). Available at: http://www.i-504
moveplus.eu/. Accessed on: 13 December 2018. 505
[44] Vaccine European New Integrated Collaboration Effort (VENICE) III. Available at: 506
http://venice.cineca.org/. Accessed on: 13 December 2018. 507
[45] DRIVE – Development of Robust and Innovative Vaccine Effectiveness (DRIVE): 508
increasing understanding of influenza vaccine effectiveness in Europe. Available at: 509
https://www.drive-eu.org/. Accessed on: 13 December 2018. 510
511
512
513
27
Figure captions 514
Figure 1: Generic ADVANCE system description, outlining the workflows, tools and 515
supporting documentation based on the ADVANCE best practice guidance that were 516
translated in operational procedures 517
Figure 2: Detailed process map for generation of evidence in the ADVANCE system 518
Figure 3: Data extraction, transformation and analysis infrastructure in the ADVANCE 519
system 520
521
28
Table 1: Implementation of the ADVANCE code of conduct for the proof-of-concept (POC) studies 522
Topic CoC Recommendations to be applied in all studies (‘must’) CoC Other recommendations to be
considered for all studies (‘should’)
Implementation in POC studies
1.
Sci
enti
fic
inte
gri
ty
1. All study team members are qualified to fulfil their role 1.Members applied for roles with CV. Qualifications
reviewed by decision making organ
2. All study team members act in accordance with core values of
honesty, accuracy and objectivity
2. Compliance statement in application letter for study
team
3. Study team members adhere to IEA Good Epidemiological Practice
and ISPE Good Pharmacoepidemiological Practices
3. Compliance statement in application letter for study
team
2.
Sci
enti
fic
ind
epen
den
ce
4. Study is conducted without undue influence of any financial,
commercial, institutional or personal interest in a particular outcome of
the study
5. Autonomy of members of study team for
making scientific decisions in their
organisation is documented
4. Declarations of interest signed and made publicly
available by all members
5. Was not documented in PoC
6. Scientific independence is safeguarded by clear and transparent roles
and responsibilities, peer review process, transparency measures and
disclosure of all funding sources
6. Full transparency on DoI, roles and comments /edits
throughout the study. No study member can decide alone.
Tracked change documents and rebuttals on Sharepoint &
detailed document histories
3.
Tra
nsp
aren
cy
7. Study is registered in a publicly accessible database before the start of
data collection or extraction
11. Final study report or summary is
uploaded in publicly accessible database
where study is registered
7. Studies registered in ENCePP registry
11. Final report in ENCePP registry and on ADVANCE
website
8. Sources of funding are made public at the time of registration, in the
protocol and in the presentation of results
12. After study completion, study information
is made available from outside the study team
in a collaborative approach
8. IMI funding is made public
12. Study information available on ADVANCE public
website
9. Declarations of Interests (DoI) are made available at an early stage of
the study, regularly updated and disclosed
13. In case of primary data collection,
participants in the study or their
representatives may receive main study
results and interpretation thereof
9. Declaration of interests for each study team member
publicly available on ADVANCE website
13. Not tested only secondary use
10. All comments received on study protocol and results with impact on
the study are documented
10. Detailed track changes and comments plus rebuttals on
protocol and report publicly available
4.
Co
nfl
ict
of
inte
rest
14. Actual or potential conflicts of interest (and perceptions thereof) are
addressed at the planning phase of the study. Research contract includes
a description of the management of conflicts of interest. All DoI are
made publicly available
15. A standard form is used to declare all
interest that may lead to conflicts
14. DoI publicly available. No process for dealing with
potential conflicts of interests was tested.
15. DoI were collected in a standard form
5.
Stu
dy
pro
toco
l
16. A study protocol is drafted as one of the first step in any research
projects
18. The process for reaching an agreement on
the design options of the study is agreed
beforehand
16. study protocol drafted
18. Design choice decided first in study team and subs. By
Steering committee
17. Study protocol is developed by persons with relevant expertise 22. Detailed draft protocol undergoes
independent scientific review
17. Study team composition had clear entry criteria
(epidemiology PhD, vaccinologists, disease experts,
statistician)
22. Draft protocol reviewed by SC and Scientific Advisory
Board
29
Topic CoC Recommendations to be applied in all studies (‘must’) CoC Other recommendations to be
considered for all studies (‘should’)
Implementation in POC studies
19. Protocol includes a section with ethical considerations involved and
information on funding, affiliations, potential conflicts of interest, data
protection and incentives to subjects. Protocol is approved by relevant
research ethics committee
23. Protocol is registered in publicly
accessible database before the start of data
collection
19. EMA PASS protocol template used with ethical,
privacy & regulatory section
23. All protocols in ENCePP database before start of data
collection
20. Protocol includes description of each party to study design, protocol
writing and work programme
20. Detailed document history in Protocol and study team
roles/responsibilities
21. Regulatory obligations and recommendations applicable to the study
are described
21.Section on regulatory obligations stated in protocol
24. Changes to the protocol that may affect the interpretation of the study
are identifiable and reported in the study report
24. All amendments listed in report. Report also has
limitation section
25. Key statistical analyses are described 25. Detailed Statistical analysis plan provided separately
and public, summary described in report
6.
Stu
dy
rep
ort
26. Set of principles are followed for reporting results including
documentation of important safety concerns and deviations from protocol
or statistical analysis plan, sources affecting data quality, strengths and
limitations, and sources of funding
27. STROBE statement and internationally
agreed guidelines are consulted when
analysing and reporting data
26. Reporting results according to EMA PASS report
template, including limitations, data quality and funding
27. STROBE and RECORD guidelines followed in report
28. Draft study report undergoes independent
scientific review
29. Study report evaluated by SAB, SC and in house
29. Study report or summary of the results is
included in the publicly accessible database
29. Study report is available online (ADVANCE website)
and summary in ENCePP study registry
7.
Pu
bli
cati
on
30. All study results are made publicly available. Authorship of
publications follows the rules of ICMJE
32. Preliminary or partial results of
discontinued study are reported and identified
as such
30. Report publicly available & in line with ICMJE
32. Monitoring of coverage, risks and benefits not done,
but part of POC1.2 (was described)
31. Research contract allows the principal investigators and relevant
study team members to publish study results independently from the
funding or data source. The study requester/funder may provide
comments
31. by IMI grant agreement & consortium processe,
publication rights.
33. Procedures are in place to rapidly inform regulatory and public health
authorities of study results, independently from submission of a
manuscript
33. Communication procedures in place and less relevant
because all stakeholders are involved
8.
Su
bje
ct
pri
vac
y
34. Privacy of study subjects in relation to personal data is core principle
of any medical research
34. Study in compliance with relevant EU directives
35. In case where personal data are collected, the applicable legislation is
followed
34. Study in compliance with relevant EU directives and
national implementation thereof. Detailed data collected
through P& E tool
9.
Sh
arin
g o
f
stu
dy d
ata
bey
on
d s
tudy 38. Sharing of study data is based on a written request specifying the
ground of the request. The study team verifies the compliance of the
request with the data protection legislation
36. There is an open and collaborative
approach to sharing study data with persons
from outside the study team
36. Study results shared publicly. Data can be accessed
through RRE after signature of confidentiality & security
documents
38. Has not occurred but processes in place
39. Requests for data sharing are justified based on public health interest 37. Data are shared only after the study report
is finalized
37. Has not occurred
39. Has not occurred
30
Topic CoC Recommendations to be applied in all studies (‘must’) CoC Other recommendations to be
considered for all studies (‘should’)
Implementation in POC studies
41. Analyses performed with shared data follow the provisions of the
ADVANCE Code of Conduct
40. Study team or delegated committee takes
the decision to share study data
40. Has not occurred
41. Has not occurred
10
. R
esea
rch
con
trac
t
42. The research contract does not lead investigators, directly or
indirectly, to act against the principles of the Helsinki Declaration or
applicable legal or regulatory obligations
44. Unique multiparty contract is preferred in
cases where several parties interact
42/44. Could not be tested due to IMI context
43. Clarity and transparency are key elements of the research contract 45. The research contracts indicates that the
study will follow the ADVANCE Code of
Conduct and provides core information
43/45. Could not be tested due to IMI context
523
524
31
Table 2: Summary of databases that responded to the ADVANCE International Research Readiness (AIRR) survey 525
Database name Country Type of database Diagnoses Availability of vaccine related
information in the database Linked population file
The Belgian network of Sentinel General
Practitioners (SGP)
Belgium
Primary health care Specific diseases Yes (incomplete) Not linked, but catchment
Belgium
Pediatric Surveillance Network (Pedisurv) Pediatric infectious disease
surveillance
Specific diseases Yes (incomplete) Not linked, but catchment
Belgium
Network National reference centers for
human microbiology
infectious disease
surveillance
Specific diseases Yes Not linked, but catchment
Belgium
Belgian Health Interview Survey Population survey Specific diseases Yes Not linked, but catchment
Belgium
National Surveillance of Influenza Like
Illness
infectious disease
surveillance
Specific diseases No Not linked, but catchment
Belgium
National Surveillance of SARI infectious disease
surveillance
Specific diseases No Not linked, but catchment
Belgium
Sentinel Laboratory Network (SLN) infectious disease
surveillance
Specific diseases No Not linked, but catchment
Belgium
MATRA communicable diseases Specific diseases No 3,600,000
MATRA-BRU communicable diseases Specific diseases No 1,200,000
Aarhus University Hospital
Denmark
Population based record
linkage
Hospitalisations &
emergency visits, death
no primary care (ICD-
10)
Yes (linked to vaccine registry) 1,700,000
The Danish Civil and Health Registration
System
Population based record
linkage
Hospitalisations &
emergency visits, death
no primary care (ICD-
10)
Yes 7,500,000
HPV-CRT Cohort Finland Cohort after trial In-outpatient (ICD-10) Yes (linked to vaccine registry) 35,000
PEDIANET (only Veneto data)
Italy
Medical record (family
paediatricians)
Primary care & reported
specialist
visits/hospitalizations
Yes (part linked to vaccine
registry) 280,000 (9,000 linked)
Agenzia regionale di sanità della Toscana
(ARS)
population based record
linkage
Hospitalisations,
emergency visits, death
no primary care (ICD-9)
Yes (linked to vaccine registry) 5,000,000
Arianna Medical record (family
doctors
Primary care & reported
specialist
visits/hospitalizations
No 1,100,000
32
Database name Country Type of database Diagnoses Availability of vaccine related
information in the database Linked population file
ASLCR/Val Padana Population based record
linkage
Hospitalisations,
emergency visits, death
no primary care (ICD-9)
Yes (linked to vaccine registry) 450,000
Integrated Primary Care Information
Netherlands
Medical record (family
doctors
Primary care & reported
specialist
visits/hospitalizations
(ICPC, text)
No, only influenza 1,800,000
Osiris Surveillance cases Specific diseases Yes (incomplete) Not linked, but catchment
NL
The Information System for the
Development of Research in Primary Care
(SIDIAP)
Spain
Population based record
linkage
Primary care & reported
specialist
visits/hospitalizations,
50% linkable to
hospital data (ICD-10)
Yes, those provided by GPs 5,800,000
BIFAP (Base de Datos para la
Investigación Farmacoepidemiológica en
Atención Primaria)
Medical record (family
doctors
Primary care & reported
specialist
visits/hospitalizations
(ICPC, ICD-9, text)
Yes those provided by GP 4,800,000
FISABIO Population based record
linkage
Primary care & reported
specialist visits linked
hospitalizations (ICD-9)
yes (linked to vaccine registry) 5,000,000
Karolinka Institute Sweden Population based record
linkage
Hospitalisations, no
primary care(ICD-10)
Available on request for part of
the population 9,400,000
The Health Improvement Network (THIN)
database
United
Kingdom
Medical record (family
doctors
Primary care & reported
specialist
visits/hospitalizations
(READv2)
Yes those provided by GPs 8,300,000
Royal College of General Practitioners
(RCGP) Research & Surveillance Centre
(RSC)
United
Kingdom
Medical record (family
doctors)
Primary care & reported
specialist
visits/hospitalizations
(READv2, READCtv3)
Yes those provided by GP 2,600,000
526
33
Table 3: Key characteristics of the ADVANCE distributed network and other organised distributed networks addressing vaccine studies 527
USA Canada EU
Sentinel VSD IMEDS CNODES ADVANCE DRIVE VENICE
III I-MOVE+
Scope and size
Scope / type of
vaccines
Effectiveness
and safety / all
vaccines
Safety / all
vaccines
Effectiveness
and safety / all
vaccines
Effectiveness
and safety / all
vaccines
Coverage,
effectiveness and
safety / all
vaccines
Effectiveness /
influenza
Coverage /
all vaccines
Effectiveness /
influenza and
pneumococcal
No. of partners 34 9 NA 8 47 15 7 26
No. of countries 1 1 1 3 19 7 31 15
Type of data used Secondary use
EHR
Secondary
use EHR
Secondary use
EHR
Secondary use
EHR
Secondary use
EHR
Primary data
collection and
secondary use
EHR
Survey
Primary data
collection and
secondary use EHR
No. EHR DAPs in
consortium 18 8 18 8 19 3 NA 8
No. lives in
eligible EHR
DAPs (active)
67 M 9 M 67 M >100 M 30 M NA NA NA
Principles and operations
Inclusiveness No VM No VM VM can
collaborate No VM
VM can
collaborate
VM can
collaborate No VM No VM
CoI handling of
financial interest Excluded Excluded Managed Managed Managed Managed Excluded Excluded
CDM Sentinel deep
CDM
VSD deep
CDM
Sentinel deep
CDM No CDM
Study specific
CDM
study specific
CDM No
Study specific
CDM
Common data
transformation
script
Yes Yes Yes No Yes No No Yes
Script language SAS SAS SAS SAS R, SAS NA NA NA
Data management
& analysis Central Rotate Central Central Rotate Central Central Central
34
PRISM: Post-licensure rapid immunization safety monitoring program; VSD: Vaccine Safety Datalink; IMEDS; Innovation in medical evidence 528
development and surveillance; CNODES; Canadian Network for Observation Drug Evaluation Studies; FDA=Food and Drug Administration; 529
CDC: Centers for Disease Control and Prevention; P&P: policy and principles; VM: vaccine manufacturer; CDM: common data model; DAP: 530
data access provider; RI: research institute; RG; regulatory agency; PHI: public health institute; NA: not available; P&R&T; protocols, reports 531
and tools; P&R protocols & reports; R: reports 532
533
35
Appendix: Members of ADVANCE consortium (October 2018) 534
Full partners 535
AEMPS: Agencia Española de Medicamentos y Productos Sanitarios (www.aemps.es) 536
ARS-Toscana: Agenzia regionale di sanità della Toscana (https://www.ars.toscana.it/it/) 537
ASLCR: Azienda Sanitaria Locale della Provincia di Cremona (www.aslcremona.it) 538
AUH: Aarhus Universitetshospital (kea.au.dk/en/home) 539
ECDC: European Centre of Disease Prevention and Control (www.ecdc.europa.eu) 540
EMA: European Medicines Agency (www.ema.europa.eu) 541
EMC: Erasmus Universitair Medisch Centrum Rotterdam (www.erasmusmc.nl) 542
GSK: GlaxoSmithKline Biologicals (www.gsk.com) 543
IDIAP: Jordi Gol Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut 544
Jordi Gol i Gurina (http://www.idiapjordigol.com) 545
JANSSEN: Janssen Vaccines - Prevention B.V. (http://www.janssen.com/infectious-diseases-546
and-vaccines/crucell) 547
KI: Karolinska Institutet (ki.se/meb) 548
LSHTM: London School of Hygiene & Tropical Medicine (www.lshtm.ac.uk) 549
MHRA: Medicines and Healthcare products Regulatory Agency (www.mhra.gov.uk/) 550
MSD: Merck Sharp & Dohme Corp. (www.merck.com) 551
NOVARTIS: Novartis Pharma AG (www.novartisvaccines.com) 552
OU: The Open University (www.open.ac.uk) 553
P95: P95 (www.p-95.com) 554
PEDIANET: Società Servizi Telematici SRL (www.pedianet.it) 555
PFIZER: Pfizer Limited (www.pfizer.co.uk) 556
RCGP RSC: Royal College of General Practitioners Research and Surveillance Centre 557
(www.rcgp.org.uk/rsc) 558
36
RIVM: Rijksinstituut voor Volksgezondheid en Milieu (www.rivm.nl) 559
SCIENSANO: Sciensano (https://www.sciensano.be) 560
SP: Sanofi Pasteur (www.sanofipasteur.com) 561
SSI: Statens Serum Institut (www.ssi.dk) 562
SURREY: The University of Surrey (www.surrey.ac.uk) 563
SYNAPSE: Synapse Research Management Partners, S.L. (www.synapse-managers.com) 564
TAKEDA: Takeda Pharmaceuticals International GmbH (www.tpi.takeda.com) 565
UNIBAS-UKBB: Universitaet Basel – Children’s Hospital Basel (www.unibas.ch) 566
UTA: Tampereen Yliopisto (www.uta.fi) 567
Associate partners 568
AIFA: Italian Medicines Agency (www.agenziafarmaco.it) 569
ANSM: French National Agency for Medicines and Health Products Safety (ansm.sante.fr) 570
BCF: Brighton Collaboration Foundation (brightoncollaboration.org) 571
EOF: Helenic Medicines Agency, National Organisation for Medicines (www.eof.gr) 572
FISABIO: Foundation for the Promotion of Health and Biomedical Research 573
(www.fisabio.es) 574
HCDCP: Hellenic Centre for Disease Control and Prevention (www.keelpno.gr) 575
ICL: Imperial College London (www.imperial.ac.uk) 576
IMB/HPRA: Irish Medicines Board (www.hpra.ie) 577
IRD: Institut de Recherche et Développement (www.ird.fr) 578
NCE: National Center for Epidemiology (www.oek.hu) 579
NSPH: Hellenic National School of Public Health (www.nsph.gr) 580
PHE: Public Health England (www.gov.uk/government/organisations/public-health-england) 581
THL: National Institute for Health and Welfare (www.thl.fi) 582
UMCU: Universitair Medisch Centrum Utrecht (www.umcu.nl) 583
37
UOA: University of Athens (www.uoa.gr) 584
UNIME: University of Messina (www.unime.it) 585
Vaccine.Grid: Vaccine.Grid (http://www.vaccinegrid.org/) 586
VVKT: State Medicines Control Agency (www.vvkt.lt) 587
WUM: Polish Medicines Agency - Warszawski Uniwersytet Medyczny 588
(https://wld.wum.edu.pl/) 589
Study question Data sources Study team Protocol Data extraction
and checks Data
transformation Analysis and dissemination
Evid
ence
gen
erati
on
IT s
up
port
tools
Catalogue Community list Methods and
design tools
CodeMapper
VaccO
SAS, R & JAVA
programs
Dashboard
website
Su
pp
ort
docu
men
ts
Study outline
example
Feasibility
assessment sheet
Declaration of
interest template
EMA
PASS/PAES
protocol
template
Instruction
template Instruction
template
EMA report
template
Gu
idan
ce
ADVANCE best practice guidance (code of conduct, quality, communication, governance)
Implementation procedures (Table 2)
Figure 1
Readiness to act Study process
Pre-study Protocol and Statistical analysis plan (SAP) Data extraction Data
transformation
Analysis &
reporting
Identification of potential
databases in catalogue
Planning Study team Protocol Statistical analysis
plan
Send instructions to
DAPs
Write data
transformation
programs
Write final analysis
and pooling
programs
Collection of experience
from DAPs for events /
vaccines
Study question Identification
of partners
and roles
Write study
protocol
Write statistical
analysis plan and
dummy tables
Data extraction /
refresh
Test programs and
quality control
Quality control
program
Code mapping Feasibility
assessment
Collect CVs
and
declaration
of interest
Review of study
protocol
Create instructions
and CDM content
for DAPs
Fingerprint and
data quality checks
Run data
transformation
programs
Review of analyses
Extraction of data and
components
Study outline Agree on
study team
Update study
protocol
Approval SAP Approve study data
extraction
Transfer output to
RRE
Draft report
Quality control and sharing
of output
Identification of
potential
databases
Select
governance
model
Approval study
protocol
Review and
approve output
Review of report
Benchmark of data Approval of
study outline
Sign contract Register study
protocol EU PAS
Finalisation of
report
Archival of codes and
algorithms
Upload report to
EU PAS
DAPs: database access providers; CDM: common data model
Figure 2
DAP1 DAP2 DAP3 DAP4 DAPX
Common Input files (CDM)
Patient file PatID DoB
Gender Startdate Enddate
Vaccination file PatID Date
Vactype ATC
Brand DoseRecorded DoseDerived
Diagnosis file PatID Date
Eventtype Code
Remote Research Environment (RRE)
Partner 1 Partner 2 Partner 3 Partner X
R, SAS & Jerboa programs Aggregated data
Loca
l C
entr
al
Figure 3
