Healthcare Interoperability Testing and Conformance Harmonisation

Healthcare Interoperability Testing and

Conformance Harmonisation

WP6:

D6.4 Final Report including consolidated Roadmap and recommendations

Version 1.0 18 July 2011

D6.4 Final Report and Roadmap Page 2 of 36

DISCLAIMER Possible inaccuracies of information are under the responsibility of the project/study team. This report reflects solely the views of its authors. The European Commission is not liable for any use that may be made of the information contained therein.


EXECUTIVE SUMMARY This Final Report provides an overview of the outcome of the Healthcare Interoperability Testing and Conformance Harmonisation (HITCH) Project. This report includes the project’s recommendations and a proposed roadmap for achieving sustainable and effective deployment of the testing and certification/labelling process to enable health information exchange interoperability within and between European member states.

As stated in the Calliope Interoperability Roadmap: “Process improvements, acceleration of testing, and adoption and use of cross-sector standards are being pursued through close co-operation with national, European and international standardisation organisations. This in turn strengthens and binds together the European health IT industry and stimulates partnerships. The immediate benefits for the public sector and health authorities mean that they can begin to rely on open and sustainable IT solutions.”

HITCH is focused on one of the most critical elements in the realisation of eHealth interoperability: the acceleration of testing for interoperability, and supporting processes.

The Project focuses on how to test for conformance with profiles and standards that have already been selected. Its ambition is to help answer questions such as:

• How can an IT system be tested for conformance with agreed standards and profiles?

• Are the necessary and reliable interoperability test tools available to validate?

• Can there be a way to declare conformance, such as a label or a certification process?

• Will it be possible to test a specific product against others to validate their interoperability?

• What needs to be organised at the European and national/regional levels to test and certify interoperability, avoiding duplicate efforts and accommodating national specificities?

Addressing these questions is of critical interest to various groups: Vendors, Users, Patients and Authorities, each with specific expectations.

The aim of the HITCH project was to involve, both within the project and through reviews, those stakeholders who are at the heart of Interoperability issues in defining and agreeing on a roadmap to establish a way forward for Interoperability Conformance Testing in the field of Healthcare.

HITCH offers an analysis of the state of the art in quality systems governing interoperability, the maturity of test tools and various alternatives in certification and labelling processes. Based on this assessment, six recommendations were identified as critical to moving forward:

1. Drive adoption of interoperability standards and profiles at European level

This is a critical precondition to enabling the development of consistent test tools and certification/labelling. Profiles recognised at EU-level would reduce the fragmentation of IT-systems and the associated cost of development and maintenance. If recognised profiles


have the necessary flexibility, such as IHE profiles, they could allow for national/regional customisation.

Such recognised profiles are expected to form the technical and semantic dimension of an upcoming EU Interoperability Framework.

2. Drive adoption of the HITCH Interoperability Quality Management System Guideline.

Such a specific interoperability testing Guideline would define policies and procedures for managing the quality of profile testing based upon well-established standards (e.g. ISO 9000 and ISO 17025). It would be used for laboratory testing and inspection bodies in the field of interoperability testing. The HITCH recommendation is to encourage the use of its proposed Quality Guide by entities such as IHE in organising the epSOS Projectathon and its Connectathon. Based on feedback, this draft guide should then be fully developed into an international standard.

3. Drive closure of key gaps in interoperability testing tools

Testing tools are essential for many aspects of smooth testing processes, critical to accelerating implementation and deployment of interoperability. The use of tools increases productivity in test execution, reliability of the results obtained, traceability of test evolution and repeatability of tests.

The evaluation of the Open Source test tools currently available has demonstrated that key gaps need to be addressed:

• Terminology services;

• Devices workflow;

• Exchange and sharing of electronic records (eHealth; eInclusion);

• Identification, authentication and signature;

• Structured documents (laboratory, oncology and other specialities).

Improvement of test management tools and languages (e.g. TTCN-3 test plans, IHE Gazelle) to increase the level of test automation wherever feasible

4. Encourage collaboration between the functionality-focused and

interoperability-focused testing/labelling initiatives

Implementations may be tested for:

• Interoperability: this ensures that information exchange meets the standards-based profiles

• Functionality: this ensures that the features expected are provided to the user of the implementation

As one may choose to test from only one or both of these points of view, they need to be distinguished from but also related to each other as combined test plans may be easier to test in some cases.


One of the conclusions of the work conducted in HITCH by IHE (Interoperability testing) and EuroRec (Functionality testing) is that it is best to keep the test scripts separate for addressing functional criteria or interoperability criteria. However, a link between those criteria exists and need to be documented.

5. Preserve flexibility across the proposed Labelling and Certification schemes

Various approaches to certification and labelling processes have been analysed by HITCH. Three typical alternatives have been identified as most relevant to eHealth interoperability:

• Labelling/Certification of products by a Third Party: The testing process is performed by a conformity assessment body (accredited third party) or a (non-accredited) third party according to the specifications and requirements within the scope of the certification. This can be seen as an a priori control.

• Self-assessment of products with external Audit: The testing process is performed by the supplier (like self-assessment) according to the specifications and requirements (that represent the scope of the labelling/certification processes) and within an agreed interoperability quality assurance framework. The vendor is subject to an external audit performed by an inspection body (accredited or non-accredited third party). This can be seen as an ex-post control.

• Self-assessment of products: The testing process is performed by the vendor according to the testing specifications and requirements (that represent the scope) and within an agreed interoperability quality assurance framework.

Each of these alternatives has benefits and limitations. The choice depends on:

• The maturity of the market;

• The cost of product development and administrative processes;

• The scale of diffusion of interoperability and the emergence of new interoperability profiles;

• The legal framework relating to patient safety and security of exchanged or shared medical data.

In a less mature or emerging market, Third Party Labelling/Certification of products seems to be a reasonable first step approach to developing an initial range of interoperable products. As the market becomes more stable and mature, there should be a natural transition to self-assessment with external audit as it is less cumbersome and more dynamic. The broad acceptance of the specifications and their adoption by suppliers will be encouraged by this scheme.

HITCH recommends a flexible approach to the choice of Labelling/Certification scheme. It is advised not to include any choice of a specific certification/labelling scheme in a regulation or law to allow progressive transition towards lighter and equally effective schemes whenever possible. This is important to reduce costs and increase market adoption while accompanying the maturation of the market.


6. Establish a two-level Labelling and Certification process (European Level and National/project Level)

At the European Level

At European level, it is proposed to limit the focus of testing and certification to the EU-recognised profiles (see recommendation 1). This foundation will create a common level of consistency and quality in eHealth interoperability, a critical step towards developing a Europe-wide market for eHealth interoperability.

It is proposed to develop a testing ecosystem by building and evolving the European IHE testing process: first by implementing the HITCH QMS Guideline, enhancing and aligning the test tools on the EU-recognised profiles, enhancing the quality of the test tools to become a reference as they have been validated by the Connectathon itself and eventually moving from the current labelling approach to a certification scheme.

An outcome of this evolutionary approach is to recognise the test plans, test cases, test tools and test management tool (Gazelle) whilst openly offering them for the self-assessment of products, either directly by the projects or at country level testing (see below).

At the Project/Country level

eHealth projects may be led by public or private organisations:

• European projects such as epSOS;

• National projects such as DMP in France, or ELGA in Austria;

• Regional projects such as Veneto and Abruzzo in Italy, or the regional PACS deployment in Wales;

• Other projects promoted by a given user category (hospital, health professional network, etc.)

The proposed certification/labelling and testing strategy applies to projects that have decided to leverage the European-recognised profiles within some local extensions (additional specific profiles or extensions of the existing profiles).

As the European-recognised profiles have an established corpus of test plans, test cases and test tools, it will be to the benefit of the project/national level to reuse these valuable resources along with their own extensions. This will ensure that consistency between European and national levels is preserved and that additional project/national level investment and testing is focused only on the project/national extensions both for the project and the vendors. By using the same testing processes based on the same Quality Management System used at European level or by accredited bodies, shared experience and cross-recognition will be possible.

Proposed Roadmap

The HITCH project proposes a roadmap in order to orchestrate the implementation of the above recommendations. This roadmap spans four years and is organised into three phases:

1. Get ready 2012-2013 (could in fact start immediately) This first phase delivers the European-Level Test Process and Certification/Labelling Process. It is assumed that the first version of the eHealth


Interoperability Framework becomes available at the end of this phase, as it is a critical prerequisite for the next phase.

2. Operationalise 2013-2014 The second phase is to perform an evaluation of the draft European testing and labelling process and the first version of the test tools developed in the previous phase. Lessons learned from this evaluation are to be used to revise and formally approve the European testing and certification process and deliver a new version of the testing tools.

3. Deploy 2015 This third pilot phase offers testing and certification of eHealth products. The approved European testing and certification process is applied using the second version of the test tools delivered by the previous phase.

For each of these phases, several detailed milestones have been defined for the underlying testing and certification processes (Labelling/Certification process, European QMS) as well as for the underlying test tool development (test management platform, test cases and test tools) (see complete Roadmap on page 36).


DOCUMENT COVER AND CONTROL PAGE Project number: FP7-ICT-2009-4 Project name: HITCH. Healthcare Interoperability Testing and Conformance

Harmonisation Document id: D6.4 Document name: Tools Selection Dissemination level* PU: Public Version: 1.0 Date: 18.07.2011 Author(s):

Karima Bourquard Eric Poiseau

Contributors *Dissemination level: PU = Public, PP = Restricted to other programme participants, RE = Restricted to a group specified by the consortium, CO = Confidential, only for members of the consortium.

ABSTRACT

This task aims to consolidate the results of the project. Based on the inputs generated by the HITCH work packages, complemented with the feedback from the major stakeholders attending the IHE European Connectathon (vendors, users, patients and Member States and European authorities), the HITCH roadmap will propose an achievable foundation for Interoperability Conformance Testing with deployment starting in 2012. The roadmap will identify specific needs in terms of strategy, process improvements and tool development. Recommendations and identified deployment activities are presented in three phases from 2012 to 2015.

KEYWORDS Interoperability, requirements, conformance testing, roadmap, eHealth


TABLE OF CONTENTS

EXECUTIVE SUMMARY ......................................................................................................... 2!

1! ABOUT THIS DOCUMENT ............................................................................................. 11!

1.1! Acknowledgments ................................................................................................. 11!1.2! HITCH Team ........................................................................................................ 11!

2! INTRODUCTION ............................................................................................................. 11!

2.1! Overview .............................................................................................................. 11!2.2! Rationale .............................................................................................................. 12!

3! OVERVIEW OF STATE OF THE ART (WHERE WE ARE TODAY) .............................. 14!

3.1! Quality Management System (QMS) ....................................................................... 14!

3.2! Tools ................................................................................................................... 15!3.3! Certification .......................................................................................................... 18!

4! VISION FOR THE NEAR FUTURE ................................................................................. 21!

4.1! Quality Management System (QMS) ....................................................................... 21!4.2! Test Tools ............................................................................................................ 23!

4.3! Certification and Labelling Process ......................................................................... 25!

5! RECOMMENDATIONS ................................................................................................... 26!

5.1! Drive the Adoption of Profiles at European Level ..................................................... 26!5.2! Drive adoption of the HITCH QMS .......................................................................... 27!5.3! Drive Closure of key Gaps in Interoperability Testing Tools. ..................................... 27!

5.4! Encourage Collaboration between Functional- and Interoperability-focused Initiatives 28!5.5! Preserve Flexibility across LabelCert Schemes ......................................................... 29!5.6! Establish a Two-level LabelCert Process .................................................................. 30!

6! ROADMAP ...................................................................................................................... 31!

6.1! Get ready: 2012-2013 ........................................................................................... 32!


6.2! Operationalise: 2013-2014 ..................................................................................... 33!

6.3! Deploy: 2015 and beyond ...................................................................................... 34!


1 ABOUT THIS DOCUMENT

1.1 Acknowledgments We thank our colleagues from the European Commission’s ICT for Health unit for their kind support, in particular our project officer Benoit Abeloos for his continuous support throughout the realisation of this project. The HITCH team is grateful to the speakers and participants at the workshop in Pisa during the IHE European Connectathon 2011, in particular Lisa Carnahan from the NIST and Benoit Abeloos.

1.2 HITCH Team This project is realised by partners already at the heart of eHealth testing: IHE Europe and EuroRec from Belgium, INRIA and ETSI from France as well as MedCom from Denmark and OFFIS from Germany.

This document is the final outcome of work supported by the European Commission, Directorate Information Society, through the EU Seventh Framework Programme (FP7). The HITCH project was executed under Support Action contract number FP7-248288.

2 INTRODUCTION

2.1 Overview Today, delivering healthcare to patients includes a multitude of computer systems that collaborate with each other by exchanging patient data. This is necessary as many different devices and departments as well as local, regional and national institutions are engaged in the treatment of a patient.

At European level, the directive on patients' rights in cross-border healthcare “aims to establish rules for facilitating the access to safe and high quality cross-border healthcare in the Community and to ensure patients mobility in accordance with the principles established by the Court of Justice and to promote cooperation on healthcare between Member States”. To promote and test this coordination, the epSOS project is experimenting with exchanges of medical data between countries by specifying a technical framework based on integration profiles and standards.

The European Calliope project has clearly stated that “Process improvements, acceleration of testing, and adoption and use of cross-sector standards are being pursued through close co-operation with national, European and international standardisation organisations. This in turn strengthens and binds together the European health IT industry and stimulates partnerships. The immediate benefits for the public sector and health authorities mean that they can begin to rely on open and sustainable IT solutions.”

HITCH is focused on one of the most critical elements in the realisation of eHealth interoperability: the acceleration of testing for interoperability, and supporting processes.


2.2 Rationale

All collected information such as medical images (X-Rays, ultrasound, etc.), and medical reports need to be available to the patient's care providers in order to facilitate optimal treatment and at the same time avoid redundant examinations and adverse drug reactions. Furthermore, the current trend of having all patient-related data ubiquitously available (e.g. in the patient's holiday resort) requires a high level of interaction between the computer systems involved.

The underlying communication of images, reports and other information is highly complex and needs to be supported by international standards including terminologies to express semantics such as medical terms (for example, terminology for describing anatomical regions, diseases, physical units, medication, pharmaceutics and many more). The final goal would be to have all information shared such that a computer system may "understand" and evaluate it. Furthermore, the systems should be able to provide an understandable presentation to the user utilising the system, e.g. to a physician or the patient themselves.

This only works if all involved systems communicate in an internationally designed, familiar and standardised way. If two systems can talk to each other, then there is "interoperability" between those systems. According to the IEEE Standard Computer Dictionary, interoperability can be defined as the ability of two or more systems or components to exchange data and to use the information that has been exchanged. In addition, in the field of eHealth the definition can be extended to include the ability of two or more systems to exchange data:

• between linguistically and culturally disparate health professionals, patients and other actors and organisations;

• within and across health system jurisdictions in a collaborative manner.

Interoperability must first be addressed by the conformance harmonisation of eHealth systems. This is sometimes also referred to as profiling, i.e. selecting (subsets of) standards and options in order to uniquely identify the communication means to be used. As described, these standards are fairly complex, for example the DICOM standard describing the content and communication of medical images spans over 4,000 pages of technical documentation. So, besides describing how the communication should work, it has to be

ensured that the different software products actually do what is defined in the standards. In this context, two different testing methods are known. On the one hand, testing whether a system conforms to a set of standards is called conformance testing. On the other hand, interoperability testing "only" tests whether two systems are able to exchange information with each other1.

Interoperability Conformance testing is a

1 i.e. aims to test whether the communication goal is reached regardless of whether both are also conformant to the standards they implement

Interoperability “The ability of two or more systems or components to exchange information and to use the information that has been exchanged.” (IEEE 610)

Profile

A profile is a selection of definitions and options from existing standards.

Profiling is usually conducted in order to achieve interoperability between different products and implementations as a profile aims to harmonise all systems implementing it to use the same standards and contents.


topic of interest to various groups: Vendors, Users, Patients and Authorities all have their views on the subject. Vendors wish to avoid developing interfaces for information exchange specific to each site. Users expect products to interoperate smoothly with minimal integration effort, even when the products are provided by different vendors. For health professionals, conformance testing is critical to ensuring quality transfer of clinical information and enabling a safe and secured environment. Patients care about their safety, and would like to receive timely and effective care avoiding redundant examinations. Regional and national health authorities are concerned with the quality of care and its costs. Interoperability Conformance Testing reduces the deployment costs and risks of multi-system IT projects. It results in projects that have a more predictable deployment schedule and reduced disruption to existing care delivery activities. The aim of the HITCH (Healthcare Interoperability Testing and Conformance Harmonisation) project was to involve major stakeholders already at the heart of Interoperability issues in defining and agreeing on a roadmap to establish a foundation for the Interoperability Conformance Testing in the field of Healthcare. Three main topics have been identified in order to obtain a credible and applicable testing process:

• The eHealth Interoperability Testing Quality Management System (QMS) which describes policies and procedures for managing the quality of profile testing;

• eHealth Test Tool analysis which describes the current situation with regard to available Open Source test tools, the gaps and needs for the future;

• The strategy for labelling and certification of eHealth products in Europe and, by extension, the links between projects and national programme level on the one hand and the international level on the other.

In the following sections, the state of the art of each of the above topics is discussed in further detail, followed by the vision of the near future.

This report concludes with a set of recommendations and a proposed roadmap for achieving sustainable and effective deployment of the testing and certification/labelling process to enable health information exchange interoperability within and between European member states. This roadmap complements the other roadmaps proposed by other European projects such as Smart Personal Health, CALLIOPE, EHR-Q, etc. and more generally could contribute to the deployment of the European Cross Border Health Care directive.


3 OVERVIEW OF STATE OF THE ART (WHERE WE ARE TODAY)

3.1 Quality Management System (QMS) The main thrust of a QMS for Interoperability Testing is in defining the processes, which will result in the production of quality products and services, rather than in detecting defective products or services after they have been produced. Developing and implementing a QMS is not a new discipline and has been conducted in industry (e.g. construction and car production) for many years. The search for projects and experiences regarding QMS for Interoperability Testing did not detect much useful material. As a consequence, it was necessary to gather and analyse information (Figure 1) that could form a common understanding defining the QMS requirements for Interoperability Testing.

Figure 1: Methodology for defining QMS requirements

A number of interesting existing interoperability initiatives have been identified and evaluated based on the eight interoperability QMS principles defined in ISO 90002.

The list of reviewed initiatives was not exhaustive and served as input to the interoperability QMS requirements. The reviewed initiatives were CCHIT, Continua, ETSI, EuroRec, IHE, Infoway, KITH, MedCom and OpenECG.

2 http://www.iso.org/iso/iso_catalogue/management_standards/iso_9000_iso_14000/qmp.h

!"

!"#$%&'(#)#*+,+)&

-'.&+,

/-012333

#"

45#$"#&%6)167/)&+869+8#:%$%&'

%)%&%#&%5+.

$;

4<%.&%)*%)&+869+8#:%$%&'

#)=1>6)768,#)>+/)%&%#&%5+.

%"

?+7%)%&%6)1671&@+18+A"%8+,+)&.B

!(-1%)&+869+8#:%$%&'

&+.&%)*

&"

-&#C+@6$=+88+A"%8+,+)&.

'"

4D96$%>%+.18+$#&+=1

&6%)&+869+8#:%$%&'

A QMS can be defined as “a set of co-ordinated activities to direct and control the work in an organisation in order to continually improve the effectiveness and efficiency of its performance”2.


Figure 2: Outcome of the evaluation of interoperability initiatives

The evaluation led to the following conclusions for the existing interoperability initiatives:

• In general they have good knowledge regarding the eHealth domain and are involved in the process of supporting and improving interoperability, including the necessary stakeholders (principles 1-3 and 8) in that process.

• Quality is not monitored systematically and decision-making could be improved (principles 6-7).

• They lack formalised procedures and governance structures dealing with quality-related matters (principles 4-5).

3.2 Tools Reasons for interoperability problems are always related to a lack of standards, ambiguous standards or the incorrect interpretation of standards. To some extent, interoperability problems can be identified and resolved by testing systems before their deployment. Test tools are an important factor in conformance and interoperability testing and the HITCH project therefore reviewed over 30 tools and libraries of interest in the context of interoperability conformance testing of healthcare information systems. To this end, an evaluation approach was established based on the ISO PECA methodology. It proposed a detailed set of evaluation criteria ranging from name, version, author and programming language to quality management measures such as bug management and public source code repositories.

The review revealed a rich range of test tools and libraries, many of them freely available under a non-restrictive Open Source licence. The detailed evaluation of each tool provides a strong support to users such as vendors, authorities, certifiers, service technicians and others interested in eHealth interoperability testing in selecting the testing tools best suited to their specific requirements. For each tool under review, a radar plot-like visualisation was

Interoperability QMS principle 1 2 3 4

5 6 7 8

1

2

3

4

0

5

Score

Not existing

Initial

Repeatable

Defined

Managed

Optimized

Focus in HITCH Principle 4 & 5.

The eight interoperability QMS principles:

1. Customer focus 2. Leadership 3. Involvement of people 4. Process approach 5. System approach to management 6. Continual improvement 7. Factual approach to decision making 8. Mutually beneficial supplier relationships


created, giving a visual illustration of the tool’s usefulness and serving as a quality indication (Figure 3). Each individual evaluation was based as far as possible on objective measures. However, in order to provide helpful indications of a tool’s usefulness, some subjective influences cannot (and should not) be avoided, but are documented where applicable. The HITCH evaluation team members’ eHealth testing experience reflected in some ratings is even seen as a valuable addition to the tool facts that were collected from documentation and, in rare cases, source code.

Figure 3: Example of Radar Plot (NIST PIX/PDQ Pre-Connectathon Test Tool)

A common task for a vendor implementing a specific product or a certifier testing those products is to find the correct testing tool to check the interoperability of the implemented software. Besides the rich tool descriptions offered within deliverable D2.13, the evaluation provided an overview of the most common test areas such as HL7, DICOM and test management. For these domains, Domain Maps were introduced showing all the relevant tools of a specific domain at a glance. The tools within a domain map are displayed in a coordinate system offering rapid understanding of whether a tool is more generic or specialised and whether it works more on a high application level or on the technical protocol side.

3 HITCH Deliverable 2.1

!


Figure 4: Example of the domain map for tools in the field of test management.

As a high level overview, deliverable D2.1 also presented a tool landscape visualisation (Figure 4) which brought together all evaluated tools in a single figure that organises the world of eHealth testing tools into different areas (test management, privacy, physical transport, etc). The figure provides a quick overview of how many tools exist for each area.

Against this background, it is noted that the tool support for test management tools and tools for processes and methodologies as well as health information exchange infrastructure, seem to be thin if looking at the number and completeness of the tools available.

In general, the review revealed trends towards building large-scale eHealth infrastructure tools with projects such as Open Health Tools (OHT) or the Open eHealth Integration Platform.

As a result, from the different levels reviewed (detailed tool information, radar plot, domain maps and tool landscape), the following statements can also be derived:

• As the maturity of the tools increases, the quality management established for each tool increases.

• The more mature a standard or specification is, the higher the quality of tools for testing it.

!"#$%&#'()*+,-,#,(.

Functional/applicative

specialized generic

TestLink

Test Management

Gazelle

Selenium

SoapUI

TTCN3 based Test Management tools

Eurorec Use ToolsTM

Certification Based on ISO 9001:2000 (or ISO 9001:2008) and ISO 14001:2004, “certification” could be defined as an independent accredited external body issuing written assurance (the “certificate”) that it has audited and verified that the product or software conforms to the specified requirements.


3.3 Certification

In contrast to the US, where the certification approach is defined at federal regulation level, the situation in Europe differs and depends on the individual European countries. Figure 5 below has been designed based on the definition of the certification and labelling procedures identified in different European countries. The purpose of the figure is to describe the approach commonly used in certification. It provides an overview of the relations between relevant bodies (e.g. entity requesting certification, accredited entity and testing or auditing entities) and the tasks of each of these organisations within the certification process.

Audit An audit is an independent, objective assurance and consulting activity designed to add value and improve an organisation’s operations. It helps an organisation accomplish its objectives by bringing a systematic, disciplined approach to evaluating and improving the effectiveness of risk management, control, and governance processes. Based on the definition of The Institute of Internal Auditors on June 29th 1999.


Figure 5: Certification Process

Certification processes exist in many domains4 (security, automotive, banking and others) and in general the following is observed for these processes. The accreditation authority is a European/National entity that conforms to the ISO 17011 and/or guide 65 (for example, COFRAC in France). The accreditation authority has the competence to accredit inspection or laboratory bodies. This accreditation authority can be a member of a certification body network where the activities of such entities are coordinated. The network may be established at European or international level. The network gives the accreditation authorities recognition of their competency. For example, the multilateral agreement (MLA) between accreditors provides a means for goods and services to cross boundaries in Europe and throughout the world (example: IAF- “Certified once, accepted everywhere”).

To become a conformity assessment body, an organisation needs to conform to requirements based on technical competence, impartiality, availability of skills and equipment, professional privacy protection and civil liability insurance.

The certification is based on the reliability of and confidence in the competence of the body performing the certification. However, a laboratory or inspection body may not be certified but nonetheless provide a high level of confidence due to its dedication to meeting the necessary requirements in fields where no CABs exist.

To build a certification process in a new field takes time and is a long process that depends on:

• Definition of the specifications and requirements;

• Definition of the requirements for the laboratory and/or inspection bodies;

4 http://www.ifs-certification.com/

Conformity Assessment

Bodies (CABs)

Accreditation Authority

International (IAF), European (EA) or

national Accreditation forum/organisation

International Accreditation Forum European co-operation for accreditation Member State (coordination of the activities of the accreditation bodies)

Third party assess the competence of CAB who is conform to a define scope of activities

Notifies

Products including services Supplier

Conformity of products, services and suppliers to requirements and criteria

Testing Laboratory

Inspection Body Assess

Certification Process

Derived from ISO 17011:2004


• Setting up a certification process;

• Developing competence, skills and equipment;

• Etc.

In the eHealth interoperability sector, the first attempts at starting a process leading to recognition that products conform to specifications have been made by bodies such as IHE and Continua Alliance, as well as EuroRec for functional aspects. These bodies are not accredited and the labels they deliver are not fully recognised by the whole market for the following reasons:

• The specifications and requirements are recently defined and still need to be improved;

• The processes, tools and equipment needed to launch product certification are in a developmental stage.

Considering that the labelling and certification processes are very similar and that the difference is a question of the “authority” i.e. organisation that has legal and or regulatory authority to perform labelling or certification, we decided to introduce the term LabelCert process in this document.

The distinction between the LabelCert scenarios (using the LabelCert process) is based on the following principles:

• Bodies participating in the LabelCert process: CAB (for certification), testing laboratory body (e.g. IHE) or inspection body (e.g. EuroRec for functional aspects) and participants who present products and their own testing processes. Note that some CABs are able to support both activities (inspection and testing laboratory).

• The scope of the activities of the LabelCert scenarios focuses primarily on functional auditing and testing of both conformance and interoperability. In addition, in the case of self-assessment with external audit, it also includes auditing the testing process as described in the QMS.

Figure 6 shows the generic LabelCert process a supplier has to follow in order to obtain his seal.


Figure 6: Generic LabelCert processes

The requirements, specifications and test criteria are provided by the organisation that sets up a label/certificate for products and services. The supplier signs an agreement with this organisation or, by delegation, with the accredited authority (called CAB). Once the process starts, the supplier submits a log (test results and/or documentation) to the CAB. After validation of the input data, an inspection or lab testing is launched. Following validation, the label/certificate is given to the product or testing services and published.

4 VISION FOR THE NEAR FUTURE Based on the state of the art described above, this chapter describes the desired future of interoperability testing in Europe, which it is believed can be reached within the next five years. The vision is presented for the three areas addressed by HITCH: QMS, testing tools and the LabelCert process.

4.1 Quality Management System (QMS) HITCH presented5 how a Quality Management System for interoperability can be set up, based on principles taken from ISO 9001. The vision is for the current interoperability initiatives organised by parties such as IHE and EuroRec to achieve an initial implementation of the HITCH-defined Interoperability Testing QMS in the next years. Eventually, the QMS could be based on a new international standard dealing specifically with this topic. A group of experts from Europe and the US will extend the first version of the HITCH QMS to a Quality Guide for laboratory testing and inspection bodies in the field of interoperability

5 HITCH Deliverable 1.1 and Deliverable 2.2

Requirements definition

Signature of agreement with

the software company

V

Submission of the logs to the

NO/AC/TO

V

Audit and/or test V

Results and agreement validations

Software Certified

Publication of the certified software list

V


testing. The Quality Guide could then be further developed into an international standard by ISO.

In the same timeframe, IHE and EuroRec (at least) should instantiate the necessary QMS processes (see Figure 7) and actors as shown in Table 1.

Figure 7: Interoperability QMS Processes

All processes and actors are described in a Quality Manual which should be known by all stakeholders in the testing process and permeate all corresponding organisations (e.g. IHE, EuroRec and, in the medium term, the implementers’ organisations also) and their staff. There should be websites promoting interoperability testing QMS and organisations or companies offering training and workshops on that issue.

!"#$%&'()%

*"+),-.)%%

*"#,/)%$"#$#%

*"0"'.1%.2%3"'"4$%%!"#$%!..'#%

&2"1(2"%!"#$%3"##,.)%

5(',6($"%

78"49$"%!"#$%3"##,.)%

:"1.2$%!"#$%3"##,.)%

&2.+'"%31"4,+4(-.)%:"0,";%

:"<9,2"=")$#%>)('?#,#%

!"#$%34")(2,.#%@;.2AB.;C%

!"#$%D(#"#%

!"#$%D2,$"2,(%

!"#$%D(#"#%:"0,";%3,=9'($.2#E5(',6($.2#E342,1$#E*($(%

F")"2($.2#%*.49=")$(-.)%

:"/,#$2(-.)%.G%&(2-4,1()$#%

784H()/"%.G%4.)+/92(-.)%

&(2-4,1()$%;.2A',#$%

I(J%$"#-)/%#"##,.)%

!"#$%&2.4"692"#%

!"#$%34")(2,.ED(#"#%

!"#$%$..'#%

&'(K.2=%#"##,.)%

!"#$%I./#%L.),$.2,)/%

!"#$%39==(2?%:"1.2$%

!"#$%&'(')"*"($+%,-.'$"%%!"#$%

&'(')"*"($%

/0'12$3%41'((2()%

%


Table 1: Interoperability QMS Actors

Interviews and questionnaires evaluated6 at Connectathon 2011 in Pisa also indicated that the neutral persons validating the tests at Connectathon (the “monitors”) expect to benefit from such a Quality Management System.

Implementing organisations like IHE and EuroRec need to identify measurable quality indicators which will permit them to assess the quality of their QMS and improve relevant quality processes whenever necessary.

4.2 Test Tools Test tools are important for making large parts of a Quality Management System operational and feasible. Therefore, a tool ecosystem7 should be established in the future that fosters active tool development and maintenance. Against this background HITCH envisages an ecosystem with the characteristics described below.

First of all, an international agreement on (a set of) common standards, respectively profiles, should build a common ground for tool development, thus concentrating the efforts of the developer community. All standards developed (if necessary) are always accompanied by the development of tools needed to test their implementation in systems. This can be achieved by expressly, politically and monetarily fostering tool development within large-scale projects such as the epSOS8 project. These tools are used by the system developers as well as in testing events that take place on a regular basis. Most probably, the tools will be used in certification or labelling initiatives and programs (see section 0) and support functional as well as interoperability (including conformance) testing approaches. It is important that profiles and related tests are built on realistic use cases.

The test tool development process will implement the interoperability QMS, in particular for the activities of test plan definition, test design, test tool development and selection, test case, test session execution, test session reporting and all required validation activities.

6 HITCH Deliverable 3.3 7 HITCH Deliverable 2.2 8 http://www.epsos.eu/


Tools support and reference interoperability, and functional criteria are linked to the corresponding standards. For each test to be executed, test partners (in the case of interoperability testing) can be assigned manually or by the test management tool itself, along with the necessary tool chain (e.g. validators, sniffers or simulators for missing test partners). Based on the tests run, the test management tool will enable evaluation of a system’s test summary, for example, in order to find out whether it fulfils the requirements of the underlying certification program. All performed tests, partners, test results and tools used should be available within that test management tool.

• What is needed to address the interoperability challenge in the eHealth sector?

o International agreement on common standards or profiling of standards (cross-references/mappings between standards in the least);

o Close cooperation between political mandates, SDOs, testing/auditing organisations and implementers;

o Standard development systematically accompanied by test tools, already available or developed specially;

o Regular testing events such as the IHE Connectathon; o Quality labelling/certification programmes that include testing of eHealth systems; o Alignment of functional auditing and interoperability testing, based on realistic

and jointly agreed use cases; o An ecosystem that supports the continued existence of testing tools.

• Test Plan Definition o Creation and organisation of test criteria (functional statements and

interoperability criteria); o Links between criteria and the underlying standards; o Composition of test plans based on those criteria.

• Test Design o Test design tools that combine functional requirements and interoperability

criteria into a test case that can be assigned to a number of test partners and test tools (simulators, validators);

o Tools aware of the concepts to be tested (model-aware) in order to guide the test designer.

• Test Tool Development and Selection o Develop those test tools still missing today (gap analysis); o Improve existing tools where needed.

• Test Case and Test Tool Validation o Make sure test software works correctly and tests comprehensively; document in

validation report; o Maintain bug tracking tools for bugs in test cases and test tools.

• Test Session Execution o Provide training material for auditors and testers; o Integrate test tools into the test management tool.

• Test Session Reporting o Evaluation of the system under test: features tested, failed tests, list of non-

conformities, list of third party systems used for testing; o Evaluation of the test session: number of tests performed / verified / failed / not

verified; o Evaluation of the monitor work: was staffing sufficient? o Evaluation of the testing quality:

! Satisfaction questionnaire for participants and monitors; ! Report to the QA management in order to identify weaknesses and

improvements for the next cycle.


During and after testing, correct and sufficient tool functionality has to be checked and reported so that tools can be enhanced where needed. This also requires bug-tracking facilities. The use of free Open Source software in tool development is seen as the optimal approach, at least from the user’s perspective, concentrating developer’s efforts on common projects. Another major benefit is that such tools can be freely used by the testing organisations (such as companies and certifiers), as well as be checked by them in terms of security problems and functionality. However, it is understood that other business models may be applied such as dual licensing (e.g. free Open Source for not-for-profit organisations and cost-based licence for commercial applications) or offering commercial support for free Open Source tools.

It is clear that many test tools from the different areas closely interact with each other, for example, test tools such as sniffers and validators produce results that are then fed into test reporting tools which consolidate them into a unified summary. The test management tool in particular (as the “dashboard” for testing) should and must interact with many other tools. Therefore, tools should support a common description language for input and output data. The same applies to the tools’ order and execution details. This is where ETSI’s TTCN-3 family of standards9 and the corresponding TTCN-3 language is seen as one example of a possible solution in the future.

Finally, in any phase of the testing process, tools should support feedback regarding the auditors validating the tests, the tools used, the profiles tested, and so on. At the same time all persons participating in the testing processes must be offered training and documentation before testing.

4.3 Certification and Labelling Process The purpose is to define a labelling or certification strategy suitable to the eHealth domain. Three main scenarios already used in different domains were investigated and discussed and these represent alternatives:

• LabelCert of products (Third party LabelCert): The testing process is performed by a conformity assessment body (accredited third party) or a third party according to the specifications and requirements within the scope of the certification.

• Self-assessment with external audit: The testing process is performed by the supplier (like self-assessment), according to the specifications and requirements (that represent the scope of the LabelCert investigation) and within an agreed interoperability quality assurance framework. The vendor is subject to an external third party audit.

• Self-assessment of products: The testing process is performed by the vendor according to the testing specifications and requirements (that represent the scope) and under an agreed interoperability quality assurance framework.

The legal framework regarding responsibility for the LabelCert process needs to be discussed by the actors involved and is therefore not within the scope of this document.

The main conclusion is that the LabelCert process in eHealth depends on

• The maturity of the market: the more mature the market, the less a strong certification process is needed;

9 TTCN-3 2010: Standards ES 201 873-1 to ES 201 873-10, ES 202 781, ES 202 782 and ES 202 785, see http://www.ttcn-3.org/StandardSuite.htm


• A flexible process encourages innovation by guiding and promoting interoperability between systems (the suppliers will focus more on their research and development of their products than on duplicate efforts in the interoperability field);

• The diffusion of interoperability profiles on a large scale and within time constraints is generally better supported in the scenario of self-assessment with external audit.

All of these key principles need to be related to the legal framework in order to respond to the main objective of the eHealth LabelCert process, which is patient safety and the secured exchange of shared medical data.

5 RECOMMENDATIONS Based on the assessment of the state of the art and the vision, HITCH proposes the following recommendations for the interoperability labelling and certification of Healthcare information systems:

1. Drive adoption of Interoperability standards and profiles at European level 2. Drive adoption of the HITCH QMS 3. Drive closure of key gaps in interoperability testing tools 4. Encourage collaboration between the functionality-focused and interoperability-

focused testing/labelling initiatives 5. Preserve flexibility across the proposed LabelCert schemes 6. Establish a two-level LabelCert process (European and National/project levels)

These keys elements are described in the following subsections

5.1 Drive the Adoption of Profiles at European Level Today, many profiles are promoted at European level (epSOS) and by national projects. In France, ASIP-Santé references profiles from the IHE IT-Infrastructure, Laboratory and Patient Care Coordination domains within the “Cadre d’interopérabilité”. In Austria, IT-Infrastructure, Patient Care Coordination and Pharmacy are in use. In Italy, regional projects are also implementing IT-Infrastructure profiles. We recommend following the example of IHE in order to promote the adoption of profiles at European Level.

HITCH notes that recognised profiles allow for national/regional customisation and thus reduce fragmentation of IT-systems and the associated cost of development and maintenance.

The benefit of recognised profiles is that they encourage the development of test tools that help customers to improve their solutions. We think it is time to recognise these profiles to allow wider usage and a harmonised market.

HITCH therefore recommends supporting an end-to-end-process for profiles at European level: creating profiles (when necessary, e.g. when profiles are not available at international level), from use-case to testing – independent of which entity is actually performing such a process.

We recommend adopting the Process for the conception, approval and recognition of the profiles and their diffusion through testing events as well as through publishing lists of conformant implementers.


5.2 Drive adoption of the HITCH QMS The adoption of the HITCH-proposed interoperability QMS and guidelines to interoperability testing processes is a pre-requisite to certification and/or labelling of interoperability. At this stage, usage of the QMS is not required per se by the Labelling/Certification process, however it does help ensure the quality of the process.

The HITCH QMS is still not fully developed and therefore needs to improve and evolve with usage. HITCH’s proposal is to promote the adoption of the HITCH QMS by IHE, epSOS and MedCom. Feedback will result from the adoption of the QMS by those organisations, making it a reference for wider adoption. HITCH recommends creating and publishing a reference ISO International Standard or ISO Technical Report. This document will then be used for accrediting testing laboratories and inspection bodies for eHealth interoperability certification.

In order to increase the quality of the testing session already in place in Europe with the IHE Connectathon, we recommend:

• Developing the IHE Connectathon QMS, which is primed in HITCH and needs to evolve over the coming years;

• The need to strengthen documentation of all procedures;

• Increasing the number of indicators that could be used to measure progress;

• Improving training materials and the means of obtaining knowledge where it is needed;

• Analysing the requirements of ISO/IEC 17025, most likely leading to an update of the HITCH QMS;

• Strengthening IHE and EuroRec organisational processes such that testing can proceed in an efficient way.

5.3 Drive Closure of key Gaps in Interoperability Testing Tools.

The evolution of testing in the context of IHE Connectathon or EuroRec labelling has clearly demonstrated that good test management tools represent an essential component to achieving a high level of coordination between testing activities, eventually resulting in an overall high throughput of the testing event.

In the last few years, IHE Connectathon has been using Gazelle as the test management tool. It is very positive that Gazelle has reached such a level of maturity that future events can profit from it. The feedback from the 2011 IHE European Connectathon questionnaire positively highlights the above claim, as almost all participants expressed their satisfaction with the registration of devices and their implementation profiles, a task that was fully handled by Gazelle.

Testing tools are essential for many aspects of smooth testing processes. The use of tools increases productivity in test execution, reliability of the results obtained, traceability of test evolution and repeatability of tests and has other positive effects.

Testing tools are increasingly used to improve the interoperability of eHealth systems. In future, the range of tools and their functionality need to be extended. The most important extensions are listed below:


• Increase the coverage in terms of the number of test cases per profile and number of profiles that have appropriate test tools. The most important examples are as identified by HITCH:

o Terminology services; o Devices workflow; o Exchange and sharing of electronic records (eHealth; eInclusion); o Identification, authentication and signature; o Network services for images; o Structured documents (laboratory, oncology and other specialities).

• Increase the level of test automation wherever feasible;

• Improve Gazelle management tools by integrating profile-specific testing tools;

• Automate complex tests spanning several profiles;

• Improve the ergonomic aspects of tool usage;

• Improve tool feedback on problems detected during the test;

• Further examine usage of dedicated testing techniques and languages. This examination should include web-services and testing language TTCN-3, although other techniques may also be considered.

5.4 Encourage Collaboration between Functional- and Interoperability-focused Initiatives

Taken from the HITCH project, the following guidelines can be proposed regarding the ‘Requirements definition’ box in Figure 6: Generic LabelCert processes:

• The starting point is the definition of the test cases (what needs to be tested?) and the selection of the relevant criteria (addressing functional and interoperability requirements).

• Based on the chosen test cases, both types of criteria, i.e. functional and interoperability requirements, need to be investigated in detail in order to avoid overlapping aspects and address the test cases from both perspectives as far as possible.

• Once the final criteria/sources (e.g. selection of EuroRec functional descriptive statements and relevant aspects within IHE technical frameworks and profiles) have been selected (cf. overlapping and missing criteria mentioned in the bullet point above) and agreed upon, test scripts (i.e. actions users and/or systems have to perform) and test scenarios have to be developed. One of the conclusions of the work conducted is that it is better to have separate scripts for addressing functional and interoperability criteria. This enables functional and interoperability testing to be performed together or separately.

• Auditors/testers can use these scenarios and scripts within an auditing or testing process.


5.5 Preserve Flexibility across LabelCert Schemes

Figure 8: LabelCert Processes Diagram

HITCH recommends a flexible LabelCert process in order to accompany the maturation of the market.

Functional certification can be tested on a standalone product and does not generally depend on other partner products, and is therefore easier to perform.

As interoperability depends on partners, the LabelCert of products tested in a controlled environment cannot guarantee the interoperability of the products when they are deployed in the operational field (hospitals, clinics, healthcare providers, etc.). Defining the scope of the LabelCert process is a way to better control the success of the interoperability of the products once deployed in the field.

The LabelCert scheme may be supported by policy initiatives and even to some extent by European or individual country legislation. However, the details should be left for the standards and profiles and should not be dealt with in legislation itself as this would stifle innovation and evolution.

In an emerging market, Third Party LabelCert of products seems to be a reasonable approach to developing a critical range of interoperable products and encouraging the creation of an ecosystem around them. As the market becomes more stable and mature, self-assessment with external audit may be considered as an alternative scheme. The recognition of the specifications and their adoption by suppliers will be encouraged by this scheme.

The third party LabelCert process can be seen as an a priori (ex-ante) control and the self-assessment with external audit as an a posteriori (ex-post) control.

High Maturity

Low Maturity

•  Dynamic domain (fast developing)

•  Better with specification not yet widely adopted

•  Easy with informal third party quality labelling

•  Stable Environment •  Accelerates the recognition of

the specifications •  Depends on level of sanctions

when audit fails e.g lose right to use certification in commercial activity

3rd Party LabelCert of

products

Self-assessment with External Reference

Self-assessment of Products


5.6 Establish a Two-level LabelCert Process

Figure 9: The Two-level LabelCert Process

At European Level

Europe is not responsible for eHealth governance in each country, but the development of cross-border data exchange for citizens leads to the consideration of common interoperability policies. There is now a consensus among countries that eHealth interoperability in Europe shall be based on international interoperability standards and profiles.

Several European projects (Mandate 403, Calliope and epSOS) show that the recognition of a set of profiles based on prior needs is the first step before developing any type of certification. The selection of this set of profiles can be managed at eHealth governance level, whatever that may be (project, organisation, etc.) but this entity must be officially recognised by the countries.

Today, with the lessons learned from the IHE Connectathon, the LabelCert of products is the first step towards deploying interoperability in Europe. Connectathon is not a complete certification process because the entity that performs the tests is not an accredited body, although several organisations worldwide recognise the value of such a process. By improving the IHE process with the HITCH QMS, IHE will increasingly become a reference body and recognised laboratory testing body. The IHE Connectathon is a forum where participants, standard bodies and user associations can improve standards, profiles and solutions.

To become an accredited laboratory testing body, a candidate organisation must not only be compliant with the ISO 17025 standards, but also with the extension of the HITCH QMS once

European Level

Project/Country Level

Ehealth Governance body (TBD)

Recognises Catalog of Profiles Test tools and Test

cases as reference

Used to develop

Testing and evaluation

Used at

Selection of Profiles

Project

Extension specifications

Selection of tools and Test cases as

reference

SpecialiseTest tools and test cases

Testing and evaluation

Specific profiles

International Profiles Test tools

Catalog of Business cases

International Level International Standards

Interoperability Framework


validated by ISO. With time, the recommendation is to evolve to the LabelCert of products and testing processes (self-assessment with external audit) for the profiles that are stable and robust and largely deployed in projects in Europe.

The benefit of having recognised profiles is that they encourage the development of test tools which help customers improve their solutions. The test tools become the reference for the set of profiles because they are validated by the Connectathon itself, as are the products. Test tools and tested solutions are the outcomes of the Connectathon.

The recognition of the test plans and test cases, test tools and the test management tool (Gazelle) offer the possibility of their use for the self-assessment of products or directly by the projects. This is part of the process to stabilise profiles and develop their robustness in relation to their test environment.

At Project/Country level

The projects are led by public or private organisations:

• European projects such as epSOS; • National projects such as DMP in France, or Elga in Austria; • Regional projects such as Veneto and Abruzzo in Italy, or the Pacs deployment in

Wales; • Other projects promoted by a given user category (hospital, health professional

network, etc.).

The projects will use the European profiles but may need further specific profiles. In some cases, they need to specify extensions to the existing profiles. These extensions must be as minimal as possible. If new features are needed, the project is encouraged to present new proposals at international level and if accepted, these become new profiles that will be recognised at European level.

In cases where the recognised profiles are tested at European level, it is recommended that a certification process that includes the national extensions specifications be established at national level. However, SMEs which do not have the capability to participate in the LabelCert process at European level will benefit from the possibility of conducting testing for the European level by participating in testing events at national level. Based on the same corpus of test plans, test cases and test tools, consistency between European and national levels is preserved.

For the specific and additional extensions defined by the project, a dedicated set of test tools will be developed. The certification process must be based on the same Quality Management System used at European level or by accredited bodies.

To recap, Figure 9 summarises the link between the European and project levels.

6 ROADMAP The HITCH project proposes the following roadmap in order to establish a LabelCert process for the interoperability of eHealth systems in Europe. We recommend the following action plan for the next five years. The plan consists of three levels and three phases, as shown in Figure 10.


Figure 10: Presentation of the three phases and levels of the roadmap

The first level consists of establishing strategic milestones at European level in order to drive a European operational Testing and LabelCert process.

The second level is the organisational level, providing the quality management system and necessary processes.

The third level is the technical level and concerns the development of the necessary tools and test plans.

6.1 Get ready: 2012-2013 Figure 11 shows the first phase of the roadmap (“Get Ready”), which extends to the end of 2013.

Figure 11: Roadmap Phase 1. Getting ready for the operationalise phase

The objective of this phase is to ground the LabelCert process in Europe. This phase starts early in 2012 and will end with the publication of the eHealth Interoperability Framework. Based on a set of use cases, the interoperability framework will list profiles and standards responding to the functional requirements. Thus, we encourage the classification of profiles

2012-2013 2013-2014 2015 and beyond

Strategic level

Organisational level

Technical level

Phases

Levels


by priority linked to their usage. For example, profiles and standards related to the interoperability infrastructure need to be defined with high priority before the deployment of the profiles linked to a specific clinical use case. The framework can also provide some interoperability architecture based on a set of profiles responding to implementation deployment scenarios.

The first draft of the Interoperability Framework will be submitted to the eHealth Governance Initiative for comments and feedback.

The first set of profiles will be used for the evaluation and pilot in phase 2. Feedback from phase 2 will allow some adjustment of the framework.

Testing and Certification Processes

During that first period and in parallel with the work on establishing the eHealth Interoperability Framework, we strongly recommend driving the adoption of the HITCH QMS. MedCom in Denmark and IHE Europe are candidates interested in adopting the QMS. They should, as along with others, be encouraged and their experiences should then feed back into the publication of the first version of the QMS (V1.0).

This HITCH QMS will be refined and extended in order to for the document to be submitted to a standard body such as ISO as a guide for laboratory testing bodies. It will complete the corpus of requirements an accredited body needs to follow. A group of experts from Europe (those who participated in drawing up the first draft) and other parts of the world (experts from NIST, for example, who are involved in the certification process) has to be nominated.

The first versions of the Testing Process and the LabelCert Process can be defined in parallel. The first document based on the QMS describes to which organisation and processes it will be contributing in Europe. The second document will be more focused on policy, describing the process and roadmap for establishing a LabelCert process in Europe. In this document, scenarios are chosen according to European policy.

The drafts can also be submitted to the eHealth Governance initiative for comments and feedback.

Testing Tools

This phase shall be used to drive closure of key gaps in interoperability testing tools. This “Get Ready” phase should result in a test methodology and Test Management platform.

Several initiatives are working on test methodology (ETSI, IHE, NIST, etc). To increase the quality of the test plan and test cases to ensure good coverage of the tests and their validation, the methodology, using all the experience held by different organisations, has to be formalised and clarified and will help to support transparency in the testing process.

Once the methodology is clarified, the test management platform, such as the Gazelle management tool from IHE, will benefit from this progress and new features will be implemented.

6.2 Operationalise: 2013-2014 We recommend evaluating the testing and LabelCert of the profiles selected in the eHealth Interoperability Framework and then preparing a pilot test of the Labelling/Certification.

This involves the development of test cases and missing test tools for the profiles selected by the eHealth Interoperability Framework. The process of elaborating the test cases and


test tools developed in that phase shall conform to the guide (or the QMS) established in the “Get Ready” phase and be compatible with the methodology and “Test Management Platform”. The test cases and test tools shall be ready for the Evaluation milestone.

To evaluate all these outputs, a specific project has to be selected (for example, a project such as epSOS can be used as a platform for testing the tools).

In parallel, solutions and methodology will be found for the gaps between functionality-focused and interoperability-focused LabelCert initiatives already detected in the work to date. This will complete the first versions of testing and LabelCert processes produced in phase 1 and result in updates to the test management platform, test cases and test tools.

The LabelCert process already defined in phase 1 will be refined and adjusted during the evaluation period. This process will be analysed according to national/project schemes in European countries in order to avoid duplication or inconsistency between European and national/project levels.

The phase will end with the launch of a pilot testing and LabelCert process at European level.

Figure 12: Roadmap Phase 2. Operationalise phase

6.3 Deploy: 2015 and beyond The final phase of the roadmap concerns the effective deployment of the Testing platform and LabelCert process.

The main input for that phase comes from the outcome of the Pilot Testing and Labelling/Certification platform. The pilot shall be re-usable to support national/regional and

European Strategic Milestones

Testing Tools

Ehealth Interoperability Framework

Test Management Platform V1.0

Test Tools V1.0

European-Level Testing Process

European-Level LabelCert Process


Test Tools V2.0

Interop/functional Linkage

Proposed European-Level Label/Cert. Process

European-Level QMS V2.0

Pilot Testing & Label/ Cert.

Evaluation of Testing & Label/Cert.

Test Cases V1.0 Test Cases V2.0

Operationalise 2013-2014

Testing & Certification Processes


European projects, thus ensuring a coherent process across Europe and facilitating the recognition of the Labelling/Certification across nations for profiles that are similar.

The process can then be re-used and extended for a new set of selected profiles within the eHealth Interoperability Framework.

In conclusion, phase 1 can be managed directly by a one-shot project and could be a continuation of this work. From phase 2 and beyond, the question of organisation(s) managing the test management platform should be raised and discussed. The requirements of such organisations depend on the chosen LabelCert scenario.


Figure 13: The three phases of the roadmap

European Strategic Milestones

Testing Tools

Test Methodology

Ehealth Interoperability Framework


Test Tools V1.0

European-Level Testing Process

European-Level LabelCert Process


Test Tools V2.0

Interop/functional Linkage

Proposed European-Level Testing Process

Proposed European-Level LabelCert. Process



Pilot Testing & LabelCert.

Evaluation of Testing & LabelCert.

EuropeanTesting & LabelCert. Operational

Support of Project-Level Test &

LabelCert Process

Support of Project-Level Test Tools

Test Cases V1.0 Test Cases V2.0

Get ready 2012-2013 Operationalise 2013-2014 Deploy 2015-20XX

Testing & Certification Processes

Documents

Healthcare Interoperability Testing and Conformance Harmonisation