Embed Size (px)
Citation preview
1
Technical Guidance Series (TGS)
Risk management for
manufacturers of in
vitro diagnostic
medical devices
TGS–07
Draft for comment 25 September 2017
© World Health Organization 2017 All rights reserved. Publications of the World Health Organization can be obtained from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail: [email protected]). Requests for permission to reproduce or translate WHO publications – whether for sale or for non-commercial distribution – should be addressed to WHO Press, at the above address (fax: +41 22 791 4806; e-mail: [email protected]). The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use. Contact: Irena Prat, EMP Prequalification Team Diagnostics WHO – 20 Avenue Appia – 1211 Geneva 27 Switzerland
3
WHO Prequalification – Diagnostic Assessment: Technical Guidance Series
The World Health Organization (WHO) Prequalification Programme is coordinated through
the Department of Essential Medicines and Health Products. The aim of WHO
prequalification of in vitro diagnostic medical devices (IVDs) is to promote and facilitate
access to safe, appropriate and affordable IVDs of good quality in an equitable manner.
Focus is placed on IVDs for priority diseases and their suitability for use in resource-limited
settings. The WHO Prequalification Programme undertakes a comprehensive assessment of
individual IVDs through a standardized procedure aligned with international best regulatory
practice. In addition, the WHO Prequalification Programme undertakes post-qualification
activities for IVDs to ensure their ongoing compliance with prequalification requirements.
Products that are prequalified by WHO are eligible for procurement by United Nations
agencies. The products are then commonly purchased for use in low- and middle-income
countries.
IVDs prequalified by WHO are expected to be accurate, reliable and able to perform as
intended for the lifetime of the IVD under conditions likely to be experienced by a typical
user in resource-limited settings. The countries where WHO-prequalified IVDs are
procured often have minimal regulatory requirements. In addition, the use of IVDs in these
countries presents specific challenges. For instance, IVDs are often used by health care
workers who lack extensive training in laboratory techniques, in harsh environmental
conditions, without extensive pre- and post-test quality assurance (QA) capacity, and for
patients with a disease profile different from those encountered in high-income countries.
Therefore, the requirements of the WHO Prequalification Programme may be different
from the requirements of high-income countries, and/or of the regulatory authority in the
country of manufacture.
The Technical Guidance Series was developed following a consultation, held on 10–13
March 2015 in Geneva, Switzerland, which was attended by experts from national
regulatory authorities, national reference laboratories and WHO prequalification dossier
reviewers and inspectors. The guidance series is a result of the efforts of this and other
international working groups.
This guidance is intended for manufacturers interested in WHO prequalification of their
IVD. It applies in principle to all IVDs that are eligible for WHO prequalification for use in
WHO Member States. It should be read in conjunction with relevant international and
national standards and guidance.
The Technical Guidance Series guidance documents are freely available on the WHO
website.
WHO
Prequalification
– Diagnostic
Assessment
Procurement of
prequalified
IVDs
Prequalification
requirements
About the
Technical
Guidance
Series
Audience and
scope
4
Contents Contents .................................................................................................................................... 4
1 Abbreviations and definitions ............................................................................................ 6
1.1 Abbreviations .................................................................................................................... 6
1.2 Definitions ......................................................................................................................... 6
1.2.1 Definitions related to risk management .................................................................. 6
1.2.2 General definitions .................................................................................................. 7
2 Introduction .................................................................................................................... 10
2.1 Standards, guidance and WHO prequalification assessment ......................................... 10
2.2 Key concepts ................................................................................................................... 11
2.3 Cautions ........................................................................................................................... 12
2.3.1 Issues observed during WHO prequalification assessment .................................. 13
2.4 Risk management in a regulated environment............................................................... 13
3 Risk management process ............................................................................................... 14
3.1 Responsibilities ................................................................................................................ 15
3.1.1 Top management ................................................................................................... 15
3.1.2 Departments .......................................................................................................... 16
3.2 Policies and planning ....................................................................................................... 18
3.3 Training for risk management ......................................................................................... 19
3.4 Risk management file ...................................................................................................... 19
3.4.1 Demonstrating regulatory compliance .................................................................. 21
4 Tools and methods .......................................................................................................... 22
4.1 FMEA ............................................................................................................................... 23
4.1.1 The start-up work .................................................................................................. 24
4.1.2 The FMEA process ................................................................................................. 26
4.1.3 Risk evaluation ....................................................................................................... 29
4.1.4 Risk grid .................................................................................................................. 30
5 Risk control options ......................................................................................................... 32
6 Risk management – selected activities ............................................................................. 33
6.1 Quality management system .......................................................................................... 33
6.2 Risk management and design control ............................................................................. 34
5
6.2.1 Design risk assessment .......................................................................................... 37
6.3 Verification and validation .............................................................................................. 39
6.4 Analytical and clinical performance studies (design validation) ..................................... 40
6.5 Change controls............................................................................................................... 41
6.5.1 Equipment change ................................................................................................. 42
6.5.2 Change of supplier of a critical component of an IVD ........................................... 42
6.5.3 Change of intended use ......................................................................................... 42
6.6 The IFU and other labelling ............................................................................................. 43
6.6.1 IFU .......................................................................................................................... 43
6.6.2 Package labelling ................................................................................................... 47
6.7 Manufacturing ................................................................................................................. 48
6.7.1 Suppliers ................................................................................................................ 48
6.7.2 Manufacturing processes ...................................................................................... 52
6.7.3 Safe documentation .............................................................................................. 54
6.7.4 Process changes in manufacturing ........................................................................ 54
References ............................................................................................................................... 55
Acknowledgements
The document Risk management for manufacturers of in vitro diagnostic medical devices
was developed as part of the Bill & Melinda Gates Foundation Umbrella Grant and the
UNITAID grant for “Increased access to appropriate, quality-assured diagnostics, medical
devices and medicines for prevention, initiation and treatment of HIV/AIDS, TB and
malaria”. The first draft was prepared in collaboration with Dr Julian Duncan, London,
England, and with input and expertise from Ms Jeanette Twell. This document was
produced under the coordination and supervision of Kim Richards and Deus Mubangizi
WHO/HIS/EMP, Geneva, Switzerland.
The draft guidance was posted on the WHO website for public consultation on 25
September 2017.
6
1 Abbreviations and definitions
1.1 Abbreviations
CAPA corrective and preventive action
CLSI Clinical and Laboratory Standards Institute
FTA fault tree analysis
FMEA failure mode and effects analysis
FRACAS failure reporting and corrective action system
IFU Instructions for Use
GHTF Global Harmonization Task Force
IMDRF International Medical Device Regulators Forum
ISO International Organization for Standardization
IVD in vitro diagnostic or in vitro diagnostic device
KPI key performance indicators
QA quality assurance
QC quality control
QMS quality management system
R&D research and development
RPN risk prioritization number
RDT rapid diagnostic test
SQA supplier quality agreements
TMV test method validation
US CFR United States Code of Federal Regulations
1.2 Definitions
The definitions below related to risk management of in vitro diagnostic devices (IVDs) are
transcribed from ISO 14971:2007 Medical devices – application of risk management to
medical devices (1) and are generally used in this guidance. When a source other than ISO
14971 is used, the source is indicated.
1.2.1 Definitions related to risk management
7
Harm: Physical injury or damage to the health of people, or damage to property or the 1
environment 2
Hazard: Potential source of harm 3
Hazardous situation: Circumstance in which people, property, or the environment are exposed 4
to one or more hazard(s) 5
Residual risk: Risk remaining after risk control measures have been taken 6
Risk: Combination of the probability of occurrence of harm and the severity of that 7
harm. (Note: The definition of risk in (2) is broader and more generally applicable 8
than this, and is used by preference in this guide.) 9
Risk analysis: Systematic use of available information to identify hazards and to estimate the 10
risk 11
Risk assessment: Overall process comprising a risk analysis and a risk evaluation 12
Risk control: Process in which decisions are made and measures implemented by which risks 13
are reduced to, or maintained within, specified levels 14
Risk estimation: Process used to assign values to the probability of occurrence of harm and the 15
severity of that harm 16
Risk evaluation: Process of comparing the estimated risk against given risk criteria to determine 17
the acceptability of the risk 18
Risk management: Systematic application of management policies, procedures and practices to 19
the tasks of analysing, evaluating, controlling and monitoring risk 20
Risk management plan: For the particular IVD being considered, the manufacturer shall establish 21
and document a risk management plan in accordance with the risk management 22
process 23
Severity: Measure of the possible consequences of a hazard 24
Safety: Freedom from unacceptable risk 25
1.2.2 General definitions 26
The following definitions are used throughout this guide. 27
Design input: The physical and performance requirements of an IVD that are used as a basis for 28 IVD design. 29
Source: (3), definition (f). 30
8
Evidence: Information that can be proved true based on facts obtained through 31 observation, measurement, test or other means. 32
Source: Modified from (4), definition 3.8.1. 33
Instructions for Use (IFU): Information supplied by the manufacturer to enable the safe and 34 proper use of an IVD. 35
Note: Includes the directions supplied by the manufacturer for the use, 36 maintenance, troubleshooting and disposal of an IVD, as well as warnings and 37 precautions. 38
Source: (5), definition 3.30. 39
In the United States, the acronym IFU occasionally stands for “indications for 40 use”, and the acronym IU stands for “intended use” or “indications for use”. The 41 ISO definition and requirements (5) for IFU cover the intended use and the 42 precise method of use. 43
Intended use Use for which a product, process or service is intended according to the 44 specifications, instructions and information provided by the manufacturer. 45
Source: (1), definition 2.5. 46
Note 1: The clinical use for which the procedure was designed. 47
Note 2: The concept includes definition of the measurand, the target condition 48 and the clinical use of the measurement procedure, which may include 49 screening, diagnosis, prognosis, and/or monitoring of patients. (these notes are 50 from the Clinical and Laboratory Standards Institute (CLSI) website 51 http://htd.clsi.org.) 52
WHO note: The concept includes the physical, economic and resource limitations 53 in the environments of intended use. 54
In vitro diagnostic (IVD): A medical device, whether used alone or in combination, intended by 55 the manufacturer for the in vitro examination of specimens derived from the 56 human body solely or principally to provide information for diagnostic, 57 monitoring or compatibility purposes. 58
Note 1: IVDs include reagents, calibrators, control materials, specimen 59 receptacles, software, and related instruments or apparatus or other articles and 60 are used, for example, for the following test purposes: diagnosis, aid to 61 diagnosis, screening, monitoring, predisposition, prognosis, prediction, 62 determination of physiological status. 63 Note 2: In some jurisdictions, certain IVDs may be covered by other regulations. 64
Source: (5) and definition 3.27. 65
IVD reagent: Chemical, biological or immunological components, solutions or preparations 66 intended by the manufacturer to be used as an IVD. 67
Source: (5), definition 3.28. 68
This guide uses the terms IVD and IVD reagent interchangeably. 69
9
Life cycle: All phases in the life of a medical device, from the initial conception to final 70 decommissioning and disposal. 71
Source: (1), definition 2.7. 72
Measurand Quantity intended to be measured 73
NOTE 1 The specification of a measurand in laboratory medicine requires 74 knowledge of the kind of quantity (e.g., mass concentration), a description of the 75 matrix carrying the quantity (e.g., blood plasma), and the chemical entities 76 involved (e.g., the analyte). 77
NOTE 2 The measurand can be a biological activity 78
Source : (5), definition 3.39 79
Performance claim: Specification of a performance characteristic of an IVD as documented in the 80 information supplied by the manufacturer. 81
Note 1: This can be based upon prospective performance studies, available 82 performance data or studies published in the scientific literature. 83
Source: (5), definition 3.51. 84
“Information supplied by the manufacturer” includes but is not limited to: 85 statements in the IFU, in the dossier supplied to WHO and/or other regulatory 86 authorities, in advertising or on the Internet. 87
Referred to simply as “claim” or “claimed” in this document. 88
Process Set of interrelated or interacting activities which transforms inputs into outputs 89
Note 1: Inputs to a process are generally outputs of other processes. 90
Note 2: Processes in an organization are generally planned and carried out under 91 controlled conditions to add value. 92
Source: (4) definition 3.4.1. 93
Risk: Effect of uncertainty on objectives. 94
Note 1: An effect is a deviation from the expected — positive and/or negative. 95
Note 2: Objectives can have different aspects (such as financial, health and 96 safety, and environmental goals) and can apply at different levels (such as 97 strategic, organization-wide, project, product and process). 98
Note 3: Risk is often characterized by reference to potential events and 99 consequences, or a combination of these. 100
Note 4: Risk is often expressed in terms of a combination of the consequences of 101 an event (including changes in circumstances) and the associated likelihood of 102 occurrence. 103
Note 5: Uncertainty is the state, even partial, of deficiency of information related 104 to understanding or knowledge of an event, its consequence, or likelihood. 105
Source: (2) definition 1.1 and (4) definition 3.7.9 106
10
State of the art: What is currently and generally accepted as good practice. Various methods can 107 be used to determine” state of the art” for a particular medical device. 108
Examples are: 109
standards used for the same or similar devices, 110
best practices as used in other devices of the same or similar type, 111
results of accepted scientific research. 112
State of the art does not necessarily mean the most technologically advanced 113 solution. 114
Source: (1) paragraph D.4. 115
Top management: Person or group of people who direct(s) and control(s) a manufacturer at the 116 highest level 117
Source: (1) definition 2.26. 118
2 Introduction 119
2.1 Standards, guidance and WHO prequalification assessment 120
Risk management is essential to the competent manufacture of a safe in vitro diagnostic 121
medical device (IVD). Evidence of appropriate risk management within a quality 122
management system must be provided in dossiers supplied to the World Health 123
Organization (WHO) for prequalification assessment (6), including through formal dossier 124
review and information from the dossier request during manufacturing site inspection (7). 125
If no dossier is submitted for formal dossier review, as in the case of the abridged WHO 126
prequalification assessment, evidence of a risk management process will be reviewed 127
during inspection of the manufacturing site and at the stage one inspection, if applicable. 128
This guidance is intended as an aid for manufacturers of IVDs in compiling a product 129
dossier for submission to WHO and in preparation for the site inspection aspect of the 130
WHO prequalification assessment. The guidance does not cover instrumentation, 131
analysers nor software, except in a general way. It must be read in conjunction with the 132
internationally accepted regulations, requirements and guidance documents. 133
The principal international standard for risk management of IVDs is ISO 14971:2007 134
Medical devices – Application of risk management to medical devices (1), together with 135
the harmonized European version, EN ISO 14971. 136
11
ISO 31000 (8) and its supporting standard, ISO 31010:2009 Risk management – Risk 137
management techniques (9) and guide, ISO Guide 73:2009 Risk management – 138
Vocabulary (2) are generic standards providing the framework for managing (analysing, 139
evaluating, controlling and monitoring) all types of risk including, but not specifically for, 140
those relevant to IVD design, manufacture and use. They are powerful adjuncts to 141
ISO 14971 (1) when considering the complete life cycle of an IVD, including commercial, 142
financial and manufacturing aspects. In addition, they provide essential information on 143
the development and application of any aspect of risk management. 144
Other internationally recognized guidance on the techniques and tools for risk 145
management is available from the CLSI (10,25) and GHTF (12). The CLSI guide (11) is 146
concerned, like ISO 14971 (1), with safety at the point of use, whereas CLSI EP23-A (10) 147
covers many aspects of safety in clinical laboratories. The guidance from GHTF [12] is also 148
primarily concerned with safety in use and considers all stages of the product life cycle 149
from that viewpoint. 150
2.2 Key concepts 151
Four characteristics of successful risk management underlie the methods and practices: 152
Risk management is specific for a process (.e.g. development of a product, use of a 153
product, design of labelling, installation of equipment, managing a change of use 154
to a product, evaluation of a production procedure) and not generic. Although 155
lists of factors to consider with respect to use of an IVD are available (13, 14), 156
these factors need to be evaluated and appropriately extended depending on the 157
intended use, the measurand, the environment of use and the specific features of 158
the IVD involved. 159
Risk management is a whole life cycle activity and is not retrospective. It begins 160
with the conception of the IVD and ends only when the IVD is withdrawn from the 161
market. Lessons learnt can be applied to subsequent IVDs. This means that risk 162
management of an IVD is not a one-time process: it is iterative. The risk 163
management documentation must be revisited and revised, within change 164
control, as knowledge and circumstances alter, including as post-market 165
information becomes available. 166
12
Note: For an IVD that was developed and marketed prior to the enforcement of risk 167
management regulatory requirements, the development of a product specific risk 168
management file might have to be retrospective. Updates and revisions, however, 169
should be made in “real time”. 170
Risk management is an important activity helping to produce market-leading, 171
technically innovative products and in driving cost reduction. 172
Risk management is a company-wide activity and is not restricted to a specific 173
department. 174
2.3 Cautions 175
Different groups of similarly qualified people assessing the same system, process 176
or problem with the same risk management methods will produce different lists of 177
hazards and will assign different priorities (section 4.1.3) to those that they find in 178
common (15). 179
The different tools used in risk management (section 4) produce different lists and 180
priorities of hazards for the same subject (16). 181
The validity of the weightings (criticality and risk prioritization number, section 182
4.1.2) calculated in FMEA is disputable (17, 18): because “the concept of 183
multiplying ordinal scales to prioritize failures is mathematically flawed” (18). 184
Risk management can be time-consuming (and hence expensive) and without top 185
management support is unlikely to be useful, despite being a regulatory 186
requirement and one that is valuable in practice. 187
Risk management documentation must reflect the input from all departments 188
(including the quality department) involved in the IVD development, manufacture 189
and user environment of the organization. 190
Despite the cautions listed above, risk management, however practised, is a necessary, 191
important and useful tool in the production of safe and innovative IVD. It is important to 192
have the outcomes in mind, to think constructively, and not to become obsessed with the 193
assessment processes and marginal quantitation. With appropriate training within the 194
organization and skilledfacilitation of risk management, groups of suitably experienced 195
13
people can pool their knowledge. They can then reason in a constructive, structured 196
manner about ways to eliminate potential harm, solve current problems, produce novel 197
solutions and optimize manufacturing and financial factors. 198
2.3.1 Issues observed during WHO prequalification assessment 199
Some deficiencies, only a few examples of which are outlined in Box 1, will result in 200
noncompliance with the requirements of ISO 13485:2016 (13). Of greater concern is that 201
these deficiencies indicate the possibility of unsuspected problems with the IVD resulting 202
from poor risk management of the development and verification processes. 203
Box 1 Examples of issues identified during WHO prequalification assessment
FMEA are submitted in the form of a tick-list related to the Essential
Principles (13) with little analysis of the particular measurand, IVD format
or use of the IVD.
Risk analysis content is poorly written; sometimes it is not even possible
to discern the IVD format or measurand in question.
The risk management documentation has not been updated properly
during the product life cycle; for example risk assessment for IVDs that
have been marketed in high-resource settings has not been changed
before entering into commerce in more challenging environments.
Insufficient consideration has been given to the skills required of users
and the reproducibility of the results obtained with the IVD in their
hands, their physical environment and potentially interfering substances
that may be present.
2.4 Risk management in a regulated environment 204
Risk management has long been an expectation in IVD design, manufacture and 205
commercialization. ISO 13485:2016 requires “a risk based approach to the control of the 206
appropriate processes needed for the quality management system”, clause 4.1.2 b. 207
Compliance with ISO 13485:2016 and ISO 14971: 2007 satisfies the basic regulatory 208
requirement for risk management of most authorities. However, the standards might not 209
reflect the state of the art nor all of the scientific and business aspects of the IVD life 210
14
cycle. As noted previously, the IVD regulatory requirements are predominantly safety 211
related but the organization will also want to manage business, manufacturing and 212
environmental risks. 213
The primary responsibility for the preparation of the risk management plans for any IVD 214
development project or subproject in its early phases is generally contrlled by research 215
and development department. As the project progresses, the department of 216
manufacturing, and then customer services, normally take the lead roles. The QA 217
department is usually responsible for ensuring that risk management is documented in 218
the QMS of an IVD manufacturer. This includes preparing the policies (in association with, 219
and with the approval of, top management), procedures (in association and agreement 220
with multiple departments) and documenting allocated responsibilities. The QA 221
department is involved with all risk management activities throughout the organization 222
but not necessarily in a leading role. 223
3 Risk management process 224
The required risk management process from an IVD safety viewpoint is set out in 225
ISO 14971 (1), and as a flow diagram in Annex B of that document. This information is 226
summarized here in Figure 1. The general descriptions will be expanded with examples of 227
risk management at various stages of the life cycle in later sections of this guide. 228
15
Figure 1 Risk management process for IVDs
Risk analysis
intended use and identification of characteristics related to the safety of the IVD
identification of hazards
estimation of the risk(s) for each hazardous situation
Risk assessm
en
t
Risk m
anage
men
t
Risk evaluation
Risk control
risk control option analysis
implementation of risk control measure(s)
residual risk evaluation
risk–benefit analysis
risks arising from risk control measures
completeness of risk control
Evaluation of acceptability of residual risk
Risk management report
Production and post-production information
Risk management is a process that involves feedback as knowledge of the topic being 229
managed increases, as is shown in the process flow diagram. 230
The process summarized in Figure 1 is concerned primarily with safety of an IVD, but this 231
general process of risk management is applicable to any aspect, for example, financial, 232
commercial, regulatory or manufacturing. Reference can be made to the ISO 31000 series 233
(2, 8, 9) for substantial guidance on these processes. Further information is available in 234
the WHO Prequalification mock dossiers (20–22). 235
3.1 Responsibilities 236
3.1.1 Top management 237
1. Responsibility for establishing the criteria for acceptability of residual risk: this is a 238
key responsibility that cannot be delegated, although developing the criteria should 239
16
be a team effort. The documented criteria with their justifications and verification 240
must be recorded in the files associated with the IVD – the design control files or 241
the risk management files. The risk management report for an IVD must be 242
approved and signed by top management, in particular the statement of 243
acceptability of the overall residual risk. 244
2. Review of the suitability and progression of the risk management process at 245
planned intervals. This would normally be included in policies regarding routine 246
reviews of the QMS and design progression (19). Organizing the review is commonly 247
an activity prepared for top management under the control of the management 248
representative (19). 249
3. Appropriate resources and suitably qualified personnel for risk management must 250
be provided throughout the product life cycle. Sufficient numbers of competent 251
personnel must be trained in risk management and have sufficient time and the 252
physical resources needed to perform the tasks required of a risk management 253
team. Without top management’s total support, risk management may be poorly 254
performed, as the activities require significant resources (23). 255
3.1.2 Departments 256
It is important that all departments within the company are involved in all risk 257
management planning and subsequent activities, except certain aspects that only affect a 258
few departments. These exceptions would need to be documented in the overall quality 259
policies and the justification for excluding some departments would be noted in the risk 260
assessment. For example, a safety risk assessment of the chemicals for a manufacturing 261
process might not require the involvement of the marketing and financial departments. 262
Everyone involved in the development and implementation of the risk management 263
process must understand the objectives – i.e. they must have an understanding of the risk 264
management plan, the available risk management tools and the methods of assigning the 265
level of risk. High-level technical knowledge of each process being evaluated is not 266
necessary for every individual on the team; however, there must be at least one person 267
with such knowledge involved. 268
17
Specific training for risk management has proven invaluable in ensuring sufficiently 269
knowledgeable individuals who can conduct the meetings, collect and present the 270
information arising in a systematic way and write effective reports. They should also be 271
able to apply, in a consistent manner, the organization’s methods for assignment of 272
degree of probability, severity and detectability to the hazards that have been identified. 273
The extent of training needed will vary between those who prepare company policies and 274
plans (high-level), those who lead risk management meetings, and those who attend risk 275
management meetings to provide insight into specific processes. To meet the 276
requirements of ISO 13485:2016 (19) on risk management, formal training is essential 277
organization-wide and many training companies already exist to provide this, either in 278
person or via the Internet. 279
Evidence of successful training in the principles and application of risk management must 280
be readily available for assessment during an audit. This would include training records, 281
interviews with relevant staff members and observation of the effectiveness of 282
implementation of risk management within the organization. 283
Example: Job description requirements for a senior risk manager 284
The risk manager will need to be able to: 285
Develop and maintain a strategic risk management policy, framework, annual plan 286
and budget for risk management activities that will help achieve the objectives of 287
the organization and meet stakeholder expectations. 288
Manage communication about risk management activities throughout the 289
organization including timely reporting to top management. 290
Collate and analyse the results of risk assessments and contribute to managing the 291
actions required. 292
Ensure risk management activities meet current regulatory requirements, both in 293
the country of manufacture and countries of distribution, including audit readiness. 294
Manage risk management training requirements within the organization, attract 295
and retain suitably qualified personnel. 296
Maintain current knowledge of risk as applied to the IVDs of the organization or 297
similar IVDs both nationally and internationally. 298
18
Promote a proactive and performance-based risk aware culture within the 299
organization. 300
3.2 Policies and planning 301
ISO 14971 (1) requires documentation showing that risk management activities are 302
planned in detail. This standard lists activities that must be covered by a risk management 303
policy and plan and Annex F gives a comprehensive guide to planning. Detailed guidance 304
for preparing quality plans is available in ISO 10005:2005 (24), ISO 31000 (8) and 305
ISO 31010 (9). Together these documents cover all aspects of policies and plans for risk 306
management. CLSI QMS02 (11) provides guidance on the contents of - and the 307
relationship between - policies, plans and procedures and is helpful in the preparation of 308
clear documentation that allows good traceability. 309
There will usually be an overall plan for risk management of an IVD covering all phases of 310
its life cycle and more specific plans for the management of the risks for each phase. Risk 311
management policies must always contain definitions of the occasions in the life cycle of a 312
product when risk management activities will take place. At a minimum these will be: 313
when an IVD or one of its related processes is being designed, after the design 314
inputs have been obtained. 315
before and after design verification and validation studies and prior to launch of a 316
new or modified product. 317
before using an existing IVD in a different way, for example with new specimen 318
types, new environments of use (such as when an IVD that has been used in a 319
high-resource setting is to be used in a low resource setting, or one that has been 320
used in major clinical laboratories is to be used at primary level testing sites), or 321
new intended users (such as extending from professional use to self-testing). 322
before and after any change to the manufacturing processes, whether as an 323
improvement of any kind or in response to problems. 324
before and after any field safety corrective actions – whether following complaints 325
from users or for other reasons. 326
19
at regular, frequent, defined intervals during the commercial life of a product to 327
ensure that no information has been overlooked. 328
The risk management policy must include methods for assigning degrees of severity to 329
effects of potential failure modes and also to the categories of probability and 330
detectability (see 4.1.2 ). 331
3.3 Training for risk management 332
Risk management is central to the development and maintenance of a quality system and 333
at all stages of the life cycle of an IVD as outlined in ISO 13485 (19). Hence, a 334
comprehensive knowledge of the philosophy and tools of risk identification and analysis 335
must be available within an organization commercializing an IVD. 336
Responsibilities listed within a risk management policy would include preparing the risk 337
management plans, organizing the meetings, preparing the reports, updating the risk 338
analyses, preparing any change control documentation and ensuring timely completion of 339
tasks identified as a result of the analyses. Responsibilities would also include post-340
commercialization activities such as searching sources (for example, the Internet and 341
reviewed literature) for information that may affect the IVD risk management, integrating 342
feedback (for example from customers and the manufacturing process) and checking 343
continuing compliance with regulatory requirements. 344
3.4 Risk management file 345
The results of risk management activities must be collected in a risk management file, 346
which can be managed in various ways. For example, the information could be held with 347
the design control documentation, which might be an electronic system (possibly a 348
database or custom software). However, the file format must allow ready access to all 349
interconnected aspects of the risk management of each process, and to the relationship 350
of that process to other processes, within the overall risk management plan for the IVD. 351
The risk management file will provide traceability. It will include such information as the 352
risk analyses on the hazards identified and a record of the risk assessment and evaluation. 353
It will also include information on risk controls (including verification of adequacy) and 354
summarize the acceptability of residual risk in a risk management report. Top 355
20
management must sign the documentation of the acceptability of any overall residual 356
risk. 357
The file contents might not be held in a single place. However, references to and locations 358
of associated documentation must be included. A link to meeting minutes and to top 359
management reviews and approvals must also be included. This documentation must be 360
retrievable without delay for review by auditors. 361
Example: Table of contents of a risk management file. (The risk management file might 362
not physically contain all items listed in the table of contents, but must include links or 363
instructions on how to locate the information quickly.) Contents will be replicated for each 364
of the risk evaluations at key points in the life cycle of the product, e.g. for design input, 365
design verification, design validation, IFU validation, post-market surveillance. 366
Table of contents 367
Description of the IVD including intended use; safety data sheet 368
Description of risk management scope, timeline and tools to be used 369
Design and development risk management documentation 370
Risk management plan 371
Hazard identification, analysis and evaluation (biological, physical and 372
environmental) 373
Residual risk acceptance or risk mitigation (criteria and outcome) 374
Verification and validation of risk control measures 375
Production risk documentation: risk assessment of each production process 376
Post-production data analysis report (for example using data from manufacturing, 377
customer feedback etc.) 378
List of participants in risk management teams; minutes of meetings including 379
attendees, action items etc. 380
Risk management summary report including risk–benefit statement 381
Sign-off by stakeholders and top management 382
21
3.4.1 Demonstrating regulatory compliance 383
“The manufacturer shall establish, document, and maintain throughout the life cycle an 384
ongoing process for identifying hazards associated with a medical device, estimating and 385
evaluating associated risks, controlling these risks and monitoring the effectiveness of 386
controls and shall include: 387
– Risk analysis 388
– Risk evaluation 389
– Risk Control 390
– Production and post-production information”(1). 391
As previously mentioned, the basis for requirements can be found in ISO 13485 (19) and 392
ISO 14971 (1). These standards offer the broad outline of the principles and practices that 393
are required. In addition, the annexes provided, particularly in ISO 14971 (1), together 394
with many other resources such as those in the reference list, are very useful to suitably 395
trained and qualified personnel. 396
Within this context, the intent of the standards is used by WHO to assure compliance with 397
internationally recognized best practice in the manufacture of IVDs and to meet the 398
needs of WHO and its stakeholders. WHO’s technical guidance (including this guidance 399
document), together with published procedures and mock dossiers available on the WHO 400
website (http://www.who.int/diagnostics_laboratory/evaluations/en/) are intended to 401
assist manufacturers in their understanding of this approach. 402
Throughout this document there are examples of frequently encountered occurrences of 403
failure to comply with requirements set out in the two main standards used by WHO 404
ISO 14971 (1) and ISO 13485 (19). These examples come from dossiers submitted to, and 405
on-site inspections performedon behalf of, the WHO Prequalification Programme and are 406
given here to assist manufacturers in avoiding similar deficiencies and nonconformities. 407
Examples: The main examples of inadequate risk management resulting in 408
nonconformities include: 409
22
Resourcing (and approval of outcomes) of risk management activities by top 410
management is underestimated and hence insufficient to meet compliance 411
expectations. 412
The qualifications of personnel performing risk management activities are 413
inadequate. 414
Risk management activities are superficial in nature, generic, and do not consider 415
all of the layers of complexity required in risk management as outlined in the 416
standards. 417
Risk management is not applied to all aspects of the product life cycle as it must 418
be according to ISO 13485:2016 and according to best practice. It is rare to find 419
risk management applied to manufacturing and business processes as would be 420
expected to ensure product quality and continuity of supply. 421
The risk management documentation submitted with the dossier is incomplete. 422
Documentation, in terms of a risk management file, is scattered and not easily 423
accessible: it is not retrievable within a reasonable time frame (within one hour) at 424
on-site inspections. 425
Traceability of risk management activities throughout the whole life cycle of the 426
IVD is inadequate. 427
Review and updating of the risk management assessment at timely intervals is not 428
performed. 429
The “worst-case” environment of the end user is not considered. 430
Comment 1: As with all submissions to WHO, accuracy of information and data 431
(truthfulness) is considered to be an essential requirement. 432
Comment 2: Evidence of effectively implemented risk management across all aspects of 433
the life cycle of an IVD presented to WHO for prequalification builds significant trust in 434
the manufacturer’s ability to provide a high quality IVD. 435
4 Tools and methods 436
ISO 31010 (9) has a comprehensive list of risk assessment and problem-solving tools, their 437
applicability and their strengths and weaknesses. Both ISO 31010 (9) and CLSI EP18-A2 438
(25) provide excellent guidance. 439
23
The techniques most commonly used in the IVD industry are: 440
failure mode and effects analysis (FMEA) 441
fault tree analysis (FTA) 442
failure reporting and corrective action system (FRACAS) 443
When using these tools, it must be remembered that: 444
FMEA deals with single failure modes 445
FTA can lead to discovery of some effects caused by two or more failure modes 446
occurring simultaneously 447
FRACAS can be used to feed information into the other two tools about existing 448
failure modes with known causes and effects 449
This interdependency explains why the tools are most effective when used together. In 450
addition, usage, descriptions and techniques for the “seven basic tools” for problem 451
solving (including those of risk management) can be found in most manuals about quality 452
processes including that of the American Society for Quality (26). 453
4.1 FMEA 454
FMEA will be described in detail as it is the most commonly used basis for hazard 455
discovery and quantification during the processes of IVD design, manufacture and use. 456
However FMEA is not always the most appropriate basis for risk management. The 457
techniques described in ISO 31010 (9) are of more general applicability because they are 458
concerned with risk defined as “effect of un certainty on objectives” (2 and4), not simply 459
with potential harm as in ISO 14971 (1). For example, management of the risks in 460
preparation of the IFU is probably not efficiently based on an FMEA: the potential harms 461
arising from the IVD and its use should have been assessed during the various design and 462
development evaluations. Development of the IFU, especially in an established IVD 463
manufacturing organization, is more likely to relate to provision of correct and complete 464
information in a user accessible fashion and regulatory compliance than in identifying, 465
quantifying and minimizing the effect of hazards as defined in (1). Similarly some aspects 466
of performance evaluation and supplier management are best risk assessed using 467
techniques from (9) other than FMEA (but probably not those involved with defining 468
24
criteria for incoming goods inspections) because the related hazards should have been 469
evaluated during design and development of the IVD. Whatever the techniques of risk 470
management, their use and outcome must be recorded for compliance with (19). 471
4.1.1 The start-up work 472
The following sections are written predominantly from an ISO 14971˗compliant safety 473
stance (1). However, the principles and comments are applicable to risk assessment for 474
most processes in the IVD life cycle including R&D, manufacturing and post-market 475
surveillance. These sections also apply to most business-related activities when read with 476
reference to guidance in ISO 31010. 477
The first step in preparing an FMEA is to obtain a complete description of the process – 478
the scope – with as much detail as practicable. Often a flowchart of the process can be 479
prepared from the scope, giving a detailed overview of the activities in the process in the 480
sequence in which they take place, as this makes the subsequent analysis easier to 481
document. 482
A template for the FMEA should have been developed from the policy and planning for 483
risk management in the QMS of the organization. Conventionally this is prepared as an 484
electronic spreadsheet with a worksheet with headings as in Figure 2 (25). 485
Existing situation After action taken
Ite
m in
sco
pe:
ch
arac
teri
stic
Failu
re m
od
e
Effe
ct o
f fa
ilure
mo
de
Seve
rity
Po
ten
tial
cau
se o
f fa
ilure
mo
de
Co
ntr
ols
Occ
urr
en
ce
Det
ecta
bili
ty
Cri
tica
lity
Ris
k p
rio
rity
nu
mb
er
Act
ion
to
be
take
n
Act
ion
tak
en
Seve
rity
Occ
urr
en
ce
Cri
tica
lity
RP
N
Figure 2 Example of an FMEA template 486
There are examples of spreadsheets in the WHO HIV self-test sample dossier (20), and the 487
two nucleic acid testing sample dossiers (21, 22). The spreadsheet must have a start-up 488
25
worksheet with details of QMS factors such as version control, participants in the various 489
meetings, signatures and dates. The organization’s risk grid (see section 4.1.4) is also 490
usually added to the spreadsheet. 491
Comment: There are many other ways of capturing hazard data. However, the FMEA 492
format seems to be the most frequently used and is easy to read and understand. 493
It is generally the task of the risk management leader to assemble the scope 494
documentation, draw up the flowchart of the process and from that to prepare the 495
spreadsheet using the template. The risk management leader then completes as much as 496
possible of the start-up information and also the first column (“Item in scope: 497
characteristics”) of the main worksheet. 498
With this preparation completed, a team of people from across the organization who 499
have diverse knowledge of the topic should be assembled to develop the risk assessment. 500
The team should include representatives from various levels of seniority and all should 501
have basic training in risk management and FMEA processes. They should be well 502
informed about the agenda of the meeting, having reviewed the scope document and the 503
flowsheet and considered independently the specific risk management needed. 504
A useful aid to simplify and hasten progress is to project the worksheet onto a screen and 505
complete it electronically as the meeting progresses. After the meeting the leader can 506
add to the worksheet from his or her notes, agree about follow-up work with other staff 507
and if necessary use it to report to senior management. 508
Risk management is iterative and spreadsheets will be reworked several times during the 509
lifetime of a process, whether that process is commercialization of an IVD or risk 510
management of a factory process. Controls will be added, new failure modes discovered, 511
changes in probabilities and detectability calculated, new characteristics added (for 512
example submission of an IVD to a new regulatory authority or addition of a new 513
intended use). Use of a spreadsheet helps in managing this activity, as extra rows and 514
worksheets can be added, all within a document change control system. 515
26
4.1.2 The FMEA process 516
517
Figure 3 The FMEA process 518
The assessment group starts work by creating lists of ways that each item in the scope 519
could fail (or become noncompliant or nonconforming). These are the failure modes and 520
must be as comprehensive as possible. Hazards in normal use not caused by a failure 521
mode of the IVD (although perhaps caused by a failure mode of the design), such as 522
contamination of the user with specimen, must also be considered during the design and 523
use FMEA processes. 524
For each failure mode the group must next generate a list of all the effects of that type of 525
failure on the output of the process concerned. When compiling this list every aspect that 526
might be affected should be taken into consideration, for example, manufacturability, 527
safety of the user or the patient, continuity of supply, cost and so forth. These listings are 528
Prepare for the meeting Input documents
Flow sheets Worksheets
Circulate to attendees
Identify the effects of the failure mode
Identify failure modes For each item in the scope and in
logical order
Determine severity
Identify the potential causes of each failure mode
Use problem solving tools Possibly several potential causes for
each failure modeEnsure the real (root) cause is identified
Determine probability of occurrence of HARM from each
cause
? Determine probability of detection with current controls
Calculate criticality and risk priority number
Risk evaluation and control measures
Re-evaluate FMEA with new controlsNew hazards?
Severity cannot be changed
27
the effects of the failure modes. The following sections give guidance on this for each 529
major point in the life cycle of an IVD. 530
The next action, determining the severity of all the possible harms of each of the effects, 531
presents difficulties. It is common practice to give each harm a score between 1 and 5 or 532
between 1 and 10 (higher being more serious) but different groups will nearly always 533
assign different degrees of severity to the same harm for the same process, sometimes 534
markedly different (15, 17). One approach is to take the best opinions available from the 535
members of the assessment group and calculate the median of the values. Another 536
possibility is to obtain a consensus value through group discussion. (See reference (17) for 537
apparently more rigorous methods but which still face the same difficulty.) 538
Some failure modes might have more than one effect. In that case it is possible that only 539
the most serious effect needs to be considered further. However, in view of the difficulty 540
of assigning severity, this could present problems. This is particularly likely if the range of 541
severity values assigned to an effect is wide but the value finally chosen is low (relative to 542
other effects from the same failure mode). However, failure modes with effects on a 543
critical outcome of the process (for example safety, continuity of supply, user perceptions 544
or cost in a manufacturing environment) are usually given high scores. 545
All the potential causes of each failure mode must next be listed (generally using routine 546
problem solving tools, for example Ishikawa (fishbone) diagrams) and bearing in mind 547
that there might be more than one potential cause for each failure mode. The potential 548
causes are given an estimated probability of occurrence using the current state of the 549
process and its existing controls if any. As with severity, the probability of occurrence can 550
be debatable. It might be possible to obtain probabilities from similar processes, failures 551
and causes, but it is more likely that a consensus view will be necessary. A procedure for 552
estimating qualitative frequencies and probabilities from available data is presented in 553
CLSI QMS11 (27, Appendix D) and a thorough approach is described in (28). This 554
probability is known as P1 (see Annex E of ISO 14971), it is the probability of the 555
occurrence of the hazard. Risk, however, is defined in relationship to harm, and the 556
existence of a hazard or hazardous situation does not always lead to harm – usually a 557
second effect or event must occur to bring about the harm. The probability of this second 558
28
effect is known as P2. Thus the probability of harm being caused by the hazard is 559
composed of the combined probability of the hazard and the second event, P1*P2, which 560
is lower than the probabiity of either event alone. This overall probability of causing 561
harm is given the score (from 1–5 or 1–10, higher being more probable) to be used in the 562
occurrence column of the spreadsheet and in the variious risk calculations. As discussed 563
above assigning such probabilities and scores is subjective! 564
Examples: 565
1.) Consider an effect being a false-positive result, a harm of this is a misdiagnosed and 566
maltreated patient. A recombinant protein might occasionally have an impurity, 567
depending on the efficacy of its manufacture, that could be incorporated into an 568
assay. The probability of the presence of the impurity is P1, the probability of the 569
hazard. An individual tested in the assay might have antibodies that react with the 570
impurity (the probability of this second event is P2) giving rise to a false-positive 571
reaction, probabiility P1*P2, which is the probability of occurrence of the harm. In 572
the presnce of either the impurity or the individual with the antibody, but not both, 573
no harm will occur. This is frequently the case in HIV and HCV testing, when the 574
recombinant proteins purified from E. coli cultures are not subjected to stringent 575
evaluation prior to use and some individuals have strong anti-E. coli reactivity. 576
2.) Consider an effect being a technician becoming infected as a result of using the IVD. A 577
potential cause being exposure to patient body fluid because of a poorly designed 578
sample entry port. The hazard is that the user might touch the specimen, the 579
hazardous situation arises if this happens (probability P1), and with probability P2 the 580
patient has an infection (not necessarily that being tested) that affects the technician. 581
The probability of harm is then P1*P2 – coincidence of both hazard and second event. 582
3.) Two events are not always directly involved. Consider an IVD which has been subject 583
to maltreatment during transport so that it no longer detects reactive specimens (the 584
probablity is P1, the hazard). If the IVD has a control line which becomes visible when 585
an assay is performed but which only monitors flow, not function, and the presence 586
of the line is said in the IFU to validate the assay: harm (wrong diagnosis and its 587
consequences) will be caused with probability P1. 588
29
In IVD manufacturing organizations, detectability (probability of detection) of the failure 589
mode is occasionally taken into consideration during an FMEA. As with severity and 590
probability, opinions on detectability can differ; indeed whether detectability should be 591
included in risk management at all is subject to debate (29). If the failure can be detected 592
(for example by the user, or by the operator during manufacturing) the effect should not 593
occur, but the process of detection itself should form part of the risk evaluation of the 594
control mechanism. For example, could a visually impaired user notice that a control line 595
was unusually weak? The probability of detection is given a score of 1–5 or 1–10. A score 596
of 1 is assigned if the failure will always be detected (100%) and a score of 5 or 10 when 597
there is no possibility of detection. Whichever methods of assigning severity, probability 598
and detectability are used, it is important to validate them as thoroughly as possible and 599
to re-evaluate the FMEA as knowledge increases. 600
Finally, the risk priority number (RPN: severity × occurrence × detectability) and the 601
criticality (severity × occurrence) might be calculated for each effect of the failure modes 602
(see Box 2). 603
Box 2 Explanatory notes on probability × severity × detectability
Probability (occurrence) is related to likelihood, occurrence or
frequency of the HARM arising. A probability estimate can be
quantitative (using data and statistics) or qualitative (based on
experience and considered opinion). Each hazard and hazardous
situation will give rise to a risk estimate.
Severity measures the possible consequence of a hazard.
Detectability (probability of detection) of the hazard or hazardous
situation before it leads to harm and so reduces the likelihood of
harm and reduces the estimated risk.
4.1.3 Risk evaluation 604
Once the FMEA has been completed, the risks can be placed in order of criticality or 605
priority (possibly with aid of Pareto diagrams) and also measured against the company 606
risk grid. The risks that are unacceptable can be dealt with by introducing appropriate 607
controls (see section 5 and later in this section) and subsequently reviewed by further 608
30
FMEA for newly introduced hazards. Decisions can then be made about how to proceed 609
with risks that are unacceptable. Every measurement or action by QC or QA should be 610
the outcome of, and traceable to and a risk evaluation. 611
For IVDs that are to be CE-marked under the EU directive on In vitro diagnostic medical 612
devices (30), risk must be reduced “as far as possible” through safe design and 613
construction. This is usually interpreted as meaning that the cost of ameliorating a risk 614
cannot be used as a factor in deciding acceptability. While CE marking is not a 615
prerequisite for acceptance to the WHO Prequalification Programme, if an IVD is 616
CE-marked, the dossier would be expected to prove compliance with this aspect. 617
ISO 14791 (1) is more lenient in that risk must “as low as reasonably practicable”. 618
4.1.4 Risk grid 619
The most common form of risk evaluation is against a risk grid similar to that shown in 620
Table 1, which uses a grading scheme with scores of 1–5. The risk grid is usually 621
incorporated as a worksheet on the FMEA spreadsheet together with a table of actions 622
similar to those shown in Table 2, which must be taken subsequent to the FMEA. The 623
detailed action is added to the “action to be taken column” either after discussion at the 624
FMEA meeting or by the risk management leader in consultation with technical staff. 625
Table 1 Risk Grid Severity of effect
5 Critical
4 Major
3 Moderate
2 Minor
1 Minimal
Pro
bab
ility
of
occ
urr
ence
5 Frequent High High High High Medium
4 Probable High High High Medium Low
3 Occasional High High Medium Medium Low
2 Rare High Medium Medium Low Low
1 Improbable Medium Low Low Low Low
31
Table 2 Actions to be taken following the FMEA
Table of actions dependent on zone in risk grid
Outcome zone Risk assessed Action to be taken
High Harm A Process must be redesigned, or, if that is not possible, a mode of control must be added. This must be agreed by top management.
Medium Harm B Process to be redesigned or a control added. This may need agreement by top management.
Low Harm C Control to be added.
How the grades for severity and probability are assigned and what action should be taken 626
must be defined in the QMS policies and might vary from process to process. There is no 627
regulatory standard and very little guidance on quantifying any of the factors, nor for 628
determining the acceptability of overall risk for an IVD. 629
For safety aspects, ISO 14971:2007 (1) suggests that severity could be classified 630
qualitatively using three grades as shown in Table 3 (1). 631
Table 3 Qualitative classification of severity using three grades 632
Term Description of the harm
Significant Death or loss of function or structure Moderate Reversible or minor injury Negligible Will not cause injury or will injure slightly
But more usually severity is classified in at least five grades as shown in Table 4. 633
Table 4 Qualitative classification of severity using five grades 634
Severity Description of the harm
5 Critical Life-threatening, loss of limb, threat to community 4 Major Severe lasting effects, requiring medical action 3 Moderate Short-term effects, requiring medical action 2 Minor No lasting effects 1 Minimal Slight or no effects to users or patients
For an IVD these harms might be caused by: 635
Misdiagnosis. 636
physical, chemical or microbiological failure mode of the device itself. 637
a hazard present in the use of the IVD with no failure mode. 638
32
use of the device in an unintended fashion (“off-label use”). 639
Annex D of ISO 14971 (1) provides guidance on estimation of risk. 640
No regulatory standards have been established for probability and detectability of risk for 641
IVD. The usually accepted levels for probability are listed in Table 5. 642
Table 5 Commonly used probability levels for risk assessment of IVDs 643
Level Description
5 Frequent 1:100 4 Probable 1:1 000 3 Occasional 1:10 000 2 Rare 1:100 000 1 Improbable 1:1 000 000
Whatever characteristics are selected for severity, occurrence of harm and detectability 644
of the hazardous situation, the QMS must include a policy on how to choose and justify 645
them, together with the justification of acceptability of overall residual risk (ISO 13485 646
(13)). Given the variability in outcome of assessing risk quantitatively (see the comments 647
above and references in section 2.3) the policy must encourage the exercise of caution 648
and be carefully and comprehensively written. Use of a qualitative risk grid and action 649
table as described above avoids the need to calculate RPN or criticality scores and this in 650
turn avoids much debate about calculating “exact” values and their interpretation. 651
5 Risk control options 652
Once risks have been quantified using the tools described in previous sections they must 653
be controlled. ISO 14971 (1) prescribes the options for risk control and the order in which 654
they must be applied: 655
Safety by design comes first and foremost and is usually interpreted as meaning 656
“inherently safe design and construction” (30). For this reason, risk management 657
must start from the inception of an IVD (or any process) so that the design can be 658
planned safely and controls do not need to be built in later on to overcome design 659
flaws that should have been avoided. Experience shows that risk management at 660
the design input stage can also lead to much greater user satisfaction, as ideas for 661
novel and functional features often arise during the risk management meetings. 662
33
Protective measures in the IVD itself or in the manufacturing process are features 663
that prevent a failure mode in the first place. They are not features such as run 664
controls (31) that warn users that a failure has already occurred. 665
Example: A shield over the specimen addition port of a rapid diagnostic test to prevent 666
contact with potentially infectious material. 667
Information for safety. Well-written and validated IFU provide control of risk, but 668
this is the least effective method of control. It is not the same as a warning, which 669
alerts users to the existence of risk but neither prevents nor ameliorates it. 670
Run controls might be viewed as providing information for safety but they are 671
weak controls in that, if activated, indicate the failure of that test run. 672
Control measuress must be evaluated to ensure they do not present new hazards. They 673
must be verified and validated in the user environments. 674
Example: Many run controls for rapid diagnostic tests indicate successful flow of 675
reagents only and do not confirm that the test would have detected a positive 676
specimen. Such a run control might not indicate thermal inactivation of the IVD, and 677
so give a false sense of security to a user. Limitations must be clearly stated. 678
6 Risk management – selected activities 679
6.1 Quality management system 680
As noted in section 2.4 ISO 13486:2016, unlike earlier versions of the standard, 681
specifically states that the QMS must be developed using risk management principles. 682
Quality management and risk management of all processes of the organization must be 683
linked and have feedback interchange mechanisms between them. All steps in the risk 684
management process must be performed in accordance with the organization’s quality 685
manual. For example, document control will apply to risk management activites and 686
include document identification, version control, traceability, review intervals, 687
identification of responsible personnel, links to competence and training records of 688
personnel, records of meetings and links to management review, among others. The risk 689
management documentation – the risk management file – is managed within the 690
34
manufacturer’s QMS and is an essential component of the risk management 691
documentation linking and feeding back to the quality manual 692
Input leading to changes in the quality manual and the QMS from risk management of 693
activites within the system might include for example: 694
potential lack of staff competence determined during particular risk 695
management activities and any training needs documented. 696
Overall risk to the organization from customer feedback mechanisms such as the 697
CAPA system; problems found when using particular transportation suppliers or 698
modes; lack of clarity in IFU and subsequent unintended uses; reviews of 699
literature related to the manufacturer’s products, analytes, assay methods or 700
failure to notify regulators and WHO about critical change to a product. 701
Information from a manufacturing risk assessmentto assess the need for a 702
general change in documentation (e.g. styles, use of language, font size in 703
policies, procedures) within the QMS that might be needed to minimize risk 704
from lack of readability by non-technical personnel, sometimes through the use 705
of photographs, diagrams or translation into the language in use on the 706
manufacturing floor. 707
Overall risk to the organization caused by poor suppliers found from particular 708
failings in the supplier audit methods noticed during supply-risk analyses 709
6.2 Risk management and design control 710
Risk management is an integral part of design control. The two processes have the same 711
goal from a manufacturer’s viewpoint: to produce a safe, efficacious, 712
regulatory-compliant product with wide customer appeal, good profitability and 713
continuity of supply. 714
A typical design and development flowchart with integral risk management is shown in 715
Figure 4. This is a suggested process flow only but is typical of current practices. A risk 716
management process is associated with each critical stage of the life cycle of the product, 717
within the overall plan (see section 3.2). The risk management performed as the product 718
is withdrawn from commerce (mainly user satisfaction and business oriented but also 719
35
summarizing the life cycle of the IVD) is not shown in Figure 4, but the design information 720
obtained from this activity should be used in the development of any future IVD. 721
36
Figure 4 Design control and risk management
Design inputs User requirements Regulatory requirements Manufacturing requirements and capabilities Management expectations
Product specifications Numeric design requirements for R&D Product will be verified against this R&D phase 1 (under design change control) Format and instrumentation developed Processes developed and qualified. Manufacturing documentation started Guard bands for all process parameters defined and validated QA and QC parameters and materials defined, sourced and documented Calibrators and internal controls developed and metrologically traceable Beginning of stability work for in-process intermediates and final device IFU initiated
User, patient, manufacturing risk analyses started, re-evaluated regularly
R&D phase 2 Transitioning to factory Change control begins Instruments and material suppliers finalised and audited Instrumental PQ, OQ in the factory Pilot batches made, tested against putative QC Interfering substances evaluated, efficacy of microbiocides proven Process documentation finalised and approved All aspects of device and specimen stability using devices made to approved specifications IFU finalised and approved QA and QC specifications finalised and approved
"Customer" requirements document Design will be validatedi against this Design change control begins when this document is approved
R&D phase 3 Design verification using material made in the factory to approved documentation (R&D evaluation of all aspects of product specification document e.g.performance, repeatability, reproducibility , lot to lot variability, all aspects of stability)
Design and development plan writtenincluding risk managment plan
R&D phase 4 Design validation using material made in the factory at scale to approved documentation (User evaluation of all aspects of product specification document and customer requirements document) e.g.performance, repeatability, reproducibility , lot to lot variability, functionality of IFU, training manuals, software) Normally done in three user-labs with three independent lots of reagent
Manufacture CE marking declarations and inspections Full scale manufacture under change control On-market surveillance Review of scientific literature for independent clinical evaluations
Continuous on-market monitoring and risk re-assessment
Design risk analysis
Pre-commercial launch risk analyses Production of: Declaration of acceptable residual overall risk
Pre-verification and validation risk analyses
Final IFU risk analysis
37
6.2.1 Design risk assessment 722
This is the first stage of risk management for the whole life cycle of the IVD. It begins as 723
soon as the concept of the product is finalized, after the customer inputs and 724
requirements have been obtained. “Customer” in this context is any entity that the 725
potential product will be affected by or will affect. Customers will be both outside the 726
organization, for example patients, users, distributors, purchasers and regulators and 727
within the organization, for example finance department, patents managers, QA, 728
manufacturing, R&D, sales, marketing and customer support. Design input from all of 729
these sources will define the intended use and lead, in turn, to defining the design 730
requirements of the proposed IVD. 731
Once the design requirements are available, the basic features of the proposed IVD will 732
become apparent and the design risk management planning can begin. Normally the 733
outline of the plan will be described in the QMS and policies, but the details will need to 734
be defined. These include determining responsibilities overall and identifying the 735
participants in the initial risk assessment meetings. Design risk management planning will 736
consider everything that is known about the potential product, in line with and taking into 737
account the Essential Principles (13, 14). This will be focused specifically on the proposed 738
IVD and its uses and is not a generic assessment. 739
The initial design risk management meeting should address as a minimum the factors 740
listed in Table 6. 741
Table 6 Factors related to the specific IVD to be considered in initial design risk 742
management 743
Factor Example characteristics to consider
Specimen Method of obtaining specimen from patient, type, storage, transport, likely interfering substances and cross-reacting materials in that specimen type
Patient Environment, age, sex, likely concurrent and similar illnesses, potential pharmaceutical treatments, vaccinations, potential drugs and other social factors
User Environment (especially temperature, humidity, altitude, microbial flora), training, skill level, social factors (intolerance of some constituents)
38
Factor Example characteristics to consider
IVD
Known failure modes with a similar IVD using the same format (for example prozone effect, lack of or poor specimen/reagent flow, reversal of specimen/reagent flow, insufficient specimen addition, cross-contamination, failure to link result to patient, contamination with enzymes)
Measurand
Known failure modes with all formats of IVD for the measurand (for example cross-reaction with other substances, vaccination status of patient, drug treatment interference, confusion with disease that has similar symptoms, specific problems in certain populations or age groups)
Many of these factors will interact with other factors. For example, the patient’s 744
environment is likely to affect the possible concurrent illnesses and possible treatments 745
unrelated to, but potentially affecting, the proposed IVD. This assessment requires 746
considerable knowledge of the disease, technicalities of the measurand, the device 747
formats available and state of the art for the proposal. The initial design risk management 748
meetings must also take into account the internal requirements of the organization such 749
as business and commercial factors, manufacturing factors including training and 750
equipment and regulatory issues. 751
The control factors coming from the design risk assessment should lead to features of the 752
design of the IVD, to requirements for many of its performance features, to supplier 753
considerations and to in-built controls. Many risks should be removed by appropriate 754
design of the format of the device, choice of reagents and method of addition of 755
reagents. This process might lead to at least some features that will give the product 756
commercial and possibly societal advantages and provide more than a “me too” device 757
that would be perceived by potential purchasers as competing simply on cost. 758
As the design is developed, new ideas will be generated, new problems will arise and the 759
risk assessment and controls will need to be re-evaluated. Once the main design risk 760
management process, usually an FMEA supported by an FTA, has been completed just 761
after design initiation, subsequent amendments are simple to document. Review of the 762
design risk management together with the progress of the whole project at the regular 763
design control review meetings (19) is usually sufficient. 764
39
As the various phases of the design progress, risk assessment will become necessary for 765
manufacturing processes, the IFU, verification and validation work and pre-launch 766
matters (see Figure 4 and subsequent sections in this guide). All of these assessments 767
should be specified in the QMS policies and will provide information especially related to 768
safety and QA matters and to the main design risk management process, although they 769
are to some extent independent of it. 770
Any changes to the design, manufacturing process or IFU at any stage and for any reason 771
(for example because input requirements cannot be met, supplier change, regulation 772
change, CAPA, a new intended use, a new population of patients or users of the IVD, or a 773
new environment of use) must trigger input to the design risk management files and a 774
new assessment, even if only to report and justify that there is “no change” in risks. 775
6.3 Verification and validation 776
Design verification, in accordance with the manufacturer’s QMS, will confirm that the 777
design output meets the design input requirements. Results of the design verification will 778
usually be documented in, for example, a design history file. Verification occurs at 779
multiple stages and includes testing, inspection and analysis of results. Verification will 780
include hazards associated with handling and use, environmental effects, packaging 781
integrity tests, biocompatibility testing of materials to be used, bio-burden testing and 782
comparison with existing similar designs. 783
Design validation confirms that products conform to user needs and are suitable for their 784
intended use. Design validation must include risk analysis. Consideration must be given to 785
the robustness of the process. That is, expected variations of components, materials, 786
manufacturing processes and the user and user’s environment must be taken into 787
account. Validation must use routine production units tested under actual or simulated 788
conditions. 789
Process validation confirms that a process produces a result or product that consistently 790
meets requirements (and hence that the product consistently meets the correct 791
specifications). 792
40
Comment: The absence of “re-validation” (ongoing verification) programmes for 793
equipment and processes and inadequate qualifications of personnel performing such 794
tasks have resulted in nonconformities being recorded during WHO PQ inspections. 795
6.4 Analytical and clinical performance studies (design validation) 796
The risk management methodology and tools already described apply to performance 797
studies. 798
Existing WHO guidance and mock dossiers support a risk management approach to 799
analytical performance studies to confirm intrinsic performance capabilities relative to 800
design specifications and to clinical performance studies to confirm that the expected 801
performance of the IVD is achieved in its intended use by intended users (19). 802
The WHO publication Principles of performance studies (Technical Guidance Series 3) 803
refers to risk management in the context of analytical and performance studies 804
throughout (33). In addition, the sample dossiers (20, 21, 22) have information that can 805
be used when preparing for risk management of the processes. 806
The risk management plan, assessment, evaluation and control actions related to the 807
studies must be documented and executed. 808
Note: “Clinical performance studies should ensure that the rights, safety, and well-being 809
of subjects participating in a clinical performance study must be protected … That is, each 810
clinical performance study should generate new data, the benefits to health must 811
outweigh risks to study participants and any risks must be minimized, and confidentiality 812
must be respected” (33 Study rationale 4.6.1). 813
Furthermore, “The risk assessment conducted as part of product development should 814
also include a component that accounts for any hazards posed (to user and/or patient) by 815
the product during the course of the clinical study” (33 Study method 4.6.3). 816
As an example, assessment would need to include the following: 817
a well-defined study protocol (to include risk assessment and plans, assess data 818
collection, amendments and changes protocol). 819
41
monitoring of study (monitoring plan that considers complexity of study design, 820
clinical complexity of study population, geography of study location, experience 821
of investigators, relative safety of the product and data collection methods). 822
recruitment risk (selection of sites and patients, adequacy of medical records, 823
informed consent risk). 824
risk of deviation from the protocol (rules for cessation of study). 825
data collection risk (quality of data and staff turnover). 826
6.5 Change controls 827
Changes made to, for example, any design, manufacturing processes or intended use of 828
the product must be implemented and documented in line with the manufacturer’s QMS 829
change control requirements. The QMS will have procedures for the identification, 830
documentation, validation (or where appropriate verification), review and approval of 831
changes before implementation. The reason for and justification of the change and any 832
retraining required must also be documented. 833
Many changes will be related to the product design and so the risk management will likely 834
be based on hazard evaluation using FMEA as the main tool, closely linked to the overall 835
design risk management. Other changes, for example to labelling, would be expected to 836
be linked to risk management documentaton already established for both the design and 837
the process concerned for that IVD. Risk evaluation of change must be initiated before 838
any changes are made so that the planning for the change will take into consideration any 839
risk to be minimized and ensure that appropriate verification or validation of the process 840
and IVD is performed in a timely fashion. Any changes must be assessed and 841
documented in relation not only to the product or process that is changed but also taking 842
into account the possible repercussions across subsystems and the system as a whole 843
because modification of one aspect of a process might well introduce hazards elsewhere. 844
The following paragraphs exemplify some of the change processes which are frequently 845
found to be poorly managed during inspections by WHO PQ. The changes are often not 846
managed in compliance with (19) and any risk evaluation is not documented. Aspects of 847
particular concern are absence of the following: traceability, justification for change, 848
verification and validation of the change and notification to the user of the change. 849
42
6.5.1 Equipment change 850
Criticality of the equipment or process will affect the level of risk management of change 851
control. High-risk equipment or processes will require a higher level of qualification, 852
change control, maintenance and monitoring. 853
In addition, the category of equipment change will affect the risk management activities. 854
If it is an “identical” replacement, then assessment may be limited to 855
demonstrating that the equipment is identical as defined in the manufacturer’s 856
procedures, and documenting the process within the change control procedures, 857
recording specifications and operating parameters to demonstrate they are 858
identical. An abridged functional qualification may suffice. 859
If the equipment has the same dimensions, uses the same methodology and has 860
the same performance characteristics, then the activities described for identical 861
replacement plus additional performance testing may suffice. 862
A “true” change (neither of the above) would require a full risk management 863
approach covering the whole life cycle of the equipment and the products 864
concerned. 865
6.5.2 Change of supplier of a critical component of an IVD 866
Example: A recombinant protein is purchased from a new supplier. 867
The processes and the intermediate products tmust be re-validated to ensure 868
the new protein meets all of the requirements. 869
The design must be re-validated as the change will potentially affect assay 870
stability, sensitivity and specificity (ideally validation by users although this 871
depends on the risk evaluation). 872
The change must be notified to regulatory bodies as it is the change of a 873
critical component. 874
6.5.3 Change of intended use 875
Example: Risk management of change of the types of anticoagulants for plasma used in 876
the IVD: a change in the IFU is required. 877
The design must be re-validated, at least partly, because: 878
43
– the change will affect the intention in the IFU. 879
– the stability of the new plasma type must be documented. 880
– performance claims must be maintained. 881
The change may need to be notified, certainly if plasma types are restricted 882
and perhaps if increased. 883
6.6 The IFU and other labelling 884
Risk management for some processes such as the design, safety in use and development 885
of most of the manufacturing and QA procedures for an IVD can be based efficiently on 886
an FMEA and associated tools for quality management and improvement. However, as 887
noted previously (section 4) ISO 31010 (9) provides a comprehensive list of techniques 888
which might be better than FMEA for managing risks for processes for which the basic 889
hazards should be well understood and documented from the design and manufacturing 890
assessments. 891
6.6.1 IFU 892
As the IFU is the main communication between user and manufacturer it is regarded with 893
particular importance by WHO PQ. The contents of the IFU and their validation must be 894
evidence based. The following must be read in conjunction with TGS 5 “Designing 895
Instructions for use for in vitro diagnostic medical devices” (34) which presents WHO 896
expectations for IFU, and is consistent with international regulation (5 and 35). For 897
established manufacturing companies the use of checklists and properly facilitated 898
brainstorming as described in (9) should be sufficient to control the risks to be managed 899
during the IFU development process, perhaps with organization of the output using FMEA 900
like spreadsheets but with different column headings. The main residual risks related to 901
using, storing and disposing the IVD, its specimens and accessories should be available 902
from the design and manufacturing groups, the risks from not satisfying the regulatory 903
requirements laid out in the ISO 18113 (5) series from the company’s regulatory group 904
and risks related to the supply and physics of the IFU from the logistics managers. Each of 905
these departments plus the customer support group should be present at the IFU risk 906
management meetings of which records must be maintained (19). A specific risk, 907
relevant for WHO, is that the users’ language might not be included in the instructions for 908
44
use or the level of language might be not be appropriate for the level of training of the 909
intended user. As a consequence, diagrammatic aids for use are provided frequently but 910
this needs considerably thought to ensure they are appropriate for all intended users in 911
all environments. The culture of the users must be evaluated in each intended setting to 912
ensure the information is appropriate and accurate. Whatever “simple” user aids are 913
made available it is critical that the risk management process ensures that the intended 914
method and the depicted method are identical, changed in tandem as necessary, and 915
validated relative to the clinical evaluation of the product. 916
Although the risks will be product dependent and risk management must make specific 917
reference to the IVD concerned there are recognized, repeated deficiencies (and non-918
compliances with (5)) found in the IFU presented to WHO PQ. These deficiencies might 919
be found from the IFU itself, in comparison of the claims in the IFU with the data in 920
dossiers, or during on-site inspections. The following table, which can help with the risk 921
management process by contributing to checklists, brings together the principle issues 922
found (many of which should have been dealt with and made acceptable during IVD 923
development). Reading through the table it will be apparent that risk management for 924
the IFU needs to evaluate that all statements and numerical data required from (5, 3 and 925
35) are present and that there is evidence in the design history file for each. In addition it 926
will be apparent that consideration must be given to users who might have different 927
educational, language and technical skills from those with equivalent roles familiar to the 928
manufacturer. 929
Vulnerability Description Detail Examples
Safety
disposal method inappropriate
or not defined IVD, specimens,
accessories
warnings none or inadequate harmful chemicals
biocides, thiomersal, azide
control specimens inactivation of positive
specimens not proven
SOP invalid, equipment inadequate
Intent intended use not
defined no statement of
purpose
diagnostic, screening, quantitation, prognostic
45
Vulnerability Description Detail Examples
population to be tested
no restricting statement of what has been validated – or no exclusions for populations not tested
not validated in claimed medical setting
paediatric, elderly,
treated
point of care; clinics, laboratories for screening, diagnosis
intended users not stated or no
supporting data
evaluated only by expert local laboratories or only by manufacturer
limitations
known limitations of method, reagents, patient type: some or all not stated
IgM not detected, strains of target organism not detected (malaria, cholera, HIV), treated patients
Claims
specimen types
plasma types not specified or not validated
no statement of what has been validated – or no exclusion of types not tested
performance not evaluated on
appropriate populations
specificity not tested appropriately
precision not evaluated by
intended users
only by expert users or in manufacturers own facility and staff
stability
not proven rigorously partial or no stability
data for specimens
based on QC panel not on the critical specimens
46
Vulnerability Description Detail Examples
timings and volumes
not validated rigorously or not at all
Reading time not validated at beginning and end of assigned life
controls not proven to warn of
damaged IVD with lower performance
EIA controls set too high
line controls fail to mirror state of active constituents
Documentation
legibility poor users and
environments not considered
font too small type face
unsuitable use of white space
inappropriate ink smudges with
time
intelligibility poor
no consideration of educational level of intended users in intended environment
complex language poor translations languages in text
not appropriate for intended areas of use
symbols used not in accord with (36)
or absent
mismatches in instructions
IFU, “simple” method guide, training manuals differ
pictures inaccurate number of drops angle of addition
different methods (times, volumes)
product code not present or not
under version control
product changes nor reflected in the product code
IFU content validation
not traceable through the change system
version of IFU provided not the same as, or untraceable to, that used in performance verification and validation
Regulation ISO 18113 series requirements not met sections missing
47
Vulnerability Description Detail Examples actual
manufacturer not clear
OEM, inadequate addresses, contact details
supplier control
no audit of printers
specifications not appropriate or not checked
improvements IFU not updated as
appropriate
CAPA data not applied between IVD for different measurands
A spreadsheet related to a checklist like this could have further columns indicating what 930
measures need to be taken and by whom. 931
6.6.2 Package labelling 932
Checklists and facilitated brainstorming (9) should be appropriate for managing risks 933
related to package labelling for IVD because the hazards related to transport, storage and 934
use of the IVD need to be well understood from its development processes. WHO PQ 935
expects that package labelling will be suitable for users in environments that might be 936
more extreme than in manufacturers’ home countries and where transport operatives 937
might not read English. It is particularly important that the labelling on the outer package 938
for transport is clear, of a size providing easy legibility under all conditions and uses 939
international symbols (36) appropriately and correctly. Restrictions related to handling 940
and allowable temperatures must be very obviously displayed. Regulatory requirements 941
for labelling are set out in the ISO 18113 (5) series and in (35). The following table lists 942
some of the common, repeated failings of package labelling submitted to WHO PQ. It is 943
intended as a resource for a manufacturer’s checklist but each IVD, in each pack size, 944
must be risk managed individually by the manufacturer. Any statement on labelling (e.g. 945
transport conditions, expiry dating) must be supported by evidence in the design history 946
file of the IVD. 947
Issue Detail Examples
48
Clarity
Labels obscure
Font too small Only English on shipping labels:
No symbols used Symbols used incorrectly
Ink used not permanent
Ink poor quality and smears during transport or when handled in use
Ink fades in strong light
Labels detach from containers
Glue of poor quality: heat labile or water soluble
Claims Stability not valid
Evidence for stability only available for one bottle size although different kit sizes use different bottles
Shelf-life or transport studies performed using unlabelled materials
Regulatory
Not fully compliant with (36) and (5) or (35)
No product code No lot number No expiry date No storage conditions No single use symbol
Supplier audits inadequate
6.7 Manufacturing 948
6.7.1 Suppliers 949
Control of both suppliers and goods supplied have always been expectations according to 950
ISO 13485; however ISO 13485:2016 now explicitly states that the criteria for their 951
control and management must be proportionate to the risk associated with the medical 952
device (19). Formal, documented risk management in relation to suppliers and goods 953
(which should always include assessments of continuity of supply in addition to the 954
assessment of quality of the goods supplied) is hence a regulatory requirement. The 955
policies and procedures of the organization should specify how this risk management will 956
be performed and documented, and these activities must form part of the design control 957
of the IVD. The mechanisms for these risk management activities are as described 958
previously, probably with an emphasis on ISO 31000 (8) and the methods it specifies. 959
Following from this assessment will be the documented rationale of the extent and 960
frequency of supplier audits and the nature and methods of verifying the incoming goods 961
against the specifications (The incoming goods QA specifications, developed to minimize 962
effects of identified hazards.) As with all risk management, the process is iterative: once 963
49
controls have been developed, the system must be re-assessed to ensure that no new 964
hazards have been created. Then, once the system is operative, it must be continuously 965
reviewed to ensure that it is functional and does not need amendment. 966
In addition to the use of suppliers for basic materials for the manufacture of IVDs, 967
outsourcing of services and component manufacturing is increasingly common. Particular 968
hazrds are associated with outsourcing of manufacturing processes of either subsystems 969
or complete systems. Good risk management can help maintain the balance between 970
quality and cost. This, together with the increased regulatory requirements of risk 971
management (19), make it essential to have a robust risk based approach for evaluating 972
new and existing suppliers. 973
The steps to be taken are as follows: 974
a. Use the already identified critical control points of the product noted. 975
b. Identify specifications that the supplier needs to meet. Document how these are 976
to be evaluated (for example initial product check, audit of supplier) and 977
monitored (for example incoming product QC data). 978
c. Prepare a qualification plan to: 979
– assess the supplied product risk. 980
– assess the supplier risk. 981
– schedule audits. 982
– ensure effective follow-up of nonconformities. 983
– determine frequency of formal review of supplier. 984
d. Prepare supplier quality agreements (SQA). Note: SQA apply to both internal 985
suppliers (for example subsidiary providers under a single ownership) and to 986
external suppliers. 987
The SQA should indicate, for example: 988
– commitment by the supplier to quality. 989
– agreement on how quality will be monitored. 990
frequency and scope of audits defined; allow purchaser’s auditors 991
to review audit reports from other external auditors, especially 992
when related to QMS regulatory approvals. 993
50
access to supplier QMS management review. 994
change control notification obligation, for example timely 995
notification of changes to product design, manufacturing 996
equipment, critical personnel, QC changes, change in vendor of raw 997
materials etc. 998
the product acceptance criteria defined. 999
for nonconforming product, a description of actions taken by the 1000
supplier within their QMS. 1001
complaints from other customers of the supplier 1002
field corrective actions – supplier’s involvement and 1003
responsibilities. 1004
environmental controls. 1005
distribution of product, for example shipping conditions 1006
(temperature, humidity, dust, vibration), packaging. 1007
e. Use of tools. 1008
– use FMEA or other such tools. 1009
– develop supplier risk scorecards to rank suppliers based on past and 1010
current performance. 1011
– define key metrics and key performance indicators (KPIs) 1012
f. Demonstrate compliance to auditors and/or regulators. Documentation and 1013
qualified personnel need to be available for interview to support the following: 1014
– qualification plan for each supplier (referred to in supplier audit plans). 1015
– approved supplier list that includes explanation of criticality of supplier, 1016
how this is determined and what impact this has on risk management. 1017
– supplier audits: 1018
that qualifications of auditors are suitable for the task. 1019
objective evidence that specific requirements for the 1020
manufacturer’s particular IVD have been met; compliance with 1021
relevant technical standards may also be required. 1022
well-documented audit findings (reports contain sufficient detail to 1023
illustrate thoroughness of audit). 1024
51
supplier corrective action plan (CAP) or corrective action request 1025
(CAR) is documented and follow-up is completed in an appropriate 1026
time frame. 1027
triggers for action are defined – for example, critical 1028
nonconformities may initiate an immediate meeting with the 1029
supplier, an additional audit or cessation of supply. Justification for 1030
actions taken must be documented. 1031
Evidence that top management has been informed of findings (as 1032
required by the manufacturer’s QMS) and any decisions for action 1033
or no action were taken by qualified personnel. 1034
– Traceability at all steps is essential. Documentation must be readily 1035
accessible for each supplier. For example, although a spreadsheet of KPIs 1036
of all suppliers may be available, and all audit reports kept together, all 1037
data related to a single supplier must be readily available in an assembled 1038
single supplier-specific file for review by an external auditor for regulatory 1039
and compliance purposes. Note that by assembling all notifications of 1040
nonconforming material and other reports from a single supplier, trends 1041
may become evident. 1042
– The risk to the manufacturer’s QMS posed by a poorly performing supplier 1043
must be considered broadly for example the problem could occur with 1044
another supplier and could thus be prevented. 1045
– Evidence of good communication between the supplier and manufacturer 1046
must be available. That is, it should be shown that the supplier is quick to 1047
respond to the manufacturer’s quality concerns, for example with a 1048
corrective action plan; cooperative when scheduling audits; and readily 1049
provides evidence of QMS compliance, for example, certification and audit 1050
reports from EU notified bodies or the US Food and Drug Administration. 1051
Comment: The work of external auditors, although becoming more 1052
harmonized, varies in quality. Regulatory approvals such as certification are 1053
not valuable unless they are supported by the review of the actual audit 1054
reports on which they are based. The manufacturer’s auditor can thus assess 1055
52
the auditors’ skills and the thoroughness of the external audit and hence the 1056
validity of regulatory approvals provided by a supplier. 1057
6.7.2 Manufacturing processes 1058
Each manufacturing process should be risk assessed to ensure the safety of 1059
manufacturing staff together with the capability of the process to produce the planned 1060
results leading to a consistent, safe product. These top level assessments should take into 1061
account the local health and safety regulations as well as the expectations of ISO 14971 1062
(1). There should also be an assessment of: 1063
the materials used in relation to safety, to local environmental regulations and 1064
possibly to ISO 14001 (32). 1065
equipment (purchase, maintenance and cleaning), training needs as 1066
communicated in the manufacturing section of the input documentation, and 1067
cost. 1068
the written procedures to be followed by the operators (legibility under 1069
manufacturing conditions, intelligibility, completeness, and the presence of any 1070
warnings and precautions). 1071
any in-process controls necessary to ensure consistency within the process and 1072
the methods for those controls. 1073
potential effects of differences in the scale of the process (especially in stability 1074
and specificity of the final IVD), and the necessity for validation of different scales 1075
of manufacture. 1076
the necessity for validation or verification of the product of the process, and the 1077
methods for performing those activities. 1078
The mechanism for process risk management is most often an FMEA led by the 1079
manufacturing department with input at least from R&D and QA. A flow diagram of the 1080
process being evaluated is essential as is detailed technical knowledge of the materials 1081
being used, the capability of the manufacturing department (before and after any 1082
controls introduced as a result of the assessment) and the reasons for the specifications 1083
for the product of the process. Figure 5 shows fragments of the risk management 1084
53
documentation for a manufacturing process. It shows only a part related to the actual 1085
manufacturing steps, not all the aspects listed above. Note that the work instructions are 1086
written in a structured fashion that relates easily to a flowsheet and then to the FMEA. 1087
This makes the process easy to manage, explain and update as necessary. 1088
It should be possible to trace each control action back to the hazard that it controls and 1089
forward to the validation of the control and the test methods involved. (See the Test 1090
Method Validation guide, TGS-4 in this series (37)) 1091
Fragment of a process instruction showing the hierarchy of actions for a single step
Fragment of the flowchart for the same process, showing several steps
1. Ink Jet Mark the platesRemove plates from boxes
Transport to coating
Cover print site with label
Print label
Check the label and sign for it
Ink Jet mark whole batch
Stack marked plates
Put stacks in cabinets
Record cabinet number
2. Prepare for coatingRecord room temperature
Check bed of instrument
Collect coating reagents
Record time of spiking
3. Set the dispensed volumeAdjust the dispenser to 110 l
Load 40 plates, no frames
Calibrate the dispenser
54
Fragment of the FMEA document for the same process
Figure 5 Risk management for a manufacturing process
6.7.3 Safe documentation 1092
It is essential that each procedure is assessed for the safety of the process concerned. 1093
This should be a priority of management of manufacturing and needs emphasis. In 1094
particular, appropriate warnings (about hazards to operator safety) and cautions (about 1095
hazards to equipment) must be included and evaluated. 1096
6.7.4 Process changes in manufacturing 1097
Any change to a process must be managed within the change control system of the QMS 1098
but the policies concerning change must also make reference to re-assessing the effect of 1099
any changes on the established risk profile. This applies even to minor changes to a 1100
manufacturing process. Such changes must be managed, details recorded and the risk 1101
management documents updated accordingly, even if the outcome is that there is no 1102
change to the risks. 1103
Example: Nonconformities noted during WHO inspections of production lines have 1104
included the following: 1105
A lack of staff well trained in risk management techniques and hence the 1106
inadequate application of such techniques to the production line. This has 1107
resulted in observing unsafe practices of personnel, for example not 1108
55
wearing hearing protection and unsafe handling of infectious materials, as 1109
well as hazard to product quality, for example inappropriate handling of 1110
labile biological materials. 1111
Well-structured and sufficiently detailed batch manufacturing records 1112
(BMRs) play an important role in reducing manufacturing risk. Analysis of 1113
BMRs by suitably qualified personnel was poorly performed leading to lost 1114
opportunities for preventive action to maintain quality. Quality control data 1115
in BMRs (checking of the output being within specifications and trend 1116
analysis of the data) were not adequately reviewed and thus trends 1117
towards failure modes were not detected. 1118
Reporting on deviations (products not meeting specifications) and 1119
appropriate follow-up actions were often inadequate. 1120
References 1121
1. ISO 14971:2007 and EN ISO 14971:2012 Medical devices – Application of risk management to 1122
medical devices. Geneva: International Organization for Standardization; 2007. 1123
2. ISO Guide 73:2009 Risk management — Vocabulary. Geneva: International Organization for 1124
Standardization; 2009. 1125
3. United States CFR 21- Code of Federal Regulations Title 21. Sec. 820.3 Definitions. 1126
4. ISO 9000:2005 Quality management systems – Fundamentals and vocabulary. Geneva: 1127
International Organization for Standardization; 2005. 1128
5. ISO 18113-1:2009 In vitro diagnostic medical IVDs – Information supplied by the manufacturer 1129
(labelling) – Part 1: Terms, definitions and general requirements. Geneva: International 1130
Organization for Standardization; 2009. 1131
6. WHO PQDx_018: Instructions for Compilation of a Product Dossier. Geneva: World Health 1132
Organization; 2014. 1133
7. WHO PQDx_014: Information for Manufacturers on the Manufacturing Site(s) Inspection 1134
(Assessment of the quality management system). Geneva: World Health Organization; 1135
2017. 1136
56
8. ISO 31000:2009 Risk management — Principles and guidelines. Geneva: International 1137
Organization for Standardization; 2009. 1138
9. ISO 31010:2009 Risk management – risk assessment techniques. Geneva: International 1139
Organization for Standardization; 2009. 1140
10. CLSI: Laboratory quality control based on risk management, approved guideline (CLSI 1141
document EP23-A). Wayne, PA: Clinical and Laboratory Standards Institute; 2011. 1142
11. CLSI: Quality management system: development and management of laboratory documents: 1143
approved guideline (CLSI document QMS02-A6, sixth edition). Wayne, PA: Clinical and 1144
Laboratory Standards Institute; 2013. 1145
12. GHTF/SG3/N015:2005 Implementation of risk management principles and activities within a 1146
quality management system. Global Harmonization Task Force (GHTF); 2005. 1147
13. GHTF/SG1/N068:2012 Essential principles of safety and performance of medical devices. 1148
Global Harmonization Task Force (GHTF); 2012. 1149
14. ISO/DIS 16142-2:2016 Medical devices – Recognized essential principles of safety and 1150
performance of medical devices – Part 2: General essential principles and additional 1151
specific essential principles for all IVD medical devices and guidance on the selection of 1152
standards. Geneva: International Organization for Standardization; 2016. 1153
15. Shebl NA, Franklin BD, Barber N. Is failure mode and effect analysis reliable? J Patient Saf. 1154
5;2009;86–94. 1155
16. Potts HWW, Anderson JE, Colligan L, et al. Assessing the validity of prospective hazard analysis 1156
methods: a comparison of two techniques. BMC Health Serv Res. 14;2014:41–50. 1157
17. Ashley L, Armitage GJ. Failure mode and effects analysis: an empirical comparison of failure 1158
mode scoring procedures. Patient Saf. 6;2010:210–215. 1159
18. Shebl NA, Franklin BD, Barber N. Failure mode and effects analysis outputs: are they valid? 1160
BMC Health Serv Res. 12:2012;150–160. 1161
19. ISO 13485:2016 Medical devices – Quality management systems – Requirements for regulatory 1162
purposes. Geneva: International Organization for Standardization; 2016. 1163
57
20. WHO Prequalification: Sample product dossier for a qualitative test for HIV-1 and HIV-2: 'Risk 1164
analysis and control summary' 5.4 and Annexes II, III, IV and V. Geneva: World Health 1165
Organization 1166
(http://www.who.int/diagnostics_laboratory/guidance/160613_sample_product_dossier1167
_for_qualitative_nucleic_acid_test_hiv1_2.pdf?ua=1). 1168
21. WHO Prequalification: sample product dossier for a quantitative nucleic acid test to detect 1169
HIV-1 RNA’: 'Risk analysis and control summary' 5.4 and Annexes II, III, IV and V. Geneva: 1170
World Health Organization 1171
(http://www.who.int/diagnostics_laboratory/guidance/161026WHO_QuantHIV_sample_1172
dossier.pdf?ua=1). 1173
22. WHO Prequalification: Sample product dossier for an IVD intended, for HIV self-testing 1174
Geneva: World Health Organization 1175
(http://www.who.int/entity/diagnostics_laboratory/guidance/160613_sample_product_1176
dossier_for_intended_hiv_self_testing.pdf?ua=1). 1177
23. Franklin DB, Shebl NA, Barber N. Failure mode and effects analysis: too little for too much? J 1178
Qual Saf. 21:2012; 607–611. 1179
24. ISO 10005:2005 Quality management systems — Guidelines for quality plans. Geneva: 1180
International Organization for Standardization; 2005. 1181
25. CLSI: Risk management techniques to identify and control laboratory error sources; approved 1182
guideline. Second edition CLSI document EP18-A2 (ISBN 1-56238-712-X). Wayne, PA: 1183
Clinical and Laboratory Standards Institute; 2009. 1184
26. American Society for Quality (ASQ) (http://asq.org/learn-about-quality/seven-basic-quality-1185
tools/overview/overview.html). 1186
27. CLSI: Nonconforming event management. Second edition (CLSI guideline QMS11) Wayne, PA: 1187
Clinical and Laboratory Standards Institute; 2015. 1188
28. Kusselman I. Pennechi F., UPAC/CITAC Guide: Classification, modeling and quantification of 1189
human errors in a chemical analytical laboratory. Pure Appl Chem. 88;2016:477–515. 1190
58
29. Krouwer J. (https://jkrouwer.wordpress.com/2016/11/01/westgards-detection-and-iqcp/, 1191
accessed 6 January 2017). 1192
30. Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on 1193
in vitro diagnostic medical devices. 1194
31. ISO 15198:2004 Clinical laboratory medicine – In vitro diagnostic medical devices – Validation 1195
of user quality control procedures by the manufacturer. Geneva: International 1196
Organization for Standardization; 2004. 1197
32. ISO 14001:2015 Environmental management systems – Requirements with guidance for use. 1198
Geneva: International Organization for Standardization; 2015. 1199
33. WHO prequalification: ‘Principles of performance studies’ Technical Guidance Series (TGS) 3. 1200
Geneva: World Health Organization 1201
(http://www.who.int/diagnostics_laboratory/guidance/160613_tgs3_principles_for_perf1202
ormance_studies.pdf?ua=1). 1203
34. WHO prequalification: “Designing Instructions for use for in vitro diagnostic medical devices” 1204
TGS 5. Geneva: World Health Organization 1205
35. US FDA “In Vitro Diagnostic Device Labeling Requirements” 2014 1206
https://www.fda.gov/medicaldevices/deviceregulationandguidance/overview/devicelabe1207
ling/invitrodiagnosticdevicelabelingrequirements/default.htm 1208
36. ISO 15223-1:2016. Medical Devices - Symbols to be used with medical device labels, labelling 1209
and information to be supplied - Part 1: General requirements. Geneva. International 1210
Organization for Standardization; 2016. 1211
37. WHO prequalification: ‘Guidance on Test Method Validation for in vitro diagnostic medical 1212
devices’ Technical Guidance Series (TGS) 4 Geneva: World Health Organization 1213
