Eight Common Myths And Misunderstandings About Risk Managements3.amazonaws.com/rdcms-aami/files/production/... · About Risk Management Patrick Caines, Pat Baird, and Kathleen Whanger

13Horizons Spring 2015

Perspectives on Risk Management

Risk management is a critical process in the design of medical devices and extends into the postmarket surveillance of product performance. A safe product comes from the robust applica-tion of risk management principles and processes. Unfortunately, we have found that many common myths and misunderstandings undermine the results of a well-intentioned team.

This article seeks to highlight common myths and misunderstandings of risk manage-ment, offer explanations of where they come from, and clarify what manufacturers can do to resolve them.

Myth: FMEA Is Our Risk Management FileThe failure mode and effects analysis (FMEA) is a popular technique used to assess what can go wrong and understand the impact of that failure. Part of the success of an FMEA approach is that it can be used in a wide variety of circumstances. For example, an electrical engineer can use it to better understand the impact of a component-level failure on a circuit board, a process engineer can use it to understand the impact of an error during the manufacturing process, and a human factors engineer can create a use-misuse FMEA to understand how use errors can affect product performance.

Developing an FMEA can be extremely methodical, and the FMEA itself can be very detailed. Engineers often love details, and FMEAs are a favorite tool.

Per ANSI/AAMI/ISO 14971:2007, the risk management process needs to identify hazards, estimate and evaluate risks, control the risks, and monitor the effectiveness of the controls.1 Although it may appear that an FMEA has many of these elements covered, an FMEA by itself is not a risk management file.

One reason why an FMEA alone is not sufficient is that an FMEA does not identify hazards. Some variations of FMEA may express the effects of a failure as a resulting hazardous situation; an FMEA itself does not explain how the hazards were identified. Annex E in ANSI/AAMI/ISO 14971:2007 provides some guidance on hazard identification.

In addition, FMEA is one of many tech-niques, with certain advantages and disadvantages. A robust risk file uses a diversity of approaches to estimate and evaluate risks; an FMEA alone is not sufficient to estimate and evaluate risks. Other complementary approaches include preliminary hazard analysis, fault tree analysis, hazard and oper-ability analysis, and hazard analysis and critical control points.

Misunderstanding: Hazard vs. Hazardous Situations vs. HarmANSI/AAMI/ISO 14971:2007 includes the following definitions:• Harm refers to physical injury or damage to

the health of people or damage to property or the environment.

• A hazard is a potential source of harm.

Eight Common Myths And Misunderstandings About Risk ManagementPatrick Caines, Pat Baird, and Kathleen Whanger

About the Authors

Patrick Caines, PhD, MBA, is di-rector of product surveillance for GE Healthcare in Milwaukee, WI. E-mail: patrick.

[email protected]

Pat Baird is director of engineering at Baxter Health-care Corpora-tion in Round Lake, IL. E-mail:

[email protected]

Kathleen Whanger is quality assurance manager for Teleflex Medical in Asheboro, NC. E-mail: kathleen.

[email protected]

© Copyright AAMI 2015. Single user license only. Copying, networking, and distribution prohibited.

14 Horizons Spring 2015


• A hazardous situation is a circumstance in which people, property, or the environment are exposed to one or more hazard(s).Despite these definitions, debate frequently

occurs regarding whether something is a hazard, hazardous situation, or harm. Teams often will find that members have widely different interpretations of these terms and spend a substantial amount of time arguing over the category in which a certain term belongs.

For example, in an electrically powered medical device, one thing that the design typi-cally should avoid is electrocuting the patient. What one person might call “electrocution,” another person might call “exposure to high voltage,” and another person might confuse with “inadequate insulation or grounding.” A medical professional on the team might decide to refine electrocution into the medical signs of “burns,” “involuntary muscular contraction,” or “ventricular fibrillation.” Further compounding the confusion is that certain devices such as automated external defibrillators are intended to deliver high voltage to the patient.

After a certain point, these time-consuming debates over terminology provide less and less value. An analysis of ANSI/AAMI/ISO 14971:2007 shows that these terms are used to help the thinking process in identifying categories of things that can go wrong (e.g., “energy hazards”) and identifying the severity of the outcome. A much more thorough and thoughtful risk analysis can be produced by breaking the thought process into following stages:• What are the sources of harm?• What sequence(s) of events leads to

someone being exposed to those sources of harm?

• In what sequence(s) of events does the exposure lead to harm?One approach to reducing the time spent

aligning the team on hazards, hazardous situations, and harms is to establish these categories and examples early in the risk management process and to accept that situations will arise in which overlap occurs among these categories. The main objective is to use these thinking tools to develop a better risk analysis. If, for example, renaming “electrocution” to “exposure to high-voltage

energy” does that, then make the change. If it doesn’t add value, then don’t make the change.

Misunderstanding: Brainstorming ‘Sharknado’ ScenariosEngineers love puzzles. If you ask an engi-neer, “What is the worst possible thing that can happen?,” they inevitably will come up with an extremely unlikely scenario in which the most innocuous failure results in the worst possible catastrophic outcome.

While understanding how little failures can lead to big disasters is important, the process must be balanced with the mantra, “If everything is important, then nothing is important.” Incremental time spent develop-ing situations in which every failure leads to a catastrophic outcome is not nearly as useful as time spent identifying key points of control and ensuring that mitigations work as intended.

It can sometimes be difficult to know when a team is working with an abundance of caution versus when a team has switched to “sharknado” mode. One method is to periodically ask the team if they are discov-ering anything new about the design (e.g., deliberations are adding value) or if they are simply developing more scenarios to develop more scenarios.

Misunderstanding: Implementing Risk Control But Not Validating EffectivenessIn this misunderstanding, developers, following good requirements-based testing philosophies, verify that a risk control is properly implemented but fail to validate that the risk control is actually effective at manag-ing the risk.

ISO 9001:2008 defines verification and validation as follows2: • Verification: ensure the design and devel-

opment outputs have met the design and development input requirements

• Validation: ensure the resulting product is capable of meeting the requirements for the specified application or intended useOne way in which this can happen is when

risk management is brought into the design cycle after the input requirements have been transposed into output requirements. As a

Incremental time spent

developing situations

in which every failure

leads to a catastrophic

outcome is not nearly

as useful as time spent

identifying key points

of control and ensuring

that mitigations work

as intended.




result, the risk controls end up being stuck in a “risk management bubble.” In this bubble, the risk controls are either not traced back to the input or output requirements or are considered stand-alone input requirements and only trace down to output requirements.

In addition, this “risk management bubble” leans heavily on risk controls, which can be categorized as protective measures or information for safety. ANSI/AAMI/ISO 14971:2007 provides three broad categories of risk controls: 1) inherent safety by design, 2) protective measures in the medical device itself or in the manufacturing process, and 3) information for safety.

As the risk management bubble grows throughout the design process, the protective measures and information for safety grows as well. After device testing begins, verification is used to confirm whether protective meas-ures and information for safety risk controls exist for the product and product labeling. However, validation of risk controls does not occur because assumptions are made that protective measures and information for

safety do not need to be validated by the user. As a result, the risk controls are not confirmed to be effective and, as a result, not confirmed to meet intended use or user needs.

The following steps can be taken to minimize the chance of this occurring:1. Ensure that the requirements management

plan includes a clear explanation for how risk controls will trace back into product requirements.

2. Ensure that the validation plan and labeling validation plan include confirmation risk controls that meet the intended use and user needs. One way to support this is by develop-ing use cases, then creating a risk management file based on when the use cases are not met.

Myth: Postmarket Surveillance Is the Same as Complaint HandlingIt is easy to understand why postmarket surveillance is thought to be synonymous with complaint handling. Companies spend enor-mous efforts being compliant with U.S. Food and Drug Administration (FDA) complaint


16 Horizons Spring 2015


handling requirements and either fail to understand the distinction or believe postmar-ket activities are discretionary.

Although definitions vary, postmarket surveil-lance can be thought of as a comprehensive set of activities to monitor the performance of products in the marketplace. A useful model is to define the four main postmarket surveillance subprocesses, starting with 1) collecting the right data; 2) analyzing the data and assess-ing patient safety and product risk; 3) taking the right actions, including global regulatory reporting and escalation into corrective and preventive action system as needed; and 4) communicating and disseminating the information to a broad group of internal and external stakeholders.

Data collected should be from passive (reactive) surveillance (e.g., complaints, evaluation of service events), as well as from active surveillance (e.g., literature reviews, customer feedback, postlaunch patient studies). Passive postmarket surveillance is always required. Active postmarket surveillance also is required, but the level and type of activity is dependent on product risk. Annex F.7 in ANSI/AAMI/ISO 14971:2007 states, “The risk management plan should include documenta-tion of decisions, based on risk analysis, about what sort of post-market surveillance is appropriate for the device, for example, whether reactive surveillance is adequate or whether proactive studies are needed.”

These four elements define the basis of an effective and compliant postmarket surveillance system. Both postmarket surveillance and its subset, complaint handling, are required.

Myth: Complaint Investigation Is Not Needed Unless a Device Is ReturnedSome firms mistakenly believe that they are absolved from conducting a complaint investi-gation if they are unable to get back the device. According to FDA Code of Federal Regulations (CFR) 820.198(b), “Each manufacturer shall review and evaluate all complaints to determine whether an investigation is necessary. When no

investigation is made, the manufacturer shall maintain a record that includes the reason no investigation was made and the name of the individual responsible for the decision not to investigate.” In addition, the agency states in CFR 820.198(c), ”Any complaint involving the possible failure of a device, labeling, or packag-ing to meet any of its specifications shall be reviewed, evaluated, and investigated, unless such investigation has already been performed for a similar complaint and another investiga-tion is not necessary.”3

If a firm is unable to get back the device or retrieve the information needed to investigate the complaint, its “good-faith efforts” to perform these functions should be docu-mented. Although no firm rule exists, typically three attempts should be made, with at least one written request for information. The good-faith effort should appropriately balance obtaining both product- and patient-centric information. Too often, firms fail to make good-faith efforts to determine whether there was any patient involvement and the patient’s condition before, during, and after the malfunc-tion of the device. This information is needed, not only for conducting a good investigation, but also for accurate and complete medical device and regulatory reporting.

If all attempts fail, the firm is still on the hook for conducting an investigation. This can be done, for example, by reviewing trend data for similar complaints, batch records of the suspected device, or examining any retained samples. Keep in mind that the level of effort made to obtain the device will depend on the nature and severity of the event. Reviewing the product’s risk management file and assessing likely failures and mitigations also may be useful.

Misunderstanding: Complaint Information Tells the Whole StoryRecently, a company complaint team received 20 complaints from Japan about an iodine sponge breaking down during surgery. The complaints then were passed on to the manu-facturing site that packaged the iodine sponges into kits. Then, three months were spent by the manufacturing site investigating all potential root causes of the issue with the help of their supplier. The supplier performed a failure investigation and looked to see if they had received any other complaints, if any anomalies

Passive postmarket surveillance is always required. Active

postmarket surveillance also is required, but the level and type

of activity is dependent on product risk.




were present in the shipping conditions of the iodine sponges, if any anomalies occurred in the packaging conditions, and if their supplier had any anomalies at the manufacturing facility.

After a few months of receiving the com-plaints, the failure investigation provided no significant indication of risk in the process and the investigation went cold. However, based on other recent events, the manufacturing site that packaged the sponges into kits requested that the company’s complaint team question the hospitals that issued the complaints. It then was discovered that the hospitals were soaking the sponges in a solution for at least five minutes before use. Further analysis revealed that the soaking was causing the sponges to break down.” Changing the soaking instructions addressed the issue and resolved the complaints.

Myth: Users Are Responsible for Use Error, not the DesignOne of the easiest ways to mitigate risks is to provide safety information in the reading material for the user. The reality, however, is that this information is not always read. Nonetheless,

knowing that people don’t always read safety information is not a reason to always identify the root cause of the risk as a use error. Therefore, it is so critical to focus on whether any changes inherent to the design can be made or any protective measures can be implemented. Safety information alone should not be relied on for risk control. n

References1. Association for the Advancement of Medical

Instrumentation. ANSI/AAMI/ISO 14971:2007:

Medical devices—Application of risk management to

medical devices. Arlington, VA: Association for the

Advancement of Medical Instrumentation; 2007.

2. International Standards Organization.

ISO 9001:2008, Quality Management Systems.

Geneva, International Standards Organization;

2008.

3. U.S. Food and Drug Administration. Code of

Federal Regulations, title 21, subchapter H–

Medical Devices. Available at: www.accessdata.

fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.

cfm?fr=820.198. Accessed January 29, 2015.



Documents

Eight Common Myths And Misunderstandings About Risk Managements3.amazonaws.com/rdcms-aami/files/production/... · About Risk Management Patrick Caines, Pat Baird, and Kathleen Whanger