Copyright © 2014 MPR Associates, Inc. All Rights Reserved

Design Verification Testing: The FDA Cares, You Should Too

MPR Product Development

By Lynessa Erler

Director of Product Testing and Test Services

Design Validation

Design Changes

Design Controls - General

Design Verification

Design History File

Design Review

Design Input

Design & Development Planning

Design Output

Design Transfer

Number of Warning Letters0 25 50 10075 125

2

Design Verification failures are costly...may

result in rejection of submissions, warning letters, recalls, loss of market share, or in the

worse cases, patient harm.

Why Should You Care about Design Verification Testing?

Design Verification failures are costly, and these costs may be paid not just by development budget and schedule, but may result in rejection of submissions, warning letters, recalls, loss of market share, or in the worse cases, patient harm. Design Verification is one of the most commonly cited sections in recent warning letters issued by the FDA that include findings related to specific sections of the Design Control regulation (21 CFR 820.30).

Figure 1. Number of Warning Letters issued by FDA by Design Control Section1,2,3,4

3

Design Verification is the process used by medical device manufacturers to ensure that the outputs from the design process meet the requirements specified in the design inputs. Activities such as Tests, Inspections, or Analyses provide objective evidence that the specified requirements have been fulfilled. This paper will examine three categories of design verification test failures and discuss what can be done to prevent them.

Figure 1. Number of Warning Letters issued by FDA by Design Control Section1,2,3,4

Category 1: Failing When You SHOULD Have Passed

Poor planning of Design Verification testing activities often leads to failures that do not reflect a deficiency of the device to meet it requirements, rather they reflect a deficiency in test design. Test criteria may not reflect device requirements or test methods may not adequately control the test environment, resulting in the execution of a test for which the device was not designed. Test tools may not accurately or reliably measure the device performance, resulting in inaccurate test data.

For example: A verification procedure uses a method to trigger a device alarm. However, during testing at low temperatures, the device fails to alarm in accordance with the test criteria. Through investigation it is discovered that the device is functioning properly, but that the test method does not reliably trigger the alarm at low temperatures.

These types of failures typically do not require revision to the device design, only revisions to test design and retesting. In the worst cases, revised or new test tools and methods may be required.

ProductionRequirements DesignPlanning

$Best Case

Worst Case

$$

Failing When You Should Have Passed

What Will This Cost You?

4

Category 2: Failing When You SHOULD Have Failed

When a device does not meet the requirements specified as inputs to the design process, it is expected that the device will fail design verification testing. This commonly occurs when insufficient engineering characterization testing was performed during the design phase to fully understand the device’s design margins. It can also occur when late stage revisions to requirements are made without fully understanding the change’s impact on the device design and the need for rework.

For example: A device manufacturer made a late change to the requirements, extending the device’s battery life, but failed to consider how this change affected the low battery alarm thresholds in the device software. During verification testing the low battery alarm sounded before the desired time remaining. This failure required additional engineering characterization tests to determine the appropriate alarm thresholds, modifications to device software, and retesting.

In the best case scenario, these types of failures require revision to device design and subsequent revisions to test design and retesting. In the worst case scenario, these types of failures require revisions to requirements.

ProductionPlanning

$$$Best Case

Worst Case

$$$$

Failing When You Should Have Failed

What Will This Cost You?

5

Passing When You SHOULD Have Failed

When design verification testing fails to identify design deficiencies, the consequences are costly. This can occur when both the design and verification teams have an incomplete understanding of device requirements. The consequences of these failures depend upon how and when these deficiencies are revealed. If found during validation activities, the costs are to development budget and schedule. If found later, the manufacturer’s reputation and patient’s safety may be at risk.

Other types of failures which fall in this category include incomplete testing and testing that is not adequately documented. When your testing fails to convince the regulator that your device meets its requirements, then your testing has failed.

For example: During an onsite inspection, the FDA determined that a device manufacturer failed to verify that a device’s battery could consistently meet specifications over time. This resulted in a warning letter from the FDA and implementation of a CAPA to address the issue.

These types of failures most often point to inadequate requirements and may require rework through all phases of the product development lifecycle.

Planning

$$$$$$$$$$

Passing When You Should Have Failed

What Will This Cost You?

Key Takeaways: What You Can Do To Avoid Catastrophe

It is unrealistic to expect to prevent all design verification test failures, as it is often the first time sufficient devices are available to test significant sample sizes or to perform pre-conditioning, such as accelerated aging or sterilization. However, the following methods, implemented as part of the planned verification activities, can significantly reduce the occurrence of design verification test failures.

6

1. Trace Requirements. The tracing of requirements through all stages of the device development is the single most effective tool for ensuring that personnel designing the device and personnel designing verification tests are working from the same set of criteria.

2. Design For Test. Often, if a requirement is not easily tested, it is not completely understood. Establishing exactly how a requirement will be verified often goes hand in hand with characterizing the design features and functions necessary to meet the requirement .

3. Treat your test tools as a sub system of your device. The planning process for design verification testing should start as device requirements are defined. Identify the requirements for test methods and tools at the same time that you analyze requirements to ensure that they are verifiable. Then treat the development of these methods and tools as if they are a sub-system of your device. Integrate test development into the overall device development schedule so that the tools are ready to assist engineering characterization testing during design.

4. Perform and document validation of test methods and tools. Properly validating the methods and tools that are used to simulate operating conditions and measure device performance is crucial to good test design. Also, appropriate validation of test methods is an expected part of design verification documentation.

5. Perform Dry runs of design verification test procedures. When the time comes to actually start design verification testing you should be confident that your device will pass. This confidence comes from adequate preparation. This includes performing dry runs of high risk methods, such as those which use a newly written or modified test protocols or test device features that have not previously been fully tested.

7

References

1. U.S. Food and Drug Administration, 2013 Annual FDA Medical Device Quality System Data: FDA Form 483 Observations and Warning Letter Citations, <http://xavierhealth.org/wp-content/ uploads/7.0-Bracket_Geniusz_Dastoli_FDA-Investigator-Insights.pdf>, accessed on June 16, 2014.

2. U.S. Food and Drug Administration, Food and Drug Administration (FDA) Medical Device 2012 Quality System Data: FDA Inspectional Observations (Form 483) and Warning Letter Citations, <http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobac co/CDRH/CDRHTransparency/UCM395850.pdf>, accessed on June 16, 2014.

3. U.S. Food and Drug Administration, FDA Medical Device 2011 Quality System Data: FDA Form 483 Observations and Warning Letter Citations, <http://www.fda.gov/downloads/AboutFDA/ CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHTransparency/UCM345510. pdf>, accessed on June 16, 2014.

4. U.S. Food and Drug Administration, FDA Medical Device 2010 Quality System Data: Analysis of 2010 Warning Letter Cites, <http://www.fda.gov/downloads/AboutFDA/CentersOffices/Officeof MedicalProductsandTobacco/CDRH/CDRHTransparency/UCM256354.pdf>, accessed on June 16, 2014.

Thinking About Outsourcing Your Design Verification Testing???Make Sure To Consider These Things!

1. Is your test service provider capable of designing your product? If not, they may lack the knowledge to effectively and efficiently design your tests. Expect to provide the oversight necessary to bridge this gap.

2. Is your test service provider prepared to offer customized solutions? Or are they looking to utilize their existing test tools and procedures? When testing to recognized standards, consistency and repeatability are virtues. But if you are looking to outsource customized testing, individualized attention is required. Beware of providers looking to adjust your test to the tools they have.

3. What are all the costs? Cost and schedule are always important issues, but also consider the hidden costs of inadequate test designs and rework.

About MPR

our costs are scope-driven so the basis is clear

we excel when others say "it can't be done"

we don't just design the device, we design the product ecosystem

we have experts in design, development and testing under one roof

MPR Associates320 King Street, Alexandria VA 22314

703-519-0523pd@mpr.com

www.breakthroughproducts.com