RJT_HealthPack07

8/14/2019 RJT_HealthPack07

http://slidepdf.com/reader/full/rjthealthpack07 1/39

TEST METHODTEST METHOD

VALIDATIONVALIDATION

Randall J. Troutman CPP

Senior Engineering Manager

Oliver Medical



Presentation Disclaimer

Oliver MedicalOliver Medical is devoted to building awareness whileis devoted to building awareness while

enhancing educational value among medical packagingenhancing educational value among medical packagingprofessionals through seminars such as this.professionals through seminars such as this.

The views, topics, and suggestions discussed today areThe views, topics, and suggestions discussed today arefor educational purposes only and therefore should notfor educational purposes only and therefore should not

be considered the last word on these issues.be considered the last word on these issues.

It is the sole responsibility of the attendee and his or herIt is the sole responsibility of the attendee and his or hercompany to determine the suitability of informationcompany to determine the suitability of information

presented today. As well as to investigate all otherpresented today. As well as to investigate all othersources of information in regards to application of thesources of information in regards to application of thecontent.content.



Presentation Overview

• Background

• ASTM Guidance / Definitions

• ASTM F88-05 (overview)

• ASTM F1929-98 (overview)

• Getting Started in your lab

• References



What is all the buzz?

• FDA “hot button” issue -

– Recent release of revised 11607 documents have causedconfusion in regards to listed test methods

– Test method validation statement in both parts• Part 1 - Clause 4.4 Test Methods

• Part 2 - Clause 4.3 Test Methods

– This statement was included in the previous revision

• ISO 11607 Clause 4.3.2



11607 part 14.4 Test Methods

4.4.1 All test methods used to show compliance with this part of ISO 11607 shall be

validated and documented.

NOTE Annex B contains a list of suitable test methods.

4.4.2 The test method validation shall demonstrate the suitability of the method used. The

following elements shall be included:

- establishment of a rationale for the selection of the appropriate test for the packaging system;

- establishment of acceptance criteria;

NOTE Pass/fail is a type of criteria;

- determination of test method repeatability;

- determination of test method reproducibility; and

- establishment of test method sensitivity for integrity tests.



11607 part 24.3 Test Methods

4.3.1 All test methods used to show compliance with this part of ISO 11607 shall be

validated and documented.

NOTE Annex B in ISO 11607-1 contains a list of suitable test methods.

4.3.2 The test method validation shall demonstrate the suitability of the method used. The

following elements shall be included:

- establishment of a rationale for the selection of the appropriate test for the packaging system;

- establishment of acceptance criteria;

NOTE Pass/fail is a type of criteria;

- determination of test method repeatability;

- determination of test method reproducibility; and

- establishment of test method sensitivity for integrity tests.



11607 part 1 – Annex B

• What does the list include?

– Standard guides

– Standard practices

– Standard test methods• Including some that can not be validated

• From various organizations…..



11607 part 1 – Annex B

• Includes standards from the following organizations;

– ASTM (Examples – peel, dye, bubble emission, etc.)

– ISO (Examples – air permeance, biological evaluation, etc.)

– EN (Example – sterilization wrap requirements and test methods)

– USP (Examples – Biological reactivity test, in vitro)

– TAPPI (Example – dirt in paper)

– DIN (Example – testing of paper board – bending stiffness)



General DefinitionsValidation1

<general> confirmation by examination and provision of objective

evidence that the particular requirement for a specific intended usecan be consistently fulfilled

Verification2

confirmation by examination and provision of objective evidence that

specified requirements have been fulfilled

Qualificationreferred to as the process of events used to establish feasibility of a

concept, process, or method

1. 11607 part 1

2. 21 CFR Part 820



ASTM GuidanceATSM E 177

Standard Practice for Use of Terms

Precision and Bias in ASTM Test Methods

ATSM E 1169

Standard Guide for Conducting Ruggedness Tests

ATSM E 691

Standard Practice for Conducting an Inter laboratory Study

to Determine the Precision of a Test Method



ASTM Guidance cont.ASTM E 2282

Standard Guide for Defining the

Test Result of a Test Method

ATSM E 1488

Standard Guide for Statistical Procedures to Use in

Developing and Applying Test Methods

ATSM E 456

Standard Terminology Relating to Quality and Statistics



ASTM E 177

Standard Practice for Use of Terms the

Precision and Bias in ASTM Test Methods

– Purpose is to provide a basis for;

• Precision & Bias statements (for quantitative methods)

• Describes methods of expressing precision and bias

• Gives examples of how P&B statements may be written



Definitionsprecision

the closeness of agreement among test results obtained under

prescribed conditions

repeatability

addresses variability between independent test results gatheredfrom within a single laboratory (otherwise known as intralaboratorytesting)

reproducibility

addresses variability among single test results gathered fromdifferent laboratories (otherwise known as inter laboratory testing)



Definitionsbias

a generic concept related to a consistent or systematic difference

between a set of test results from the process and an acceptedreference value of the property being measured

accuracy

a generic concept of exactness related to the closeness ofagreement between the average of one or more test results and anaccepted reference value

– Accuracy should be used when comparing test results, not in termsof the output result generated from execution of test a process. Butas - is this test result accurate, verses what is the accuracy of theseresults compared to previously generated results.



ASTM E 1169

Standard Guide for

Conducting Ruggedness Tests


• Finding the variables that influence measurements

– experimental factors using Plackett-Burman designs

• Determining how closely these factors need to be controlled

– The test does not determine optimum conditions for

the test method



Definitionsruggedness (robustness)

determination of variables associated with the performance of a testmethod that is ultimately critical to the accuracy of the precision

statistical control

point when the test results obtained vary in a predictable manner,showing no unassignable trends, cycles, abrupt changes, excess

scatter, or other unpredictable variations as determined by

application of appropriate statistical methods



ASTM E 691

Standard Practice for Conducting an InterlaboratoryStudy to Determine the Precision of a Test Method

– Purpose is to provide a basis for;• Addressing inherent variability in test methods

– Includes (a.) Operator, (b.) equipment used, (c.) calibrationof equipment, (d.) environment (temp, humidity, & etc.)

• Developing protocol w/ details

– Calibration, specific methods of testing, etc.

• Sample preparation – Sample preparation tips, including conditioning

– Statistically valid sampling plans

• Statistical evaluation methods



ASTM E 2282

Standard Practice for

Defining the Test Result of a Test Method


• Identifying the elements that comprise the testresults of a test method and to illustrate how

these elements combine into the test result.



Definitions

test method

a definitive procedure for the identification, measurement and evaluation of one ormore qualities, characteristics, or properties of a material, product, system, or servicethat produces a test result

observation or observed value

the most elemental single reading or corrected reading obtained in the process of

making a measurement

test determination

(1) the process of calculating from one or more observations a property of a singletest specimen, or (2) the value obtained from the process

test result

the value obtained by carrying out the complete protocol of the test method once,being either a single test determination or a specified combination of a number of testdeterminations





ASTM E 456

Standard Terminology Relating to Quality and

Statistics


• Terminology used in quality and statistical based ASTMmethods in which standard definitions appear

– attribute data: observed values or determinations which

indicate the presence of absence of specific characteristics.

– variables data: measurements which vary and may take anyof a specified set of numerical values.



Formal Process to Evaluate

• As taken from 11607;

– Suitability of particular test method

– Robustness (ruggedness)

– Precision and / or bias – Repeatability (within lab precision)

– Reproducibility (between lab precision)

– Range and / or Sensitivity of Detection



Document the Exercise

• The engineering exercise should include:

– Protocol• Objective / Scope

• Sampling plan, w/ statistical rationale

• Predetermined acceptance criteria

– Report w/ supporting data

• Demonstrates all requirements have been met – Statement of validation / successful execution

• Includes “proof of work”



Applying the process….

• ASTM F88-05 – Quantitative test method application

– Statistical comparison of test results w/ controlled variation

• ASTM F1929-98

– Qualitative test method application

– Determining sensitivity levels for pass / fail tests

• “In house” test method validation – Modified standard test method for use by MDM

– Internal test method developed by MDM



1. Scope

1.1 method covers the measurement of the strengthof seal in flexible materials

1.2 method may be conducted on seals between a

flexible and rigid material1.3 the seals tested may be from any source,

laboratory, or commercial

1.4 method measures the force required to separatea test strip of material containing a seal. Also

identifies the mode of specimen failure



Conducting a Test Method Evaluation

• Evaluate / Read the Test Method

– Scope meets intended use

– Understand the sources of variation

• Material combinations – Flexible vs. rigid application

• Number and types of equipment involved

– Equipment calibration details» What, how, and when

• Test sample conditioning




• Conduct an evaluation to expose how these

factors affect the test result

– “ruggedness test”

• Objective is to evaluate how the test method variablesaffect the test determination

• The results of this test will provide you with an

accurate “sample” to use in the repeatabilityphase of your evaluation




• Conduct “reproducibility” studies with other pieces of likeequipment in other locations using “samples” proven in

the ruggedness and repeatability studies

• Use a statistician to help determine the predetermined

“Precision” acceptance criteria for the method in your lab

• Based on the data compiled within your R&R study youmay use a one-way analysis of variance (ASTM E 691) to

evaluate the data

• Compare your data to the “Precision” data recorded inthe actual standard (Section 11)






• ASTM F1929-98

– Qualitative test method application – Determining sensitivity levels for pass / fail tests





1. Scope1.1 defines materials and a procedure that will

detect and locate a leak equal or greater

than a channel of 0.002 in.1.2 intended for use on packages with an

edge seal formed between a transparent

film and a porous material

1.3 requires a contrasting dye penetrant

solution be used




• Evaluate / Read the Test Method

– Scope meets intended use

• Objective is to determine if the method that

you conduct internally meets the sensitivityclaim in the standardized method

– For this particular method the sensitivity level is

a channel down to 0.002 in. in seal




• Overall evaluation should consider;

– Sufficient sample sizes

– Variation in dye mixture methods

– Variation in sourced ingredients – Variation in exposure times

• 5 seconds vs. 20 seconds

– Variation in the actual procedure / process

– Variation in voids detected




• While considering those variables you

should prove the sensitivity of the method – Establish a range, w/ control limits

• Example range –

» Control (no void)

» 0.001” void (min limit)

» 0.002” void (detectable)








• ASTM F1929-98

– Qualitative test method application – Determining sensitivity levels for pass / fail tests







Getting started in your lab….

• Review the standard test methods that you

currently execute within your company – How was the standardized method evaluated?

• Review the ASTM standards mentioned in this

presentation for guidance

• Develop a plan to “validate” (evaluate) your

methods separately and be prepared to employvarying strategies to fulfill the requirement



Thank You

Randall Troutman CPP

Oliver Medical800.253.3893 ext. 7914

616.456.7711 ext. 7914

[email protected]

www.olivermedical.com





Additional Research

• Test Method Validation Techniques

– Analytical Chemistry Test Methods

– Toxilogical Test Methods

– Forensic Test Methods

Documents

RJT_HealthPack07