1 International Federation for Consulting GmbH IFC Validation by Dr. Michael Scheutwinkel

1

International Federation forConsulting GmbH IFC

ValidationValidation

by

Dr. Michael Scheutwinkel

2

• Governmental regulations

• Books ….

• Recommendations of associations

3

ReferencesReferences

EURACHEM GuideThe Fitness for Purpose of Analytical Methods -

A Laboratory Guide to Method Validationand Related Topics (12/1998)

www.eurachem.com

EURACHEM GuideThe Fitness for Purpose of Analytical Methods -

A Laboratory Guide to Method Validationand Related Topics (12/1998)

www.eurachem.com

CITAC / EURACHEM GuideGuide to Quality in Analytical Chemistry

An Aid to Accreditation (2002)www.eurachem.com

CITAC / EURACHEM GuideGuide to Quality in Analytical Chemistry

An Aid to Accreditation (2002)www.eurachem.com

4

5

Which chapter of ISO/IEC 17025 is dedicated to validation?

6

Do not ask me, I am

only a simple guy

from Alemania!

Do not ask me, I am

only a simple guy

from Alemania!

7

Validation is the confirmation by examination

and the provision of objective evidence that

the particular requirements for a specific

intended use are fulfilled.

Validation (ISO/IEC 17025)

Definition in clause 5.4.5.1 of ISO/IEC 17025

….fit for purpose….fit for purpose

8

Validation of an analytical test method

Which analytes can be determined in which matrix in the presence of which interferences?

Within these conditions what levels of precision and accuracy can be achieved?

Which analytes can be determined in which matrix in the presence of which interferences?

Within these conditions what levels of precision and accuracy can be achieved?

Validation is the process of establishing the

• Performance characteristics

• Limitations of a method

• Identification of the influences which may change these characteristics and to what extent.

Validation is the process of establishing the

• Performance characteristics

• Limitations of a method

• Identification of the influences which may change these characteristics and to what extent.

9

Equipment and method valdiation

… different items… different items

Method validationMethod validation

Soft and hardwareSoft and hardware

Equipment validationEquipment validation

IQIQ PQPQ

OQOQDQDQ

10

Validation in order to analytical method

ValidationValidation

Modified standardized andin-house methods

Modified standardized andin-house methods

VerificationVerification

Standardized-methodsStandardized-methods

11

Verification of an analytical test method

The tools are the same as for validation,

the laboratory needs only to verify that

the documented performance characteristics

can be met (e.g. accuracy).

The tools are the same as for validation,

the laboratory needs only to verify that

the documented performance characteristics

can be met (e.g. accuracy).

Verification is the process to demonstrate the

competence in test performance

of an already validated standardized test method.

Verification is the process to demonstrate the

competence in test performance

of an already validated standardized test method.

12

When should be validated?

• Changes in established methods• New method developed for a particular problem• Established method used in different

laboratories, different equipment or different staff

• Out-of-control situations within internal quality assurance

• Non-successful participation in PTs• Demonstration of equivalence between two

methods (e.g. a rapid new test against a standard method)

• Changes in established methods• New method developed for a particular problem• Established method used in different

laboratories, different equipment or different staff

• Out-of-control situations within internal quality assurance

• Non-successful participation in PTs• Demonstration of equivalence between two

methods (e.g. a rapid new test against a standard method)

13

Validation Strategy

Characterizationof the test methodCharacterization

of the test method

Comparison of method

charcacteristicswith

requirements

Comparison of method

charcacteristicswith

requirements

Verification to give

proof that the requirements are

fulfilled

Verification to give

proof that the requirements are

fulfilled

BB CC DD

Specific test requirements defined by customer or marketSpecific test requirements defined by customer or market

AA

14

Description of test method (clause 5.4.4)

Identification / ScopeIdentification / Scope Parameters in which matrixParameters in which matrix

Reagents and materials, CRMsReagents and materials, CRMsEquipment and environmental

conditionsEquipment and environmental

conditions

CalibrationCalibrationSampling and

sample preparationSampling and

sample preparation

Test performanceincluding safety instructions

Test performanceincluding safety instructions

Quality assuranceQuality assurance

Documentation , reportingevaluation criteria

Documentation , reportingevaluation criteria

Uncertainty or procedure forits estimation

Uncertainty or procedure forits estimation

15

Please show us the way…

Please show us the way…

16

Tools

Determination of method characteristics

Characteristics of calibration


Characteristics of the range


Characteristics of accuracy


Selectivity orSpecifity


Robustness /Ruggedness


17

Tools


Characteristics of calibration Linearity Sensitivity


Perform morepoint calibration Perform more

point calibration Correlation coefficient:r² > 0.99 ??

Correlation coefficient:r² > 0.99 ??

Linearity is defined by the correlation coefficient

Sensitivity is defined by the slope of the calibration graph

18

Commission Decision 2002/657/EC

19

Calibration

• Calibration values should be applied within the working range.

• Lowest point should be the limit of quantification.

• For calibration certified standard solutions shall be used.• In addition, also standard solutions have to be added to blank materials, analysed over all process steps.

• For linear calibration functions at least five concentrations are

necessary.

• Test calibration functions for linearity and highlight failing linearity.

Validation of testing methods

20

Sensitivity (resolution)

What is sensitivity?

It is the difference in an analytical concentration that corresponds to the smallest difference of a signal in a method which is still detectable.

Sensitivity can be extracted from the calibration curve or defined by using samples with different concentrations.


21

Linearity

Definition based on samples with varying concentrations and the calculation of the regression of results.

Signal and concentration do not have to be fully related.

Five standards suffice to produce a calibration curve if linearity is o.k.

More standards are necessary once the linearity is unsatisfactory.

Examine repeat samples and standards over a particular working range to experience whether a reliable line can be drawn between proof and detection limit.


22

Homogeneity of variances

Characteristic values for the comparison

of standard variations for differing

concentration.

In case of great differences between

standard deviations

(in-homogeneity of variances),

the working range needs to be split.

23

Tools





Limit of detection Limit of quantification









24

Working range

Finding out the appropriate working range using

• different matrices

• varying concentrations.

Concentration range within the achievement of

acceptable

• accuracy and

• precision is possible.

In general the working range is broader than the linear range.


25

Prerequisites for calibration

Standards must be faultless.

Precision has to be similar throughout the entire working range.

The model function is applicable: either linear or curved.

Errors only may occur randomly within signals.

Errors have to follow the normal distribution.


26

LODCharacteristics of the range

Limit of detectionLimit of detection

LODLOD xLOD = 3 . sL / b (Blank value method)

xLOD = 4 . sxo (Calibration function method)

xLd = Limit of detectionsL = Standard deviation of the blanks

Sxo = Standard deviation of the calibration function

b = Slope of calibration function

Various conventionsVarious

conventionsXLOD = Mean of blanks + 3 x SL

bycalculation

bycalculation

27

LOQ


Limit of quantificationLimit of quantificationby

calculationby

calculation

xLOQ = 9 . sL/ b (Blank value method)

xLOQ = 11 . sxo (Calibration function method)

xLOQ = Limit of quantificationsL = Standard deviation of the blanks

Sxo = Standard deviation of the calibration function

b = Slope of calibration function

Various conventionsVarious

conventionsXLOQ = 5 or 6 or 10 x SL

LOQ LOQ

28

Commission Decision 2002/657/EC

29

LOD


Limit of detectionLimit of detection bysignal to

noise ratio

bysignal to

noise ratio

From 5 : 1 to 10 : 1From 5 : 1 to 10 : 1

Limit of quantificationLimit of quantification

From 2 : 1 to 5 : 1From 2 : 1 to 5 : 1

30

Effect of peak shape on LOD / LOQ

31

Tools










Robustness or Ruggedness


32

Selectivity / Specifity

SelectivitySpecifity

SelectivitySpecifity

Use ofantibodiesUse of

antibodies

Extent to which particular analytes can be determined in complex mixtures.

A method which is selective for an analyte is said to be specific.

Extent to which particular analytes can be determined in complex mixtures.

A method which is selective for an analyte is said to be specific.

Selective detectors in instrumental

analysis

Selective detectors in instrumental

analysis

Selective media in microbiology

Selective media in microbiologyCross-reactions ?Cross-reactions ?

33

Selectivity

• A selective method gives correct results for all interesting analytes whereas a specific test method gives correct results for the interesting analyte whereas other analytes might interfere each other.

The difference between selectivity and specificity

Specifity

• Outlines the analytical extent to which an analytical substance or substance group can be determined without interference from sample related components.

No problem, usually both words are mixed …No problem, usually both words are mixed …

34

Selectivity

• Outlines the extent to which an analytical substance can be determined without interference from other components.

• Method selective to one analytical substance specific.

• Assess selectivity while application to pure solution up to complex matrices.

• Document disturbances and restrictions of the method.


35

Tools












36

Robustness

• Part of the routine.

• Learn from other analytical institutes, as the method-establishing laboratory will have proved robustness before publishing a new method.

Degree of proneness of a method to conscious and unconscious alterations of a working instruction.


37

Robustness

Sensitivity degree of the test method against small deviations in experimental conditions.

Examples:

• Times within process steps

• Environmental conditions (e.g. temperatures)

• Minor process changes (e.g. pH, flow rates in

HPLC)

Sensitivity degree of the test method against small deviations in experimental conditions.

Examples:

• Times within process steps

• Environmental conditions (e.g. temperatures)

• Minor process changes (e.g. pH, flow rates in

HPLC)

38

Ruggedness, according to USP

Ruggedness is the degree of reproducibility obtained under a variety of conditions, expressed as relative SD, e.g.:

• different analysts,• different equipment,• different trade marks of reagents,• etc.

Ruggedness is the degree of reproducibility obtained under a variety of conditions, expressed as relative SD, e.g.:

• different analysts,• different equipment,• different trade marks of reagents,• etc.

Internal reproducibility

39

Tools






Characteristics

of accuracy: Correctness Uncertainty

Precision (Repeatability/

Recovery) Reproducibility

Characteristics of accuracy: Correctness Uncertainty

Precision (Repeatability/

Recovery) Reproducibility





40

Accuracy = Precision and correctness I

AccuracyAccuracyExactness of an

analytical methodExactness of an

analytical method

PrecisionPrecision

Degree ofrepeatability of ananalytical method

Degree ofrepeatability of ananalytical method

UncertaintyUncertaintyReproducibilityReproducibility

Recovery /Bias

Recovery /Bias

RepeatabilityRepeatability

Correctness /Trueness

Correctness /Trueness

Systematic error /Correct value

Systematic error /Correct value

41

precise

and

correct

not precise

but correct

precise but wrong

not precise and wrong

Random and systematic deviations

42


Correct-ness

Correct-ness

PrecisionPrecision

Measure of correctness, covering systematic and non-systematic mistakes (use of CRM, comparison

to a well characterized method)

Measure of correctness, covering systematic and non-systematic mistakes (use of CRM, comparison

to a well characterized method)

Measure of the degree of repeatability,covering systematic mistakes

Measure of the degree of repeatability,covering systematic mistakes

Repeatability: one laboratory;Better: internal reproducibilityRepeatability: one laboratory;Better: internal reproducibility

Reproducibility: more than one laboratoryReproducibility: more than one laboratory

Accuracy = Precision and correctness II

43

Repeatability

Measurement of reference materials.

Fortification of a blank sample with the analytical substance.

Comparison to a reference procedure.

Participation in a co-operative (interlaboratory) test in which the “true content” is known.

Use at least one of the following procedures for determination:


44

Precision

Information on accuracy of two coinciding and independent analytical results.

Precision depends on the concentration of the analyte.

Repeatability is part of the precision with respect to repeated measurements:

– same material,

– same method,

– same analyst,

– same laboratory and

– short time-span in-between the analyses.


45

Correctness / Trueness

Use the one of the following methods to assure correctness:

• Analysis of certified reference material.

• Participation in interlaboratory comparisons.

• Comparison to a known procedure.

If not available:

• Document any data that prove correctness.

• Make an approximation as a first approach to correctness.• Purchase reference material.• Use a similar method in parallel.


46

Recovery

• Performed by addition of the analytical substance to the matrix.

• A matrix shall be free of the analytical substance.

• If not the sample will be enriched with a weak concentration of the analytical sample.

• Or a simulated matrix will be used.

• Recovery can be determined only when the analytical sample is available in pure form.

• List the recovery rate (in %) and the standard deviation when recovery rates are constant.

• Otherwise issue recovery rate as a function.


47

Frame conditions for the determination of the recovery rate

Frame conditions for the determination of the recovery rate

The analytical substance needs to be added in that form in which it occurs in nature.

The samples requires good homogenisation.

The native content shall be below the determination limit.


48

Reproducibility

Statistically

Comparative standard deviation Replicate standard deviation

Through participation in proficiency testing schemes

or participation in interlaboratory comparisons.


internallyinternally

externallyexternally

Prove reproducibilityProve reproducibility

49

Tools









Correctness Uncertainty Precision

(Repeatability/Recovery)

Reproducibility


Correctness Uncertainty Precision

(Repeatability/Recovery)

Reproducibility



Robustness orRuggedness

Robustness orRuggedness

50

Classification of the test method

• Analysis of contaminants (e.g. pesticides, heavy metals, food additives)• Analysis of food constituents (salt and minerals)

• Methods which are determining true values

• Test methods which have to be calibrated with instruments

• Physical measurement (pH value, conductivity)

• Qualitative determinations

• Microbiological methods

Distinguish between


51


Parameter Analysis of

trace

contaminants

Analysis of

constituents

Conventional

methods

Methods not

necessary to

calibrate

Physical

methods

Qualitative

methods

Bacteriological

methods

Description of

test method

X X X X X X X

Calibration X X X X

Working range X X X X X X X

Detection limit X X X X

Determination

limit

X X

Recovery X X X

Repeatability X X X X X X

Correctness X X X X

Specificity X X X X X

Reproducibility if possible if possible if possible if possible if possible if possible if possible

Parameters for validation

52

Common validation needs of known methods

Method Accuracy Precision Specificity/Selectivity

Linearity Detection/ Determinationlimits

Robustness

AAS, AES, ICP, RF X X (X) X X X

Polarography/ Voltametry X X X X X

Titration X X X

Water assay X X X X

Chromatography/ CapillaryElectrophoresis

X X X X X

Spectroscopy X X X X X

Release X X X X X

TOC X X X X

Enzyme Immuno Assays X X X X X X


53

The Analysis of Standard Reference Materials

• Generally accepted method for validation.

• Such standards are provided with guarantee on the market.

• It may be necessary to contract the preparation of a unique sample in particular matrix in order to utilise this procedure for method validation.

• The analyst must demonstrate that the method provides accuracy and precision.

Procedures for method validation

54

The participation in PT-schemes or in a laboratory Collaborative Study

The most widely accepted procedure for method validation.

Serious practical draw backs: Costly and time consuming Effort in co-ordination Shipping of samples and data Statistical analysis and interpretation of results.

This method is rarely used for the first description of a method in the literature.


55

Comparison with a currently accepted method

• Usually done by one analyst

• or two using a split-sample.

• using results from the currently accepted method for verification.

Agreement suggests validation.

Disagreement could also suggest that the currently accepted method is invalid.

• In such case, another procedure has to be employed for the method validation.

• The more samples are analysed and the wider the range of concentration the higher the credibility of the validation method.


56

The Zero-Blind Method

• One analyst.

• Using samples at known levels of analyte to demonstrate recovery, accuracy and precision.

• Fast, simple and useful but

it leads to subjective results.

• Suitable for a first approximation requiring minimal time, manpower, samples and cost.

• In general, a good start for the overall validation process.

If this methods fails there is no reason to proceed with further validation of the method.


57

The Single-Blind Method

• One analyst at the start.

• Samples are given to a 2nd analyst to whom levels of analyte are unknown.

• Results are compiled and compared by the 1st analyst.

• Comparison by the first analyst makes objectivity questionable.

• This method still is biased on behalf of the 1st analyst.

• Suitable after the zero-blind method has been successful and before additional analysts or the management will be involved.


58

The Double-Blind Method

• Three analysts.

• 1st analyst prepares samples at known levels.

• 2nd analyst does the actual analysis.

• 3rd analyst (or administrator) compares both data from the first two analysts.

• Only the 3rd analyst has access to these data.

Most effective approach to objectiveness


59

Clearance of validated test procedures

Validation results

Customer requirements

Quality Manager


60


Decision criteria for the extent of a validation

Technical feasibility and staff disposition?

Analytical viewpoints Measuring principle, method complexity?

Occurrence of the sample, wide-spread?

Risk potential for client/ company?

Political decisions Actual aim of the validation?

Consideration of normative

and other requirements?

facts

super-ordinated criteria

61

Extent of validation in analytical sciences

Purpose

• Suitability of the method

• „Marketability“ of the method

Analytical sensibility

Costs

Requirements from the outside


62

Validation is always a balance between costs, risks and technical possibilities.

Validation is always a balance between costs, risks and technical possibilities.

The validation shall be completed by a statement by the laboratory that the

method is fit for the intended use.

ISO/IEC 17025 states:ISO/IEC 17025 states:

63

Validation is always a balance between

costs, risks and technical possibilities.

Validation is always a balance between

costs, risks and technical possibilities.

64

Self developed software or excel sheets have to be validated.

Self developed software or excel sheets have to be validated.

Records for that must be available.

Do not forget:Do not forget:

65

Muito

obrigado!

66

67

Bring them together ….Bring them together ….

68

69

70

71

Fuente de incertidumbre SímboloValor

± [Unidades]

Distribución de probabilidad

DivisorIncertidum-bre estándar

[unidades]

Calibración de la balance uEst + 0.003 g Tipo B / rect 0.7% 0.4%

Calibración de la estufa uPip 0 % Tipo B / rect

Tiempo uBurInd 0 %

Repetibilidad uRep + 0.1 % Tipo B 0.1%

Trabajo en grupos ”Humedada en Alimentos”

33

ucombinada = 3.43 % Uexpandida = 7 %

33

72

IfEP Workshop: Proficiency Testing - Background and general aspects

Why proficiency testing?

PT schemes are just one important tool in quality control and quality management.

QM

Participating in Proficiency Tests

Use of certified reference material

Validation of methods

Documents

1 International Federation for Consulting GmbH IFC Validation by Dr. Michael Scheutwinkel