Roles and responsibilities in traceability implementation · Mario Plebani, Laura Sciacovelli, Daniela Bernardi, Ada Aita, Giorgia Antonelli, Andrea Padoan The role of external quality

Roles and responsibilities in Roles and responsibilities in

traceabilitytraceability

implementationimplementation

Federica BragaFederica Braga

University of Milan Medical School University of Milan Medical School

12th International 12th International ScientificScientific Meeting Meeting

STANDARDIZATION IN LABORATORY MEDICINE STANDARDIZATION IN LABORATORY MEDICINE

AND PATIENT SAFETYAND PATIENT SAFETY

Laboratory measurement paradigm:Laboratory measurement paradigm:

•• Assays that claim to measure the same Assays that claim to measure the same analyteanalyte

should give equivalent measurement results (for should give equivalent measurement results (for

long term and within clinically meaningful limits)long term and within clinically meaningful limits)

•• Measurement results should be independent of:Measurement results should be independent of:

−− TimeTime

−− Location/laboratoryLocation/laboratory

−− Assay systemAssay system

Laboratory results should be Laboratory results should be

equivalent no matter where they equivalent no matter where they

are performedare performed

ISO/EN 17511 ISO/EN 17511 -- MeasurementMeasurement of of quantitiesquantities in in samplessamples of of biologicalbiological originorigin -- MetrologicalMetrological traceabilitytraceability of of

valuesvalues assignedassigned toto calibratorscalibrators and control and control materialsmaterials..

Objective of traceability Objective of traceability

implementation:implementation:

to enable the results obtained by the to enable the results obtained by the

calibrated routine procedure to be calibrated routine procedure to be

expressed in terms of the values obtained at expressed in terms of the values obtained at

the highest available level of the calibration the highest available level of the calibration

hierarchyhierarchy..

EU 98/79/ECEU 98/79/EC--IVD IVD DirectiveDirective

To become To become equivalent for long termequivalent for long term, ,

results must be traceable to higherresults must be traceable to higher--

order referencesorder references

requires manufacturers to ensure requires manufacturers to ensure

traceability of their analytical systems to traceability of their analytical systems to

recognized higher order referencesrecognized higher order references

REGULATION (EU) 2017/746REGULATION (EU) 2017/746 OF THE EUROPEAN PARLIAMENT OF THE EUROPEAN PARLIAMENT

AND OF THE COUNCIL of 5 April 2017 on AND OF THE COUNCIL of 5 April 2017 on in vitro in vitro diagnostic diagnostic

medical devices and medical devices and repealing Directive 98/79/ECrepealing Directive 98/79/EC and and

Commission Decision 2010/227/EUCommission Decision 2010/227/EU

•• Establishment of a calibration hierarchy Establishment of a calibration hierarchy

starting from the unequivocal definition of starting from the unequivocal definition of

the the measurandmeasurand

•• Elimination of measurement biasElimination of measurement bias

•• Adequate estimation of measurement Adequate estimation of measurement

uncertaintyuncertainty

Basic requirements to Basic requirements to

establish traceabilityestablish traceability

Primary Ref. ProcedurePrimary Ref. Procedure

Secondary Ref. Secondary Ref.

ProcedureProcedure

ManufacturerManufacturer’’ss

Internal ProcedureInternal Procedure

EndEnd--useruser’’s Routines Routine

ProcedureProcedure

Primary Reference MaterialPrimary Reference Material

(e.g. pure (e.g. pure analyteanalyte))

Secondary Ref. MaterialSecondary Ref. Material

(matrix(matrix--based)based)

ManufacturerManufacturer’’s Calibrators Calibrator

Routine SampleRoutine Sample

Test ResultTest Result

SI UnitsSI Units

Tra

cea

bility

Tra

cea

bility

*Adapted from ISO 17511

Reference Measurement SystemReference Measurement SystemU

nce

rtain

tyU

nce

rtain

ty

2. Diagnostic manufacturers:

Implement suitable measuring

systems (platform, reagents,

calibrators, controls) fulfilling the

above established goals

2. 2. DiagnosticDiagnostic manufacturersmanufacturers::

ImplementImplement suitablesuitable measuringmeasuring

systemssystems ((platformplatform, , reagentsreagents, ,

calibratorscalibrators, , controlscontrols) ) fulfillingfulfilling the the

aboveabove establishedestablished goalsgoals

PanteghiniPanteghini M,M,

ClinClin ChemChem Lab Lab MedMed

2010;48:72010;48:7

Roles and responsibilities in traceability implementationRoles and responsibilities in traceability implementation

3. End users (clinical laboratories):3. End users (clinical laboratories):

Survey assay and laboratory performance through:Survey assay and laboratory performance through:

-- IQC component I: testing system controls to confirm and verify IQC component I: testing system controls to confirm and verify manufacturermanufacturer’’s declared performance s declared performance

(CE marked (CE marked –– virtually unbiased)virtually unbiased)

-- IQC component IIIQC component II: : estimation of the MEASUREMENT UNCERTAINTY due to the random effeestimation of the MEASUREMENT UNCERTAINTY due to the random effects and cts and

calculation of uncertainty of laboratory measurements (calculation of uncertainty of laboratory measurements (UUcalcal + + UUrandomrandom))

-- EQA: true value in commutable materials for defining measuremenEQA: true value in commutable materials for defining measurement error of laboratoryt error of laboratory

1. Profession (e.g., IFCC, JCTLM):

Define analytical objectives:

reference measurement systems

(traceability chain) and associated

clinically acceptable

uncertainty (fit for purpose)

1. 1. ProfessionProfession (e.g., IFCC, JCTLM):(e.g., IFCC, JCTLM):

Define analytical objectives: Define analytical objectives:

reference measurement systems reference measurement systems

(traceability chain) and associated (traceability chain) and associated

clinically acceptable clinically acceptable

uncertainty (fit for purpose)uncertainty (fit for purpose)













2010;48:72010;48:7





















1.1. Roles and responsibilities ofRoles and responsibilities of

Professional OrganizationProfessional Organization

••UnequivocalUnequivocal definitiondefinition of the of the

MEASURANDMEASURAND asas the the quantityquantity

subjectsubject toto measurementmeasurement

•• DefinitionDefinition of the of the REFERENCE REFERENCE

MEASUREMENT SYSTEMMEASUREMENT SYSTEM

•• DefinitionDefinition of of ANALYTICAL ANALYTICAL

PERFORMANCE SPECIFICATIONSPERFORMANCE SPECIFICATIONS

MEASURANDMEASURANDMEASURAND

TYPE A MEASURANDSTYPE A MEASURANDS

• Well defined Well defined

compoundscompounds

•• Traceable to SI unitsTraceable to SI units

•• Results are not Results are not

methodmethod--dependentdependent

•• Approx. 65 Approx. 65 analytesanalytes

(metabolites, (metabolites,

electrolytes)electrolytes)

TYPE B MEASURANDSTYPE B MEASURANDS

• OftenOften not well defined not well defined

(heterogeneous mixtures)(heterogeneous mixtures)

•• AnalytesAnalytes can be bound or in can be bound or in

free statefree state

•• Not traceable to SI units, Not traceable to SI units,

but to arbitrary units (e.g. but to arbitrary units (e.g.

WHO International Units)WHO International Units)

•• Immunochemical Immunochemical

procedures show inherent procedures show inherent

variability (different variability (different epitopesepitopes))

•• 400400--600 600 analytesanalytes ((tumourtumour

markers, viral antigens, markers, viral antigens,

clotting factors)clotting factors)

PanteghiniPanteghini M. M. ClinClin BiochemBiochem 2009;42:2362009;42:236

Joint Committee for Traceability in Joint Committee for Traceability in

Laboratory Medicine (JCTLM)Laboratory Medicine (JCTLM)

The WorldThe World’’s only qualitys only quality--assured database of:assured database of:

a)a) Higher Order Reference MaterialsHigher Order Reference Materials

b)b) Higher Order Reference Measurement ProceduresHigher Order Reference Measurement Procedures

c)c) Accredited Laboratory Reference Measurement Services Accredited Laboratory Reference Measurement Services

For use by (primarily):For use by (primarily):

a) IVD industry (a) IVD industry (to assist them in following the EU Directive on compliance and to assist them in following the EU Directive on compliance and

traceability of commercial systemstraceability of commercial systems))

b) Regulators (to verify that results produced by b) Regulators (to verify that results produced by IVDsIVDs are traceable to)are traceable to)

REFERENCE MEASUREMENT SYSTEMREFERENCE MEASUREMENT SYSTEMREFERENCE MEASUREMENT SYSTEM

ANALYTICAL PERFORMANCE

SPECIFICATIONS

ANALYTICAL PERFORMANCE ANALYTICAL PERFORMANCE

SPECIFICATIONSSPECIFICATIONS

Some models are better suited for certain Some models are better suited for certain

measurandsmeasurands than for others. The attention is than for others. The attention is

therefore primarily directed towards the therefore primarily directed towards the

measurandmeasurand and its biological and clinical and its biological and clinical

characteristics.characteristics.

•• Model 1: Based on the effect of analytical Model 1: Based on the effect of analytical

performance on clinical outcomeperformance on clinical outcome

•• Model 2: Based on components of biological Model 2: Based on components of biological

variation of the variation of the measurandmeasurand

•• Model 3: Based on state of the art of the Model 3: Based on state of the art of the

measurement (i.e., the highest level of measurement (i.e., the highest level of

analytical performance technically achievable)analytical performance technically achievable)

The The measurandmeasurand hashas a high a high

homeostatichomeostatic controlcontrol

NeitherNeither centralcentral diagnosticdiagnostic

rolerole nornor sufficientsufficient

homeostatichomeostatic controlcontrolThe The measurandmeasurand has a central has a central

role in diagnosis and role in diagnosis and

monitoring of a specific monitoring of a specific

disease disease

a. Done by direct outcome studies – investigating the impact of analytical performance

of the test on clinical outcome.

b. Done by indirect outcome studies – investigating the impact of analytical

performance of the test on clinical classifications or decisions and thereby on the

probability of patient outcome, e.g., by simulation analysis.

a.a. Done by direct outcome studies Done by direct outcome studies –– investigating the impact of analytical performance investigating the impact of analytical performance

of the test on clinical outcome.of the test on clinical outcome.

b. Done by indirect outcome studies b. Done by indirect outcome studies –– investigating the impact of analytical investigating the impact of analytical

performance of the test on clinical classifications or decisionsperformance of the test on clinical classifications or decisions and thereby on the and thereby on the

probability of patient outcome, e.g., by simulation analysis.probability of patient outcome, e.g., by simulation analysis.

IMPRECISION:IMPRECISION: ≤≤ 0.25 CV0.25 CVII (O)(O)

≤≤ 0.5 CV0.5 CVII (D)(D)

≤≤ 0.75 CV0.75 CVI I (M)(M)

BIAS: BIAS: < 0.125 (CV< 0.125 (CVII22 + CV+ CV

GG22))1/21/2 (O)(O)

< 0.25 (CV< 0.25 (CVII22 + CV+ CV

GG22))1/21/2 (D)(D)

< 0.375 (CV< 0.375 (CVII22 + CV+ CV

GG22))1/21/2 (M) (M)

IntraIntra--individualindividual BV: BV: CVCVII%%

InterInter--individualindividual BV: BV: CVCVGG%%

OUTCOME APPROACHOUTCOME APPROACHOUTCOME APPROACH

BIOLOGICAL VARIATION APPROACHBIOLOGICAL VARIATION APPROACHBIOLOGICAL VARIATION APPROACH

<0.75 x CV<0.75 x CVII (Minimum)(Minimum)

<0.50 x CV<0.50 x CVII ((DesirableDesirable))

<0.25 x CV<0.25 x CVII (Optimum)(Optimum)

[Note that these are goals for [Note that these are goals for random variabilityrandom variability, as , as at at

the calibrator level the systematic error (bias), in the calibrator level the systematic error (bias), in

agreement with the metrological traceability theory, agreement with the metrological traceability theory,

must be corrected if present in a non negligible amount]must be corrected if present in a non negligible amount]

<(0.75 x CV<(0.75 x CVII) x 2 (Minimum)) x 2 (Minimum)

<(0.50 x CV<(0.50 x CVII) x 2 () x 2 (DesirableDesirable))

<(0.25 x CV<(0.25 x CVII) x 2 (Optimum)) x 2 (Optimum)

kuU ×=Total budget for uncertainty (GTotal budget for uncertainty (G

UU) should be fulfilled by:) should be fulfilled by:

-- uncertainty of the measuring system employed in the uncertainty of the measuring system employed in the

individual laboratory (random uncertainty) +individual laboratory (random uncertainty) +

-- that accumulated along all the steps of metrological that accumulated along all the steps of metrological

traceability chain andtraceability chain and

-- expanded by a coverage factor (k=2 for a 95%Coverage expanded by a coverage factor (k=2 for a 95%Coverage

Interval)Interval)

APS FOR UNCERTAINTYAPS FOR UNCERTAINTY

APS FOR EXPANDEDAPS FOR EXPANDED

UNCERTAINTYUNCERTAINTY

Measuring system imprecision

Measuring system calibration

uncertainty

Individual laboratory

performance

Measurement

uncertainty

budget

Uncertainty of

references

Measurand definition

Patient result

Un

certa

inty

ClinClin BiochemBiochem 57 (2018) 757 (2018) 7--1111

Uncertainty of measurement must be defined across the entire

traceability chain,

starting with the provider of reference materials,

extending through the IVD manufacturers and their processes for

assignment of calibrator values, and

ultimately to the final result reported to clinicians by end users (i.e.

clinical laboratories).

Measuring system imprecision

Measuring system calibration

uncertainty

Individual laboratory

performance

Measurement

uncertainty

budget

Uncertainty of

references


Patient result

Un

certa

inty

ClinClin BiochemBiochem 57 (2018) 757 (2018) 7--1111

33%33%

50%50%

100%100%

To obtain that a given measurement can fulfill the established GTo obtain that a given measurement can fulfill the established GUU

when clinical samples are measured, it is necessary to when clinical samples are measured, it is necessary to define how define how

much of the Gmuch of the GUU should be used across the different steps of should be used across the different steps of

metrological traceability chainmetrological traceability chain, by establishing combined , by establishing combined

uncertainty limits at different chain levels as a proportion of uncertainty limits at different chain levels as a proportion of GGUU

Editorial

Foreword

Mauro Panteghini

Opinion Piece

Measurement uncertainty: Friend or foe?

Ilenia Infusino, Mauro Panteghini

Defining permissible limits for the combined uncertainty budget in the implementation of metrological traceability

Federica Braga, Mauro Panteghini

Performance specifications and six sigma theory: Clinical chemistry and industry compared

Wayne P. Oosterhuis, Michel J. Severens

Reviews

What information on measurement uncertainty should be communicated to clinicians, and how?

Mario Plebani, Laura Sciacovelli, Daniela Bernardi, Ada Aita, Giorgia Antonelli, Andrea Padoan

The role of external quality assessment in the verification of in vitro medical diagnostics in the traceability era

Federica Braga, Sara Pasqualetti, Mauro Panteghini

Opinion Piece

Sources and performance criteria of uncertainty of reference measurement procedures

Andrea Mosca, Renata Paleari

Deriving proper measurement uncertainty form Internal Quality Control data: An impossible mission?

Ferruccio Ceriotti

Original Research

Measurement uncertainty in laboratory reports: A tool for improving the interpretation of test results

Andrea Padoan, Laura Sciacovelli, Ada Aita, Giorgia Antonelli, Mario Plebani

Ensuring suitable quality of clinical measurements through design

Anthony Orzechowski, Victoria Petrides, Richard Scopp

Different top-down approaches to estimate measurement uncertainty of whole blood tacrolimus mass concentration values

Raül Rigo-Bonnin, Aurora Blanco-Font, Francesca Canalias

Short Communication

Random uncertainty of photometric determination of hemolysis index on the Abbott Architect c16000 platform

Elena Aloisio, Assunta Carnevale, Sara Pasqualetti, Sarah Birindelli, Alberto Dolci, Mauro Panteghini

Special Issue: Measurement Uncertainty in Medical Laboratories: Friend or Foe?Guest Editors: Federica Braga and Mauro Panteghini













2010;48:72010;48:7






















Diagnostic manufacturersDiagnostic manufacturers

••Identification of higher order metrological Identification of higher order metrological

REFERENCESREFERENCES

••Definition of a Definition of a CALIBRATION HIERARCHYCALIBRATION HIERARCHY to to

assign traceable values to their system assign traceable values to their system

calibrators calibrators

•• Estimation of combined expanded Estimation of combined expanded

MEASUREMENT UNCERTAINTYMEASUREMENT UNCERTAINTY of calibratorsof calibrators

••Verification that Verification that UNCERTAINTY GOALSUNCERTAINTY GOALS are are

fullfilledfullfilled

EU 98/79/ECEU 98/79/EC--IVD IVD

DirectiveDirective

[[FulfillmentFulfillment of of thethe EU IVD EU IVD Directive andDirective and

REGULATION (EU) 2017/746 RequirementsREGULATION (EU) 2017/746 Requirements]]

1. IDENTIFICATION OF HIGHER ORDER REFERENCES

httpshttps://www.bipm.org/://www.bipm.org/jctlmjctlm//

2. DEFINITION OF A CALIBRATION HIERARCHY TO ASSIGN

TRACEABLE VALUES TO THEIR SYSTEM CALIBRATORS

ERMERM--DA470DA470

U.S. National Reference U.S. National Reference

Preparation no. 12Preparation no. 12--0575C0575C

ERMERM--DA470k/IFCCDA470k/IFCC

ManufacturerManufacturer’’s working calibrators working calibrator

(master lot)(master lot)

ManufacturerManufacturer’’s product calibrators product calibrator

Routine sample resultRoutine sample result

value transfer protocolvalue transfer protocol

ManufacturerManufacturer’’s standing s standing

immunoassayimmunoassay

Commercial immunoassayCommercial immunoassay

value transfer value transfer

protocolprotocol

value transfer protocolvalue transfer protocolDiagnostic manufacturersDiagnostic manufacturers

concentration

coverage interval

B

best

estimate

uncertainty

bias="error"

CRM

assigned

CRM

measured

A


[[AdaptedAdapted fromfrom KallnerKallner A, A,

Scand J Scand J ClinClin & Lab Invest 2010; & Lab Invest 2010;

70(Suppl 242): 34]70(Suppl 242): 34]

3. ESTIMATION OF MEASUREMENT 3. ESTIMATION OF MEASUREMENT

UNCERTAINTY OF CALIBRATORSUNCERTAINTY OF CALIBRATORS

4. VERIFICATION THAT UNCERTAINTY 4. VERIFICATION THAT UNCERTAINTY

GOALS ARE FULLFILLED GOALS ARE FULLFILLED

GUMGUM

Eurachem/

CITAC Guide

Eurachem/

CITAC Guide

CLSI document EP29-ACLSI document EP29-A

PARADIGM SHIFT IN THE THINKINGPARADIGM SHIFT IN THE THINKING

•• IfIf the the manufacturermanufacturer assumesassumes total total responsibilityresponsibility forfor supplyingsupplying productsproducts of of acceptableacceptable qualityquality in in termsterms of of traceabilitytraceability and and uncertaintyuncertainty of the system (of the system (““CE CE markedmarked””), ), itit isisno no longerlonger possiblepossible toto considerconsider separatelyseparately the the componentscomponents of of eacheach measuringmeasuring system (i.e., system (i.e., platformplatform, , reagentsreagents, , calibratorscalibrators and control and control materialsmaterials), ), whichwhich in in termsterms of performance can of performance can onlyonly bebe guaranteedguaranteed and and certifiedcertified byby the the manufacturermanufacturer asas a a wholewhole..

•• AnyAny changechange introducedintroduced byby usersusers or or thirdthirdpartiesparties (e.g., the (e.g., the useuse of of reagentsreagents, , calibratorscalibrators or or control control materialsmaterials fromfrom otherother supplierssuppliers) ) maymaysignificantlysignificantly alter the alter the qualityquality of the of the measuringmeasuringsystem performance, system performance, removingremoving anyanyresponsibilityresponsibility fromfrom the the manufacturermanufacturer and and deprivingdepriving the system (and, the system (and, consequentlyconsequently, the , the producedproduced resultsresults) of the ) of the certificationcertification originallyoriginallyprovidedprovided through CE through CE markingmarking..













2010;48:72010;48:7






















Clinical laboratoriesClinical laboratories

•• Verification of availability and quality of Verification of availability and quality of

INFORMATION INFORMATION about IVD metrological about IVD metrological

traceability and uncertaintytraceability and uncertainty

•• DAILY SURVEILLANCEDAILY SURVEILLANCE of IVD system of IVD system

traceabilitytraceability

••Estimation of the Estimation of the MEASUREMENT MEASUREMENT

UNCERTAINTYUNCERTAINTY due to the random effects due to the random effects

and calculation of uncertainty of and calculation of uncertainty of

laboratory measurements (laboratory measurements (UUcalcal + + UUrandomrandom))

Post-marketingsurveillance

PostPost--marketingmarketing

surveillancesurveillance

In principle, laboratory users should be able to access the In principle, laboratory users should be able to access the

following:following:

a) an indication of higher order references (materials and/or procedures) used to assign traceable values to calibrators,

b) which internal calibration hierarchy has been applied by the manufacturer, and

c) a detailed description of each step,

d) the expanded combined uncertainty value of commercial calibrators, and

e) which, if any, acceptable limits for uncertainty of calibrators were applied in the validation of the analytical system.

a) an indication of higher order references (materials and/or procedures) used to assign traceable values to calibrators,

b) which internal calibration hierarchy has been applied by the manufacturer, and

c) a detailed description of each step,

d) the expanded combined uncertainty value of commercial calibrators, and

e) which, if any, acceptable limits for uncertainty of calibrators were applied in the validation of the analytical system.

Braga F & Panteghini M, Clin Chim Acta 2014;432:55Braga F & Panteghini M, Clin Chim Acta 2014;432:55

(ideally all this information should be available in (ideally all this information should be available in

the assay or calibrator package inserts)the assay or calibrator package inserts)

VERIFICATION OF AVAILABILITY AND QUALITY OF INFORMATIONVERIFICATION OF AVAILABILITY AND QUALITY OF INFORMATION

Currently, the full information about

calibration is usually not available

Currently, the full information about

calibration is usually not available

Some Some organisationsorganisations are are frequentlyfrequently

mentionedmentioned ((oftenoften withoutwithout explanationexplanation): ):

usedused asas a a ““trustedtrusted brandbrand””

•• NIST, IRMM, IFCC, CLSI (NIST, IRMM, IFCC, CLSI (protocolsprotocols))

GCGC--IDMS @ NISTIDMS @ NIST

NIST SRM 917NIST SRM 917

ManufacturerManufacturer’’s s

internal procedureinternal procedure

Commercial Commercial

systemsystem


(glucose in human serum)(glucose in human serum)

CommercialCommercial

calibratorcalibrator

PatientPatient’’s sample resultss sample results

GCGC--IDMS IDMS [accredited reference laboratory][accredited reference laboratory]

ManufacturerManufacturer’’s internals internal

procedureprocedure


systemsystem




ComparisonComparison onon

biologicalbiological samplessamples

AA BB

CC DD NIST SRM 917NIST SRM 917

ManufacturerManufacturer’’s s

internal procedureinternal procedure


CDC CDC HexokinaseHexokinase[accredited reference laboratory][accredited reference laboratory]

ManufacturerManufacturer’’s internals internal

procedureprocedure


systemsystem




ComparisonComparison onon

biologicalbiological samplessamples


Commercial Commercial

systemsystem




Types of metrological chains that can be used to implement the tTypes of metrological chains that can be used to implement the traceability of blood glucose results*raceability of blood glucose results*


*all*all JCTLM JCTLM recognizedrecognized

Chain A = 1.45% vs. Chain C = 3.26%

Are the Are the measuringmeasuring systemssystems commerciallycommercially availableavailable forfor glucoseglucose determinationdetermination ableable

toto achieveachieve the the desirabledesirable limitlimit forfor combinedcombined uncertaintyuncertainty in a in a clinicalclinical settingsetting??

uurefref

(u(u22refref ++ uu22

calcal))½½

((uu22ref ref + + uu22

calcal + + uu22randomrandom))½½

System imprecision

System calibration

uncertainty

Individual lab

performance

(IQC safety margin)

Measurement

uncertainty

budget

Uncertainty of

references


Patient result

5.4%5.4% desirabledesirable


Pasqualetti S, Braga F, Panteghini M, Pasqualetti S, Braga F, Panteghini M, Clin Biochem 2017; 50:587Clin Biochem 2017; 50:587--594.594.

L M H

•• Measured in triplicate in 3 Measured in triplicate in 3

consecutive daysconsecutive days byby participatingparticipating

laboratorieslaboratories

59

.9 U

/L5

9.9

U/L

18

6.9

U/L

18

6.9

U/L

40

1.5

U/L

40

1.5

U/L

•• ALP target values ALP target values

assigned with assigned with IFCC RMPIFCC RMP

The importance of the postThe importance of the post--marketing surveillancemarketing surveillance

EQA experimentEQA experiment

(performed 5(performed 5 years later years later the availability of the IFCC RMPthe availability of the IFCC RMP) )

to evaluate the level of ALP standardizationto evaluate the level of ALP standardization

The ability to meet or not the

desirable APS for bias is clearly

dependent on the measuring

system used.

The ability to meet or not the

desirable APS for bias is clearly

dependent on the measuring

system used.

Schumann G, et al.

Clin Chem Lab Med 2011;49:1439-46

Schumann G, et al.Schumann G, et al.

ClinClin ChemChem Lab Lab MedMed 2011;49:14392011;49:1439--4646

The importance of the postThe importance of the post--marketing surveillancemarketing surveillance

CompanyCompany PlatformPlatformPrinciple of Principle of

commercial methodcommercial methodCalibratorCalibrator

Declared standard Declared standard


HigherHigher--order reference order reference

employedemployed

Abbott Abbott ArchitectArchitect pp--NPP/AMPNPP/AMP Calibration factorCalibration factor NANA IFCC reference method (2011)IFCC reference method (2011)

pp--NPP/AMPNPP/AMP Calibration factorCalibration factor NANA Molar extinction coefficientMolar extinction coefficient

Beckman Beckman AUAU IFCC (1983)IFCC (1983) System calibratorSystem calibrator 6.00%6.00% Beckman Coulter Master CalibratorBeckman Coulter Master Calibrator

pp--NPP/DEANPP/DEA System calibratorSystem calibrator NANA Beckman Coulter Master CalibratorBeckman Coulter Master Calibrator

SynchronSynchron pp--NPP/AMPNPP/AMP Enzyme Enzyme ValidatorValidator level 1level 1 6.22%6.22% IFCC reference method (2011)IFCC reference method (2011)

Enzyme Enzyme ValidatorValidator level 2level 2 1.86%1.86% IFCC reference method (2011)IFCC reference method (2011)

pp--NPP/AMPNPP/AMP Enzyme Enzyme ValidatorValidator level 1level 1 3.64%3.64% DGKC standard methodDGKC standard method

Enzyme Enzyme ValidatorValidator level 2level 2 1.27%1.27% DGKC standard methodDGKC standard method

Roche Roche CobasCobas cc IFCC gen. 2IFCC gen. 2 C.f.a.sC.f.a.s.. 0.59%0.59% IFCC reference method (1983)IFCC reference method (1983)

IntegraIntegra IFCC gen. 2IFCC gen. 2 C.f.a.sC.f.a.s 1.22%1.22% IFCC reference method (1983)IFCC reference method (1983)

ModularModular IFCC liquidIFCC liquid C.f.a.sC.f.a.s 1.65%1.65% IFCC reference method (1983)IFCC reference method (1983)

SiemensSiemens Dimension VistaDimension Vista pp--NPP/AMPNPP/AMP ALPI calibratorALPI calibrator 4.51%4.51%cc IFCC reference method (2011)IFCC reference method (2011)

AdviaAdvia pp--NPP/AMPNPP/AMP Chemistry calibrator control 1Chemistry calibrator control 1 3.70%3.70% IFCC reference method (2011)IFCC reference method (2011)

Chemistry calibrator control 2Chemistry calibrator control 2 1.00%1.00% IFCC reference method (2011)IFCC reference method (2011)

pp--NPP/DEANPP/DEA Chemistry calibrator control 1Chemistry calibrator control 1 1.40%1.40%cc Molar extinction coefficientMolar extinction coefficient

Chemistry calibrator control 2Chemistry calibrator control 2 1.30%1.30%cc Molar extinction coefficientMolar extinction coefficient

• The availability of an internationally agreed reference

measurement system does not automatically mean that

the traceability to it is implemented!

• If a post-marketing surveillance is lacking, alternatives

that do not comply with the EU Directive can remain

undisturbed on the market.

• The availability of an internationally agreed reference The availability of an internationally agreed reference

measurement system does not automatically mean that measurement system does not automatically mean that

the traceability to it is implemented!the traceability to it is implemented!

•• If a postIf a post--marketing surveillance is lacking, alternatives marketing surveillance is lacking, alternatives

that do not comply with the EU Directive can remain that do not comply with the EU Directive can remain

undisturbed on the market.undisturbed on the market.

Roche recalibrated Roche recalibrated

its analytical its analytical

methods for the methods for the

ALP measurement ALP measurement

according to 2011 according to 2011

IFCC RMP!IFCC RMP!

Trueness evaluation and verification of interTrueness evaluation and verification of inter--assay agreement of 11 serum assay agreement of 11 serum

immunoglobulin A measuring systems: implications for medical decimmunoglobulin A measuring systems: implications for medical decisionsisions

Braga F, Infusino I, Frusciante E, Ceriotti F, Panteghini M [under review]

Evaluating the status of standardization of available Evaluating the status of standardization of available IgAIgA measuring systems to measuring systems to

investigate if a common lower reference limit can be implementedinvestigate if a common lower reference limit can be implemented to identify to identify

an an IgAIgA deficiencydeficiency, , especially for pediatric populationespecially for pediatric population

• Trueness evaluationERM-DA470k/IFCCCertified value: 1.80 g/L

Expanded uncertainty (U) (k =2): 0.05 g/L

[immediately shipped to centres]

• Inter-assay agreement18 SERUM POOLS- 5 ≤0.7 g/L

- 10 between 0.7 to 4.0 g/L

- 3 ≥ 4.0 g/L

[stored at -80°C until the distribution to centres]

AimAim

Laboratories were requested to perform analysis within two days Laboratories were requested to perform analysis within two days of delivery, in a single of delivery, in a single

analytical run, by applying the manufactureranalytical run, by applying the manufacturer’’s recommended instructions and strictly s recommended instructions and strictly

following a specific analysis sequencefollowing a specific analysis sequence

ResultsResults

• Commutability assessment of ERM-DA470k/IFCC

1. CLSI 1. CLSI guidelineguideline EP30EP30--AA

[based on the pair-wise comparison of sets of IgA results on patient pools by the 11 MS]

DemingDeming regressionregression

95% 95% predictionprediction

intervalsintervals

2. IFCC 2. IFCC recommendationsrecommendations

ReferenceReference

materialmaterial

Results confirmed the acceptable commutability of Results confirmed the acceptable commutability of

ERMERM--DA470k/IFCC for all MS we evaluated, DA470k/IFCC for all MS we evaluated,

suggesting that this material was appropriate for suggesting that this material was appropriate for

estimating the trueness of the procedures.estimating the trueness of the procedures.

[CLSI. Document EP30-A Wayne (PA): CLSI; 2010]

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0 1 2 3 4 5

Mean conc both (x+y)/2 (g/L)

Bia

s ln

(AU

48

0)-

ln(A

rch

ite

ct

c4

00

0)_

ReferenceReference materialmaterial

Minimum Minimum

quality level quality level

for bias from for bias from

BVBV

[Nilsson G, et al Clin Chem 2018;64:455–64]

ResultsResults

• Trueness evaluation of IgA MS

Significantly

overestimated

Significantly

underestimated

ExcludedExcluded

fromfrom the the

followingfollowing

data data

elaborationelaboration

ResultsResults

• Verification of agreement among

IgA MS

PassingPassing--BablokBablok regression analysis and average percentage bias of measured regression analysis and average percentage bias of measured

IgAIgA values for patient pools vs. median values for patient pools vs. median IgAIgA concentrations for MS. concentrations for MS.

not fulfilling

desirable bias

(±9.1%)

not fulfilling

minimum bias

(±13.6%)

All methods median, g/L

0 1 2 3 4 5

Arc

hitect

c400

0 -

All

meth

ods m

ed

ian, %

-100

-50

0

50

100


0 1 2 3 4 5

AU

48

0 -

All

me

tho

ds m

ed

ian

, %

-100

-50

0

50

100


0 1 2 3 4 5

Imm

age

800

- A

ll m

eth

od

s m

edia

n,

%

-100

-50

0

50

100


0 1 2 3 4 5

Optilit

e -

All

meth

ods m

ed

ian,

%

-100

-50

0

50

100


0 1 2 3 4 5

Ro

ch

e C

OB

AS

c7

02

- A

ll m

eth

od

s m

ed

ian

, %

-100

-50

0

50

100


0 1 2 3 4 5

Min

dra

y B

S4

80

- A

ll m

eth

od

s m

edia

n,

%

-100

-50

0

50

100


0 1 2 3 4 5

Vis

ta -

All

me

thod

s m

edia

n,

%

-100

-50

0

50

100


0 1 2 3 4 5

Ta

uru

s -

All

me

tho

ds m

ed

ian

, %

-100

-50

0

50

100


0 1 2 3 4 5

ILA

B 6

50

- A

ll m

eth

od

s m

ed

ian,

%

-100

-50

0

50

100


0 1 2 3 4 5

Advia

- A

ll m

eth

ods m

edia

n, %

-100

-50

0

50

100

PercentagePercentage differencedifference plotsplots

In In generalgeneral, the , the situationsituation clearlyclearly worsenedworsened whenwhen the the

analysisanalysis focusedfocused on the on the lowlow IgAIgA concentrationsconcentrations

typicaltypical of of childrenchildren

ResultsResults

• Measurement uncertainty of IgA MS on patient pools

For suitable clinical application of For suitable clinical application of IgAIgA measurements, the expanded measurement measurements, the expanded measurement

uncertainty at the patient sample level should remain within uncertainty at the patient sample level should remain within ±±2.7%, 2.7%, ±±5.4%, and 5.4%, and ±±8.1%8.1%

for the optimum, desirable, and minimum quality levels, respectifor the optimum, desirable, and minimum quality levels, respectivelyvely..

ConclusionConclusion

Although the harmonization among marketed Although the harmonization among marketed IgAIgA MS MS

is substantially good, is substantially good, the implementation of the implementation of

traceability to higher order references is still far from traceability to higher order references is still far from

perfectperfect. Most of the MS evaluated in this study . Most of the MS evaluated in this study

showed unacceptable standardization of showed unacceptable standardization of IgAIgA

measurements, especially at concentrations measurements, especially at concentrations ≤≤0.4 g/L. 0.4 g/L.

The definition of traceable reference intervals in The definition of traceable reference intervals in

pediatric population should consider this analytical pediatric population should consider this analytical

background.background.

Daily surveillance of IVD system traceability Daily surveillance of IVD system traceability

1.1. Verification of the consistency of declared performance Verification of the consistency of declared performance

during routine operations performed in accordance with the during routine operations performed in accordance with the

manufacturermanufacturer’’s instructionss instructions

(checking of system alignment by (checking of system alignment by IQC component IIQC component I))

2.2. Participation to appropriately structured EQASParticipation to appropriately structured EQAS

((““that meet metrological criteriathat meet metrological criteria””))

Braga F & Panteghini M, Braga F & Panteghini M,

Clin Chim Acta 2014;432:55Clin Chim Acta 2014;432:55

IVDIVD

TraceabilityTraceability

SurveillanceSurveillance








Braga F & Panteghini M, Braga F & Panteghini M,

Clin Chim Acta 2014;432:55Clin Chim Acta 2014;432:55

IVDIVD



Analytical quality

of measurement

Check alignment Imprecision

qualify

Reliability of the analytical system

AnalyticalAnalytical QualityQuality Control in the Traceability EraControl in the Traceability Era

PanteghiniPanteghini M, M, ClinClin ChemChem Lab Lab MedMed 2010;48:72010;48:7


10-M

ay

25-M

ay

28-J

un

13-Jul

28-J

ul

11-A

ug

27-Aug

11-Sep

26-Sep

13-O

ct

24

+3SD

+2SD

+1SD

19

29

34

Monitoring the reliability of the measuring system through IQC: Monitoring the reliability of the measuring system through IQC:

Component I. Check alignment (Component I. Check alignment (““system traceabilitysystem traceability””))A

cce

pta

ble

Acc

ep

tab

lera

ng

era

ng

eo

f co

ntr

ol

ma

teri

al

of

con

tro

l m

ate

ria

l

This program checks whether in the course of an analytical run tThis program checks whether in the course of an analytical run the performance of a he performance of a

measuring system complies with the set goals, represented by themeasuring system complies with the set goals, represented by the acceptable acceptable

ranges of control materials. ranges of control materials.

Clinical laboratories must verify the consistency of declared peClinical laboratories must verify the consistency of declared performance during routine operations rformance during routine operations

performed in accordance with the manufacturerperformed in accordance with the manufacturer’’s instructions, by s instructions, by checking that values of control checking that values of control

materials provided by the manufacturer as component of the measumaterials provided by the manufacturer as component of the measuring system are in the ring system are in the

established control rangeestablished control range, with no clinically significant changes in the assumed traceabl, with no clinically significant changes in the assumed traceable results.e results.

PlatformPlatform

CalibratorsCalibratorsReagentsReagents

Control material(s)Control material(s)

PlatformPlatform


PlatformPlatform


Control material(s)Control material(s)

Acceptance/rejection of

the analytical run in

“real time”

Acceptance/rejection of Acceptance/rejection of

the analytical run in the analytical run in

““real timereal time””

Internal Quality Control

(Component I)

Internal Quality Control Internal Quality Control

(Component I)(Component I)

Testing alignmentTesting alignment

[[““system traceabilitysystem traceability””]]

Any “out of control” signal must be made available with

sufficient time to allow immediate corrective actions

to bring again the situation under control (virtually

“unbiased”) and before reports related to the samples

analyzed in the affected analytical run are issued.

Any Any ““out of controlout of control”” signal must be made available with signal must be made available with

sufficient time to allow immediate corrective actions sufficient time to allow immediate corrective actions

to bring again the situation under control (virtually to bring again the situation under control (virtually

““unbiasedunbiased””) and before reports related to the samples ) and before reports related to the samples

analyzed in the affected analytical run are issued.analyzed in the affected analytical run are issued.








Braga F & Panteghini M, Clin Braga F & Panteghini M, Clin

Chim Acta 2014;432:55Chim Acta 2014;432:55

IVDIVD



Analytical quality

of measurement


qualify





Requirements for the applicability of EQAS results in the Requirements for the applicability of EQAS results in the

evaluation of the performance of participating laboratories evaluation of the performance of participating laboratories

in terms of traceability of their measurementsin terms of traceability of their measurements

FeatureFeature AimAim

EQAS materials valueEQAS materials value--assigned with assigned with

reference procedures by an accredited ref. reference procedures by an accredited ref.

laboratorylaboratory

To check traceability of commercial To check traceability of commercial

system to reference systemsystem to reference system

Proved commutability of EQAS materialsProved commutability of EQAS materials To allow transferability of To allow transferability of

participating laboratory performance participating laboratory performance

to the measurement of clinical to the measurement of clinical

samplessamples

Definition and use of the clinically Definition and use of the clinically

allowable measurement errorallowable measurement error

To verify the suitability of laboratory To verify the suitability of laboratory

measurements in clinical settingmeasurements in clinical setting

PanteghiniPanteghini M, M, ClinClin ChemChem Lab Lab MedMed 2010;48:7 2010;48:7

InfusinoInfusino I I etet al., al., ClinClin ChemChem Lab Lab MedMed 2010;48:3012010;48:301

Braga F & Braga F & PanteghiniPanteghini M. M. ClinClin ChemChem Lab Lab MedMed 2013;51:17192013;51:1719


[Ferraro S, Braga F, Panteghini M. Clin Chem Lab Med 2016;54:523]

Analytical quality

of measurement


qualify





Internal Quality Control

(Component II)

Internal Quality Control Internal Quality Control

(Component II)(Component II)

Testing the uncertainty Testing the uncertainty

due to the random effectsdue to the random effects

((““imprecisionimprecision””))

System stability at

medium/long term

System stability at System stability at

medium/long termmedium/long term

This program provides, through

mechanisms of retrospective

evaluation, data useful to the

knowledge of variability of

measuring system and of its use by

the individual laboratory.

This program provides, through This program provides, through

mechanisms of retrospective mechanisms of retrospective

evaluation, data useful to the evaluation, data useful to the

knowledge of variability of knowledge of variability of

measuring system and of its use by measuring system and of its use by

the individual laboratory.the individual laboratory.

System

Reagent lots

Laboratory

Monitoring the reliability of the measuring system through IQC: Monitoring the reliability of the measuring system through IQC:

Component II. Evaluate the system + individual lab imprecisionComponent II. Evaluate the system + individual lab imprecision

System imprecision

System calibration

(combined) uncertainty

Individual lab

performance

(IQC safety margin)

Measurement

uncertainty

budget

CardiacCardiac troponintroponin T high sensitiveT high sensitiveMonthlyMonthly monitoringmonitoring of of imprecisionimprecision byby IQC materialIQC material

0

1

2

3

4

5

6

7

8

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Imp

recis

ion

, C

V%

Requirements for IQC material

(Component II)

RequirementsRequirements forfor IQC material IQC material

((ComponentComponent II)II)

RequirementRequirement CommentComment

MatrixedMatrixed material from a thirdmaterial from a third--

party independent source party independent source

should be used (e.g., freshshould be used (e.g., fresh--

frozen pool)frozen pool)

Material must be different from the Material must be different from the

system control material used for system control material used for

checking alignmentchecking alignment

Specimens closely resembling Specimens closely resembling

authentic patient samples authentic patient samples

(commutability)(commutability)

Commercial nonCommercial non--commutable controls commutable controls

may provide a different impression of may provide a different impression of

imprecision performanceimprecision performance

Specimens of concentrations Specimens of concentrations

appropriate to the clinical appropriate to the clinical

application of the application of the analyteanalyte

When clinical decision cutWhen clinical decision cut--points are points are

employed, samples around these employed, samples around these

concentrations should preferentially be concentrations should preferentially be

selected selected

F. BragaF. Braga

http://http://users.unimi.itusers.unimi.it//cirmecirme/home//home/index.phpindex.php

Research Centre for Metrological Traceability in Laboratory Medicine (CIRME),

University of Milan, Milan, Italy

Documents

Roles and responsibilities in traceability implementation · Mario Plebani, Laura Sciacovelli, Daniela Bernardi, Ada Aita, Giorgia Antonelli, Andrea Padoan The role of external quality