EXTERNAL QUALITY EXTERNAL QUALITY ASSESSMENT PROGRAM ASSESSMENT PROGRAM

(EQAP)(EQAP)(EQAP)(EQAP)BIOCHEMISTRY DEPARTMENTBIOCHEMISTRY DEPARTMENT

R. MohammadiR. MohammadiBiochemist (Ph.D.)Biochemist (Ph.D.)

Faculty member of Medical FacultyFaculty member of Medical FacultyFaculty member of Medical FacultyFaculty member of Medical Faculty

TWO COMPLEMENTARY TWO COMPLEMENTARY COMPONENETS OF TQM ARECOMPONENETS OF TQM ARECOMPONENETS OF TQM ARECOMPONENETS OF TQM ARE

Internal Quality Control (IQC) Internal Quality Control (IQC)

External Quality Assessment (EQA)External Quality Assessment (EQA)

INTERNAL QUALITY CONTROL INTERNAL QUALITY CONTROL ( QC)( QC)(IQC)(IQC)

Is necessary for the daily monitoring of Is necessary for the daily monitoring of precision and accuracy of analytical methodprecision and accuracy of analytical method

Only Detects changes between present Only Detects changes between present performance and previous stable performance and previous stable p pp pperformance (Baseline)performance (Baseline)

Undetected systematic errors duringUndetected systematic errors duringUndetected systematic errors during Undetected systematic errors during baseline period would included in the mean baseline period would included in the mean that was used to calculate the control limits that was used to calculate the control limits

EXTERNAL QUALITY EXTERNAL QUALITY SS SS ( Q )SS SS ( Q )ASSESSMENT (EQA)ASSESSMENT (EQA)

Is a process by which a laboratory uses an outside Is a process by which a laboratory uses an outside unbiased source to verify the quality of patients unbiased source to verify the quality of patients resultsresultsresults results

allows a laboratory to confirm that its results are allows a laboratory to confirm that its results are i t t ith th f th l b t i i thi t t ith th f th l b t i i thconsistent with those of other laboratories using the consistent with those of other laboratories using the

same or similar methods for an same or similar methods for an analyteanalyte and thus to and thus to confirm it is using a method correctly confirm it is using a method correctly

Is important for maintaining the longIs important for maintaining the long--term accuracy term accuracy of the analytical methodsof the analytical methodsof the analytical methodsof the analytical methods

EQA HAS FOUR STAGESEQA HAS FOUR STAGESEQA HAS FOUR STAGESEQA HAS FOUR STAGES

Providing and Sending Quality Control (QC) Providing and Sending Quality Control (QC) Material to Participating LaboratoryMaterial to Participating Laboratory

Measuring Measuring AnalyteAnalyte(s) and Reporting Result(s) by (s) and Reporting Result(s) by Participating LaboratoryParticipating Laboratory

Data Analysis and Sending Evaluation Report(s) to Data Analysis and Sending Evaluation Report(s) to Participating LaboratoryParticipating Laboratory

Interpretation of Evaluation Report(s) and Taking Interpretation of Evaluation Report(s) and Taking Corrective Action for Unacceptable Result(s) in Corrective Action for Unacceptable Result(s) in Participating LaboratoryParticipating LaboratoryParticipating LaboratoryParticipating Laboratory

PROVIDING AND SENDING PROVIDING AND SENDING QC MATERIALQC MATERIAL

General RequirementsGeneral RequirementsA l tA l t V lid tiV lid tiAnalyteAnalyte ValidationValidationHomogenityHomogenity (Uniformity) Testing(Uniformity) TestingStability TestingStability TestingStress TestingStress TestingStress TestingStress TestingShipping TestingShipping Testing

MEASURING ANAYTE(S) AND MEASURING ANAYTE(S) AND REPORTING RESULT(S)REPORTING RESULT(S)

StorageStorageReconstituationReconstituationM i QC M i l Lik P iM i QC M i l Lik P iMeasuring QC Material Like Patient Measuring QC Material Like Patient SpecimenSpecimenReporting Results and Method CharactersReporting Results and Method CharactersReporting Results and Method Characters Reporting Results and Method Characters 11) Kit Producer) Kit Producer22) Kit Lot No.) Kit Lot No.33) Procedure) Procedure44) Principle of Method) Principle of Method

REASONS FOR REDUCINGREASONS FOR REDUCINGREASONS FOR REDUCING REASONS FOR REDUCING CONSISTENCY BETWEEN DIFFERENT CONSISTENCY BETWEEN DIFFERENT METHODS AND PROCEDURES AREMETHODS AND PROCEDURES AREMETHODS AND PROCEDURES AREMETHODS AND PROCEDURES ARE

Diff t A l ti l S ifi itDiff t A l ti l S ifi itDifferent Analytical SpecificityDifferent Analytical SpecificityDifferent Analytical SensitivityDifferent Analytical SensitivityDifferent CalibrationDifferent CalibrationMatrix EffectMatrix EffectMatrix EffectMatrix Effect

DATA ANALYSIS FOR DATA ANALYSIS FOR O O S SO O S SINTERPRETATION OF RESULTSINTERPRETATION OF RESULTS

Evaluation of performance of each participant needs Evaluation of performance of each participant needs to to stablishstablish two values:two values:

11) Assigned (target) value of the test material) Assigned (target) value of the test material22) Acceptable range) Acceptable range

Different methods can be used to establish these Different methods can be used to establish these estimates, but there is no standard protocol estimates, but there is no standard protocol statistical parametersstatistical parametersstatistical parametersstatistical parameters

WHAT AREWHAT ARETARGETTARGET VALUEVALUE& & TRUETRUE VALUE ?VALUE ?

Definitive Method True Value

Primary ReferenceMaterial

MethodValidation

Reference Method Traceability

Secondary Reference

MethodValidation& ExternalReference

Material& External

QualityAssurance

Field Method

Internal

Observed Value

STRUCURE OF ControlMaterial

InternalQuality

Assurance

STRUCURE OF ACCURACY –BASED

SYSTEM

ESTABLISHING ASSIGNED VALUEESTABLISHING ASSIGNED VALUEESTABLISHING ASSIGNED VALUEESTABLISHING ASSIGNED VALUE

There are three methodsThere are three methods

11) The addition of a known amount or concentration ) The addition of a known amount or concentration ))of of analyteanalyte to a base material containing noneto a base material containing none

22) The use of a Consensus value produced by a group) The use of a Consensus value produced by a group22) The use of a Consensus value produced by a group ) The use of a Consensus value produced by a group of expert or referee laboratories using best possible of expert or referee laboratories using best possible methodsmethods

33) The use of a consensus value produced in each ) The use of a consensus value produced in each round of EQA, and based on the results by round of EQA, and based on the results by participantsparticipantsparticipantsparticipants

ESTABLISHING ASSIGNED VALUEESTABLISHING ASSIGNED VALUEFROM PARTICIPANT RESULTSFROM PARTICIPANT RESULTSFROM PARTICIPANT RESULTS FROM PARTICIPANT RESULTS

assigned value is consensus value (trimmed mean assigned value is consensus value (trimmed mean l ) d i d f ll lt b itt d bl ) d i d f ll lt b itt d bvalue) derived from all results submitted by value) derived from all results submitted by

participants in the scheme of that participants in the scheme of that analyteanalyte

P i l i hP i l i h hh h hh hPractical experiences has Practical experiences has shwonshwon that the that the consensus value usually agrees closely with the true consensus value usually agrees closely with the true value in schemes with a large number participantsvalue in schemes with a large number participants

Consensus value may not be valid in two Consensus value may not be valid in two condition:herecondition:here are three methodsare three methods

11) Numbers of laboratories ) Numbers of laboratories is smallis small22) A large proportion of participants have a significant) A large proportion of participants have a significant

analytical biasanalytical biasyy

VALIDATING THE CONSENSUS VALUEVALIDATING THE CONSENSUS VALUE

By By analysinganalysing the control material by reference the control material by reference methods, which needs laboratories with adequate methods, which needs laboratories with adequate training and experience of such methodstraining and experience of such methods

The routine methods by participants in the scheme The routine methods by participants in the scheme may also be used , but laboratories assigning the may also be used , but laboratories assigning the

l t b fid t f th i bi d i il t b fid t f th i bi d i ivalues must be confident of their bias and precisionvalues must be confident of their bias and precision

To compare the consensus values obtained for the To compare the consensus values obtained for the same control material from different schemes in for same control material from different schemes in for example example differerntdifferernt countries or regionscountries or regions

ESTABLISHING ACCEPTABLE RANGEESTABLISHING ACCEPTABLE RANGEESTABLISHING ACCEPTABLE RANGEESTABLISHING ACCEPTABLE RANGE

After calculating method relating consensus value, After calculating method relating consensus value, acceptability criteria must be establishacceptability criteria must be establish

For this, statistical parameters are calculated, For this, statistical parameters are calculated, includingincluding

11) Mean (X)) Mean (X)) ( )) ( )22) Standard Deviation (SD)) Standard Deviation (SD)33) Coefficient of Variation (CV)) Coefficient of Variation (CV)

CV% = SD

Xx 100

X

ESTABLISHING ACCEPTABLE RANGEESTABLISHING ACCEPTABLE RANGEESTABLISHING ACCEPTABLE RANGEESTABLISHING ACCEPTABLE RANGE

Acceptability criteria may beAcceptability criteria may be11) Interval based on group SD (e.g. X +/) Interval based on group SD (e.g. X +/-- 22SD)SD)22) Fixed percentage (e.g. X +/) Fixed percentage (e.g. X +/-- constant percent)constant percent)) p g ( g) p g ( g p )p )33) ) Fixed interval (X Fixed interval (X +/+/-- constant amount)constant amount)

Alternatively scoring system may be usedAlternatively scoring system may be usedAlternatively, scoring system may be usedAlternatively, scoring system may be used11) ) Bias Index Score (BIS)Bias Index Score (BIS)22) Variance ) Variance Index Score (VISIndex Score (VIS))33) Standard ) Standard Deviation Index or Interval (SDIDeviation Index or Interval (SDI))

Z ScoreZ ScoreZ ScoreZ Score

Z = Xlab - Xpeer

SDSDpeer

BIS =

Xlab - Xpeer

Xpeer x 100SDI = Xlab - Xpeer

SD BIS = CCV%SDpeer

Chosen Coefficient of VariationChosen Coefficient of Variation(CCV)(CCV)(CCV)(CCV)

CCV are the lowest CVs obtained CCV are the lowest CVs obtained for particular determinations for particular determinations d i fi t t f th EQASd i fi t t f th EQASduring first two years of the EQASduring first two years of the EQAS

It is kept constant so that It is kept constant so that improvements in the performance improvements in the performance

f l b t i b d t t df l b t i b d t t dof laboratories can be detectedof laboratories can be detected

Performance Indicators Resulting from Performance Indicators Resulting from CumulationCumulation of BIS & VIS Over Time of BIS & VIS Over Time

(typically (typically 6 6 months or months or 10 10 scores)scores)

Mean Running VIS (MRVIS)Mean Running VIS (MRVIS)g ( )g ( )

Mean Running BIS (MRBIS)Mean Running BIS (MRBIS)g ( )g ( )

Standard Deviation of BIS (SDBIS)Standard Deviation of BIS (SDBIS)( )( )

Random Error Systematic Error Total Errory

Imprecision Untrueness Inaccuracy

Random Bias Systematic BiasRandom &

Systematic Bias

SDBIS MRBIS MRVIS

Four Rules UsuallyFour Rules Usually EmpolyedEmpolyed forforFour Rules Usually Four Rules Usually EmpolyedEmpolyed forforSDI EvaluationSDI Evaluation

22//5511SDISDI11SDISDIXX11..55SDISDI1133SDISDI33SDISDIRR44SDISDI

Peer GROUP PROGRAMPeer GROUP PROGRAMPeer GROUP PROGRAMPeer GROUP PROGRAMPeer Group Program Is A Combination of Peer Group Program Is A Combination of p gp gInternalInternal And And ExternalExternal Quality ControlQuality ControlWhen EQC Is Used In Conjunction With When EQC Is Used In Conjunction With D il IQC Thi P Will GiD il IQC Thi P Will GiDaily IQC, This Program Will Give Daily IQC, This Program Will Give Laboratories Laboratories Added ConfidenceAdded Confidence in Their in Their Patient Test ResultsPatient Test ResultsPatient Test ResultsPatient Test ResultsAll Labs Use The Same Control Material All Labs Use The Same Control Material and Report Their Results and Report Their Results DailyDailyData Are Analyzed And Reported Data Are Analyzed And Reported MonthlyMonthlyAs As SDISDI and and CVRCVR

Is Numbers of member in peer group adequate?

No

Is CV% of

Data analysis is not valid

Yes

peer group suitable?

YesNo

What Is the result of data analysis?

UnacceptableWarnningExcellent or Good pg

Need error detection and correction

Follow next EQA ResultNeed no action

CAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURES

Problem with QC MaterialsProblem with QC Materials11) ) AnalyteAnalyte UnstabilityUnstability22) Mishandling of Specimen during ) Mishandling of Specimen during ) g p g) g p g

Shipping to LaboratoryShipping to Laboratory33) Interfering Substance in QC Materials) Interfering Substance in QC Materials33) Interfering Substance in QC Materials) Interfering Substance in QC Materials

CAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURES

Incorrect Handling of QC Incorrect Handling of QC M i lM i lMaterialsMaterials

11) Incorrect) Incorrect ReconstituationReconstituation11) Incorrect ) Incorrect ReconstituationReconstituation22) Incorrect Storage Conditions) Incorrect Storage Conditions33) Evaporation of Prepared QC Materials) Evaporation of Prepared QC Materials33) Evaporation of Prepared QC Materials) Evaporation of Prepared QC Materials

CAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURES

Technical Problem with a MethodTechnical Problem with a Method11) Calibration Problem) Calibration Problem))22) Inadequate Maintenance Causing Increased ) Inadequate Maintenance Causing Increased

ImprecisionImprecision33) D t i ti f R t d Oth) D t i ti f R t d Oth33) Deterioration of Reagents and Other ) Deterioration of Reagents and Other

ComponentsComponents44) Inadequate Environmental Control In) Inadequate Environmental Control In44) Inadequate Environmental Control In ) Inadequate Environmental Control In

LaboratoryLaboratory

CAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURES

Incorrect ProcedureIncorrect Procedure11) Incorrect Dilution Process) Incorrect Dilution Process22) Incorrect Mixing ) Incorrect Mixing 33) Incorrect Calculation) Incorrect Calculation44) Incorrect Unit) Incorrect Unit55) Transcription Error) Transcription Error

CAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURESCAUSES FOR EQAP FAILURES

Incorrect Laboratory Grouping Incorrect Laboratory Grouping A d F lli i GA d F lli i G O hO hAnd Falling in Group And Falling in Group OthersOthersWhich Is Compared with Which Is Compared with AlloverAllover