Upload
camron-mathews
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
QCL Training Seminar, Tanzania | 5-7 Dec 071 |
Proficiency Tests
John H McB Miller
Laboratory Department (DLab)
European Department for the Quality of Medicines
Council of Europe
Strasbourg, France
QCL Training Seminar, Tanzania | 5-7 Dec 072 |
Interlaboratory StudiesInterlaboratory Studies
Interlaboratory Testing
• Proficiency testing
• Collaborative study
• Certification study
• Co-operative study
Description
• Continuing assessment of technical competence
• Validation of a specific method
• Establishing the best estimate of the time value of an analyte in a reference material
• Laboratory assessment of samples and methods (eg educational studies)
QCL Training Seminar, Tanzania | 5-7 Dec 073 |
Proficiency testing by interlaboratory comparisons
To determine the competence of individual laboratories to perform specific tests or measurements
To monitor the performance of laboratories overtime
QCL Training Seminar, Tanzania | 5-7 Dec 074 |
ISO Guide 43-1
Interlaboratory comparisons may be used to:
a) determine the performance of individual labs for specific tests or measurements and to monitor labs’ continuing performance;
b) identify problems in labs & initiate remedial actions which may be related to, for example, individual staff performance or calibration of instrumentation;
QCL Training Seminar, Tanzania | 5-7 Dec 075 |
ISO Guide 43-1
c) establish the effectiveness & compatibility of new test or measurement methods & similarly to monitor established methods
d) provide additional confidence to lab clients;
e) identify interlaboratory differences;
QCL Training Seminar, Tanzania | 5-7 Dec 076 |
ISO Guide 43-1
Interlaboratory comparisons are conducted for a number of purposes and may be used by participating laboratories and other parties eg establishment of a reference material
QCL Training Seminar, Tanzania | 5-7 Dec 077 |
ISO Guide 43-2
6. Use of results by laboratory accreditation bodies
6.1 The results from proficiency testing schemes are useful for both participating laboratories & accreditation bodies. There are, however, limitations ............. that proficiency testing alone should not be used by laboratory accreditation bodies in their accreditation processes.
QCL Training Seminar, Tanzania | 5-7 Dec 078 |
6.2 If a laboratory submits a result(s) which fall outside acceptance criteria for a specific scheme, a laboratory accreditation body should have procedures for acting on such results
6.3 Such procedures should include early reporting to the laboratory of its results with an invitation for the laboratory to investigate and comment on its performance.
ISO Guide 43-2
QCL Training Seminar, Tanzania | 5-7 Dec 079 |
Quality Manual
Laboratories should have a section in their Quality Manuals:
- covering participation in proficiency testing
- how the results are used to demonstrate the competence of the laboratory
- procedures to be followed when unsatisfactory performance is reported
- records of participation in PT scheme
- corrective action reports
QCL Training Seminar, Tanzania | 5-7 Dec 0710 |
Z-score = c tSD
c = calculated reported value t = true value SD = target
RSZ = Zm
m = number of tests
RSSZ = Z2m
PT Scoring System
QCL Training Seminar, Tanzania | 5-7 Dec 0711 |
Assigned Value,
The assigned value ( ) may be the “true” or the consensus value.
True value may be a theoretical value or known value from “spiking’ of known quantity of a known quantity of analyte to a sample.
Consensus value based on the results of the participants. The consensus value is determined by the application of robust statistics (eg median value, mean interquartile range, Huber’s robust mean), to avoid the influence of “outliers” in the overall mean.
x
x
QCL Training Seminar, Tanzania | 5-7 Dec 0712 |
Target standard deviation (TSD)
This is set based on experience, reported or expected precision of the techniques used and according to fitness-for-purpose.
The TSD must be realistic.
The TSD should be consistently applied from round to round for the same technique/procedure in a PT scheme so that performance can be assessed over time.
QCL Training Seminar, Tanzania | 5-7 Dec 0713 |
Ranking
Z-score ≼ 2 satisfactory
≽ 2≼ 3 questionable> 3 unsatisfactory
QCL Training Seminar, Tanzania | 5-7 Dec 0714 |
Outliers are indicated using three test statistics:
Cochran’s test for outlying variances, Grubbs’ single test for outlying means and Grubbs’ paired test for outlying means, to be applied in this order. If a laboratory is excluded, the cycle is repeated from Cochran’s test until no outliers remain.
QCL Training Seminar, Tanzania | 5-7 Dec 0715 |
CochranÕs test for outlying variances is applied as follows: Calculate for each
laboratory the within laboratory variance 2ö and then 2
2
ö
ömax
. This value is compared
with the tabled critical values for the number of individual determinations r. If it exceeds the critical value, the laboratory with the maximum variance is removed and the cycle is continued with CochranÕs test.
QCL Training Seminar, Tanzania | 5-7 Dec 0716 |
Grubbs’ single test for outlying means is applied as follows: Calculate the between laboratory standard deviation of the mean values. Calculate the standard deviations L and H excluding the lowest and the highest mean value respectively
and let * be the smallest of these two values. Calculate
ˆ
ˆ1
*
and compare it with the tabled critical values. If it exceeds the critical value, the corresponding laboratory is excluded and the cycle continued with Cochran’s test
QCL Training Seminar, Tanzania | 5-7 Dec 0717 |
GrubbsÕ paired test for outlying means is applied as follows: Calculate the between laboratory standard deviation ö of the mean values. Calculate the standard deviations L2ö , H2ö and LHö excluding the two lowest, the two highest, and the
highest and the lowest mean value respectively and let *ö be the smallest of these
three values. Calculate
ö
ö1
*
and compare it with the tabled critical values. If it
exceeds the critical value, the corresponding laboratories are excluded and the cycle continued with CochranÕs test.
QCL Training Seminar, Tanzania | 5-7 Dec 0718 |
Robust Statistics
Huber’s mean for calculation of the « consensus » value is preferred to using elimination of results by tests for outliers
QCL Training Seminar, Tanzania | 5-7 Dec 0719 |
Scoring over time
RSD - Used for detecting consistent bias
RSSZ - Magnitude of deviations.
Cancellation of significant Z-scores if opposite sign is limited
QCL Training Seminar, Tanzania | 5-7 Dec 0720 |
The rescaled sum of squared z-scores RSSZ is calculated as m
z 2
=RSSZ where m
is the number of squared z-scores added (one for each sample). Critical values of RSSZ depend on m according to the table:
QCL Training Seminar, Tanzania | 5-7 Dec 0721 |
m ‘Doubtful’ ‘Unacceptable’ 2 3 4 5 6
> 3.00 > 2.60 > 2.37 > 2.21 > 2.10
> 4.61 > 3.78 > 3.32 > 3.02 > 2.80
QCL Training Seminar, Tanzania | 5-7 Dec 0722 |
Proficiency Testing
Corrective actions:
- PT schemes can be either mandatory or voluntary
- Voluntary schemes: PT records examined by external auditors during ISO 17025 assessment
- Mandatory schemes: Corrective action reports must be set within a defined time limit to the organisers for assessment and approval (or not). Failure to do so will result in a sanction
QCL Training Seminar, Tanzania | 5-7 Dec 0723 |
OMCL NetworkProficiency Testing Scheme
Initially open to OMCls of the European Union & other OMCLs associated with the European Pharmacopoeia (member & observer states)
Now open to any lab. on a fee paying basis
minimum of 4 tests/year
voluntary scheme
QCL Training Seminar, Tanzania | 5-7 Dec 0724 |
WHO External Quality Assurance Assessment Scheme (EQAAS)
Started in 2001 for selected regional medicines control laboratories
Phase 4 began in 2007
QCL Training Seminar, Tanzania | 5-7 Dec 0725 |
PTS 20Atomic Absorption Spectrophotometry
Incorrect programming of instrument resulting in insufficient significant figures
Calibration 1 significant figure 1200 ppm K
3 significant figure 1290 ppm K
11/28 questionnaire/unsatisfactory results
QCL Training Seminar, Tanzania | 5-7 Dec 0726 |
PTS 025Titration of the conjugate acid of organic bases
* Corrected for water content (not required)A lab. reported a difference of 2% in results between potentiometric & visual end-point. However, incorrect standardisation procedure was employed
* Deterioration in response of the electrode
23/46 questionnaire/unsatisfactory results
QCL Training Seminar, Tanzania | 5-7 Dec 0727 |
Liquid Chromatography Assay of Indapamide
98.33}98.64} Mean = 98.7 RSD = 0.4098.11}
True value:- 99.75 z-score: 2.13
Repeatability of ref. sol. (n=6) :- 0.57
QCL Training Seminar, Tanzania | 5-7 Dec 0728 |
Liquid Chromatography Assay of Indapamide
Mg Area Corrected
Area
Mean ²% Content
1 2
Ref.
Sol.
20.28 1698
1691
1675
1668
1672
1.8
- -
20.32 1669
1668
1643
1642
1643 - -
Test
Sol.
20.79
20.97
20.56
1696
1701
1716
1721
1690
1696
1632
1636
1637
1641
1644
1649
1634
1639
1647
< 1.0
98.33
98.65
99.11
99.1
99.46
99.94
QCL Training Seminar, Tanzania | 5-7 Dec 0729 |
Phase 4 : Procedure 1 - Table 1
Raw data & scoring of participating laboratoriesSemi-micro determination of water
QCL Training Seminar, Tanzania | 5-7 Dec 0731 |
EQAS on water content by Karl FischerNumber & percentage of participating laboratories having shown satisfactory
performance (z-score < 2)
Water content Phase 1 (2001)
Phase 3 (2005)
Phase 4 (2007)
No of Participants 7 33 35
No of labs with satisfactory results
4 19 23
% of labs with satisfactory results
57% 58% 66%
QCL Training Seminar, Tanzania | 5-7 Dec 0732 |
EQAAS
Comments:
- From the 3 participants who failed on the first exercise, 2 of them did not participate in the subsequent studies. Concerning the 3rd one their results slightly improve Phase 3 and were satisfactory in Phase 4.
- From the 14 participants who failed in the second exercise, 8 of them reported satisfactory results on the 3rd one (3 did not participate and 3 didn’t show any improvement)
- There doesn’t seem to be an improvement in the general trend. The overall performance of laboratories using this technique is not very satisfactory and could be improved. However, it has to be pointed out that the determination of water by Karl-Fischer is problematic even for experienced laboratories as we can see from the results reported by the laboratories (including OMCLs) participating in our regular PTS programme.
QCL Training Seminar, Tanzania | 5-7 Dec 0733 |
References
International Organization for S tandardization, ISO/IEC Guide 43-1: 1997, Proficiency testing by interlaboratory comparisons Ń Part 1: Development and operation of proficiency testing schemes.
International Organization for S tandardization, ISO/IEC Guide 43-2: 1997, Proficiency testing by interlaboratory comparisons Ń Part 2: Selection and us e of proficiency testing schemes by laboratory accreditation bodies.
International Organization for Standardization, ISO 5725-2-1994: Statistical methods.
Association of O fficial Analytical Chemists, Guidelines fo r Collaborative Study Procedure to Validate Ch aracteristics of a Method of Analysis, AOAC Arlington, (1990).
The Royal Society of Chemistry, Proficiency Testing in Analytical Chemistry, Teddington, (1997).
International Organisation of Standardisation ISO/IEC Standard ISO13528 : 2005, Statistical Methods for use i n Proficiency Testing by Interlaboratory Comparisons.