Warschau Nov. 2008 1
Dr. K. Luden
Lab Quality Assurance by Interlaboratory
Comparisons
Proficiency Testing
Dr. Katrin Luden
Dr. K. Luden
Drinking water directive 98/83/EC
Annex III
Specifications for the analysis of Parameters
Each Member State must ensure that any laboratory at which samples are analysed has a system of analytical quality control that is subject from time to time to checking by a person who is not under the control of the laboratory and who is approved by the competent authority for that purpose.
Warschau Nov. 2008 2
Dr. K. Luden
German drinking water directive
§ 15 Methods of Analysis and Laboratories
(4) The nessessary analysis including sampling are to be conducted by laboratories that work by “generally accepted rules of the technology” (allgemein anerkannte Regeln der Technik), have a system of internal quality control, participate at least once a year in external quality assurance schemes, have sufficiently qualified personell and are accredited by a generally recognized accreditation body. The responsible highest federal authority will publish a list of laboratories in its jurisdiction.
(5) The authorities (of the federal state) independent of the labs will have to check regularly wether the conditions of (4) are still fullfilled.
Warschau Nov. 2008 3
Dr. K. Luden
Proficiency testing
Participation in proficiency testing programs allows laboratories to:
identify areas where improvement in their testing and measurement methods is needed
identify further training of their staff
foster confidence in the performance of their testing and measurements
assure laboratory performance in their accredited test and calibration results
Warschau Nov. 2008 4
Dr. K. Luden
Jungle of standards for PT / EQA
Warschau Nov. 2008 5
There are several standards and guidelines for providing PT / EQA schemes (in chronological order)
• ISO/IEC Guide 43-1:1996 Proficiency testing by Interlaboratory Comparisons – Part 1: Development and Operation of Proficiency testing Schemes
• ISO 13528:2005 Statistical Methods for use in Proficiency Testing by Interlaboratory comparisons
• IUPAC Technical Report 1/2006 The International Harmonized Protocol for the Proficiency Testing of Analytical Chemistry Laboratories
• ILAC-G13:2007 ILAC Guidelines for the Competence of Providers of Proficiency Testing Schemes
• ISO/TS 20612 Water Quality – Interlaboratory comparisons for proficiency testing of analytical chemistry laboratories
• ISO/IEC CD 17043:2008 Conformity Assessment – General Requirements for Proficiency Testing
Dr. K. Luden
External Quality Assurance
Recommendation of Federal Environmental Agency 2002 - Microbiology
Warschau Nov. 2008 6
Dr. K. Luden
External Quality Assurance
Warschau Nov. 2008 7
Parametersuccessfull participation in
proficiency testing schemes
E. coli/Coliform bacteriaEnterococciColony Count
2x / year
Clostridium perfringensLegionellaPseudomonas aeruginosa
1x / year
Enteropathogenic VirusesCryptosporidium parvum
International PTS
Recommendation of Federal Environmental Agency 2002 - Microbiology
Dr. K. Luden
External Quality Assurance
Warschau Nov. 2008 8
Parametersuccessfull participation in
proficiency testing schemes
All parameters offered by a laboratory
1x within 2-3 years
Recommendation of Federal Environmental Agency 2003 - Chemistry
Proficiency testing (PT) according to DIN 38402-45
Dr. K. Luden
German drinking water PT providers
Warschau Nov. 2008 9
Microbiology and “hygenic-chemical PT
Chemistry PT
Chemistry PT
Chemistry PT
Regular meetings, harmonized schedules, identical statistical evaluation → comparability for labs and authorities
Dr. K. Luden
DIN 38402-45
Warschau Nov. 2008 10
Standard may be applied if:
Parameter analysed can be considered continuously measureable
true for most chemical and phisico-chemical parameters
not allways true for biological and/or microbiological parameters
Application in Chemistry: Yes
Application in microbiology: by way of trial / mostly
Dr. K. Luden
Microbiology PT
Warschau Nov. 2008 11
The governmental Institute of Public Health of Lower Saxony (NLGA) is Proficency Test (PT) provider for microbiological measurements in drinking water in Germany
Every year:
4 PTS for E. coli/coliform bacteria, Enterococci, and Colony Counts
2 PTS for Pseudomonas aeruginosa, Clostridium perfringens and Legionella
1 PTS for EU bathing water (E. coli, coliforme B.) 1 PTS for selected “hygenic-chemical” parameters
Dr. K. Luden
Microbiology PT
Warschau Nov. 2008 12
Parameter Qualified methods1986 -1999
2000
2001
2002
2003
2004
2006
2007
2008
E. coli-coliform bacteria
presence/absence testDIN EN ISO 9308-1Colilert®-18/Quanti-Tray®
+(+)
+(+)
+(+)
+(+)
(+)++
(+)++
(+)++
(+)++
(+)++
Colony CountGerman DWD 1990DIN EN ISO 6222
+ + ++
++
++
++
++
++
++
EnterococciEN ISO 7899-2Chromocult®-Enterococci-Agar
(+) (+) + + + ++
++
++
Clostridium perfringens
DWD (mCP-Agar)TSC
(+)(+)
++
++
++
++
++
++
Pseudomonas aeruginosa
EN 12780/ISO 16266 + + + + + +
Legionella spec. GVPC (ISO 11731-2) + + + + +
EC bathing water directive
Feacal coliform bacteria
+ + + + + + (+)
E. coli-coliform bacteria
DIN EN ISO 9803-3 (+) + +
Intestinal Enterococci
DIN EN ISO 7899-2DIN EN ISO 7899-1
(+)
(+)
+
+
+
+
Dr. K. Luden
Microbiology PT
Warschau Nov. 2008 13
Parameter Qualified methods1986 -1999
2000
2001
2002
2003
2004
2006
2007
2008
E. coli-coliform bacteria
presence/absence testDIN EN ISO 9308-1Colilert®-18/Quanti-Tray®
+(+)
+(+)
+(+)
+(+)
(+)++
(+)++
(+)++
(+)++
(+)++
Colony CountGerman DWD 1990DIN EN ISO 6222
+ + ++
++
++
++
++
++
++
EnterococciEN ISO 7899-2Chromocult®-Enterococci-Agar
(+) (+) + + + ++
++
++
Clostridium perfringens
DWD (mCP-Agar)TSC
(+)(+)
++
++
++
++
++
++
Pseudomonas aeruginosa
EN 12780/ISO 16266 + + + + + +
Legionella spec. GVPC (ISO 11731-2) + + + + +
EC bathing water directive
Feacal coliform bacteria
+ + + + + + (+)
E. coli-coliform bacteria
DIN EN ISO 9803-3 (+) + +
Intestinal Enterococci
DIN EN ISO 7899-2DIN EN ISO 7899-1
(+)
(+)
+
+
+
+
Dr. K. Luden
PT samples - two options
Warschau Nov. 2008 14
Liquid samples Freeze dried material
Bacterial solution in defined mineral medium
Bacteria adsorbed to milk powder
Very realistic samples
Handling exactly like a real sample
A solution has to be prepared and first step of handling procedure is different from real samples
Stability 4-6 days after shipment at <10°C
Stability at -20°C weeks or months
Germany United Kingdom, France, Sweden, Australia...
Dr. K. Luden
Shipping
Warschau Nov. 2008 16
300 - 500 participants/PT
Registered:
> 700 Labs from GermanyAustria (26) Switzerland (8) Lithuania (2) Hungary (2) Great Britain (1) Luxembourg (1)
Dr. K. Luden
Schedule of a proficiency testing scheme
Warschau Nov. 2008 17
Registration of participants
Confirmation of the registration (4 weeks before PT)
Preparation of the samples and shipment
Internal quality control (two weeks before and 6 days after shipment)
Reporting of the results (within 16 days of shipment)
Evaluation of the results usually within 4 weeks after the deadline for reporting
Dr. K. Luden
What do the labs get?
Warschau Nov. 2008 18
Report on the PT design, special problems, interestingresults etc.
Certificate and evaluation report(target value as consensus value of participants results)
Quality control charts about the influence of temperature and the duration of shipping
Z(u)-Scores of all measurements in tables
Z(u)-Scores in graphic form
Dr. K. Luden
Warschau Nov. 2008 19
Statistics of DIN 38402-45
Robust statistics (no outlier tests); consensus mean (used as assigned value) calculated by Hampel Estimator
ZU-scores “normalize” the results for comparing different levels of bacterial concentration
Z-score: lZUl 2 - sufficient
lZUl 3 - questionable
lZUl > 3 - insufficient
m-4,5s m-3s m-1,5s m m+1,5s m+3s m+4,5s
t
t
s
mxZ
)(
Dr. K. Luden
Warschau Nov. 2008 20
Name/Adress of PT-provider
Address of participant
Identification of PT (e.g. 3-2008)
Table of parameters, successfully analyzed param. And method used
Signatures of PT provider
Certificate
Dr. K. Luden
Microbiology PT
Warschau Nov. 2008 21
ZU-s
core
Group B III-2008 E. coli (assigned value 21 CFU/100 ml)
Labcode
Dr. K. Luden
Interlaboratory comparisons
Warschau Nov. 2008 22
method validation studies:aptitude/qualification of the lab is prerequisitereference material is fit for purpose
to certify reference material:method is supposed to be fit for purpose/validlabs are expert labs
Quality control/proficiency testing:performance of a lab is tested, reference material and method are supposed to be valid/fit for purpose
Dr. K. Luden
Warschau Nov. 2008 23
PT 1-2003
E. coli
0
5
10
15
20
25
30
35
40
45
R R R R R R R R R R 5 428
3 35
183
12
162
406
445
543
178
56
340
437
459
219
328
442
469
284
367
460
116
279
149
545
522
391
60
386
36
Labcode
E. c
oli
CF
U /
100
ml
E. coli / 100 ml X-3s X-2s X X+2s X+3s
Dr. K. Luden
Warschau Nov. 2008 25
Clostridium perfringens
Medium/method usedRecovery
rate
Columbia blood agar / membrane filtration 100 %
Columbia blood agar / membrane filtration - no shaking of the sample
39 %
DEV-Nutrient agar / pour plate method 78 %
mCP Agar / Filtration (24 hours TNT) 43 %
mCP Agar / Filtration (24 hours 4°C) 43 %
mCP Agar / Filtration (PT participants mean) 11 %
Dr. K. Luden
Warschau Nov. 2008 26
Clostridium perfringens
Medium/method usedRecovery
rate
Columbia blood agar / membrane filtration 100 %
Columbia blood agar / membrane filtration - no shaking of the sample
39 %
DEV-Nutrient agar / pour plate method 78 %
mCP Agar / Filtration (24 hours TNT) 43 %
mCP Agar / Filtration (24 hours 4°C) 43 %
mCP Agar / Filtration (PT participants mean) 11 %
Dr. K. Luden
Pseudomonas aeruginosa
Warschau Nov. 2008 28
E. coli
P. spe
cies
P. put
ida
Neg
ative
cont
r.
P. aer
ugino
sa
In PT 1-200730 % of labs failed because they reported a numerical result for P. aeruginosawhen the strain provided was not a P. aeruginosa
Possible explanation:
problems with confirmation reaction - Ammonia formation from acetamide
Dr. K. Luden
E. coli
ISO 9308-1 (n = 246)
E. coli Coliform bacteria delta
61 68 70 25 25
30 54 2415 16 121 22 124 25 1
Warschau Nov. 2008 29
A pure culture of E. coli was used in PT 4-2004
False results:
ISO 9308-1: 2,4 %Colilert®-18: 18,8 %
Colilert (n = 80)
E. coli Coliform bacteria delta
0 66 6666 70 462 70 874 78 40 78 780 83 83
74 83 930 43 1348 50 248 53 550 53 353 59 619 21 224 25 127 45 18
Dr. K. Luden
Conclusions
Warschau Nov. 2008 31
Interaction between filter and nutrient agar might be unfavorable for growth of the target organism
Performance of the material combinations used has to be checked (even if supplied with a certificate)
Negative controls are as important as positive controls
One can learn about the methods while testing performance of the lab
....
Dr. K. Luden
Most likely errors - E. coli / coliforms
Warschau Nov. 2008 33
Formal mistakes in the results: E. coli is part of the group of coliforms, therefore the number of E. coli cannot be larger than the number of total coliforms
Commercial identification systems (that are not to be used for the PT) might give conflicting results with the PT method
ISO 9803-1: 44°C Indole formation, wrong or not exactly controlled incubation temperature (use a waterbath)
Colilert®-18: measurements carried out at the wrong wavelength; cavities not properly filled.
Dr. K. Luden
Most likely errors colony counts
Warschau Nov. 2008 34
Contamination of the molten agar or the sample
Temperature of the agar too high: a slight increase in temperature might cause considerable decrease in numbers of bacteria that are able to form colonies; too low temperatures might lead to uneven distribution of bacteria as the agar starts clumping and proper mixing is prevented
Volume of agar is too small