Warschau Nov. 20081 Dr. K. Luden Lab Quality Assurance by Interlaboratory Comparisons Proficiency Testing Dr. Katrin Luden

Warschau Nov. 2008 1

Dr. K. Luden

Lab Quality Assurance by Interlaboratory

Comparisons

Proficiency Testing

Dr. Katrin Luden

Dr. K. Luden

Drinking water directive 98/83/EC

Annex III

Specifications for the analysis of Parameters

Each Member State must ensure that any laboratory at which samples are analysed has a system of analytical quality control that is subject from time to time to checking by a person who is not under the control of the laboratory and who is approved by the competent authority for that purpose.


Dr. K. Luden

German drinking water directive

§ 15 Methods of Analysis and Laboratories

(4) The nessessary analysis including sampling are to be conducted by laboratories that work by “generally accepted rules of the technology” (allgemein anerkannte Regeln der Technik), have a system of internal quality control, participate at least once a year in external quality assurance schemes, have sufficiently qualified personell and are accredited by a generally recognized accreditation body. The responsible highest federal authority will publish a list of laboratories in its jurisdiction.

(5) The authorities (of the federal state) independent of the labs will have to check regularly wether the conditions of (4) are still fullfilled.


Dr. K. Luden

Proficiency testing

Participation in proficiency testing programs allows laboratories to:

identify areas where improvement in their testing and measurement methods is needed

identify further training of their staff

foster confidence in the performance of their testing and measurements

assure laboratory performance in their accredited test and calibration results


Dr. K. Luden

Jungle of standards for PT / EQA


There are several standards and guidelines for providing PT / EQA schemes (in chronological order)

• ISO/IEC Guide 43-1:1996 Proficiency testing by Interlaboratory Comparisons – Part 1: Development and Operation of Proficiency testing Schemes

• ISO 13528:2005 Statistical Methods for use in Proficiency Testing by Interlaboratory comparisons

• IUPAC Technical Report 1/2006 The International Harmonized Protocol for the Proficiency Testing of Analytical Chemistry Laboratories

• ILAC-G13:2007 ILAC Guidelines for the Competence of Providers of Proficiency Testing Schemes

• ISO/TS 20612 Water Quality – Interlaboratory comparisons for proficiency testing of analytical chemistry laboratories

• ISO/IEC CD 17043:2008 Conformity Assessment – General Requirements for Proficiency Testing

Dr. K. Luden

External Quality Assurance

Recommendation of Federal Environmental Agency 2002 - Microbiology


Dr. K. Luden



Parametersuccessfull participation in

proficiency testing schemes

E. coli/Coliform bacteriaEnterococciColony Count

2x / year

Clostridium perfringensLegionellaPseudomonas aeruginosa

1x / year

Enteropathogenic VirusesCryptosporidium parvum

International PTS

Recommendation of Federal Environmental Agency 2002 - Microbiology

Dr. K. Luden



Parametersuccessfull participation in

proficiency testing schemes

All parameters offered by a laboratory

1x within 2-3 years

Recommendation of Federal Environmental Agency 2003 - Chemistry

Proficiency testing (PT) according to DIN 38402-45

Dr. K. Luden

German drinking water PT providers


Microbiology and “hygenic-chemical PT

Chemistry PT

Chemistry PT

Chemistry PT

Regular meetings, harmonized schedules, identical statistical evaluation → comparability for labs and authorities

Dr. K. Luden

DIN 38402-45


Standard may be applied if:

Parameter analysed can be considered continuously measureable

true for most chemical and phisico-chemical parameters

not allways true for biological and/or microbiological parameters

Application in Chemistry: Yes

Application in microbiology: by way of trial / mostly

Dr. K. Luden

Microbiology PT


The governmental Institute of Public Health of Lower Saxony (NLGA) is Proficency Test (PT) provider for microbiological measurements in drinking water in Germany

Every year:

4 PTS for E. coli/coliform bacteria, Enterococci, and Colony Counts

2 PTS for Pseudomonas aeruginosa, Clostridium perfringens and Legionella

1 PTS for EU bathing water (E. coli, coliforme B.) 1 PTS for selected “hygenic-chemical” parameters

Dr. K. Luden

Microbiology PT


Parameter Qualified methods1986 -1999

2000

2001

2002

2003

2004

2006

2007

2008

E. coli-coliform bacteria

presence/absence testDIN EN ISO 9308-1Colilert®-18/Quanti-Tray®

+(+)

+(+)

+(+)

+(+)

(+)++

(+)++

(+)++

(+)++

(+)++

Colony CountGerman DWD 1990DIN EN ISO 6222

+ + ++

++

++

++

++

++

++

EnterococciEN ISO 7899-2Chromocult®-Enterococci-Agar

(+) (+) + + + ++

++

++

Clostridium perfringens

DWD (mCP-Agar)TSC

(+)(+)

++

++

++

++

++

++

Pseudomonas aeruginosa

EN 12780/ISO 16266 + + + + + +

Legionella spec. GVPC (ISO 11731-2) + + + + +

EC bathing water directive

Feacal coliform bacteria

+ + + + + + (+)


DIN EN ISO 9803-3 (+) + +

Intestinal Enterococci

DIN EN ISO 7899-2DIN EN ISO 7899-1

(+)

(+)

+

+

+

+

Dr. K. Luden

Microbiology PT


Parameter Qualified methods1986 -1999

2000

2001

2002

2003

2004

2006

2007

2008


presence/absence testDIN EN ISO 9308-1Colilert®-18/Quanti-Tray®

+(+)

+(+)

+(+)

+(+)

(+)++

(+)++

(+)++

(+)++

(+)++

Colony CountGerman DWD 1990DIN EN ISO 6222

+ + ++

++

++

++

++

++

++

EnterococciEN ISO 7899-2Chromocult®-Enterococci-Agar

(+) (+) + + + ++

++

++


DWD (mCP-Agar)TSC

(+)(+)

++

++

++

++

++

++


EN 12780/ISO 16266 + + + + + +

Legionella spec. GVPC (ISO 11731-2) + + + + +

EC bathing water directive

Feacal coliform bacteria

+ + + + + + (+)


DIN EN ISO 9803-3 (+) + +

Intestinal Enterococci

DIN EN ISO 7899-2DIN EN ISO 7899-1

(+)

(+)

+

+

+

+

Dr. K. Luden

PT samples - two options


Liquid samples Freeze dried material

Bacterial solution in defined mineral medium

Bacteria adsorbed to milk powder

Very realistic samples

Handling exactly like a real sample

A solution has to be prepared and first step of handling procedure is different from real samples

Stability 4-6 days after shipment at <10°C

Stability at -20°C weeks or months

Germany United Kingdom, France, Sweden, Australia...

Dr. K. Luden

Preparation at NLGA


Dr. K. Luden

Shipping


300 - 500 participants/PT

Registered:

> 700 Labs from GermanyAustria (26) Switzerland (8) Lithuania (2) Hungary (2) Great Britain (1) Luxembourg (1)

Dr. K. Luden

Schedule of a proficiency testing scheme


Registration of participants

Confirmation of the registration (4 weeks before PT)

Preparation of the samples and shipment

Internal quality control (two weeks before and 6 days after shipment)

Reporting of the results (within 16 days of shipment)

Evaluation of the results usually within 4 weeks after the deadline for reporting

Dr. K. Luden

What do the labs get?


Report on the PT design, special problems, interestingresults etc.

Certificate and evaluation report(target value as consensus value of participants results)

Quality control charts about the influence of temperature and the duration of shipping

Z(u)-Scores of all measurements in tables

Z(u)-Scores in graphic form

Dr. K. Luden


Statistics of DIN 38402-45

Robust statistics (no outlier tests); consensus mean (used as assigned value) calculated by Hampel Estimator

ZU-scores “normalize” the results for comparing different levels of bacterial concentration

Z-score: lZUl 2 - sufficient

lZUl 3 - questionable

lZUl > 3 - insufficient

m-4,5s m-3s m-1,5s m m+1,5s m+3s m+4,5s

t

t

s

mxZ

)(

Dr. K. Luden


Name/Adress of PT-provider

Address of participant

Identification of PT (e.g. 3-2008)

Table of parameters, successfully analyzed param. And method used

Signatures of PT provider

Certificate

Dr. K. Luden

Microbiology PT


ZU-s

core

Group B III-2008 E. coli (assigned value 21 CFU/100 ml)

Labcode

Dr. K. Luden

Interlaboratory comparisons


method validation studies:aptitude/qualification of the lab is prerequisitereference material is fit for purpose

to certify reference material:method is supposed to be fit for purpose/validlabs are expert labs

Quality control/proficiency testing:performance of a lab is tested, reference material and method are supposed to be valid/fit for purpose

Dr. K. Luden


PT 1-2003

E. coli

0

5

10

15

20

25

30

35

40

45

R R R R R R R R R R 5 428

3 35

183

12

162

406

445

543

178

56

340

437

459

219

328

442

469

284

367

460

116

279

149

545

522

391

60

386

36

Labcode

E. c

oli

CF

U /

100

ml

E. coli / 100 ml X-3s X-2s X X+2s X+3s

Dr. K. Luden


PT 2-2004

Clostridium perfringensC

FU

/100

ml

Labcode

Dr. K. Luden



Medium/method usedRecovery

rate

Columbia blood agar / membrane filtration 100 %

Columbia blood agar / membrane filtration - no shaking of the sample

39 %

DEV-Nutrient agar / pour plate method 78 %

mCP Agar / Filtration (24 hours TNT) 43 %

mCP Agar / Filtration (24 hours 4°C) 43 %

mCP Agar / Filtration (PT participants mean) 11 %

Dr. K. Luden



Medium/method usedRecovery

rate

Columbia blood agar / membrane filtration 100 %

Columbia blood agar / membrane filtration - no shaking of the sample

39 %

DEV-Nutrient agar / pour plate method 78 %

mCP Agar / Filtration (24 hours TNT) 43 %

mCP Agar / Filtration (24 hours 4°C) 43 %

mCP Agar / Filtration (PT participants mean) 11 %

Dr. K. Luden


Dr. K. Luden



E. coli

P. spe

cies

P. put

ida

Neg

ative

cont

r.

P. aer

ugino

sa

In PT 1-200730 % of labs failed because they reported a numerical result for P. aeruginosawhen the strain provided was not a P. aeruginosa

Possible explanation:

problems with confirmation reaction - Ammonia formation from acetamide

Dr. K. Luden

E. coli

ISO 9308-1 (n = 246)

E. coli Coliform bacteria delta

61 68 70 25 25

30 54 2415 16 121 22 124 25 1


A pure culture of E. coli was used in PT 4-2004

False results:

ISO 9308-1: 2,4 %Colilert®-18: 18,8 %

Colilert (n = 80)

E. coli Coliform bacteria delta

0 66 6666 70 462 70 874 78 40 78 780 83 83

74 83 930 43 1348 50 248 53 550 53 353 59 619 21 224 25 127 45 18

Dr. K. Luden

Explanation?


ß-Galactosidase

ß-Glucuronidase

Dr. K. Luden

Conclusions


Interaction between filter and nutrient agar might be unfavorable for growth of the target organism

Performance of the material combinations used has to be checked (even if supplied with a certificate)

Negative controls are as important as positive controls

One can learn about the methods while testing performance of the lab

....

Dr. K. Luden

Thank you for your attention!


Dr. K. Luden

Most likely errors - E. coli / coliforms


Formal mistakes in the results: E. coli is part of the group of coliforms, therefore the number of E. coli cannot be larger than the number of total coliforms

Commercial identification systems (that are not to be used for the PT) might give conflicting results with the PT method

ISO 9803-1: 44°C Indole formation, wrong or not exactly controlled incubation temperature (use a waterbath)

Colilert®-18: measurements carried out at the wrong wavelength; cavities not properly filled.

Dr. K. Luden

Most likely errors colony counts


Contamination of the molten agar or the sample

Temperature of the agar too high: a slight increase in temperature might cause considerable decrease in numbers of bacteria that are able to form colonies; too low temperatures might lead to uneven distribution of bacteria as the agar starts clumping and proper mixing is prevented

Volume of agar is too small

Documents

Warschau Nov. 20081 Dr. K. Luden Lab Quality Assurance by Interlaboratory Comparisons Proficiency Testing Dr. Katrin Luden