199

Accred Qual Assur (2001) 6:199–202© Springer-Verlag 2001

Manfred Golze

Information system and qualifying criteriafor proficiency testingschemes

Council (DAR), jointly took the initiativeto set up an information system on inter-laboratory comparisons in Austria and Ger-many. As a result the Internet informationsystem IRIS was published in 1998 [2] andis still available.

However, this was only a limited solu-tion and at the European level an appropri-ate information system was still lacking.Therefore, the European Co-operation forAccreditation (EA), EURACHEM andEUROLAB have established a network ofcompetent organizations in 16 MemberStates of the European Union and the European Free Trade Association (EFTA).A project proposal was submitted to and accepted by the European Commis-sion. Thus within the 4th framework pro-gram the European Commission has spon-sored a concerted action “InformationSystem and Qualifying Criteria for Proficiency Testing Schemes” (contractSMT4-CT98–8002) since October 1998.As a major result of this project the European Information System on Profi-ciency Testing Schemes (EPTIS) is nowpublicly available on the World Wide Web(http://www.eptis.bam.de) since March2000.

EPTIS

EPTIS is an online database of regularlyorganized proficiency testing schemes(PTS) from the 16 participating Europeancountries. As of September 2000 it con-tains approximately 640 schemes from abroad range of different testing fields.Metrological and clinical PTS are not in-cluded.

The procedure to develop EPTIS

The information presented by EPTIS wascollected by means of a questionnairewhich had been developed on the basis ofthe ISO/IEC Guide 43 Part 1 “Develop-ment and operation of proficiency testingschemes” [3] and agreed by all partners atthe beginning of the project. In most coun-tries this questionnaire was translated intothe national languages and then forwardedto the known proficiency test providers.The national coordinators collected thecompleted questionnaires and after a plau-sibility check and, if necessary, in consul-tation with the respective provider they fedthe data into the system. It is a feature ofEPTIS which is physically located at aBAM server that the input of data into the database is done via the Internet (seeFig. 1) and that each national coordinatoris responsible for the maintenance of all in-formation concerning the PTS in his coun-try. However, it should be emphasized thatall information is based on self-declarationof the PT providers and that the EPTISconsortium cannot take any responsibilityfor the correctness of the data.

The structure and content of the information system

EPTIS consists of three parts:– A searchable database with general in-

formation– Linked web-pages with complementary

information– Links to homepages of the providers (if

available)The general information, i.e., informationabout the PT provider and the basic fea-tures of his schemes (for more details seeTable 1), is the kind of information whichone would expect from each catalogue of

POLICIES AND CONCEPTS

Abstract Since October 1998 the Europe-an Commission has financed a concertedaction on Information System and Qualify-ing Criteria for Proficiency TestingSchemes within the 4th framework pro-gram. As a major result of this project EP-TIS, the European Information System onProficiency Testing Schemes which isavailable on the Internet since March 2000,is presented in this paper. Today EPTIScontains comprehensive information on ap-proximately 640 proficiency testingschemes from 16 European countries pro-viding information on the state of the art inproficiency testing in Europe. Finally somepossible approaches for interlinkages andrecognition of proficiency testing schemesare discussed.

Keywords Proficiency testing · EPTIS · Quality characteristics · Accreditation

Introduction

In recent years both laboratories and ac-creditation bodies have increasingly real-ized the importance of proficiency tests.They are an excellent tool to assess and todemonstrate the performance of laborato-ries with regard to specific measurement,testing, and analytical tasks and they canunderpin the usual accreditation proce-dures. Therefore, proficiency tests are con-fidence building means in the laboratoriesinternally and in the relation to their cli-ents, and indirectly they add to the reputa-tion of accreditation. Despite these benefitsthe participation in proficiency tests washampered not only by the restricted offerof suitable proficiency testing schemes inmany testing areas but simply by a lack ofinformation on their availability [1].

To improve this situation the FederalInstitute for Materials Research and Test-ing (BAM) and the Austrian Federal Min-istry for Economy and Labour (BMWA), ata suggestion of the German Accreditation Fig. 1 Entry of data into EPTIS by national coordinators is done via the Internet

PTS. It is provided by a relational dat-abase, which makes a selective inquirypossible. However, the complementary in-formation (see Table 1), e.g., on the man-agement of the scheme, the test item char-acterization, and the statistical analysisused, is a special feature of EPTIS. It isoriented in accordance with the ISO/IECGuide 43 [3] and should enable the potential users of PTS to make a carefuland profound choice. We hope that theparticipation in proficiency tests mightthus be augmented and the quality of PTSmight be fostered and improved. Thiscomplementary information is supplied bya web page, which is linked to the respec-tive data set in the database. Such a struc-ture offers better possibilities of the main-tenance of the data compared to a lessflexible and more complex database. Lastbut not least the links to the homepages ofthe PT providers, if available, involvethem in the information system and enablethem to provide additional and up-to-dateinformation, e.g., on the date of the nextround.

A preliminary analysis of the EPTIScontent was performed in July 2000 basedon 590 PTS at that time. The distributionof the PTS and of the PT providers amongthe participating countries is shown in Fig.2. As can be clearly seen in some Europe-an countries rather few PTS are offeredand thus an information system at a supra-national level can facilitate the PT partici-pation of laboratories from the smallercountries.

Figure 3 presents a summary of theEPTIS content in terms of, e.g., testingfields, product groups, and testing meth-ods. From a breakdown of the testing fieldscovered by the PTS (Fig. 4) it can be de-duced that chemical PTS dealing with ana-lytical chemistry in general or with envi-ronmental analysis are greatly dominating.A growing field with regard to proficiencytesting is microbiology. The most oftentested products are food and beverages,water, and soil, sludge, and contaminatedsites. It can be concluded that at the presenttime PT schemes are still lacking in manyfields of materials and product testing.

200

The future of EPTIS

The concerted action will be finished verysoon and there will be no further fundingby the European Commission. Neverthe-less during the last project meeting thepartners agreed to keep EPTIS running.This means that all the partners are willingto act as national coordinators, to keep thecontacts with the relevant PT providers, toupdate the information about their respec-tive national PTS, and to give support toEPTIS users. In addition all the partnerswill jointly act as a steering committee totake decisions on the EPTIS development.Thus the mid-term existence of EPTIS isensured and this is encouraging consider-ing the growing resonance from the EPTISusers that has been registered since EPTISwas launched (March 2000).

Because of the flexible structure, EP-TIS can easily be extended with regard tothe regional as well as the technical scope.From a European point of view it might beadequate to include also those countrieswhich are applicants to the European

Table 1 Overview of the information offered for each PTS

General information Complementary information(searchable information given in a data base) (available as web-pages)

Information about the PT-provider Conditions for participationName of the provider Documentation / management of the PTSAddress Test item characterizationTelephone, fax Statistical analysis of testing results and assessment of the laboratoriesContact person Exchange of information between the PTS provider and the participantsE-mail, www-address

Βasic information about the PTSDesignation of the PTSTesting fieldProduct/set of productsTest itemTested propertyTesting methodStandard series/guidelines

Fig. 2 Distribution of the PTS and PT pro-viders among the participating countries(as of September 2000)

201

Union. Concerning the multilateral recog-nition arrangements (MRAs) of the accred-itation bodies an even broader scope wouldbe helpful. Consequently APLAC (AsianPacific Laboratory Accreditation Co-opera-tion) and ILAC (International LaboratoryAccreditation Co-operation) have indicatedsome interest to join EPTIS. However, anyextension would cause additional costs andmight give rise to the need for new coordi-nators responsible for the additional tasks.

Existing databases with a similar con-tent could of course be easily linked withEPTIS.

Quality characteristics for proficiency testing schemes

As mentioned above the questionnaire usedto collect the information from the PT pro-viders was based on the ISO/IEC Guide 43Part 1 [3]. Therefore, the data contained inEPTIS can be used to analyze the degree ofcompliance of an individual PT provider orof the majority of the providers with the re-quirements of this guide. In Table 2 a listof items concerning

– Documentation/management of thescheme

– Test item characterization and manage-ment

– Analysis of testing results and evalua-tion of performance

– Exchange of information between thePT providers and the participants

Fig. 3 Summary of the EPTIS content (as of July 2000)

Fig. 4 Distribution of testing fields covered by the PTS (as of July 2000)

Table 2 List of quality characteristics for PTS (without ranking priority)

Documentation/Management1. Written information describing the PTS, its aims and its operation2. Advisory/steering group for organizational, technical, and statistical questions3. Documentation for essential operations of the PTS4. Policy to maintain confidentiality of individual participants’ identity

Test item characterization and management5. Information about the producer and the production of the test items (e.g., sampling, sample processing, homogeneity testing,

stability testing, and assigned values)6. Information about special requirements on packaging, transport, and storage7. Documentation for management (e.g., storage) of retained test items8. Information about any test items left for quality assurance testing after completion of the PTS

Analysis of testing results and evaluation of performance9. Description of the procedure used to establish the assigned values (e.g., with reference to standards or publications)

10. Specification of the uncertainty of the assigned values11. Description of the statistical analysis of the testing results including the treatment of outliers (e.g., with reference to standards

or publications)12. Description of the assessment of the testing results (e.g., with reference to standards or publications)

Exchange of information between the PTS provider and the participants13. Information/instructions for the laboratories about the handling of the test items and, if applicable, for the performance of the test14. Instructions with regard to the reporting of the results and their format15. Information about the time schedule of the whole PTS16. Elaboration of a PT-protocol or PT-report for the participants of the PT-scheme17. Technical advice for improvement of laboratories’ performance

is given which are addressed in ISO/IECGuide 43 and which are fulfilled at least bythe majority of the PT providers accordingto their self-declaration. From this analysisone gets an idea of the state-of-practice ofthe PT providers in Europe. It can be con-cluded that about 85% of the providershave already implemented or are at leastplanning to establish a quality system fortheir respective PT-schemes and that arather high quality level has already beenreached. Of course there are some issuesthat need more consideration in the future,e.g., uncertainty statements. The EPTISconsortium has decided to include this listof quality characteristics in EPTIS so thatlaboratories, which are looking for suitablePTS, can take them into account whenmaking their choice.

Again it should be emphasized that thisoverview of the state-of-practice is merelybased on the information provided by thePT organizers themselves. No third partyassessment was performed.

International comparabilityof proficiency testing schemesA mutual recognition of PTS which isbased on their international comparabilityis an important issue which would help

– The participating laboratories becausethey have to rely on the validity of thePTS

– The PT organizers because it couldstrengthen their market position

– The accreditation bodies with regard topeer evaluations by other accreditationbodies

The whole issue is dealt with in more de-tail in a discussion paper which was pre-pared by the EPTIS consortium and intend-ed as an input for the 3rd EURACHEM/EQALM Workshop on proficiency testingin analytical chemistry, microbiology andlaboratory medicine [4]. Different ap-proaches, such as

– Accreditation of PT providers– Intercomparisons of PT providers– Mutual participation in different PT-

schemes

are mentioned there.It is not possible to discuss in this paper

the pros and cons of a possible accredita-tion of PT providers, a service which is al-ready being offered by some accreditationbodies and is based on ISO/IEC Guide 43

[3] and ILAC Guide G13 [5]. Here I wantto focus on intercomparisons between PTproviders. Such intercomparisons might behelpful in assessing to what extent the re-quirements of ISO/IEC Guide 43 [3] arefulfilled and to what extent they provide asufficient basis for comparability. Compa-rability is certainly not only based on howa PTS is organized or on the statistics ap-plied, but it strongly depends on the trace-ability of the reference values, which areused in the PT.

One way in which such an intercompar-ison could be accomplished would be thatthe same test material would be distributedin similar schemes and the data be pro-cessed in the usual way. In addition, thedata from all schemes would be processedby a coordinator enabling a comparison ofthe results. The procedure of the intercom-parison could be as follows:

– First meeting of the participating PTproviders to agree on technical, meth-odological and statistical issues

– Feasibility study on statistics to checkthe appropriateness of the agreed statis-tical method for the overall evaluation

– Preparation and distribution of testitems with the reference values of thetested properties ascertained by refer-ence laboratories but not disclosed tothe PT-providers

– Performance of the PTS by the differentproviders in their usual way and reporton their results

– Evaluation and statistical analysis ofthe results by the coordinator

– Final discussion with the PT-providersand conclusions

The EPTIS consortium proposed to orga-nize a workshop to establish the willing-ness of PT providers for assessing theequivalence of the existing schemes in Eu-rope.

Conclusions

As a main result of the concerted action,EPTIS was developed and is now publiclyavailable on the World Wide Web. Becauseof the increasing demand for informationon proficiency tests and the encouragingresonance from EPTIS users so far, the de-cision of the partners to maintain EPTISafter ceasing of the EU funding is impor-tant. Possible extensions of the informationsystem might open up new possibilities forat least preliminary new funding.

Based on the data collected in EPTISan analysis with regard to the state-of-prac-tice could be performed which shows arather high level of compliance withISO/IEC Guide 43. To assess the compara-bility of similar PTS an intercomparison ofPT providers is suggested.

202

Acknowledgements Financial support ofthe concerted action by the European Com-mission under contract SMT4-CT98–8002is gratefully acknowledged. The results ofthe project presented in this paper are dueto a close and fruitful cooperation of allpartners in the EPTIS consortium forwhich I am extremely grateful. In additionessential contributions were provided by aproject team at BAM. In particular I feelbound to underline the important role ofNazmir Presser in the application and start-ing phase of the project. Last but not leastthe support from EA, EURACHEM, andEUROLAB was necessary from the verybeginning and will remain so in the futureto keep the EPTIS consortium alive whenthe financial support by the EuropeanCommission will have ceased.

References

1. EURACHEM (1999) Draft guide on se-lection, use and interpretation of profi-ciency testing schemes by laboratories.http://www.vtt.fi/ket/eurachem

2. Presser N (1998) DAR-Aktuell 3:1–23. ISO/IEC Guide 43–1 (1997) Proficiency

testing by interlaboratory comparisons –part 1: development and operation ofproficiency testing schemes. ISO/IEC,Geneva

4. EPTIS consortium (2000) Discussionpaper on interlinkages and recognitionof proficiency testing schemes, present-ed at 3rd Eurachem/EQALM workshopon proficiency testing in analyticalchemistry, microbiology and laboratorymedicine, Borås, 25–26 September 2000

5. ILAC Guide G13 (2000) Guidelines forthe requirements for the competence ofproviders of proficiency testingschemes. http://www.ilac.org

Presented at the EURACHEM/EQUALMWorkshop “Proficiency Testing in Analyti-cal Chemistry, Microbiology and Labora-tory Medicine”, 24–26 September 2000,Boras, Sweden

M. GolzeFederal Institute for Materials Researchand Testing (BAM), 12200 Berlin,Germanye-mail: manfred.golze@bam.de