PT/ILC Requirements -

R48 and some

implementation ideas.

(R80 also covered)

By

John Wilson

Accreditation and Metrology

Services (Pty) LTD.

Accreditation & Metrology Services -John Wilson 2017 1

What is PT ?

Accreditation & Metrology Services -John Wilson 2017 2

Unfortunately - In our case

– it is not ….

What is PT?

• PT stands for Proficiency Testing

• The Collins dictionary defines Proficiency as

an exam which test how proficient or skilledsomeone is in a particular activity, field of study, language, etc

• Similarly test (ing) (the noun) is defined as

A test is a series of questions that you must answer or actions that you must perform in order to show how much you know about a subject or how well you are able to do something.

Collins English Dictionary. Copyright © HarperCollins Publishers

Accreditation & Metrology Services -John Wilson 2017 3

What is an ILC ?

• ILC is the acronym for Inter Laboratory

Comparison.

• It is basically the same thing as PT and the

two words are interchangeable.

• Traditionally, PT has been used by Analytical

and Chemical Metrology Laboratories and ILC

by Physical Metrology Laboratories.

Accreditation & Metrology Services -John Wilson 2017 4

What is an ILC ?

• In medical laboratories, it is sometimes

known as EQA (external quality assurance).

• There is an international move to phase out

the use of ILC (words) and just use PT.

Accreditation & Metrology Services -John Wilson 2017 5

What is PT/ILC ?

• Neither PT or ILC are defined in the VIM

(Vocabulaire international de métrologie)

• More appropriately, there are definitions in

ISO/IEC 17025, ISO/IEC 17011, ISO/IEC 17043,

ILAC P9, SANAS A01, SANAS PM01, etc.

• We are today particularly interested in the

SANAS R48 and R80 documents and how they

effect the laboratory.

• R48-04 - Proficiency Testing and other comparison

programme requirements for Calibration laboratories.

• R80-03 – Proficiency Testing … for Testing & Medical

laboratories and Blood transfusion services.

Accreditation & Metrology Services -John Wilson 2017 6

What do SANAS require?

• The SANAS documents reflect largely what

the International documents require and the

requirements are similar to what most

accreditation bodies require of a laboratory.

• The R48 and R80 documents define the

detailed requirements for laboratories. (Read

them)!

• It all starts with a plan!

Accreditation & Metrology Services -John Wilson 2017 7

What do SANAS require?

• Calibration laboratories need to have a plan

that reflects the activities done in the last 5

years AND the plan for next 5 years.

• This plan must cover the listed parameters

and the ranges on the accreditation schedule

at least once in each 5 year period.

Accreditation & Metrology Services -John Wilson 2017 8

What do SANAS require?

• The Testing laboratories are required to have

a plan which covers their scope over two

accreditation cycles, one cycle in arrears and

one in the future.

• These plans differ from initial assessment PT

requirements where the lab has to prove it

can achieve the range and CMCs requested.

Accreditation & Metrology Services -John Wilson 2017 9

What do SANAS require?

• All plans must address the risk profile of the

laboratory and it’s activities. The new 17025

puts a much larger emphasis on risk analysis.

• All plans must be addressed during the

Laboratory Management meetings for

applicability, progress and performance.

• Details of what must be in the plan are listed

in the appropriate R48 or R80 documents.

Accreditation & Metrology Services -John Wilson 2017 10

What do SANAS require?

• Calibration laboratories are required to

participate in the National Audit Program and

may use these audits as part of the PT plan.

• All laboratories must preferably use PT

schemes that have independently been shown

to comply with the requirements of ISO/IEC

17043.

• All laboratories shall participate in PT

schemes for the items on their schedule

including specific tests or methods or where

specific instruments have been separately

listed. Calibration should also consider

including different methods in their plans.

Accreditation & Metrology Services -John Wilson 2017 11

What do SANAS require?

• Where no formal PT is available, Cal Labs shall

indicate other interlaboratory activities. Such an

ILC must still comply with the full R48 PT/ILC

activity and reporting requirements.

• Where no formal PT is available, Testing Labs

shall indicate suitable alternative means such as

reference materials, intralaboratory or other

comparisons. Again R80 compliance is required.

• The lack of commercially available PTs is not

considered a valid reason for non participation.

• Where PTs are not available or practical, suitable

alternatives may be proposed by the laboratory to

SANAS for consideration.

Accreditation & Metrology Services -John Wilson 2017 12

What do SANAS require?

• PT reports shall be clear and comprehensive

and have the information required by the R48

or R80 documents. (Identification (people and

items), responsibilities, evaluation methods

and conclusions are key).

• All laboratories shall review their own

performance and investigate all measurement

results. If some fail to meet the minimum

acceptance criteria then document the root

cause analysis and all corrective and

preventative actions taken. (i.e. raise a non

conformance in the quality system).

Accreditation & Metrology Services -John Wilson 2017 13

Common problems found

• Assessors have found the following problems

during assessments:-

• Failure to have a comprehensive plan.

• Failure to keep the plan up to date.

• Failure to adhere to the plan.

• Not using commercially accredited schemes

when they are available. (False economy).

• Participation with another lab and each lab

draws up it’s own report with different

conclusions. (i.e. no protocol done before ILC)

• Inappropriate choice of artefact and/or

reference value/material.

Accreditation & Metrology Services -John Wilson 2017 14

Common problems found

• Trying to pass off a PT for one device as

being adequate as competence to calibrate

another (Drying oven PT given for Autoclave).

• Failing to prepare a measurement protocol

before commencing the PT/ILC. (This means

different labs measure different points and

report differently so evaluation is not

possible).

• The report does not cover the required items

listed in the SANAS documents.

Accreditation & Metrology Services -John Wilson 2017 15

Common problems found

• Inadequate information in the report to

validate the findings. (e.g.. Results, En value

but no uncertainties recorded).

• No evaluation of the PT and it’s adequacy.

• One word reporting. (e.g. Conclusion –

“Acceptable” (even when results do not

overlap).

Accreditation & Metrology Services -John Wilson 2017 16

Considering Alternatives

• There is no benefit from evaluating a device

which has also been the source of

traceability.

• Intralaboratory comparisons only evaluates

one metrologist against another, usually the

procedure, equipment, traceability is exactly

the same. It does not confirm that the

procedure, equipment or traceability is valid.

Accreditation & Metrology Services -John Wilson 2017 17

Considering Alternatives

• To do a good ILC, it is not a matter of two or

more labs doing an intercomparison and then

expecting the assessor to do the evaluation

and decision making regarding who passes

and who fails. Things to consider are:-

• Decide on an artefact and check if it will be

adequate for the required accuracy or UoM

• Decide how the reference value will be

established.

• How will drift be monitored.

• Who will write an acceptable protocol.

Accreditation & Metrology Services -John Wilson 2017 18

Considering Alternatives

• Define how evaluation will be done. (En, Z)

• Who will do the evaluation.

• Who will write the report to the required

standard defined in the documents.

• The organising lab cannot usually use the ILC

to demonstrate it’s own performance,

especially if it provides the reference value

and the reference UoM.

• How will confidentiality be taken care of?

Accreditation & Metrology Services -John Wilson 2017 19

Conclusion

• PT/ILCs are definitely an integral part of an

accredited laboratory’s life.

• They should add value and confidence to the

laboratory (not just satisfy the assessor).

• They are a wonderful training tool for

evaluating new metrologists progress and

provides a source of CPD points for the

metrologist concerned.

Accreditation & Metrology Services -John Wilson 2017 20

Conclusion

• Be creative and innovative when planning the

PT schedule so that it is interesting and value

adding.

Accreditation & Metrology Services -John Wilson 2017 21

Conclusion

• Thanks to Neville Tayler from SANAS for his

valuable information.

• Thanks to Steve Sidney from the NLA for his

contribution based on running the NLA PT/ILC

accredited schemes.

• Thanks to all the assessors who diligently

work with the laboratories during

assessments to make sure that the PT/ILC

plans are reasonable and within the correct

risk profile for the laboratory

• Thanks to you for listening.

Accreditation & Metrology Services -John Wilson 2017 22

PT/ILCs – here to stay!

Accreditation & Metrology Services -John Wilson 2017 23

John WilsonCell : +27 83 652 0770eMail : jgpwilson@xsinet.co.za