A new sampling algorithm for use with ATP testing

Greg Whiteley Managing Director – Whiteley Corporation

Declaration of conflict of interest

• None to declare: with respect to this presentation

• I have no association with any brand of ATP device, nor does my company, nor do any of my affiliates

• I have no financial links to ATP testing, other than to use these devices and conduct validation research through WSU

• All data referenced in regards to any ATP testing device has been peer reviewed and internationally published except where noted that writing for submission, or current peer review is still underway

Project Design features

Cleaning processes

Lets change just one item and use a control group

Outcome: Measurable cleanliness

Should be a snack!

We looked for an ICU

The ICU manager wasn’t happy with the micro results after the hygiene audit

Background on ATP testing

Advantages:

Easy to use; real time results; broad indicator of cellular contamination

Problems:

Variability & imprecision; relative scaling; lack of brand to brand interoperability; sampling error

Early Validation work on ATP testing devices

• Each device uses a unique

scale – all named ‘RLU’

[Relative Light Units]

• The Lower Limit of

Quantitation is different

from the lower level of

detection for some brands

• Several brands do not

read down to zero, so a

practical zero is required

leading to user confusion

• Imprecision is virtually

undetectable in use Whiteley et al., Healthcare Infection:2012:17:91-97

y = 6E-05x - 0.0242R² = 1

0

2

4

6

8

10

12

0 20000 40000 60000 80000 100000 120000 140000 160000 180000 200000

Co

nce

ntr

atio

n (

pp

m)

Mean Peak Area

HPLC Calibration curve using pure ATP

Calibration testing of ATP testing devices compared to a

laboratory standard analytical tool - HPLC

Variance measurements Cv = σ /

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 10 20 30 40 50 60

Co

V

Number of dilutions tested for each unit

Coeffecient of Variance CoV for three portable ATP bioluminometers

Cleantrace 3M [n=57 (246 swabs)]

Kikkoman [n=49 (222 swabs)]

Hygiena [n=47(199 swabs)]

LCMS [n=22 (72 runs)]

n = number of separatedilutions tested per brand

Whiteley et al., Infect Control Hosp Epidemiol:2013:34:538-540

Whiteley et al., ICHE 2015

Summary findings on variability

Whiteley et al., Infect Control Hosp Epidemiol:2015:36:658

Key Findings

1. The variability with bacteria is the same as the variability with the pure source ATP

2. With a Cv of 0.4, any reading has a 20% chance to be wrong by a factor of two: i.e. the error potential on a reading of 100 RLU is from 50 RLU to 200 RLU.

3. Readings using more than a single point are required for statistical validity.

Finding the bad bugs in a busy ICU

Ref: Whiteley et al., Am J Infect Control:2015:43:1270-5

So where are the bad bugs?

A cross sectional ICU pilot survey

Data: pre-publication Knight, Whiteley, Jensen, Gosbel and others., 2016

SO WE DEVELOPED A NEW SAMPLING ALGORITHM

We needed to improve certainty over the ATP readings

Key features of the new sampling algorithm

• Starts with duplicate sampling on proximal surfaces of an HTO or RMD

• Uses a standardised sampling area – normally 2cm x 5cm = 10cm2

• Compares results to a predetermined initial cleanliness threshold

• Uses a cleaning intervention step for internal validation of cleanability & cleanliness

A new ATP testing Sampling Algorithm

2 samples both

RLU <100 RLU*

Cleaning Intervention

Step

Resample aiming for cleanliness at

< 50 RLU

2 samples both

RLU > 100 RLU*

Cleaning Intervention

Step

Resample aiming for cleanliness at <50RLU

Cleanliness intervention step to repeat and sample

repeat until cleanliness is < 50 RLU

2+ samples: one < 100 RLU*

& one > 100 RLU*

Continue sampling for up to four samples to indicate outlier effect

Cleaning Intervention

Step

Cleanliness intervention step to repeat and sample

repeat until cleanliness is <50RLU

* 100 RLU measure is specific only to Hygiena

Food Premises Cleanliness Study*

• Conducted in regional location in NSW

• 8 Food Premises & 72 items examined

• Statistics using Mann Whitney and Wilcoxon

• Cleaning Intervention Step using disposable detergent (anionic) wipe

• Cleaning Principle: “one wipe, used on one surface, wiping in only one direction”**[**Sattar & Maillard, 2013. Am J Infect Control:41:S97-S104]

*Paper: Whiteley Nolan & Fahey: J Env Health (USA) under peer review, 2016

Results: Spread of duplicate samples with

Line of best fit between duplicate samples

Paper: Whiteley Nolan & Fahey: J Env Health (USA) under peer review, 2016

Before and after Cleaning Intervention Step results

Paper: Whiteley Nolan & Fahey: J Env Health (USA) under peer review, 2016

Key findings from the Food Premises survey

• Before and after findings – all statistically significant (P>.001) for all categories except <25 RLU (P=0.136)

Classification Before After Significance

Clean 2x<100 RLU

19 6 P = 0.001

Unclean2x > 100RLU

922 10 P = 0.001

Mixed± 100RLU

192 5 P = 0.001

Very CleanAv < 25 RLU

8 5 P = 0.136

Paper: Whiteley Nolan & Fahey: J Env Health (USA) under peer review, 2016

The algorithm mapped – IDH 2016

Paper: Whiteley Glasbey Fahey, IDH 2016: pending DOI

Key comparative charts

Cleanliness Threshold

Hygiena Cleantrace Kikkoman

Initial Cleanliness threshold TC1 100 RLU 500 RLU 460 RLU

Secondary Cleanliness threshold TC2 50 RLU 250 RLU 230 RLU

Tertiary Cleanliness threshold TC3 25 RLU 125 RLU 115 RLU

Lower Limit of Quantitation LLQ 0 RLU 100 RLU 90 RLU

Identified Cleanliness Threshold

Defining the Cleanliness Thresholds relationships

TC1 TC1 = TC2 x 2 = TC3 x 4

TC2 TC2 = TC1 ÷ 2 = TC3 x 2

TC3 TC3 = LLQ + 25 RLU (may be higher for some devices)

Table 1 from the paper

Table 2 from the paper

Paper: Whiteley Glasbey Fahey, IDH 2016: pending DOI

Medical Device Study

• Survey is currently underway and on-going

• Tested 258 individual surfaces and items so far

• Over 1000 swabs so far

• Cleaning intervention step using two different wipes – testing difference in cleaning outcomes between wipes

• Statistical analysis – standard methods

Conclusions

1. New sampling algorithm improves certainty when using ATP testing devices;

2. Algorithm is simple to use in field hygiene assessments with slightly increased costs;

3. New sampling algorithm requires further study in well designed trials to test validity and precision improvement of ATP testing…

Acknowledgements

• Dr Trevor Glasbey, Regulatory and Research Manager, Whiteley Corporation

• Paul Fahey, Statistician, Western Sydney University

• Mark Nolan, Environmental Health Officer, National Parks and Wildlife Service of NSW

• ASUM

• ASUM Staff: Lyndal Macpherson, Associate Professor Sue Westaway, Dr Jocelyne Basseal, Ann-Marie Gibbons

• Jessica Knight, Professor Iain Gosbell, and Associate Professor Slade Jensen – all based at Western Sydney University and the Ingham Research Institute

• Thank youThank you

Questions?