Principles of microbiological testing: Statistical basis ... · ‘Probability that an acceptable lot is rejected’ Three-Class Attributes Sampling Plans Three-class sampling plans

1

Principles of microbiological Principles of microbiological testing: Statistical basis of testing: Statistical basis of

samplingsampling

Martin ColeSymposium on Relating Microbiological Testing and Microbiological Criteria to Public Health GoalsOctober 31-November 1, 2005

OverviewOverview

Sampling plansSampling plansICMSF CasesICMSF CasesStatistical basisStatistical basisPerformance of sampling plansPerformance of sampling plansSummary and ConclusionsSummary and Conclusions

2

Sampling PlansSampling Plans

Define the probability of Define the probability of detecting a microorganisms or detecting a microorganisms or other hazards in a lotother hazards in a lotNone can ensure the absence of a None can ensure the absence of a particular hazardparticular hazardShould be administratively and Should be administratively and economically feasibleeconomically feasible

Types of Microbiological Types of Microbiological Sampling PlansSampling Plans

Attributes plans:

Qualitative analytical results (presence/absence) orquantitative results that have been grouped(e.g. <10 cfu/g, 10 to 100 cfu/g, >100 cfu/g)

Variables plans:

Non-grouped quantitative analytical results

Require distributional assumptions be made

3

TwoTwo--Class Attributes Class Attributes Sampling PlansSampling Plans

TwoTwo--class sampling plansclass sampling plans designed to decide designed to decide on acceptance or rejection of a lot consist ofon acceptance or rejection of a lot consist of

nn –– number of sample units to be chosen number of sample units to be chosen independently and randomly from the lotindependently and randomly from the lotmm –– a microbiological limit (i.e. in cfu/g);a microbiological limit (i.e. in cfu/g);a sample is defined to be positive, if its a sample is defined to be positive, if its microbial content exceeds this limitmicrobial content exceeds this limitcc –– maximum allowable number of sample maximum allowable number of sample units yielding a positive result units yielding a positive result (presence/absence testing) or exceeding the (presence/absence testing) or exceeding the microbiological limit m; for pathogens c is microbiological limit m; for pathogens c is usually set to 0usually set to 0

Log cfu/g

Pro

babi

lity

Den

sity

0 1 2 3 4 5 6

0.0

0.1

0.2

0.3

0.4

0.5

0.6m

Proportion defective

Two-class sampling plan:

4

OC Curve for TwoOC Curve for Two--Class Class PlansPlans

Operation characteristics (OC) or performance for two-class sampling plans:

Probability of lot acceptance calculated for possible proportions defective in lot

Plot of OC curve to visualize

sampling plan performance

dependency on n and c

Proportion defectiveA

ccep

tanc

e pr

obab

ility

Proportion Defective

Pro

babi

lity

of A

ccep

tanc

e

0.0 0.2 0.4 0.6 0.8

0.0

0.2

0.4

0.6

0.8

1.0

P(acceptance)

P(rejection)

Probability of Acceptance by Proportion Defective

n=5, c=0

5


Prob

abilit

y of

Acc

epta

nce

0.0 0.2 0.4 0.6 0.8

0.0

0.2

0.4

0.6

0.8

1.0

P(acceptance)

P(rejection)

Probability of Acceptance by Proportion Defective

n=5, c=0

Consumer’s risk‘Probability of accepting A defective lot’

Producer’s risk‘Probability that anacceptable lot is rejected’

ThreeThree--Class Attributes Class Attributes Sampling PlansSampling Plans

ThreeThree--class sampling plansclass sampling plans consist ofconsist ofnn –– number of sample units to be chosen number of sample units to be chosen independently and randomly from the lotindependently and randomly from the lotmm –– a microbiological limit that separates a microbiological limit that separates good quality from marginally acceptable good quality from marginally acceptable quality quality MM –– a microbiological limit above which a microbiological limit above which sampling results are unacceptable or sampling results are unacceptable or defectivedefectivecc –– maximum allowable number of sample maximum allowable number of sample unitsunitsyielding results between m and M yielding results between m and M (marginally acceptable);(marginally acceptable);the number of sample units allowed to the number of sample units allowed to exceed M is usually set to 0 exceed M is usually set to 0

6

Log cfu/g

Prob

abilit

y D

ensi

ty

0 1 2 3 4 5 6

0.0

0.1

0.2

0.3

0.4

0.5

0.6m M


Proportion marginally acceptable

Three-class sampling plan:

OC Function for ThreeOC Function for Three--Class Class PlansPlans

Operation characteristics (OC) or performance for three-class plans:

Probability of lot acceptance depending on two proportions

marginally acceptable: between m and M

defective: above M

OC function plotted as a three-dimensional graph A

ccep

tanc

e pr

obab

ility


Prop. marginally acceptable

7

Operating Characteristic Curves, 3-Class Plans

P(acc)

ICMSF CasesICMSF Cases

15 cases which reflect:15 cases which reflect:Degree of riskDegree of riskConditions of useConditions of useIntended PopulationIntended Population

8

Risk categorization matrixRisk categorization matrix

Food handling conditionsFood handling conditionsa b ca b c

AAHealthHealthhazardhazard BB

CC

increasedrisk

A) ModerateA) Moderate::

B) SeriousB) Serious::

C) SevereC) Severe: :

S. aureus toxinV. parahaemolyticusB. cereusEPEC

Salmonella (non typhi)ShigellaListeria monocytogenes

EHEC (STEC, VTEC)V. cholerae O1EPEC for infants

Categories of hazardsCategories of hazards

9

Plan Stringency (Case) in Relation to Degree of Plan Stringency (Case) in Relation to Degree of Health Concern and Conditions of Use.Health Concern and Conditions of Use.

Type of Hazard Reduce Degreeof Hazard

Cause No Changein Hazard

May IncreaseHazard

No direct healthhazard

Utility (generalcontamination)

Case 1 Case 2 Case 3

Health HazardLow, indirect(indicator)


Moderate, direct,limited spread


Moderate, direct,potentiallyextensive spread


Severe, direct Case 13 Case 14 Case 15

TwoTwo--Class Plans (c=0): Probabilities of AcceptanceClass Plans (c=0): Probabilities of Acceptance

Composition of Lot% Acceptable % Defective

Number of Sample Units Tested5 10 20 60 100

98

95

90

80

70

50

40

30

2

5

10

20

30

50

60

70

.90

.77

.59

.17

.03

.01

<

.82

.60

.35

.11

.03

<

.67

.36

.12

.01

<

.30

.05

<

.13

.01

<

Typical way of expressing performance of sampling plans

10

Important Properties of Sampling

Gonick & Smith, Harper Resource, 1993.

Microbial Sampling is a Bernoulli Trial


11

Operating Characteristic Curve

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n=0


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n=1Example:

0.1 Proportion Defective

Probability of Acceptance =

0.9 = 0.9

12


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n=2Example:



0.9 x 0.9 = 0.81


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n=3Example:



0.9 x 0.9 x 0.9 = 0.73

13


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n=4

Example:



0.9 x 0.9 x 0.9 x 0.9= 0.66


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n = 5Example:



0.9 x 0.9 x 0.9 x 0.9 x 0.9 = 0.59

14


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n = 10


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n = 20

15


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n = 30


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

‘Idealized’ Situation

P(acc) =0P(rej) = 1

P(acc) =1P(rej) = 0

16


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


Prob

abili

ty o

f acc

epta

nce

n = 5Example: 0.1 Proportion DefectiveProbability of Acceptance = 0.9 x 0.9 x 0.9 x 0.9 x 0.9 = 0.59

c = 10.1 x 0.9 x 0.9 x 0.9 x 0.9 = 0.0650.9 x 0.1 x 0.9 x 0.9 x 0.9 = 0.0650.9 x 0.9 x 0.1 x 0.9 x 0.9 = 0.0650.9 x 0.9 x 0.9 x 0.1 x 0.9 = 0.0650.9 x 0.9 x 0.9 x 0.9 x 0.1 = 0.065i.e. 5 ways = 0.329

So probability of accepting lot is 0.59 + 0.329 = 0.918

Performance of sampling plansPerformance of sampling plansand concentration controlledand concentration controlled

Alternative approach for quantitative data:

Distributional assumption for sampling resultse.g. log-normal with standard deviation known fromprevious experience

Determine proportions acceptable, (marginally acceptable), and defectivefor possible mean log cfu/g

Calculate acceptance probabilities and plot against mean log cfu/g

17


‘When the error is proportional to the measurement the use of logarithms is likely to produce normal curves……’

Gaddum, Nature No.3964, October 20, 1945

18

Log Normality of Total Viable Counts in Log Normality of Total Viable Counts in Batches of FoodsBatches of Foods

7.87.81281128110010066OverallOverall

12.712.7118118151522PowderedPowdered

9.69.652525511Frozen dairyFrozen dairy

2.42.441411111Frozen Frozen vegveg

8.28.2159159131322Frozen meatFrozen meat

7.97.9393393313111Frozen Frozen crust.crust.

6.76.7518518353522Frozen fishFrozen fish

% of % of batchesbatchesNOT LOGNOT LOGNORMALNORMAL

No. of No. of batchesbatchesExaminedExamined

No. of No. of batches batches NOT LOGNOT LOGNORMALNORMAL

No. ofNo. ofSuppliersSuppliers

CommodityCommodity

Fillibens (1975), Techometrics, 17, 111-117

Campylobacter jejuni/coli in raw ground chicken-source Nationwide Raw Chicken Micro.survey,

Mar 1995-May1995 mean=0, sigma =0.8

0.00

0.20

0.40

0.60

0.80

1.00

-6.0 -5.0 -4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0

Log MPN/g

Prob

abili

ty

Distribution

AccumulatedDistributionActual Values

19

APC in raw ground chicken-source Nationwide Raw Chicken Micro.survey, Mar 1995-May1995

mean=4.5, sigma =0.8

0.00

0.20

0.40

0.60

0.80

1.00

-6.0 -5.0 -4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0

Log MPN/g

Prob

abili

ty

Distribution


Assumed Distribution of Enterbacter sakazakii in PIF based on all available published data

mean=-4.0, sigma =.8

0.000.200.400.600.801.00

-6.0 -5.0 -4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0

Log MPN/g

Prob

abili

ty

Distribution


20

Prob

abili

ty D

ensi

ty

Log cfu/g

m

0.0 1.0 2.0 3.0 4.0 5.0 6.0

Linking the Performance of attribute sampling plans to the concentration of bacteria controlled

m

pa

0.0 1.0 2.0 3.0 4.0 5.0 6.0

Prob

abili

ty D

ensi

ty

Log cfu/g


21

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pa

Prob

abili

ty D

ensi

ty

Log cfu/g

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pa pd

Prob

abili

ty D

ensi

ty

Log cfu/g

22

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pa pd

Prob

abili

ty D

ensi

ty

Log cfu/g

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pa pd

Prob

abili

ty D

ensi

ty

Log cfu/g

23

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pdpa

Prob

abili

ty D

ensi

ty

Log cfu/g

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pa pd

Prob

abili

ty D

ensi

ty

Log cfu/g

24

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pd

Prob

abili

ty D

ensi

ty

Log cfu/g

0.0 1.0 2.0 3.0 4.0 5.0 6.0

m

pd

Prob

abili

ty D

ensi

ty

Log cfu/g

25

0.0 1.0 2.0 3.0 4.0 5.0 6.0

mPr

obab

ility

Den

sity

Log cfu/g

Mean Log cfu/g

0.0

0.2

0.4

0.6

0.8

1.0

Prop

ortio

n de

fect

ive,

pd

m

26

0.0

0.2

0.4

0.6

0.8

1.0

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

m

0.0

0.2

0.4

0.6

0.8

1.0

m

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

27

0.0

0.2

0.4

0.6

0.8

1.0

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

m

0.0

0.2

0.4

0.6

0.8

1.0

m

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

28

0.0

0.2

0.4

0.6

0.8

1.0

m

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

0.0

0.2

0.4

0.6

0.8

1.0

m

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

29

0.0

0.2

0.4

0.6

0.8

1.0

m

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

0.0

0.2

0.4

0.6

0.8

1.0

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

m

30

0.0

0.2

0.4

0.6

0.8

1.0

m

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

0.0

0.2

0.4

0.6

0.8

1.0

m

Prop

ortio

n de

fect

ive,

pd

Mean Log cfu/g

31

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)

OC curven = 10,c = 2

1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

32

0.00.20.40.60.81.0

p d

pdP(

acce

pt)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

33

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

34

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

35

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

36

0.00.20.40.60.81.0

p d

pd

P(ac

cept

)


1.0

0.0

0.2

0.4

0.6

0.8

Mean log cfu/g

Prob

abili

ty o

f acc

epta

nce

Performance of Sampling Performance of Sampling PlansPlans

Sampling plan stringency, steepness of OC curve, location of critical lot qualities (95% probability of rejection, 95% probability of acceptance)depend on

Plan specifications n and c

Microbiological limits m and M

Standard deviation s.d.

Difference M-m in relation to s.d.

37

0-3 -2 -1 1 2 3

1000/g100/g10/g1g1cfuin

10g

1cfuin

100g

1cfuIn1 Kg

LOG Conc.

Effect of number of samples(m=1/g, s.d. =0.8)

123510

Mean Concentration Controlled with a 95% Probability

30

0

0-3 -2 -1 1 2 3

1000/g100/g10/g1g1cfuin

10g

1cfuin

100g

1cfuIn1 Kg

LOG Conc.

Effect of limit m/sample size(n=5, s.d. =0.8)

100cfu/g10cfu/g1cfu/g

Absencein

25g

Absencein

15 x 25g


0

38

0-3 -2 -1 1 2 3

1000/g100/g10/g1g1cfuin

10g

1cfuin

100g

1cfuIn1 Kg

LOG Conc.

Effect of Standard Deviation(n=3, m = 1/g)

0.12.4

1.60.8


0

ICMSF ThreeICMSF Three--Class Plans: Mean CFU/G Class Plans: Mean CFU/G Rejected With 95% ProbabilityRejected With 95% Probability

Case 9:Case 9:n=10, c=1n=10, c=1575 cfu/g575 cfu/g






With:m = 1000 cfu/g, M = 10 000 cfu/g,and standard deviation s.d. = 0.8

39

ICMSF ThreeICMSF Three--Class Plans: Mean Class Plans: Mean Implied POs(based on mean count +3sds)Implied POs(based on mean count +3sds)

Case 9:Case 9:n=10, c=1n=10, c=1144435 144435 cfu/gcfu/g






With:m = 1000 cfu/g, M = 10 000 cfu/g,and standard deviation s.d. = 0.8

ICMSF TwoICMSF Two--Class Plans: Mean CFU/G Class Plans: Mean CFU/G Rejected With 95% ProbabilityRejected With 95% Probability

Case 15:Case 15:n=60, c=0n=60, c=01 cfu / 526g1 cfu / 526g






With:m = 0 cfu / 25g,and standard deviation s.d. = 0.8

40

ICMSF TwoICMSF Two--Class Plans: Class Plans: Implied PO (based on mean count +3sds)Implied PO (based on mean count +3sds)

Case 15:Case 15:n=60, c=0n=60, c=00.4cfu/g0.4cfu/g






With:m = 0 cfu / 25g,and standard deviation s.d. = 0.8

Relationship of Microbiological Relationship of Microbiological Criteria to Criteria to FSOFSO’’ss

FSOFSO: A statement of the maximum : A statement of the maximum frequency and/or concentration of a frequency and/or concentration of a microbiological hazard in a food microbiological hazard in a food considered tolerable for consumer considered tolerable for consumer protectionprotectionMicrobiological CriteriaMicrobiological Criteria: The : The acceptability of a product or a food lot, acceptability of a product or a food lot, based on the absence or presence, or based on the absence or presence, or number of microorganisms, and/or of number of microorganisms, and/or of mass, volume, area, or lot .mass, volume, area, or lot .

41

Summary and ConclusionsSummary and Conclusions

No feasible sampling plan can ensure absence No feasible sampling plan can ensure absence of a pathogenof a pathogenMicrobial sampling is a Microbial sampling is a ‘‘Bernoulli trialBernoulli trial’’Stringency depends on key parametersStringency depends on key parametersPossible to link performance to mean Possible to link performance to mean concentration controlledconcentration controlled

ICMSF Sampling Plan Spreadsheetwww.icmsf.org

Acknowledgments

Resources

ICMSF members & consultantsSusanne DahmsRuss FlowersMike van SchothorstBob BuchananBruce TompkinDavid Legan

Documents

Principles of microbiological testing: Statistical basis ... · ‘Probability that an acceptable lot is rejected’ Three-Class Attributes Sampling Plans Three-class sampling plans