Determining the Heterogeneity of Reference Materials

Thomas Bagley, Dr. Cliff Stanley, Dr. John MurimbohDepts. of Earth & Environmental Science and Chemistry

Acadia University

Geoscientists commonly use geochemistry to address geological and environmental problems. The geological samples are sent to the lab, and the results indicate the major or trace element compositions. But how do we know that the results are true to the sample?

Geoscientists send reference material with the batch of geological samples. Reference materials have an accepted concentration, so the quality of the analysis can be assessed from the result. My thesis concerns the heterogeneity of reference materials – how reproducible are the results from reference materials? Current practice assumes they are homogeneous, and attributes any variation to lab error.

2

Acknowledgements

• ACME Analytical Labs, Vancouver• Bureau Veritas, Perth

• Acadia Faculty Research Funding• Canada Summer Jobs Funding•MAC Student Research Funding•SEG Student Research Funding

Participating Laboratories & CRM Manufacturers

Grants

• African Mineral Standards, Johannesburg Geostats Proprietary, Vancouver• CDN Resource Laboratories• Ore Research/Exploration, Melbourne• Rocklabs, Auckland

Background

• Certified Reference Materials (CRMs): – Pulverized rock samples – With accepted element concentrations– With accepted standard deviations– Used to monitor analytical quality

Certified reference materials are pulverized rock or soil samples. They are useful in quality assessment/quality control (QAQC) because they are attributed with an accepted concentration & error.

4

Background

• How are CRMs used ?– Analyzed in a batch of samples

• CRM measured concentrations are compared with accepted concentrations to monitor accuracy– CRM measured concentrations are compared

with each other to monitor precision– These provide an assessment of data quality

for the batch

Certified Reference Materials (CRMs) are used by the mining/resource and environmental industries to monitor analytical quality. Certified reference materials are sent with a batch of samples for analysis. To assess accuracy, the measured concentration can be compared to the accepted concentration. To measure precision, the results can be compared against themselves.

Control Chart

This is an example of a traditional control chart. Each CRM is plotted in the order that it was analyzed, from left to right.

The concentration is represented on the y axis. The control chart has tolerances above and below the mean (1, 2 and 3 SD). Samples are unlikely to have concentrations outside 2SD from the mean, and very unlikely to have concentrations outside 3SD from the mean. In this chart there is an acceptable variation in the results from an analysis. A sample beyond 2SD would probably not be an outlier, but multiple samples beyond 2 SD would be anomalous. There also aren’t any obvious patterns. Patterns can be cause for concern – they can indicate problems like contamination. If one of two crushers is contaminated then the results on a control chart would alternate high – low.

Background

• How are CRMs prepared ?– Pulverized– Homogenized – Sub-sampled– Chemically analyzed

(in multi-lab round robin)– Statistically analyzed

(to remove outliers)

•The rock or soil sample is pulverized in a ball mill.•The sample are blended. homogenized•Sub samples are taken from the population which has been ‘homogenized’.•The sub samples are analyzed at external labs in a ‘round robin’ analysis.•Statistical processes are applied to remove outliers from the round robin: t-test (iso 5725), z-test, iso/iec 43-1, iso 3207..•http://upload.wikimedia.org/wikipedia/commons/c/c2/Ball_mill.gif

Image of a ball mill from google images

http://bestservices.co.in/images/equipment/ball-mill-dry-type.jpg

Problem

• The variance observed in CRM concentrations is not all laboratory error

• CRMs are heterogeneous, and thus exhibit sampling error too

• But how heterogeneous are CRMs ?

-Despite the best efforts of the manufacturer, the homogenization is imperfect – the CRMs are heterogeneous-Also, segregation can occur during shipping – graded beddingTherefore, the mining industry cannot use CRMs to monitor analytical quality unless the magnitudes of their heterogeneity are known.So, how heterogeneous are CRMs?

BackgroundOutliers

Larger Samples ?

This diagram shows the results from a round robin CDN-GS-P4B. What contributes to the variation that we see?

Background

• What contributes to the total variance of a CRM ?– Sampling Error– Laboratory Error– Inter-Laboratory Error

2222InterLabSampTot

Variance (gesture toward a sigma) is the standard deviation squared, it represents the spread of the data. Variation is high when replicate values differ significantly in concentration. Variance describes the magnitude of error. There are three sources of variation in a certification/round-robin: •Sampling error is the heterogeneity of the sample – this is relevant to all elements. Gold is especially prone to high sampling error (nuggets..)•Laboratory error is introduced during the analysis – variation in reagent quality, internal pressure in the mass spec, flame flicker..•Inter lab error is introduced when the labs in the round robin produce different distributions.

BackgroundOutlying sample

Analysis oflarger samples ?Outlying lab

0.45

0.4

0.5

0.55

Here is our round-robin again: •The red labs produced results that differ from the other labs, they’re outlying labs. (different means)•The purple results from lab 14 differ substantially from most of lab 14’s results, and are outlying samples.Statistical procedures are used to distinguish outliers in this sort of scenario, to improve the quality of the certification. •Also notice that the blue lab was much more precise than the other labs, this can happen if they analyze using a larger sample mass.

Background

• Inter-laboratory error and outliers are removed using inferential statistical tests

• Observed variation is now only the sum of lab error and sample heterogeneity

• Standard practice assumes that CRMs are homogenous; suggesting that observed variation is only lab error (NOT TRUE!)

222LabSampTot

By removing the outliers we also remove the inter-lab error. •Since inter-lab error has been removed, the error equation becomes: (gesturetoward the equation on ppt)•This is the point where it has generally been assumed that the reference material is homogeneous and that the total variation is lab error. THIS IS NOT TRUE, BUT IT IS STANDARD PRACTICE.

Objective

• Need to determine the magnitude of sampling error in CRMs

To improve the quality assessment methods, we must be able to determine the magnitude of the sampling error.

Strategy

• Employ Dr. Stanley’s method to measure CRM lab and sampling errors simultaneously

– Measure CRM concentrations in small and large samples

– Solve using 3 equations / 3 unknowns

2.

2.

22.

2,

22.

2,

SmSampSmLgSampLg

LabSmSampSmTot

LabLgSampLgTot

MM

There is a statistical approach that was developed by my supervisor, Dr Cliff Stanley that’s separates sample and lab error.•It is known that the sample mass is proportional to sampling error. Assuming lab error to be constant then the sample error can be determined.•In this way, sampling error and lab error can both be determined.

Methods – Analysis of CRMs

• Aqua Regia/ICP-MS for trace elements• 9 * 2.00 g samples; • 36 * 0.50 g samples; • Data quality assessment samples• Calculate large and small CRM sampling

errors and laboratory error

2.

2.

,

,

SmTotSm

LgTotLg

M

M

22.

2. LabSmSampLgSamp

The lab procedure was developed with Dr John Murimboh. This is the summary of the procedure that we use for each batch of samples. There are large and small samples, measured in replicate to estimate the variance. The procedure blank + control sample are for QAQC purposesUsing this procedure we estimate each error: •Small sampling error•Large sampling error•Lab error

Sources of Error in Co

0

2

4

6

8

10

12

14

16

2P 1M 4E 7F 12A

GS5

0

20B

BA

S-1

P4B

BL-

10 P6

QU

A-1

(1)

QU

A-1

(2)

%R

SDSmall Sampling ErrorLarge Sampling ErrorAnalytical Error

Results%

Rel

ativ

e St

anda

rd D

evia

tion

The method allows us to determine the sampling error for both the large and small samples, and the analytical error. These graphs illustrates the relative proportions of analytical and sampling error for large and small samples. The small samples have much more sampling error! No surprise there! Samples that have no bars plotted did not produce valid estimates of sampling error and analytical error (to be discussed in detail later on)

Fundamental Sampling Constant

• Unique to pulverized material• Relates sample size to sampling error

(inversely proportional)• For a given sample size, sampling error is

known

ΨσMσM SmSampSmLgSampLg 2,

2,

More background:In the last graphs we saw that the sampling error was smaller for large samples. It turns out that the variance is inversely proportional to sample mass. That is, they multiply to define a constant, the fundamental sampling constant (FSC). The FSC can be used to compare homogeneity, and it allows us to CALCULATE THE ERROR AT ANY SAMPLE MASS – very practical information for QAQC.

18

ResultsVariance Ratio Plot of Co in Reference Materials

0.1

1

10

2P 1M 4E 7F 12A

GS5

0

20B

BA

S-1

P4B

BL-

10 P6

QU

A-1

(1)

QU

A-1

(2)

Reference Material Batch

Larg

e Va

rianc

e / S

mal

l Var

ianc

e

02Lab σ

0and

0

2Sm.Samp

2Lg.Samp

σ

σ

The ratio of the total variances determines if the sampling and analytical errors can be decomposed. The lower tolerance is equal to the mass ratio (Ms/ML = 0.25 in this case), the upper tolerance is 1: the large total variance equalling the small total variance. Above the upper tolerance, negative sampling errors result and below the lower tolerance we get negative analytical errors.

19

Discussion• The large & small sampling, and

laboratory variances exhibit error• Sampling and Laboratory error

measurements are dependent on adequate total variance estimates

• How do we achieve adequate estimates of the total variance?

The cause of failed sampling & analytical error estimates is likely to be a low standard error on the total variance (caused by too few replicates or an ineffective sampling strategy). How can we achieve the best estimates?

20

Discussion

• Standard error is dependent on the sampling & analytical variances, and the number of samples

• It is best to estimate total variances with the same standard error

1n2s

1n2sSE

Lg

2Lab

Lg

2Lg.Samps2

Lg.Tot

1n2s

1n2sSE

Sm

2Lab

Sm

2Sm.Samps2

Sm.Tot

Standard error on the total variance is a function of the sampling & analytical error, and the number of samples. The best strategy to maximize the precision of the analytical and sampling error estimates is to make the standard error of the large total variance equal to the standard error on the small total variance (Stanley & Smee 2007).

21

Discussion

Analytical variance is equal to the small sampling variance in this diagram, the large sampling variance is ¼ of the small sampling variance for a sample mass 4 times the size (Stanley 2007).

1) Small samples need to be analyzed more times (as expected) to achieve the same standard error

22

Discussion

2) The optimal sampling strategy uses the same standard error on the variance for both large and small samples; 36 and 15 in this case (we would have achieved more consistent results with fewer invalid estimates had we used 15 large samples instead of 9)

23

Discussion

• To determine the optimal sampling strategy (equal standard errors)

1)(

)1n()1(n 2

S2

L

κλλ

2Lg.Samp

2Lab

Sm

Lg

ss

,MM

λκ

The previous example was for fixed sampling and analytical errorsThe sampling strategy changes depending on the proportions of sampling and analytical error. This system models the number of small to large samples that it takes to achieve an equal standard error on the total variances.

24

Discussion

4MM

Sm

Lg

The optimal proportion of small to large samples to analyze based on the % large sampling variance (of the large total variance). The orange line is the same case that we examined previously (50% sm. Sampling error), not that 15 large samples to approximately 36 small samples. This diagram also illustrates that the sampling strategy for a reference material exhibiting no sampling error is an equal proportion of large and small samples, a case where there is only analytical error (slope = 1, blue line, 0.001%). The sampling strategy for a reference material that exhibits only sampling error (a perfect geochemical analysis!) would use a proportion of 1:16 large to small samples (slope of 16), when using a large sample mass 4 times the small sample mass.

25

Discussion

8MM

Sm

Lg

By using samples with twice the difference (1:8 instead of 1:4 by mass), the number of large samples is greatly decreased – only 8 would need to be analyzed for 36 small samples.

Conclusions• Reference material heterogeneity it is

not zero• Fundamental sampling constants can be

used to estimate sampling error at different sample masses

• CRM manufacturers should provide fundamental sampling constants with their accepted values

• Analytical procedures should be designed to optimize standard error on the variance

27

Future work

• Investigate the controls on sampling strategy, to maximize precision

• Develop QAQC methods that accommodate reference material heterogeneity

28

References• Stanley, C.R. 2007. The Fundamental Relationship

Between Sample Mass and Sampling Variance in Real Geological Samples and Corresponding Statistical Models. Exploration and Mining Geology, 16: 109-123.

• Stanley, C.R., and Smee, B.W. 2007. Strategies for Reducing Sampling Errors in Exploration and Resource Definition Drilling Programmes for Gold Deposits. Geochemistry: Exploration, Environment, Analysis, 7: 1-12.

• Stanley, C.R, O'Driscoll, N., and Ranjan, P. 2010. Determining the magnitude of true analytical error in geochemical analysis. Geochemistry: Exploration, Environment, Analysis, 10: 355–364

29