DQOs and the Development DQOs and the Development of MQOsof MQOs
Carl V. GogolakCarl V. Gogolak
USDOEUSDOE
Environmental Measurements Lab Environmental Measurements Lab
22
Data Quality Objectives
DQOs define the performance criteria that limit the probabilities of making decision errors by: considering the purpose of collecting the data; defining the appropriate type of data needed; and specifying tolerable probabilities of making decision errors.
33
Measurement Quality Objectives
MQOs are statements of performance objectives or requirements for a particular analytical method performance characteristic.
MQOs are acceptance criteria for quality attributes, usually measured by project DQIs.
One of the most important of these is the analytical measurement uncertainty (accuracy).
44
UncertaintyUncertainty
UncertaintyUncertainty is defined in the ISO Guide to is defined in the ISO Guide to the Expression of Uncertainty in the Expression of Uncertainty in Measurement (GUM) as:Measurement (GUM) as:
““a parameter associated with the result of a parameter associated with the result of a measurement that characterizes the a measurement that characterizes the dispersion of values that could reasonably dispersion of values that could reasonably be attributed to the measurand.”be attributed to the measurand.”
55
UncertaintyUncertainty
The uncertainty of a measured value is typically The uncertainty of a measured value is typically expressed as an estimated standard deviation, expressed as an estimated standard deviation, called a called a standard uncertainty.standard uncertainty.
The standard uncertainty of a calculated result is The standard uncertainty of a calculated result is usually obtained by propagating the standard usually obtained by propagating the standard uncertainties of a number of other measured uncertainties of a number of other measured values. In this case it is called a values. In this case it is called a combined combined standard uncertainty.standard uncertainty.
66
DQOs and Uncertainty DQOs and Uncertainty
No measurement program or sampling plan No measurement program or sampling plan can be adequately designed without some can be adequately designed without some estimate of the uncertainty in the data estimate of the uncertainty in the data relative to the action level.relative to the action level.
77
Uncertainty and the Action LevelUncertainty and the Action LevelRelatively large uncertainty can be tolerated:
Either more accuracy or more samples are needed:
88
Connecting the MQOs to the DQOsConnecting the MQOs to the DQOs
Decision errors are made because there is Decision errors are made because there is uncertainty in the datauncertainty in the data
One component of the uncertainty is One component of the uncertainty is analytical measurement uncertaintyanalytical measurement uncertainty
To limit decision errors, the analytical To limit decision errors, the analytical measurement uncertainty should be measurement uncertainty should be limited to a level appropriate to the DQOslimited to a level appropriate to the DQOs
99
MARLAP MARLAP emphasizesemphasizes Method UncertaintyMethod Uncertainty as an MQO. as an MQO.
Method UncertaintyMethod Uncertainty refers to the predicted refers to the predicted uncertainty of a measured value that would be uncertainty of a measured value that would be calculated if the method were applied to a calculated if the method were applied to a hypothetical laboratory sample with a specified hypothetical laboratory sample with a specified analyte concentration.analyte concentration.
The Method Uncertainty is a characteristic of the The Method Uncertainty is a characteristic of the analytical method and the measurement analytical method and the measurement process.process.
1010
Developing MQOs for Method Developing MQOs for Method UncertaintyUncertainty
Data are collected so that decisions can be Data are collected so that decisions can be made...made...
Decisions can be made about individual samplesDecisions can be made about individual samples……as for bioassaysas for bioassays
Decisions can be made about the mean of a Decisions can be made about the mean of a sampled populationsampled population……as for MARSSIM final status surveysas for MARSSIM final status surveys
1111
Decisions made about individual Decisions made about individual samplessamples
HH00: Sample contains no radioactivity: Sample contains no radioactivity
HHaa: Sample contains radioactivity: Sample contains radioactivity
Type I error: Decide there is radioactivity when there isn’tType I error: Decide there is radioactivity when there isn’t
Type II error: Decide there is no radioactivity when there isType II error: Decide there is no radioactivity when there is
This is the familiar framework for LLD calculations.This is the familiar framework for LLD calculations.
1212
Desired Limits on Decision ErrorsDesired Limits on Decision Errors
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5
Concentration
Pro
babi
lity
of D
etec
tion
AL 0
Gray Region:The probability of detectiondecreases with decreasingconcentration
Limit the probability of missing a sampleabove the action level
Limit the probability of flagging a blank sample
1313
Lower Limit of DetectionLower Limit of Detection
Distributionof net countsunder nullhypothesis
Distribution of net counts under alternativehypothesis
LC0 LD
1414
-3 -2 -1 0 1 2 3 4 5 6
Concentration (in units of measurement standard deviation)
0.0
0.1
0.2
0.3
0.4
3
1.645
1
The probability of detection increases with increasing concentration
1515
Power CurvePower Curve
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5
Concentration
Pro
babi
lity
of D
etec
tion
LDLC0
Gray Region
1616
The MQO for Method UncertaintyThe MQO for Method UncertaintyThe width of the gray region is = AL – 0 = AL.
To limit the probability of decision errors, and , to 0.05, we have to limit the standard deviation of the analytical method, M
So that / M > 1.645 + 1.645 = 3.29.
The performance requirement is that the upper bound for the measurement standard deviation is MR = / 3.29 = AL / 3.29.
This is essentially the same as requiring that the minimum detectable concentration (MDC) not exceed the action level.
1717
Required Method UncertaintyRequired Method Uncertainty The true standard deviation of the measurement The true standard deviation of the measurement
process, process, M , is a theoretical quantity and is never , is a theoretical quantity and is never
known exactly.known exactly.
The laboratories estimate of The laboratories estimate of M is denoted uM, the method uncertainty.
The value MR is the required upper bound for the
unknown M. Therefore in practice MR is an upper bound on uM,
the method uncertainty.
Used this way, MR is called the required method uncertainty and is denoted by uMR.
1818
Decisions made about the mean of a Decisions made about the mean of a sampled populationsampled population
HH00: The mean exceeds the action level.: The mean exceeds the action level.
HHaa: The mean is below the action level.: The mean is below the action level.
Type I error: Decide mean exceeds AL when it doesn’t.Type I error: Decide mean exceeds AL when it doesn’t.
Type II error: Decide mean does not exceed AL when it does.Type II error: Decide mean does not exceed AL when it does.
1919
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5
Desired Limits on Decision ErrorsDesired Limits on Decision Errors
Mean Concentration
Pro
babi
lity
of D
ecid
ing
Mea
n <
AL
AL DL0
Limit the probability of missing a meanabove the action level
Limit the probability of missing a meanbelow the DL
Gray Region
The widthof the grayregion, ,represents the smallest concentration difference that it is important to detect.
2020
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5
Power CurvePower Curve
Mean Concentration
Pro
babi
lity
of D
ecid
ing
Mea
n <
AL
AL DL0
Gray Region
2121
The MQO for Method UncertaintyThe MQO for Method UncertaintyThe width of the gray region is = AL – DL.The total variance of the data is
222SM
The sampling standard deviation, S , depends on the variability in the spatial distribution of the analyte concentrations and other factors having to do with how the sampling is performed.
The analytical standard deviation, M , is affected by laboratory sample preparation, subsampling and analysis procedures.
2222
The MQO for Method UncertaintyThe MQO for Method Uncertainty
The sample size needed to conduct the hypothesis test with specified limits on and depends on the relative shift, / .
To keep sample sizes reasonable, should be such that 1< / < 3. Ideally, / 3.
The cost in samples rises rapidly when / < 1, but there is little benefit from increasing / above 3.
2323
The MQO for Method UncertaintyThe MQO for Method UncertaintyGenerally it is easier to control M than S.
If S is large, then the best one can do is make M small relative to S.
If M ~ S/3, then the analytical method variance is contributing less than 10% to the total variance 2. Reducing it further will not reduce very much.
This implies that the upper bound for M should be
MRMR u 10/)103/(10/
2424
Required Method UncertaintyRequired Method UncertaintyIf the lower bound of the gray region is zero, then
the required method uncertainty is
10/ALu MRMR This is essentially the same as requiring that the
relative standard deviation of the measurements near the action level be 10%.
In other words, the minimum quantifiable concentration MQC should be no larger than the action level.
2525
Required Method UncertaintyRequired Method Uncertainty The required method uncertainty is The required method uncertainty is
specified at the Action Levelspecified at the Action Level At concentrations below the action level, At concentrations below the action level,
the bound on the standard deviation the bound on the standard deviation ReqReq = = MRMR is constant. is constant.
At concentrations above the action level, At concentrations above the action level, the bound on the the bound on the relativerelative standard standard deviation deviation ReqReq = = MRMR = = MRMR/AL is constant./AL is constant.
The combined standard uncertainty The combined standard uncertainty uucc of of each result may be compared to these each result may be compared to these bounds.bounds.
2727
Required Method UncertaintyRequired Method Uncertainty
The required method uncertainty, The required method uncertainty, uuMRMR , and , and
the required method relative uncertainty, the required method relative uncertainty, MR MR , can be used for both method , can be used for both method
selection and to develop acceptance selection and to develop acceptance criteria for QC sample results.criteria for QC sample results.