7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
1/33
Danish Fundamental Metrology DFM- 2004-R78 Phone: +45 4593 1144
B 307 Matematiktorvet 6406 HDJ Fax: +45 4593 1137DK-2800 Kgs. Lyngby, DENMARK 2004-11-30 Web: www.dfm.dtu.dk
DRAFT(please send comments to [email protected])
Practical guide for the estimation of uncertainty in testing
Hans Dalsgaard Jensen
Danish Fundamental Metrology
B 307, Matematiktorvet,
DK-2800 Kgs. Lyngby
and
George Teunisse
Nederlands Meetinstitut
Schoemakerstraat 97
NL-2628 VK Delft
Abstract:
Measurement uncertainty plays a fundamental role in the documentation of trace-
ability and the reliability of measurement results. Physical testing often relies on
quantitative test results, and such are to be regarded as any other quantitative
measurement, and thus requires a reliable uncertainty statement for conclusions to
be drawn. However, a tradition and routine in calculating measurement uncertain-
ties for test results is only in the beginning of its development.
This guide seeks to develop and present some tools for the physical testing com-
munity, showing how to approach the problem of assigning uncertainty using the
philosophy and methods of the ISO Guide to the Expression of Uncertainty inMeasurement.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
2/33
2
6406 HDJ 2004 11 30
1 Table of contents
1 Table of contents ................................................................................................................ 2
2 Introduction ........................................................................................................................ 2
3 Connection with other Guides............................................................................................ 2
4 Considerations for evaluation of uncertainty in testing ..................................................... 34.1 Why uncertainty in the first place? ............................................................................ 4
5 Methods for uncertainty estimation.................................................................................... 4
5.1 General - GUM........................................................................................................... 5
5.1.1 Hints on establishing model functions ............................................................... 8
5.2 Generic models........................................................................................................... 9
5.3 Repeatability - reproducibility ................................................................................. 14
5.3.1 Basic assumptions and use of nested designs................................................... 15
5.3.2 Intercomparison data ........................................................................................16
5.4 Other basic tools....................................................................................................... 17
5.4.1 Translation of instrument specifications....................................................... 17
5.4.2 Calculation, use and uncertainty of calibration curves .................................... 18
5.4.3 Drift ..................................................................................................................21
5.4.4 Handling ill-defined measurement conditions .................................................21
6 Examples from electrical testing ...................................................................................... 22
6.1 Measurement of temperature using thermocouples .................................................22
6.2 Heating in Black Test Corner................................................................................... 24
6.3 Glow wire test .......................................................................................................... 25
6.4 EMC - Radiated Immunity Test (IEC 61000-4-3) ................................................... 26
7 Appendix .......................................................................................................................... 28
7.1 Table of distribution functions often used for type-B evaluation: ........................... 32
8 References ........................................................................................................................ 33
2 Introduction
The widespread use of accreditation in testing as a means to ensure the proper level of process
and quality control of test results has lead to an increased focus on the estimation of uncer-
tainty of quantitative test results. The introduction of ISO 17025 as the laboratory standard
has further increased the focus on estimation of uncertainty in quantitative testing.
A main problem is the distinction between measurand and characteristic, where the latter
is a property determined by a test. However, for quantitative testing, where a quantitative
characteristic is determined, there is no practical distinction to that of a measurement quantity.
Another problem is the fact that due to ill defined test set up or test environment in quite a
number of standards, the result of a test can be strongly influenced by these conditions.
The evaluation of all the influences on the measurement quantity could be a very complex and
time-consuming operation. This guide aims to supply solutions to some of these problems.
3 Connection with other Guides
There exist several other guides for the evaluation of uncertainty in testing. Accreditation
bodies (e.g. A2LA, UKAS), regional cooperation organisations (e.g. EA) and laboratory asso-
ciations (Eurolab) have all published documents concerning this issue, see section 8References. Some specific information on classical concepts in testing, such as repeatability
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
3/33
3
6406 HDJ 2004 11 30
and reproducibility, are described in detail in ISO 5725 and brought into the GUM framework
in ISO 21748. Some of the documents fall in the category of a more or less policy documents,
and do not provide specific guidance for the combination and calculation of uncertainty.
Other documents provide examples of calculation of uncertainty, but with very specific ex-
amples without overall guidance on how to handle specific types of uncertainty contributions.
In the present document we will try to bridge this gap and focus on the method of assigning
uncertainty, generalize some of the specific uncertainty contributions, open a toolbox of ge-
neric measurement methods and discuss how to assign uncertainties for the results from use of
these methods.
4 Considerations for evaluation of uncertainty in testing
Paper standards (norms) from the beginning of their existence have served as a means to be
able to produce or reproduce a property by the extensive description of methods for manufac-
turing or testing of an object.
This method of describing in itself causes the first sources of uncertainty. Differences in in-terpretation could already lead to different outcome. Therefore it is necessary that there al-
ways is some platform where people can exchange their information in order to diminish the
possible influence of this source.
It even appears (which is often the case in e.g. EMC test methods) that the testing conditions
are ill defined.
The increasing accuracy and measurement techniques have lead to a growing awareness of the
imperfections in many of the described methods in standards, especially when the measurand
is presented as an absolute measurement value.
However due to the fact that most of the standards (norms) for testing often have been applied
for many years it will lead to a heavy oppositions to modify the described methods and espe-cially in the case that these standards do not contain any margins to the measurand in terms of
uncertainty.
Would an inventory been made on the uncertainty contributions and their values in these
standards then there could be made a comparison between such a method and any new
method to be introduced.
Like mentioned this is not often the case so there will be no strong arguments to change a
method to a more sophisticated method even when this appears to be a far more reproducible
method.
The only elegant way to alter this defensive attitude to a more constructive attitude is toclearly show the benefits of the knowledge of the uncertainties in the measurements by intro-
ducing in the standards a dependency of the values of the testing limits from the magnitude of
these uncertainties.
Of course when starting such an approach it will need an inventory on the measurement un-
certainties in all existing (applicable) standards.
Though this inventory could be assumed a quite time consuming and extensive job it does not
at the same time mean that it is neither impossible nor unprofitable.
The following chapters will show that when applying a systematic method in which a distinc-
tion is made between the contributions coming from the instrumentation on the one hand and
from the test set up on the other hand.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
4/33
4
6406 HDJ 2004 11 30
It will also be shown that in a number of cases it is not always necessary that the testing lab
produce the values of all the contributors to the uncertainty as long as the normalization
committees use the same methods of evaluation when integrating test methods in the stan-
dards they produce.
In the next chapter first of all the use of a uniform method for the estimation of uncertainty
will be described, which is already common practice for accredited calibration laboratories.
Extending this method for testing set ups is possible when introducing some additional tools.
4.1 Why uncertainty in the first place?
The argument has been heard, that uncertainty in testing is not worth the effort, because the
test is performed within the limits of the influence factors as specified in the standard, and
the uncertainty of the measurement results themselves are small in comparison with the
overall test result, hence there is no reason to evaluate or report measurement uncertainty.
This of course does not make sense; compliance with as well the conditions of the test as
the result of the test itself, does require evaluation of uncertainty. But there are more im-portant arguments for performing a thorough investigation of the uncertainty of measure-
ments, although the task can be time-consuming. These arguments are also stressed by the
accreditation organisations:
Reporting a qualified measurement uncertainty statement assists in issues such as riskcontrol and the credibility of test results, and can represent a direct competitive advan-
tage by adding value and meaning to the result.
The knowledge of quantitative effects of single quantities on the test result improves thereliability of the test procedure. Corrective measures may be implemented more effi-
ciently and hence become more cost-effective.
The evaluation of measurement uncertainty enables one to optimise the test proceduresand calibration costs may be reduced if it is found that particular influence quantities do
not contribute substantially to the uncertainty.
When evaluating the result of a test and stating compliance with a specification, it isnecessary with information on the uncertainty associated with results.
5 Methods for uncertainty estimation
Quantitative measurement results without a stated uncertainty are basically meaningless.
However, we must realize that in many, maybe even most, cases in our daily lives this is how
results are reported. Often there is an implicit assumption on the accuracy of the numbergiven: When taking of the age of a person, we implicitly assume an uncertainty of less than
one year, when reading a reference value in a data book we assume the accuracy is of the or-
der of the last significant digit. When we buy a measurement instrument, we assume that most
of the digits shown have meaning; otherwise the manufacturer would probably have chosen a
cheaper display. However, for scientifically based measurements or for measurements in
which we wish to put just the slightest amount of trust, these implicit assumptions simply are
not enough. We need some quantitative indication of the quality of the result; otherwise we
have no means of comparing results among themselves, with other measurement results, with
reference values or limits. And most importantly, we must be very explicit on the assumptions
that are made in the process, because assumptions will still be necessary.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
5/33
5
6406 HDJ 2004 11 30
According to the newest version of the Vocabulary of Basic and General Terms in Metrology
(VIM, 3rd ed), measurement uncertainty is defined as
a parameter that characterizes the dispersion of the quantity values that are
being attributed to a measurand, based on the information used.
Hence, the purpose of estimation of uncertainty is to quantify this dispersion, and the re-mainder of this paper is devoted to the discussion of the methods used for this.
It is the expressed policy of the accreditation bodies as well as of the national metrology insti-
tutes, that measurement uncertainty shall be estimated as a realistic, scientifically based quan-
tity, and the method by which to estimate uncertainty should be universal (applicable to all
kinds of measurements), consistent (independent of the grouping and decomposition into sub-
components) and transferable (the results of one uncertainty calculation can be directly ap-
plied in a second calculation). The method universally agreed upon is described in The ISO
Guide on the Expression of Uncertainty in Measurement, or short, GUM.
5.1 General - GUMThe GUM, is the Master document used on the calcula-
tion and reporting of uncertainty for accredited laborato-
ries. The GUM employs the BIPM Method developed
by BIPM in the late 1980s, which takes the basic start-
ing point of handling all measurement quantities, includ-
ing their factors of influence, as stochastic variables.
A stochastic variable describes a quantity as having
many possible different outcomes, with a given prob-
ability distribution.
The value of an input quantity is the estimate of the ex-pectation value of the distribution function, and the associated standard uncertainty of the
value of the quantity is the square-root of the estimate of the variance of the distribution
function. The methods to ascribe the distribution function of the input quantities may be
performed either through statistical means (called type-A evaluation) or by other means
(called type-B evaluation), e.g. by assumption of a probability distribution. For a primer on
how to evaluate and assign probability distribution functions, see section 7.
The measured quantity or the output quantity is described in terms of a model function in
terms of the input quantities. Finally, the distribution function of the output quantity/-ies is
found using the calculation rules for the propagation of variances.
The GUM method is described in various details with examples in many other documents,
so a thorough guide will not be given here. For completion, the method may be summa-
rised to comprise of the following steps:
1. Specification of the measurand. Consider the nature of the quantity intended as the
result of the measurement. Is it a direct value, is it a deviation from some reference,
e.g. the nominal value, is it a dispersion from an average (or is it the average itself)
or is it some ensemble property, e.g. the average temperature of a selection of
measurement points. The specification of the measurand is naturally critical in any
kind of assignment of uncertainty. Often the focus is only on uncertainty
components, and are collected without regard to their comparability and how the
value of the component combines with the result, thus, the measurand becomesimplicitly defined; one of the most dangerous starting points for uncertainty evalua-
tion.
Probability
Value
Fig. 1 A stochastic variable as a quan-
tity with a probability distribution
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
6/33
6
6406 HDJ 2004 11 30
2. Input quantities. Identify all sources that have a direct relation to the end result, e.g.
instrument readings, applied correction values, reference data, and any source that
may affect the result of the measurement indirectly, i.e. influence quantities such as
environmental conditions, operator variability,
3. Model function. Set up a model function, which relates the final result (the output
quantity), the value of the measurand, to the sources in (2) (the input quantities). The
model function may reflect the physical measurement process or may be a purely
empirical relation. The starting point for any model function should naturally reflect
the way in which the measurement result is derived, even if it consists of a direct
reading of an instrument. The model function must then be expanded to take into ac-
count all the input quantities identified in (2). For tips on taking the indirect quanti-
ties into account, see below.
4. Uncertainty distributions. For each of the input quantities, an expectation value and
the variance must be retrieved, either from a statistical treatment or from an assumed
distribution function of the input quantity. This part of the process sometimes require
just as much thought as setting up the model function itself. Again the documentsmentioned above and listed in the appendix provides many examples of derivations
for different types of influence quantities. This subject is treated more extensively
below.
Consider possible correlations between the input quantities, e.g. when using differ-
ences between readings. Consider whether two or more of the input quantities are re-
lated to each other, even via a third parameter, so that a change in one would be ob-
served at the same time as a change in the other. For example, two quantities may
depend on temperature. If this underlying temperature fluctuates, the values of both
parameters fluctuate, in the same direction or in the opposite; there is a relation (a
correlation) between them. When this interdependence is found or expected to besignificant, the degree of correlation must be taken into account when calculating
measurement uncertainty. To the extent possible and practical, such a third parameter
should be put into the model function as an independent input quantity, and the influ-
ence it has on the two parameters in question be described by the model function.
The important aspect is the conversion of any uncertainty statement that may be
the initial point of discussion, to that of a standard uncertainty in terms of the
square root of the estimate of the variance of the distribution function assigned to the
quantity. For different types of uncertainty statements, e.g. the very common rec-
tangular distribution a, calculate the standard uncertainties by applying the factor
to the parameter appropriate for the distribution function assigned, e.g. for the rec-
tangular distribution 31a , and in the case of statistical treatment based on aver-ages, be sure to consider whether the quantity refers to one observation or an average
value of several observations. See section 7.1 for a list of distribution functions, their
parameters and conversion to standard uncertainties.
5. Propagation of uncertainty. Apply the rule of propagation of uncertainty, i.e. calcu-
late the sensitivity factors for the model function for each input quantity and multiply
by the standard uncertainty of that quantity, and include the terms due to correlation.
It is recommended to use some software calculation tool, to avoid the tedious, error
prone calculations when the model function is more than a simple sum/difference or
product/quotient.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
7/33
7
6406 HDJ 2004 11 30
6. Coverage probability. Due to the nature of the calculation of uncertainty based on
standard uncertainties, the interval covered by the result plus and minus the standard
uncertainty, will only cover 68% of the expected outcomes, if all the assumptions
hold. It is therefore common practice to quote an expanded uncertainty, U, in that the
standard uncertainty is multiplied by a coverage factor to achieve either 95 % or
99 % coverage probability. For an output quantity that follows a Normal distribution,a coverage factor ofk= 2 is used to achieve approximately 95% coverage probabil-
ity. If the output quantity is dominated by, say a rectangular distributed input quan-
tity, the appropriate coverage factor is k= 1,65.
In cases where the confidence in the evaluation of the individual uncertainties is
small, it is necessary to take into account an appropriate number ofdegrees of free-
dom. When the combined uncertainty has a small number of degrees of freedom (less
than 10), the factork= 2 above is not sufficient to achieve a 95 % coverage probabil-
ity. The relevant coverage factor may be found from a table of the two-tailed Stu-
dents t-distribution with the given number of degrees of freedom.
7. Reporting the result. A measurement result should be quoted:x Uand provide in-formation on the coverage probability and the coverage factor used.
Two simple cases are mentioned here, which may be applied in maybe 90 % of the cases
relevant for the intended audience: A sum/difference of contributions and a prod-
uct/quotient of contributions.
Sum and difference
Given a measurement of a quantityy found as a sum or differences of other quantities, say
a direct reading of a value plus a series of corrections, all stated in the same unit as the
quantity being measured (an important point: you cannot add apples and pears!) The model
function then becomes a simple sum:
=+++=i
ixxxxy ...321
The corrections applied to the direct reading could be a known deviation, an estimated drift
or the dispersion determined by experience of the type of instrument used. Again, it is im-
portant to stress, that both a value and an uncertainty must be assigned to each of the input
quantitiesxi (even if the appropriate value is 0, thus indicating that no numerical correction
is performed).
If the input quantities are uncorrelated, the combined standard uncertainty for the output
quantity becomes
=+++=i
ixuxuxuxuyu22
3
2
2
2
1 )(...)()()()(
Product and quotient
A model function in the form of products and quotients may be handled in a similar way. If
the measurand may be modelled in the form
=+=i
ixxxxy ...321
where the factors are e.g. corrections factors to the main input quantity, say x1. Such cor-
rection factors will then typically have a value of 1 (hence not changing the numericalvalue of the result) and their uncertainty could be in units of % or or similar. For such a
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
8/33
8
6406 HDJ 2004 11 30
product (or quotient) model, it is convenient to work in relative uncertainties w(x) = u(x) /
x.
If the input quantities are uncorrelated, the combined standard uncertainty for the output
quantity becomes
=+++= i ixwxwxwxwyw22
32
22
1 )(...)()()()(
and u(y) = w(y) y.
In an very large number of cases, and especially in the cases where more empirically based
model functions are used, the sum/difference or the product/quotient, or a combination of
the two, is all that is needed. This is due to the consistency of the method; it does not de-
pend on any specific way of grouping the uncertainties, and may be evaluated in an arbi-
trary number of steps. For example, given the expression
( )
65
4321
xx
xxxxy
+
++=
This may be reduced in the following way:
89
4719
658
327xxy
xxxx
xxx
xxx=
++=
+=
=
with the calculation of uncertainty done at each step as well, i.e. for the auxiliary quantities
x7,x8, and x9. If the quantities are correlated or the same quantity appears in several places
in the expression, this approach cannot be used without taking the correlations into account
in the steps.
SoftwareIt is strongly recommended to use a software package for the tedious calculation of uncer-
tainty to avoid the necessity for reductions as shown above. Choose a program that allows
you to input quantities, distribution functions, correlations and most importantly general
model functions, and that calculates the sensitivity coefficients, uncertainty contributions
and handles limited degrees of freedom.
5.1.1 Hints on establishing model functions
The establishment of a model function is most often an iterative process. Always start
out by the relation that is used to calculate the measurement result, even if it as simple
as y = x, e.g. the reading of an instrument. In that case always consider a reading assimply a number thrown at you with no direct meaning; not until a calibration (estab-
lishing the relation between the indication of an instrument and the physical quantity)
is performed. Even a fixed standard with a stamped on nominal value, say a 100 re-sistor, should not be considered as a resistance of 100 without taking into account thatthere is a difference between the physical quantity presented and the collection of sym-
bols 1-0-0- stamped on the box. There will always be a term describing the differ-ence between the nominal value and the true physical value; in simple cases it may have
the value zero, but its standard uncertainty will enter the equation for the standard un-
certainty of the quantity being measured.
Given a quantity that is known to affect the total uncertainty of a measurement process,but for which it may be difficult to establish the functional relationship with the output
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
9/33
9
6406 HDJ 2004 11 30
quantity, one hint is to exaggerate its influence, assume perfect knowledge of its value
and determine how a correction would be applied. First give the quantity a name and a
symbol so that it may be handled algebraically.
E.g. in a test, the influence of the experience of the operator is known to affect, maybe
not the result, but at least the expected variation in results. Using the hint, an assump-
tion to make in the process of setting up the model function could be: I known that the
operator always measures a value for quantity A which is 25% lower than the true
value. If that was so, one would correct quantityA and instead useA/(100 % 25 %) =
A/0,75. Hence, the quantity I look forf is the influence quantity, the relative magni-
tude by which the operator measures too low, and it enters as A/(1 f). Now, given an
observation that for a specific test, operator variability is found as a standard deviation
in the results equal to 2%, and the fact that no specific bias (too high values or too low)
may be assigned, and hence no correction can be applied, the influence quantity f may
now be specified asf= 0, u(f) = 0,02.
It is important to list all possible sources of influence and quantify them in a form that
may be expressed in a model function. Whether that needs to be in absolute or relativequantities, with one or another unit (remember also scaling factors in the model func-
tion; e.g. not mixing mV and V and similar!) depends of course of the situation, but a
decision on some specific quantity is necessary. The method above may help in that re-
spect, when formulating a specific exaggeration, and that will then aid in the proper in-
clusion into the model function.
Second, it is not necessary to start by formulating the model function directly in terms
of the quantity to be determined = an expression with all the input quantities. Write
up the physical relations, add the appropriate influence components and then perform
the algebra to achieve the direct functional form. E.g. say we wish to determine the
mass of water in a bottle and we need to take into account the thermal expansion of wa-ter and bottle given that the temperature is different from a reference temperature. The
physical volume in the flask equals the physical volume of the water, hence we have the
physical relations, and the result of the derivation:
( )
( ) ( ) ( )( )TVTTVmVmTVTV
waterglassalnoflaskwaterwaterglassalnoflaskwaterwater
waterwaterwaterwateralnowaterglassalnoflask
+++=
=+=+
11/1
,11
min,min,
min,min,
5.2 Generic models
Many measurement processes are covered by three generic measurement methods: 1) di-rect measurement, 2) substitution, and 3) balance or difference measurement.
For each of these three types, it is possible to set up a generic model function
Direct measurement
Direct measurements come in two flavours: generation of a known signal onto a measure-
ment instrument (DUT = device under test), or determination of an unknown signal on a
calibrated indicator. Calibration of either device is typically performed using the comple-
mentary method.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
10/33
10
6406 HDJ 2004 11 30
When calibrating a measurement instru-
ment, the measurand is the error of indica-
tion on the measurement instrument,
x_err.
The direct input quantities may be speci-
fied as:
The setting of calibrator/source,x_source
The indicated value on the DUT,x_indicate.
The first approximation of the model function can then be formulated as,
the error of indication equals the indication minus the setting of the source,
x_err=x_indicate x_source.
Now we may build up other influence factors to the model. If the indication (reading of themeter) is limited by instrument resolution or the need to interpolate an observation on a
analog scale, it is appropriate to add a quantity for resolution loss, x_res, containing the
part of the reading not accessible, e.g. the digits truncated off the display, or the inability to
determine the subdivision of a scale.
The value of the physical quantity generated by the source is not equal to the setting (dial
setting or an entered value), but there is some deviation, x_transfer, which has been deter-
mined in a prior calibration. Hence we must add this correction (or subtract a deviation,
depending on the how this information is obtained) to the value of the setting of the source.
The manufacturer of the source may furthermore have specified the performance of the in-
strument under different conditions, i.e. the change in the value of the generated physical
quantity with respect to the setting,x_stab, e.g. drift over time, influence of environmental
conditions and other influence factors.
Combining these we get the model function
x_err= (x_indicate +x_res) (x_source + x_transfer + x_stab)
One can then collect information on each of these input quantities:
x_indicate: From repeated observations we may find an average value and a standard de-
viation, which may then serve as our estimate for the mean and the standard uncertainty,
respectively. If the indication is stable, i.e. a constant value, we may take the uncertainty of
indication as zero (the resolution is taken care of by thex_res).
x_res: In the first case an appropriate assumption is that digit is lost due to rounding,
i.e. digit
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
11/33
11
6406 HDJ 2004 11 30
x_transfer: This quantity is the difference between the value of actual physical quantity
transferred to the DUT from the setting or nominal value of the source. This quantity
would be determined by a calibration of the source, and the information needed should be
available from a calibration certificate. To find the standard uncertainty we must examine
the calibration certificate to find the confidence interval or the coverage probability or the
coverage factor. For most accredited calibration certificates a coverage factor ofk= 2 isused, hence we find the standard uncertainty as the quoted uncertainty divided by 2:
u = U/ k.
x_stab: The influence of the environment or usage conditions to the output of a source is
often described by the specifications of the manufacturer. For a more detailed discussion
on the use of manufacturers specifications, see section 5.4.1. The most used strategy, given
no other information, is to assign a rectangular distribution to the manufacturers specifica-
tion, i.e. we obtainx_stab = 0, and u(x_stab) = specification / 3.
If determining the indication requires skill and hence depend on the operator, it may be ap-
propriate to include such a component, i.e. add x_operatorto x_indicate, with a value of
zero and standard uncertainty equal the experimental standard deviation of readings by dif-ferent operators under repeatability conditions.
The model function is that of a simple sum/difference, hence we obtain the resulting stan-
dard uncertainty as the root-sum-square of the standard uncertainties of the components.
The use of a calibrated instrument for di-
rect measurement follows closely the dis-
cussion above, with a slight rearrange-
ment of the components. The basic rela-
tion for a direct measurement, is to con-
sider the indication on the instrument
equal the value of the physical quantitypresented by the object being measured
(DUT). The basic model is thus:
x_dut=x_indicate
But again we have the influence quantities as described above: The limited resolution of
the indication, , the instrument error, as determined above, , and the influence of environ-
ment, usage and time on the instrument, . All of these quantities would simply add to the .
However, not that the definition of error is error = indication true value, hence the
error determined above would need to be subtractedthe indication. We arrive at the model
function
x_dut=x_indicate +x_res x_err + x_stab
The source and handling of the components is equivalent to that discussed above.
Substitution and scaling
Substitution and scaling: A transfer instrument is used to generate an indication equal to
(or with a small difference from) the measurement object and a known standard using an
assumption of a stability and linearity of the indicating instrument.
The quantity to determine is the value of the DUT,x_dut.
The basic quantities that determine the value ofx_dutare: the value of the reference,x_ref,
and the indications on the transfer instrument,x_i, refandx_i, dut.
INSTRx_errx_indicate x dut
x_stabx_res
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
12/33
12
6406 HDJ 2004 11 30
The basic assumption used in the substitu-
tion method is, that the relationship be-
tween the indication on the transfer in-
strument, when measuring the reference
and the DUT is equal to the relationship
between the value of the reference and theDUT.
The basic model function, i.e. the way we
calculate the value is derived from the re-
lation
refix
dutixrefxdutx
refix
dutix
refx
dutx
,_
,___
,_
,_
_
_==
Again, we may now expand the basic model function to take into account relevant influ-
ence factors. The value of the reference, which we assume to know from its calibration,
may still be subject to a change over time or operating conditions. This influence we assignas an additive valuex_stab.
When reading out the indication we may again be limited by the resolution of the instru-
ment or the readability of the scale. We assign the quantityx_res for the missing part of the
reading due to rounding. In this type of measurement setup we have two contributors to the
loss in resolution, one from the indication of the reference and one from the DUT. Note
that we may assume that the two quantities are independent; there is no immediate connec-
tion between the rounding in the two cases.
The final contribution we consider is the capability of the transfer instrument to perform
the transfer,x_transfer. One essential part of the quantity is the linearity of the scale, how-
ever more important in the present case is the variation of the deviation from linearity. Toillustrate this point, let us assume the relation between the reading on the instrument and
the physical quantity is x_true = x_indicate (1 + lin_err), where lin_err is the deviation
from the scale factor 1. For such an instrument we find that the ratio of two readings is un-
affected by the linearity error. Hence, we must quantify the deviations from linearity and
the stability over the measurement time. Thus, we assignx_transferas the change in linear-
ity over the part of the scale used, and will include it in our model function as a factor on
the ratio of the indications.
Adding the components to the basic model function we obtain:
( ) ( )transferxresxrefixresxdutix
stabxrefxdutx _1_,_
_,_
___ 2
1
++
+
+=
We collect the information for each of the input quantities:
x_i,ref and x_i,dut: From repeated observations we use the average and the standard devia-
tion. For a constant reading, u(x_i,..) = 0 and the uncertainty is handled byx_res.
x_res1,2: Again the appropriate contribution isx_res = 0 and u(x_res) = 1 digit / 2 3.
x_ref: The value and the uncertainty should be obtained from a calibration certificate.
x_stab: The expected change in the reference value over time or over usage conditions
may be found either from own analysis based on the results of previous calibrations, ex-
perience or from specifications from the manufacturer of the standard. Depending on thesource of the information different methods of arriving at the value (which would be 0 if
INSTRx_i, ref
x_i, dutx_dut
x_ref
x_transfer
x_stab
x_resx_res
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
13/33
13
6406 HDJ 2004 11 30
no correction is applied) and the standard uncertainty may be employed. E.g. a manufac-
turers specification may be assigned a rectangular distribution and treated accordingly, or
if a historic drift curve is used, the calculated standard uncertainty from the linear regres-
sion analysis is appropriate.
x_transfer: The transfer component will at best be characterised by the short term repro-
ducibility of the indicating instrument. Thus it may be characterised by performing re-
peated measurements. If more information is available, such as a specific relation for the
short term stability of the indicating instrument, this should be taken into account. In case
even slightly different values are read, the non-linearity of the scale should be considered,
either from a calibration or from information such as the manufacturers specification.
The model function is unfortunately not a simple sum/difference form hence other calcula-
tion tools must be taken into use, or calculated in steps by evaluating the various sums
separately and combine the factors using relative uncertainties.
Balance or difference method
The unknown quantity is balanced againsta known reference, with a small differ-
ence nulled out or with an absolute meas-
urement of a small difference.
An unknown quantity is balanced against
a known reference and a small difference
is measured or nulled out using an ad-
justment. The measurement principle is
similar to that of substitution, except the
DUT and reference will typically be physically connected, and the indicating instrument
will be used on its lowest range.
The quantity to determine is the value of the DUT,x_dut.
The basic quantities that determine the value ofx_dutare: the value of the reference,x_ref,
the indication on the null instrument,x_null and the value of the adjustmentx_adjust.
The basic model function when balanced, i.e. the way we calculate the value, is found as
adjustxrefxdutx ___ +=
Again, we may now expand the basic model function to take into account relevant influ-
ence factors. The value of the reference, which we assume we know from its calibration,
may still be subject to change over time or operating conditions. This influence we assign
as an additive valuex_stab.
When the null indication is achieved, we may again be limited by the resolution of the in-
strument or the readability of the scale. We assign the quantityx_res for missing part of the
reading due to rounding.
The measurement of the null is also subject to uncertainty. If the instrument is used as a
true null detector by applying an adjustment signal x_adjustto achieve a null reading, our
x_null should reflect the ability to achieve a true null, e.g. if the null instrument is a volt-
meter, a comparison with a short circuit is appropriate. If polarity reversal is possible, the
null value may be derived from the forward and reverse polarity readings.
Finally, thex_adjustmay be either the direct difference reading on the null instrument if noadjustment is performed, or the value of the additional signal we apply to achieve a null.
INSTRx_nullx_dut
x_ref
x_adjust
x_stab
x_res
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
14/33
14
6406 HDJ 2004 11 30
The demands on the relative uncertainty ofx_adjustare much less stringent than that of
x_ref; if, say,x_adjustis 1 % ofx_refand its relative uncertainty is 1 %, the overall rela-
tive uncertainty contribution becomes 10-4.
Adding the components to the basic model function we obtain:
( ) )__(____ resxnullxadjustxstabxrefxdutx ++++= We collect the information for each of the input quantities:
x_ref: The value and the uncertainty should be obtained from a calibration certificate.
x_stab: The expected change in the reference value over time or over usage conditions may
be found either from ones own analysis based on the results of previous calibrations, ex-
perience or from specifications from the manufacturer of the standard. Depending on the
source of the information different methods of arriving at the value (which would be 0 if
no correction is applied) and the standard uncertainty may be employed. E.g. a manufac-
turers specification may be assigned a rectangular distribution and treated accordingly, or
if a historic drift curve is used, the calculated standard uncertainty from the linear regres-
sion analysis is appropriate.
x_adjust: The additional signal applied may come from a source, and hence handled simi-
lar tox_ref. Ifx_adjustis a direct measurement of a small difference, we may thus need the
manufacturers specification for the range used and the value read.
x_null: From repeated observations we use the average and the standard deviation. For a
constant reading, u(x_i,..) = 0 and the uncertainty is handled byx_res.
x_res: Again the appropriate contribution isx_res = 0 and u(x_res) = 1 digit / 2 3.
The model function is again a simple sum/difference and the root-sum-square of the uncer-
tainty contributions gives the standard uncertainty of the result.
5.3 Repeatability - reproducibility
Often in testing, series of trails are used to validate the measurement set-up, the method
employed as well as in the measurement process itself. Employing statistical methods on
the final test results is a powerful method of obtaining an overall estimate of the total un-
certainty. However, the requirement for this to be true is that the influence quantities are
varied suitably over their full range during such a series of trails.
The ISO 5725 series of standards cover much of the detail on the design of experiments
and the analysis. The basic idea is to quantify from each level of repetition, the variance of
one or more influence quantities in terms of a standard deviation. From the results of twoor more levels, it is then possible to quantify the consistency of the assumption on the
number of independent influence quantities.
This process of determining repeatability and reproducibility may be summarized with the
termprecision. Secondly when it becomes important to consider comparability outside the
laboratory, the concept of trueness, or the determination of bias, must be taken into ac-
count.
At this point it is important to stress:
Precision, i.e. repeatability and reproducibility, is a measure for the ability of the lab togenerate consistentresults, and can be estimated from internal sources.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
15/33
15
6406 HDJ 2004 11 30
Trueness or bias, is a measure for the ability of the lab to generate correctresults orcomparable results, and therefore, must be estimated from external sources: calibra-
tion, reference materials, intercomparisons, etc.
5.3.1 Basic assumptions and use of nested designs
It is important to stress, that the methods described in ISO 5725 and ISO 21748 are
based on a pragmatic or empirical model function and not model functions based on the
physics of the measurement procedure. There is no contradiction with the philosophy of
the GUM and the methods are fully applicable for this type of treatment.
The empirical model can be stated as (ISO 21748)
( ) excBmy ii ++++= where y is the observed value, m is the ideal or true (unknown) value, is the in-trinsic method bias, B is the laboratory bias and e is an error term. The sum of cixi
represents other effects than those giving rise to andB.Hence, m in this treatment could be the reading of the (complex) measurement equip-
ment in the form of a black box, which we attribute to the quantity we wish to assign
a value to. The parameter is the bias or error the particular method of measurementhas been assigned or that one must assume exists when comparing results based on
(very) different measurement principles or similar. The parameterB is also a bias or er-
ror, but related to the laboratory implementation of the method, i.e. if several laborato-
ries implement the same measurement using the same method, the parameterB would
signify the dispersion expected; i.e. the method reproducibility as implemented in the
laboratory. In some cases it may not be possible or relevant to distinguish between and
B. The terms cixi are physically modelled influence factors. The formulation used al-lows one to, e.g. take environmental influences into account, say it is assumed the tem-
perature in the laboratory affects the result with 1 %/C (i.e. the sensitivity factor), one
would then be able to take into account the uncertainty in temperature combined with
sensitivity factor. Finally, e signifies the pure random errors, or the repeatability term
(an average value or other statistically derived value would be part ofm).
The assumptions in the treatment are, that the expectation ofB and e is zero (hence no
correction is applied for the laboratory bias!), and have variances L and r respec-tively. The quantity e basically describes the repeatability of the measurement, whileB
signifies the reproducibility under ideal conditions.
The reproducibility observed would be the combination 22rLR += .
The whole idea of the exercise is to find from the observed reproducibility and the ob-
served repeatability to find whether the ideal reproducibility is significant or not, and
hence if there are contributors to the total uncertainty that must be described separately.
Given two sources of variability, the purpose of the nested design experiments and
ANOVA is to determine the significance of one over the other. Letx_A be a component
that signifies the within-day variability of some measurement results, and letx_B be a
component that signifies the between-day variability. How, and to what extent can we
determine the significance ofx_B and does it make sense to try and split the observed
dispersion into these two components?
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
16/33
16
6406 HDJ 2004 11 30
Lets assume a balanced nested design. Each day we perform Nmeasurements. The ex-
perimental variance 2As of one of the within-day results would be an estimate of the
variance of the individual measurements. The average of the within-day results would
then have an estimated variance of NsA2 , where N is the number of measurement
within a day. Hence, when we calculate the observed variance of the between-day re-
sults, 2as , each result being the average of the results of that day, we should expect a
value compatible with NsA2 . If the observed 2as is significantly larger than this, we
must assume there is a contributorx_B with variance2
Bs to the variability and which we
have not taken into account, so that 222BAa sNss += .
To test for significance, an f-test may be performed, by comparing the critical fvalue
with the ratio of two estimates of 2As . One we obtain by pooling the2
As values, which
for the balanced design is the average of these, Mss Ab = 22 , whereMis the numberof days we conduct our experiment; this estimate has (N 1)Mdegrees of freedom the
other as 2asN , which hasM 1 degrees of freedom. We calculate the ratio
22
ba ss and
compare to the critical value F0,95(M1, (N1)M)1. E.g. if we do 5 measurements each
day for 10 days, the critical value is F0,95(9, 40) = 2,12.
If the ratio is smaller than the critical value, we cannot conclude that 2as is significantly
larger than NsA2 , hence there is nox_B component.
If the ratio is larger than the critical value, we can conclude that 2as is significantly lar-
ger than NsA2 , hence there is ax_B component. We may even estimate the magnitude
of 2Bs as Nsss AaB
222 = (but be careful if these numbers are close; remember the uncer-tainty of the variance estimates!)
Thus, appropriate nested designs allow us to extract information of separate uncertainty
components based on an empirical model and statistical treatment of the measurement
data. However, the method has its limitations. A simple systematic effect such as drift in
the measurement value, e.g. caused by a change in a reference standard or the measure-
ment instrument, cannot be distinguished from purely random sources. Hence, it is im-
portant, before relying on empirical models, to examine the measurement problem from
a physical-technical standpoint, and incorporate the influences identified into the model
function. The general expression above allows partly taking such physically modelled
effects into account through the terms cixi.
5.3.2 Intercomparison data
From the discussion above, it is found that the characterisation of repeatability and in-
house reproducibility, together with the technical-physical characterisation of the meas-
urement problem may give an overall estimate of the total uncertainty. However, it is
not until a comparison is made with an outside source that a systematic laboratory bias
may be uncovered.
In order for the laboratory bias, i.e. the systematic difference between the laboratory in
question and the outside world, may be stated to be in control, the laboratory must
1 In Excel, the F-test critical value can be generated as FINV(1-p, 1, 2), wherep is the confidence level (e.g.95%), and 1 and 2 are the degrees of freedom of the variance estimates being compared.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
17/33
17
6406 HDJ 2004 11 30
establish that its reproducibility with variation of several of the influence factors cov-
ered by the statistical treatment is within the standard deviation of the between-
laboratory results of a comparison. Unfortunately, intercomparisons in the field of LVD
testing are so infrequent and with a small number of participants, that a good statistical
basis for this evaluation is very difficult to obtain.
Another method of establishing the magnitude of any bias, is the use of certified refer-
ence materials. Again, in the field of LVD testing, such items or materials are usually
not available.
5.4 Other basic tools
To generate a realistic uncertainty budget many sources must be taken into consideration.
However, many of these a common to measurement situations across the fields of metrol-
ogy. Below are given some examples of such effects that may be treated in the same way
independent of the field of measurement.
5.4.1 Translation of instrument specifications
Measurement instruments are typically accompanied by a set of specifications, indicat-
ing the performance to be expected by the instrument under specific conditions of use,
as determined by the manufacturer. These are often given as direct or relative limits of
the error of indication or generation of the measurement quantity in question. However,
there are almost as many ways of indicating the performance of instruments, as there are
manufacturers.
First, it is necessary to determine to which level of confidence the specifications are
given. In few cases statements such as 99% confidence or 95% confidence are
given. If it assumed, that the specification may be interpreted as a quantity with a Nor-mal distribution, a 99 % confidence interval corresponds to a coverage factor ofk= 2,58
and a 95 % interval corresponds to k= 2,58. To use the specification one would divide
the specification limit with the appropriate k-factor to get the standard uncertainty.
However, often no such indication is given and it is most appropriate to assume a rec-
tangular distribution for the limit, and hence divide by a factor3.
Second, it is necessary to determine which limitations apply to the specifications given.
Do they cover all secondary settings (e.g. integration time, filter settings), or are there
additional terms which must be added, e.g. for measurement of resistance, specifications
are often quoted for four-wire measurements, and an additional term is quoted for two-
wire measurement.
Third, in many cases, specifications are often quoted in not-so-rigorous language, and
caution must be applied in translating these into more strict mathematical form, includ-
ing separating absolute value terms and relative value terms. In some cases, absolute
value terms are even given as a relative term of a range or maximum value, which may
not be the possible maximum reading for a range, and even visa-versa. E.g. for some in-
struments a range may be called 1, but has a maximum reading of 2 and the con-
stant contribution for a measurement on that range is to be calculated from 2 times the
range uncertainty.
An example is a common statement such as a table heading (% reading + % range)
and table entries for specific measurement ranges, e.g. 100 mV 0,005+0,004. Thisshorthand would then be interpreted for a reading of V, as the interval (510-5 V +
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
18/33
18
6406 HDJ 2004 11 30
4 V). With the assumption of a rectangular distribution, we achieve the standard uncer-
tainty u(V) = 2,910-5V+ 2,3 V.
5.4.2 Calculation, use and uncertainty of calibration curves
Under many measurement conditions, some transducer is used to convert from onephysico-chemical property into a property, say voltage, resistance or similar, that is
more easily handled and measured. To characterise the transducer and allow the transfer
back and forth the two properties, a calibration curve or transfer function is needed.
The calibration curve or transfer function is in some cases given by a parameterised, but
standardised expression, e.g. the R(T) for resistance thermometers and thermistors, or
the V(T) for thermocouples.
A calibration curve will typically be
mapped by performing a calibration: ap-
plying a series of known signals and re-
cording the responses. From the valuesand standard uncertainties of the inputs
and outputs, and appropriate parameterised
calibration curve may be determined, by
determining the value of the parameters
and their covariance (because the parame-
ters are all determined from the same set
of inputs, they will be correlated, thus we
must derive and use the full covariance
matrix of the parameters). With the pa-
rameters and covariances in hand, we may
for a given observed output derive the ap-propriate input and its standard uncertainty.
Hence the starting point is a set of applied stimuli {xi , u(xi)}, the measured responses
{yi , u(yi)} and an empiric or physical bondf(x,y) = 0.
The simplest transfer function is a linear relation,f(x,y;a,b) = a+ bx y, with parameters
a and b.
Linear regression,y =a +b x
If the uncertainties in x are so small that we may ignore their contribution, we obtain
the standard formulas (following Ref. 7):
( )2xxx
yxxy
xyxyxx
SSS
SSSSb
SSSSa
=
=
=
where
==
===
==
===
N
i i
ii
xy
N
i i
i
xx
N
i i
iy
N
i i
ix
N
i i
yu
yxS
yu
xS
yu
yS
yu
xS
yuS
1 21 2
2
12
12
12
)()(
)()()(
1
x
y
Fig. 1 Calibration curve
The red points signify the calibration points,
known x with observed y, giving the green
calibration curve Given an observation (blue)
we derive the stimuli oran e .
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
19/33
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
20/33
20
6406 HDJ 2004 11 30
they-values in the vectorY, then the solution a to the least square estimation (the esti-
mate ofa) becomes
CaaXXCYXCa === ),(.)(where, 111 TTT u
The fitted values are aXY = and a predicted value for an input becomes ax =y withT
yu xCx=2)(
Linear regression in matrix notation
How does this look in an implementation? For the linear regression above we have a =
(a, b) and x = (1,x). With a data set of {xi,yi, u(yi)} without correlations between they-
values we have
=
=
=2
2
111
)(00
00
00)(
,,
1
1
1
NNN yu
yu
y
y
x
x
OMM YX
We then get (using the notation above)
( ) ( )
=
==
=
xy
yT
x
xxxT
xxx
xT
S
S
SS
SSC
SS
SSYXXXXX
1111 ,1
and finally
( )
=
=
==
SS
SSu
SSSS
SSSS
S
S
SS
SS
x
xxxT
yxxy
xyxyxx
xy
y
x
xxxT 1),(,11
1 aaYXCa
which is the same result as quoted above. If the y-values are correlated, the matrix
above is replaced by the general covariance matrix fory.
The inclusion of uncertainties in the x-values can be included by replacing the covari-
ance matrix by
=+=2
21
21
2
1
2
)(
),(
),()(
,
Nxu
xxu
xxuxu
b
LM
MO
L
xxy
For a polynomial expression we would have something like x = (1,x, x2, x3, ) and the
procedure to derive the formulas and expressions similar to that shown above.
Using the general linear least squares
We may obtain the combined uncertainty when using the calibration curve to look up
the value from a measured output y including its uncertainty u(y). One requirement is
that they is independent from they-values used to determine the calibration curve and
hence not correlated with the calibration curve parameters.
Let us for simplicity assume that x = x(t), e.g. x(t) = (1, t, t2); e.g. ifa = (R0,A,B) this
could be the calibration curve for a resistance thermometer, with tas temperature andy
as resistance:y = a x =R0 +At+Bt2.
The general formula for the least squares solution above gives
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
21/33
21
6406 HDJ 2004 11 30
( )( ))(,
)()()()(
2
22
tyt
yutttu
T
T
xaxa
Cxx=
+=
For the temperature curve above, the denominator term becomes
( ) tBAtBARt TT 2)2,1,0(),,( 0 +== xa
The calculation of the covariance matrix C is in this case a bit more complicated, but
easy to handle numerically. Such least squares algorithms can easily be implemented in
a spreadsheet program such as Microsoft Excel, which supports matrix algebra.
The inclusion of uncertainties in x may be included in the determination of the model
parameters, as was shown for the linear regression above. It requires the scaling ofu(xi)
ontoy using the first approximation ofa. In the case where x = x(t) and we assume no
covariances, one would replace u(yi)2 by ( )[ ] 22 )()()( iii tuttyu + xa .
Non-linear general least squares
If it is necessary to be able to take both uncertainties in x andy into account, e.g. if theuncertainties in the independent variable are significant, numerical methods must be
used. An algorithm for such full calculation is given in Ref. 8. The method described
there allows also for implicitly given model functions, i.e. general f(x, y) = 0, and de-
scribes also how to test for the consistency of the fit obtained.
5.4.3 Drift
Most material standards are affected by internal and external sources, which changes
their characteristics over time. Hence the value represented by the standard changes
over time. In many cases the change may be sufficiently small and the change is insig-
nificant. Regular calibration may be used to quantify this change and one may be able topredicted the drift between calibrations. Thus, the change may be compensated and/or
the knowledge of the time-dependence can be used to fix calibration intervals; if the
standard proves stable enough, longer intervals are possible and visa-versa.
The most common method is prediction from linear regression. Hence the methods and
formula described in the section on calibration curves may be applied to drift as well. A
drift component would then be handled as an additive contribution to the value of the
standard, with a standard uncertainty as calculated from the linear regression.
5.4.4 Handling ill-defined measurement conditions
If the variability of the test results is caused by a variation in the measurement condi-
tions, this variability should be quantified by including the influence quantities into the
modelling of the measurement situation. This may be done by direct physical modelling
or by including a empirical model quantity of the form cixi.
However, it could be argued, that the variation due to ill-defined measurement condi-
tions is not the responsibility of the testing laboratory, but should be part of the method
validation studies. That is, such studies should be performed in the process of develop-
ing the test method, and should be information provided in the standard. It is the respon-
sibility of the laboratory to evaluate measurement uncertainties of the measurements
performed. It would then be appropriate to compare the measurement uncertainty
against such method uncertainty in the validation of a specific test setup, and in theprocess of intercomparisons.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
22/33
22
6406 HDJ 2004 11 30
Alternatively, in order to facilitate comparison of test results, the actual measurement
conditions should be specified when reporting the test result. If differences are found,
e.g. in a comparison, and it is observed that measurement conditions vary, it would be
appropriate for an investigation as to the magnitude of the influence factor. If, however,
the dispersion is much smaller than the test limits, this will only be an academic exer-
cise and not influence the conclusions or decisions drawn from test results.
6 Examples from electrical testing
6.1 Measurement of temperature using thermocouples
In the next two examples of basic LVD testing, the measurement of temperature using
thermocouples is the basic physical measurement used. We therefore present here an ex-
ample of an uncertainty budget for such basic measurement.
Note, however: This example is constructed to present some of the considerations, inter-
pretations and arguments in setting up the model function and assigning uncertainties, andshould not be taken as a direct example on what uncertainty to assign a thermocouple tem-
perature measurement!
Thermocouples are made from two wires of dissimilar metals brought together in a joint.
Due to the thermoelectric effect and different coefficient for different metals, different
voltage drops will occur along the two wires from the common joint, and a voltage differ-
ence can be measured at the open end of the thermocouple wires. Given a thermocouple of
a specific type, there are standard expressions for the voltage-temperature relationship, e.g.
for type K thermocouples a 9th degree polynomial expression plus an exponential term is
the reference expression. An inverse expression, giving the temperature as a function of the
voltage difference, is also available stated in the same form as a 9 th degree polynomial. The
conversion error is typically 0,05 C.
To avoid the additional thermal voltages when connecting the thermocouple to the voltme-
ter, an additional thermocouple is inserted held at a standard temperature, 0 C, allowing
for the same junction type at the voltmeter. Alternatively, a correction based on the tem-
perature of the junctions at the voltmeter is applied. For the purpose of the measurements
for heating testing, we do not seek the best uncertainties, and will assume an internal refer-
ence junction compensation is used.
The basic for the measured temperature thus becomes
T= t(Vt) with Vt= Vm V0,
where t(Vt) is the reference conversion curve, Vm is the thermocouple voltage differenceand V0 is the compensation voltage.
Assume we wish to measure 950 C using a type K thermocouple. The reference voltage is
Vref = 39,3139 mV and the sensitivity c950 = dT/dV= 25,3077 C/mV. We perform a first
order expansion of the reference curve and obtain
T= 950 C + c950(Vm V0 Vref).
We may then begin to expand the model function based on other contributions. First, the
deviation of the thermocouple in question to that of an ideal thermocouple. We will state
that as the additional voltage generated by the thermocouple at the temperature T, Vtc.
Second, for the voltmeter we have its measurement errorVm from calibration, the resolu-tion VI, the drift and operating conditions from the specification VD, and the error in the
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
23/33
23
6406 HDJ 2004 11 30
compensation voltage (internally based on the reference curve and the measured tempera-
ture on the connection block) V0. Finally, we have the conversion error itselfTc.
To expand the model function we may argue: If the meter reading is perfect Vm and the
temperature is T, the thermocouple generates too high a voltage Vtc. Hence we must sub-tract this from the reading. Also the reading presented is corrected for the cold junction
voltage, so V0 is in reality included in Vm. Next, the reading is not perfect, but is in error by
Vm as determined by calibration, hence we must also subtract this from the reading. Alsothe drift and operating conditions give too high a reading, so we must subtract VD. Themeter compensates the cold junction by calculating the cold junction voltage and subtract-
ing it to the measured voltage, however, this voltage is in error by V0, hence we must sub-tract that from V0 i.e. add to Vm. Finally, the imperfectness of the temperature-voltage con-
version gives an errorTc in temperature, which we must subtract the reference tempera-tur. With these arguments, we arrive at the expanded model function
T= 950 C + Tc + c950(Vm + V0 Vtc Vm VDVref).
Now we examine each of the components and derive the value to assign and the standarduncertainty.
Tc: The reference curve specification gives the conversion error as 0,05 C. We will as-sume there is equal probability in this range, and assign a rectangular distribution function.
Hence we obtain the value Tc = 0 C, with standard uncertainty u(Tc) = 0,05 C/3 =0,029 C.
Vm: During measurement we acquire 10 readings. We calculate an average 39,151 mV and
a standard deviation 0,011 mV. Hence, Vm = 39,151 mV, u(Vm) = 0,011 mV/10 =0,0034 mV.
V0: The instrument specifies that the cold junction compensation is done with an accuracyof 0,5 C. Converted to voltage, this corresponds to 20 V. We choose to understand this,as the compensation voltage is accurate in the interval 20 V to +20 V. Hence, we as-
sign a symmetric rectangular distribution and get V0 = 0 V and u(V0:) = 20 V / 3 =11,5 V.
Vtc: The thermocouples are made in the laboratory using good quality thermocouple wire.A batch of thermocouples are made and calibrated in an oven and measure their voltage
output in a specialised setup. It is found at 950 C that the average voltage generated is
39,250 mV with a standard deviation of 21 V. The uncertainty of the generated tempera-
ture is estimated as 0,2 C measured by a standard thermometer, and the uncertainty of the
voltage measurement is 5 V. The average voltage error is (39,250 mV 39,314 mV) =
0,064 mV. The 0,2 C temperature uncertainty corresponds to 8 V uncertainty in the gen-
erated voltage, hence the combined uncertainty of the voltage error is (212 +52 + 82) V =23 V. Hence, for a thermocouple from this batch we use Vtc = 0,064 mV, u(Vtc) =0,023 mV.
Vm: The meter has been calibrated at an accredited laboratory. At 40 mV on the 100 mVrange, the error was found to be 6 V with an uncertainty of 5 V. The uncertainty is
stated as a 95% confidence value, i.e. overage factor k= 2. Hence we use Vm = 6 V,u(Vm) = 5 V / 2 = 2,5 V.
VD: The specification of the meter is stated in the form 0,02 % reading + 0,03 % range.
The meter is used on the 100 mV range, thus 0,03 % range is 30 V. The reading is 40 mVand 0,02 % reading is 8 V. Added we get 38 V. This is the manufacturers estimate of
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
24/33
24
6406 HDJ 2004 11 30
the absolute limits of variation of the error for 1 year and under the prescribed measure-
ment conditions. Hence, we interpret this as a symmetric rectangular distribution, (0 38)
V, i.e. VD = 0 V, u(VD) = 38 V / 3 = 22 V. The resolution is much lower than thisterm, hence we regard this as included in the specification.
We will evaluate the uncertainty in steps:
The voltage terms form a sum and difference expression; hence we may combine the un-
certainties directly by root-sum-square. We thus obtain V= 0,105 mV, and u(V) = 34 V,
dominated by the meter specification and the thermocouple error.
We may note, that the uncertainty found for the reading is a factor of 10 lower, hence does
not contribute significantly. We may therefore without loss of accuracy reduce the number
of readings to, say, four; we have an estimate of the standard deviation, 11 V, and the av-
erage of four readings would give a standard uncertainty of 5,5 V. With this value the
combined voltage uncertainty remains 34 V within the rounding.
The combination of the voltage and the scale factor is a product, but we have not assigned
an uncertainty to the scale factor because the conversion uncertainty is covered by Tc.Hence the uncertainty in voltage scales directly to an uncertainty in temperature, and we
obtain Tc = c950V = 25,3077 C/mV (0,105) mV = 2,65 C and u(Tc) = c950u(V) =
25,3077 C/mV 0,034 mV = 0,86 C.
The final result is thus T= 950 C + 0 C 2,65 C = 947,35 C. The standard uncertainty
is the combination of the converted voltage uncertainty and the reference curve uncer-
tainty. The latter is more than a factor of 10 lower, thus we may safely ignore it and obtain
u(T) = 0,86 C.
6.2 Heating in Black Test Corner
The tests specified in EN 60335-1 relates to heating of household appliances. The test re-
quires the measurement of temperature on and close to the surface of the appliance under
normal operating conditions.
A black test corner is implemented as an array of thermocouples connected to Cu disc
mounted on wood plates. By using a number of plates, a corner with dimensions of the or-
der 1 m x 1 m x 1 m can be constructed. This arrangement is typically used to measure side
and bottom temperatures of large appliances.
If we look at the measurement situation, we again must measure the voltage of the thermo-
couples. The voltage of the thermocouples is related to their temperature, but we are inter-
ested in the temperature at the surface of the appliance. Thus we must take into account the
difference in temperature from the thermocouple and the position of the surface.
Alternatively, we may apply a temperature to the thermocouple array and thus indirectly
take into account the transfer, however, we will then be subjecting the thermocouple wire
to a different temperature distribution under calibration than under use.
Hence we must carefully specify our measurand: Do we report the temperature of the
thermocouples in the arrangement specified by the standard, or do we try to evaluate the
temperature of the surface of the appliance?
In the first case, our measurement model is clear: We have the voltage measurement, trans-
lated into temperature using the standard transfer curve, combined into the variable TV, the
calibration measurement of the thermocouple voltages when subjected to temperaturesgenerated in an airbath, combined into the variable Tc, and, if we treat all thermocouple
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
25/33
25
6406 HDJ 2004 11 30
alike, and do not perform individual corrections, we have the deviation of the thermocou-
ple voltage from the reference voltage, again scaled to temperature, Tc.
Our model function:
T= TV+ Tc + Tt , with standard uncertainty222 )()()()(
tcV TsTuTuTu ++=
Given, for example, the voltage measurement uncertainty and the reference curve conver-
sion has specification 0,4 C, we use u(TV) = 0,4 C / 3 = 0,23 C, the total calibrationuncertainty of the temperature bath u(Tc) = 0,04 C, and the observed standard deviationof the individual thermocouple temperatures, u(Tt) = s(Tt) = 0,69 C. Combined weachieve u(T) = 0,73 C, and we see the dominating contribution comes from the spread of
the thermocouples. Because this component is near a Normal distribution, we may use
coverage factor k 0 2 and achieve U(T) = 1,5 C.
If we were to specify our measurand as the surface temperature, we would need to evaluate
the possible temperature difference between our thermocouple and the surface, across the
airgap present. One way of evaluating this would be to subject the thermocouple plate to acalibration plate. The difference may then be measured and an appropriate component
added, with the uncertainty evaluated from, e.g. the maximum deviation combined with the
temperature measurement uncertainty.
6.3 Glow wire test
The objective of a glow wire test (EN 60695-2), is to investigate the non-flammability of
insulation materials. A piece of material, typically plastic, is subjected to a hot wire, typi-
cally NiCr, heated by an electric current. It is observed that the material does not ignite af-
ter contact with the wire for a specific time at the specified temperature. The temperature is
measured by a thermocouple in the wire near the point of contact.There are some physical dimensions (diameter of wire and penetration) and mechanical
specifications (pressure applied) that must kept for the test procedure to be fulfilled. Also
the construction of the thermocouple and its integration into the glow wire is specified in
the standard.
The measurement itself is specified as the temperature of the glow wire. The specification
for the thermocouple to use is 0,4 % T(in C), and a verification measurement specified is
the generation of the melting of silver with a tolerance of 15 C; a somewhat relaxed re-
quirement in comparison.
Let the thermocouple voltage be read by a voltmeter with an internal calculation of a type
K thermocouple temperature. The 1-year specification of the instrument is given in theform:
The specification for a Type K thermocouple is thus 1,0 C. We interpret this as a rectan-
gular distribution of the instrument error of (0,0 1,0) C, hence the instrument error is
TV = 0 C, u(TV) = 1 C / 3 = 0,58 C.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
26/33
26
6406 HDJ 2004 11 30
A certified class 1 type K thermocouple, the tolerance (above 375 C) is given as 0,4 % of
the temperature. Hence, when the voltage generated equals the reference value at tempera-
ture T, the true temperature is in the interval T (1 0,4 %). The temperature indicated on
the display of the meterTI will be that of the reference equation, the error due to the probe
id thus Tc = 0 C, with u(Tc) = 0,4 % / 3 TI . The value ofTIwill be given by the read-
ing, and let us assume a constant reading is obtained, i.e. s(TI) = 0. There will then be acontribution from the resolution of the display, a rounding of 0 0,05 C; hence TI = 0,u(TI) = 0,05 C / 3 = 0,029 C.
Our basic model function becomes:
T= TI+ TI+ Tc + TV
The uncertainty is derived from the simple sum/difference formula:
u(T)2 = (0 C)2 + (0,029 C)2 + (0,23 % TI)2 + (0,58 C)2,
varying from 2,1 C at 900 C to 2,4 C at 1000 C.
Because our dominating term is the tolerance of the thermocouple, which we assigned arectangular distribution, the correct coverage factor is in fact k= 1,65. The expanded un-
certainty at 950 C thus becomes U(950 C) = 3,7 C.
Is it necessary to evaluate uncertainty contributions from the measurement conditions to
the result?
No, it should not!
The laboratory must document that the conditions specified (mechanical properties and
dimensions) are fulfilled, but it would be the task of the standardization committee to
translate the measurement conditions into an overall method uncertainty. This method
uncertainty would then be the appropriate uncertainty to compare against in interlaboratory
comparisons.
The uncertainty reported by a laboratory for a quantitative test result should be the meas-
urement uncertainty obtained for the measurement.
6.4 EMC - Radiated Immunity Test (IEC 61000-4-3)
For EMC compliance an instrument must be immune to an applied electromagnetic field.
A test is performed where an instrument is placed in a electric field of known field strength
and the ability of the instrument to function is monitored. To fulfil the test conditions, the
field distribution must be uniform over the test volume within 6 dB, and the field strength
be known. Thus, the test conditions rely on a calibrated field probe and measurementequipment, and compliance with the test specification of a uniform field.
The measurement uncertainty budget below is based on the assumption that it has been
demonstrated during calibration that the 6 dB field uniformity has been achieved. Once the
expanded uncertainty has been calculated then the value is related to the standard severity
level (e.g. 3 V/m) to provide an increased test level, which accounts for the uncertainty in
the test configuration.
The mathematical model for the measurement process is assumed to be:
F = FSM+ FSAW+ PD + PAH+ FD + RS+ REUT
It should be noted that this model does not account for the unknown influence of the un-specified semi-anechoic chamber and the significant variations caused by reflections from
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
27/33
27
6406 HDJ 2004 11 30
the EUT. The contribution form the test set up will need to be evaluated and given a value.
In the mean time the 6 dB field uniformity restriction is a means to at least have some con-
trol over this contribution for an empty chamber. Thus, to include the lack of informa-
tion/characterisation of the measurement setup, the 6 dB requirement should be added to
the uncertainty forFD.
Sym-
bolQuantity Units Comment
FSM Field Strength monitor dBV As reported on the Field Probe calibration certificate
FSAWField Strength accept-
ability windowdB
Usually a set parameter to allow the software to ac-
cept a value for field strength that is within an ac-
ceptable window of the calibration level.
PDForward power Meas-
urement DriftdB
A function of the power meter or other device con-
nected to monitor forward power.
PAHPower Amplifier
Harmonics
dBContribution due to inclusion of amplifier distortion
in power meter reading.
FDEffect of field distur-
bancedB
Possible effects due to table and/ supporting struc-
tures.
Mismatch Not applicable.
RSMeasurement System
RepeatabilitydB
The repeatability of the measurement system is de-
termined by obtaining the standard deviation of a
series of repeated readings on a stable EUT. It
should include typical variations that will occur in
normal testing.
REUT Repeatability of EUT dB
This will only need to be considered if the measured
result is close to the specification limit to the extent
that variations due to the EUT may affect compli-
ance.
7/27/2019 0341-ID-A Practical Guide for the Estimation of Uncertainty in Testing
28/33
28
6406 HDJ 2004 11 30
7 Appendix
Statistics and uncertainty primer
A stochastic variable or a random variable is a variable that may assume different values, atdifferent trials or outcomes. A stochastic variable, as it may appear in measurements, may be
fully characterized by a probability distribution function.
A distribution function is a function,f(x) describing the probability of occurrence of the out-
come of a random variable. For a continuous distribution, the probability that Xfalls in the
interval x < X< x + dx equals f(x)dx. For a discrete distribution, such as the outcome of a
throw with a dice,f(x) is a set of values. It is so that 1)( =xall
dxxf .
The expectation value (or mean value) of a distribution function is given by
= xall dxxfxXE )()(The variance of a distribution function is given by
( ) =xall
dxxfXExXV )()()(2
The GUM method relies on the basic concept:
The resultis an estimate of the expectation value and the standard uncertainty is the square-
root of an estimate of variance.
TheNormal distribution or Gaussian distribution has a special significance. It can be shown
(the Central Limit Theorem) that a combination of random variables with any well-behaved
distribution functions will approach the Normal distribution. Hence, in many physical proc-
esses, where sources from the scale of molecular motion and up affect the outcome, quantities
follow a Normal distribution.
The Normal distribution is described by two parameters, and , and the probability distribu-
tion function is given by
( )2
2
2
2
1)(),;(
==
x
exfxN .
For the Normal distribution, the expectation value isE(X) = and the variance is V(X) = 2
.
Combining uncertainties
Given a model function connecting input quantities nXXX ,,, 21 K and an output quantity Y
by a mathematical function ),,( 1 NXXfY K= , the propagation of uncertainties follows froma first order Taylor expansion off:
The result is given by ),,(