Uncertainty of Measurement Results - uni-plovdiv.bgweb.uni-plovdiv.bg/kiril/Students/Masters-MYK/... · 2015-03-21 · Uncertainty - definition • “Non-negative parameter characterizing

Uncertainty ofMeasurement Results

A measurement result is an information. Without knowledge of its uncertainty it is just a rumour.

Albert Weckenmann

Quality Assurance and Quality Control

Overview

• Uncertainty – definition;• Uncertainty and Error;• Why do we need uncertainty?• When should you evaluate uncertainties of

measurement results?• Uncertainty sources and uncertainty components;• Low of propagation of uncertainty;• Procedure’s steps for uncertainty evaluation;• Examples;• Spreadsheet method of Kragten for uncertainty

calculation and assessment.

Uncertainty – definition

Basic documents

• GUM: “Guide to the Expression of Uncertainty in Measurement” (JCGM 100:2008), first published in 1993 by ISO in collaboration with BIPM, IEC, IFCC, ILAC, IUPAC, IUPAP and OIML;

(freely available from www.bipm.org)

• EURACHEM/CITAC-Guide “Quantifying Uncertainty in Analytical Measurement”,3rd edition published in 2012 (QUAM:2012.P1);

(freely available from www.eurachem.org)

• EUROPEAN CO-OPERATION for ACCREDITATION “Evaluation of the Uncertainty of Measurement in Calibration” (EA-4/02 M: 2013);

(freely available from http://www.european-accreditation.org/)

Quantity and Uncertainty/Error approach

• Quantity - property of a phenomenon, body, or substance, where the property has a magnitude that can be expressed as a number and a reference (VIM: 2012; § 1.1);

• Measurand – the quantity intended to be measured;(VIM: 2012; § 2.3);

• Error Approach (Traditional Approach or True Value Approach) –to determine an estimate of the true value that is as close as possible to that single true value. The deviation from the true value is composed of random and systematic errors.

• The objective of measurement in the Uncertainty Approach is not to determine a true value as closely as possible. Rather, it is assumed that the information from measurement only permits assignment of an interval of reasonable values to t he measurand , based on the assumption that no mistakes have been made in performing the measurement.

Uncertainty - definition

• “Non-negative parameter characterizing the dispersion of the quantity values being attributed to a measurand, based on the information used”(VIM: 2012, § 2.26)

• “A parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand”(GUM: 2008, § 2.2.3)

result = value ± uncertainty

Uncertainty and Error

+U-U

YA

YA

Ytrue value

Error

Yestimated valueof the measurand

measurand

Correction for all recognized systematic effects

Why do we need uncertainty?

• It is an intrinsic part of every measurement result;• It is required by ISO/IEC 17025: 2005 – Accreditation;

5.4.6.2 Testing laboratories shall have and shall apply procedures for estimating uncertainty of measurement;

5.4.6.3 When estimating the uncertainty of measurement, all uncertainty components which are of importance in the given situation shall be taken into account using appropriate methods of analysis.

• It improves the knowledge about the measurement procedure;

• It allows comparison of results – GUM does not require statistical tests unless you need it….


The uncertainty of the result demonstrates the metrological QUALITY of the measurements (not measuring with the smallest achievable uncertainty):

• In laboratory � documents in transparent way the measurement procedure;

• For end-user � gives the result with proper confidence;

Measurement uncertainty - is it a criterion for judgement and placing orders of testing laboratories?

• Measurement uncertainties should not be misused as a quality indicator for laboratories - otherwise there will be a tendency among some laboratories to report unrealistically low measurement uncertainties;

• There is no need to have lower uncertainties then requested from the customer.


Customer needs

• Before calculating or estimating the measurement uncertainty, it is recommended to find out what are the needs of the customers;

• The main aim of the actual uncertainty calculations will be to find out if the laboratory can fulfill the customer demands;

• Customers often are not used to specify demands, so in many cases the demands have to be set in dialogue with the customer.


A well documented uncertainty statement: • Underpins your results and provides transparency;• Identifies major uncertainty contributors -

find out ways to improve the procedure;• Demonstrates compliance with limits (legal or contractual).

� Your best defence in discussions!

� Repeating the measurement 2, 10 or 100 times gives you only repeatability or over time within-lab reproducibility (not total uncertainty)!


Comparison of results

Lab ALab ALab ALab A Lab BLab BLab BLab BLab ALab ALab ALab A Lab BLab BLab BLab BLab ALab ALab ALab A Lab BLab BLab BLab B

10.5

11.5

11.0

12.0

12.5

mg

kg-1

Mean value

± 1 s ± U


Repeatability / Reproducibility / Precision

Measurement precision: closeness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditions (VIM: 2012, § 2.15).

• Measurement repeatability: measurement precisionunder repeatability conditions of measurement (VIM: 2012, § 2.21);

• Measurement reproducibility: measurement precisionunder reproducibility conditions of measurement (VIM: 2012, § 2.25);


Compliance against limits

Example: QUAM:2012

A decision rule must be defined!

Guidance can be found in EURACHEM/CITAC (2007)“Use of uncertainty information in compliance assessment”.

When should you evaluate uncertainties of measurement results?

• When a procedure is introduced inside your laboratory;

• When a critical factor changes in the procedure (instrument, operator, sample type …);

• During / together with procedure validation.

� An individual evaluation is not needed for every individual result produced!

When should you evaluate uncertainties of measurement results?

� In case the laboratory needs tocomplains with regulation � followsthe respective document;

� In case of no guidelines available:(i) Use a common sense for setting the

frequency;(ii) Monitor the data from control chart.

How often re-evaluation of uncertainty is needed?

Uncertainty sources

� Sampling� Storage conditions� Instrument effects� Reagent purity� Assumed stoichiometry� Measurement conditions� Sample effects� Computation effects� Blank correction� Operator effects� Random effects

What is usually taken into account when calculating a

“traditional” Confidence Interval (CI)?

Uncertainty components

Uncertainty “type”

Uncertainty components are formally grouped into two categories based on their method of evaluation, “A” and “B”.

• Type A evaluation (of uncertainty) - method of evaluation of uncertainty by the statistical analysis of series of observations;

• Type B evaluation (of uncertainty) - method of evaluation of uncertainty by means other than the statistical analysis of series of observations;(previous experiments, literature data, manufacturer’s information, expert’s estimate).

(GUM: 2008, § 0.7, §2.3.2, §2.3.3)


Standard uncertainty - us

• When expressed as a standard deviation, an uncertainty component is known as a standard uncertainty – �(��).(QUAM:2012; § 2.3.1)

• Uncertainty of the result of a measurement expressed as a standard deviation.

(GUM: 2008; § 2.3.1)


Combined (standard) uncertainty - uc

For a measurement result y, the total uncertainty, termed combined standard uncertainty and denoted by uc(y), is an estimated standard deviation equal to the positive square root of the total variance obtained by combining all the uncertainty components, however evaluated, using the law of propagation of uncertainty or by alternative methods (e.g. the spreadsheetmethod of Kragten).

(QUAM:2012; § 2.3.2)

��

� �

y

For example:

� = (��, ��, … , ��, ��)In general:



Law of propagation of uncertainty:

� = (��, ��, … , ��, ��)

where, �� = �� - sensitivity coefficients;

�(��), �(��) - standard uncertainties of �� and �� ;

�� - correlation coefficient between �� and �� ;

�� = ��. � ��

��+ 2. � ��

�

�,��"�

. �� . � �� . � �� . ��,�



Law of propagation of uncertainty:

� = (��, ��, … , ��, ��)

When the function involves only sum or difference (e.g.):

� = �� + �� − �$ �� = �(��)� + �(��)� + �(�$)�

�� = � %�%��

�. � ��

�

��

When all variables��, ��, … , ��, ��are independent:

When the function involves only product or quotient (e.g.):

� = ��$�� = �(��)

��+ �(��)

��+ �(�$)

�$�

�� = ��. � ��

��+ 2. � ��

�

�,��"�

. �� . � �� . � �� . ��,�


Expanded uncertainty - U

• U is obtained by multiplying the combined standard uncertainty uc(y), with a coverage factor k;

• The expanded uncertainty U provides an interval within which the value of the measurand is believed to lie with a higher level of confidence (QUAM:2012; § 2.3.3);

• The choice of the factor k is based on the level of confidence desired:– k = 2 corresponded to P = 95%;– k = 3 corresponded to P = 99%;

For normal distribution:µ ± 2σµ ± 3σ

(Re)-calculating standard uncertainty (standard deviation)

Confidence Interval (CI)

The used level of confidence (α) anddegree of freedom (f = N-1) are known.

�̅ ± (),* . +, �̅ ± -. -. = (),* . +,

�/ = + =-.. ,(),*


Rectangular distribution

• Rectangular distribution is usually described in terms of:the average value and the range (±a);

• One can only assume that it is equally probable for the value to lie anywhere within the interval;

• Certificates or other specification give limits where the value could be, without specifying a level of confidence (or degree of freedom).

• Example:Concentration of calibration standard is quoted as (1000 ± 2) mg/l.

x

p(x)0 � =

122, −2 < � < 2

0,otherwise

�/ = + =2

3


Triangular distribution

• Distribution used when it is suggested that values near the centre of range are more likely than near to the extremes (Values close to mathematical expectation are more likely than near the boundaries);

• Example:Volumetric glassware.

x

p(x)0 � =

2 + �2� , −2 < � < 02 − �2� , 0 < � < 2

0, otherwise

�/ = + =26

If any time in doubt, use the rectangular distribution !

Procedure’s steps for uncertainty evaluation

EURACHEM/CITAC Guide

• In GUM: 2008 (§ 8) is presented summary of procedure for evaluating and expressing uncertainty;

• Approach for uncertainty estimation is also suggested in QUAM: 2012 (a procedure containing 4 steps):

Measurand = quantity intended to be measured.(VIM: 2012; § 2.3)

Step 1

Step 2

Measurand = the particular kind of quantity to be measured, usually the concentration or mass fraction of an analyte (QUAM: 2012; § 5.2).



Step 2

Step 3

Step 4

� = (��, ��, … , ��, ��)

Express mathematically the relationship between the measurand y and the input quantities �� on which y depends:

Input quantities (��) may be quantities whose value and standard uncertainty are directly determined in the current measurement (evaluation of Type A, statistical analysis of series of observation) or brought into the measurement from external sources (evaluation of Type B).

Should any correction factor be included?



Step 4

(-� ± ?) mg/g, k = 2


Understanding the measurement

The evaluation of uncertainty is neither a routine task nor a

purely mathematical one; it depends on detailed knowledge

of the nature of the measurand and of the measurement.

(GUM: 2008, 3.4.8)

The ‘state of the art’ of uncertainty evaluation is determined by the ‘degree’ of understanding of measurement, that includes, critical thinking, professional skills and good understanding of

the processes and chemistry involved!

Examples

Volume of liquid (solution)

Example 1. Calculate the combined uncertainty of the volume of water solution prepared in 1000 ml volumetric flask if it is known:

• In the certificate of the volumetric flask is declared:(1000.0 ± 0.2) ml at 20 0C;

• The temperature in the laboratory varies within the interval:(20 ± 5) 0C. The coefficient of volume expansion for water is2.1x10-4 0C-1 (the volume expansion of the flask itself is neglected);

• Repeatability - the uncertainty due to variations in filling is assessed to be 0.3 ml.

Volumetric flaskcertificate

Examples

Concentration of calibration standard

Example 2. A calibration standard solution of Zn (0.102 mg/l) was prepared by transferring 1 ml of commercial reference material (ICP Multi VI - 102 mg/l Zn) into 1000 ml volumetric flask. The flask was filled to the mark with 0.1 % HNO3 v/v. Calculate the combined uncertainty of the conc. of the obtained Zn calibration solution, if:

• In the certificate of the commercial RM is declared that the concentration of Zn is (102 ± 5) mg/l – no information is supplied about the degrees of freedom and confidence level;

• The standard uncertainty of the pipetted volume is 0.005 ml;

• For the standard uncertainty of the volumetric flask volume take the obtained value from the previous example (Problem 1).

ICP Multi VIcertificate

CPAchem Muilti-element #2Acertificate

Spreadsheet method of Kragten

Applicable when:• Either� = ��, ��, … , ��, �� is linear in ��;

• or�(��) is small compared to ��;

Method of Kragten

Calculating the uc of solution volume

1.4%

79.2%

19.4%

Ser.

Temp.

Repeat.

Example 1.V flask (ml)= 1000 Variable mean u_s Ser. Temp. Repeat.

Certificate uncertainty (ml)= 0.2 mlSer. 1000 0.08 1000.1 1000.0 1000.0

Temp. variation (0С)= 5 mlTemp. 0 0.61 0.0 0.6 0.0

Fill repeatability (ml)= 0.3 mlRepeat. 0 0.30 0.0 0.0 0.3

V flask(ml)= 1000 0.68 1000.08 1000.61 1000.30

diff 0.082 0.606 0.300

diff^2 0.007 0.367 0.090 0.464sum

Contribution, % 1.4% 79.2% 19.4%

Method of Kragten

Calculating the uc of standard concentration

96.9%

3.0%

C1

V1 pipette

V2 flask

Example 2.

C1 (mg/l)= 102 Variable mean u_s C1 V1 pipette V2 flask

C1 certificate uncertainty (mg/l)= 5 mg/lC1 102.0 2.89 104.9 102.0 102.0

V1 pipette (ml)= 1.000 mlV1 pipette 1.000 0.005 1.000 1.005 1.000

V1 u_s (ml)= 0.005 mlV2 flask 1000.0 0.68 1000.0 1000.0 1000.7

V2 flask (ml)= 1000.0

V2 u_s (ml)= 0.68 C2 (mg/l)= 0.102 0.003 1.05E-01 1.03E-01 1.02E-01

diff 2.89E-03 5.10E-04 -6.93E-05

diff^2 8.33E-06 2.60E-07 4.80E-09 8.60E-06sum

Contribution, % 96.9% 3.0% 0.1%

Method of Kragten

Calculating the uc of Cx - Bracket calibration

6.3%

64.5%1.7%

7.3%

20.1% C3

C4

A3

A4

Aх

Example 3. Calculating the combined uncertainty of the concentration of Zn in surface water (real sample determined by FAAS).

34

3443 )()(

AA

AACAACCx XX

−−+−=

C3 Cx C4

A3 Ax A4

Advantages:• The uncertainty of the concentration of the

calibration standards is included;• True homoscedastic model;

mean u_s RSD% Variable mean u_s C3 C4 A3 A4 Aх

C3 (mg/l)= 1.02 0.0296 2.90 mg/lC3 1.02 0.03 1.05 1.02 1.02 1.02 1.02

C4 (mg/l)= 2.04 0.0594 2.91 mg/lC4 2.04 0.06 2.04 2.10 2.04 2.04 2.04

A3 (signal)= 0.406 0.006 1.48 signalA3 0.406 0.006 0.406 0.406 0.412 0.406 0.406

A4 (signal)= 0.806 0.008 0.99 signalA4 0.806 0.008 0.806 0.806 0.806 0.814 0.806

Aх (signal)= 0.652 0.008 1.23 signalAх 0.652 0.008 0.652 0.652 0.652 0.652 0.660

Cx (mg/l)= 1.65 0.05 1.659 1.684 1.641 1.635 1.668

diff= 0.011 0.037 -0.006 -0.012 0.020

diff^2= 0.00013 0.00133 0.00004 0.00015 0.00042 0.00207sum

Contribution, % 6.3% 64.5% 1.7% 7.3% 20.1%

Uncertainty of Measurement Results

Conclusions

Uncertainty estimation according to the GUM is a useful and accepted concept:

� It allows the analyst to combine prior knowledge and observations in a consistent and well defined way;

� It allows others (e.g. assessors) to understand what and how things were done;

� It doesn’t requires to measure with smallest achievable uncertainty, but with the most realistic one.


Conclusions

Uncertainty adopted and accepted by:

• National Metrology Institutes and BIPM;

• ISO/IEC 17025: 2005 (required for accreditation);

• IUPAC, OIML and accreditation community such as EA and ILAC;

• CEN is incorporating these concepts.


Thank you for your attention!

Documents

Uncertainty of Measurement Results - uni-plovdiv.bgweb.uni-plovdiv.bg/kiril/Students/Masters-MYK/... · 2015-03-21 · Uncertainty - definition • “Non-negative parameter characterizing