Upload
agatha-pitts
View
218
Download
1
Embed Size (px)
Citation preview
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
1
Wolfgang Garche
Saxony-Anhalt Environmental Protection AgencyDepartment Air Quality Monitoring, Information, Assessment
Estimation of Measurement Uncertainties
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
2
Measurementerror
Systematicerror
Randomerror
Known systematic error
Unknown systematic error
Correction Residual error
Measurement result Measurement uncertainty
(Source: EUROLAB Technical Report 1/2006 „Guide to the Evaluation of Measurement Uncertanty for Qualitative Test Results“)
Types of measurement error and their consideration in determining the result of a measurement and the associated uncertainty
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
3
Measured values for simultaneous occurring of random and systematic errors
(Source: EUROLAB Technical Report 1/2006 „Guide to the Evaluation of Measurement Uncertanty for Qualitative Test Results“)
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
4
Uncertainty of measurement is a parameter, associated with the result of a measurement, that characterises the dispersion of the values that could reasonably be attributed to the measurand.
Uncertainty of the resultEstimated quantity intended to characterise a range of values which contains the reference value.
Definitions
Standard uncertainty (u)Uncertainty of the result of a measurement expressed as a standard deviation.
Combined standard uncertainty (uc )Standard uncertainty of the result of a measurement when that result is obtained from the values of a number of other quantities, equal to the positive square root of a sum of terms, the terms being the variances or covariances of these other quantities weighted according to how the measurement result varies with changing these quantities.
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
5
Expanded uncertainty (Up)Quantity defining an interval about the result of a measurement that may be expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand.
Coverage factor (k)Numerical factor used as a multiplier of the (combined) standard uncertainty in order to obtain an expanded uncertainty.
AccuracyThe closeness of agreement between a test result and the accepted reference value.
TruenessThe closeness of agreement between the average value obtained from a large series of test results and an accepted reference value.
PrecisionThe closeness of agreement between independent test results obtained under stipulated conditions.
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
6
EN ISO 20988:2007 „Air quality – Guidelines for estimating measurement uncertainty“
A five-step procedure for uncertainty estimation is described:
1. Problem specification
2. Statistical analysis
3. Estimation of variances and covariances
4. Evaluation of uncertainty statements
5. Reporting
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
7
Objective is to specify
- the measurement
- the wanted uncertainty statement
- the experimental data
- effects not described by experimental data
Problem specification
Element Direct approach Indirect approach
experimental design one design more than one design
experimental data y(j) with j = 1 to NyR (reference value)
x1(j) with j = 1 to N, x1R
x2(j) with j = 1 to N, x2R
….
Additional deviation, if appropriate
Y Y
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
8
Statistical analysis
Element Direct approach Indirect approach
Method model equation not required y= f(x1, x2,…)
Statistical model equation Y=y + Y Y= f(x1, x2,…)+Y
Variance budget equation var(Y)=var(y)+var(Y) var(Y)=c1² var(x1)+ c2² var(x2)+….
+2c1c2cov(x1,x2)+ … var(Y)
A statistical model equation shall be established to describe the relationship between the statistical population of possible results of measurement Y and the input data.
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
9
Direct approach
A direct approach provides a series of results of measurement observed in a specified experimental design and, if appropriate, expert judgement of additional deviations Yi caused by effects not described by the series of observations
iYyY
variance budget equation:
where: Y a possible result of measurement y a realized result of measurement (input data) dYi an additional deviation of result of measurement y not described by the experimental data
(can be neglected, if the corresponding variance contributes less than 5 % to the variance estimate var(Y) used in the uncertainty estimation)
statistical model equation:
)var()var()var( iYyY
where: var(Y) estimate of the variance of possible results of measurement Y var(y) estimate of the variance of a series of results of measurement y var(Yi) estimate of the variance of additional deviation Yi , obtained by a type B evaluation
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
10
Indirect approachprovides for each input quantity xi of a known method model equation y = f(x1,..,xK ) either a series of experimental data xi(j) collected in a specified experimental design, or a variance estimate var(xi). If appropriate, additional deviations δYj , which are not described appropriately by the experimental data to be evaluated, are assessed by expert judgement.
iK YxxfY ),...( 1statistical model equation:where Y possible result of measurement; xi input quantity of the method model equation y = f (x1,.., xK )dYi additional deviation of result of measurement y not described by the experimental data
variance budget equation:
M
jj
K
i
K
i
K
ijKijii YxxccxcY
11 1 11
2 )var(),cov(2)var()var(
where ci sensitivity coefficient with respect to variations of input quantity xi
cov(xi, xj) estimate of covariance between input quantities xi and xj
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
11
Sensitivity coefficients
The sensitivity coefficient ci is the partial derivative of the method model function
y = f (x1,.., xK )
i
Ki x
xxfc
),....,( 1
ci - can be calculated numerical from the partial derivative
N
j
ii N
jxjyc
1
)(/)(
ci - as mean value of the ratio of observed changes of the results of measurement y(j) to the the changes of the input data xi(j)
V
My Example: y= mass concentration of particles
)var(),cov(2)var()var()var( 22 YVMccVcMcY VMVM
VM
ycM
1
2V
M
V
ycV
0),cov( VM
)var()var()var(1
)var(4
2
2YV
V
MM
VY
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
12
Estimation of variances and covariances
Type A: Evaluation using statistical analysis of measurement series
The calculation method is depending on the experimental design used to collect the input data.
Type B: Evaluation using means other than statistical analysis of measurement series (expert judgement)
If information are available• on the expected range of variation [min(δYj) < δYj < max(δYj)] of the
deviation δYj
• on the expected type of the statistical population of δYj
Statisticalpopulation
Rangemax(δYj) = – min(δYj)
Estimated variancevar(δYj)
Rectangular a a²/3
Triangular a a²/6
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
13
Covariances
The covariance associated with the values xi and xk assigned to two input quantities of the applicable method model equation shall be zero if:
• xi and xk have not been observed repeatedly in the same experimental design
• either xi or xk was kept constant, when providing repeated observations of the other quantity repeatedly.
Calculation of covariances can be avoided often by appropriate choice of experimental designs!
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
14
Evaluation of uncertainty statements
Combined standard uncertainty: )var()( Yyuc
y
yuyu c
r
)()(
where var(Y) is the estimate of the variance of the population of possible results of measurement Y
Relative standard uncertainty:
Expanded Uncertainty: )()( yukyU cp
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
15
Expanded uncertainty – coverage faktor k
)()( yukyU cp coverage factor k and coverage probability p shall be stated when an expanded uncertainty Up(y) is reported
relationship between k and p:
1. y is a mean value (N>1) of independent observations with the same measuring system
2. y is obtained by single application of a measuring method, distribution of possible results approximately is Gaussian
3. y is obtained by single application of a measuring method, distribution of possible results are not described by a Gaussian distribution
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
16
Case 1. and 2.: k = t(p,(direct approach)
where:
t(pis the (1-p)-quantile of Students t- distribution of ν degrees of freedom
p is the coverage probability of interval [–t(p,ν); +t(p,ν)] by Students t-distribution with ν degrees of freedom
is the number of degrees of freedom; = N – 1
Uncertainty interval for a coverage probability: )]();([ yukyyuky
Indirect approach: Welch-Satterthwaite-equation:
K
i
M
j j
j
i
i
eff Yxc
Y
1 1
224
2
)(var)(var
)(var
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
17
A report on execution of a specified task of uncertainty estimation shall include (at least) the following items:
• Problem specification including- method of measurement- wanted uncertainty statement- statistical population of possible results of
measurement considered- experimental data and experimental design- effects not described by experimental data
• statistical analysis, describing the applied statistical model equation and the variance (budget) equation
• evaluation methods, describing the applied evaluation methods
• numerical value of the wanted uncertainty statement and its range of application
Reporting
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
18
Uncertainty estimation in practise
What is required ??• Analysers have to fulfil the data quality objectives of the Directive
2008/50/EC “on ambient air quality and cleaner air for Europe”.(Uncertainty and minimum data capture)
• Analysers have to fulfil the relevant performance characteristics and criteria of the EN standards.(Type approval test)
Test reports of the type approval should contain all needed uncertainty information for a specific type of analyser.
But the user has to show that an analyser fulfil the requirements on measurement uncertainty also under the specific conditions on the monitoring site. suitability evaluation
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
19
Type approval of analysers
1. the value of each individual performance characteristic tested in the laboratory shall fulfil the requirements
2. the expanded uncertainty calculated from the standard uncertainties due to the values of the specific performance characteristics obtained in the laboratory tests fulfils the requirements
3. the value of each of the individual performance characteristics tested in the field shall fulfil the requirements
4. the expanded uncertainty calculated from the standard uncertainties due to the values of the specific performance characteristics obtained in the laboratory and field tests fulfils the requirements
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
20
When a type approved analyser has been chosen for a particular measuring task, then the suitability of this analyser shall be evaluated at a specific measuring location.
The analyser at the specific site has been judged to conform with the EU data quality objectives
An expanded uncertainty calculation for the type approved analyser shall be made according to the specific circumstances at the monitoring station or site.
If the site-specific conditions are outside the conditions for which the analyser is type approved, then the analyser shall be retested under these site-specific conditions and a revised type approval will be issued.
If the analyser complies with the requirements, then that particular analyser may be installed and used at that monitoring station.
General Requirements:
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
21
Parameters Remarks
Sample pressure variation The sample gas pressure variation during a whole period of a year shall be estimated.
Sample gas temperature
variation
The sample gas temperature variation during a whole period of a year shall be estimated. Sample gas temperature may be controlled by heating or thermostating.
Surrounding air temperature
variation
The temperature fluctuation shall be within the range specified in the type approval test. Temperature may be controlled thermostatically.
Voltage variation The voltage fluctuation shall be within the range in the type approval test. Voltage fluctuations may be controlled by means of voltage stabilizers.
H2O concentration range The H2O concentration range during a whole period of a year shall be estimated.
H2S, NH3, NO, NO2 and
m-xylene concentration ranges
The concentration range of each compound during a whole period of a year shall be estimated.
Expanded uncertainty of the
calibration gas
The expanded uncertainty of the calibration gas shall be included. This implies the uncertainty of the calibration gas itself as well as the uncertainty of any dilution system (where applicable)
Calibration frequency The intended calibration frequency shall be used for the calculation of the influence of the drift.
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
22
Choosing of the needed performance characteristics from the type approval test report
Evaluation or estimation of the site-specific conditions and of the uncertainty of the span gas used for calibration
Calculation of the combined expanded uncertainty with inclusion of the site-specific conditions
Comparison with the uncertainty requirements
Operating Sequence:
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
23
Example: Uncertainty of the certification of a transfer standards
Measurand y: NO concentration of the test gasMethod: analyzer according to EN 14211Input data: daily measurements of a certified NO test gas
uncertainty of the test gas concentration
Model equation: )()( jeyjy R Ryjyje )()(
)()()var( 22 euyuy R
with
Variance equation:
0),cov( eyR
Residual standard deviation:
N
jRyjy
Neu
1
2))((1
)(
Standard uncertainty: )()()( 22 euyuyu R
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
24
Calculated:mean value 132,65 ppb
residual standard dev. u(e)= 0,538 ppb
Date y(j) y(j)-yR
22.10.2008 133,3 0,5
23.10.2008 132,5 -0,3
24.10.2008 133,0 0,2
27.10.2008 133,0 0,2
28.10.2008 133,1 0,3
29.10.2008 133,2 0,4
30.10.2008 132,7 -0,1
03.11.2008 132,7 -0,1
04.11.2008 133,0 0,2
05.11.2008 133,1 0,3
06.11.2008 133,0 0,2
07.11.2008 132,7 -0,1
10.11.2008 132,8 0,0
14.11.2008 131,7 -1,1
17.11.2008 132,9 0,1
18.11.2008 133,1 0,3
19.11.2008 133,2 0,4
20.11.2008 133,3 0,5
21.11.2008 133,0 0,2
24.11.2008 132,7 -0,1
25.11.2008 132,7 -0,1
26.11.2008 131,7 -1,1
27.11.2008 131,6 -1,2
28.11.2008 132,2 -0,6
01.12.2008 131,8 -1,0
02.12.2008 132,2 -0,6
03.12.2008 132,8 0,0
04.12.2008 132,4 -0,4
05.12.2008 132,7 -0,1
08.12.2008 132,5 -0,3
09.12.2008 131,5 -1,3
certified NO concentration yR = 132,8 ppb
stated expanded uncertainty U(yR) = 2% with k=2
standard uncertainty u(yR)=1,328 ppb
ppbyu
euyuyu R
433,1538,0328,1)(
)()()(
22
22
y
Expanded uncertainty:
%2,2022,0)(
92,2)()(
04,2)31;05,0(
95,0
95,0
y
yU
ppbyukyU
tk
y
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
25
Ozone measurements:(example C.2 from EN ISO 20988)
Method: automatically measurements according EN 14625 (UV-absorption)
function control every 25 h with zero and span gas
daily correction of zero offsets
Measurand: 1-h mean value of the ozone concentration in ambient air
Described effects: variations of surrounding air temperature and pressure
Method model equation: y = x – e(j)
x is the observed value without correction
e(j) is the offset for day j
Wanted uncertainty statement: standard uncertainty u(y), expanded uncertainty U0,95(y)
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
26
Experiment: 1. every 25 hours providing of zero gas and determination of the zero offset e(j) = x0(j)2. every 25 hours providing of span gas and evaluation of the
span factor ß(j) = xs(j)/ys
Input data: series of offset corrections (N=20)series of span factors (N=20)
Reference values: Zero gas y0 = 0 µg/m³Span gas ys = 280 µg/m³ u(ys) = 2,8 µg/m³
Effects not described: Influence of sampling device(Influence of humidity and other compounds in the
ambient air)
Variance budget equation: var(y) = u²(x) + u²(e) + 2cov(x,e) cov(x,e) = 0
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
27
Index j Observed measuring value for zero gas e/j)
Span factor ß(j)
1 -0,7 1,00
2 -0,9 0,96
3 -1,4 0,98
4 -0,9 0,99
5 -1,1 1,04
6 -0,3 1,05
7 -0,8 1,04
8 -0,8 1,03
9 -1,0 1,04
10 -1,0 1,03
11 -0,9 1,02
12 -0,8 1,02
13 -1,1 1,03
14 -0,8 1,07
15 -0,8 1,04
16 -0,6 1,02
17 -0,5 1,02
18 -1,0 1,05
19 -0,7 1,05
20 -1,0 0,97
Standard uncertainty of zero offset e:
³/89,0)(1
)(1
20 mµgjx
Neu
N
j
(includes the bias uB(e))
Model equation for x(j): Syjjx )()(
Variance equation:
),cov()(2)()(
)var(22
2S
S
S yjxy
yuuxx
(cov(ß,yS) = 0)
Standard uncertainty of ß:
036,0))(1(1
)(1
2
N
j
jN
u
(includes the bias uB(ß) )
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
28
Standard uncertainty of y:
)(
)()()( 2
22
2 euy
yuuyyu
S
S
Degrees of freedom:= 20 coverage factor: k = t(0,05,20) = 2,1
Uncertainty depends on the corrected measured value!
Now it is possible to calculate an uncertainty statement for the 1-hour mean values measured in the observed period depending on the corrected measured value.
EU-Twinning Project RO 2006 IB EN 09
Bucharest, 03.02.2009Saxony-Anhalt Wolfgang Garche State Environmental Protection Agency
29
Helpful documents and programs:
EN ISO 20988 “Air quality – Guidelines for estimating measurement uncertainty”
Eurolab Technical Report 1/2006 “Guide to the Evaluation of Measurement Uncertainty for Quantitative Test Results”
Eurolab Technical Report 1/2007 “Measurement Uncertainty Revisited: Alternative Approaches to Uncertainty Evaluation”
Excel-Program Nordtest TR 537 “Measurement Uncertainty Estimation”