Measurement Uncertainty Part I - Fundamentalsstatic.sif.it/SIF/resources/public/files/va2012/bich_i.pdfOrdinal quantities ordinal quantity quantity, defined by a conventional measurement

ISTITUTO NAZIONALE DI RICERCA METROLOGICA

1

Measurement Uncertainty

Part I - Fundamentals

Walter Bich

INRIM – Istituto Nazionale di Ricerca Metrologica

Torino (Italia)

International School of Physics Enrico Fermi "Metrology

and Physical Constants" Varenna 17 - 27 July 2012


2

Framework

• 1977-79 BIPM questionnaire on uncertainties

• 1980 Recommendation INC-1

• 1981 Establishment of WG3 on uncertainties under ISO TAG4: BIPM, IEC,

IFCC, ISO, IUPAC, IUPAP, OIML

• 1981 Recommendation CI-1981

• 1986 Recommendation CI-1986

• 1993 Guide to the expression of uncertainty in measurement

• 1995 Reprint with minor corrections

• 1997 Establishment of the Joint Committee for Guides in Metrology JCGM.

ILAC joins in 1998




3

• Present Chair: the BIPM Director



Joint Committee for Guides in

Metrology

• The JCGM has two working groups (WGs)

•WG 1 has responsibility for the Guide to the expression of uncertainty in

measurement, GUM

•WG 2 has responsibility for the International vocabulary of Basic and General

Terms in metrology, VIM

•See also www.bipm.org


4

WG1 published documents

JCGM 101:2008 (Monte Carlo)

JCGM 100:2008 (GUM 1995 with minor corrections)

JCGM 104:2009 (Introduction to uncertainty)

JCGM 102:2011 (Any number of output quantities)

All these are also published by OIML, ISO and IEC. Adopted by

IFCC, ILAC, IUPAC and IUPAP



JCGM 106:2012 (conformity assessment). Approved, to be published

soon


5

WG1 planned documents

JCGM 103 (Modelling)

JCGM 105 (Fundamental principles)

JCGM 100 (GUM revision)

JCGM 107 (Least-squares applications)

JCGM 108 (Markov Chain Monte Carlo)




6

WG2 document

JCGM 200:2012 (VIM3) The third version of this basic document.

Source of definitions of concepts and terms

Need for the two WGs to be as consistent as possible.



Not always an easy task!

http://www.bipm.org/utils/common/documents/jcgm/JCGM_101_2008_E.pdf


7

Measurement, properties and

quantities quantity

property of a phenomenon, body, or substance, where the property has a

magnitude that can be expressed as a number and a reference (VIM3,1.1)

a reference can be a measurement unit, a measurement procedure, a

reference material, or a combination of such



‘property’, ‘magnitude’ and ‘reference’ are explicitly considered as

“primitive”, therefore undefined


8

Measurement units

measurement unit

real scalar quantity, defined and adopted by convention, with which any other

quantity of the same kind can be compared to express the ratio of the two

quantities as a number (VIM3 1.9)



A measurement unit is thus a particular kind of reference


9

Ordinal quantities

ordinal quantity

quantity, defined by a conventional measurement procedure, for which a total

ordering relation can be established, according to magnitude, with other

quantities of the same kind, but for which no algebraic operations among

those quantities exist (VIM3, 1.26)

EXAMPLE 1 Rockwell C hardness.

EXAMPLE 2 Octane number for petroleum fuel.

EXAMPLE 3 Earthquake strength on the Richter scale.

EXAMPLE 4 Subjective level of abdominal pain on a

scale from zero to five.

NOTE 1 Ordinal quantities can enter into empirical

relations only and have neither measurement units nor

quantity dimensions. Differences and ratios of ordinal

quantities have no physical meaning.




10

VIM3 definitions

quantity value (value of a quantity)

number and reference together expressing magnitude of a quantity

(VIM3,1.19)



Length of a given rod: 5.34 m or 534 cm

numerical quantity value

number in the expression of a quantity value, other than any number serving

as the reference (VIM3,1.20)

Numerical values of the length of a given rod: 5.34 (unit metre) or 534

(unit centimetre)


11

VIM definitions

quantity calculus

set of mathematical rules and operations applied to quantities other than

ordinal quantities (VIM3, 1.21)



quantity numerical quantity value

unit

quantity value!


12

VIM3 definitions

measurement

process of experimentally obtaining one or more quantity values that can

reasonably be attributed to a quantity (VIM3, 2.1)

measurand

quantity intended to be measured (VIM3, 2.3)



My loose definition: process aimed at improving the state

of knowledge about a measurand


13

VIM definitions

measurement result

set of quantity values being attributed to a measurand together with any other

available relevant information

NOTE 1 A measurement result generally contains “relevant information” about the

set of quantity values, such that some may be more representative of the measurand

than others. This may be expressed in the form of a probability density function

(PDF).

NOTE 2 A measurement result is generally expressed as a single measured quantity

value and a measurement uncertainty. If the measurement uncertainty is considered

to be negligible for some purpose, the measurement result may be expressed as a

single measured quantity value. In many fields, this is the common way of

expressing a measurement result.

NOTE 3 In the traditional literature and in the previous edition of the VIM,

measurement result was defined as a value attributed to a measurand and explained

to mean an indication, or an uncorrected result, or a corrected result, according to

the context.




14

VIM definitions

2.10

measured quantity value

quantity value representing a measurement result

2.11

true quantity value

true value of a quantity

true value

quantity value consistent with the definition of a quantity




15

VIM definitions

2.27

definitional uncertainty

component of measurement uncertainty resulting from the finite amount of detail in

the definition of a measurand

NOTE 1 Definitional uncertainty is the practical minimum measurement uncertainty

achievable in any measurement of a given measurand.

Examples: Distance Varenna – Paris

Atomic weight of C

Air density in my laboratory




16

GUM scheme




17 International School of Physics Enrico Fermi "Metrology


Error (horror?)




More horror


19

Probable error

Now, re-interpret 𝜀 as probable error

Probability comes into play, and all its tools are at hand

First, 𝜀 is taken as a random variable

From here, and for a while, apologies to those who already know!

Most of the material taken from Wikipedia…




20

Random variables A random variable (RV) is a variable whose value is subject to variations due to

chance, or can take on a set of possible different values, each with an associated

probability.

They may also conceptually represent either the results of an "objectively" random

process (e.g. rolling a die), or the "subjective" randomness that results from

incomplete knowledge of a quantity.

The meaning of the probabilities assigned to the potential values of a random

variable is not part of probability theory itself, but instead related to philosophical

arguments over the interpretation of probability.

The mathematics works the same regardless of the particular interpretation in use.




21

Probability distributions




22

Probability density function




23

Cumulative distribution function




24

RVs and pdfs




25

Gaussian pdf



From Wikipedia


26

Gaussian CDF



From Wikipedia


27

Moments of RVs




28

Moments of RVs




29

A relevant pdf




30

Student’s t-distribution



From Wikipedia


31

Random vectors




32

Covariance matrix




33

Covariance




34

Correlation




35

Covariance in measurement

The role of covariance in measurement uncertainty is fundamental, and typically

underevaluated

«Neglecting covariances» is not a way to get rid of them.

Rather, it is a very stringent statement that they are equal to zero, a situation rarely

occurring in multivariate measurements




36

Multivariate Gaussian pdf




37

From the ugly duckling …




38

… to the beautiful swan




39

A question




40

Orthodox solution




41

Orthodox solution cont.




42

Orthodox solution fails When there is no sample, no statistics can be applied and the frequentist approach

fails.

In a Bayesian approach, a pdf can be associated to any quantity to describe the

state of knowledge on it. Guidance in JCGM 101:2008.

In the GUM this approach is adopted only in Type B evaluations, to associate an

uncertainty to the corresponding estimates. The uncertainty is the standard deviation

of the pdf.

However, to remain in the frequentist framework, degrees of freedom are artificially

attached also to these uncertainties!

The Bayesian framework works both for random and systematic errors

The GUM is being revised in this sense




43

A dirty trick




44

Multivariate generalization




45

Multivariate generalization




46

Limitations

We talked so far about pdfs, but we used only their first and second moments.

This procedure has two orders of limitations:

The reliability of the variance of the output quantity depends on the amount of non-

linearity in the neighbourhood of the estimates.

The variance of the output quantity is of little interest to end users, who typically

need a coverage interval, i.e., an interval containing the value of the measurand with

a stated probability, based on the information available.




47

Solution Instead of propagating only first and second moments of the input quantities Xi, their pdfs are propagated through the model. The method is more demanding in terms of amount of knowledge. One has to assign a pdf to each input quantity, based on the experimental data or other knowledge.




Assignment of input pdfs

•The principle of maximum entropy –you maximize a functional S, the

“information entropy”, under constraints given by the information

•According to the available information, it is based on

•Bayes’ theorem, typically when a series of indications is available




Assignment of input pdfs

•FOCUS: if a sample of indications is available, you assign a Student’s t-

distribution. Departure from the GUM!

•Luckily, extensive literature and guidance in Supplement 1 do most of the

job

•A dozen pdfs for the various cases are suggested




Problem Given the joint pdf of the N input quantities X, find the pdf of the output

quantity y.

Using the Jacobian method (any textbook on mathematical statistics)

1d...d... NY fgg X

Is the Dirac function




Solutions Analytical: A closed-form solution can be found only in the most simple (and

therefore uninteresting) cases. In general, the integrals involved in this solution must

be solved numerically. Not viable.

•Numerical 1: by numerical integration of the formal expression




The Supplement 1 approach: MCM

Numerical 2 (adopted in Supplement 1):

• Numerical simulation.

• Method selected: Monte Carlo (MCM).

• Tools: suitable random number generators for the various pdfs, reasonable computing power.

• Outcome: a numerical approximation for the output distribution (in various possible forms).

YG




Output of MCM

• From the numerical approximation for the output distribution, the required statistics, such as

• the best estimate for the measurand,

• its standard uncertainty, and

• the endpoints of a prescribed coverage interval can be obtained.




The method in a nutshell

From each input pdf draw at random a value xi for the random variable

Xi.

Use the resulting vector xr (r = 1,…M) to evaluate the model, thus obtaining

a corresponding value yr. The latter is a possible value for the measurand

Y.

Iterate M times the preceding two steps, to obtain M values yr for Y .




Representations of the probability

distribution for y

Sort the M values yr for Y in non-decreasing order.

•Discrete (G):

Take G as the set Mry r ,...,1 ,

•Sufficient for most applications




Representations of the probability

distribution for y

In the form of a piecewise-linear function, suitably obtained from G

(details in the Supplement).

Useful, e.g. for further samplings

•Continuous: YG~




Representations of the pdf for y

•Assemble the (M) yr values into a histogram with suitable cell widths

(subjective!) and normalize to one.

Useful for visual inspection of the pdf, or

when the sorting time (M large and simple model) is excessive




Coverage interval(s)

•The novelty with respect to the GUM is that, since the pdf is usually

asymmetric, there is more than one coverage interval (for a given coverage

probability p)

Shortest (contains the mode)

Probabilistically symmetric 21 p

•Endpoints quantiles p and




Comparison of the two approaches

• The Monte Carlo approach works in a broader class of problems than the GUM approach. In this sense, it is more general, therefore

• It can be used to validate the results provided by the GUM uncertainty framework, however

• It is based on the same principles underlying the GUM

• Descends naturally from the GUM

• It is to be used in conjunction with the GUM







Red dotted: GUM

Solid blue: S1



(continued)

• The best estimate (as the expectation of the numerical approximation

for the output distribution) does not necessarily coincide with that

provided by the GUM.

• Also the standard uncertainties do not coincide. The GUM value may be

smaller. This is a consequence of the pdf recommended for sampled data

(Student).

• The main output is a coverage interval, not the standard uncertainty




• There is no longer any need for degrees of freedom.

• No longer uncertainty of the uncertainty!

Type A and B do not apply to pdfs (luckily…)

• Actually, the approach of Supplement 1 is intrinsically Bayesian.


(continued)




Internal (in)consistency of the GUM

Poor, if an interval of confidence is required

Acceptable, as far as the issue is standard uncertainty

presence of two different views of probability, and

frequentist choice concerning expanded uncertainty

Reason(s):




(In)consistency between the GUM and

Supplement 1 (its first creature…)

Resulting measurand values and associated standard uncertainties are different, but also

standard uncertainty

is uncertain with the GUM,

has no uncertainty with Supplement1




Remedy

(Mildly) revise the GUM, by

Adopting a Bayesian evaluation of standard uncertainty also for

sampled data (Type A), for example

GUM1GUM2

3

1ii xu

n

nxu




To go back to reality…

If your experiment needs statistics, you ought to

perform a better experiment

(Lord Rutherford)


and Physical Constants" Varenna 17 - 27 July 2012 66


Or, if you prefer…

the only statistics you can trust are those you

falsified yourself

(Churchill)

67


End of part I



Documents

Measurement Uncertainty Part I - Fundamentalsstatic.sif.it/SIF/resources/public/files/va2012/bich_i.pdfOrdinal quantities ordinal quantity quantity, defined by a conventional measurement