Roberti: NEON's approach to uncertainty estimation for sensor-based measurements

© 2013 National Ecological Observatory Network, Inc. ALL RIGHTS RESERVED.

NEON’S APPROACH TO UNCERTAINTY

ESTIMATION FOR SENSOR-BASED

MEASUREMENTS

Joshua A Roberti

Jeffrey R Taylor

Henry W Loescher

Janae L Csavina

Derek E Smith

5 August 2013 Ecological Society of America 98th Annual Meeting


Why?

• An example: Temperature change (2013 - 2042)

• It is crucial that uncertainties are identified and quantified to

determine statistical interpretations about mean quantity and

variance structure; both are important when constructing higher level

data products and modeled processes.


Why?






Why?






Why?






Why?






Why?






Why?






Standardized – Traceable – Transparent


• Following the Joint Committee for Guides in Metrology’s (JCGM

100:2008) Guide to the Expression of uncertainty in measurement

(GUM). This is an updated version of the International Organization

for Standardization’s (ISO 1995) GUM







“The evaluation of uncertainty is neither a routine task nor a

purely mathematical one; it depends on detailed knowledge of the

nature of the measurand and of the measurement method and

procedure used. The quality and utility of the uncertainty quoted

for the result of a measurement therefore ultimately depends on

the understanding, critical analysis, and integrity of those who

contribute to the assignment of its value.” (Eurachem-Citac 2000)







“The evaluation of uncertainty is neither a routine task nor a

purely mathematical one; it depends on detailed knowledge of the

nature of the measurand and of the measurement method and

procedure used. The quality and utility of the uncertainty quoted

for the result of a measurement therefore ultimately depends on

the understanding, critical analysis, and integrity of those who

contribute to the assignment of its value.” (Eurachem-Citac 2000)

• Algorithm Theoretical Basis Documents (ATBD)

• Theory of measurement

• Equations (converting from raw, uncalibrated data)

• QA/QC; temporal averaging

• Uncertainty estimates



Data & Uncertainty Flow Example: temperature

CALIBRATION

Standards/Procedures

AD[08,10,14,15]

Field measurement

ASPIRATION

HEATER

L1 DP:TEMPERATURE

± combined uncertainty

Equations:1: Ωi to ○Ci

2: Averaging

DAS

Calibrated Field PRT

Bridge Voltage

Bridge Resistance

Current SupplyNoise

Field PRT

Bridge Voltage

Bridge Resistance

Current Supply

Fig 1. Diagram outlining the data flow and

potential sources of uncertainty associated with

air temperature data



CALIBRATION


AD[08,10,14,15]

Field measurement

ASPIRATION

HEATER

L1 DP:TEMPERATURE



2: Averaging

DAS


Bridge Voltage

Bridge Resistance

Current SupplyNoise

Field PRT

Bridge Voltage

Bridge Resistance

Current Supply

Uncertainties associated with PRTs and

their calibration processes propagate into

a combined uncertainty. This combined

uncertainty represents

i) the variation of an individual sensor

from the mean of a sensor

population,

ii) uncertainty of the calibration

procedures and

iii) uncertainty of coefficients used to

convert resistance to calibrated

station temperature






CALIBRATION


AD[08,10,14,15]

Field measurement

ASPIRATION

HEATER

L1 DP:TEMPERATURE



2: Averaging

DAS


Bridge Voltage

Bridge Resistance

Current SupplyNoise

Field PRT

Bridge Voltage

Bridge Resistance

Current Supply

• Air temperature measured with the

aid of an aspirated shield are more

accurate than those made with a

naturally ventilated (passive) shield

(World Meteorological Organization

2006)

• wind + insolation = error






CALIBRATION


AD[08,10,14,15]

Field measurement

ASPIRATION

HEATER

L1 DP:TEMPERATURE



2: Averaging

DAS


Bridge Voltage

Bridge Resistance

Current SupplyNoise

Field PRT

Bridge Voltage

Bridge Resistance

Current Supply

• Any measurements recorded

during times of heating, and for

a specified time after the heater

is turned off, will be flagged.






CALIBRATION


AD[08,10,14,15]

Field measurement

ASPIRATION

HEATER

L1 DP:TEMPERATURE



2: Averaging

DAS


Bridge Voltage

Bridge Resistance

Current SupplyNoise

Field PRT

Bridge Voltage

Bridge Resistance

Current Supply






The resulting value is multiplied by the partial derivative of the L1 DP. Since the DP

is a temporal average, the partial derivative with respect to an individual

measurement is simply:

Where n represents the number of valid observations made during the averaging

period. The absolute value of Eq. (2) is then multiplied by Eq. (1):

(1)

(2)

(3)

Finally, the combined uncertainty of the L1 mean DP is calculated via quadrature:

(4)



The resulting value is multiplied by the partial derivative of the L1 DP. Since the DP

is a temporal average, the partial derivative with respect to an individual

measurement is simply:

Where n represents the number of valid observations made during the averaging

period. The absolute value of Eq. (2) is then multiplied by Eq. (1):

(1)

(2)

(3)

Finally, the combined uncertainty of the L1 mean DP is calculated via quadrature:

(4)

SIGNAL : NOISE


The Uncertainty Budget

Source of

uncertainty

Standard

uncertainty

component

u(Xi)

Value of

standard

uncertainty

[○C]

Degrees of

Freedom

L1 Temp. DP Eq. (11) -- -- Eq. (13)

1 Hz Temp. Eq. (8) Eq. (9) Eq. (10) Eq. (12)

Sensor/calibration AD[15] 1 AD[15] AD[15]

Noise (DAS) Eq. (4) [Ω] Eq. (5) Eq. (6) AD[15]

Aspiration Eq. (7) 1 Eq. (7) 100

Table 1: Uncertainty budget for L1 mean temperature DPs. Shading denotes the order of uncertainty propagation

(from lightest to darkest).


Further Propagation…

Pressure corrected to Sea Level

• Air temperature data are used in the following equation:

• Partial derivative with respect to temperature is:

• Things get a bit messy…. May be better suited to solve with a Monte Carlo

Method (JCGM 101:2008)

(1)

(2)

(3)

• And associated uncertainty propagates to the following equation:


Identifying (and quantifying?)

Precipitation: Tipping buckets (also for throughfall)

• Evaporative losses

• Undercatchment

• Splash-out

• Wind

• Wetting

• Representativeness

Fine-root turnover: Minirhizotrons

• Proper quantification

• Sampling frequency

• Resolution of the sensor

• Representativeness

Fig 2. wind flow as a function of rain gauge size (Sevruk

and Nespor 1994)

Fig 3. Minirhizotrons at NEON

headquarters


The National Ecological Observatory Network is a project sponsored by the

National Science Foundation and managed under cooperative agreement by

NEON Inc.

Contact: [email protected]

• Traceable, standardized approach by which measurement

uncertainties can be quantified.

– Transparency!

• It is our hope that current and future ecological networks

will adapt this method, thereby strengthening ecological

datasets while promoting interoperability.

TAKE HOME:

ReferencesEurachem-Citac (2000) Quantifying uncertainty in analytical

measurement. Technical Report. Second Edition

Joint Committee for Guides in Metrology (JCGM) (100:2008) Evaluation of

measurement data – Guide to the expression of uncertainty in

measurement.

JCGM (101:2008) Evaluation of measurement data – Supplement 1 to the

“Guide of uncertainty in measurement” – Propagation of

distributions using a monte carlo method

International Organization for Standardization (ISO) (1995) Guide to the

expression of uncertainty in measurement.

Sevruk B. and Zahlavova L. (1994) Classification system of precipitation

gauge site exposure: Evaluation and application. International

Journal of Climatology, 14, pp. 681 – 689.

World Meteorological Organization (WMO) (2006) Guide to meteorological

instruments and methods of observation: Measurement of

Temperature. WMO-No. 8.

Technology

Roberti: NEON's approach to uncertainty estimation for sensor-based measurements