Celya Summer School ‘Atmospheric sound propagation’ 14 ......• Standard (ISO/CEI 61672 for ex.) • Technical spec., publications Specific value • Estimated by the user CFA

Uncertainties and variabilityCelya Summer School ‘Atmospheric sound propagation’

14-06-2018

David EcotièreUMRAE

o A start with a few truisms (… but not always considered as such)

It is impossible to measure (or predict) EXACTLY a physical quantity (evenif you are Nobel prize in metrology)

A measurement result given without its associated uncertainty should make no sense

Several dB uncertainties are common in environmental acoustics (depending on the quantity observed)

Introduction

Celya Summer School – Uncertainties and variability 14/06/20182

I’m 1.6m tall I’m 1m tall Who is right ?

Both !

o Uncertainties : where are they come from?

Epistemic uncertainties : unknown (or bad known) parameters

Model limitations, measuring instrument uncertainties, methodologyuncertainty …

‘Natural’ variability of the observed quantity, sampling uncertainty

Introduction


o Uncertainties : what is it ?

Measurement (or prediction) -> measurement (or prediction) error

Introduction


Referencevalue

Measureresult

Measurand value

Systematic random

Error

o Uncertainties: what do I do with them? what do I need it for?

Uncertainty information = information on the quality of its measurement/calculation against a defined requirement

Comparison to a threshold

Introduction


Threshold

dB

Estimation

Estimation

dB

dB

dB

I’m 1.6m tall I’m 1m tall

Who will you trust in ?

o Several kinds of uncertainties

Accuracy

• e.g. due to the instrument resolution

Dispersion

Systematic error (trueness)

Introduction


-> Uncertainty is a combination of all of these

o Roadmap

Define precisely the quantity of interest

Investigate the measurement process to identify the possible causes of error (if possible)

Choose a method for estimating standard uncertainties :• Depending on the knowledge we have of the measurement process

• Depending on the possibility to model the process

• Depending on whether the measurement process can be repeated easily or not

Calculate the expanded uncertainty and a confidence interval

Methodology


Methodology


[Désenfant, Priel, A road map for uncertainty estimation approaches, 2006]

o Bias, expanded standard uncertainty, confidence interval

Result:

𝐿𝑐 = 𝐿𝑚𝑒𝑠 − 𝑏

in the x% confidence interval: [𝐿𝐶 − 𝑈min ;𝐿𝐶 + 𝑈max ]

Methodology


k=2 : IC 95%

- b : bias (systematic error)- Umin et Umax : expanded standard uncertainty

-- u : standard uncertainty (standard deviation)

utu

utu

maxmax

minmin

Referencevalue

Measureresult

MeasurandvalueSystematic rand

omError

o GUM method (ISO/CEI Guide 98-3) – analytical approach(errors propagation law)

An analytical model is required:

• Estimations:

• Combined uncertainty:

• If influence quantities Xi are not correlated:

Methods : GUM


1

1

,

11

2

2

2 2N

i

xx

j

N

ij i

N

i

x

ijii x

f

x

f

x

fu

),...,,( 21 NXXXfY

N

i

x

iix

fu

1

2

2

2

),...,,( 21 Nxxxfy

o In France : new standard XP S 31-115 (GUM compliant)

o General process

a) identification of influence quantitiesb) bias induced by each influence quantityc) standard uncertainty induced by each influence quantityd) calculation of combined uncertainty and total biase) calculation of the expanded uncertainty and confidence intervalf) presentation of the measurement result and uncertainty

information


Methods : GUM

o a) Main influence quantities

Instrumentation Implementation and environmentalconditions

- Microphone Directivity- Linearity of levels- Frequency weighting and frequency

linearity (A, C, Z)- Air temperature and humidity- Ambient air pressure- Field standard level- Wind screen- Power supply- Proper noise of the instrumentation- Time Weighting (F, S, I)

- Duration of measurement;- Post-treatment ;- Local weather conditions at the

microphone (wind, rain,...) ;- Electromagnetic fields outside the acoustic

measuring device


Methods : GUM

o b) and c) bias and standard uncertainty of each influence quantity

Standard uncertainty estimated by standard deviation

o (d) total bias and combined uncertainty

Total Bias :

-> biases can compensate each other (and the total bias can be null)

Combined uncertainty (decorrelated quantities) : 𝑢𝐶 =

𝑖

𝑢𝑖2

𝑏 =

𝑖

𝑏𝑖

-> standard uncertainties do not compensate each other (the composed standard uncertainty cannot be zero)


Methods : GUM

o e) Expanded uncertainty and confidence level

Expanded uncertainty

k=coverage factor (depend on the confidence level)

Confidence interval

𝑈 = 𝑘. 𝑢𝐶(𝑦

k=2 : IC 95%

Methods : GUM


o Values for standard uncertainties

Standard values (overestimation)

• Standard (ISO/CEI 61672 for ex.)

• Technical spec., publications

Specific value

• Estimated by the user

27/04/2018CFA 2018 - norme XP S 31-115-1 15

Methods : GUM

o Ex : from NF EN 61672-1 (sound level meter spec.) for class I

Influence quantity u (dB)Level linearity error 0.46Supply voltage 0.06Static air pressure 85 kPa à 108 kPa 0.23

65 kPa à 85 kPa 0.52Air temperature -10 à + 50°C 0.28

0 à + 40°C -Air humidity 20% à 90% 0.28Electromagnetic fields 0.57Calibrator 0.17Wind screen 63 Hz à 2 kHz 0.28

2 kHz à 8 kHz 0.46Time weighting F and S 0.06Leq measurement 0.06Rounding of the measurement 0.03Adjustment 0.01Measurement of C-weighted peak sound pressure levels

1.15


Methods : GUM

o Ex : from NF EN 61672-1 (sound level meter spec.)

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

10 100 1000 10000

1/3 octave (Hz)

u (

dB

)

Classe 1

Classe 2

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

10 100 1000 10000

f (Hz)

u (

dB

)

30° - Classe 1

30° - Classe 2

90° - Classe 1

90° - Classe 2

150° - Classe 1

150° - Classe 2

Microphone directivity‘A’ frequency weighting


Methods : GUM

o Wet wind screen [Ribeiro, Ecotière et al, 2014] [XP S 31-115]

Correction C j

(protections mouillées)

-0,8

-0,6

-0,4

-0,2

0

0,2

0,4

0,6

10 100 1000 10000 100000

Fréquence (Hz)

dB

Modèle 1

Modèle 2

Incertitude-type u Cj

(protections mouillées)

0

0,2

0,4

0,6

0,8

1

1,2

10 100 1000 10000 100000

Fréquence (Hz)

dB

Modèle 1

Modèle 2


Methods : GUM


o Wind induced noise [Ecotière, Rumeau et al. 2012, 2018] [XP S 31-115]

Bias :

Standard uncertainty :

54 56 58 60 62 64

01

23

45

6

Biais

Lmes (dBA)

dB

A

54 56 58 60 62 64

01

23

4incertitude-type

Lmes (dBA)

dB

A

Ex : wind speed 5/m/s, microphone height 1.5m.

Methods : GUM

𝑏𝑤𝑖𝑑 = 𝑎𝑏,1 +𝑎𝑏,2

𝐿𝑚𝑒𝑠 − 𝐿0

𝑢𝑤𝑖𝑛𝑑 = 𝑎𝑢,1 +𝑎𝑢,2


2

+ 𝑢𝑤𝑖𝑛𝑑,𝑚𝑜𝑑2

o GUM method (ISO/CEI Guide 98-3) – stochastic approach(distributions propagation, Monte Carlo)

Methods : GUM


Y=f(x1,x2,x3)

x1

x2

x3

n samples

n calculations

Y

IC95%

Y

Ymin Ymax

YY Result :

within the 95% confidence interval =[Ymin, Ymax]

n samples

n samples

o An example of Monte Carlo application : uncertainties of windinduced noise in a screened microphone

Methods : GUM


o An example of Monte Carlo application : uncertainties of windinduced noise in a screened microphone

Methods : GUM


8,74)()(

/log7,26/log4,69 00,

CzF

lDVVL Aat

)./215.0log(20)(

,)/ln(/log20)(

),3/(

0

2

0

0

3/1

pVC

zzDzzF

DVf c

Input parameters :

Wind screen: zD ,,

HTzV ,,, 00Environment:

[van den Berg, 2006]

Bias and bias uncertainty estimation by Monte Carlo technique

Methods : GUM


Lsource

105 samples

80dB20dB

105 calculations for each wind class

V

Vi Vi+1

T0

-20oC 40oC

H

700W/m2-150W/m2

105 samples

105 samples

105 samples

10/10/1010log10 sourcewind LL

mesL

messource LLC

windL : van den Berg model

Monte Carlo method

[Ecotière,2012, 2018]

Methods : GUM

V=5 m/s

V=6 m/s


Bias and bias uncertainty

Methods : GUM


negligible biasBig uncertainty[Ecotière,2012, 2018]

Bias and standard uncertainty for wind induced noise

Methods : GUM


[Ecotière,2018]

𝑏𝑤𝑖𝑑 = 𝑎𝑏,1 +𝑎𝑏,2


𝑢𝑤𝑖𝑛𝑑 = 𝑎𝑢,1 +𝑎𝑢,2


2

+ 𝑢𝑤𝑖𝑛𝑑,𝑚𝑜𝑑2

o Interlaboratory approach (CEI Guide 98-3) – approach

Requires numerous experimental trials with several labs

Based on a statistical method of analysis of variance (ANOVA)

Allows the estimation of 2 uncertainties :

• Repeatability: 𝑢𝑟

• Reproducibility: 𝑢𝑅

• Final Uncertainty: u2 = ur2 + 𝑢𝑅

2

Possibility to calculate intermediate precision (repeat. per lab, per operator, etc.)

Methods : ISO 5725


Methods : ISO 5725

1 2 3 4 5

Laboratories

Deviation / ref

Repeatability L1

Reproducibility

Repeatability L3

Ex of interlab. approach

o An example of ISO 5725 application : uncertainties of an in situ method for measuring ground acoustic impedance

A 2 mic. Method [Hess,1990] [Sabatier,1990] [Sabatier,1993][Guillaume,2015]

Methods : ISO 5725


2

2

110log10

p

pL

HPMicrophone 1

Microphone 2

Rd1

Rd2

Rr1 Rr2Z

- Measurement of the spectrum transfer function between 2 points

- Estimation of the Z parameters by fitting calc/measured of ΔL

The round robin test : protocol

Methods : ISO 5725


-5 laboratories, whith the same kind of exp. device-3 repetitions / point

-5 outdoor grounds-3 points / ground

S *

R *

dSR

S *

R *

dSR

S *

R *

dSR

Ground i

Point 1

5m mini 5m mini

Point 2 Point 3x 3 x 3 x 3

Lab j

[Ecotière, Glé et al 2015]

The round robin test : ground samples

Methods : ISO 5725


Synthetic lawn

Natural lawn

Natural grass

Compacted ground

Non porous asphalt

Reprod. and repeat. vs ground type

Methods : ISO 5725


Asphalt

Compacted

Lawn

Grass

Synth_lawn

10 50 100 500 5000 50000

Air flow resistivity

kNsm-4

Air flow resistivity (kNsm-4) Mean sR (reprod.) Sr (repeat.) sL (interlab.)

Non porous Asphalt 19 609 15 070 13 065 7 510

Compacted ground 3 519 567 512 245

Natural lawn 1 154 248 217 119

Grass 277 34 34 6

Synthetic lawn 51 7 5 5

Uncertainties on sound levels estimation

Methods : ISO 5725


hs=0.05 m hs=1 m hs=3 m hs=10 m

σ =3 519 1,7 dB 2,1 dB 1,2 dB 1 dB

σ = 1 154 2,1 dB 2,3 dB 1,6 dB 1 dB

σ = 277 2,6 dB 2,9 dB 2,2 dB 1 dB

Maximum type-uncertainty of SPL 1/3rd octave band (under reproducibility conditions)

example for hr=2m, d=1:200m and f=100:5000 Hz

Standard uncertainty (1/3rd oct bands) <3 dB

Uncertainties on sound levels estimation

Methods : ISO 5725


Maximum type-uncertainty of SPL 1/3rd octave band (under repeatability conditions)

example for hr=2m, d=1:200m and f=100:5000 Hz

hs=0.05 m hs=1 m hs=3 m hs=10 m

σ =3 519 0,7 1 0,6 0,6

σ = 1 154 1 1,2 0,8 0,6

σ = 277 1,2 1,3 1,3 0,6

Standard uncertainty (1/3rd oct bands) <1,3 dB

Journée SFA/GABE – Lyon – 13/10/2014

Incertitude élargie (k=2)

Incertitude composée

Linéarité de niveau

Tension d''alimentation

Pression statique

Température

Humidité

Calibreur

Ecran anti-vent

Filtre A

Directivité (30°) Classe 1

Classe 2

Incertitudes - NF CEI 61672-1

dB

0.0 0.5 1.0 1.5 2.0 2.5 3.0

Sound level meter : minimum ± 2dB (class I)± 3dB (class II)


About sound level measurement uncertainties

Ex of vertical sound speed gradient measurement

• Influence of method : sonic vs mast

• Influence of the devices : sonic vs sonic

• Influence of the height meas.


Uncertainties due to meas. processes

Ex of vertical sound speed gradient measurement

• Influence of devices



Uncertainties on sound level estimation due to uncertainties on vertical sound speed gradient measurement



How my measurement result is representative ?

‘Natural’ variability

• Source emission variability

• Meteorological variability

• Ground properties variability

• ...

• Spatial and temporal variability

Sampling uncertainty


Variability

Variability due to meteo


LAeq(15min)

30

35

40

45

50

55

60

65

70

13/06/2005

00:00

14/06/2005

00:00

15/06/2005

00:00

16/06/2005

00:00

17/06/2005

00:00

18/06/2005

00:00

19/06/2005

00:00

20/06/2005

00:00

21/06/2005

00:00

22/06/2005

00:00

23/06/2005

00:00

dB

(A)

d=50m

d=100m

d=200m15dB(A)

50m 100m 200m0m

2m

Lannemezan 2005 (Junker et al, 2006)

)( cfLAeq z

Step 1

Acoustic model (PE)

[Gauvreau, JASA, 2002]

[Lihoreau et al., JASA, 2006]

Site data(geometry, ground …)

+ celerity gradients

LAeq(1h) over x years

Statistical analysis

Step 2

Micrometeo model(Choisnel model based on Monin-Obukhov theory)[Brunet et al., LRSP, 1996]

Meteo data measured hourly over x years

Hourly celerity gradient over x

yearswzzz VTTgc )(


Noise variability due to meteo

o Estimation of variability due to meteo : LTS method[Zouboff, 2000][Ecotière 2008]

2

4

6

8

10

12

14

16

%

0 5 10 15 20

45

50

55

60

65

70

Mean

Mean +/- 1 st. dev.

Point source (300m,300°) - LAeq(1h) - July

Hours of the day (h)

LA

eq(1

h)

(dB

(A),

arb

. re

f.)

Point source (300m,300°)

LAeq(1h) - July.

>20 dB(A)

>8 dB(A)

Daily fluctuations of LAeq(1h) – seasonal influence

2

4

6

8

10

12

14

16

%

0 5 10 15 20

45

50

55

60

65

70

Mean

Mean +/- 1 st. dev.

Point source (300m,300°) - LAeq(1h) - Jan.

Hours of the day (h)

LA

eq(1

h)

(dB

(A),

arb

. re

f.)

>13 dB(A)>6 dB(A)

Point source (300m,300°)

LAeq(1h) - Jan.



[Ecotière 2008]


5

10

15

20

%

Jan. Apr July Oct Dec

50

55

60

65

Mean

Mean +/- 1 st. dev.

Point source (300m,300°) - LAeq(6h22h)

Month

LA

eq(6

h22h)

(dB

(A),

arb

. re

f.)

>10 dB(A)>4 dB(A)


5

10

15

20

%

Jan. Apr July Oct Dec

50

55

60

65

Mean

Mean +/- 1 st. dev.


Month

LA

eq(2

2h6h)

(dB

(A),

arb

. re

f.)

>10 dB(A)>1 dB(A)


[Ecotière 2008]


Seasonal fluctuations of LAeq(day/night)

North

Receiver

2m

0m

0.05m

Di

Si

Application for a road source

Site : Avrillé (49) - meteo data : 1960 to 1990 (>250 000 samples LAeq(1h))

Flat ground, Model of impedance Miky ( =300 kNms-4 )

Road source - Road noise spectrum




Error of estimation of the LAeq(day/night) – Road source (150m, 0°)

0 50 100 150 200 250 300 350

02

46

81

0

Error of estimation of LAeq(6h22h) - Road(150m,0°)

Duration of observation (day)

Err

or

(dB

(A))

12

34

56

78

9

Mean

Range

Error of estimation of LAeq(6h22h)

0 50 100 150 200 250 300 350

05

10

15

Error of estimation of LAeq(22h6h) - Road(150m,0°)


Err

or

(dB

(A))

13

57

91

11

41

7

Mean

Range






- Summer : error>2 dB(A)

- Winter : error>1 dB(A)

0 5 10 15 20 25

02

46

81

0

Error on estimation of LAeq(6h22h) - Road(150m,0°)

Duration of estimation (day)

Err

or

(dB

(A))

12

34

56

78

91

0

Jan.

Apr

July

Oct.





6h22h =50% =90%

50j 1.5 dB(A) 2.7 dB(A)

100j 1.2 dB(A) 2.4 dB(A)

22h6h =50% =90%

50j 1.5 dB(A) 2.5 dB(A)

100j 1 dB(A) 1.8 dB(A)

Maximum error - confidence level

and duration of observation

0 50 100 150 200 250 300 350

02

46

81

01

2

Error of the estimation of LAeq


Err

or

(dB

(A))

50%

90%

LAeq(6h22h)

LAeq(22h6h)


Confidence level - LAeq(day/night) – Road source (150m, 0°)


Level confidence as a function of

the duration of observation (max

error = 1 dB(A))

0 50 100 150 200 250 300 350

20

40

60

80

10

0

Confidence levels for a max. error of 1dB(A)


Co

nfid

en

ce

le

ve

l (%

)

LAeq(6h22h)

LAeq(22h6h)

Minimum number of days to have an error < 1 dB(A) with a confidence level of 50% :

- LAeq(6h22h) : 150 days

- LAeq(22h6h) : 60 days

10/02/2017Séminaire scientifique 49

o Spatial and temporal ground variability

[Junker et al, 2006][Baume 2006]

Spatial variability

Temporal variability

Noise variability due to the ground



o Temporal ground variability



o Spatial ground variability


50

100

150

200

250

300

350

05/0

6/0

5 0

0:0

0

09/0

6/0

5 0

0:0

0

13/0

6/0

5 0

0:0

0

17/0

6/0

5 0

0:0

0

21/0

6/0

5 0

0:0

0

25/0

6/0

5 0

0:0

0

29/0

6/0

5 0

0:0

0

03/0

7/0

5 0

0:0

0

07/0

7/0

5 0

0:0

0

11/0

7/0

5 0

0:0

0

15/0

7/0

5 0

0:0

0

19/0

7/0

5 0

0:0

0

23/0

7/0

5 0

0:0

0

27/0

7/0

5 0

0:0

0

31/0

7/0

5 0

0:0

0

04/0

8/0

5 0

0:0

0

08/0

8/0

5 0

0:0

0

12/0

8/0

5 0

0:0

0

16/0

8/0

5 0

0:0

0

20/0

8/0

5 0

0:0

0

24/0

8/0

5 0

0:0

0

28/0

8/0

5 0

0:0

0

01/0

9/0

5 0

0:0

0Date/heure

Sig

ma

(k

Ns

m-4

)

I3-10 Suivi temporel (EDF)

I3-10 Suivi spatial (LCPC)















Noise variability due to ground

o Spatial ground variability


o Uncertainties: what do I do with them?

Comparison to a threshold


Threshold

dB

Estimation

Estimation

dB

dB

dB

Conclusion

and/or

Lower uncertainty

Lower confidence level

IC 95%, 75% …

X+kucX-kuc X

The real world is uncertain, uncertainties will always exist and you can’tdo without it

Uncertainties of several dB are common in environmental acoustics

Confidence level : uncertainties are a tool for risk management


Conclusions

Thank you for your attention!

o Contact :

[email protected]

55Celya Summer School – Uncertainties and variability 14/06/2018

www.umrae.fr

mailto:[email protected]

Documents

Celya Summer School ‘Atmospheric sound propagation’ 14 ......• Standard (ISO/CEI 61672 for ex.) • Technical spec., publications Specific value • Estimated by the user CFA