Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1
SCL Simulator version 1.0 (QS 1684)
Software Manual
1 Introduction
“Guide to the expression of uncertainty in measurement” (GUM) [1] defines the GUM
Uncertainty Framework (GUF) for the evaluation of uncertainties by applying the law of
propagation of uncertainty (LPU). However, there are limitations in the application of GUF.
The Supplement 1 to the GUM (GUM-S1)[2] introduces a Monte Carlo method (MCM) to
implement the propagation of the distribution, and also provides a validation method for the
GUF.
The Standards and Calibration Laboratory (SCL) has developed a simulator software (the SCL
simulator) for evaluation of measurement models in accordance with Supplement 1. With the
SCL simulator, users need only to encode the measurement model using the Visual Basic for
Application (VBA) programming language included in Microsoft Excel and to specify relevant
parameters of the uncertainty components (also called input quantities), such as estimates,
standard uncertainties and probability distribution function (PDF), in table form on an Excel
worksheet.
2 Hardware and Software Requirements
For the tool to perform uncertainty calculation, the following hardware and software are
required:
2.1 Hardware Requirements
Personal computer with Intel CPU
2.2 Software Requirements
(1) MS Windows 7 or later version (32-bit or 64-bit version)
(2) MS Excel 2013 or 2016 version (32-bit or 64-bit version)
2
3. Simulator software and the libraries
The SCL simulator, written in Visual C++ and VBA, is tightly integrated with Microsoft Excel
which serves as front-end user interface. It consists of the following 3 files.
a). scl_simulator_vx.x.xls : The SCL simulator in excel format (x.x is the version number),
written in VBA
b). simulator_64.dll : Dynamic Link Library (DLL) for 64 bit Excel
c). simulator2.dll : DLL for 32 bit Excel
Computational intensive routines that require faster execution speed were developed in Visual
C++ and compiled into Dynamic Link Libraries (DLL) for MS Excel in both 32 bit version and
64 bit version. User should copy all the 3 files into the same folder for execution. The software
will detect the version of Excel being used and load the correct DLL for the simulation. No
installation is required.
4. Enabling the Developer Tab and the Macro in Excel
4.1 Enabling the Developer Tab
Microsoft Excel includes a VBA editor, allowing users to employ a subset of the VBA
programming language to develop software routines to enhance and customize the features of
Excel. The SCL simulator makes use of this built-in VBA programming language to encode the
measurement model and the execution of the simulation. The VBA editor is available in the
Developer tab in Excel.
The Developer tab of MS Excel 2016 can be enabled through the following ways:
a). On the File tab, click the Options button. Choose ‘Customize the Ribbon’ in the popup menu
and check the Developer tab in the right column.
3
Fig. 4-1 Developer tab in ‘Customize the Ribbon’ popup menu
b). Enable the Developer through the Enable content button
Fig. 4-2 Enabling the developer
c). Or using the keyboard shortcut key “ALT+F11”.
4.2 Enabling the Macro
To enable the Macro, click the Macro Settings at the Develop tap (Fig. 4-3), enable the Macros
and select ‘Trust access to the VBA project object model’ at the Trust centre (Fig. 4-4).
Fig. 4-3 Macro settings (Develop tap)
4
Fig. 4-4 Macro settings (Trust Centre)
5. Worksheet in the Simulator
The SCL simulator consists of 4 worksheets namely ‘MODEL’, ‘Sheet1’, ‘Sheet2’ and
‘ Sheet3’. The functions of the worksheets are listed below:
Worksheet Function
Model Input of the measurement model
Sheet1 1). Definition of the probability distribution function (PDF)
of the Input quantities
2). Setting up the MCM simulation mode
3). Reporting the MCM simulation results
Sheet2 Storage of the output distribution results
Sheet3 Storage of the output correlation results
6. Setting up the measurement model
User can enter the measurement model starting from row 22 at the ‘MODEL’ worksheet as
shown below.
5
Measurement model
coding area (from row 22)
Model input button
Fig. 6-1. ‘MODEL’ worksheet for the input of measurement model
The input quantities of the measurement model are represented by an array x() with a maximum
of 100 elements, x(1) to x(100). The output quantities are returned in an array y( ). User should
specify the number of output in y(0) and define the measurement models of the output quantities
from y(1) to y(100).
For example, in GUM-S1 example 9.2, the model is
Y1 = X1 + X2 + X3 + X4 - Eq. (1)
There is only 1 output quantity, so y(0) = 1. The output y(1) is modelled by adding the four
input quantities x(1), x(2), x(3) and x(4) as shown in Fig. 6-1.
The following example is an additive measurement model with two output quantities as
described in example 9.2.4 in GUM Supplement 2.
Y1 = X1 + X3 - Eq. (2)
Y2 = X2 + X3 - Eq. (3)
In the SCL simulator, this measurement model is encoded as below
y(0) = 2 y(1) = x(1) + x(3) y(2) = x(2) + x(3)
After encoding the output quantities and the models, press the ‘Model Input’ Button ( )
6
and select the column of codes as shown in Fig. 6-2. Press ‘ok’ after the selection to load the
model(s) into the simulator module.
Fig. 6-2. Loading the model to the simulator
Note: To use this model input function, user should enable the Microsoft Visual Basic For
Applications Extensibility 5.3 in the VBA editor by selecting Tools menu in the VBA editor,
choose References and enable Microsoft Visual Basic For Applications Extensibility 5.3 as
shown in Fig. 6-3.
Fig. 6-3. Enable Microsoft Visual Basic For Applications Extensibility 5.3
7
7. Setting up the input quantities and the PDF
Worksheet ‘Sheet1’ is the main operation page of the SCL simulator. There are 3 areas for
user input and selection as shown in Fig. 7-1.
(a) Parameters of the input quantities
(c) Selection of the Simulation Mode
(b) Correlation of the input parameters
Fig. 7-1. SCL Simulator user input areas: (a) Parameters of the input quantities, (b) Correlation
of the input parameters and (c) Selection of the simulation mode.
7.1 Parameters of the input quantities
The SCL simulator supports a maximum of 100 input quantities. User can specify the
parameters of the input quantities in a table on the SCL simulator (see input area (a) in Fig. 7-
1). An example is shown in Fig. 7-2. The "PDF" column is used to specify the PDF for the
input quantities. The following 8 types of PDF are supported: CTP - curvilinear trapezoid; E -
exponential; G - Gaussian; R - rectangular; TP - trapezoidal; TR - triangular; T - student-t; U -
8
arc sine. The other columns are used for entering the expectation, standard deviation, semi-
range or other parameters relevant to the particular type of PDF. Details are given in Table 7-1.
The histograms generated by the SCL simulator for x1 to x8 in Fig. 7-2 are shown in Figure
Fig. 7-3.
PDF* valueG, T : std.devR,U,TR : semi-rangeCTP, TP : a
T : DOFCTP, TP : b
TPCTP : d
X1 G 0.000E+00 1.000E+00
X2 T 0.000E+00 1.000E+00 5
X3 R 0.000E+00 1.000E+00
X4 U 0.000E+00 1.000E+00
X5 TR 0.000E+00 1.000E+00
X6 CTP -1.000E+00 1 0.3
X7 TP -1.000E+00 1 0.5
X8 E 1.000E+00
Fig. 7-2. Examples of specifying input quantities.
PDF column 1 column 2 column 3 column 4 Remark
G : Gaussian best estimate standard uncertainty N/A N/A
T : student-t average standard deviation degrees of
freedom
N/A
R : rectangular (upper limit +
lower limit)/2
semi-range =
(upper limit - lower
limit)/2
N/A N/A
U : arc sinc (upper limit +
lower limit)/2
semi-range =
(upper limit - lower
limit)/2
N/A N/A
TR : triangular (upper limit +
lower limit)/2
semi-range =
(upper limit - lower
limit)/2
N/A N/A
CTP : curvilinear
trapezoid
N/A a b d rectangular distribution
with lower limit of a d
and upper limit of b d
TP : trapezoid N/A a b refer to GUM
supplement 1 section
6.4.4
E :exponential best estimate N/A N/A N/A
Table 7-1. Types of PDF supported and their parameters
9
(a) x1 : Gaussian (b) x2 : Student-t
(c) x3 : Rectangular (d) x4 : Arc sine
(e) x5 : Triangular (f) x6 : Curvilinear trapezoid
(g) x7 : trapezoid (h) x8 : exponential
-4 -2 0 2 4 -6 -4 -2 0 2 4 6
-1.5 -1 -0.5 0 0.5 1 1.5 -1.5 -1 -0.5 0 0.5 1 1.5
-1.5 -1 -0.5 0 0.5 1 1.5 -1.5 -1 -0.5 0 0.5 1 1.5
-1.5 -1 -0.5 0 0.5 1 1.5 -2 0 2 4 6
Figure 7-3. PDF of x1 to x8 in Table 7-1 generated by the SCL simulator.
7.2 Correlation of the input quantities
The SCL simulator supports the generation of up to a maximum of 10 input quantities with
multivariate Gaussian distribution.
In reference [2] section 6.4.8.1, the uncertainty matrix of N input quantities is defined as
, … ,,
…
, Eq. (4)
⋮ ⋮ ⋱ ⋮ , , …
10
with u(xi, xj) = u(xj, xi) = u(xi) u(xj) r(xj, xi) and r(xj, xi) is the correlation coefficient between
the two input quantities xi and xj [1]. Eq. (4) can further simply as
… 1 , … , … , 1 … , Eq. (5) ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋱ ⋮ … , , … 1
with the second matrix on the right represents the correlation coefficient matrix of the input
quantities.
In the SCL simulator, the correlation coefficients can be entered in a table at input area (b) in
Fig. 7-1. Table 7-2 shows a correlation table for 4 input quantities x1 to x4. The lower triangular
matrix of the correlation coefficient matrix in Eq. 5 is a mirror image of the upper triangular
matrix about the diagonal, therefore the user is only required to input the upper triangular
matrix of these coefficients and the program will generate the lower triangular matrix (the
darkened cells) for computation.
x1 x2 x3 x4
x1 1
Correlation Coefficient
for x1 and x2
Correlation Coefficient
for x1 and x3
Correlation Coefficient
for x1 and x4
x2 Correlation Coefficient
for x1 and x2 1
Correlation Coefficient
for x2 and x3
Correlation Coefficient
for x2 and x4
x3 Correlation Coefficient
for x1 and x3
Correlation Coefficient
for x2 and x3 1
Correlation Coefficient
for x3 and x4
x4 Correlation Coefficient
for x1 and x4
Correlation Coefficient
for x2 and x4
Correlation Coefficient
for x3 and x4 1
Table 7-2 Correlation Table for 4 input quantities. The darkened cells are copied from the
upper triangular matrix by the program.
Fig. 7-4 shows a zoom-in view with the input parameters and the correlation table putting side-
by-side. The correlation coefficient between x3 and x4 is 0.9 and the other pairs of quantities
are treated as uncorrelated with zero correlation coefficient (the empty cells refer).
11
Fig. 7-4. Correlation coefficient between x3 and x4 is set as 0.9 in the correlation table.
7.3 Setting of the Simulation Mode
GUM-S1 describes a MCM for the propagation of distributions. The idea is to make large
number of draws from the PDF of the input quantities and to derive the output quantities for
each draw. There are two ways to select the number of Monte Carlo trials: (a) a fixed number
of trials defined by the user and (b) an adaptive algorithm. An adaptive algorithm will determine
the number of trials on-the-fly based on the dispersion of the simulated output. The user can
specify the precision of the simulation by selecting the number of significant digits ndig [2] for
the output quantities.
User can select the MCM simulation mode in input area (c) in Fig. 7-1. A zoom-in view of
the fixed sample size setting and adaptive setting are shown in Fig.7-5. For the “fixed sample
size” mode, user can specify the number of MCM trials in the “Sample Size” field. The
allowable range of trials is from 20,000 to 3,000,000. For the “adaptive” mode, the user can
specify the number of significant digits (1 or 2) required for the output quantities. User can also
configure the “Confidence Interval Type” to be “Symmetrical” or “Shortest”. Finally, the
“Coverage Probability” of the confidence interval is selectable between 95% and 99 %.
(a) (b)
Fig. 7-5. (a) Fixed sample size mode with 1,000,000 MCM trials selected ; (b) Adaptive
mode selected with output quantities having 2 significant digits.
12
After the completion of the setup procedure in Section 6 and 7, click the ‘Start Simulation’
button to run the simulation program. The simulation time and the total number of trial are
shown in the same area in Fig. 7-5.
8. Reporting the results
Fig. 8-1 shows the display areas of the MCM simulation results in ‘Sheet1’.
(a) Results of the MCM
(b) Histogram of the PDF for y(1) to y(10)
(c) Correlation of the output
quantities
(d) Contour map
Fig. 8-1 SCL Simulator output areas: (a) Results of the MCM, (b) Histogram of the PDF for
y(1) to y(10), (c) Correlation of the output quantities and (d) Contour map.
8.1 MCM result table and GUF validation
After MCM trials, the estimates, standard uncertainties and the confidence intervals (95 % or
99 %) of the output quantities y(i) (i = 1 to 100) are shown in area (a) in Fig. 8-1. The results
of GUM-S1 Example 9.2.2 are shown in Fig. 8-2.
Fig. 8-2. Result table of GUM-S1 Example 9.2.2
The MCM results are compared with the results obtained from GUM uncertainty framework
(GUF). In adaptive simulation mode with assigned number of significant digits ndig, the
software will check the validity of the GUF according to Section 8.1.3 of GUM-S1[2]. If the
absolute differences of the respective endpoints of the two coverage intervals obtained by the
MCM method and the GUF are within the numerical tolerance, the GUF is validated and the
last column of the result table will show as ‘Yes’.
13
8.2 Visualization of the PDF for y(i)
Area (b) in Fig. 8-1 displays the histogram of the PDF of the first 10 output quantities. An
example of the plot is shown in Fig. 8-3 with the blue line showing the PDF obtained by MCM
and the pink line represents the results by GUF.
Fig. 8-3 Histogram of the PDF for the output quantity y(1).
8.3 Correlated output quantities
Section 8.1 and Section 8.2 demonstrate the capabilities of the SCL Simulator for MCM
computation and GUF validation according to GUM-S1 [2]. In addition, the SCL simulator
also calculates the correlations of the output quantities to provide further insight of the results
to the user (in Area (c) and Area (d) of Fig. 8-1). Fig. 8-4 shows the correlation coefficient table
of the output quantities y(1) and y(2) of Eq. (2) and Eq. (3). The correlation coefficient between
y(1) and y(2) is 0.9. The correlation of y(1) and y(2) can be visualized through the contour map
as shown in Fig. 8-5.
Fig. 8-4 Correlation coefficient table of the output quantities
14
Fig. 8-5 Contour map of the joint PDF for the first two output quantities.
9. Practical example from GUM-S1
This example from section 9.2 of Supplement 1 is used to explain the procedure for validation
of GUF using the SCL simulator. It is an additive model described by Eq. (1) in section 6 of
this manual, where
Y1 = X1 + X2 + X3 + X4
Three cases are considered. In case (i) all input quantities X1 to X4 have Gaussian PDF with
standard uncertainty of 1. In case (ii) X1 to X4 have rectangular PDF with standard uncertainty
of 1. Case (iii) is similar to case (ii) except X4 has standard uncertainty of 10.
Step 1 – Setting up the models
The measurement models for Case (i) to (iii) are encoded as below, with y(1) for case (i), y(2)
for case (ii) and y(3) for case (iii). The total number of output quantities y(0) is 3.
y(0) = 3
y(1) = x(1) + x(2) + x(3) + x(4) 'GUM-S1 Section 9.2.2 case (i)
y(2) = x(5) + x(6) + x(7) + x(8) 'GUM-S1 Section 9.2.3 case (ii)
y(3) = x(9) + x(10) + x(11) + x(12) 'GUM-S1 Section 9.2.4 case (iii)
15
Fig. 9-1. Step 1 setting up the models for case (i) to (iii)
Step 2 – Setting up the input quantities
The parameters for the input quantities are entered into the SCL simulator are shown in Figure
9-2.
Fig. 9-2. Step 2 setting up the input quantities
16
Step 3 – Setting up the simulation mode
The “adaptive” simulation mode with 2 significant digits are selected as shown in Fig. 9-3,
with symmetrical confidence interval of 95 % coverage.
Fig. 9-3. Step 2 setting up the simulation model
Step 4 – Results
After the SCL simulator has executed 1.13 million trials, the results are shown in Fig. 9.-4.
Case (i) and case (ii) are validated but not case (iii). The histogram of the 3 outputs are shown
in Fig. 9-5. It is easy to see why case (iii) is not validated.
Fig. 9-4 Results of the validation of GUF using MCM.
17
(a) y1 for case (i) (b) y2 for case (ii) (c) y3 for case (iii)
-10 -5 0 5 10 -10 -5 0 5 10 -40 -20 0 20 40
Fig. 9-5 Histogram of the 3 outputs
10 Reference
1. Evaluation of measurement data — Guide to the expression of uncertainty in measurement,
JCGM 100:2008
2. Evaluation of measurement data — Supplement 1 to the“Guide to the expression of uncertainty
in measurement” — Propagation of distributions using a Monte Carlo method, JCGM 101:2008
First Edition
3. Evaluation of measurement data — Supplement 2 to the “Guide to the expression of uncertainty
in measurement” – Extension to any number of output quantities, JCGM 102:2011