Embed Size (px)
Citation preview
SMU-DS-TR-257
A Bayesian Method for Testing TTBT Compliance with Unknown Intercept and Slope
Jangsun Baek Henry L. Gray Gary D. McCartor Wayne A. Woodward
Southern Methodist University Department of Statistical Science Dallas, Texas 75275
September 1992
Scientific Report No. 101 Project Title: "Statistical Research in Nuclear Monitoring" DARPA Contract No. F29601-91-K-DB25
Approved for public release; distribution unlimited
Department of Defense Phillips Laboratory United States Air Force Kirkland AFB, NM 87117-5320
'W0J75 w,
REPORT DOCUMENTATION PAGE I'cnn Approved
OMB No 070'W) i88
-»•,ni'- --»tv i n: o-.'jen tor in i. ,:lii;<!i ;n >f i".T.::.T,it.:n *, .-,[ ~* tt«.} '.o i.-»r i :<? ■ -. i," ipr •■■y.O'rr-,..'. -txair i *r-n V.Tp •■-: -P.IP-.V.I ; -■'<:r'i-:*.:;r<. ,-?Hf - '-■j ; i*--'f'fg md .71 t.Mtimr.} th*1 ajt.l neecoa. jr.a ccrru.etin ; jrc r-?vi-?.vrr^ :ne ^;lle'-'i',n or n'efn it.cn >ena ;;mni<?nti .•■M.irnin.j tris uur 3--n .".tiTi.ite r i
■'»'■M'Cri ■' .^».-<ffr if;.p. .nrli.oin^ iugqeitio'n 'nt r<?auarq this nuraen to // isn<nqton Me-iaaujrTO's v?r»,ic.i?s. Directorate for :nt oifTMlion Ooer morn ma ''eoeav :i.iii":r.'.i/,;u'!!'.'Ci .•.iiTjirt ;J .'2212 -»M2 ma !■: i-'O*' ■>' vnini»»'»«! .i«i 'luowt »soervicf« Äeaucncn c-oi«'.tn)'t4-0'i)i) #v.isn.n.ji..-n : c ."05.
g -3.its vur"^ rr iiuea of :nii .1215 jelterson i
1. AGENCY USE ONLY (Leave ülank) 2. REPORT DATE
September 1992 3. REPORT TYPE AND DATES COVERED
Scientific Report No. 101 4. TITLE AND SUBTITLE
A Bayesian Method for Testing TTBT Compliance with Unknown Intercept and Slope
o. «u i m.m\j/
Jangsun Baek Henry L. Gray Gary D. McCartor
Wayne A. Woodward
5. FUNOING NUMBERS
DARPA Contract F29601-91-K-DB25
1
;. rEnrCA.Vlli'JG OKGAWI2ATICN .MAM£(S) AflO AOCKtSSiiSj
Southern Methodist University Department of Statistical Science Dallas, Texas 75275
REPORT NUMBER
9. SPONSORING/MONITORING AGENCY MAME(S) AND ADDRESS(ES)
Phillips Laboratory Department of the Air Force Kirtland Air Force Base, NM 87117-5320
Contract Manager: Del Patten
10. SPONSORING/MONITORING AGENCY REPORT NUMBER
11. SUPPLEMENTARY NOTES
'.Zl. ulSTmSUTiCN, AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
12b. DiST?l3lJT!O.M CODE
13. A8STRACT (Maximum 200 words)
In this report we examine the Bayesian method for testing for compliance to a given threshold studies by Nicholson, Mensing and Gray. It is noted that although this test and accompanying confidence intervals are valid for single event, it is incorrect to apply it or the confidence intervals to repeated events at the same eite unless the number of calibration events is large. Since in any foreseeable future the number of calibration events is likely to be small, this report studies the applicability of the Bayesian test in this case. The results suggest that in many instances the Bayesian method examined here should be used on repeated events with caution if the number of calibration events is less than three.
14. SUBJECT TERMS testing TTBT compliance, unknown intercept and slope, Bayesian method
15. NUMBER OF PAGES 59 pages
16. PRICE CODE
17. SECURITY CLASSIFICATION OF REPORT Unclassified
18. SECURITY CLASSIFICATION OF THIS PAGE Unclassified
19. SECURITY CLASSIFICATION OF ABSTRACT Unclassified
20. LIMITATION OF ABSTRACT
SAR
NSN 7540-01-280-5500 Standard Form 298 (Rev 2-39) Pr»».:nn«<1 hv -fi".! '.Id <'i").lg
A Bayesian Method for Testing TTBT Compliance
with Unknown Intercept and Slope
Jangsun Baek, Henry L. Gray, Gary D. McCartor and Wayne A. Woodward
Southern Methodist University
Feb. 19, 1992
Abstract
In this report we examine the Bayesian method for testing for compliance
to a given threshold studied by Nicholson, Mensing and Gray. It is noted that
although this test and accompanying confidence intervals are valid for a single
event, it is incorrect to apply it or the confidence intervals to repeated events
at the same site unless the number of calibration events is large. Since in any
foreseeable future the number of calibration events is likely to be small, this
report studies the applicability of the Bayesian test in this case. The results
suggest that in many instances the Bayesian method examined here should be
used on repeated events with caution if the number of calibration events is less
than three.
1 Introduction
Over the last few years much of the interest in yield estimation and threshold
test ban treaty monitoring has shifted to the problem of properly monitoring
yields that are somewhat smaller than the current test ban limit of 150 Kt.
As a result of this interest in smaller yields it has become more important to
include the effects of unknown slope (in the standard magnitude/yield relation)
on estimated yields, associated confidence intervals, and related hypothesis tests.
The most popular approach for addressing this problem thus far has been
through the Baysian methodology. See W. L. Nicholson, R. W. Mensing and H.
L. Gray, or R. H. Shumway and Z. A. Der for example. In each of these papers
the authors make use of prior distributions on the parameter spaces to obtain
estimates of yield, confidence intervals for yield, threshold type test of hypotheses,
and associated F-numbers which allow for errors in estimating geological bias and
slope as well as several other unknown parameters. Although such results are
exactly what was needed in one sense they present a problem in another. That
is, although the confidence intervals and hypothesis tests are valid when related
to a single event from all possible parameter configurations they do not represent
such intervals or hypothesis tests when applied repeatly to a fixed test site (This
will be explained in detail in section 4). This problem was noted by Fisk, Gray,
McCartor and Wilson (1991) for the case where the slope is known.
In this report we examine the current Baysian approach to yield estimation
from several practical aspects. That is we consider:
1. The power of the tests for several different parameter configurations and
yield training sets.
2. The maximum benefit of previous no yield data regarding its contribu-
tion to increasing the power or decreasing the F-number.
3. The actual error rate or confidence interval (CI) that results when these
Baysian tests or CI's are applied to repeated tests at the same site.
Item 3 is of special interest if the number of calibration events is small and the
particular test site is an anomaly, i.e. a site whose parameters differ substantial
2
from their corresponding Bayesian means. We shall refer to our investigation of
item 3 as a robustness study.
2 Notation and Background
Let Yj denote the jth yield at a given test site and let m,j denote the ith
magnitude associated with the jth yield,
mij = A{ + BiW0j + etj (1)
i = l,2,---,p and j = l,2,---,n, where WQJ = \ogYj - logF0 = Wj -
WQ, with Wo given and the e,-j represent random errors. Further let A =
(A\, • • •, Ap), B = (2?i, • • •, Bp), and ey = (ey, e<ij, • • •, epj)' where the prime
denotes transpose, and the e^- are normal random vectors with mean (0,0, • • •, 0)'
and known variance Ee. We can now write (1) in the matrix form
mj=A + BW0j + ej. (2)
In the model defined by Equation (1) A and B are vectors of parameters
that depend on the test site and the particular magnitude being considered. For
example my may refer to the jth. mj value while m^ might be the jth mL
value. Ideally A and B in (2) would be known. This is in general not the case.
However there may be sufficient information regarding A and B to restrict
their possible values. That is, it is arguable that one can reasonably impose
a probability distribution on A and B a priori. This is in fact the reasoning
that leads to a Bayesian approach to the problem. Specifically we suppose that
ß — (A , B ) has a prior normal distribution with known mean fio and covariance
E^. In the future we will denote this by ß ~ N(ftß,Eß). Therefore in Equation
(2) we no longer treat A and B as fixed parameters but as random variables or, if
you like, "parameters" which take on their possible values with some probability.
Now suppose n calibration events are available, ». e. n events at a given site
for which the yields Wj are known (or at least known sufficiently well that we
can neglect the errors in the observed Wj). Then we can determine a compliance
test and its associated F-number which properly integrates the information in the
3
prior distribution with the data from the calibration events. This is the subject
of the next section.
3 A Bayesian Test of Compliance
In order to determine a test for compliance which makes use of prior informa-
tion regarding ß and the calibration events, we need to determine the probability
density function (pdf) for m = mn+i given mi, 1112, • • •, mn. We will denote this
pdf by /(m|mn), where m^ = (mi, m2, • • •, mn)'. Given n events for which the
yields are known we wish to develop a compliance test for an (n + l)st event for
which the yield is unknown.
Note that the model in (2) can be written in the form
mj = Djß + ej, j = l,2,---,n, (3)
where
D,=
f\ 0 ••• 0 Woj 0
0 1 ••• 0 0 WQJ
o \ 0
w0jJ
= (l,W0j)®Ip
\0 0 ••• 1 0 0 •
and ® denotes the kronecker product.
Case 1: ß known
The problem is a simple one when ß is known since in that event m is
independent of the previous mi,m2, • • • ,m„, i.e. 23^ = 0. The hypothesis Ho,
to be tested is
Ho : W < WT
against (4)
Hi : W > WT,
where W = Wn+i. If we shorten the notation Wo}n+l t° Wc, i.e. take Wo)n+i =
Wc we can write the hypothesis test in Equation (4) in the form
H0: Wc< WcT
Hi: Wc> WcT,
4
where WcT = WT - W0. In this case /(m|inn) = /(m). Now let
P mr = Sr'm«' (5)
t=l
where the r,- are known weights with 0 < r,- < 1 and £{Li r,- = 1. It is well known
that if ej ~ N(0, Ee), then m ~ N(Dß, Ee) where D = (1, Wc) <g> Ip, and it then
follows at once that mr ~ NtfDß, r'Eer), where r = (rl5 r2, • • •, rp)'. Therefore,
under HQ, we take Wc = Wcj< so that a test of the hypothesis in Equation (4) at
the 100a: percent significance level is given by the following rule
Reject Ho if mr > T\a, (6)
where
Tla = r'DT)9 + za v/r'Eer, (7)
Dr = (l,WcT)<8)Ip,
and za is the 100(1 -a)th percentile point of N(0,1) distribution. We shall refer
to the test denned by the rule given by Equation (6) as Test 1.
Case 2: ß unknown
Of course ß is not known and therefore Test 1 cannot be used in practice.
It does however furnish us a base line for comparison purposes. What can be
reasonably assumed, as we have already mentioned, is that ß ~ N(fta, E«), where
fiß and Hß are known. In this case m and nt„ are not independent and therefore
the problem is a bit more difficult. It can however be solved by making use of
the following theorem, the proof of which we include in the Appendix.
Theorem 1. Let m = m„^i be a p-dimensioned magnitude related to
Wo,n+1 = Wc by the model of Equation (3). For k = 1,2, • • •, n + 1, let rh/fc) =
£j=l mj/K ^W{k) = E*=i Wojmj/k, and WQk = £j=i W0j/k. Suppose ß
has the prior density N(fiß, E^). Then the probability density of m given mn,
5
/(m|ni„), is #(/*,£), where
i-l E = Ee Ee — H *^e>
H = E{(1, We) ® Ee1}S/?[s/?+ {En+i ® (se/(n + 1))}]~
.{En+i®(Ee/(n + l))| ElV/J + nfe^E"1)! m(n) ]
(8)
, (9)
and
H={(l,Wc)®Ip}E^[E^+{En+i®(Ee/(n + l))}]
. {En+1 ® (Ee/(n + l))} | ( J ®lJ ,
1 Wo.n+1
-1
(10)
E„+i = .Wb,n+1 EJil <•/(" + !)-
Given Theorem 1, the problem is once again trivial and we can again write
down a 100a% significance level test. If mr is defined by Equation (5), then it
follows from Theorem 1 that the pdf of mr given m„ is iV^/i, r'Er), where fi
and E are given by (8) and (9) respectively. The desired test is then
where
Reject HQ if mr > T2a,
T2a = r'/i + ZaVr^r, (11)
fi and E are defined in Theorem 1 with Wc = WCT, and zQ is the 100(1 - a)
percentile point of JV(0,1). We shall refer to the test of Equation (11) as Test 2.
From Theorem 1 one can also obtain a confidence interval for W. For a
general treatment of the confidence interval problem with a Bayesian prior see
R. H. Shumway and Z. A. Der.
4 A Constrained Bayesian Test
In a recent report Nicholson, Mensing and Gray (1991) show how previous
magnitude data can be used to define a Bayesian prior for ß even though the
associated yields are not available. We shall refer to such data as "no yield"
data as before, and we also assume that n calibration events are available. In
this section we consider the question, "What is the maximum information that
can be gained by this approach ?" In order to accomplish this we will consider
the problem of the previous section but we let the number of no-yield events
go to infinity. That is, we consider the case where the "no yield" data set is
sufficiently large that the parameters that are estimable from that data can be
estimated without error, i.e. they are known. By developing a test for this
case and comparing its power to Test 2 we are able to determine the maximum
improvement in power (or reduction in F-number) obtainable in the approach of
Case 2.
Specifically we note that the parameters
c,_i = Bi/Bi, i' = 2,3,---,p
and (12)
/x,_l = Ai-ci_iA\,
do not depend on yield and hence consistent estimates for c,-_i and [i(_i can be
obtained from the "no yield" data. Thus in this case we take c,_i and m_\ as
known, i = 2, • • • ,p. Moreover under these constraints the model of Equation (3)
becomes
mj=PL + VLjßl + ej, (13)
where^i =(^1,ß1)', /iX = (0,/ii,/X2,---,^-l)'and
/ 1 ci c2 ••• cp_i
\W0j CIWQJ C2WQJ ••• cp_ iW0j,
Thus the original 2p dimensional parameter space for ß is reduced to the 2
dimensional one due to the constraints in Equation (12).
7
To determine a test for the model of Equation (13) we need the following
theorem, the proof of which is included in the Appendix.
Theorem 2. Let m = mn+l De a p-cfimensionaJ magnitude related to
Won+1 = Wc by the model of Equation (13). Suppose ß\ = (A\,B\)' has the
prior pdf N(m, Ei). Then the pdf of m given m„ is JV(/ic,Ec), where
Ec = [ip - DI)n+iQ^+1Din+1Ej J Ee, -In n-l HC = I*L + EcE-^i^+iQ-^Rn,
and
n+1 Qn+1 = EJ-1 + ^(Dij-E-^),
3=1 n
R„ = E^Vl + £ DiiEe Vj - /<l), i=i
/ 1 ci c2 ••• cp_i Y
L,n+l~\Wc ClWc c2Wc ••• Cp-iWc/
From Theorem 2 it follows that mr ~ i\^(r'/4c, r'Ecr) and hence a 100(a)%
significance test of the hypothesis in Equation (4) is given by the following rule:
Reject HQ if mr > T$a,
where
T3a = r'/ic + W^Ecr, (14)
and za is the 100(1 — a) percentile point of a JV(0,1). We shall refer to the test
of Equation (14) as Test 3.
5 Power Curve Comparisons
In order to assess the impact of imposing the prior information, we compare
the power of the following tests:
8
Test 1 : a test of hypothesis based on the assumption that the population
parameters are known.
Test 2 : a test of hypothesis based on the unconstrained Bayesian ap-
proach and the assumption that the parameters are unknown.
Test 3 : a test of hypothesis based on the constrained Bayesian approach
and the assumption that the population parameters are un-
known.
The power at W is given by
Pawer(W) = P(mr > Ta|mn, W).
Also the F-number of the test is given by
p _ IQWF-WT
where WF is the value of the log yield at which the power is 0.5.
Since we specified the critical values of Test 1, Test 2, and Test 3 in (7), (11),
and (14), respectively, it is easy to show that the power of Tests 1, 2, and 3 are
Power(W)1 = 1 - $ (za + £^)§=^) , (15)
Power(W02 = 1 - * (^-^)^zaV?Wv\ \ Vr'Epyr )
Power(W03 = 1 - $ M^c - Kw) + W?%F\ V VT'2cWr /'
where D = (1, W) <g> Ip, * is the cumulative distribution function of N(0,1), and
{PW, Spy}, {HcW, ^cw) are defined as in Theorem 1, and Theorem 2, respec-
tively, with W given.
From (16) and (17) it is clear that the power of Test 2 and Test 3 depends on
the value of iitin. Therefore in order to compare with Test 1 we generate two equiv-
alent data sets for Test 2 and Test 3 with fixed values of {Ee,^,2^,ci,/ii,n}
when p = 2. With the known parameter and the generated data sets, we com-
puted the power of Test 1, Test 2, and Test 3 on the 100 equally spaced grid
values between log 150 and log 300 for W from (15), (16), and (17), respectively
9
and Wo = log 125. We ran this simulation 20 times to get the mean of the powers
for Test 2 and Test 3. Pawer(W)l» mean Power(W)2, and mean Power(VT)3 are
plotted on Figure 1 through Figure 8 for various values of {Ee,Hß,E^, c\, fi\,n}.
Now we summarize some findings from the simulation. As we can see in Fig-
ure 1 through Figure 3, mean Power(W)2 and mean Power(W)z rapidly converge
to Power(W)i as n gets large. Similarly average F2 and average F3 converge to
Fi as n grows, where Fi, F2 and F3 are the F-numbers of Test 1, Test 2, and
Test 3, respectively.
The relatively better performance of Test 3 over Test 2 is observed regardless
of the values of c\ in Figure 1 and Figure 4. However, Figure 5 and Figure 6
show that the overperformance of Test 3 against Test 2 diminishes as the standard
deviations of A2 and B<i ((TA2,^B2) decrease to those of A\ and B\, respectively.
Figure 7 and Figure 8 show the same phenomenon as <rC2 becomes small enough
to be similar to aty. Thus it would appear that if the values used here for ftß, E^
and Ee are representative, additional no yield data would be of little value.
6 Robustness
In the previous sections we have developed a test of the hypothesis of com-
pliance of the (n + l)st event given n calibration events when ß is unknown. We
referred to this as Test 2. In making use of this test it is important to under-
stand the nature of the false alarm rate or significance level a. Possibly the best
way to interpret a is to think through a simulation for estimating a. In order
to simulate the process one would first generate ß from N(fiß^ß) and then,
given ß and Wt-, t = l,---,n, generate ei,e2,---,e„ to obtain m„. Now let-
ting Wn+\ = log 150 and generating en+i to obtain mn+i, one would apply the
test and note the decision. This simulates the senario of obtaining n calibration
events and one additional event of unknown yield. This entire process would be
repeated a large number of times and the proportion of incorrect decisions would
approach a. Table 1 below describes the method. The ßi denote the values of
ß generated on simulation # i. Let mr(T1+i(t) = r^mj^+i, where mj)T1+i is the
10
(n + l)st magnitude vector generated in the ith simulation, i = 1,2, • • •, /.
Table 1. Simulation procedure for estimating the false alarm rate
Simulation # 1 Simulation # 2 Simulation # /
Given A,Wi," -,Wn faWu — ,Wn
Generate mn,---,mi„ m2i,---,m2n
Generate mi>n+i,W = log 150 m2in+i,W = log 150
Decision Reject if Reject if
mr>n+i(l) > r2a(l) mrfB+i(2) > T2a(2)
m/,n+l.^ = logl50
Reject if
mr>B+i(/) > T2a(l)
Now, if we define a random variable X such that X = 1 if mr>n+i(j) > T2a(j),
and otherwise Jf = 0, it follows that
a=Hm S^Xj 1= lim X, a.s.
We should note, however, that in practice the application of these tests will
be to events mn+i,m„4.2,-- -,mn+s at the same site. That is, what is needed
is essentially a test so that the empirical false alarm rate or significance level
approaches a as s gets large rather than as / gets large. We shall refer to the
sample false alarm rate as s —► oo as the "actual significance level" or "actual
false alarm rate" and denote it by a(mn|n,jö). Thus
a(r3„|n,0) = P(mn+S > T2a|m„, W„+s = log 150,0) (18)
It can be shown that
= Urn (# of mn+k > T2a)/k.
lim a(nt„|n,/9) = a.
Thus when n is large, a(mn\n,ß) « a regardless of the observed value of ß.
However in most instances n will be small and therefore the question which
arrises is, "How robust is a to small values of n and unusual values of ß, i.e.
values of ß far removed from fiß ?" That is, "How close is a to a(mn\n,ß),
11
the actual false alarm rate, when n is small and ß is substantially different from
HßT In addition to the "actual false alarm rate" we need to obtain the probability
of rejecting HQ as s —► oo. We shall refer to this as the "actual power" or the
"actual probability of detection", and denote it by P(W\n,ß). Thus
P(W\n,ß) = P(mn+a > T2a\v&„J,Wn+, = W) (19)
= lim (# of mn+k > T2a)/k. «—►00
Then it also can be shown that
lim P(W\n,ß) = PoweT(W).
From (18) and (19) it is clear that a(mn\n,ß) and P(W\n,ß) depend on mR.
Thus for every sample of mn these quantities will be different. We can however
estimate E[a(mn\n,ß)] and E[P(W\n,ß)] for various values of ß and n. This is
the topic of the remaining portion of this section.
In order to investigate the robustness of the actual false alarm rate,
a(rnn\n, /?), a small simulation was performed for a variety of values of n and ß.
Specifically, taking p = 2,fiß = (/Mj,/^,^,/^)' = (Mfl.l)'» <*AX = *A3 =
^Bi = °B2 ~ °-05> PA = PB = 0.5, PAB = 0, ffei = <^e2 = 0.05, pe = 0.5, W =
log 150 and Wo = log 125, we considered the cases
0 = /*/? + <?• (^,^,0,0)',
where C = 0,±1, ±2, for n = 1,2,3,5,10 and 100.
For each case a value of m„+i was obtained 10,000 times (or equivalently
mn.f,-, t = 1, • • •, 10,000 was obtained) and a(mn|n,/?) was estimated by
Ä/_+ . .. # of rejections ,ääS 6&.\n,n~* 10
]000 • (20)
As already noted a(xnn\n,ß) depends on m„ and clearly the same is true about
&(mn\n,ß). Therefore a reasonable measure of the robustness of Test 2 when n
is small is the E[a(nin\n,ß)] = /ia. To obtain an estimate of fia, for each case
we generated 20 repetitions of a(mn\n,ß), i.e.
1 2° _♦ V<* = ^^2&iC^n\nJ). (21) 20 ,=.
12
The results of these simulations are given in Table 2 for a = 0.025. It is worth
noting the relatively large standard deviation of a(mn\n,ß). In view of the val-
ues of fia one can conclude that the distribution of a(mn\n,ß) is quite skewed
to the right or at least contains some extreme values on the right side. That
is, values of a(mn\n,ß) much larger than jxa are more frequent than values of
a(tnn\n,ß) less than /xa, or substantially larger values of a(mn\n,ß) than p,Q
may not be unusual. Since a(tnn\n,ß) is obtained from 10,000 repetitions, it fol-
lows that a(mn\n,ß) « a(mn\n,ß). So similar remarks can be made regarding
a(mn\n,ß). The result of this is that Table 2 presents these results in a conser-
vative way since most people would interpret the mean as a typical value of the
false alarm rate. What we are cautioning here is that, in fact false alarm rates
substantially larger than the mean values shown in Table 2 will be much more
common than in a symmetric distribution. We probably should have included
the median in Table 2, but that was not calculated.
It should be noted that if C < 0, the Bayesian estimator of yield will un-
derestimate yield and hence the true false alarm rate will be too small while if
C > 0 the estimator will overestimate yield and hence the false alarm rate will
be too large. From inspection of Table 2, it appears that if we have only 1 or 2
calibration events, this effect can be large, and hence in this case the Bayesian
significance level or CI may be seriously in error. On the other hand if n > 5 the
method might be considered adequate, even though for C < 0 the false alarm
rate may still be sufficiently too small that it could very adversely effect the
power, i.e. the chances of detecting a violation.
Power Considerations
Figures 1 - 8 compare the power of Test 1, Test 2, and Test 3 for various
parameter configurations. As in the case of the false alarm rate, if these parame-
ter values are representative, little is to be gained from additional no yield data.
Also, from the comparison of the F-numbers it does not appear that a great deal
is to be gained by taking n > 2. Unfortunately these rather pleasant results do
not uniformly extend to the actual power.
13
Figure 9 through Figure 36 compare the "actual" power of Test 2 to the
power of Test 2, i.e. they compare P(W|n, ß) to P(W). The figures also compare
the F-number for Test 2 to the "actual" F-number. For n < 2 it is clear that
both the power and the F-number are seriously effected if C = ±2 and the same
is true for C = ±1 if n = 1. It should be pointed out that the small F-numbers
associated with C < 0 are a result of very large false alarm rates and should not
be viewed as improved tests.
Concluding Remarks
In this report we have investigated the robustness of the Bayesian method
(referred to as Test 2) for testing compliance of an observed yield to a threshold.
Although the simulations reported here were not exhaustive, they were adequate
to demonstrate that the Bayesian method for testing compliance is probably not
satisfactory if there are only one or two calibration events. Moreover it is highly
desirable to have five or more calibration events to guarantee good agreenent
with the stated significance level. Similar remarks could be made regarding the
corresponding confidence intervals.
The consequence of these findings is that if it is unlikely that several cali-
bration events will be available, Test 2 and confidence intervals associated with
Test 2, the Bayesian tests and CI discussed by Nicholson, Mensing and Gray, and
those introduced by Shumway and Der should be used with care. In fact if the
number of calibration events is less than 3 it would probably be wise to consider
a constrained likelihood method as an alternative to the Bayesian method, or, if
possible, the Bayesian method should be extended to include the case of several
events following the calibration events.
14
Table 2. Estimate of Actual False Alarm Rate E[a(n,ß)], a = 0.025
C = -2 C = -l C = 0 C = l C = 2
Aa 0.0000
st. dev. fia 0.0000
st. dev. a(mn\n,ß) 0.0000
£a 0.0006
st. dev. jia 0.0003
st. dev. a(mn\n,ß) 0.0012
£a 0.0034
st. dev. /ia 0.0014
st. dev. d(m„|n,/9) 0.0062
£a 0.0046
st. dev. fia 0.0018
st. dev. a(mn\n,ß) 0.0079
/ia 0.0076
st. dev. fia 0.0019
st. dev. a(mn\n,ß) 0.0084
Aa 0.0119
st. dev. fia 0.0017
st. dev. d(m„|n,/9) 0.0075
/xa 0.0229
st. dev. fia 0.0015
st. dev. a(tnn\n,ß) 0.0065
n = 0
0.0000 0.0023 0.0488 0.3165
0.0000 0.0001 0.0006 0.0011
0.0000 0.0004
n = l
0.0025 0.0050
0.0034 0.0149 0.0532 0.1418*
0.0013 0.0044 0.0112 0.0205
0.0058 0.0197
n = 2
0.0499 0.0919
0.0089 0.0225 0.0498 0.0985
0.0030 0.0061 0.0107 0.0173
0.0132 0.0273
n = 3
0.0479 0.0774
0.0099 0.0193 0.0364 0.0643
0.0033 0.0055 0.0087 0.0132
0.0146 0.0246
n = 5
0.0389 0.0592
0.0121 0.0192 0.0283 0.0425
0.0029 0.0042 0.0058 0.0081
0.0128 0.0190
n = 10
0.0259 0.0363
0.0155 0.0201 0.0255 0.0316
0.0025 0.0030 0.0034 0.0040
0.0110 0.0132
n = 100
0.0152 0.0179
0.0230 0.0234 0.0238 0.0240
0.0013 0.0016 0.0014 0.0015
0.0058 0.0072 0.0063 0.0065
15
* note: For symetric confidence intervals a 100(1 — 2a)% two sided confidence
interval corresponds to a one sided a—level significance test. For example, for
Test 1 of size 0.025, the corresponding two sided confidence interval is a 95%
C.I. This suggests that if the "actual" significance level is 0.14, the actual C.I.
could be a 72% C.I. That is, if the site geological bias is 2<r greater than the
expected bias, \i£, then even though the Bayesian significance level is 0.025 and
the Bayesian C.I. is 0.95, the actual significance level is estimated here as 0.14
and one would assume that the actual two sided C.I. is around 72%.
16
POWER COMPARISON *—1
-m r-p-r TT TT TT i i i i i i i i i i i i i i i i i i i i i i i i i ii ii ii i i M rr - -
n = = 2
co o
_
Pow
er
0.4
0.6
-
/
//■■''
•V V
/ / _
- C3
TEST 1 rj = 1.37
O
ill!
s?' ?■
<<£
1
TEST 2 Fg - 1.44
TEST 3 F3 = 1.40 -
o UlAJ-i-L-LJ-lJ ......... i . i .. . . , , 140 160 180 200 220 240 260 280 300
Yield
"«. = 0.05 MA, = 4.00 MBj = 1.00 CTA, = 0.05 °Bt = 0.05 PAB = 0.00
°*z = 0.10 MA2 = 4.00 MB2 = 0.90 °A2 = 0.10 aB2 = 0.10 <=1 = 0.90
Pe = 0.50 PA = 0.45 PB = 0.45 Ml = 0.40
Figure 1
17
POWER COMPARISON , i ! , I | | | | | II II I I I I I | I II II I I I I | | I I I I I I I I I | I I I I I I I I I | I I I I I I IMJJJJJJJ-U
TEST 1 Fi = 1.37
TEST 2 F2 = 1.41
TEST 3 F3 = 1.39
■ ■ ■ i ' ' 11 11 11 i I 111 i i i i 11 11 11 i i i 11
140 160 180 200 220 240 260 280 300 Yield
a, = 0.05 MA, = 40° AB, = 10° "A, = 005 "B, = 005 PAB = 0.00
C, = 0.10 fiAg = 4.00 MB2 = 0-90 <r*2 = 0.10 ohz = 0.10 Ci = 0.90
p. = 0.50 PA = °-45 PB = 045 in = 040
Figure 2
18
POWER COMPARISON
TEST 1 F, = 1.37
TEST 2 F8 = 1.39
TEST 3 F3 = 1.38
11 ' ■ '' ■ ' ■ ■ ■ ' * ' * * ■' ■ * *' ■ ' * ■ ■ ' > ■ * * * ' * ■
140 160 180 200 220 240 260 280 300 Yield
aei = 0.05 ßAs = 4.00 ßBl = 1.00 oAj = 0.05 CTBl = 0.05 pAB = 0.00
cet = 0.10 ßAz = 4.00 MB2 = 0.90 cAz = 0.10 (TBj = 0.10 Cj = 0.90
Pe = 0.50 pA = 0.45 pB = 0.45 /ij = 0.40
Figure 3
19
POWER COMPARISON T-l
i i i i i i | i i i i i i i i i | i i i i i i ii i | ii i ii ii ii | ii IM ii ii | i ii MI ill i i i ii i iiMjjjjjm ~tr
_ n = 2 ^S0^ -*' _ y^ ^ s^ ^
co / ' _ / ' — o / '
■ / ' - / / - / / _
CO / / ^d
■ / /
/ /
/ /
"
c> / / . PLH^ // _
o / / //■■'
•
//.■■
w //.■■ TEST 1 Fi = 1.35
d //. - // TEST 2 Fg = 1.41
s" yP TEST 3 F3 = 1.39
o tTii i i In ui ii,u.iji,iJ.jj.JiJ JiJj-iJi.i.j-i-Li.i.JiJ±j.jj,JjJi.jj.j.iiJ.JiJ.iiJjjji li ii in i , ,
140 160 180 200 220 240 260 280 300 Yield
a,, = 0.05 ßKl = 4.00 fiBi = 1.00 CTA( = 0.05 aBl = 0.05 pAB = 0.00
on = 0.10 fiAl = 4.00 MB2 = 100 trÄ2 = 0.10 <TB2 = 0.10 Cj = 1.00
p% = 0.50 pk = 0.50 pB = 0.50 ß} = 0.00
Figure 4
20
POWER COMPARISON 11111111111111111111111111111111111 ii i 11111111111111111
TEST 1 Ft = 1.35
TEST 2 Fg = 1.41
TEST 3 F3 = 1.39
1 ' ' ' ' ■ ' ' ■ i 11 11 i i
140 160 180 200 220 240 260 280 300 Yield
oej = 0.05 /iAj = 4.00 fiBl = 1.00 aAl = 0.05 aB, = 0.05 PtS = 0.00
at2 = 0.10 /i^ = 4.00 rtj2 = 1.00 akz = 0.08 CTB2 = 0.08 ci = 1.00
pe = 0.50 pA = 0.63 pB = 0.63 Hi = 0.00
Figure 5
21
POWER COMPARISON *-( rrj-n 11 11 T> "^
n = 2
CO
d - / y
Pow
er
0.4
0.6
~
/
/ ' / /
/ /
/y '/ /
/ /
- cv TEST 1 Fi - 1.35
o TEST 2 Fj - 1.39
. i i 11
/P' V
\ |
TEST 3 F3 = 1.39 -
o iii ii iiiii , ,, , ,
140 160 180 200 220 240 260 280 300 Yield
oei = 0.05 ßk% = 4.00 fiBl = 1.00 aAl = 0.05 aB, = 0.05 pig = 0.00
o,2 = 0.10 fik2 = 4.00 fiBl = 1.00 ckt = 0.06 CBJ = 0.06 Ci = 1.00
p. = 0.50 pA - 0.83 pB = 0.83 ^i = 0.00
Figure 6
22
POWER COMPARISON 1—1
n = 2 j^^ •"* ■■■'
oo d
_ / / -
Pow
er
0.4
0.6
- / /
/ ' / ' / ' / /
/ A
/ ' / '
-
- / ,: ~ / /
//■'
Fj = 1.27
F2 = 1.32 C\2
TEST 1
O TEST 2 //
TEST 3 F3 = 1.31 -
o 11 Li.hjujgm 1 i i i i i i i i i 1
140 160 180 200 220 240 260 280 300 Yield
oei = 0.05 fiAi = 4.00 ßBi = 1.00 <TAI = 0.05 aBl = 0.05 PAB = 0.00
o,2 = 0.07 MA2 = 4.00 MB2 = 1.00 ckl = 0.10 aBz = 0.10 Ci = 1.00
Pe = °-50 PA = 0.50 pB = 0.50 Hi = 0.00
Figure 7
23
POWER COMPARISON
cq 6
«D
-2
d
q d
| | | | | I | I I I | I I I I I I I I I | I I I I I I I I I IMjiiU'l I |IMUUI-»riVIMM | I I I
TEST 1 F, = 1.22
TEST 2 F8 = 1.26
TEST 3 F3 = 1.26
■ ' ' ■ ■ ■ '' '' ' 11 11 i i i i 11
140 160 180 200 220 240 260 280 300 Yield
ce, = 0.05 MA, = 4.00 ßBi = 1.00 aA( = 0.05 aBl = 0.05 PAB = 0.00
o„ = 0.05 /zAj = 4.00 /zBa = 1.00 aAi = 0.10 crBj = 0.10 Cj = 1.00
p. = 0.50 pA = 0.50 pB = 0.50 H! = 0.00
Figure 8
24
Actual Power Comparison *-H '
n = 1 -
oo •
^n CD ■
£ - o,„ • P^ / -
o TJ _
CD -+-> TH
V^ / • " CD * OH - X / . H^ - / o TEST 2 r 2—1.27
■
Act.TEST 2 Act.Fg—1.42
o lill 1 1 liJJJJ.Ml.U.llliMjJ i , , . if,,,.,.
140 160 180 200 220 240 260 280 300 Yield
o.j = 0.05 /iAl = 4.00 ßBl = 1.00 aAl = 0.05 aBl = 0.05 p/s = 0.00
o,s = 0.05 /iAz = 4.00 /iBl! = 1.00 CTÄ2 = 0.05 oBj = 0.05 C = -2.00
pe = 0.50 pA = 0.50 pB = 0.50
Figure 9
25
Actual Power Comparison *-l i i 1111 | | 11 III_I ir' r~——n M 11 11 1111
- n = 1 -
CO
^o " ~ OJ - - £ - - o ,„ .
PU^ . o
■Ö OJ ^^ ?rs - CD - PH - X
o -
TEST 2 Fj-l.Zv ■
Act.TEST 2 Act.F3-l.34
o '''■ 11 1 11 11 1 t-i-i.iii 11 1111 11 1 1 1 1 1 1
140 160 180 200 220 240 260 280 300 Yield
ati = 0.05 fj.A% = 4.00 fiBl = 1.00 CTAJ = 0.05 oBl = 0.05 pAB = 0.00
a,2 = 0.05 fi^ = 4.00 /iBl = 1.00 aAz = 0.05 aBl <= 0.05 C = -1.00
p, = 0.50 pA = 0.50 pB = 0.50
Figure 10
26
Actual Power Comparison *-l
n = 1 -
CO
^o ~~ <D • £ - o ,„ .
PL,^ _ o
T3 CD
-•-> Th
!r!o ~ 0) ■
PH - X .
o - TEST 2 Fj—1.27 - Act.TEST 2 Act.F2-1.27
o 140 160 180 200 220 240 260 280 300
Yield
CTe] = 0.05 (j,Ai = 4.00 /iB] = 1.00 CTAJ = 0.05 aB, = 0.05 PAB = 0.00
a.j, = 0.05 ßkz = 4.00 ßBl - 1.00 CTAJ = 0.05 aBj = 0.05 C = 0.00
p„ = 0.50 pA = 0.50 pB = 0.50
Figure 11
27
Actual Power Comparison n^TTTTTTTm
TEST 2 F2=l-27
Act.TEST 2 Act.F2=M9
IJ-I I i UJ i liiiii i, n il„i i ii ill AxJ,.i,i„i 11 u i i -JI i I i ii i i i 1 i i i I i,i I J I 1 1 Ll.l J .1 LI.L
140 160 180 200 220 240 260 280 300 Yield
cej = 0.05 HKl = 4.00 tiBi = 1.00 CTAI = 0.05 cBj = 0.05 pAB = 0.00
a,2 = 0.05 MA2 = 4.00 /iBj = 1.00 aAj = 0.05 CTBj = 0.05 C = 1.00
p. = 0.50 pk = 0.50 pB = 0.50
Figure 12
28
Actual Power Comparison 1—1
n = 1 /^
CO ^o <D / £ / o „„ /
Pu^ / o / -d / 0)
+-> <«
?>* CD • / PH / X / H™ -
o ■
Act.TEST 2 Act. F2— 1.13
o " " ' " ■ ■ 1 ■ ■ ■ ■ 1 1 1 ■ 1 1 1 1 1 1 t t 1 1 t 1 1 1 1 1 1 1 1 1 1 1 , ,
140 160 180 200 220 240 Yield
260 280 300
a„ = 0.05 MA, = 400 /iBl = 1°0 CTA> = 0.05 (TBl = 0.05 PXB = 0.00
ae2 = 0.05 ßk2 = 4.00 fiBl = 1.00 CTA, = 0.05 CTB2 = 0.05 C = 2.00
pe = 0.50 p* = 0.50 pB = 0.50
Figure 13
29
Actual Power Comparison r-1
n = 8 /^
CD - /
t^ / CD / £ / o „„ / HH^ / o -Ö
CD -t-> «.
^ / 0) / PH / X / Hcv -
O TEST 2 Fg—1.20 - Act.TEST 2 Act.F2-l.34
o Mil Jj.Ul.lJJ.il 111 IllJlllJ
140 160 180 200 220 240 260 280 300 Yield
atl = 0.05 /iAl = 4.00 nBl = 1.00 CTA] = 0.05 trBl = 0.05 pAB = 0.00
c,2 = 0.05 /ikz = 4.00 /iB2 = 1.00 CTA2 = 0.05 cB2 = 0.05 C = -2.00
p. = 0.50 pA = 0.50 pB = 0.50
Figure 14
30
Actual Power Comparison ■*-1
1111 1111111111 11 u 111111 i 1 1 l l l 1 1 1 ILjJ-U 1 1 1 1 l | l 1 l l l l l 1 1 "| 1 1 1 l l 1 l .
n = 2
-
Pow
er
0.6
0.8
E
xp
ecte
d
.2
0
.4
-
■ o
o " rTT-t ii'i 111111 ' ' , ,
140 160 180 200 220 240 260 280 300 Yield
c., = 0.05 MA, = 4.00 /xBl = 1.00 aA, = 0.05 CTBI = 0.05 PAB = 0.00
aez = 0.05 M*2 = 400 /iBs = 100 CTAI = 0.05 aBz = 0.05 C = -1.00
Pe = 0.50 PA = 0-50 pB = 0.50
Figure 15
31
Actual Power Comparison *—1
n = 2
CD
Vn " (D ■ ■
£ - - o „ . •
cuw. _ . o
-d . a;
■+-> Tt
^^ ~
CD • C^ - X .
W™ - O TEST 2 r2—1.20
■
1.1111 111111 1111 i. 11111111 11111111 JJ.
Act.TEST 2 Act.F2-l.c4
_i_i_ o 1.2 212 ii i-i-i-i-i 11 11 11 1 J 1111111 11 1 i 11 j 111
140 160 180 200 220 240 260 280 300 Yield
".. = 0.05 MA, = 4.00 MB, = 1.00 "A, = 0.05 °B, = 0.05 PAB = 0.00
°*2 = 0.05 MA2 = 4.00 MB2
= 1.00 °*2 = 0.05 <*Ba = 0.05 C = 0.00
P. = 0.50 PA = 0.50 PB = 0.50
Figure 16
32
Actual Power Comparison *—1
IT 1 1 | 1 1 1 II 1 1 1 1 | 1 1 1 1 1 1 1 1 | i | 1 1 I.I.I .J.I 1 MJ_I_UI 1 1 1 1 1 1 | 1 1 1 1 1 1 | 1 | | | | | | | | | 1 '
n = 2 yS
00 / ^^
~ / <u - / £ - / O „ - : /
PU^ _ o
Ti <L>
+> * /
?* / CD / iX ■ / -
X [VI 02
o ■
TEST 2 Fa=1.25
Act.TEST 2 Act.F2=1.19
o 1 1 I 1 1 1 1 1 1 1 1 1 1 1 1 1X1.1
140 160 180 200 220 240 260 280 300 Yield
atj = 0.05 ßAl = 4.00 ßB> = 1.00 CTA, = 0.05 CTBI = 0.05 pAB = 0.00
cea = 0.05 /iAs = 4.00 fiBl = 1.00 aKx = 0.05 aBz = 0.05 C = 1.00
pe = 0.50 pA = 0.50 pB = 0.50
Figure 17
33
Actual Power Comparison I | | | | I I I I | I I I I I I I I I | I I I I I I I I I | I I I I I I I I I | I I I 'I I I I I MJJJ.,1 I I I WTTTTTI I I I I | I I I I I I I I I
TEST 2 F2=1.25
Act.TEST 2 Act.F2=1.15
M ■ ' I ■ » ■ i i i i i ■ I t i . . i i i i i i
140 160 180 200 220 240 260 280 300 Yield
ce, = 0.05 /iAl = 4.00 fiB> = 1.00 oA] = 0.05 CTBI = 0.05 PAB = 0.00
o,2 = 0.05 fikz = 4.00 /*B2 = 1.00 CTAS = 0.05 crBa = 0.05 C = 2.00
p. = 0.50 pA = 0.50 pB = 0.50
Figure 18
34
Actual Power Comparison 1-1
- n = 3
CO ^ri - CD . £ - o ,„ d^ o T3
CD tJ <* ■
?ri - oj . a . X
o _ TEST 2 Fg=1.24
■
Act.TEST 2 Act.F8=1.31
o ■ ■' ■ < ■■••■■ i ■ i ■ ■ ' '' i' -i-» J i 11 11 11 11 11 111 i
140 160 180 200 220 240 260 280 300 Yield
a., = 0.05 fikl = 4.00 fiBl = 1.00 aAt = 0.05 aBl = 0.05 pAB = 0.00
a., = 0.05 fikz = 4.00 ßBz = 1.00 aAj = 0.05 CTB2 = 0.05 C = -2.00
P„ = 0.50 pA = 0.50 pB = 0.50
Figure 19
35
Actual Power Comparison *—1
n = 3 /^
CO
^o <D / -'
£ / o ,„ . / PU^ _ f
o / -Ö /
OJ -4-> ,*
üo ~
CD / OH / X / H™ -
o - - Act.TEST 2 Act.r2-127
o , ,
140 160 180 200 220 240 260 280 300 Yield
oe, = 0.05 /iAl = 4.00 fiBl = 1.00 oAl = 0.05 <7Bl = 0.05 pw = 0.00
o„2 = 0.05 fik2 = 4.00 MBa = 1-00 aAj = 0.05 aBa = 0.05 C = -1.00
p„ = 0.50 pA = 0.50 pB = 0.50
Figure 20
36
Actual Power Comparison r-t
1111 11111 M 1111 1111 i 1 ■
n = 3 \f
GO . y
5HO •'/ OJ - ./ £ - •7 o,„ .7
PU^ ■'/ _
o id 0)
-•-> * "o / oj / O. / X
o - TEST 2 F2-1.24
■
Act.TEST 2 Act.Fg-1.24
o ■ ■ ■' in •'
140 160 180 200 220 240 260 280 300 Yield
otl = 0.05 MA, = 4.00 MBl = 1.00 aAl = 0.05 aB, = 0.05 pAB = 0.00
a„2 = 0.05 MA2 = 4.00 fiBl = 1.00 aAj = 0.05 aBj = 0.05 C = 0.00
pe = 0.50 pA = 0.50 pB = 0.50
Figure 21
37
Actual Power Comparison y-i ".
n = 3 yS
Pow
er
0.6
0.8
/
Expec
ted
.2
0.
4 -
/ / ■ <->
Act.TEST Z Act.r2—1.Z0
o ■ , i i i , , i i i 11 111 i
140 160 180 200 220 240 260 280 300 Yield
oej = 0.05 MA, = 40° MB, = I0° aAl = 0.05 aBt - 0.05 Pig = 0.00
cei! = 0.05 ßkl = 4.00 MB2 = 1.00 ak2 = 0.05 <j„2 = 0.05 C = 1.00
p, «= 0.50 pA = 0.50 pB = 0.50
Figure 22
38
Actual Power Comparison T-4
n = 3 -
CD ^rS - CD ■
£ - O Ä
.
PH^ _ o
T* cu
-*->'*
?* ~ CD - OH - X .
o - TEST 2 F2-1.24 • Act.TEST 2 Act.F2-1.17
o -rf^\, -1 1 1 1 I I^J-LJ-Ll.l.L.J-U-1.1 J J J J J J 1 1 1 11 11 1 1 1 1 111 11 11 11 1 n 1 11 1 1
140 160 180 200 220 240 Yield
260 280 300
oei = 0.05 IJ.kl = 4.00 fiBl = 1.00 aAl = 0.05 aBj = 0.05 pAB = 0.00
o,2 = 0.05 ^A2 = 4.00 /iBi, = 1.00 CT>2 = 0.05 aBz = 0.05 C = 2.00
p„ = 0.50 pA = 0.50 pB = 0.50
Figure 23
39
Actual Power Comparison *-! 1 1 1 1 1 1 1 1 1 | 1 1 1 1 II 1 1 1 | 1 1 II 1 1 1 1 1 | 1 1 1 1 1 1 1 1 1 | 1 II 1 IMU-I 1 ' 1 1 II 1 1 | 1 l-T'I I I 1 I 1 | 1 1 M 1 1 ! ! 1
n = 5 jf
co / ^o <L> / £ / O „ /
OH^ 1 o / -d / 0)
-t-j -^
^ 0) / p. / X / H™ - o TEST 2 T2—1.Z4 -
o --/■;,
140 160 180 200 220 240 260 280 300 Yield
otl = 0.05 /iAl = 4.00 fiBi = 1.00 aAl = 0.05 aB, = 0.05 pAB = 0.00
a,2 = 0.05 fikj, = 4.00 ^B2 = 1.00 tj>2 = 0.05 aBz = 0.05 C = -2.00
p. = 0.50 pA = 0.50 pB = 0.50
Figure 24
40
Actual Power Comparison *—1
1 11 1 1 1 1 11 1 1 1 1111 11 11 11111 i 11111 11111 IIIJJ,^ i 11 i 11111111 11 | -rr
n = 5
CO ^n - <D . £ - o en
OU^ o T3 <D
•+-> T^ *
^ -
<D - a _ X
o _ TEST 2 Fg=1.24 -
tni i i 111111
Act.TEST 2 Act.F2=1.27
o ' '' ' i 11 11 11 i 11 i i i i 11 j i i 11 11 11 11 i i 11 i i
140 160 180 200 220 240 260 280 300
Yield
ce, = 0.05 ßAl = 4.00 ßBl = 1.00 <TAI = 0.05 aB, = 0.05 pis = 0.00
ae2 = 0.05 fik2 = 4.00 MBJ = 1.00 oAj = 0.05 CTBJ = 0.05 C = -1.00
Pe = 0.50 pk = 0.50 pB = 0.50
Figure 25
41
Actual Power Comparison W M I I I | I T I I I I I I 'I [ 1 1 I ! I I I T I
TEST 2 F2=1.24
Act.TEST 2 Act.F2=1.24
iiij.,1 j i ii 111 ii
140 160 180 200 220 240 260 280 300 Yield
oej = 0.05 /xA, = 4.00 fj.Bl = 1.00 cAl = 0.05 cBj = 0.05 pAB = 0.00
o,2 = 0.05 /iÄ2 = 4.00 fiBl = 1.00 oAa = 0.05 oBl = 0.05 C = 0.00
pe = 0.50 pA = 0.50 pB = 0.50
Figure 26
42
Actual Power Comparison r-l
n = 5 / CO
y /
fr" •" / - 0)
■'/
£ / o ,„ ■ /
Pu^ / / _ o
XI OJ
-4-3 Tj<
^ / 0) / ex // X
o - TEST 2 F2-I.24 - Act.TEST 2 Act.F2=1.22
o ,, ,, ,
140 160 180 200 220 240 Yield
260 280 300
oei = 0.05 fiAi = 4.00 fiBi = 1.00 (TA, = 0.05 (7Bl = 0.05 PXB = 0.00
au = 0.05 ^iA2 = 4.00 ßBi = 1.00 aAz = 0.05 aB2 = 0.05 C = 1.00
pe = 0.50 pA = 0.50 pB «= 0.50
Figure 27
43
Actual Power Comparison T—I
n = 5 /
co S
Vri CL> / £ / O „ /
&««? / o •Ü
<D -+-> Tf
?^ / 0) / OH / X ■■ /
H^ - o TEST Z Fj—1.24 " Act.TEST 2 Act.rg=1.20
o 1 1 > 1 i 1 1 1-LII 1 1 1 1 1 1 Ill 1 1 I 1 I 1 1 1 1 1 1 1 M 1 1 1 1 > 1 1 1 1 1 1 1 J 1 1 1 1 1 1 1 1 1 1
140 160 180 200 220 240 260 280 300 Yield
CT«. = 0.05 "A, = 4.00 MB, = 1.00 "*> = 0.05 °B> = 0.05 PAB = 0.00
°*z = 0.05 *A2 = 4.00 MBj = 1.00 "A, = 0.05 °B* = 0.05 C = 2.00
Pe = 0.50 PA = 0.50 PB = 0.50
Figure 28
44
Actual Power Comparison I I I II I I I I I | I I I I I I I 1 I | | | | | | | | | M I I I I | I I I I | | | | | | I | | | I I I I I | I I I I I I I I
TEST 2 F2=1.44
Act.TEST 2 Act.F2=1.72
111111111 '•'TVl'i'i't ' I I I I , , , , |
140 160 180 200 220 240 260 280 300 Yield
°e, = °05 /iAj = 4.00 ßBi = 1.00 aA> = 0.05 cB% = 0.05 PAB = 0.00
c,8 = 0.10 fiAl = 4.00 /uB, = 1.00 aAz = 0.10 CTBS, = 0.05 C = -2.00
P» = °-50 pA = 0.50 pB = 0.50
Figure 29
45
Actual Power Comparison T—1
111111111111111111111111 ii 1111111M1111111111 ii 11111 ii11111111111111111M iii M i
n = 1 ^^^^ ...-
CO jS*^
^ OJ y^
£ / o .^ / PU^ / o X3 (D
^Tt<
?* / 0) / ft / X / w~ - o TEST Z Fg—1.44 .
Act.TEST 2 Act.F2=1.57 -
o 140 160 180 200 220 240 260 280 300
Yield
a., = 0.05 pki = 4.00 MB, = J 00 CTA, = 0.05 aBl = 0.05 p^ = 0.00
a.a = 0.10 /2Aj = 4.00 MB2 = 1-00 aAj = 0.10 CB2 = 0.05 C = -1.00
p. = 0.50 pA = 0.50 pB = 0.50
Figure 30
46
Actual Power Comparison 1111111111 111 MI 11 ii 1111111111111111M ii 111111111111111111
TEST 2 Fz=1.41
Act.TEST 2 Act.F8=1.53
140 160 180 200 .220 240 260 280 300 Yield
atl = 0.05 ßkl = 4.00 MB, = 100 cAi = 0.05 aB, = 0.05 pAB = 0.00
ae2 = 0.10 ßkz = 4.00 MB2 = 1.00 a^ = 0.10 (TB2 = 0.05 C = -2.00
Pe = 0.50 pA = 0.50 PB = 0.50
Figure 31
47
Actual Power Comparison 1111M11111 111111111111 111M1111111111111111111111M111
TEST 2 F2=1.41
Act.TEST 2 Act.F2=1.44
' ' ■ ■ ' ' 11 . i i i 11 i i i i 1 ■'■■■■■■ ■ '' *' *
140 160 180 200 220 240 260 280 300 Yield
«■ = 0.05 MA, = 4.00 MB, = 1.00 °*, = 0.05 OB, = 0.05 PAB = = 0.00
«2 = 0.10 MA2
= 4.00 MB2 = 1.00 CTA2 = 0.10 °B2
= 0.05 C = -1.00
e = 0.50 PA = 0.50 PB = 0.50
Figure 32
48
Actual Power Comparison TH
Mil | 1 1 1 1 1 1 1 1 1 | 1 1 1 1 1 1 1 1 1 | I I I I I I i I I 11 i i i i i i i i | i i i i i i i i i | i i i i i i i TT-
n = 3 •""-
CD
^o OJ ■
£ - o „ -
OH^ - o
-Ö <D
-t-3 Tt
^ri " <P ■
OH - X . H™ -
o "
mill li.ixuj JJJJJJJJJJJ-J
Act.TEST 2 Act.F2-l.06
J_l_ o
140 160 180 200 220 240 260 280 300
Yield
ae, = 0.05 /iA| = 4.00 pBi = 1.00 aA, = 0.05 aBl = 0.05 PXB = 0.00
o„2 = 0.10 fiki = 4.00 MB2 = 1°° °k2 = 010 aB2 = 0.05 C = -2.00
pe = 0.50 pA = 0.50 pB = 0.50
Figure 33
49
Actual Power Comparison »—I
11 11 11 11 111 11 i 11111 11111 | i 1111 11 11 111 11 11 11 i [iiiiiiiii 11 11 111 i
—
n = 3 ^^^^
Pow
er
0.6
0.8
- /
-Ö
ecte
0.4
- /
ft X
"
■
o Act.TEST 2 Act.Fg—1.50
o IllilllllJjl IJIJJllJjUllJJJjJjJJllU
140 160 180 200 220 240 260 Yield
280 300
oej = 0.05 fiAl = 4.00 fiBl = 1.00 oA, = 0.05 aBl = 0.05 PXB = 0.00
an - 0.10 nAl = 4.00 fiBl = 1.00 akt - 0.10 aBj = 0.05 C = -1.00
p. = 0.50 pk = 0.50 pB = 0.50
Figure 34
50
Actual Power Comparison ■*-<
III 111111111111111111 1111 111111111111111111 M i
- n = 5 -
GO ■
^n J0^ - <L> s - £ / - o,„ / .
PL«"* / o
T3 CÜ
-i-5 ^
?* / - 0) - a _ X
o -
TEST 2 F2—1.38 - Act.TEST 2 Act.F2=1.47
• o ■ 11 1 11 i-i_i-i_i..j.i 1.1 11 1111 1 ■ i J 1 1 1
140 160 180 200 220 240 260 280 300 Yield
oti = 0.05 fikl = 4.00 MB, = 1 00 aAl = 0.05 aB, = 0.05 PAB = 0.00
aez = 0.10 nkz = 4.00 fiBz = 1.00 aA2 = 0.10 oB2 = 0.05 C = -2.00
pe = 0.50 pA = 0.50 pB = 0.50
Figure 35
51
Actual Power Comparison i r i r t i i' i i | i i i ii i r r i | T T T r FT M r ] TTTI IIIII | TTT'TTTTTTTTT'T'TTI i "i i | M i I T T m n| i i i i r r T T_L
TEST 2 F2=1.38
Act.TEST 2 Act.F2=1.42
I t t t t i i i i t i I i i i i i i M n | x
140 160 180 200 220 240 260 Yield
280 300
«s = 0.05 "A, = 4.00 MB, = 1.00 "*! = 0.05 "B, = 0.05 PtS = 0.00
°; = 0.10 MA2 = 4.00 MB, = 1.00 CTA2
= 0.10 °*z = 0.05 c = ■1.00
P. = 0.50 PA = 0.50 PB = 0.50
Figure 36
52
APPENDIX: PROOFS
Proof of Theorem 1
For the new observation m related to Wc, the conditional pdf of m given
iitin, /(m|mn) is as follows:
f(m\ttLn) = J fi(mj\vtn)dß
= J f2(m\ßA)h(ß\mn)dß
f2(m\ß)fz(ß\^n)dß, (Al) /
where /i,/2, and /3 are the probability densities. The last equation is obtained
due to the independence between m and in„ when ß is given. The conditional
distribution of ß given mn may be computed using Bayes' law as follows:
Mfi\^)=lhUW^"W ■ (A2)
j h(ß)L(T&„\ß)dß '
where h is the prior density of the parameter vector ß and L is the likelihood
function for the data mn, given values of ß. If we assume ej are independent
multivariate normal, then
n
L(^n\ß) = H^mj\ß\ 3=1
where
rKmjW = (27r)-P|Ee|-1/2exp{ - I(m,- - D^/E^m; - D^)}.
Note
f2(m\ß)L(nln\ß) = L(^n+1\ß,mn+l = m,Wn+1 = Wc)
since e^ are independent. Thus referring to (Al) and (A2) leads to
/(mlm-n) = .W)£("*"+llftm"+l = m>Wn+l = Wc)dß f h(ß)L(xän\ß)dß
Thus if h(ß) is available, / is completely determined.
53
Note
h(ß)L(TZn\ß) oc cxp[-±{(ß--ltß)'Vß1(ß-»ß)
+ ^(mi-D^)VK-D^)}
The exponential of (A3) is -1/2 times
(A3)
;=i
Let W0n = E"=l W0j/n. Since D; = (1, Woj) 0 Ip,
(A4)
2DJB"^ = E ((1. Wbi)' ® IP)2e*((1, Wbi) ® Ip) i=i i=i
-tff1 *iW) 'e
-1 0 (Ee/n)
IV Won £?=!</"/ V '
= JE„ 0 (Ee/n) } ,
= <
-1
where -1
En
1 w0n
Let ih(n) = YJj=\ mj/n and "V(n) = £j=l Wojtnj/n. Then Jt is easy to
verify that
E npe^j = Hn)^W(n)) (*2 ® (Se/n)"1).
54
We can rewrite (A4) as follows:
0 [E^1 + {En 0 (Ee/n) }" ]ß - 2 [jfö1 + (m'(n), m^(n))
—1 n
• (l2 0 (se/n) )] 0 + ^E^/i/J + 53 mJEe lmi i=i
= (0 - Zn)' [E^1 + {En 0 (Ee/n) }_1] (^ - Z„) -In
Z'„ [E^1 + {En 0 (Ee/n) } ] Z„ + ^E^ + £ mJ^lm. lj>
where
Z« = -1-1-1
n= [E^ + {En®(Ee/n)} '] l Ttfpß + {l2 0 (Ee/n) *} ( "^
Since -(l/2)(0 - Z^E*1 + {E„ 0 (Ee/n)}_1](0 - Z„) is the exponential
of the multivariate normal density with mean Zn and variance [Eä + {En 0
(Ee/n)}-1]"1, and / exp[-(l/2)(ß - Zn)'[E^ + {E„ 0 (Ee/n)}"1]^ - Zn))dß
is a constant, it can be shown that
J h(ß)L(TÄn\ß)dß oc exp[ - (l/2){ - Z'n [E^1 + {E„ 0 (Ee/n) }_1]z„
n + ^E-1^ + Em;E-1mi}].
i=i
For the new observation m, let /(m|m„) be the conditional density of m
given mn in the unconstrained case. Then
/(m|m„) = f h(ß)L(iZn+i\ß,mn+i = m,Wb,n+l = We)dß
oc exp
f h(ß)L(xZn\ß)dß
- (l/2){ - Z'B+1 [E^1 + {En+1 ® (Ee/(n + 1)) }_1] Zn+1
+ Z'n [E^1 + {En 0 (Ee/n) }_1] Z„ + m'E-1™}], (A5)
55
where
Zn+1 = R-n+i E^+{l2®(=e/(n + l)) ^(P""^ .mW(n+l)
Rn+1 = E^1 + {En+i ® (Ee/(n +1)) } ,
En+1"Uo,n+l E3ilX'/(» + 1)/ ' n+1
i=i n+1
™»T(n+l) = J2 W0jmj/(n + 1), i=l n+1
^0,n+l = £ ^0j7(" + 1), J=l
with mn+i = m and Wo,n+l = Wc-
Note
m(n+l) = (n/(n + l))«»(n) + (V(n + l))m
m^(„+i) = (n/(n + l))nV(n) + (wc/(n + l))m.
Then
® m. Wc
-li
Therefore the exponential of (A5) is —1/2 times
- M'n+1Rn+iMn+i + Z'n [E^1 + {E„ ® (Ee/n)p]z„ - /I'E"
1/*
(m-^E-^m-zi), (A.6) +
56
where
Mn+i = Rn |j V^ + («A» + 1)) {h ® (WC« + 1)) ' } f ^^
E = So Ee — H l-l
mW(n) / J
Be,
with
/i = E{(l,Wc)®E-1}E/,[E^+{En+1®(Ee/(n + l))}] *
.{En+i®(Ee/(n + l))} E^V/J + nfc^S-1)^ "^ ) , L \ mW(n) ) .
H = {(1, Wc) ® IP}S^ [fy + {En+1 ® (Ee/(n + 1)) }] "1
{EB+i®(Ee/(n + l))}|f ®\v\-
Since the first three terms in (A6) are not function of m, which are constants,
the theorem holds.
Proof of Theorem 2
Note the distribution of nij given ß\ is the multivariate normal with mean
PL + ^Ljßl an(i variance Ee.
h{ßi)I4&n\ßl) oc exp[ - (l/2){(/*i - /ii/E^l " Ml) n
+ E(mJ " ^ - I>Ljßl)'2e\™j -ML- Djyft)}] .(A7) 3=1
The exponential of (A7) is —1/2 times
(ßl ~ Z»y [Sr1 + £ (ü^D^)] (ft - Z„) - Z'n [EJ-I + £ (D^-E-^^)] i=i
n
i=i
i=l
where
Z„ = i-l
V+E (P'LJ^^LJ))' [=rVi+EDi;EeV; -^)' • i=i i=i
57
Hence
J h{ßl)L{^n\ßl)dßl
(l/2){ - Z'n [EJ-1 + £ (D^D^JZ» oc exp
J=l
For the new observation m, let /c(m|ntn) be the conditional density of m
given mn in the constrained case. Then
f h(ßi)L(mn+i\ßi,mn+i = m, W0)n+1 = We)dßi /c(m|m„) =
n+1 ex exp[ - (1/2){ - Z'B+1 [EJ-1 + £ (D^E"1^)] Zn+1
i=i
+ Z'n [^r1 + E (DliEe ^Lj)] Zn + (m - /i^E'V - |IL)}],
(A8)
DX,n+l =
where Zn_|.i is defined as Zn with mn+i = m, and
1 c\ C2 ••• cp_i
,WC ci^c c2Wc ••• Cp-iWc
For A; = l,2,---,n + 1, let Rk = E^Vl + £*=i DijSe Hmj ~ 0L)I and let
Q* = Sr^EjtiCD^E-iD^)- Then with Zn+1 = Rn+D^E-^m-iij), it is easy to show that (A8) is
exp [ - (l/2){ - R'nQ^Rn + Z'nQ„Z„ + (m - HL)' [E"1 - E^D^+iQ"^
■ Di,n+iSe*] (m - /iL) - 2R'nQ^1Di)„+1E-1(m - |iL)}]
oc exp[ - (l/2){(m - /ic/E^m - /ic)}],
where
Ec = [lp - D^+xQ-^D'^E-1] _1Ee,
Hc = ML + VcVe1VL,n+lQnllR«-
The last result is obtained because Qn, Qn+i,Rn, and Zn are not function of m.
58
REFERENCES
Anderson, T. W. (1984), An Introduction to Multivariate Statistical Analysis, John Wiley & Sons, New York.
Fisk, M. D., Gray, H. L., McCartor, G. D. and Wilson, G. L. (1991), "A Constrained Bayesian Approach for Testing TTBT Compliance," F19628-90- C-0135.
Rao, C. R. (1973), Linear Statistical Inference and Its Applications, John Wiley & Sons, New York.
Shumway, R. H. (1991), "Multivariate Calibration and Yield Estimation with Errors in Variables," Air Force Technical Applications Center, Patrick AFB, FL 32925.
Shumway, R. H. and Der, Z. A. (1990), "Multivariate Calibration and Yield Estimation for Nuclear Tests," Unpublished Manuscript.
59
