411 IEEE TRANSACTIONS ON RELIABILITY, VOL. 45, NO. 3, 1996 SEPTEMBER

Hierarchical Bayes Estimation for the Exponential-Multinomial Model in Reliability and Competing Risks

Alex S. Papadopoulos

Ram C. Tiwari

Jyoti N. Zalkikar

University of North Carolina, Charlotte

University of North Carolina, Charlotte

Florida International University, Miami

Keywords - Bayes risk, Gamma distribution, Dirichlet distribution, Quadratic loss, Multivariate normal distribution, Gibbs sampler.

Summary & Conclusions - The exponential-multinomial distribution arises from: 1) observing the system failure of a series system with p Components having independent exponential lifetimes, or 2) a competing-risks model with p sources of failure, as well as 3) the Marshall-Ob multivariate exponential distribution under a sen” sampling scheme. Hierarchical Bayes (HB) estimators of the component sub-survival function and the system reliability are obtained using the Gibbs sampler. A large-sample approximation of the posterior pdf is used to derive the HB estimators of the parameters of the model with respect to the quadratic loss func- tion. The exact risk of the HB estimator is obtained and is com- pared with those corresponding to some other estimators such as Bayes, maximum likelihood, and minimum variance unbiased estimators.

1. INTRODUCTION

Acronyms

MLE maximum likelihood estimator U M W E uniformly minimum variance unbiased estimator EM exponential multinomial HB hierarchical Bayes .

Notation

9( 0 )

p , n Y , J

Zl

indicator function: S(true) = 1, S(fa1se) =O number of [components, samples] lifetime of component j in sample i, j = l , . . . ,p ; i= I , ..., n min(Y,,, ,..., Y, ,p) , 0 < 2, < 03

T 21

1 = 1

M i J (‘1 = x,J) 4 6 MIJ

2=1

j = 1

j = 1

.pyp(A) - l , 0 < y, v < 03 //v/I2 v’v for any p x 1 vector v ET{. ] s-expectation with respect to the joint distribution of

8 and ( T , N ) rB(O*) Bayes risk: Et{l18* - Oil'} - implies: distributed as r(h) (Vl(h),... . VP(h))’, 0 < V] < 1,-j=1,.. . ,p, r ’ (h&

= 1

Dirichlet pdf with parameter vector v(h) D ( ~ ( h - >>

’The singular & plural of an acronym are always spelled the same

0018-9529/96/$5.00 01996 IEEE

478 ’ IEEE TRANSACTIONS ON RELIABILITY, VOL. 45, NO. 3, 1996 SEPTEMBER

V (VI, ..., vp)’, 0 < Y, < 03

P

VJ V

J = 1 XI Y - implies: conditional on Y, X is distributed ad betd(a,b) pdf of a beta distribution with parameters a & b.

Other, standard notation is given in “Information for Readers & Authors” at the rear of each issue.

Nomenclature

* Sub-distribution function, F , ( z ) . Pr(ZL 5 z , M L J = l , for k ~ j } .

Incidence reliability function, S J ( z ) . f‘r(z, > Z, M L J = I , Mz,k=O, for k f j } .

* System Sf, S(Z). Pr(Z > z } . -4

Assumptions

1-1. Component lifetimes rJ - expd(l/Oj) and are

1-2. The prior distribution for (X, p) is gamma-Dirichlet

-4 Under assumption 1-1, the pdf{Z,M}, at ( z ,m) , is given

by fp (z,m 18) ; it is referred to as EM pdf with parameters p & 8, and is written as EM(p,B). Typically p is known and 8 is unknown. For a multivariate exponential distribution, estima- tion of 8 has been discussed in [2,3,8]; a Bayes approach to estimating 13 is in [6,9]. Ref [5] showed that the MLE & UM- VUE of 8 are s-inadmissible, and developed s-adaptive estimators.

From 1( A, p ) , the ( T,N) is jointly s-sufficient for (X,p). Under assumptions 1-2 & 1-3:

s-independent .

pdf n(X, PI. 1-3. v ( X ) =v, v(X) =v.

From (1-l), h & p are s-independent and, using the transformation 19 = Ab, 81,. . . ,ep are s-independent r.v. with OJ - gamd(y; vJ) , j = 1,. . . ,p. The gamma-Dirichlet family (1-1) is a natural conjugate family for (A, p ) . A posteriori, 0, are s-independent with OJ - gamd(y+ T; NJ+ vJ), j = 1 ,..., p . Under the quadratic loss function,

which is a linear combination of the prior guess (v iy) of 0 and the MLE 6 = N / T . In (1-3) if y - 0 in such a way that (v/y) remains fixed, then - 8 with probability 1. Straight- forward calculations yield:

= c.6, and c = a positive constant.

&, with c= 1, and ( n - 1 ) / n correspond to the MLE and UM- VUE respectively. Whenever ( ( n - 2 ) / n ) < c1 < c2, then rB(&c, ) < r B ( J c * ) .

Under the prior ( 1 - 1 ) :

To reduce the lmension of the vi that need to be specified and to include the dependency of ~ ( h ) on X, section 2 considers the HB approach, wherein only the hyper-parameters v & y are known, and the uncertainties in ‘v (X) given X’ are modeled by another Dirichlet pdf. In practice, it is often easy to elucidate the hyper-parameters y & v of the prior for A, eg, through its mean & variance.

2. HIERARCHICAL BAYES ESTIMATION

Assumptions

2-1. (Z,M) - EM(p,8) 2-2. Same as 1-2. 2-3. q(X) - D(1,). -4

Reparameterize v(X) as v ( X ) = v(X)q(X), where q(h) follows assumption 2-3. Under the HB model (assumptions 2-1 - 2-3), the full conditional posterior pdf are:

E{PJX) = v ( A ) / v ( X ) = q(h), and under assumption 2-3,

E{P} = l p /p , ie, the cause-specific hazard rates are equally likely. The Gibbs sampler [lo] is implemented to find the HB estimators of 8, Sj, S.

2.1 Implementation of Gibbs Sampler for EM ( p , @ Model

Assumptions

2.1-1. p = 2. 2.1-2. VI@) = V I , vz(X) = v2, v(h) = Y , s-independent

of A. 2.1-3. v ~ ( X ) = v 1 . S v ~ ( X ) = v ~ ’ X , ~ ( h ) = v.A. -4

PAPADOPOULOS ET A L HIERARCHICAL BAYES ESTIMATION FOR THE EXPONENTIAL-MULTINOMIAL MODEL 479

Case I. Under assumptions 2.1-1 & 2.1-2,

E 71 = Vl/V,

72(A) = 1-71;

and of the estimates of S1 & S2 when y= 1 and v = 2. Since the simulation results for a case 111 where v(X) = v + Xl, and v ( A ) = v + Asp did not provide any additional insight, those graphs are not reported.

q l - betd(1, 1).

Case 11. Under assumptions 2.1-1 & 2.1-3,

V,(A) = v 1 ( A ) / v ( A ) = U l h ,

2 0 0 - 72(A) = 1-9dA) = 1-71;

- betd(A-u.Tl, A . v . ( l - q , ) ) ,

q l - betd(1, 1).

The vl(X) & vz(A) depend on A, but ql(A) & vZ(A) are

To derive estimates of the posterior pdf of 01, 02, ql , q2, S, ( z ) , computer simulation was performed according to

100 -

s-independent of A. 4

algorithm 1.

Algorithm 1

1. A value of A is generated from a gamd(y, v). Values for 71 are generated from betd( 1, 1); values for p1 are generated from -

betd(v.ql, ~ - ( 1 - - 7 ~ ) ) for case I,

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

- estimated posterior pdf{Ol I T i N }

______ estimated posterior pdf (0, I T,N}

Figure la. Case I: y = l , v = 2

betd(A-v.ql, h ~ v . ( l - r ] ~ ) ) for case 11.

Then p2 = 1-Pl.

2. A random sample Zl,...,Zn is generated from an expd(A); T i s computed.

3. A value for Nl is generated from binm(. ; p1 ,n) . N2 = n - N 1 .

4. Use as initial values the generated values of A, q l , q2, pl, p2; use 100 iterations to generate values of A, P1, P 2 , 71, q2 from the posteriors (2-1).

5. Step 4 is repeated lo3 times; the values of -

112 = 1 - 71,

2 5 0 1

S j ( z ) for different z , are all stored. 6 . The stored values are used to draw smoothed S j ( z ) &

4

Figure l a shows the smoothed estimates of posterior pdf of O1 & e2; figure lb shows the smoothed estimates of posterior pdf of v l & v2. Figure IC shows the estimates of S1 & S2 for case I when y = 1 and v = 2. For case 11, figures 2a - 2c pre- sent the smoothed estimates of posterior pdf of 01, 02, 71, 72,

pdffor 81, 62, 71, 112- 0 -. .-. I . . _ _ .

-..___ .-_ I I I I I I I I I I

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

- estimated posterior pdf{ql I T,N} estimated posterior pdf(q2 I T,N}

Figure l b . Case I: y = l , v = 2

______

480

0.6

0.5

0.4

0.3

0.2

0.1

0

IEEE TRANSACTIONS ON RELIABILITY, VOL. 45, NO. 3, 1996 SEPTEMBER

16C

14C

12c

100

80

6 0

40

20

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 o I) i n I I I 1 I I

I I I - estimated posterior pdf(ql I T,N} 0 0.9 ______ estimated posterior pdf(7, I T,N) 0.3 0.8

. A , \ __ esrimatea 2, ( t i ______ estimated i2 ( t )

Figure IC. Case I : ? = I , u=2

Figure 2b. Case II: y = 1, U = 2

2 0 0

150

100

5 0

0

0.0 0 1 0 2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2

~ estimated posterior pdf{O, 1 T,N} ---___ estimated posterior pdf(0, I T,N}

Figure 2a. Case II: ? = I , v = 2

0.E

0.4

0.2

0.2

0.1

0 0 1.5

~ estimated 3, ( t ) estimated $, ( t )

Figure 2c. Case II: y = I , = 2

______

3.0 4.5

PAPADOPOULOS ET A L HIERARCHICAL BAYES ESTIMATION FOR THE EXPONENTIAL-MULTINOMIAL MODEL 481

2.2 Large-Sample Approximation of the Posterior pdf of q

Assumptions

2.2-1. X & P are s-independent with X - gamd(y, v), and P - D(v) .

2.2-2 q - D(lp ) . 4

Under assumptions 2.2-1 & 2.2-2, and reparameterization of v as v.q, the transformation 8 = X/3 yields the prior for 8 as a mixture of gamd(y, vq,) pdf with the mixing pdf D ( lp). Under the loss function (1-2), the mean of the posterior pdf of 8 (the HB estimator of 8) is:

Thus to obtain &, we need the posterior mean of q given the data ( T , N). After some routine calculations, we get the posterior pdf of (ql,. . . , q P - l ) given ( T, N ) as:

(2-3)

U- 1

J = 1

Theorem 1 gives the large-sample approximation to the posterior pdf of q.

Theorem 1. L e t j = l , ..., p-1. Let (qr ,..., v ; - ~ ) ' , 0 < q;< 1, CfZl1 Yl; < 1, be a non-null vector around which the posterior pdf of (ql,...,qpPl)' is centered. Then, for large n , the posterior pdf of (ql , . . . ,qp - ) ' is approximately a ( p - 1)- dimensional multivariate s-normal pdf with mean vector p =

(PI,. . . , pp - 1 ) ' and variance-covariance matrix %1)X (p-l)2 where:

Pj = - (bj/2aj), (2-4)

c-' = ((a')),

. . a l ' l - - -2ai,

tJ N,/n,

P-1 4 n/v, Gp = 1- tJ = ( N p / n ) - 7;

J = 1

P - 1

= 1- q;. ] = I

Pro08 See appendix A-1.

4

The mean vector in (2-4) is a function of the vector q* = (q?,. . . ,$) ' around which the posterior pdf is centered, the vector f j = ( f 1 , ...,$,)I of the sample proportions, and the parameter v. In most of the known s-normal approximations of the posterior pdf [ 1,4], q* is chosen to be the posterior mode.

Corollary 1. Let 7;" = f j J . Let j = 1,. . . ,p - 1. For large n , the posterior pdf of (ql,. . . ,qp- ' is approximately multivariate s-normal with mean vector (ql, . . . ,7jp- ) ' and the variance- covariance matrix E = ( ( u ~ , ~ ) ) ,

aiJ = - ( v + n ) .7ji.7jjl(v.n), i#j . 4

Pro08 See appendix A-2.

Lemma 1 is a straightforward consequence of Corollary 1.

Lemma 1. In HB, under the conditions of corollary 1 for large n, the posterior mean vector and variance-covariance matrix of 8, given (T ,N) , are:

Cov {8,8 'I ( T,N) } = (v + n ) . [(Y + n ) . Diag { fj l,. . . ,ep}

From Lemma 1,

4

P = E ' { ( ~ + n ) [ ( ~ + n ) * r j j - v.$$(.* ( Y + T ) ~ ) ) .

J = 1

Take s-expectation with respect to all the r.v. involved and simplify:

482 IEEE TRANSACTIONS ON RELIABILITY, VOL. 45, NO. 3,1996 SEPTEMBER

The Bayes risk of e^, = c .6 is: & B * j = (N, + ~ * p j ) / ( y + T ) ,

+ n.c2 + The average r ( tHB*) over lo3 values is reported in table

1, column 3. Column 4 gives rHB(6HB) using (2-8); rHB(#HB*)

> rHB( &B). This justifies our choice of ( j (as qj7 in corollary 1 a n d l a m " m .

$l(c,n) = n . ( n - l ) . ( c - 1 ) 2 +- 2n. (c -1) + 2,

G2(c,n) = ( n . ( c - l ) + 1 ) 2 + n . ( c - l ) + c - n + 1,

c is a positive constant.

As in section 1, (%-9) with the appropriate choice of c gives the risk of the MLE and the UMVUE in HB.

Let j = 1,. . . ,p - 1 ; and see Notation in section 2.3 ' 'Ex- ample". The JHB in (2-7) and rHB(&B) in (2-8) are based on setting qT(theorem 1) to be qY One could use mod(qj) for q,?, and compute In that case, results equivalent to corollary 1 and lemma 1 can not be obtained analytically; hence we resort to numerical computation.

2.3 Example2

Notation

mod(qj) posterior mode of q, eHB* HB estimator of 8.

Let the system be I-out-of-2:F ( p = 2 ) and y =1; j=1,2.

For each v = 2,5,10, and for given n=10(10)50 perform algorithm-2 lo3 times.

Algorithm 2

1. Generate 91 from betd(1, 1); calculate,

2. Generate 0, from gamd(v,, l ) , and compute = 0,/h,

3. Generate M = (N1,N2) ' from multinomial pmf with

4. Maximize (2-3) with respect to q l ; obtain mod(ql);

A = 01+e2.

parameters IZ & 0; generate T from gamd(n, A).

calculate,

from theorem 1

'The number of significant figures is not intended to imply any ac- curacy in the estimates, but to illustrate the arithmetic.

Table 1. Bayes Risks rHB(&B*) and rHB(&B)

@=2, y = l ]

2 10 20 30 40 50

5 10 20 30 40 50

10 10 20 30 40 50

0.5188 0.2759 0.1906 0.1568 0.1157

2.7296 1.7924 1.0951 0.6748 0.6059

14.2077 13.5433 5.4393 4.0156 2.8186

APPENDIX

A.l Proof of Theorem 1

Notation

0.4800 0.2664 0.1844 0.1410 0.1142

2.1094 1.2300 0.8706 0.6743 0.5503

6.6667 4.0591 2.9449 2.3 162 1.9104

Use the Stirling approximation,

to the gamma functions in (2-3) and simplify.

PAPADOPOULOS ET AL: HIERARCHICAL BAYES ESTIMATION FOR THE EXPONENTIAL-MULTINOMIAL MODEL 483

+ O{n-'}]

For large n & v, the O(n- ' ) is negligible in (A-1) and may be ignored without losing much in the approximation. Use a Taylor

= (qr ,..., qa'. Expand the log functions. 1 p-1 p-1

series expansion of the exponent in (A-1) around the vector q* '5-l. [ ( t * f j p + vi ) - '* [ 2 2 (qZ-q;Y)' (qJ-qj*)

+ o{(T;-$)21; (A-2) c2 is a constant not depending on qy

Sum (A-4) over index j , add (A-5), and then multiply by n. The approximation to the exponent in (A-1):

c1 is a constant independent of q;,

D- 1

j= 1

Therefore,

Thus the posterior pdf of (ql , . . . ,qp- 1) ' in (A-1) is approx- imated by:

exp[ 5 aj-[qj - (77 - ?hbj/aj) ;=l

Q. E. D

Letj=1,2 ,..., p-1. If q? = fj,, then b,=O and p, = 7iJ. Also,

E-' = ( n . v / ( n + v ) )

.[ Diag{Gl- ',...,

Diag{dl,. . . ,dk} is a diagonal matrix with elements dl,.. .,dk Q.E.D.

} + fj;llp.-ll>-l],

Use [7: (2.5.6)] to obtain C in the required form.

484 IEEE TRANSACTIONS ON RELIABILITY, VOL. 4.5, NO. 3, 1996 SEPTEMBER

r41

REFERENCES

J.A. Achcar, A.F.M. Smith, “Aspects of reparameterization in approx- imate Bayesian inference”; see [ l I]. B. Arnold, “Parameter estimation for a multivariate exponential distribu- tion”, J . Anzer. Statistical Assoc, vol 67, 1968, pp 928-929. B. Bemis, L. Bain, J. Higgins, “Estimation and hypothesis testing for the parameters of bivariate exponential distribution’ ’ , J. Amer. Statistical

R.E. Kass, L. Tierney, J.B. Kadane, “The validity of posterior expan- sions based on Laplaces’s method”; see [ l l ] . E. Pefia, “Improved estimation for a model arising in reliability and com- peting risks”, J . Multivariate Analysis, vol 36, 1991, pp 18-34. E. Peiia, A. Gupta, “Bayes estimation for the Marshall-Olkin exponen- tial distribution”, J. Royal Statistical SOC, vol B52, 1990, pp 379-389. J. Press, Applied Multivariate Analysis: Using Bayesian and Frepen- tist Methods of Inference (2nd ed), 1982; Robert Krieger. F. Proschan, P. Sullo, “Estimating the parameters of a multivariate ex-

ASSOC, VOI 67, 1972, pp 927-929.

AUTHORS

Dr. Alex S . Papadopoulos; Dept. of Mathematics; Univ. of North Carolina; Charlotte, North Carolina 28223 USA. Internet (e-mail) : apapdpls@unccvx. uncc . edu

Alex S. Papadopoulos received the BS (1968) in Electrical Engineering and the MS (1969) in Mathematics from the University of Rhode Island and the MS (1971) and PhD (1972) from Virginia Tech, both in Statistics. He is a Professor in the Dept. of Mathematics at the University of North Carolina at Charlotte, and has published several technical articles in statistics & reliability related journals.

Dr. Ram C. Tiwari; Dept. of Mathematics; Univ. of North Carolina; Charlotte, North Carolina 28223 USA. Internet (e-mail): fmaOOrct@unccvm.uncc.edu

Ram C. Tiwari is a Professor & Chair’n of the Mathematics Depart- ment at the University of North Carolina at Charlotte. For biography, see IEEE Trans. Reliability, vol 41, 1992 Dec, p 607.

.. - .

ponential distribution”, J. Amer. Statistical Assoc, vol 71, 1976, pp 465-472.

tion for a bivariate exponential distribution”, J. Economics, vol24, 1984,

R.C. Tiwari, Y . Yang, J.N. Zalkikar, “Bayes estimation for the Pareto failure-model using Gibbs sampling’’, IEEE Trans. Reliability, vol 45, 1996 Sep.

[l 11 Bayesian and Likelihood Methods in Statistics and Econometrics (Geisser, Hodges, Press, Zellner, Eds), 1990; North-Holland.

Dr. Jyoti N. Zalkikar; Dept. of Statistics; Florida International Univ; Univer-

Internet (e-mai1): zalkikar@servms.fiu.edu Jyoti N. Zalkikar is an Associate Professor at the Florida International

university. For biography, see IEEE Trans. Reliability, vol41, 1992 Dec, p 607.

Manuscript received 1996 June 30

Publisher Item Identifier S 0018-9529(96)07362-9

[9] A. Shamseldin, S . Press, “Bayesian parameter and reliability estima- sitY Park; Miami, 33199

pp 363-378. [lo]

+TRb

MANUSCRIPTS RECEIVED MA NUSCRIP TS RECEIVED MANUSCRIPTS RECEIVED MANUSCRIPTS RECEIVED

“Moments of order statistics from the extreme-value distribution under t y p e 4 progressive censoring” (U. Balasooriya, et al.), Dr U. Balasooriya e Dept. of Math & Statistics * Nat’l Univ of Singapore * Kent Ridge - 119 260 e Rep. of SINGAPORE. (TR96-093)

“An exact compound-Poisson distribution for loss-of-load in power-generation reliability” (M. Sahinoglu, et al.), Dr. Mehmet Sahinoglu, Professor * Dept. of Statistics * Dokuz Eylul University Isciler Cad No. 143 35230 Alsancak-Izmir 35230 * TURKEY. (TR96-099)

“Degradation of InP-based solid-state laser diodes: Comparison o f field data to accelerated aging data” (J, Osenback, et J, W. Osenback. Microelec- “Optimal maintenance decisions over unbounded horizons on the basis of

exPertjudgment” (J. Noomijk), Jan M. van NoomQk 0 POBOX 2120

HKV tronics Optoe~ectronic Ctr . Lucent Technologies . 9999 Hamilton Blvd . ~ ~ ~ i ~ i ~ ~ ~ i l l ~ , pennsylvania 18031 , USA, (TR96-101) 8203 AC Lelystad - The NETHERLANDS. (TR96-095)

“Testing the stochastic order and the IFR (DFR), NBU (NWU) ageing classes39 (J, Ruiz, et Mathematics

Dr, Jose M, Ruiz, Professor . “MTBF, M n R , availability for a repairable system” (A. Liu, et 4 , Aimin Liu - Dept. of Electronics - Peking Univ. * Beijing - 100 871 * P.R. CHINA. Statistics . Murcia University, Apdo, 4021 . 30100

SPAIN. (TR96-096) (TR96-103)

“Reliability and redundancy-optimization for a large compound system” (2. Tan), Dr. Zhongfu Tan 700 - Harbin - 1 SO 008 - P.R. CHINA. (TR96-097)

“Correction to: Reliability of a consecutive-k-out-of-r-from-n:F system” (T. Szantai, et al.), Dr. Tamas Szantai - Dept. of Mathematics - Technical Univ. * Budapest, Muegyetem rkp 3, 11 1 1 * HUNGARY. (TR96-098)

“Redundancy optimization for series-parallel multi-state systems” (A. Lisnianski, D. Elmakis, et al.), Anatoly Lisnianski * Reliability Dept. Israel Electric Corp. * POBox 10 * Haifa 31 000

“Finding a minimal test-set for analog fault-diagnostic dictionary” (C. Shi), Dr. C.-J. Richard Shi * Dept. of Electrical & Computer Eng’g * Univ. of Iowa

Iowa City, Iowa 52242 a USA. (TR96-105)

Inst. of Eng’g Theory & Application . ISRAEL. (TR96-104)