Journal of Statistical Planning andInference 82 (1999) 147–149

www.elsevier.com/locate/jspi

A note about a curious generalization of Simes’ theorem

Ester Samuel-CahnDepartment of Statistics and Center for Rationality and Interactive Decision Theory, The Hebrew

University of Jerusalem, Mount Scopus, Jerusalem 91905, Israel

Received 1 January 1997; accepted 1 December 1997

Abstract

Simes (1986, Biometrika 73, 751–754) proposed a modi�ed Bonferroni procedure for con-ducting multiple comparisons. He showed that when the n test statistics are independent, hisprocedure has exact size �. Here we generalize Simes’ theorem and show that exact size �is obtained also when one considers the order statistics of independent identically distributedrandom variables, each of which appears with any multiplicity. c© 1999 Elsevier Science B.V.All rights reserved.

MSC: 62J15

Keywords: Exact probability statement; Multiple comparisons; Simes’ test; Simes’ weighted test

1. Introduction

Let H = {H1; : : : ;Hn} be a set of null-hypotheses with corresponding test statisticsT1; : : : ; Tn, and p-values P1; : : : ; Pn. Let H0 be the hypothesis that all Hi (i=1; : : : ; n) aretrue. The modi�ed Bonferroni procedure suggested by Simes (1986) rejects H0 if andonly if there exists some value j (16j6n) such that P( j)6j�=n, where P(1)6 · · ·6P(n)are the ordered values of P1; : : : ; Pn. Simes proves that, when Pj are independentlyuniformly distributed on [0; 1], then, for 06�61,

P

n⋃j=1

(P( j)6j�=n)

= �; (1)

which is equivalently stated as

P

n⋂j=1

(P( j)¿j�=n)

= 1− �: (2)

E-mail address: msdoronz@mscc.huji.ac.il (E. Samuel-Cahn)

0378-3758/99/$ - see front matter c© 1999 Elsevier Science B.V. All rights reserved.PII: S0378 -3758(99)00038 -5

148 E. Samuel-Cahn / Journal of Statistical Planning and Inference 82 (1999) 147–149

If the test statistics T1; : : : ; Tn are continuous but not independent, the Pjs will notbe independent, but, for simple {Hj}, they will still be uniformly distributed. In caseswhere the equality sign in (1) can be replaced by6 we say that the test is conservative,and if it is replaced by ¿ we say that the test is anticonservative.Samuel-Cahn (1996) shows that for n= 2 and jointly normally distributed (T1; T2),

for one-sided tests Simes’ procedure is conservative if the Tis are positively correlated,and is anticonservative when they are negatively correlated. Sarkar and Chang (1997)generalize the result, and show that the procedure is conservative for a large classof n-dimensional positively dependent test statistics. This class includes the family ofn equicorrelated multivariate normally distributed variables, with positive correlation.(See example 1, there.) In a quest to generalize this result to any multivariate normallydistributed random variables with positive correlations, we considered the special casewhere some of the correlations are zero and the others are one. After standardizationthis clearly is equivalent to observing just k independent N(0; 1) random variables,where the ith random variable has multiplicity ji¿1 and

∑ki=1 ji = n. Considering the

p-values instead, we obtain k independent Yi; i = 1; : : : ; k, uniformly distributed on[0; 1]. The argument below now applies to any continuous random variables, includingN(0; 1). Let P1; : : : ; Pn be the n random variables obtained by giving Yi multiplicityji; i = 1; : : : ; k, and let P(1)6 · · ·6P(n) be their order statistic. Clearly, if ji ¿ 1 forsome i, then some of the P(j)’s will be equal. The content of the following theorem isthat Simes’ equality (1) remains valid also for the latter P(j)’s. This is rather surprising,since if one was to consider the order statistics Y(1)6 · · ·6Y(k) of the k independentYi’s only, the corresponding statement would be

P

k⋃j=1

(Y( j)6j�=k)

= �; (3)

that is, the constants are now j�=k and not j�=n. (If all ji = s and so n= ks then (1)and (3) are equivalent.)

Theorem. Let Yi; i = 1; : : : ; k; be independent random variables uniformly distributedon [0; 1]. Let P1; : : : ; Pn be the random variables obtained by generating ji identicalreplications of Yi; i = 1; : : : ; k; ji¿1; where n =

∑ki=1 ji; and let P

n(1)6 · · ·6Pn(n) be

their order statistics. For any 06�61; (1) (and; equivalently (2)) holds.

2. The proof

We shall prove the theorem by induction on n. For n=1; (2) is trivial. Now suppose(2) is true for all n6m− 1. We shall prove it for n= m.If k = 1; the left-hand side of (2) is just P{Y ¿�} and (2) is again trivial. Hence

assume k¿2. We shall evaluate the left-hand side of (2) by conditioning on the valueof Pm(m) = max{Y1; : : : ; Yk}, the density of which is kxk−1I (06x61). In order for (2)to hold we must consider x¿� only. Let R be the random index for which Pm(m) = Yi.

E. Samuel-Cahn / Journal of Statistical Planning and Inference 82 (1999) 147–149 149

Then R takes on the values 1 to k each with equal probability 1=k, and is independentof the value of Pm(m). Thus, the value of the left-hand side of (2) for n = m can bewritten as

1k

k∑i=1

∫ 1

�P(Pm(1)¿

�m; : : : ; Pm(m−ji)¿

(m− ji)�m

|Pm(m) = x; R= i)kxk−1 dx: (4)

Now Pm(1); : : : ; Pm(m−ji); given P

m(m)=x, behave like m−ji order statistics generated by k−1

independent random variables uniformly distributed on [0; x]. Since ji¿1 it thereforefollows by the induction hypothesis (and k¿2) that the value of (4) equals

1k

k∑i=1

∫ 1

�

{1− (m− ji)�

mx

}kxk−1 dx= 1− �k −

k∑i=1

(m− ji)�m(k − 1) (1− �

k−1)

= 1− �k − (km− m)�m(k − 1) (1− �

k−1) = 1− �;(5)

where the next to last equality in (5) uses∑k

i=1 ji = m.

3. Generalizations

Suppose test i has weight !i¿0; i=1; : : : ; k, and let i1; : : : ; ik be the random orderingof the tests corresponding to the ordering of the p-values. Consider the (conditionally)modi�ed Simes’ procedure where, in (3), j�=k is replaced by �

∑jt=1 !it =

∑ki=1 !i. As

has been pointed out by a referee, to whom we are very grateful, this procedure stillhas an exact error rate � when the k tests are independent. The proof is essentiallythe same, the induction is now on k, and conditioning on ik . This result has earlierbeen obtained in Benjamini and Hochberg (1997) (see Theorem 3). The connection tocomplete dependence was not noted there.

References

Benjamini, Y., Hochberg, Y., 1997. Multiple hypotheses testing with weights. Scand. J. Statist. 24, 407–418.Samuel-Cahn, E., 1996. Is the Simes improved Bonferroni procedure conservative?. Biometrika 83, 928–933.Sarkar, S.K., Chang, C.-K., 1997. The Simes method for multiple hypothesis testing with positively dependenttest statistics. J. Amer. Statist. Assoc. 92, 1601–1608.

Simes, R.J., 1986. An improved Bonferroni procedure for multiple tests of signi�cance. Biometrika 73,751–754.