The Poisson Approximation to the Poisson Binomial DistributionAuthor(s): J. L. Hodges, Jr. and Lucien Le CamSource: The Annals of Mathematical Statistics, Vol. 31, No. 3 (Sep., 1960), pp. 737-740Published by: Institute of Mathematical StatisticsStable URL: http://www.jstor.org/stable/2237582 .

Accessed: 07/12/2014 06:21

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

Institute of Mathematical Statistics is collaborating with JSTOR to digitize, preserve and extend access to TheAnnals of Mathematical Statistics.

http://www.jstor.org

This content downloaded from 128.235.251.160 on Sun, 7 Dec 2014 06:21:46 AMAll use subject to JSTOR Terms and Conditions

THE POISSON APPROXIMATION TO THE POISSON BINOMIAL DISTRIBUTION1

BY J. L. HODGES, JR. AND LUCIEN LE CAM

University of California, Berkeley

1. Introduction. It has been observed empirically that in many situations the number S of events of a specified kind has approximately a Poisson dis- tribution. As examples we may mention the number of telephone calls, accidents, suicides, bacteria, wars, Geiger counts, Supreme Court vacancies, and soldiers killed by the kick of a horse.

Many textbooks in probability content themselves with an explanation of this phenomenon that runs something like this: There is a large number, say n, of events that might occur-for example, there are many telephone subscribers who might place a call during a given minute. The chance, say p, that any specified one of these events will occur (e.g., that a specified telephone sub- scriber will call), is small. Assuming that the events are independent, S has exactly the binomial distribution, say 6B(n, p). If we now let n - o and p -O 0, so that np -* X where X is fixed and 0 < X < *, it is shown that 6B(n, p) tends to the Poisson distribution 6P(X) with expectation X.

As was pointed out by von Mises [4], such an explanation is often not satis- factory because the various trials cannot in many applications reasonably be regarded as equally likely to succeed. Let pi denote the success probability of the ith trial, i = 1, 2, *., n. Then S has the distribution sometimes called "Poisson binomial." Starting from this more realistic model von Mises shows that S has in the limit the distribution 6P(X), provided n -X o and the pi vary with n in such a way that 2pi = X is fixed and a = max{pi, P2, X * pn} tends to 0. This result is given in a few textbooks [1]; [5].

The limit theorem of von Mises suggests that the Poisson approximation will be reliable provided that n is large, a is small, and X is moderate. But even these requirements are unnecessarily restrictive, as may be seen from a general approximation theorem of Kolmogorov [2]. When this theorem is applied to our problem, it asserts that there is some constant C, independent of n and the pi, such that the maximum absolute difference D between the cumulative distribu- tions of S and of 6P(2pi) satisfies the bound D < C a. Thus, the Poisson ap- proximation will be good provided only a is small, whether n is small or large, and whatever value 2pi may have. It seems to us that this is the type of theorem that best "explains" the empirical phenomenon of the "law of small numbers."

The purpose of our note is to present an elementary and relatively simple

Received November 6, 1959. 1 This paper was prepared with the partial support of the Office of Naval Research (Nonr-

222-43). This paper in whole or in part may be reproduced for any purpose of the United States Government.

737

This content downloaded from 128.235.251.160 on Sun, 7 Dec 2014 06:21:46 AMAll use subject to JSTOR Terms and Conditions

738 J. L. HODGES JR. AND LUCIEN LE CAM

proof of a bound of Kolmogorov's type. By using special features of the Poisson distribution, we are able to get the improved bound 3-/_a for D, and to accom- plish this in a good deal simpler way than is required for the general result. We believe that our proof is suitable for presentation to an introductory class in probability theorv.

2. The approximation theorems. Let Xi indicate success on the ith trial, so that P(Xi = 1) -pi and P(Xi = 0) = 1 - pi. Our proofs will be based on the device of introducing random variables Yi that have the Poisson dis- tribution with E( Yi) = pi, and are such that P(Xi = Yi) is as large as pos- sible. Specifically, we give to Xi and Yi the joint distribution according to which

P(Xi = Yi = 1) = -pie-, P(Xi = 1, Yi = 0) = pi(1 -e-pi),

P(Xi =Yi = 0) = e-Pi pi(1-ePi),

and

P(Xi = 0, Yi-y) = pe/y! fory = 2, 3,Y * *.

We let the Yi be independent of each other. (The construction is valid if pi < 0.8, insuring P(Xi = Yi = 0) _ 0. For pi > 0.8 the results below are trivially correct.)

From the familiar additive property of Poisson variables, we know that T = 2Yi has exactly the Poisson distribution P(2pi). Our objective is to show that S = 22Xi has nearly this distribution. Specifically, if we let

D = sup I P(S < u) - P(T ? u) u u

denote the maximum absolute difference between the cumulatives of S and T, we want to find conditions under which D is small.

THEOREM 1. D < 21ps. Using the inequality e-P' 1 - Pi, it is easy to check that

P(Xi s Yi) = 1 + pi - (1 + 2pi)e-P; < 2p2.

Therefore, by Boole's inequality, P(S -4 T) < 2P(Xi $ Yi) _ 2Zp2. But since I P(S < u) - P(T < u) I < P(S $ T), the theorem follows. In order to prove our next theorem, we shall need a uniform bound on the indi-

vidual terms p(k, X)=. e-)Xk/k! of the Poisson distribution. It is well known that for large X, the maximum term is of the order KX, but we will give a spe- cific upper bound.

LEMMA. The maximum term of the distribution GP(X) is less than (1 + 1/12X)/ (27XA)i.

PROOF. Suppose k < X < k + 1. The maximum term is then e -Xk/k!, as may be seen by looking at the ratio of successive terms. Since (X)1e-\Xk is maxi- mized at X = k + 2, and since 1 + 1/12X > 1 + 1/12(k + 1), it will suffice to show that

(27r)1(k + .)k+iek-1 < k![l + 1/12(k + 1)]

This content downloaded from 128.235.251.160 on Sun, 7 Dec 2014 06:21:46 AMAll use subject to JSTOR Terms and Conditions

POISSON BINOMIAL DISTRIBUTION 739

for k = 0, 1, 2, -. This inequality may easily be checked by direct computa- tion for k = 0, 1, and 2, and for k > 3 by using the Stirling bound

k ! > (27r)ikk+?e k+(1/12k)-(1/360k3)

Let us denote 2pi by X and 2p2 by ,. THEOREM 2. D ? (3,u/a 2) + (a + 1)(1 + 1/12X)/(27rX)'. To prove this, we shall consider the random variables Zi -Yi- X.

E(Zi) = 0,

while

Var(Z ) = E(Zg) = pi(l-e ) + k )k! k=2

-pi(l - e-p') + E(Y2) -pie-P = p2 + 2pi(1 - ep') < 3p2.

Let Zi = U. ThenE(U) = OandVar(U) < 3,.

Let a be any positive number. If T = S + U < v - a, then either S _ v or U < -a, so that P(T < v - a) < P(S < v) + P(U < -a) and

P(T ? v) - P(S < v) ? P(v - a < T ? v) + P(U ? -a).

Similarly, if S = T - U < v, then either T ? v + a or U > a, so that

P(S _ v) _ P(T ? v + a) + P(U _ a)

and P(S ? v) -P(T ? v) < P(v < T < v + a) + P(U _ a). Combin- ing, we see that

D = sup I P(S < v) - P(T ? v) v

< sup P(v < T < v + a) + P(I U I _ a).

By the Chebycheff inequality, P(I U I _ a) < Var (U)/a2 ? 3,/a2. Using the lemma, we see that

sup P(v < T < v + a) < (a + 1)(1 + (1/12X))/(27rX)', v

since there are at most a + 1 Poisson terms in the interval from v to v + a. This completes the proof.

We now combine Theorems 1 and 2 to obtain our main result. THEOREM 3. D ? 3 Yao. We prove this by considering two cases. If 2, _ 3Va, the theorem is an im-

mediate consequence of Theorem 1. On the other hand, if 2m > 3-Y/-, we have by virtue of , < aX the inequality X > 3/2a213 > 1. Now suppose that a > 1. Then

[(a + 1)(1 + (1/12X))]/[(2irX)'] < a/XV

This content downloaded from 128.235.251.160 on Sun, 7 Dec 2014 06:21:46 AMAll use subject to JSTOR Terms and Conditions

740 J. L. HODGES, JR. AND LUCIEN LE CAM

and

D < (3AI/a2) + (a/Xi).

This is minimized when a = = ao. Since X > ,q/a and , > 3(a)'/2, we see that A(X)j > (3)312 or ao > (35/2)116 > 1, so the restriction a ? 1 is satisfied, and the theorem is proved.

3. Remarks. (i) We have presented our results as approximation theorems rather than as

limit theorems. We believe it is better pedagogy to do so, since in the applica- tions there will be definite values of n and the pi, which are not "tending" to anything. However, if limit theorems are desired they follow at once. For ex- ample, Theorem 1 implies that D -O 0 as ,u = -p 0, whereas Theorem 3 implies that D -0 as a - max {pi, , pn -?0.

(ii) Our Theorem 1 gives a simple and elementary proof of the standard textbook result that 63(n, p) -@ 6P(X) as n -- o, p -> 0, and np -- X, since under these conditions MP2 = Xp -- 0., Furthermore, Theorem 1 implies the more realistic theorem of von Mises, since if 2pi- X is fixed while a -- 0, we must have lp2 <aSpi = aX -- 0.

(iii) As is customarily the case with bounds for the accuracy of approxima- tions, our bound has only theoretical interest, being much too crude for prac- tical usefulness. By pushing the method of proof, the constant factor 3 in the inequality D < 3-<1a can be reduced, but the result would still be of only theoretical value. It can be shown [3], using a much less simple argument, that D ? 9a. While it is clearly a theoretical improvement to have a bound of order a rather than one of order '/ax, even the bound 9a is of limited applicational use. Fortunately, approximations are usually found in practice to be much better than the known bounds would indicate them to be.

(iv) The condition that a -- 0 is sufficient but not necessary for D -O 0. It is easy to see that S will have approximately a Poisson distribution even if a few of the pi are quite large, provided these values contribute only a small part of the total 2pi .

REFERENCES

[1] WILLIAM FELLER, An Introduction to Probability Theory and Its Applications, Vol. 1, 2nd ed., New York, John Wiley and Sons, 1957.

[2] A. N. KOLMOGOROV, "Deux th6oremes asymptotiques uniformes pour sommes des variables al6atoires," Teor. Veroyatnost. i Primenen., Vol. 1 (1956), pp. 426-436.

[31 LUcCIEN LE CAM, "An approximation theorem for Poisson binomial distributions," to be published in the Pacific J. Math.

(4] R. v. MISES, "tiber die Wahrscheinlichkeit seltener Ereignisse," Z. Angew. Math. Mech., Vol. 1 (1921), pp. 121-124.

[5] RICHARD V. MISES, Wahrscheinlichkeitsrechnung und ihre Anwendung in der Statistik und theoretischen Physik, Wien, Deuticke, 1931.

This content downloaded from 128.235.251.160 on Sun, 7 Dec 2014 06:21:46 AMAll use subject to JSTOR Terms and Conditions