Bayesian optimal designs for estimating a set of symmetrical quantiles

STATISTICS IN MEDICINEStatist. Med. 2001; 20:123–137

Bayesian optimal designs for estimating a setof symmetrical quantiles

Wei Zhu1;∗;† and Weng Kee Wong2

1Department of Applied Mathematics and Statistics; State University of New York; Stony Brook;NY 11794-3600; U.S.A.

2Department of Biostatistics; University of California; Los Angeles; CA 90095-1772; U.S.A.

SUMMARY

We propose multiple-objective Bayesian optimal designs for the logit model. As an example, we consider thedesign problem for estimating several percentiles with possibly unequal interest in each of the percentiles.Characteristics of these designs are studied and illustrated for the case when the interest lies in estimatingthe three quartiles. We compare these optimal designs with the sequential designs generated via a generalizedP�olya urn model and found the latter to be highly e�cient. In addition, comparisons are made between locallyoptimal designs and Bayesian optimal designs. Copyright ? 2001 John Wiley & Sons, Ltd.

1. BACKGROUND

1.1. Introduction

A quantal dose–response experiment is typically used to study the relationship between the doselevel of a drug and the probability of a response. A popular model is the simple logit model

log�(x)

1− �(x) = �(x − �)

where �(x) = 1=[1+exp(−�(x−�))] is the probability of a response at dosage x ∈ � and � is thedose range of interest. The parameter � is the slope in the logit scale and � is the dose level atwhich the response probability is 0:5. The parameter � is often referred to as the ‘median e�ectivedose’ and denoted by ED50. More generally, we let ED100� denote the dose level x at which theprobability of a response is �. This means that the 100� percentile or ED100� is equal to �+ =�,where = log[�=(1− �)].

∗ Correspondence to: Wei Zhu, Department of Applied Mathematics and Statistics, State University of New York, StonyBrook, NY 11794-3600, U.S.A.

† E-mail: [email protected]

Contract=grant sponsor: NIH; contract=grant number: R29 AR44177-01A1

Received December 1998Copyright ? 2001 John Wiley & Sons, Ltd. Accepted February 2000

124 W. ZHU AND W. K. WONG

Previous design work for this model focuses on the problem of constructing a single-objectiveoptimal design [1–6]. Frequently, the objective is to estimate ED50 [1; 3; 6] but other percentilesmay be of interest as well. For instance, in heart de�brillator design problems, the ED95 is a moreinteresting percentile to estimate [2; 4].Sequential or adaptive designs have also been developed for estimating quantiles [7–9]. Rosen-

berger and Grill [10] described a neurological stimulus–response sequential experiment where thegoal is to study the relationship between stimulus level and response by estimating quantiles ofthe stimulus–response curve. The principal goal is to estimate the median of the stimulus–responsecurve, and the secondary goals are to estimate the 25th and the 75th percentiles. This is an ex-ample of a three-objective quantal dose–response experiment. The various goals may or may notbe of equal interest. For example, depending on whether there is interest in estimating ED25 andED75, researchers may seek a design which is optimal for estimating:

1. ED25 and ED75 with equal interest subject to the constraint that the design is at least 90 percent e�cient for estimating ED50 (example 1);

2. ED50 subject to the constraints that the design is at least 65 per cent e�cient for estimatingboth ED25 and ED75 (example 2);

3. ED75 subject to the constraints that the design is at least 90 per cent e�cient for estimatingED50 and 60 per cent for ED25 (example 3);

4. ED50 subject to the constraints that the design is at least 50 per cent e�cient for estimatingED25 and 80 per cent for ED75 (example 4).

The aim of this paper is to construct multiple-objective optimal designs in a binary experimentfor estimating several percentiles simultaneously with possibly unequal interest in each of thepercentiles. We focus on the design problem of estimating the three quartiles in a psychophysicalexperiment described in Rosenberger and Grill [10] but the method applies to any set of symmetricpercentiles. In addition, we compare locally optimal designs and sequential designs based on ageneralized P�olya urn model [10] with Bayesian multiple-objective optimal designs.

1.2. Single-objective Bayesian optimal designs

We assume that the total number of observations N for the experiment is determined in advance,usually by cost or time constraint. Any design � in this paper can be represented in the form

�={x1 x2 · · · xKp1 p2 · · · pK

}

where K is the number of dose levels and pi is the proportion of subjects assigned to dose level xi.Each of the xi is selected from a predetermined interval � representing the dose range of interest.This design assigns roughly Npi subjects to the dose level xi for each i, i = 1; 2; : : : ; K .Let y be the indicator of a response at dose level x and let �T = (�; �). Since the probability of

observing a response at dose level x is �(y = 1 | x; �) = �(x), it follows that the (i; j)th elementof the observed Fisher information matrix M (�; �) of a design � is

[M (�; �)]ij = −∫

@2

@�i@�jlog (�(y | x; �))�(dx) (1)

Chaloner and Larntz [11] argued for using a prior distribution on the parameters � to constructBayesian optimal designs. The prior can be obtained by elicitation. The design criteria have the

Copyright ? 2001 John Wiley & Sons, Ltd. Statist. Med. 2001; 20:123–137

BAYESIAN OPTIMAL DESIGNS FOR ESTIMATING SYMMETRICAL QUANTILES 125

form �(�) = E�[M (�; �)], where is a convex function of the information matrix. For instance,if we wish to estimate a given function of �, say c(�), we set

�(�)=E�[(∇c(�))TM−1(�; �)(∇c(�))]where ∇c(�) is the gradient of c(�) and the expectation is over the prior distribution of �. TheBayesian c-optimal design is the one that minimizes �(�) over the set of all designs on �. If thetotal sample size in the study is N , the variance of the estimated c(�) is �(�)=N and so the optimaldesign has the smallest expected variance for estimating c(�). In practice, the optimal design isfound from any of the standard computer algorithms used for generating a single objective optimaldesign. The optimality of the design is veri�ed using the Bayesian general equivalence theorem(BGET) described in the Appendix.As an example, consider �nding an optimal design for estimating ED100�. We set c(�)=

ED100� and (∇c(�))T = (1;− �−2). For the design � given above, a direct calculation shows thatits Fisher information matrix (1) is

M (�; �)=[

�2t −�t(�x − �)−�t(�x − �) s+ t(�x − �)2

]

where t=∑K

i=1 piwi, wi= �(xi)(1 − �(xi)), �x= t−1∑K

i=1 piwixi and s=∑K

i=1 piwi(xi − �x)2. Itfollows that the Bayesian c-optimality criterion for estimating ED100� is

� (�)=E�{�−2[t−1 + ( − �(�x − �))2�−2s−1]}Note that a subscript = log[�=(1 − �)] has been added to emphasize the dependence on thepercentile to be estimated. For example, substituting =0 yields the design criterion for estimating� alone. Likewise, substituting = − log 3 or log 3 gives us the design criterion for estimatingED25 or ED75.The design e�ciency of an arbitrary design � relative to the c-optimal design �∗ is de�ned as

E(�)=�(�∗)=�(�)This ratio compares the expected variances for the estimated c(�) given by the two designs. It is in-dependent of the sample size N and 0¡E(�)61. Since the expected variance �(�)=N =�(�∗)=[NE(�)], designs with high e�ciencies are desirable because they provide more accurate estimates fora given sample size. Speci�cally, we need, for example, two replicates of a design with e�ciency0.5 in order to achieve the same variance given by the optimal design.

2. MULTIPLE-OBJECTIVE OPTIMAL DESIGNS

2.1. An equivalence result

Suppose there are m objectives in the study and each of them is represented by a convex cri-terion �i ; i=1; : : : ; m. Two approaches have been suggested for constructing a multiple-objectiveoptimal design. One way is to �nd a compound optimal design [12; 13] and the other is to �nda constrained optimal design [14]. The compound optimal design is the design which minimizesthe convex combination

∑mi=1 �i�i(�), where each �i ∈ [0; 1] is user-selected and

∑mi=1 �i=1.



For each convex combination, the compound optimal design is found using any of the designalgorithms for generating a single objective design. Let Ei(�) denote the e�ciency of a design �relative to the ith objective, and assume that the �rst objective is the least important one amongthe m objectives. The constrained optimal design is the design which maximizes E1(�), subject tothe constraints that Ei(�)¿ei, i=2; 3; : : : ; m, and each ei is a user de�ned constant between 0 and1. In practice, these constants are assigned larger values for the more important objectives in thestudy.Compound optimal designs are easier to generate, however, the choice of the weights in the

convex combination is problematic. On the other hand, constrained optimal designs are harderto �nd but easier to interpret because of the intuitive formulation of the design problem. Cookand Wong [12] considered linear models when there are two objectives and established that everyconstrained optimal design is a compound optimal design and vice versa. Clyde and Chaloner [13]extended this equivalence result to non-linear models where there are two or more objectives andBayesian optimal designs are sought. This result enables us to �nd the constrained optimal designfrom the more easily available compound optimal designs. For two-objective design problems, thisis achieved by �rst plotting the two e�ciencies of the compound optimal designs versus values of� between 0 and 1. The desired constrained optimal design is then determined from the e�ciencyplot as explained in Section 3.1.

2.2. Compound optimal designs for several percentiles

Let [T = (�1; : : : ; �m) where each �i ∈ [0; 1] is user-selected and∑m

i=1 �i=1. The compound opti-mality criterion for estimating m percentiles, ED100�i; i=1; : : : ; m, can be expressed as

�(� | [)=m∑i=1�i� i(�)

where i= log[�i=(1 − �i)], i=1; 2; : : : ; m. For �xed [, the compound optimal design �[ is thedesign that minimizes �(� | [). In what is to follow, for �xed [; we let ei denote the e�ciencyof the compound optimal design for estimating ED(25i), i=1; 2; 3.We assume that the priors for � and � are independent and without loss of generality that the

prior expectation of � is 0. Under this setting, we justify in the Appendix that the compoundoptimal designs have the following properties:

(i) If the compound optimal design for minimizing∑m

i=1 �i� i(�) is

�∗[;`={x1 x2 · · · xKp1 p2 · · · pK

}

then the compound optimal design for minimizing∑m

i=1 �i�− i(�) is

�∗[;1−` ≈{−xK −xK−1 · · · −x1pK pK−1 · · · p1

}

This means that for the same weight [; the compound optimal design for estimating ED100�i,i=1; : : : ; m, is symmetrical to that for estimating ED100(1− �i), i=1; : : : ; m.

(ii) A symmetrical design has approximately equal e�ciency for estimating any two symmetricalpercentiles ED100� and ED100(1− �).



(iii) A special case of (i) is that for any design & and its re ection with respect to 0, &, the design& has the same e�ciency for estimating ED100� as the design & for estimating ED100(1−�).

In the next section, we apply these properties to simplify the search of �nding constrainedoptimal designs for estimating the three quartiles.

2.3. The Logit-Design program

Chaloner and Larntz [11] developed a software program named Logit-Design to �nd single-objective Bayesian optimal designs including the ED50 optimal design and the ED95 optimaldesign. This program is based on the Nelder and Mead [15] version of the simplex algorithm andit requires that the number of design support points be speci�ed in advance. The search for the com-pound optimal design begins with two support points and if the optimal design is not found withinthe class of two-point designs, the search extends to the class of three-point designs and so forth.Consequently, our compound optimal design has the smallest possible number of support points.We modi�ed the Logit-Design program to generate compound optimal designs for estimating

several symmetric percentiles and the standard deviations of the estimates. Compound optimaldesigns were found for all combinations of [ where each �i is a multiple of 0.1. As in Chalonerand Larntz [11], we assumed that the design region � is a su�ciently large compact intervalincluding the origin and for illustrative purposes the priors are two independent uniform priors,with �∼U[−0:1; 0:1] and �∼U[6:9; 7:1].Table I lists compound optimal designs for estimating the three quartiles simultaneously. We

observed that for the selected priors, the optimal designs have two support points and they per-formed the way we had expected. For example, the ED25 and the ED75 optimal designs areapproximately symmetric to each other and the ED50 optimal design is symmetric about 0. Simi-larly, when �1 = �3, the compound optimal designs are approximately symmetric about 0, and thedesign e�ciencies for estimating ED25 and ED75 are about equal, that is, e1 ≈ e3.We can use the symmetrical properties (i) of these optimal designs to �nd other optimal designs

not listed in Table I. For example, the compound optimal design when [T = (0:1; 0:9; 0) is supportedat −0:148 and 0:138 with the mass at 0:138 equal to 0:466. This is deduced by symmetry fromthe optimal design with [T = (0; 0:9; 0:1) in Table I.The last three columns in Table I list the design criterion values for estimating ED25, ED50

and ED75 using di�erent compound optimal designs. The �rst three of these compound optimaldesigns are the optimal designs for estimating each of the quartiles. E�ciencies of the compoundoptimal designs for estimating each of the quartiles are given in the middle columns. For example,using values in the second to the last column, the e�ciency of the compound optimal designwith [T = (0:1; 0:8; 0:1) for estimating ED50 is e2 = 0:1101=0:1134 ≈ 0:971. If the sample sizeis N =100, the expected variance for estimating ED50 using this compound optimal design is0:1134=100=0:001134 and its expected standard deviation is

√0:001134 ≈ 0:03367.

3. CONSTRAINED OPTIMAL DESIGNS

3.1. Equal interest in ED25 and ED75

In example 1 we sought an optimal design for estimating ED25 and ED75 with equal interestsubject to the constraint that the design is at least 90 per cent e�cient for estimating ED50. This



Table I. Compound optimal designs for estimating � and � under two independent priors:� ∼ U[−0:1; 0:1] and � ∼ U[6:9; 7:1].

�1; �2; �3 x1; x2 p1; p2 e1 e2 e3 �− log(3) �0 �log(3)

1; 0; 0 −0:210; 0:093 0:800; 0:200 1 0.444 0.202 0.1419 0.2479 0.70360; 1; 0 −0:127; 0:127 0:500; 0:500 0.528 1 0.528 0.2685 0.1101 0.26850; 0; 1 −0:093; 0:210 0:200; 0:800 0.202 0.444 1 0.7036 0.2479 0.14190:1; 0:8; 0:1 −0:155; 0:155 0:500; 0:500 0.618 0.971 0.618 0.2296 0.1134 0.22960:2; 0:6; 0:2 −0:172; 0:172 0:500; 0:500 0.650 0.935 0.650 0.2183 0.1177 0.21830:3; 0:4; 0:3 −0:184; 0:184 0:500; 0:500 0.662 0.904 0.662 0.2144 0.1218 0.21441=3; 1=3; 1=3 −0:187; 0:187 0:500; 0:500 0.664 0.894 0.664 0.2138 0.1231 0.21380:4; 0:2; 0:4 −0:193; 0:193 0:500; 0:500 0.666 0.876 0.666 0.2131 0.1256 0.21310:5; 0; 0:5 −0:200; 0:200 0:500; 0:500 0.668 0.853 0.668 0.2125 0.1290 0.21250; 0:1; 0:9 −0:121; 0:206 0:235; 0:765 0.268 0.563 0.983 0.5302 0.1955 0.14430; 0:2; 0:8 −0:134; 0:202 0:268; 0:732 0.318 0.647 0.954 0.4460 0.1702 0.14870; 0:3; 0:7 −0:142; 0:197 0:297; 0:703 0.361 0.715 0.921 0.3930 0.1539 0.15400; 0:4; 0:6 −0:146; 0:192 0:325; 0:675 0.399 0.776 0.886 0.3556 0.1419 0.16010; 0:5; 0:5 −0:148; 0:186 0:353; 0:647 0.434 0.823 0.849 0.3273 0.1338 0.16720; 0:6; 0:4 −0:148; 0:179 0:380; 0:620 0.465 0.868 0.807 0.3050 0.1268 0.17580; 0:7; 0:3 −0:146; 0:171 0:407; 0:593 0.494 0.910 0.760 0.2871 0.1210 0.18670; 0:8; 0:2 −0:143; 0:161 0:435; 0:565 0.519 0.949 0.704 0.2734 0.1160 0.20160; 0:9; 0:1 −0:138; 0:148 0:466; 0:534 0.535 0.982 0.633 0.2651 0.1121 0.22430:1; 0; 0:9 −0:159; 0:211 0:290; 0:710 0.371 0.686 0.926 0.3827 0.1604 0.15320:1; 0:1; 0:8 −0:163; 0:206 0:317; 0:683 0.407 0.736 0.899 0.3486 0.1496 0.15780:1; 0:2; 0:7 −0:165; 0:202 0:342; 0:658 0.441 0.780 0.870 0.3217 0.1411 0.16310:1; 0:3; 0:6 −0:166; 0:197 0:367; 0:633 0.473 0.820 0.839 0.2997 0.1343 0.16910:1; 0:4; 0:5 −0:166; 0:191 0:391; 0:609 0.505 0.856 0.806 0.2812 0.1286 0.17610:1; 5; 0:4 −0:165; 0:184 0:416; 0:584 0.535 0.890 0.769 0.2652 0.1237 0.18450:1; 0:6; 0:3 −0:163; 0:177 0:442; 0:558 0.565 0.920 0.727 0.2513 0.1197 0.19510:1; 0:7; 0:2 −0:160; 0:168 0:470; 0:530 0.593 0.948 0.678 0.2393 0.1161 0.20920:2; 0; 0:8 −0:179; 0:209 0:355; 0:645 0.468 0.777 0.852 0.3035 0.1417 0.16660:2; 0:1; 0:7 −0:179; 0:205 0:378; 0:622 0.498 0.809 0.825 0.2850 0.1361 0.17190:2; 0:2; 0:6 −0:179; 0:200 0:400; 0:600 0.529 0.840 0.797 0.2682 0.1311 0.17800:2; 0:3; 0:5 −0:178; 0:194 0:424; 0:576 0.558 0.867 0.767 0.2545 0.1270 0.18510:2; 0:4; 0:4 −0:177; 0:188 0:447; 0:553 0.588 0.893 0.733 0.2415 0.1233 0.19360:2; 0:5; 0:3 −0:175; 0:181 0:473; 0:527 0.618 0.916 0.694 0.2297 0.1202 0.20450:3; 0; 0:7 −0:189; 0:207 0:408; 0:592 0.542 0.823 0.787 0.2618 0.1338 0.18040:3; 0:1; 0:6 −0:188; 0:202 0:429; 0:571 0.571 0.847 0.760 0.2485 0.1300 0.18670:3; 0:2; 0:5 −0:187; 0:197 0:452; 0:548 0.600 0.868 0.731 0.2364 0.1268 0.19410:3; 0:3; 0:4 −0:186; 0:191 0:475; 0:525 0.630 0.887 0.698 0.2251 0.1241 0.20320:4; 0; 0:6 −0:196; 0:204 0:455; 0:545 0.607 0.846 0.727 0.2336 0.1301 0.19530:4; 0:1; 0:5 −0:194; 0:199 0:477; 0:523 0.636 0.863 0.698 0.2230 0.1276 0.2032

means that we want a constrained optimal design with e2¿0:90 and e1 = e3: In the Appendix weargued that when �1 = �3 = �, the compound optimal design is approximately symmetric about0 and has the same e�ciency for estimating ED25 and ED75, that is, e1 ≈ e3. This implies thatthe original three-objective design problem can be reduced to a two-objective problem, and thedesired constrained optimal design can be found from an e�ciency plot [12]. In this case, sincee1 ≈ e3, we plot e1 (or e3) and e2 versus � (Figure 1) where � ranges from 0 to 0.5.To �nd the design, we draw a horizontal line e2 = 0:9 in the e�ciency plot (Figure 1) and

determine where it meets the graph of e2. At the cross point, a vertical line is drawn to intersect the



Figure 1. E�ciency plot of symmetrical compound optimal designs when� ∼ U[−0:1; 0:1] and � ∼ U[6:9; 7:1].

� axis at � ≈ 0:31. It follows that the compound optimal design ��≈0:31 is the desired constraintoptimal design. Using � ≈ 0:31, the modi�ed Logit-Design program was rerun and the designfound from the algorithm is equally supported at ±0:185. It is 90 per cent e�cient for estimatingED50 and 66.3 per cent e�cient for ED25 and ED75.In example 2, we sought a constrained optimal design for estimating ED50 subject to the

constraints e1 = e3¿0:65. Proceeding as before, the vertical line at the cross point meets the �axis at � ≈ 0:2, giving e2 ≈ 0:935. The desired optimal design is ��≈0:2 with equal mass at thetwo points ±0:172 (Table I). This design is 65 per cent e�cient for estimating ED25 and ED75,and 93.5 per cent e�cient for ED50.

3.2. Unequal interest in ED25 and ED75

In this case, it is not possible to reduce the design problem into one with two distinct objectives.We are now confronted with a design problem for estimating the three quartiles, each quartile withdi�erent precision. This is a three-objective design problem. It is now harder to �nd the constrainedoptimal design from the compound optimal design because the e�ciency plot is three-dimensionaland the e�ciency curves are no longer monotonic as they were in two-objective design problems



[12]. For example, we observe from Table I that the design with [T = (0:3; 0:4; 0:3) is moree�cient for estimating ED50 than the design with [T = (0; 0:5; 0:5) although the latter has moreweight on �2.We found the following rules quite useful for identifying the constrained optimal design from the

compound optimal designs. They seem to work consistently well for estimating three symmetricalpercentiles simultaneously:

Rule 1. If we �x �1 and increase �2, then e1 and e2 increase and, e3 decreases. Similarly, ifwe �x �3 and increase �2, then e3 and e2 increase, and e1 decreases.Rule 2. If we �x �2, then e2 increases as |�1 − �3| decreases. When �1 = �3, e2 reaches itsmaximum for the given �2.Rule 3. (symmetry). Let [T = (�1; �2; �3), [T = (�3; �2; �1), ei = ei(�[) and ei = ei(�[); i =1; 2; 3: Then we have e1 ≈ e3; e2 ≈ e2 and e3 ≈ e1 and, �[ and �[ are approximatelysymmetric about 0.

In example 3, we sought a design which is optimal for estimating ED75 subject to the con-straints that the design is at least 90 per cent e�cient for estimating ED50 and 60 per cent forED25. This means that we want a constrained optimal design which has the highest e�ciency forestimating ED75 subject to the constraints that e2¿0:9 and e1¿0:6. From Table I, we found thatthe compound optimal design with weights [T = (0:2; 0:4; 0:4) is close to the desired constrainedoptimal design. The weights can be further re�ned by applying rules 1–3. After a few iterations,we arrive at [T = (0:205; 0:427; 0:368) and the desired constrained optimal design is supported at−0:177 and 0:186 with mass at 0:186 equal to 0:543. This design has the following e�ciencies:e1 = 0:600; e2 = 0:900 and e3 = 0:719.If we want to �nd the constrained optimal design that is at least 90 per cent e�cient for

estimating ED50 and 60 per cent for ED75, the compound optimal design can be found bysymmetry. In this case, the new design has weight [T = (0:368; 0:427; 0:205) and is supportedat −0:186 and 0:177 with mass at the latter point equal to 0:457. The design e�ciencies aree1 = 0:719; e2 = 0:900 and e3 = 0:600.The constrained optimal design sought in example 4 can be found similarly. The design sought

should have e1¿0:5 and e3¿0:8 and subject to these constraints is the most e�cient design forestimating ED50. Omitting details, the required �-values are now [T = (0:067; 0:485; 0:448) andthe optimal design is supported at −0:161 and 0:186 with mass at the latter point equal to 0.607.The e�ciencies of this design for estimating the three quartiles are e1 = 0:500; e2 = 0:868 ande3 = 0:800, respectively.

3.3. Di�erent priors and di�erent percentiles of interest

The multiple-objective Bayesian optimal designs found previously have two support points, butin general, the number of support points depends on the prior distribution. Frequently, the opti-mal designs have more support points if the priors are di�use. Table II lists compound optimaldesigns for estimating the three quartiles for di�erent priors on � when � ∼ U[6:9; 7:1] and[T = (1=3; 1=3; 1=3). The number of support points of the optimal design varies from two to six asthe � prior becomes less informative. However, the compound optimal designs appear more robustto variation of the � prior distribution. For any of the priors shown in Table II, the compoundoptimal designs have the same number of support points located at similar dose levels when the� prior is changed from U[6:9; 7:1] to U[6; 8].



Table II. Compound optimal designs using di�erent priors on � when �∼U[6:9; 7:1] and [T = (1=3; 1=3; 1=3).Prior on � Design support points Design weights

U[−0:1; 0:1] −0:187 0.187 0.500 0.500U[−0:3; 0:3] −0:302 0.000 0.302 0.380 0.240 0.380U[−0:5; 0:5] −0:480 −0:115 0.115 0.480 0.286 0.214 0.214 0.286U[−0:7; 0:7] −0:676 −0:286 0.000 0.287 0.676 0.218 0.203 0.159 0.202 0.218U[−1; 1] −0:973 −0:560 −0:185 0.185 0.560 0.973 0.163 0.171 0.166 0.166 0.171 0.163

Figure 2. Sensitivity plot.

We also studied the performance of the multiple-objective Bayesian optimal designs when wehave non-uniform priors and other percentiles are of interest. Independent beta priors for � and� were used and similar conclusions were obtained. In addition, we gained some insights aboutthe robustness properties of these optimal designs for estimating other percentiles by consideringoptimal designs found in examples 1 and 3. Figure 2 shows that their e�ciencies for estimatingany other percentiles are at least 50 per cent or higher, and about 80 per cent if the percentileis near ED50. This suggests that these constrained optimal designs are quite robust for estimatingother percentiles as well.



Table III. Equally supported Bayesian compound optimal designs for the simulationstudy with �∼U[33:5; 40:5].

[T �∼U[0:1; 0:3] �∼U[0:2; 0:4] �∼U[0:3; 0:5](0; 1; 0) 32.3 41.7 33.3 40.7 33.7 40.3(0:5; 0; 0:5) 28.7 45.3 31.8 42.2 33.0 41.0

4. COMPARISON OF THE SEQUENTIAL DESIGN AND THE OPTIMAL DESIGNS

4.1. A simulation study

Optimal designs can be used to gauge the performance of other designs, for example, sequentialdesigns. Rosenberger and Grill [10] described a sequential design based on a generalized P�olyaurn model for estimating the three quartiles in a dose–response study. They �rst conducted asimulation study [10] to evaluate the performance of the sequential design. Subsequently theyapplied the sequential design to a psychophysical experiment. In this and the next sections, wewill compare the performance of this sequential design with the Bayesian optimal design underthe settings of the simulation study and the psychophysical experiment. The underlying model isthe simple logit model.In the simulation study [10], the dose range is divided into 71 levels (from level 1 to level 71)

with the ED25 at level 32, ED50 at level 37 and ED75 at level 42. It follows from the equation2 log 3=� = ED75− ED25 that the model parameter � is 0:22. The true values of � and � for theunderlying logit model are therefore level 37 and 0:22, respectively.For the simulation study, Rosenberger and Grill [10] started with an initial urn of seven par-

ticles at levels 34 to 40 centred around the true median. The trial was conducted as follows. Aparticle was drawn at random and then replaced. The stimulus level labelled on the particle wasadministered to the subject. If the subject was responsive, one particle for each of the next �velower levels were added to the urn; otherwise, one particle for each of the next �ve higher levelswere added. This addition rule is referred to as the 5-up=5-down rule [10].We have selected the following priors for the construction of the multiple-objective Bayesian

optimal designs. Since the initial urn composition could be viewed as a discrete uniform distributionfrom level 34 to level 40, an appropriate prior for � is U[33:5; 40:5]. The prior on � could bedetermined from the pilot study. Here, in the absence of a pilot study, three priors for � werechosen to re ect the possibility of misspeci�cation; they are U[0:1; 0:3]; U[0:2; 0:4], and U[0:3; 0:5].The true value for �(0:22) lies in the centre, on the boundary and outside the interval, respectively.Since we are equally interested in estimating ED25 and ED75, and the primary objective is toestimate the ED50 in this study, we compare the sequential design with the ED50 optimal design,that is, �[ with [T = (0; 1; 0), and the optimal design for estimating ED25 and ED75 with equalinterest, that is, �[ with [T = (0:5; 0; 0:5). The Bayesian optimal designs for these priors are equallysupported at two points (Table III).Table IV displays the standard deviations for estimating the quartiles using various designs and

three sample sizes. These standard deviations are obtained with respect to the true underlying logitmodel with �=37 and �=0:22. Since the prior on � is centred at the true median at level 37,the standard deviations of the estimated ED25 and ED75 are the same using these designs. Theoverall conclusion from Table IV is that the sequential design performs quite well compared to



Table IV. Standard deviations of the estimated quartiles given by two com-pound optimal designs �[ assuming �∼U[33:5; 40:5]: (i) [T = (0; 1; 0) and

(ii) [T = (0:5; 0; 0:5) (in parentheses).

N Sequential design �∼U[0:1; 0:3] �∼U[0:2; 0:4] �∼U[0:3; 0:5]ED25 ED50 ED75 ED25 ED50 ED25 ED50 ED25 ED50

25 3.7 2.9 3.8 3.0 (3.1) 2.1 (2.6) 3.3 (3.0) 2.0 (2.1) 3.5 (3.2) 1.9 (2.0)50 2.3 1.9 2.3 2.1 (2.2) 1.5 (1.9) 2.3 (2.1) 1.4 (1.5) 2.5 (2.3) 1.4 (1.4)100 1.5 1.2 1.7 1.5 (1.5) 1.0 (1.3) 1.6 (1.5) 1.0 (1.1) 1.8 (1.6) 1.0 (1.0)

the Bayesian optimal designs, especially if one considers the non-parametric nature of the formerdesign.

4.2. The psychophysical experiment

Rosenberger and Grill [10] conducted a psychophysical experiment where subjects sequentiallyreceive di�erent levels of a stimulus, and data are recorded on response or non-response to thestimulus. In responding to sensory stimuli, motor responses may proceed in less than 200 ms. Theresponses occur unconsciously since sensory stimuli probably do not reach consciousness for up to500 ms. Knowledge on the timing of the central nervous system processing would greatly enhanceour understanding of normal and pathological brain functioning. The goal of the experiment wasto elicit information e�ciently about the relationship between stimulus levels and responses byestimating quantiles of the stimulus–response curve.The experiment had 11 normal volunteers, aged 41 to 70, and 132 trials were performed on

each of the subjects. Prior to the experiment, a preliminary trial was carried out to determine areasonable span of the dose–response curve and to obtain a guess for the median. The dose rangewas divided uniformly into 71 levels where level 1 corresponds to dose 250 ms and the distance(dose) between adjacent levels was 25 ms. This means that, for example, level 31 corresponds todose 250 + 25× 30=1000 (ms).For the sequential design, Rosenberger and Grill [10] chose the initial urn with nine parti-

cles from level 29 to 37 centred at level 33 (1050 ms), the presumed median. The additionrule adopted was the 10-up=10-down rule [10]. For the Bayesian optimal design, we selected theprior for � to be U[28:5; 37:5] accordingly. We could have also obtained a reasonable prior forthe slope parameter � based on the preliminary trial. However, since no detailed information onthe preliminary trial was available, we calculated the range of the estimated � over the 11 sub-jects from Table III in Rosenberger and Grill [10]. The range was [0:12; 0:54] and we assigneda uniform prior for � on this interval. The Bayesian optimal design for estimating ED50 in theoriginal dose scale is symmetrically supported at 925, 1050 and 1175 with the mass at 925 equalto 0:42.To evaluate the performance of the optimal design, we estimated the quartiles by averaging the

estimated quartiles of the 11 subjects (Table III [10]). These estimates were 1032 ms, 1114 ms,and 1206 ms, respectively. Consequently we found the estimates of the model parameters to be�=1114 ms (level 35.56) and �≈ 0:015 (= 0:015× 25≈ 0:38 after scaling to levels).Now consider a subject whose stimulus–response curve is logit with the three quartiles equal

to the above estimates, and suppose a total of N =132 trials is performed on this subject using



the above ED50 optimal design. The standard deviations of the estimated quartiles are generatedfrom the modi�ed Logit-Design program and they are 19 ms, 15 ms and 20 ms, respectively. Thestandard deviations of the estimated ED25 and ED75 are unequal because the prior on � is centredaround the assumed median at level 33 (1050 ms) instead of the estimated median of 1114 ms(level 35.56).These standard errors are superior to the estimates given by the sequential designs, which were

33 ms, 19 ms and 27 ms, respectively. This could be attributed to using a rather informativeprior for the slope parameter �. We repeated the above calculations using two less informativepriors � ∼ U[0:1; 2:0] and � ∼ U[1:0; 5:0]. Using the corresponding ED50 optimal design, thestandard deviations of the estimated quartiles are (17 ms, 15 ms, 24 ms) and (16 ms, 16 ms, 28ms), respectively. The standard deviation of the estimated ED75 has increased from 20 ms to 24ms, and from 24 ms to 28 ms, exceeding the estimate of 27 ms given by the sequential design.This shows that prior assumptions used in the construction of multiple-objective Bayesian optimaldesigns are important as they can a�ect the accuracy of our conclusions.The sequential design considered here is non-parametric in nature and its induced frequency

distribution (Theorem 1 [10]) has nice convergence properties. The simulation study provides someevidence that the sequential design based on a generalized P�olya urn is quite e�cient. Since thisdesign is a randomized design, it can mitigate selection bias in practice. The Bayesian compoundoptimal designs are also randomized designs if we randomly assign subjects to the various doselevels (support points) according to the optimal allocation schemes.

4.3. Bayesian and locally compound optimal designs

We now compare Bayesian and locally compound optimal designs for the psychophysical experi-ment. In the previous section, we found the Bayesian compound optimal designs with independentpriors � ∼ U[28:5; 37:5] and � ∼ U[0:12; 0:54]. For the locally compound optimal designs, weadopted �=33 (1050 ms), the assumed median, and �=0:33 (0:013 in the original scale), themedian of the estimated range, [0:12; 0:54], for �. These locally compound optimal designs havetwo support points (Table V) unless we are interested in estimating ED25 or ED50 or ED75alone; in the latter case, the locally optimal design is supported at the targeting quartile. Onlysymmetrical designs are displayed in Table V to re ect the equal interest in estimating ED25 andED75 in the psychophysical experiment.When we apply the estimates obtained from the previous section with �=1114 ms (level 35.56)

and �≈ 0:015 (0:38 after scaling to levels) in the logit model, the standard deviations of theestimated quartiles using the locally compound optimal designs are generated from the modi�edLogit-Design program and shown in the last three columns of Table V. On average, they aresuperior to those provided by the Bayesian compound optimal designs shown in columns 5; 6 and7 in the same table. This is because the true values of � and � are rather close to the assumedvalues of �=33 and �=0:33 used in constructing the locally compound optimal designs.Now suppose that the true values are �=1200 ms (level 39) and � remains to be 0:015 (0:38).

Now the true value of � is further away from the assumed value. The locally compound optimaldesigns, with only two support points, are now ine�cient for estimating ED50 and ED75. Thiscan be seen from the greatly increased standard deviations displayed in the parentheses in thelast two columns of Table V. In contrast, the Bayesian compound optimal designs with priors� ∼ U[28:5; 37:5] and � ∼ U[0:12; 0:54] have three support points and these points are furtherspread out than those of the corresponding locally compound optimal designs. Consequently, the



Table V. Comparison of Bayesian and locally compound optimal designs. The priors for the Bayesian designsare � ∼ U[28:5; 37:5] and � ∼ U[0:12; 0:54]. For the locally designs, � = 33 and � = 0:33. Standard deviations

when � = 35:56 and � = 0:38, and when � = 39 and � = 0:38 (in parentheses) are displayed.

[T �∗[ ED25∗ ED50∗ ED75∗ �†[ ED25† ED50† ED75†

28.08 33 37.92‡ 33‡

(0,1,0) 19 (17) 15 (22) 20 (36) — — —0.42 0.16 0.42§ 1.00§

26.60 33 39.40 30.20 35.80(0.1,0.8,0.1) 20 (19) 17 (18) 21 (28) 16 (20) 16 (39) 27 (63)

0.36 0.28 0.36 0.50 0.5025.88 33 40.12 29.64 36.36

(0.2,0.6,0.2) 20 (20) 17 (18) 21 (25) 17 (18) 15 (32) 24 (53)0.36 0.28 0.36 0.50 0.5025.40 33 40.60 29.32 36.68

(0.3,0.4,0.3) 21 (21) 18 (18) 22 (24) 18 (17) 15 (29) 22 (49)0.36 0.28 0.36 0.50 0.5025.12 33 40.88 29.08 36.92

(0.4,0.2,0.4) 22 (22) 19 (18) 22 (23) 18 (17) 15 (27) 21 (45)0.37 0.26 0.37 0.50 0.5024.88 33 41.12 28.92 37.08

(0.5,0,0.5) 22 (23) 19 (18) 22 (22) 18 (17) 15 (25) 21 (43)0.37 0.26 0.37 0.50 0.50

∗ Bayesian. †Locally. ‡Design support points. §Design weights.

Bayesian compound optimal designs tend to be more robust for estimating all three quartiles. Thisis con�rmed by the relatively small standard deviations displayed in parentheses in columns 5, 6and 7 of Table V.The above examples show that if the assumed values are not too di�erent from the true model

parameter values, the locally compound optimal designs would outperform the Bayesian optimaldesigns. However, when the misspeci�cation is more severe, the converse is true. Bayesian com-pound optimal designs with more than two support points have a further advantage of permittingmodel adequacy checking if necessary. We conclude with a cautionary note that our approach hereassumes that we have large samples. More work needs to be done to examine the properties ofthe Bayesian compound optimal designs when the sample sizes are small.

APPENDIX

We now justify the properties of compound optimal designs claimed in Section 2. The Bayesiancriterion for estimating ED100� using design � is � (�)=E� (�; �), where (�; �)= �−2t−1 +( − �(�x − �))2�−2s−1. The directional derivative of � (�) is d (x; �)=E�[� (x; �; �) + (�; �)],where � (x; �; �)= − w(x; �)(�2st)−2[t(�x − x)(�(�x − �) − ) + �s]2. To check if a design �∗ isBayesian optimal for estimating ED100�i, it su�ces to check if d (x; �

∗) is non-negative; seeChaloner and Larntz [11] where this result is referred to as the Bayesian general equivalencetheorem (BGET).



When there are several percentiles of interest, the compound optimality criterion for estimatingm percentiles is

�S(� | [)=m∑i=1�i� i(�)=E�

m∑i=1�i i(�; �)

where i= ln[�i=(1− �i)], i=1; : : : ; m and S′=( 1; : : : ; m). Its directional derivative is

d(x; � | [)=m∑i=1�id i(x; �)=E�

m∑i=1�i[� i(x; �; �) + i(�; �)]

If � puts mass pi at xi, i=1; 2; : : : ; K and

��={2�− xK 2�− xK−1 · · · 2�− x1pK pK−1 · · · p1

}

the two designs satisfy

� i(x; �; �)= �− i(2�− x; �; ��) and i(�; �)=− i(�; ��); i=1; : : : ; m

This is because the functions w(x; �), (�x− x)2, (�x− �)2, s and t are all symmetric with respect to�, that is, f(2�− x)=f(x), and the functions (�x − x) and (�x − �) are symmetric with respect to� at the origin, that is, f(2�− x)= − f(x). It follows that

m∑i=1�i� i(x; �; �)=

m∑i=1�i�− i(2�− x; �; ��)

andm∑i=1�i i(�; �)=

m∑i=1�i− i(�; ��)

Since E(�)= 0, it follows from the delta method that

�S(� | [) ≈ �−S(� | [) and dS(x; � | [) ≈ d−S(−x; � | [)where �= ��= 0. An application of the BGET shows that if � is a compound optimal design forestimating ED100�i for a given set of weights �i, i=1; : : : ; m, then � must be approximately acompound optimal design for estimating ED100(1−�i) for the same set of weights �i, i=1; : : : ; m.Clearly, if we set [T = (1; 0; : : : ; 0), the above argument shows that the ED100� optimal design

and the ED100(1 − �) optimal design are symmetrical. Furthermore, if a design � is symmetric,�= �, and � (�) ≈ �− (�)=�− (�). Consequently, e (�) ≈ e− (�) holds by the de�nition ofdesign e�ciency. This means that the e�ciencies of a symmetrical design for estimating ED100�and ED100(1 − �) are approximately equal. Similarly, for any design & and its re ection &, wehave � (&) ≈ �− (&) and e (&) ≈ e− (&) follows.

ACKNOWLEDGEMENTS

We thank Dr William Rosenberger who provided the motivation behind our work. We thank the reviewersfor their constructive and helpful comments. Special thanks also go to Drs Chaloner and Larntz for kindlysending us their Logit-Design program. The work of Wong is partially supported by as NIH grant R29AR44177-01A1.



REFERENCES

1. Flournoy N. A clinical experiment in bone marrow transplantation: estimating a percentage point of a quantal responsecurve. In Case Studies in Bayesian Statistics, Gatsonis C, Hodges JS, Kass RE, Singpurwalla ND (eds). Springer-Verlag, 1993; 324–335.

2. Clyde M, Muller P, Parmigiani G. Optimal design for heart de�brillators. In Case Studies in Bayesian Statistics II,Gatsonis C, Hodges JS, Kass RE, Singpurwalla ND (eds). 1995; 278–292.

3. Freeman PR. Optimal Bayesian sequential estimation of the median e�ective dose. Biometrika 1970; 57:79–89.4. Malkin RA, Pilkington TC, Burdick DS, Swanson DK, Johnson EE, Ideker RE. Estimating the 95% e�ectivede�brillation dose. IEEE Transactions on EMBS 1993; 40(3):256–265.

5. Tsutakawa R. Selection of dose levels for estimating a percentage point on a logistic quantal response curve. AppliedStatistics 1980; 29:25–33.

6. Wu CFJ. Optimal design for percentile estimation of a quantal response curve. In Optimal Design and Analysis ofExperiments, Dodge Y, Fedorov VV, Wynn HP (eds). Elsevier Science Publication: North Holland, 1988; 213–223.

7. Durham SD, Flournoy N. Random walks for quantile estimation. In Statistics Decision Theory and Related TopicsV, Gupta SS, Berger JO (eds). Springer: New York, 1994; 4670-476.

8. Giovagnoli A, Pintacuda N. Properties of frequency distributions induced by general ‘Up-And-Down’ methods forestimating quantiles. Journal of Statistical Planning and Inference 1998; 74:51–63.

9. Rosenberger WF. New directions in adaptive designs. Statistical Science 1996; 11:137–149.10. Rosenberger WF Grill SE. A sequential design for psychophysical experiments: an application to estimating timing of

sensory events. Statistics in Medicine 1997; 16:2245–2260.11. Chaloner K, Larntz K. Optimal Bayesian design applied to logistic regression experiments. Journal of Statistical

Planning and Inference 1989; 21:191–208.12. Cook RD, Wong WK. On the equivalence of constrained and compound optimal designs. Journal of the American

Statistical Association 1994; 89:687–692.13. Clyde M, Chaloner K. The equivalence of constrained and weighted designs in multiple objective design problems.

Journal of the American Statistical Association 1996; 91:1236–1244.14. Lee CMS. Constrained optimal designs for regression models. Communication in Statistics Part A – Theory and

Methods 1987; 16:765–783.15. Nelder JA, Mead R. A simplex method for function minimization. Computer Journal 1965; 7:308–313.


Documents

Bayesian optimal designs for estimating a set of symmetrical quantiles