Design and Interpretation of Vaccine Field Studiescourses.washington.edu/b578a/readings/halloran1999.pdf · 2 Escola Nacional de Saude Publica, Fundacao Oswaldo Cruz, Rio de Janeiro,

Epidemiological ReviewsCopyright © 1999 by The Johns Hopkins University School of Hygiene and Public HealthAll rights reserved

Vol.21, No. 1Printed in U.S.A.

Design and Interpretation of Vaccine Field Studies

M. Elizabeth Halloran,' Ira M. Longini, Jr.,1 and Claudio J. Struchiner2

INTRODUCTION

Vaccine efficacy and effectiveness (VE) are gener-ally estimated as one minus some measure of relativerisk (RR) in the vaccinated group compared with theunvaccinated group:

VE - 1 - RR. (1)

Due to the dependent happenings in infectious diseases(1), vaccination can produce several different kinds ofeffects, at both the individual and the population levels.The groups being compared could be composed ofindividuals or of populations or communities.Vaccination can induce a biologically protectiveresponse in a vaccinated individual or reduce thedegree or duration of infectiousness for other individ-uals. Widespread vaccination in a population canreduce transmission and produce indirect effects, evenin individuals who were not vaccinated. Vaccinatedpeople might change their rate of making contacts withpotentially infectious sources and alter the overall ben-efits of vaccination. In designing a study to evaluatethe effects of vaccination, the question of interestguides the choice of unit of observation, comparisongroups, parameter of effect, and level of informationrequired (2).

In this presentation we review different measures ofeffect of vaccination and vaccination programs. Weconsider various study designs for estimating the dif-ferent measures of effect based on the choice of corn-

Received for publication December 10, 1998, and accepted forpublication March 16, 1999.

Abbreviations: BS-WC, B subunit killed whole cell; Cl, cumulativeincidence; HIV, human immunodeficiency virus; /, infectiveness; IR,incidence rate; PH, proportional hazards; RR, relative risk; S, sus-ceptibility; SAR, secondary attack rate; T, susceptibility and infec-tiveness combined effects; VE, vaccine efficacy and effectiveness;WC, whole cell.

1 Department of Biostatistics, Rollins School of Public Health,Emory University, Atlanta, GA.

2 Escola Nacional de Saude Publica, Fundacao Oswaldo Cruz,Rio de Janeiro, Brazil.

Reprint requests to Dr. M. Elizabeth Halloran, Department ofBiostatistics, Rollins School of Public Health, Emory University, 1518Clifton Road NE, Atlanta, GA 30322.

parison groups, the unit of observation, the choice ofparameter, and the amount of information about thetransmission system required for estimation. Our focusis on field studies that may be double-blinded, placebocontrolled phase III trials, or phase IV post-licensurestudies that may be randomized or observational. Weregard the paradigm of the randomized, double-blindedstudy as the point of departure for interpreting observa-tional and nonrandomized studies (3, 4). Case-controlstudies produce approximate estimators of the appro-priate population parameters (5, 6). Our primary con-cerns here are concepts of design and interpretation ofthe estimates. For example, what does it mean exactlywhen we say that a vaccine is 85 percent efficacious?

We begin by discussing some biologic aspects ofvaccination that are of interest or that need to be takeninto account when estimating vaccine efficacy.Following that, we discuss designs for estimating thedirect protective effect of vaccination. Next we presentdesign options for evaluating how vaccination altersthe infectiousness of a person who becomes infected.We then consider that a vaccination may result inchange in behavior, altering the exposure to infection.Continuing on, we present community-based studydesigns for estimating the indirect, total, and overalleffectiveness of widespread vaccination in popula-tions. We conclude by touching upon the problem ofevaluating safety and adverse events.

EFFECTS OF INTEREST

Table 1 (2) provides an overview of several differenttypes of effects and the parameters used to estimatethem. Historically, the primary effect of interest ofvaccination has been how well it protects the vacci-nated individual. Biologically, the protective immuneresponse can reduce the probability that a personbecomes infected given a specified exposure to orinoculum of an infectious agent. That is, it can reducethe transmission probability. If a vaccinated personbecomes infected, the immune response might reducethe degree or duration of disease or the probability ofdying from the disease. It may also alter the rate of dis-ease progression. For many infectious agents with

73

74 Halloran et al.

TABLE 1. Parameters used

Level Parameterchoice

for measuring various

Susceptibility

effects of vaccination*

Comparison groups and effect

Combined change inInfectiousness susceptibility and

infectiousness

Conditional on exposureto infection:

Transmission probability, p + - 1 - —Secondary attack rate s " ~ p00

(SAR)

VE, = 1 - —'•" Poo

VEro = 1 - —Poo

Unconditional:

II Incidence rate (IR)

Hazard, A.

Ill Proportional hazards

IV Cumulative incidence

Attack rates (AR)

(PH)

(Cl)

VEalR

V E «

Idirect

= 1 -

VES,PH = 1

VESCI = 1 -

IRAI

IRAO

* - A 1

^ • A O

- e *

CI A 1

CUo

IIAindirect

NAf

VE,1 A .C I = 1 -

Study design

MBtotal

I R vtiiB.iR - i

XA0

x«, VE|IM

NA

i f VE-.c,=1-

IRAI

IRBO

xM

^BO

CIBO

VEM | t n _ 1

V E | | I X = 1

IIIoverall

aA 1 +

NA

rciA) 4

- (1 -

IRBO

(1 -

Xeo

- ( 1 -

CIBO

OIRAO

OÂO

OCIAO

* From Halloran et al., Am J Epidemiol 1997;146;789-803. Reproduced with permission. The subscripts 0 and 1 denote unvaccinated andvaccinated people, respectively. Population A contains both vaccinated and unvaccinated people. All people in population B are unvaccinat-ed (see figure 3). The subscripts S, /, and T denote susceptibility, infectiousness, and combined effects, respectively. The Cox proportionalhazards estimator is denoted by epi. Time has been omitted from the table for notational clarity.

t VE, vaccine efficacy/effectiveness; NA, not applicable.

short incubation periods, disease is used as the out-come of interest in vaccine trials rather than infection.Becoming infected results with some probability fromcontact with an infectious source, while developingdisease depends on the within host interaction subse-quent to successful infection. In many vaccine studies,the distinction between infection and disease as out-come is not made. Studies with either of these out-comes are sometimes used to measure vaccine efficacyfor susceptibility (VES) (third column, table 1), thoughthe distinction between infection and disease shouldalways be kept in mind.

Another measure of effect evaluates the degree ofprotection once a person has become infected. We callthis vaccine efficacy for progression (VEP) (not intable 1). With infectious agents that have long incuba-tion periods, such as tuberculosis or human immuno-deficiency virus (HIV), evaluation of this sort of effectis particularly important. Another example of VE/. isthe comparison of the degree of illness conditional on

becoming sick. For example, vaccinated persons whocontract chickenpox generally have much milder dis-ease than unvaccinated persons who contract the samedisease.

The main distinction between VE5 and VEP is thatstudies to estimate VE5 evaluate susceptibles and theexposure to infection would need to be taken intoaccount. Studies to estimate VEP are conditional onthe participants already being infected, so the pro-gression within infected individuals is important.Studies to evaluate VES that use disease as an outcomeoften do not differentiate the protective effects againstinfection and against disease conditional upon infec-tion. Nonlinearities of the pathway from infection tomanifest disease could mean that the efficacy as mea-sured by the observed outcome would be quite differ-ent from the biologic efficacy if it could be measuredalong the pathway (7). The difference should be keptin mind when designing, analyzing, and interpretingthe study.

Epidemiol Rev Vol. 21, No. 1, 1999

Vaccine Field Studies 75

A vaccinated person who becomes infected may alsobe less infectious to other susceptibles or be infectiousfor a shorter period of time. The vaccine efficacy forinfectiousness (VE;) (table 1, top row, columns 4 and5), is of interest because a vaccine that reduces infec-tiousness could have important public health conse-quences. Vaccination could also change the distribu-tion of carriers in a population.

Widespread vaccination can have indirect effects forunvaccinated people as well as for vaccinated people.The indirect effects are due to the change in collectivelevel of immunity in the population due to vaccination.The collective level of immunity in a populationagainst a particular parasite is called herd immunity. Itis important to differentiate between the indirecteffects in the unvaccinated and vaccinated groups.These are called the indirect and total effectiveness,respectively (table 1, middle, columns 4 and 5). Theindirect effects in unvaccinated people might not bethe same as those in the vaccinated people. The over-all effectiveness of a vaccination strategy or allocationwithin a particular population is the weighted averageof the outcomes in the vaccinated and the unvaccinatedpeople (table 1, middle, column 6).

To evaluate the direct protective effects of vaccina-tion, VES and VEP, usually the individual is the unit ofobservation. To evaluate VE/5 generally small trans-mission units, such as households or partnerships inwhich contacts can be defined, are needed. This typeof study in small transmission units can also be used toevaluate VES (table 1, top row). To evaluate the popu-lation level effects, the unit of observation becomes thepopulation, so that several populations need to beincluded in the study (table 1, bottom right portion; seebelow in the discussion of figure 3).

Vaccination can induce heterogeneous response, andprotection might wane or be boosted by exposure towild-type infection. Exposure to infection might beheterogeneous within comparison groups. Vaccineantigen mixes might be effective against some strains,but not others. Belief in the protective effects of vacci-nation could result in people increasing their exposureto infection. These characteristics affect the interpreta-tion of the vaccine efficacy estimates. In the followingsections, we review some of the issues related to eval-uating each of the various effects.

VACCINE EFFECT ON SUSCEPTIBILITY, VES

We first consider study designs for estimating theprotective effects of vaccination, VES. In table 1, theseare represented in the third column. Important forchoosing methods to estimate VES and for interpretingVES estimates is to consider some model for the pro-tection conferred by the vaccine (8). To begin, sup-

pose that vaccination provides a multiplicative reduc-tion in the probability of being infected given a spec-ified exposure to infection or a specified inoculum.That is, the transmission probability in a vaccinatedperson is a fraction, denoted by 0, of that in an unvac-cinated person. We then define VE5 = 1 - 9 , whereVES represents the proportionate reduction in thetransmission probability given a specified exposure toinfection.

Under the assumption of equal exposure to the infec-tious agent in the vaccinated and unvaccinated groups(9), the estimates of VES are obtained from the relativerisk of infection or disease in the vaccinated individu-als compared with the unvaccinated individuals:

V E c =R (vaccinated people)

R (unvaccinated people)'

where R denotes one of the measures of risk. Given ourassumption about the multiplicative effect of the vac-cine on the transmission probability, if we can estimatethe relative transmission probability to the vaccinatedcompared with the unvaccinated individuals, then wecan estimate VES = 1 - 9 . However, this requiresinformation on exposure to infection, which is oftendifficult or impossible to obtain. The incidence rate,hazard rate, and cumulative incidence (or incidenceproportion) are measures of disease frequency thatgenerally do not require knowledge of contacts withinfectives to estimate. They also can be used to esti-mate VE5, but then the interpretation of the estimatedefficacy is different.

VES based on the transmission probability

Let the transmission probability, denoted /?,-,, be theprobability that, conditional upon a contact between aninfective source with covariate status / and a suscepti-ble host with covariate status j , successful transfer andestablishment of the infectious agent will occur. Thetransmission probability could also be defined condi-tional on a specified level of inoculum. A related con-cept is the secondary attack rate, (SAR,-,), defined asthe proportion of susceptibles with covariate status jmaking contact with an infectious person of covariatestatus i who become infected.

Let 0 and 1 denote being unvaccinated and vacci-nated, respectively. Then, for example, p0l denotes thetransmission probability per contact from an unvacci-nated infective person to a vaccinated uninfected per-son. Let p0 andpj, denote the transmission probabilityto unvaccinated and vaccinated susceptibles, respec-tively, where the dot in the subscript can denote anyvaccine status or an average across the population.


76 Halloran et al.

Then VESp based on the transmission probability orsecondary attack rate (table 1, top row) is estimatedfrom

= i -EA= i S A R 'Pa SARn

1 -

vaccinated infections

vaccinated exposures

unvaccinated infections

unvaccinated exposures

Estimating vaccine efficacy from the transmissionprobability ratios requires information on who is infec-tious and when, and whom they contact and how. Theconcept of a contact is very broad and must be definedin each particular study. Often it is defined within asmall transmission unit such as a household or sexualpartnership. For a sexually transmitted disease, thecontact could be defined per sex act or per partnership.For pertussis, a contact could be defined as attendingschool the same day as an infectious person or as liv-ing in the same household during the entire period ofinfectiousness of a case. The mode of transmission ofa parasite determines what types of contacts are poten-tially infectious.

The type of contact and the infectiousness of theinfective source will determine the inoculum level percontact. A vaccine that is protective against a lowinoculum might not protect against a high inoculum. Ifit were possible to measure the infectiousness of theinfectives, the covariate^ in the transmission probabil-ity might include information about this, and, there-fore, provide a means to stratify by inoculum level incomputing the efficacy estimates. Similarly, if it werepossible to measure the different types of contacts,then the transmission probability for each type of con-tact could be estimated, and the VE5p estimates couldbe stratified by type of contact. If it is not possible tomeasure the levels of infectiousness, the inoculumlevel, or the different types of contacts, then the esti-mates will reflect the unmeasured heterogeneities.

There are two main ways to design a study to esti-mate the relative transmission probabilities. The firstmethod, called the secondary attack rate (10-13), orcase-contact rate method, has been used since the per-tussis vaccine trials in 1930s (14) to estimate vaccineefficacy. In this case, the focus is on identifying theinfected individuals and the proportion of peopleexposed to them who become infected. Anothermethod of estimating the transmission probability isbased on the binomial model. In this case, we observesusceptible people, count the number of contacts they

make with infectives, and count the number of thesesusceptible people who become infected. The trans-mission probability is estimated using the binomialmodel. Secondary attack rate studies are commonlyused for directly transmitted infectious agents withhigh transmission probabilities, such as measles,chickenpox, mumps, pertussis, and tuberculosis.Contacts are often defined within small transmissionunits such as households. In tuberculosis, contact trac-ing and testing are often used to estimate the secondaryattack rates. The binomial model is commonly used instudies with low transmission probabilities, such asHIV, in which susceptibles often make more than onecontact before becoming infected. The ascertainmentof the susceptibles or infectives can occur prospec-tively or retrospectively, depending on the design ofthe study.

VES not conditional on knowledge of exposure toinfection

Information on exposure to infection is often diffi-cult or impossible to collect. More commonly, studiesare designed to estimate VE5 from events per person-time of potential rather than actual exposure or simplyfrom the proportion of people who become infected inthe vaccinated compared with the unvaccinatedgroups. Standard parameters for estimating VES areincidence rates, hazard rates, or cumulative incidence.Halloran and Struchiner (5, 15) called these compar-isons study design type I (table 1, third column, lowerrows). Here information on actual exposure to infec-tion is not required to estimate VES. The assumption ismade that the vaccinated and unvaccinated groups areequally exposed to infection (9), so that any differ-ences in the risk in the two groups is due to the bio-logic effects of the vaccine.

Estimation of the different VES parameters requiresdiffering levels of information and make differentdemands on study design and data collection (16).Incidence rates or hazard rates require the time toevent and the period of potential exposure of each per-son under study. The hazard rate in infectious diseasesis often called the force of infection. A Cox propor-tional hazards model requires only the ordering of theevent times. An estimate of cumulative incidencerequires only final value data, that is, whether an infec-tion occurred by the end of the study or not.Correspondingly, in table 1, VE5IR based on incidencerates (IR) and VESJl are level II parameters, VE5PH

based on Cox proportional hazards (PH) is level III,and VE5 CI based on cumulative incidence (CI) or finalvalue data is level IV. Each level requires less infor-mation about the transmission system, with only levelI requiring actual contact information. Thus, the levels



form a hierarchy. Since VE S p based on the transmis-sion probability is defined conditional on exposure toinfection, it is called a conditional parameter, while theother measures are called unconditional parameters.

Primary vaccine efficacy studies often report VES I R

based on relative events per person time, or level Elinformation,

VES,IR = 1 -vaccinated events/person-person-time

unvaccinated events/person-time

The usual assumption is that the numbers of events fol-low a Poisson distribution. Similarly, investigatorsmay estimate the hazard rates in the vaccinated andunvaccinated Xx(t) and ^ ( 0 , respectively, using sur-vival analysis methods. Then the VES is based on the

hazard rate ratio VE5 x(t) = 1 — . When covariates

such as age and gender are added, the analyses arestratified by the covariates or Poisson regression canbe used. Under the assumption that the effect of thevaccine is multiplicative, constant, and homogeneous,the Cox proportional hazards model can be used. Inthis case, it is not necessary to estimate the hazard ratein the unvaccinated group, but only the relative hazardrate. This requires only the ordering of the infectiontimes, as mentioned above. Covariates including time-dependent covariates can easily be incorporated usingstandard software.

An early example of estimating VE5IR is the study byKendrick and Eldering of pertussis vaccine in the1930s (14), which reported both the proportion of peo-ple exposed to infection who developed pertussis, aswell the number of cases per person-time. The vacci-nated and control groups had 1,815 and 2,397 children,respectively, who contributed 2,268 and 2,307 person-years at risk, respectively. There were 52 cases in thevaccinated and 348 cases in the control group, so

52 cases

VESJR = 12,268 person-years

348 cases

2,307 person-years

= 0.85. (2)

More recently, Urdaneta et al. (17) present estimatesof VES I R as the result of a randomized, placebo con-trolled field trial of SPf66 malaria vaccine in CostaMarques, Rondonia, Brazil. A total of 572 participantscompleted the three dose vaccine schedule and werefollowed up for 18 months. The 287 vaccinated indi-viduals contributed a total of 12,178 person-weeks atrisk, and 76 first Plasmodium falciparum malaria

episodes were observed among them. In the placebogroup, 285 individuals contributed 11,698 person-weeks at risk and 85 cases leading to an estimate ofVE5>IR = 0.14.

In some studies, it is possible to compute both a con-ditional and an unconditional estimate of vaccine effi-cacy from a single study. The Kendrick and Elderingstudy on pertussis vaccine mentioned above also hadinformation on children who had been exposed to per-tussis within their own households. In the vaccinatedgroup, 29 of 83 exposed children developed pertussis,while 143 of 160 exposed children in the unvaccinatedgroup developed pertussis. Thus, the estimate of VEXp is

29 cases/83 vac exposeds'p 143 cases/160 unvac exposed

While everyone is included in the estimate of VE5IR,only the children with (presumed) exposure to infec-tion are included in the VE5p estimate. The interpreta-tions of the two estimates are also different, since onemeasures the protection conferred as measured byinfections per person time and the other by the proba-bility of an infection per potentially infectious contact.

Estimation of VE5CI(r) based on the cumulativeincidence requires only information about whetherpersons are infected or not by the end of the study attime T, that is, final value data:

VEs,a(T) =

vaccinated infection events/persons-at-risk1 -

unvaccinated infection events/persons-at-risk

1 CI,(7Q

As an example, Greenwood and Yule (9) used thecumulative incidence in studying the efficacy oftyphoid vaccination in the troops in the early part of thetwentieth century. In one analysis, Greenwood and Yuleassumed that the denominators were based on the vac-cinated and unvaccinated groups at the beginning of thestudy. They had 56 cases of typhoid in 10,378 vacci-nated soldiers, and 272 cases in 8,936 unvaccinated sol-diers. The estimated efficacy based on these numbers is

— 56 cases/10,378 at-riskVES C I (7) = 1 0.82 .(4)5>CIV ' 272 cases/8,936 at-risk

A more recent example is the estimation of VE5 C I(7)from a double-blinded live attenuated, cold-adapted


78 Halloran et al.

influenza vaccine trial in children (18). In this trial, ofthe 1,070 children who received vaccine, 14 wereinfected, and of the 532 children who receivedplacebo, 95 were infected. Infection was defined asbeing culture positive for influenza virus during theacute phase of an influenza-like illness. Based on these

— 14/1,070numbers, VE5,CI(r) = 1 - ^ = 0.93.

Because of the dependent happening structure ofevents in infectious diseases, there is an intrinsic rela-tion among the different parameters on which the VE5

estimators are based. Understanding this relation helpsto see the relation of the different estimators of VE5 toone another. Let py be the transmission probability asdefined above. Let c denote the contact rate in a popu-lation assuming that people are randomly mixing, andlet P(i) denote the prevalence of infectives at time t.Then the hazard rate X(t) (or incidence rate or force ofinfection) at time t can be expressed as the product ofthe contact rate, the transmission probability, and theprobability that a contact is infectious:

= cPijP{t). (5)

So even if the different components of the hazard rateare not measured, we can consider the underlyingprocess that is producing the infections we observe.Similarly, the cumulative incidence, 0 (7} , at sometime T is a function of the hazard rate during thefollow-up period, and thus also a function of the trans-mission probability, contact rate, and prevalence ofinfection in the contacts. Even though the cumulativeincidence estimate is a sort of black-box estimator, it isuseful in vaccine studies to think about the underlyingtransmission system that would produce the observedfinal values.

axis into piece-wise constant components. Then a sep-arate constant VEslR(t) is estimated for each time inter-val. If there is time dependence, then the estimates willvary across the time intervals. This method of analysiswas used to estimate VE5IR(f) of killed whole-cell-only (WC) and B subunit killed whole-cell (BS-WC)oral cholera vaccines over 3 years (1985-1988) of arandomized, double-blinded vaccine trial in ruralMatlab, Bangladesh (19). The placebo was a killedEscherichia coli strain. Participants included 89,596subjects aged 2-15 years (male and female) andgreater than 15 years (females only). The first 3 yearsof the vaccine trial were partitioned into 1-year seg-ments. Cholera incidence was defined as the numberof culture-confirmed cholera episodes per 106 person-days of follow-up during each of 3 years.

In the first year of the trial, the estimated incidence ratefor those vaccinated with the BS-WC vaccine was 41/106

person-days and for the placebo HO/106 person-days.41

Thus, in the first year VEWR(1) = 1 - — = 0.63

(one tailed 95 percent confidence limit 0.50). The VEestimates for the BS-WC vaccine for the second andthird year of the trial were 0.57 (one tailed 95 percentconfidence limit 0.42) and 0.17 (one tailed 95 percentconfidence limit -0.15), respectively. Thus, we see awaning time trend in efficacy, with no significant pro-tection by the third year. Since the partitioning bound-aries are selected at 1-year intervals, it is not clear ifthe waning protection is continuous or precisely atwhat point in time significant protection is lost. Withuse of a Poisson regression including covariates, theproblem still remains of how to partition the time axisinto piece-wise constant components. The problem canbe solved by the use of survival analysis methodsdescribed in the next section.

When estimates of VES vary with time

In introducing the various VE$ estimators, we madethe very simple assumptions that the protective effectwas the same multiplicative effect in everyone, thatprotection did not wane nor was it boosted, and thatexposure to infection was not only the same in the vac-cinated and unvaccinated groups, but that everyonewas exposed equally. Unmeasured heterogeneities insusceptibility, protection, and exposure to infectioncan produce time-varying estimates of VESIR(r) orVE5Jt(f) that are artifacts of the choice of analysis,while true waning of protection or boosting can lead toreal time-varying effects.

Consider first the situation that protective efficacyactually wanes with time. One way to build timedependence into the analysis is to partition the time

Nonparametric estimation of time-varying vaccineeffects

In this section, we assume that individual or groupedtime-to-event data (level II) are available. Durham etal. (20) adapted and compared two basic approachesfor the nonparametric estimation of smoothed curves

for VEsx(t) = 1 - RR(0 = 1 - r ~ . The first is a

generalized additive models approach that involvesusing a time-varying coefficient (21) version of theproportional hazards model assuming a Poisson model(22). It is useful for diagnostics to ascertain the shapeof (3(0, but it cannot provide an estimator for VEsx(t).The other method uses Schoenfeld residuals (23, 24).The general idea is to fit an ordinary proportional haz-ards model to the data, then to compute the scaled dif-



ferences between the actual and expected covariatevalues at each event time, called Schoenfeld residuals.The scaled residuals are added to the coefficient fromthe proportional hazards model. The time-varyingregression coefficient (3(0 is recovered by smoothingthe rescaled Schoenfeld residuals. Conceptually, weare nonparametrically estimating the instantaneoushazard rate ratio e^'\ thus VEsx(t). Both methods pro-vide a hypothesis test for the null Ho: P(/) = |3 for allt, that is, for no time-varying effects. The method usingthe Schoenfeld residuals is easy to use, provides anestimate of e^(I) on the natural scale, and allows easyincorporation of time-dependent covariates, so we rec-ommend this approach in general.

Durham et al. (25) used the method involvingSchoenfeld residuals to estimate smooth plots of theVE5^(0 for the two oral cholera vaccines from thecholera vaccine trial described above. Figure 1 showsthe plot of the VE5^(0 estimates and the 95 percentconfidence intervals for the two vaccines. The bendingdownward of the curves is indicative of waning. The pvalues for the hypothesis test for departures from theproportional hazards assumption are 0.008 and 0.002for the estimated model of the WC and BS-WC vac-cines, respectively. The WC vaccine gives fairly con-stant and significant protection, with a VE55t(O ofabout 0.50 for the first 2.5 years of the trial, but thenprotection appears to wane rapidly. After 3 years of thetrial (May 1988), the point estimate of the VE5X(0 is0.245 and the 95 percent confidence interval coverszero. Protection from WC-BS vaccine starts out higherthan that from WC vaccine, i.e., 0.713 versus 0.430,but then gradually wanes at a fairly constant rate, i.e.,about 2-3 percent per month. This analysis provides amore complete description of the VES than that basedon yearly incidence ratios described above.

Unmeasured heterogeneity (frailty mixturemodels)

Consider now heterogeneous protection. Smith et al.(8) considered two models for vaccine protection, onewhich acted homogeneously and multiplicatively onthe hazard and the other which protected some peoplecompletely against infection while not having anyeffect on the others. They showed that whether an esti-mator varies with time depends on the distribution ofprotection. We extended the discussion of mechanismof protection to the action being directly at the level ofthe transmission probability (26, 27). We then devel-oped more general distributions of protection (28-31)that can include some people who are completely pro-tected, some who have no protection, and the rest hav-ing a continuous distribution of protection. Vaccineswith a multiplicative effect are called "leaky" (32),

WC vaccine

Ma/85 Nov"85 Ma/86 N0V86 Ma/87 Nov-87 Ma/88 N0V88 Ma/89 Nov-89

time

BS-WC vaccine

Ma/85 Nov-85 Ma/86 N0V86 Ma/87 No/87 Ma/88 N0V88 Ma/89 NovTO

time

FIGURE 1 . Smoothed plots of VE(f) versus /, with 95% confidenceintervals, for the whole killed cell (WC) and B subunit whole killed cell(BS-WC) vaccines, Matlab, Bangladesh, May 1, 1985, throughNovember 31,1989. The smoothing was carried out with regressionsplines with 4 degrees of freedom. These plots were constructedcontrolling for age as a covariate. VE, vaccine efficacy; t, time. FromDurham et al., Am J Epidemiol 1998; 147:948-59. Reproduced withpermission.

while those conferring complete protection or none atall are called "all-or-none."

With an all-or-none vaccine, the estimator VEX.CI

does not vary with time, and provides an estimate ofthe proportion of the vaccinated completely protected(8). If VESIR or VES x is

use(* t o estimate protection ofa vaccine conferring all-or-none protection, the esti-mates will increase with time. This is due to the peoplewith no protection becoming infected, so the unin-fected vaccinated group remaining becomes enrichedwith highly protected people. Thus, the VES^ appearsto increase with time. If a proportion of the vaccinated


80 Halloran et al.

group is completely protected, and another portion isonly partially protected, then there are additional vac-cine efficacy parameters of interest to estimate. Theseinclude a, the proportion of people completely pro-tected, and 6, the mean reduction in susceptibility inthose individuals still at least partially susceptible. Thesummary measure under heterogeneity is VE5SUM =1 - (1 - a)9.

Longini and Halloran (30) derived a frailty mixturemodel for estimating a, 0, and VESSUM when there isunmeasured heterogeneity in the vaccine effects on thehost's immune response. The model is also applicable ifthere is heterogeneity in exposure to infection, thoughthe interpretation of the estimates is different. Themodel falls into the general category of frailty models(33) employed in survival analysis, but like cure models(34), allows for a point mass at zero. It also incorporatesthe infection process into the baseline hazard rate. Themodel includes a continuous family of vaccine effectdistributions ranging between the two extremes of leakyand all-or-none effects (28). The model also can be usedfor individual time-to-event data (35). Longini andHalloran (30) applied the methods to estimate the VES

of measles vaccine in children from an observationalstudy of a measles outbreak in Muyinga, Burundi. Theyestimated that a proportion a = 0.80 of the vaccinatedchildren were fully protected (95 percent confidenceinterval: 0.69,0.92), but the 20 percent without full pro-tection had a higher hazard of measles illness than didthe unvaccinated children. This lead to an estimatedsummary VE5SUM of only 0.46 (95 percent confidenceinterval: 0.32, 0.67). The frailty model was shown toprovide an adequate fit to the measles data as indicatedby a nonsignificant chi-square goodness-of-fit statistic.This validated the use of the frailty model for estimatingmeasles vaccine efficacy in this setting.

Halloran et al. (31) explored the potential use of theabove-described frailty mixture model for the estima-tion of VE5 over the parameter space that covers thepossibilities of most vaccine studies. They showed thatthe parameters are identifiable under reasonable fieldconditions as long as there is not too much right cen-soring. Most importantly, they showed that the con-ventional VE5 estimators, i.e., proportional hazardsand cumulative incidence, can be considerably biasedwhen unmeasured heterogeneity is present. This bias isremoved when the correct frailty mixture model isused.

A general strategy for estimating VESi

We present a general strategy for estimating VEsx(t)from time-to-event or incidence data. The first step isto conduct diagnostics. Then, with the help of the diag-nostics, we find the best estimator of the VE5.

Diagnostics. We begin by constructing log-minus-log plots of the Kaplan-Meier or actuarial estimates ofthe survival curves for the unvaccinated and vacci-nated groups, such as those in figure 2a (31). Theseplots provide information about whether the vaccineeffect is leaky, all-or-none, or a mixture. In addition,they provide some information about whether vaccineinduced protection is waning. If the curves are parallel,then the effect is mostly leaky (multiplicative), and weshould model the vaccine effect with a proportionalhazards model. Any divergence from parallelism indi-cates time-varying effects and the presence of someform of heterogeneity and/or waning protection. In thiscase, a model other than the proportional hazardsmodel is needed. If the curves tend to diverge, thenthere is all-or-none effect, and if they tend to converge,then the model still may be leaky, but with an unmea-sured random effect (heterogeneity). Convergencecould also indicate waning protection. Although con-struction of log-minus-log plots is an important firstdiagnostic step, they are sometimes difficult to inter-pret (30). If there are a sufficient number of events, amore informative plot is a smoothed hazard ratio plotof VEsx{t) = 1 ~~ î(?)/A^(f) as described above.The possible patterns associated with different vaccineeffects are shown in figure 2b (31). A line with zeroslope indicates a purely leaky or multiplicative effect.The researcher can construct a formal hypothesis testfor zero slope (20, 24, 25).

Estimation. If there is no evidence of time-varyingeffects from the diagnostics, then the VES PH = 1 - (rcan be estimated by fitting a proportional hazardsmodel. If there is evidence of time-varying effects,then the investigator should fit the full family of frailtymixture models. If these models provide an adequatefit to the data, then the estimated parameters may be,but are not necessarily, the appropriate measures of theVES. If there is evidence of waning or other time-varying effects not attributable to unmeasured hetero-geneity, then the nonparametric estimate of VEs(f)itself will provide the best estimate. In this case, it maybe possible to construct a time-dependent parametricmodel of the VEs(r) that would provide tighter confi-dence intervals than the nonparametric approach.

Interpretation of measures. Which parameter to useto estimate VES in a particular study depends on thetype and duration of the study, the infectious agent andits transmission mode, the resources available, and theassumptions of the distribution of protection within thevaccinated group. Given the above discussion, thereare clear limits on the interpretability and generaliz-ability of estimates of VES. Even if time-dependenteffects are detected, knowledge of the underlying biol-ogy will need to be used to interpret the effects and to



Time60 100

1

0.8

fee

^ 0.4

0.2

0

ball-or-none

^ ' mixed effect

partial protection

20 40 60 80 100

Time

FIGURE 2. a, Diagnostic natural log In (-In Sy{t) survival plots checking the proportional hazards assumption for a vaccine conferring homo-geneous partial protection, an all-or-none vaccine, and a mixed degenerate vaccine model compared with the unvaccinated group, b, differentpossible 1 - hazard ratio shapes over time. Plots of 1 - hazard ratios for homogeneous partial protection (6 = 0.5), the all-or-none vaccine (a,= 0.5), and the mixed degenerate model (6 = 0.75, a, = 0.33). VESUM = 0.5 at time t0 = 0 in these three cases. VE, vaccine efficacy; SUM, sum-mary. From Halloran et al., Am J Epidemiol 1996;144:83-97. Reproduced with permission.

help choose between actual waning, boosting, or hetero-geneities. Most vaccine trials nowadays collect data onimmune response, some using a data structure of longi-tudinal data and repeated measures. This informationcan be used to help estimate and interpret vaccineeffects. Also, measuring actual or potential exposure toinfection in individuals will help identify heterogeneitiesin exposure to infection. Some trials of vaccines forvector-borne diseases have entomologic data. Thesehelp in quantifying potential exposure to infection.

That randomization does not control for confound-ing of estimates has been discussed by several authorsin the noninfectious disease setting (36-41).Struchiner and Halloran (42) show that these resultshold as well in randomized vaccine trials, but a newdimension is added when the possibly omitted covari-ate is exposure to infection, as is the case when goingfrom VE5p based on the transmission probability toVES based on the unconditional estimators.Differences in transmission intensity, previous expo-


82 Halloran et al.

sure to infection, and preexisting partial immunity andheterogeneities across communities result in differentVES estimates, even when the actual biologic actionof the vaccine is the same conditional on these factors.Reporting of any estimate of VES should include thegeneral elements of the study design, for instancewhether the estimate is based on events-per-persontime, the transmission probability, or final value data.It should also include indications concerning hetero-geneities or time-dependence of efficacy. However,Struchiner et al. (7) demonstrate that the complexbiology and possibly nonlinear relation between thebiologic mechanism of action of vaccine and the mea-sured outcome make it important to interpret the esti-mates with caution. Reviews of pertussis vaccinetrials in different populations using different estima-tors consider some of these issues (13, 43).

In addition, systems of causal inference that definecausal effects in terms of potential outcomes (44, 45),that is the potential outcome if a person receives a cer-tain treatment, say a vaccine, compared with if the per-son received another treatment, such as a placebo orother vaccine, generally assume that the outcomes inany individual are independent of the treatment assign-ment of other individuals. It is obvious that thisassumption does not hold in most infectious diseasesdue to the dependent happenings (1). Halloran andStruchiner (46) discuss several of the open questionsof causal inference in relation to vaccine studies.

Additional concerns. The definition of a case in avaccine study is important. The sensitivity and speci-ficity of the case definition can be crucial in determin-ing the magnitude of the efficacy or effectiveness esti-mate. Often several case definitions are used in theanalysis to explore how changing the case definitionalters the estimates of vaccine effect.

It is possible that there are several wild-type strainsof a parasite circulating in the host population. Thevaccine under investigation may contain just one anti-genic variant, or it may contain a cocktail of the sev-eral variants, but still not have all the wild-type vari-ants. As molecular epidemiology and immunediagnostic measures improve, strain-specific estimatesof VES and VE, will be in increasing demand. Gilbertet al. (47) discuss statistical methods for inferring howVES may vary with viral type when several wild-typestrains might be circulating in a vaccine study. Themethods are presented for final value data, that is levelIV, with the further additional information on eachinfection of what strain of wild-type parasite causedthe infection. Additional considerations includingadjusting for covariates, multiple events, and complexoutcomes, such as malaria in which morbidity is verycomplex, go beyond the scope of this review.

EFFECT ON PROGRESSION, VEp

As discussed above, the protection conferred by avaccine if a vaccinated person does become infected isof interest. A vaccine could alter post-infection diseasein several possible ways. The outcome of interestcould be the probability of developing disease or ofdeath in some specified time interval after becominginfected, the rate of progression to disease after infec-tion, or the severity of illness after becoming infected.Evaluation of the effect of prophylactic vaccination ondisease progression, VE/,, requires comparison of mor-bidity or mortality in vaccinated people who havebecome infected with that in infected unvaccinatedpeople. Thus, infection status must be determined sep-arately from disease status. If VEP were evaluated bythe relative rate of progression to disease, then obser-vation of the infected individuals over time would berequired. If VE/. is based on relative morbidity, thenappropriate definitions of morbidity levels would benecessary. Similar to the VES measures discussedabove, VE/, would be estimated by one minus the cor-responding ratio in the vaccinated compared with theunvaccinated, including in the calculation only thosepeople who had become infected.

VACCINE EFFECT ON INFECTIOUSNESS, VE,

The efficacy of a vaccine in reducing infectiousness,VE7, can be estimated epidemiologically by comparingthe per-contact transmission probability from vacci-nated people who become infected with the transmis-sion probability from unvaccinated people whobecome infected. The relative risk comparison groupsare defined according to the vaccination status of theinfectious person contacting the susceptible person(46). In table 1, the VE, estimate is shown in the sec-ond column of the top row of conditional parameters.For completeness, the third column contains the esti-mate of combined effect of the vaccine in reducing thetransmission probability if both the infectious personand the susceptible person in the contact are vacci-nated, (VEr). If we assume that the vaccine has a mul-tiplicative effect in reducing the transmission probabil-ity from a vaccinated infectious person to anunvaccinated person, then the efficacy for infectious-ness is VE/ = 1 — <{). In contrast to VE5, which can beestimated using either conditional or unconditionalparameters, the VE/ can generally be estimated usingonly conditional measures such as the transmissionprobability or secondary attack rate (46, 48-50). Bymaking strong modeling assumptions, Longini et al.(51) suggest a method for estimating the effect of thevaccine on infectiousness using grouped time-to-eventdata from studies in multiple populations.



Studies for estimating VE, can be incorporated intothose for estimating VE5p based on the transmissionprobability, if the vaccination status of the infectiousperson in a contact is known. The analysis can thensimply stratify on the vaccination status of both theinfectious and susceptible persons in the contact to getestimates of VE5, VE,, and VEr. In the case of thebinomial model, the likelihood can simply be con-structed from the different contributions of each con-tact, where the parameters 6 for relative susceptibilityand § for relative infectiousness are built directly intothe likelihood (49, 52). In this case, it is simplest toassume that the effect on infectiousness and suscepti-bility are multiplicative and independent, so that/?n, =

Estimating VES and VE, from multilevelinformation

In some vaccine studies, there may be informationon contacts within transmission units such as house-holds or sexual partnerships, but the individuals mayalso be exposed to infection outside of the transmis-sion unit. Such an example is the pertussis study byKendrick and Eldering (14) mentioned above. It mayalso be that some individuals in a study are not mem-bers of clearly defined transmission units. In thesecases, it is possible to develop a statistical model toexpress both the within household transmission proba-bility and the probability of being infected from thecommunity at large (53). Rather than presenting twoseparate analyses for conditional and unconditionalestimates, the probability model includes the probabil-ity of being infected within the transmission unit fromcontact with an infective and the probability of beinginfected outside the unit. In essence, then, the modelcombines elements of level I conditional parameterswith elements of levels II, III, or IV unconditionalparameters. People who are members of transmissionunits can contribute information to estimation of bothVE5 and VE,, while people who are not in transmissionunits and on whom no contact information is availablecontribute only to estimation of VE5.

Augmented vaccine trial design. It is possible todesign studies prospectively that intentionally makeuse of multilevel information in estimating vaccineefficacy. One such design is the augmented trialdesign (49, 52). For example, the trial may initiallyrecruit and randomize individuals by some usual eli-gibility criteria. Then the trial can be augmented byincluding information on contacts and transmissionunits such as households or partnerships of the pri-mary trial participants. This is one method to preservethe individual level analysis and randomization thatsome investigators think is important in randomized

controlled trials. The primary analysis can still befocused on estimating VE5, although estimation ofVE/ is also possible.

Using validation sets in vaccine studies. Collectinginformation on contacts between infectives and sus-ceptibles, though necessary for estimating VE, anduseful for estimating VE5, is difficult, expensive, andinherently prone to mismeasurement. An area of cur-rent research is the possible use of validation sets forexposure to infection to improve estimation of vaccineefficacy (54-56). For example, it might be possible toget good contact information on a small subset ofstudy participants, but only a coarse estimate on every-one else as well as the validation set. Then using meth-ods developed for missing data (57) and two-phasesampling designs (58-62), it is possible to decreasebias and increase efficiency by combining the differentlevels of information.

Efficacy estimates are seriously attenuated whenthe diagnosis of a particular disease is not culture-confirmed but based on a nonspecific case definition.For example, if the case definition in an efficacystudy of an influenza vaccine is fever and coughing,then the efficacy estimates could be much lower thanthe efficacy estimates based on culture-confirmedinfluenza. We are exploring the use of validation setsfor outcomes in vaccine efficacy and effectivenessstudies. In these designs, the diagnosis of the diseaseof interest would be confirmed by a specific test in aselected group of the study population. Statisticalmethods for combining validation sets with the largermain study in which the diagnosis is less specificwould be used to obtain efficient, unbiased estimatesof vaccine efficacy or effectiveness.

Sample size calculations. Sample size calculationsshould be based on the analysis method that is goingto be used. For example, if the analysis is going to bebased on a Cox proportional hazards model withcovariates, then the sample size calculation should bebased on that model, not on a model using final valuedata. Sample size calculations are frought with uncer-tainty. Even if baseline transmission intensities aremeasured, these may change abruptly during the trial.It is better to be generous in estimating the number ofpeople required in the study. Precise estimation ofboth VE, and VE/> require larger samples than esti-mates of VES because they both are measured on peo-ple who have already become infected, which mightbe a small fraction of the participants. If more com-plex effects such as waning, frailty models, or VE,are to be estimated, it is good to simulate the pro-posed vaccine study to examine what numbers of par-ticipants are required to produce the precisiondesired.


84 Halloran et al.

CONTACT RATES AND EXPOSURE EFFICACY

Vaccinated people may alter their contact and expo-sure to infection patterns if they believe the vaccine isprotective. Exposure or behavior efficacy is the relativeincrease or decrease in the relative risk of infection ordisease due to the change in exposure to the infectiousagent (54). For example, if we consider the componentsof the hazard rate as discussed above, changes in expo-sure to the infectious agent can occur in the rate of con-tacts, in the prevalence of infection in the contactgroups, or in the transmission probability throughchanging the type of contact. In nonrandomized orobservational studies, the vaccinated and unvaccinatedgroups often differ in their exposure to infection, result-ing in biased estimates of VE5. Although VE5 estimatesbased on the transmission probability require moreinformation than those based on the unconditionalparameters, they are less sensitive to bias from unequalexposure to infection in the two groups. The overalleffect of biologic protection and change in exposure toinfection might be of interest for understanding thepublic health consequences of vaccination. Studydesigns need to be explicit about differentiating factorsrelated to susceptibility, such as vaccination status, andfactors related to exposure to infection.

INDIRECT, TOTAL, AND OVERALLEFFECTIVENESS

Interest in evaluating the indirect and overall effectsof vaccination strategies as part of phase III as well as

POPULATION A POPULATION B

DESIGN moverall

A

DESIGN!direct

DESIGN Haindirect

DESIGN l ibdirect + indirect

FIGURE 3. Study designs for dependent happenings. Types ofeffects of interventions against infectious disease, and differentstudy designs based on comparison populations for their evalua-tions. From Halloran and Struchiner, Epidemiology 1991;2:331-8.Reproduced with permission.

post-licensure is increasing (63, 64). Struchiner et al.(5) and Halloran and Struchiner (15) define studydesigns for dependent happenings that allow evalua-tion of the indirect and overall effects of vaccination(figure 3). Since the population-level effects of a vac-cine are defined within the context of a particular inter-vention program, or allocation of vaccination, the unitof inference is the population, and several populationsor communities should be included in the study.

Defining the intervention program or allocation ofinterest and what the comparison program or alloca-tion of interest is can be complicated. The comparisonor control populations might have no vaccination atall. The controls may be the same populations thatreceive the vaccination, but before the vaccination pro-gram started. In table 1 and figure 3, the different typeof population level effects are considered on the sim-ple example that no vaccination has taken place inpopulation B, and a proportion of people are vacci-nated in population A. The indirect effects of the vac-cine given a specific allocation of vaccination is thenthe comparison of the incidence or other outcome ofinterest in the unvaccinated people in community Acompared with the unvaccinated people in the unvac-cinated community B. These comparisons are calleddesigns type IIA. The indirect effectiveness measuresare designated VEIIA. The total effects of the combina-tion of being vaccinated and the allocation is the out-come in the vaccinated people in the communities Acompared with that of the unvaccinated people in theunvaccinated communities B. These comparisons arecalled designs type IIB, and the total effectivenessmeasures are designated VEIIB. The overall effective-ness of the vaccine and allocation compare the averageoutcomes in the vaccinated communities with those ofthe unvaccinated communities. These comparisons arecalled designs type III, and the overall effectivenessmeasures are designated VEni. Table 1 contains exam-ples of the VEnA, VEUB, and VEm based on the usualunconditional measures incidence rate, hazard rate,and cumulative incidence. Many other measures couldbe used, including average age of infection or the basicreproductive number, RQ.

It is important in choosing the communities or pop-ulations to assure that they are separated as much aspossible in every way that is relevant for transmission.If the populations are not transmission dynamicallyseparated, then the indirect effects of the programs willnot actually differ among the groups, and the studywill yield an attenuated estimate of the potential effectof intervention. Transmission patterns can differgreatly among communities. The variability couldmask the effects of vaccination. Matching by transmis-sion characteristics is an option to consider (65).



Exactly what the intervention program of interest iswill depend on the vaccine and which subgroups suf-fer the greatest morbidity. The comparisons may bemade between different levels of vaccination cover-age, between allocation within different age groups orotherwise defined subgroups. For example, the inter-vention of interest may be vaccination of school-agechildren against influenza, with the primary outcomeof interest the reduction in influenza incidence inunvaccinated adults. That is, the primary goal of thestudy might be to evaluate the pure indirect effects inunvaccinated adults. The comparison of interest wouldbe between unvaccinated adults in populations inwhich children were not vaccinated and unvaccinatedadults in populations with vaccinated children. If therisk measure used was cumulative incidence, then themeasure would be VEIIACI in unvaccinated adults.Another outcome of interest might be the overallreduction of cumulative incidence in one influenzaseason in school-age children, VEincI, the indirecteffects in unvaccinated children of vaccinating a por-tion of the children, VEIIA CI, or the total effects in thevaccinated children, VEtIB CI, all compared with unvac-cinated children in the unvaccinated populations.

Example

Monto et al. (66) estimated both the protective effi-cacy, VES, and the overall effect, VEni, of aninfluenza vaccination program. They vaccinated 85percent of the school-age children in Tecumseh,Michigan, against Hong Kong influenza just beforethe epidemic in 1968. The 10-week epidemic periodwas from November 17, 1968, to January 26, 1969.The weekly mean influenza illness rates in vaccinatedand unvaccinated children were 0.072 and 0.090,respectively. This yields an approximate estimate of

0.072VE5 jR ~ 1 = 0.20, which is rather low. The

overall influenza illness cumulative incidence inTecumseh for the epidemic period was 0.14, while theadjusted overall influenza cumulative incidence inunvaccinated, neighboring Adrian, Michigan, was 0.42for the same period. Using the methods of study designIII, the overall effectiveness of vaccinating 85 percentof Tecumseh's school children is estimated to beVEUIjCI~ 1 - 0.14/0.42 = 0.67.

When designing a population-level study, it is nec-essary to give some thought to the likely transmissionpatterns and sources of exposure to infection in a pop-ulation. These transmission patterns will greatly influ-ence the magnitude of the indirect effects. Forinstance, if one is interested in evaluating the indirecteffects in preschool children of pneumococcal vacci-

nation, one may decide to vaccinate a large fraction ofpreschool children in several populations and compareit with incidence in preschool children in unvaccinatedpopulations. However, if a large amount of the sourceof exposure to infection is from older children who arenot included in the study, then the indirect effects ofvaccination could be quite low. Vaccination of theolder children as well may be necessary to have mea-surable indirect effects.

Because of the great variability among communities,precise definition of the actual intervention of interestwill usually be difficult. When designing a study thatincludes several populations, the definition of theentire allocation and its implementation play an impor-tant role in the interpretability of the outcome. Therewill be limits in the interpretability and general applic-ability of the results to other settings.

Comparisons across communities would also allowstudy of other biologic questions. For example, vac-cines might contain only particular serotypes or strainsof an organism. Widespread vaccination could allowthe expansion of nonvaccine serotypes that had beenless important before vaccination (67, 68) or put evo-lutionary pressure on the existing strains. Comparisonacross populations or before and after introduction ofvaccination would be the method to evaluate suchchanges.

Many issues related to the design and interpretationof studies to evaluate indirect and overall effects areactive areas of research and go beyond the scope ofthis review. Community trials fall into the category ofcluster or group randomized trials where whole socialunits, rather than independent individuals are ran-domly assigned to treatment groups (65, 69-73). Theanalysis and sample size calculations need to take theclustering and possible group randomization intoaccount. Unmeasured heterogeneity may be dealt withusing random effects models in the analysis phase,although such models for use in the dependent hap-penings setting have not yet been fully developed.Other approaches for examining population leveleffects include time series and population dynamicmodels.

Comparison with prevented fractions in noninfec-tious diseases

The unconditional vaccine effect parameters areanalogous to the family of prevented fraction parame-ters discussed by Greenland and Robins (74, 75), withsome essential differences. In their work, the preventedfraction in the group with a risk factor is estimated bycomparing the cumulative incidence in the individualswith a risk factor to cumulative incidence in individu-als without the risk factor in a study design similar to

Epidemiol Rev Vol.21, No. 1, 1999

86 Halloran et al.

study design I with level IV information. The groupwithout the risk factor is supposed to represent whatwould have happened to the group with the risk factorhad it not had the risk factor. The number of preventedcases in the group with the risk factor can be estimatedfrom the prevented fraction based on the cumulativeincidence if it is known how many people had the riskfactor. Under dependent happenings, however, in studydesign I, say in population A, the number of cases in theunvaccinated individuals does not represent the numberof cases that would have occurred in the unvaccinatedindividuals had the vaccination program not takenplace. If vaccination is widespread enough, the cumu-lative incidence in the unvaccinated group will usuallybe lower in the presence of the vaccination programthan if no one had been vaccinated. Thus, the estimatednumber of cases prevented generally will underesti-mate the actual number of cases prevented in the vacci-nated group if calculated using methods for noninfec-tious diseases as in study design I. The comparisonneeds to be made between the cumulative incidence inthe vaccinated group and what the cumulative inci-dence would have been in the unvaccinated group if novaccination program had taken place, as in study designIIB. A similar argument applies to estimation of the pre-vented hazard fraction.

Multiple questions within a study

Conducting a trial to evaluate effectiveness acrossseveral different populations or communities does notpreclude evaluating VES or VE, of vaccination withinthe populations. A phase lH vaccine trial can bedesigned to answer several questions at the same time.Randomization within a population can be used toanswer efficacy questions, while comparison acrosspopulations can be used to evaluate the indirect andoverall effects of vaccination. Consider a study of vac-cination in several populations to measure the indirect,total, and overall effects of vaccination with differentlevels of coverage in each population. Within eachpopulation, a comparison can be made of the relevantvaccinated and unvaccinated portions of the popula-tion to estimate VEIIA, VEIIB and VEm. If informationis gathered within the populations on actual contacts,then the effect of the vaccine on infectiousness as wellas susceptibility could be evaluated. The most impor-tant consideration in designing a vaccine study is to beclear about the effect(s) or question(s) of interest, andthe level of information that can be gathered. Then theparameters of interest and the choice of comparisonpopulations should be chosen to provide the effectmeasures of interest. There is a tradeoff in designingstudies to measure both direct and indirect effects of

vaccination between vaccinating high numbers of peo-ple so that indirect effects are high, and vaccinating toomany people so that the number of events in the vacci-nated populations is too low to estimate VE5 or VE,well (51). Combining estimates of VE5 and VE, acrosspopulations is not necessarily straightforward (42) dueto lack of exchangeability (76).

SAFETY

The Food and Drug Administration in the UnitedStates is putting more emphasis on evaluating rareadverse events related to the vaccine during phase IIItrials. Generally, evaluation of rare adverse events hastaken place after licensure when large numbers of peo-ple are vaccinated. After licensure, most studies will notbe randomized, however, creating challenges for thedesign and interpretation of such studies. Although thisbrief section cannot do justice to the problem of safetyevaluation, we include it as a reminder that safety eval-uation belongs to all phases of vaccine studies.

An example of an adverse event is the possible asso-ciation between inactivated influenza vaccine andGuillian-Barre syndrome. Since there may be a back-ground level of such events that are not vaccinerelated, the rate of adverse events in the vaccinated iscompared with that in the unvaccinated. If we let theseadverse event rates be rx and r0 in the vaccinated andunvaccinated, respectively, then the null hypothesisthat the adverse events are not associated with the vac-cine is HQ : /-, < r0. We have the inequality in the nullhypothesis since we will suspect that the vaccine isassociated with the adverse events only if the adverseevent rate is greater in the vaccinated. Usually the totalnumber of adverse events is small, thus the event rateis assumed to follow a Poisson distribution. A goodstrategy for carrying out the above test is to conditionon the total number of adverse events in both groups,and then construct the uniformly most powerful unbi-ased test based on the resulting binomial model (77). Ifthe number of vaccinated and unvaccinated people atrisk is reasonably small, then Fisher's exact test isused. For a large number of vaccinated and unvacci-nated people, the binomial approximation is used.Several further approximations for large samples arealso available (78).

In a randomized, double-blinded, phase EH vaccinetrial, the unvaccinated group consists of people whoget placebo or some other vaccine. However, forunblinded trials, such as large community trials orpost-licensure phase IV trials and surveillance, thecomparison group must be carefully selected fromamong unvaccinated comparable people. These typesof studies provide an excellent opportunity for adverseevents analysis since large numbers of people are vac-



cinated, but issues concerning comparability must beaddressed (79, 80).

SUMMARY

There are many different effects to consider whenevaluating vaccines in the field. In this review, wehave covered some of the various measures and issuesrelated to study design and interpretation of the differ-ent measures. We emphasize that in designing andunderstanding vaccine studies, it is necessary to bespecific about what the effect of interest is and aboutthe assumptions underlying the interpretation of theresults. Halloran et al. (81) present design, analysis,and interpretation of vaccine studies in more detail.

ACKNOWLEDGMENTS

This work was partially supported by NIH NIAID grantsR01-AI32042, R01-AI40846, and the Brazilian ResearchCouncil (CNPq).

REFERENCES

1. Ross R. An application of the theory of probabilities to thestudy of a priori pathometry, part 1. Proc R Soc Series A1916;92:204-30.

2. Halloran ME, Struchiner CJ, Longini IM Jr. Study designsfor evaluating different efficacy and effectiveness aspects ofvaccines. Am J Epidemiol 1997;146:789-803.

3. Rosenbaum PR. Observational studies. New York, NY:Springer-Verlag, 1995.

4. Robins JM, Rotnitzky A, Scharfstein DO. Sensitivity analy-sis for selection bias and unmeasured confounding in missingdata and causal inference models. In: Halloran ME, BerryDA, eds. Statistics in epidemiology, environment and clinicaltrials. New York, NY: Springer-Verlag (in press).

5. Struchiner CJ, Halloran ME, Robins JM, et al. The behaviourof common measures of association used to assess a vacci-nation programme under complex disease transmission pat-terns—a computer simulation study of malaria vaccines. IntJ Epidemiol 1990;19:187-96.

6. Rodrigues LC, Smith PG. Case-control approach in vaccineevaluation: efficacy and adverse effects. Epidemiol Rev1999;21:56-72.

7. Struchiner CJ, Halloran ME, Brunet RC, et al. Malaria vac-cines: lessons from field trials. Cadernos do Saiide Publica1994; 10(Suppl2):310-26.

8. Smith PG, Rodrigues LC, Fine PEM. Assessment of the pro-tective efficacy of vaccines against common diseases usingcase-control and cohort studies. Int J Epidemiol 1984;13:87-93.

9. Greenwood M, Yule GU. The statistics of anti-typhoid andanti-cholera inoculations, and the interpretation of such sta-tistics in general. Proc R Soc Med 1915;8(part 2): 113-94.

10. Fox JP, Hall CE, Elveback LR. Epidemiology: man and dis-ease. New York, NY: MacMillan, 1970.

11. Orenstein WA, Bernier RH, Dondero TJ, et al. Field evalua-tion of vaccine efficacy. Bull World Health Organ 1985;63:1055-68.

12. Orenstein WA, Bernier RH, Hinman AR. Assessing vaccineefficacy in the field: further observations. Epidemiol Rev198810212U

13. Fine PEM, Clarkson JA, Miller E. The efficacy of pertussisvaccines under conditions of household exposure: furtheranalysis of the 1978-80 PHLS/ERL study in 21 area healthauthorities in England. Int J Epidemiol 1988; 17:635^2.

14. Kendrick P, Eldering G. A study in active immunizationagainst pertussis. Am J Hyg 1939;29B: 133-53.

15. Halloran ME, Struchiner CJ. Study designs for dependenthappenings. Epidemiology 1991;2:331-8.

16. Rhodes PH, Halloran ME, Longini IM Jr. Counting processmodels for differentiating exposure to infection and suscep-tibility. J R Stat Soc B 1996;58:751-62.

17. Urdaneta M, Prata A, Struchiner CJ, et al. Evaluation ofSPf66 malaria vaccine efficacy in Brazil. Am J Trop MedHyg 1998;58:378-85.

18. Belshe RB, Mendelman PM, Treanor J, et al. The efficacy oflive attenuated, cold-adapted, trivalent, intranasal influenza-virus vaccine in children. N Engl J Med 1998,338:1405-12.

19. Clemens JD, Sack DA, Harris JR, et al. Field trial of oralcholera vaccines in Bangladesh: results from three-yearfollow-up. Lancet 1990;335:270-3.

20. Durham LK, Halloran ME, Longini IM Jr, et al. Smoothingmethods for exploring time-dependent vaccine effects. ApplStat 1999;48:395^07.

21. Hastie T, Tibshirani R. Varying-coefficients models. J R StatSoc B, 1993;55:757-96.

22. Whitehead J. Fitting Cox's regression model to survival datausing GLIM. Appl Stat 1980;29:268-75.

23. Schoenfeld D. Partial residuals for the proportional hazardsregression model. Biometrika 1982;69:239Û.

24. Grambsch PM, Therneau TM. Proportional hazards tests anddiagnostics based on weighted residuals. Biometrika 1994;81:515-26.

25. Durham LK, Longini IM Jr, Halloran ME, et al. Estimationof vaccine efficacy in the presence of waning: application tocholera vaccines. Am J Epidemiol 1998; 147:948-59.

26. Halloran ME, Haber M, Longini IM Jr, et al. Direct and indi-rect effects in vaccine field efficacy and effectiveness. Am JEpidemiol 1991; 133:323-31.

27. Haber M, Longini IM Jr, Halloran ME. Measures of theeffects of vaccination in a randomly mixing population. Int JEpidemiol 1991;20:300-10.

28. Halloran ME, Haber M, Longini IM Jr. Interpretation andestimation of vaccine efficacy under heterogeneity. Am JEpidemiol 1992; 136:328-43.

29. Brunet RC, Struchiner CJ, Halloran ME. On the distributionof vaccine protection under heterogeneous response. MathBiosci 1993; 116:111-25.

30. Longini IM Jr, Halloran ME. A frailty mixture model for esti-mating vaccine efficacy. Appl Stat 1996;45:165-73.

31. Halloran ME, Longini IM Jr, Struchiner CJ. Estimability andinterpretation of vaccine efficacy using frailty mixing mod-els. Am J Epidemiol 1996;144:83-97.

32. Halloran ME, Struchiner CJ, Spielman A. Modeling malariavaccines. II: Population effects of stage-specific malaria vac-cines dependent on natural boosting. Math Biosc 1989;94:115^19.

33. Vaupel JW, Manton KG, Stallard E. The impact of hetero-geneity in individual frailty on the dynamics of mortality.Demography 1979; 16:439-54.

34. Farewell VT. The use of mixture models for the analysis ofsurvival data with long-term survivors. Biometrics 1982;38:1041-6.

35. Haber M, Miao G. Frailty models for estimating vaccine effi-cacy from continuous time data. In: Bulletin of theInternational Statistical Institute, 51st Session, Istanbul,Turkey, August 1997:247-8.

36. Greenland S, Robins JM. Identifiability, exchangeability, andepidemiological confounding. Int J Epidemiol 1986;15:412-19.


88 Halloran et al.

37. Greenland S. Interpretation and choice of effect measures inepidemiologic analyses. Am J Epidemiol 1987;125:761-8.

38. Gail MH. Adjusting for covariates that have the same distri-bution in exposed and unexposed cohorts. In: MoolgavkarSH, Prentice RL, eds. Modern statistical methods in chronicdisease epidemiology. New York, NY: Wiley, 1986:3-18.

39. Gail MH. The effect of pooling across strata in perfectly bal-anced studies. Biometrics 1988;44:151-62.

40. Gail MH, Tan WY, Piantadosi S. Tests for no treatment effectin randomized clinical trials. Biometrika 1988;75:57-64.

41. Gail MH, Wieand S, Piantadosi S. Biased estimates of treat-ment effect in randomized experiments with nonlinearregressions and omitted covariates. Biometrika 1984;71:431-44.

42. Struchiner CJ, Halloran ME. Randomization and baselinetransmission in vaccine field trials. Technical report, biosta-tistics. Atlanta, GA: Emory University, 1996.

43. Fine PEM, Clarkson JA. Reflections on the efficacy of per-tussis vaccines. Rev Infect Dis 1987;9:866-83.

44. Rubin DB. Bayesian inference for causal effects: the role ofrandomization. Ann Stat 1978;7:34-58.

45. Holland PW. Statistics and causal inference. J Am Stat Assoc1986;81:945-60.

46. Halloran ME, Struchiner CJ. Causal inference for infectiousdiseases. Epidemiology 1995 ;6:142-51.

47. Gilbert PB, Self SG, Ashby MA. Statistical methods forassessing differential vaccine protection against humanimmunodeficiency virus types. Biometrics 1998;54:799-814.

48. Koopman JS, Little RJ. Assessing HIV vaccine effects. Am JEpidemiol 1995; 142:1113-20.

49. Longini IM Jr, Datta S, Halloran ME. Measuring vaccineefficacy for both susceptibility to infection and reduction ininfectiousness for prophylactic HIV-1 vaccines. J AcquirImmune Defic Syndr Hum Retrovirol 1996;13:440-7.

50. Halloran ME. [Evaluating HIV vaccines] Discussion. StatMed 1998;15:2405-12.

51. Longini IM Jr, Sagatelian K, Rida WN, et al. Optimal vac-cine trial design when estimating vaccine efficacy for sus-ceptibility and infectiousness from multiple populations. StatMed 1998; 17:1121-36.

52. Datta S, Halloran ME, Longini IM Jr. Augmented HIV vac-cine trial design for estimating reduction in infectiousnessand protective efficacy. Stat Med 1998;17:185-200.

53. Longini IM Jr, Koopman JS. Household and communitytransmission parameters from final distributions of infectionsin households. Biometrics 1982;38:115-26.

54. Halloran ME, Longini IM Jr, Haber MJ, et al. Exposure effi-cacy and change in contact rates in evaluating prophylacticHIV vaccines in the field. Stat Med 1994; 13:357-77.

55. Golm GT, Halloran ME, Longini IM Jr. Semi-parametricmodels for mismeasured exposure information in vaccinetrials. Stat Med 1998; 17:2335-52.

56. Golm GT, Halloran ME, Longini IM Jr. Semiparametricmethods for multiple exposure mismeasurement and a bivari-ate outcome in HIV vaccine trials. Biometrics 1999;55:94-101.

57. Little RJA, Rubin DB. Statistical analysis with missing data.New York, NY: Wiley, 1987.

58. Prentice RL. A case-cohort design for epidemiologic cohortstudies and disease prevention trials. Biometrika 1986;73:1-11.

59. Flanders WD, Greenland S. Analytic methods for two-stage

case-control studies and other stratified designs. Stat Med1991; 10:739^*7.

60. Pepe MS, Fleming TR. A nonparametric method for dealingwith mismeasured covariate data. J Am Stat Assoc 1991;86:108-13.

61. Reilly M, Pepe MS. A mean score method for missing andauxiliary covariate data in regression models. Biometrika1995;82:299-314.

62. Robins JM, Rotnitzky A, Zhao LP. Estimation of regressioncoefficients when some regressors are not always observed.J Am Stat Assoc 1994;89:846-66.

63. Fine PEM. Herd immunity: history, theory, practice.Epidemiol Rev 1993;15:265-302.

64. Clemens J, Brenner R, Rao M, et al. Evaluating new vaccinesfor developing countries: efficacy or effectiveness? JAMA1996;275:390-7.

65. Hayes R, Mosha F, Nicoll A, et al. A community trial of theimpact of improved sexually transmitted disease treatmenton HIV epidemic in rural Tanzania. 1. Design. AIDS 1995 ;9:919-26.

66. Monto AS, Davenport FM, Napier JA, et al. Effect of vacci-nation of a school-age population upon the course of an A2-Hong Kong influenza epidemic. Bull World Health Organ1969;41:537^2.

67. Lipsitch M. Vaccination against colonizing bacteria with mul-tiple serotypes. Proc Natl Acad Sci U S A 1997;94:6571-6.

68. Lipsitch M. Bacterial vaccines and serotype replacement:lessons fron Haemophilus influenza and prospects forStreptococcus pneumoniae. Emerg Infect Dis 1999;5:336-45.

69. Koepsell TD, Wagner EH, Cheadle AC, et al. Selectedmethodological issues in evaluating community-based healthpromotion and disease prevention programs. Annu RevPublic Health 1992;13:31-57.

70. Donner A. Some aspects of the design and analysis of clusterrandomization trials. Appl Stat 1998;47:95-114.

71. Prentice RL, Sheppard L. Aggregate data studies of diseaserisk factors. Biometrika 1995;82:113-25.

72. Klar N, Gyorkos T, Donner A. Cluster randomization trials intropical medicine: a case study. Trans R Soc Trop Med Hyg1995;89:454-9.

73. Murray DM. Design and analysis of group-randomized trials.New York, NY: Oxford University Press, 1998.

74. Greenland S, Robins JM. Conceptual problems in the defini-tion and interpretation of attributable fraction. Am JEpidemiol 128:1185-97, 1988.

75. Robins JM, Greenland S. Estimability and estimation ofexcess and etiologic fraction. Stat Med 1989;8:845-59.

76. De Finetti B. Theory of probability: a critical introductorytreatment. New York, NY: Wiley, 1974.

77. Norwood PK, Sampson AR. A statistical methodology forpostmarketing surveillance of adverse drug reaction reports.Stat Med 1988 ;7:1023-30.

78. Tsong Y. False alarm rates of statistical methods used indetermining increased frequency of reports on adverse drugreaction. J Biopharm Stat 1992;2:9-30.

79. Fine PEM, Chen RT. Confounding in studies of adverse reac-tions to vaccines. Am J Epidemiol 1992; 136:121-35.

80. Farrington CP. Relative incidence estimation from case seriesfor vaccine safety evaluations. Biometrics 1995;51:228-35.

81. Halloran ME, Longini IM Jr, Struchiner CJ. Design, analysis,and interpretation of vaccine studies. New York, NY:Springer-Verlag (in press).


Documents

Design and Interpretation of Vaccine Field Studiescourses.washington.edu/b578a/readings/halloran1999.pdf · 2 Escola Nacional de Saude Publica, Fundacao Oswaldo Cruz, Rio de Janeiro,