pharmacoepidemiology and drug safety 2004; 13: 511–513Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/pds.999

EDITORIAL

Clinical judgment, common sense and adverse reactionreporting{

In this issue of the Journal, Zhou et al.1 report a signalof a possible increase in the risk of Bell’s palsy in reci-pients of parenteral inactivated influenza vaccine. Theycarried out their investigation because of 46 adversereaction reports received in Switzerland following theintroduction of an inactivated nasal vaccine.2

In data collected from 1991 to 2001 in the UnitedStates Vaccine Adverse Event Reporting System(VAERS), out of a total of 154 verified cases of Bell’spalsy in individuals who received influenza vaccine,there were 130 who received the vaccine alone, forwhom the onset dates were known. Over that timeinterval some 560 million doses were distributed, andbased on the numerator of 130 cases the estimatedreporting rate was 0.23 per million, with rates perinfluenza season varying between 0.15 and 0.46 permillion. The proportional reporting ratio (PRR) (whichis a ratio of two ratios: numerator, the ratio of reports,following influenza vaccination, of Bell’s palsy toreports of all other adverse events; denominator, thesame ratio for all other vaccines combined) was 3.78—and significantly in excess of 2.0, the rate deemed toconstitute a signal of a possible drug-attributableadverse association.2 Times of onset after influenzavaccination, expressed as percentages of all reports ofBell’s palsy, were compared with the correspondingpercentages of Bell’s palsy among all adverse eventreports after all vaccinations: the respective propor-tions during the first day were approximately 7 and35%, whereas during the first 30 days they were 77 and43%. The authors conclude that the signal they haveidentified justifies ‘a population-based controlled

study to determine whether this association could becausal and to quantify the risk’.

In technical terms, the hypothesis that one form ofinfluenza vaccine may increase the risk of Bell’s palsyhas been generated in Switzerland, and at this stage itmay be reasonable to ask whether there is anyevidence, even if tentative, to suggest that a relatedvaccine should also be suspected, and that furtherepidemiological investigation may be justified. Herethe issue is whether the study of Zhou et al. constitutesan adequate signal to support such a suspicion. Andbeyond that immediate concern, the global questionraised by this study is whether the concept of signalgeneration based on PRRs is valid. Unfortunately, thestudy has many limitations some minor, and somemajor; but it does serve to shed light on the globalquestion.

First some examples of minor deficiencies: contraryto what was stated, Bell’s palsy is commonlypermanent, not temporary; the definition of theoutcome was nonspecific and inevitably subject toerror because of the incompleteness (especially thelongitudinal incompleteness) of adverse reactionreporting (multiple sclerosis e.g. could not beexcluded); and limited numbers were overinterpreted(e.g. it was claimed that there were more reports ofBell’s palsy among males than among females agedless than 18 years, based on numerators of 3 and 2respectively).

Standing alone these and other minor deficienciesmight perhaps have been insufficient to nullify apurported signal, but there were also major deficien-cies. The authors acknowledged the problem ofunder-reporting, and of potentially biased reporting.However, they failed to consider either the magnitudeof the under-reporting or the extent and complexity ofthe possible bias.

With regard to under-reporting, the estimatedincidence of Bell’s palsy cited by the authors is 13–40.2 per 100 000 person-years.3,4 Influenza vaccine is

Received 22 December 2003Copyright # 2004 John Wiley & Sons, Ltd. Accepted 22 December 2003

* Correspondence to: S. Shapiro, MB, FRCP(E), Emeritus Director,Slone Epidemiology Center, Boston University, Boston; VisitingProfessor, Mailman School of Public Health, Columbia University;Visiting Professor, University of Cape Town Medical School.E-mail: sydzeev@aol.com{Conflicts of interest: the author currently acts, and has in the pastacted, as a consultant to pharmaceutical manufacturers.

administered only once in any given year; thus thecorresponding reporting rate was roughly 0.023 per100 000 per year (or slightly higher if all reports, ratherthan verified ones only, are considered), or about 0.06–0.2% of the estimated incidence. Or put another way,under the null hypothesis the under-reporting of Bell’spalsy among influenza vaccine recipients was of theorder of at least 500-fold, and it could have been inexcess of 1600-fold. In addition, since the PRR was3.78, the under-reporting among recipients of othervaccines was some at least some 2000-fold, and quitepossibly much greater. Moreover, those estimates areconservative since the above mentioned incidence rateswere observed either in the population at large,4 or in amilitary population,3 not in an elderly populationrepresentative of influenza vaccine recipients.

In short, the under-reporting of Bell’s palsy in theVAERS database was so massive that it constitutedvirtually no information relevant to the hypothesis atissue.

With regard to potential biases, some may argue thatunder the null the reporting rate should be the same forall vaccines—the massive under-reporting notwith-standing. However, it is likely that there were so manypotential biases that there is no reason why the ratesshould have been the same. To begin with, influenzahas a viral etiology, Bell’s palsy is suspected to haveone, and both diseases are more common in winter.3

It is possible, even likely, that persons who haveearly symptoms of as yet undiagnosed Bell’s palsy(e.g. fever) at the commencement of an influenzaepidemic might tend selectively to seek an influenzavaccination: that is, there could be confounding byindication, or violation of time order, or both.

The assumption of equal under-reporting also ignoresthe likelihood of bias because an outcome such asBell’s palsy would be susceptible to different degreesof under-reporting, and different patterns of under-reporting, for different vaccines—and that the varia-bility is also likely to change over time. In addition, it islikely that the reporting of the wide array of adverseevents constituting the denominator of the PRR wouldalso vary in the same or even more complex ways.

That general point is well illustrated in the study ofZhou et al. by the lower percentage of reporting, amonginfluenza vaccine recipients relative to recipients of allvaccines, of Bell’s palsy (as a proportion of all reportedreactions) on the first post-vaccination day, followed bya reversal to a much higher proportion during the first 30days. As is clinically obvious, the first day would tend tobe dominated by acute events (e.g. anaphylactoidreactions, urticaria, fever), while in the days and weeksthereafter reactions of slower onset (e.g. encephalo-

pathy, renal failure, jaundice) would become morecommon. Thus, for example, if acute immune typereactions are less commonly reported for influenza thanfor other vaccines, this would be sufficient to explainthe findings. And if that were not enough, clinicalsuspicion might wane, and wane differently, fordifferent vaccines, and for different outcomes, depend-ing on the time lapse after vaccination.

And still further, when it comes to the computationof PRRs the fallacies multiply exponentially. Considerthe reactions just mentioned: not only do the aboveconsiderations limit their interpretability, but thedifferent reactions also affect different organ systems,and have different covariate risk factors, differentpathologies and different likelihoods of being reported.The act of combining all outcomes other than the one ofinterest into a single entity to be used as a denominatoris to combine unlike things; then, to specify a singleoutcome whose reporting is also variable as thenumerator produces an uninterpretable result; and thento compute two uninterpretable ratios, one for thevaccine under suspicion, and one for all other vaccinescombined—and then to divide the one ratio by theother in order to obtain a further ratio is to compoundthe uninterpretability of already uninterpretable datastill further.

Thus, in considering the first question, whether thereis evidence to signal a possible increase in the risk ofBell’s palsy in influenza vaccine recipients, the study ofZhou et al. is not contributory. If it is neverthelessjudged that there are sufficient grounds to justify anepidemiological investigation, that judgment must bebased on considerations other than the findings in thisstudy. And with regard to the global question, whetherthe concept of PRRs is valid, the findings demonstratewith a ‘real life’ example that it is not.

The use of PRRs is not the first attempt tosuperimpose algorithms of some sort on adversereaction reporting. This approach is unfortunatebecause case reports, properly interpreted using welldeveloped clinical skills and judgment, combined withcommon sense, have always played a cruciallyimportant role in the discovery and documentation ofadverse drug reactions: quite possibly they have playeda more important role than any other approach. Inaddition, informed collaboration between clinicalmedicine and epidemiology has often commencedwith astute clinical observation, and then gone furtherto produce some striking successes. It is beyond thescope of this Editorial to consider that topic any further,beyond noting that such collaboration has resultedin the documentation and quantification of associa-tions such as thalidomide and phocomelia,5 oral

512 editorial

Copyright # 2004 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2004; 13: 511–513

contraceptives and liver cancer6 and felbamate andaplastic anemia,7 to give just a few examples.

The ascendancy of algorithms is also unfortunatebecause it debases clinical medicine, and insteadaccords the leadership role in causal research tostatistics. That discipline, to be sure, is an indispen-sable tool in such research, but it is a tool. Adversereactions are clinical phenomena, and all research,clinical, epidemiological or experimental, must begoverned by that fact. I believe that one task facingthose who are concerned with how to improve adversedrug reaction reporting must be to explore how furtherto make optimal use of the best features of clinicalmedicine, clinical pharmacology and related disci-plines—and, of course, epidemiology. And for thatpurpose clinical medicine should occupy the driver’sseat.

Samuel Shapiro MB, FRCP(E)*

REFERENCES

1. Zhou W, Pool V, DeStefano F, et al. A potential signal of Bell’spalsy after parenteral inactivated influenza vaccines: reports tothe Vaccine Adverse Event Reporting System (VAERS)—United States, 1991–2001. Pharmacoepidemiol Drug Safe2004; 13: 505–510.

2. Mutsch M, Zhou W, Rhodes P, et al. Use of the inactivatedintranasal influenza vaccine and the risk of Bell’s palsy inSwitzerland. N Engl J Med 2004; 350: 896–903.

3. Campbell KE, Brundage JF. Effects of climate, latitude, andseason on the incidence of Bell’s palsy in the US armed forces,October 1997 to September 1999. Am J Epidemiol 2002; 156:32–39.

4. Bleicher JN, Hamiel S, Gengler JS. A survey of facial paralysis:etiology and incidence. Ear Nose Throat J 1996; 75: 355–358.

5. Lenz W. Thalidomide and congenital abnormalities. Lancet1962; 1: 45.

6. Palmer JR, Rosenberg L, Kaufman DW, et al. Oral contracep-tive use and liver cancer. Am J Epidemiol 1989; 130: 878–882.

7. Kaufman DW, Kelly JP, Anderson T, Harmon DC, Shapiro S.Evaluation of case reports of aplastic anemia among patientstreated with felbamate. Epilepsia 1997; 38: 1265–1269.

editorial 513

Copyright # 2004 John Wiley & Sons, Ltd. Pharmacoepidemiology and Drug Safety, 2004; 13: 511–513