Acta Med Scand, Suppl. 683: 23-27

The Epidemiological Evaluation of Drugs

Samuel Shapiro, M.B., F.R.C.P.(E)

From the Drug Epidemiology Unit, School of Public Health, Boston University School of Medicine, Boston, Mass., USA

ABSTRACT. Depending on the research problem, the epidemiological evaluation of drugs must be based on a combination of research strategies that include experimental methods (controlled trials), and non-experimental methods. Among the latter are correlational studies; spontaneous reporting systems and published case reports; cohort studies; and case- control studies. The spread of computer technology may, by record linkage, facilitate cohort or case-con- trol studies. Comprehensive drug surveillance must be based on the combined use of each research strategy, as appropriate.

This paper is my first presentation since the death of my colleague, Dennis Slone, on May 10, 1982. Over the years we had considered the issues discus- sed here so many times that I can no longer tell which ideas were his, and which mine-and I do not care to make the distinction.

When I was invited to participate in this meeting, I was informed that one purpose was “to stimulate colleagues in epidemiology to link drug utilization data to statistics on morbidity, mortality, and ad- verse effects”. Another aim was “to assess whether drug utilization is rational or not, a ques- tion which cannot be answered easily from conven- tional, controlled clinical trials”, since these are done in “rather selected samples of the population at large that might not be representative” of those ultimately to be treated.

With these aims in mind I will attempt sketch a general picture of the epidemiological approach to the evaluation of drugs, giving brief consideration to each of the main methods. I will not consider basic epidemiological principles, but concentrate on the application of epidemiological strategies to drug evaluation.

Epidemiological methods can broadly be divided into experimental and non-experimental ap- proaches.

EXPERIMENTAL METHODS (CONTROLLED TRIALS)

Experimental studies designed to determine short- term drug efficacy (e.g., hypnotics) are usually

small-scale, and considered to fall within the do- main of clinical pharmacology. Although the dis- tinction is blurred, experimental studies in the epidemiological domain are usually large-scale studies concerned with long-term effects. Examples are the University Group Diabetes Program (UGDP) in the United States (1) and the Clofibrate Study (2) in Europe. One feature common to both studies should be noted. The UGDP was designed to determine whether anti-diabetic drugs reduce the morbidity or mortality due to diabetes. Randomiza- tion was essential because diabetes is itself a risk factor for the principal outcome of interest, is- chemic heart disease. Randomization was also es- sential in the Clofibrate study, once again because the indication, a high cholesterol level, is a risk fac- tor for ischemic heart disease.

Thus, in each study the indication for drug use was confounded with the outcome. In the absence of randomization, the validity of the studies would, have been in question regardless of whether any treatment group fared better, worse, or no different- ly than the reference group.

If we turn to another example, the polio vacci- nation trials (3) carried out in the United States in the 1950s, a valid determination of vaccine efficacy could have been made without random- ization. In this example, there were no known strong risk indicators (apart, perhaps, from socio- economic status) that would have confounded the evaluation. In fact, both experimental and non-experimental studies were carried out, but a non-experimental study would have been sufficient to show that the vaccine worked.

One problem with experimental drug studies is that while they are designed to be large enough to evaluate efficacy, they are rarely large enough to detect adverse effects, although on occasion they have done so (e.g., the association between clofibrate and cholecystitis) (4).

Another problem with experimental drug studies, already referred to, is that the study populations are often restricted by strict selection criteria and treatment regimens, whereas drugs subsequently are used in different populations, and in different

Acta Med Scand, Suppl. 683

24 S. Shapiro

ways. In this situation, randomization remains es- sential if the indication for drug use (e.g., hyperten- sion) is confounded with the outcome (e.g., stroke). We have suggested ( 5 ) that randomization of “usual patterns of use” may yield useful information with respect to the effectiveness of drugs in the com- munity at large.

Finally, I should mention that while experimental studies are generally designed to evaluate efficacy, careful analysis of the data can sometimes yield early clues as to morbidity; such clues may, in principle, alert epidemiologists to the need for sur- veillance. In the case of practolol(6), after the drug’s effects on the eye and the peritoneum were already known, it was found that there was an ex- cess of minor ocular symptoms in exposed subjects given the drug in premarketing studies. Today most of us would suggest that such a finding should alert us to the need for surveillance. Another drug, triazure (7), was found effective for the treatment of psoriasis in controlled trials, but there was an ex- cess of thrombotic complications in exposed sub- jects. With careful surveillance after marketing, this excess was again noted and the drug was with- drawn.

NON-EXPERIMENTAL METHODS Correlational studies Sometimes, if the prevalence of drug use is high, it is possible either by examining trends over time, or differences in patterns of drug use according to geographic area, to suspect that a drug may cause some effect. One recent example (8) is the demon- stration that the incidence of uterine cancer increased contemporaneously with an increase of the use of female hormone supplements for menopausal symptoms; the incidence declined as the hormone use declined.

Correlational studies, however, have serious limitations, particularly if the proportion of the population using the drug is low. In addition, since these studies are indirect, and since it is sometimes not known what trends, or differences, would be apparent under the null hypothesis, such studies are of extremely limited value. Generally they can only be used as ancilliary evidence for studies based upon more rigorous methods.

Spontaneous reports Since the early 1960s, many governments, and the World Health Organization, have organized spon- taneous reporting systems which register cases of suspected drug-induced morbidity. Medical jour- nals, of course, also provide forums for case re- ports. Spontaneous reports are useful if their limita-

Actu Med Scund, Suppl. 683

tions are clearly understood. What is well under- stood is that spontaneous reports cannot provide reliable estimates of incidence because reporting is incomplete. What is not generally appreciated, however, is that spontaneous reports have their greatest utility when the disease of interest is rare, and when the prevalence of drug use is low. In that circumstance, it is sometimes even possible almost solely on the basis of case reports to infer cause and effect. For example, a clearcut association has been demonstrated between the use of oxymetholone and liver tumors (9): the use of the drug is exceed- ingly uncommon, virtually the only indication being aplastic anemia and Fanconi’s anemia-and, of course, liver tumors are also exceedingly uncom- mon. The existence of even a few case reports would be highly suspicious; in fact, there have been hundreds. Similarly, the receipt of some 70 case reports of acute hepatic failure (lo), shortly after ticrynafen (a new uricosuric agent) was introduced to the market, resulted in the recall of the drug by the United States Food and Drug Administration.

Case reports can also serve, somewhat less effec- tively, to raise suspicions about relationships be- tween relatively rare diseases and common expo- sures. The suspicion that oral contraceptives cause liver tumors was first raised in this way (1 1) ; sub- sequently the association was confirmed in formal case-control studies (12) .

Spontaneous reports are least reliable, and most subject to substantial bias, in relating common exposures to common events. It has been in this domain that most false alarms have been sounded because, by chance, many associations can be observed. Quite often considerable publicity has been given to putative associations that have been rebutted subsequently in formal epidemiological studies (13).

If the drug use is uncommon and an event is exceedingly common, spontaneous reports are vir- tually useless.

For all spontaneous reports inferences of cause and effect can be made more readily if there is a rapid onset (e.g., deafness following intravenous ethacrynic acid or furosemide) (14) and if a compli- cation occurs on rechallenge (e.g., rash). Probably there would have been fewer children born with phocomelia if there had not been a six to nine month interval betwenn in utero exposure and observation of the birth defect.

Thus, to conclude, spontaneous reporting sys- tems are useful if used with circumspection and understanding, and if their limitations are clearly understood. Suspicions raised by such systems can be tested formally in more rigorous epidemiological studies.

The epidemiological evaluation of drugs 25

to a specific hypothesis is more efficient, more rapid, and usually less costly in obtaining the same information as would be yielded by an equivalent cohort study. The case-control method also has its drawbacks.

Since Professor Vessey will be discussing this method in detail I will only confine myself to a few remarks. The first is that it is generally claimed that case control studies are more subject to bias than cohort studies. A more accurate claim is that both methods are subject to various forms of bias. In fact, it is the very ease with which case control studies can be mounted (as compared with cohort studies) that entails the danger they may be poorly designed, and therefore biased. In both cohort studies and case-control studies, problems of bias in selecting subjects, and in obtaining information can be minimized in many (but not all) situations. When they cannot, the interpretation of validity, with either method, is a matter of judgement. Such judgement involves consideration of the nature of the bias and its possible impact on the results; the magnitude of the effect in terms of relative risk, as well as its stability (statistical significance); biologi- cal plausibility; consistency of the data; whether hypotheses under the main hypothesis can be con- firmed, and whether the findings are consistent with other studies.

Another observation is that case-control studies can more readily be used to examine drug expo- sures that took place in the distant past. If we con- sider the example of adenocarcinoma of the va- gina (19) in relation to in utero exposure to dieth- ylstilbestrol one or more decades earlier, the advan- tage of the method is obvious. On the other hand, if interviews are used to determine drug exposure, there is a substantial question of bias in the recall of drug use in the distant past. My own view is that if the interview is conducted with care, and if the independent variable is defined as relatively long- term drug use, such bias is not likely to be great. This proposition is a matter of judgement, but it is interesting to note that while our group had the prejudice that oral contraceptive use was likely, if anything, to increase the risk of cancer of the uterus and ovary we found, to our great surprise, that there appeared to be reduction in the risk that per- sisted for many years (20, 21).

Another observation is that although the case control approach cannot, in principle, be used to discover all unsuspected connections between a specific drug and an array of diseases, it can partial- ly serve that objective if several diseases, together with all antecedent drugs that were used, are studied at the same time. This approach, known as case-control surveillance (22), has identified previ-

Cohort studies Cohort studies, in which exposed and non-exposed patients are followed for events, have a major ad- vantage in that they are capable of identifying pre- viously unsuspected associations between a specific drug and many outcomes. One limitation is that the size of the study population is usually fixed and cannot be augmented easily. Unless cohort studies are very large, either in terms of numbers, or years of follow-up (or both), they possess only modest strength, over and above clinical trials, for detecting important but relatively uncommon events. Usually they are costly and must, therefore, be limited to certain very specific situations, such as the use of pharmacologically potent drugs by a large segment of a healthy population. Thus, oral contraceptives, and indeed contraceptive methods in general, are important public health concerns. Large cohort studies are needed, and they have indeed yielded important information (15, 16, 17). Without doubt, they will continue to do so, and if continued for long enough, they may be able to yield important information about safety, or other- wise, even with regard to effects that can occur after a latent interval (such as cancer) (17).

Another situation that may call for cohort studies arises when new drugs are released to the market, having been shown to be effective, but with ques- tions of safety still unanswered. As a matter of principle, this consideration applies to all drugs. However, it is impractical to conduct a cohort study of every newly released drug. Instead, we have suggested that if there are concerns based upon pharmacological considerations, or suspicions aroused during the premarketing studies, it may be advisable to assemble and follow a cohort of the earliest drug users immediately after release to the market. Exactly such a policy was followed with triazure (7). Similarly, a follow-up study of ci- metidine is now in progress in England (18); since this drug affects hydrogen ion transport throughout the body, a cohort study to ensure that serious side effects are not occurring at more than a modest frequency seems prudent.

Other situations arise which may require life-time follow-up. One such example is daughters exposed to high doses of diethylstilbestrol in pregnancy (19). From the information already available there are grounds for concern about several outcomes over the entire lifetimes of the exposed subjects.

Case-control studies In this approach cases with a specific disease are compared with controls without the disease, and the rates of drug use are determined in the two comparison groups. A case-control study directed

Actu Med Scund, Suppl. 683

26 S. Shapiro

ously unsuspected associations, such as the recent- ly reported association between alcohol consump- tion and risk of breast cancer (23).

My final observation is that certain diseases known to be induced by a variety of drugs (e.g., agranulocytosis; liver tumors) ought to be moni- tored continuously by case-control surveillance. Such monitoring would enable rapid detection of drugs that cause these rare but important illnesses.

Computer data systems

With the spread of computer technology prescrip- tions and diagnoses can, in theory, be linked and analyzed, either by the cohort or case-control ap- proach. Preliminary attempts have been made to do so (24). But there are some limitations, perhaps the most important of which is that computer data banks have been designed primarily for administra- tive purposes. They seldom contain information on confounding variables (other than age and sex). Another disadvantage is one of limited population size, so that the prevalence of users of some specific drug, or of cases with some specific dis- ease, may be too low to be useful. Obviously, how- ever, limitations of size will disappear if, as seems likely, computer systems come into more general use.

I suggest that computer systems are likely to have the greatest utility in validating information on drug use obtained by other methods, and in providing sources of cases and control, together with access to unbiased information on drug exposure. Ad hoc case control studies can then be conducted, as needed, to obtain additional information needed in order to control confounding. In addition, when the computer systems become large enough it should prove possible to evaluate hypotheses, and detect associations, between specific drugs and specific diseases in situations where confounding does not appear to be an important problem.

DRUG SURVEILLANCE The idea that comprehensive drug surveillance is needed to ensure reasonable drug safety became widely accepted in the 1960s. However, there was confusion about how to conduct surveillance. With subsequent experience a more clear picture is now emerging. First, controlled trials may sometimes be essential, even after a drug has been introduced to the market. Second, spontaneous reporting systems and published case reports have an important, if limited, role. Third, certain drugs are important enough to require cohort studies to detect adverse effects of moderate frequency. Fourth, the case- control surveillance of uncommon as well as rare

diseases can serve to monitor a large range of drugs. Fifth, ad hoc studies of specific hypotheses will always be a necessary component of compre- hensive drug surveillance.

Most of these methods are being used with var- ying degrees of completeness. Perhaps the most important deficiency in present-day surveillance is the failure to monitor, systematically, rare diseases that are known to be induced by drugs.

REFERENCES 1 .

2.

3 .

4.

5 .

6.

7.

8.

9.

10. 1 1 .

12.

13.

14.

15.

16.

The University Group Diabetes Program. A study ofthe effects of hypoglycemic agents on vascular complica- tions in patients with adult-onset diabetes. 11. Mortali- ty results. Diabetes 1970 (Suppl2); 787-830. A cooperative trial in the primary prevention of is- chemic heart disease using clofibrate. Report from the committee of principal investigators. Br Heart J 1978; 40: 1069-1118. Sabin AB, Ramos-Alvarez M, Alvarez-Amezquita J, et al. Effects of rapid mass immunization of a popula- tion with live, oral poliovirus vaccines under condi- tions of massive enteric infection with other viruses. Second International Conference on Live Poliovirus Vaccines. Washington, D.C.: Pan American Sanitary Bureau, 1960: 377-385. The Coronary Drug Project Research Group. Clo- fibrate and niacin in coronary disease. JAMA, 1975; 23 1 : 360-365. Slone D, Shapiro S, Miettinen OS, et al. Drug evalua- tion after marketing. Ann Intern Med 1979; 90: 257- 261. Hazards of non-practolol beta-blockers. Editorial. Br Med J 1977; 1: 529-530. Azaribine (triazure) recalled. FDA Drug Bulletin 1976; 6: 34. Weiss NS, Szekely DR, Austin DF. Increasing inci- dence of endometrial cancer in the United States. N Engl J Med 1976; 294: 1259-1262. Sale GE, Lerner KG. Multiple tumors after androgen therapy. Arch Pathol Lab Med 1977; 101: 600-603. FDA drug bulletin. Ticrynafen recalled 1980; 10: 3-4. Baum JK, Holtz F, Bookstein JJ, Klein EW. Possible association between benign hepatomas and oral con- traceptives. Lancet 1973; 2: 926-929. Bournerooks J , Ory HW, Ishak KG, et al. The as- sociation between oral contraception and hepato-cel- lular adenoma--a preliminary report. Int J Gynecol Obstet 1977; 15: 143-149. Mitchell AA, Rosenberg L, Shapiro S, et al. Birth de- fects related to Bendectin use in pregnancy: I. Oral clefts and cardiac defects. JAMA 1981; 245: 2311- 2314. Cooperman LB, Rubin IL. Toxicity of ethacrynic acid and furosemide. Am Heart J 1973; 85: 831-834. Royal College of General Practitioners. Oral con- traceptives and health. London: Pitman Medical, 1974. Vessey M, Doll R, Pet0 R, et al. A long-term follow- up study of women using different methods of con-

Acta Med Scand, Suppl. 683

The epidemiological evaluation of drugs 27

traception-an interim report. J Biosoc Sci 1976; 8: 373427.

17. Rosenberg L, Hennekens CH, Rosner B, et al. Oral contraceptive use and non-fatal myocardial infarc- tion. Am J Epidemiol 1980; 1 1 1: 59-66.

18. Vessey M, Lawson D. A follow-up study of cimetidine. Presented at the August 1981 meeting of the International Epidemiological Association in Edinburgh, Scotland.

19. Herbst AL, Ulfelder H, Poskanzer DC. Adenocar- cinoma of the vagina: Association of maternal stilbes- trol therapy with tumor appearance in young women. N Eng J Med 1971; 284: 878-881.

20. Kaufrnan DW, Shapiro S, Slone D, et al. Decreased risk endometrial cancer among oral contraceptive users. N Engl J Med 1980; 303: 18, 1045-1047.

21. Rosenberg L, Shapiro S, Slone D, et al. Epithelial ovarian cancer and the use of combination oral con- traceptives. JAMA. In press.

22. Slone D, Shapiro S, Miettinen 0s. Case-control sur- veillance of serious illnesses attributable to ambula- tory drug use. In: Colombo F, Shapiro S, Slone D, et al, eds. Epidemiologic evaluation of drugs. New York: ElsevierlNorth Holland Biomedical Press, 1977: 59-70.

23. Rosenberg L, Slone D, Shapiro S, et al. Breast cancer and alcoholic-beverage consumption. Lancet 1982; i: 267-270.

24. IMS America Ltd. Health Care Services Research Group. Final report-Task C. An experiment in early post-marketing surveillance of drugs. Contract 223- 78-3007. DHHS/PHS/FDA. Rockville, MD, 1978.

Acta Med Scand, Suppl. 683