ELSEVIER

Masking Drug Treatments in the Cardiac Arrhythmia Pilot Study (CAPS)

Clair Haakenson, MS, Toshio Akiyama, MD, Alfred Hallstrom, PhD, Mike R. Sather, MS, FASHP for the CAPS Investigators VA Cooperative Studies Program and University of New Mexico, 2100 Ridgecrest Drive SE 0514, Albuquerque, New Mexico 87208 (C.H., M.R.S.), University of Rochester; Rochester, New York (T.A.), and University of Washington, Seattle, Washington (A.H.)

ABSTRACT: The effectiveness of masking in the Cardiac Arrhythmia Suppression Study (CAPS) was assessed by surveying investigators and study coordinators. CAPS patients were assigned one of five treatments: encainide, flecainide, imipramine, moricizine, or placebo. Had all treatments appeared identical and equal numbers of patients been assigned to each, 20% of guesses of treatment assignment would be correct by chance alone. Since neither was possible in CAPS, higher rates of correct guessing were expected. Overall, respondents correctly identified treatment 39% of the time. Investigators identified the drug 30% of the time with rates of 20%, 24%, 37%, 20%, and 55% for the five treatment groups, respectively, whereas coordinators identified the treatments 47% of the time with scores of 42%, 45%, 50%, 40%, and 60%. Side effects and a suboptimal masking design detracted from masking; electrocardio- graphic changes did not. lmipramine, which caused characteristic side effects, was the most frequently identified active treatment. Scores were higher for investigators who had prior experience with the drugs, but scores did not improve over the course of the trial. Findings suggest that to improve masking all drugs should have been matched in appearance or persons evaluating treatments should not have been allowed to see the drugs. Controlled Clin Trials 1996;17:294-303

KEY WORDS: Clinical trials, double-blind, double-masked, placebo, trial design

INTRODUCTION

Masking of patients and investigators to the assigned treatment is essential in many clinical trials. Despite sound experimental methodology and appropriate statistical models, the results of trials may be questioned if the study fails to mask the participants. The objective of masking is to reduce bias and to counteract the placebo effect, but masking is not always feasible and seldom is perfect. When masking is attempted, however, effectiveness of the mask should be assessed.

Address reprint requests to: Clair Haakenson, MS, VA Cooperative Studies Program and University of New Mexico, 2100 Ridgecrest Drive SE (151-l), Albuquerque, New Mexico 87108.

Received August 18, 1995; revised October 30, 1995.

Controlled Clinical Trials 17:294-303 (1996) 0 ELsevier Science Inc. 1996 655 Avenue of the Americas, New York, NY 10010

0197-2456/96/$15.00 SSDI 0197-2456(95)00195-6

Masking Drug in CAPS

Table 1 Druns Used in the Cardiac Arrhvthmia Pilot Studv

295

Planned Enrollment (actual) Drug” Active Drug Matching Placebo

Encainide formulations 100 (99) 25 (28) Flecainide formulations 100 (103) 25 (24) Imipramine formulations 100 (102) 25 (23) Moricizine formulations 100 (98) 25 (25) TOTAL 400 (402) 100 (100)

“Drugs and corresponding placebo were donated as follows: encainide by Bristol-Myers Research Laboratories; flecainide by Riker Laboratories; imipramine by Ciba-Geigy; moricizine by DuPont Pharmaceuticals.

Several clinical trials have attempted such assessments by administering question- naires to investigators and patients at the conclusion of the trial [l-3]. Surveyors generally have found that participants can guess the correct treatment more often than can be attributed to chance. Ability to identify the treatment at the end of the trial does not guarantee that bias has occurred, just as inability to guess the treatment is not a certain indicator that bias has not occurred. However, such findings raise skepticism about the ability to mask any trial and have prompted the suggestion that effectiveness of masking should routinely be assessed in clinical trials [4].

The Planning Committee of the Cardiac Arrhythmia Pilot Study (CAPS) recog- nized at the outset that several complicating factors would make effective masking difficult to achieve. Four drugs, each with a different appearance, were to be tested. Characteristic side effects and pharmacologic actions of the study drugs also presented potential problems. For these reasons, considerable effort was de- voted both to reducing the influence of potentially unmasking factors and to moni- toring success in maintaining the mask. In this paper we evaluate the masking strategy of CAPS and identify important issues in masking clinical drug trials.

METHODS

CAPS was a double-masked, placebo-controlled, randomized trial conducted at 10 clinical centers as a pilot for a full-scale trial of drug treatment of postinfarction ventricular arrhythmia. The experimental design and methods of CAPS have been published elsewhere [5,6]. Briefly, patients entered a drug and dose selection phase on one of five equally and randomly assigned treatments: encainide, flecainide, imipramine, moricizine, or placebo (Table 1). If the first treatment assigned did not adequately suppress arrhythmia or if proarrhythmia or other intolerable reaction occurred, the patient was randomly assigned a second CAPS drug: to one of the other three active drugs if original assignment was active; to a second placebo if original assignment was placebo. If a tolerable and at least partially efficacious drug and dose was found for the patient, treatment was continued for 1 year. Patients were evaluated by physical examination and electrocardiogram (ECG) at 3,6,9, and 12 months. Patients reported symptoms and adverse effects to the study coordinators at the clinic visits with physician evaluation generally limited to more significant events. Study coordinators performed pill counts. Unmasking of the drug treatment was permitted during the trial for medical emergencies or to guide

296 C. Haakenson et al

management of patients completing the trial. In 14 instances CAPS personnel were unmasked, but questionnaires concerning the involved patients were either not completed or were completed prior to unmasking.

Study Drug Formulations and Masking Strategy

The manufacturers supplied antiarrhythmic agents and matching placebo for each. Thus appearance was matched within but not across pairs. Each patient received only one drug (in either the active or placebo form). Under this plan, 80% of patients receiving a drug of a given appearance received the active form, whereas 20% received placebo (Table 1).

Formulations of encainide, flecainide, and moricizine prepared for CAPS were not commercially available. Only imipramine resembled a currently marketed for- mulation. Descriptions of the drugs and masking strategy were not available to the investigators. Drugs were packaged into opaque plastic dispensing bottles at the centralized pharmacy coordinating center and were distributed to centers la- beled with the common name, “CAPS DRUG”. All drugs were prescribed three times daily.

Survey on Masking

At the end of 1 year follow-up, the investigators and coordinators who had cared for a patient were asked either to guess the patient’s treatment assignment or to indicate “no idea.” The question was posed for each patient assigned a CAPS study drug at completion of dosing. Investigators and coordinators were also surveyed about prior experience with the drugs used in CAPS either in clinical practice or in other clinical trials. Patients were not surveyed.

Statistical Analysis

Results were tabulated separately for investigators and coordinators for each of the five treatment groups. The expected rate of correct guessing by CAPS investiga- tors and coordinators depended not only on how well drug treatments were matched in appearance but on opportunity to observe the drugs, awareness of the specific appearance of each of the four drugs, and prior knowledge of the masking strategy and treatment allocation scheme. If CAPS had used identically appearing treatments with equal allocation of patients to each, 20% of guesses could be expected correct by chance alone. Because these conditions did not exist in CAPS, higher guessing scores were expected for personnel who saw the drug products.

Despite the use of generic labeling and a common code name for all CAPS drugs, it was presumed that coordinators generally would know which agent appearance (encainide, flecainide, imipramine, or moricizine) the patient was re- ceiving, but not whether the agent was active or placebo. On the other hand, investigators rarely would see the pills. The expected guess rate depends on whether the physician or coordinator used prior knowledge of probabilities in guessing or ignored prior probabilities and guessed on available evidence. If coordinators used the probabilities suggested by the 4: 1 ratio of active to placebo and an awareness of drug appearance, they would always guess the active form of the apparent drug type and thus would be right 80% of the time. The breakdown of correct guessing

Masking Drug in CAPS 297

scores by treatment group would be lOO%, lOO%, 1000/o, lOO%, and 0% for encainide, flecainide, imipramine, moricizine, and placebo, respectively. If they ignored prior probabilities and chose equally between the active and placebo forms of the appar- ent drug type, 50% of the guesses would be correct for each active drug and for placebo.

If investigators used probabilities based on prior knowledge of the allocation ratio, they would also always guess active but would not know which active because, unlike coordinators, they did not see the drugs. They would be right 20% of the time overall with a breakdown of guessing scores by treatment of 25%, 25%, 25%, 25%, and 0%, respectively. If they did not use probabilities based on prior knowledge, they would again be right 20% of the time, but the breakdown would be 20%, 20%, 20%, 20%, and 20%, respectively-scores arising purely from chance.

Although physicians are likely to understand the implications of prior probabili- ties, we believe that they are motivated to use empirical evidence to guess the therapy. Likewise, we believe that coordinators also relied on empirical evidence but, notably, evidence for coordinators also included awareness of the drug type given to the patient. Therefore, for the purpose of identifying unmasking influences, we have assumed that the scores expected to be correct by chance are 50% for coordinators and 20% for investigators. Higher scores than these can be attributed to unmasking. For testing purposes we assumed that “no-idea“ responses would have been uniformly distributed among the other possible responses. Chi-square tests were employed to assess statistical significance.

RESULTS

A total of 940 guesses were registered, 430 by 36 investigators and 510 by 27 coordinators. Responses were received for 396 of the 448 patients assigned long- term CAPS study drug (88%). Ideally, two (and sometimes more) responses would be registered for each patient. Actual response rate was reduced because of staff turnover. Of the 940 guesses registered, 14% involved an assigned treatment with which the respondent had prior experience.

Table 2 presents the results of the masking survey for study coordinators and Table 3 for investigators. The top of the column shows the actual treatment assigned whereas the figures in the column represent the percentage of responses registered for each of the six possible choices about the patient’s treatment. Overall, coordina- tors correctly guessed the treatment 47% of the time (53% corrected for “no idea”) compared to 30% for investigators (38% corrected for “no idea”). Combining coordi- nator and investigator responses reported in Tables 2 and 3, the overall rate of correct guessing is 39% (46% corrected for “no idea”), which is significantly different (~7 < 0.0001) from the expected combined rate of 36%. Tested separately, the propor- tion of correct guesses was significantly different from that expected for investiga- tors (I? < O.OOOl), but not for coordinators (~7 = 0.2); however, homogeneity of guessing across treatment groups does not hold for coordinators (x2 with 5 df = 19.3, p = 0.007) or for investigators (x2 with 5 df = 10.2, p = 0.036). Rates of correct guessing were significantly higher for the placebo and imipramine groups for both coordinators and investigators.

Adverse effects and drug discontinuations occurred in all treatment groups including placebo (Table 4). Imipramine, however, produced more gastrointestinal

298 C. Haakenson et al

Table 2 Guessing of Treatment Assignment by CAPS Coordinators (o/o) Treatment Assigned

Encainide Flecainide Imipramine Moricizine Placebo

Treatment guessed Encainide 42” 5 3 6 2 Flecainide 6 45 2 3 6 Imipramine 3 2 50 1 5 Moricizine 0 2 3 4ob 3 Placebo 16 16 11 22 6ob No idea 33 30 31 28 24

Number of guesses n = 128 n = 127 n = 67 n = 87 n = 101 Coordinator guesses correct-all treatment groups combined 47% (241/510) Coordinators with no idea-all treatment groups combined 29% (149/510)

“Correct guesses are highlighted. Column figures are percent of correct guesses (six choices) about this treatment.

b p < 0.05.

problems and a much higher incidence of reactions requiring discontinuation of therapy. As shown in Table 5, guessing scores were generally higher among study personnel who had prior experience with CAPS drugs. Scores of investigators and coordinators were not analyzed separately with respect to prior experience because subdividing these data would have yielded extremely small subgroups. Improve- ment in scores associated with prior experience was least pronounced for imipra- mine. Encainide does not appear in Table 5 because prior experience with this drug was so limited that little can be said about an effect of such experience. In contrast to experience with the drugs gained outside of CAPS, experience gained during the trial did not improve guessing as indicated by the similar scores for the first 100 patients and last 100 patients enrolled in CAPS (Table 6).

Table 3 Guessing of Treatment Assignment by CAPS Investigators (%) Treatment Assigned

Encainide Flecainide

Treatment guessed Encainide Flecainide Imipramine Moricizine Placebo No idea

Number of guesses

20” 15 16 24

7 8 8 6

10 13 39 34

n = 95 n = 123

Imipramine

4 9 0 11 10 5 37b 11 1 10 20 4 9 16 55b

29 34 35 n = 52 n = 80 n = 80

Moricizine Placebo

Investigator guesses correct-all treatment groups combined Investigators with no idea-all treatment groups combined

30% (128/430) 35% (149/430)

“Correct guesses are highlighted. Column figures are percent of correct guesses (six choices) about this treatment.

* p < 0.001.

Masking Drug in CAPS

Table 4 Drun Effects Observed in CAPS

299

Drug

Encainide Flecainide Imipramine Moricizine Placebo

Efficacy of first drug tried: (n = 99) (n = 103) (n = 102) (n = 98) (?I == 100) Effective by ECG (%) 79 83 52 66 37

Adverse effects of drug assigned long term: (n = 108) (n = 119) (n = 57) (n = 77) (n = 87)

Gastrointestinal (%) 13 19 44 26 21 Dizziness on standing (%) 5 5 9 10 0 Any symptom (%) 49 55 67 64 60 Stopped drug because

of symptom (%) 2 3 18 6 7

DISCUSSION

The objective of CAPS was to determine if any available antiarrhythmic agent could adequately suppress postinfarction arrhythmias in the setting of a multicenter clinical trial. A secondary goal of CAPS was to identify problems in keeping physicians and study coordinators masked as they titrated doses and monitored patients for therapeutic and adverse effects.

Notable findings in CAPS concerning masking were as follows: (1) Overall, identification of the therapy was greater than might be expected by chance given assumed rates of chance guessing of 50% for coordinators and 20% for investigators. (2) Imipramine was more frequently identified than other active drugs by both investigators and coordinators. (3) Placebo was identified more often than any of the active drugs. In evaluating these findings, potential sources of this unmasking were identified and broadly categorized under the headings of inadequately matched drug formulations and drug-induced effects.

Matching of Drug Formulations

Unlike drug-induced effects, which held implications for both CAPS and the trial to follow, concern about appearance of the drugs was largely limited to the current trial. It was assumed that for the follow-up trial all drugs could be obtained in identically matching forms, if necessary. The decision in CAPS to use a separate

Table 5 Prior Experience with CAPS Drugs and Effect on Masking Percent Correct Guesses

By Personnel with No Prior Experience

Treatment Assigned” Using This Drug Flecainide 31% (n = 61)” Moricizine 28% (n = 41) Imipramine 42% (n = 36) ” Prior experience with encainide was extremely limited.

‘ Number of correct guesses about patients receiving this treatment.

By Personnel with Prior Experience Using This Drug

48% (n = 26) 56% (n = 10) 50% (n = 17)

300 C. Haakenson et al

Table 6 Experience Gained During CAPS and Effect on Masking Percent Correct Guesses Percent Correct Guesses about First 100 Patients about Last 100 Patients

Investigators Coordinators

33% (n = 40)” 30% (n = 26) 51% (n = 52) 54% (n = 53)

u Number of correct guesses about these 100 patients.

placebo for each drug rather than having all drugs reformulated to a common appearance was largely due to time constraints. Several months would have been required to develop, test, and manufacture the necessary formulations. While not conducive to masking, the approach chosen held the advantage of eliminating questions of bioequivalence between formulations used in CAPS and those used in other trials or clinical practice. Compounding the potential problem of using treatments of unmatched appearance was the use of an imipramine formulation resembling a commercial product pictured in the Physician’s Desk Reference thus rendering it readily identifiable as imipramine or its placebo. Fortunately, the other three drugs used in CAPS either were not commercially available during CAPS or were obtained in formulations unfamiliar to investigators.

The remedy typically employed in trials saddled with dissimilar appearances among drugs is to give the patient the assigned drug plus the placebo forms of the other drugs. Using this multiple placebo technique in CAPS, however, would have required the patient to take four drugs (one active form plus three placebo forms). Such a plan would likely deter patient adherence. The Planning Committee decided, therefore, that each patient would receive a single drug formulation masked only as to active or placebo within that drug appearance.

Did this less-than-perfect match of drugs affect masking as we had assumed it would? Under the assumption that coordinators saw the pills whereas physicians did not, we expected chance scores of 50% for coordinators (a choice between active and placebo) and 20% for investigators (a choice among five treatments). Actual scores for coordinators and investigators, respectively, were 47% (53% if corrected for “no idea”) and 30% (38% if corrected). These findings support our assumption that coordinators (1) saw the pills, (2) were aware of the specific appearance of each of the four drugs, and (3) used this knowledge in guessing the treatments; and that investigators either did not see the pills or were unable to associate appearance with specific drug type. The data indicate that coordinators and investi- gators ignored the prior probabilities inherent in the study design in making their judgments. The findings also suggest that it is easier to identify placebo than to distinguish among active therapies, so that there would appear to be little to gain in the follow-up trial to have all treatments identically matched, particularly if imipramine were excluded (as it was).

Drug-Induced Effects

The second major category of potential unmasking influences is drug-induced effects. For a drug effect to jeopardize the masking of a trial, several conditions must exist. The effect must be specific to the drug, consistently produced, and large enough to be perceived by the observer. Furthermore, the observer must have

Masking Drug in CAPS 301

opportunity to see the effect and must then be aware of its significance. The planners of CAPS sought to anticipate such effects, both adverse and therapeutic, and elimi- nate or reduce their unmasking potential.

ECG changes were of a major concern in CAPS because ECG tracings were obtained at each visit to monitor for therapeutic and adverse (proarrhythmic) effects. In theory, ECG changes induced by the various agents can differ qualitatively and quantitatively. In CAPS, however, qualitative differences proved to be small, somewhat variable, and not useful to differentiate between drugs without the aid of sophisticated computer analyses and access to baseline tracings. Neither was routinely available to investigators and coordinators. Moreover, the apparent effi- cacy of placebo in 37% of patients receiving this treatment plus lack of efficacy among active treatments (approximately 25% across active drugs) illustrates the high degree of variability inherent in ventricular arrhythmias. These findings and lack of access to baseline information argue that identification of placebo probably did not occur through the ECG.

Side effects, on the other hand, may have had a profound influence on guessing. While all treatments including placebo produced a high incidence of adverse effects, the nature and severity of reactions varied among the treatments. One of the more troublesome adverse effects, dizziness upon standing, was reported only among the active treatments (Table 4). This highly visible effect may have been viewed as an indication of active drug assignment. Furthermore, whereas all active drugs caused gastrointestinal symptoms and dizziness, those caused by imipramine were apparently more severe and resulted in significantly more drug discontinuations 161. These two reactions have long been associated with imipramine whereas ad- verse effects of the other drugs were at the time of the trial less well known. Thus both the nature and severity of observed reactions may have produced the higher guessing scores seen in the imipramine group whereas, conversely, the absence of notable effects may have lead to the higher scores for placebo (Tables 2 and 3).

Effect of Prior Experience with CAPS Drugs If effects produced by a drug are capable of unmasking the observer, then

experience in using the drug might be expected to amplify this unmasking effect. About 14% of guesses in CAPS were made by investigators or coordinators claiming previous experience with the assigned CAPS drug. Experience, however, was largely limited to flecainide and imipramine with virtually none for encainide. Experienced personnel in general registered higher scores than those without prior experience (Table 5). Surprisingly, the difference was least pronounced for imipra- mine, the drug producing the most troublesome side effects. The difference in guessing scores between respondents with imipramine experience and those with- out such experience was only 8%, whereas differences associated with prior experi- ence were 17% and 28%, respectively, for flecainide and moricizine. A possible explanation for this apparent discrepancy is that the effects of imipramine may have been known to both experienced and nonexperienced investigators because they are well documented in the literature. Use of an imipramine formulation readily identifiable in the Physician’s Desk Reference may also have been a factor. A notable effect of prior experience with the assigned drug was to reduce the number of investigators indicating that they had no idea about the drug given to the patient.

302 C. Haakenson et al

In contrast to prior experience with these drugs, experience gained by study personnel as the CAPS trial progressed apparently did not help discriminate among the treatments. Nearly identical guessing scores were registered by respondents for patients enrolled early in CAPS as compared to those enrolled near the end of the trial. This lack of improvement in guessing suggests that study procedures were adequate to prevent a learning effect. A learning effect would probably have occurred had the treatment code been routinely broken as individual patients completed follow-up or as they were taken off drug for intolerable side effects.

CONCLUSION

The difficulties encountered in masking CAPS were significant but in no way unique to this trial. Treatment-specific side effects or other obvious pharmacologic effects are present to some extent in most clinical drug trials. Use of multiple drug appearances is often unavoidable in trials featuring multiple treatments. Imperfect match of drug and placebo is a common problem in large and small trials alike. As anticipated prior to starting the trial, masking in CAPS was not perfect; however, sufficient uncertainty existed about the identify of the treatments to make masking satisfactory for the purposes of this trial. The survey clearly suggests that masking was weakened by imperfectly matched treatments and the distinct side effect profile of imipramine. Also apparent, and perhaps more disturbing, was the difficulty in masking placebo.

Implications of CAPS for the masking of clinical drug trials are as follows: (1) Agents with prominent side effect profiles such as imipramine will be difficult to mask. (2) Study drug formulations should not resemble commercially available products unless the masking strategy is sufficient to make such clues irrelevant. (3) Ideally, all treatments should be perfectly matched in appearance or persons evaluating treatments should not be allowed to see the drugs; however, if the various therapies have similar effect profiles (as with encainide, flecainide, and moricizine but not imipramine), identical matching may not be necessary. Attention to these details will be necessary if a clinical drug trial is to be conducted in a double-masked manner.

CAPS was funded by the National Heart, Lung and Blood Institute and supported by the Coopera- tive Studies Program of the Medical Research Service, Department of Veterans Affairs through an interagency agreement. Theauthors also acknowledge Naricy L. Armijo for her helpful suggestions.

REFERENCES

1. Howard J, Whittemore AS, Hoover JJ, et al. How blind was the patient blind in AMIS? Clin Pharmacol Ther 1982;32:543-553.

2. Byington RI’, Curb JD, Mattson ME. Assessment of double-blindness at the conclusion of the B-blocker heart attack trial. ]AMA 1985;253:1733-1736.

3. Moscucci M, Byrne L, Weintraub M, et al. Masking, unmasking, and the placebo effect: an analysis of patients’ guesses of treatment assignment in a double-blind clinical trial. CIin Pharmacol They 1987;41:259-265.

4. Ney PG, Collins C, Spensor C. Double-blind: double talk or are there ways to do better research? Med Hypotheses 1986;21:119-126.

Masking Drug in CAPS 303

5. The CAPS Investigators. The cardiac arrhythmia pilot study. Am J Cardiol 1986;57:91-95.

6. The Cardiac Arrhythmia Pilot Study (CAPS) Investigators. Effects of encainide, flecai- nide, imipramine and moricizine on ventricular arrhythmias during the year after acute myocardial infarction: the CAPS. Am J Cardiol 1988;61:501-509.