Version: 27 May 2007

Whose Outcome Matters Most?View From the Reviewing Division

Russell Katz, M.D.Director

Division of Neurology ProductsCenter for Drug Evaluation and Research

2

Legal Requirements

As used in this subsection and subsection (E), the term “substantial evidence” means evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of the drug involved, on the basis of which it could fairly and responsibly be concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling or proposed labeling thereof.

3

Statutory/Legal Requirements

• The definition raises several questions– 1) Who are these experts?– 2) Is any claim that can be described in

labeling acceptable?

4

Warner-Lambert v Heckler 1986U.S. Court of Appeals, Third Circuit

A case about FDA’s withdrawal of oral proteolytic enzymes once used to treat a variety of tissue injuries, episiotomy scars, etc.

5

Warner-Lambert V Heckler 1986 (Cont’d)

The fact that the drug, not chance, can be assumed to have contributed to the factor measured does not necessarily establish that patients will receive a benefit from the drug. The Commissioner has consistently required a showing of some benefit as an element of the statutory requirement of effectiveness. See Benylin; Final Order, 44 Fed. Reg. 51512, 51521 (1979); Luxtrexin; Withdrawal of Approval of New Drug Application, 41 Fed. Reg. 14406, 14419 (1976).

6

Warner-Lambert V Heckler 1986 (Cont’d)

Given the strength of the congressional concern with the protection to the public underlying the Drug Amendments of 1962, it would be anomalous to hold that drug manufacturers may demonstrate effectiveness merely by showing statistical significance. Therefore, we hold that the Commissioner did not err in requiring petitioners to show that OPEs are therapeutically effective. This interpretation accords with long-standing FDA policy and with the policies embodied in the Food, Drug and Cosmetic Act.

7

Outcome measures

• Primary consideration is that the outcome must measure something that:– 1) matters clinically– 2) to the patient (some outcome measures

that have been proposed seemed to assess caregivers’ problems)

8

Outcome measures

• What about the size of the treatment effect ? (That’s what the court addressed) Do we worry about that?

• We do care about this, but, in general, we don’t know how to decide (numerically) what is an important effect

9

Outcome measures

• Because we don’t know (in any given case) what an important effect size is, we “require” that the outcome measures themselves “ensure” that the effect is clinically meaningful

• This results in many types of acceptable/required outcome measures, depending upon the clinical setting

10

Outcome measures

• Acceptable types of outcome measures that are considered a priori clinically meaningful

• Counts of events– Epilepsy-seizures– Migraine-proportion of patients without a

given symptom (responder rates)– MS-relapses– Many others

11

Outcome measures

• Acceptable types of outcome measures that are considered a priori clinically meaningful

• Mean changes in symptom scales– Parkinson’s Disease-UPDRS sub-scales– ALS-ALSFRS

12

Outcome measures

• Acceptable types of outcome measures that are considered a priori clinically meaningful

• Time-related outcomes– Parkinson’s Disease-time spent “off”, “on”– Insomnia-time to fall asleep, time staying

asleep (but need objective and subjective confirmation)

– Time to diagnosis of Alzheimer’s Disease

13

Outcome measures

• Types of outcome measures that are not considered a priori clinically meaningful– Mean changes in scales in which a very small

change is not obviously clinically meaningful.– In these cases (increasing number of these

due to new indications), we require a second, co-primary outcome

14

Outcome measures

• Types of outcome measures that are not considered a priori clinically meaningful– Alzheimer’s Disease-cognitive measure AND

a global/functional measure– RLS-symptom scale AND a global/functional

measure– Spasticity-Ashworth AND a global/functional

measure

• The global measure can be general or specific (e.g., ADL-preferred over QoL)

15

Outcome measures

• Types of outcome measures that are not considered a priori clinically meaningful– In some of these cases, if a sponsor can

justify the clinical meaningfulness of a particular change on a scale, a Responder Rate based on this change can be used

– The treatment effect, though, is still the difference between drug and control!

16

Outcome measures

• Composite measures– Mathematical construct combining several

different scales/outcomes– Major concern: No absolute assurance that all

scales will move in the same direction (a requirement that this be so can be built into the analysis)

• But there is another objection…

17

Outcome measures

• Composite measures– The measure is not “clinically interpretable”

• There is a difference between “clinically interpretable” and “clinically meaningful”

• If we are convinced that what the composite is measuring (and we are convinced that the effect) is clinically meaningful, the clinical “opacity” of the actual measure is not really a problem

18

Outcome measures

• Composite measures

• The measure is not “clinically interpretable”– I don’t think that most (any?) of the results we

present in labeling are really what people think they are (statistics!)

– The results (treatment effects) in any given trial probably do not reflect anything that will occur in any given patient in practice

19

Outcome measures

• Composite measures

• The measure is not “clinically interpretable”– In any event, if the results on the primary

outcome are truly opaque, we can (and often do) present, in labeling, the results of analyses that may be more “clinically interpretable”

20

Outcome measures

• An outcome measure does not have to measure all the symptoms/signs of a condition

• In fact, none of the outcomes we routinely employ do this

• Is this more important for a disease modifying claim?

21

Outcome measures• Although we say that the interpretation of

a study rests on the results of the primary outcome, this is not really (ever) true

• Examples abound– PD-”off”, but we look at “on with troublesome

dyskinesias”– Tx of dyskinesias-but we look at motor control– Recent treatment “won” on primary outcome,

but “lost” on standard secondary outcomes

22

Outcome measures• Although we believe in prospectively

designated outcomes, even this is not etched in stone

• Sabril-infantile spasms– Primary outcome involved 2 hour EEG– This was misleading, and 24 hour EEG was

used instead (chosen after the study was over!)

– Sponsor may have chosen “wrong” primary outcome; we can change this

23

Outcome measures

• New/novel outcomes– We like these, though not so common– There are many aspects of many conditions

that we routinely ignore as outcomes, but we shouldn’t

– These would not necessarily need to be “validated” fully, or against the “standard”

24

Outcome measures

• New/novel outcomes

• Ampyra (dalfampridine), to increase walking speed in patients with MS (novel claim)– Primary outcome-the proportion of

Responders, defined as a patient with a faster walking speed on at least 3 of 4 tests compared to any off-tx value (pre-or post)

25

Outcome measures

• New/novel outcomes

• Ampyra– To assure clinical meaningfulness of this

Responder definition, sponsor had to show that Responders do better than non-Responders on the MS Walking Scale (measures disability)

26

Outcome measures

• Outcomes for new indications– Sometimes, an appropriate scale does not

exist for a new indication– If the domain to be assessed is considered

important, we’ll want a measure for it (it’s not adequate to say the domain shouldn’t be assessed because there is [currently] no way to assess it)

27

Outcome Measures

• Depending upon the setting, the outcome may be physician-rated (e.g., MS), patient and/or caregiver rated (e.g., epilepsy, migraine), or both (e.g., hypnotics).

• Physician-rated is not the same thing as “objective”, and patient-rated is not the same thing as “subjective”, though…

• Patient ratings can be problematic if CNS active (hypnotics)

28

Outcome Measures-Summary

• There is one fundamental rule: outcome measures must (accurately) reflect an effect of the treatment that is considered to have clinical meaning to the patient

• This can be, and is, achieved in many different ways

• There are not any other a priori rules