1

Mohamed Alosh, Ph.D. Kathleen Fritsch, Ph.D.

Shiowjen Lee, Ph.D. DBIII, OB, CDER, FDA

Efficacy Evaluation in Acne Clinical Trials

2

An Outline

1. Re-visit choice of primary endpoints

(statistical viewpoint)

2. Statistical analysis methods and datatransformations

3. Repeated measurements vs. final assessment

4. Effect of baseline severity

5. Final Comments

3

1. Primary Endpoints in Acne Trials

(Statistical Viewpoint)

Inflammatory, non-inflammatory, and Total Lesion Counts can be analyzed as final lesion counts change from baseline percent change

Investigator Global Evaluation (IGE)

4

Analysis of Change Scores

Pros Easy to interpret &

analyze

Attempts to remove influence of baseline counts

Cons Baseline may still have

influence since change is negatively correlated with final counts

Change and percent change scores may have highly skewed distributions (violates parametric tests)

5

Figure 1. Mean Lesion Counts by Type over Time(Drug X, Study 1)

0

20

40

60

80

100

0 4 8 12Week

Mea

n l

esio

n c

ou

nt

Tot. (act)Tot. (veh.)N-Inf (act.)N-Inf. (veh.)Inf. (act.)Inf. (veh.)

6

Figure 2. Mean Lesion Counts by Type over Time(Drug Y, Study 2)

0

20

40

60

80

100

0 1 2 3 4 5 6Cycle

Mea

n le

sion

cou

nt

Tot. (act.)Tot. (pbo.)N-Inf. (act.)N-Inf. (pbo.)Inf. (act.)Inf. (pbo.)

7

Figure 3. Inflammatory Counts at Week 12 vs. Baseline Count (Drug X, Vehicle Arm)

Vehicle Arm - Drug X

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Infl. Count at Baseline

Infl

. C

ou

nt

at W

eek

12

0 - 100% Reduction

0 - 100% Increase

>100% Increase

8

Figure 4. Inflammatory Counts at Week 12 vs. Baseline Count (Drug X, Active Arm)

Active Arm - Drug X

0

20

40

60

80

100

120

0 20 40 60 80 100 120

Infl. Count at Baseline

Infl

. C

ou

nt

at W

eek

12

0 - 100% Reduction

0 - 100% Increase

>100% Increase

9

Figure 5. Mean % Change in Infl. Lesions over Time (Drug X, Study 1)

-60

-40

-20

0

20

40

60

80

100

0 4 8 12

Week

mea

n %

ch

ang

e

Act. (mean)Veh. (mean)

+/- sd (Act.)+/- sd (Veh.)

10

0

50

100

150

200

250

0 4 8 12Week

Mea

n le

sion

cou

nt

Subj.1 (veh.)Subj.2 (veh.)Subj.3 (veh.)Subj.4 (veh.)Subj.5 (act.)Subj.6 (act.)Subj.7 (act.)Subj.8 (act.)

Figure 6. Subject’s Total Lesion Count over Time -- Center A (Drug X, Study 1)

11

2. Statistical Analysis Issues(a) Analysis units

original transformed data (ranks, log, etc.)

(Pros & Cons)

(b) Analysis methods for final study endpoint Simple comparisons ANOVA (treatment, center, interaction) ANCOVA (include baseline count as covariate)

(Comparison of results presented in Tables 1a-c & Tables 2a-c)

12

3. Repeated Measures vs. Final Assessment

Repeated measures may increase power for detecting treatment effect

Must select number of timepoints to be included in repeated measurements model

Repeated Measures Models:

MANOVA, GLM, MIXED (Comparison of results presented in Tables 1a-c &

Tables 2a-c))

13

Table 1a: Treatment effect p-values for various statistical methods (Drug X, Study 1)

COUNTS Infl. Lesions Non-Inf. Les. Total Lesions

Data Ranks Data Ranks Data Ranks

Week 12 0.130 0.044 0.002 <0.001 0.002 <0.001

Week 8 0.308 0.222 0.003 0.002 0.005 <0.001

Week 4 0.834 0.917 0.270 0.418 0.325 0.293

ANCOVA 0.062 0.013 <0.001 <0.001 <0.001 <0.001

MANOVA 0.306 0.069 <0.001 <0.001 <0.001 <0.001

GLM (R) 0.316 0.219 0.010 0.006 0.013 0.003ANCOVA (R) 0.155 0.075 <0.001 <0.001 <0.001 <0.001

Mean Count (18.4, 20.6)* (37.4, 46.7)* (55.8, 67.2)*

*(active, vehicle) (R) = Repeated Measures

14

Table 1b: Treatment effect p-values for various statistical methods (Drug X, Study 1)

CHANGE Infl. Lesions Non-Inf. Les. Total Lesions

Data Ranks Data Ranks Data Ranks

Week 12 0.031 0.040 <0.001 <0.001 <0.001 <0.001

Week 8 0.116 0.132 <0.001 0.001 <0.001 <0.001

Week 4 0.429 0.789 0.020 0.021 0.027 0.030

ANCOVA 0.035 0.050 <0.001 <0.001 <0.001 <0.001

MANOVA 0.176 0.158 <0.001 <0.001 <0.001 <0.001

GLM (R) 0.081 0.134 <0.001 <0.001 <0.001 <0.001ANCOVA (R) 0.091 0.143 <0.001 <0.001 <0.001 <0.001

Mean Chnge (9.2, 6.4)* (26.0, 15.5)* (35.1, 21.9)*

*(active, vehicle) (R) = Repeated Measures

15

Table 1c: Treatment effect p-values for various statistical methods (Drug X, Study 1)

% CHANGE Infl. Lesions Non-Inf. Les. Total Lesions

Data Ranks Data Ranks Data Ranks

Week 12 0.012 0.008 <0.001 <0.001 <0.001 <0.001

Week 8 0.201 0.108 <0.001 0.001 <0.001 <0.001

Week 4 0.498 0.712 0.050 0.037 0.023 0.034

ANCOVA 0.015 0.009 <0.001 <0.001 <0.001 <0.001

MANOVA 0.067 0.046 <0.001 <0.001 <0.001 <0.001

GLM (R) 0.079 0.061 <0.001 <0.001 <0.001 <0.001ANCOVA (R) 0.061 0.053 <0.001 <0.001 <0.001 <0.001

Mean % Ch. (32.1, 21.4)* (41.1, 24.8)* (38.8, 24.8)*

*(active, vehicle) (R) = Repeated Measures

16

Comments on Results for Drug Xresults for total are similar to non-infl.

(strong corr., highly signif.)no general pattern for ranks vs. original datafor infl. lesions, % change has smaller

p-values than counts or change for change and % change, ANCOVA has

similar results to Week 12 analysisp-values for rep. meas. in general are larger

than final study endpoint analysis (prev. weeks not signif.)

17

Table 2a: Treatment effect p-values for various statistical methods (Drug Y, Study 2)

COUNTS Infl. Lesions Non-Inf. Les. Total Lesions

Data Ranks Data Ranks Data Ranks

Cycle 6 0.020 0.018 0.294 0.071 0.161 0.022

Cycle 5 0.031 0.024 0.636 0.231 0.327 0.072

Cycle 4 0.160 0.375 0.803 0.099 0.561 0.098

ANCOVA 0.014 0.032 0.044 0.017 0.015 <0.001

MANOVA 0.065 0.058 0.479 0.154 0.354 0.021

GLM (R) 0.020 0.031 0.415 0.120 0.211 0.023ANCOVA (R) 0.015 0.016 0.072 0.030 0.021 0.001

Mean Count (13.7, 16.2)* (43.6, 49.2)* (57.3, 65.3)*

*(active, vehicle) (R) = Repeated Measures

18

Table 2b: Treatment effect p-values for various statistical methods (Drug Y, Study 2)

CHANGE Infl. Lesions Non-Inf. Les. Total Lesions

Data Ranks Data Ranks Data Ranks

Cycle 6 0.060 0.036 0.027 0.001 0.010 <0.001

Cycle 5 0.016 0.007 0.204 0.024 0.063 0.002

Cycle 4 0.117 0.124 0.240 0.014 0.118 0.001

ANCOVA 0.014 0.023 0.044 <0.001 0.015 <0.001

MANOVA 0.097 0.046 0.087 0.018 0.035 0.003

GLM (R) 0.037 0.017 0.043 0.008 0.013 0.001ANCOVA (R) 0.015 0.009 0.072 0.003 0.021 <0.001

Mean Chnge (8.1, 3.7)* (6.6, -2.4)* (14.7, 3.3)*

*(active, vehicle) (R) = Repeated Measures

19

Table 2c: Treatment effect p-values for various statistical methods (Drug Y, Study 2)

% CHANGE Infl. Lesions Non-Inf. Les. Total Lesions

Data Ranks Data Ranks Data Ranks

Cycle 6 0.097 0.017 0.021 0.005 0.002 <0.001

Cycle 5 0.078 0.010 0.078 0.020 0.010 0.004

Cycle 4 0.249 0.188 0.039 0.012 0.021 0.004

ANCOVA 0.096 0.016 0.037 0.004 0.004 <0.001

MANOVA 0.203 0.030 0.071 0.016 0.010 <0.001

GLM (R) 0.076 0.009 0.035 0.005 0.003 <0.001ANCOVA (R) 0.074 0.008 0.061 0.005 0.005 <0.001

Mean % Ch. (31.0, 22.0)* (13.2, -5.2)* (22.6, 8.5)**(active, vehicle) (R) = Repeated Measures

20

Comments on Results for Drug Y

results for total & non-infl. are similar, as for X (but here less signif.)

no general pattern for ranks vs. original datafor infl. lesions, % change has larger

p-values than counts or change for change and % change, ANCOVA has

similar results to Cycle 6 analysisp-values for GLM(R)/ANCOVA(R) in

general are smaller than final study endpoint

21

4. Effect of Baseline Severity

Divide subjects into equal sized groups (e.g., quartiles) based on baseline lesion countPlots of lesion counts by baseline category

Compare efficacy results by baseline category Tables 3a-b present results for lesion counts Tables 4a-b present results for IGE

22

Figure 7. Mean Week 12 Lesion Counts by Type over Baseline Category (Drug X, Study 1)

0

10

20

30

40

50

60

70

80

90

1 2 3 4

Baseline Category

Me

an

Le

sio

n C

ou

nt

Infl - Act

Infl - Veh

Non-Inf - Act

Non-Inf - Veh

23

Figure 8. Mean Cycle 6 Lesion Counts by Type over Baseline Category (Drug Y, Study 2)

0

20

40

60

80

100

120

1 2 3 4

Baseline Category

Me

an

Le

sio

n C

ou

nt

Infl - Act

Infl - Pbo

Non-Inf - Act

Non-Inf - Pbo

24

Table 3.a. Treatment Effect at Week 12 by Baseline Category (Drug X, Study 1)

Baseline Category1 2 3 4

Counts Act Veh Dif Act Veh Dif Act Veh Dif Act Veh DifInf. 13.3 13.2 0.1 17.0 20.0 -3.0 19.4 22.9 -3.5 23.2 26.8 -3.6Non. 18.1 23.5 -5.4 25.7 36.7 -11.0 41.3 47.0 -5.7 59.7 85.6 -25.9Total 31.4 36.7 -5.3 42.7 56.6 -13.9 60.8 69.8 -9.0 82.9 112.5 -29.5ChangeInf. 6.2 4.3 1.9 7.6 6.0 1.6 8.3 6.9 1.4 13.6 8.9 4.7Non 14.7 10.9 3.8 19.1 7.8 11.3 24.1 17.5 6.6 42.5 26.6 15.9Total 20.9 15.3 5.6 26.7 13.9 12.8 32.3 24.4 7.9 56.1 35.4 20.7% ChgInf. 30.9 24.4 6.5 28.4 18.4 10.0 32.2 17.7 14.5 35.9 26.8 9.1Non 44.0 31.2 12.8 41.3 17.0 24.3 35.8 27.2 8.6 42.6 21.9 20.7Total 40.2 29.4 10.8 38.3 19.1 19.2 35.3 26.2 9.1 40.8 23.5 17.3

25

Table 3.b. Comparison of IGE Success* Rates at Week 12 by Baseline Category (Drug X, Study 1)

*Success = None or Minimal Acne

26

Table 4.a. Treatment Effect at Cycle 6 by Baseline Category (Drug Y, Study 2)

Baseline Category1 2 3 4

Counts Act Pbo Dif Act Pbo Dif Act Pbo Dif Act Pbo DifInf. 9.6 10.2 -0.6 11.8 12.0 -0.3 14.5 17.4 -2.9 18.9 24.4 -5.5Non. 10.3 13.2 -2.9 17.7 22.5 -4.8 38.9 44.0 -5.2 110.7 112.3 -1.6Total 21.7 25.8 -4.1 33.5 35.8 -2.2 50.2 60.0 -9.8 127.6 139.5 -11.9ChangeInf. 1.9 1.5 0.4 39. 3.5 0.4 7.4 4.2 3.2 19.7 13.9 5.8Non 1.0 -1.8 2.7 4.6 -0.5 5.1 7.3 0.2 7.1 13.7 -7.1 20.8Total 5.4 0.8 4.6 9.4 7.5 1.9 19.5 11.2 8.3 25.1 -5.6 30.7% ChgInf. 14.8 12.3 2.5 25.1 21.4 3.6 33.9 19.4 14.5 49.9 36.2 13.7Non 5.4 -17.7 23.1 20.2 -6.7 26.9 14.8 3.1 11.7 11.8 0.3 11.5Total 19.2 2.8 16.4 23.0 17.7 5.3 28.3 15.5 12.7 19.5 -0.8 20.3

27

Baseline CategoryTreat.

1 2 3 4

Overall

Active 28/43

(65%)

25/51

(49%)

21/46

(46%)

15/44

(34%)

89/184

(48%)

Placebo 28/49

(57%)

17/42

(40%)

12/46

(26%)

12/47

(26%)

69/184

(38%)

Trt effect()

8% 9% 20% 8% 10%

(% success in active group – % success in placebo group)

Table 4.b. Comparison of IGE Success* Rates by Baseline Category (Drug Y, Study 2)

*Success = ‘Clear’ or ‘Almost Clear/Mild’

28

Comments about efficacy results by baseline category

For the two drugs considered, no general pattern for results for lesion counts by type, their change, or percent change

Similarly, for the two drugs considered, there is also no general pattern for the IGE

For the range of lesion counts in these studies, efficacy results do not appear to vary by baseline severity

29

5. Final Comments

(a) Analysis of change from baseline, percent change, and final counts with baseline as covariate attempt to account for variability at baseline

(b) Percent change data could have extreme outliers when the baseline count is relatively small (e.g., inflammatory lesions)

30

Final Comments (cont’d)(c) Repeated measurements approach attempts

to reduce the influence of outliers (flares) by ‘averaging’ over time.

The impact of the repeated measures on the p-value depends on whether efficacy reached a plateau at previous time points.

(d) For the data sets considered, treatment efficacy did not vary by baseline severity whether one considers analysis of lesion counts or IGE