Arie Gafson 6th MS Research Day

Why is it important?

Why is it important?

Why is it important?

Why is it important?

Statistics – a hard nut to crack

Example 1

• 1996 – Mammography screening reports 21%

reduction in breast cancer mortality

• Represented a reduction from five to four in every thousand

women i.e. 0.1%

• In a representative sample, 92% of about 5000 women

overestimated the benefit 10-fold, 100-fold, and more.

!!DOCTORS ALSO STRUGGLE WITH STATISTICS!!

• 31% of 150 gynaecologists answered that for every 1000 women

who were screened, 25 or 250 fewer would die

Statistics – a hard nut to crack

Example 2

• 1995 - UK Committee on Safety of Medicines issues

a warning that third generation oral contraceptive pills

increase the risk of potentially life threatening thrombosis

twofold.

• 13 000 additional abortions in the next year in England and Wales

• For every 7000 women who took the earlier, second generation

pills, one had a thrombosis, and this number increased to two in

women who took third generation pills.

• This study was only reporting on the relative risk

Good News – common threadsRelative Risk Reduction (%)

Annualized Relapse Rate for Placebo

Absolute Risk Reduction

No. needed to treat

IFN – β1b ScPivotal Trial

34 1.27 0.43 2.3

IFN – β1a ScPRISMS Trial

33 1.28 0.42 2.4

IFN – β1a IMMSCRG

32 0.90 0.29 3.4

GA Pivotal Trial 29 0.84 0.24 4.2

Natalizumab AFFIRM Trial 68 0.78 0.50 2.0

FingolimodFREEDOMS

54 0.40 0.22 4.5

BG – 12 DEFINE 53 0.36 0.19 5.3

CladribineCLARITY

58 0.33 0.19 5.3

TEMSO 31.5 0.54 0.17 5.9

Relative risk reduction = difference

original

2 – 1 1= = 50%

2 2

Relative risk reduction = difference

original

Relative risk reduction = difference

original

2 – 1 1= = 50%

2 2

Absolute risk reduction = simply the difference

between the two

2 – 1 = 1 = 10% 10 10 10

Relative risk reduction = difference

original

Relative risk reduction = difference

original

8 – 6 2= = 25%

8 8

8 – 6 = 2 = 20% 10 10 10

Absolute risk reduction = simply the difference

between the two

Learning points!

• The relative risk reduction does not take

into account the baseline risk making an

insignificant finding appear significant

• Always check the baseline risk

• Always check the absolute risk reduction

Number Needed to Treat

“The average number of patients who need

to be treated to prevent one additional bad

outcome”

NNT = 100

ARR

NNT = 100 = 5 20

Number needed to Treat = 100

Absolute risk reduction

Trial Example 1A double blind Placebo controlled study with a new M.S.

drug (x) included 100 Placebo cases and 100 treated cases

(x). The primary end point was the occurrence of a relapse.

4 relapses in the placebo group versus 2 relapses in the

treated group at the end of the trial.

Event Rate (Relapse)

Control Drug ARR RRR NNT

4 = 4%100

2 = 2%100

4 – 2 = 2 2 = 50%4

100 = 502

Trial Example 2A double blind Placebo controlled study with a new M.S.

drug (y) included 100 Placebo cases and 100 treated cases

(y). The primary end point was the occurrence of a relapse.

40 relapses in the Placebo group versus 20 relapses in the

treated group (y) at the end of the trial.

Event Rate (Relapse)

Control Drug ARR RRR NNT

40 = 40%100

20 = 20%100

40 – 20 = 20

20 = 50%40

100 = 520

Trial Example 3

A double blind Placebo controlled study with a new M.S.

drug (z) included 100 Placebo cases and 100 treated cases

(z). The primary end point was the occurrence of a relapse.

90 relapses in the placebo group versus 60 relapses in the

treated group at the end of the trial.

Event Rate (Relapse)

Control Drug ARR RRR NNT

90 = 90%100

60 = 60%100

90 – 60 = 30 30 = 33.3%90

100 = 3.3 30

Comparison

Trial (1) Trial (2) Trial (3)

Risk (Relapse) in Placebo Group 4% 40% 90%

Relative Risk Reduction (RRR) 50% 50% 33.3%

NNT 50 5 3.3

• Treatments with very large RRR may have a small effect

in conditions where the control group has a very low

event rate.

• On the other hand, modest RRR can assume major

clinical importance if the control event rate is large.

Two-year Relapse Reduction: Relative Risk

Reduction Vs. Absolute Risk Reduction

Relative Risk Reduction (%)

Annualized Relapse Rate for Placebo

Absolute Risk Reduction

No. needed to treat

IFN – β1b ScPivotal Trial

34 1.27 0.43 2.3

IFN – β1a ScPRISMS Trial

33 1.28 0.42 2.4

IFN – β1a IMMSCRG

32 0.90 0.29 3.4

GA Pivotal Trial 29 0.84 0.24 4.2

Natalizumab AFFIRM Trial 68 0.78 0.50 2.0

FingolimodFREEDOMS

54 0.40 0.22 4.5

BG – 12 DEFINE 53 0.36 0.19 5.3

CladribineCLARITY

58 0.33 0.19 5.3

TEMSO 31.5 0.54 0.17 5.9

Conclusions

• The value of the relative risk reduction of the Relapse

Rate as a clinical outcome, depends on the risk of an

event in the control group.

• Low relapse rates in the control group may yield high

relative risk reductions yet they are clinically less

significant, as reflected by a high NNT.

• Modest relative risk reductions may be clinically more

significant if the relapse rate of the control group is high.

Food for thought• An analysis of the articles published in the Annals of Internal

Medicine, BMJ, JAMA, Journal of the National Cancer

Institute, Lancet, and the New England Journal of Medicine,

2003-4, showed that 68% (150/222) failed to report the

underlying absolute risks in the abstract.

• An analysis of 119 systematic reviews in BMJ, JAMA, and

Lancet from 2004 to 2006 showed that every second article

discussed only relative risks or odds ratios.

Misleading communication of

risk

BMJ 2010;