SJS Harpenden: Competence and Equivalence 1 Competence and Equivalence Stephen Senn Department of Statistical Science UCL

SJS Harpenden: Competence and Equivalence

1

Competence and Equivalence

Stephen Senn

Department of Statistical Science

UCL


2

Outline

• Why this is important– Practical contexts in which the general problem

occurs• Particular emphasis on bioequivalence

• A simple model for competence and equivalence

• Conclusions


3

Difference or Equivalence?

• Much of statistical theory is devoted to methods of proving that putative causes can effect outcomes– streptomycin on course of TB– smoking and lung cancer

• Sometimes we wish to prove non-effect– MMR does not increase risk of autism– generic is no different from brand-name drug


44

Bioequivalence Studies

These are special examples of equivalence studies in which we try to show that two formulations of a drug are equivalent by comparing AUC for the serum concentration over time.

Uses:

1) generic versus brand name comparisons

2) development bridging

3) new formulations

4) interaction studies.


5

When is a drug not a drug?

• Taylor et al, Lancet 2001– 581 samples, 27 drugs, 35 pharmacies, Lagos and

Abuja, Nigeria

– 48% did not comply with pharmacopoeial limits

• Newton et al Lancet 2001– 104 samples of anti-malarial artesunate in Burma,

Cambodia, Laos, Thailand and Vietnam

– 39 (38%) had no active ingredient


65

A C Ce I

DPK s PK e

PD

R

Based on Sheiner (1992)

Absorption, Concentration, Effect-Site Concentration

Response

Dose

Equality here implies equality of all response


77

Treatment Difference

A

B

C

D

E

F

G


88

The Basic Position (Testing View)

H0A : ( )

(Exact equality) 0

H0B :

(

H1

(Diff. allowed for in power calculation)

H1B

H1A

H1 = H1A H1B

H0 = H0A H0B()


9

Critical Values for 3 Different Approaches to Bioequivalence

0.0 0.1 0.2 0.3 0.4 0.5

Standard Error

0.0

0.1

0.2

0.3

Crit

ical

Val

ue o

f log

-AU

C

TOSTLindley's proposed critical valuesNeyman-Pearson critical values

Limit of equivalence = 0.223

0.136 = 0.223/1.645

0.229

{(k-)/SE}- {(-k-)/SE}


10

Swatch and Rolex

• There is more than one quality of a pharmaceutical

• As soon as you can no longer use serum concentration there is a problem

• Pharmacodynamics is multi-dimensional

• Does this not imply that equality has to be proved multi-dimensionally?


11

How Should Equivalence Trials in Asthma be Designed?

The assumption behind the question is that this is possible. I am not convinced that it is. Conventional equivalence trials work like this.

Route of absorption 1

Route of absorption 2 Circulation

Other sites

Equality here implies equality for target and all other sites


12

Treatment Placebo MT&A 6 MT&A 12 MT&A 24

FEV1 (L)

2.0

2.5

Minute

0 180 360 720

Placebo and the 3 doses of the new formulation


13

Treatment Placebo MT&A 6 MT&A 12 MT&A 24ISF 6 ISF 12 ISF 24

FEV1 (L)

2.0

2.5

Minute

0 180 360 540 720

With the 3 doses of reference formulation added


14

-0.1 0.0 0.1 0.2

log-AUC FEV1

Formoterol

Formulation

Dose

ParallelismParallelism

Curvature

Opposing Curvature

Orthogonal Contrasts for Parallel Assay


15

Other Examples

• Drug interaction issues– no food interaction

– no sex interaction

• Model adequacy issues– no carry-over

• Safety issues– Breast implants are safe?

– Eating beef is safe?

– Accidents do not increase risk of MS?


16

Blinding and Equivalence

• Running a double blind trial does not protect you against a conclusion of equivalence

• You do not need to know the treatment code to bias results towards equivalence


17

The Unscrupulous Pharmacokineticist• Take the 12 test tubes for day one for a given

volunteer– hour 1,2…12

• Take the 12 test tubes for day two for the same volunteer– hour 1,2…12

• Mix each pair (by hour) together• Divide them into two• Et voila

– Perfect equivalence without having to unblind


18

Fanciful?

• Pharmaceutical companies commonly prosecute cheating doctors

• Reason– Trial fails to show any effect whereas others do

• Explanation– The trial never took place– The data have been invented– This will produce a conclusion of equivalence


19

Fairness and Competence

• Experiment is fair if treatments are handled equivalently– in all aspects except those that form the essence

(definition) of the treatment

– cannot be determined by looking at outcomes

• Competence is the ability to detect differences– can only partly be determined on external grounds

– can be established if difference is detected


20

A Model for Competencecompetent, not competent

equivalent, inequivalent

observed difference, no difference

Likelihoods

( ) ( ) 1

( ) ( ) 1

( ) ( ) 1

( ) ( ) 1

1 0

"Priors" (

C C

E E

D D

P D E C P D E C

P D E C P D E C

P D EC P D EC

P D EC P D EC

P E

) , ( )P C E


21

Interpretation of These Parameters

• and 1- reflect the ‘precision’ of our experiments – analogous to type I and II error rates – can be reduced by more and more precise

experiments

• Joint effect of and represents factors beyond our control


22

Notes

Under this formulation of the likelihoods it is irrelevant as to whether the trial is competent if the treatments are equivalent.

We could require the combination EC as impossible

We require > , but this is a linguistic convention


23

For those who like formulae

1 (1 )( )

(1 ) (1 )

as 1 and 0

( ) 1

(1 )( )

(1 )(1 ) (1 )(1 )(1 ) (1 )

as 1 and 0

( )(1 )(1 )(1 )

P E D

P E D

P E D

P E D


24

Pi

po

ste

rio

r p

rob

ab

ility

of

no

n-e

qu

iva

len

ce

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Pi

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Probability of non-equivalence given observed difference as a function of the strength, Pi, of the study

Alpha=1Alpha=0.5


25

Pi

po

ste

rio

r p

rob

ab

ility

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Pi

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Probability of competence and equivalence given no difference as a function of the strength, Pi, of the study

CompetenceEquivalence


26

Consequences

• Assymetry between concluding equivalance and difference– The former is inherently more problematic

• Not just a matter of reformulating the problem

• Conditional on an assumption of competence we can conclude equivalence– However if we have any doubts regarding

competence these increase by failing to find a difference


27

In summary

“Equivalence is different”

28

A KoanIf not Zen then Senn

“The jealous husband fears that his wife is unfaithful and determines to try and set his mind at ease. He hires a private detective to investigate her. Several weeks and large sums of money later the detective issues his report. He has followed the wife for weeks, investigated her phone calls and observed her every movement. In an extensive report he explains that there is no evidence whatsoever that she is having an affair.

The husband is much relieved. His prior doubt has been changed into posterior belief. He goes to bed with a light heart for the first time in months. At three o' clock in the morning he wakes with a frightening thought. ...Suppose his wife is having an affair with the detective?” From Chapter 4 of Dicing with Death, Cambridge University Press 2003 (to appear)

Documents

SJS Harpenden: Competence and Equivalence 1 Competence and Equivalence Stephen Senn Department of Statistical Science UCL