Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Florian Klinglmueller*
Ack: Andreas Brandt, Thomas Lang, Ina Rondak
Operating characteristics of
frequently used similarity rules
*Austrian Medicines & Medical Devices Agency
The contents of this presentation are my personal opinion.
My remarks do not necessarily reflect the official view of AGES.
Comparison of an originator product to a biosimilar product with respect to
„critical quality attributes“ (i.e. physical, chemical, biological, or microbiological
properties that ensure product quality) with the aim to conclude similarity on the
quality level.
One quality attribute on a continuous scale. Comparison between samples from
originator and biosimilar.
Different rules to decide whether samples from the biosimilar are similar to
samples from the originator - based on sample data
Explore operating characteristics of commonly used rules under different scenarios
of similarity and dissimilarity.
Similarity assessment of quality attributes
Introduction
Decision rule
Problem: Similar is not equal!
⇒ Specify what is similar enough
Average similarity
• E.g.: Biosim on average within ± 1.5
standard deviations of reference mean
• E.g.: Ratio of CQA on average 80%-125%
Population based similarity
• Reference samples define margin of what
is safe (e.g. min-max, TI)
• Future/observed biosim batches fall into
range
Translate into a criterion
How to compute „similar/not similar“ from data
Decision rule
Problem: Similar is not equal!
⇒ Specify what is similar enough
Average similarity
• E.g.: Biosim on average within ± 1.5
standard deviations of reference mean
• E.g.: Ratio of CQA on average 80%-125%
Population based similarity
• Reference samples define margin of what
is safe (e.g. min-max, TI)
• Future/observed biosim batches fall into
range
Translate into a criterion
How to compute „similar/not similar“ from data
Decision rule
Problem: Similar is not equal!
⇒ Specify what is similar enough
Average similarity
• E.g.: Biosim on average within ± 1.5
standard deviations of reference mean
• E.g.: Ratio of CQA on average 80%-125%
Population based similarity
• Reference samples define margin of what
is safe (e.g. min-max, TI)
• Future/observed biosim batches fall into
range
Translate into a criterion
How to compute „similar/not similar“ from data
Min-Max: gives a the range of the observed values. Purely descriptive i.e. permits
little inference about future samples of the process, except that true range is wider
X-SD: estimates the variation in the sample around the sample mean. Purely
descriptive, many statistical intervals are constructed by choosing x such that
probabilistic statements hold
• E.g. 95% CI: x=1.96; 95% PI (n=10): x=2.16, 95/95 TI (n=10): x=3.38
Confidence Interval: estimates a range that should cover an unknown parameter
(e.g. mean) of the distribution assumed to generate the data
Prediction Interval: estimates a range that should cover the value of the next
sample from distribution assumed to generate the data
β-content Tolerance Interval: estimates a range that should cover a certain
proportion β of future samples from the distribution assumed to generate the data
Statistical Intervals Probabilistic interpretation of different interval types
Min-Max: gives a the range of the observed values. Purely descriptive i.e. permits
little inference about future samples of the process, except that true range is wider
X-SD: estimates the variation in the sample around the sample mean. Purely
descriptive, many statistical intervals are constructed by choosing x such that
probabilistic statements hold
• E.g. 95% CI: x=1.96; 95% PI (n=10): x=2.16, 95/95 TI (n=10): x=3.38
Confidence Interval: estimates a range that should cover an unknown parameter
(e.g. mean) of the distribution assumed to generate the data
Prediction Interval: estimates a range that should cover the value of the next
sample from distribution assumed to generate the data
β-content Tolerance Interval: estimates a range that should cover a certain
proportion β of future samples from the distribution assumed to generate the data
Statistical Intervals Probabilistic interpretation of different interval types
Frequentist confidence: in repeat experimentation range estimate
computed in this way will cover the quantity (parameter, next sample,
all future samples) a certain proportion of times (e.g. 95%)
Rules for concluding biosimilarity
Min-Max: All samples of the biosim are between min-max of the originator
X-Sigma: All samples from the biosim are within x-standard deviations of the
originators mean
(75%/90%) Tolerance interval: All samples from the biosim are within a P/Q
Tolerance interval of the originator
TI Specs: The P/Q tolerance interval of the biosim is within „specifications“ (e.g.
Min-Max) of the originator
FDA Rule: The 90% confidence interval for mean difference between originator
and biosim is within a similarity margin of 1.5 standard deviations of originator
A selection of frequently used decision rules
Differences in mean, equal variance
Simulation scenario 1
Originator and Biosim samples follow
standard normal distribution
Equal variance
Distance between distributions
expressed as multiples of the
(common) standard deviation
Settings considered:
• M2-M1= {0, 0.5, 0.8, 1, 1.5} * SD
Differences in mean, equal variance
Simulation scenario 1
Originator and Biosim samples follow
standard normal distribution
Equal variance
Distance between distributions
expressed as multiples of the
(common) standard deviation
Settings considered:
• M2-M1= {0, 0.5, 0.8, 1, 1.5} * SD
Differences in mean, equal variance
Simulation scenario 1
Originator and Biosim samples follow
standard normal distribution
Equal variance
Distance between distributions
expressed as multiples of the
(common) standard deviation
Settings considered:
• M2-M1= {0, 0.5, 0.8, 1, 1.5} * SD
Differences in mean, equal variance
Simulation scenario 1
Originator and Biosim samples follow
standard normal distribution
Equal variance
Distance between distributions
expressed as multiples of the
(common) standard deviation
Settings considered:
• M2-M1= {0, 0.5, 0.8, 1, 1.5} * SD
Equal means (biosimilar) shown with dashed lines, unequal means solid lines.
Probability to conclude similarity decreases with increasing dissimilarity (m=10,n=10)
Simulation results: Equal variances Most simple scenario
Equal means (biosimilar) shown with dashed lines, unequal means solid lines.
Probability to conclude similarity decreases with increasing dissimilarity (m=10,n=10)
If we increase the sample size (m=20,n=20) several things happen
Simulation results: Equal variances Most simple scenario
Equal means (biosimilar) shown with dashed lines, unequal means solid lines.
Probability to conclude similarity decreases with increasing dissimilarity (m=10,n=10)
If we increase the sample size (m=20,n=20) several things happen
Similarity conclusions increase, with increasing originator samples (except TI)
Simulation results: Equal variances Most simple scenario
Equal means (biosimilar) shown with dashed lines, unequal means solid lines.
Probability to conclude similarity decreases with increasing dissimilarity (m=10,n=10)
If we increase the sample size (m=20,n=20) several things happen
Similarity conclusions increase, with increasing originator samples (except TI)
With increasing Biosim samples similarity conclusions increase for for interval based
criteria
Simulation results: Equal variances Most simple scenario
Difference in means, unequal variance
Simulation scenario 2
Originator and biosim distribution may
differ in mean and in variance
Standard deviation of originator is
sdratio times larger than biosim
Values for sdratio: .25 - 2
sdratio=1 corresponds to
Scenario 1
Both cases, assuming equal and
unequal means, were investigated
Difference in means, unequal variance
Simulation scenario 2
Originator and biosim distribution may
differ in mean and in variance
Standard deviation of originator is
sdratio times larger than biosim
Values for sdratio: .25 - 2
sdratio=1 corresponds to
Scenario 1
Both cases, assuming equal and
unequal means, were investigated
Difference in means, unequal variance
Simulation scenario 2
Originator and biosim distribution may
differ in mean and in variance
Standard deviation of originator is
sdratio times larger than biosim
Values for sdratio: .25 - 2
sdratio=1 corresponds to
Scenario 1
Both cases, assuming equal and
unequal means, were investigated
Difference in means, unequal variance
Simulation scenario 2
Originator and biosim distribution may
differ in mean and in variance
Standard deviation of originator is
sdratio times larger than biosim
Values for sdratio: .25 - 2
sdratio=1 corresponds to
Scenario 1
Both cases, assuming equal and
unequal means, were investigated
Difference in means, unequal variance
Simulation scenario 2
Originator and biosim distribution may
differ in mean and in variance
Standard deviation of originator is
sdratio times larger than biosim
Values for sdratio: .25 - 2
sdratio=1 corresponds to
Scenario 1
Both cases, assuming equal and
unequal means, were investigated
Impact of different variances on decision rules:
Biosimilar more variable left of horizontal line, reference right of line
Monotone relationship between ratio of variances and probability of concluding
similarity, i.e. more variable reference process -> more likely to conclude similarity
x-Sigma and TI rules often (erroneously) conclude biosimilarity even if Biosim
samples are more variable and means different.
Larger variance in reference to the right
Shift in originator process
Scenario 3
Note: Illustrations use shift of +-5*SD to get a bimodal distribution
Originator samples come from a
mixture of normal distributions with
different means
Means of originator (mixture)
distribution are a multiple of SD apart
Positive shifts are into the direction of
the biosim process
Negative shifts are away from the
biosim process
Impact of shift on decision rules
Dotted line reports probability of self-similarity conclusion
Horizontal line indicates scenario where biosimilar is equal to post-shift process
Probability to conclude similarity is smaller compared to no shift when shift is
slightly opposite to test mean
It increases when shift is towards test mean
However, probability to conclude similarity for acceptance ranges based on
reference SD converges to 1 both for large shifts towards and opposite test mean
Summary & Outlook
Dichotomy of Type I and Type II errors does not apply to an equivalence decision -
boundaries between success and Type I error are fuzzy
Some rules (TI, X-Sigma) have undesirable properties (decreasing power with
increasing sample size, increasing error probability for shifts away from the biosim)
We have only considered simple scenarios; have not considered:
• Alternative designs, use of historical data
• Issues with sampling (originator from the market, biosim from production process under
development)
• Multiplicity (typically there are more than one CQA)
• Sequential decision making (issues with one CQA are discredited using data from another
CQA)
BASG -
Austrian Federal Office for Safety in Health Care
www.basg.gv.at
Traisengasse 5
1200 Vienna
Florian Klinglmueller
Biostatistician
T + 43 (0) 50 555 36624