Is it potent? Can these results tell me?

Statistics for assays

Ann Yellowlees PhDQuantics Consulting Limited

Contents

Role of statistics in bioassayRegulationsEstimating Relative PotencyChoice of model for RP estimationParallelismCase study

Role of statistics in bioassayDesign / optimisation

Analysis

Validation

Why use statistics - regulationsValidation of routine and custom assays:ICH Q6B SPECIFICATIONS: TEST PROCEDURES AND ACCEPTANCE CRITERIA FOR BIOTECHNOLOGICAL/BIOLOGICAL PRODUCTSAssessment of biological properties constitutes an ....essential step in establishing a complete characterisation profileAppropriate statistical analysis should be appliedMethods of analysis, including justification and rationale, should be described fullyA relevant, validated potency assay should be part of the specifications for a biotechnological or biological drug substance and/or drug product

Why use statistics - regulationsStability testing, more defined by ICH:ICH Q5C QUALITY OF BIOTECHNOLOGICAL PRODUCTSAt time of submission, applicants should have validated the methods that comprise the stability-indicating profile and the data should be available for reviewICH Q1A (R2) STABILITY TESTINGAn approach for analyzing data of quantitative attribute that is expected to change with time is to determine the time at which the 95% one-sided confidence limit for the mean curve intersects the acceptance criterion

Estimating Relative PotencyAnalysis of dose response dataChoosing the best model for estimating RPChecks for parallelismEstimating RPCalculations: RP and its precision

Improving precision

Data typesResponse per unit (animal, welI, etc):Binary Dead / alive at a given time pointDiseased / disease free at a given time pointSummarised as % or proportionContinuous Antibody levelTime of deathOptical density

Continuous responseNote: Log concentration S shape Noise level varies

Continuous response means

What is Relative Potency?Potency of the sample in comparison with the reference

Mathematically this is the same as:

Estimating RP

When is it valid to calculate RP?When the bioassay is a dilution assaythe unknown preparation to be assayed is supposed to contain the same active principle as the standard preparation, but in a different ratio of active and inert compounds **Implies RP constant across concentrationsOne curve is a horizontal shift of the otheri.e. parallel curves** European Pharmacopoeia 3.1.1

Check for parallelismPh. Eur approach:Residual sum of squares (RSS) and the F-testArbitrary p valuePenalises good dataUSPConfidence intervals on differences between parametersArbitrary confidence levelArbitrary limits on widthPenalises bad dataOthersChi squared test Similar to (1)

Check for parallelismEP, USP are guidance only

No simple, generally applicable statistical solution exists to overcome these fundamental problems. The appropriate action has to be decided on a case-by-case basis USP Workshop 2008

Estimating RP from dataChoose a model; fit to each material

Check system suitability reference is behaving as expectedparallel models are appropriate

Calculate RP and 95% confidence interval

Choose a model

Choose a model

Linear model(4 concentrations)Assume: Middle 4 concentrations of interestParallel when the same for both materials

Four parameter logisticmodelNote: If A = 0 and B = 1 this is a simple logistic model: proportions.

Four parameter logisticmodel

Parallel: when A, Band scale are the same for both materials

Five parameter logisticmodel

Parallel: when A, B scale and asymare the same for both materials

Which model to use?

Which model to use?Consider the relevant range of concentrationsHow much do you need to know about the ends?

Pros and cons Need more data to fit curvesMore data = more precision?Weighting / variability at ends

Formal statistical tests for fitIs a 5PL model really necessary or is it a statistical remedy for a bad assay?

R CapenChair, USP workshop 2008

4 PL parallel model chosen

Calculate RP and 95% CIParallel model provides an estimate of logeRPHorizontal distance between the curveslogeRP = 0.233 Back-transform for RPRP = e0.233 = 1.26

95% confidence interval for RP (1.26)(0.84, 1.90)

Assay development / optimisationChoose statistical modelDesign assay Number of replicates per concentrationOperators, days etcto achieve required precision for RP

Set suitability criteria for assayReference behaviourParallelism

Model selection for an assayModel must:Fit the data

Allow RP calculation most of the timei.e. the curves are parallel

Provide precise estimates of RP

Example with 12 plates12 development plates runWide range of concentrations0.001 2000 IU/mlReference and sample3 replicates4 statistical models examinedLinear (4), Linear (6), 4PL, 5PL ParallelismPrecisionFit

Summary: Parallelism test (F)* Denominator = 12

Summary: Precision

Linear model: 4 points, parallel

Summary: Model selectionLinear model based on 4 concentrations

All 12 pairs passed linearity test

All 12 pairs passed parallelism testProvided the best precision No apparent bias

Improving precisionIf the linear model can be justified:Allows extra replication Better precision within plate? fewer plates required

How low can you go?2 doses: test for linearity cannot be done3 doses: test for linearity has low power

SummarySystem software provides most of the required statistics per plate

When do you need a statistician?Choosing modelAppropriate statistical analysis should be appliedMethods of analysis, including justification and rationale, should be described fullyDesigning and validating assayAssessing sources of variationSimulationSetting suitability criteriaA relevant, validated potency assay should be part of the specifications for a biotechnological or biological drug substance and/or drug product

Thank youBioOutsourceThe invitationThe data

Quantics staffKelly Fleetwood, Catriona KeerieAnalysis and graphics

Testing should have been done by timeof submission for marketing approvalBatch testing also needs to be done for marketed products- stability, shelf-life, purity*Testing should have been done by timeof submission for marketing approvalBatch testing also needs to be done for marketed products- stability, shelf-life, purity*Look at this plate a few more timesTriplicate wellsLog concentration*Look at this plate a few more timesTriplicate wellsLog concentration***Shown what parallelism looks like on the graphReal world ...Sample is similar to the reference*All except xmid (or alpha in linear case*All except xmid (or alpha in linear case*Data may show a definite curveData may show a definite curveslope at the midpoint = k * (B A)/ scale*Mention symmetry.

A looks the sameB does not look the same*A: sameB: not the same.Slope not the sameAll look good IN THE MIDDLE.. Depends partly on the range of concentrations that are of interest (SAMPLE DATA)***NB **NOTE: No apparent biases********