View
34
Download
0
Category
Tags:
Preview:
DESCRIPTION
Statistics for Clinical Trials in Neurotherapeutics. Barbara C. Tilley, Ph.D. Medical University of South Carolina. Funding:. NIA Resource Center on Minority Aging 5 P30 AG21677. NINDS Parkinson’s Disease Statistical Center U01NS043127 and U01NS43128. Sample Size. - PowerPoint PPT Presentation
Citation preview
11
Statistics for Statistics for Clinical Trials in Clinical Trials in NeurotherapeuticsNeurotherapeutics
Barbara C. Tilley, Ph.D.Barbara C. Tilley, Ph.D. Medical University of South Medical University of South CarolinaCarolina
22
Funding:Funding:
NIA Resource Center on Minority Aging5 P30 AG21677
NINDS Parkinson’s Disease Statistical Center U01NS043127 and U01NS43128
33
Sample SizeSample Size
44
Issues in Issues in NeurotherapeuticsNeurotherapeutics What is the outcome?What is the outcome? How will this be measuredHow will this be measured
– One or many measures of outcome?One or many measures of outcome? How will you analyze the data?How will you analyze the data?
(Nquery $700, STPLAN free, etc.)(Nquery $700, STPLAN free, etc.)
55
Sample Size: Sample Size: Putting it all togetherPutting it all together
Continuous (Normal) DistributionContinuous (Normal) Distribution
Need all but one: Need all but one: , , , , 22, , , N , N Z Z = 1.96 (2 sided, 0.05);= 1.96 (2 sided, 0.05); ZZ = 1.645 (always one-sided, 0.05,= 1.645 (always one-sided, 0.05, 95% power)95% power) = difference between means= difference between means 22 = pooled variance= pooled variance
2
22
)Z4(Z2n
66
Adjusting for Adjusting for Drop-outs/Drop-insDrop-outs/Drop-ins
10% dropout, increasing 10% dropout, increasing sample size by 10% is not sample size by 10% is not enoughenough
Use: 1/(1-R)Use: 1/(1-R)22
Friedman, Furburg, DeMetsFriedman, Furburg, DeMets
77
Sample Size for Sample Size for Multiple Primary Multiple Primary OutcomesOutcomes Choose largestChoose largest
sample size for anysample size for any
single outcome.single outcome. If multiple aims, useIf multiple aims, use
largest sample size forlargest sample size for
any aim. any aim.
88
Sample Size: Sample Size: Food for ThoughtFood for Thought
Is detectable difference Is detectable difference biologically/clinically meaningful?biologically/clinically meaningful?
Is sample size too small to be Is sample size too small to be believable? WHERE DID YOU GET believable? WHERE DID YOU GET the estimate????the estimate????
Report power (for design), not Report power (for design), not conditional power conditional power for negative for negative study.study.
99
Sample Size: Sample Size: Keeping It SmallKeeping It Small
Study continuous outcomeStudy continuous outcome (if variability does not increase)(if variability does not increase)
Updrs Score rather “above or below cut-Updrs Score rather “above or below cut-point”point”
Study surrogate outcome whereStudy surrogate outcome where effect is largeeffect is large
Rankin at 3 months rather than stroke Rankin at 3 months rather than stroke mortalitymortality
Reduce variability (ANCOVA, training, Reduce variability (ANCOVA, training, equipment, choosing model)equipment, choosing model)
1010
Sample Size: KeepingSample Size: Keeping It Small It Small Difference between two means = Difference between two means =
11
Standard deviation = Standard deviation = 22; ; N = N = 6464/group/group
Standard deviation = Standard deviation = 11; ; N = N = 1717/group/group
1111
AnalysisAnalysis
Parametric?Parametric?– NormalNormal– BinomialBinomial
Nonparmetric?Nonparmetric?– RankedRanked
1212
Distribution of Distribution of Barthel IndexBarthel Index
05
101520253035404550
rt-PA
Placebo
100
1313
Sample SizeSample Size
Sample size to detect effect of sizeSample size to detect effect of sizeobserved in NINDS t-PA Stroke Trialobserved in NINDS t-PA Stroke Trial BarthelBarthel::
Non-parametric N = 507Non-parametric N = 507Binary N = 335Binary N = 335
RankinRankin::Non-parametric N = 394Non-parametric N = 394Binary N = 286Binary N = 286
1414
Multiple ComparisonsMultiple Comparisons
Different questions, can argueDifferent questions, can argue no adjustment (O’Brien, 1983)no adjustment (O’Brien, 1983)
– Effect on blood pressureEffect on blood pressure– Effect on quality of lifeEffect on quality of life
All pair-wise comparisons or All pair-wise comparisons or multiple measures of same multiple measures of same outcome, adjustoutcome, adjust
– Pairwise comparisons ofPairwise comparisons ofDrugs A, B, C (same outcome)Drugs A, B, C (same outcome)
1515
Multiple Multiple ComparisonsComparisons
Bonferroni (or less conservative Bonferroni (or less conservative Simes, or Hockberg)Simes, or Hockberg)– /#tests = 0.05/5 = 0.01/#tests = 0.05/5 = 0.01
– Sample size, use adjusted Sample size, use adjusted ANOVA methods – Tukey’s, etc.ANOVA methods – Tukey’s, etc.
– Sample size for ANOVASample size for ANOVA
1616
Bonferroni for Different Bonferroni for Different Primary Outcomes, Same Primary Outcomes, Same ConstructConstruct
All outcomes measure same constructAll outcomes measure same construct– Stroke recoveryStroke recovery– PD progressionPD progression
May lack power when most measures May lack power when most measures of efficacy are improved, but no single of efficacy are improved, but no single measure is overwhelmingly so.measure is overwhelmingly so.
Problem exacerbated when outcomes Problem exacerbated when outcomes are highly correlated.are highly correlated.
1717
Use Global Tests Use Global Tests When:When: No one outcome sufficient or No one outcome sufficient or
desirabledesirable Outcome is difficult to measure Outcome is difficult to measure
and combination of correlated and combination of correlated outcomes usefuloutcomes useful
1818
Properties of Properties of Global TestGlobal Test If all outcome measures perfectly If all outcome measures perfectly
correlated, correlated, test statistic, p-value same as for test statistic, p-value same as for
single (univariate) test single (univariate) test power = power of univariate testpower = power of univariate test
Assumes common dose effectAssumes common dose effect Power increases as correlation Power increases as correlation
among outcomes decreasesamong outcomes decreases
1919
O’Brien’s Non-O’Brien’s Non-parametric Procedure parametric Procedure (Biomet., 1984)(Biomet., 1984)
Separately rank each outcome in Separately rank each outcome in the two treatment groups the two treatment groups combined.combined.
Sum ranks for each subject.Sum ranks for each subject. Compare mean ranks in the two Compare mean ranks in the two
treatment groups usingtreatment groups using– Wilcoxon or t-test Wilcoxon or t-test – ANOVA if more than two treatmentsANOVA if more than two treatments
2020
Sample Size forSample Size forGlobal TestGlobal Test Use largest sample size for single Use largest sample size for single
outcomeoutcome
2121
NINDS t-PA Stroke NINDS t-PA Stroke Trial Binary Outcomes Trial Binary Outcomes (Part II)(Part II)
Outcome rt-PA
Placebo
Odds R. 95% C.L. P
Barthel 50 38 1.63 1.06-2.49 0.03
Rankin 39 26 1.68 1.09-2.59 0.02
Glasgow 44 32 1.64 1.06-2.53 0.03
NIHSS 31 20 1.72 1.05-2.84 0.03
Global 1.73 1.16-2.60 0.008
2222
NINDS t-PA Trial NINDS t-PA Trial Observed Agreement & Observed Agreement & Correlations for Binary Correlations for Binary OutcomesOutcomesMeasure % Agreement Phi Coeff.Barthel, NIHSS 77 0.55Barthel, Rankin 87 0.76Barthel, Glasgow 89 0.78NIHSS, Rankin 86 0.67NIHSS, Glagow 85 0.69Rankin, Glasgow 94 0.88
2323
RandomizationRandomization
2424
RandomizationRandomization
StratificationStratification– Age, prior stroke, years with PD, siteAge, prior stroke, years with PD, site– Greatest gain if N < 20Greatest gain if N < 20– Too many strata, difficult to balanceToo many strata, difficult to balance
3 age x 2 years with PD x gender = 123 age x 2 years with PD x gender = 12 Blocking – balance number in each Blocking – balance number in each
treatment grouptreatment group– Important if number expected per site is Important if number expected per site is
smallsmall Minimization Minimization
– Can be complicated to implement, cause Can be complicated to implement, cause delays delays
2525
Interim AnalysesInterim Analyses
Who?Who? Why?Why? When?When? How?How?
2626
1 2 3 4 5 # Looks
5.0
3.0
2.0
-2.0
-3.0
-5.0
Sta
ndard
Norm
al Sta
tist
ic (
Zi)
Reject Ho
Reject Ho
Continue Fail to Reject Ho
O’Brien-Fleming
Pocock
Peto
Stopping “Guidelines”
0
2727
Intent-to-Treat (ITT)Intent-to-Treat (ITT)
Intent-to-treat means Intent-to-treat means analyzinganalyzing
ALLALL patients as randomized. patients as randomized.Patients lost to follow-up (LTF)Patients lost to follow-up (LTF)Patients who do not adhere to Patients who do not adhere to
treatmenttreatmentPatients who were randomized Patients who were randomized
and did not receive treatmentand did not receive treatmentPatients incorrectly randomizedPatients incorrectly randomized
2828
ImputationImputation
Definition - replacing a value for Definition - replacing a value for those lost to follow-up or not those lost to follow-up or not adhering.adhering.
Imputation may or may not be Imputation may or may not be ITT.ITT.
2929
Optimal ApproachOptimal Approach
MAKE IMPUTATION UNECESSARY!MAKE IMPUTATION UNECESSARY!
3030
Optimal Approach Optimal Approach ContinuedContinued Make follow-up a high priorityMake follow-up a high priority Monitor follow-up closelyMonitor follow-up closely Build in patient incentives Build in patient incentives
– ““gifts” for patients (t-shirts, mugs, gifts” for patients (t-shirts, mugs, etc.)etc.)
– free parking, meal ticketfree parking, meal ticket– TransportationTransportation
Follow even those off treatmentFollow even those off treatment
3131
Hypertension Detection and Hypertension Detection and Follow-up Program/MRFITFollow-up Program/MRFIT
Outcome was mortalityOutcome was mortality HDFP 21/10,940HDFP 21/10,940 MRFIT 30/12,866MRFIT 30/12,866 Used Death Index, Social Used Death Index, Social
Security, detectivesSecurity, detectives
3232
NINDS t-PA Stroke NINDS t-PA Stroke TrialTrial Four 3-month outcomes Four 3-month outcomes
– Barthel,NIHSS,GOS, RankinBarthel,NIHSS,GOS, Rankin NINDS Project Officer pushed for NINDS Project Officer pushed for
complete ascertainmentcomplete ascertainment Study staff made house calls, searched Study staff made house calls, searched
medical recordsmedical records 5/612 (<1%) lost to follow-up on at 5/612 (<1%) lost to follow-up on at
least one of the four outcome measuresleast one of the four outcome measures Used worst value possibleUsed worst value possible
3333
NET-PD Futility StudiesNET-PD Futility StudiesLTF for 1-year outcomeLTF for 1-year outcome(Used worst outcome in assigned group)(Used worst outcome in assigned group)
FS-1FS-1 3/ 3/200200– Creatine 2Creatine 2– Minocycline 0Minocycline 0– Placebo 1Placebo 1
FS-2FS-2 4/ 4/213213– GPI 3GPI 3
– CoQCoQ10 10 11
– Placebo 0Placebo 0
3434
Handling Missing Handling Missing ValuesValues Why?Why? How?How?
3535
When Data Are When Data Are Missing:Missing:Common ApproachesCommon ApproachesApproach ITT Imputation
Completers NO NO
Missing at Random ? NO
Last Obs. Carried Forward
YES YES
Worst Case YES YES
Best/Worst YES YES
Rubin (1998) NO YES
Little & Lau/Others YES YES
3636
Subgroup AnalysesSubgroup Analyses (Sub-set) (Sub-set) Pre-specified based on rationalePre-specified based on rationale
– NINDS t-PA Stroke TrialNINDS t-PA Stroke Trial Those randomized 0-90 minutes and 91-Those randomized 0-90 minutes and 91-
180 minutes from stroke onset180 minutes from stroke onset
Post-hoc in the presence of Post-hoc in the presence of interactioninteraction– (Yusuf, 1991)(Yusuf, 1991)
3737
Subgroup AnalysesSubgroup Analyses
The more subgroups examined, The more subgroups examined, the more likely analyses will lead the more likely analyses will lead to finding a difference by chance to finding a difference by chance alone.alone. 10 mutually exclusive subgroups;10 mutually exclusive subgroups; 20% chance that in one group the 20% chance that in one group the
treatment will be better than control treatment will be better than control and that the converse will be true in and that the converse will be true in anotheranother
3838
Example of Interaction Example of Interaction (Effect Modification) (Effect Modification)
0
10
20
30
40
50
60
Placebo Treatment
% M
ort
alit
y Men
Women
3939
Example of InteractionExample of Interaction(Effect Modification)(Effect Modification)
0
10
20
30
40
50
60
Placebo Treatment
% M
ort
alit
y
Men
Women
4040
Lack of InteractionLack of Interaction
0
10
20
30
40
50
60
70
Placebo Treatment
% M
ort
alit
y
Men
Women
4141
Trial of Org10172 for Trial of Org10172 for Stroke (TOAST) TrialStroke (TOAST) Trial
0
10
20
30
40
50
60
70
80
90
Placebo Org 10172
% 3
mo
nth
Fav
. O
utc
om
eMen
Women
N = 379(M) 238 (F) N=372(M) 239 (F)
Test for interaction p = 0.251
4242
Pooled AnalysisPooled AnalysisCarotid Carotid EndarterectomyEndarterectomy
0
10
20
Medical Surgical
% w
/ P
rim
ary
Ou
tco
me
Men
Women
N (men) 4175 N(women) 1718 Test for interaction p = 0.007 (Cox model)
Rothwell, 2004 NASCET &ECSTRothwell, 2004 NASCET &ECST
4343
Pooled Analysis Pooled Analysis ECASS, Atlantis, NINDSECASS, Atlantis, NINDS Kent 2005Kent 2005
0
10
20
30
40
50
Placebo t-PA
% M
RS
<=
1Men
Women
N (men) 4175 N(women) 1718 Test for interaction p = 0.04 (logistic model)
4444
4545
ReferencesReferences
Rubin, DB. More powerful randomization-based p-values in Rubin, DB. More powerful randomization-based p-values in double blind trials with non-compliance. Statistics in double blind trials with non-compliance. Statistics in Medicine (1998) 17:317-385.Medicine (1998) 17:317-385.
Little R, Yau L. Intent-to-treat analysis for longitudinal Little R, Yau L. Intent-to-treat analysis for longitudinal studies with drop-outs. Biometrics (1996) 52:1324-1333.studies with drop-outs. Biometrics (1996) 52:1324-1333.
NINDS t-PA Stroke Trial Study Group. Tissue Plasminogen NINDS t-PA Stroke Trial Study Group. Tissue Plasminogen Activator for Acute Stroke (1995) 333:1581-1587.Activator for Acute Stroke (1995) 333:1581-1587.
Curb JD, et al. Ascertainment of vital status through the Curb JD, et al. Ascertainment of vital status through the national death index and social security administration. A J national death index and social security administration. A J Epi (1985)121:754-766.Epi (1985)121:754-766.
Multiple Risk Factor Intervention Trial Research Group. Multiple Risk Factor Intervention Trial Research Group. Multiple risk factor intervention trial: risk factor changes and Multiple risk factor intervention trial: risk factor changes and mortality results. JAMA (1982) 248:1466-77.mortality results. JAMA (1982) 248:1466-77.
4646
EXTRA slides not EXTRA slides not presentedpresented
4747
CompletersCompleters
Retain only those patients who Retain only those patients who remain on treatmentremain on treatment
Was used frequently in past in Was used frequently in past in trials in rheumatoid arthritistrials in rheumatoid arthritis
Not intent-to-treatNot intent-to-treat Obvious potential for biasObvious potential for bias
– patients not responding to treatment patients not responding to treatment drop-outdrop-out
4848
Last Observation Last Observation Carried ForwardCarried Forward For those missing a final value, For those missing a final value,
use most recent previous use most recent previous observation.observation.
Potential for bias in disease with Potential for bias in disease with downward coursedownward course
4949
Worst case Worst case
Replace missing values with worst Replace missing values with worst outcomeoutcome– assumes that those who are lost to assumes that those who are lost to
follow-up were not successfully follow-up were not successfully treatedtreated
– generally variance is not inflatedgenerally variance is not inflated– could inflate or deflate differencescould inflate or deflate differences
5050
Best Case/Worst CaseBest Case/Worst Case
Replace missing values in Replace missing values in treatment group by worst outcome treatment group by worst outcome and missing values in comparison and missing values in comparison group with best outcome.group with best outcome.– Rarely usedRarely used– Generally overly conservative as both Generally overly conservative as both
treatment and placebo group drop-out treatment and placebo group drop-out for lack of efficacy.for lack of efficacy.
5151
Missing at RandomMissing at Random
Drop-out at time t does not depend Drop-out at time t does not depend on unobserved outcomes at times on unobserved outcomes at times t’t’>> t, after conditioning on data up t, after conditioning on data up to time t.to time t.
Example:Example:– a patient misses follow-up visit a patient misses follow-up visit
because she is not feeling well (small because she is not feeling well (small TIA’s) then has a major stroke a week TIA’s) then has a major stroke a week later.later.
5252
Missing at randomMissing at random
Ignore missing valuesIgnore missing values In survival analyses, censor at date In survival analyses, censor at date
of last follow-upof last follow-up Use generalized estimating Use generalized estimating
equations equations Difficulties in assessing missing at Difficulties in assessing missing at
randomrandom Rarely is this assumption expectedRarely is this assumption expected
5353
Rubin’s Approach for Rubin’s Approach for Non-ComplianceNon-Compliance Assume assignment to treatment (T) Assume assignment to treatment (T)
or control (C) has no effect on or control (C) has no effect on outcome for non-complying patients.outcome for non-complying patients.
Model compliance status under the Model compliance status under the null hypothesis (no effect on null hypothesis (no effect on outcome)outcome)
Compute average effect of Compute average effect of assignment to T versus C for assignment to T versus C for subset subset of T compliers.of T compliers.
5454
Rubin’s Approach Rubin’s Approach ContinuedContinued Few studies have “pure” non-Few studies have “pure” non-
compliers.compliers. Pure non-compliersPure non-compliers
– those refusing surgery in surgical trialthose refusing surgery in surgical trial– those refusing medication after those refusing medication after
randomizationrandomization If patients take some medication, If patients take some medication,
there may be carryover treatment there may be carryover treatment effectseffects
5555
Little’s Approach to Little’s Approach to ImputationImputation Uses multiple imputation for patients Uses multiple imputation for patients
who are missing information based on who are missing information based on actual dose after drop-out if known or actual dose after drop-out if known or assumption.assumption.
Accounts for uncertainty in parameter Accounts for uncertainty in parameter estimates.estimates.– Model parameters drawn from posterior Model parameters drawn from posterior
distn’, then missing values drawn from distn’, then missing values drawn from predictive distn’ conditional on drawn predictive distn’ conditional on drawn parameters.parameters.
5656
Geller, et alGeller, et al
Raynaud’s Treatment StudyRaynaud’s Treatment Study Model missing values using patient Model missing values using patient
covariates at baseline to identify covariates at baseline to identify similar patient(s) with follow-up similar patient(s) with follow-up (neighbor)(neighbor)
Weights neighbor, sets weight for Weights neighbor, sets weight for missing patient to zero missing patient to zero
(Propensity Score)(Propensity Score)
5757
Sample Size for Sample Size for Composite Favorable Composite Favorable Outcome*Outcome*
Comp.Outcome rt-PA Placebo N/Group
At least 1 0.54 0.41 309
At least 2 0.43 0.32 405
At least 3 0.39 0.27 321
All four 0.27 0.16 289
*Power 90%, = 0.05, two-sided test
5858
LTF Groups And Imputation Methods in WARSS
GroupGroup Sample ReasonSample Reason
for LTF for LTF MethodMethod
1 Endpoint Imminent1 Endpoint Imminent““Terminal ALS” on CRF Terminal ALS” on CRF oror
rapidly worsening rapidly worsening symptoms symptoms
Impute endpointImpute endpoint
at LTFat LTF
2 Cause of LTF2 Cause of LTF
is is independent of independent of timetime
to future endpointto future endpoint
Daughter moves Daughter moves
to Puerto Rico, patient to Puerto Rico, patient moves with hermoves with her
Censor at LTFCensor at LTF
3 Cause of LTF 3 Cause of LTF is is notnot
independent of time independent of time toto
future endpointfuture endpoint
Patient has a seriesPatient has a series
of TIAs, is then LTFof TIAs, is then LTF
Model time to Model time to endpoint (multiple endpoint (multiple
imputation)imputation)
5959
Baseline risk factorsAgeNo College Education Low or High ETOH Consumption Sedentary life style Hx Diabetes Hx Cardiac Disease Hx Diabetes and Hx Stroke Hx Diabetes and Glasgow <5
Among the 12 group 3 patients: Primary endpoints imputed for 2 patients Event-free follow-up imputed for 10 patients
Variables in The Imputation Model
Recommended