52
Sample Size Sample Size Determination Determination

Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Embed Size (px)

Citation preview

Page 1: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Sample Size Sample Size DeterminationDetermination

Page 2: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

IntroductionIntroduction Integral part of vast majority of Integral part of vast majority of

quantitative studiesquantitative studies Important in ensuring validity, accuracy, Important in ensuring validity, accuracy,

reliability & scientific & ethical integrityreliability & scientific & ethical integrity Don’t give in to temptations of taking a Don’t give in to temptations of taking a

shortcutshortcut Highly recommended to ask a Highly recommended to ask a

professional statistician to conduct the professional statistician to conduct the sample size calculation (they may even sample size calculation (they may even show you methods to decrease your show you methods to decrease your sample size!)sample size!)

Page 3: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

IntroductionIntroduction

Freiman JA, Freiman JA, NEJM, NEJM, 1978;299:690-41978;299:690-4 Reviewed the power of 71 published RCTs Reviewed the power of 71 published RCTs

which had failed to detect a differencewhich had failed to detect a difference Found that 67 could have missed a 25% Found that 67 could have missed a 25%

therapeutic improvementtherapeutic improvement 50 could have missed a 50% improvement50 could have missed a 50% improvement

Page 4: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

IntroductionIntroduction

Three main parts in its calculationThree main parts in its calculation Estimation (depends on a host of items)Estimation (depends on a host of items) Justification (in the light of budgetary or Justification (in the light of budgetary or

biological considerations)biological considerations) Adjustments (accounting for potential Adjustments (accounting for potential

dropouts or effect of covariates)dropouts or effect of covariates)

Page 5: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

IntroductionIntroduction Consequences of getting it wrongConsequences of getting it wrong

ScientificScientific EthicalEthical EconomicalEconomical

Problems can be approached in two Problems can be approached in two waysways Patients I need Patients I need approach: based on approach: based on

calculations of sample size for a given calculations of sample size for a given power, significance, and clinically power, significance, and clinically meaningful differencemeaningful difference

Patients I can get Patients I can get approach: based on approach: based on calculations of power for a given sample calculations of power for a given sample size & level of significancesize & level of significance

Page 6: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Pilot StudiesPilot Studies

It is a preliminary study intended to test It is a preliminary study intended to test the feasibility of a larger study, data the feasibility of a larger study, data collection methods, collect information for collection methods, collect information for sample size calculationssample size calculations

It should not be regarded as a study which It should not be regarded as a study which is too small to produce a definitive answeris too small to produce a definitive answer

It should be regarded as a tool in finding It should be regarded as a tool in finding the answer as long as it is followed throughthe answer as long as it is followed through

Sample size calculations may not be Sample size calculations may not be requiredrequired

Page 7: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Importance of Sample Size Importance of Sample Size calculationcalculation

Scientific reasonsScientific reasons Ethical reasonsEthical reasons Economic reasonsEconomic reasons

Page 8: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Scientific ReasonsScientific Reasons

In a trial with In a trial with negativenegative results and a results and a sufficient sample sizesufficient sample size, the result is , the result is concreteconcrete

In a trial with In a trial with negativenegative results and results and insufficient power (insufficient sample insufficient power (insufficient sample size)size), may mistakenly conclude that the , may mistakenly conclude that the treatment under study made no differencetreatment under study made no difference

Page 9: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Ethical ReasonsEthical Reasons

An An undersized undersized study can expose subjects study can expose subjects to potentially harmful treatments without to potentially harmful treatments without the capability to advance knowledgethe capability to advance knowledge

An An oversizedoversized study has the potential to study has the potential to expose an unnecessarily large number of expose an unnecessarily large number of subjects to potentially harmful subjects to potentially harmful treatmentstreatments

Page 10: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Economic ReasonsEconomic Reasons

Undersized study Undersized study is a waste of resources is a waste of resources due to its inability to yield useful resultsdue to its inability to yield useful results

Oversized studyOversized study may result in may result in statistically significant result with statistically significant result with doubtful clinical importance leading to doubtful clinical importance leading to waste of resources (Cardiac Studies)waste of resources (Cardiac Studies)

Page 11: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Classic Approaches to Classic Approaches to Sample Size CalculationSample Size Calculation

Precision analysisPrecision analysis BayesianBayesian FrequentistFrequentist

Power analysisPower analysis Most commonMost common

Page 12: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Precision AnalysisPrecision Analysis

In studies concerned with estimating In studies concerned with estimating some parametersome parameter PrecisionPrecision AccuracyAccuracy prevalenceprevalence

Page 13: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Power AnalysisPower Analysis

In studies concerned with detecting an effectIn studies concerned with detecting an effect Important to ensure that if an effect deemed Important to ensure that if an effect deemed

to be clinically meaningful exists, then there to be clinically meaningful exists, then there is a high chance of it being detectedis a high chance of it being detected

Page 14: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Factors That Influence Factors That Influence Sample Size CalculationsSample Size Calculations

The objective (precision, power analysis)The objective (precision, power analysis) Details of intervention & control Rx.Details of intervention & control Rx. The outcomesThe outcomes

Categorical or continuousCategorical or continuous Single or multipleSingle or multiple PrimaryPrimary SecondarySecondary Clinical relevance of the outcomeClinical relevance of the outcome Any missed dataAny missed data Any surrogate outcomesAny surrogate outcomes

WhyWhy Will they accurately reflect the main outcomeWill they accurately reflect the main outcome

Page 15: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Factors That Influence Factors That Influence Sample Size CalculationsSample Size Calculations

Any covariates to controlAny covariates to control The unit of randomizationThe unit of randomization

IndividualsIndividuals Family practicesFamily practices Hospital wardsHospital wards CommunitiesCommunities FamiliesFamilies EtcEtc

The unit of analysisThe unit of analysis Same as aboveSame as above

Page 16: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Factors That Influence Factors That Influence Sample Size CalculationsSample Size Calculations

The research designThe research design Simple RCTSimple RCT Cluster RCTCluster RCT EquivalenceEquivalence Non-randomized intervention studyNon-randomized intervention study Observational studyObservational study Prevalence studyPrevalence study A study measuring sensitivity & specificityA study measuring sensitivity & specificity A paired comparisonA paired comparison Repeated-measures studyRepeated-measures study Are the groups equalAre the groups equal

Page 17: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Factors That Influence Factors That Influence Sample Size CalculationsSample Size Calculations

Research subjectsResearch subjects Target populationTarget population Inclusion & exclusion criteriaInclusion & exclusion criteria Baseline riskBaseline risk Pt. compliance ratePt. compliance rate Pt. drop-out ratePt. drop-out rate

Page 18: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Factors That Influence Sample Size Factors That Influence Sample Size CalculationsCalculations

Is the F/U long enough to be of any clinical relevanceIs the F/U long enough to be of any clinical relevance Desired level of significanceDesired level of significance Desired powerDesired power One or two-tailed testOne or two-tailed test Any explanation for the possible ranges or variations in Any explanation for the possible ranges or variations in

outcome that is expectedoutcome that is expected The smallest differenceThe smallest difference

Smallest clinically important differenceSmallest clinically important difference The difference that investigators think is worth detectingThe difference that investigators think is worth detecting The difference that investigators think is likely to be detectedThe difference that investigators think is likely to be detected Would an increase or decrease in the effect size make a sig. Would an increase or decrease in the effect size make a sig.

clinical differenceclinical difference Justification of previous dataJustification of previous data

Published dataPublished data Previous workPrevious work Review of recordsReview of records Expert opinionExpert opinion

Software or formula being usedSoftware or formula being used

Page 19: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Statistical TermsStatistical Terms

The numerical value summarizing The numerical value summarizing the difference of interest (effect)the difference of interest (effect) Odds Ratio (OR)Odds Ratio (OR) Null, OR=1Null, OR=1 Relative Risk (RR)Relative Risk (RR) Null, RR=1Null, RR=1 Risk Difference (RD)Risk Difference (RD) Null, RD=0Null, RD=0 Difference Between MeansDifference Between Means Null, Null,

DBM=0DBM=0 Correlation CoefficientCorrelation Coefficient Null, CC=0Null, CC=0

Page 20: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Statistical TermsStatistical Terms P-value:P-value: Probability of obtaining an Probability of obtaining an

effect as extreme or more extreme than effect as extreme or more extreme than what is observed by chancewhat is observed by chance

Significance level of a test:Significance level of a test: cut-off point cut-off point for the p-value (conventionally it is 5%)for the p-value (conventionally it is 5%)

Power of a test:Power of a test: correctly reject the correctly reject the null hypothesis when there is indeed a null hypothesis when there is indeed a real difference or association (typically real difference or association (typically set at least 80%)set at least 80%)

Effect size of clinical importanceEffect size of clinical importance

Page 21: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Statistical TermsStatistical Terms One sided & Two sided tests of significanceOne sided & Two sided tests of significance

Two-sided testTwo-sided test Alternative hypothesis suggests that a difference exists Alternative hypothesis suggests that a difference exists

in either directionin either direction Should be used unless there is a very good reason for Should be used unless there is a very good reason for

doing otherwisedoing otherwise One-sided testOne-sided test

when it is completely inconceivable that the result when it is completely inconceivable that the result could go in either direction, or the only concern is in could go in either direction, or the only concern is in one directionone direction

Toxicity studiesToxicity studies Safety evaluationSafety evaluation Adverse drug reactionsAdverse drug reactions Risk analysisRisk analysis

The expectation of the result is not adequate The expectation of the result is not adequate justification for one-sided testjustification for one-sided test

Page 22: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

ApproachApproach Specify a hypothesisSpecify a hypothesis Specify the significance level alphaSpecify the significance level alpha Specify an effect sizeSpecify an effect size Obtain historical valuesObtain historical values Specify a powerSpecify a power Use the appropriate formula to calculate Use the appropriate formula to calculate

sample sizesample size

After the study is finished compare the After the study is finished compare the variance of actual data with the one used in variance of actual data with the one used in sample size calculationssample size calculations

Page 23: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

FormulaeFormulae

Page 24: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Sample Size AdjustmentsSample Size Adjustments

Separate sample size calculation Separate sample size calculation should be done for each important should be done for each important outcome & then use the max. outcome & then use the max. estimateestimate

When two variables are correlated When two variables are correlated with a factor with a factor pp, then sample size can , then sample size can be reduced by a factor of 1-be reduced by a factor of 1-pp22

Another option is to use Bonferroni Another option is to use Bonferroni correction for multiple outcomescorrection for multiple outcomes

Page 25: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Sample Size AdjustmentsSample Size Adjustments

Allowing for response rates & other Allowing for response rates & other losses to the samplelosses to the sample The expected response rateThe expected response rate Loss to f/uLoss to f/u Lack of complianceLack of compliance Other lossesOther losses

nnnewnew=n/(1-L)=n/(1-L) when when LL is the is the

loss to f/u rateloss to f/u rate

Page 26: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Sample Size AdjustmentsSample Size Adjustments

Adjustment for unequal group sizeAdjustment for unequal group size Assuming Assuming nn11/n/n22=k=k Calculate Calculate nn assuming equal assuming equal ThenThen

nn22=0.5n(1+1/k)=0.5n(1+1/k) &&

nn11=0.5n(1+k)=0.5n(1+k)

Page 27: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Reporting Sample Size Reporting Sample Size CalculationsCalculations

Clear statement of the primary objectiveClear statement of the primary objective The desired level of significanceThe desired level of significance The desired powerThe desired power The statistics that will be used for analysisThe statistics that will be used for analysis Whether the test would be one or two-tailedWhether the test would be one or two-tailed The smallest differenceThe smallest difference

Smallest clinically important differenceSmallest clinically important difference The difference that investigators think is worth The difference that investigators think is worth

detectingdetecting The difference that the investigators think is The difference that the investigators think is

likely to be detectedlikely to be detected

Page 28: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Reporting Sample Size Reporting Sample Size CalculationsCalculations

Justification for prior estimates used in calculationsJustification for prior estimates used in calculations Clear statements about the assumptions made about Clear statements about the assumptions made about

the distribution or variability of the outcomesthe distribution or variability of the outcomes Clear statement about the scheduled duration of the Clear statement about the scheduled duration of the

studystudy Statement about how the sample size was adjustedStatement about how the sample size was adjusted The software or formulae that was usedThe software or formulae that was used Take the reporting seriously as your documentation Take the reporting seriously as your documentation

may be used in the future for sample size may be used in the future for sample size calculationscalculations

Page 29: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Example: Comparing Two Example: Comparing Two Means Means

Scenario: A randomized controlled trial has been Scenario: A randomized controlled trial has been planned to evaluate a brief psychological planned to evaluate a brief psychological intervention in comparison to usual treatment intervention in comparison to usual treatment in the reduction of suicidal ideation amongst in the reduction of suicidal ideation amongst patients presenting at hospital with deliberate patients presenting at hospital with deliberate self-poisoning. Suicidal ideation will be self-poisoning. Suicidal ideation will be measured on the Beck scale; the standard measured on the Beck scale; the standard deviation of this scale in a previous study was deviation of this scale in a previous study was 7.7, and a difference of 5 points is considered 7.7, and a difference of 5 points is considered to be of clinical importance. It is anticipated to be of clinical importance. It is anticipated that around one third of patients may drop out that around one third of patients may drop out of treatment of treatment

Page 30: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Example: Comparing Two Example: Comparing Two MeansMeans

Required information Required information Primary outcome variable = The Beck Primary outcome variable = The Beck

scale for suicidal ideation. A continuous scale for suicidal ideation. A continuous variable summarized by means. variable summarized by means.

Standard deviation = 7.7 points Standard deviation = 7.7 points Size of difference of clinical importance Size of difference of clinical importance

= 5 points = 5 points Significance level = 5% Significance level = 5% Power = 80% Power = 80% Type of test = two-sidedType of test = two-sided

Page 31: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Example: Comparing Two Example: Comparing Two MeansMeans

The formula for the sample size for The formula for the sample size for comparison of 2 means (2-sided) is as comparison of 2 means (2-sided) is as follows follows nn = [ = [AA + + BB]2 x 2 x ]2 x 2 x SDSD2 / 2 / DIFFDIFF22 where where nn = the sample size required in each = the sample size required in each

group (double this for total sample). group (double this for total sample). SDSD = standard deviation, of the primary = standard deviation, of the primary

outcome variable - here 7.7. outcome variable - here 7.7. DIFFDIFF = size of difference of clinical = size of difference of clinical

importance - here 5.0.importance - here 5.0.

Page 32: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Example: Comparing Two Example: Comparing Two MeansMeans

AA depends on desired significance level (see table) - depends on desired significance level (see table) - here 1.96. here 1.96.

BB depends on desired power (see table) - here 1.28. depends on desired power (see table) - here 1.28. Table of values for A and B Table of values for A and B Significance level A 5% Significance level A 5%

1.961.96, 1% 2.58 , 1% 2.58 Power BPower B 80% 0.84, 80% 0.84, 90% 1.2890% 1.28, 95% 1.64 , 95% 1.64 Inserting the required information into the formula Inserting the required information into the formula gives: gives:

nn = [1.96 + 0.84]2 x 2 x 7.72 / 5.02 = = [1.96 + 0.84]2 x 2 x 7.72 / 5.02 = 3838 This gives the number required in each of the trial's This gives the number required in each of the trial's

two groups. Therefore the total sample size is double two groups. Therefore the total sample size is double this, i.e. this, i.e. 7676. .

To allow for the predicted dropout rate of around one To allow for the predicted dropout rate of around one third, the sample size was increased to 60 in each third, the sample size was increased to 60 in each

group, a group, a total sample oftotal sample of 120120. .

Page 33: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Example: Comparing Two Example: Comparing Two MeansMeans

Suggested description of this sample size Suggested description of this sample size calculation calculation

""A sample size of 38 in each group will be A sample size of 38 in each group will be sufficient to detect a difference of 5 points on sufficient to detect a difference of 5 points on the Beck scale of suicidal ideation, assuming a the Beck scale of suicidal ideation, assuming a standard deviation of 7.7 points, a power of standard deviation of 7.7 points, a power of 80%, and a significance level of 5%. This 80%, and a significance level of 5%. This number has been increased to 60 per group number has been increased to 60 per group (total of 120), to allow for a predicted drop-(total of 120), to allow for a predicted drop-out from treatment of around one thirdout from treatment of around one third" "

Page 34: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Inappropriate Wording or Inappropriate Wording or ReportingReporting

““A previous study in this area recruited 150 A previous study in this area recruited 150 subjects & found highly sign. Results”subjects & found highly sign. Results” Previous study may have been luckyPrevious study may have been lucky

““Sample sizes are not provided because there Sample sizes are not provided because there is no prior information on which to base them”is no prior information on which to base them” Do a pilot studyDo a pilot study Standard Deviation could be estimated from rangeStandard Deviation could be estimated from range

SD=(max-min)/4SD=(max-min)/4 Number decided based on available pts aloneNumber decided based on available pts alone

Extend the lengthExtend the length Consider a multi-center studyConsider a multi-center study

Page 35: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Failure to Achieve Required Failure to Achieve Required Sample SizeSample Size

Pt. refusal to consentPt. refusal to consent Bad time of the study (heavy clinic Bad time of the study (heavy clinic

study in the winter)study in the winter) Adverse media publicityAdverse media publicity Weak recruiting staffWeak recruiting staff Lack of genuine commitment to the Lack of genuine commitment to the

projectproject Lack of staffing in wards or unitsLack of staffing in wards or units Too many projects attempting to Too many projects attempting to

recruit the same subjectsrecruit the same subjects

Page 36: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Possible SolutionsPossible Solutions

Pilot studiesPilot studies Have a plan to regularly monitor Have a plan to regularly monitor

recruitment or create recruitment targetsrecruitment or create recruitment targets Ask for extension in time and/or fundingAsk for extension in time and/or funding Review your staffs commitment to other Review your staffs commitment to other

ongoing trials or other distractersongoing trials or other distracters Regular visits to trial sitesRegular visits to trial sites

Page 37: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Strategies For Maximizing Strategies For Maximizing Power & Minimizing the Power & Minimizing the

Sample SizeSample Size Use common outcomes (the power is Use common outcomes (the power is

driven more by the number of events driven more by the number of events than the total sample size)than the total sample size)

Use paired design (such as cross-over Use paired design (such as cross-over trial)trial)

Use continuous variablesUse continuous variables Choose the timing of the assessments Choose the timing of the assessments

of primary outcomes to be when the of primary outcomes to be when the difference is most likely to be optimaldifference is most likely to be optimal

Page 38: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Recalculation of Sample Recalculation of Sample Size Mid-TrialSize Mid-Trial

Two main reasonsTwo main reasons Changing input factors Changing input factors

Changes in the anticipated control group Changes in the anticipated control group outcomeoutcome

Changes in the anticipated treatment compliance Changes in the anticipated treatment compliance raterate

Changing opinions regarding min. clinically Changing opinions regarding min. clinically important difference (MCID)important difference (MCID)

Increasing accrual ratesIncreasing accrual rates Increasing the sample size to increase the power Increasing the sample size to increase the power

to detect the same MCIDto detect the same MCID Increasing the sample size to allow smaller Increasing the sample size to allow smaller

differences to be detecteddifferences to be detected

Page 39: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Retrospective Sample Size Retrospective Sample Size CalculationsCalculations

ControversialControversial Most recommend to avoid it as it Most recommend to avoid it as it

really doesn’t add more information really doesn’t add more information in most cases and may confuse or in most cases and may confuse or misguide the conclusionmisguide the conclusion

Page 40: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

General Rules of ThumbGeneral Rules of Thumb Don’t forget multiplicity testing corrections Don’t forget multiplicity testing corrections

(Bonferroni)(Bonferroni) Overlapping confidence intervals do not Overlapping confidence intervals do not

imply non-significance (up to 1/3 can overlap imply non-significance (up to 1/3 can overlap even when significant)even when significant)

Use the same statistics for both sample size Use the same statistics for both sample size calculation and your analysis (superiority, calculation and your analysis (superiority, equality, etc)equality, etc) Otherwise you may alter the anticipated powerOtherwise you may alter the anticipated power

Usually better to adopt a simple approachUsually better to adopt a simple approach Better to be conservative (assume two-sided)Better to be conservative (assume two-sided)

Page 41: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

General Rules of ThumbGeneral Rules of Thumb

Remember that sample size calculation Remember that sample size calculation gives you the minimum you requiregives you the minimum you require

If the outcome of interest is “change”, If the outcome of interest is “change”, then use the standard deviation (SD) of then use the standard deviation (SD) of the change and not each individual the change and not each individual outcomeoutcome

Page 42: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

General Rules of ThumbGeneral Rules of Thumb Non RCTs generally require a much larger Non RCTs generally require a much larger

sample to allow adjustment for confounding sample to allow adjustment for confounding factors in the analysisfactors in the analysis

Equivalence studies need a larger sample size Equivalence studies need a larger sample size than studies aimed to demonstrate a differencethan studies aimed to demonstrate a difference

For moderate to large effect size (0.5<effect For moderate to large effect size (0.5<effect size<0.8), 30 subjects per groupsize<0.8), 30 subjects per group

For comparison between 3 or more groups, to For comparison between 3 or more groups, to detect a moderate effect size of 0.5 with 80% detect a moderate effect size of 0.5 with 80% power, will require 14 subjects/grouppower, will require 14 subjects/group

Use Use sensitivity analysis sensitivity analysis to create a sample size to create a sample size table for different power, significance, or effect table for different power, significance, or effect size and then sit and ponder over it for the size and then sit and ponder over it for the optimal sample sizeoptimal sample size

Page 43: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Rules of Thumb for Rules of Thumb for AssociationsAssociations

Multiple RegressionMultiple Regression Minimal requirement is a ratio of 5:1 for Minimal requirement is a ratio of 5:1 for

number of subjects to independent variablesnumber of subjects to independent variables The desired ratio is 15:1The desired ratio is 15:1

Multiple CorrelationsMultiple Correlations For 5 or less predictors (m) use For 5 or less predictors (m) use

n>50 + 8mn>50 + 8m For 6 or more use 10 subjects per predictorFor 6 or more use 10 subjects per predictor

Logistic RegressionLogistic Regression For stable models use 10-15 events per For stable models use 10-15 events per

predictor variablepredictor variable

Page 44: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Rules of Thumb for Rules of Thumb for AssociationsAssociations

Large samples are neededLarge samples are needed Non-normal distributionNon-normal distribution Small effect sizeSmall effect size Substantial measurement errorSubstantial measurement error Stepwise regression is usedStepwise regression is used

For chi-squared testing (two-by-two For chi-squared testing (two-by-two table)table) Enough sample size so that no cell has less Enough sample size so that no cell has less

than 5than 5 Overall sample size should be at least 20Overall sample size should be at least 20

Page 45: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Rules of Thumb for Rules of Thumb for AssociationsAssociations

Factor analysisFactor analysis At least 50 participants/subjects per At least 50 participants/subjects per

variablevariable Minimum 300Minimum 300

N=50 very poorN=50 very poor N=100 poorN=100 poor N=200 fairN=200 fair N=300 goodN=300 good N=500 very goodN=500 very good

Page 46: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Sample Sizes for Time-to-Sample Sizes for Time-to-Event StudiesEvent Studies

Most software require the use of event-Most software require the use of event-free rates (survival) and not event rates free rates (survival) and not event rates (death), because the log rank test is based (death), because the log rank test is based on event-free rateson event-free rates

Beware if your software is giving you total Beware if your software is giving you total number of subjects or events.number of subjects or events.

Page 47: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Sample Size for Cluster Sample Size for Cluster RCTRCT Clusters or units are randomizedClusters or units are randomized

ReasonsReasons LogisticalLogistical

Administrative convenience-easier than Administrative convenience-easier than individual pt. recruitment or randomization individual pt. recruitment or randomization

EthicalEthical Hard to randomize part of a family or Hard to randomize part of a family or

communitycommunity ScientificScientific

Worry about Worry about treatment contaminationtreatment contamination-changing -changing behavior or knowledge during the trialbehavior or knowledge during the trial

Plan for cluster level intervention-family Plan for cluster level intervention-family physician or hospital unitsphysician or hospital units

Cluster action for an intervention-communitiesCluster action for an intervention-communities

Page 48: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

There are Many Other There are Many Other Types of StudiesTypes of Studies

Specialized sample size calculationsSpecialized sample size calculations Cross-over designCross-over design

Needs half as many as an RCTNeeds half as many as an RCT There should be no carry-over effect of Rx.There should be no carry-over effect of Rx. Most suited for chronic conditions (pain, Most suited for chronic conditions (pain,

insomnia), not acute (death)insomnia), not acute (death) Analysis of change from baselineAnalysis of change from baseline Comparisons of means for two Poisson Comparisons of means for two Poisson

populationpopulation Testing for a Single Correlation CoefficientTesting for a Single Correlation Coefficient Comparing Correlation Coefficients for Comparing Correlation Coefficients for

Two Independent SamplesTwo Independent Samples

Page 49: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

TransformationsTransformations Most of the statistical testing is based on a Most of the statistical testing is based on a

Normal distributionNormal distribution Quite often the assumed distribution may not Quite often the assumed distribution may not

fit the datafit the data Duration of symptomsDuration of symptoms CostCost

Changing the scale of the original data Changing the scale of the original data (transforming) and assuming the distribution (transforming) and assuming the distribution for the transformed data may provide a for the transformed data may provide a solutionsolution

Log-transformation may normalize the Log-transformation may normalize the distribution leading to a log-normal distribution leading to a log-normal distribution to work withdistribution to work with

Page 50: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Non-parametric TestsNon-parametric Tests Non-parametric (also called Non-parametric (also called distribution freedistribution free) )

methods are designed to avoid distributional methods are designed to avoid distributional assumptionsassumptions

AdvantagesAdvantages Fewer assumptions are requiredFewer assumptions are required Only nominal (categorical data) or ordinal (ranked) Only nominal (categorical data) or ordinal (ranked)

are required, rather than numerical (interval) dataare required, rather than numerical (interval) data DisadvantagesDisadvantages

Less efficientLess efficient Less powerfulLess powerful Overestimates varianceOverestimates variance

Do not lend themselves easily to sample size Do not lend themselves easily to sample size calculations and CIcalculations and CI

Interpretation of the results is difficultInterpretation of the results is difficult Most software don’t do themMost software don’t do them

Page 51: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Software for Sample Size Software for Sample Size CalculationsCalculations

nQuery Advisor 2000nQuery Advisor 2000 Power and Precision 1997Power and Precision 1997 Pass 2000Pass 2000 UnifyPow 1998UnifyPow 1998

Page 52: Sample Size Determination. Introduction Integral part of vast majority of quantitative studies Integral part of vast majority of quantitative studies

Freeware on The Web (User Freeware on The Web (User beware)beware)