Herramientas básicas para un buen diseño Epidemiológico

Oct-2007Oct-2007

[email protected]@uab.es 11

Ferran Torres, MD PhDFerran Torres, MD PhD

Unidad de Soporte en Estadística y Metodología (USEM)Servei de Farmacologia Clínica (UASP). Hospital Clínic

Profesor Bioestadística. Facultat Medicina. UAB

Herramientas básicas Herramientas básicas para un buen diseño para un buen diseño

EpidemiológicoEpidemiológico

Oct-2007Oct-2007 [email protected]@uab.es 22

2 extreme views about 2 extreme views about observational studiesobservational studies

Observational studies aren’t useful. Observational studies aren’t useful. RCTs are the gold standard and the RCTs are the gold standard and the only valid design for “truth”only valid design for “truth”

Observational study evidence trumps Observational study evidence trumps RCT evidence. RCTs are not RCT evidence. RCTs are not applicable to real-world practiceapplicable to real-world practice


Epidemiological studies. Epidemiological studies. TerminologyTerminology

““Observational” studies: Lack investigator Observational” studies: Lack investigator allocation to an interventionallocation to an intervention

Studies not trialsStudies not trials

Include: case series, cross-sectional, case-Include: case series, cross-sectional, case-control & cohort studies, before and after, control & cohort studies, before and after, time-series, database studies, historical time-series, database studies, historical controlscontrols

““Non-randomized studies” is broader termNon-randomized studies” is broader term


NRS vs RCT evidenceNRS vs RCT evidence

– Well-designed cohort or case-control & Well-designed cohort or case-control & RCTs have similar effect sizes (24 clinical RCTs have similar effect sizes (24 clinical topics). topics). Concato et al. Benson et al NEJM 2000;342:1887-92.Concato et al. Benson et al NEJM 2000;342:1887-92.

– ““Strong evidence that quasi-randomized Strong evidence that quasi-randomized trials provided biased effect size trials provided biased effect size estimates of about 30%”--at least for estimates of about 30%”--at least for medical Rx. medical Rx. Cochrane NRSG approach (Cochrane NRSG approach (http://www.cochrane.dk/nrsmghttp://www.cochrane.dk/nrsmg))

– Results of RCTs and NRS sometimes but Results of RCTs and NRS sometimes but not always differ. not always differ. Deeks. HTA report. 2003;Vol 7:no. 27.Deeks. HTA report. 2003;Vol 7:no. 27.


Extending findings from available Extending findings from available RCTs RCTs

Population limitations (homogeneous, limited Population limitations (homogeneous, limited co-morbidities, unstudied vulnerable groups)co-morbidities, unstudied vulnerable groups)

Small sample sizes (adverse events, rare Small sample sizes (adverse events, rare events)events)

Short follow-up (maintenance of benefits, Short follow-up (maintenance of benefits, adverse events)adverse events)

Important outcomes not available (patient Important outcomes not available (patient priorities, long-term effects, natural priorities, long-term effects, natural history/background rate)history/background rate)



Esperanza de vida al nacer

años

0

20

40

60

80

1900 1996

47,3

78,1


Key points in study designKey points in study design

Economical- BudgetEconomical- Budget

Logistic-organizationLogistic-organization

EthicalEthical

ScientificScientific


Errors in researchErrors in research

There are basically 2 types of error in research. There are basically 2 types of error in research.

1.1. One is One is random errorrandom error due to random variation due to random variation in subjects’ responses or measurement. in subjects’ responses or measurement. Inferential statistics (the p value and 95% Inferential statistics (the p value and 95% confidence interval) measure the amount of confidence interval) measure the amount of random error and thus allow us to draw random error and thus allow us to draw conclusion based on our research data.conclusion based on our research data.

2.2. However, there is another type of error, However, there is another type of error, Bias Bias or systematic erroror systematic error..

Oct-2007Oct-2007 [email protected]@uab.es

Types Of Error: Random ErrorTypes Of Error: Random Error

– Larger sample produce less variable Larger sample produce less variable estimate and more likely to reflect the estimate and more likely to reflect the experience of the total populationexperience of the total population

p < 0.05 : 5 %, or 1 in 20, probability of p < 0.05 : 5 %, or 1 in 20, probability of observing a result as extreme as that observed observing a result as extreme as that observed solely by chance solely by chance

BUT, a composite measure affected by both the BUT, a composite measure affected by both the magnitude of the difference between groups magnitude of the difference between groups and the sample sizeand the sample size


Bias Bias - Definition- Definition DDeviations of results (or inferences) from eviations of results (or inferences) from

the truth, or processes leading to such the truth, or processes leading to such deviation. Any trend in the selection of deviation. Any trend in the selection of subjects, data collection, analysis, subjects, data collection, analysis, interpretation, publication or review of interpretation, publication or review of data that can lead to conclusions that are data that can lead to conclusions that are systematically different from the truth. systematically different from the truth.

(Last Dictionary of epidemiology)(Last Dictionary of epidemiology)

Systematic deviation from the truth that Systematic deviation from the truth that distorts the results of research.distorts the results of research.

(Sitthi Lancet 1993)(Sitthi Lancet 1993)


Válidas y precisas

Válidas e imprecisas

No Válidas y precisas

No Válidas e imprecisas

SesgoSesgo

Precisas pero con SESGO

Imprecisas y con SESGO


1.1. Selection biasSelection bias

2.2. Confounding biasConfounding bias

3.3. Measurement bias Measurement bias

4. Information bias 4. Information bias

Bias –ClassificationBias –Classification


Selection bias Selection bias (in entire study group(in entire study group))

Error due to systematic differences in Error due to systematic differences in characteristics between those who are characteristics between those who are selected for study and those who are not. selected for study and those who are not.

(Last Dictionary of Epidemiology)(Last Dictionary of Epidemiology)

The selected sample is not representative of The selected sample is not representative of the universe of which it is a part.the universe of which it is a part.

(Hill Principles of Medical Statistics 1971)(Hill Principles of Medical Statistics 1971)

The control or population experience may The control or population experience may not be representative of the counterfactual not be representative of the counterfactual of the casesof the cases


Types of Selection BiasTypes of Selection Bias Berksonian biasBerksonian bias – – There may be a spurious There may be a spurious

association between diseases or between a association between diseases or between a characteristic and a disease because of the different characteristic and a disease because of the different probabilities of admission to a hospital for those with probabilities of admission to a hospital for those with the disease, without the disease and with the the disease, without the disease and with the characteristic of interestcharacteristic of interest

Berkson J. Limitations of the application of fourfold table analysis to Berkson J. Limitations of the application of fourfold table analysis to hospital data. hospital data. BiometricsBiometrics 1946;2:47-53 1946;2:47-53

Response BiasResponse Bias – – those who agree to be in a study may those who agree to be in a study may

be in some way different from those who refuse to be in some way different from those who refuse to

participateparticipate

– Volunteers may be different from those who are Volunteers may be different from those who are enlistedenlisted


Confounding biasConfounding bias

1.1. Associated with the exposure being studiedAssociated with the exposure being studied– imbalance in the comparison groupsimbalance in the comparison groups

2.2. Independently associated with the diseaseIndependently associated with the disease3.3. Not an effect of the exposureNot an effect of the exposure

Confounding bias arises when the confounder is unequally distributed between the

group with the study risk factor and the control group without the study factor.


Types of Information BiasTypes of Information Bias Interviewer BiasInterviewer Bias – an interviewer’s knowledge may – an interviewer’s knowledge may

influence the structure of questions and the manner of influence the structure of questions and the manner of presentation, which may influence responsespresentation, which may influence responses

Recall BiasRecall Bias – those with a particular outcome or – those with a particular outcome or exposure may remember events more clearly or amplify exposure may remember events more clearly or amplify their recollectionstheir recollections

Observer BiasObserver Bias – observers may have preconceived – observers may have preconceived expectations of what they should find in an examinationexpectations of what they should find in an examination

Loss to follow-upLoss to follow-up – those that are lost to follow-up or – those that are lost to follow-up or who withdraw from the study may be different from who withdraw from the study may be different from those who are followed for the entire studythose who are followed for the entire study


Types of Information BiasTypes of Information Bias Hawthorne effectHawthorne effect – an effect first documented at a – an effect first documented at a

Hawthorne manufacturing plant; people act Hawthorne manufacturing plant; people act differently if they know they are being watcheddifferently if they know they are being watched

Surveillance biasSurveillance bias – the group with the known – the group with the known exposure or outcome may be followed more closely exposure or outcome may be followed more closely or longer than the comparison groupor longer than the comparison group

Misclassification biasMisclassification bias – errors are made in – errors are made in classifying either disease or exposure statusclassifying either disease or exposure status


Misclassification Bias (cont.)Misclassification Bias (cont.)

250250100100150150

10010050505050NonexposedNonexposed1501505050100100ExposedExposed

TotalTotalControlsControlsCasesCases

OR = ad/bc = 2.0; RR = a/(a+b)/c/(c+d) = 1.3

True Classification

250250100100150150

909050504040NonexposedNonexposed

1601605050110110ExposedExposed

TotalTotalControlsControlsCasesCases

OR = ad/bc = 2.8; RR = a/(a+b)/c/(c+d) = 1.6

Differential misclassification - Overestimate exposure for 10 cases, inflate rates



CasesCases ControlsControls TotalTotalExposedExposed 100100 5050 150150NonexposeNonexpose

dd5050 5050 100100

150150 100100 250250OR = ad/bc = 2.0; RR = a/(a+b)/c/(c+d) = 1.3

True Classification

CasesCases ControlsControls TotalTotalExposedExposed 9090 5050 140140NonexposedNonexposed 6060 5050 110110

150150 100100 250250OR = ad/bc = 1.5; RR = a/(a+b)/c/(c+d) = 1.2

Differential misclassification - Underestimate exposure for 10 cases, deflate rates



CasesCases ControlsControls TotalTotalExposedExposed 100100 5050 150150NonexposeNonexpose

dd5050 5050 100100

150150 100100 250250OR = ad/bc = 2.0; RR = a/(a+b)/c/(c+d) = 1.3

True Classification

CasesCases ControlsControls TotalTotal

ExposedExposed 110110 6060 170170

NonexposNonexposeded

4040 4040 8080

150150 100100 250250OR = ad/bc = 1.8; RR = a/(a+b)/c/(c+d) = 1.3

Nondifferential misclassification - Overestimate exposure in 10 cases, 10 controls – bias towards null


• Avoiding bias: DesignAvoiding bias: Design

• Correcting bias: AnalysisCorrecting bias: Analysis• Estimating magnitude and Estimating magnitude and

direction of bias: Sensitivity direction of bias: Sensitivity analysis of Biasanalysis of Bias

Dealing with Dealing with biasbias

Oct-2007Oct-2007

[email protected]@uab.es 2424

Study designStudy design–SamplingSampling–Sample SizeSample Size–Sources of data collectionSources of data collection–Methods of data collectionMethods of data collection–Content of informationContent of information–Statistical Analysis PlanStatistical Analysis Plan

Prevention of Bias (?)Prevention of Bias (?)


Avoiding bias: Avoiding bias: DesignDesign Standard source of informationStandard source of information

– More than one source: Multiple standard sources to confirm More than one source: Multiple standard sources to confirm informationinformation

Methods to assure participation and compliance and Methods to assure participation and compliance and ffollow-upollow-up

Strategy to maximise participation rate (response, Strategy to maximise participation rate (response, consent), and to maximise complete follow upconsent), and to maximise complete follow up

Defining study population: Defining study population: – population based study less vulnerablepopulation based study less vulnerable

– Define, a priori, who is a case or what constitutes exposure so that there Define, a priori, who is a case or what constitutes exposure so that there is no overlapis no overlap

Define categories within groups clearly (age groups, aggregates of Define categories within groups clearly (age groups, aggregates of person years)person years)

“ Prevention is

better than cure”


Avoiding bias: Avoiding bias: DesignDesign

Well defined populationWell defined population• In Cohort studies, the population should be chosen In Cohort studies, the population should be chosen

independent of the risk of disease in question.independent of the risk of disease in question.• In Case control studies, the selection of the controls In Case control studies, the selection of the controls

should be independent of the exposure in questionshould be independent of the exposure in question

Set up strict guidelines for data collectionSet up strict guidelines for data collection TrainTrain observers or interviewers to obtain data in the same fashion observers or interviewers to obtain data in the same fashion It is It is preferablepreferable to use to use more than one observer or interviewermore than one observer or interviewer, but , but

not so many that they cannot be trained in an identical mannernot so many that they cannot be trained in an identical manner

Use of a good control groupUse of a good control group


Be purposeful in the study design to minimize the Be purposeful in the study design to minimize the chance for bias, ex. chance for bias, ex. more than one control groupmore than one control group

Selection of control in case control study : to equalise incentive Selection of control in case control study : to equalise incentive or motivation to recall, use a third control arm that has similar or motivation to recall, use a third control arm that has similar disease but not disease under study. Example, congenital disease but not disease under study. Example, congenital abnormality study, case mothers, normal control mothers, a abnormality study, case mothers, normal control mothers, a third group of other abnormalitythird group of other abnormality

Sampling : probability sampling required to ensure Sampling : probability sampling required to ensure representative sample representative sample external validity external validity

Experimental design for RCT : parallel groups design Experimental design for RCT : parallel groups design best. Others for example self-controlled design, best. Others for example self-controlled design, historical control etc prone to biaseshistorical control etc prone to biases


Avoiding bias - DesignAvoiding bias - Design

Randomisation: Randomisation: – random allocation to comparison groups to random allocation to comparison groups to

avoid selection bias by investigators as well as avoid selection bias by investigators as well as to minimise confounding bias. to minimise confounding bias.

Randomly allocate observers/interviewer data collection Randomly allocate observers/interviewer data collection assignmentsassignments

Matching on important confoundersMatching on important confounders

Blinding of subjects, investigators and / or Blinding of subjects, investigators and / or statisticianstatistician


Restriction of subjects to obtain Restriction of subjects to obtain homogenous group. homogenous group.

Quality control procedures in data Quality control procedures in data collectioncollection

……all detailed in advance in a all detailed in advance in a written protocolwritten protocol

Avoiding bias - DesignAvoiding bias - Design


Current GuidelineCurrent GuidelineInitiatives…Initiatives…

Observational & Non-randomized Observational & Non-randomized StudiesStudies– STROBE: STROBE: www.strobe-www.strobe-

statement.orgstatement.org– TREND: TREND: www.trend-statement.orgwww.trend-statement.org

Randomized Clinical TrialsRandomized Clinical Trials – CONSORT: CONSORT: www.consort-statement.orgwww.consort-statement.org


Med Clin (Barc) Dic-2005. Vol 125, Med Clin (Barc) Dic-2005. Vol 125, Supl.1Supl.1

Estudios epidemiológicos (Estudios epidemiológicos (STROBESTROBE). ).

Metaanálisis (Metaanálisis (QUOROM, MOOSEQUOROM, MOOSE). ). Estudios de intervención no aleatorizados Estudios de intervención no aleatorizados

((TRENDTREND). ). Estudios de precisión diagnóstica (Estudios de precisión diagnóstica (STARDSTARD) y ) y

pronóstica (pronóstica (REMARKREMARK). ). Otros:Otros:

– Recomendaciones metodológicas de las Recomendaciones metodológicas de las agencias agencias reguladorasreguladoras. .

– Instrumentos de medida de Instrumentos de medida de calidad de vida calidad de vida relacionada con la saludrelacionada con la salud y de otros resultados y de otros resultados percibidos por los pacientes. percibidos por los pacientes.

– Estudios de Estudios de evaluación económicaevaluación económica en salud. en salud. – Ensayos clínicos aleatorizados (Ensayos clínicos aleatorizados (CONSORTCONSORT).).– Ensayos clínicos aleatorizados comunitarios Ensayos clínicos aleatorizados comunitarios

((CONSORTCONSORT CLUSTERCLUSTER). ).


ICHE9ICHE9 Statistical PrinciplesStatistical Principles for Clinical Trials for Clinical Trials CPMP/EWP/908/99CPMP/EWP/908/99 CPMP Points to Consider on CPMP Points to Consider on MultiplicityMultiplicity issues issues

in Clinical Trials in Clinical Trials (Apr 2003)(Apr 2003)

CPMP/EWP/2863/99CPMP/EWP/2863/99 Points to Consider on Adjustment for Points to Consider on Adjustment for BaselineBaseline CovariatesCovariates (Nov 2003)(Nov 2003)

CPMP/2330/99CPMP/2330/99 Points to Consider on Application with 1.) Points to Consider on Application with 1.) Meta-Meta-analysesanalyses and 2.) One Pivotal study and 2.) One Pivotal study (May 2001)(May 2001)

CPMP/EWP/2158/99CPMP/EWP/2158/99 Guideline on the Choice of a Guideline on the Choice of a Non-Inferiority Non-Inferiority MarginMargin (Jan2006)(Jan2006)

CPMP/EWP/482/99CPMP/EWP/482/99 Points to Consider on Points to Consider on SwitchingSwitching between between Superiority and Non-inferiority Superiority and Non-inferiority (Feb 2001)(Feb 2001)

CPMP/EWP/1776/99CPMP/EWP/1776/99 Points to Consider on Points to Consider on Missing DataMissing Data (Jan 2002)(Jan 2002)

CHMP/EWP/83561/05CHMP/EWP/83561/05 Guideline on Clinical Trials in Guideline on Clinical Trials in Small Small PopulationsPopulations (Feb2007)(Feb2007)

CHMP/EWP/2459/02CHMP/EWP/2459/02 Reflection Paper on Methodological Issues Reflection Paper on Methodological Issues in Confirmatory Clinical Trials with in Confirmatory Clinical Trials with Flexible DesignFlexible Design and and Analysis Plan Analysis Plan (Draft)(Draft)

NormativasNormativas


Population of Oldenburg, Germany, 1930-1936(Ornithologische Monatsberichte 44, Jahrgang, 1936, Berlin)

Storks (1000s)

Humans(1000s)


http://ferran.torres.name/docencia/hcphttp://ferran.torres.name/docencia/hcp

Do Storks Bring Do Storks Bring Babies?Babies?

Documents

Herramientas básicas para un buen diseño Epidemiológico