Evidence-Based Evidence-Based Medicine 4Medicine 4

More Knowledge and Skills More Knowledge and Skills for Critical Readingfor Critical Reading

Karen E. Schetzina, MD, MPH

Review: Relative RiskReview: Relative Risk

The ratio of the risk of disease or The ratio of the risk of disease or death among the exposed to the risk death among the exposed to the risk among the unexposed; same as risk among the unexposed; same as risk ratio.ratio. Interpretation:Interpretation:

>1 suggests positive association>1 suggests positive association <1 suggests negative association<1 suggests negative association =1 suggests no difference between groups=1 suggests no difference between groups

Review: Relative RiskReview: Relative Risk D+ D-D+ D-

E+E+

E-E-

RR = RR = Risk of disease for E+Risk of disease for E+ = = A/(A + A/(A + B)B)

Risk of disease for E- C/(C + D)Risk of disease for E- C/(C + D)

A B

C D

Review: Odds RatioReview: Odds Ratio Odds Ratio – The ratio of two odds. Odds Ratio – The ratio of two odds.

For For rare diseasesrare diseases, this , this approximates relative risk. approximates relative risk. Commonly calculated in cross-Commonly calculated in cross-sectional studies and case control sectional studies and case control studies, and from logistic regression.studies, and from logistic regression. Interpretation:Interpretation:

>1 suggests positive association>1 suggests positive association <1 suggests negative association<1 suggests negative association =1 suggests no difference between groups=1 suggests no difference between groups

Review: Odds Ratio – Review: Odds Ratio – General DefinitionGeneral Definition

D+ D-D+ D-

E+E+

E-E-

OR =OR =odds of disease for E+odds of disease for E+ = = A/BA/B==ADAD

odds of disease for E- = C/D BCodds of disease for E- = C/D BC

A B

C D

Review: Exposure Odds Review: Exposure Odds Ratio – Case Control StudyRatio – Case Control Study

D+ D-D+ D-

E+E+

E-E-

OR =OR =odds of exposure for D+odds of exposure for D+ = = A/CA/C==ADAD

odds of exposure for D- = B/D BCodds of exposure for D- = B/D BC

A B

C D

Review: Absolute RiskReview: Absolute Risk D+ D-D+ D-

E+E+

E-E-

AR = (Risk for E+) - (Risk for E-) = AR = (Risk for E+) - (Risk for E-) =

A/(A + B) - C/(C + D)A/(A + B) - C/(C + D)

A

B

C

D

Review: Hypothesis Review: Hypothesis TestingTesting Random sampling error exists in all Random sampling error exists in all

epidemiological studies. Hypothesis epidemiological studies. Hypothesis testing allows us to account for this testing allows us to account for this random error and to determine whether a random error and to determine whether a result is “statistically significant.”result is “statistically significant.”

Hypothesis TestingHypothesis Testing – Statistically test – Statistically test the study hypothesis against the the study hypothesis against the null null hypothesishypothesis (the null hypothesis is the (the null hypothesis is the nothing hypothesis - says there is no nothing hypothesis - says there is no association between two variables – i.e. association between two variables – i.e. between risk factor and disease).between risk factor and disease).

Study HypothesisStudy Hypothesis – i.e. - There – i.e. - There is an is an associationassociation between sex & race and between sex & race and physicians’ recommendations for cardiac physicians’ recommendations for cardiac catheterization.catheterization.

Review: p-ValueReview: p-Value Test statisticTest statistic – A value quantifying the – A value quantifying the

degree of association between two degree of association between two variables that is calculated from the variables that is calculated from the statistical test procedure. For example, a statistical test procedure. For example, a chi-square statistic.chi-square statistic.

p-Valuep-Value - - The probability of The probability of obtaining a value for the test statistic obtaining a value for the test statistic as extreme or more extreme as that as extreme or more extreme as that observed if the null hypothesis were observed if the null hypothesis were truetrue (also calculated from the statistical (also calculated from the statistical test procedure). A p-Value quantifies the test procedure). A p-Value quantifies the degree of random variability in the degree of random variability in the sampling process.sampling process.

Review: p-ValueReview: p-Value

Statistical SignificanceStatistical Significance – Most – Most researchers are willing to declare that researchers are willing to declare that a relationship is statistically significant a relationship is statistically significant if the chances of observing the if the chances of observing the relationship in the sample when relationship in the sample when nothing is going on in the population nothing is going on in the population are less than 5%. This is why the are less than 5%. This is why the commonly accepted cut point for commonly accepted cut point for calling a result “statistically significant calling a result “statistically significant is p<0.05.is p<0.05.

Review: Confidence Review: Confidence IntervalsIntervals Another value that can be calculated Another value that can be calculated

from statistical test procedures that from statistical test procedures that accounts for random sampling error.accounts for random sampling error.

95% Confidence Intervals (95% CI) are 95% Confidence Intervals (95% CI) are commonly reported.commonly reported.

95% CI – A range of values 95% CI – A range of values computed from the sample that computed from the sample that should contain the true population should contain the true population parameter with 95% probability in parameter with 95% probability in repeated collections of the datarepeated collections of the data (i.e. (i.e. a range of values that is almost sure to a range of values that is almost sure to contain the true population parameter).contain the true population parameter).

Review: Confidence Review: Confidence IntervalsIntervals

The width of a confidence interval is The width of a confidence interval is inversely proportionate to the sample size of inversely proportionate to the sample size of the study.the study.

For risk ratios and odds ratios, if the For risk ratios and odds ratios, if the confidence interval includes the value “1,” confidence interval includes the value “1,” the association is not “statistically the association is not “statistically significant.”significant.”

If the confidence intervals for measures in If the confidence intervals for measures in two groups overlaps, the two groups do not two groups overlaps, the two groups do not differ “significantly” with respect to that differ “significantly” with respect to that measure.measure.

If the confidence interval for the difference If the confidence interval for the difference between two groups does not include “0,” between two groups does not include “0,” this difference is “statistically significant.”this difference is “statistically significant.”

Review: Important!Review: Important!

p-Values and Confidence Intervals p-Values and Confidence Intervals assume that there is no assume that there is no bias, or bias, or systematic errorsystematic error, in the study - i.e., , in the study - i.e., they do not account for bias in the they do not account for bias in the study. They do not assure that the study. They do not assure that the association is real. They do not quantify association is real. They do not quantify clinical significanceclinical significance. It is important . It is important not to completely discount values that not to completely discount values that are not statistically significant. One are not statistically significant. One must also look at trends and how the must also look at trends and how the results compare to previous studies.results compare to previous studies.

PrecisionPrecision vs. Accuracy vs. Accuracy

Precision vs. Precision vs. AccuracyAccuracy

Accuracy=Validity=TruthAccuracy=Validity=Truth

External Validity - Are the results of the External Validity - Are the results of the study generalizable to other populations study generalizable to other populations of interest? Are the results valid for this of interest? Are the results valid for this other population?other population?

Internal Validity - Do the study results Internal Validity - Do the study results represent the truth for the population represent the truth for the population studied? All studies are flawed to some studied? All studies are flawed to some degree. To reduce the effect of bias and degree. To reduce the effect of bias and confounding on a study’s results, the confounding on a study’s results, the study must be correctly designed, study must be correctly designed, executed, and analyzed.executed, and analyzed.

BiasBias Systematic Error – Deviation of Systematic Error – Deviation of

results from the truth or - any results from the truth or - any process or effect at any stage of a process or effect at any stage of a study from its design to its execution study from its design to its execution to the application of information from to the application of information from the study, that produces results or the study, that produces results or conclusions that differ systematically conclusions that differ systematically from the truth.from the truth. Initial selection of participants for a Initial selection of participants for a

studystudy Continued participation in a studyContinued participation in a study Methods of measurementMethods of measurement

Selection BiasSelection Bias Selection Bias – Selection Bias – A bias in assignment A bias in assignment

that arises from study design rather than that arises from study design rather than by chance. These can occur when the by chance. These can occur when the study and control groups are chosen so study and control groups are chosen so that they differ from each other by one or that they differ from each other by one or more factors that may affect the outcome more factors that may affect the outcome of the study (a potential problem in case of the study (a potential problem in case control studies).control studies). From last week’s study: Infants with From last week’s study: Infants with

intussuception were compared to others born intussuception were compared to others born at the same hospital A bias could have arisen at the same hospital A bias could have arisen if they had instead been compared to infants if they had instead been compared to infants born in other communities (with different born in other communities (with different immunization practices) or to other immunization practices) or to other hospitalized (sick) infants.hospitalized (sick) infants.

Non-Response BiasNon-Response Bias Non-response biasNon-response bias - How do - How do

respondents and non-respondents differ respondents and non-respondents differ in regard to the study question? In in regard to the study question? In general, respondents tend to be more general, respondents tend to be more educated compared to non-respondents.educated compared to non-respondents. From Week 2 – Authors of the cardiac From Week 2 – Authors of the cardiac

catheterization study did not give physician catheterization study did not give physician response rates – not only could physicians response rates – not only could physicians attending these national meetings been attending these national meetings been more educated & aware, but out of those more educated & aware, but out of those attending, those who chose to participate attending, those who chose to participate may be even more educated and aware may be even more educated and aware compared to the general population of compared to the general population of physicians.physicians.

Loss-to-Follow-Up BiasLoss-to-Follow-Up Bias

Loss-to-follow-up BiasLoss-to-follow-up Bias – Even if the – Even if the study sample was representative of study sample was representative of the population from which it was the population from which it was derived at the beginning of a study, it derived at the beginning of a study, it may not be by the end of the study. may not be by the end of the study. This is a potential problem in cohort This is a potential problem in cohort studies and clinical trials.studies and clinical trials. It may be more difficult to maintain long-It may be more difficult to maintain long-

term follow-up of patients of lower SES.term follow-up of patients of lower SES. Patients may drop out of a clinical trial Patients may drop out of a clinical trial

because of symptoms they are having because of symptoms they are having that may be due to the study drug.that may be due to the study drug.

Measurement BiasMeasurement Bias Measurement bias – Were measurement Measurement bias – Were measurement

methods consistently different between methods consistently different between groups in a study?groups in a study? Lead-Time BiasLead-Time Bias: If study patients are not : If study patients are not

enrolled at similar, well-defined points in the enrolled at similar, well-defined points in the course of their illness, differences in outcome over course of their illness, differences in outcome over time may simply reflect differences in the duration time may simply reflect differences in the duration of their illness. For example, persons diagnosed of their illness. For example, persons diagnosed using screening tests will be observed to live using screening tests will be observed to live longer than those diagnosed based on clinical longer than those diagnosed based on clinical symptoms.symptoms.

Recall BiasRecall Bias: Systematic Error due to the : Systematic Error due to the differences in accuracy or completeness of recall differences in accuracy or completeness of recall to memory of past events or experiences. A to memory of past events or experiences. A potential problem in case-control studies, for potential problem in case-control studies, for example.example.

ConfoundingConfounding

ConfoundingConfounding may be considered "a may be considered "a confusion of effects" - attributing a confusion of effects" - attributing a result or disease to a specific risk result or disease to a specific risk factor when it is in fact due to factor when it is in fact due to another factor It can lead to over- or another factor It can lead to over- or under-estimation of an effect or can under-estimation of an effect or can even change the direction of the even change the direction of the effect.effect.

ConfoundingConfounding Researchers may attempt to control Researchers may attempt to control

confounding in several different ways:confounding in several different ways: Matching: Infants with intussusception were Matching: Infants with intussusception were

matched to controls of the same age and matched to controls of the same age and birth location (they attempted to match birth location (they attempted to match them to infants born in the same hospital on them to infants born in the same hospital on the same day). Age is related both to the the same day). Age is related both to the probability of having been vaccinated with probability of having been vaccinated with RRV-TV and to the risk of intussusception. RRV-TV and to the risk of intussusception.

Regression (a statistical procedure): Regression (a statistical procedure): “Variables used to adjust the odds ratios “Variables used to adjust the odds ratios were related to both the risk of were related to both the risk of intussusception and to vaccination with intussusception and to vaccination with RRV-TV.” The reported adjusted odds ratios RRV-TV.” The reported adjusted odds ratios were adjusted for sex, mother’s level of were adjusted for sex, mother’s level of education, education, type of health insurancetype of health insurance, type of , type of mild or formula used for feeding, and time of mild or formula used for feeding, and time of first intake of solids.first intake of solids.

Effect ModificationEffect Modification Does the relationship between the Does the relationship between the

predictor variable (risk factor) and predictor variable (risk factor) and outcome variable (disease) vary among outcome variable (disease) vary among different subgroups of a population? different subgroups of a population? (Statistical term is “interaction”).(Statistical term is “interaction”). Example: “The risk of intussusception three Example: “The risk of intussusception three

to seven days after the first dose of RRV-TV to seven days after the first dose of RRV-TV was lower among infants fed breast milk was lower among infants fed breast milk (adjusted odds ratio, 10.7; 95%CI, 1.4 to 78.7) (adjusted odds ratio, 10.7; 95%CI, 1.4 to 78.7) than among other vaccinated infants (adjusted than among other vaccinated infants (adjusted odds ratio, 43.3; 95%CI, 12.7 to 148.1). odds ratio, 43.3; 95%CI, 12.7 to 148.1). However, the difference between these two However, the difference between these two estimates was not statistically significant estimates was not statistically significant (p=0.22).”(p=0.22).”

Sources of DefinitionsSources of Definitions

Last, John M. Last, John M. A Dictionary of A Dictionary of EpidemiologyEpidemiology, 4, 4thth edition, Oxford edition, Oxford Press (2001).Press (2001).

““Clinical Epidemiology & Evidence-Clinical Epidemiology & Evidence-Based Medicine Glossary,” Based Medicine Glossary,” http://www.vetmed.wsu.edu/courses-http://www.vetmed.wsu.edu/courses-jmgay/GlossClinEpiEBM.htm#Introdjmgay/GlossClinEpiEBM.htm#Introduction and Usage, June 22, 2002.uction and Usage, June 22, 2002.