Evidence-Based Medicine 3

Evidence-Based Evidence-Based Medicine 3Medicine 3

More Knowledge and Skills More Knowledge and Skills for Critical Readingfor Critical Reading

Karen E. Schetzina, MD, MPH

EpidemiologyEpidemiology Definition - the study of the distribution Definition - the study of the distribution

and determinants of health-related states and determinants of health-related states or events in specified populations, and or events in specified populations, and the application of this study to the control the application of this study to the control of health problems.of health problems.

Epidemiologists and clinical researchers Epidemiologists and clinical researchers study samples of populations, collect study samples of populations, collect information on variables of interest from information on variables of interest from persons in the samples, and then look for persons in the samples, and then look for associations between the variables of associations between the variables of interest.interest.

Through this process, hypotheses are Through this process, hypotheses are generated, causes of disease are generated, causes of disease are identified, treatments are discovered, etc.identified, treatments are discovered, etc.

The Effect of Race and Sex on Physicians’ The Effect of Race and Sex on Physicians’ Recommendations for Cardiac Recommendations for Cardiac CatheterizationCatheterization Design: Computerized SurveyDesign: Computerized Survey Surveyed 720 primary care physicians Surveyed 720 primary care physicians

attending two national meetingsattending two national meetings Physicians viewed video recorded interviews of Physicians viewed video recorded interviews of

black and white male and female patients ages black and white male and female patients ages 55 – 70 with chest pain, as well as results of 55 – 70 with chest pain, as well as results of their electrocardiography and thallium stress their electrocardiography and thallium stress teststests

The physicians then were asked whether they The physicians then were asked whether they wished to refer the patient for cardiac wished to refer the patient for cardiac catheterizationcatheterization

Critical Reading - ReviewCritical Reading - Review


What is a variable?What is a variable?

What were the main predictor What were the main predictor variables in this study?variables in this study?

What was the main outcome variable What was the main outcome variable in this study?in this study?

Critical Reading – ReviewCritical Reading – Review Results – Table 4: Referral for Cardiac Results – Table 4: Referral for Cardiac

Catheterization According to Catheterization According to Experimental FactorExperimental FactorFactorFactor Mean Mean

Referral Referral Rate (%)Rate (%)

Odds Odds Ratio Ratio (95% CI)(95% CI)

P ValueP Value

MaleMale

FemaleFemale90.690.6

84.584.51.01.00.60.6 (0.4-(0.4-0.9)0.9)

0.020.02

WhiteWhite

BlackBlack90.690.6

84.784.71.01.00.60.6 (0.4-(0.4-0.9)0.9)

0.020.02

Critical Reading – ReviewCritical Reading – Review Odds Ratio – The ratio of two odds. Odds Ratio – The ratio of two odds.

For For rare diseasesrare diseases, this , this approximates relative risk. approximates relative risk. Commonly calculated in cross-Commonly calculated in cross-sectional studies and case control sectional studies and case control studies, and from logistic regression.studies, and from logistic regression. Interpretation:Interpretation:

>1 suggests positive association>1 suggests positive association <1 suggests negative association<1 suggests negative association =1 suggests no difference between groups=1 suggests no difference between groups

Odds Ratio – Cardiac Odds Ratio – Cardiac Catheterization ArticleCatheterization Article

Referred Not ReferredReferred Not Referred

FemaleFemale

MaleMale

OR =OR =odds of referral for femalesodds of referral for females = = A/BA/B==ADAD

odds of referral for males = C/D BCodds of referral for males = C/D BC

A B

C D

Odds Ratio – General Odds Ratio – General DefinitionDefinition

D+ D-D+ D-

E+E+

E-E-

OR =OR =odds of disease for E+odds of disease for E+ = = A/BA/B==ADAD

odds of disease for E- = C/D BCodds of disease for E- = C/D BC

A B

C D

Exposure Odds Ratio – Case Exposure Odds Ratio – Case Control StudyControl Study

D+ D-D+ D-

E+E+

E-E-

OR =OR =odds of exposure for D+odds of exposure for D+ = = A/CA/C==ADAD

odds of exposure for D- = B/D BCodds of exposure for D- = B/D BC

A B

C D

Relative RiskRelative Risk D+ D-D+ D-

E+E+

E-E-

RR = RR = Risk of disease for E+Risk of disease for E+ = = A/(A + A/(A + B)B)

Risk of disease for E- C/(C + D)Risk of disease for E- C/(C + D)

A B

C D

Absolute RiskAbsolute Risk D+ D-D+ D-

E+E+

E-E-

AR = (Risk for E+) - (Risk for E-) = AR = (Risk for E+) - (Risk for E-) =

A/(A + B) - C/(C + D)A/(A + B) - C/(C + D)

A

B

C

D


““In univariate analysis, the race and In univariate analysis, the race and sex of the patient were sex of the patient were significantlysignificantly associated with the physicians’ associated with the physicians’ decisions about whether to make decisions about whether to make referrals for cardiac catheterization, referrals for cardiac catheterization, with men and whites more likely to with men and whites more likely to be referred than women and blacks, be referred than women and blacks, respectively.”respectively.”


Results – Table 4: Referral for Cardiac Results – Table 4: Referral for Cardiac Catheterization According to Catheterization According to Experimental FactorExperimental FactorFactorFactor Mean Mean

Referral Referral Rate (%)Rate (%)

Odds Odds Ratio Ratio (95% CI)(95% CI)

P ValueP Value

MaleMale

FemaleFemale90.690.6

84.584.51.01.00.6 0.6 (0.4-0.9)(0.4-0.9) 0.020.02

WhiteWhite

BlackBlack90.690.6

84.784.71.01.00.6 0.6 (0.4-0.9)(0.4-0.9)

0.020.02

Hypothesis TestingHypothesis Testing Random sampling error exists in all Random sampling error exists in all

epidemiological studies. Hypothesis epidemiological studies. Hypothesis testing allows us to account for this testing allows us to account for this random error and to determine whether a random error and to determine whether a result is “statistically significant.”result is “statistically significant.”

Hypothesis TestingHypothesis Testing – Statistically test – Statistically test the study hypothesis against the the study hypothesis against the null null hypothesishypothesis (the null hypothesis is the (the null hypothesis is the nothing hypothesis - says there is no nothing hypothesis - says there is no association between two variables – i.e. association between two variables – i.e. between risk factor and disease).between risk factor and disease).

Study HypothesisStudy Hypothesis – i.e. - There – i.e. - There is an is an associationassociation between sex & race and between sex & race and physicians’ recommendations for cardiac physicians’ recommendations for cardiac catheterization.catheterization.

p-Valuep-Value Test statisticTest statistic – A value quantifying the – A value quantifying the

degree of association between two degree of association between two variables that is calculated from the variables that is calculated from the statistical test procedure. For example, a statistical test procedure. For example, a chi-square statistic.chi-square statistic.

p-Valuep-Value - The probability of obtaining a - The probability of obtaining a value for the test statistic as extreme or value for the test statistic as extreme or more extreme as that observed if the null more extreme as that observed if the null hypothesis were true (also calculated from hypothesis were true (also calculated from the statistical test procedure). A p-Value the statistical test procedure). A p-Value quantifies the degree of random variability quantifies the degree of random variability in the sampling process.in the sampling process.

p-Valuep-Value

Statistical SignificanceStatistical Significance – Most – Most researchers are willing to declare that researchers are willing to declare that a relationship is statistically significant a relationship is statistically significant if the chances of observing the if the chances of observing the relationship in the sample when relationship in the sample when nothing is going on in the population nothing is going on in the population are less than 5%. This is why the are less than 5%. This is why the commonly accepted cut point for commonly accepted cut point for calling a result “statistically significant calling a result “statistically significant is p<0.05.is p<0.05.

Confidence IntervalsConfidence Intervals Another value that can be calculated Another value that can be calculated

from statistical test procedures that from statistical test procedures that accounts for random sampling error.accounts for random sampling error.

95% Confidence Intervals (95% CI) are 95% Confidence Intervals (95% CI) are commonly reported.commonly reported.

95% CI – A range of values computed 95% CI – A range of values computed from the sample that should contain the from the sample that should contain the true population parameter with 95% true population parameter with 95% probability in repeated collections of probability in repeated collections of the data (i.e. a range of values that is the data (i.e. a range of values that is almost sure to contain the true almost sure to contain the true population parameter).population parameter).

Confidence IntervalsConfidence Intervals

The width of a confidence interval is The width of a confidence interval is inversely proportionate to the sample size of inversely proportionate to the sample size of the study.the study.

For risk ratios and odds ratios, if the For risk ratios and odds ratios, if the confidence interval includes the value “1,” confidence interval includes the value “1,” the association is not “statistically the association is not “statistically significant.”significant.”

If the confidence intervals for measures in If the confidence intervals for measures in two groups overlaps, the two groups do not two groups overlaps, the two groups do not differ “significantly” with respect to that differ “significantly” with respect to that measure.measure.

Important!Important!

p-Values and Confidence Intervals p-Values and Confidence Intervals assume that there is no assume that there is no bias, or bias, or systematic errorsystematic error, in the study - i.e., , in the study - i.e., they do not account for bias in the they do not account for bias in the study. They do not assure that the study. They do not assure that the association is real. They do not quantify association is real. They do not quantify clinical significanceclinical significance. It is important . It is important not to completely discount values that not to completely discount values that are not statistically significant. One are not statistically significant. One must also look at trends and how the must also look at trends and how the results compare to previous studies.results compare to previous studies.

Hill’s Causal CriteriaHill’s Causal Criteria

StrengthStrength ConsistencyConsistency SpecificitySpecificity TemporalityTemporality Biologic gradientBiologic gradient PlausibilityPlausibility CoherenceCoherence

Experimental Experimental evidenceevidence

AnalogyAnalogy

Test Your KnowledgeTest Your Knowledge From Table 3 in “Factors Associated From Table 3 in “Factors Associated

with Hypertension Control in the with Hypertension Control in the General Population of the United General Population of the United States”States” Age- and sex- adjusted odds ratios and Age- and sex- adjusted odds ratios and

95% confidence intervals for the 95% confidence intervals for the association between hypertension association between hypertension control and having private health control and having private health insurance (compared to no insurance):insurance (compared to no insurance):NHW: 1.64 (0.99-2.70)NHW: 1.64 (0.99-2.70)NHB: 2.62 (1.62-4.26)NHB: 2.62 (1.62-4.26)MA: 1.16 (0.52-2.60)MA: 1.16 (0.52-2.60)

Test Your KnowledgeTest Your Knowledge From Table 4 in “Factors Associated with From Table 4 in “Factors Associated with

Hypertension Control in the General Hypertension Control in the General Population of the United States”Population of the United States”

Multivariate Adjusted Odds Ratio and 95% Multivariate Adjusted Odds Ratio and 95% Confidence Intervals and p-Values for the Confidence Intervals and p-Values for the association between hypertension control association between hypertension control and marital status:and marital status:

Currently married (compared to never Currently married (compared to never married):married):

OR=2.39 (1.52-3.71) p-Value<0.001OR=2.39 (1.52-3.71) p-Value<0.001

Next LectureNext Lecture We will discuss sources of systematic error We will discuss sources of systematic error

(bias) and confounding.(bias) and confounding. Some examples are: Some examples are: 1. 1. Selection-biasSelection-bias (people who volunteer for studies (people who volunteer for studies

may be different, "healthy-worker effect"). From the may be different, "healthy-worker effect"). From the study: "Physicians who attend professional meeting study: "Physicians who attend professional meeting may be better informed than those who do not attend may be better informed than those who do not attend . . . may have a greater interest than others in . . . may have a greater interest than others in coronary heart disease." How might findings differ if coronary heart disease." How might findings differ if they sampled all practicing physicians?they sampled all practicing physicians?

2. 2. Non-response biasNon-response bias (How do respondents and (How do respondents and non-respondents differ in regard to the study non-respondents differ in regard to the study question?). This study does not give response rates - question?). This study does not give response rates - only says that 720 physicians participated - at least only says that 720 physicians participated - at least they did not know that it was a study of the effects of they did not know that it was a study of the effects of race and sex.race and sex.

3. 3. Measurement biasMeasurement bias (How accurately were the (How accurately were the predictor and outcome variables measured?)predictor and outcome variables measured?)

Next LectureNext Lecture ConfoundingConfounding may be considered "a confusion of effects" - attributing a may be considered "a confusion of effects" - attributing a

result or disease to a specific risk factor when it is in fact due to result or disease to a specific risk factor when it is in fact due to another factor It can lead to over- or under-estimation of an effect or another factor It can lead to over- or under-estimation of an effect or can even change the direction of the effect.can even change the direction of the effect.

Researchers may attempt to control confounding in several difference Researchers may attempt to control confounding in several difference ways. From the study: these authors reported that they clothed ways. From the study: these authors reported that they clothed patients identically and listed them as having the same type of patients identically and listed them as having the same type of insurance and occupations to help to remove the potential confounding insurance and occupations to help to remove the potential confounding effects of SES and insurance. effects of SES and insurance.

Another way the authors attempted to control for confounding was by Another way the authors attempted to control for confounding was by using a "multivariate logistic regression analysis." From the study: Are using a "multivariate logistic regression analysis." From the study: Are the differences between rates of referral by race and sex due to other the differences between rates of referral by race and sex due to other factors besides just race and sex? Physicians are aware of many factors besides just race and sex? Physicians are aware of many different risk factors for coronary heart disease (several are reported in different risk factors for coronary heart disease (several are reported in the other article you read). Perhaps they referred the white males in the other article you read). Perhaps they referred the white males in the study more often because they thought that they were at higher the study more often because they thought that they were at higher risk for coronary artery disease based on their clinical presentation? risk for coronary artery disease based on their clinical presentation? Well, the authors attempted to account for this possibility in the Well, the authors attempted to account for this possibility in the analysis as well as for other factors (age, level of risk, type of chest analysis as well as for other factors (age, level of risk, type of chest pain, results of thallium test) in an attempt to determine the differences pain, results of thallium test) in an attempt to determine the differences in referral rates just based on sex and race as independent factors. in referral rates just based on sex and race as independent factors.

Documents

Evidence-Based Medicine 3