Confidence intervals

Kristin Tolksdorf(based on previous EPIET material)

18th EPIET/EUPHEM Introductory course01.10.2012

Inferential statistics

• Uses patterns in the sample data to draw inferences about the population represented, accounting for randomness.

• Two basic approaches: – Hypothesis testing– Estimation

2

Criticism on significance testing

“Epidemiological application need more than a decision as to whether chance alone could have produced association.” (Rothman et al. 2008)

→ Estimation of an effect measure (e.g. RR, OR) rather than significance testing.

→ Estimation of a mean

→ Estimation of a proportion3

Why estimation?

Norovirus outbreak on a Greek island: “The risk of illness was higher among people who ate raw seafood (RR=21.5).”

How confident can we be in the result?What is the precision of our point estimate?

4

The epidemiologist needs measurements rather than probabilities

2 is a test of association

OR, RR are measures of association on a continuous scale infinite number of possible values

The best estimate = point estimate

Range of “most plausible” values, given the sample data

Confidence interval precision of the point estimate

5

Confidence interval (CI)

Range of values, on the basis of the sample data, in which the population value (or true value) may lie.

• Frequently used formulation: „If the data collection and analysis could be replicated many times, the CI should include the true value of the measure 95% of the time .”

6

α/2

Lower limit upper limitof 95% CI of 95% CI

= 5%

s

α/2

Confidence interval (CI)

Indicates the amount of random error in the estimate Can be calculated for any „test statistic“, e.g.: means, proportions, ORs, RRs

95% CI = x – 1.96 SE up to x + 1.96 SE

1 - α

7

CI terminology

RR = 1.45 (0.99 – 2.13)

Confidence intervalPoint estimate

Lower confidence limit

Upper confidence limit

8

• amount of variability in the data

• size of the sample

• level of confidence (usually 90%, 95%, 99%)

Width of confidence interval depends on …

A common way to use CI regarding OR/RR is :If 1.0 is included in CI non significant If 1.0 is not included in CI significant

9

Study A, large sample, precise results, narrow CI – SIGNIFICANTStudy B, small size, large CI - NON SIGNIFICANT

Looking at the CI

Study A, effect close to NO EFFECTStudy B, no information about absence of large effect

RR = 1

A

B

Large RR

10

More studies are better or worse?

1RR

20 studies with different results...

11

clinical or biological significance ?

Norovirus on a Greek island

• How confident can we be in the result?• Relative risk = 21.5 (point estimate)• 95% CI for the relative risk:

(8.9 - 51.8)

The probability that the CI from 8.9 to 51.8 includes the true relative risk is 95%.

12

Norovirus on a Greek island

“The risk of illness was higher among people who ate raw seafood (RR=21.5, 95% CI 8.9 to 51.8).”

13

Example: Chlordiazopoxide use and congenital heart disease (n=1 644)

Cases Controls

C use 4 4

No C use 386 1 250

OR = (4 x 1250) / (4 x 386) = 3.2

p = 0.080 ; 95% CI = 0.6 - 17.5

From Rothman K

3.2

p=0.080

0.6 – 17.515

Example: Chlordiazopoxide use and congenital heart disease – large study (n=17 151)

Cases Controls

C use 240 211

No C use 7 900 8 800

OR = (240 x 8800) / (211 x 7900) = 1.3

p = 0.013 ; 95% CI = 1.1 - 1.5

Precision and strength of association

Strength

Precision17

Confidence interval provides more information than p value

• Magnitude of the effect (strength of association)

• Direction of the effect (RR > or < 1)

• Precision of the point estimate of the effect (variability)

p value can not provide them ! 18

2 Test of association, depends on sample size

p value Probability that equal (or more extreme) results can be observed by chance alone

OR, RR Direction & strength of associationif > 1 risk factor if < 1 protective factor(independently from sample size)

CI Magnitude and precision of effect

What we have to evaluate the study

19

Comments on p-values and CIs

• Presence of significance does not prove clinical or biological relevance of an effect.

• A lack of significance is not necessarily a lack of an effect: “Absence of evidence is not evidence of absence”.

20

Comments on p-values and CIs

• A huge effect in a small sample or a small effect in a large sample can result in identical p values.

• A statistical test will always give a significant result if the sample is big enough.

• p values and CIs do not provide any information on the possibility that the observed association is due to bias or confounding.

21

Cases Non-cases Total 2 = 1.3E 9 51 60 p = 0.13NE 5 55 60 RR = 1.8Total 14 106 120 95% CI [ 0.6 - 4.9 ]

Cases Non-cases Total 2 = 12E 90 510 600 p = 0.0002NE 50 550 600 RR = 1.8Total 140 1060 1200 95% CI [ 1.3-2.5 ]

2 and Relative Risk

22

Exposure Cases Non-cases AR%Yes 15 20 42.8%No 50 200 20.0%

Total 65 220

Common source outbreak suspected

REMEMBER: These values do not provide any information on the possibility that the observed association is due to a bias or confounding.

2 = 9.1 p = 0.002RR = 2.195%CI = 1.4 - 3.4

23%

23

The ultimative (eye) test

24

• Hypothesis testing: X²-Test– Question: Is the proportion of facilitators wearing

glasses equal to the proportion of fellows wearing glasses?

• Estimation of quantities: Proportion– What is the proportion of fellows/facilitators

wearing glasses?

The ultimative (eye) test

25

Proportion = 11/38 = 0.29SE = 0.07495%CI = 0.14 - 0.44

Glasses among fellows : Yes 11No 27

Total 38

Glasses among facilitators :

Yes 6No 8

Total 14

Proportion = 6/14 = 0.43SE = 0.13295%CI = 0.17 - 0.69

Recommendations

• Always look at the raw data (2x2-table). How many cases can be explained by the exposure?

• Interpret with caution associations that achieve statistical significance.

• Double caution if this statistical significance is not expected.

• Use confidence intervals to describe your results.

26

Suggested reading

• KJ Rothman, S Greenland, TL Lash, Modern Epidemiology, Lippincott Williams & Wilkins, Philadelphia, PA, 2008

• SN Goodman, R Royall, Evidence and Scientific Research, AJPH 78, 1568, 1988

• SN Goodman, Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy, Ann Intern Med. 130, 995, 1999

• C Poole, Low P-Values or Narrow Confidence Intervals: Which are more Durable? Epidemiology 12, 291, 2001

27

Previous lecturers

28