Confidence Interval & P-Value Tutorial

Lecture & tutorial material prepared by: Dr. Shaffi Shaikh

Tutorial presented by:Dr. Rufaidah Dabbagh Dr. Nurah Al-Amro

What is a confidence interval?When we say that our confidence interval is

95% this means that : If we repeat our study several times we will

find that the outcome we are calculating lies in the interval 95% of the times.

ExampleSay you conducted a study to estimate the risk

of lung cancer among smokers. You did your data analysis and you found out

that the OR (odds ratio) for lung cancer comparing smokers with non-smokers. OR was 3.8 with a 95% CI of (2.5, 4.1).

This means: If we conducted this same study several times

and calculated the odds ratio for lung cancer, the OR value will be between 2.5 and 4.1, 95% of the time.

What is a P-ValueP-Value is defined as: the probability of getting an outcome as extreme or

even more extreme than the actual observed value under the null hypothesis.

(in other words, it is the probability of rejecting the null hypothesis when the null hypothesis is actually true)

P-value is usually assigned by the researcher at the beginning of the study.

The significant p-value agreed upon must be ≤ 0.05

Hypothesis testingWhen we conduct a study, we are interested

in showing proof that what we are hypothesizing is actually true.

In statistics, it is difficult for you to prove your hypothesis is true on its own. Instead, you have to prove that the “opposite” of what you are hypothesizing is false.

We call this refuting the null hypothesis.

What is the null-hypothesisThe null hypothesis is basically the opposite of what you

hypothesize.

If you hypothesize that there will be a difference in incidence of a specific disease between males and females, the null hypothesis will be that:

there is NO difference in incidence between males and females

If you are hypothesizing that oral contraceptives increases the risk of ovarian cancer in your study, the null hypothesis will be that:

There is no difference in ovarian cancer risk between those who take oral contraceptives and those who don’t take oral contraceptives.

What is the association between the null hypothesis and the p-value?As mentioned before, the p-value is the probability

of rejecting the null hypothesis when it is actually true.

In our studies, we want this probability to be very small, so that we can convince people that our results will have a good impact on health.

When you get a statistically significant p-value, you can reject the null hypothesis and conclude that what you hypothesized (the alternative hypothesis) is most probably true.

What if you don’t get a statistically significant P-Value?

In this situation, you won’t be able to reject the null hypothesis.

You will conclude that you don’t have enough evidence to reject the null hypothesis and therefore, you won’t be able to comment whether what you hypothesized is actually true or false.

You will need to do further studies.

Q1. Definition of “p-value”

Mark correct and false statements as: (Yes/No)

Statement Response

A “p” stands for probability and it ranges from 0 to 1.

A p-value of ≤ 0.05 is considered as not statistically significant

A p-value of >0.05 is considered as statistically significant

Statistically significant is more important than clinical significant

The p-value is the probability of getting an outcome as extreme as or more extreme than the actually observed outcome (sample) under the null hypothesis.

Usually the null hypothesis is a statement of "no effect", "no difference" or "=0" and we are eager to find evidence against it.

When large samples are available, even small deviations from the null hypothesis will be significant.