What do you think about a doctor who uses the wrong treatment, either wilfully or through ignorance, or who uses the right treatment wrongly (such as by

What do you think about a doctor who uses the wrong treatment, either wilfully or through ignorance, or who uses the right treatment wrongly (such as by giving the wrong dose of a drug)?

Most people would agree that such behaviour is unprofessional, arguably unethical, and certainly unacceptable.

Derived from: Altman DG. The Scandal of Poor Medical Research. BMJ, 1994; 308:283

What do you think about researchers who use the wrong techniques (either wilfully or in ignorance), use the right techniques wrongly, misinterpret their results, report their results selectively or draw unjustified conclusions? We should be appalled… but numerous studies of the medical literature have shown that all of the above phenomena are common.

Derived from: Altman DG. The Scandal of Poor Medical Research. BMJ, 1994; 308:283

Understanding your results Research Talk

2015

Dr Emily [email protected]

Office for Research, Western Centre for Health Research & EducationCentre for Epidemiology and Biostatistics, Melbourne School of Population

and Global Health, University of Melbourne

Overview

• Defining your research question – PICOS• Describing data• Understanding the results

– Estimates reported in the literature– Interpreting 95% confidence intervals and p-

values ~ Statistical Inference

Research question

Participants / population• neonates

Intervention / exposure• 14 day administration of antenatal corticosteroids

Comparison• 7 day administration of antenatal corticosteroids

Outcome • Neonatal mortality and neonatal morbidity

Study design• RCT

Murphy et al. The Lancet, 2008; 372:2143-2151.

Research question

Research question

Participants / population• Neonates




Study design• RCT

Research question

Participants / population• Women at high risk of preterm birth




Study design• RCT

Study designs

The general idea…

– Evaluate whether a risk factor (or

preventative factor) increases (decreases)

the risk of an outcome (e.g. disease, death,

etc)

exposure

outcome

time

Overview




Study designs

The general idea…




etc)

exposure

outcome

time

Murphy et al. The Lancet, 2008; 372:2143-2151.

Summarising the data


Dreyfus et al. Journal of Pediatrics, 2015 online.



Numerical

Categorical

Continuous (age, weight, height)

Discrete (length of stay, # of hospital visits)

Nominal (sex, blood group)

Ordinal (tumour stage, quintile of SES)

Discrete

Nominal

Ordinal

• Which variables are categorical? – Sex (Male/Female)– Country of birth (Australia/Elsewhere)

• Which variables are continuous? – Age (years)– Length of stay (days)






0.0

5.1

.15

.2D

ens

ity

44 46 48 50 52 54Age (years)

Stata command: histogram Age


Standard deviation

Mean = 49.8 years

= 2.1 years

Note, 95% of observations lie within approximately ±2×SD of the mean. In this example, 95% of observations lie within 45.6 and 54.0 years.




0.0

5.1

.15

.2D

ens

ity

0 5 10 15 20 25 30 35Length of stay (days)

Stata command: hist LOS


Stata command: hist LOS, normal


Mean = 5 days


Mean = 5 days

Median = 50th percentile= 4 days


Mean = 5 days

Standard deviation

Median = 4 days

Mean is not a good measure of central tendency and standard deviation is not a good measures of spread for a skewed distribution

Note, 95% of observations lie within approximately ±2SD of the mean. In this example, 95% of observations lie within -4.8 and 14.8 daysBUT they don’t because LOS can’t be negative!


Inter-quartile range (IQR) = lower quartile – upper quartile= 25th percentile – 75th percentile = 2 to 6 days

Median = 50th percentile= 4 days



Spread

Spread

Central tendency

Central tendency



Positive skew

Negative skew

Data variable - numerical

Plot histogram

Normally distributed

NOT normally distributed

Unimodal Multimodal

MeanStandard deviationMinimum-maximum

MedianInter-quartile rangeMinimum-maximum

Categorise variable


Simpson et al. J Fam Plan and Rep Health Care, 2001; 27:234-236.


Absolutely critical to choosing the appropriate form of statistical analysis


SkewedNumerical

Categorical





Overview




Study designs

The general idea…




etc)

exposure

outcome

time

Estimates reported in the literature

– Risk differences– Odds ratios / risk ratio – logistic regression– Beta-coefficients – linear regression



SkewedNumerical

Categorical





Measures of association – binary outcome

Binary variables – two categories only

(also termed – dichotomous variable)

Examples: • Outcome – diseased or healthy; alive or dead• Exposure – male or female; smoker or non-smoker;

treatment or control group

Comparing two proportions

With outcome (diseased)

Without outcome (disease free)

Total

Exposed (group 1)

d1 h1 n1

Unexposed (group 0)

d0 h0 n0

Total d h n

• Proportion of all subjects experiencing outcome, p = d/n

• Proportion of exposed group, p1 = d1/n1

• Proportion of unexposed group, p0 = d0/n0

Comparing two proportions - TBM Trial

Adults with tuberculous meningitis randomly allocated into 2 treatment groups:

1. Dexamethasone

2. Placebo

Outcome measure: Death during 9 months following start of treatment.

Research question:

Can treatment with dexamethasone reduce the risk of death among adults with tuberculous meningitis?

Thwaites et al 2004


Death during 9 months post start of treatment

Treatment group Yes No Total

Dexamethasone(group 1)

87 187 274

Placebo (group 0)

112 159 271

Total 199 346 545

Thwaites et al 2004


Measure of effect Formula

Risk difference p1-p0

Risk Ratio (RR) p1/p0

Odds Ratio (OR) (d1/h1)/(d0/h0)

When there is no association between exposure and outcome:

– Risk difference = 0 – Risk ratio (RR) = 1– Odds Ratio (OR) = 1


Death during 9 months post start of treatment

Treatment group Yes No Total

Dexamethasone(group 1)

87 (d1) 187 (h1) 274 (n1)

Placebo (group 0)

112 (d0) 159 (h0) 271 (n0)

Total 199 346 545

Risk difference = p1-p0 = (87/274)-(112/271) = -0.095

Risk ratio = p1/p0 = (87/274)/(112/271) = 0.77

Odds ratio = (d1/h1)/(d0/h0) = (87/187)/(112/159) = 0.66

Thwaites et al 2004


Estimates reported in the literature

– Risk differences– Odds ratios / risk ratio – logistic regression– Beta-coefficients – linear regression



SkewedNumerical

Categorical





Linear regression

Dreyfus et al. Journal of Pediatrics, 2015 online.

There are four assumptions underlying our linear regression model:

Linearity (outcome and exposure)

Normality (residual variation)

Independence (of observations)

Homoscedasticity (constant variance)

Linear regression

Overview




Statistical Inference

We follow a standard four-step process

1) Sample size

2) Estimate of the effect size

3) Calculate a confidence interval

4) Derive a p-value to test the hypothesis of no association



P-value

How likely is it we would see a

difference this big

IF

There was NO realdifference betweenthe populations?

What is theprobability (P-value)

of finding theobserved difference

IF

The null hypothesis is true?


0.0001

0.001

0.01

0.1

1P-v

alu

e

Increasing evidence againstthe null hypothesis with

decreasing P-value

Weak evidence againstthe null hypothesis

Interpretation of p-values

Strong evidence againstthe null hypothesis



Overweight and obese adults living in the UK

300 adults participating in a RCT comparing 2 dietary interventions

Mean weight loss after 4 weeksAtkins group – 4.40 kgWeight Watchers group – 2.86 kg

Source: Truby H et al. BMJ 2007

Example: Randomised controlled trial of weight loss programmes in the UK

Group nSample mean

Weight loss after 4 weeks (kg)

Sample standard deviation

Sample standard error

Atkins 57 4.40 2.45 0.32

Weight Watchers

58 2.86 2.23 0.29

1) Estimate of difference in population mean weight loss after 4 weeks between Atkins & Weight Watchers groups = 4.40 – 2.86 = 1.54 kg2) 95% CI: 0.67 kg to 2.41 kg

Source: Truby H et al. BMJ 2007


Interpretation

1) We found a difference of 1.54 kg in mean weight loss

after 4 weeks between the Atkins & Weight Watchers

diet groups.

2) From the 95% confidence interval, the true difference

could be as much as 2.41 kg (much greater weight

loss for Atkins diet) or 0.67 kg (marginally greater

weight loss for the Atkins diet compared with Weight

Watchers).


P-value: comparing two groups

How likely is it we would see a

difference this big

IF

There was NO realdifference betweenthe populations?

What is theprobability (P-value)

of finding theobserved difference

IF

The null hypothesis is true?


Null hypothesis – There is no difference in the population mean weight loss after 4 weeks between the Atkins and Weight Watchers groups

2-sided p-value <0.001

Thus the probability of observing a difference of at least 1.54 kg in the sample means of the two groups, assuming the null hypothesis is true,

is <0.001 or <0.1%.


Presenting the results

1) Sample size300 adults participating in a RCT comparing 2 dietary

interventions

2) Estimate of the effect sizeMean weight loss after 4 weeks for Atkins group compared to Weight watchers: 1.54 kg

3) Calculate a confidence interval 95% CI for difference in population means: 0.67 kg to 2.41 kg

4) Derive a p-value to test the hypothesis of no association

P-value < 0.001


Overview




Documents

What do you think about a doctor who uses the wrong treatment, either wilfully or through ignorance, or who uses the right treatment wrongly (such as by