Critical Appraisal of Clinical Research Evidence

Critical appraisal of clinical research evidence

Chris Lewis – May 2008

How to read a “paper”

Objectives:

To enable VTS members to have a good working knowledge of:

• Processes of EBM• Conduct of a RCT • Sources of bias in a RCT • Risk assessment terminology (RR ARR NNT)

Evidence Based Medicine

Incorporating the best available research evidence into clinical decision making

Processes of Evidence Based Medicine

• Asking answerable questions (PICO)• Accessing the best information• Appraising the information for validity and

relevance• Applying the information to patient care

Asking an answerable question

• Population• Intervention• Comparator• Outcome(s)

Types of “paper” research evidence• Primary studies

– Case studies– Experiments– Surveys– Clinical Trials

• Secondary studies– Non-systematic reviews– Systematic reviews

• Meta-analyses • Guidelines • Decision analyses

• Economic analyses

Advantages DisadvantagesEvidence-based Guideline

Summarises all relevant research about all possible interventions for a clinical problem. Explores benefits and harms.

May become out-of-date quickly.Expert opinion often fills gaps in evidence.

Systematic Review Summarises all research about an intervention.

Usually only one of several possible interventions is considered. May not explore benfits vs harms.

Primary Study Very specific information Not comprehensive

Types of evidence

Topics of Primary Study and Types of Study Design

PhenomenaObservation / qualitative studies

AetiologyCohort studies (or Case-control studies)

Diagnosis and screeningCross-sectional analytical studies

PrognosisCohort studies

InterventionRandomised Controlled Trials

How do you read a clinical research paper?

How to read a (clinical research) paper

• Scan abstract for a few seconds– Are the authors conclusions of interest?– Briefly assess study design– Briefly assess statistical precision of results– Formulate a brief summary

• Critically appraise methods & results sections for validity

• Critically appraise results section (especially the tables and figures) for relevance

• Draw your own conclusions about clinical applicability

What conclusions have you drawn concerning clinical application of the

Heart Protection Study?

How to read a (clinical research) paper

• Scan abstract for a few seconds– Are the authors conclusions of interest?– Briefly assess study design– Briefly assess statistical precision of results– Formulate a brief summary

• Critically appraise methods & results sections for validity

• Critically appraise results section (especially the tables and figures) for relevance

• Draw your own conclusions about clinical applicability

WHAT INFORMATION WOULD YOU INCLUDE IN A BRIEF SUMMARY OF A CLINICAL RESEARCH PAPER?

What is the essential information you want to know about any clinical research evidence?

Information to include in summary• Type of study• Size• Study Population • Intervention• Comparator• Duration• Outcome(s)• Main findings (with relevant statistics)• Conclusion(s)

Produce a brief summary for the Heart Protection Study(6 to 8 short sentences)

Note which parts of the paper you have to read to produce your summary.

Heart Protection Study (Lancet.v360.pp7-22.6/7/2002)

• Randomised placebo-controlled trial • 20536 UK adults, aged 40 to 80, with CHD,

other occlusive arterial disease or diabetes. • Effect of Simvastatin 40mg vs placebo on

mortality, fatal and non-fatal vascular events. • 5 years follow-up

• All cause mortality reduced in the simvastatin group1328/10269 (12.9%) vs 1507/10267 (14.7%); p=0.0003; RR 0.87 (0.81-0.94); NNT=55

• First vascular event rate reduced in the simvastatin group2033/10269 (19.8%) vs 2585/10267 (25.2%); p<0.0001; RR 0.76 (0.72-0.81); NNT=18.

• No significant harms identified

• All people similar to the study population should be treated with Simvastatin 40mg

How much of the paper have we actually read to get to this summary?

How to read a (clinical research) paper

• Scan abstract for a few seconds– Are the authors conclusions of interest?– Briefly assess study design– Briefly assess statistical precision of results– Formulate a brief summary

• Critically appraise methods & results sections for validity

• Critically appraise results section (especially the tables and figures) for relevance

• Draw your own conclusions about clinical applicability

Flow Diagram for a RCT / cohort study1 Selection and sampling 4 Outcomes2 Allocation (with or without Randomisation) 5 Analysis3 Follow-up

Validity- External Validity

To whom do the results of this trial apply?

Can the results be reasonably applied to a definable group of patients in a particular clinical setting in routine practice?

Are the results generalisable beyond the trial setting?

Appraisal of External Validity

• Where were the participants recruited from (primary care / referral centre)?

• Do the inclusion and exclusion criteria make sense?

• What proportion of the screened population was recruited?

Where were the participants recruited from?

Where were the participants recruited from?

Methods: Recruitment p8• 69 UK hospitals

Inclusion Criteria:

Inclusion Criteria:

• Methods: Eligibility p8 • Men and women aged 40 to 80 +• Blood total cholesterol >= 3.5mmol/L +• Past medical history of any one or more of

CHD, CVA, TIA, PVD, DMOR• Men, 65 to 80, treated for hypertension

Exclusion criteria:

Exclusion criteria:

1. Anyone already on a statin or Dr considered statin to be clearly indicated.

2. Contraindications – Chronic liver disease– ALT >67 IU/L (1.5 x ULN)– Child-bearing potential

3. Conditions requiring a dose reduction– Severe renal disease– Creatinine >200 mmol/L

4. Interactions– Treatment with ciclosporin,

fibrates, niacin

5. Conditions similar to known unwanted effects– Inflammatory muscle disease– CK >750 IU/L (3 x ULN)

6. Patient unlikely to survive 5 years follow-up– Severe heart failure– Another life threatening

condition7. Conditions limiting

compliance – Severely disabling stroke – Dementia

What proportion of the screened population was recruited?

What proportion of the screened population was recruited?Results: patient enrolment & Fig.1 p10 • 49% (31458/63603) of screened population excluded or refused

We are not given a break-down of the reasons • 36% (11609/32145) of population accepted for run-in were not

subsequently randomised.– 26% chose not to enter or “did not seem likely to be compliant for 5

years”– 5% considered to have clear indication for statin– 3% raised ALT, CK or Creatinine at pre-treatment screen– 2% attributed various problems to run-in treatment– 1% cholesterol <3.5mmol/L

• Only 32% (20536/63603) of screened population were randomised.

Validity - Internal Validity

The extent to which the observed difference in outcomes between the two comparison groups can be attributed to the intervention rather than other factors.

What are the possible causes of an “effect” in a RCT?

What are the possible causes of an “effect” in a RCT?

• Bias• Placebo• Chance• Real effect

Bias• Allocation (Selection) Bias – Failure of randomisation

Systematic differences in comparison groups • Performance Bias

Systematic differences in interventions received by the two groups

• Attrition Bias

Systematic differences in withdrawals from the trial • Detection (Measurement) Bias – Failure of blinding

Systematic differences in outcome assessment

Internal Validity - Sources of Bias in a RCT2 Allocation Bias (Failure of Randomisation)3 Follow-up – Performance Bias and Attrition Bias4 Outcomes – Detection Bias (Failure of Blinding)

CONSORT definition:

Selection bias—a systematic error in creating intervention groups, causing them to differ with respect to prognosis. The groups differ in measured or unmeasured baseline characteristics because of the way in which participants were selected for the study or assigned to their study groups.

Confounding—a situation in which the estimated intervention effect is biased because of some difference between the comparison groups apart from the planned interventions - such as baseline characteristics, prognostic factors, or concomitant interventions. For a factor to be a confounder, it must differ between the comparison groups and predict the outcome of interest.

Comparison of Cohort and RCT

Cohort• Population diverse• Allocation by clinical decision• Outcomes can be defined

retrospectively• Outcomes may be rare• Follow-up may be

retrospective and may be long-term

• Analysis complex multivariate

RCT• Population highly selected• Allocation by chance• Outcomes defined

prospectively• Outcomes must be common• Follow-up pre-determined

and usually short-term• Analysis relatively simple

Possible types of comparisons in cohort study

• General population – Intervention v alternative intervention– Intervention v no intervention

• Restricted population – Intervention v alternative intervention– Intervention v no intervention

Do patients who receive atypical antipsychotic drugs have an increased risk of hip fracture?

All older people Older people with dementia

Atypical antipsychotic

(n=34 960)

No Intervention (n=1 251 435)

Atypical antipsychotic

(n=21 427)

No intervention (n=58 754)

Mean (SD) age 80.46 (7.63) 74.50 (6.58) 81.69 (7.11) 80.95 (7.64)

No (%) with dementia

21 427 (61.3) 58 754 (4.7) 21 427 (100) 58 754 (100)

Effect on age distribution and sample size of restricting comparison of atypical antipsychotic with no intervention to individuals with dementia

Consider the difference between the three sets of figures here:

Atypical antipsychotic (n=21 427)

No intervention (n=58 754)

Mean (SD) age 81.69 (7.11) 80.95 (7.64)

Mean (SD) age 81.69 (1.11) 80.95 (7.64)

Mean (SD) age 81.69 (7.11) 80.95 (1.64)

Selection / Allocation BiasAssessed by looking at the Table of Baseline Characteristics

Womens Health InitiativeJAMA v288, pp321-333, 17th July 2002

• A randomised placebo-controlled trial.• 16608 American women, aged 50-79, with intact

uterus.• Effect of conjugated equine oestrogens 0.625mg

od + medroxyprogesterone acetate 2.5mg od on incidence of CHD and Breast Cancer

• 8.5 years follow-up planned, but stopped after 5.2 years.

• CHD rate increased in oest+prog group

164/8506 (1.93%) vs 122/8102 (1.51%); RR 1.29 (1.02-1.63); ARI 0.42%; NNH 238

• Breast Ca rate increased in oest+prog group166/8506 (1.95%) vs 124/8102 (1.53%); RR 1.26 (1.00 – 1.59); ARI 0.42%; NNH 238

• Treatment group also had increased rate of venous thrombo-embolism and reduced rates of fractures and colo-rectal carcinoma.

• Overall long-term harms exceeded benefits751/8506 (8.83%) vs 623/8102 (7.69%); RR 1.15 (1.03-1.28); ARI 1.14%; NNH 88

• When prescribing combined HRT in the over-50’s short-term

benefits should be balanced by consideration of long-term harms.

Table of Baseline Characteristics (WHI)

• Are all important characteristics listed?• Are any of the differences between the treatment

and placebo groups statistically significant?• Are there any differences in the two groups that may

bias the results?• What age range includes 95% of the Placebo group?• Assuming HRT has no effect – which group would

you expect to have more heart attacks?• Does this introduce a bias?• If so, in which direction does it operate?

Performance Bias

• Contamination: Provision of the intervention to the control group

• Compliance: Poor compliance with the allocated intervention

• Co-interventions Provision of unintended additional interventions to either group

Attrition Bias

• Count (drop-out)

Loss to follow-up rate should not exceed outcome event rate and should be equal in all groups.

Detection (Measurement) Bias

Best: Double-blindBoth patient and investigator unaware of treatment allocation

Less important if outcome is objective (e.g. death)Critical if outcome is subjective

Impossible for some comparisons eg medical vs surgical intervention

What are the possible causes of an “effect” in a RCT?

• Bias• Placebo• Chance• Real effect

Placebo Effect

You can only know the size of a placebo effect if a placebo has been used!

Appraisal of Internal Validity

• Was assignment of patients to treatments randomised?

• Were groups similar at start of trial?• Were groups treated similarly, apart from the

experimental treatment?• Were all participants accounted for in the conclusions?• Were all participants analysed in the groups to which

they were randomised (Intention To Treat analysis)?• Were participants and clinicians kept “blind” to

treatment received?

Randomisation?(with allocation concealment) Best: Centralised computer randomisationShould be independent of investigators

Where: Methods

HPS Methods: Recruitment p8 “The central telephone randomisation system…”

Similar Groups?Table of baseline characteristics (Note p-values) Where: Results

HPS:

Methods: Recruitment p8“The central telephone randomisation system used a minimisation algorithm to balance the treatment groups with respect to eligibility criteria and other major prognostic factors”

Results: Patient Enrollment p10

“…good balance between the groups for the main pre-randomisation prognostic features…” But – there is no table of baseline characteristics!We are left to make our assessment from the numbers allocated to each sub-group in Fig.8 p16

Treated equally?

Where: Methods - for intended scheduleResults - for actual treatment

HPS:Methods: Recruitment p8

“..randomly allocated to receive 40mg simvastatin daily or matching placebo tablets in specially prepared calendar packs..”

HPS:Results: Compliance p11

Placebo-allocated group more likely to be prescribed a non-study statin (32% vs 5% at end of study).Averaged over the 5 years of study 85% of simvastatin group and 17% of placebo group received a statin.

Non-equal treatment – Classic example

1948 – Trial of Vitamin E in pre-term infants

Vitamin E prevented retrolental fibroplasia

(by removal from 100% Oxygen to give the frequent doses of Vit E)

Loss to follow-up

Rough guide: 5% - OK >20% - validity doubtfulANDMust not exceed outcome event rate

Where: Results

HPS – Loss to Follow-up

Simvastatin Placebo

Mortality 0.03% 0.04%

Morbidity 0.33% 0.25%

Total 0.36% 0.29%

Outcome ER 19.8% 25.2%

Results: Fig.1 p10

Intention-to-treat analysis

Maintains the randomisation

Where: Results

HPS:Summary: Methods p7

“Analyses…compare all simvastatin-allocated versus all placebo-allocated participants. These ‘intention-to-treat’ comparisons…”

Results: Compliance p11

Blinding?Best: Double-blindWhere: Methods

HPS:Does not mention “blinding”

But we are given some suggestion the randomisation process kept the allocation concealed from both patients and investigators.

Methods: Recruitment p9“…matching placebo tablets in specially prepared calendar packs.”

Methods: Follow-up p9

“….coordinating centre clinical staff…were kept unaware of the study treatment allocation.”

How to read a (clinical research) paper

• Scan abstract for a few seconds– Are the authors conclusions of interest?– Briefly assess study design– Briefly assess statistical precision of results– Formulate a brief summary

• Critically appraise methods & results sections for validity

• Critically appraise results section (especially the tables and figures) for relevance

• Draw your own conclusions about clinical applicability

Appraisal of Relevance / Impact

• Were all the outcomes studied important?• Were all the important outcomes studied?• Was sub-group analysis pre-planned?• Could the treatment effect have arisen by

chance? • How large was the treatment effect?

HPS: Outcomes

• Primary Outcomes– Mortality– Non-fatal vascular events

• MI, CVA, Revascularisation

• Secondary Outcomes– Cancer– Other major morbidity

Where: Summary

Was sub-group analysis pre-planned?

Where: Methods, Statistical Analysis“The data analysis plan was prespecified…before any analysis of the effects of treatment were available…”

Could the treatment effect have arisen by chance?

p-valuesStatistical test of the (“null”) hypothesis that the intervention had no effectIf p<0.05 result is statistically significanti.e. the effect would occur by chance less than 5% of the timeThe smaller the p-value the less likely is the effect to occur by chance

Confidence Intervals (95%)

Range of values that has a 95% chance of including the true value

How large was the treatment effect?

RR RRR ARR NNT

Death 0.87 0.13 1.8% 55

Event 0.76 0.24 5.4% 18

Expressions of RiskIn the study population, treatment with Simvastatin 40mg for 5 years, compared with placebo, resulted in: • A 13% reduction in risk of death• A 24% reduction in risk of a major vascular event • A 1.8% reduction in deaths• A 5.4% reduction in major vascular events

• We need to treat 55 people to defer one death• We need to treat 18 people to prevent / defer a major

vascular event.

Expressions of Risk

• Relative Risk (RR) = EER / CER• Relative Risk Reduction (RRR) = 1 – RR• Absolute Risk Reduction (ARR) = CER – EER• Numbers Needed to Treat (NNT) = 1/ARR

EER = Experimental Event Rate CER = Control Event Rate

Relative Risk

RR & RRR may remain constant despite huge differences in absolute event rates

They are useful to determine whether a

biological effect exists ….

BUTThey do not discriminate between huge treatment effects and trivial ones.

CER EER RRR

0.16 0.10 37.5%

0.016 0.010 37.5%

0.0016 0.0010 37.5%

Absolute Risk Reduction

ARR reflects baseline risk and does discriminate between huge and trivial treatment effects.

CER EER RRR ARR

0.16 0.10 37.5% 6%

0.016 0.010 37.5% 0.6%

0.0016 0.0010 37.5% 0.06%

Numbers Needed to Treat

The number of people you need to treat for one of them to have the desired outcome over a specified period of time.

A good measure of clinical relevance.

Allows calculation of cost per desired outcome.

CER EER RRR ARR NNT

0.16 0.10 37.5% 6% 16.7

0.016 0.010 37.5% 0.6% 167

0.0016 0.0010 37.5% 0.06% 1667

NNT for various CER’s and RRR’s

RRRCER 50% 40% 30% 20% 10%0.9 2 3 4 6 110.3 7 8 11 17 330.1 20 25 33 50 1000.01 200 250 333 500 10000.001 2000 2500 3333 5000 10000

Note that a small RRR for a condition with a high CER is more clinically important than a large RRR for a condition with a low CER

Cost: HPS

Can be calculated from NNT

• It costs (£25 x 12 x 5 x 55) £82500 to defer one death.

• It costs (£25 x 12 x 5 x 18) £27000 to prevent / defer one major vascular event.

Women’s Health Initiative (WHI)

Estrogen + Progestin

Placebo

Women (n) 8506 8102

Fractures ~5 yrs 650 788

Rate (annualised) 0.0147 0.0191

Can you calculate RR, RRR, ARR and NNT?

Women’s Health Initiative (WHI)

Estrogen + Progestin

Placebo

Women (n) 8506 8102

Fractures ~5 yrs 650 788

Rate (annualised) 0.0147 0.0191

RR = EER/CER = 0.0147/0.0191 = 0.77RRR = 1-RR = 1 – 0.77 = 0.23ARR = CER–EER = 0.0191 – 0.0147 = 0.0044NNT = 1/ARR = 1/0.0044 = 227

Women’s Health Initiative (WHI)

HRT for one year:• Reduces the risk of fracture by 23%• Reduces the number of fractures by 0.44%• We need to treat 227 post-menopausal

women for one year to prevent one fracture

At ~£20/month (20x12x227) = £54480 per fracture prevented

Number Needed to Harm

Estrogen + Progestin

Placebo

Women (n) 8506 8102

Breast Cancer ~5 yrs 166 124

Rate (annualised) 0.0038 0.0030

Can you calculate RR, RRI, ARI, NNH?

Number Needed to Harm

Estrogen + Progestin

Placebo

Women (n) 8506 8102

Breast Cancer ~5 yrs 166 124

Rate (annualised) 0.0038 0.0030

RR = EER/CER = 0.0038/0.0030 = 1.27RRI = RR-1 = 1.27-1 = 0.27ARI = EER-CER = 0.0038-0.0030 = 0.0008NNH = 1/ARI = 1250

Women’s Health Initiative (WHI)

HRT for one year:• Increases the risk of breast cancer by 27%• Increases the number of breast cancers by

0.08%• We need to treat 1250 post-menopausal

women for one year to give one of them breast cancer.

How to read a (clinical research) paper

• Scan abstract for a few seconds– Are the authors conclusions of interest?– Briefly assess study design– Briefly assess statistical precision of results– Formulate a brief summary

• Critically appraise methods & results sections for validity

• Critically appraise results section (especially the tables and figures) for relevance

• Draw your own conclusions about clinical applicability

Appraising Applicability

• Is my patient similar to the study population?• Is the treatment feasible in my clinical setting?• Will potential benefits of treatment

outweigh potential harms of treatment for my patient?

Outcomes with CER 25% and NNT=20

www.nntonline.net

CURE. NEJM 2001 v345 p394

Worse with Rx

Discuss the evidence for and against the following conclusions regarding clinical applicability in relation to

the Heart Protection Study:

Group 1

• There is no need to check LFT’s prior to commencing

statin therapy.

• There is no need to check LFT’s during statin therapy.

• There is no need to check CK prior to commencing statin therapy.

• There is no need to check CK during statin therapy.

Group 2

• All people with diabetes should be treated

with a statin. • All men >65 who are treated for hypertension

should be treated with a statin.

Group 3

• Amongst people similar to the study population

there is no need to test blood cholesterol levels prior to commencing a statin.

• Once a decision has been taken to start statin

treatment there is no need to monitor blood cholesterol levels.

Group 4

• Statins should be stopped at age 80.

• Women benefit from statin treatment to the same extent as men.

• Once started statin treatment should be continued indefinitely.

Group 1

• There is no need to check LFT’s prior to commencing

statin therapy.

• There is no need to check LFT’s during statin therapy.

• There is no need to check CK prior to commencing statin therapy.

• There is no need to check CK during statin therapy.

• People with raised ALT and CK were excluded from the study.

• We are not told how many screened people were excluded for this reason, but 3% of pre-randomisation run-in group were excluded for raised ALT, CK or Creatinine (Results para1 p10)

Simva Placebo ARI NNH pALT 2-4x 1.35% 1.28% 0.07% 1428ALT >4x 0.42% 0.31% 0.11% 909CK 4-10x 0.19% 0.13% 0.06% 1667CK >10x 0.11% 0.06% 0.05% 2000Myo 0.05% 0.01% 0.04% 2500 0.2Rhabdo 0.05% 0.03% 0.02% 5000

Persistant ALT >4x

0.09% 0.04% 0.05% 2000 0.3

Persistant CK >4x

0.07% 0.01% 0.06% 1667 0.07

• There is good evidence that it is not necessary to

monitor LFT’s or CK in follow-up of Simvastatin 40mg treatment in the absence of relevant symptoms.

• The low (not statistically significant) attributable risk of

persistently raised ALT or CK due to Simvastatin 40mg makes it improbable that pre-treatment testing will be of value to the patient.

• This study does not provide evidence about generalisability to other statins (or other doses of simvastatin).

Group 2

• All people with diabetes should be treated

with a statin. • All men >65 who are treated for hypertension

should also be treated with a statin.

Event rate

Simva Placebo ARR NNT

Diabetes 20.2% 25.1% 4.9% 20

There is good evidence that people with diabetes aged 40 to 80, with total cholesterol >=3.5 mmol/L should be offered Simvastatin 40mg.

• Only 1% of the trial population was male >65 on treatment for hypertension and had no history of vascular disease or diabetes. This subgroup was not separately analysed (or if it was analysed it was not reported).

• This study provides no evidence concerning the benefits treatment for this subgroup

Group 3

• Amongst people similar to the study population

there is no need to test blood cholesterol levels prior to commencing a statin.

• Once a decision has been taken to start statin

treatment there is no need to monitor blood cholesterol levels.

• Patients with Total Cholesterol <3.5 mmol/L were

excluded. • Outcome event rate benefit from simvastatin

40mg did not vary with starting cholesterol level or pre-randomisation LDL response.

• But the study does not tell us whether much

larger reductions in LDL cholesterol would give rise to larger reductions in outcome event rates.

• This study provides no evidence of need to check cholesterol levels during treatment with simvastatin 40mg.

• This information needs to be taken in the context that other studies have reported greater benefit for greater reduction in cholesterol levels, and QOF provides a financial incentive to treat to a target level.

Group 4

• Statins should be stopped at age 80.

• Women benefit from statin treatment to the same extent as men.

• Once started statin treatment should be continued indefinitely.

Trial population were aged 40 to 80 at outset – so the oldest were 85 at end of trial.

Event rate Simva Placebo ARR NNT<65 16.9% 22.1% 5.2% 1965 – 69 20.9% 27.2% 6.3% 16>70 23.6% 28.7% 5.1% 20

Considerable overlap of confidence intervals of event rate ratios for different age subgroups suggests no statistically significant difference in treatment effect with age within the age groups studied. Trend chi2 0.73 (nb 3.84 ~ p<0.05)

Note: 67% of trial participants were male

Event rate Simva Placebo ARR NNTMale 21.6% 27.6% 6.0% 17Female 14.4% 17.7% 3.3% 30

Note the pattern of overlap of confidence intervalsHeterogeneity chi2 0.76 The actual results tell us that there is a difference in benefit between male and female, but the statistical tests tell us that this difference is not significant.

• Study duration was 5 years, so strictly it only informs us of benefits and harms over the first 5 years of treatment.

• But figures 5 & 6 give some information which allows us to predict that benefits would continue for longer than 5 years – how much longer is a matter of opinion / judgment.

Event rateYear simva placebo ARR NNT1 4.7% 5.1% 0.4% 2502 3.9% 5.6% 1.7% 593 3.9% 5.6% 1.7% 594 3.8% 5.2% 1.4% 715+ 5.8% 7.3% 1.5% 67

Event rate

Diff in Adjusted

Year ARR Statin use

ARR NNT

1 0.4% 85% 0.5% 2002 1.7% 76% 2.2% 453 1.7% 67% 2.5% 404 1.4% 59% 2.4% 425+ 1.5% 50% 3.0% 33

• Before we can apply the results of any individual

primary study we have to place it in the context of all other relevant research.

• > Systematic Reviews, Guidelines

• What is already known• What this study adds• Has anything been added since this study?

Levels of Evidence:

I Systematic Review of all relevant RCT’sII At least one good quality RCTIIIGood non-randomised trials; cohort or case-

control studies.IV‘Expert’ opinion

Further Reading

• Cochrane handbook for systematic reviews of interventions http://www.cochrane.org/resources/handbook/Handbook4.2.6Sep2006.pdf

• Consort Statements http://www.consort-statement.org/?o=1001

• Bandolier• BMJ

Odds Ratios Odds of an event = no. of events / no. of non-events e.g. 51 boys/100 birthsOdds of boy = 51/49 = 1.04 Odds >1 means event more likely to happen than notOdds of an impossibility are zeroOdds of a certainty are infinity

Odds ratio = odds in intervention group / odds in control group

Odds Ratios (cont.)

When events are rare Odds and Risk are similarOR and RR are similar

As prevalence (control event rate) and OR increase the error in using OR as an approximation for RR becomes unacceptable.

In the simvastatin group: the odds of an event are 2033/8236 = 0.247 or 24.7%the risk of an event is 2033/10269 = 0.198 or 19.8% In the placebo group:The odds of an event are 2585/7684 = 0.336 or 33.6%The risk of an event is 2585/10267 = 0.252 or 25.2%

Event No EventSimvastatin 2033 8236 10269Placebo 2585 7684 10267

The odds ratio (=relative odds) is 0.247/0.336 = 0.735 or 73.5%The risk ratio (=relative risk) is 0.198/0.252 = 0.786 or 78.6% An absolute difference of 51:1000 and a relative error of ~7%

In HPS:

Odds ratios are used because of their superior mathematical properties:

• They can always take values between 0 and infinity; values RR can take are dependent on CER.

• With OR’s the relationship between the two possible outcomes (event or not event) is reciprocal – not so for RR.

• OR’s always used in case-control studies where disease prevalence is not known.

• When adjusting for confounding factors which affect event rates, logistic regression models (the correct approach) use odds and report effects as odds ratios.