1 Clinical Epidemiology Using Evidence to Guide Practice A bit of thinking and a few simple sums

1

Clinical EpidemiologyUsing Evidence to Guide Practice

A bit of thinking and a few simple sums

2

What you need to know for the MRCGP

“Sums”• ARR, RRR, NNT (calculate) plus OR• Impact of baseline risks• P values, Confidence Intervals (principles, not how to calculate)• Power (principles) • Diagnosis and screening

– Sensitivity, specificity, PPV, NPV (LRs unlikely?)

Concepts• Types of studies, including SR & MA, qualitative research

(principles), health economics (principles)• Where to find evidence (in the real world)• How to use EBM in the consultation

3

Who likes shopping?(Oh no, I really hate that)

Apples – were £30 a bag, now only £20 a bag.

Would you go out and buy apples if the saving was ONLY described as “ONE THIRD OFF”?

Apples – 3p a bag, now 2p a bag1. Saving is 1p a bag2. Saving is STILL one third

1. Saving is £10 per bag (Original rate – new rate).

2. Saving is one third or 33%. (original rate – new rate / original rate; i.e. 3-2 = 1, 1/3 = one third, 1/3 x 100 = 33%

4

Describing differences between treatments

• In a RCT, 50% of people died using medicine A. • Only 45% of people died when they were given medicine B.

– How much better is B than A?

– Does it matter how we describe those differences?

– What is the best way, or the “fairest way”, of describing differences?

5

In a RCT, 50% of people died using medicine A. Only 45% of people died when they were given medicine B.

1. The difference is 5%.

(Control rate – experimental rate = 50% - 45% = 5%.)

Absolute risk reduction (ARR) or risk difference


Control rate – experimental rate / control rate;

50% - 45% = 5% / 50% = 1/10 = 10%.

Relative risk reduction (RRR) or risk ratio

Which of these is “best” or “fairer”?

6

Same medicines, different people

In a RCT, 5% of people died using medicine A. Only 4.5% of people died when they were given medicine B.

1. The difference is 0.5%.

(Control rate – experimental rate = 5% - 4.5% = 0.5%.)

ARR = 0.5%.


Control rate – experimental rate / control rate;

5% - 4.5% = 0.5% / 5% = 1/10 = 10%.

RRR = 10%.

7

Imagine two trials…

8

So let’s get this straight

• The RRR stays constant in different populations.• The ARR alters in different populations – it will be

much more impressive if the population has a lot of events – i.e. “has a high baseline risk”.

• But if there are not many events then a 10%, 20% or even a 30% reduction in a rare event doesn’t amount to much benefit. And EVERYBODY has to take the intervention (and so is at risk of side effects).

9

Numbers needed to treat (NNT)

Medicine A cures 50% of people

Medicine B cures 60% of people

ARR = 10%

RRR = 20%

Another way of looking at the absolute rate is to divide it into 100:

• In this case 100/10 = 10.• i.e. treat ten people with B rather than A and 1 will benefit.

10

Same medicines, different people again

In RCT 1:• 50% of people died

using medicine A• Only 45% of people

died using medicine B.• ARR 5%, RRR 10%,

NNT 20.

In RCT 2:

• 5% of people died using medicine A.

• Only 4.5% of people died using medicine B.

• ARR = 0.5%. RRR = 10%. NNT = 200.

In the higher risk population, we

would only need to treat 20 people with B rather than

A to save one.

But in the lower risk population

we would need to treat 200 people

with B rather than A to save one.

11

Effect of baseline risk on ARR

0%

2%

1%

Event = a coronary death or a non-fatal MI

0% 6%3%1% 2% 4% 5%

Baseline annual risk of an event

ARR4S

CARE

LIPID

WOSCOPS

ACTC

12

Mini-test (1)

Calculate the ARRs, RRRs and NNTs for these trials:

1. Medicine A 15% have an MI, Medicine B 12% have an MI.

2. Medicine A 7% have an epileptic fit, Medicine B 5% have an epileptic fit.

3. Medicine A 12% develop diabetic retinopathy, Medicine B 6% develop diabetic retinopathy.

4. Medicine A 27% are readmitted with heart failure, Medicine B, 24% are readmitted with heart failure.

13

Mini-test (1) answers

Calculate the ARRs, RRRs and NNTs for these trials:

ARR RRR NNT

1. 3% 20% 33

2. 2% 29% 50

3. 6% 50% 17

4. 3% 11% 33

14

Clopidogrel in ACS

• It’s really beneficial; I’d want all of my patients to be taking it

• Most people do fine just on aspirin. Adding clopidogrel prevents only a few people having an event and there’s the increased bleeding risk

• The specialist starts it and I don’t question that

15 N Engl J Med 2001; 345: 494-502.

20% relative risk reduction2.1% absolute risk reductionNNT 48

38% relative risk increase1% absolute risk increaseNNT 100

16

CURECURE Conclusions

Clopidogrel significantly reduces the risk of:

a) CV Death, MI, Stroke by about one-fifth (p < 0.001)

b) CV Death, MI, Stroke, and Refractory Ischemia by about one-sixth (p < 0.001)

c) Early revascularization, severe and recurrentischemia and heart failure by about one-fifth to one-quarter in hospital

There is a small (absolute 1%) significant excess of major, but not life threatening, bleeds

Translation: Clopidogrel significantly reduces the absolute risk of:

a) CV Death, MI, Stroke taken together by 2.1% (p < 0.001) NNT 48

b) CV Death, MI, Stroke, and Refractory Ischaemia taken together by 2.3% (p < 0.001) NNT 43

Most benefit is achieved by 30 days with MI

There is no effect on all cause mortality

There is a large (relative 38%) significant excess of major bleeds

17

This is appallingNuovo J, et al. JAMA 2002; 287: 2813 –4.

• Ann Intern Med, BMJ, JAMA, Lancet, NEJM• 1989, 1992, 1995, 1998.• Treatment RCTs• 359 eligible articles. • NNT reported in 8 (6 of these in 1998)• ARR reported in 18 (10 of these in 1998).

Put another way, 93% of all RCTs only report relative risk.

18

Odds ratios or relative risks?Macfarlane J et al. BMJ 2002; 13: 105-9

Patients who took antibiotics

Patients who did not take antibiotics

TOTAL

Patients who were given a

leaflet

49 55 104

Patients not given a leaflet

63 38 101

TOTAL 112 93 205

19



TOTAL

Patients who were given a leaflet

49 55 104


63 38 101

TOTAL 112 93 205

Relative risk: (49/104) / (63/101) = 0.76.i.e the relative risk of patients taking an antibiotic if they were given a leaflet is reduced by 24%. (Also called risk ratio)

20

Odds ratio: (49/55) / (63/38) = 0.54.There was a 46% reduction in the ratio of those taking antibiotics who had a leaflet compared with the ratio of those taking antibiotics who did not have a leaflet.



TOTAL


49 55 104


63 38 101

TOTAL 112 93 205

21

Absolute risk reduction: (63/101) – (49/104) = 0.15.Also known as the risk difference. i.e. the difference in the riskof taking antibiotics depending on whether a leaflet was usedor not.



TOTAL


49 55 104


63 38 101

TOTAL 112 93 205

22

NNT: 1 / 0.15 = 7. i.e. 7 people need to be given a leafletIn order for 1 additional person not to take antibiotics



TOTAL


49 55 104


63 38 101

TOTAL 112 93 205

23


• ARR, RRR, NNT (calculate) – plus OR• Impact of baseline risks• P values, Confidence Intervals (principles, not how to

calculate)• Power (principles) • Diagnosis and screening

– Sensitivity, specificity, PPV, NPV (LRs unlikely)

• Types of studies, including SR & MA, qualitative research (principles), health economics (principles)

• Where to find evidence (in the real world)• How to use EBM in the consultation

24

matters

25

How does the size of the study affect things?

Counsell CE, et al. BMJ 1994 309: 1677-1681. [Bandolier Nov 2002]

• Investigators used a dice to simulate outcomes in a trial

• ‘Treatment’ arm vs. control arm• Roll of a dice = outcome in the trial:

1-5 survival 6 = death

• Did for ‘treatment’ group then repeated for control group

• Number of times the dice was rolled varied from 5 to 100.

26

But it’s the size that matters

Results according to number of times the dice was rolled:

• Variation in ‘outcome’ was largest in the ‘smallest’ studies

• i.e the chance of a spurious result decreased with increasing numbers included in the trial

More consistency in results

Wide variation in results

27

How good is the evidence for the management of schizophrenia

Thornley B, et al. BMJ 1998; 317: 1181-84

Size of trials (n=1941; 59 studies did not report study size)

28

Sub-group analyses – caveat emptor

ISIS 2 trial:• 17,187 patients, 417 hospitals up to 24 hours after MI. • Randomised to either streptokinase, aspirin or placebo in 2x2

factorial design• Streptokinase alone and aspirin alone each produced a highly

significant reduction in 5-week vascular mortality: ARR 2,8%, together ARR vs double placebo 5.2%.

• To try and allay concerns re benefit:safety ratio in subgroups, the Lancet pushed for subgroup analyses.

• The authors agreed – but with the proviso that they should analyse by astrological star signs and that this should appear first in the table of subgroup results.

• The result?Gemini and Libra: aspirin of no benefit.

All other star signs: aspirin strongly beneficial

29





• Types of studies, including SR & MA, and qualitative research (principles)


30

P< 0.05

The Shrine of

Statistics

The Sacred P-Value

31

Are you happy with 1 in 20?

32

Did he just say P =

0.05 ????

P = 0.027 means…… this result occurs

BY CHANCE 1 time in 36;

If P = 0.0001, 1 time in 10,000by chance

33

Confidence intervals are the range of

values between which we could be 95%

certain that this result would lie if this

intervention was applied to the general

population

34

Confidence IntervalsEstrogen Replacement Therapy in Women with a History of

Proliferative Breast Disease

35

0.910.68 1.2

Risk could be this low

Risk could be this high

95% C.I.

Since the 95% CI crosses 1.0, the difference is not significant

1.0

Confidence intervals are the range of values between

which we could be 95% certain that this result would lie

if this intervention was applied to the general

population

36

6 month data: Incidence rate per year

Celecoxib NSAIDs NNT

All patients:

upper GI ulcer complications alone 0.76% 1.45% (P=0.09) -

combined with symptomatic ulcers 2.08% 3.54% (P=0.02) 68

For patients not taking aspirin:

upper GI ulcer complications alone 0.44% 1.27% (P = 0.04) 121

combined with symptomatic ulcers 1.40% 2.91% (P= 0.02) 66

For patients taking aspirin:

upper GI ulcer complications alone 2.01% 2.12% (P = 0.92) -

combined with symptomatic ulcers 4.70% 6.00% (P = 0.49). -

Silverstein FE, et al. JAMA 2000; 284: 1247-1255

37

Kaplan-Meier estimates for ulcer complications according to traditional definition

Jüni P, et al. BMJ 2002; 324: 1287-1288

38







39

0 10

20

30

NoDisease A

BPercentof population

VALUEArbitrary Units

Set cut off at A A lot of people who do not have the disease arelabeled as having it (false positives)

Set cut off at B A lot of people who do have the disease arelabeled as not having it (false negatives)

Disease

C

40

How many people in the study?

Positive

Negative

Test

Disease

Present Absent

100

41

How many had the disease?

Positive

Negative

Test

Disease

Present Absent

50 10050

42

How many with the disease had a positive test?How many without the disease had a negative

test?

Positive

Negative

Test

Disease

5

45

45

5

Present Absent

50 50 100

50

50

43

What was the prevalence of disease in those tested?

Positive

Negative

Test

Disease

5

45

45

5

Present Absent

Prevalence = 50/100= 50%

50 50 100

50

50

44

So …………Sensitivity and specificity

Positive

Negative

Test

Disease

5

45

45

5

Present Absent

5/50 false negatives;i.e. sensitivity = 45/50

=90%

5/50 false positives;i.e. specificity = 45/50

= 90%

50 50 100

50

50

45

Positive Predictive Value and Negative Predictive Value

Positive

Negative

Test

Disease

5

45

45

5

Present AbsentPPV 45/50

= 90%

NPV 45/50= 90%

50 50 100

50

50

46

Watch what happens when the prevalence drops to 10%NB. PLEASE remember this bit!!!!!!!

Positive

Negative

Test

Disease

1

9

81

9

Present Absent

Prevalence = 10/100= 10%

1/10 false negatives;i.e. sensitivity = 9/10

=90%

9/90 false positives;i.e. specificity = 81/90

= 90%

PPV 9/18= 50%

NPV 81/82= 99%

10 90 100

18

82

47

H Pylori infection in a population with a 25% prevalenceMeReC Bulletin 2001; 12 (1): 1-4

646945475.586Near-patient serological tests

32597759091Laboratory serological tests

112998895.597.5Breath test (14C)

11199899696.5Breath test (13C)

False Negative results (%)

False positive results (%)

Negative predictive value (%)

Positive predictive value (%)

Specificity (%)

Sensitivity (%)

48

So what does all this mean?

• In primary care many people have a low chance of having the disease they are being tested for.

• If they get a positive test then they may have the disease – or it could be a false positive. They may need more tests to sort out whether they truly, truly have the disease.

• (But what will the patient think when you tell them their initial test indicates they may have something and they need further tests?)

• If they get a negative test, and they are unlikely to have the disease, then it’s really very unlikely that they have it when they have tested negative.

• And MOST IMPORTANTLY, try only to test people for anything if they are in a high risk group for having the disease. Testing lots of people will do more harm than good.

49


“Sums”• ARR, RRR, NNT (calculate) – plus OR• Impact of baseline risks• P values, Confidence Intervals (principles, not how to calculate)• Power (principles) • Diagnosis and screening


Concepts• Types of studies, including SR & MA, qualitative research

(principles), health economics (principles)• Where to find evidence (in the real world)• How to use EBM in the consultation

50

My brain hurts

Mr T F Gumby

51

Take a break

52







53

Three Steps to (primary care consultation)

Heaven

Now there are Three Steps To HeavenJust listen and you will plainly seeAnd as life travels onAnd things do go wrongJust follow steps one, two and three

54

Acknowledgements

• Prof Allen Shaughnessay, Tufts University, Boston• Prof Dave Slawson, University of Virginia,

Charlottesville• Prof Trisha Greenhalgh, University College, London

55

Step 1

• Make the best attempt at an initial (often provisional) diagnosis

– open questions– clarify ideas, concerns and expectations (Can work in biomedical, anticipatory and hermeneutic relationships with patients)– examine appropriately – exclude ‘red flags’– Beware of the limits of pattern recognition; use Bayesian approaches where possible

Step one - you find a girl to love

56

Development of Medical Expertise Psychology of Medical Decision Making: Vanderbilt University.

Stage 1. Elaborated causal networks– Learned during the basic science years– Facts and relationships– Nodes and links– Causal models of disease processes

57

Target condition

Anatomy, physiology, biochemistry

Epidemiology, risk factors

Pathophysiology

Clinical features-Forceful features

-relevant cues

Clinical management options-Acute

- chronic

Consultation skills

58

Stage 2. Compilation of abridged networks

– Starts when exposed to real patients– Knowledge gets compiled (rewritten,

automated)

simplified causal modelsexplain signs and symptomsassociated with diagnostic labels

59

Stage 3. Illness Scripts.– Based on repeated experience with patients– Illness scripts are sufficient to diagnose and

treat diseases.

Lists of features that characterize the disease

Specification of what to do

Information about context

Information about temporal features of disease

60

Stage 4. Cases.• Patient encounters stored as instance scripts.

– Based on long experience– Physician remembers many individual patients– Each has a different variant of the disease– New (or newly sick) patients are

recognized as “similar to Patient X”

treated as Patient X was treated

61

Making a diagnosisElstein AS and Schwarz A. BMJ 2002; 324: 729–732

• Problem solving – Testing hypotheses (novices), (complex)

Failure to generate correct hypothesisdata collected thoroughly but ignore /

misunderstand / misinterprettoo economical, but interpret accurately what

is available

– Pattern recognition (experts), (familiar)Mental models

– Neither are proof against error (see Klein, BMJ 2005; 330: 781–784)

• Decision making – Pre test probability, LR, post test probability (see Gill CJ, et al. BMJ

2005; 330: 1080–1083.)

– A minority sport (consciously)

62

63

Learn the basic patterns

64

Then can see them in new situations.

65

66

Dwivedi G, et al. BMJ 2006; 332: 406

• Mrs Patel (28 years, 7 month old baby) presents as an emergency with a severe episode of R chest pain.

• Pain started 2 months ago, well localised over the R chest wall with no radiation. Precipitated by exertion, particularly by pushing daughter in pushchair, limits recreational activities with child

• Pain not exacerbated by deep inspiration, coughing or twisting; no dyspnoea, palpitations or dizziness.

• Previous consultation = musculoskeletal pain.• PMH: Type 2 diabetes 3 years, well controlled on oral

medication (HbA1c 7.5%); hypothryoidism, l-thyroxine 25micrograms daily

• Never smoked, no history of hypertension, lipid status unknown

67

• On examination– Obese (BMI 34.6)

– P 70/min, regular, BP 130/70

– All peripheral pulses palpable, no bruits

– Heart sounds are normal, no murmurs

– Chest clear

– No breast lumps or tenderness

– Abdo examination normal

– Resting ECG = sinus rhythm, poor R wave progression in anterior chest leads, inverted and biphasic T waves in V2 and V3.

68

• What’s the diagnosis?– Musculoskeletal?– Dyspepsia?– Pulmonary embolism?– Cholecystitis?– Ischaemic heart disease?– Something else?

69

• Mr Patel, 55 years, presents as an emergency with a severe episode of R chest pain.

• Pain started 2 months ago, well localised over the R chest wall with no radiation. Precipitated by exertion, particularly by pushing granddaughter in pushchair, limits recreational activities with child

• Pain not exacerbated by deep inspiration, coughing or twisting; no dyspnoea, palpitations or dizziness.

• Previous GP consultation = musculoskeletal pain.• PMH: Diabetes 3 years, well controlled on oral medication

(HbA1c 7.5%); hypothryoidism, l-thyroxine 25micrograms daily• Never smoked, no history of hypertension, lipid status

unknown

70

Mrs Patel• Seen and admitted by cardiologist• HB 13.1, ESR 20 (slightly raised), Biochemistry normal• D-Dimer normal (<250ng/ml)• TC 6, LDL 3.13, TG 3.47• TFTs: free T4 <5pmol/l, TSH 151mIU/l• Treadmill: 3min51sec Bruce protocol – chest discomfort,

maximum HR 155, no ECG changes indicative of ischaemia• Echo after exercise test = large area of reduced systolic wall

thickening affecting anterior septum and apex with obvious LV dilatation

• Angiogram = critical lesion proximal LAD artery. • Angioplasty + DE stent• Aspirin, clopidogrel and atorvastatin started, thyroxine dose

doubled• 6 months plus later is doing well

71

Has my neighbor won the lottery?

• Pre test probability – extremely low• Test 1 – my neighbour says he’s won – still low

likelihood (he’s a joker) LR+• Test 2 – (he says he’s won and) he disappears on

holiday for three weeks and comes back with QE2 stickers on his luggage– slightly more likely LR+

• Test 3 – (he says he’s won, he’s had an expensive holiday and) he pulls up in a gleaming new Bentley Continental – more and more likely LR+

• Test 4 – (he says he’s won, has had an expensive holiday and bought an expensive car and) he is no longer my neighbor, having bought a very big house in the country LR+

• Test 5 – (the police arrest him and charge him with a £50million gold bullion robbery) LR-

72

How many times more (or less) likely are you to have an MI if you develop chest pain and have these signs /

symptoms?

• Chest pain sharp or stabbing • Chest pain radiates to left arm• Chest pain most important symptom• Nausea or vomiting• Perspiring• BP < 80mmHg• Chest pain reproduced by palpation

e.g. 1 = no change in likelihood; 2 = twice as likely; 10 = ten times more likely; 0.5 = half as likely;

0.1 = ten time less likely etc.

73

You’ve just estimated likelihood ratios

• The likelihood ratio is the number of times more or less likely it is that someone with that symptom has the disease

• The greater the LR, the more likely the person is to have the disease– A +LR more than 10 tends to rule in disease (but beware if

the disease is very unlikely)

• The smaller the LR, the less likely person is to have the disease– A –LR less than 0.1 tends to rule out disease (but beware if

the disease is very likely)

74

Positive

Negative

Test

Disease

5

45

45

5

Present Absent

50 50 100

50

50

Positive Likelihood ratio:45/50 (= 0.90) divided by 5/50 (= 0.10). LR+ = 9

Likelihood ratios express how many more times (or less times) a test result is to be found in diseased people compared with non-diseased

people.

75

Positive

Negative

Test

Disease

5

45

45

5

Present Absent

50 50 100

50

50

Negative likelihood ratio:5/50 (= 0.10) divided by 45/50 (= 0.90). LR- = 0.11

76

You use LRs without knowing it…Gill CJ, et al. BMJ 2005; 330: 1080-1083

• Most symptoms have +LRs around 2 and –LRs around 0.5

• On their own they don’t add much• The clever bit is that you can link them together…

77

Is This Patient Having a Myocardial Infarction?Panju AA, et al. JAMA 1998; 280: 1256-1263

78

79

80

Step 1

Make the best attempt at an initial (often provisional) diagnosis

– open questions– clarify ideas, concerns and expectations– examine appropriately – exclude ‘red flags’– Beware of the limits of pattern recognition; use Bayesian approaches where possible

Step one - you find a girl to love

81

You have just moved to a new town and are looking for a GP practice for your own care and that of your new family. Two doctors are accepting patients:-

Jane completed her VTS six months ago. She gained the highest marks out of the 2000 candidates when she sat her MRCGP exam and won the Fraser Rose medal.

Susan completed her postgraduate training 10 years ago. She passed her MRCGP exam at the end of her VT with distinction.

• Who do you choose as your doctor?

82

Systematic review: the relationship between clinical experience and quality of health care

Choudhry NK, et al. Annals of Internal Medicine 2005; 142: 260–273

• 32 of the 62 evaluations (52%) reported decreased performance with increasing years in practice for all outcomes assessed.

• 13 (21%) reported decreasing performance with increasing experience for some outcomes but no association with other assessed.

• 1 (2%) reported increasing performance with increasing experience for all outcomes assessed.

83

Step 2

Be aware of the evidence base in relation to the possible management

options– balance efficacy, safety, cost, and patient factors– never ‘always’, and never ‘never’ – get used to not knowing everything and have a plan for finding the ‘best answer’– check regularly on the evidence for the management of the conditions seen often– be prepared to look up the evidence for the management of conditions not seen often

Step two – she falls in love with you

84

EFFECTIVE SAFE

COSTPATIENT

FACTORS

Benificence Non-malfeasance

Justice Patient autonomy

Barber N. BMJ 1995; 310: 923-925

85

QUALITY OF CARE

SYSTEMATIC APPLICATIONOF POLICY

Low

High

0% 100%

86

Chief Medical Officer Annual Report 2005

“The way in which clinical decisions are made, the extent to which they depart from research evidence, and the extent to which they depart from research evidence, and the factors that determine compliance with best practice have….been extensively studied….Despite this, the solution to the problem of clinical practice variation has not been found…..”

87

88

• Financial incentives• Guidelines• Data feedback• Academic detailing• Educational• Decision support (IT)• Audit• Opinion leaders• Media• Patient – mediated

89

Benefits of implementation strategiesGrimshaw J, et al. Journal of Continuing Education in the

health professions 2004; 24: S31-S37

• Changes in practitioner behaviour; in the desired direction, were reported in 86% of the comparisons made.

• The median effect size overall was approximately 10% improvement in absolute terms.

90

"We surveyed one acute medical take in our hospital. In a relatively quiet take, we saw 18 patients with a total of 44 diagnoses. The guidelines that the on call physician should have read remembered and applied correctly for those conditions came to 3679 pages. This number included only NICE, the Royal Colleges and major societies from the last 3 years. If it takes 2 min to read each page, the physician on call will have to spend 122h reading to keep abreast of the guidelines" (for one 24h on-call period).

Allen D, Harkins KJ. Lancet 2005; 365: 1768

91

• There are 1500 pages indexed in Medline each day.

Prof Sir JA Muir Gray Best Current Evidence Strategy.

March 2006

(So which ones will you choose to read?)

92

Abstracts lie (lots)Pitkin RM, et al JAMA 1999; 281: 1110-1111

• Random samples from 44 articles and their abstracts from Annals, BMJ, Lancet, JAMA, NEJM (12 months from July 1996), and 44 articles CMAJ (15months from July 1996) were compared with the original articles

• 19% of abstracts contained statements that were inconsistent with the full article

• 11% of abstracts contained statements that were not present in the full article

93

20% of RCTs don’t report all outcomes

Chan A-W, Altman DG. BMJ 2005; 330: 753-756

• 519 RCTs in 553 publications were examined for incompletely reported outcomes per trial. Original authors were surveyed (response rate 69%).

• 32% denied the existence of unreported outcomes when there was evidence to the contrary in their publications.

• On average, 20% of outcomes measured in RCTs were incompletely reported.

• These outcomes were more likely to be non significant (OR 2.0, 95% CI 1.6 to 2.7) for efficacy outcomes; OR 1.9, 95% CI 1.1 to 3.5 for harm outcomes.

94

95

• So you can’t keep up to date by reading primary research in journals.

• And if you do read some primary research in journals, there’s a decent chance what you read is poorly designed, poorly conducted and poorly reported.

• And you’ve only read that research – not all of the research which may be relevant. What if you’ve just read the one aberrant study and there are lots of others showing results that are the exact opposite – and you haven't seen those?

96

Clinician reading journals

Evidence-based treatment for my patient

97

Evidence based medicine1. Formulate question

2. Efficiently track down bestavailableevidence

3. Critically review thevalidity and usefulnessof the evidence

4. Implement changesin clinical practice

5. Evaluate performance

Recognise lack of certainty

98

• If you ask doctors, they say they need information about once a week. But if you debrief them, they raise about 2 questions for every three patients Covell DG et al. Ann Intern Med 1985 103: 596–599

• Many potential questions are not recognised by general practitioners (over confidence?, failure to recognise uncertainty?) Barrie AR et al. BMJ 1997; 315: 1512–1515

• Answers to most questions are not immediately pursued. Ely JW et al. BMJ 1999; 31: 358-361

• Doctors spent an average of less than 2 minutes pursuing an answer, and they used readily available print and human resources. Only two questions (out of over 1100) led to a formal literature search.

99

Travel agent’s consultation

100

101

• Pattern recognition, mental maps and short cuts are often used FOR DIAGNOSTIC DECISION MAKING

• Management decision making– Is the same problem (lots of complex

information to compute)– But creating mental maps to

make sense of this is NOT the preferred approach when there are better ways to keep up to date with the evidence for interventions

102

MindlinesGabbay and le May. BMJ 2004; 329: 1013–1016

“Clinicians rarely accessed, appraised, and used explicit evidence directly from research or other formal sources; rare exceptions were where they might consult such sources after dealing with a case that had particularly challenged them.”

103

MindlinesGabbay and le May. BMJ 2004; 329: 1013–1016

Not once was a guideline read.

Expert computer systems were rarely used (never in real time).

Instead, they relied on what we have called "mindlines,"

collectively reinforced, internalised tacit guidelines, which were informed by brief reading, but mainly by their interactions with each other and with opinion leaders, patients, and pharmaceutical representatives and by other sources of largely tacit knowledge that built on their early training and their own and their colleagues' experience.

The clinicians, in general, would refine their mindlines

by acquiring tacit knowledge from trusted sources, mainly their colleagues, in ways that were mediated by the organisational features of the practice, such as the nature and frequency of meetings, the practice ethos, and its financial and structural features, including the computer system.”

104

Reading and critical appraisal can (largely) be replaced by using brief

summaries of evidence from trusted sources

105

Finding the ‘best answer’, first time

Cochrane LibraryNICE, (NSFs)

EBM DTB MeReCBandolier

“Ivy League” journals

Clinical EvidenceInfoPOEMs, ProdigyBestTreatments NPC ref sheets

Textbooks

Use

fuln

ess

Medline, Google scholar

InfoRetriever, DrCompanion, self-assembly

106

Post-modern EBM skills

1. Recognise the need to check mindlines

2. Find the summarised

evidence

3. Understand the language of the evidence

4. Translate that into terms your patient

can understand

5. Enable your patient to play

a full part in shared

decision making

(if they want to)

107

Individuals need……

• Basic clin epi (how to read a summary)– Using Evidence to Guide Practice

• Information Mastery induction– Where to find summaries from trusted sources

• Point of Care Information Tools– DrCompanion, InfoRetriever

• Consultation translation skills• Local trusted colleague(s) – networked• CPD programme for COMMON conditions,

unbiased, evidence-based, summarised, reinforced

• Programme to spread the skills required to hunt for answers efficiently using the above tools

108

Step 3

Reach agreement on a management plan

• present options• where data is available, use “ x in 100 will benefit, y in 100 will be harmed” (not percentages)• consider supporting with visual representation e.g. ‘smiley faces’, CV risk charts• if an immediate decision is not required, consider materials for patient self-study• safety netting• housekeeping

Step three – you kiss and hold her tightly

109

Clopidogrel in ACS

• It’s really beneficial; I’d want all of my patients to be taking it

• Most people do fine just on aspirin. Adding clopidogrel prevents only a few people having an event and there’s the increased bleeding risk

• The specialist starts it and I don’t question that

110 N Engl J Med 2001; 345: 494-502

111

• Absolute risk• Relative risk• Numbers needed to treat• P values• Confidence Interval

112

113

114

Clopidogrel in ACS

• It’s really beneficial; I’d want all of my patients to be taking it.

• Most people do fine just on aspirin. Adding clopidogrel prevents only a few people having an event and there’s the increased bleeding risk.

• The specialist starts it and I don’t question that.

115

What would happen to 100 people like you who take sleeping tablets for more than a week.

☺ ☺ ☺ ☺ ☺ ☺ ☺

These SEVEN people sleep better, which

means they get an extra 25

minutes sleep a night

They also wake up once less

every 2 nights

For 76 people the tablets do

NOTHING, good or bad

These SEVENTEEN people have side effectsOne of them may be serious, like a fall or car crash

116

Antibiotics for bronchitisLittle P, et al. JAMA 2005; 293: 3029–3035

117

Level 3-4 diagnostic skills

Can work in biomedical, anticipatory

and hermeneutic relationships with

patients (ICE)

Finds and understands the strength

and the language of summaries of

evidence from trusted sources

Aware of diagnostic cognitive biases; uses

baseline probability and decision rules routinely

Translates that evidencefor patients,

uses DAs appropriately, discusses OICJ

routinely

The New Generalist Pinwheel

118

Back to hard core exam prep

119

Web based resources

An Introduction to Information Mastery www.poems.msu.edu/InfoMasteryNational Prescribing Centre www.npc.nhs.ukDrug And Therapeutics Bulletin www.which.net/health/dtb/main.html

Bandolier www.ebandolier.comClinical Evidence www.clinicalevidence.orgEffective Health Care Bulletin www.york.ac.uk/inst/crdCochrane Collaboration www.cochrane.co.ukNational Institute for Clinical Excellence www.nice.org.ukUnited Kingdom Medicines Information www.ukmi.nhs.ukCentre for Evidence Based Medicine www.cebm.netInfoPOEMS www.infopoems.comCASP www.phru.nhs.uk/casp/casp.htmNational electronic library for health www.nelh.nhs.uk

120

How to read a paper Trisha Greenhalgh

BMJ 1997; 315 : 180 – 3 : 243 – 6 : 305 – 8 : 364 – 6 : 422 – 5 : 480 – 3 : 540 – 3 : 596 – 9 : 672 – 5 : 740 – 3

www.bmj.com

Getting research findings into practiceVarious authors

BMJ 1998; 317 : 72 – 5 : 139 – 42 : 200 – 3 : 273 – 6 : 339 – 42 : 405 – 9 : 465 – 8

www.bmj.com

NB Health economics

121

Books

• Greenhalgh T How to read a paper 2nd edn London: BMJ Books 2001

• Sackett DL, Richardson WS, Rosenberg WMC, Haynes RB. Evidence-based medicine: how to practice and teach EBM. London: Churchill-Livingstone, 1996.

• Muir Gray JA. Evidence-based Healthcare. Churchill Livingstone. 1997.

122

123

Are you suffering from information overload? There are over 30,000 biomedical journals in print.

How do you cope?

Using Evidence to Guide Practice 5 interactive modules available via the Internet covering the

principles of EBM and the influences on prescribing

E-LearningLearn at a pace and

time to suit you

ALL FIVE MODULES FOR ONLY £35

Go to www.npc.co.uk for more details

Documents

1 Clinical Epidemiology Using Evidence to Guide Practice A bit of thinking and a few simple sums