Chapter

39 Evidence-based medicineC. Heneghan, P. Glasziou

IntroductionThe process of lifelong, selfdirected learning includesrecognition of clinically important questions about diag-nosis, therapy, prognosis and other aspects of healthcarewhen caring for our patients. The methods of evidence-based medicine (EBM) aim to provide skills that help cli-nicians to rapidly answer these questions and assimilatenew evidence and ideas and put them into practice.

We can summarise the EBM approach as a fourstep process (Straus et al 2005):• asking answerable clinical questions• searching for the best research evidence• critically appraising the evidence for its validity

and relevance• applying the evidence to groups and individuals.

To remind us to evaluate and improve our skills atthese steps, a fifth 'step' is often suggested: 'Evaluatingyour own self education performance.

In this chapter we outline a possible curriculumplan for teaching EBM, moving from awareness andprinciples to skills and practice.

Introductory lecture - an houron raising awareness

"Evidence Based Medicine is the conscientious,explicit and judicious use of the best currentevidence in making decisions about the care ofindividual patients."

Initial concepts in an introductory lecture shouldinclude: what EBM (sometimes called 'evidencebased practice') is and isn't, the problem of our infor-mation overload, our need to discriminate betweengood and poor quality evidence, and how EBM skillscan help.

Important themes to develop are trying to keep upto date and the problem of overcoming the increasing

world literature: over 1500 new medical research arti-cles are published daily (Fig. 39.1). You might stop andask your learners to reflect on how they currently learnand keep up to date. You might also ask them how muchtime is being spent on each process. Activities usuallyidentified include: attending lectures, and conferences,reading articles in journals, tutorials, textbooks, clinicalguidelines, clinical practice, small group learning, studygroups, using electronic resources, and speaking to col-leagues and specialists. There is no right or wrong way tolearn, a mixture of all of these methods will be benefi-cial in the overall process of gathering information.

It is helpful to think of learning needs as a process ofgathering information in two different ways 'push' and'pull*. The 'push' is learning from the deluge of infor-mation that arrives in our post or email, on a variety oftopics. This type of learning can be thought of as 'just incase learning'. When the information is filtered to pro-vide only what is important to clinical practice and hasalready been appraised for certain validity criteria, it canbe very useful. But most of our learning should focuson information 'pull': answering the questions that arisein practice - also called 'just in time' learning.

Evaluate your own learning strategics andyour learners• Write down one recent patient problem• What was the critical question?

• Did you answer it? If so, how?

Samples of an introductory presentation are avail-able free from the Centre for Evidence Based Medicinewebsite (Oxford Centre for Evidence-Based Medicinedownloads, http://www.cebm.net/downloads.asp).

Asking answerable questionsYou should consider following or combining this ses-sion (1-2 hours) with searching skills laboratory (1-2hours). Your librarians may have skills unbeknown toyou that might help in this area. Use the PICO prin-ciple, to formulate the clinical questions:

FWidence-based medicine 295

2500000

2000000

1500000

1000000

500000

0Trials MEDLINE BioMedlcat

Fig. 39.1 The amount of medical research

P Population/patient - what is the disease problem?/ Intervention/indicator - what main treatment is

being considered?C Comparator/control - is this in relation to an

alternative?O Outcome - what are the main outcomes of

interest to the patient?(And sometimes we add a T for the Time to theoutcome, e.g. 5 year survival).

Learners should practice dissecting the questioninto its component parts and then restructure it sothat the components can be used to direct the search.Prior to clinical work, scenarios are useful to teachquestion building. For the less experienced, you mightbegin with short scenarios that lend themselves todissecting the question (s) easily.

One way to begin is by giving an initial patientpresentation, and ask students what medical knowl-edge they need to 'solve' the clinical problem. Listall the questions, and then classify using the typol-ogy below (the type will affect where you look forthe answer and what type of research you can expectto provide it)

Typology for question building

Clinical findings: how to interpret findings fromthe history and examination.

Aetiology: the causes of diseases.

Diagnosis: what tests are going to aid you in thediagnosis?

Prognosis: the probable course of the diseaseover time and the possible outcomes.

Therapy: which treatment are you going tochoose based on beneficial outcomes, harms, costand your patient's values?

Prevention: primary and secondary risk factorswhich may or may not lead to an intervention.

Cost-effective ness: is one intervention morecost effective than another.

Quality of life: what effect does the interventionhave on the quality of your patient's life?

Phenomena: the qualitative or narrative aspectsof the problem

I low you are going to integrate askingquestions into your own practice?The challenge to the teacher is to identify questionsthat are both problem based and orientated to thelearners needs, at the same time you will also be iden-tifying gaps in your own knowledge which may needto be addressed. Think of how you and your studentsare going to keep track of the questions and come to aclinically useful answer.

Remember to recognise potential questions whichare clinically useful to you and your learners, decidewhich of the questions to focus on, guide your learnersinto developing useful questions, and asses their per-formance in building and asking answerable questions.

Searching for the evidence

Resources which you will need:• if available, a computer laboratory with internet

access" one terminal for every two students

• if available, data projection linked up to acomputer with database access for students toview searches in action

• internet access or access to medical researchdatabases.

We recommend starting with MEDLINE forsearching for the evidence.

PubMed: http://www.nc hi.nlm.nih.gov/PubMed/Using PubMed allows all the concepts of proficientsearching to be taught. The most useful is the 'ClinicalQueries' section: a question focused interface with f i l -ters for identifying the more appropriate studies forquestions of therapy, prognosis, diagnosis and aetiol-ogy. Ask students to do a search with and without thefilters, and contrast the number of 'hits' found. Warnstudents that the first searches will take 15-30 minutes,but with regular practice they should get key informa-tion in a few minutes.

The session will take 1--2 hours, and there are sev-eral good guided tutorials on the web.

The success of problem-based learningdepends on the abi l i ty to find the currentbest evidence effectivelyFinding answers to questions when done well withspeed can be highly rewarding or when done poorlycan be frustrating and time consuming. A study of103GPs showed that they generated about 10 ques-tions over a 2.5 day periotl. They tried to find theanswers for about half of them. The most criticalfactor influencing which questions they followed up

296 SECTION S: Curriculum themes

was how long they thought it would take to obtain ananswer. If they thought the answer would be availablein less than a couple of minutes, they were preparedto look for il. If they thought it would take longer,they would not bother. Only two questions in thewhole study (0.2%) were followed up using a properelectronic literature search (Ely et al 1999).

How are you i^oing to n i ; u n t ; i i nsearching s k i l l s ?

In teaching searching skills think of the resources youhave available to you. Librarians often have knowledgein this area far beyond the skills of a busy clinician.They often run courses and workshops, if they aren'tthen consider approaching them to begin setting themup. On a personal level you should be proficient insearching, often you will be asked questions aboutsearching strategics: Which database to use? Why doI seem to get so many articles?

It is often best to have a prepared scenario whichgenerates a lour part question and leads on to demon-strating some of the key components of an effectivesearch strategy.

• In a five year old with a fever and a red bulgingtympanic membrane should I prescribe antibioticsor watch and wait?

(Clinically useful systematic review in Glasziouet al 2004)

• A 35-year-old lady came to clinic with a fresh dog-bite and she wanted to know whether she shouldhave some prophylactic antibiotics.(Clinically useful answer in PubMed/Medline, in

Cummings 1994).

There are two main strategies for searching bib-liographic databases: thesaurus searching (all articlesare indexed under subject headings) and text-wordsearching (where you search for specific words orphrases in the title or abstract). Once the questionhas been broken down into its components, it can becombined using Boolean operators 'AND' and 'OR'(Fig. 39.2, Table 39.1).

In combining terms into a search strategy, it can beuseful to represent them as a Venn diagram (Fig. 39.3).Complex combinations can then be structured.

Truncation and wildcard (') NEAR = AND plus wordsI close together

(furunc* OR (staphylococc* NEAR skin)) AND recur':TI

t t tBOOLEANS IN Group words with () Word must be

CAPITALS in TITLE

Fig, 39.2 Tips and tactics for searching

Retrieve all articles with either word

AND

Retrieves articles only with both words

Fig. 39.3 A search strategy represented as a Venndiagram

Table 39.1

Convert your question into a search strategy

.̂ Question part^f*-~3tff. -t y ' .

Patient/problem

Intervention

Comparison

Question termAdult, heart failure

(J-blocker

Usual care, standard therapy

Synonyms

Left ventricular failure, congestive heart failure, NYHAclassification

Metoprolol, carvedilol

Outcome Mortality Death, survival

Evidence-based medicine 297

Critically appraising the evidencefor its validity and relevanceInitially it is worth putting over to your audience in3 lecture format some of the key concepts of criti-cal appraisal and why it is important and some of thepotential pitfalls. Flecainide use in the treatment ofventricular arrhythmias (Anderson et al 1981, Echtet al 1991) is an illustration of potential pitfalls.

If you are about to embark on teaching appraisalthink of how proficient you are in appraisal in the fol-lowing study types:• therapy• diagnosis• systematic reviews• harm/aetiology• prognosis• optional - economic analysis.

If you are not feeling confident then refer backto the EBM book by Straus et al (2005) or considerattending a workshop on Teaching and Practising EBM(Oxford Centre for Evidence-Based medicine; http://www.cebm.net).

Critical appraisalTeaching the details of critical appraisal is best done assmall group work, with a specific article and flipchartor whiteboard. Small group work allows generationof free discussion of these new concepts between thegroup leader and the participants, enabling them togain knowledge from their peers. As with many intel-lectual skills, practice, discussion and feedback arehelpful for faster and deeper learning. Each of the sixtopics above will take a session of 60-90 minutes, sothe full set may take 6-9 hours of small group work.

Working through critical appraisalTo help you in the process, we suggest that you havesome prepared critical appraisal worksheets available toguide the process. You can design your own but thereare many available which will help you, for example:• Oxford Centre for Evidence-Based Medicine free

downloads of critical appraisal worksheets (www.cebm.net/downloads.asp)

• Centre for Health Evidence University of Alberta(www.cche.net/che/home.asp)

• Centre for Evidence-Based Medicine, MountSinai, Toronto (www.cebm.utoronto.ca/J.

Crunching the numbersLearners should become proficient with most com-mon outcome measures used in trials. Results can

be expressed in many ways and you should feelcomfortable in demonstrating the calculationsinvolved:• Relative risk (RR)• Relative risk reduction (RRR)• Absolute risk reduction (ARR)• Number needed to treat (NNT)• Odds ratios (OR)• Sensitivity and specificity of diagnostic tests• Pre-test probabilities likelihood ratios and post-

test probabilities.Do not expect your learners to calculate all of

these every time, but they should be able to calcu-late NNTs and relative risks; however, the more com-plex calculations may put them off. Introduce the useof a nomogram (Fig. 39.4) to aid diagnostic decisionmaking.

Confidence intervals and F valuesStatistics will often bring fear to some of your students.Remember that most people will only need to be usersof statistics not doers. So emphasise interpretation notcalculation. So you might explain that all studies are

0.1

0.2

0.5

1

2

5

10

20

304050607080 -

90

95

98

100

2000100050020010050-20-10521

0.50.20.1

- 0.05- 0.020.01

- 0.0050.0020.0010.0005

99

98

95

90

80

7060

- 50- 40- 30

20

- 10

_ 5

- 2

- 1

- 0.5

0.2

-0.1

Pre-testProbability (%)

Likelihoodratio

Post-testprobability (%)

Fig. 39.4 Nomogram for converting pre-test probabilitiesinto post-test probablilties for a dianostic test result witha given likelihood ratio. (Reproduced with permission fromthe Oxford Centre for Evidence-Based Medicine fromhttp://www.cebm.net/index.aspx?o=1 161)

298 SECTION 5: Curriculum themes

subject to some random error, and that the best we cando is estimate the true risk based on the sample of sub-jects in a trial (called the 'point estimate'). Statisticsprovide two ways of assessing chance:

• P-values (hypothesis testing)

• Confidence intervals (estimation).

Do not go into the statistical methods of calcu-lating the answers. Provide discussion about the rel-evant methods and why an intervention can only beconsidered useful if the 95% confidence interval (CI)includes a clinically important treatment effect. A use-ful exercise is a 'forest plot' from a meta-analysis. Askstudents to read off which is the largest and smalleststudies, which are statistically significant, and whichare clinically significant, etc.

Make clear the distinction between statistical sig-nificance and clinical importance:

• Statistical significance relates to the size of the effectand the 95% CIs in relation to the null hypothesis

• Clinical importance relates to the size of the effectand to a minimum effect that would be consideredto be sufficiently important to change practice.

Applying the evidenceThe questions you should think of before you decideto apply the results of" the study to your patient art*:

• Is the treatment feasible in my setting?

• Arc your patients similar to those of the study?• What alternatives are available?

• Will the potential benefits outweigh the harms?

" What does the patient value?

Evaluating your own performanceThe most important evaluations are the ones you andyour learners design and carry out yourselves. Thequestions you might want to ask of yourself include:

• How many questions am I recording?

• Are you using different databases in your searchstrategies?

• Arc you challenging your colleagues abouteveryday decisions?

• Are you answering your questions?• Has your practice changed?

Think of" how you might want to be evaluated. Smallpresentations such as in a journal club can be a valuableway of appraising students, whilst increasing everyone'sknowledge. Arc you creating critically appraised topics(CATs)? These are structured appraisals incorporatingall of the relevant points you would want to know froma study. There is a free downloadable version of a CAT-

maker available at the CEBM web-site (Oxford Centrefor Evidence-Based medicine, http://www.cebm.net,Evidence-Based Medicine. http;//www.evidencebased-rncdicine.com).

Are you setting clearly defined objectives in yoursmall groups, and is this meeting the needs of yourstudents. If your learners arc creating important infor-mation, think of how you might want to store thisresource. You could create a database that could beaccessed by others in your related fields.

Integrating EBM into yourcurriculumThink of finding a like-minded individual who will helpyou to design and carry out the introduction and main-tenance of an EBM course. You can have a rapid intro-duction to EBM in one-day courses: there are severalgood workbooks available to guide you in this process(Glasziou et al 2007). A one-day course would com-prise an introductory lecture, searching skills and criti-cal appraisal (of just 1 study type) in small groups- Itis possible to integrate EBM throughout the course ofa year, making your scenarios relevant to the differentdisciplines that a student will face. It is essential todemonstrate that you are practicing EBM in your ownclinical duties; leading by example is an excellent wayto illustrate how evidence can be incorporated into yourpatients care. Be adventurous and try different strate-gies, mix and match lectures and small group work,think about introducing a journal club for all thosequestions that still need answering. Join up with prac-titioners from different areas, and consider refreshingyour skills at one of the many courses available.

SummaryThe traditional Continuing Medical Education flow isfor research to be formulated into guidelines or reviews(evidence-based we hope'), packaged into CME, andpresented written (in journals) or at educational ses-sions to be later retrieved and applied at the appro-priate clinical moment. Instead we suggest that thetraditional flow be reversed, and the choice of topicson which to gather information should come directlyfrom caring lor our patients (Del Mar & Glasziou2001). Use a variety of strategies to incorporate thefive steps of EBM into your course.

Essential to this process is the teacher's role as aneffective practitioner of EBM.

ReferencesAnderson J L, Stewart J K, Perry H A el al 1 981 Oral

fleicanide acetate for the treatment of ventriculararrhythmias. New England Journal of Medicine305:473-477

Evidence-based medicine 299

CurnminKs P 1994 Antibiotics to prevent infection inpatients with dog bite wounds: a meta-analysis ofrandomized trials. Annals of Emergency Medicine23:535-540

pel Mar C B, Glasziou P 2001 Ways of using evidence-based medicine in general practice. Medical Journalof Australia 174:374-350

l-cht D S, l.iebson P R, Mitchell L B et al 1991Mortality and morbidity in patients receivingccainidc, flecainide, or placebo. The CardiacArrhythmia Suppression Trial. NEJM 324:781-788

Ely J W, Osheroff J A, Ebell M H et al. 1999 Analysis ofquestions asked by family doctors regarding patientcare. British Medical Journal 319:358-361

Glas/jou P P, Del Mar C B, Sanders S L, Hayem M2004 Antibiotics for acute otitis media in children(Cochrane Review). In: The Cochrane Library, Issue 1.John Wiley, Chichester

Glasziou P, Del Mar C, Salibusry J 2007 Evidence-basedpractice workbook. BMJ Books, Oxford

Straus et al 2005 Evidence-based medicine: howto practice and teach EBM, 3rd edn. ChurchillLivingstone, New York

Further readingClinical Evidence. Online. Available: http://www.

clinicalevidcnce.c.omEvidente-Based Medicine. Online. Available: http://

www. cvidencebasedmedicine.comGreen M L, Ciampi M A, Ellis P J 2000 Residents'

medical information needs in clinic: are they beingmet? American Journal of Medicine 109:218-233

Oxford Centre for Evidence-Based medicine. Online.Available: http://www.ccbm.net/

Oxford Centre for Evidence-Based Medicinedownloads. Online. Available: http://www.cebm.net/downloads.asp