Evidence Based Neurology Maurizio A. Leone Neurologic Clinic University Hospital “Maggiore della...

Evidence Based Evidence Based NeurologyNeurology

Maurizio A. Leone

Neurologic Clinic University Hospital

“Maggiore della Carità”,Novara, Italy

I International Course on Neuroepidemiology in Eastern Europe, Chisinau, Moldova. September 24-28, 2012

• The need for EBM

• How can I practice EBM ?

• What is EBM ?

• The story

• What are the “products” of EBM ?• Limitations of and criticisms to EBM

On the need of Evidence Based Medicine

1. Few medical decisions are based on evidence

Why do we need RANDOMIZED CONTROLLED TRIALS ?

In the early 1980s newly introduced antiarrhythmics were found to be highly successful at suppressing arrhythmias.

Not until a RCT was performed was it realized that, although these drugs suppressed arrhythmias, they actually increased mortality.

The CAST trial revealed Excess mortality of 56/1000.

By the time the results of this trial were published, at least 100,000 such patients had been taking these drugs.

Caveat

Many medical decisions are taken on the basis of:

• etiopathogenetic theories • experience• other factors (pharma industry, other

interests, ideology…)

Probability of a result favouring study drug in industry-sponsored trials OR = 4.05 (2.98-5.51)

On the need of Evidence Based Medicine

1. Few medical decisions are based on evidence

2. Deterioration of knowledge after graduation

Performance deteriorates ….

1. Level of blood pressure.2. Patient’s age.3. The physician’s year of

graduation from medical school.

4. The amount of target-organ damage.

Determinants of the clinical decision to treat some, but not other, hypertensives:

On the need of Evidence Based Medicine

1. Few medical decisions are based on evidence

2. Deterioration of knowledge after graduation

3. Delay in transferring results from research to clinical practice

Systematic review of bed rest after spinal tap

• 10 trials of bed rest after spinal puncture – no change in headache with bed rest– Increase in back pain

• Protocols in UK neurology units - 80% still recommend bed rest after LP

Serpell M, BMJ 1998;316:1709–10

• …evidence of harm available for 17 years preceding... Allen, Glasziou, Del Mar. Lancet, 1999

On the need of Evidence Based Medicine

1. Few medical decisions are based on evidence

2. Deterioration of knowledge after graduation

3. Delay in transferring results from research to clinical practice

4. Limited resources

Information epidemic: how to keep up-to-date ?

MEDLINE 20102,000 articles / day

approx 75 new trials published every day

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

Median minutes/week spent reading about my patients (self report):

• Medical Students: 90 minutes• House Officers: 0 (up to 70%=none)• Senior House Officers: 20 (up to 15%=none)• Registrars: 45 (up to 40%=none)• Senior Registrars 30 (up to 15%=none)• Consultants:

– Grad. Post 1975: 45 (up to 30%=none)– Grad. Pre 1975: 30 (up to 40%=none)

• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM

Evidence (from Latin “evidentia”)

In Italian (Rumanian-Moldovan):• Anything that is clear and obvious, that

doesn’t need any further demonstration

In English:• The available body of information

indicating whether a belief or a proposition is true or valid (= proof, testimony, sign).

‘Background’ Questions

• About the disorder, test, treatment, etc.

2 components:a. Root* + Verb: “What causes …”b. Condition: “… stroke?”

• * Who, What, Where, When, Why, How

Foreground learning: Clinical decision-making, history taking, examining, diagnosing, and therapeutic intervening.

Background learning: basic neurosciences, neuroanatomy, neurophysiology, neuropharmacology, amd neuropathology

Evidence-Based Medicine:The Practice

1. Translation to an answerable question (patient/manoeuvre/outcome).

2. Efficient track-down of the best evidence3. Critical appraisal of the evidence for its validity

and clinical applicability4. Integration of that critical appraisal with clinical

expertise and the patient’s unique biology and beliefs.

5. Evaluation of one’s performance.

• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM

Dave Sackett

Evidence-based Medicine: the storyPierre Charles Alexandre Louis (Paris, 1830): “A true science is merely a summary of facts which have no value unless they are expressed in numbers ... The statistics represent the only and fundamental basis of all the medical studies ‘. He was the promoter of “La Societé d’Observation Medicale”, a cultural movement supporting the concept that knowledge about a disease, its history, clinical presentation and treatment, could be derived from aggregated patient data rather than from individual experience.

Archibald Cochrane (1972): “It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.”

1986: The focus of Sackett and coll. gradually moves from "how to read the biomedical literature" to "how to use the biomedical literature to solve clinical problems."

1981: (McMaster Medical School, Canada): “How to read clinical journals", a series of articles describing the strategies of critical approach to the biomedical literature. These articles are among the top reprinted in the history of biomedical literature.

1993 JAMA: first article of the movement of Evidence-Based Medicine

1993: founding of the Cochrane Collaboration, an international network to “prepare, update and disseminate systematic reviews of controlled trials on the effects of health care and, where there is no controlled trials, systematic reviews of existing evidence, however.”

http://www.cochrane.org

• Back• Consumers and

communication• Dementia & cognitive

improvement• Incontinence • Infectious diseases• Injuries• Movement disorders• Multiple sclerosis

• Muskuloskeletal • Musculoskeletal injuries• Neuromuscular disease• Pain, palliative and

supportive care• Drugs and alcohol• Epilepsy• Eyes and vision• Stroke

Review Groups

• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM

Evidence-Based Medicine:The Practice

1. Translation to an answerable question (patient/manoeuvre/outcome).

????

1st part PopulationClinical characteristics of the patient or of the reference patients population

2nd part

3rd part

Structure of the clinical question

Courtesy of Luca Vignatelli

1st part PopulationClinical characteristics of the patient or of the reference patients population

2nd part InterventionTherapy: drug / device /

procedureDiagnosis: test

ComparatorAlternative clinical act (placebo / no therapy / other therapy / other test)

Structure of the clinical question

1st part PopulationClinical characteristics of the patient or of the reference patients population

2nd part InterventionTherapy: drug / device /

procedureDiagnosis: test

ComparatorAlternative clinical act (placebo / no therapy / other therapy / other test)

3rd part OutcomeClinical outcome of interest (to be increased/reduced)

TimeTime dimension of the observation of the outcome

Structure of the clinical question

Can coffee reduce daytime drowsiness?

In healthy adults engaging in normal activity, does coffee reduce daytime drowsiness compared with de-caffeinated coffee?

Control

PopulationIntervention

Outcome

Formulate a research question that can be

answered

Components: PICO(T)• Population• Intervention• Control • Outcome • T (Time)

Evidence-Based Medicine:The Practice

1. Translation to an answerable question (patient/manoeuvre/outcome).

2. Efficient track-down of the best evidence

• The expert (colleague): lack of obiectivity and completeness of information.

• Medical books (printed): often incomplete, outdated, lack of quantitative data. Selection of scientific evidence is not systematic neither explicit; generally organized by disease and not by clinical presentations.

• Journals: too many, fragmentation of topics, rarity of definitive studies to be transferred to clinical practice.Traditional (narrative) reviews: same methodological shortcomings of the medical books: selection bias, self-quotation and harmony with the opinion of the author.Randomized Clinical Trials (RCTs): difficult to adapt their results to the individual patient. Selected populations, complex patients excluded. Ideal world and peculiar competence and motivation of doctors. Often surrogate rather than clinically significant end-points, often relative and not absolute (NNT) measures are reported.

Traditional information instruments

GUIDELINES / HTA

SECONDARY LITERATURE

PRIMARY LITERATURE

CLINICAL STUDIES

Pyramid of the productionof medical literature

Courtesy of Luca Vignatelli

DATA-BANKS OF GUIDELINES / HTA

SECONDARY DATA-BANKS

PRIMARY DATA-BANKS

RCTs DATA-BANKS

How to organize the searchof evidence in the literature

Clinical research, synthesis of studies and recommendations

Primary Studies

Results ofresearch

. .. .

.

. .. .

.

. .. .

.

. .. .

.

. .. .

.

. .. .

.

Systematic ReviewsMeta-analyses

Sinthesis

HealthTechnology Assessments (HTA)

Guidelines Recommendations

Evidence-Based Medicine:The Practice

1. Translation to an answerable question (patient/manoeuvre/outcome).

2. Efficient track-down of the best evidence3. Critical appraisal of the evidence for its validity

and clinical applicability

We needn’t always carry out all 5 steps to provide E-B Care

• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM

Guidelines are:

• Recommendations of clinical practice,• produced through a systematic process,• to assist physicians and patients• in deciding which are the most

appropriate method of care• in specific clinical circumstances.

Aims of the guidelines

To ensure the highest degree of appropriateness of the interventions, reducing the possible variability in clinical decisions (due to lack of knowledge and subjectivity in the definition of strategies of care).

Basic methodological elements

• Multidisciplinarity• Use of systematic reviews• Explicit evaluation of quality of

evidence • Strenght of recommendations

Requirements of a guideline

• Validity• Reproducibility• Representativeness • Applicability• Flexibility• Clarity• Documentation• Update

Who produce guidelines?

• National and Regional Agencies

• Scientific Societies (EFNS,….)

• Research Institutions• Ad hoc Associations• ..

Data Bank IstituzioneNational Guideline Clearinghouse™ Agency for

Healthcare Research and Quality (USA)

National Library of Guidelines (GuidelinesFinder)

NHS Evidence (UK)

CMA Infobase Canadian Medical Association (CMA)

International Guidelines Library G-I-N – the Guidelines International Network

Where to find guidelines?

The quality of evidence (levels of evidence) depends on:

• Appropriatness of study design• Quality of study conduction and analysis• Effect size• Appropriatness and relevance of outcomes

The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.

EFNS Guidelines: classes of evidence • Class I: An adequately powered prospective, randomized, controlled clinical trial with

masked outcome assessment in a representative population or an adequately powered systematic review of prospective randomized controlled clinical trials with masked outcome assessment in representative populations. The following are required:

• (a) randomization concealment• (b) primary outcome(s) is/are clearly defined• (c) exclusion/inclusion criteria are clearly defined• (d) adequate accounting for dropouts and crossovers with numbers sufficiently low to

have minimal potential for bias• (e) relevant baseline characteristics are presented and substantially equivalent among

treatment groups or there is appropriate statistical adjustment for differences• Class II: Prospective matched-group cohort study in a representative population with

masked outcome assessment that meets a–e above or a randomized, controlled trial in a representative population that lacks one criteria a–e

• Class III: All other controlled trials (including well-defined natural history controls or patients serving as own controls) in a representative population, where outcome assessment is independent of patient treatment

• Class IV: Evidence from uncontrolled studies, case series, case reports, or expert opinion

• Level A: (established as effective, ineffective, or harmful) requires at least one convincing class I study or at least two consistent, convincing class II studies

• Level B: (probably effective, ineffective, or harmful) requires at least one convincing class II study or overwhelming class III evidence

• Level C: (possibly effective, ineffective, or harmful) rating requires at least two convincing class III studies

EFNS Guidelines: rating of recommendations

Grades of Recommendation Assessment,

Development and Evaluation

Consensus Conference

Drafting of recommendations by a jury at the end of a presentation and consultation of experts summarizing scientific knowledge on a given topic.Agreement among different figures regarding controversial and complex health issues, to provide patients the best quality of care in relation to available resources.

Clinical pathway: result of guidelines adaptation to local situations, with their specific organizational and management characteristics.

Protocol: the detailed outline of the steps to be followed in the treatment or the diagnosis of a patient.

What is a systematic review?

• Is a research, conducted with an explicit and repeatable method to determine what is the best available evidence on the effectiveness of an intervention or the accuracy of a diagnostic test.

• The meta-analysis is the most well-known statistical tool associated with the systematic reviews. It is used to comprehensively analyze all the observations of controlled clinical trials and to provide a summary estimate.

FEATURES TRADITIONAL REVIEWS

SYSTEMATIC REVIEWS

Cook DJ et al Ann Int Med 1997; 126:320, adapted

Question often large often focused

Protocol no yes

Literature search strategy not specified explicit and reproducible

Selection of studies criteria not specified inclusion/exclusion criteria

Evaluation of studies variable rigorous and critical

Synthesis often qualitative summary quantitative summary (meta-analyses)

Inferences not always evidence-based generally evidence-based

Systematic Reviews seek to:

• Identify all relevant published and unpublished primary studies• Select studies or reports for inclusion• Critically appraising the quality of each study or report• Extract data• Quantitatively synthetise findings from individual studies or reports in an unbiased way• Interpret the findings and present a balanced and impartial summary with due consideration of any flaws.• SR may examine quantitative or qualitative evidence.

If we believe it,does it apply to our patient?

• Is our patient (or population) so different from those in the primary studies that the results may not apply?

• consider differences in: - time - many things change. - culture - both treatments and values of

outcomes can be different. - stage of illness or prevalence can affect results.

We believe it ! But —>> does it matter?

• Is the benefit worthwhile to my patient?Ask the patient about cultural values.

• Are the outcomes relevant for my patient?Think about Relative Risk Reduction vs. Absolute Risk to your patient. Potential benefit is the Absolute risk avoided in our patient!

Event Rate = 1 relapse at 2 years

RRR ARR NNT

Control Event Rate (CER)

Experimental Event rate (EER)

(CER-EER) /CER

(CER-EER)

1/AAR

IFNBMSG 76% 64%

(76-64)/76 = 16%

(76-64)=12%

100/12=8

Event Rate = 1 relapse at 2 years

RRR ARR NNT

Control Event Rate (CER)

Experimental Event rate (EER)

(CER-EER) /CER

(CER-EER)

1/AAR

IFNBMSG 76% 64% (76-64)/76 = 16%

(76-64)= 12%

100/12= 8

Hypothesis A

7,6% 6,4% (7,6-6,4)/7,6 = 16%

(7,6-6,4) = 1,2%

100/1,2= 83

Event Rate = 1 relapse at 2 years

RRR ARR NNT

Control Event Rate (CER)

Experimental Event rate (EER)

(CER-EER) /CER

(CER-EER)

1/AAR

IFNBMSG 76% 64% (76-64)/76 = 16%

(76-64)= 12%

100/12= 8

Hypothesis A

7,6% 6,4% (7,6-6,4)/7,6 = 16%

(7,6-6,4) = 1,2%

100/1,2= 83

Hypothesis B

0,76% 0,64% (0,76-0,64) /0,76= 16%

(0,76-0,64) = 0,12%

100/0,12= 833

Infezioni nosocomialiProfilassi vs controllo

Risk ratio.01 .05 .2 1 5

Study

Risk ratio (95% CI)

0.29 (0.17,0.49) Abele 0.07 (0.01,0.52) Aerdts 0.33 (0.18,0.62) Blair 0.82 (0.49,1.37) Boland 0.33 (0.11,0.99) Cockerill 0.69 (0.23,2.01) Finch 0.11 (0.01,2.05) Jacobs 1 0.15 (0.07,0.36) Kerver 0.39 (0.21,0.73) Palomar 0.32 (0.15,0.68) Rocha 0.57 (0.40,0.81) Sanchez 0.61 (0.47,0.78) Stoutnbek 0.28 (0.13,0.59) Ulrich 0.53 (0.33,0.85) Verwaest1 0.72 (0.47,1.10) Verwaest 2 0.18 (0.05,0.59) Winter

For(r)est plot

Punctual estimate

Favours treatment Favours control

No difference line

Confidence Intervals

Not statistically significant results

The pooled analysis of the results of all studies is   represented by a diamond (weighted average).If the confidence interval includes the unity, this

means that we have not found a statisticallysignificant difference between the two treatments.

Total (95% CI) 410/1194 301/925 100.0 0.96 (0.85, 1.08)Test for heterogeneity 2=18.11 df=14 p=0.2020 I2=23.3%Test for overall effect z=0.68 p=0.50

Standard Meta-analysis (Left)&Cumulative Meta-analysis (Right)

Cochrane Database of Systematic Reviews

DARE: http://www.crd.york.ac.uk/crdweb

Where to find systematic reviews?

• The systematic evaluation of properties, effects, and/or impacts of health care technology

• It may address the direct, intended consequences of technologies as well as their indirect, unintended consequences

• Its main purpose is to inform technology-related policymaking in health care

• HTA is conducted by interdisciplinary groups using explicit analytical frameworks drawing from a variety of methods

(HTA glossary - www.htaglossary.net)

Health Technology Assessment

Were to find Health Technology Assessments

http://www.crd.york.ac.uk/crdweb/

• The need for EBM• What is EBM ?• The story• How can I practice EBM ?• What are the “products” of EBM ?• Limitations of and criticism to EBM

Limitations of EBM

1. Shortage of consistent scientific evidence2. Difficulties to apply evidence to the careof

individual patient3. Economic barriers to the practice of high-

quality medicine4. Need to develop new skills5. Limited time and resources6. Paucity of evidence that EBM “works”

Misperceptions of EBM

1. EBM denigrates clinical expertise2. It ignores patients’ values and preferences3. It promotes a cookbook approach to medicine4. It is simply a cost-cutting tool5. It is an ivory-tower concept6. It is limited to medical research7. It leeds to therapeutic nihilism in the absence of

evidence from randomized clinical trials

Patients’ values and

expectations

EBM for clinical decision-making

Sackett et al. 2000

Best available evidence

from medical research

Individual clinical

expertise

Available resources

Best Evidence Resources for Neurology

Guidelines (free access): American Academy of Neurology www.aan.com/go/practice/guidelines

• European Federation of Neurological Societies (EFNS) www.efns.org

CATs Data bank: Evidence Based Neurology, University of Western Ontario www.uwo.ca/cns/ebn/

Courtesy of Luca Vignatelli