What does this mean, and what do I do with it?

Level of evidence, strength of recommendations, and patient-centered care

Objectives

1. Describe the concept of “levels of evidence (LOE)” and explain which research study designs lead to certain levels of evidence.

2. Define the term “strength of recommendation taxonomies (SORT)” and give examples of two organizations’ SORT criteria.

3. Evaluate clinical practice guidelines by using LOE and SORT, and summarize the evidence-based applicability of these guidelines.

The situation

While covering the ER, an 18 year-old male presents with acute shortness of breath. His symptoms had present over the past 8 hours, but acutely worsened 30 minutes ago.

On arrival, he is tachypneic, using accessory muscles, and only able to speak in short sentences. His RR is 36, with supra-sternal retractions. Pulse Ox is 94%, and his peak flow is <40% predicted.

The situation

The patient is placed on supplemental O2, and he is given 3 combined albuterol/ipratropium nebulizer treatments. Despite this, he does not improve significantly.

The question

What are your next treatment options?

How will you know if what you *think* is the next step will be an effective next step?

How can we quickly and efficiently assess the quality of evidence?

Two tools

Level of evidence (LOE)

Strength of recommendation (SOR)

Level of evidence (LOE)

Provides a quick overview

How reliable is the study’s findings?

How much bias is likely?

How “good” is this study?

Level of evidence

Study design affects likely LOE: more robust studies usually lead to better LOE

Review of study design: Randomized-controlled trials Cohort studies Case-control studies Case series or case reports

Also: reviews and syntheses of prior studies

Pyramid of Evidence

Pyramid of Evidence

Level of evidence

Oxford Centre for Evidence-Based Medicine Updated 2011 Divides criteria by the question being asked A “gold standard”?

Level 1-5 Level 1 is the highest/least-biased/”best”

evidence, usually from RCTs, systematic reviews, etc.

Level 5 evidence is mechanism-based reasoning.

Level of evidence

Strength of recommendation Assesses the usefulness of clinical

recommendations

Usually seen in the context of point-of-care tools, evidence-based summaries, etc.

Provides a patient-centered review of clinical evidence

Incorporates patient-centered outcomes and level of evidence

Patient vs. disease

Disease-oriented outcomes: “These outcomes include intermediate, histopathologic, physiologic, or surrogate results (ie, blood sugar, blood pressure, flow rate, coronary plaque thickness) that may or may not reflect improvements in patient outcomes”

Patient-centered outcomes: “These are outcomes that matter to patients and help them live longer or better lives, including reduced morbidity, reduced mortality, symptom improvement, improved quality of life, or lower cost”

Patient vs. disease

Disease-oriented outcomes: Measurements that scientifically or physiologically “make sense” in terms of disease management (BP, A1c, FEV1, etc.); surrogate outcomes.

Patient-centered outcomes: Events that actually make a tangible difference in patients’ lives (hospitalizations, death, etc.)

Patient vs. disease

Studies that show apparently beneficial changes in surrogate outcomes may not improve (or may worsen) patient outcomes

Surrogate outcomes are easier to study, and those studies are more common

SOR taxonomies allow a rapid overview of patient-oriented recommendations

Strength of recommendation Study designs also tend to predispose to

certain SOR…if those studies used patient-oriented outcomes, better studies lead to better SOR

Different organizations use different approaches to define SOR

Are you familiar with any particular SOR taxonomies?

Strength of recommendation USPSTF

A, B, C, D, I

AAFP A, B, C

AAP: strong recommendation, recommendation, option, no

recommendation

ACP = USPSTF

Cochrane/BMJ GRADE strong, weak

Evidence-based guideline exercise

How would you determine the next steps in treating our patient?

Search for guidelines that answer the clinical question (5 mins)

Report back re: what you found and the quality of the evidence

Evidence-based guideline exercise

What is the LOE or SOR associated with the guidelines you found?

How confident are you in these guidelines?

Cautions

LOE and SORT will NOT Provide definitive judgment Provide definitive treatment

recommendations unless ▪ the study population represents your

population▪ the outcome is patient-centered and

clinically-relevant, treatment is appropriate▪ no other treatment is superior

Summary

LOE and SOR are interrelated and complementary

LOE focuses on individual studies, SOR on the aggregate

LOE and SOR help guide clinical decision-making, but do not dictate it

Summary

“What does this mean?” LOE How good is this evidence? How much can I trust it?

“What do I do with it?” SOR How can I best use this evidence? What does this evidence mean to me and

my patients?