What does this mean, and what do I do with it?
Level of evidence, strength of recommendations, and patient-centered care
Objectives
1. Describe the concept of “levels of evidence (LOE)” and explain which research study designs lead to certain levels of evidence.
2. Define the term “strength of recommendation taxonomies (SORT)” and give examples of two organizations’ SORT criteria.
3. Evaluate clinical practice guidelines by using LOE and SORT, and summarize the evidence-based applicability of these guidelines.
The situation
While covering the ER, an 18 year-old male presents with acute shortness of breath. His symptoms had present over the past 8 hours, but acutely worsened 30 minutes ago.
On arrival, he is tachypneic, using accessory muscles, and only able to speak in short sentences. His RR is 36, with supra-sternal retractions. Pulse Ox is 94%, and his peak flow is <40% predicted.
The situation
The patient is placed on supplemental O2, and he is given 3 combined albuterol/ipratropium nebulizer treatments. Despite this, he does not improve significantly.
The question
What are your next treatment options?
How will you know if what you *think* is the next step will be an effective next step?
How can we quickly and efficiently assess the quality of evidence?
Two tools
Level of evidence (LOE)
Strength of recommendation (SOR)
Level of evidence (LOE)
Provides a quick overview
How reliable is the study’s findings?
How much bias is likely?
How “good” is this study?
Level of evidence
Study design affects likely LOE: more robust studies usually lead to better LOE
Review of study design: Randomized-controlled trials Cohort studies Case-control studies Case series or case reports
Also: reviews and syntheses of prior studies
Pyramid of Evidence
Pyramid of Evidence
Level of evidence
Oxford Centre for Evidence-Based Medicine Updated 2011 Divides criteria by the question being asked A “gold standard”?
Level 1-5 Level 1 is the highest/least-biased/”best”
evidence, usually from RCTs, systematic reviews, etc.
Level 5 evidence is mechanism-based reasoning.
Level of evidence
Strength of recommendation Assesses the usefulness of clinical
recommendations
Usually seen in the context of point-of-care tools, evidence-based summaries, etc.
Provides a patient-centered review of clinical evidence
Incorporates patient-centered outcomes and level of evidence
Patient vs. disease
Disease-oriented outcomes: “These outcomes include intermediate, histopathologic, physiologic, or surrogate results (ie, blood sugar, blood pressure, flow rate, coronary plaque thickness) that may or may not reflect improvements in patient outcomes”
Patient-centered outcomes: “These are outcomes that matter to patients and help them live longer or better lives, including reduced morbidity, reduced mortality, symptom improvement, improved quality of life, or lower cost”
Patient vs. disease
Disease-oriented outcomes: Measurements that scientifically or physiologically “make sense” in terms of disease management (BP, A1c, FEV1, etc.); surrogate outcomes.
Patient-centered outcomes: Events that actually make a tangible difference in patients’ lives (hospitalizations, death, etc.)
Patient vs. disease
Studies that show apparently beneficial changes in surrogate outcomes may not improve (or may worsen) patient outcomes
Surrogate outcomes are easier to study, and those studies are more common
SOR taxonomies allow a rapid overview of patient-oriented recommendations
Strength of recommendation Study designs also tend to predispose to
certain SOR…if those studies used patient-oriented outcomes, better studies lead to better SOR
Different organizations use different approaches to define SOR
Are you familiar with any particular SOR taxonomies?
Strength of recommendation USPSTF
A, B, C, D, I
AAFP A, B, C
AAP: strong recommendation, recommendation, option, no
recommendation
ACP = USPSTF
Cochrane/BMJ GRADE strong, weak
Evidence-based guideline exercise
How would you determine the next steps in treating our patient?
Search for guidelines that answer the clinical question (5 mins)
Report back re: what you found and the quality of the evidence
Evidence-based guideline exercise
What is the LOE or SOR associated with the guidelines you found?
How confident are you in these guidelines?
Cautions
LOE and SORT will NOT Provide definitive judgment Provide definitive treatment
recommendations unless ▪ the study population represents your
population▪ the outcome is patient-centered and
clinically-relevant, treatment is appropriate▪ no other treatment is superior
Summary
LOE and SOR are interrelated and complementary
LOE focuses on individual studies, SOR on the aggregate
LOE and SOR help guide clinical decision-making, but do not dictate it
Summary
“What does this mean?” LOE How good is this evidence? How much can I trust it?
“What do I do with it?” SOR How can I best use this evidence? What does this evidence mean to me and
my patients?